Embedding Visualizer
Visualize word embeddings in 2D space and explore semantic relationships
This is a simplified 2D projection of word embeddings. Real embeddings have hundreds of dimensions — this demo uses pre-computed positions to illustrate the concept of semantic similarity.
2D Embedding Space
Word Analogy Demo
The famous "king - man + woman = queen" analogy demonstrates that embeddings capture semantic relationships as vector arithmetic.
Related Tools
Perplexity Explainer
Calculate and understand perplexity from probability distributions
ROUGE Score Calculator
Calculate ROUGE-N and ROUGE-L scores for summarization tasks
Temperature Visualizer
Visualise how temperature and top-p sampling affect next-token probabilities
Tokenization Visualizer
See how text is broken down into tokens by different tokenizers (BPE, WordPiece)
Nucleus Sampling (Top-p) Demo
Interactive demo explaining how Nucleus Sampling filters token selection
Vector Dimension Guide
Reference for default embedding dimensions of popular models (OpenAI, Cohere, etc.)
What Are Word Embeddings?
Word embeddings are dense vector representations of words in a continuous vector space. Words with similar meanings are positioned closer together in this space. For example, "king" and "queen" will be near each other, as will "dog" and "puppy".
Popular embedding models include Word2Vec, GloVe, FastText, and modern transformer-based embeddings like those from BERT and GPT.
Why Visualize Embeddings?
🔍 Explore Relationships
Discover which words the model considers similar.
🐛 Debug Issues
Identify unexpected clustering or bias in embeddings.
📊 Understand Data
See how your document collection is distributed.
🎓 Learning
Build intuition about how models represent meaning.
Dimensionality Reduction
Real embeddings have hundreds of dimensions (e.g., 768 for BERT, 1536 for OpenAI). To visualize them in 2D, we use dimensionality reduction techniques:
t-SNE
Preserves local structure. Great for clusters but distances aren't meaningful.
UMAP
Faster than t-SNE, better preserves global structure.
PCA
Simple and fast. Shows principal axes of variation.
Pro Tip: Real Visualization
For real embedding visualization, try TensorBoard's Embedding Projector, Nomic Atlas, or Python libraries like matplotlib with UMAP/t-SNE.
Related Tools
Cosine Similarity
Calculate similarity between vectors.
Vector Dimensions
Compare embedding model dimensions.