Why this matters
- PolicyDesk's first search was keyword-only. "PTO" never matched "privilege leave," so the right page was invisible.
- Embeddings are why "time off" and "leave policy" can land on the same chunk.
- Interviewers ask for the cosine formula, the dimension of a common model, and what breaks if you mix models.
- The vector is useless if you cannot explain what distance means.
What a text embedding is
A text embedding model reads a chunk and returns a vector, for example 384 or 1536 numbers. Chunks about leave cluster. Chunks about mortgage rates sit somewhere else. The model was trained so that texts a human would call similar end up with vectors that point in a similar direction.
Nothing in the vector is a readable word. You cannot scan dimension 12 and find "casual leave." You can compare two vectors and decide they are neighbors.
What embeddings buy you
- Similar text maps to similar vectors, even when the words differ.
- Different topics map apart, which is what makes search selective.
- Retrieval becomes a geometry problem: find the nearest stored vectors to the query vector.
- Machines compare meaning without a human tagging every synonym.
From chunk to vector
The steps inside the model
- Preprocess. Normalize weird whitespace and decide whether to keep case. Most modern embedding models handle raw text; do not invent a stemmer unless the model card tells you to.
- Tokenize. Break text into subword tokens. A long chunk may be truncated. If your chunker emits pieces longer than the embedding model's max length, the tail is silently dropped and you embed a fragment.
- Encode. The network turns tokens into contextual vectors.
- Pool. Many embedding models average token vectors (mean pooling) into one vector for the whole chunk. That single vector is what you store.
- Normalize (often). Scale the vector to length 1. After normalization, dot product and cosine similarity are the same number, and search can use the cheaper dot product.
If you embed "home loan eligibility and documents required," you might store a 384-dimensional vector from a small open model or a 1536-dimensional vector from a hosted model. The dimension is a property of the model, not of the sentence. A longer sentence does not create a longer vector. It creates a different point in the same dimension.
Models you should be able to name
| Model | Typical dimension | Why people pick it |
|---|---|---|
| OpenAI text-embedding-3-small | 1536 | Strong general quality, lower cost than the large model. Dimension can be shortened. |
| OpenAI text-embedding-3-large | 3072 | Higher quality when the corpus is large and near-misses are expensive. |
| BAAI bge-base-en | 768 | Strong open English model. You run it yourself. |
| BAAI bge-small-en | 384 | Faster and cheaper, still useful for many policy corpora. |
| all-MiniLM-L6-v2 | 384 | The classic small sentence-transformer. Fine for prototypes, often beaten by BGE on retrieval. |
| Instructor embeddings | 768 | You can prepend an instruction such as 'represent this for retrieval' so the same model serves different tasks. |
| Cohere embed English v3 | 1024 | Hosted multilingual-capable family. The light variant is smaller (384). Match the dimension you index. |
OpenAI text-embedding-3-small
Typical dimension1536Why people pick itStrong general quality, lower cost than the large model. Dimension can be shortened.OpenAI text-embedding-3-large
Typical dimension3072Why people pick itHigher quality when the corpus is large and near-misses are expensive.BAAI bge-base-en
Typical dimension768Why people pick itStrong open English model. You run it yourself.BAAI bge-small-en
Typical dimension384Why people pick itFaster and cheaper, still useful for many policy corpora.all-MiniLM-L6-v2
Typical dimension384Why people pick itThe classic small sentence-transformer. Fine for prototypes, often beaten by BGE on retrieval.Instructor embeddings
Typical dimension768Why people pick itYou can prepend an instruction such as 'represent this for retrieval' so the same model serves different tasks.Cohere embed English v3
Typical dimension1024Why people pick itHosted multilingual-capable family. The light variant is smaller (384). Match the dimension you index.
Dimensions above are the defaults people mean in interviews. Always read the model card before you create the collection. If you change models, the old vectors are not comparable. You re-embed the corpus. There is no conversion that turns a 384-d MiniLM space into a 1536-d OpenAI space.
Similarity
Once both the query and the chunks are vectors, you rank chunks by a metric.
| Metric | What it measures | When it is used |
|---|---|---|
| Cosine similarity | The angle between two vectors. Ignores magnitude. | The default for text embeddings. 1 means the same direction, 0 means orthogonal, negative means opposite. |
| Euclidean distance | Straight-line distance. Smaller is closer. | Useful in some ANN indexes. Sensitive to magnitude. Less ideal as dimension grows, because distances concentrate. |
| Dot product | Sum of coordinate-wise products. | Equals cosine when both vectors are normalized. Fast. If you skip normalization, longer vectors win even when the angle is worse. |
Cosine similarity
What it measuresThe angle between two vectors. Ignores magnitude.When it is usedThe default for text embeddings. 1 means the same direction, 0 means orthogonal, negative means opposite.Euclidean distance
What it measuresStraight-line distance. Smaller is closer.When it is usedUseful in some ANN indexes. Sensitive to magnitude. Less ideal as dimension grows, because distances concentrate.Dot product
What it measuresSum of coordinate-wise products.When it is usedEquals cosine when both vectors are normalized. Fast. If you skip normalization, longer vectors win even when the angle is worse.
Cosine, written so you can say it:
cos(A, B) = (A · B) / (||A|| * ||B||)
A · B = A1*B1 + A2*B2 + ... + An*Bn
||A|| = sqrt(A1^2 + A2^2 + ... + An^2)
For normalized embeddings, ||A|| and ||B|| are 1, so cosine equals the dot product. That is why vector databases often store normalized vectors and search with dot product: same ranking, less arithmetic.
A score of 0.82 does not mean "82 percent correct." It means "this neighbor is closer than the others." You still read the chunk. Teams pick a minimum score only after they plot score distributions for known-good and known-bad questions. A global threshold copied from another product will either hide the right page or accept garbage.
A picture of the space
Two chunks can be close and both wrong for the user if the corpus has three versions of the leave policy. Geometry does not know which version is current. Metadata filters do. Embeddings find "about leave." Filters decide "the March HR edition."
A minimal embedding call
from openai import OpenAI
client = OpenAI()
text = "Home loan eligibility and documents required"
response = client.embeddings.create(
model="text-embedding-3-small",
input=text,
)
vector = response.data[0].embedding
print(len(vector)) # 1536
The same call, with the user question as input, produces the query vector. Batch your document chunks (input can be a list) during indexing. Do not send one HTTP request per chunk if the provider allows batches. Do send the query alone at request time so you are not re-embedding the handbook.
What goes wrong in practice
Embedding pitfalls
- Mixed models or mixed dimensions. The index and the query must share a space. Re-embed everything when you switch.
- Truncation. Chunks longer than the model's input window are cut. The embedding then ignores the very sentence you hoped to find.
- Domain jargon. A general model may place an internal product code far from the words employees actually type. If recall is poor after chunking is clean, try a stronger model or a domain-tuned one. Do not start there.
- Multilingual queries. An English-only MiniLM will mishandle a Hindi or mixed query. Pick a multilingual model if the workforce writes in more than one language.
- Stale vectors. The handbook changed and the point in space still represents March's words. Re-embed the changed chunks.
- Magnitude bugs. You switched the index from cosine to raw dot product but did not normalize. Longer chunks outrank better chunks.
Quick recall
Everything you need if you only revisit this box.
- Embeddings turn text into fixed-length vectors so similar meanings land nearby.
- Pipeline: preprocess, tokenize, encode, pool to one vector, often normalize.
- Know the usual dimensions: 1536 and 3072 for OpenAI's small and large v3 models, 768 and 384 for BGE base and small, 384 for MiniLM, 1024 for Cohere's full English v3.
- Cosine compares angle. Euclidean compares distance. Dot product matches cosine only when vectors are unit length.
- Similarity scores are rankings, not probabilities of correctness.
- Changing the embedding model means re-indexing. There is no compatible mix.
Test yourself
Answer these before moving on — recall is what makes it stick.