What is Embedding?
Also called Vector Embedding, Text Embedding.
An embedding is a list of numbers that represents a piece of content, such as a sentence, image, or record, in a way that places related items close together in a shared coordinate space. A model produces the vector, and the distance between two vectors approximates how similar their meanings are. Embeddings make similarity computable.
An embedding model reads input and outputs a fixed length array of floating point numbers, often between a few hundred and a few thousand values. The individual numbers carry no readable meaning on their own. What matters is relative position, since content the model treats as related lands nearby and unrelated content lands far apart.
Because position encodes meaning rather than spelling, embeddings match paraphrases that share no words. A query about canceling a subscription can retrieve a passage titled ending your plan. This behavior is what separates semantic retrieval from keyword matching, and it also explains why embeddings struggle with rare identifiers, product codes, and exact numbers that carry meaning only as literal strings.
Embeddings are model specific. Vectors from two different models are not comparable, so a corpus must be embedded again when the model changes, and query and document vectors must come from the same model. Dimensionality, maximum input length, and language coverage vary between models and shape both storage cost and retrieval quality.
Beyond retrieval, embeddings support clustering, deduplication, classification, and recommendation. They also carry the biases and blind spots of their training data, and they can leak information, since an embedding of sensitive text may permit partial reconstruction. Treating stored vectors with the same care as the source text is standard practice.
Key points
- Numeric vector where distance approximates semantic similarity.
- Matches paraphrases that share no literal words.
- Vectors from different models are not comparable.
- Weak on exact codes, identifiers, and numbers.
- Also used for clustering, deduplication, and recommendation.
In practice
The phrases reset my password and I cannot log in produce vectors whose cosine similarity is high, while reset my password and quarterly revenue report produce vectors that sit far apart. A search system stores a vector for every help article, embeds the incoming question with the same model, and returns the articles whose vectors sit closest.