A user types "how do I get my money back?" The chunk that answers them says "refunds are issued to the original payment method within five business days." Read those two lines again: they share almost no words. "Money" isn't "refund." "Get back" isn't "issued." A keyword search, the kind that hunts for matching words, finds nothing and hands back an empty page. Yet you, reading it, matched them in about half a second.
That half-second is the whole problem embeddings solve. Your documents are written in one set of words; your users ask in another. Somewhere in between, a machine has to know that "money back" and "refund" point at the same thing, without reading either sentence the way you just did.
Here's how it manages that. An embedding turns a piece of text into a point on a map: a long list of numbers that act as its coordinates. The map is built so that texts meaning similar things land near each other. "Money back" and "refund" end up as close neighbors even though they share no letters, because the model that drew the map learned, from an enormous amount of text, that people use them the same way.
So if you keep one picture from this page, keep this: an embedding is a location in meaning-space — near means similar, far means unrelated. Searching stops being about matching words and becomes something simpler. Drop the question on the map. Look at what's nearby.
How it works
An embedding model reads a chunk of text and returns a vector — usually a few hundred to a few thousand numbers. Each number is a coordinate along some axis, and the whole list is one point in a space with that many dimensions. You can't picture a thousand dimensions, and you don't have to. The flat, two-dimensional intuition carries you the whole way: near is similar, far is different.
You don't get to decide what the axes mean. The model worked them out on its own from reading, and most of them line up with nothing a human would name. One might loosely track "is this about money." Another tracks something no word quite fits. That's fine, because the axes were never the point. The arrangement was. The model placed everything so that meaning turned into distance.
To ask "how close are these two?", the system compares their vectors, usually by the angle between them rather than the straight-line gap. Point in nearly the same direction and they score as similar. Point apart and they don't. That score is what retrieval sorts on.
Which gives you the search loop. Ahead of time, you embed every chunk and store the vectors. When a question arrives, you embed the question the same way and pull the chunk vectors closest to it. One rule holds the whole thing together: the documents and the questions must be embedded by the same model. Two models draw two different maps, and a coordinate from one map means nothing on the other. Mix them and every result is noise.
Where it breaks
The map is an approximation of meaning, and approximations have seams. Knowing where they run is most of what separates a RAG system that works from one that's wrong in ways nobody can explain.
• Opposites can look alike. An embedding captures what a text is about more reliably than what it claims. "The drug is effective" and "the drug is not effective" concern the same drug and the same trial, so they can land as neighbors — the one small word that flips the meaning barely moves the point. Negation is a known blind spot. When exact logical meaning matters, the map alone won't protect it.
• Your jargon is off the map. A general model learned from public text. It has never met your internal product names, your SKUs, your error codes, so it places them by their surroundings and often gets them wrong. The further your language sits from the public internet, the more the map warps.
• Questions and answers don't look alike. "How do I reset my password?" and "Open Settings, then Security, then Reset" are an obvious pair to you, but they're built in different shapes, and their vectors can sit further apart than you'd expect. Raw similarity quietly under-ranks the passage that answers the question.
• Exact strings get fuzzed. Meaning-search is the wrong tool when the user wants an exact match: an order ID, a function name, a version number. The map holds no special respect for v2.4.1. It will cheerfully offer v2.1.4 as a close cousin.
• A bad chunk makes a bad point. Everything here inherits the problem from the last piece. Embed a bloated, mixed chunk and its vector is the average of everything inside it, pointing vaguely at nothing. An embedding is only ever as sharp as the text you handed it.
Choosing a model (and when to fix it)
Most teams should start with a strong, current general-purpose embedding model and touch nothing else. The families you'll run into — OpenAI's embedding models, and open ones like the E5, BGE, and sentence-transformer lines — are good enough that the model is rarely your early bottleneck. Chunking and retrieval usually are.
Reach past the default only when you have a specific reason:
• Your domain is genuinely foreign (heavy internal jargon, a specialized field) and you've confirmed retrieval is failing on exactly those terms. Then a domain-tuned or fine-tuned model earns its cost. Not before.
• The exact-match failures above are hurting. Don't swap the model for this; add keyword search beside it. Running vector search and plain keyword search together, called hybrid search, covers both "find me something that means this" and "find me this exact string." It settles more retrieval complaints than any model upgrade.
• Scale is biting. Bigger vectors hold more nuance and cost more to store and search. Size the vector to how much you have to index.
Whatever you choose, measure it against real questions before and after. A new embedding model is exactly the kind of change that feels like an upgrade while quietly reshuffling which chunks win. (More on that when we reach evals.)
Where to go next
Embeddings are the second piece of the knowledge retrieval model: chunking cut the documents, embeddings turned each piece into a searchable point. The rest of the map:
• Vector databases — where millions of these vectors live, and how "find the nearest points" stays fast at scale
• Re-ranking — a second, sharper pass that fixes the "close, but not the best" ordering embeddings leave behind
• Evals — proving an embedding or model change helped, instead of hoping it did
• then the project: build a production RAG, end to end.
New here, or want the ground this stands on? Start at What is RAG, really?. The piece right before this one, Chunking, covers the cuts that become these vectors. And to watch retrieval quietly fail and then get fixed in production, that's the war story: Your RAG worked in the demo. Then real users arrived.
An embedding feels like the mathematical heart of RAG: the neural network, the thousand dimensions, the part you're not really meant to follow. It's the simplest idea in the whole stack. Put meaning on a map, and finding the right passage becomes finding what sits near the question. Everything that comes after, the databases and the re-rankers, is just machinery for searching that map faster and tidying what it returns. The map is the idea.
