Hybrid search: words plus meaning
By TryRAG · Updated
Hybrid search means using two kinds of search together: one that matches exact words and one that matches meaning. Each catches cases the other misses, and combining them is one of the most common ways to improve what a RAG system retrieves.
Two ways to search, two ways to fail
Keyword search looks for the words of the question in each chunk. It is exact and predictable. It fails when the question and the document say the same thing in different words: a search for "money back" finds nothing in a chunk that only says "refund".
Vector search, also called semantic or meaning search, compares embeddings produced by a language model. It handles rephrasing well: "money back" and "refund" land close together. It fails in the opposite direction, on exact strings. To an embedding model, the order numbers A-10437 and A-10438 look almost identical, and a rare product name it never saw in training may be placed near unrelated text.
A concrete case
Take a one-page bank form containing the line "I hold the above account with branch code: BOFAUS3N."
- The question "What is the branch code?" works with keyword search, because "branch" and "code" both appear in that line.
- The question "Which office identifier is on the form?" shares no words with the line. Keyword search returns nothing; meaning search can still find it.
- The question "Where is BOFAUS3N mentioned?" is the reverse. Keyword search finds the exact string at once; a meaning model may not know what to do with it.
Real users ask all three kinds of question, which is why neither method alone is enough.
How the two are combined
There are two common approaches.
- Weighted blend. Each chunk gets a keyword score and a meaning score, and the final score is a mix, for example 70 percent meaning plus 30 percent keyword. This is simple and gives one setting to tune, but it assumes the two scores are on comparable scales.
- Rank fusion. Each method produces its own ranked list, and chunks are scored by their positions in the two lists rather than by raw scores. A chunk ranked highly by either method rises to the top. This avoids the scale problem and needs little tuning.
Production keyword search usually uses a formula called BM25, which improves on plain word counting by giving rare words more weight and limiting the effect of a word repeated many times.
Choosing the blend
- Lean towards keywords when the documents are full of codes, part numbers, names, legal references or other exact terms that people will search for directly.
- Lean towards meaning when people ask in everyday language about documents written in formal or technical language.
- Start in the middle if you do not know, then measure.
As with chunk size, the right answer depends on the documents and the questions, so the dependable approach is to test with real questions.
Trying it
In the RAG tool, choose "Hybrid" under the document. The "Share of word matching" slider that appears mixes word matching into the meaning score, up to 100 percent (words only), and "Combine by" switches between a blend of scores and rank fusion. Each retrieved chunk shows its position in the meaning list and in the word list, and the Step by step panel has a table of the two lists side by side with the combined result. On the Evaluate tab, the auto-tuner tries keyword, semantic, three hybrid mixes and rank fusion against your test questions and recommends one.
The tool's word matching is plain word counting by default. Under Settings, Keyword scoring switches it to BM25 (with its usual constants, k1 = 1.2 and b = 0.75), for keyword search and for the word half of hybrid search, so you can compare the two on your own questions. BM25 scores are shown as a share of the best score a chunk could get for the question, so they sit between 0 and 1 like the others. Words are matched by their stem and share 4,096 slots, so in a very large document two different words can occasionally count as one.
What hybrid search does not fix
Blending helps with how chunks are matched. It does not help if the answer is split across two chunks, if the chunks are too large to match anything precisely, or if the document simply does not contain the answer. Those are covered in common RAG mistakes. When the right chunk is found but not ranked first, the usual next step is re-ranking.