Hybrid search: words plus meaning

Hybrid search means using two kinds of search together: one that matches exact words and one that matches meaning. Each catches cases the other misses, and combining them is one of the most common ways to improve what a RAG system retrieves.

Two ways to search, two ways to fail

Keyword search looks for the words of the question in each passage. It is exact and predictable. It fails when the question and the document say the same thing in different words: a search for "money back" finds nothing in a passage that only says "refund".

Vector search, also called semantic or meaning search, compares embeddings produced by a language model. It handles rephrasing well: "money back" and "refund" land close together. It fails in the opposite direction, on exact strings. To an embedding model, the order numbers A-10437 and A-10438 look almost identical, and a rare product name it never saw in training may be placed near unrelated text.

A concrete case

Take a one-page bank form containing the line "I hold the above account with branch code: BOFAUS3N."

• The question "What is the branch code?" works with keyword search, because "branch" and "code" both appear in that line.
• The question "Which office identifier is on the form?" shares no words with the line. Keyword search returns nothing; meaning search can still find it.
• The question "Where is BOFAUS3N mentioned?" is the reverse. Keyword search finds the exact string at once; a meaning model may not know what to do with it.

Real users ask all three kinds of question, which is why neither method alone is enough.

How the two are combined

There are two common approaches.

• Weighted blend. Each passage gets a keyword score and a meaning score, and the final score is a mix, for example 70 percent meaning plus 30 percent keyword. This is simple and gives one setting to tune, but it assumes the two scores are on comparable scales.
• Rank fusion. Each method produces its own ranked list, and passages are scored by their positions in the two lists rather than by raw scores. A passage ranked highly by either method rises to the top. This avoids the scale problem and needs little tuning.

Production keyword search usually uses a formula called BM25, which improves on plain word counting by giving rare words more weight and limiting the effect of a word repeated many times.

Choosing the blend

• Lean towards keywords when the documents are full of codes, part numbers, names, legal references or other exact terms that people will search for directly.
• Lean towards meaning when people ask in everyday language about documents written in formal or technical language.
• Start in the middle if you do not know, then measure.

As with chunk size, the right answer depends on the documents and the questions, so the dependable approach is to test with real questions.

Trying it

In the RAG tool, turn on meaning search and open Settings. The "Word blend" slider mixes word matching into the meaning score, from 0 percent (meaning only) to 100 percent (words only), and the retrieved passages update as you move it. Step 4 of the Step by step panel shows how each score is made up. On the Tune tab, "Try many settings" tests three blend levels against your test questions and lists the best first.

The tool's word matching is plain word counting, not full BM25, so treat it as a way to understand the idea, not as a benchmark of a production system.

What hybrid search does not fix

Blending helps with how passages are matched. It does not help if the answer is split across two passages, if the passages are too large to match anything precisely, or if the document simply does not contain the answer. Those are covered in common RAG mistakes.

More guides

What is RAG? · How to choose a chunk size for RAG · What are embeddings? · What are tokens, and what do they cost? · How to test RAG retrieval · Seven common RAG mistakes