Hybrid search: words plus meaning

By TryRAG · Updated

Hybrid search means using two kinds of search together: one that matches exact words and one that matches meaning. Each catches cases the other misses, and combining them is one of the most common ways to improve what a RAG system retrieves.

Two ways to search, two ways to fail

Keyword search looks for the words of the question in each chunk. It is exact and predictable. It fails when the question and the document say the same thing in different words: a search for "money back" finds nothing in a chunk that only says "refund".

Vector search, also called semantic or meaning search, compares embeddings produced by a language model. It handles rephrasing well: "money back" and "refund" land close together. It fails in the opposite direction, on exact strings. To an embedding model, the order numbers A-10437 and A-10438 look almost identical, and a rare product name it never saw in training may be placed near unrelated text.

A concrete case

Take a one-page bank form containing the line "I hold the above account with branch code: BOFAUS3N."

Real users ask all three kinds of question, which is why neither method alone is enough.

How the two are combined

There are two common approaches.

Production keyword search usually uses a formula called BM25, which improves on plain word counting by giving rare words more weight and limiting the effect of a word repeated many times.

Choosing the blend

As with chunk size, the right answer depends on the documents and the questions, so the dependable approach is to test with real questions.

Trying it

In the RAG tool, choose "Hybrid" under the document. The "Share of word matching" slider that appears mixes word matching into the meaning score, up to 100 percent (words only), and "Combine by" switches between a blend of scores and rank fusion. Each retrieved chunk shows its position in the meaning list and in the word list, and the Step by step panel has a table of the two lists side by side with the combined result. On the Evaluate tab, the auto-tuner tries keyword, semantic, three hybrid mixes and rank fusion against your test questions and recommends one.

The tool's word matching is plain word counting by default. Under Settings, Keyword scoring switches it to BM25 (with its usual constants, k1 = 1.2 and b = 0.75), for keyword search and for the word half of hybrid search, so you can compare the two on your own questions. BM25 scores are shown as a share of the best score a chunk could get for the question, so they sit between 0 and 1 like the others. Words are matched by their stem and share 4,096 slots, so in a very large document two different words can occasionally count as one.

What hybrid search does not fix

Blending helps with how chunks are matched. It does not help if the answer is split across two chunks, if the chunks are too large to match anything precisely, or if the document simply does not contain the answer. Those are covered in common RAG mistakes. When the right chunk is found but not ranked first, the usual next step is re-ranking.