Hybrid Search BM25 Plus Vectors

Vector search alone misses exact matches. BM25 alone misses semantics. Hybrid search — both, fused — is the default, not the upgrade, for any RAG pipeline that has to work in production.

The RAG community has a recurring debate: BM25 or vectors. The production answer is both. Each catches what the other misses, and fusion is the cheap mechanism that combines them.

• **BM25.** Exact terms, rare terms, names, IDs, codes. The things vectors average away.
• **Vectors.** Semantics, paraphrase, intent. The things BM25 can't match without the exact term.

A pipeline that ships only one misses the other's wins — and the misses show up on the queries your users actually run.

Hybrid search is a design, not a toggle. The fusion — typically reciprocal rank fusion — combines the rankings from BM25 and vectors into one ordered result. The design question is the fusion weights and the rerank step, not which retriever to pick.

• Run both retrievers.
• Fuse the rankings (reciprocal rank fusion is the cheap default).
• Rerank the fused top-K with a cross-encoder.

It's the default because the cost is low and the wins are real. BM25 is cheap; vectors are cheap; fusion is cheap. The pipeline that ships hybrid pays slightly more and catches the queries that either alone misses — which is most production traffic.

Hybrid search must be evaled against single-retriever baselines on the same eval set. The hybrid win is the recall and precision delta — and if the delta isn't there on your data, hybrid isn't justified. On most real data, it is.

Hybrid search is the production default because BM25 and vectors catch different things, and fusion is the cheap mechanism that combines them. Run both, fuse, rerank — and eval against either alone to confirm the win.