Hybrid search and reranking for RAG, measured: BM25, vectors, Weaviate and a cross-encoder
Build BM25, dense, RRF and Weaviate hybrid retrieval plus a cross-encoder reranker in Python, then score all of them on a labelled query set.
Most "advanced RAG" tutorials add hybrid search and a reranker, run one query, print three chunks and call it done. That proves the code runs. It does not tell you whether retrieval got better, and in my runs below one of the two upgrades made things worse.
This post builds four retrievers over the same small ops handbook: BM25 alone, dense vectors alone, hybrid fusion, and hybrid plus a cross-encoder reranker. It builds them twice, once in plain Python with rank_bm25 and sentence-transformers, and once in Weaviate 1.40 with the v4 Python client. Then it scores every variant on 25 labelled queries with recall@k and MRR. All code below ran on an Apple M5 laptop on CPU, with no API keys. There is no generation step: the LLM only sees what retrieval hands it, so retrieval is the part to measure first.
If you want the wider map of where hybrid search sits among RAG designs, read RAG architectures compared. For a one-page summary of every stage, see the RAG cheat sheet.
Why one retriever is not enough#
Dense retrieval embeds the query and each chunk into vectors and ranks by cosine similarity. It handles paraphrase well: "who gets woken up when the site is slow" lands near a chunk about paging the on-call engineer, even though the two share almost no words.
BM25 ranks by term overlap, weighted so rare terms count more. It is good at exact strings that an embedding model has never seen as a unit: error codes, config keys, header names, ticket IDs. To a small embedding model, ERR-4031 and ERR-4033 look almost the same. To BM25 they are different tokens.
Hybrid search runs both and merges the two ranked lists. A reranker then reads each (query, chunk) pair together and rescores the top few. The pipeline looks like this:
query
|
+----------+----------+
| |
BM25 top N dense top N
| |
+------ fusion -------+ RRF, or Weaviate relativeScore / ranked
|
hybrid top K
|
cross-encoder rerank (optional)
|
final top k -> LLM prompt
The test data#
The corpus is 32 short chunks from an imaginary engineering handbook: cache fallback, token expiry, payment retries, backups, alerts, Kubernetes exit codes. I wrote it so that some chunks differ only by an identifier, because real runbooks are full of those. There are four ERR-403x codes, three pod exit codes and three PAYMENTS_* config keys.
The 25 queries come in two kinds. Nine contain an exact identifier ("What does ERR-4031 mean?", "pod exit code 137"). Sixteen are paraphrases with little word overlap ("How much data can we lose in a disaster?" should find the chunk about a 5 minute recovery point objective). Each query has one relevant chunk.
An excerpt of data.py:
DOCS = {
"cache-fallback": "If the Redis cluster stops answering, the API reads straight from PostgreSQL. Latency rises from about 40 ms to 180 ms but no request fails.",
"auth-err-4031": "Error ERR-4031 means the refresh token was reused after rotation. The session is revoked and the user must sign in again.",
"db-backup": "Full database snapshots run every 6 hours and write-ahead logs ship every 5 minutes, which gives a recovery point objective of 5 minutes.",
# ... 29 more, including near-duplicates such as:
"auth-err-4033": "Error ERR-4033 means the token was issued for a different tenant. The session is revoked and an alert goes to security.",
}
QUERIES = [
("What does ERR-4031 mean?", "auth-err-4031"),
("How much data can we lose in a disaster?", "db-backup"),
# ... 23 more
]
Twenty-five queries is small. One query moving changes recall@1 by 0.04, so treat differences of one or two queries as noise. The point is the method, and the method scales to the 100 to 300 queries you should write for a real system. The RAG evaluation pipeline post covers how to build that labelled set and how to score the generation half too.
Setup#
python3 -m venv .venv
.venv/bin/pip install weaviate-client sentence-transformers rank_bm25
That installed weaviate-client 4.24.1, sentence-transformers 6.3.0 and rank-bm25 0.2.2. The two models are all-MiniLM-L6-v2 for embeddings (384 dimensions) and ms-marco-MiniLM-L6-v2 as the cross-encoder. Both download on first use, about 90 MB each.
Version 1: plain Python#
search.py holds all four retrievers:
import re, numpy as np
from rank_bm25 import BM25Okapi
from sentence_transformers import SentenceTransformer, CrossEncoder
from data import DOCS
IDS = list(DOCS)
TEXTS = [DOCS[i] for i in IDS]
def tokenize(text):
# keep identifiers like ERR-4031 and PAYMENTS_RETRY_MAX as single tokens
return re.findall(r"[a-z0-9][a-z0-9_\-]*", text.lower())
bm25 = BM25Okapi([tokenize(t) for t in TEXTS])
embedder = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
doc_vecs = embedder.encode(TEXTS, normalize_embeddings=True)
reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
def bm25_search(q, k=10):
scores = bm25.get_scores(tokenize(q))
# drop zero-score docs: BM25 found no shared term, so they are not matches
return [IDS[i] for i in np.argsort(-scores)[:k] if scores[i] > 0]
def dense_search(q, k=10):
qv = embedder.encode(q, normalize_embeddings=True)
return [IDS[i] for i in np.argsort(-(doc_vecs @ qv))[:k]]
def rrf(rankings, k=60, top=10):
scores = {}
for ranking in rankings:
for rank, doc_id in enumerate(ranking, start=1):
scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
return sorted(scores, key=scores.get, reverse=True)[:top]
def hybrid_search(q, k=10):
return rrf([bm25_search(q, 20), dense_search(q, 20)], top=k)
def rerank(q, candidate_ids, k=10):
scores = reranker.predict([(q, DOCS[d]) for d in candidate_ids])
order = np.argsort(-scores)
return [candidate_ids[i] for i in order][:k]
def hybrid_rerank(q, k=10, pool=10):
return rerank(q, hybrid_search(q, pool), k)
Two details here bite people. First, the tokenizer. A default split on non-word characters turns ERR-4031 into err and 4031, which weakens the exact match you added BM25 for. Second, the zero-score filter. rank_bm25 returns a score for every document, so a plain argsort hands RRF a list of documents that share no term with the query, ranked by nothing. My first run skipped that filter and hybrid scored worse because of it.
Reciprocal rank fusion comes from Cormack, Clarke and Buettcher (SIGIR 2009). Each document gets 1 / (k + rank) from every list it appears in, with k = 60 as in the paper. It ignores raw scores, so you never have to put BM25 scores and cosine similarities on one scale.
The metric code in evaluate.py:
def metrics(search_fn, k_values=(1, 3, 5)):
hits = {k: 0 for k in k_values}
rr = 0.0
for q, gold in QUERIES:
ranked = search_fn(q, 10)
if gold in ranked:
pos = ranked.index(gold) + 1
rr += 1 / pos
for k in k_values:
hits[k] += pos <= k
n = len(QUERIES)
return {f"R@{k}": hits[k] / n for k in k_values} | {"MRR@10": rr / n}
Recall@k is the share of queries whose relevant chunk is in the top k. MRR@10 averages 1 / rank of that chunk, counting 0 when it is missing from the top 10. With one relevant chunk per query these are the right two numbers: recall@k tells you if the answer reaches the prompt, MRR tells you how high.
The output:
system R@1 R@3 R@5 MRR@10
bm25 0.56 0.72 0.72 0.645
dense 0.72 0.92 0.96 0.823
hybrid_rrf 0.68 0.80 0.80 0.739
hybrid+rerank 0.80 0.84 0.84 0.826
Hybrid with RRF scored below dense alone. The per-query ranks show why (columns are BM25, dense, hybrid, rerank; - means not in the top 10):
1 4 1 1 What does ERR-4031 mean?
1 2 1 1 pod exit code 137
- 1 - - Can I ship code during the holidays?
- 1 - - What happens if the recommendation model is too slow?
The identifier queries behave as promised. Dense put ERR-4031 at rank 4 behind its look-alike siblings, and hybrid pulled it back to 1. The paraphrase queries break. BM25 finds nothing useful for "holidays", but it still returns a few weak matches on words like "code". Under RRF a document at rank 10 in both lists scores 2 / 70 = 0.029, while dense's correct rank 1 document, absent from the BM25 list, scores 1 / 61 = 0.016. Any document that shows up in both lists, even near the bottom, outranks a document that only one retriever found at the top.
With 32 documents and a depth of 20, the dense list contains almost the whole corpus, so this happens on every query where BM25 finds weak matches. Cutting the depth of each input list fixes much of it:
rrf depth 20 0.68 0.80 0.80 0.739
rrf depth 5 0.68 0.88 0.96 0.793
rrf depth 3 0.68 0.88 0.96 0.800
On a corpus of millions of chunks, depth 20 or 50 is a small slice and the effect is milder. The lesson still holds. RRF treats both retrievers as equally trustworthy, and when one of them has nothing to say, it adds noise rather than staying quiet.
Version 2: Weaviate hybrid search#
Weaviate runs BM25 and vector search inside one query and fuses them on the server. Start a single node with Docker (the Docker install guide has a compose file if you prefer that):
docker run -d --name wv-blog -p 8080:8080 -p 50051:50051 \
-e AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=true \
-e PERSISTENCE_DATA_PATH=/var/lib/weaviate \
-e DEFAULT_VECTORIZER_MODULE=none \
-e CLUSTER_HOSTNAME=node1 \
cr.weaviate.io/semitechnologies/weaviate:1.40.7
Port 50051 is gRPC, which the v4 client uses for queries. I send my own vectors instead of using a vectorizer module, so the dense side is the same MiniLM model as version 1 and the comparison stays fair. If you want Weaviate to call Ollama for embeddings, the local RAG with Ollama post covers running the embedding model; Weaviate's text2vec-ollama module points at that same server.
Creating the collection and loading data with the v4 client:
import weaviate
from weaviate.classes.config import Configure, Property, DataType
from weaviate.classes.query import HybridFusion, MetadataQuery
client = weaviate.connect_to_local() # localhost:8080 (REST) and 50051 (gRPC)
runbook = client.collections.create(
name="Runbook",
vector_config=Configure.Vectors.self_provided(), # we send our own vectors
properties=[Property(name="doc_id", data_type=DataType.TEXT),
Property(name="content", data_type=DataType.TEXT)],
)
vectors = embedder.encode(list(DOCS.values()), normalize_embeddings=True)
with runbook.batch.fixed_size(batch_size=100) as batch:
for (doc_id, text), vec in zip(DOCS.items(), vectors):
batch.add_object(properties={"doc_id": doc_id, "content": text},
vector=vec.tolist())
print("failed objects:", len(runbook.batch.failed_objects),
"| count:", runbook.aggregate.over_all(total_count=True).total_count)
Older tutorials pass vectorizer_config= to collections.create. Current clients use vector_config= with the Configure.Vectors helpers; check the collection config reference if your client version differs. Check failed_objects after every batch. A batch that partly fails does not raise.
A hybrid query sends the text for BM25 and the vector for the dense side:
q = "What does ERR-4031 mean?"
res = runbook.query.hybrid(
query=q,
vector=embedder.encode(q, normalize_embeddings=True).tolist(),
alpha=0.5, limit=3,
return_metadata=MetadataQuery(score=True, explain_score=True))
for o in res.objects:
print(f"{o.metadata.score:.3f} {o.properties['doc_id']}")
print(res.objects[0].metadata.explain_score)
failed objects: 0 | count: 32
0.881 auth-err-4031
0.500 auth-err-4033
0.464 auth-err-4030
Hybrid (Result Set keyword,bm25) Document 3cf9...: original score 4.21371, normalized score: 0.5 -
Hybrid (Result Set vector,hybridVector) Document 3cf9...: original score 0.48810434, normalized score: 0.38140234
explain_score shows the fusion arithmetic. With the default relativeScoreFusion (the default since Weaviate 1.24, per the hybrid search concepts page), each result set is min-max scaled so its best hit is 1 and its worst is 0, then weighted by alpha. The BM25 side gave this chunk the top raw score, 4.21, so it scales to 1.0 and becomes 0.5 after the (1 - alpha) weight. Its vector similarity of 0.488 scaled to 0.763 and became 0.381 after the alpha weight. The total is 0.881. Alpha 0 is pure BM25, alpha 1 is pure vector.
The other option, rankedFusion, scores by rank position only, like RRF. The fusion algorithms post explains the difference. The docs also warn that the effective alpha when you leave it unset depends on client and server versions, so set it explicitly.
The sweep:
def wv_hybrid(alpha, fusion):
def fn(q, k):
res = runbook.query.hybrid(
query=q,
vector=embedder.encode(q, normalize_embeddings=True).tolist(),
alpha=alpha, fusion_type=fusion, limit=k)
return [o.properties["doc_id"] for o in res.objects]
return fn
for fusion in (HybridFusion.RELATIVE_SCORE, HybridFusion.RANKED):
for alpha in (0.0, 0.25, 0.5, 0.75, 1.0):
m = metrics(wv_hybrid(alpha, fusion))
fusion alpha R@1 R@3 R@5 MRR@10
RELATIVE_SCORE 0.0 0.60 0.76 0.76 0.687
RELATIVE_SCORE 0.25 0.56 0.92 0.96 0.741
RELATIVE_SCORE 0.5 0.56 0.92 0.96 0.750
RELATIVE_SCORE 0.75 0.80 0.96 0.96 0.873
RELATIVE_SCORE 1.0 0.72 0.92 0.96 0.823
RANKED 0.0 0.60 0.76 0.76 0.687
RANKED 0.25 0.68 0.80 0.92 0.778
RANKED 0.5 0.68 0.80 0.96 0.774
RANKED 0.75 0.56 0.76 0.96 0.707
RANKED 1.0 0.72 0.92 0.96 0.823
Relative score fusion at alpha 0.75 is the best retriever in this post: recall@1 0.80, recall@3 0.96, MRR 0.873, against 0.72 / 0.92 / 0.823 for dense alone. It keeps the identifier wins and loses fewer paraphrase queries than RRF did, because a BM25 list with weak, flat scores contributes little after min-max scaling when the vector side has a clear winner. Rank-based fusion cannot see that a BM25 match is weak, which is why RANKED and my RRF both trail.
Weaviate's BM25 at alpha 0 (MRR 0.687) differs from rank_bm25 (0.645). Weaviate's default word tokenizer and BM25 parameters are not the same as mine, which is one more reason to measure your own stack instead of trusting a number from elsewhere.
Adding a cross-encoder reranker#
A bi-encoder embeds query and chunk separately. A cross-encoder reads them together in one forward pass and outputs a relevance score, which is more accurate per pair and far too slow to run over a whole corpus. So it only rescores the top few hybrid results (usage docs):
def retrieve(q, k, alpha=0.75):
res = runbook.query.hybrid(query=q, vector=embedder.encode(q, normalize_embeddings=True).tolist(),
alpha=alpha, fusion_type=HybridFusion.RELATIVE_SCORE, limit=k)
return [o.properties["doc_id"] for o in res.objects]
for pool in (5, 10, 20):
m = metrics(lambda q, k: rerank(q, retrieve(q, pool), k))
alpha .75 + rerank pool 5 0.72 0.92 0.96 0.810
alpha .75 + rerank pool 10 0.72 0.88 0.96 0.805
alpha .75 + rerank pool 20 0.72 0.92 0.96 0.822
hybrid only: 9.3 ms/query | hybrid + rerank 10: 23.2 ms/query
The reranker made the best retriever worse at every pool size, and it more than doubled latency. Here is where its ranks differ from the hybrid ranks:
2 -> 1 pod exit code 137 (rerank top1: k8s-oom)
3 -> 1 Why was a customer logged out after using an old refresh token? (rerank top1: auth-err-4031)
1 -> 3 How much data can we lose in a disaster? (rerank top1: cache-retry)
2 -> 3 Who gets woken up when the site is slow? (rerank top1: db-replicas)
1 -> 2 Can I ship code during the holidays? (rerank top1: k8s-exit-143)
1 -> 5 Where should API keys and passwords be kept? (rerank top1: obs-logs)
1 -> 4 What happens if the recommendation model is too slow? (rerank top1: db-replicas)
It fixed two queries and broke five. The ones it broke need a domain hop: "data we can lose" means recovery point objective, "recommendation model" means "ranking models". This cross-encoder was trained on MS MARCO, web search questions over web passages. On short ops text that hop is outside what it learned, and it falls back on surface overlap.
In the plain Python version, the same reranker on top of RRF lifted recall@1 from 0.68 to 0.80, because RRF had left more correct chunks at ranks 2 to 5 for it to promote. A reranker helps most when the first stage has good recall and poor ordering. When the first stage already orders well, a weak reranker can only shuffle.
What I would try next, without claiming results I did not measure: a larger reranker (the L12 MiniLM variant or a BGE reranker), a reranker fine-tuned on a few hundred of your own labelled pairs, or a threshold that only lets the reranker override the hybrid order when its score gap is large.
Query rewriting, briefly#
The paraphrase misses above are also a case for query rewriting: have an LLM turn "how much data can we lose" into "recovery point objective, backup frequency" before retrieval, or generate a hypothetical answer and embed that. It costs an LLM call per query, so measure it the same way, on the same labelled set. The RAG cheat sheet lists the common rewriting patterns.
Failure modes to check in your own stack#
- Tokenizer splits your identifiers. Search for one of your own error codes with BM25 alone. If it misses, change the tokenizer (Weaviate lets you set
tokenizationper property, for examplefieldorwhitespace). - Zero-score BM25 results in fusion. If you build RRF yourself, filter them out.
- Fusion depth too large relative to the corpus. Sweep it.
- Alpha left at the default. Sweep 0 to 1 on your own queries; 0.75 won here, and the curve was not smooth.
- Reranker trained on a different domain. Run it with and without, per query, before paying the latency.
- Silent batch failures. Check
failed_objectsand the object count after every import. - Evaluating on five queries. You will tune to noise. Twenty-five is the floor for a demo; aim for a few hundred.
Clean up#
docker rm -f wv-blog
docker rmi cr.weaviate.io/semitechnologies/weaviate:1.40.7
rm -rf ~/.cache/huggingface/hub/models--sentence-transformers--all-MiniLM-L6-v2 \
~/.cache/huggingface/hub/models--cross-encoder--ms-marco-MiniLM-L6-v2
On this data the ranking was: Weaviate relative score fusion at alpha 0.75, then dense alone, then any combination with this particular reranker. Your corpus will give a different order, and the labelled set plus the sweep above is how you find it.
Sources#
- Weaviate: hybrid search how-to
- Weaviate: hybrid search concepts and fusion methods
- Weaviate blog: hybrid search fusion algorithms
- Weaviate: collection configuration reference
- Weaviate: Docker installation
- weaviate-client on PyPI and GitHub
- Cormack, Clarke, Buettcher: Reciprocal Rank Fusion (SIGIR 2009)
- rank_bm25 on GitHub
- sentence-transformers/all-MiniLM-L6-v2
- cross-encoder/ms-marco-MiniLM-L6-v2
- Sentence Transformers: cross-encoder usage