Retrieval fails quietly. You ask about exit code 137 and the pipeline returns a paragraph on MemoryError, because both texts sit close together in embedding space. The model then writes a fluent answer over the wrong chunk, and nothing in the logs tells you which step went wrong.
Vector search compares meaning. It is weak on exact tokens: error codes, version strings, function names, order IDs. BM25 does the opposite, matching tokens precisely while having no idea that "process killed" and OOMKilled describe the same event. Combining the two beats either one alone, and a cross-encoder reranker on top repairs ranking mistakes that fusion cannot see because fusion only reads positions, never content.
Everything below ran while writing this: Qdrant 1.19.1 in Docker, qdrant-client 1.19.1, fastembed 0.8.1, a 16-document index, 2 vCPU. Every score and timing quoted here comes from that run.
Prerequisites
- Docker Engine 24 or newer (
docker version) - Python 3.10 or newer in a virtualenv
- Around 1 GB of free disk for the ONNX models and 2 GB of RAM
- Working knowledge of embeddings and cosine similarity
Step 1: run Qdrant
docker run -d --name qdrant-rag \
-p 6333:6333 -p 6334:6334 \
-v "$(pwd)/qdrant_storage:/qdrant/storage" \
qdrant/qdrant:latest
curl -s http://localhost:6333/
The version check returns what the server actually runs, which matters because the pieces you are about to use landed at different times:
{"title":"qdrant - vector search engine","version":"1.19.1","commit":"6ab21cac..."}
Hybrid queries with prefetch exist since v1.10.0, the k parameter for RRF since v1.16.0, and weighted RRF since v1.17.0. Anything older than 1.10 will reject the query shape in Step 5.
Step 2: client and models
python3 -m venv .venv && source .venv/bin/activate
pip install "qdrant-client[fastembed]"
FastEmbed runs the models through ONNX Runtime, so you get local embeddings and a local reranker without installing torch or a CUDA stack.
Step 3: one collection, two named vectors
from qdrant_client import QdrantClient, models
COLLECTION = "kb"
DENSE = "dense"
SPARSE = "sparse"
client = QdrantClient(url="http://localhost:6333")
client.create_collection(
collection_name=COLLECTION,
vectors_config={DENSE: models.VectorParams(size=384, distance=models.Distance.COSINE)},
sparse_vectors_config={SPARSE: models.SparseVectorParams(modifier=models.Modifier.IDF)},
)
The modifier=models.Modifier.IDF line is the part people skip. Qdrant/bm25 in FastEmbed emits term frequencies, not finished BM25 scores, and its model card marks it as requires_idf: True. The inverse document frequency half of BM25 has to be applied somewhere, and letting the server do it means adding documents later shifts those weights across the whole collection instead of freezing the values you computed at index time.
Then index the chunks:
from fastembed import TextEmbedding, SparseTextEmbedding
dense_model = TextEmbedding(model_name="BAAI/bge-small-en-v1.5")
sparse_model = SparseTextEmbedding(model_name="Qdrant/bm25")
dense_vecs = list(dense_model.embed(DOCS))
sparse_vecs = list(sparse_model.embed(DOCS))
client.upsert(
collection_name=COLLECTION,
points=[
models.PointStruct(
id=i,
vector={
DENSE: d.tolist(),
SPARSE: models.SparseVector(indices=s.indices.tolist(), values=s.values.tolist()),
},
payload={"text": t},
)
for i, (t, d, s) in enumerate(zip(DOCS, dense_vecs, sparse_vecs))
],
wait=True,
)
Embedding all 16 documents with both models took 394 ms on this box. The point IDs here are just list indices; in a real index use a content hash so a re-index updates chunks instead of duplicating them.
Step 4: query each retriever alone
This is where you learn which of the two is carrying your workload. Three queries against the same 16 documents:
| Query | Dense top-1 | BM25 top-1 |
|---|---|---|
exit code 137 |
exit-code-137 doc, cosine 0.7536 | exit-code-137 doc, BM25 16.9190 |
why does my process die with no stack trace when the app itself looks healthy |
exit-code-137 doc, cosine 0.6679 | Node.js heap doc, BM25 24.4693 |
OOMKilled |
OOMKilled doc, cosine 0.7922 | OOMKilled doc, 6.7338, one hit total |
Three things stand out. BM25 returns a single document for OOMKilled, because exactly one chunk contains that token, while dense returns five with cosine scores packed between 0.53 and 0.79 and most of them are noise about MemoryError. Dense misses the Node.js heap chunk for the paraphrase query, which is the correct answer, and BM25 finds it because the chunk literally says "stack trace". Scores from the two retrievers live on different scales and cannot be compared, added, or thresholded against each other, which is the reason fusion needs ranks rather than scores.
Step 5: fuse the two lists with RRF
dense_q = list(dense_model.embed([query]))[0].tolist()
sq = list(sparse_model.embed([query]))[0]
sparse_q = models.SparseVector(indices=sq.indices.tolist(), values=sq.values.tolist())
hits = client.query_points(
collection_name=COLLECTION,
prefetch=[
models.Prefetch(query=dense_q, using=DENSE, limit=20),
models.Prefetch(query=sparse_q, using=SPARSE, limit=20),
],
query=models.RrfQuery(rrf=models.Rrf(k=60)),
limit=5,
).points
Each prefetch runs as its own search, and the outer query merges them. Reciprocal rank fusion scores a document by summing 1 / (k + rank) over every list that returned it, so only positions matter. Qdrant's documentation notes the constant defaults to 2, and I set 60 explicitly, the value from the original RRF paper. The reason to touch it: a small k lets one list's rank-1 result outvote agreement at ranks 4 and 5 in the other list, which is exactly the situation hybrid search exists to avoid.
Real output for exit code 137:
1. 0.0333 A container killed by the kernel usually reports exit code 137 ...
2. 0.0328 The timeout command exits with status 124 when the deadline fires
3. 0.0304 Vector search compares embeddings, which capture meaning but drop exact tokens
4. 0.0161 OOMKilled is the Kubernetes container status ...
5. 0.0159 A JVM inside a container reads the cgroup limit
Those fused scores sit around 0.03 because they are summed ranks, not similarities, so treat them as ordering only.
Now the failure mode you should expect. For the paraphrase query, hybrid ranked the timeout chunk first at 0.0328 and the Node.js heap chunk second at 0.0323:
1. 0.0328 The timeout command exits with status 124 ...
2. 0.0323 Node.js sets a default old space cap around 2 GB ...
The timeout chunk was mid-table in both lists, and that agreement beat one strong result in a single list. Fusion has no way to notice the Node.js chunk is the one that actually answers the question. This is the ceiling of rank-based merging, and it is where a reranker earns its place.
Step 6: rerank the candidate pool
from fastembed.rerank.cross_encoder import TextCrossEncoder
reranker = TextCrossEncoder(model_name="Xenova/ms-marco-MiniLM-L-6-v2")
candidates = client.query_points(
collection_name=COLLECTION,
prefetch=[
models.Prefetch(query=dense_q, using=DENSE, limit=20),
models.Prefetch(query=sparse_q, using=SPARSE, limit=20),
],
query=models.RrfQuery(rrf=models.Rrf(k=60)),
limit=20,
).points
texts = [p.payload["text"] for p in candidates]
scores = list(reranker.rerank(query, texts))
order = sorted(range(len(texts)), key=lambda i: scores[i], reverse=True)[:3]
A cross-encoder reads query and chunk in one pass and outputs a relevance score, unlike the dot product between two vectors that were encoded separately. On the paraphrase query it reorders the fused list to:
1. 2.2160 Node.js sets a default old space cap around 2 GB ...
2. -10.5071 The timeout command exits with status 124 ...
3. -11.0184 A container killed by the kernel usually reports exit code 137 ...
The right chunk moved from second to first, and the wrong one dropped. For exit code 137 the reranker kept the correct chunk on top at 8.2984 and pushed weak matches down, with the gap between the first and second score landing around 15 points. Scores are unbounded in both directions and are not probabilities, so order the list and do not threshold it.
Measured latency on the same machine, per query:
| Stage | Time |
|---|---|
| Dense search, top-5 | 3 to 4 ms |
| BM25 search, top-5 | 2 ms |
| Hybrid RRF, two prefetches, top-5 | 3 ms |
| Rerank 20 candidates, MiniLM-L-6 | 199 to 250 ms |
Reranking costs two orders of magnitude more than retrieval and scales linearly with the candidate pool, so keep the pool between 20 and 50 and cut it to three or five before the prompt. Reranking a thousand chunks per query is a way to turn a fast system into a slow one.
Step 7: the retrieval function
def retrieve(query: str, k: int = 3, pool: int = 20):
dense_q = list(dense_model.embed([query]))[0].tolist()
s = list(sparse_model.embed([query]))[0]
sparse_q = models.SparseVector(indices=s.indices.tolist(), values=s.values.tolist())
candidates = client.query_points(
collection_name=COLLECTION,
prefetch=[
models.Prefetch(query=dense_q, using=DENSE, limit=pool),
models.Prefetch(query=sparse_q, using=SPARSE, limit=pool),
],
query=models.RrfQuery(rrf=models.Rrf(k=60)),
limit=pool,
).points
texts = [p.payload["text"] for p in candidates]
scores = list(reranker.rerank(query, texts))
ranked = sorted(range(len(texts)), key=lambda i: scores[i], reverse=True)[:k]
return [{"text": texts[i], "score": round(scores[i], 3)} for i in ranked]
Feed those three chunks into your prompt with their identifiers, and log the rerank scores next to the question. When an answer is wrong, the log tells you whether retrieval failed or generation failed, and those two need different fixes.
When to use what
- Dense only: natural-language questions over a large corpus where exact tokens rarely matter.
- BM25 only: logs, source code, identifiers, and any index you cannot afford to embed.
- Hybrid: the default for a mixed knowledge base. The cost is one more vector per point and one more query per search.
- Rerank: for when top-3 precision matters more than 200 ms. It cannot rescue a corpus that never contained the answer.
One correction that helps more than tuning: make chunks small enough that one chunk equals one fact. A 1500-token chunk covering six topics gives both retrievers and the reranker the same problem, because relevance becomes a property of the chunk as a whole.
Checklist
- Create the collection with two named vectors and set
Modifier.IDFon the sparse one. - Embed a handful of documents and query each retriever alone, so you know which one is doing the work.
- Fuse with
prefetchplus RRF, and decide on k instead of accepting the default. - Rerank a pool of 20 to 50 candidates with a local cross-encoder, then keep 3 to 5.
- Log fused scores and rerank scores for every query, and re-read them when answers go wrong.