strawmANN
A deliberately unfair strawman for approximate nearest neighbour search
A Qdrant-wire-compatible vector engine in Zig, built as a performance oracle: no compatibility debt, no distribution, no product. These are its benchmark runs against Qdrant 1.19.
Headline results
Queries per second, higher is better. A ratio above 1x means strawmANN serves more. The bar runs from 0x to 5x and its tick marks 1x. parity is a difference inside the dataset's noise band; - is a row the comparison refuses, and its caption says why.
sift1m · d=128, euclid, 1M vectors
measured 2026-10-12 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| At equal recall§7.4's comparison, over 8 anchors | best at recall 0.9625, falling to 1.34x at 0.9977 | 2.40x |
|
| Saturatingclosed loop, every core busy; not compared, the full table says why | 27,947 | 14,226 | - |
| Batched16 queries per request | 28,156 | 17,217 | 1.64x |
| Exactbrute force, no index | 191 | 123 | 1.56x |
| One query at a timesingle client, latency bound | 2,142 | 2,013 | 1.06x |
| Quantized, PQrecall 0.9829 against 0.9780 | 27,153 | 19,240 | 1.41x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-sift1m.md.
These ratios are this dimension's and do not carry to another.
Full report for this run
dbpedia-openai-1m · d=1536, cosine, 1M vectors
measured 2026-10-07 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| At equal recall§7.4's comparison, over 8 anchors | best at recall 0.8936, falling to 1.06x at 0.9894 | 1.36x |
|
| Saturatingclosed loop, every core busy | 3,572 | 3,384 | parity |
| Batched16 queries per request | 3,542 | 3,441 | parity |
| Exactbrute force, no index | 13 | 10 | 1.27x |
| One query at a timesingle client, latency bound | 1,059 | 627 | 1.69x |
| Quantized, PQrecall 0.9650 against 0.9659 | 6,115 | 5,067 | 1.21x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-dbpedia-openai-1m.md.
These ratios are this dimension's and do not carry to another.
Full report for this run
laion-small-clip · d=512, cosine, 100K vectors
measured 2026-10-12 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| At equal recall§7.4's comparison, over 8 anchors | best at recall 0.9619, falling to 1.76x at 0.9977 | 1.93x |
|
| Saturatingclosed loop, every core busy | 11,341 | 10,924 | parity |
| Batched16 queries per request | 11,193 | 12,591 | 0.89x |
| Exactbrute force, no index | 321 | 318 | parity |
| One query at a timesingle client, latency bound | 1,659 | 1,662 | parity |
| Quantized, PQrecall 0.9926 against 0.9859 | 18,462 | 13,247 | 1.39x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-laion-small-clip.md.
These ratios are this dimension's and do not carry to another.
Full report for this run
h-and-m-2048-angular-filters · d=2048, cosine, 105,100 vectors
measured 2026-10-13 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| At equal recall§7.4's comparison, over 8 anchors | best at recall 0.9941, falling to 1.21x at 0.9993 | 1.65x |
|
| Saturatingclosed loop, every core busy | 4,658 | 5,673 | 0.82x |
| Batched16 queries per request | 4,649 | 5,953 | 0.78x |
| Exactbrute force, no index | 71 | 72 | parity |
| One query at a timesingle client, latency bound | 1,286 | 995 | 1.29x |
| Quantized, PQrecall 0.9980 against 0.9962 | 6,507 | 4,839 | 1.34x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-h-and-m-2048-angular-filters.md.
These ratios are this dimension's and do not carry to another.
Full report for this run
BM25 over a text index
Full-text search on BEIR corpora: Qdrant's text query
against strawmANN's, the same documents at the same BM25 scores (checked against an f64
oracle). Queries per second, higher is better; nDCG@10 is against each corpus's relevance
judgements, for the engine on each side.
scifact · BM25, 5,183 documents, 300 queries
measured 2026-10-14 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| One query at a timesingle client; nDCG@10 0.6886 / 0.6886 | 5,286 | 3,452 | 1.53x |
| Saturatingclosed loop, every core busy; nDCG@10 0.6886 / 0.6886 | 78,477 | 31,643 | 2.48x |
| Filtereda keyword filter keeping half the corpus; nDCG@10 0.3542 / 0.3542 | 69,781 | 29,546 | 2.36x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-scifact.md.
Qdrant served BM25 from optimized segments only: all 5,183 points in immutable indexes (indexing threshold 1 KB). strawmANN served it from its one text index. Every ratio in this panel is against that Qdrant index.
Full report for this run
fiqa · BM25, 57,638 documents, 648 queries
measured 2026-10-14 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| One query at a timesingle client; nDCG@10 0.2500 / 0.2500 | 2,373 | 1,447 | 1.64x |
| Saturatingclosed loop, every core busy; nDCG@10 0.2500 / 0.2500 | 24,824 | 12,674 | 1.96x |
| Filtereda keyword filter keeping half the corpus; nDCG@10 0.1616 / 0.1616 | 16,371 | 10,181 | 1.61x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-fiqa.md.
Qdrant served BM25 from optimized segments only: all 57,638 points in immutable indexes (indexing threshold 1 KB). strawmANN served it from its one text index. Every ratio in this panel is against that Qdrant index.
Full report for this run
arguana · BM25, 8,674 documents, 1,406 queries
measured 2026-10-14 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| One query at a timesingle client; nDCG@10 0.3138 / 0.3139 | 1,635 | 298 | 5.50x |
| Saturatingclosed loop, every core busy; nDCG@10 0.3138 / 0.3139 | 16,486 | 2,631 | 6.27x |
| Filtereda keyword filter keeping half the corpus; nDCG@10 0.2615 / 0.2615 | 9,166 | 2,488 | 3.68x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-arguana.md.
Qdrant served BM25 from optimized segments only: all 8,674 points in immutable indexes (indexing threshold 1 KB). strawmANN served it from its one text index. Every ratio in this panel is against that Qdrant index.
Full report for this run
quora · BM25, 522,931 documents, 10,000 queries
measured 2026-10-14 · 3 passes · gate
pass both arms
| workload | strawmANN | qdrant 1.19 | ratio |
|---|---|---|---|
| One query at a timesingle client; nDCG@10 0.7906 / 0.7905 | 1,761 | 1,450 | 1.21x |
| Saturatingclosed loop, every core busy; nDCG@10 0.7906 / 0.7905 | 19,927 | 13,792 | 1.44x |
| Filtereda keyword filter keeping half the corpus; nDCG@10 0.4334 / 0.4334 | 16,011 | 6,755 | 2.37x |
Every row, with its latency percentiles, recall and the rows the comparison
refuses, is in
comparison-quora.md.
Qdrant served BM25 from optimized segments only: all 522,931 points in immutable indexes (indexing threshold 1 KB). strawmANN served it from its one text index. Every ratio in this panel is against that Qdrant index.
Full report for this run