strawmANN benchmark report

strawmANN against Qdrant · AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M (24 logical) · generated 2026-09-29 20:15:09 CEST

Dataset: sift1m
Vectors: 1,000,000 × 128 · 10,000 held-out queries
Metric: euclid
Storage datatype: strawmANN float32 · Qdrant float32 (default)
Quantization: none (fp32) · separate rows measure binary (1 bit/dim), PQ (product), SQ8 scalar

strawmANN: 43/43 ok Qdrant: 43/43 ok

Summary

At equal recall, strawmANN serves 1.81x to 2.16x Qdrant's throughput. That range spans the recall levels measured; allowing for the uncertainty in each recall figure widens it to 0.86x to 5.14x.

throughput at equal recall

1.81 to 2.16x

strawmANN over Qdrant, recall@10 0.909 to 0.999

p99 latency at 90% load

strawmANN1.41 ms

Qdrant5.27 ms

W4-sat90, open loop, one offered rate

upload and index build

strawmANN42 s

Qdrant80 s

W1 + W2

peak memory (RSS)

strawmANN5.2 GiB

Qdrant5.2 GiB

largest before the concurrent-write rows (W11); of it, anonymous 2.1 GiB / 833.8 MiB

written to disk

strawmANN2.5 GiB

Qdrant12.2 GiB

before the concurrent-write rows (W11); they wrote 122.6 MiB / 10.1 GiB more

Where strawmANN is slower, past the noise floor:

search, fp32, single query (W3): 0.85x, strawmANN at recall 0.9888, Qdrant at 0.9875
quantized: scalar (W6, 2 settings): 0.94x

Throughput against recall

Read it vertically: at any recall both engines reach, the higher curve is faster. Throughput from bfb W10, recall from the conformance sweep, joined on ef. ef is the same unit here: Qdrant held 2 segments for this run, one populated and 1 empty (read back per segment from the engine), and strawmANN held one graph, so equal ef is equal traversal width and the curves may be read at equal x. That is a property of this run, not of the engines.

Every figure is the median of 3 passes per engine, run alternately (A/B/A/B). What this does not establish.

Glossary

recall@10: Of the ten nearest neighbours a query really has, the share the engine returned. 1.0 is a perfect answer. "Really has" is settled by an exhaustive fp64 search, not by the other engine.
ef: The candidate-list size: how many nodes the index keeps in play while searching. Larger is slower and more accurate. It does not mean the same amount of work in both engines, which is why the headline holds recall fixed instead.
MRDE: Mean relative distance error: when the engine returns a wrong neighbour, how much further away it is than the right one. Small numbers mean the misses were near-misses.
qps: Queries per second. On a batched row one request carries several queries, and the request rate is shown under it.
p50 / p99 / p99.9: The latency half of requests beat, that 99% beat, that 99.9% beat. The tail is what a user notices.
closed / open loop: A closed loop sends the next request only when the last one comes back, so a slow server receives less work and its tail looks better than it is. An open loop sends at a fixed rate regardless.
noise floor: How much a number moves between identical runs on this machine. A ratio inside it is shown grey with ≈: no measured difference. Hover a ratio to see its band.
conformance tier: What the two engines were shown to agree on before any speed was quoted. T1 licenses a single engine's own numbers; T3 licenses comparing the two, and requires their recall to be statistically indistinguishable.
§ numbers: Sections of docs/spec.md, the written rule each claim in the appendix is measured against. §7.1 is the host gate, §7.4 the comparison rules, §8 what may be published.

Throughput

Queries per second; higher is better. The ratio is strawmANN over Qdrant: green is faster, red slower, grey ≈ inside the noise floor (hover a ratio for its band). A dash means the pair is not compared, and the note says why. An ef sweep is one row showing its range; every measurement is under All throughput rows.

workload		strawmANN	Qdrant	ratio	notes
W3	search, fp32, single query	1,702	2,008	0.85x
W4	search, saturating (closed loop)	23,059	14,135	1.63x
W5	search batched (16 distinct dataset queries per request)	23,4001,437 requests/s	4,859304 requests/s	4.82x
W6	quantized: scalar	4,393	4,690	0.94x
W6 ef 32 to 512	SQ8 recall control (latency only)	1,835 to 9,375	1,830 to 7,754	0.94x	3 of 5 points at parity 1 of 5 points not compared recall differs
W7	quantized: binary + oversampling	4,508	4,655	-	recall below the usable floor: 0.0618 and 0.0660, both under 0.50 — this row characterises the encoding, not the engines
W8	quantized: PQ	4,490	4,165	1.08x
W9	exact / brute force	186	124	1.50x
W10 ef 32 to 512	recall control (latency only)	6,622 to 40,820	3,963 to 22,022	1.67 to 2.03x	1 of 5 points not compared recall differs
W12-sel1	filtered search, one keyword (~1% of bench12)	5,807	3,884	1.49x
W12-sel10	filtered search, any of 10 keywords (~10% of bench12)	1,359	2,222	-	recall differs: 1.0000 vs 0.9893
W12-sel1 ef 32 to 512	filtered recall control, one keyword (latency only)	5,996 to 6,419	1,701 to 6,432	1.19 to 3.66x	1 of 5 points at parity
W12-sel10 ef 32 to 512	filtered recall control, any of 10 keywords (latency only)	1,358 to 3,172	798 to 4,685	1.02 to 1.70x	3 of 5 points not compared recall differs
W13	scroll / pagination	51,192	46,560	1.10x	strawmANN: 92% is client and socket, not server Qdrant: 82% is client and socket, not server
W11-steady	mixed read/write below the rebuild threshold: search bench2 while 50,000 synthetic points append	18,228	5,537	-	search-during-write; no recall join
W11	mixed read/write: search bench2 while 200,000 synthetic points append (runs last)	8,0333,800 requests/s	2,387	-	search-during-write; no recall join

Throughput by workload

Higher is better. Hatched bars are rows the table does not compare (W7, W11): each bar is that engine's own rate, and the pair is not a result.

Recall

recall@10 against an exact fp64 search, not against the other engine. Equal ef is not equal work in the two engines, so the comparison that counts holds recall fixed and compares throughput there.

At matched recall

recall@10	measured at	strawmANN q/s	Qdrant q/s	strawmANN / Qdrant	range over recall CI
0.9089	strawmANN at ef=32, Qdrant interpolated	40,820	20,768	1.97x	1.88 – 2.06x
0.9603	Qdrant at ef=64, strawmANN interpolated	31,958	15,494	2.06x	2.02 – 2.11x
0.9639	strawmANN at ef=64, Qdrant interpolated	31,425	14,665	2.14x	1.93 – 2.38x
0.9875	Qdrant at ef=128, strawmANN interpolated	21,066	10,159	2.07x	1.99 – 2.16x
0.9888	strawmANN at ef=128, Qdrant interpolated	20,608	9,553	2.16x	1.68 – 2.77x
0.9967	Qdrant at ef=256, strawmANN interpolated	12,622	6,591	1.91x	1.76 – 2.08x
0.9974	strawmANN at ef=256, Qdrant interpolated	12,089	5,747	2.10x	0.86 – 5.14x
0.9992	Qdrant at ef=512, strawmANN interpolated	7,162	3,963	1.81x	1.51 – 2.17x

How this is computed

Interpolated linear in log(q/s) between the two bracketing measurements, nothing extrapolated past the measured range. Throughput is bfb's W10 sweep, recall the conformance sweep, joined on ef — same m and ef_construct, but separate builds, so the pairing assumes two builds with identical parameters are equivalent.

The last column is why the ratio is not a result on its own. The ratio treats the anchor's recall as exact; it is an estimate, and moving it across its 95% interval moves the interpolated rate with it — near the top of the sweep 0.003 of recall spans a factor of 1.8, so two decimals there quote the interpolation. Even that is the narrow reading: it moves one recall and not the other, adjacent anchors share bracketing segments, and the throughputs behind it are medians of the run's passes.

At matched recall, SQ8

The same reading over the scalar-quantized collection. Under the `pool` policy both engines rescore `ef` candidates, so W6 is compared row by row as well; held at equal recall, the reading does not depend on the rows landing in one recall band. Note where each engine's curve stops, because a recall only one of them reaches is the more useful fact about an encoding than any ratio.

recall@10	measured at	strawmANN q/s	Qdrant q/s	strawmANN / Qdrant	range over recall CI
0.9087	strawmANN at ef=32, Qdrant interpolated	9,375	7,461	1.26x	1.23 – 1.28x
0.9600	Qdrant at ef=64, strawmANN interpolated	6,578	6,179	1.06x	1.03 – 1.10x
0.9646	strawmANN at ef=64, Qdrant interpolated	6,372	5,896	1.08x	1.04 – 1.12x
0.9871	Qdrant at ef=128, strawmANN interpolated	4,535	4,684	0.97x	0.93 – 1.00x
0.9890	strawmANN at ef=128, Qdrant interpolated	4,411	4,321	1.02x	0.93 – 1.12x
0.9968	Qdrant at ef=256, strawmANN interpolated	3,090	3,069	1.01x	0.95 – 1.07x
0.9974	strawmANN at ef=256, Qdrant interpolated	3,013	2,724	1.11x	0.88 – 1.39x
0.9993	Qdrant at ef=512, strawmANN interpolated	1,969	1,830	1.08x	0.93 – 1.24x

How this is computed

Interpolated linear in log(q/s) between the two bracketing measurements, nothing extrapolated past the measured range. Throughput is bfb's W6 sweep, recall the conformance sweep, joined on ef — same m and ef_construct, but separate builds, so the pairing assumes two builds with identical parameters are equivalent.

The last column reads as it does in the fp32 table above.

At matched recall, filtered to 10%

The same reading under a keyword filter over the points the condition matched, 19,882 to 20,160 across 3 builds in strawmANN and 19,933 to 20,042 across 3 builds in Qdrant (bfb draws the keyword payloads unseeded at each upload). The per-row table refuses W12-sel10 a ratio because the two engines land just outside the recall band at the one ef it measures; held at equal recall instead, the comparison exists at every recall both engines reach. Note the shape rather than any single number: one engine's curve is flat in ef and the other's is steep, so where you match decides the ratio.

recall@10	measured at	strawmANN q/s	Qdrant q/s	strawmANN / Qdrant	range over recall CI
0.9777	strawmANN at ef=32, Qdrant interpolated	3,172	2,427	1.31x	1.27 – 1.34x
0.9893	Qdrant at ef=128, strawmANN interpolated	2,417	2,226	1.09x	1.03 – 1.15x
0.9954	strawmANN at ef=64, Qdrant interpolated	2,098	1,616	1.30x	1.21 – 1.40x
0.9992	Qdrant at ef=256, strawmANN interpolated	1,470	1,325	1.11x	1.04 – 1.18x
1.0000	Qdrant at ef=512, strawmANN interpolated	1,364	798	1.71x	1.66 – 1.76x

How this is computed

Interpolated linear in log(q/s) between the two bracketing measurements, nothing extrapolated past the measured range. Throughput is bfb's W12-sel10 sweep, recall the conformance sweep, joined on ef — same m and ef_construct, but separate builds, so the pairing assumes two builds with identical parameters are equivalent.

The last column reads as it does in the fp32 table above.

Recall by `ef`

ef	strawmANN				Qdrant
ef	recall@1	recall@10	recall@100	MRDE	recall@1	recall@10	recall@100	MRDE
ef=32	0.9455	0.9089	-	3.14e-03	0.9360	0.8986	-	3.58e-03
ef=64	0.9802	0.9639	-	1.13e-03	0.9771	0.9603	-	1.25e-03
ef=128	0.9954	0.9888	0.9509	3.28e-04	0.9936	0.9875	0.9440	3.34e-04
ef=256	0.9994	0.9974	0.9855	1.32e-04	0.9983	0.9967	0.9835	7.66e-05
ef=512	0.9999	0.9995	0.9968	9.10e-06	0.9995	0.9992	0.9961	2.26e-05

How recall was measured

10,000 held-out queries, limit 10, ε=4.172e-07, against the fp64 oracle rather than against the other engine. Base checksum 7bf7ead7809a45cd: the same corpus the latency rows were measured on. Each pass rebuilds the collection. Across 3 independent builds of it, recall@10 at ef=512 spread by: strawmANN 0.00611; Qdrant 0.00017. strawmANN's graphs disagree with each other 36x as much as Qdrant's do, so a ratio read at the top of this table reports which graph the build produced as much as which engine built it. Over the same builds the engine reported unreachable nodes: strawmANN 0 of 200,000 and 0 of 1,000,000 — a node nothing points at is invisible at any ef, so that is where a graph-quality difference shows rather than being inferred from the recall beside it. Level seed: strawmANN 0x57ea3111. Recorded as provenance: under this engine's level draw, six builds at four seeds sit within 0.00003 of recall@10 at ef 512, well inside the spread above.

What each quantization costs in recall

Colour is the encoding here, not the engine — the engines are the line style. Each sweep is joined on the quantization parameters it actually sent, so a curve speaks only for the search its row ran. Higher is better, and the encodings are not free: read this against the throughput their rows bought.

Latency

Client-side round trip. A closed loop understates the tail (a stalled server stops receiving requests), so this keeps the fixed-rate, open-loop rows, which offer the same load to both engines, plus single-client W3 and batched W5. Every row is under All latency rows.

workload	strawmANN			Qdrant
workload	p50	p99	p99.9	p50	p99	p99.9
W3 search, fp32, single query	598 µs	715 µs	792 µs	505 µs	594 µs	666 µs
W4-sat50 search, fixed rate at 50% of saturation (open loop)	493 µs	1.01 ms	1.56 ms	753 µs	1.83 ms	2.32 ms
W4-sat70 search, fixed rate at 70% of saturation (open loop)	532 µs	1.12 ms	1.22 ms	838 µs	2.02 ms	2.87 ms
W4-sat90 search, fixed rate at 90% of saturation (open loop)	718 µs	1.41 ms	1.53 ms	1.24 ms	5.27 ms	9.40 ms
W5 search batched (16 distinct dataset queries per request)	1.36 ms	1.52 ms	2.63 ms	6.56 ms	7.10 ms	7.74 ms

Ingest and index build

Qdrant indexes while it ingests, so its upload time already contains most of the indexing; strawmANN uploads raw and builds afterwards. Compare the sum, not the upload line.

Where the load time goes

Solid is upload, hatched is the wait for Green. The bar's whole length is the sum this section asks you to compare; the split is why the upload line alone is not comparable between these engines.

workload		strawmANN	Qdrant
W0-upload	d=4 floor: graph traversal with the distance taken out	19.06 s	26.06 s
W1	ingest throughput (no index wait)	1.08 s	11.83 s
W2	index build, time to Green	41.13 s	68.16 s
W6-upload	search, SQ8 scalar quantization	41.12 s	53.12 s
W7-upload	search, binary quantization	41.13 s	49.12 s
W8-upload	search, product quantization	50.16 s	88.20 s
W12-upload	upload	9.03 s	30.08 s
W1 + W2	upload and index, together	42.2 s	80.0 s	1.89x

Memory and disk

What each engine held in memory and moved to and from disk over the whole run. Per-row figures are under Storage and I/O.

	strawmANN	Qdrant
storage on disk	3.1 GiB	2.9 GiB
peak memory (RSS)	5.2 GiB	5.2 GiB
read from disk	16.0 KiB	1.9 GiB
written to disk	2.5 GiB	12.2 GiB

Peak memory and storage on disk are read before the concurrent-write rows (W11): an engine rewriting segments maps old and new files at once, and RSS counts each mapping. Bytes written are summed over the same rows, since W11's volume is set by the harness's write rate; the other disk figures are totals over every row.

Appendix

How the run was set up, every row of every table, and the diagnostics behind them. Charts on a logarithmic axis say so on the axis: read the positions there, not the distances.

Run conditions and conformance

Measured as-deployed: the scheduler was left as it ships, so a difference is what a user would see rather than the engine in isolation, and ambient load is part of the measurement. Qdrant ran equal-work: asked for one populated graph (--segments 1), as strawmANN serves, so ef means the same thing on both sides. This is the configuration §8's comparative licensing is built around, and it is a control rather than a deployment. Its optimizer treats the count as a target; what it actually held is read back from the engine and stated under Collection config.

Conformance tier reached: T4 quantization fidelity. T1 passed, so single-engine performance rows are licensed (§8). T3 passed, so a strawmANN-vs-Qdrant throughput comparison is licensed: the engines are at equal recall (§7.4). Conformance hash 0ace76ed961b7820.

On exact search the two engines' scores differ by at most 0.000e+00 (p99 0.000e+00), against a calibrated relative ε of 4.172e-07. §8.1: bit-exactness is unachievable between two different summation orders, so this — equal values within a measured tolerance — is the claim the speed rests on.

The conformance harness's single-client rate agrees on the ordering (3.54x to 3.78x); a different instrument, so no ratio is formed from it.

Every figure is the median of 3 passes per row per engine, alternated A/B/A/B (§7.4), and the spread is this run's own over 36 of 43 rows. The ingest and index-build rows (W0-upload, W1, W2, W6-upload, W7-upload, W8-upload, W12-upload) have none, so no build time carries a verdict. Each pass rebuilds the index, so the bands include build variance and are wider than a floor measured against one standing graph.

Units are queries per second. bfb reports rps, which counts batch requests: at --search-batch-size 16 the two differ by 16x, and reading one as the other once turned a 2.2x speedup into an apparent 7x regression. The tables show the request rate wherever it diverges.

The open-loop rows are not a speed. W4-sat50/70/90 offer a fixed fraction of measured saturation; serving it means the engine kept up, not that it was faster, so no ratio is printed. They exist because --parallel is a closed loop, where a stalled server stops receiving requests and understates its own tail.

ef is the same unit here: Qdrant held 2 segments for this run, one populated and 1 empty (read back per segment from the engine), and strawmANN held one graph, so equal ef is equal traversal width and the curves may be read at equal x. That is a property of this run, not of the engines.

What was measured builds, dataset, host

	strawmANN `sm-sift-perf-0930`	Qdrant `qd-sift-perf-0930`
engine	commit `1fb2e437e858` `ReleaseFast`, native build, vnni on binary sha256 `e9f47255fbea5cee` built 2026-09-29T15:06:47Z with zig 0.16.0	version 1.19.2-dev binary `~/.cache/strawmann/qdrant-dbeb0f73/qdrant` sha256 `dbeb0f73dea2d371` build `878843e6` (from the server's banner, not a checkout) native binary outside a checkout: sha256 is the identity (§8.9) network `native` no container in the path, like strawmann
measured	2026-09-29T16:00:24Z to 2026-09-29T17:44:04Z (3 passes)	2026-09-29T16:20:36Z to 2026-09-29T18:07:07Z (3 passes)
profile	`as-deployed` · both engines
load generator	bfb dev @ fc6632e5 (qdrant/bfb#176; carries #172, findings 32's --rps reaping fix) · both engines
client	qdrant-client 1.16.1-dev (git dev branch) · both engines
launched as	`~/Workspace/strawmann/zig-out/bin/strawmann --port 6334 --capacity 1250000 --connections 64 --workers 7 --io-threads 1 --pin --cpus 4-11 --data-dir ~/.cache/strawmann/strawmann-storage --default-placement cached`	`~/.cache/strawmann/qdrant-dbeb0f73/qdrant`
cores the engine could use	`4-11` (8 cores) requested by the harness `4-11` (8 cores) observed on 8 threads `0-23` (24 cores) observed on 1 thread	`4-11` (8 cores) requested by the harness `4-11` (8 cores) observed on 42 threads

Dataset

sift1m, 1000000 × 128, euclid, 10000 held-out queries

ground truth: shipped, and diffed against our fp64 recompute

1 file, each pinned by sha256 in datasets.json

Host §7.1 gate pass

AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M

24 logical cores · 58.6 GB · kernel 7.0.0-34-generic

memory bandwidth: 73.5 GB/s aggregate (24 threads) · 44.8 GB/s single core (61% of bus)

strawmann startup banner, 2026-09-29T16:00:24Z; the Qdrant run agrees within 10%

strawmANN run start: environment hash 373e8531fd3d1bf6

Qdrant run start: environment hash 373e8531fd3d1bf6

All throughput rows every measurement, with its notes

workload		strawmANN	Qdrant	ratio	notes
W0	d=4 floor: graph traversal with the distance taken out	4,395	3,580	1.23x	1.23x is 1.66x less work per query x 0.68x cores busy during the row (0.69 against 1.02) x 1.00x clock x 1.10x counted on-CPU share: the middle term is occupancy, not search speed d=4 floor: this measures graph traversal and heaps with the distance taken out, not request plumbing (findings 28: 76% of the server's CPU was HNSW search, 24% kernel; the RPC path is a small remainder)
W3	search, fp32, single query	1,702	2,008	0.85x
W4	search, saturating (closed loop)	23,059	14,135	1.63x
W4-sat50	search, fixed rate at 50% of saturation (open loop)	7,156	7,156	offered
W4-sat70	search, fixed rate at 70% of saturation (open loop)	10,017	10,017	offered
W4-sat90	search, fixed rate at 90% of saturation (open loop)	12,879	12,880	offered
W5	search batched (16 distinct dataset queries per request)	23,4001,437 requests/s	4,859304 requests/s	4.82x	per-batch latency (16 queries/request); queries: dataset, random-sample 4.82x is 1.32x less work per query x 3.60x cores busy during the row (7.05 against 1.96) x 1.00x clock x 1.01x counted on-CPU share: the middle term is occupancy, not search speed batched 16, so qps and rps differ by 16x; the ratio uses qps
W6	quantized: scalar	4,393	4,690	0.94x
W6-ef32	SQ8 recall control, ef=32 (latency only)	9,375	7,754	-	recall unequal: 0.9087 vs 0.8983; §7.4 compares at equal recall; the rescore pools match, so the encoders differ
W6-ef64	SQ8 recall control, ef=64 (latency only)	6,372	6,179	parity	1.03x is 1.18x less work per query x 0.80x cores busy during the row (1.57 against 1.97) x 1.00x clock x 1.10x counted on-CPU share: the middle term is occupancy, not search speed within the ±6.1% band this dataset's noise floor puts on W6-ef64: no measured difference, not a small one
W6-ef128	SQ8 recall control, ef=128 (latency only)	4,411	4,684	0.94x
W6-ef256	SQ8 recall control, ef=256 (latency only)	3,013	3,069	parity	within the ±2.0% band this dataset's noise floor puts on W6-ef256: no measured difference, not a small one
W6-ef512	SQ8 recall control, ef=512 (latency only)	1,835	1,830	parity	within the ±2.0% band this dataset's noise floor puts on W6-ef512: no measured difference, not a small one
W7	quantized: binary + oversampling	4,508	4,655	-	recall below the usable floor: 0.0618 and 0.0660, both under 0.50 — this row characterises the encoding, not the engines
W8	quantized: PQ	4,490	4,165	1.08x
W9	exact / brute force	186	124	1.50x	brute force over the whole collection, no index involved
W10-ef32	recall control, ef=32 (latency only)	40,820	22,022	-	strawmANN: -n 100000 (2x the table's 50000) recall unequal: 0.9089 vs 0.8986; §7.4 compares at equal recall
W10-ef64	recall control, ef=64 (latency only)	31,425	15,494	2.03x	strawmANN: -n 100000 (2x the table's 50000)
W10-ef128	recall control, ef=128 (latency only)	20,608	10,159	2.03x
W10-ef256	recall control, ef=256 (latency only)	12,089	6,591	1.83x
W10-ef512	recall control, ef=512 (latency only)	6,622	3,963	1.67x
W12-sel1	filtered search, one keyword (~1% of bench12)	5,807	3,884	1.49x	filtered search over a keyword payload index both engines were asked to build; whether each did is read back from the engine and refused beside the row when it did not. Recall is measured under the filter, against ground truth restricted to the matching set
W12-sel10	filtered search, any of 10 keywords (~10% of bench12)	1,359	2,222	-	recall unequal: 1.0000 vs 0.9893; §7.4 compares at equal recall filtered search over a keyword payload index both engines were asked to build; whether each did is read back from the engine and refused beside the row when it did not. Recall is measured under the filter, against ground truth restricted to the matching set
W12-sel1-ef32	filtered recall control, one keyword, ef=32 (latency only)	6,419	6,432	parity	within the ±6.8% band this dataset's noise floor puts on W12-sel1-ef32: no measured difference, not a small one
W12-sel1-ef64	filtered recall control, one keyword, ef=64 (latency only)	6,269	5,258	1.19x
W12-sel1-ef128	filtered recall control, one keyword, ef=128 (latency only)	5,996	3,862	1.55x
W12-sel1-ef256	filtered recall control, one keyword, ef=256 (latency only)	6,271	2,687	2.33x
W12-sel1-ef512	filtered recall control, one keyword, ef=512 (latency only)	6,228	1,701	3.66x
W12-sel10-ef32	filtered recall control, any of 10 keywords, ef=32 (latency only)	3,172	4,685	-	recall unequal: 0.9777 vs 0.8007; §7.4 compares at equal recall
W12-sel10-ef64	filtered recall control, any of 10 keywords, ef=64 (latency only)	2,098	3,377	-	recall unequal: 0.9954 vs 0.9331; §7.4 compares at equal recall
W12-sel10-ef128	filtered recall control, any of 10 keywords, ef=128 (latency only)	1,358	2,226	-	recall unequal: 1.0000 vs 0.9893; §7.4 compares at equal recall
W12-sel10-ef256	filtered recall control, any of 10 keywords, ef=256 (latency only)	1,358	1,325	1.02x
W12-sel10-ef512	filtered recall control, any of 10 keywords, ef=512 (latency only)	1,359	798	1.70x
W13	scroll / pagination	51,192	46,560	1.10x	strawmANN: -n 200000 (4x the table's 50000) Qdrant: -n 100000 (2x the table's 50000) strawmANN: 0.13 ms of the client's 0.14 ms p50 is not server time (12x), so 92% of this row is the load generator and the socket Qdrant: 0.13 ms of the client's 0.16 ms p50 is not server time (6x), so 82% of this row is the load generator and the socket 1.10x is 3.27x less work per query x 0.33x cores busy during the row (1.39 against 4.22) x 1.00x clock x 1.02x counted on-CPU share: the middle term is occupancy, not search speed
W11-steady	mixed read/write below the rebuild threshold: search bench2 while 50,000 synthetic points append	18,228	5,537	-	strawmANN: qps over the 2.7 s the append ran (18,221 over the whole search); write overlap 11%; search covered 11% of the append; append 2,000 points/s Qdrant: qps over the 9.0 s the append ran (5,533 over the whole search); write overlap 36%; search covered 37% of the append; append 2,000 points/s search-during-write; no recall join; the search saw under 90% of either append, so its qps is the append's start and not all of it
W11	mixed read/write: search bench2 while 200,000 synthetic points append (runs last)	8,0333,800 requests/s	2,387	-	strawmANN: qps over the 6.2 s the append ran (8,029 over the whole search); write overlap 10%; search covered 10% of the append; append 3,300 points/s Qdrant: qps over the 20.9 s the append ran (2,386 over the whole search); write overlap 35%; search covered 35% of the append; append 3,300 points/s search-during-write; no recall join; the search saw under 90% of either append, so its qps is the append's start and not all of it

W10: throughput against ef

All latency rows closed loop included, p50 to max

A row flagged “… not server time” is one where the client saw far more than the server reported: below the rate at which a queue can form, what is left is the load generator's own scheduling.

workload		strawmANN p50	strawmANN p95	strawmANN p99	strawmANN p99.9	strawmANN max	Qdrant p50	Qdrant p95	Qdrant p99	Qdrant p99.9	Qdrant max
W0	d=4 floor: graph traversal with the distance taken out closed loop	223 µs	251 µs	272 µs	331 µs	890 µs	272 µs	324 µs	352 µs	414 µs	1.55 ms
W3	search, fp32, single query closed loop	598 µs	680 µs	715 µs	792 µs	3.01 ms	505 µs	564 µs	594 µs	666 µs	2.04 ms
W4	search, saturating (closed loop) closed loop	2.77 ms	2.99 ms	3.07 ms	17.73 ms	22.06 ms	4.01 ms	8.93 ms	12.78 ms	27.43 ms	37.33 ms
W4-sat50	search, fixed rate at 50% of saturation (open loop) open loop	493 µs	834 µs	1.01 ms	1.56 ms	2.30 ms	753 µs	1.38 ms	1.83 ms	2.32 ms	5.29 ms
W4-sat70	search, fixed rate at 70% of saturation (open loop) open loop	532 µs	997 µs	1.12 ms	1.22 ms	2.37 ms	838 µs	1.67 ms	2.02 ms	2.87 ms	9.43 ms
W4-sat90	search, fixed rate at 90% of saturation (open loop) open loop	718 µs	1.26 ms	1.41 ms	1.53 ms	3.12 ms	1.24 ms	3.18 ms	5.27 ms	9.40 ms	23.12 ms
W5	search batched (16 distinct dataset queries per request) closed loop	1.36 ms	1.47 ms	1.52 ms	2.63 ms	4.15 ms	6.56 ms	6.92 ms	7.10 ms	7.74 ms	9.78 ms
W6	quantized: scalar closed loop	442 µs	573 µs	605 µs	649 µs	1.26 ms	426 µs	473 µs	502 µs	581 µs	3.04 ms
W6-ef32	SQ8 recall control, ef=32 (latency only) closed loop	210 µs	244 µs	262 µs	293 µs	936 µs	250 µs	307 µs	334 µs	400 µs	1.32 ms
W6-ef64	SQ8 recall control, ef=64 (latency only) closed loop	315 µs	377 µs	401 µs	440 µs	1.31 ms	314 µs	390 µs	422 µs	466 µs	1.47 ms
W6-ef128	SQ8 recall control, ef=128 (latency only) closed loop	441 µs	573 µs	604 µs	647 µs	1.29 ms	427 µs	474 µs	506 µs	592 µs	1.75 ms
W6-ef256	SQ8 recall control, ef=256 (latency only) closed loop	669 µs	800 µs	933 µs	1.01 ms	1.63 ms	659 µs	724 µs	756 µs	848 µs	2.04 ms
W6-ef512	SQ8 recall control, ef=512 (latency only) closed loop	1.12 ms	1.27 ms	1.31 ms	1.39 ms	2.93 ms	1.11 ms	1.23 ms	1.27 ms	1.34 ms	2.83 ms
W7	quantized: binary + oversampling closed loop	439 µs	500 µs	533 µs	598 µs	1.13 ms	427 µs	485 µs	513 µs	584 µs	1.60 ms
W8	quantized: PQ closed loop	448 µs	491 µs	513 µs	569 µs	1.29 ms	472 µs	573 µs	609 µs	657 µs	1.80 ms
W9	exact / brute force closed loop	38.90 ms	64.03 ms	71.04 ms	79.22 ms	86.18 ms	64.57 ms	74.95 ms	78.79 ms	87.90 ms	90.57 ms
W10-ef32	recall control, ef=32 (latency only) closed loop	188 µs	227 µs	255 µs	344 µs	1.33 ms	335 µs	536 µs	657 µs	861 µs	3.51 ms
W10-ef64	recall control, ef=64 (latency only) closed loop	251 µs	290 µs	315 µs	406 µs	1.50 ms	476 µs	774 µs	965 µs	1.30 ms	2.56 ms
W10-ef128	recall control, ef=128 (latency only) closed loop	390 µs	451 µs	485 µs	726 µs	1.92 ms	716 µs	1.26 ms	1.50 ms	1.98 ms	4.39 ms
W10-ef256	recall control, ef=256 (latency only) closed loop	668 µs	807 µs	890 µs	1.02 ms	3.16 ms	1.09 ms	1.94 ms	2.33 ms	3.02 ms	4.98 ms
W10-ef512	recall control, ef=512 (latency only) closed loop	1.22 ms	1.54 ms	1.72 ms	1.94 ms	4.24 ms	1.85 ms	3.17 ms	3.68 ms	4.69 ms	7.14 ms
W12-sel1	filtered search, one keyword (~1% of bench12) closed loop	318 µs	444 µs	470 µs	581 µs	1.22 ms	510 µs	560 µs	587 µs	660 µs	1.96 ms
W12-sel10	filtered search, any of 10 keywords (~10% of bench12) closed loop	1.49 ms	1.57 ms	1.61 ms	1.69 ms	3.32 ms	904 µs	1.03 ms	1.08 ms	1.16 ms	3.19 ms
W12-sel1-ef32	filtered recall control, one keyword, ef=32 (latency only) closed loop	293 µs	412 µs	444 µs	493 µs	1.16 ms	304 µs	359 µs	399 µs	455 µs	1.55 ms
W12-sel1-ef64	filtered recall control, one keyword, ef=64 (latency only) closed loop	302 µs	422 µs	448 µs	493 µs	1.14 ms	376 µs	421 µs	448 µs	531 µs	1.69 ms
W12-sel1-ef128	filtered recall control, one keyword, ef=128 (latency only) closed loop	310 µs	434 µs	458 µs	495 µs	1.12 ms	513 µs	564 µs	593 µs	679 µs	2.90 ms
W12-sel1-ef256	filtered recall control, one keyword, ef=256 (latency only) closed loop	302 µs	426 µs	452 µs	498 µs	1.12 ms	740 µs	796 µs	830 µs	936 µs	2.40 ms
W12-sel1-ef512	filtered recall control, one keyword, ef=512 (latency only) closed loop	304 µs	425 µs	452 µs	497 µs	1.19 ms	1.17 ms	1.27 ms	1.32 ms	1.40 ms	3.35 ms
W12-sel10-ef32	filtered recall control, any of 10 keywords, ef=32 (latency only) closed loop	625 µs	711 µs	861 µs	958 µs	1.78 ms	422 µs	495 µs	534 µs	603 µs	1.88 ms
W12-sel10-ef64	filtered recall control, any of 10 keywords, ef=64 (latency only) closed loop	972 µs	1.06 ms	1.10 ms	1.39 ms	2.29 ms	591 µs	677 µs	717 µs	790 µs	2.16 ms
W12-sel10-ef128	filtered recall control, any of 10 keywords, ef=128 (latency only) closed loop	1.49 ms	1.57 ms	1.61 ms	1.72 ms	3.20 ms	902 µs	1.03 ms	1.07 ms	1.14 ms	2.74 ms
W12-sel10-ef256	filtered recall control, any of 10 keywords, ef=256 (latency only) closed loop	1.49 ms	1.57 ms	1.61 ms	1.75 ms	3.30 ms	1.52 ms	1.74 ms	1.82 ms	1.93 ms	3.64 ms
W12-sel10-ef512	filtered recall control, any of 10 keywords, ef=512 (latency only) closed loop	1.49 ms	1.57 ms	1.61 ms	1.71 ms	3.30 ms	2.52 ms	2.88 ms	3.00 ms	3.16 ms	5.24 ms
W13	scroll / pagination closed loop	137 µs	204 µs	230 µs	270 µs	1.10 ms	158 µs	221 µs	292 µs	441 µs	1.52 ms
W11-steady	mixed read/write below the rebuild threshold: search bench2 while 50,000 synthetic points append closed loop	394 µs	571 µs	1.96 ms	3.01 ms	6.64 ms	1.26 ms	2.69 ms	4.79 ms	9.92 ms	18.90 ms
W11	mixed read/write: search bench2 while 200,000 synthetic points append (runs last) closed loop	430 µs	2.96 ms	4.09 ms	6.06 ms	10.97 ms	2.62 ms	7.65 ms	14.09 ms	25.83 ms	101.30 ms

Storage and I/O block layer and syscalls, per row

The syscall rows count every descriptor, sockets included, so on a search row they measure the network rather than the disk.

	strawmANN	Qdrant
storage on disk	3.1 GiBbefore W11-steady, W11	2.9 GiBbefore W11-steady, W11
peak RSS	5.2 GiB	14.7 GiB5.2 GiB before the writers
of which anonymous	2.1 GiB	1.7 GiB833.8 MiB before the writers
of which file-backed	3.1 GiB	12.8 GiB3.7 GiB before the writers
disk read bytes	16.0 KiB	1.9 GiB
disk write bytes	2.7 GiB	22.3 GiB
disk read ops	12	49,347
disk write ops	35,822	393,702
syscall reads (all fds)	10,638,208	46,122
syscall writes (all fds)	4,883,704	7,673,060

measured via strawmANN: proc+cgroup / Qdrant: proc+cgroup. proc supplies syscall counts and block-layer bytes, cgroup supplies block-layer operations and bytes, so a row one interface does not carry reads n/a via …. unknown means it was not measured, and 0 means it was: an engine started with no --data-dir has no store, which is the row this section exists for. storage on disk is the level before the rows with a concurrent writer: during those an engine that rewrites segments is caught mid-rewrite, and the same row has read 3.47 and 10.11 GiB on two runs of one binary. peak RSS does include them, being a peak.

Per workload

A search row doing block-layer reads is an engine going to disk to answer a query.

workload	strawmANN read	strawmANN written	strawmANN read ops	strawmANN write ops	Qdrant read	Qdrant written	Qdrant read ops	Qdrant write ops
W0-upload	4.0 KiB	61.1 MiB	3	987	130.2 MiB	406.7 MiB	3,369	8,812
W0	0	0	0	3	0	0	0	6
W1	0	488.4 MiB	0	0	6.9 MiB	1.3 GiB	309	21,489
W2	8.0 KiB	488.4 MiB	6	7,884	170.8 MiB	2.4 GiB	4,329	42,780
W3	0	0	0	3	0	0	0	0
W4	0	0	0	0	0	0	0	0
W4-sat50	0	0	0	0	0	0	0	0
W4-sat70	0	0	0	0	0	0	0	0
W4-sat90	0	0	0	0	0	0	0	0
W5	0	0	0	0	0	0	0	0
W6-upload	0	488.4 MiB	0	7,882	161.9 MiB	2.7 GiB	4,101	48,154
W6	0	0	0	3	0	0	0	0
W6-ef32	0	0	0	0	0	0	0	0
W6-ef64	0	0	0	0	0	0	0	0
W6-ef128	0	0	0	0	0	0	0	0
W6-ef256	0	0	0	0	0	0	0	0
W6-ef512	0	0	0	0	0	0	0	0
W7-upload	0	488.4 MiB	0	7,882	194.3 MiB	2.7 GiB	4,917	47,819
W7	0	0	0	3	0	0	0	0
W8-upload	4.0 KiB	488.4 MiB	3	7,882	145.1 MiB	2.2 GiB	3,621	38,723
W8	0	0	0	3	0	0	0	0
W9	0	0	0	0	0	0	0	0
W10-ef32	0	0	0	0	0	0	0	0
W10-ef64	0	0	0	0	0	0	0	0
W10-ef128	0	0	0	0	0	0	0	0
W10-ef256	0	0	0	0	0	0	0	0
W10-ef512	0	0	0	0	0	0	0	0
W12-upload	0	97.8 MiB	0	0	52.0 MiB	558.7 MiB	1,428	10,707
W12-sel1	0	0	0	1,577	0	0	0	0
W12-sel10	0	0	0	3	0	0	0	0
W12-sel1-ef32	0	0	0	0	0	0	0	0
W12-sel1-ef64	0	0	0	0	0	0	0	0
W12-sel1-ef128	0	0	0	0	0	0	0	0
W12-sel1-ef256	0	0	0	0	0	0	0	0
W12-sel1-ef512	0	0	0	0	0	0	0	0
W12-sel10-ef32	0	0	0	0	0	0	0	0
W12-sel10-ef64	0	0	0	0	0	0	0	0
W12-sel10-ef128	0	0	0	0	0	0	0	0
W12-sel10-ef256	0	0	0	0	0	0	0	0
W12-sel10-ef512	0	0	0	0	0	0	0	0
W13	0	0	0	0	0	0	0	0
W11-steady	0	24.5 MiB	0	3	365.6 MiB	3.5 GiB	8,973	59,229
W11	0	98.1 MiB	0	1,707	736.2 MiB	6.6 GiB	18,300	115,983

Scheduler and memory CPU use, run-queue wait, migrations

Whether the engine was running while it ran. of wall is CPU over elapsed, waiting is runnable-but-not-scheduled, migrations checks the pinning claim, and switches gives voluntary over involuntary — the scheduler taking the core away against the engine choosing to sleep, which per unit of work is the cheapest signal of lock contention there is. A dash is not a zero: an index build's threads exit before the row does, and threads shows what fraction of the row the survivors account for.

Time spent waiting for a core

Lower is better; the bar between a pair is the gap.

Peak memory per row

Peak RSS while the row ran, from the engine's own process. Unlike the disk counters this is not refused across the two engines: residency decides where bytes live, and this is what the process held either way.

workload	strawmANN							Qdrant
workload	cpu	of wall	waiting	switches vol/invol	migrations	faults min/maj	threads	cpu	of wall	waiting	switches vol/invol	migrations	faults min/maj	threads
W0-upload	126.9 s	566%	52.7 ms	47,057 / 723	7	114,500 / 0	9 1%	181.7 s	515%	15,239.2 ms	156,563 / 35,027	14,207	431,061 / 190,409	39 5%
W0	8.2 s	71%	3.6 ms	161,774 / 46	0	0 / 0	9	14.4 s	103%	3.5 ms	561,464 / 30	93	408 / 0	36 0%
W1	2.0 s	64%	1.8 ms	110,702 / 2	0	160,269 / 0	9	84.0 s	604%	33,583.0 ms	131,555 / 50,552	16,232	180,060 / 259,097	53 0%
W2	305.7 s	690%	119.3 ms	147,939 / 1,575	1	228,032 / 0	9 1%	518.7 s	626%	27,187.5 ms	152,557 / 42,934	14,311	499,998 / 298,071	39 3%
W3	25.6 s	87%	7.9 ms	173,310 / 40	0	5 / 0	9	25.7 s	103%	3.7 ms	562,070 / 57	181	422 / 0	38 0%
W4	16.2 s	735%	1.4 ms	56,793 / 28	0	50 / 0	9	27.5 s	769%	76,836.9 ms	63,155 / 45,149	16,561	2,804 / 0	76
W4-sat50	75.7 s	270%	4.4 ms	292,354 / 47	0	13 / 0	9	122.4 s	436%	133,078.5 ms	1,120,132 / 160,982	373,210	227 / 0	68 95%
W4-sat70	70.3 s	350%	5.7 ms	240,306 / 55	0	18 / 0	9	121.9 s	606%	167,455.3 ms	1,040,817 / 213,639	367,583	477 / 0	68
W4-sat90	69.8 s	446%	2.2 ms	203,139 / 58	0	24 / 0	9	107.0 s	683%	114,892.9 ms	472,846 / 122,618	139,382	1,409 / 0	44 0%
W5	15.1 s	703%	0.6 ms	9,816 / 27	0	0 / 0	9	20.3 s	197%	10.2 ms	33,335 / 84	691	393 / 0	42 0%
W6-upload	304.7 s	679%	117.0 ms	53,110 / 1,555	1	277,859 / 0	9 1%	347.3 s	545%	21,501.8 ms	162,119 / 38,056	16,873	595,971 / 298,941	40 0%
W6	19.5 s	171%	3.3 ms	171,686 / 37	0	0 / 0	9	21.6 s	202%	11.4 ms	493,837 / 38	9,908	551 / 0	44 0%
W6-ef32	7.5 s	140%	2.5 ms	138,186 / 6	0	0 / 0	9	12.8 s	198%	10.5 ms	465,752 / 31	10,249	179 / 0	44
W6-ef64	12.5 s	159%	10.5 ms	159,938 / 12	0	0 / 0	9	16.1 s	198%	14.3 ms	472,820 / 42	10,364	72 / 0	44
W6-ef128	19.4 s	170%	4.8 ms	172,966 / 14	0	0 / 0	9	21.7 s	202%	11.2 ms	494,074 / 87	9,854	95 / 0	44
W6-ef256	29.9 s	180%	2.2 ms	178,963 / 24	0	0 / 0	9	33.2 s	203%	10.8 ms	523,021 / 124	13,276	148 / 0	44
W6-ef512	51.2 s	188%	6.1 ms	180,667 / 59	0	0 / 0	9	55.5 s	203%	15.3 ms	542,027 / 192	17,613	364 / 0	44
W7-upload	304.6 s	679%	120.6 ms	53,686 / 1,616	0	247,539 / 0	9 1%	352.0 s	570%	21,874.3 ms	156,692 / 39,491	18,020	572,310 / 337,442	43 0%
W7	18.7 s	168%	2.7 ms	168,670 / 25	0	0 / 0	9	21.8 s	203%	11.5 ms	503,873 / 59	10,474	601 / 0	46 0%
W8-upload	376.0 s	698%	139.5 ms	52,110 / 1,927	0	249,688 / 0	9 0%	562.0 s	573%	10,770.0 ms	2,177,353 / 80,472	58,573	474,960 / 240,680	44 1%
W8	18.8 s	168%	8.0 ms	166,890 / 25	0	0 / 0	9	24.3 s	202%	14.4 ms	497,464 / 73	10,385	534 / 0	48
W9	74.8 s	695%	6.9 ms	3,205 / 175	0	0 / 0	9	126.3 s	781%	8,408.8 ms	17,482 / 7,967	5,536	220 / 0	50
W10-ef32	11.6 s	462%	2.0 ms	157,561 / 12	0	0 / 0	9	13.2 s	568%	5,006.8 ms	228,193 / 18,122	63,572	1,061 / 0	50
W10-ef64	18.9 s	582%	2.1 ms	182,607 / 26	0	0 / 0	9	19.2 s	588%	10,527.1 ms	259,766 / 28,977	98,077	891 / 0	75
W10-ef128	16.3 s	661%	1.2 ms	106,480 / 21	0	0 / 0	9	29.3 s	591%	20,877.8 ms	288,104 / 35,837	110,017	147 / 0	75
W10-ef256	28.9 s	693%	1.9 ms	114,718 / 35	0	0 / 0	9	47.0 s	617%	34,985.1 ms	298,809 / 37,179	125,296	145 / 0	69 0%
W10-ef512	53.4 s	703%	4.0 ms	122,860 / 87	0	0 / 0	9	80.3 s	635%	49,392.0 ms	315,538 / 42,945	152,309	215 / 0	69
W12-upload	48.6 s	414%	19.5 ms	11,659 / 279	0	117,282 / 0	9 1%	116.8 s	339%	2,585.1 ms	51,033 / 5,678	5,037	175,816 / 152,068	52 0%
W12-sel1	14.0 s	162%	8.8 ms	175,682 / 18	0	12 / 0	9	25.9 s	200%	16.3 ms	484,978 / 72	9,442	499 / 0	57
W12-sel10	69.8 s	190%	3.9 ms	155,286 / 118	0	0 / 0	9	45.6 s	203%	12.8 ms	533,768 / 155	15,863	396 / 0	57
W12-sel1-ef32	12.8 s	163%	2.6 ms	171,090 / 9	0	0 / 0	9	15.5 s	198%	10.7 ms	473,623 / 27	9,702	61 / 0	57
W12-sel1-ef64	13.0 s	162%	2.0 ms	172,605 / 7	0	0 / 0	9	19.1 s	200%	12.6 ms	477,029 / 49	9,517	65 / 0	57
W12-sel1-ef128	13.5 s	162%	2.0 ms	175,035 / 11	0	0 / 0	9	26.0 s	200%	17.1 ms	486,023 / 87	9,573	64 / 0	57
W12-sel1-ef256	13.0 s	162%	2.5 ms	175,444 / 10	0	0 / 0	9	37.5 s	201%	21.4 ms	494,796 / 126	9,041	199 / 0	57
W12-sel1-ef512	13.1 s	162%	2.4 ms	175,891 / 6	0	0 / 0	9	59.4 s	202%	30.5 ms	521,897 / 252	13,852	435 / 0	57
W12-sel10-ef32	28.1 s	178%	1.7 ms	179,286 / 28	0	0 / 0	9	21.7 s	202%	9.0 ms	507,298 / 45	11,149	166 / 0	57
W12-sel10-ef64	44.2 s	185%	1.2 ms	179,360 / 37	0	0 / 0	9	30.1 s	203%	10.2 ms	519,898 / 92	13,038	135 / 0	57
W12-sel10-ef128	69.8 s	189%	2.6 ms	159,072 / 112	0	0 / 0	9	45.5 s	202%	10.5 ms	534,036 / 125	15,511	250 / 0	57
W12-sel10-ef256	69.8 s	190%	5.0 ms	156,530 / 127	0	0 / 0	9	76.1 s	202%	31.4 ms	546,095 / 200	18,323	407 / 0	57
W12-sel10-ef512	69.8 s	190%	4.8 ms	157,305 / 93	0	0 / 0	9	126.1 s	201%	57.6 ms	554,359 / 464	18,614	1,068 / 0	57
W13	5.6 s	138%	3.2 ms	243,033 / 8	0	0 / 0	9	9.1 s	411%	864.2 ms	382,488 / 4,639	112,331	708 / 0	70
W11-steady	18.9 s	662%	1,345.8 ms	110,268 / 134	0	229,802 / 0	9 88%	88.5 s	968%	54,590.2 ms	313,517 / 88,025	116,172	936,740 / 224,007	59 34%
W11	49.5 s	781%	17,678.2 ms	117,472 / 4,680	111	810,544 / 0	9 0%	267.7 s	1,271%	135,269.8 ms	347,677 / 138,412	108,738	1,842,780 / 391,257	59 20%

Waiting and stalls where the time went when the engine was not running

waiting is runnable and not scheduled, blocked on disk is not runnable at all, and pressure is PSI for the engine's cgroup (some = at least one task stalled, full = every runnable task).

Time stalled on I/O

Time every task in the engine's cgroup was stalled on I/O. Lower is better. 34 rows never stalled on I/O and are not drawn.

workload	strawmANN					Qdrant
workload	waiting	blocked on disk	cpu some/full	io some/full	memory some/full	waiting	blocked on disk	cpu some/full	io some/full	memory some/full
W0-upload	52.7 ms	0 ms	15 / 10 ms	1 / 1 ms	0 / 0 ms	15,239.2 ms	0 ms	1,369 / 17 ms	31 / 29 ms	0 / 0 ms
W0	3.6 ms	0 ms	48 / 48 ms	0 / 0 ms	0 / 0 ms	3.5 ms	0 ms	80 / 80 ms	0 / 0 ms	0 / 0 ms
W1	1.8 ms	0 ms	11 / 11 ms	0 / 0 ms	0 / 0 ms	33,583.0 ms	0 ms	2,765 / 11 ms	24 / 12 ms	0 / 0 ms
W2	119.3 ms	0 ms	26 / 14 ms	24 / 24 ms	0 / 0 ms	27,187.5 ms	0 ms	2,269 / 25 ms	86 / 65 ms	0 / 0 ms
W3	7.9 ms	0 ms	48 / 48 ms	0 / 0 ms	0 / 0 ms	3.7 ms	0 ms	88 / 88 ms	0 / 0 ms	0 / 0 ms
W4	1.4 ms	0 ms	2 / 2 ms	0 / 0 ms	0 / 0 ms	76,836.9 ms	0 ms	2,881 / 3 ms	0 / 0 ms	0 / 0 ms
W4-sat50	4.4 ms	0 ms	65 / 65 ms	0 / 0 ms	0 / 0 ms	133,078.5 ms	0 ms	8,512 / 88 ms	0 / 0 ms	0 / 0 ms
W4-sat70	5.7 ms	0 ms	39 / 39 ms	0 / 0 ms	0 / 0 ms	167,455.3 ms	0 ms	9,485 / 60 ms	0 / 0 ms	0 / 0 ms
W4-sat90	2.2 ms	0 ms	27 / 27 ms	0 / 0 ms	0 / 0 ms	114,892.9 ms	0 ms	7,199 / 32 ms	0 / 0 ms	0 / 0 ms
W5	0.6 ms	0 ms	0 / 0 ms	0 / 0 ms	0 / 0 ms	10.2 ms	0 ms	13 / 12 ms	0 / 0 ms	0 / 0 ms
W6-upload	117.0 ms	0 ms	22 / 10 ms	0 / 0 ms	0 / 0 ms	21,501.8 ms	0 ms	1,856 / 25 ms	250 / 232 ms	0 / 0 ms
W6	3.3 ms	0 ms	32 / 32 ms	0 / 0 ms	0 / 0 ms	11.4 ms	0 ms	54 / 53 ms	0 / 0 ms	0 / 0 ms
W6-ef32	2.5 ms	0 ms	24 / 24 ms	0 / 0 ms	0 / 0 ms	10.5 ms	0 ms	46 / 45 ms	0 / 0 ms	0 / 0 ms
W6-ef64	10.5 ms	0 ms	30 / 30 ms	0 / 0 ms	0 / 0 ms	14.3 ms	0 ms	51 / 50 ms	0 / 0 ms	0 / 0 ms
W6-ef128	4.8 ms	0 ms	31 / 31 ms	0 / 0 ms	0 / 0 ms	11.2 ms	0 ms	54 / 53 ms	0 / 0 ms	0 / 0 ms
W6-ef256	2.2 ms	0 ms	30 / 30 ms	0 / 0 ms	0 / 0 ms	10.8 ms	0 ms	59 / 59 ms	0 / 0 ms	0 / 0 ms
W6-ef512	6.1 ms	0 ms	33 / 33 ms	0 / 0 ms	0 / 0 ms	15.3 ms	0 ms	69 / 68 ms	0 / 0 ms	0 / 0 ms
W7-upload	120.6 ms	0 ms	23 / 11 ms	0 / 0 ms	0 / 0 ms	21,874.3 ms	0 ms	1,915 / 23 ms	172 / 160 ms	0 / 0 ms
W7	2.7 ms	0 ms	34 / 34 ms	0 / 0 ms	0 / 0 ms	11.5 ms	0 ms	54 / 53 ms	0 / 0 ms	0 / 0 ms
W8-upload	139.5 ms	0 ms	25 / 12 ms	0 / 0 ms	0 / 0 ms	10,770.0 ms	0 ms	1,891 / 151 ms	104 / 96 ms	0 / 0 ms
W8	8.0 ms	0 ms	33 / 33 ms	0 / 0 ms	0 / 0 ms	14.4 ms	0 ms	58 / 57 ms	0 / 0 ms	0 / 0 ms
W9	6.9 ms	0 ms	2 / 2 ms	0 / 0 ms	0 / 0 ms	8,408.8 ms	0 ms	980 / 11 ms	0 / 0 ms	0 / 0 ms
W10-ef32	2.0 ms	0 ms	15 / 15 ms	0 / 0 ms	0 / 0 ms	5,006.8 ms	0 ms	570 / 15 ms	0 / 0 ms	0 / 0 ms
W10-ef64	2.1 ms	0 ms	14 / 14 ms	0 / 0 ms	0 / 0 ms	10,527.1 ms	0 ms	1,002 / 17 ms	0 / 0 ms	0 / 0 ms
W10-ef128	1.2 ms	0 ms	6 / 6 ms	0 / 0 ms	0 / 0 ms	20,877.8 ms	0 ms	1,789 / 20 ms	0 / 0 ms	0 / 0 ms
W10-ef256	1.9 ms	0 ms	6 / 6 ms	0 / 0 ms	0 / 0 ms	34,985.1 ms	0 ms	2,982 / 22 ms	0 / 0 ms	0 / 0 ms
W10-ef512	4.0 ms	0 ms	6 / 6 ms	0 / 0 ms	0 / 0 ms	49,392.0 ms	0 ms	4,537 / 26 ms	0 / 0 ms	0 / 0 ms
W12-upload	19.5 ms	0 ms	5 / 3 ms	0 / 0 ms	0 / 0 ms	2,585.1 ms	0 ms	261 / 12 ms	47 / 45 ms	0 / 0 ms
W12-sel1	8.8 ms	0 ms	31 / 31 ms	0 / 0 ms	0 / 0 ms	16.3 ms	0 ms	56 / 54 ms	0 / 0 ms	0 / 0 ms
W12-sel10	3.9 ms	0 ms	36 / 36 ms	0 / 0 ms	0 / 0 ms	12.8 ms	0 ms	65 / 64 ms	0 / 0 ms	0 / 0 ms
W12-sel1-ef32	2.6 ms	0 ms	25 / 25 ms	0 / 0 ms	0 / 0 ms	10.7 ms	0 ms	49 / 49 ms	0 / 0 ms	0 / 0 ms
W12-sel1-ef64	2.0 ms	0 ms	27 / 27 ms	0 / 0 ms	0 / 0 ms	12.6 ms	0 ms	52 / 51 ms	0 / 0 ms	0 / 0 ms
W12-sel1-ef128	2.0 ms	0 ms	29 / 29 ms	0 / 0 ms	0 / 0 ms	17.1 ms	0 ms	56 / 54 ms	0 / 0 ms	0 / 0 ms
W12-sel1-ef256	2.5 ms	0 ms	27 / 27 ms	0 / 0 ms	0 / 0 ms	21.4 ms	0 ms	60 / 58 ms	0 / 0 ms	0 / 0 ms
W12-sel1-ef512	2.4 ms	0 ms	27 / 27 ms	0 / 0 ms	0 / 0 ms	30.5 ms	0 ms	72 / 69 ms	0 / 0 ms	0 / 0 ms
W12-sel10-ef32	1.7 ms	0 ms	31 / 31 ms	0 / 0 ms	0 / 0 ms	9.0 ms	0 ms	54 / 53 ms	0 / 0 ms	0 / 0 ms
W12-sel10-ef64	1.2 ms	0 ms	31 / 31 ms	0 / 0 ms	0 / 0 ms	10.2 ms	0 ms	58 / 57 ms	0 / 0 ms	0 / 0 ms
W12-sel10-ef128	2.6 ms	0 ms	36 / 36 ms	0 / 0 ms	0 / 0 ms	10.5 ms	0 ms	64 / 63 ms	0 / 0 ms	0 / 0 ms
W12-sel10-ef256	5.0 ms	0 ms	36 / 36 ms	0 / 0 ms	0 / 0 ms	31.4 ms	0 ms	95 / 90 ms	0 / 0 ms	0 / 0 ms
W12-sel10-ef512	4.8 ms	0 ms	36 / 36 ms	0 / 0 ms	0 / 0 ms	57.6 ms	0 ms	124 / 117 ms	0 / 0 ms	0 / 0 ms
W13	3.2 ms	0 ms	28 / 28 ms	0 / 0 ms	0 / 0 ms	864.2 ms	0 ms	121 / 26 ms	0 / 0 ms	0 / 0 ms
W11-steady	1,345.8 ms	0 ms	258 / 13 ms	0 / 0 ms	0 / 0 ms	54,590.2 ms	0 ms	4,204 / 27 ms	212 / 193 ms	0 / 0 ms
W11	17,678.2 ms	0 ms	3,366 / 25 ms	10 / 10 ms	0 / 0 ms	135,269.8 ms	0 ms	10,206 / 61 ms	412 / 318 ms	2 / 1 ms

A dash under blocked on disk is not a zero: most hosts ship with kernel.task_delayacct=0, which reports the field as a permanent zero and would manufacture the strongest claim here — that the memory-resident engine never waits for a device — out of a sysctl. bench/setup.py apply turns it on.

Hardware counters cycles, DRAM loads, TLB walks, IPC per query

What the core did per query, from perf stat attached to the engine for the same window as the /proc counters. §5's cost model is a set of claims about instructions per cycle, DRAM traffic and TLB reach; these are those quantities, measured on the row the headline quotes rather than on a microbenchmark.

What one query cost, in cycles

Lower is better; the bar between a pair is the gap. Load rows are absent: they have no queries to divide by, and an absolute count under a per-query axis would be a different quantity wearing this one's label.

Demand loads from DRAM per query

Demand loads served from DRAM, times the cache line this host reports. Lower is better. Demand only: the hardware prefetcher's fills are a separate counter and are not in this number, so a row that streams — an exact scan above all — moved far more than this says. On the graph rows, where there is little for a prefetcher to predict, it is most of the traffic and is the outstanding-miss quantity §5.2 argues the engine is limited by.

TLB walks per query

Data TLB misses that reached a page walk, per query. Lower is better. §5.5 argues a memory-resident index needs hugepage care; this is what not taking it costs, and `--no-huge-pages` is the A/B arm that prices it.

Instructions per cycle

How well the core was fed while it ran. Not a score: a low IPC on a memory-bound row is what §5.2 predicts, and a high one on a row that does more work per query is not a win. Read it beside the two charts above.

Branch mispredictions per 1k instructions

A graph traversal is a chain of data-dependent branches and none of them is predictable from the last query, so this is the column that separates a slow kernel from an unpredictable walk. Per thousand instructions rather than per branch: `branches` costs a PMU counter that `ref-cycles` needs more, and MPKI is the comparable form anyway.

workload	strawmANN						Qdrant
workload	IPC	GHz	branch MPKI	cycles/query	demand DRAM/query	TLB walks/query	IPC	GHz	branch MPKI	cycles/query	demand DRAM/query	TLB walks/query
W0-upload	1.52	2.011.005x nominal	5.40	-	-	-	1.33	2.011.005x nominal	4.69	-	-	-
W0	1.27	2.011.005x nominal	6.06	273,380	7.4 KiB	1,213.0	1.26	2.011.005x nominal	3.22	452,847	46.2 KiB	1,319.0
W1	1.80	2.011.005x nominal	0.80	-	-	-	1.41	2.011.005x nominal	3.94	-	-	-
W2	1.29	2.001.004x nominal	4.53	-	-	-	1.08	2.011.004x nominal	4.02	-	-	-
W3	0.75	2.011.005x nominal	6.06	947,761	776.9 KiB	3,340.2	1.19	2.011.005x nominal	3.20	900,759	1,023.0 KiB	3,671.2
W4	1.08	2.011.006x nominal	5.25	641,138	765.9 KiB	2,816.9	0.91	2.011.006x nominal	3.74	1,045,275	1,104.6 KiB	3,102.1
W4-sat50	0.94	2.011.006x nominal	5.94	739,002	777.9 KiB	3,077.9	0.96	2.011.006x nominal	3.79	1,093,904	1,100.1 KiB	3,539.4
W4-sat70	1.01	2.011.005x nominal	5.71	686,638	775.1 KiB	2,980.1	0.96	2.011.005x nominal	3.76	1,088,257	1,102.4 KiB	3,420.5
W4-sat90	1.01	2.011.005x nominal	5.71	685,960	774.2 KiB	2,932.1	0.96	2.011.005x nominal	3.72	1,012,569	1,076.0 KiB	3,213.4
W5	1.06	2.001.004x nominal	5.59	602,015	776.1 KiB	2,827.0	1.00	2.011.006x nominal	4.33	795,668	1,088.9 KiB	2,787.6
W6-upload	1.29	2.011.005x nominal	4.51	-	-	-	1.80	2.011.005x nominal	4.56	-	-	-
W6	1.00	2.011.005x nominal	6.59	727,296	172.5 KiB	4,507.3	1.70	2.011.006x nominal	3.08	754,218	126.0 KiB	3,176.8
W6-ef32	1.04	2.011.006x nominal	3.61	270,638	50.2 KiB	1,791.6	1.60	2.011.006x nominal	2.24	405,966	39.7 KiB	1,458.2
W6-ef64	0.95	2.011.006x nominal	5.28	457,566	93.4 KiB	2,799.3	1.60	2.011.006x nominal	2.64	540,471	68.1 KiB	2,148.9
W6-ef128	1.00	2.011.006x nominal	6.59	724,184	172.2 KiB	4,508.4	1.69	2.011.006x nominal	3.12	753,540	127.5 KiB	3,176.6
W6-ef256	1.12	2.011.005x nominal	7.20	1,143,584	310.7 KiB	7,747.7	1.69	2.011.006x nominal	3.59	1,207,544	236.6 KiB	4,938.1
W6-ef512	1.18	2.011.005x nominal	7.68	1,986,580	566.3 KiB	14,644.6	1.64	2.011.006x nominal	4.01	2,090,557	453.1 KiB	8,019.3
W7-upload	1.29	2.001.004x nominal	4.55	-	-	-	2.12	2.011.005x nominal	3.95	-	-	-
W7	1.28	2.011.006x nominal	7.65	696,492	54.6 KiB	4,029.0	1.77	2.011.006x nominal	3.30	762,270	103.5 KiB	2,896.8
W8-upload	2.39	2.001.004x nominal	2.23	-	-	-	4.50	2.001.004x nominal	0.93	-	-	-
W8	2.14	2.011.006x nominal	3.45	701,324	56.7 KiB	2,820.2	2.18	2.011.006x nominal	1.97	857,079	102.8 KiB	2,838.3
W9	2.01	2.011.005x nominal	0.01	74,904,816	34,264.3 KiB	33,969.3	1.33	2.011.006x nominal	0.15	125,896,181	10,931.2 KiB	126,608.9
W10-ef32	1.25	2.011.006x nominal	3.60	217,779	243.1 KiB	978.7	1.14	2.011.005x nominal	2.66	458,682	363.5 KiB	1,277.9
W10-ef64	1.17	2.011.006x nominal	4.45	361,769	424.7 KiB	1,611.6	1.04	2.011.006x nominal	3.19	682,559	630.7 KiB	2,049.3
W10-ef128	1.11	2.011.005x nominal	5.23	632,461	767.1 KiB	2,860.7	0.97	2.011.006x nominal	3.75	1,065,767	1,094.2 KiB	3,362.0
W10-ef256	1.07	2.011.006x nominal	5.94	1,138,522	1,396.4 KiB	5,391.8	0.93	2.011.005x nominal	4.27	1,752,445	1,913.4 KiB	5,486.9
W10-ef512	1.03	2.011.006x nominal	6.57	2,116,574	2,527.4 KiB	10,566.6	0.89	2.011.005x nominal	4.72	3,063,231	3,399.6 KiB	9,095.5
W12-upload	1.41	2.011.005x nominal	5.00	-	-	-	1.59	2.011.005x nominal	3.90	-	-	-
W12-sel1	1.03	2.011.006x nominal	4.88	511,298	693.8 KiB	2,279.3	1.33	2.011.006x nominal	4.03	925,913	411.5 KiB	2,568.2
W12-sel10	1.31	2.011.006x nominal	1.14	2,738,308	7,811.4 KiB	7,120.1	1.42	2.011.006x nominal	2.73	1,700,049	924.7 KiB	3,907.2
W12-sel1-ef32	1.12	2.011.006x nominal	4.60	469,316	681.0 KiB	2,232.2	1.29	2.011.006x nominal	2.98	513,455	208.6 KiB	1,577.6
W12-sel1-ef64	1.10	2.011.006x nominal	4.69	478,534	687.3 KiB	2,249.2	1.32	2.011.006x nominal	3.53	655,811	301.7 KiB	2,055.7
W12-sel1-ef128	1.05	2.011.006x nominal	4.76	496,626	690.6 KiB	2,253.6	1.32	2.011.006x nominal	4.04	931,805	420.3 KiB	2,563.9
W12-sel1-ef256	1.11	2.011.006x nominal	4.68	474,473	685.9 KiB	2,261.3	1.38	2.011.006x nominal	4.46	1,385,109	548.5 KiB	3,120.4
W12-sel1-ef512	1.10	2.011.005x nominal	4.74	477,695	687.3 KiB	2,267.2	1.41	2.011.006x nominal	4.83	2,250,713	741.8 KiB	3,744.4
W12-sel10-ef32	1.51	2.011.005x nominal	3.57	1,066,755	385.2 KiB	2,742.4	1.41	2.011.006x nominal	2.40	752,520	339.6 KiB	2,060.3
W12-sel10-ef64	1.48	2.011.006x nominal	4.05	1,710,408	669.7 KiB	3,903.1	1.41	2.011.006x nominal	2.54	1,084,807	555.5 KiB	2,791.9
W12-sel10-ef128	1.31	2.011.006x nominal	1.14	2,738,193	7,806.5 KiB	7,120.0	1.42	2.011.006x nominal	2.72	1,698,423	923.2 KiB	3,909.8
W12-sel10-ef256	1.31	2.011.006x nominal	1.14	2,738,607	7,812.9 KiB	7,117.4	1.39	2.011.006x nominal	3.14	2,898,569	1,545.5 KiB	5,491.6
W12-sel10-ef512	1.31	2.011.006x nominal	1.14	2,737,526	7,808.8 KiB	7,128.4	1.43	2.011.005x nominal	3.31	4,866,386	2,421.6 KiB	7,610.4
W13	1.36	2.011.007x nominal	1.22	42,234	1.0 KiB	13.1	1.47	2.011.006x nominal	1.40	137,999	1.4 KiB	41.8
W11-steady	1.23	2.011.005x nominal	4.74	732,549	802.7 KiB	2,974.4	1.91	2.011.005x nominal	2.01	3,380,270	1,691.8 KiB	4,485.8
W11	1.72	2.011.004x nominal	2.49	1,913,392	1,614.5 KiB	3,878.7	1.79	2.011.004x nominal	1.64	10,457,605	6,698.1 KiB	17,066.6

Nothing here is scaled. Where perf had to multiplex the group, the values it prints are extrapolations from the fraction of the row each counter was on, and they are withheld rather than shown — the same rule the scheduler table applies to a thread that exited. An event this host does not implement is likewise blank, never zero.

The two instruments disagree on these rows. strawmANN: perf and /proc disagree on context switches by up to 16% on W11; Qdrant: perf and /proc disagree on minor faults by up to 10% on W2, W6-upload, W7-upload, W8-upload, W12-upload, W11-steady, W11; Qdrant: perf and /proc disagree on context switches by up to 13% on W8-upload. perf keeps an exiting thread's counts and the /proc sums do not, so a gap here is usually the same lost-thread effect the scheduler table reports as coverage.

Collection config what each engine says it built

Read back from the engine rather than taken from the request. --segments 1 sets Qdrant's default_segment_number, which its optimizer treats as a target, and the segment count is the largest confound in the ef comparison. strawmANN held 1 segment; Qdrant held 2 segments.

Where the engines disagree a cell reads strawmANN / Qdrant, and is marked.

collection	segments	populated segments	requested segments	points	indexed vectors	vector size	hnsw m	hnsw ef_construct	quantization	shards	vector residency
bench0	1 / 2	- / 1	- / 1	1,000,000	1,000,000	4	16	100	-	1	cached
bench1	1 / 5	- / 5	- / 1	1,000,000	0 / 101,100	128	16	100	-	1	cached
bench2	1 / 2	- / 1	- / 1	1,000,000	1,000,000	128	16	100	-	1	cached
bench6	1 / 2	- / 1	- / 1	1,000,000	1,000,000	128	16	100	scalar	1	cached
bench7	1 / 2	- / 1	- / 1	1,000,000	1,000,000	128	16	100	binary	1	cached
bench8	1 / 2	- / 1	- / 1	1,000,000	1,000,000	128	16	100	product	1	cached
bench12	1 / 2	- / 1	- / 1	200,000	200,000	128	16	100	-	1	cached

After the mutating rows bench2 held strawmANN 1,250,000 (+250,000), 1,070,900 of them indexed; Qdrant 1,250,000 (+250,000), 1,223,700 of them indexed. The table above is the state the throughput, latency and recall rows searched; this is what W11 left, and is that row's subject rather than theirs.

bench1 was read back straight after its row and dropped, with no settle between, so an engine still building it reports a snapshot taken mid-ingest. Its cells are not marked as a disagreement.

Per workload every metric of every row, engine beside engine

Metric down, engine across, so a comparison is two adjacent cells. Metrics a row did not measure are dropped rather than shown empty.

W0-upload transport floor: load

upload · collection=bench0 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=1,000,000

metric	strawmANN	Qdrant
wall clock	22.4 s	35.3 s	1.57x
of which upload	1.3 s	7.2 s	5.59x
of which index wait	19.1 s	26.1 s	1.37x
time to Green	19.1 s	26.1 s	1.37x
cpu	126.9 s	181.7 s	1.43x
cpu, % of wall	566%	515%
waiting for a core	52.7 ms	15,239.2 ms	289.41x
migrations	7	14,207	2,029.57x
faults min/maj	114,500 / 0	431,061 / 190,409
peak RSS	443.8 MiB	1.5 GiB	3.37x
disk written	61.1 MiB	406.7 MiB
disk read	4.0 KiB	130.2 MiB

W0 d=4 floor: graph traversal with the distance taken out 1.23x

closed-loop · collection=bench0 · client -p 1 -t 2 -c 1 (-t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	4,395	3,580	1.23x
client p50	223 µs	272 µs	1.22x
client p99	272 µs	352 µs	1.29x
client p99.9	331 µs	414 µs	1.25x
server p50	132 µs	171 µs	1.30x
server p99	145 µs	228 µs	1.57x
wall clock	11.4 s	14.0 s	1.23x
queries sent	50,000	50,000
cpu	8.2 s	14.4 s	1.76x
cpu, % of wall	71%	103%
waiting for a core	3.6 ms	3.5 ms	0.95x
migrations	0	93
faults min/maj	0 / 0	408 / 0
peak RSS	443.8 MiB	1.5 GiB	3.37x
disk written	0	0
disk read	0	0

1.23x is 1.66x less work per query x 0.68x cores busy during the row (0.69 against 1.02) x 1.00x clock x 1.10x counted on-CPU share: the middle term is occupancy, not search speed d=4 floor: this measures graph traversal and heaps with the distance taken out, not request plumbing (findings 28: 76% of the server's CPU was HNSW search, 24% kernel; the RPC path is a small remainder)

W1 ingest throughput

upload · collection=bench1 · client -p 8 -t 8 -c 1 (-c bfb default) · n=1,000,000

metric	strawmANN	Qdrant
wall clock	3.2 s	13.9 s	4.34x
of which upload	1.1 s	11.8 s	10.96x
cpu	2.0 s	84.0 s	41.20x
cpu, % of wall	64%	604%
waiting for a core	1.8 ms	33,583.0 ms	18,323.08x
migrations	0	16,232
faults min/maj	160,269 / 0	180,060 / 259,097
peak RSS	1.1 GiB	2.3 GiB	2.04x
disk written	488.4 MiB	1.3 GiB
disk read	0	6.9 MiB

W2 index build time

upload · collection=bench2 · client -p 8 -t 8 -c 1 (-c bfb default) · n=1,000,000

metric	strawmANN	Qdrant
wall clock	44.3 s	82.8 s	1.87x
of which upload	1.1 s	12.3 s	11.27x
of which index wait	41.1 s	68.2 s	1.66x
time to Green	41.1 s	68.2 s	1.66x
cpu	305.7 s	518.7 s	1.70x
cpu, % of wall	690%	626%
waiting for a core	119.3 ms	27,187.5 ms	227.95x
migrations	1	14,311	14,311.00x
faults min/maj	228,032 / 0	499,998 / 298,071
peak RSS	1.4 GiB	3.2 GiB	2.36x
disk written	488.4 MiB	2.4 GiB
disk read	8.0 KiB	170.8 MiB

W3 search, fp32, single query 0.85x

ef=128 · closed-loop · collection=bench2 · client -p 1 -t 2 -c 1 (-t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	1,702	2,008	0.85x
recall@10	0.9888	0.9875	~equal
client p50	598 µs	505 µs	0.84x
client p99	715 µs	594 µs	0.83x
client p99.9	792 µs	666 µs	0.84x
server p50	492 µs	409 µs	0.83x
server p99	597 µs	481 µs	0.81x
wall clock	29.4 s	24.9 s	0.85x
queries sent	50,000	50,000
cpu	25.6 s	25.7 s	~equal
cpu, % of wall	87%	103%
waiting for a core	7.9 ms	3.7 ms	0.46x
migrations	0	181
faults min/maj	5 / 0	422 / 0
peak RSS	1.4 GiB	3.2 GiB	2.36x
disk written	0	0
disk read	0	0

W4 search, saturating (closed loop) 1.63x

ef=128 · closed-loop · collection=bench2 · client -p 64 -t 16 -c 2 · n=50,000

metric	strawmANN	Qdrant
queries/second	23,059	14,135	1.63x
recall@10	0.9888	0.9875	~equal
client p50	2.77 ms	4.01 ms	1.45x
client p99	3.07 ms	12.78 ms	4.16x
client p99.9	17.73 ms	27.43 ms	1.55x
server p50	2.69 ms	2.80 ms	1.04x
server p99	2.97 ms	9.77 ms	3.28x
wall clock	2.2 s	3.6 s	1.62x
queries sent	50,000	50,000
cpu	16.2 s	27.5 s	1.70x
cpu, % of wall	735%	769%
waiting for a core	1.4 ms	76,836.9 ms	55,996.19x
migrations	0	16,561
faults min/maj	50 / 0	2,804 / 0
peak RSS	1.4 GiB	3.2 GiB	2.36x
disk written	0	0
disk read	0	0

W4-sat50 search, fixed rate at 50% of saturation (open loop) offered

ef=128 · open-loop · offered=7,156/s (50% of saturation, pinned reference 14,312 qps) · collection=bench2 · client -p n/a under --rps -t 16 -c 2 · n=200,000

metric	strawmANN	Qdrant
queries/second	7,156	7,156	offered
recall@10	0.9888	0.9875	~equal
client p50	493 µs	753 µs	1.53x
client p99	1.01 ms	1.83 ms	1.82x
client p99.9	1.56 ms	2.32 ms	1.48x
server p50	380 µs	588 µs	1.55x
server p99	852 µs	1.57 ms	1.84x
wall clock	28.1 s	28.1 s	~equal
queries sent	200,000	200,000
cpu	75.7 s	122.4 s	1.62x
cpu, % of wall	270%	436%
waiting for a core	4.4 ms	133,078.5 ms	30,255.65x
migrations	0	373,210
faults min/maj	13 / 0	227 / 0
peak RSS	1.4 GiB	3.2 GiB	2.36x
disk written	0	0
disk read	0	0

W4-sat70 search, fixed rate at 70% of saturation (open loop) offered

ef=128 · open-loop · offered=10,018/s (70% of saturation, pinned reference 14,312 qps) · collection=bench2 · client -p n/a under --rps -t 16 -c 2 · n=200,000

metric	strawmANN	Qdrant
queries/second	10,017	10,017	offered
recall@10	0.9888	0.9875	~equal
client p50	532 µs	838 µs	1.57x
client p99	1.12 ms	2.02 ms	1.80x
client p99.9	1.22 ms	2.87 ms	2.35x
server p50	410 µs	641 µs	1.56x
server p99	942 µs	1.72 ms	1.83x
wall clock	20.1 s	20.1 s	~equal
queries sent	200,000	200,000
cpu	70.3 s	121.9 s	1.73x
cpu, % of wall	350%	606%
waiting for a core	5.7 ms	167,455.3 ms	29,377.87x
migrations	0	367,583
faults min/maj	18 / 0	477 / 0
peak RSS	1.4 GiB	3.2 GiB	2.36x
disk written	0	0
disk read	0	0

W4-sat90 search, fixed rate at 90% of saturation (open loop) offered

ef=128 · open-loop · offered=12,881/s (90% of saturation, pinned reference 14,312 qps) · collection=bench2 · client -p n/a under --rps -t 16 -c 2 · n=200,000

metric	strawmANN	Qdrant
queries/second	12,879	12,880	offered
recall@10	0.9888	0.9875	~equal
client p50	718 µs	1.24 ms	1.72x
client p99	1.41 ms	5.27 ms	3.73x
client p99.9	1.53 ms	9.40 ms	6.13x
server p50	597 µs	887 µs	1.49x
server p99	1.22 ms	3.80 ms	3.11x
wall clock	15.7 s	15.7 s	~equal
queries sent	200,000	200,000
cpu	69.8 s	107.0 s	1.53x
cpu, % of wall	446%	683%
waiting for a core	2.2 ms	114,892.9 ms	52,563.19x
migrations	0	139,382
faults min/maj	24 / 0	1,409 / 0
peak RSS	1.4 GiB	3.2 GiB	2.36x
disk written	0	0
disk read	0	0

W5 search batched (16 distinct dataset queries per request) 4.82x

ef=128 · closed-loop · collection=bench2 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	23,400	4,859	4.82x
recall@10	0.9888	0.9875	~equal
client p50	1.36 ms	6.56 ms	4.83x
client p99	1.52 ms	7.10 ms	4.67x
client p99.9	2.63 ms	7.74 ms	2.94x
server p50	1.22 ms	6.17 ms	5.07x
server p99	1.38 ms	6.61 ms	4.80x
wall clock	2.1 s	10.3 s	4.80x
queries sent	50,000	50,000
cpu	15.1 s	20.3 s	1.35x
cpu, % of wall	703%	197%
waiting for a core	0.6 ms	10.2 ms	15.73x
migrations	0	691
faults min/maj	0 / 0	393 / 0
peak RSS	1.4 GiB	3.2 GiB	2.36x
disk written	0	0
disk read	0	0

per-batch latency (16 queries/request); queries: dataset, random-sample 4.82x is 1.32x less work per query x 3.60x cores busy during the row (7.05 against 1.96) x 1.00x clock x 1.01x counted on-CPU share: the middle term is occupancy, not search speed batched 16, so qps and rps differ by 16x; the ratio uses qps

W6-upload scalar quantization: load

upload · collection=bench6 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=1,000,000

metric	strawmANN	Qdrant
wall clock	44.9 s	63.7 s	1.42x
of which upload	1.6 s	10.1 s	6.14x
of which index wait	41.1 s	53.1 s	1.29x
time to Green	41.1 s	53.1 s	1.29x
cpu	304.7 s	347.3 s	1.14x
cpu, % of wall	679%	545%
waiting for a core	117.0 ms	21,501.8 ms	183.73x
migrations	1	16,873	16,873.00x
faults min/maj	277,859 / 0	595,971 / 298,941
peak RSS	2.4 GiB	4.6 GiB	1.88x
disk written	488.4 MiB	2.7 GiB
disk read	0	161.9 MiB

W6 quantized: scalar 0.94x

ef=128 · closed-loop · collection=bench6 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=12.8 · rescore=true

metric	strawmANN	Qdrant
queries/second	4,393	4,690	0.94x
recall@10	0.9890	0.9871	~equal
client p50	442 µs	426 µs	0.96x
client p99	605 µs	502 µs	0.83x
client p99.9	649 µs	581 µs	0.89x
server p50	349 µs	329 µs	0.94x
server p99	495 µs	386 µs	0.78x
wall clock	11.4 s	10.7 s	0.94x
queries sent	50,000	50,000
cpu	19.5 s	21.6 s	1.11x
cpu, % of wall	171%	202%
waiting for a core	3.3 ms	11.4 ms	3.47x
migrations	0	9,908
faults min/maj	0 / 0	551 / 0
peak RSS	2.4 GiB	4.6 GiB	1.88x
disk written	0	0
disk read	0	0

W6-ef32 SQ8 recall control, ef=32 (latency only)

ef=32 · closed-loop · collection=bench6 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=3.2 · rescore=true

metric	strawmANN	Qdrant
queries/second	9,375	7,754	-
recall@10	0.9087	0.8983
client p50	210 µs	250 µs
client p99	262 µs	334 µs
client p99.9	293 µs	400 µs
server p50	117 µs	153 µs
server p99	156 µs	213 µs
wall clock	5.4 s	6.5 s
queries sent	50,000	50,000
cpu	7.5 s	12.8 s
cpu, % of wall	140%	198%
waiting for a core	2.5 ms	10.5 ms
migrations	0	10,249
faults min/maj	0 / 0	179 / 0
peak RSS	2.4 GiB	4.6 GiB
disk written	0	0
disk read	0	0

recall unequal: 0.9087 vs 0.8983; §7.4 compares at equal recall; the rescore pools match, so the encoders differ

W6-ef64 SQ8 recall control, ef=64 (latency only) parity

ef=64 · closed-loop · collection=bench6 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=6.4 · rescore=true

metric	strawmANN	Qdrant
queries/second	6,372	6,179	parity
recall@10	0.9646	0.9600	~equal
client p50	315 µs	314 µs	~equal
client p99	401 µs	422 µs	1.05x
client p99.9	440 µs	466 µs	1.06x
server p50	217 µs	216 µs	~equal
server p99	291 µs	299 µs	1.03x
wall clock	7.9 s	8.1 s	1.03x
queries sent	50,000	50,000
cpu	12.5 s	16.1 s	1.28x
cpu, % of wall	159%	198%
waiting for a core	10.5 ms	14.3 ms	1.36x
migrations	0	10,364
faults min/maj	0 / 0	72 / 0
peak RSS	2.4 GiB	4.6 GiB	1.88x
disk written	0	0
disk read	0	0

1.03x is 1.18x less work per query x 0.80x cores busy during the row (1.57 against 1.97) x 1.00x clock x 1.10x counted on-CPU share: the middle term is occupancy, not search speed within the ±6.1% band this dataset's noise floor puts on W6-ef64: no measured difference, not a small one

W6-ef128 SQ8 recall control, ef=128 (latency only) 0.94x

ef=128 · closed-loop · collection=bench6 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=12.8 · rescore=true

metric	strawmANN	Qdrant
queries/second	4,411	4,684	0.94x
recall@10	0.9890	0.9871	~equal
client p50	441 µs	427 µs	0.97x
client p99	604 µs	506 µs	0.84x
client p99.9	647 µs	592 µs	0.91x
server p50	348 µs	329 µs	0.95x
server p99	494 µs	386 µs	0.78x
wall clock	11.4 s	10.7 s	0.94x
queries sent	50,000	50,000
cpu	19.4 s	21.7 s	1.12x
cpu, % of wall	170%	202%
waiting for a core	4.8 ms	11.2 ms	2.32x
migrations	0	9,854
faults min/maj	0 / 0	95 / 0
peak RSS	2.4 GiB	4.6 GiB	1.88x
disk written	0	0
disk read	0	0

W6-ef256 SQ8 recall control, ef=256 (latency only) parity

ef=256 · closed-loop · collection=bench6 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=25.6 · rescore=true

metric	strawmANN	Qdrant
queries/second	3,013	3,069	parity
recall@10	0.9974	0.9968	~equal
client p50	669 µs	659 µs	~equal
client p99	933 µs	756 µs	0.81x
client p99.9	1.01 ms	848 µs	0.84x
server p50	578 µs	561 µs	0.97x
server p99	825 µs	641 µs	0.78x
wall clock	16.6 s	16.3 s	~equal
queries sent	50,000	50,000
cpu	29.9 s	33.2 s	1.11x
cpu, % of wall	180%	203%
waiting for a core	2.2 ms	10.8 ms	4.92x
migrations	0	13,276
faults min/maj	0 / 0	148 / 0
peak RSS	2.4 GiB	4.6 GiB	1.88x
disk written	0	0
disk read	0	0

within the ±2.0% band this dataset's noise floor puts on W6-ef256: no measured difference, not a small one

W6-ef512 SQ8 recall control, ef=512 (latency only) parity

ef=512 · closed-loop · collection=bench6 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=51.2 · rescore=true

metric	strawmANN	Qdrant
queries/second	1,835	1,830	parity
recall@10	0.9996	0.9993	~equal
client p50	1.12 ms	1.11 ms	~equal
client p99	1.31 ms	1.27 ms	0.96x
client p99.9	1.39 ms	1.34 ms	0.96x
server p50	1.02 ms	1.01 ms	~equal
server p99	1.22 ms	1.15 ms	0.95x
wall clock	27.3 s	27.4 s	~equal
queries sent	50,000	50,000
cpu	51.2 s	55.5 s	1.08x
cpu, % of wall	188%	203%
waiting for a core	6.1 ms	15.3 ms	2.52x
migrations	0	17,613
faults min/maj	0 / 0	364 / 0
peak RSS	2.4 GiB	4.6 GiB	1.88x
disk written	0	0
disk read	0	0

within the ±2.0% band this dataset's noise floor puts on W6-ef512: no measured difference, not a small one

W7-upload binary quantization: load

upload · collection=bench7 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=1,000,000

metric	strawmANN	Qdrant
wall clock	44.9 s	61.7 s	1.38x
of which upload	1.6 s	10.5 s	6.40x
of which index wait	41.1 s	49.1 s	1.19x
time to Green	41.1 s	49.1 s	1.19x
cpu	304.6 s	352.0 s	1.16x
cpu, % of wall	679%	570%
waiting for a core	120.6 ms	21,874.3 ms	181.45x
migrations	0	18,020
faults min/maj	247,539 / 0	572,310 / 337,442
peak RSS	3.4 GiB	5.2 GiB	1.54x
disk written	488.4 MiB	2.7 GiB
disk read	0	194.3 MiB

W7 quantized: binary + oversampling

ef=128 · closed-loop · collection=bench7 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=12.8 · rescore=true

metric	strawmANN	Qdrant
queries/second	4,508	4,655	-
recall@10	0.0618	0.0660
client p50	439 µs	427 µs
client p99	533 µs	513 µs
client p99.9	598 µs	584 µs
server p50	337 µs	329 µs
server p99	424 µs	403 µs
wall clock	11.1 s	10.8 s
queries sent	50,000	50,000
cpu	18.7 s	21.8 s
cpu, % of wall	168%	203%
waiting for a core	2.7 ms	11.5 ms
migrations	0	10,474
faults min/maj	0 / 0	601 / 0
peak RSS	3.4 GiB	5.2 GiB
disk written	0	0
disk read	0	0

recall below the usable floor: 0.0618 and 0.0660, both under 0.50 — this row characterises the encoding, not the engines

W8-upload PQ: load

upload · collection=bench8 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=1,000,000

metric	strawmANN	Qdrant
wall clock	53.9 s	98.1 s	1.82x
of which upload	1.6 s	7.8 s	4.83x
of which index wait	50.2 s	88.2 s	1.76x
time to Green	50.2 s	88.2 s	1.76x
cpu	376.0 s	562.0 s	1.49x
cpu, % of wall	698%	573%
waiting for a core	139.5 ms	10,770.0 ms	77.21x
migrations	0	58,573
faults min/maj	249,688 / 0	474,960 / 240,680
peak RSS	4.3 GiB	5.2 GiB	1.20x
disk written	488.4 MiB	2.2 GiB
disk read	4.0 KiB	145.1 MiB

W8 quantized: PQ 1.08x

ef=128 · closed-loop · collection=bench8 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000 · oversampling=12.8 · rescore=true

metric	strawmANN	Qdrant
queries/second	4,490	4,165	1.08x
recall@10	0.9825	0.9776	~equal
client p50	448 µs	472 µs	1.05x
client p99	513 µs	609 µs	1.19x
client p99.9	569 µs	657 µs	1.16x
server p50	348 µs	373 µs	1.07x
server p99	396 µs	486 µs	1.23x
wall clock	11.2 s	12.0 s	1.08x
queries sent	50,000	50,000
cpu	18.8 s	24.3 s	1.29x
cpu, % of wall	168%	202%
waiting for a core	8.0 ms	14.4 ms	1.80x
migrations	0	10,385
faults min/maj	0 / 0	534 / 0
peak RSS	4.3 GiB	5.2 GiB	1.20x
disk written	0	0
disk read	0	0

W9 exact / brute force 1.50x

exact · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=2,000

metric	strawmANN	Qdrant
queries/second	186	124	1.50x
client p50	38.90 ms	64.57 ms	1.66x
client p99	71.04 ms	78.79 ms	1.11x
client p99.9	79.22 ms	87.90 ms	1.11x
server p50	38.31 ms	63.45 ms	1.66x
server p99	70.31 ms	77.79 ms	1.11x
wall clock	10.8 s	16.2 s	1.50x
queries sent	2,000	2,000
cpu	74.8 s	126.3 s	1.69x
cpu, % of wall	695%	781%
waiting for a core	6.9 ms	8,408.8 ms	1,211.84x
migrations	0	5,536
faults min/maj	0 / 0	220 / 0
peak RSS	4.3 GiB	5.2 GiB	1.20x
disk written	0	0
disk read	0	0

brute force over the whole collection, no index involved

W10-ef32 recall control, ef=32 (latency only)

ef=32 · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=100,000

metric	strawmANN	Qdrant
queries/second	40,820	22,022	-
recall@10	0.9089	0.8986
client p50	188 µs	335 µs
client p99	255 µs	657 µs
client p99.9	344 µs	861 µs
server p50	96 µs	208 µs
server p99	132 µs	447 µs
wall clock	2.5 s	2.3 s
queries sent	100,000	50,000
cpu	11.6 s	13.2 s
cpu, % of wall	462%	568%
waiting for a core	2.0 ms	5,006.8 ms
migrations	0	63,572
faults min/maj	0 / 0	1,061 / 0
peak RSS	4.3 GiB	5.2 GiB
disk written	0	0
disk read	0	0

strawmANN: -n 100000 (2x the table's 50000) recall unequal: 0.9089 vs 0.8986; §7.4 compares at equal recall

W10-ef64 recall control, ef=64 (latency only) 2.03x

ef=64 · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=100,000

metric	strawmANN	Qdrant
queries/second	31,425	15,494	2.03x
recall@10	0.9639	0.9603	~equal
client p50	251 µs	476 µs	1.90x
client p99	315 µs	965 µs	3.07x
client p99.9	406 µs	1.30 ms	3.20x
server p50	170 µs	335 µs	1.97x
server p99	215 µs	761 µs	3.55x
wall clock	3.2 s	3.3 s	~equal
queries sent	100,000	50,000
cpu	18.9 s	19.2 s	~equal
cpu, % of wall	582%	588%
waiting for a core	2.1 ms	10,527.1 ms	4,907.24x
migrations	0	98,077
faults min/maj	0 / 0	891 / 0
peak RSS	4.3 GiB	5.2 GiB	1.20x
disk written	0	0
disk read	0	0

strawmANN: -n 100000 (2x the table's 50000)

W10-ef128 recall control, ef=128 (latency only) 2.03x

ef=128 · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	20,608	10,159	2.03x
recall@10	0.9888	0.9875	~equal
client p50	390 µs	716 µs	1.83x
client p99	485 µs	1.50 ms	3.09x
client p99.9	726 µs	1.98 ms	2.73x
server p50	313 µs	561 µs	1.79x
server p99	401 µs	1.28 ms	3.19x
wall clock	2.5 s	5.0 s	2.01x
queries sent	50,000	50,000
cpu	16.3 s	29.3 s	1.80x
cpu, % of wall	661%	591%
waiting for a core	1.2 ms	20,877.8 ms	17,186.58x
migrations	0	110,017
faults min/maj	0 / 0	147 / 0
peak RSS	4.3 GiB	5.2 GiB	1.20x
disk written	0	0
disk read	0	0

W10-ef256 recall control, ef=256 (latency only) 1.83x

ef=256 · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	12,089	6,591	1.83x
recall@10	0.9974	0.9967	~equal
client p50	668 µs	1.09 ms	1.63x
client p99	890 µs	2.33 ms	2.62x
client p99.9	1.02 ms	3.02 ms	2.95x
server p50	587 µs	923 µs	1.57x
server p99	809 µs	2.03 ms	2.51x
wall clock	4.2 s	7.6 s	1.83x
queries sent	50,000	50,000
cpu	28.9 s	47.0 s	1.63x
cpu, % of wall	693%	617%
waiting for a core	1.9 ms	34,985.1 ms	18,362.54x
migrations	0	125,296
faults min/maj	0 / 0	145 / 0
peak RSS	4.3 GiB	5.2 GiB	1.20x
disk written	0	0
disk read	0	0

W10-ef512 recall control, ef=512 (latency only) 1.67x

ef=512 · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	6,622	3,963	1.67x
recall@10	0.9995	0.9992	~equal
client p50	1.22 ms	1.85 ms	1.52x
client p99	1.72 ms	3.68 ms	2.14x
client p99.9	1.94 ms	4.69 ms	2.42x
server p50	1.13 ms	1.65 ms	1.46x
server p99	1.63 ms	3.35 ms	2.05x
wall clock	7.6 s	12.7 s	1.67x
queries sent	50,000	50,000
cpu	53.4 s	80.3 s	1.51x
cpu, % of wall	703%	635%
waiting for a core	4.0 ms	49,392.0 ms	12,311.47x
migrations	0	152,309
faults min/maj	0 / 0	215 / 0
peak RSS	4.3 GiB	5.2 GiB	1.20x
disk written	0	0
disk read	0	0

W12-upload filtered search: load 200,000 with payloads

upload · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=200,000

metric	strawmANN	Qdrant
wall clock	11.7 s	34.4 s	2.93x
of which upload	0.6 s	2.0 s	3.28x
of which index wait	9.0 s	30.1 s	3.33x
time to Green	9.0 s	30.1 s	3.33x
cpu	48.6 s	116.8 s	2.40x
cpu, % of wall	414%	339%
waiting for a core	19.5 ms	2,585.1 ms	132.37x
migrations	0	5,037
faults min/maj	117,282 / 0	175,816 / 152,068
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	97.8 MiB	558.7 MiB
disk read	0	52.0 MiB

W12-sel1 filtered search, one keyword (~1% of bench12) 1.49x

ef=128 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	5,807	3,884	1.49x
recall@10	1.0000	1.0000	~equal
client p50	318 µs	510 µs	1.61x
client p99	470 µs	587 µs	1.25x
client p99.9	581 µs	660 µs	1.14x
server p50	228 µs	411 µs	1.80x
server p99	352 µs	470 µs	1.33x
wall clock	8.6 s	12.9 s	1.49x
queries sent	50,000	50,000
cpu	14.0 s	25.9 s	1.84x
cpu, % of wall	162%	200%
waiting for a core	8.8 ms	16.3 ms	1.85x
migrations	0	9,442
faults min/maj	12 / 0	499 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

filtered search over a keyword payload index both engines were asked to build; whether each did is read back from the engine and refused beside the row when it did not. Recall is measured under the filter, against ground truth restricted to the matching set

W12-sel10 filtered search, any of 10 keywords (~10% of bench12)

ef=128 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	1,359	2,222	-
recall@10	1.0000	0.9893
client p50	1.49 ms	904 µs
client p99	1.61 ms	1.08 ms
client p99.9	1.69 ms	1.16 ms
server p50	1.39 ms	796 µs
server p99	1.50 ms	963 µs
wall clock	36.8 s	22.5 s
queries sent	50,000	50,000
cpu	69.8 s	45.6 s
cpu, % of wall	190%	203%
waiting for a core	3.9 ms	12.8 ms
migrations	0	15,863
faults min/maj	0 / 0	396 / 0
peak RSS	5.2 GiB	5.2 GiB
disk written	0	0
disk read	0	0

recall unequal: 1.0000 vs 0.9893; §7.4 compares at equal recall filtered search over a keyword payload index both engines were asked to build; whether each did is read back from the engine and refused beside the row when it did not. Recall is measured under the filter, against ground truth restricted to the matching set

W12-sel1-ef32 filtered recall control, one keyword, ef=32 (latency only) parity

ef=32 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	6,419	6,432	parity
recall@10	1.0000	0.9964	~equal
client p50	293 µs	304 µs	1.04x
client p99	444 µs	399 µs	0.90x
client p99.9	493 µs	455 µs	0.92x
server p50	205 µs	203 µs	~equal
server p99	333 µs	278 µs	0.84x
wall clock	7.8 s	7.8 s	~equal
queries sent	50,000	50,000
cpu	12.8 s	15.5 s	1.22x
cpu, % of wall	163%	198%
waiting for a core	2.6 ms	10.7 ms	4.05x
migrations	0	9,702
faults min/maj	0 / 0	61 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

within the ±6.8% band this dataset's noise floor puts on W12-sel1-ef32: no measured difference, not a small one

W12-sel1-ef64 filtered recall control, one keyword, ef=64 (latency only) 1.19x

ef=64 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	6,269	5,258	1.19x
recall@10	1.0000	0.9997	~equal
client p50	302 µs	376 µs	1.24x
client p99	448 µs	448 µs	~equal
client p99.9	493 µs	531 µs	1.08x
server p50	215 µs	277 µs	1.29x
server p99	336 µs	331 µs	~equal
wall clock	8.0 s	9.5 s	1.19x
queries sent	50,000	50,000
cpu	13.0 s	19.1 s	1.46x
cpu, % of wall	162%	200%
waiting for a core	2.0 ms	12.6 ms	6.20x
migrations	0	9,517
faults min/maj	0 / 0	65 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

W12-sel1-ef128 filtered recall control, one keyword, ef=128 (latency only) 1.55x

ef=128 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	5,996	3,862	1.55x
recall@10	1.0000	1.0000	~equal
client p50	310 µs	513 µs	1.66x
client p99	458 µs	593 µs	1.30x
client p99.9	495 µs	679 µs	1.37x
server p50	222 µs	413 µs	1.86x
server p99	344 µs	476 µs	1.39x
wall clock	8.4 s	13.0 s	1.55x
queries sent	50,000	50,000
cpu	13.5 s	26.0 s	1.92x
cpu, % of wall	162%	200%
waiting for a core	2.0 ms	17.1 ms	8.67x
migrations	0	9,573
faults min/maj	0 / 0	64 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

W12-sel1-ef256 filtered recall control, one keyword, ef=256 (latency only) 2.33x

ef=256 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	6,271	2,687	2.33x
recall@10	1.0000	1.0000	~equal
client p50	302 µs	740 µs	2.45x
client p99	452 µs	830 µs	1.84x
client p99.9	498 µs	936 µs	1.88x
server p50	214 µs	638 µs	2.98x
server p99	339 µs	712 µs	2.10x
wall clock	8.0 s	18.6 s	2.33x
queries sent	50,000	50,000
cpu	13.0 s	37.5 s	2.89x
cpu, % of wall	162%	201%
waiting for a core	2.5 ms	21.4 ms	8.54x
migrations	0	9,041
faults min/maj	0 / 0	199 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

W12-sel1-ef512 filtered recall control, one keyword, ef=512 (latency only) 3.66x

ef=512 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	6,228	1,701	3.66x
recall@10	1.0000	1.0000	~equal
client p50	304 µs	1.17 ms	3.85x
client p99	452 µs	1.32 ms	2.91x
client p99.9	497 µs	1.40 ms	2.82x
server p50	216 µs	1.07 ms	4.93x
server p99	339 µs	1.20 ms	3.54x
wall clock	8.1 s	29.4 s	3.65x
queries sent	50,000	50,000
cpu	13.1 s	59.4 s	4.53x
cpu, % of wall	162%	202%
waiting for a core	2.4 ms	30.5 ms	12.48x
migrations	0	13,852
faults min/maj	0 / 0	435 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

W12-sel10-ef32 filtered recall control, any of 10 keywords, ef=32 (latency only)

ef=32 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	3,172	4,685	-
recall@10	0.9777	0.8007
client p50	625 µs	422 µs
client p99	861 µs	534 µs
client p99.9	958 µs	603 µs
server p50	532 µs	319 µs
server p99	750 µs	423 µs
wall clock	15.8 s	10.7 s
queries sent	50,000	50,000
cpu	28.1 s	21.7 s
cpu, % of wall	178%	202%
waiting for a core	1.7 ms	9.0 ms
migrations	0	11,149
faults min/maj	0 / 0	166 / 0
peak RSS	5.2 GiB	5.2 GiB
disk written	0	0
disk read	0	0

recall unequal: 0.9777 vs 0.8007; §7.4 compares at equal recall

W12-sel10-ef64 filtered recall control, any of 10 keywords, ef=64 (latency only)

ef=64 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	2,098	3,377	-
recall@10	0.9954	0.9331
client p50	972 µs	591 µs
client p99	1.10 ms	717 µs
client p99.9	1.39 ms	790 µs
server p50	880 µs	488 µs
server p99	1.00 ms	605 µs
wall clock	23.9 s	14.8 s
queries sent	50,000	50,000
cpu	44.2 s	30.1 s
cpu, % of wall	185%	203%
waiting for a core	1.2 ms	10.2 ms
migrations	0	13,038
faults min/maj	0 / 0	135 / 0
peak RSS	5.2 GiB	5.2 GiB
disk written	0	0
disk read	0	0

recall unequal: 0.9954 vs 0.9331; §7.4 compares at equal recall

W12-sel10-ef128 filtered recall control, any of 10 keywords, ef=128 (latency only)

ef=128 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	1,358	2,226	-
recall@10	1.0000	0.9893
client p50	1.49 ms	902 µs
client p99	1.61 ms	1.07 ms
client p99.9	1.72 ms	1.14 ms
server p50	1.39 ms	796 µs
server p99	1.50 ms	961 µs
wall clock	36.9 s	22.5 s
queries sent	50,000	50,000
cpu	69.8 s	45.5 s
cpu, % of wall	189%	202%
waiting for a core	2.6 ms	10.5 ms
migrations	0	15,511
faults min/maj	0 / 0	250 / 0
peak RSS	5.2 GiB	5.2 GiB
disk written	0	0
disk read	0	0

recall unequal: 1.0000 vs 0.9893; §7.4 compares at equal recall

W12-sel10-ef256 filtered recall control, any of 10 keywords, ef=256 (latency only) 1.02x

ef=256 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	1,358	1,325	~equal
recall@10	1.0000	0.9992	~equal
client p50	1.49 ms	1.52 ms	~equal
client p99	1.61 ms	1.82 ms	1.13x
client p99.9	1.75 ms	1.93 ms	1.10x
server p50	1.39 ms	1.40 ms	~equal
server p99	1.50 ms	1.69 ms	1.13x
wall clock	36.9 s	37.8 s	1.02x
queries sent	50,000	50,000
cpu	69.8 s	76.1 s	1.09x
cpu, % of wall	190%	202%
waiting for a core	5.0 ms	31.4 ms	6.33x
migrations	0	18,323
faults min/maj	0 / 0	407 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

W12-sel10-ef512 filtered recall control, any of 10 keywords, ef=512 (latency only) 1.70x

ef=512 · closed-loop · collection=bench12 · client -p 2 -t 2 -c 1 (-p, -t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	1,359	798	1.70x
recall@10	1.0000	1.0000	~equal
client p50	1.49 ms	2.52 ms	1.69x
client p99	1.61 ms	3.00 ms	1.86x
client p99.9	1.71 ms	3.16 ms	1.85x
server p50	1.39 ms	2.39 ms	1.72x
server p99	1.50 ms	2.87 ms	1.91x
wall clock	36.8 s	62.7 s	1.70x
queries sent	50,000	50,000
cpu	69.8 s	126.1 s	1.81x
cpu, % of wall	190%	201%
waiting for a core	4.8 ms	57.6 ms	12.04x
migrations	0	18,614
faults min/maj	0 / 0	1,068 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

W13 scroll / pagination 1.10x

closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=200,000

metric	strawmANN	Qdrant
queries/second	51,192	46,560	1.10x
client p50	137 µs	158 µs	1.15x
client p99	230 µs	292 µs	1.27x
client p99.9	270 µs	441 µs	1.63x
server p50	11 µs	28 µs	2.56x
server p99	16 µs	90 µs	5.58x
wall clock	4.0 s	2.2 s	0.55x
queries sent	200,000	100,000
cpu	5.6 s	9.1 s	1.64x
cpu, % of wall	138%	411%
waiting for a core	3.2 ms	864.2 ms	269.26x
migrations	0	112,331
faults min/maj	0 / 0	708 / 0
peak RSS	5.2 GiB	5.2 GiB	~equal
disk written	0	0
disk read	0	0

strawmANN: -n 200000 (4x the table's 50000) Qdrant: -n 100000 (2x the table's 50000) strawmANN: 0.13 ms of the client's 0.14 ms p50 is not server time (12x), so 92% of this row is the load generator and the socket Qdrant: 0.13 ms of the client's 0.16 ms p50 is not server time (6x), so 82% of this row is the load generator and the socket 1.10x is 3.27x less work per query x 0.33x cores busy during the row (1.39 against 4.22) x 1.00x clock x 1.02x counted on-CPU share: the middle term is occupancy, not search speed

W11-steady mixed read/write below the rebuild threshold: search bench2 while 50,000 synthetic points append

ef=128 · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	18,228	5,537	-
client p50	394 µs	1.26 ms
client p99	1.96 ms	4.79 ms
client p99.9	3.01 ms	9.92 ms
server p50	316 µs	963 µs
server p99	1.11 ms	4.08 ms
wall clock	2.9 s	9.1 s
queries sent	50,000	50,000
cpu	18.9 s	88.5 s
cpu, % of wall	662%	968%
waiting for a core	1,345.8 ms	54,590.2 ms
migrations	0	116,172
faults min/maj	229,802 / 0	936,740 / 224,007
peak RSS	5.2 GiB	8.7 GiB
disk written	24.5 MiB	3.5 GiB
disk read	0	365.6 MiB

strawmANN: qps over the 2.7 s the append ran (18,221 over the whole search); write overlap 11%; search covered 11% of the append; append 2,000 points/s Qdrant: qps over the 9.0 s the append ran (5,533 over the whole search); write overlap 36%; search covered 37% of the append; append 2,000 points/s search-during-write; no recall join; the search saw under 90% of either append, so its qps is the append's start and not all of it

W11 mixed read/write: search bench2 while 200,000 synthetic points append (runs last)

ef=128 · closed-loop · collection=bench2 · client -p 8 -t 2 -c 1 (-t, -c bfb default) · n=50,000

metric	strawmANN	Qdrant
queries/second	8,033	2,387	-
client p50	430 µs	2.62 ms
client p99	4.09 ms	14.09 ms
client p99.9	6.06 ms	25.83 ms
server p50	348 µs	2.15 ms
server p99	3.36 ms	13.04 ms
wall clock	6.3 s	21.1 s
queries sent	50,000	50,000
cpu	49.5 s	267.7 s
cpu, % of wall	781%	1,271%
waiting for a core	17,678.2 ms	135,269.8 ms
migrations	111	108,738
faults min/maj	810,544 / 0	1,842,780 / 391,257
peak RSS	5.2 GiB	14.7 GiB
disk written	98.1 MiB	6.6 GiB
disk read	0	736.2 MiB

strawmANN: qps over the 6.2 s the append ran (8,029 over the whole search); write overlap 10%; search covered 10% of the append; append 3,300 points/s Qdrant: qps over the 20.9 s the append ran (2,386 over the whole search); write overlap 35%; search covered 35% of the append; append 3,300 points/s search-during-write; no recall join; the search saw under 90% of either append, so its qps is the append's start and not all of it

Host discipline the §7.1 gate, per check, and ambient load per row

Ambient load per row

The §7.1 gate checks the machine once, at the start. Colour is the engine, as everywhere else; a hatched bar with a red edge is a row that had another process on the box while it ran, which the gate cannot see.

AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M (24 logical) gate pass

One machine: both runs were measured on it, and the checks below are the same for both. What differed at each run start is listed underneath, per run.

okAMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M (24 logical)

okgovernor=performance

okboost disabled

oksmt=on

declared rather than required (§7.1 asks for SMT on or off, not

for off), and hashed, so these rows never share a chart with

SMT-off ones

cpu N and cpu N+12 share a core (12 pairs);

a --server-cpus or --client-cpus set holding both cpus of a

pair measures the engine on half the cores it names

okprofile=as-deployed (isolcpus=none nohz_full=none)

the scheduler is running as it ships, so these numbers describe a

deployment rather than the engine in isolation; they may not be

compared against an `isolated` run, and the environment hash

enforces that

okthp=madvise numa_nodes=1 numa_balancing=0

okperf_event_paranoid=-1

oktask_delayacct=1, so time blocked on a device is measurable

okperf at /usr/bin/perf, so `workloads.py run --perf` can attach

okcgroup io controller reaches the engines' own scope, so block-layer read/write operations are measurable

okzig=0.16.0

busy: Xorg(5%)

measurement profile: as-deployed

strawmANN at run start · environment hash 373e8531fd3d1bf6

Qdrant at run start · environment hash 373e8531fd3d1bf6

What this does not establish

No Qdrant developer has used this tool. Nothing here has been run, checked against something already known, or disagreed with by anyone outside the project: every number has one author and one reviewer, and they are the same person. A result you can refute is more useful to us than one you accept.

It has never been pointed at a real Qdrant regression. The harness detects a known ISA slowdown and correctly reports no difference on a control row. That is internal consistency, not evidence it would flag a regression in your tree or stay quiet through a refactor. bench/harness/qdrant_ab.py exists to run two of your commits through it blind, and that experiment has not been done.

Part of the measured throughput gap is kernel width, not architecture. Read from a dev checkout: Qdrant's distance path tops out at four 256-bit accumulators and has no AVX-512, while these kernels use 512-bit ones on a machine that has them. That is a real difference and it is not the same claim as "the design is faster".

The engines are not equivalent, by construction. §1 removes sharding, replication, consensus, snapshots, sparse vectors, multivectors and disk-resident operation — permanently. Payload storage and filtering were in that list until 2026-09-03 and are now built (M7), so W12 measures a filtered search with a keyword index on both engines. Anything still unbuilt answers UNIMPLEMENTED naming the construct and reports n/a, never a silent degradation. A strawman that was not faster would mean it was badly built; the question is by how much, and where the model was wrong.

Reproducing this

Everything below runs from a clone. The engine has no dependencies; the analysis path is a uv project pinned by bench/uv.lock.

scripts/doctor.py                    # what can this machine measure?
bench/setup.py check                 # §7.1 host gate: governor, boost, isolation, idle
conformance/datasets/datasets.py fetch

# one engine at a time, §7.1
zig build -Doptimize=ReleaseFast
# --connections matters: W4 opens ~32 sockets (-t 16 -c 2) and a server with
# fewer closes the excess before the HTTP/2 preface, which the client reports
# only as "transport error".
./zig-out/bin/strawmann --port 6334 --connections 64
bench/harness/workloads.py run http://localhost:6334 strawmann --sink --report

# the other half of every number: recall, and the licence to compare at all
cd conformance
cargo run --release -- relevance --engine http://localhost:6334 --label strawmann \
  --base $DATA/sift1m.fbin --queries $DATA/sift1m_query.fbin \
  --ground-truth $DATA/gt/sift1m.euclid.k100.gt.json --metric euclid \
  --ef 32 64 128 256 512 --json ../bench/results/strawmann/recall.json
cargo run --release -- differ --strawmann http://localhost:6334 --qdrant http://localhost:6434 \
  --base $DATA/sift1m.fbin --queries $DATA/sift1m_query.fbin \
  --ground-truth $DATA/gt/sift1m.euclid.k100.gt.json \
  --json ../bench/results/strawmann/conformance.json

uv run --project bench bench/harness/report.py strawmann qdrant

Disagreements are the point. docs/spec.md is what all of this is measured against, docs/bugs.md lists the measurement bugs found so far (each produced a plausible wrong number rather than a failure), and docs/validation.md records what has and has not been checked.

Generated by bench/harness/report.py from bench/results/. Self-contained: charts are embedded, nothing is fetched at open time. Regenerate with uv run --project bench bench/harness/report.py sm-sift-perf-0930 qd-sift-perf-0930.

Summary

Throughput against recall

Throughput

Throughput by workload

Recall

At matched recall

At matched recall, SQ8

At matched recall, filtered to 10%

Recall by ef

What each quantization costs in recall

Latency

Ingest and index build

Where the load time goes

Memory and disk

Appendix

Dataset

Host §7.1 gate pass

W10: throughput against ef

Per workload

Time spent waiting for a core

Peak memory per row

Time stalled on I/O

What one query cost, in cycles

Demand loads from DRAM per query

TLB walks per query

Instructions per cycle

Branch mispredictions per 1k instructions

W0-upload transport floor: load

W0 d=4 floor: graph traversal with the distance taken out 1.23x

W1 ingest throughput

W2 index build time

W3 search, fp32, single query 0.85x

W4 search, saturating (closed loop) 1.63x

W4-sat50 search, fixed rate at 50% of saturation (open loop) offered

W4-sat70 search, fixed rate at 70% of saturation (open loop) offered

W4-sat90 search, fixed rate at 90% of saturation (open loop) offered

W5 search batched (16 distinct dataset queries per request) 4.82x

W6-upload scalar quantization: load

W6 quantized: scalar 0.94x

W6-ef32 SQ8 recall control, ef=32 (latency only)

W6-ef64 SQ8 recall control, ef=64 (latency only) parity

W6-ef128 SQ8 recall control, ef=128 (latency only) 0.94x

W6-ef256 SQ8 recall control, ef=256 (latency only) parity

W6-ef512 SQ8 recall control, ef=512 (latency only) parity

W7-upload binary quantization: load

W7 quantized: binary + oversampling

W8-upload PQ: load

W8 quantized: PQ 1.08x

W9 exact / brute force 1.50x

W10-ef32 recall control, ef=32 (latency only)

W10-ef64 recall control, ef=64 (latency only) 2.03x

W10-ef128 recall control, ef=128 (latency only) 2.03x

W10-ef256 recall control, ef=256 (latency only) 1.83x

W10-ef512 recall control, ef=512 (latency only) 1.67x

W12-upload filtered search: load 200,000 with payloads

W12-sel1 filtered search, one keyword (~1% of bench12) 1.49x

W12-sel10 filtered search, any of 10 keywords (~10% of bench12)

W12-sel1-ef32 filtered recall control, one keyword, ef=32 (latency only) parity

W12-sel1-ef64 filtered recall control, one keyword, ef=64 (latency only) 1.19x

W12-sel1-ef128 filtered recall control, one keyword, ef=128 (latency only) 1.55x

W12-sel1-ef256 filtered recall control, one keyword, ef=256 (latency only) 2.33x

W12-sel1-ef512 filtered recall control, one keyword, ef=512 (latency only) 3.66x

W12-sel10-ef32 filtered recall control, any of 10 keywords, ef=32 (latency only)

W12-sel10-ef64 filtered recall control, any of 10 keywords, ef=64 (latency only)

W12-sel10-ef128 filtered recall control, any of 10 keywords, ef=128 (latency only)

W12-sel10-ef256 filtered recall control, any of 10 keywords, ef=256 (latency only) 1.02x

W12-sel10-ef512 filtered recall control, any of 10 keywords, ef=512 (latency only) 1.70x

W13 scroll / pagination 1.10x

W11-steady mixed read/write below the rebuild threshold: search bench2 while 50,000 synthetic points append

W11 mixed read/write: search bench2 while 200,000 synthetic points append (runs last)

Ambient load per row

AMD Ryzen AI 9 HX PRO 370 w/ Radeon 890M (24 logical) gate pass

Recall by `ef`