Should you cap a vector shard at 100,000 vectors? We tested the advice on 446,000 passages, on one machine and on three. Smaller shards find slightly more of the nearest vectors, and no more of the right pages.
- The advice: keep each shard to 50,000 or 100,000 vectors, because large ones grow "hubs" and quietly miss passages.
- It is partly right. At the same search effort, smaller shards find more: 95.1% of the nearest passages with one shard, 99.0% with ten, at a low effort.
- But effort is spent in every shard. Compared at equal accuracy, one shard found 99.73% at 745 searches a second, and five shards found 99.65% at 440.
- Almost nothing was out of reach. Searching with every one of 445,810 passages, 98 failed to find themselves at effort 1,600 (0.02%), and one shard reached 100% of the nearest passages on effort alone.
- On three machines, copies handled 3 times the searches of shards: 2,127 a second at 99.82%, against 681 at 99.98%. Near the very top of accuracy that may reverse.
- Against an answer key written by people, shards changed nothing: the labelled page was in the top 40 for 98 of 150 questions, in every layout at effort 1,600, and for exact search.
Qdrant, FinanceBench filings, vector search alone, before keyword search and reranking. Not the score of a finished answer.
Two different things are called hubs.
A vector database finds the passages closest in meaning to a question. In a space of 1,024 dimensions, as here, an odd thing happens: a few vectors end up close to almost everything. They turn up in the results of many unrelated questions. Those are hubs in the data, and the classic example is a bland passage that vaguely resembles every query.
Cutting a collection into shards does not remove these. A search asks every shard for its best results and merges them, so the whole collection still competes and a hub still ranks high.
The second kind is where the advice has a point. Most engines, Qdrant included, search a graph (HNSW) of links between vectors, and a search walks those links without visiting everything. In a large graph, popular vectors collect many links and others get few. These are hubs in the graph, and a poorly linked passage can be hard to reach even when it is the right answer. Smaller shards mean smaller graphs, and each one returns its own best results. So the worry is reasonable: does a right passage go missing without anyone noticing?
Smaller shards find more, for more work.
We took 445,810 searchable passages, in a store of 485,343 records, from FinanceBench filings, embedded with bge-m3, and loaded them into one shard, 5 shards (about 97,000 records each) and 10 shards (about 48,500 each): the sizes the advice names. Exact search, which compares a question with every vector, gives the exact closest 40 passages, and graph search is scored on how many of those it finds, over the 150 real FinanceBench questions. Search effort is how many candidates the search examines in each graph.
| One machine | Effort | Nearest 40 found | Searches a second |
|---|---|---|---|
| 1 shard | 128 | 95.1% | 3,598 |
| 1 shard | 400 | 98.8% | 2,047 |
| 1 shard | 1,600 | 99.73% | 745 |
| 5 shards | 128 | 98.5% | 1,031 |
| 5 shards | 400 | 99.65% | 440 |
| 5 shards | 1,600 | 99.93% | 134 |
| 10 shards | 128 | 99.0% | 501 |
| 10 shards | 400 | 99.87% | 216 |
| 10 shards | 1,600 | 99.98% | 65 |
One dedicated 4-vCPU machine, 32 connections, no errors. Three builds of the one-shard layout ranged from 94.8% to 95.9% at effort 128, and from 99.73% to 99.82% at 1,600.
Read a single effort across the shards and the advice looks strong: more found at every step. But the effort is spent in every shard, so 5 shards at 400 examine about as many candidates as one shard at 2,000. The fair question is what each layout costs for the same accuracy. One shard at 1,600 finds 99.73% at 745 searches a second. Five shards at 400 find 99.65% at 440.
How far can effort alone go?
Shards reach 99.9% at a lower effort per shard, so we kept raising the effort on one shard to see if it gets there too. It does. This run used the live collection, which holds one graph of 484,975 records, a different build from the table above: that is why effort 1,600 reads 99.70% here and 99.73% there.
| One shard, effort | Nearest 40 found | One search |
|---|---|---|
| 1,600 | 99.70% | 16 ms |
| 3,200 | 99.90% | 24 ms |
| 6,400 | 99.95% | 42 ms |
| 12,800 | 99.98% | 78 ms |
| 25,600 | 100% | 143 ms |
| Exact search | 100% | 158 ms |
The earlier run's smaller shared machine, one search at a time: read the times against each other, not against the tables above.
Total effort tells the story. To find 99.98%, one shard needed 12,800, and ten shards needed 10 × 1,600 = 16,000. For about 99.9%, one shard needed 3,200, and ten shards 10 × 400 = 4,000. Throughput, meanwhile, falls almost in proportion to the number of shards at high effort: 745, 134 and 65 searches a second at effort 1,600. The top end is not free for anyone: at 25,600 the search takes nearly as long as exact search, which always finds 100% on one machine, slowly. Near the very top, shards may still return a single search sooner, because they work side by side; we only measured that roughly.
Could a passage become unreachable?
This is the failure the advice fears, so we tested it directly on the one big graph, with every searchable passage. We searched with each one's own vector: the search should find that passage, or an identical twin. Of 445,810, 98 did not at effort 1,600, or 0.02%. So the failure is real, and rare. Exact search found all 98, so it is the graph that lost them. Searching with a stored vector is the easy case, as the trap below explains, so real questions may fare a little worse.
Did we find hubs in the data?
Mildly. The standard measure counts how often each passage appears among the 10 nearest neighbours of the others. Over all 445,810 passages the average is 10, the skew of that count is 1.31, and the most frequent passage appeared 122 times, with only 3 above 100. The top repeaters are boilerplate: cover-page wording and benefit-plan text. 3,522 passages were never anyone's neighbour. That is a mild tail, not a collection dominated by hubs.
Exact search cannot mark its own homework.
Everything so far is measured against exact search. That settles whether a graph loses passages: exact search uses no graph, and it returned identical results on all 150 questions with 1, 5 and 10 shards. But it says only which vectors are nearest. If hubs crowd out the right passage, exact search returns those hubs too, so it cannot catch that failure.
That needs a key no search produced. FinanceBench has one: for every question, people marked the pages that hold the answer.
| Layout | Effort | Labelled page in top 10 | In top 40 |
|---|---|---|---|
| Exact search | none | 65 of 150 | 98 of 150 |
| 1 shard | 128 | 65 | 99 |
| 1 shard | 1,600 | 65 | 98 |
| 5 shards | 1,600 | 65 | 98 |
| 10 shards | 128 | 64 | 98 |
| 10 shards | 1,600 | 65 | 98 |
Vector search alone over the whole store, before keyword search and reranking, so this is not the score of a finished answer. One run for each layout.
Smaller shards found no more of the right pages. Even the lowest effort made almost no difference here: the passages a graph drops are rarely the ones people marked.
Exact search does miss the labelled page, on 52 of the 150 questions, so nearest is not the same as right. But those misses do not look like hubs. Their 2,080 result places held 1,935 different passages, none more than 4 times, where hubs would show as repeats. The right document was in the results for at least 49 of the 52: the search reached it and landed on its other pages. That is the limit of one vector search, and it is why a retrieval pipeline adds keywords and a reranker.
The advice's best case: low effort with a filter.
A filter can break up the links a graph search walks, and this is the strongest argument for smaller graphs. We rebuilt one shard and 10 shards with a filter field and limited each search to the part of the store holding the question's document.
| Filter | Effort | 1 shard | 10 shards | Labelled page in top 40 |
|---|---|---|---|---|
| About 20% of the store | 128 | 96.45% | 99.92% | 122 both |
| About 20% of the store | 400 | 99.28% | 100% | 122 both |
| About 20% of the store | 1,600 | 99.97% | 100% | 122 both |
| About 2% of the store | 128 | 99.77% | 100% | 130 both |
| About 2% of the store | 1,600 | 100% | 100% | 130 both |
Nearest 40 found, against exact search under the same filter. These builds hold the 445,810 searchable passages and a filter index, and read higher than the tables above even with no filter (98.05% at effort 128). We have not established why: removing the store's non-searchable records alone did not do it.
At low effort the advice has a real point: one shard fell behind, and on one question the search returned none of the nearest 40. At effort 1,600 the gap is gone. The right pages were the same in every case.
Real questions are harder than stored passages.
A common shortcut is to test a search with passages already in the store, because that needs no questions. It flatters the result. In that same earlier run, one shard at effort 128 found 99.65% of the nearest 40 for stored passages, and only 95.9% for real questions. Questions are short and look unlike the passages they match, so the graph has a harder path to them.
If you tune search effort on stored passages, you can settle on a value that looks near perfect and misses 4 in 100 of the nearest passages for real questions. Test with real questions.
Copies add searches. Shards add accuracy and room.
Shards belong on separate machines, so we gave them three. The same collection went onto the same three machines in three ways: a full copy on each, one shard on each, and 12 shards, 4 on each, which is what VectoGrid's own resharding produces.
| Three machines | Effort | Nearest 40 found | Searches a second | p50, light load | p99, light load |
|---|---|---|---|---|---|
| 3 copies of 1 shard | 400 | 98.95% | 5,879 | 2.0 ms | 3.0 ms |
| 3 copies of 1 shard | 1,600 | 99.82% | 2,127 | 5.6 ms | 9.5 ms |
| 3 shards, 1 on each | 400 | 99.48% | 1,984 | 3.4 ms | 6.1 ms |
| 3 shards, 1 on each | 1,600 | 99.98% | 681 | 9.3 ms | 14 ms |
| 12 shards, 4 on each | 128 | 99.35% | 999 | 6.5 ms | 24 ms |
| 12 shards, 4 on each | 400 | 99.92% | 459 | 14 ms | 21 ms |
| 12 shards, 4 on each | 1,600 | 99.98% | 159 | 38 ms | 84 ms |
Three dedicated 4-vCPU machines, no errors. Searches a second at 96 connections spread evenly, every machine close to 100% busy; latency at 6 connections. One machine alone: 719 at effort 1,600 in the same run.
- Copies multiplied the searches: 719 on one machine, 2,127 on three.
- Three shards on three machines added no searches over one machine: 681 against 719, because every search still visits every machine. What the extra machines bought was accuracy at the same effort, 99.98% against 99.73%, and room for a collection that does not fit on one.
- Copies did better than shards around 99.5%: 99.82% at 2,127 beats 3 shards at 99.48% and 1,984, and 12 shards at 99.35% and 999.
- Near the very top, shards may win. Copies and shards above are not at equal accuracy. To reach 99.98%, copies would need an effort near 12,800, and scaling from single-search times suggests about 450 searches a second, below the 681 of 3 shards. That is an estimate: we did not measure it.
- Shards cost most in the tail. At effort 1,600 under light load, p99 was 9.5 ms with copies, 14 ms with 3 shards and 84 ms with 12: a search waits for its slowest shard.
- Separate machines ease the cost of shards a lot: ten shards on one machine ran at 65 searches a second at effort 1,600, and twelve over three machines at 159.
Copies need the load spread
In a separate test on 50,000 passages, with vector and keyword search, filters and passage text returned, one, two and three copies gave 497, 975 and 1,490 searches a second. Writes do not grow: 145, 138 and 133 documents a second, since every copy takes every write.
- One address adds nothing. With 3 copies and all the load on one machine: 495 searches a second, and the other two machines idle. A machine answers from its own copy.
- A load balancer gives the full speed: 1,590 searches a second, with no errors.
- It needs enough connections. A load balancer spreads connections, not requests, and a connection stays on the machine it landed on: 1 connection reached 115 searches a second on one machine, 16 reached 1,380, and 64 reached about 1,590.
None of this is the number of users you can serve. A new question also needs an embedding and a reranker, and an answer needs a language model. Those models have limits of their own, which we did not measure here, and copies of the store do nothing for them.
What this says, and what it does not.
- One engine, one collection. Qdrant, 446,000 passages of financial filings, one embedding model. A much larger collection, or other data, may behave differently, and at some size a cap may pay for itself.
- This collection fits on one machine. When it does not, shards are the answer, and this test says nothing against them. It tests sharding a collection that did not need it.
- Throughput at the highest efforts was estimated, not measured. For one shard above effort 1,600 we timed single searches only.
- 150 questions, and builds vary. The same layout built three times moved by about a point at low effort and 0.1 at high effort: differences that small should not be read closely.
- Keyword search and final answers. Neither was part of the shard test.
- A graph that has just been built. Updates and deletes can wear a graph down over time, and we tested fresh ones.
- Accuracy is not the only reason for a cap. Rebuild time, snapshot and restore, and how finely data can be rebalanced are real arguments for smaller shards. This article does not weigh them.
- Graph settings. Raising m from 16 to 32 gave a small gain at low effort (98.58% against 98.05%), nothing at 1,600, and the same labelled pages.
- Our own position. VectoGrid starts every collection in one shard and reshards to 12 when it outgrows a machine, so this result suits our default. Read it knowing that.
- The labels are pages, from one benchmark. A figure can also appear on a page nobody marked, and 150 questions cannot rule out a rare effect.
How the test was run
- Main runs: Qdrant 1.20, Hetzner ccx23 (4 dedicated vCPU, 16 GB), one cx33 sending load with wrk over REST on a private network. float16 vectors, HNSW with m 16 and ef_construct 100, default segment settings (default_segment_number 0, no max_segment_size), every vector indexed, 40 results without passage text. Each shard held 2 segments.
- Reach, hubs, stored passages, high effort and the answer key: Qdrant 1.17 on a cx33, 1,000 stored passages as queries beside the 150 questions. The one-shard collection there holds one graph of 484,975 records, beside a 368-record segment that holds no graph, so these runs did test one big graph. That run's speeds are not quoted: its test client shared the machine with the server, and gave figures that were not comparable.
- Copies test: 50,000 passages, Qdrant 1.20, one shard copied with Qdrant's own replication call. Load balancer: Hetzner lb11, TCP, round robin.
For how a VectoGrid index grows from one machine to many, see From one report to millions.
Sizing search for a large library?
Published 7 November 2026 · Philippe Vergnet, VectoGrid. Revised the same day: the answer-key test and latency added, "true" results renamed "nearest", the passage count corrected, filters, a full reach and hub count and m 32 added.