Why should point uploads be batched instead of sending one upsert request per point?
Batching improves ingestion throughput
Each network request has fixed overhead: connection handling, serialization, request parsing, acknowledgement, and scheduling. Sending one point per request repeats that overhead millions of times.
Batching amortizes those costs across many points and gives Qdrant a larger unit of work for indexing and storage. This generally improves ingestion throughput and reduces application-side request overhead.
The trade-off is batch size. Very large batches can increase memory usage, request payload size, retry cost, and latency for individual failures. I would choose a starting size based on payload size and then benchmark throughput, CPU, memory, and tail latency.
A common mistake is assuming the largest possible batch is always fastest. Once batches become large enough, serialization, network limits, indexing contention, or memory pressure can dominate. The correct size is workload-dependent.
- 1
Batching amortizes per-request network and serialization overhead
- 2
Larger batches can improve indexing throughput
- 3
Oversized batches increase memory, request, and retry costs
- 4
Tune batch size using the actual payload and indexing workload
Scenario Questions
0-2 years experience
- 1Your importer sends 1 million HTTP requests for 1 million points. What obvious performance issue would you investigate first?
- 2Why does sending 100 points in one request usually reduce overhead compared with sending 100 separate requests?
2-5 years experience
- 1A batch of 10,000 points is faster than batches of 100, but occasionally times out. How would you find a better operating point?
- 2Your network has a request-size limit and embeddings are large. How would you choose a batch size safely?
5-8 years experience
- 1A bulk-ingestion pipeline is CPU-bound in Qdrant rather than network-bound. Would increasing batch size necessarily help? How would you investigate?
- 2A batch fails after partially completing downstream processing. How would you design retries so successful work is not duplicated incorrectly?
8+ years experience
- 1You need to ingest billions of vectors while keeping production query latency within an SLO. How would you architect batching, concurrency, and indexing workload isolation?
- 2Ingestion throughput is high, but large batches create p99 latency spikes for search. What controls would you introduce to prevent bulk loading from starving interactive queries?
Follow-up Questions
- What factors determine a good batch size?
- Why can an extremely large batch be slower or less reliable?