Questions
12 of 13
1What role does Qdrant play in a typical RAG architecture, and what happens on either side of it in the pipeline?
2How would you design chunking and metadata so that retrieved chunks can be traced back to their source document and section for citation?
3A RAG system is returning chunks that are topically related but don't actually answer the user's question. How would you improve retrieval quality?
4How would you handle access control in a RAG system where different users are only permitted to retrieve chunks from documents they have permission to view?
5Why might you keep conversation-turn embeddings in a separate, short-lived collection rather than mixing them into your main document knowledge base?
6How would you model 'users who liked this also liked' recommendations using Qdrant's recommend/discovery query modes?
7How would you incorporate business signals like popularity or recency into a similarity-based recommendation without abandoning vector search entirely?
8What cold-start problem exists for a new item or new user in a vector-similarity recommendation system, and how might you mitigate it?
9How would you evaluate whether a change to your recommendation retrieval pipeline actually improved results, before rolling it out to all users?
10Design a Qdrant-backed search feature for a SaaS product with thousands of small customers, each with their own private dataset. What collection and sharding strategy would you use?
11One large enterprise tenant has 100x more data than a typical tenant in your shared multitenant collection. What problems could this cause, and how would you address them?
12How would you offer per-tenant usage metrics (storage, query volume) in a shared multitenant Qdrant deployment?
13What is the tradeoff of offering tenants a 'bring your own embedding model' option in a shared collection?
12 / 13

How would you offer per-tenant usage metrics (storage, query volume) in a shared multitenant Qdrant deployment?

Application-level metering built on tagged payload counts and query logging

Qdrant does not natively bill or meter per tenant, so the metrics must be built at the application level. Storage metrics come from counting the points and the payload size per tenant: since every point carries a tenant_id, you can count points per tenant by iterating over the collection or by maintaining a running count as points are upserted. Payload size is harder because Qdrant does not report per-point payload bytes directly; the common approach is to estimate from the payload schema and the average payload size, or to maintain a per-tenant byte counter in the application as points are written. Query volume comes from logging every query with its tenant_id and aggregating the logs. The application-level metering has the advantage of being flexible - you can define whatever metrics matter to the business - and the disadvantage of being separate from the database, so it must be kept in sync and it can drift if not carefully maintained.

The mechanism has three parts: counting, logging, and aggregating. Counting storage: on every upsert, increment a per-tenant counter by the size of the point (vector bytes plus payload bytes); on every delete, decrement. This gives a real-time estimate of storage per tenant. Alternatively, periodically scan the collection and count points per tenant using the payload index - this is accurate but expensive for large collections. Logging queries: every query goes through the trusted service that enforces the tenant filter, so the service can log the tenant_id, the query type, the latency, and the result count. Aggregating: a batch job or a stream processor aggregates the logs into per-tenant metrics (queries per hour, p99 latency, error rate) and stores them in a metrics store. The metrics can then be exposed in a dashboard or used for billing. The key requirement is that every query and every write goes through the trusted service, so there is a single place where the metering happens. If clients can talk to Qdrant directly, the metering will be incomplete and can be bypassed.

  1. 1

    Storage: maintain a per-tenant counter of points and bytes, updated on upsert and delete.

  2. 2

    Payload bytes: estimate from the schema or maintain a running counter in the application.

  3. 3

    Query volume: log every query with its tenant_id in the trusted service.

  4. 4

    Aggregation: batch or stream processing to produce per-tenant metrics.

  5. 5

    Metrics store: a time-series database or a metrics platform for the aggregated data.

  6. 6

    Billing: the metrics feed into the billing system for usage-based pricing.

  7. 7

    Dashboards: per-tenant visibility for the tenant and for the operations team.

  8. 8

    Accuracy: reconcile the application counters against periodic scans of the collection.

The trade-off is between accuracy and cost. Real-time per-tenant metering with counters is cheap but can drift; periodic reconciliation with a full scan is accurate but expensive. The right approach depends on how accurate the metrics need to be - for billing, they must be accurate and reconcilable; for dashboards, an estimate may be sufficient. The common mistake is to rely on Qdrant's collection-level metrics, which are not per-tenant. The second mistake is to not log queries, so query volume is unknown. The third mistake is to let clients bypass the trusted service, so the metering is incomplete. The fourth mistake is to not reconcile the counters, so the storage numbers drift over time. The fifth mistake is to meter only successful queries and not errors, so the error rate per tenant is invisible. Version note: Qdrant's metrics and the collection info API have evolved, but they are collection-level, not tenant-level. The per-tenant metering must be built in the application. The payload index on tenant_id enables efficient counting if you choose to scan periodically.

javascript

Version-dependent: Qdrant's collection info and metrics APIs have evolved across releases, but they remain collection-level. Per-tenant metering must be built in the application. The scroll API and the payload index on tenant_id are the tools for reconciliation, and their API shapes have changed with the query_points API in qdrant-client 1.10+.

Difficulty: 6/10
Topics: Multitenancy, Metering, Observability

Scenario Questions

0-2 years experience
  1. 1

    You need to show each tenant how many queries they made last month. Describe where you would get that data.

  2. 2

    A teammate looks for per-tenant metrics in Qdrant's collection info. Explain why they are not there and where they come from.

2-5 years experience
  1. 1

    You are building usage-based billing for a multitenant search product. Describe the metering architecture and how you would reconcile the numbers.

  2. 2

    Your per-tenant storage counters drift from the actual counts. Diagnose the cause and propose a reconciliation strategy.

5-8 years experience
  1. 1

    Design a metering and billing system for a multitenant Qdrant deployment, including the storage, query, and error metrics, the aggregation, and the reconciliation.

  2. 2

    You need to expose per-tenant metrics in a dashboard with near-real-time latency. Describe the pipeline and the trade-offs.

8+ years experience
  1. 1

    You are designing a metering system that must be accurate enough for billing and cheap enough to run at scale. Describe the architecture, the reconciliation, and the error bounds.

  2. 2

    Derive the cost of the metering system as a function of query rate, tenant count, and reconciliation frequency. Where does the cost become prohibitive?

Follow-up Questions

  • How would you reconcile the per-tenant storage counters against the actual collection to detect drift, and how often would you run the reconciliation?
  • If a tenant disputes the usage metrics, what evidence would you provide to support the numbers?