Write the domain event to an outbox table in the same database transaction as the business entity write. A separate relay service polls the outbox and publishes unpublished events to Kafka, then marks them as published. This guarantees the event is eventually published even if the app crashes between the write and the publish.
The event and business entity are written atomically — no event is lost even if the process crashes after the DB write.
The relay is idempotent — if it crashes mid-relay, unpublished events are picked up on the next poll.
Use a distributed lock (Redis SETNX) if multiple relay instances run concurrently to prevent duplicate publishes.
Mark events as published after Kafka confirms receipt — not before.
Add a dead-letter column for events that fail repeatedly so operators can inspect and replay them.
Suppose you need to write a NestJS service that creates a new order and publishes an 'order.created' event to Kafka. How would you use the outbox pattern to ensure the event is only sent if the order is successfully saved?
If the outbox table contains a row that hasn't been processed yet, what will happen when your background worker picks it up and the Kafka broker is temporarily unavailable?
Can you walk me through the steps your code would take from receiving the HTTP request to committing the outbox record?
We have a microservice that writes to Postgres and then publishes to Kafka using the outbox pattern. Lately, some messages are being duplicated after a service restart. What could cause that and how would you fix it?
Imagine you need to add a new field to the outbox payload without breaking existing consumers. How would you evolve the schema while keeping exactly‑once semantics?
During a load test, the outbox processing thread becomes a bottleneck. What trade‑offs would you consider to improve throughput while preserving ordering guarantees?
Design a scalable outbox processing component for a NestJS application that handles millions of events per day. Discuss partitioning, idempotency, and how you would monitor delivery failures.
How would you integrate Kafka's transactional producer API with NestJS's TypeORM transaction to achieve exactly‑once delivery across the database and Kafka?
If a downstream consumer crashes after processing a message but before committing its offset, how does your outbox design prevent duplicate processing, and what additional safeguards would you add?
Our organization is moving from a monolith to multiple NestJS services that all use the outbox pattern with a shared Kafka cluster. What architectural guidelines would you set to avoid cross‑service ordering issues and to standardize outbox implementations?
We have legacy services that directly publish to Kafka without an outbox. How would you plan a migration strategy that minimizes downtime and ensures exactly‑once semantics across the whole system?
Consider long‑term maintenance: how would you version the outbox schema and handle data migrations without losing in‑flight messages, especially when multiple teams own different services?