04 / 10

Describe how you would approach a Go service that needs to process Kafka events with exactly-once semantics.

True exactly-once requires idempotent consumers with deduplication stored in the DB, the transactional outbox pattern for producing events, and Kafka consumer group offset commits tied to successful processing.

Idempotent consumer pattern
Exactly-once checklist
  1. 1

    Idempotent consumer: processed_events table with UNIQUE(event_id) — ON CONFLICT DO NOTHING skips duplicates

  2. 2

    Transactional outbox: write event and state change in same DB transaction — relay publishes to Kafka separately

  3. 3

    Commit offsets only after successful DB write — ensures re-processing on failure

  4. 4

    Kafka transactions + transactional producers for Kafka-to-Kafka pipelines (not DB sinks)

  5. 5

    segmentio/kafka-go or confluent-kafka-go for production Kafka consumers in Go