The full picture
@Transactional
public void handle(PaymentRequested event) {
// The key must be stable across retries: derive it from the business event,
// never from the delivery attempt.
String key = event.paymentId();
try {
processedRepo.insert(key, Instant.now()); // UNIQUE constraint on key
} catch (DuplicateKeyException dup) {
return; // already handled — drop it
}
ledger.debit(event.accountId(), event.amount()); // same transaction as the marker
gateway.charge(event.amount(), key); // key -> gateway dedupes too
}
// The uniqueness constraint is what makes this safe under CONCURRENT duplicates,
// not just sequential ones: a "SELECT then INSERT" check has a race window and
// two consumers can both pass it.| Semantic | Mechanism | Risk | Fit |
|---|---|---|---|
| At-most-once | fire and forget; ack before processing | message loss | metrics, logs, telemetry |
| At-least-once | retry until acked; ack after processing | duplicates | the default — pair with idempotency |
| Exactly-once processing | at-least-once + dedupe/idempotency | storage for keys | payments, orders, anything with side effects |
| Kafka EOS | idempotent producer + transactions | Kafka-internal only | stream processing read→process→write |
- Ack timing decides the semantic, and it's a one-line change: ack before processing = at-most-once; ack after = at-least-once. Say which one your framework does by default (Spring's
AckMode, SQS visibility timeout, Kafka'senable.auto.commit) — teams are frequently surprised by their own configuration. - The transactional outbox pattern is the answer to 'how do I update the database and publish an event atomically': write the event to an
outboxtable in the same local transaction, and let a relay (or CDC via Debezium) publish it. This avoids the dual-write problem, where a crash between the DB commit and the broker publish loses the event or invents one. - Dedupe storage needs a retention policy — you can't keep idempotency keys forever. Pick a window comfortably longer than your maximum retry/redelivery horizon (24h–7d is typical), index by key, and TTL the rest. Unbounded dedupe tables are a slow-burning operational problem.
- *Idempotency is not only about duplicates — it's about reordering too.* A retry can arrive after a subsequent update. Guard with a version/sequence check (
WHERE version = ?) so a stale replay can't overwrite newer state; otherwise your consumer is idempotent but not order-safe. - Be precise about Kafka's exactly-once: it covers consume→transform→produce within Kafka, atomically committing offsets and output records. The moment your consumer calls an external API or another database, you're back to at-least-once and idempotency keys. Overstating Kafka EOS is a common and easily-caught error.