Interview Prep Zoneby CuriouserLabs

Question 3 of 5 · senior level

You must update your database and publish an event. Why can't you just do both, and what's the correct pattern?

🎤 Say this first

Because two writes to two systems cannot be made atomic by ordering them. Commit the database first and crash before publishing → the state changed and nobody knows. Publish first and the transaction rolls back → you announced something that never happened, and consumers have already acted on it. Wrapping both in a @Transactional method fixes nothing: the broker is not enrolled in the database transaction. This is the dual-write problem, and the standard fix is the transactional outbox — insert the event into an outbox table in the same local transaction as the state change, so either both land or neither does, then have a separate relay (a poller, or CDC via Debezium) read that table and publish. The relay is at-least-once, so consumers must be idempotent; you traded an atomicity problem you can't solve for a duplicate problem you can.

The full picture

One local transaction, two rows — the whole trick
@Transactional
public void placeOrder(OrderRequest req) {
    Order order = orderRepo.save(Order.pending(req));

    // Same transaction, same database, so this is atomic with the line above.
    // No broker involved yet — that is the entire point.
    outboxRepo.save(new OutboxEvent(
        UUID.randomUUID(),          // event id -> the consumer's dedupe key
        "order",                    // aggregate type
        order.getId().toString(),   // aggregate id -> the Kafka partition key,
                                    //   which is what preserves per-order ordering
        "OrderPlaced",
        toJson(order)));
}

// A SEPARATE process publishes and marks rows sent (or Debezium tails the WAL).
// It may publish the same row twice after a crash — that is at-least-once, and
// it is why the consumer dedupes on the event id.

// The bug this replaces, which looks completely reasonable and is not atomic:
//   orderRepo.save(order);          // committed
//   kafka.send("orders", event);    // ...process dies here. Silent divergence.
ApproachAtomic?Cost
DB commit, then publishno — crash loses the eventsilent divergence, hardest bug class to find
Publish, then DB commitno — rollback invents an eventworse: consumers act on a lie
XA across DB + brokeryesblocking, and Kafka has no XA support at all
Outbox + polling relayyes (event is durable with the state)polling lag, an extra table to prune
Outbox + CDC (Debezium)yesno polling lag; a connector to operate
Listen-to-yourself / event-sourcedyes (the event is the state)biggest model change; strongest guarantee
  • Preserve ordering with the aggregate id as the partition key: the outbox is a table, so it has no inherent order across aggregates — and you don't want one. What you need is that two events for the same order arrive in order, which comes free if the aggregate id is the partition key. Volunteering this shows you've thought past 'it publishes eventually'.
  • Prune the outbox from day one. It's a hot, append-heavy table that becomes your largest and most bloated one within months if nobody deletes sent rows. Delete-after-publish or a partitioned table with a drop-old-partition job — decided up front, not after the disk alert.
  • CDC vs polling is a real trade, not a fashion: polling is trivial to operate and adds latency equal to the poll interval plus jitter; Debezium tails the WAL so latency is milliseconds and there's no query load, but it's a Kafka Connect deployment with its own offsets, snapshots and failure modes. Small team, seconds of latency acceptable → poll. High volume or tight latency → CDC.
  • The mirror image is the inbox pattern: the consumer records the processed event id in the same transaction as its own side effect, so a redelivery is dropped atomically. Outbox makes publishing exactly-once-effective on the write side; inbox does it on the read side. Naming both shows you see it as one end-to-end story rather than one trick.
  • Not every event needs this. An audit log or a metric can be fire-and-forget; the outbox costs a table, a relay and a pruning job. Apply it where a lost or invented event changes what a user sees or what money does — and say that explicitly, because pattern-application-everywhere is its own architectural smell.

🔄 Likely follow-up questions

  • Why doesn't @TransactionalEventListener(AFTER_COMMIT) solve the dual-write problem?
  • How do you keep per-aggregate event ordering when the outbox is just a table?
  • When is CDC worth operating over a simple polling relay?