Event-Driven Architecture Patterns

At a Glance

Aspect	Details
Goal	Build resilient, scalable event-driven architectures
Event Sourcing	Store state as a sequence of events, not current state
CQRS	Separate read and write models for optimization
Saga Pattern	Coordinate distributed transactions via events
Outbox Pattern	Guarantee DB + event consistency without 2PC
CDC	Capture database changes as events (Debezium)
Prerequisites	Parts 1-17 (core Kafka and Streams concepts)

What You'll Learn

Event Sourcing: storing history and rebuilding state
CQRS: optimizing reads and writes separately
Saga patterns for distributed transactions
The Outbox pattern for reliable event publishing
Change Data Capture with Debezium
Choosing the right pattern for your use case
Spring implementation for each pattern

Production Story: The Audit Trail That Saved Millions

Monday, 9:00 AM. Compliance audit at a financial services company.

"We need a complete history of every balance change for the past 5 years," the auditor demanded. "Not just the current balance—every single transaction and correction."

The traditional approach stored only current state:

SQL(2 lines)
Code
Loading syntax highlighter...

After a migration to Event Sourcing:

JAVA(8 lines)
Code
Loading syntax highlighter...

The audit was passed with flying colors. Every balance change was traceable, with full context, back to account creation. When a discrepancy was found, they could replay events to find exactly when and why it happened.

The insight: When history matters more than just current state, Event Sourcing is invaluable. Kafka's append-only log makes it a natural event store.

Mental Model: Event-Driven Architecture Overview

┌──────────────────────────────────────────────────────────────────────────┐
│                  EVENT-DRIVEN ARCHITECTURE PATTERNS                      │
└──────────────────────────────────────────────────────────────────────────┘

┌─────────────────┐      ┌─────────────────┐      ┌─────────────────┐
│ EVENT SOURCING  │      │      CQRS       │      │      SAGA       │
│                 │      │                 │      │                 │
│ Store events,   │      │ Separate read   │      │ Coordinate      │
│ not state       │      │ and write       │      │ distributed     │
│                 │      │ models          │      │ transactions    │
│ "What happened" │      │ "Optimize each" │      │ "Eventually     │
│                 │      │                 │      │  consistent"    │
└────────┬────────┘      └────────┬────────┘      └────────┬────────┘
         │                        │                        │
         └────────────────────────┴────────────────────────┘
                                  │
                          ┌───────┴───────┐
                          │     KAFKA     │
                          │ (Event Backbone)│
                          └───────┬───────┘
                                  │
         ┌────────────────────────┴────────────────────────┐
         │                        │                        │
┌────────┴────────┐      ┌────────┴────────┐      ┌────────┴────────┐
│  OUTBOX PATTERN │      │       CDC       │      │  EVENT CARRIED  │
│                 │      │                 │      │  STATE TRANSFER │
│ DB + Event      │      │ Capture DB      │      │                 │
│ atomically      │      │ changes         │      │ Full data in    │
│                 │      │                 │      │ events          │
└─────────────────┘      └─────────────────┘      └─────────────────┘

Deep Dive: Event Sourcing

Traditional vs Event Sourcing

┌──────────────────────────────────────────────────────────────────────────┐
│                TRADITIONAL STATE VS EVENT SOURCING                       │
└──────────────────────────────────────────────────────────────────────────┘

  TRADITIONAL (CRUD):                EVENT SOURCING:

  Account A123                       Account A123 Event Stream
  ┌─────────────────┐               ┌─────────────────────────────────────┐
  │ balance: $500   │               │ 1. AccountCreated { initial: $0 }   │
  │ status: ACTIVE  │               │ 2. MoneyDeposited { amount: $1000 } │
  └─────────────────┘               │ 3. MoneyWithdrawn { amount: $200 }  │
                                    │ 4. MoneyWithdrawn { amount: $300 }  │
  History? 🤷                       │ 5. AccountStatus { status: ACTIVE } │
  Why $500? Unknown                 └─────────────────────────────────────┘

                                    History? ✅ Complete
                                    Why $500? $1000 - $200 - $300

  To change state:                  To change state:
  UPDATE accounts SET               APPEND event to stream
    balance = 300                   ┌───────────────────────────────────┐
  WHERE id = 'A123';                │ 6. MoneyDeposited { amount: $200 }│
                                    └───────────────────────────────────┘
  Previous state lost               Previous state preserved

Event Sourcing with Kafka

JAVA(72 lines)
Code
Loading syntax highlighter...

Event Store Implementation

JAVA(77 lines)
Code
Loading syntax highlighter...

Snapshotting for Performance

JAVA(52 lines)
Code
Loading syntax highlighter...

Deep Dive: CQRS (Command Query Responsibility Segregation)

CQRS Architecture

┌──────────────────────────────────────────────────────────────────────────┐
│                         CQRS ARCHITECTURE                                │
└──────────────────────────────────────────────────────────────────────────┘

           ┌─────────────────────────────────────────────────────┐
           │                    CLIENTS                          │
           └───────────────────────┬─────────────────────────────┘
                                   │
              ┌────────────────────┴────────────────────┐
              │                                         │
              ▼                                         ▼
    ┌──────────────────┐                    ┌──────────────────┐
    │  COMMAND SIDE    │                    │   QUERY SIDE     │
    │  (Write Model)   │                    │  (Read Model)    │
    │                  │                    │                  │
    │ • CreateOrder    │                    │ • GetOrderById   │
    │ • UpdateOrder    │                    │ • SearchOrders   │
    │ • CancelOrder    │                    │ • GetOrderStats  │
    └────────┬─────────┘                    └────────▲─────────┘
             │                                       │
             │ Validates                             │ Optimized
             │ Business Rules                        │ Queries
             │                                       │
             ▼                                       │
    ┌──────────────────┐        Events      ┌───────┴──────────┐
    │  Event Store     │───────────────────▶│  Read Database   │
    │  (Kafka)         │                    │  (PostgreSQL/    │
    │                  │     Projector      │   Elasticsearch/ │
    │  Source of truth │                    │   MongoDB)       │
    └──────────────────┘                    └──────────────────┘

CQRS Implementation

JAVA(163 lines)
Code
Loading syntax highlighter...

Deep Dive: Saga Pattern

Saga for Distributed Transactions

┌──────────────────────────────────────────────────────────────────────────┐
│                    SAGA PATTERN - CHOREOGRAPHY                           │
└──────────────────────────────────────────────────────────────────────────┘

  Order Service          Inventory Service         Payment Service
       │                       │                        │
       │  OrderCreated         │                        │
       │──────────────────────▶│                        │
       │                       │                        │
       │                       │ InventoryReserved      │
       │                       │───────────────────────▶│
       │                       │                        │
       │                       │                        │ PaymentProcessed
       │◀──────────────────────┼────────────────────────│
       │                       │                        │
       │  OrderConfirmed       │                        │
       │──────────────────────▶│───────────────────────▶│
       │                       │                        │

  COMPENSATION (if payment fails):

       │                       │                        │
       │                       │                        │ PaymentFailed
       │◀──────────────────────┼────────────────────────│
       │                       │                        │
       │  OrderCancelled       │ ReleaseInventory       │
       │──────────────────────▶│───────────────────────▶│

Choreography-Based Saga

JAVA(209 lines)
Code
Loading syntax highlighter...

Orchestration-Based Saga

JAVA(122 lines)
Code
Loading syntax highlighter...

Deep Dive: Outbox Pattern

The Problem It Solves

┌──────────────────────────────────────────────────────────────────────────┐
│                    THE DUAL-WRITE PROBLEM                                │
└──────────────────────────────────────────────────────────────────────────┘

  DANGEROUS: Two separate operations that might not both succeed

  @Transactional
  public void createOrder(Order order) {
      orderRepository.save(order);        // 1. Save to DB    ✓
      kafkaTemplate.send("orders", ...);  // 2. Send to Kafka ?
  }

  What if Kafka is down?
  → Order saved, but event never sent
  → Other services don't know about the order
  → Data inconsistency!

  What if crash between 1 and 2?
  → Same problem


  THE OUTBOX PATTERN SOLUTION:

  @Transactional
  public void createOrder(Order order) {
      orderRepository.save(order);        // 1. Save order
      outboxRepository.save(new OutboxEvent(order));  // 2. Save event
  }

  // Same transaction! Both succeed or both fail.

  // Separate process reads outbox and publishes to Kafka
  // Even if Kafka is down, events are safe in database

Outbox Implementation

JAVA(133 lines)
Code
Loading syntax highlighter...

Deep Dive: Change Data Capture (CDC)

CDC with Debezium

┌──────────────────────────────────────────────────────────────────────────┐
│                    CHANGE DATA CAPTURE (CDC)                             │
└──────────────────────────────────────────────────────────────────────────┘

  Traditional approach: Application publishes events
  ─────────────────────────────────────────────────
  Application → Database
       │
       └──────→ Kafka (must remember to publish!)


  CDC approach: Database changes automatically become events
  ──────────────────────────────────────────────────────────
  Application → Database
                   │
                   └──────→ Debezium ──────→ Kafka

  Benefits:
  • No application changes needed
  • Captures ALL changes (even direct SQL)
  • No dual-write problem
  • Works with legacy systems

Debezium Configuration

YAML(18 lines)
Code
Loading syntax highlighter...

JSON(21 lines)
Code
Loading syntax highlighter...

Consuming CDC Events

JAVA(52 lines)
Code
Loading syntax highlighter...

CDC vs Outbox Pattern

┌──────────────────────────────────────────────────────────────────────────┐
│                    CDC VS OUTBOX COMPARISON                              │
├─────────────────────────────────┬────────────────────────────────────────┤
│           Outbox                │             CDC                        │
├─────────────────────────────────┼────────────────────────────────────────┤
│ Requires application changes    │ No application changes                 │
├─────────────────────────────────┼────────────────────────────────────────┤
│ Custom event format             │ Database row format                    │
├─────────────────────────────────┼────────────────────────────────────────┤
│ Domain events (semantic)        │ Technical events (row changes)         │
├─────────────────────────────────┼────────────────────────────────────────┤
│ Full control over event content │ Captures all changes automatically     │
├─────────────────────────────────┼────────────────────────────────────────┤
│ Works with any database         │ Requires supported database            │
├─────────────────────────────────┼────────────────────────────────────────┤
│ Use when:                       │ Use when:                              │
│ • Building new systems          │ • Legacy systems                       │
│ • Need rich domain events       │ • Can't modify application             │
│ • Want event versioning control │ • Need ALL changes captured            │
└─────────────────────────────────┴────────────────────────────────────────┘

Common Mistakes

1. Not Handling Saga Timeouts

JAVA(17 lines)
Code
Loading syntax highlighter...

2. Event Sourcing Without Snapshots

JAVA(14 lines)
Code
Loading syntax highlighter...

3. Outbox Without Idempotent Consumers

JAVA(20 lines)
Code
Loading syntax highlighter...

4. CQRS Without Eventual Consistency Awareness

JAVA(23 lines)
Code
Loading syntax highlighter...

Debug This: The Lost Event

Orders are being created but downstream services never receive the events:

JAVA(10 lines)
Code
Loading syntax highlighter...

No errors in logs, orders exist in database. What's wrong?

Answer

The issue: The Kafka send is happening inside a transaction. If kafkaTemplate is not configured for transactions, the message might be sent before the transaction commits.

Scenario:

1. Transaction starts
2. Order saved to DB (not committed yet)
3. Kafka message sent
4. Consumer receives message
5. Consumer queries order - NOT FOUND (transaction not committed)
6. Consumer fails/discards message
7. Transaction commits
8. Order now in DB, but event already lost

Solutions:

Use Outbox Pattern (recommended):

JAVA(7 lines)
Code
Loading syntax highlighter...

Transactional Kafka Template:

JAVA(5 lines)
Code
Loading syntax highlighter...

Send after transaction commit:

JAVA(11 lines)
Code
Loading syntax highlighter...

The root cause is the dual-write problem - you cannot atomically write to two systems (DB + Kafka) without coordination.

Exercises

Exercise 1: Build an Event-Sourced Account

Create an account service with:

Event Sourcing for balance changes
Snapshots every 100 events
Account history query API

Exercise 2: Implement Order Saga

Build a saga for order processing:

Order Service → Inventory Service → Payment Service
Proper compensation on failure
Saga status tracking

Exercise 3: CQRS Order System

Create separate read/write models:

Commands: CreateOrder, AddItem, ConfirmOrder
Projector to Elasticsearch
Rich query API (search, filter, aggregate)

Exercise 4: Outbox Implementation

Implement the outbox pattern:

Outbox table and entity
Publisher with retry logic
Metrics for publish latency

Exercise 5: CDC Pipeline

Set up Debezium CDC:

Capture orders table changes
Transform CDC events to domain events
Sync to separate read database

Interview Questions

Q1: "When would you choose Event Sourcing over traditional CRUD?"

What they're looking for: Understanding trade-offs

Strong answer: "Event Sourcing is valuable when:

Choose Event Sourcing when:

Complete audit trail is required (finance, healthcare, legal)
Need to replay/rebuild state from any point in time
Business logic benefits from knowing 'what happened' not just 'current state'
Multiple read models from same events (CQRS)
Debugging requires understanding state evolution

Avoid Event Sourcing when:

Simple CRUD without audit requirements
Schema changes are frequent (event versioning is complex)
Team is unfamiliar with the pattern
Queries primarily need current state, not history

The trade-offs:

Pros:
+ Complete audit trail
+ Time travel / replay
+ Natural fit for CQRS
+ Events are great for integration

Cons:
- Complexity (snapshots, projections, versioning)
- Eventually consistent reads
- Event schema evolution
- More infrastructure

I typically use it for critical domains (payments, orders) while using traditional CRUD for simpler services."

Q2: "Explain the Saga pattern. When would you use choreography vs orchestration?"

What they're looking for: Distributed transaction understanding

Strong answer: "Saga is a pattern for managing distributed transactions without 2PC, using a sequence of local transactions with compensating actions.

Choreography - services react to events:

Order → publishes OrderCreated
Inventory → hears event, publishes InventoryReserved
Payment → hears event, publishes PaymentCompleted

Orchestration - central coordinator:

Orchestrator → sends ReserveInventory command
Orchestrator ← receives InventoryReserved
Orchestrator → sends ProcessPayment command

When to choose:

Choreography is better for:

Simple sagas (2-3 steps)
Loosely coupled services
Each service owns its domain decisions
Want to avoid single point of failure

Orchestration is better for:

Complex sagas (many steps, branching)
Need clear visibility of saga state
Complex compensation logic
Centralized monitoring/debugging

In practice, I prefer orchestration for business-critical flows like orders because it's easier to understand the complete flow and debug failures."

Q3: "What is the Outbox pattern and why is it necessary?"

What they're looking for: Dual-write problem understanding

Strong answer: "The Outbox pattern solves the dual-write problem - atomically updating a database AND publishing an event.

The problem:

JAVA(6 lines)
Code
Loading syntax highlighter...

The solution:

JAVA(15 lines)
Code
Loading syntax highlighter...

Why it works:

DB transaction is atomic - both save or neither
Event guaranteed to be in outbox if order exists
Publisher can retry failures
At-least-once delivery (consumers must be idempotent)

Alternatives:

CDC (Debezium) captures outbox table changes
Transaction log tailing
Transactional Kafka (ties Kafka to DB transaction)

I prefer Outbox for new systems because it gives full control over event format, with CDC for legacy systems that can't be modified."

Q4: "How do you handle event schema evolution in Event Sourcing?"

What they're looking for: Production experience

Strong answer: "Event schema evolution is one of the hardest parts of Event Sourcing. Strategies:

1. Upcasting (transform on read):

JAVA(10 lines)
Code
Loading syntax highlighter...

2. Event versioning (store version):

JAVA(7 lines)
Code
Loading syntax highlighter...

3. Weak schema (JSON with optional fields):

JAVA(2 lines)
Code
Loading syntax highlighter...

Best practices:

Never delete fields, only add
Use optional fields with defaults
Add new event types rather than changing old
Test upcasters thoroughly
Keep old events readable forever
Document all schema changes

What I avoid:

Changing field types
Renaming fields (use aliases)
Removing fields
Complex transformations

The key is treating events like a published API - once in production, breaking changes are extremely costly."

Q5: "What's the relationship between CQRS and Event Sourcing? Can you use one without the other?"

What they're looking for: Architectural clarity

Strong answer: "They're often used together but are independent patterns:

CQRS alone (without Event Sourcing):

Write Model → Database → Read Model(s)
             (current state)

Commands update a traditional database.
Projectors sync to optimized read stores.

Use case: When you need different read/write models but don't need history.

Event Sourcing alone (without CQRS):

Commands → Event Store → Rebuild on read

Single model, events are the source of truth.
Read current state by replaying events.

Use case: When you need audit trail but reads are simple.

Together:

Commands → Event Store → Projectors → Read Models

Events are source of truth.
Multiple optimized read models.

Use case: Complex domain with audit requirements and diverse read patterns.

In practice: Event Sourcing almost always needs some form of CQRS because replaying all events for every read is impractical. At minimum, you snapshot or project current state.

But CQRS is common without Event Sourcing - any system that separates read replicas or uses Elasticsearch for search is doing CQRS at some level."

Summary & Key Takeaways

Event-Driven Patterns Summary

┌──────────────────────────────────────────────────────────────────────────┐
│                EVENT-DRIVEN PATTERNS SUMMARY                             │
├────────────────────────────────┬─────────────────────────────────────────┤
│ Pattern                        │ When to Use                             │
├────────────────────────────────┼─────────────────────────────────────────┤
│ Event Sourcing                 │ Audit trail, time travel, complex domain│
│ CQRS                           │ Different read/write optimization needs │
│ Saga (Choreography)            │ Simple distributed transactions         │
│ Saga (Orchestration)           │ Complex workflows, centralized control  │
│ Outbox                         │ Guaranteed DB + event consistency       │
│ CDC                            │ Legacy systems, capture all changes     │
└────────────────────────────────┴─────────────────────────────────────────┘

Essential Takeaways

Event Sourcing is about history - Store what happened, not just current state.
CQRS separates concerns - Optimize reads and writes independently.
Sagas replace 2PC - Eventually consistent distributed transactions.
Outbox solves dual-write - Atomic DB + event publishing.
CDC captures everything - No application changes needed.
Idempotency is essential - All consumers must handle duplicates.

Quick Reference

┌──────────────────────────────────────────────────────────────────────────┐
│                    PATTERN SELECTION GUIDE                               │
├────────────────────────────────┬─────────────────────────────────────────┤
│ Requirement                    │ Pattern                                 │
├────────────────────────────────┼─────────────────────────────────────────┤
│ Need audit trail               │ Event Sourcing                          │
│ Need time travel               │ Event Sourcing                          │
│ Read/write optimization        │ CQRS                                    │
│ Distributed transaction        │ Saga                                    │
│ Simple saga (2-3 steps)        │ Choreography                            │
│ Complex saga (many steps)      │ Orchestration                           │
│ DB + event consistency         │ Outbox pattern                          │
│ Legacy system integration      │ CDC (Debezium)                          │
│ Don't modify application       │ CDC (Debezium)                          │
│ Custom event format            │ Outbox pattern                          │
└────────────────────────────────┴─────────────────────────────────────────┘

Previous: Part 17: Windowing, Joins & Time Semantics

Next: Part 19: Error Handling & Dead Letter Queues

Series Overview: Kafka Compendium Series