Devops

Architecture & Storage Engine

June 10, 2026 at 07:19 AM
19 min reading
4 views

📋 At a Glance

AspectDetails
Difficulty🟡 Intermediate
PrerequisitesBasic Kafka concepts (topics, partitions)
Key ConceptsLog, segments, zero-copy, page cache, message format
Time Investment32 minutes read + 45 minutes practice
PayoffUnderstand why Kafka handles millions of messages/sec

🎯 What You'll Learn

After this article, you'll be able to:

  1. Explain the log abstraction and why it's perfect for messaging
  2. Understand segment files and how Kafka stores data on disk
  3. Describe zero-copy transfer and why it eliminates overhead
  4. Leverage the page cache for performance tuning
  5. Parse message format including headers, keys, and timestamps

🔥 Production Story: The Page Cache Mystery

The Setup: An e-commerce company ran a 5-broker Kafka cluster handling 500K messages/second. Performance was excellent—until they deployed a new monitoring system.
The Symptoms:
Before monitoring: Producer latency p99 = 5ms
After monitoring:  Producer latency p99 = 150ms (30x worse!)

Throughput dropped from 500K to 50K msg/sec. No code changes to Kafka. Same hardware.

The Investigation:
BASH(8 lines)
Code
Loading syntax highlighter...

Read I/O jumped 50x! But Kafka consumers hadn't changed.

The Root Cause: The new monitoring system ran heavy queries against the OS metrics, consuming memory. This evicted Kafka's data from the page cache.

Normally, Kafka serves reads directly from page cache (RAM). When data was evicted, Kafka had to read from disk—turning a memory operation into a disk operation.

The Fix:
BASH(3 lines)
Code
Loading syntax highlighter...
Lesson Learned: Kafka's performance depends heavily on the page cache. Anything that competes for memory affects Kafka. This is why Kafka servers should be dedicated.

🧠 Mental Model: Kafka's Storage Architecture

┌──────────────────────────────────────────────────────────────────┐
│                    KAFKA STORAGE ARCHITECTURE                    │
├──────────────────────────────────────────────────────────────────┤
│                                                                  │
│   PRODUCER                                                       │
│      │                                                           │
│      ▼                                                           │
│   ┌──────────────────────────────────────────────────────────┐   │
│   │                    KAFKA BROKER                          │   │
│   │                                                          │   │
│   │  ┌────────────────────────────────────────────────────┐  │   │
│   │  │              PAGE CACHE (RAM)                      │  │   │
│   │  │  ┌─────────┐ ┌─────────┐ ┌─────────┐               │  │   │
│   │  │  │ Recent  │ │ Recent  │ │ Recent  │               │  │   │
│   │  │  │ Segment │ │ Segment │ │ Segment │               │  │   │
│   │  │  │   P0    │ │   P1    │ │   P2    │               │  │   │
│   │  │  └─────────┘ └─────────┘ └─────────┘               │  │   │
│   │  └────────────────────────────────────────────────────┘  │   │
│   │                         │                                │   │
│   │                   Zero-Copy                              │   │
│   │                    sendfile()                            │   │
│   │                         │                                │   │
│   │  ┌────────────────────────────────────────────────────┐  │   │
│   │  │                    DISK                            │  │   │
│   │  │                                                    │  │   │
│   │  │  Topic: orders                                     │  │   │
│   │  │  ┌─────────────────────────────────────────────┐   │  │   │
│   │  │  │ Partition 0                                 │   │  │   │
│   │  │  │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │   │  │   │
│   │  │  │ │Segment │ │Segment │ │Segment │ │Active  │ │   │  │   │
│   │  │  │ │  .log  │ │  .log  │ │  .log  │ │Segment │ │   │  │   │
│   │  │  │ │0-1000  │ │1001-2000│2001-3000│ │ 3001+  │ │   │  │   │
│   │  │  │ └────────┘ └────────┘ └────────┘ └────────┘ │   │  │   │
│   │  │  └─────────────────────────────────────────────┘   │  │   │
│   │  └────────────────────────────────────────────────────┘  │   │
│   └──────────────────────────────────────────────────────────┘   │
│                         │                                        │
│                    Zero-Copy                                     │
│                         │                                        │
│                         ▼                                        │
│                      CONSUMER                                    │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘

Key Insight: Data flows from Producer → Page Cache → Disk
            Consumer reads: Page Cache (fast) or Disk (slow)
            Recent data is almost always in page cache!

🔬 Deep Dive

1. The Log Abstraction

At its core, Kafka is a distributed commit log. What does that mean?
A log is an append-only, ordered sequence of records:
┌─────────────────────────────────────────────────────────────────┐
│                         KAFKA LOG                               │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   Offset:  0      1      2      3      4      5      6     ...  │
│          ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐       │
│          │ M0 │ │ M1 │ │ M2 │ │ M3 │ │ M4 │ │ M5 │ │ M6 │ →     │
│          └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘       │
│                                                                 │
│   Properties:                                                   │
│   • Append-only (writes go to the end)                          │
│   • Ordered (offset determines order)                           │
│   • Immutable (existing records never change)                   │
│   • Persistent (survives restarts)                              │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
Why is a log perfect for messaging?
  1. Sequential writes: Appending to the end is O(1), regardless of data size
  2. Sequential reads: Consumers read in order, which is disk-friendly
  3. Simplicity: No complex data structures, just files
  4. Durability: Once written, data stays until explicitly deleted
Traditional message brokers vs Kafka:
AspectTraditional (e.g., RabbitMQ)Kafka
Data structureQueue (FIFO, delete on read)Log (append-only)
Message deliveryPush to consumerConsumer pulls
Message retentionUntil acknowledgedTime or size based
Random accessNoYes (by offset)
Replay capabilityNoYes

2. Segments: How Data Lives on Disk

A partition isn't stored as one giant file. It's split into segments:
┌─────────────────────────────────────────────────────────────────┐
│                    PARTITION SEGMENTS                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   /var/kafka/data/orders-0/                                     │
│   │                                                             │
│   ├── 00000000000000000000.log      (offsets 0-999)             │
│   ├── 00000000000000000000.index    (offset → position)         │
│   ├── 00000000000000000000.timeindex (timestamp → offset)       │
│   │                                                             │
│   ├── 00000000000000001000.log      (offsets 1000-1999)         │
│   ├── 00000000000000001000.index                                │
│   ├── 00000000000000001000.timeindex                            │
│   │                                                             │
│   ├── 00000000000000002000.log      (offsets 2000-2999)         │
│   ├── 00000000000000002000.index                                │
│   ├── 00000000000000002000.timeindex                            │
│   │                                                             │
│   └── 00000000000000003000.log      ← ACTIVE SEGMENT            │
│       00000000000000003000.index      (writes go here)          │
│       00000000000000003000.timeindex                            │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
Why segments?
  1. Efficient deletion: Delete whole segment files, not individual messages
  2. Efficient compaction: Process segment by segment
  3. Parallel I/O: Different segments can be read simultaneously
  4. Memory mapping: Smaller files are easier to mmap
Segment configuration:
PROPERTIES(7 lines)
Code
Loading syntax highlighter...
The index files:
┌─────────────────────────────────────────────────────────────────┐
│                    INDEX FILE STRUCTURE                         │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   .index file (offset → file position)                          │
│   ┌──────────────────────────────────────────────────────────┐  │
│   │  Offset  │  Position in .log file                        │  │
│   ├──────────┼───────────────────────────────────────────────┤  │
│   │  0       │  0                                            │  │
│   │  50      │  4096                                         │  │
│   │  100     │  8192                                         │  │
│   │  150     │  12288                                        │  │
│   └──────────┴───────────────────────────────────────────────┘  │
│                                                                 │
│   Index is sparse! Not every offset is indexed.                 │
│   To find offset 75:                                            │
│   1. Binary search index → find entry ≤ 75 (offset 50)          │
│   2. Seek to position 4096 in .log                              │
│   3. Scan forward to find offset 75                             │
│                                                                 │
│   .timeindex file (timestamp → offset)                          │
│   ┌──────────────────────────────────────────────────────────┐  │
│   │  Timestamp         │  Offset                             │  │
│   ├────────────────────┼─────────────────────────────────────┤  │
│   │  1705312800000     │  0                                  │  │
│   │  1705312860000     │  1000                               │  │
│   │  1705312920000     │  2000                               │  │
│   └────────────────────┴─────────────────────────────────────┘  │
│                                                                 │
│   Used for: offsetsForTimes() API                               │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

3. Zero-Copy Transfer

This is one of Kafka's biggest performance secrets.

Traditional data transfer (4 copies!):
┌─────────────────────────────────────────────────────────────────┐
│                 TRADITIONAL DATA TRANSFER                       │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   DISK                                                          │
│     │                                                           │
│     │ 1. read() - DMA copy to kernel buffer                     │
│     ▼                                                           │
│   KERNEL BUFFER (Page Cache)                                    │
│     │                                                           │
│     │ 2. CPU copy to application buffer                         │
│     ▼                                                           │
│   APPLICATION BUFFER (JVM Heap)                                 │
│     │                                                           │
│     │ 3. CPU copy to socket buffer                              │
│     ▼                                                           │
│   SOCKET BUFFER (Kernel)                                        │
│     │                                                           │
│     │ 4. DMA copy to NIC                                        │
│     ▼                                                           │
│   NETWORK                                                       │
│                                                                 │
│   Total: 4 copies, 2 kernel-user context switches               │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
Kafka's zero-copy transfer (sendfile):
┌─────────────────────────────────────────────────────────────────┐
│                    ZERO-COPY TRANSFER                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   DISK                                                          │
│     │                                                           │
│     │ 1. DMA copy to kernel buffer                              │
│     ▼                                                           │
│   KERNEL BUFFER (Page Cache)                                    │
│     │                                                           │
│     │ 2. sendfile() - DMA scatter/gather to NIC                 │
│     ▼                                                           │
│   NETWORK                                                       │
│                                                                 │
│   Total: 2 copies (both DMA, no CPU involved)                   │
│          0 kernel-user context switches                         │
│          Data never enters JVM heap!                            │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
The Java code that enables this:
JAVA(6 lines)
Code
Loading syntax highlighter...
Why this matters:
MetricTraditionalZero-Copy
CPU usageHighMinimal
Memory copies42
Context switches20
ThroughputLimited by CPULimited by NIC

4. The Page Cache: Kafka's Secret Weapon

The page cache is OS-managed memory that caches disk data:

┌──────────────────────────────────────────────────────────────────┐
│                      PAGE CACHE OPERATION                        │
├──────────────────────────────────────────────────────────────────┤
│                                                                  │
│   WRITE PATH (Producer → Broker)                                 │
│                                                                  │
│   Producer sends message                                         │
│         │                                                        │
│         ▼                                                        │
│   Broker writes to page cache  ← NOT directly to disk!           │ 
│         │                                                        │ 
│         ├── Acknowledgment sent to producer                      │
│         │   (if acks=1 or acks=all with ISR)                     │
│         │                                                        │
│         ▼                                                        │
│   OS flushes to disk asynchronously (later)                      │
│                                                                  │
│   ─────────────────────────────────────────────────────────────  │
│                                                                  │
│   READ PATH (Broker → Consumer)                                  │
│                                                                  │
│   Case 1: Data in page cache (FAST)                              │
│   ┌─────────────────────────────────────────────────────────┐    │
│   │  Consumer requests offset 1000                          │    │
│   │         │                                               │    │
│   │         ▼                                               │    │
│   │  Check page cache → HIT! (microseconds)                 │    │
│   │         │                                               │    │
│   │         ▼                                               │    │
│   │  Zero-copy to socket                                    │    │
│   └─────────────────────────────────────────────────────────┘    │
│                                                                  │
│   Case 2: Data not in page cache (SLOW)                          │
│   ┌─────────────────────────────────────────────────────────┐    │
│   │  Consumer requests offset 1000                          │    │
│   │         │                                               │    │
│   │         ▼                                               │    │
│   │  Check page cache → MISS                                │    │
│   │         │                                               │    │
│   │         ▼                                               │    │
│   │  Read from disk (milliseconds)                          │    │
│   │         │                                               │    │
│   │         ▼                                               │    │
│   │  Load into page cache                                   │    │
│   │         │                                               │    │
│   │         ▼                                               │    │
│   │  Send to consumer                                       │    │
│   └─────────────────────────────────────────────────────────┘    │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘
Why Kafka loves the page cache:
  1. Hot data stays in RAM: Recent messages (most commonly read) stay cached
  2. No JVM heap pressure: Page cache is outside JVM, no GC impact
  3. Survives restarts: JVM restart doesn't lose cached data (OS manages it)
  4. Automatic management: OS handles eviction, no tuning needed
Monitoring page cache:
BASH(14 lines)
Code
Loading syntax highlighter...
Page cache best practices:
BASH(14 lines)
Code
Loading syntax highlighter...

5. Message Format (Record Batch)

Messages in Kafka are stored in record batches:
┌─────────────────────────────────────────────────────────────────┐
│                      RECORD BATCH FORMAT                        │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   Record Batch (v2, Kafka 0.11+)                                │
│   ┌─────────────────────────────────────────────────────────┐   │
│   │ BATCH HEADER (61 bytes)                                 │   │
│   │  ┌─────────────────────────────────────────────────────┐│   │
│   │  │ baseOffset         (8 bytes) - First offset         ││   │
│   │  │ batchLength        (4 bytes) - Size of batch        ││   │
│   │  │ partitionLeaderEpoch (4 bytes) - Leader version     ││   │
│   │  │ magic              (1 byte)  - Format version (2)   ││   │
│   │  │ crc                (4 bytes) - Checksum             ││   │
│   │  │ attributes         (2 bytes) - Compression, etc     ││   │
│   │  │ lastOffsetDelta    (4 bytes) - Offset range         ││   │
│   │  │ firstTimestamp     (8 bytes) - Batch start time     ││   │
│   │  │ maxTimestamp       (8 bytes) - Batch end time       ││   │
│   │  │ producerId         (8 bytes) - For idempotence      ││   │
│   │  │ producerEpoch      (2 bytes) - Producer version     ││   │
│   │  │ baseSequence       (4 bytes) - For ordering         ││   │
│   │  │ recordCount        (4 bytes) - Number of records    ││   │
│   │  └─────────────────────────────────────────────────────┘│   │
│   │                                                         │   │
│   │  RECORDS (variable length, compressed together)         │   │
│   │  ┌─────────────────────────────────────────────────────┐│   │
│   │  │  Record 0                                           ││   │
│   │  │  ├── length (varint)                                ││   │
│   │  │  ├── attributes (1 byte)                            ││   │
│   │  │  ├── timestampDelta (varint)                        ││   │
│   │  │  ├── offsetDelta (varint)                           ││   │
│   │  │  ├── keyLength (varint)                             ││   │
│   │  │  ├── key (bytes)                                    ││   │
│   │  │  ├── valueLength (varint)                           ││   │
│   │  │  ├── value (bytes)                                  ││   │
│   │  │  ├── headersCount (varint)                          ││   │
│   │  │  └── headers[] (key-value pairs)                    ││   │
│   │  ├────────────────────────────────────────────────────-┤│   │
│   │  │  Record 1                                           ││   │
│   │  │  ...                                                ││   │
│   │  └─────────────────────────────────────────────────────┘│   │
│   └─────────────────────────────────────────────────────────┘   │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
Key insights about the format:
  1. Batching: Multiple records in one batch = fewer I/O operations
  2. Compression: Applied to whole batch, not individual records
  3. Varints: Variable-length integers save space
  4. Headers: Key-value metadata (tracing, routing, etc.)
  5. Idempotence fields: producerId, producerEpoch, baseSequence
Spring Kafka: Working with headers:
JAVA(38 lines)
Code
Loading syntax highlighter...

6. Log Compaction

Besides time-based retention, Kafka supports log compaction for special use cases:
┌─────────────────────────────────────────────────────────────────┐
│                      LOG COMPACTION                             │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   BEFORE COMPACTION (all records kept)                          │
│                                                                 │
│   Offset: 0    1    2    3    4    5    6    7    8    9        │
│   Key:    A    B    A    C    B    A    C    A    B    C        │
│   Value:  v1   v1   v2   v1   v2   v3   v2   v4   v3   v3       │
│                                                                 │
│   ───────────────────────────────────────────────────────────── │
│                                                                 │
│   AFTER COMPACTION (only latest per key)                        │
│                                                                 │
│   Offset: 7    8    9                                           │
│   Key:    A    B    C                                           │
│   Value:  v4   v3   v3                                          │
│                                                                 │
│   • Keeps latest value for each key                             │
│   • Offsets preserved (not renumbered)                          │
│   • Tombstones (null value) delete keys                         │
│                                                                 │
│   Use cases:                                                    │
│   • Changelog topics (Kafka Streams state)                      │
│   • Database CDC (keep current state)                           │
│   • Configuration/reference data                                │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
Compaction configuration:
PROPERTIES(11 lines)
Code
Loading syntax highlighter...
Spring Kafka topic configuration:
JAVA(13 lines)
Code
Loading syntax highlighter...

⚠️ Common Mistakes

Mistake 1: Over-allocating JVM Heap

BASH(11 lines)
Code
Loading syntax highlighter...

Mistake 2: Ignoring Disk I/O Metrics

BASH(8 lines)
Code
Loading syntax highlighter...

Mistake 3: Wrong Segment Size

PROPERTIES(12 lines)
Code
Loading syntax highlighter...

Mistake 4: Running Other Apps on Kafka Servers

BASH(5 lines)
Code
Loading syntax highlighter...

Mistake 5: Misunderstanding Flush Behavior

PROPERTIES(8 lines)
Code
Loading syntax highlighter...

🐛 Debug This

Your Kafka cluster shows these symptoms:

Consumer lag: Increasing slowly
Producer latency: p50=2ms, p99=500ms (should be <10ms)
Disk I/O: Read throughput spiking periodically
Memory: 32GB total, 24GB page cache, 6GB JVM heap

Consumers are keeping up (lag not exploding), but producer p99 latency is terrible. What's happening?

Click to reveal analysis
Analysis:
  1. Consumer lag increasing slowly: Consumers slightly behind but not badly
  2. p50=2ms, p99=500ms: Median is fine, but tail latency is awful
  3. Disk reads spiking: Something is evicting page cache periodically
  4. Memory looks OK: 24GB cache should be plenty
The clue: Disk read spikes are periodic, not constant.
Most likely cause: Consumer falling behind and requesting old data

Here's what's happening:

  1. Normal consumers read from page cache (fast)
  2. One slow consumer falls behind
  3. Slow consumer requests old data not in cache
  4. Old data must be read from disk
  5. Disk reads evict recent data from cache
  6. Now even current producers' writes miss cache
  7. Producer p99 spikes during these disk reads
Investigation:
BASH(7 lines)
Code
Loading syntax highlighter...
Fix:
  1. Find and fix the slow consumer
  2. Or use quotas to limit its fetch rate
  3. Or increase partition count to distribute load
Key insight: One slow consumer can affect all producers by thrashing the page cache.

💻 Exercises

Exercise 1: Examine Segment Files

BASH(7 lines)
Code
Loading syntax highlighter...

Exercise 2: Monitor Page Cache

BASH(7 lines)
Code
Loading syntax highlighter...

Exercise 3: Measure Zero-Copy Impact

Compare sending a large file via traditional read/write vs sendfile:

JAVA(8 lines)
Code
Loading syntax highlighter...

Exercise 4: Explore Record Format

BASH(8 lines)
Code
Loading syntax highlighter...

Exercise 5: Configure Compaction

Create a compacted topic and observe behavior:

JAVA(12 lines)
Code
Loading syntax highlighter...

🎤 Interview Questions

Q1: Why is Kafka faster than traditional message brokers like RabbitMQ?

Answer: Kafka's performance comes from several architectural choices:
  1. Append-only log: All writes are sequential appends, O(1) regardless of data size. No random I/O for writes.
  2. Zero-copy transfer: Data goes directly from page cache to network socket via sendfile(). Never enters JVM heap, no serialization/deserialization by broker.
  3. Page cache reliance: Kafka leverages OS page cache instead of in-process caching. Recent data stays in RAM automatically.
  4. Batching everywhere: Records batched at producer, transferred in batches, written in batches. Amortizes overhead.
  5. Sequential reads: Consumers read in order, which is optimal for disk and cache.
  6. No per-message index: Traditional brokers track each message for deletion. Kafka just appends and deletes whole segments.

The combination means Kafka can handle millions of messages/second with modest hardware.

Q2: Explain the role of page cache in Kafka's architecture.

Answer: Page cache is OS-managed memory that caches disk data. Kafka relies on it heavily:
Write path:
  • Producer sends message → Broker writes to page cache → Acknowledgment sent
  • Actual disk write happens asynchronously by OS
  • This is safe because Kafka has replication for durability
Read path:
  • Consumer requests data → Check page cache
  • If hit: Zero-copy from cache to socket (microseconds)
  • If miss: Read from disk into cache, then send (milliseconds)
Why it's brilliant:
  1. No duplicate caching: JVM heap cache would duplicate page cache
  2. Survives restarts: Page cache persists across JVM restarts
  3. Automatic management: OS handles eviction based on access patterns
  4. Works with zero-copy: sendfile() transfers directly from cache
Key implication: Kafka performance depends on page cache availability. Competition for memory (large heap, other processes) hurts Kafka.

Q3: What are segment files and why does Kafka use them?

Answer: Segments are the physical files that store partition data:
partition-0/
├── 00000000000000000000.log    (messages offset 0-1000)
├── 00000000000000001001.log    (messages offset 1001-2000)
└── 00000000000000002001.log    (active segment)
Why segments?
  1. Efficient deletion: When retention expires, delete entire segment file. No need to rewrite data.
  2. Efficient compaction: Process one segment at a time. Completed segments are immutable.
  3. Parallel operations: Can read old segments while writing to active segment.
  4. Memory mapping: Smaller files are easier to mmap and manage in page cache.
  5. Recovery: On restart, only need to validate active segment. Old segments are known-good.
Segment boundaries:
  • Roll to new segment when size exceeds log.segment.bytes (default 1GB)
  • Or time exceeds log.segment.ms (default 7 days)

Q4: What is log compaction and when would you use it?

Answer: Log compaction retains only the latest value for each key:
Before: key=A:v1, key=B:v1, key=A:v2, key=B:v2, key=A:v3
After:  key=A:v3, key=B:v2
Use cases:
  1. Changelog topics: Kafka Streams stores state as compacted topics. On restart, replays latest state per key.
  2. CDC (Change Data Capture): Database changes streamed to Kafka. Keep current row state, not full history.
  3. Configuration data: Store current config per key. Consumers get latest values.
  4. User profiles: Keep latest profile state without unbounded growth.
Key properties:
  • Offsets preserved (not renumbered)
  • Tombstones (null value) delete keys
  • Only applied to closed segments (active segment untouched)
  • cleanup.policy=compact enables it
Not suitable for: Event logs where you need full history (orders, transactions).

Q5: How does zero-copy transfer work and why is it important for Kafka?

Answer: Zero-copy uses sendfile() syscall to transfer data without CPU copying:
Traditional transfer (4 copies):
  1. Disk → Kernel buffer (DMA)
  2. Kernel buffer → User buffer (CPU copy)
  3. User buffer → Socket buffer (CPU copy)
  4. Socket buffer → NIC (DMA)
Zero-copy (2 copies):
  1. Disk → Kernel buffer (DMA)
  2. Kernel buffer → NIC (DMA via scatter/gather)
Why it matters for Kafka:
  1. No CPU overhead: Data never touches CPU, freeing it for other work
  2. No JVM involvement: Data bypasses JVM heap, no GC pressure
  3. No serialization: Broker doesn't parse messages, just moves bytes
  4. Higher throughput: Limited by NIC speed, not CPU
Implementation: Java's FileChannel.transferTo() maps to native sendfile().
Requirement: Only works when data is already in correct format on disk—which is why Kafka stores messages in the same format they're sent over the network.

📝 Summary & Key Takeaways

Core Concepts

  1. Log abstraction: Append-only, ordered, immutable sequence
  2. Segments: Partition split into files for efficient management
  3. Zero-copy: sendfile() bypasses CPU for data transfer
  4. Page cache: OS-managed RAM cache for disk data
  5. Record batch: Multiple messages compressed together

Performance Principles

PrincipleImplementation
Sequential I/OAppend-only writes, ordered reads
BatchingRecord batches reduce I/O calls
No parsingBroker moves bytes, doesn't understand content
OS leveragePage cache instead of JVM heap
Zero-copysendfile() for network transfer

Key Configurations

PROPERTIES(10 lines)
Code
Loading syntax highlighter...

📋 Quick Reference

BASH(15 lines)
Code
Loading syntax highlighter...

📅 Review Schedule

  • Day 1: Read and understand log abstraction
  • Day 3: Explore segment files on disk
  • Day 7: Monitor page cache during production
  • Day 14: Review zero-copy and performance implications
  • Day 30: Explain architecture without notes

📚 Series Navigation

Tags:

Distributed SystemsKafkaEvent-drivenStreamingSpring-kafka
Last updated: June 10, 2026 at 11:12 AM