[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6},"actionData","errors","routes/blog-tag",{"_7":8,"_9":10},"tag",{"_31":88,"_72":89,"_35":90},"articles",{"_11":12,"_13":14,"_15":16,"_17":16,"_18":19,"_20":21,"_22":23,"_24":23,"_25":16,"_26":27},"content",[30],"pageable",{"_28":21,"_29":19},"totalElements",1,"totalPages","size",10,"number",0,"first",true,"last","numberOfElements","empty",false,"page_number","page_size",{"_31":32,"_33":34,"_35":36,"_37":38,"_11":39,"_40":-5,"_41":23,"_42":27,"_43":44,"_45":46,"_47":48,"_49":50,"_51":52,"_53":52,"_54":55,"_56":57,"_58":59,"_60":52,"_61":55,"_62":63,"_64":65,"_66":67,"_68":69,"_70":71},"id",59,"title","Container Resource Management","slug","docker-part-9-resource-management","excerpt","Memory limits, CPU allocation, OOM killer, JVM tuning","# Docker Compendium Part 9: Container Resource Management\n\nYour Java app needs 512MB but uses 2GB. Your Node.js service gets CPU throttled mysteriously. Containers die with no logs. This article explains how Docker limits resources, how to size containers correctly, and how to avoid the OOM killer.\n\n## 📋 At a Glance\n\n| Aspect | Details |\n|--------|---------|\n| Topic | Memory limits, CPU allocation, OOM killer, JVM/Node.js tuning |\n| Complexity | Advanced |\n| Prerequisites | Part 1 (Container Internals - cgroups) |\n| Key Insight | Container limits are enforced by kernel cgroups, not the application |\n| Time to Master | 3-4 hours |\n\n---\n\n## 🎯 What You'll Learn\n\n- **Memory management** - hard limits, soft limits, swap, OOM behavior\n- **CPU management** - shares, quota, pinning, throttling\n- **Language-specific tuning** - JVM, Node.js, Python in containers\n- **Right-sizing** - how to determine appropriate limits\n- **Monitoring** - detecting resource problems before they cause outages\n\n---\n\n## 🔥 Production Story: The OOM Serial Killer\n\nA team deployed microservices to Kubernetes. Pods kept restarting randomly - some after hours, some after minutes. No logs, no errors, just disappeared.\n\n**Investigation:**\n\n```bash\n$ kubectl describe pod api-server\n...\nLast State: Terminated\n Reason: OOMKilled\n Exit Code: 137\n\n$ kubectl get events\nWarning OOMKilling 4m kernel-monitor Memory cgroup out of memory: Kill process 1234\n```\n\n**The setup:**\n\n```yaml\nresources:\n limits:\n memory: \"512Mi\"\n requests:\n memory: \"256Mi\"\n```\n\n```dockerfile\nFROM eclipse-temurin:21\nCOPY app.jar /app.jar\nCMD [\"java\", \"-jar\", \"/app.jar\"]\n```\n\n**Root cause:** JVM default heap is 25% of **physical memory** (host's 32GB), not container limit. JVM tried to allocate 8GB heap inside 512MB container.\n\n**The fix:**\n\n```dockerfile\nFROM eclipse-temurin:21\nCOPY app.jar /app.jar\n# Use container-aware memory settings\nCMD [\"java\", \\\n \"-XX:+UseContainerSupport\", \\\n \"-XX:MaxRAMPercentage=75.0\", \\\n \"-jar\", \"/app.jar\"]\n```\n\nNow JVM allocates 75% of 512MB = 384MB heap, leaving room for metaspace, threads, and OS overhead.\n\n**Lesson:** Applications don't automatically respect container limits. You must configure them to be container-aware.\n\n---\n\n## 🧠 Mental Model: Resource Limits Stack\n\n```\n┌─────────────────────────────────────────────────────────────────────────┐\n│ CONTAINER RESOURCE LIMITS │\n│ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ APPLICATION LAYER │\n│ │ │\n│ │ JVM: -Xmx, -XX:MaxRAMPercentage │\n│ │ Node: --max-old-space-size │\n│ │ Python: Various pool sizes │\n│ │ │\n│ │ ⚠️ App may try to use MORE than container allows! │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ CONTAINER LAYER │\n│ │ │\n│ │ docker run --memory=512m --cpus=1.5 │\n│ │ │\n│ │ These translate to cgroup limits in kernel │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ CGROUP LAYER │\n│ │ │\n│ │ memory.max = 536870912 (512MB) │\n│ │ cpu.max = 150000 100000 (1.5 CPUs) │\n│ │ │\n│ │ Kernel ENFORCES these limits │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ KERNEL ACTIONS │\n│ │ │\n│ │ Memory exceeded → OOM Killer → SIGKILL (exit 137) │\n│ │ CPU exceeded → Throttling (slower, not killed) │\n│ │ │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │\n└─────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 🔬 Deep Dive\n\n### Memory Limits\n\n**Setting memory limits:**\n\n```bash\n# Hard limit - container killed if exceeded\ndocker run --memory=512m nginx\n\n# Soft limit (reservation) - for orchestrator scheduling\ndocker run --memory=512m --memory-reservation=256m nginx\n\n# Memory + swap limit\ndocker run --memory=512m --memory-swap=1g nginx\n# Container can use 512MB RAM + 512MB swap\n\n# Memory only (disable swap)\ndocker run --memory=512m --memory-swap=512m nginx\n# Or equivalently:\ndocker run --memory=512m --memory-swappiness=0 nginx\n```\n\n**What counts toward memory limit:**\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│ CONTAINER MEMORY │\n│ │\n│ ┌────────────────────────────────────────────────────────────┐ │\n│ │ Application Heap │ │\n│ │ (Java: -Xmx, Node: --max-old-space-size) │ │\n│ ├────────────────────────────────────────────────────────────┤ │\n│ │ Application Off-Heap │ │\n│ │ (JVM metaspace, native memory, thread stacks) │ │\n│ ├────────────────────────────────────────────────────────────┤ │\n│ │ Kernel Buffers & Cache │ │\n│ │ (File system cache, network buffers) │ │\n│ ├────────────────────────────────────────────────────────────┤ │\n│ │ Shared Libraries │ │\n│ │ (libc, language runtime, etc.) │ │\n│ └────────────────────────────────────────────────────────────┘ │\n│ │\n│ Total counted against cgroup memory limit │\n│ │\n│ ⚠️ Kernel cache is usually reclaimable, │\n│ but still counts until reclaimed! │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n**OOM Killer behavior:**\n\n```bash\n# Container exceeds memory limit\n# 1. Kernel tries to reclaim memory (cache, etc.)\n# 2. If still over limit: OOM killer triggers\n# 3. SIGKILL sent to container's main process\n# 4. Exit code: 137 (128 + SIGKILL signal 9)\n\n# Detect OOM\n$ docker inspect container --format '{{.State.OOMKilled}}'\ntrue\n\n# Check kernel logs\n$ dmesg | grep -i \"killed process\"\nMemory cgroup out of memory: Killed process 1234 (java) total-vm:...\n```\n\n### CPU Limits\n\nDocker provides multiple ways to limit CPU:\n\n**1. CPU quota (--cpus) - Hard limit:**\n\n```bash\n# Limit to 1.5 CPUs (150% of one core)\ndocker run --cpus=1.5 myapp\n\n# This translates to:\n# cpu.max = 150000 100000\n# (150000 microseconds per 100000 microsecond period)\n```\n\n**2. CPU shares (--cpu-shares) - Relative weight:**\n\n```bash\n# Default share is 1024\n# This container gets half the CPU of default\ndocker run --cpu-shares=512 myapp\n\n# Only matters under contention!\n# If no contention, container can use all available CPU\n```\n\n**3. CPU pinning (--cpuset-cpus) - Dedicated cores:**\n\n```bash\n# Only use CPU cores 0 and 1\ndocker run --cpuset-cpus=0,1 myapp\n\n# Useful for:\n# - NUMA-aware applications\n# - Latency-sensitive workloads\n# - Preventing core hopping\n```\n\n**When to use which:**\n\n| Scenario | Use | Why |\n|----------|-----|-----|\n| Predictable workload | `--cpus` | Guaranteed limit |\n| Batch jobs | `--cpu-shares` | Fair sharing |\n| Latency-sensitive | `--cpuset-cpus` | Cache locality |\n| Resource guarantee | `--cpus` + `--cpu-shares` | Both limit and priority |\n\n**CPU throttling:**\n\n```bash\n# Unlike memory, CPU doesn't kill - it throttles\n# Container is paused until next period\n\n# Detect throttling\n$ cat /sys/fs/cgroup/docker//cpu.stat\nusage_usec 1234567890\nuser_usec 1000000000\nsystem_usec 234567890\nnr_periods 10000\nnr_throttled 500 # <-- Throttling count\nthrottled_usec 50000000 # <-- Total throttle time\n```\n\n### JVM Container Tuning\n\nModern JVMs (8u191+, 11+) are container-aware but need configuration:\n\n```dockerfile\nFROM eclipse-temurin:21\n\n# Enable container support (default in modern JVMs)\nENV JAVA_OPTS=\"-XX:+UseContainerSupport\"\n\n# Set heap as percentage of container limit\nENV JAVA_OPTS=\"$JAVA_OPTS -XX:MaxRAMPercentage=75.0\"\nENV JAVA_OPTS=\"$JAVA_OPTS -XX:InitialRAMPercentage=50.0\"\n\n# Or set exact limits\n# ENV JAVA_OPTS=\"$JAVA_OPTS -Xms256m -Xmx384m\"\n\nCMD [\"sh\", \"-c\", \"java $JAVA_OPTS -jar /app.jar\"]\n```\n\n**Memory breakdown for JVM in container:**\n\n```\nContainer: 512MB\n├── Heap: 75% = 384MB (-XX:MaxRAMPercentage=75.0)\n├── Metaspace: ~64MB (class metadata)\n├── Thread stacks: ~1MB per thread\n├── Native memory: varies\n├── JIT code cache: ~48MB\n└── Other: GC, buffers\n\n⚠️ If heap = container limit, you WILL OOM\n Always leave 20-30% for non-heap\n```\n\n**Checking JVM settings:**\n\n```bash\ndocker run --rm -m 512m eclipse-temurin:21 java \\\n -XX:+UseContainerSupport \\\n -XX:MaxRAMPercentage=75.0 \\\n -XshowSettings:vm -version\n\n# Output shows:\n# Max. Heap Size (Estimated): 384.00M\n```\n\n### Node.js Container Tuning\n\nNode.js has its own memory management:\n\n```dockerfile\nFROM node:20-slim\n\n# Set heap size (default is ~1.4GB on 64-bit)\nENV NODE_OPTIONS=\"--max-old-space-size=384\"\n\n# For container-based auto-sizing (Node 19+)\n# ENV NODE_OPTIONS=\"--max-old-space-size=$(($MEMORY_LIMIT * 75 / 100))\"\n\nCMD [\"node\", \"index.js\"]\n```\n\n**Calculating Node.js memory:**\n\n```bash\n# Container limit: 512MB\n# Node heap: 75% = 384MB\n# System/buffers: 25% = 128MB\n\ndocker run -m 512m -e NODE_OPTIONS=\"--max-old-space-size=384\" mynode\n```\n\n**UV_THREADPOOL_SIZE for I/O-bound apps:**\n\n```dockerfile\n# Default is 4 threads for fs/dns operations\n# Increase for high I/O workloads\nENV UV_THREADPOOL_SIZE=16\n```\n\n### Python Container Tuning\n\nPython's memory management is more automatic, but:\n\n```dockerfile\nFROM python:3.11-slim\n\n# Disable bytecode writing\nENV PYTHONDONTWRITEBYTECODE=1\n\n# Unbuffered output (for logging)\nENV PYTHONUNBUFFERED=1\n\n# Memory allocator (can reduce memory usage)\nENV PYTHONMALLOC=malloc\n\nCMD [\"python\", \"app.py\"]\n```\n\n**Memory-intensive Python (NumPy, Pandas):**\n\n```bash\n# Limit OpenBLAS threads\nENV OPENBLAS_NUM_THREADS=1\nENV OMP_NUM_THREADS=1\n\n# For scikit-learn\nENV JOBLIB_MULTIPROCESSING=0\n```\n\n### Monitoring Container Resources\n\n**Real-time stats:**\n\n```bash\n# Docker stats\ndocker stats\n# CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O\n# app 45.2% 384MiB / 512MiB 75% 1.5MB / 800KB\n\n# Specific container\ndocker stats app --no-stream --format \"{{.MemUsage}} {{.MemPerc}}\"\n```\n\n**Inspect cgroup limits:**\n\n```bash\n# Memory limit\ncat /sys/fs/cgroup/docker//memory.max\n\n# Current memory usage\ncat /sys/fs/cgroup/docker//memory.current\n\n# CPU limit\ncat /sys/fs/cgroup/docker//cpu.max\n\n# CPU throttling stats\ncat /sys/fs/cgroup/docker//cpu.stat\n```\n\n**Prometheus metrics:**\n\n```yaml\n# docker-compose with metrics\nservices:\n app:\n image: myapp\n deploy:\n resources:\n limits:\n memory: 512M\n cpus: '1.5'\n\n cadvisor:\n image: gcr.io/cadvisor/cadvisor\n volumes:\n - /:/rootfs:ro\n - /var/run:/var/run:ro\n - /sys:/sys:ro\n - /var/lib/docker/:/var/lib/docker:ro\n ports:\n - \"8080:8080\"\n```\n\n### Right-Sizing Containers\n\n**Process for determining limits:**\n\n```\n1. Profile application under load\n └─ Measure peak memory, average CPU\n\n2. Add safety margin\n └─ Memory: +20-30% for GC, buffers\n └─ CPU: Consider burst capacity\n\n3. Test under various conditions\n └─ Cold start\n └─ Peak load\n └─ Memory leak scenarios\n\n4. Monitor in production\n └─ Actual usage vs limits\n └─ Throttling frequency\n └─ OOM events\n\n5. Iterate\n └─ Adjust based on real data\n```\n\n**Common sizing guidelines:**\n\n| App Type | Memory Suggestion | CPU Suggestion |\n|----------|-------------------|----------------|\n| JVM microservice | 512MB - 1GB | 0.5 - 2 |\n| Node.js API | 256MB - 512MB | 0.25 - 1 |\n| Python web | 256MB - 512MB | 0.25 - 1 |\n| ML inference | 2GB - 8GB | 1 - 4 |\n| Static content | 64MB - 128MB | 0.1 - 0.25 |\n\n---\n\n## ⚠️ Common Mistakes\n\n### Mistake 1: Setting Heap = Container Limit\n\n```bash\n# ❌ JVM tries 512MB heap in 512MB container\ndocker run -m 512m myapp -Xmx512m\n# Result: OOM - no room for non-heap memory!\n\n# ✅ Leave room for non-heap\ndocker run -m 512m myapp -Xmx384m\n# Or use percentage\ndocker run -m 512m myapp -XX:MaxRAMPercentage=75.0\n```\n\n### Mistake 2: Ignoring CPU Throttling\n\n```bash\n# App seems slow but no errors\n# Check for throttling:\n$ cat /sys/fs/cgroup/docker//cpu.stat\nnr_throttled 15000 # <-- Throttled 15000 times!\n\n# Solution: Increase CPU limit or optimize app\ndocker run --cpus=2.0 myapp # Was --cpus=0.5\n```\n\n### Mistake 3: Not Setting Any Limits\n\n```bash\n# ❌ No limits - can consume entire host\ndocker run myapp\n\n# ✅ Always set limits in production\ndocker run --memory=512m --cpus=1.0 myapp\n```\n\n---\n\n## 🐛 Debug This: The Mysterious Slow Container\n\nA developer reports: \"My container runs fine with 2 CPUs but is incredibly slow with 0.5 CPU. It's not even using all 0.5 CPU according to docker stats!\"\n\n```bash\n$ docker run --cpus=0.5 myapp\n# App very slow\n\n$ docker stats myapp\nCONTAINER CPU % MEM USAGE\nmyapp 35% 256MiB\n\n# CPU only 35% - not even hitting limit?\n```\n\n**Why is the app slow if it's not hitting CPU limit?**\n\n---\n\n**✅ Solution:**\n\nThe app is being **throttled**, not fully utilizing the CPU.\n\n**Understanding CPU throttling:**\n\n`--cpus=0.5` means the container can use 50ms of CPU time per 100ms period. If the app does burst computation:\n\n```\nPeriod 1 (0-100ms):\n├── 0-50ms: App runs at 100% (using its quota)\n├── 50-100ms: App THROTTLED (waiting for next period)\n└── Apparent usage: 50% of period, but app waited 50ms!\n\nPeriod 2 (100-200ms):\n├── 100-150ms: App runs again\n├── 150-200ms: Throttled again\n└── And so on...\n```\n\n**docker stats shows average** - 35% average doesn't show the throttling pattern.\n\n**Diagnosis:**\n\n```bash\n# Check throttling stats\ncat /sys/fs/cgroup/docker//cpu.stat\nnr_throttled 10000 # App was throttled 10000 times!\nthrottled_usec 500000000 # 500 seconds total throttle time\n```\n\n**Solutions:**\n\n1. **Increase CPU limit** if app needs it:\n ```bash\n docker run --cpus=1.0 myapp\n ```\n\n2. **Use CPU shares instead** if you want burst capability:\n ```bash\n docker run --cpu-shares=512 myapp\n # Can burst to more CPU if available\n ```\n\n3. **Optimize the app** - reduce CPU-intensive operations\n\n4. **Adjust period** (advanced):\n ```bash\n docker run --cpu-period=50000 --cpu-quota=25000 myapp\n # Same 0.5 CPU but smaller periods = less latency\n ```\n\n**Key insight:** CPU throttling causes latency, not just reduced throughput. A burst-y application can feel much slower than CPU % suggests.\n\n---\n\n## 💻 Exercises\n\n### Exercise 1: Observe OOM Killer\n⭐ Difficulty: Easy | ⏱️ Time: 15 minutes\n\n```bash\n# Create memory-hungry app\ndocker run -d --name oom-test -m 100m python:3.11 python -c \"\nimport time\ndata = []\nwhile True:\n data.append('x' * 10_000_000)\n print(f'Allocated {len(data) * 10}MB')\n time.sleep(0.5)\n\"\n\n# Watch it get killed\ndocker logs -f oom-test\n\n# Check OOM status\ndocker inspect oom-test --format '{{.State.OOMKilled}}'\n\n# Check exit code (should be 137)\ndocker inspect oom-test --format '{{.State.ExitCode}}'\n\n# Check kernel logs\ndmesg | tail -20 | grep -i oom\n```\n\n---\n\n### Exercise 2: Measure CPU Throttling\n⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes\n\n```bash\n# CPU-intensive task\ndocker run -d --name cpu-test --cpus=0.5 alpine sh -c \"\nwhile true; do\n # CPU-intensive operation\n dd if=/dev/urandom bs=1M count=10 | md5sum > /dev/null\n sleep 0.1\ndone\n\"\n\n# Monitor CPU usage\ndocker stats cpu-test\n\n# Check throttling\nCONTAINER_ID=$(docker inspect cpu-test --format '{{.Id}}')\ncat /sys/fs/cgroup/docker/${CONTAINER_ID}/cpu.stat\n\n# Compare with higher limit\ndocker stop cpu-test\ndocker run -d --name cpu-test2 --cpus=2.0 alpine sh -c \"\nwhile true; do\n dd if=/dev/urandom bs=1M count=10 | md5sum > /dev/null\n sleep 0.1\ndone\n\"\n\n# Check throttling for this one\nCONTAINER_ID2=$(docker inspect cpu-test2 --format '{{.Id}}')\ncat /sys/fs/cgroup/docker/${CONTAINER_ID2}/cpu.stat\n\n# Compare nr_throttled between them\n```\n\n---\n\n### Exercise 3: JVM Container Sizing\n⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes\n\n```bash\n# Test JVM memory detection\ndocker run --rm -m 512m eclipse-temurin:21 java \\\n -XX:+PrintFlagsFinal -version 2>&1 | grep -E \"MaxHeapSize|InitialHeapSize\"\n\n# With container support (should see ~128MB = 25% of 512MB)\ndocker run --rm -m 512m eclipse-temurin:21 java \\\n -XX:+UseContainerSupport \\\n -XX:+PrintFlagsFinal -version 2>&1 | grep MaxHeapSize\n\n# With 75% percentage\ndocker run --rm -m 512m eclipse-temurin:21 java \\\n -XX:+UseContainerSupport \\\n -XX:MaxRAMPercentage=75.0 \\\n -XshowSettings:vm -version\n\n# Verify with actual app\ncat > /tmp/MemTest.java << 'EOF'\npublic class MemTest {\n public static void main(String[] args) {\n Runtime rt = Runtime.getRuntime();\n System.out.println(\"Max Memory: \" + rt.maxMemory() / 1024 / 1024 + \"MB\");\n System.out.println(\"Total Memory: \" + rt.totalMemory() / 1024 / 1024 + \"MB\");\n System.out.println(\"Free Memory: \" + rt.freeMemory() / 1024 / 1024 + \"MB\");\n }\n}\nEOF\n\ndocker run --rm -m 512m -v /tmp:/code eclipse-temurin:21 sh -c \"\ncd /code && javac MemTest.java && java -XX:MaxRAMPercentage=75.0 MemTest\n\"\n```\n\n---\n\n### Exercise 4: Find the Right Limit\n⭐⭐⭐ Difficulty: Hard | ⏱️ Time: 30 minutes\n\n```bash\n# Profile an app to find appropriate limits\n\n# Create test app with variable load\ncat > /tmp/app.py << 'EOF'\nimport time\nimport sys\nimport random\n\ndata = []\ncpu_work = 0\n\nwhile True:\n # Simulate memory growth\n if random.random() > 0.7:\n data.append('x' * 1_000_000)\n if len(data) > 100:\n data.pop(0)\n\n # Simulate CPU work\n if random.random() > 0.5:\n cpu_work = sum(range(100000))\n\n time.sleep(0.1)\n print(f\"Memory chunks: {len(data)}, CPU result: {cpu_work}\", flush=True)\nEOF\n\n# Run without limits first\ndocker run -d --name profile-test -v /tmp/app.py:/app.py python:3.11-slim python /app.py\n\n# Monitor for 1 minute\ndocker stats profile-test --no-stream\nsleep 60\ndocker stats profile-test --no-stream\n\n# Record peak memory and average CPU\n# Then set limits with 20% buffer\n\n# Clean up\ndocker stop profile-test && docker rm profile-test\n\n# Test with limits\n# docker run -d --name limited-test -m --cpus= ...\n```\n\n---\n\n### Exercise 5: Complete Resource Configuration\n⭐⭐⭐⭐ Difficulty: Expert | ⏱️ Time: 30 minutes\n\nCreate a production-ready docker-compose with proper resource management:\n\n```yaml\n# Requirements:\n# 1. Java API service (512MB, 1 CPU)\n# 2. Node.js worker (256MB, 0.5 CPU)\n# 3. Python data processor (1GB, 2 CPU)\n# 4. Redis cache (128MB, 0.25 CPU)\n# 5. PostgreSQL (512MB, 1 CPU)\n\n# Each service should have:\n# - Memory limit\n# - Memory reservation (50% of limit)\n# - CPU limit\n# - Proper JVM/Node flags where applicable\n# - Health checks\n\n# Test your configuration:\n# - All services start successfully\n# - Services stay within limits under load\n# - No OOM kills during normal operation\n```\n\n---\n\n## 🎤 Senior-Level Interview Questions\n\n### Q1: Explain the difference between --memory and --memory-reservation.\n\n**Strong Answer:**\n\n\"These are hard vs soft limits:\n\n**--memory (hard limit):**\n- Enforced by kernel cgroups\n- If exceeded: OOM killer terminates the container\n- Container cannot use more than this\n\n**--memory-reservation (soft limit):**\n- Scheduling hint for orchestrators\n- Not enforced when memory is available\n- When host is under pressure, kernel tries to reclaim down to reservation\n- Container can exceed this if host has free memory\n\n**Example:**\n```bash\ndocker run --memory=512m --memory-reservation=256m myapp\n```\n\nThe container:\n- Guaranteed 256MB (orchestrator won't over-commit below this)\n- Can burst to 512MB when available\n- Is killed if it tries to exceed 512MB\n\n**Use case:** In Kubernetes, `requests` maps to reservation, `limits` maps to hard limit. Set requests to typical usage, limits to peak + buffer.\n\nI typically set reservation to 50-70% of limit for services with variable memory patterns.\"\n\n### Q2: A Java container keeps getting OOM killed despite setting -Xmx512m in a 512MB container. Why?\n\n**Strong Answer:**\n\n\"JVM uses more than just heap memory. The OOM is because total JVM memory exceeds container limit.\n\n**JVM memory components:**\n\n```\nTotal JVM Memory = Heap (-Xmx)\n + Metaspace (class metadata, ~64MB+)\n + Thread stacks (~1MB per thread)\n + Code cache (JIT compiled code, ~48MB)\n + Native memory (NIO buffers, JNI)\n + GC overhead\n```\n\n**With -Xmx512m:**\n- Heap: 512MB\n- Metaspace: 64MB\n- 50 threads: 50MB\n- Code cache: 48MB\n- Other: varies\n- **Total: ~700MB** in a 512MB container = OOM\n\n**Solutions:**\n\n1. **Reduce heap, leave room for overhead:**\n ```bash\n -Xmx384m # Leave 128MB for non-heap\n ```\n\n2. **Use container-aware settings:**\n ```bash\n -XX:+UseContainerSupport -XX:MaxRAMPercentage=75.0\n ```\n JVM calculates 75% of container limit automatically.\n\n3. **Limit other areas:**\n ```bash\n -XX:MaxMetaspaceSize=64m\n -XX:ReservedCodeCacheSize=32m\n -Xss256k # Thread stack size\n ```\n\nRule of thumb: Heap should be 65-75% of container limit maximum.\"\n\n### Q3: How do you detect and handle CPU throttling?\n\n**Strong Answer:**\n\n\"CPU throttling happens when a container exceeds its CPU quota within a scheduling period.\n\n**Detection:**\n\n```bash\n# Check throttling stats\ncat /sys/fs/cgroup/docker//cpu.stat\nnr_throttled 10000 # Times throttled\nthrottled_usec 500000000 # Total throttle time in microseconds\n```\n\n**Symptoms:**\n- High latency spikes\n- Lower than expected throughput\n- docker stats showing less CPU usage than limit\n\n**Monitoring:**\n- Prometheus: `container_cpu_cfs_throttled_periods_total`\n- Alert when throttling rate exceeds threshold\n\n**Solutions:**\n\n1. **Increase CPU limit** if legitimate need:\n ```bash\n --cpus=2.0 # Was 0.5\n ```\n\n2. **Adjust period for lower latency:**\n ```bash\n --cpu-period=50000 --cpu-quota=25000\n # Same 0.5 CPU but smaller periods = less waiting\n ```\n\n3. **Use CPU shares for burst capability:**\n ```bash\n --cpu-shares=512\n # Can burst when other containers are idle\n ```\n\n4. **Optimize application:**\n - Reduce CPU-intensive operations\n - Add caching\n - Async processing\n\nFor latency-sensitive services, I prefer slightly over-provisioning CPU to avoid any throttling. For batch jobs, throttling is acceptable.\"\n\n### Q4: How do you right-size containers for a new application?\n\n**Strong Answer:**\n\n\"I follow a data-driven process:\n\n**Phase 1: Profile without limits**\n```bash\ndocker run myapp # No limits\ndocker stats --no-stream # Peak values over time\n```\n\n**Phase 2: Analyze memory**\n- Peak memory usage\n- Memory growth pattern\n- GC behavior (for JVM)\n\n**Phase 3: Analyze CPU**\n- Average utilization\n- Peak spikes\n- Throttling sensitivity\n\n**Phase 4: Calculate limits**\n```\nMemory limit = Peak usage * 1.2 to 1.3 (20-30% buffer)\nMemory reservation = Average usage\n\nCPU limit = Peak requirement + 20% buffer\nCPU request = Average usage\n```\n\n**Phase 5: Test under various conditions**\n- Cold start\n- Sustained load\n- Spike load\n- Memory leak scenarios\n\n**Phase 6: Monitor and iterate**\n- Track actual usage vs limits\n- Adjust based on real production data\n- Set up alerts for approaching limits\n\n**Example output:**\n```yaml\nresources:\n limits:\n memory: 512Mi # Peak was 400Mi + 25%\n cpu: \\\"1.0\\\" # Peak was 0.8 + 25%\n requests:\n memory: 256Mi # Average was 250Mi\n cpu: \\\"0.5\\\" # Average was 0.4\n```\n\nKey principle: Don't guess, measure. Profile in staging, validate in production, iterate.\"\n\n### Q5: What's the difference between --cpus, --cpu-shares, and --cpuset-cpus?\n\n**Strong Answer:**\n\n\"These control different aspects of CPU allocation:\n\n**--cpus (Hard quota):**\n- Limits total CPU time\n- `--cpus=1.5` means container can use 150% of one core\n- Enforced regardless of host CPU availability\n- Container is throttled if it tries to use more\n- Use when: Predictable resource allocation, multi-tenant environments\n\n**--cpu-shares (Relative weight):**\n- Default is 1024\n- Only matters under contention\n- `--cpu-shares=512` gets half the CPU of a 1024 container\n- Can burst to full CPU if no contention\n- Use when: Batch jobs, where burst capability is valuable\n\n**--cpuset-cpus (Core pinning):**\n- Restricts to specific CPU cores\n- `--cpuset-cpus=0,1` only runs on cores 0 and 1\n- Good for: NUMA locality, cache optimization, isolating workloads\n- Use when: Latency-sensitive apps, NUMA-aware deployments\n\n**Combined example:**\n```bash\ndocker run \\\n --cpus=1.5 \\ # Max 1.5 CPU total\n --cpu-shares=2048 \\ # Higher priority than default\n --cpuset-cpus=0,1 \\ # Only use cores 0 and 1\n myapp\n```\n\nThis container:\n- Limited to 1.5 CPUs worth of time\n- Gets priority over default containers when competing\n- Only scheduled on cores 0 and 1\"\n\n---\n\n## 📝 Summary & Key Takeaways\n\n### Resource Limits Quick Reference\n\n| Resource | Flag | Enforcement |\n|----------|------|-------------|\n| Memory hard limit | `--memory` | OOM kill |\n| Memory soft limit | `--memory-reservation` | Reclaim under pressure |\n| CPU quota | `--cpus` | Throttling |\n| CPU weight | `--cpu-shares` | Proportional sharing |\n| CPU pinning | `--cpuset-cpus` | Core restriction |\n\n### Application Tuning Summary\n\n| Runtime | Memory Flag | Recommendation |\n|---------|-------------|----------------|\n| JVM | `-XX:MaxRAMPercentage=75.0` | Leave 25% for non-heap |\n| Node.js | `--max-old-space-size=N` | Set to ~75% of limit in MB |\n| Python | N/A | Generally automatic |\n\n### Golden Rules\n\n1. **Always set limits in production** - Prevent runaway containers\n2. **Leave headroom** - Heap ≠ container limit\n3. **Monitor throttling** - It causes latency, not just reduced throughput\n4. **Profile first** - Measure, don't guess\n5. **Make apps container-aware** - JVM, Node need explicit configuration\n\n---\n\n## 📋 Quick Reference\n\n### Common Commands\n\n```bash\n# Run with limits\ndocker run --memory=512m --cpus=1.0 myapp\n\n# Check OOM\ndocker inspect container --format '{{.State.OOMKilled}}'\n\n# Monitor resources\ndocker stats\n\n# Check cgroup limits\ncat /sys/fs/cgroup/docker//memory.max\ncat /sys/fs/cgroup/docker//cpu.max\n\n# Check throttling\ncat /sys/fs/cgroup/docker//cpu.stat\n```\n\n### JVM Flags\n\n```bash\n-XX:+UseContainerSupport # Respect container limits\n-XX:MaxRAMPercentage=75.0 # Heap as % of container\n-XX:InitialRAMPercentage=50.0 # Initial heap\n-XX:MaxMetaspaceSize=128m # Limit metaspace\n```\n\n### Exit Codes\n\n| Code | Meaning |\n|------|---------|\n| 137 | SIGKILL (OOM or docker kill) |\n| 143 | SIGTERM (graceful shutdown) |\n| 139 | SIGSEGV (segmentation fault) |\n\n---\n\n## 📅 Review Schedule\n\n| Day | Task | Time |\n|-----|------|------|\n| Day 1 | Review memory/CPU limit flags | 10 min |\n| Day 3 | Do Exercise 1 (observe OOM) | 15 min |\n| Day 7 | Configure JVM for container in real project | 20 min |\n| Day 14 | Profile and right-size a service | 30 min |\n| Day 30 | Audit all services for resource limits | 30 min |\n\n---\n\n## 📚 Series Navigation\n\n| Previous | Current | Next |\n|----------|---------|------|\n| [Part 8: Build Configuration](docker-part-8-build-configuration) | **Part 9: Resource Management** | [Part 10: Volumes & Storage](docker-part-10-volumes) |\n\n**Docker Compendium Series:**\n- [Part 0: How to Use This Series](docker-part-0-how-to-use-this-series)\n- [Part 1: Container Internals](docker-part-1-container-internals)\n- [Part 2: Image Anatomy](docker-part-2-image-anatomy)\n- [Part 3: Build Process Deep Dive](docker-part-3-build-process)\n- [Part 4: Networking Internals](docker-part-4-networking-internals)\n- [Part 5: Dockerfile Optimization Patterns](docker-part-5-dockerfile-optimization)\n- [Part 6: Multi-Stage Builds: Beyond Basics](docker-part-6-multi-stage-builds)\n- [Part 7: Base Image Selection & Security](docker-part-7-base-images)\n- [Part 8: ARG, ENV & Build-Time Configuration](docker-part-8-build-configuration)\n- **Part 9: Container Resource Management** ← You are here\n- [Part 10: Volume Patterns & Data Persistence](docker-part-10-volumes)\n","featuredImage","isPublished","isFeatured","subscriptionType","BASIC","publishedAt","2026-06-10T07:19:25.471218Z","createdAt","2026-06-10T07:19:25.471802Z","updatedAt","2026-06-16T21:26:56.086200Z","readingTimeMinutes",17,"estimatedReadingTime","authorId",3,"authorUsername","tomasz_main","categoryName","Devops","readingTime","seriesId","seriesOrder",9,"author",{"_31":55,"_98":57,"_99":100,"_101":102,"_103":-5,"_104":-5,"_105":-5,"_106":-5,"_107":-5},"category",{"_31":94,"_72":59,"_35":95,"_75":96,"_97":-5},"tags",[78,79,80,81],"series",{"_31":55,"_72":73,"_35":74,"_75":76,"_77":23},"name","Docker Compendium: From Practitioner to Expert","docker-compendium","description","Master Docker internals, optimization patterns, and production best practices. For developers who use Docker daily but want deeper understanding of what's happening under the hood. Not a beginner series - we assume you can write Dockerfiles and use docker-compose. Learn what namespaces and cgroups really do, why your builds are slow, how to debug containers that crash, and how to run Docker in production like a pro.\n","isActive",{"_31":91,"_72":92,"_35":93},{"_31":88,"_72":89,"_35":90},{"_31":85,"_72":86,"_35":87},{"_31":82,"_72":83,"_35":84},233,"Memory","memory",213,"Docker","docker",234,"Cpu","cpu",220,"Cgroups","cgroups",7,"devops","Category for Devops","colorHex","username","firstName","Tomasz","lastName","Jarosz","profilePicture","bio","website","githubUrl","linkedinUrl"]