[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6},"actionData","errors","routes/blog-tag",{"_7":8,"_9":10},"tag",{"_33":84,"_74":85,"_37":86},"articles",{"_11":12,"_13":14,"_15":16,"_17":18,"_19":20,"_21":22,"_23":24,"_25":24,"_26":16,"_27":28},"content",[31,32],"pageable",{"_29":22,"_30":20},"totalElements",2,"totalPages",1,"size",10,"number",0,"first",true,"last","numberOfElements","empty",false,"page_number","page_size",{"_33":109,"_35":110,"_37":111,"_39":112,"_11":113,"_42":-5,"_43":24,"_44":28,"_45":46,"_47":114,"_49":115,"_51":52,"_53":116,"_55":116,"_56":57,"_58":59,"_60":61,"_62":116,"_63":57,"_64":117,"_66":118,"_68":119,"_70":120,"_72":121},{"_33":34,"_35":36,"_37":38,"_39":40,"_11":41,"_42":-5,"_43":24,"_44":28,"_45":46,"_47":48,"_49":50,"_51":52,"_53":54,"_55":54,"_56":57,"_58":59,"_60":61,"_62":54,"_63":57,"_64":65,"_66":67,"_68":69,"_70":71,"_72":73},"id",57,"title","Base Image Selection & Security","slug","docker-part-7-base-images","excerpt","Choosing base images, Alpine pitfalls, vulnerability scanning","# Docker Compendium Part 7: Base Image Selection & Security\n\nYour base image choice affects everything - size, security, compatibility, and debugging ability. Alpine seems perfect until DNS resolution fails mysteriously. Ubuntu feels safe until your image hits 1GB. This article helps you choose wisely and understand the tradeoffs.\n\n## 📋 At a Glance\n\n| Aspect | Details |\n|--------|---------|\n| Topic | Base image selection, Alpine pitfalls, distroless, security scanning |\n| Complexity | Intermediate |\n| Prerequisites | Part 2 (Image Anatomy), Part 5 (Optimization) |\n| Key Insight | There's no universally best base image - understand the tradeoffs |\n| Time to Master | 2-3 hours |\n\n---\n\n## 🎯 What You'll Learn\n\n- **Base image landscape** - official images, slim variants, Alpine, distroless\n- **Alpine pitfalls** - musl vs glibc issues and when they bite\n- **Security considerations** - CVE scanning, minimal attack surface\n- **Language-specific guidance** - best bases for Node, Python, Java, Go\n- **When to use scratch** - truly minimal containers\n\n---\n\n## 🔥 Production Story: The Alpine DNS Mystery\n\nA microservices team migrated from `node:20` to `node:20-alpine` to reduce image sizes. Two weeks later, production incidents started appearing randomly.\n\n**The symptoms:**\n- Intermittent 5-second delays on HTTP requests\n- Some DNS lookups failing with SERVFAIL\n- Issues appeared under load, disappeared when traffic was low\n\n**Investigation:**\n\n```bash\n# Inside Alpine container\n$ time nslookup api.internal\n;; connection timed out; no servers could be reached\n\n# Same query, 5 seconds later\n$ time nslookup api.internal\nName: api.internal\nAddress: 10.0.0.15\nreal 0m0.012s\n```\n\n**Root cause:** Alpine uses `musl libc`, which has a different DNS resolver implementation than `glibc`. The musl resolver:\n1. Sends A and AAAA queries in parallel\n2. Some corporate DNS servers can't handle this\n3. Timeout on one query causes 5-second delay\n\n**The fix:**\n\n```dockerfile\n# Option 1: Configure musl DNS behavior\nFROM node:20-alpine\nRUN echo \"options single-request\" >> /etc/resolv.conf\n\n# Option 2: Switch to slim (glibc)\nFROM node:20-slim\n```\n\n**Lesson:** Alpine's size savings come with compatibility costs. Understand musl vs glibc before migrating.\n\n---\n\n## 🧠 Mental Model: The Base Image Spectrum\n\n```\n┌──────────────────────────────────────────────────────────────────────────┐\n│ BASE IMAGE DECISION SPECTRUM │\n│ │\n│ Size Full Slim Alpine Distroless Scratch│\n│ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ │\n│ │ │ │ │ │ │\n│ ~1GB │ ~200MB │ ~50MB │ ~20MB │ ~0MB │ │\n│ │ │ │ │ │ │\n│ ┌───────────┴──────────────┴──────────────┴──────────────┴─────────┴──┐ │\n│ │ │ │\n│ │ Full (ubuntu, debian) │ │\n│ │ ✓ All debugging tools ✗ Large size │ │\n│ │ ✓ Maximum compatibility ✗ More CVEs │ │\n│ │ ✓ Easy troubleshooting ✗ Slower pulls │ │\n│ │ │ │\n│ │ Slim (debian-slim, *-slim) │ │\n│ │ ✓ Good balance ~ Limited debugging │ │\n│ │ ✓ glibc compatible ✓ Reasonable size │ │\n│ │ ✓ Package manager ✓ Fewer CVEs │ │\n│ │ │ │\n│ │ Alpine │ │\n│ │ ✓ Very small ✗ musl compatibility issues │ │\n│ │ ✓ Security focused ✗ Different tooling (apk) │ │\n│ │ ✓ Minimal attack surface ✗ DNS quirks │ │\n│ │ │ │\n│ │ Distroless (gcr.io/distroless/*) │ │\n│ │ ✓ Minimal packages ✗ No shell for debugging │ │\n│ │ ✓ Fewest CVEs ✗ Harder troubleshooting │ │\n│ │ ✓ Small attack surface ✓ Just runtime needed │ │\n│ │ │ │\n│ │ Scratch (empty) │ │\n│ │ ✓ Truly minimal (0MB) ✗ Must be static binary │ │\n│ │ ✓ No CVEs possible ✗ No debugging capability │ │\n│ │ ✓ Perfect for Go/Rust ✗ Need CA certs for HTTPS │ │\n│ │ │ │\n│ └─────────────────────────────────────────────────────────────────────┘ │\n│ │\n│ Choose based on: Language, debugging needs, security requirements │\n└──────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 🔬 Deep Dive\n\n### Base Image Comparison\n\n```bash\n# Size comparison\n$ docker images --format \"table {{.Repository}}:{{.Tag}}\\t{{.Size}}\"\n\nnode:20 1.1GB\nnode:20-slim 250MB\nnode:20-alpine 180MB\n\npython:3.11 1.0GB\npython:3.11-slim 150MB\npython:3.11-alpine 60MB\n\ngolang:1.21 800MB\ngolang:1.21-alpine 250MB\n\nopenjdk:21 470MB\neclipse-temurin:21-jre 270MB\neclipse-temurin:21-jre-alpine 180MB\n\nubuntu:22.04 77MB\ndebian:bookworm-slim 74MB\nalpine:3.18 7MB\ngcr.io/distroless/static 2MB\nscratch 0MB\n```\n\n### Official Images vs Community\n\n**Use official images when available:**\n\n```dockerfile\n# ✅ Official - maintained by Docker and language teams\nFROM node:20\nFROM python:3.11\nFROM golang:1.21\n\n# ❌ Avoid random community images\nFROM someguy/node-optimized:latest\nFROM random-company/python:3.11\n```\n\n**Finding official images:**\n- Docker Hub: Look for \"Docker Official Image\" badge\n- Verified publishers: Companies like Microsoft, Google, Oracle\n\n### Alpine: The Trade-offs\n\n**Alpine uses musl libc instead of glibc:**\n\n| Aspect | glibc | musl |\n|--------|-------|------|\n| Size | Larger | Smaller |\n| Compatibility | Standard | Some edge cases |\n| DNS resolver | Sequential | Parallel (can cause issues) |\n| Performance | Optimized | Generally good |\n| Native modules | Pre-built available | Often need compilation |\n\n**Common Alpine issues:**\n\n**1. DNS resolution delays:**\n```bash\n# Fix in Dockerfile\nRUN echo \"options single-request-reopen\" >> /etc/resolv.conf\n\n# Or in docker-compose\ndns_opt:\n - single-request-reopen\n```\n\n**2. Python native packages:**\n```dockerfile\n# ❌ Many wheels won't install\nFROM python:3.11-alpine\nRUN pip install numpy # Fails - no pre-built wheel for musl\n\n# Fix: Install build deps\nFROM python:3.11-alpine\nRUN apk add --no-cache gcc musl-dev python3-dev\nRUN pip install numpy # Compiles from source (slow!)\n\n# Better: Use slim\nFROM python:3.11-slim\nRUN pip install numpy # Pre-built wheel works\n```\n\n**3. Node.js native modules:**\n```dockerfile\n# ❌ Native modules fail\nFROM node:20-alpine\nRUN npm install bcrypt # Error: Missing python, make, g++\n\n# Fix: Add build tools\nFROM node:20-alpine\nRUN apk add --no-cache python3 make g++\nRUN npm install bcrypt\n\n# Or use slim (has build tools)\nFROM node:20-slim\nRUN npm install bcrypt\n```\n\n**When Alpine works well:**\n- Pure JavaScript/TypeScript (no native deps)\n- Go applications (static binaries)\n- Rust applications (static binaries)\n- Simple shell scripts\n\n### Distroless: Maximum Security\n\nGoogle's distroless images contain only your app and runtime dependencies:\n\n```dockerfile\n# Java distroless\nFROM eclipse-temurin:21 AS builder\nCOPY . .\nRUN ./gradlew build\n\nFROM gcr.io/distroless/java21-debian12\nCOPY --from=builder /app/build/libs/*.jar /app.jar\nCMD [\"app.jar\"]\n\n# Python distroless\nFROM python:3.11 AS builder\nCOPY . .\nRUN pip install --target=/app -r requirements.txt\n\nFROM gcr.io/distroless/python3-debian12\nCOPY --from=builder /app /app\nENV PYTHONPATH=/app\nCMD [\"main.py\"]\n\n# Static binary (Go, Rust)\nFROM gcr.io/distroless/static-debian12\nCOPY --from=builder /app/binary /binary\nCMD [\"/binary\"]\n```\n\n**Distroless variants:**\n\n| Image | Contents |\n|-------|----------|\n| `static` | CA certs, /etc/passwd, tzdata |\n| `base` | glibc, libssl, openssl |\n| `cc` | libgcc, libstdc++ |\n| `java*` | JRE |\n| `python3` | Python interpreter |\n| `nodejs*` | Node.js runtime |\n\n**Debugging distroless:**\n\n```bash\n# No shell - can't exec into container!\ndocker exec -it container /bin/sh\n# Error: executable file not found\n\n# Debug variant has busybox shell\nFROM gcr.io/distroless/java21-debian12:debug\n# Now has /busybox/sh\n```\n\n### Scratch: Truly Empty\n\n`scratch` is an empty image - literally nothing:\n\n```dockerfile\n# Go static binary\nFROM golang:1.21 AS builder\nWORKDIR /app\nCOPY . .\nRUN CGO_ENABLED=0 GOOS=linux go build -o /server\n\nFROM scratch\n# Need CA certs for HTTPS\nCOPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/\nCOPY --from=builder /server /server\nENTRYPOINT [\"/server\"]\n```\n\n**Requirements for scratch:**\n1. Static binary (no dynamic linking)\n2. CA certificates if making HTTPS requests\n3. Timezone data if needed\n4. `/etc/passwd` if dropping to non-root user\n\n```dockerfile\nFROM golang:1.21 AS builder\nWORKDIR /app\nCOPY . .\nRUN CGO_ENABLED=0 go build -ldflags=\"-s -w\" -o /server\nRUN echo \"appuser:x:1000:1000::/:\" > /etc/passwd\n\nFROM scratch\nCOPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/\nCOPY --from=builder /etc/passwd /etc/passwd\nCOPY --from=builder /usr/share/zoneinfo /usr/share/zoneinfo\nCOPY --from=builder /server /server\nUSER appuser\nENTRYPOINT [\"/server\"]\n```\n\n### Security Scanning\n\n**Scan images for vulnerabilities:**\n\n```bash\n# Docker Scout (built-in)\ndocker scout cves myimage:tag\n\n# Trivy\ndocker run --rm aquasec/trivy image myimage:tag\n\n# Grype\ngrype myimage:tag\n\n# Snyk\nsnyk container test myimage:tag\n```\n\n**Interpret results:**\n\n```bash\n$ docker scout cves node:20\n\n Target │ node:20\n Digest │ sha256:abc123...\n───────────────┼─────────────────────\n Base image │ debian:bookworm-slim\n\n## Overview\n Critical │ 2\n High │ 15\n Medium │ 45\n Low │ 89\n\n## Critical vulnerabilities\nCVE-2024-1234 │ openssl │ 3.0.2 │ Fixed in 3.0.13\nCVE-2024-5678 │ libc6 │ 2.36 │ Fixed in 2.36-9\n```\n\n**Reducing CVE count:**\n\n| Strategy | Impact |\n|----------|--------|\n| Use slim/alpine | Fewer packages = fewer CVEs |\n| Update regularly | Patches fix CVEs |\n| Multi-stage builds | Don't include build tools |\n| Distroless | Minimal packages |\n| Pin versions | Control what's included |\n\n### Language-Specific Recommendations\n\n**Node.js:**\n\n```dockerfile\n# Production\nFROM node:20-slim # Good balance, glibc compatible\n\n# If no native deps\nFROM node:20-alpine # Smaller\n\n# Maximum security\nFROM gcr.io/distroless/nodejs20-debian12\n```\n\n**Python:**\n\n```dockerfile\n# Production with native packages\nFROM python:3.11-slim # Has wheels, reasonable size\n\n# Pure Python only\nFROM python:3.11-alpine # Smaller, but pip install is slower\n\n# Maximum security\nFROM gcr.io/distroless/python3-debian12\n```\n\n**Java:**\n\n```dockerfile\n# Production\nFROM eclipse-temurin:21-jre-alpine # Small, well maintained\n\n# Needs debugging tools\nFROM eclipse-temurin:21-jre # Full Debian\n\n# Maximum security\nFROM gcr.io/distroless/java21-debian12\n```\n\n**Go:**\n\n```dockerfile\n# Static binary, maximum security\nFROM scratch\n\n# Need some debugging\nFROM gcr.io/distroless/static\n\n# Need shell for debugging\nFROM alpine:3.18\n```\n\n**Rust:**\n\n```dockerfile\n# Static binary (musl target)\nFROM scratch\n\n# Dynamic binary\nFROM debian:bookworm-slim\n```\n\n### Keeping Images Updated\n\n**Pin versions but update regularly:**\n\n```dockerfile\n# ✅ Pin major.minor, update regularly\nFROM node:20.10-slim\n\n# ❌ Too specific - never gets security updates\nFROM node:20.10.0-slim\n\n# ❌ Too vague - unpredictable changes\nFROM node:latest\nFROM node:20\n```\n\n**Automate updates:**\n\n```yaml\n# .github/dependabot.yml\nversion: 2\nupdates:\n - package-ecosystem: docker\n directory: \"/\"\n schedule:\n interval: weekly\n reviewers:\n - myteam\n```\n\n**Rebuild on base image updates:**\n\n```yaml\n# GitHub Actions - rebuild on schedule\non:\n schedule:\n - cron: '0 0 * * 0' # Weekly\n push:\n branches: [main]\n\njobs:\n build:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v4\n - run: docker build --pull -t myapp .\n # --pull ensures fresh base image\n```\n\n---\n\n## ⚠️ Common Mistakes\n\n### Mistake 1: Using :latest in Production\n\n```dockerfile\n# ❌ Unpredictable, changes without notice\nFROM node:latest\n\n# ✅ Specific version\nFROM node:20.10-slim\n\n# ✅✅ Even better - with digest\nFROM node:20.10-slim@sha256:abc123...\n```\n\n### Mistake 2: Assuming Alpine is Always Better\n\n```dockerfile\n# ❌ Switching blindly to Alpine\nFROM python:3.11-alpine\nRUN pip install pandas numpy scipy scikit-learn\n# Takes 20+ minutes to compile!\n\n# ✅ Use slim for packages with native deps\nFROM python:3.11-slim\nRUN pip install pandas numpy scipy scikit-learn\n# Pre-built wheels, installs in seconds\n```\n\n### Mistake 3: Including Unnecessary Tools\n\n```dockerfile\n# ❌ Full development tools in production\nFROM node:20\nRUN apt-get update && apt-get install -y vim curl wget git\n\n# ✅ Minimal runtime only\nFROM node:20-slim\n# No extra packages needed at runtime\n```\n\n---\n\n## 🐛 Debug This: The Mysterious Segfault\n\nA team migrated from `python:3.11` to `python:3.11-alpine`. The application crashes randomly with segmentation faults.\n\n```bash\n$ docker run myapp\nStarting server...\nSegmentation fault (core dumped)\n```\n\n**The Dockerfile:**\n\n```dockerfile\nFROM python:3.11-alpine\nRUN pip install numpy pandas\nCOPY . .\nCMD [\"python\", \"app.py\"]\n```\n\n**Why does the app crash on Alpine?**\n\n---\n\n**✅ Solution:**\n\n**Root cause:** Binary incompatibility between glibc and musl.\n\nSome Python packages distribute pre-built wheels compiled against glibc. When these run on Alpine (musl), they can crash because:\n\n1. pip installed glibc-compiled wheel (manylinux)\n2. Alpine has musl, not glibc\n3. Certain operations cause segfault\n\n**Debug steps:**\n\n```bash\n# Check what was installed\ndocker run myapp pip show numpy\n# Might show manylinux wheel\n\n# Check for musl vs glibc\ndocker run myapp ldd --version\n# musl libc\n\n# Check binary compatibility\ndocker run myapp ldd /usr/local/lib/python3.11/site-packages/numpy/core/_multiarray_umath.cpython-311-x86_64-linux-gnu.so\n# Error: Not a glibc binary!\n```\n\n**Fixes:**\n\n**Option 1: Compile from source (slow but works):**\n```dockerfile\nFROM python:3.11-alpine\nRUN apk add --no-cache gcc musl-dev python3-dev openblas-dev\nRUN pip install --no-binary :all: numpy pandas\n```\n\n**Option 2: Use musllinux wheels:**\n```dockerfile\nFROM python:3.11-alpine\nRUN pip install numpy pandas\n# Recent versions have musllinux wheels\n```\n\n**Option 3: Use slim (recommended):**\n```dockerfile\nFROM python:3.11-slim\nRUN pip install numpy pandas\n# glibc - binary wheels work perfectly\n```\n\n**Lesson:** Not all pre-built binaries work on Alpine. When in doubt, use slim.\n\n---\n\n## 💻 Exercises\n\n### Exercise 1: Compare Base Image Sizes\n⭐ Difficulty: Easy | ⏱️ Time: 15 minutes\n\n```bash\n# Pull different variants\ndocker pull node:20\ndocker pull node:20-slim\ndocker pull node:20-alpine\n\n# Compare sizes\ndocker images node --format \"{{.Tag}}\\t{{.Size}}\"\n\n# Create simple app\ncat > app.js << 'EOF'\nconsole.log(\"Hello from Node!\");\nEOF\n\n# Build with each base\nfor variant in \"\" \"-slim\" \"-alpine\"; do\n cat > Dockerfile << EOF\nFROM node:20${variant}\nCOPY app.js /app.js\nCMD [\"node\", \"/app.js\"]\nEOF\n docker build -t test:node${variant:-\"-full\"} .\ndone\n\n# Compare final sizes\ndocker images test --format \"{{.Tag}}\\t{{.Size}}\"\n\n# Questions:\n# - How much smaller is alpine vs full?\n# - Does the app run the same on all variants?\n```\n\n---\n\n### Exercise 2: Alpine DNS Issue Reproduction\n⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes\n\n```bash\n# Create container to test DNS\ndocker run --rm -it alpine:3.18 sh\n\n# Inside container:\napk add --no-cache bind-tools\n\n# Test DNS (compare timing)\ntime nslookup google.com\ntime nslookup google.com\n\n# Now test with single-request option\necho \"options single-request\" >> /etc/resolv.conf\ntime nslookup google.com\n\n# Compare behavior:\n# - Try with your corporate DNS if available\n# - Note any timing differences\n```\n\n---\n\n### Exercise 3: Security Scan Comparison\n⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes\n\n```bash\n# Scan different base images\ndocker scout cves node:20\ndocker scout cves node:20-slim\ndocker scout cves node:20-alpine\ndocker scout cves gcr.io/distroless/nodejs20-debian12\n\n# Or with Trivy\ndocker run --rm aquasec/trivy image node:20\ndocker run --rm aquasec/trivy image node:20-alpine\n\n# Compare:\n# - Number of CVEs\n# - Severity distribution\n# - Which packages have vulnerabilities\n\n# Questions:\n# - Which has fewest critical CVEs?\n# - What's the tradeoff for fewer CVEs?\n```\n\n---\n\n### Exercise 4: Build Go App for Scratch\n⭐⭐⭐ Difficulty: Hard | ⏱️ Time: 25 minutes\n\n```bash\nmkdir scratch-test && cd scratch-test\n\ncat > main.go << 'EOF'\npackage main\n\nimport (\n \"fmt\"\n \"net/http\"\n \"time\"\n)\n\nfunc main() {\n loc, _ := time.LoadLocation(\"Europe/London\")\n fmt.Println(\"Time zone:\", loc)\n\n resp, err := http.Get(\"https://httpbin.org/get\")\n if err != nil {\n fmt.Println(\"HTTPS Error:\", err)\n } else {\n fmt.Println(\"HTTPS Status:\", resp.Status)\n }\n\n fmt.Println(\"Server running on :8080\")\n http.HandleFunc(\"/\", func(w http.ResponseWriter, r *http.Request) {\n fmt.Fprintf(w, \"Hello!\")\n })\n http.ListenAndServe(\":8080\", nil)\n}\nEOF\n\ncat > go.mod << 'EOF'\nmodule myapp\ngo 1.21\nEOF\n\n# Create Dockerfile for scratch that:\n# 1. Builds static binary\n# 2. Includes CA certificates (for HTTPS)\n# 3. Includes timezone data (for time.LoadLocation)\n# 4. Runs as non-root user\n\n# Test:\n# - HTTPS request should work\n# - Timezone should load\n# - Container should run as non-root\n```\n\n---\n\n### Exercise 5: Choose the Right Base Image\n⭐⭐⭐⭐ Difficulty: Expert | ⏱️ Time: 30 minutes\n\nFor each scenario, choose and justify the best base image:\n\n**Scenario 1:** Python ML application using TensorFlow, NumPy, Pandas\n- Requirements: Fast pip install, GPU support possibility\n\n**Scenario 2:** Node.js API with only pure JS dependencies (no native modules)\n- Requirements: Minimal size, fast startup\n\n**Scenario 3:** Java Spring Boot application\n- Requirements: Debugging capability in production\n\n**Scenario 4:** Go microservice making HTTPS calls\n- Requirements: Minimal CVEs, smallest possible size\n\n**Scenario 5:** Legacy PHP application with many extensions\n- Requirements: Compatibility, reasonable size\n\n```bash\n# For each scenario:\n# 1. Choose base image\n# 2. Explain why\n# 3. List tradeoffs\n# 4. Write minimal Dockerfile\n```\n\n---\n\n## 🎤 Senior-Level Interview Questions\n\n### Q1: What are the tradeoffs between Alpine and Debian-slim base images?\n\n**Strong Answer:**\n\n\"The core tradeoff is size vs compatibility.\n\n**Alpine:**\n- Uses musl libc (~7MB base)\n- Smaller images\n- Different C library can cause issues:\n - DNS parallel queries (causes timeouts with some DNS servers)\n - Python/Node native modules may not have pre-built wheels\n - Rare but real binary incompatibilities\n- Different package manager (apk)\n- Good for: Go, Rust, pure JS/Python\n\n**Debian-slim:**\n- Uses glibc (~74MB base)\n- Standard library - all pre-built binaries work\n- No DNS quirks\n- Familiar apt package manager\n- More CVEs (more packages)\n- Good for: Python with native deps, Node with native deps, anything needing glibc\n\n**My decision process:**\n1. Do I have native dependencies? → Likely slim\n2. Pure interpreted code, no native? → Alpine is fine\n3. Compiled static binary (Go/Rust)? → Alpine or scratch\n4. Production critical, low tolerance for weird issues? → Slim\n5. Need absolute smallest? → Alpine with testing\n\nI default to slim for production services because debugging musl issues in production is painful. Use Alpine when size is critical and you've tested thoroughly.\"\n\n### Q2: When would you use distroless vs scratch images?\n\n**Strong Answer:**\n\n\"Both are minimal, but serve different needs:\n\n**Scratch:**\n- Truly empty - 0 bytes\n- For fully static binaries only\n- You must provide: CA certs, timezone data, /etc/passwd\n- No shell - debugging requires rebuild or docker cp\n- Best for: Go, Rust with static linking\n\n**Distroless:**\n- Google-maintained minimal images\n- Has runtime necessities (CA certs, tzdata, libc)\n- Variants for different runtimes (java, python, nodejs)\n- No shell by default, but debug variant available\n- Best for: Java, Python, Node, dynamically linked binaries\n\n**My decision:**\n\n```\nStatic binary (Go/Rust)?\n├── Need debugging tools? → Distroless static\n└── Maximum minimal? → Scratch\n\nInterpreted/JVM?\n└── Distroless (java/python/nodejs variant)\n\nNeed shell sometimes?\n└── Distroless debug variant\n```\n\nFor Go services, I typically use scratch in production with distroless/static-debug for staging. For Java, distroless/java is my default because managing JRE dependencies manually is tedious.\"\n\n### Q3: How do you keep base images secure and updated?\n\n**Strong Answer:**\n\n\"Multi-layered approach:\n\n**1. Pin versions but not too specifically:**\n```dockerfile\nFROM node:20.10-slim # Major.minor, not patch\n```\nAllows security patches while avoiding breaking changes.\n\n**2. Regular automated rebuilds:**\n- Weekly scheduled builds with `--pull`\n- Dependabot/Renovate for Dockerfile updates\n- CI triggers on base image updates\n\n**3. Security scanning in CI:**\n```yaml\n- run: docker scout cves --exit-code --only-severity critical,high\n```\nFail builds with critical CVEs.\n\n**4. Runtime scanning:**\n- Tools like Prisma Cloud, Aqua, Twistlock\n- Continuous monitoring of deployed images\n\n**5. Base image lifecycle:**\n- Track upstream EOL dates\n- Plan migrations before support ends\n- Test with new versions in staging first\n\n**6. Minimize attack surface:**\n- Use slim/distroless where possible\n- Multi-stage builds - no build tools in runtime\n- Remove unnecessary packages\n\n**Example policy:**\n- Critical CVE: Fix within 24 hours\n- High CVE: Fix within 1 week\n- Monthly base image updates\n- Quarterly major version evaluation\"\n\n### Q4: Explain the musl vs glibc issue and how to debug it.\n\n**Strong Answer:**\n\n\"musl and glibc are different C library implementations. Alpine uses musl, most other distros use glibc.\n\n**Where it matters:**\n\n1. **Pre-built binaries:** Many pip/npm packages distribute binaries compiled against glibc. On Alpine, these either fail to install or crash at runtime.\n\n2. **DNS behavior:** musl sends A and AAAA queries in parallel. Some DNS servers (especially older corporate ones) can't handle this, causing timeouts.\n\n3. **Memory allocation:** Different implementations, usually fine but can cause issues with specific software.\n\n4. **Thread handling:** Slight differences that rarely matter.\n\n**Debugging approach:**\n\n```bash\n# 1. Check if binary is glibc or musl\nldd /path/to/binary\n# glibc shows libc.so.6\n# musl shows ld-musl-*.so.1\n\n# 2. Check which libc the image has\ndocker run image ldd --version\n# musl shows musl version\n# glibc shows GNU libc version\n\n# 3. For segfaults, check LD_DEBUG\ndocker run -e LD_DEBUG=all image ./binary 2>&1 | head -100\n\n# 4. For DNS issues\ndocker run image time nslookup google.com\n# > 1 second usually means parallel query issue\n```\n\n**Solutions:**\n1. Use slim base (has glibc)\n2. Compile from source on Alpine\n3. Use musllinux wheels (Python)\n4. Add `options single-request` to resolv.conf (DNS)\n\nI recommend defaulting to slim for anything with native dependencies. Alpine savings aren't worth production debugging nightmares.\"\n\n### Q5: How do you choose a base image for a new project?\n\n**Strong Answer:**\n\n\"I follow a decision tree based on language and requirements:\n\n**Step 1: Language runtime**\n- Go/Rust with static binary → scratch or distroless/static\n- Java → eclipse-temurin or distroless/java\n- Python → python:slim or distroless/python\n- Node → node:slim or distroless/nodejs\n\n**Step 2: Dependencies**\n- Native modules (bcrypt, numpy)? → Need glibc → slim\n- Pure interpreted? → Alpine is safe\n- System packages needed? → Consider full image\n\n**Step 3: Operations requirements**\n- Need shell for debugging? → Avoid scratch, use slim\n- Regulatory/security? → Distroless or minimal\n- Easy troubleshooting valued? → Slim with tools\n\n**Step 4: Size vs convenience**\n- CI/CD pull time critical? → Prioritize small\n- Cold start latency matters? → Prioritize small\n- Otherwise → Prioritize compatibility\n\n**My defaults:**\n```\nPython (data science): python:3.11-slim\nPython (web service): python:3.11-slim or distroless\nNode.js (most): node:20-slim\nNode.js (simple): node:20-alpine\nJava: eclipse-temurin:21-jre-alpine\nGo: scratch with CA certs\n```\n\nI always test the chosen base in staging environment before committing, especially when switching from a familiar base to a minimal one.\"\n\n---\n\n## 📝 Summary & Key Takeaways\n\n### Base Image Decision Guide\n\n| Requirement | Recommended Base |\n|-------------|-----------------|\n| Maximum compatibility | debian:slim |\n| Smallest with compatibility | alpine (test first!) |\n| Minimal CVEs, interpreted | distroless |\n| Minimal CVEs, compiled | scratch |\n| Need debugging | slim or distroless:debug |\n| Native Python packages | python:slim |\n| Native Node modules | node:slim |\n| Java production | eclipse-temurin:jre-alpine |\n| Go production | scratch |\n\n### Key Principles\n\n1. **Pin versions** - Use major.minor, not :latest\n2. **Test Alpine** - Don't assume it works like Debian\n3. **Scan regularly** - CVEs accumulate over time\n4. **Update proactively** - Don't wait for incidents\n5. **Match to workload** - No universal best choice\n\n---\n\n## 📋 Quick Reference\n\n### Size Comparison\n\n| Base | Size |\n|------|------|\n| ubuntu:22.04 | 77MB |\n| debian:bookworm-slim | 74MB |\n| alpine:3.18 | 7MB |\n| gcr.io/distroless/static | 2MB |\n| scratch | 0MB |\n\n### Alpine DNS Fix\n\n```dockerfile\n# In Dockerfile\nRUN echo \"options single-request-reopen\" >> /etc/resolv.conf\n\n# In docker-compose.yml\nservices:\n app:\n dns_opt:\n - single-request-reopen\n```\n\n### Security Scanning\n\n```bash\n# Docker Scout\ndocker scout cves IMAGE\n\n# Trivy\ndocker run aquasec/trivy image IMAGE\n\n# Grype\ngrype IMAGE\n```\n\n---\n\n## 📅 Review Schedule\n\n| Day | Task | Time |\n|-----|------|------|\n| Day 1 | Review base image decision tree | 10 min |\n| Day 3 | Run security scan on current project images | 15 min |\n| Day 7 | Do Exercise 1 (compare base sizes) | 15 min |\n| Day 14 | Test Alpine migration for one service | 25 min |\n| Day 30 | Audit all project base images | 30 min |\n\n---\n\n## 📚 Series Navigation\n\n| Previous | Current | Next |\n|----------|---------|------|\n| [Part 6: Multi-Stage Builds](docker-part-6-multi-stage-builds) | **Part 7: Base Image Selection** | [Part 8: Build Configuration](docker-part-8-build-configuration) |\n\n**Docker Compendium Series:**\n- [Part 0: How to Use This Series](docker-part-0-how-to-use-this-series)\n- [Part 1: Container Internals](docker-part-1-container-internals)\n- [Part 2: Image Anatomy](docker-part-2-image-anatomy)\n- [Part 3: Build Process Deep Dive](docker-part-3-build-process)\n- [Part 4: Networking Internals](docker-part-4-networking-internals)\n- [Part 5: Dockerfile Optimization Patterns](docker-part-5-dockerfile-optimization)\n- [Part 6: Multi-Stage Builds: Beyond Basics](docker-part-6-multi-stage-builds)\n- **Part 7: Base Image Selection & Security** ← You are here\n- [Part 8: ARG, ENV & Build-Time Configuration](docker-part-8-build-configuration)\n","featuredImage","isPublished","isFeatured","subscriptionType","BASIC","publishedAt","2026-06-10T07:19:25.448461Z","createdAt","2026-06-10T07:19:25.449116Z","updatedAt","2026-06-16T21:26:56.086200Z","readingTimeMinutes",17,"estimatedReadingTime","authorId",3,"authorUsername","tomasz_main","categoryName","Devops","readingTime","seriesId","seriesOrder",7,"author",{"_33":57,"_99":59,"_100":101,"_102":103,"_104":-5,"_105":-5,"_106":-5,"_107":-5,"_108":-5},"category",{"_33":65,"_74":61,"_37":96,"_77":97,"_98":-5},"tags",[80,81,82,83],"series",{"_33":57,"_74":75,"_37":76,"_77":78,"_79":24},"name","Docker Compendium: From Practitioner to Expert","docker-compendium","description","Master Docker internals, optimization patterns, and production best practices. For developers who use Docker daily but want deeper understanding of what's happening under the hood. Not a beginner series - we assume you can write Dockerfiles and use docker-compose. Learn what namespaces and cgroups really do, why your builds are slow, how to debug containers that crash, and how to run Docker in production like a pro.\n","isActive",{"_33":93,"_74":94,"_37":95},{"_33":90,"_74":91,"_37":92},{"_33":87,"_74":88,"_37":89},{"_33":84,"_74":85,"_37":86},229,"Security","security",213,"Docker","docker",231,"Distroless","distroless",230,"Alpine","alpine","devops","Category for Devops","colorHex","username","firstName","Tomasz","lastName","Jarosz","profilePicture","bio","website","githubUrl","linkedinUrl",62,"Container Security Hardening","docker-part-12-security","Non-root, read-only fs, capabilities, seccomp","# Docker Compendium Part 12: Container Security Hardening\n\nContainers provide isolation, but not security by default. Running as root, with all capabilities, writable filesystem - these defaults are convenient but dangerous. This article covers practical hardening: non-root users, read-only filesystems, capability dropping, and defense in depth.\n\n## 📋 At a Glance\n\n| Aspect | Details |\n|--------|---------|\n| Topic | Non-root users, capabilities, read-only fs, seccomp, AppArmor |\n| Complexity | Advanced |\n| Prerequisites | Part 1 (Container Internals), Part 7 (Base Images) |\n| Key Insight | Security is layers - each control reduces attack surface |\n| Time to Master | 3-4 hours |\n\n---\n\n## 🎯 What You'll Learn\n\n- **Non-root containers** - why and how to run without root\n- **Linux capabilities** - granular permissions instead of all-or-nothing\n- **Read-only filesystems** - preventing runtime modifications\n- **Seccomp profiles** - limiting system calls\n- **Defense in depth** - combining multiple security controls\n\n---\n\n## 🔥 Production Story: The Container Escape\n\nA security audit found a critical vulnerability: an attacker who compromised the web application could escape the container and access the host.\n\n**The vulnerable setup:**\n\n```yaml\nservices:\n web:\n image: myapp\n privileged: true # \"Needed for X feature\"\n volumes:\n - /:/host # \"For debugging\"\n```\n\n**The attack path:**\n\n```bash\n# Inside compromised container (as root)\n$ cat /host/etc/shadow # Read host passwords\n$ echo \"attacker ALL=(ALL) NOPASSWD:ALL\" >> /host/etc/sudoers\n$ chroot /host /bin/bash # Escape to host\n```\n\n**The fix:**\n\n```yaml\nservices:\n web:\n image: myapp\n user: \"1000:1000\"\n read_only: true\n security_opt:\n - no-new-privileges:true\n cap_drop:\n - ALL\n tmpfs:\n - /tmp\n```\n\n**Lesson:** Never use `privileged: true` or mount host root. Every unnecessary permission is an attack vector.\n\n---\n\n## 🧠 Mental Model: Defense in Depth\n\n```\n┌─────────────────────────────────────────────────────────────────────────┐\n│ CONTAINER SECURITY LAYERS │\n│ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ LAYER 1: IMAGE SECURITY │\n│ │ • Minimal base image (distroless, alpine) │\n│ │ • No secrets in layers │\n│ │ • Regular vulnerability scanning │\n│ │ • Signed/verified images │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ LAYER 2: USER PERMISSIONS │\n│ │ • Non-root user │\n│ │ • no-new-privileges │\n│ │ • Minimal file permissions │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ LAYER 3: CAPABILITIES │\n│ │ • Drop ALL capabilities │\n│ │ • Add only what's needed │\n│ │ • Never use --privileged │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ LAYER 4: FILESYSTEM │\n│ │ • Read-only root filesystem │\n│ │ • tmpfs for writable paths │\n│ │ • No host mounts (or read-only) │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ LAYER 5: SYSCALL FILTERING │\n│ │ • Seccomp profiles │\n│ │ • AppArmor/SELinux │\n│ │ • Block dangerous syscalls │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │ │\n│ ▼ │\n│ ┌──────────────────────────────────────────────────────────────────────┐\n│ │ LAYER 6: NETWORK │\n│ │ • Isolated networks │\n│ │ • No unnecessary port exposure │\n│ │ • Network policies │\n│ └──────────────────────────────────────────────────────────────────────┘\n│ │\n│ Each layer reduces attack surface. Compromise of one layer │\n│ doesn't mean full compromise. │\n└─────────────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 🔬 Deep Dive\n\n### Non-Root Containers\n\nBy default, containers run as root (UID 0). This is dangerous:\n\n```bash\n# Default - running as root\n$ docker run --rm alpine id\nuid=0(root) gid=0(root)\n\n# Root in container = root on host filesystem\n# If attacker escapes container, they're root on host\n```\n\n**Create non-root user in Dockerfile:**\n\n```dockerfile\nFROM node:20-alpine\n\n# Create non-root user\nRUN addgroup -g 1001 -S appgroup && \\\n adduser -u 1001 -S appuser -G appgroup\n\n# Set ownership of app files\nWORKDIR /app\nCOPY --chown=appuser:appgroup . .\n\n# Switch to non-root user\nUSER appuser\n\nCMD [\"node\", \"index.js\"]\n```\n\n**Run as non-root at runtime:**\n\n```bash\n# Override user at runtime\ndocker run --user 1000:1000 myapp\n\n# Use nobody user\ndocker run --user nobody myapp\n```\n\n**Docker Compose:**\n\n```yaml\nservices:\n app:\n image: myapp\n user: \"1000:1000\"\n```\n\n**Verify non-root:**\n\n```bash\n$ docker run --rm myapp id\nuid=1001(appuser) gid=1001(appgroup)\n\n$ docker run --rm myapp whoami\nappuser\n```\n\n### no-new-privileges\n\nPrevents processes from gaining additional privileges:\n\n```bash\n# Without no-new-privileges, setuid binaries work\ndocker run --rm alpine sh -c \"\n chmod u+s /bin/ping\n su -s /bin/sh nobody -c 'ping -c 1 google.com'\n\"\n# Works - nobody user can use setuid ping\n\n# With no-new-privileges\ndocker run --rm --security-opt no-new-privileges alpine sh -c \"\n chmod u+s /bin/ping\n su -s /bin/sh nobody -c 'ping -c 1 google.com'\n\"\n# Fails - setuid doesn't grant privileges\n```\n\n**Always enable:**\n\n```yaml\nservices:\n app:\n security_opt:\n - no-new-privileges:true\n```\n\n### Linux Capabilities\n\nCapabilities split root power into granular permissions:\n\n```bash\n# See default capabilities\ndocker run --rm alpine sh -c \"apk add libcap && capsh --print\"\n\n# Default capabilities include:\n# cap_chown, cap_dac_override, cap_fowner, cap_fsetid,\n# cap_kill, cap_setgid, cap_setuid, cap_setpcap,\n# cap_net_bind_service, cap_net_raw, cap_sys_chroot,\n# cap_mknod, cap_audit_write, cap_setfcap\n```\n\n**Common capabilities:**\n\n| Capability | Purpose |\n|------------|---------|\n| NET_BIND_SERVICE | Bind to ports < 1024 |\n| NET_RAW | Use raw sockets (ping) |\n| CHOWN | Change file ownership |\n| SETUID/SETGID | Change user/group |\n| SYS_ADMIN | Many admin operations (dangerous!) |\n| SYS_PTRACE | Debug processes |\n\n**Drop all, add only needed:**\n\n```bash\n# Drop ALL capabilities\ndocker run --cap-drop ALL myapp\n\n# Drop all, add specific\ndocker run --cap-drop ALL --cap-add NET_BIND_SERVICE myapp\n```\n\n```yaml\n# docker-compose.yml\nservices:\n app:\n cap_drop:\n - ALL\n cap_add:\n - NET_BIND_SERVICE # Only if binding to port < 1024\n```\n\n**Never use --privileged:**\n\n```bash\n# ❌ NEVER in production\ndocker run --privileged myapp\n# Gives ALL capabilities + host device access + disables seccomp\n# Effectively no container isolation\n```\n\n### Read-Only Filesystem\n\nPrevent runtime modifications to container filesystem:\n\n```bash\ndocker run --read-only myapp\n```\n\n**Handle writable requirements:**\n\n```bash\n# Read-only with tmpfs for temp files\ndocker run --read-only --tmpfs /tmp:size=100m myapp\n\n# Multiple tmpfs mounts\ndocker run --read-only \\\n --tmpfs /tmp \\\n --tmpfs /var/run \\\n --tmpfs /var/cache \\\n myapp\n```\n\n```yaml\nservices:\n app:\n read_only: true\n tmpfs:\n - /tmp:size=100m\n - /var/run\n - /app/cache\n```\n\n**Identify needed writable paths:**\n\n```bash\n# Run app and see what it tries to write\ndocker run --rm myapp 2>&1 | grep -i \"read-only\\|permission denied\"\n\n# Or use strace\ndocker run --rm --cap-add SYS_PTRACE myapp strace -f -e write ./app 2>&1\n```\n\n### Seccomp Profiles\n\nSeccomp filters system calls:\n\n```bash\n# Default Docker seccomp blocks ~44 dangerous syscalls\n# Including: reboot, mount, ptrace, etc.\n\n# Use default profile explicitly\ndocker run --security-opt seccomp=default myapp\n\n# Custom profile\ndocker run --security-opt seccomp=./profile.json myapp\n\n# Disable seccomp (dangerous!)\ndocker run --security-opt seccomp=unconfined myapp\n```\n\n**Custom seccomp profile:**\n\n```json\n{\n \"defaultAction\": \"SCMP_ACT_ERRNO\",\n \"architectures\": [\"SCMP_ARCH_X86_64\"],\n \"syscalls\": [\n {\n \"names\": [\"read\", \"write\", \"open\", \"close\", \"stat\", \"fstat\",\n \"mmap\", \"mprotect\", \"munmap\", \"brk\", \"exit_group\",\n \"socket\", \"connect\", \"accept\", \"bind\", \"listen\"],\n \"action\": \"SCMP_ACT_ALLOW\"\n }\n ]\n}\n```\n\n**Generate profile for your app:**\n\n```bash\n# Use strace to see what syscalls app uses\ndocker run --rm myapp strace -c -f ./app 2>&1\n\n# Or use OCI seccomp profiler\ndocker run --rm -v /var/run/docker.sock:/var/run/docker.sock \\\n genuinetools/amicontained\n```\n\n### AppArmor\n\nAppArmor provides Mandatory Access Control:\n\n```bash\n# Check if AppArmor is enabled\ndocker info | grep -i apparmor\n\n# Default profile\ndocker run --security-opt apparmor=docker-default myapp\n\n# Custom profile\ndocker run --security-opt apparmor=my-profile myapp\n\n# Disable (not recommended)\ndocker run --security-opt apparmor=unconfined myapp\n```\n\n### Complete Hardened Container\n\n```dockerfile\nFROM node:20-alpine\n\n# Security: non-root user\nRUN addgroup -g 1001 -S app && \\\n adduser -u 1001 -S app -G app\n\nWORKDIR /app\n\n# Security: minimal dependencies\nRUN apk add --no-cache dumb-init\n\n# Copy with correct ownership\nCOPY --chown=app:app package*.json ./\nRUN npm ci --only=production && npm cache clean --force\nCOPY --chown=app:app . .\n\n# Security: non-root user\nUSER app\n\n# Security: use init for proper signal handling\nENTRYPOINT [\"dumb-init\", \"--\"]\nCMD [\"node\", \"index.js\"]\n```\n\n```yaml\n# docker-compose.yml\nservices:\n app:\n image: myapp\n read_only: true\n user: \"1001:1001\"\n security_opt:\n - no-new-privileges:true\n cap_drop:\n - ALL\n tmpfs:\n - /tmp:size=50m,noexec,nosuid\n deploy:\n resources:\n limits:\n memory: 256M\n cpus: '0.5'\n```\n\n---\n\n## ⚠️ Common Mistakes\n\n### Mistake 1: Running as Root\n\n```dockerfile\n# ❌ No USER instruction - runs as root\nFROM node:20\nCOPY . .\nCMD [\"node\", \"index.js\"]\n\n# ✅ Explicit non-root user\nFROM node:20\nRUN useradd -r -u 1001 appuser\nUSER appuser\nCOPY --chown=appuser . .\nCMD [\"node\", \"index.js\"]\n```\n\n### Mistake 2: Using --privileged\n\n```yaml\n# ❌ Never use privileged\nservices:\n app:\n privileged: true\n\n# ✅ Add only specific capabilities needed\nservices:\n app:\n cap_add:\n - SYS_PTRACE # Only if actually needed\n```\n\n### Mistake 3: Mounting Sensitive Host Paths\n\n```yaml\n# ❌ Never mount host root or sensitive paths\nservices:\n app:\n volumes:\n - /:/host\n - /var/run/docker.sock:/var/run/docker.sock\n\n# ✅ Mount only what's needed, read-only when possible\nservices:\n app:\n volumes:\n - ./config:/app/config:ro\n```\n\n---\n\n## 🐛 Debug This: The Permission Denied Mystery\n\nA developer reports: \"My app works with root but fails as non-root user!\"\n\n```dockerfile\nFROM node:20-alpine\nRUN adduser -D appuser\nWORKDIR /app\nCOPY . .\nRUN npm install\nUSER appuser\nCMD [\"node\", \"index.js\"]\n```\n\n```bash\n$ docker run myapp\nError: EACCES: permission denied, open '/app/data/cache.json'\n```\n\n**Why does the non-root user fail?**\n\n---\n\n**✅ Solution:**\n\nThe `/app` directory and files are owned by root because COPY and npm install ran before USER instruction.\n\n**The problem:**\n1. COPY runs as root → files owned by root\n2. npm install runs as root → node_modules owned by root\n3. USER appuser switches user\n4. appuser can't write to root-owned directories\n\n**Fix 1: Set ownership explicitly:**\n\n```dockerfile\nFROM node:20-alpine\nRUN adduser -D appuser\nWORKDIR /app\n\n# Change ownership of app directory\nRUN chown -R appuser:appuser /app\n\n# Copy with correct ownership\nCOPY --chown=appuser:appuser . .\n\nUSER appuser\nRUN npm install\nCMD [\"node\", \"index.js\"]\n```\n\n**Fix 2: Create writable directories:**\n\n```dockerfile\nFROM node:20-alpine\nRUN adduser -D appuser\nWORKDIR /app\nCOPY --chown=appuser:appuser . .\n\n# Create writable data directory\nRUN mkdir -p /app/data && chown appuser:appuser /app/data\n\nUSER appuser\nRUN npm install\nCMD [\"node\", \"index.js\"]\n```\n\n**Fix 3: Use tmpfs for temporary data:**\n\n```yaml\nservices:\n app:\n user: \"1001:1001\"\n tmpfs:\n - /app/data:uid=1001,gid=1001\n```\n\n**Key lesson:** Order matters. COPY ownership, directory permissions, and USER instruction must be coordinated.\n\n---\n\n## 💻 Exercises\n\n### Exercise 1: Audit Container Security\n⭐ Difficulty: Easy | ⏱️ Time: 15 minutes\n\n```bash\n# Run a container and check security posture\ndocker run -d --name audit-test nginx\n\n# Check user\ndocker exec audit-test id\n# Is it root?\n\n# Check capabilities\ndocker exec audit-test sh -c \"apk add libcap && capsh --print\"\n\n# Check if filesystem is writable\ndocker exec audit-test touch /test-file\ndocker exec audit-test rm /test-file\n\n# Check seccomp\ndocker inspect audit-test --format '{{.HostConfig.SecurityOpt}}'\n\n# Clean up\ndocker rm -f audit-test\n```\n\n---\n\n### Exercise 2: Create Non-Root Image\n⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes\n\n```bash\nmkdir nonroot-test && cd nonroot-test\n\n# Create app\ncat > index.js << 'EOF'\nconst http = require('http');\nconst fs = require('fs');\n\n// Write to check permissions\nfs.writeFileSync('/app/data/test.txt', 'Hello');\n\nhttp.createServer((req, res) => {\n res.end(`Running as UID: ${process.getuid()}\\n`);\n}).listen(3000);\nconsole.log('Server running on port 3000');\nEOF\n\n# Create Dockerfile that:\n# 1. Uses non-root user\n# 2. Sets correct ownership\n# 3. Can write to /app/data\n\n# Verify:\n# - Container runs as non-root\n# - App can write to data directory\n# - App cannot write to /etc or other system dirs\n```\n\n---\n\n### Exercise 3: Capability Analysis\n⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes\n\n```bash\n# Test what capabilities are needed\n\n# Run with all capabilities dropped\ndocker run --rm --cap-drop ALL alpine ping -c 1 google.com\n# Fails - needs NET_RAW\n\n# Add only what's needed\ndocker run --rm --cap-drop ALL --cap-add NET_RAW alpine ping -c 1 google.com\n# Works\n\n# Test other capabilities:\n# 1. Binding to port 80 (needs NET_BIND_SERVICE?)\n# 2. Changing file ownership (needs CHOWN?)\n# 3. Running setuid binary (needs SETUID?)\n\n# Document minimum capabilities for each scenario\n```\n\n---\n\n### Exercise 4: Read-Only Filesystem\n⭐⭐⭐ Difficulty: Hard | ⏱️ Time: 25 minutes\n\n```bash\nmkdir readonly-test && cd readonly-test\n\ncat > app.py << 'EOF'\nimport os\nimport tempfile\n\n# App needs to:\n# 1. Read config from /app/config\n# 2. Write logs to /app/logs\n# 3. Create temp files\n# 4. Write to /tmp\n\nprint(\"Testing filesystem...\")\nprint(f\"Config: {open('/app/config/app.conf').read()}\")\n\nwith open('/app/logs/app.log', 'w') as f:\n f.write('Log entry\\n')\nprint(\"Wrote to logs\")\n\nwith tempfile.NamedTemporaryFile(dir='/tmp', delete=False) as f:\n f.write(b'temp data')\n print(f\"Temp file: {f.name}\")\n\nprint(\"All tests passed!\")\nEOF\n\n# Create Dockerfile and compose that:\n# 1. Uses read-only root filesystem\n# 2. Allows writing to /app/logs via tmpfs\n# 3. Allows /tmp via tmpfs\n# 4. Mounts config as read-only\n\n# Verify container can't write to / or /app\n```\n\n---\n\n### Exercise 5: Complete Hardened Setup\n⭐⭐⭐⭐ Difficulty: Expert | ⏱️ Time: 35 minutes\n\nCreate a fully hardened containerized application:\n\n```yaml\n# Requirements:\n# 1. Non-root user with specific UID\n# 2. All capabilities dropped\n# 3. Read-only filesystem with needed tmpfs\n# 4. no-new-privileges enabled\n# 5. Resource limits\n# 6. Health check\n# 7. Proper signal handling (init process)\n# 8. No unnecessary ports exposed\n# 9. Minimal base image\n\n# Verify security:\n# - Cannot write to filesystem (except tmpfs)\n# - Cannot gain privileges\n# - Cannot use dangerous syscalls\n# - Runs as non-root\n# - Has minimal capabilities\n```\n\n---\n\n## 🎤 Senior-Level Interview Questions\n\n### Q1: Why should containers run as non-root?\n\n**Strong Answer:**\n\n\"Running as root in containers is dangerous for several reasons:\n\n**1. Container escape amplification:**\nIf an attacker exploits a container escape vulnerability, they're root on the host. Non-root limits the damage.\n\n**2. Defense in depth:**\nEven without escape, root can do more damage inside the container - modify binaries, read sensitive files, etc.\n\n**3. Shared kernel:**\nContainer root is the same UID as host root. Kernel doesn't distinguish between 'root in container' and 'root on host'.\n\n**4. Best practices:**\nMost security frameworks (CIS Docker Benchmark, NIST) require non-root containers.\n\n**Implementation:**\n```dockerfile\nFROM node:20-alpine\nRUN adduser -D -u 1001 appuser\nCOPY --chown=appuser:appuser . .\nUSER appuser\n```\n\n**Exceptions:**\n- Some software legitimately needs root (port 80, raw sockets)\n- Solution: Drop privileges after initialization\n- Or use capabilities instead of full root\n\nI always start with non-root and only add specific capabilities if absolutely necessary.\"\n\n### Q2: Explain Linux capabilities and how they relate to container security.\n\n**Strong Answer:**\n\n\"Linux capabilities break root's all-or-nothing power into 40+ granular permissions.\n\n**Traditional model:**\n- UID 0 = can do everything\n- Other UIDs = restricted\n\n**Capabilities model:**\n- Each capability grants specific power\n- Can have root UID without capabilities\n- Can have capabilities without root UID\n\n**Key capabilities:**\n- `CAP_NET_BIND_SERVICE` - Bind ports < 1024\n- `CAP_NET_RAW` - Raw sockets (ping)\n- `CAP_SYS_ADMIN` - Mount, namespace ops (dangerous!)\n- `CAP_CHOWN` - Change file ownership\n\n**Docker default:**\nContainers get ~14 capabilities by default. Not minimal, but not full root either.\n\n**Best practice:**\n```yaml\ncap_drop:\n - ALL\ncap_add:\n - NET_BIND_SERVICE # Only if needed\n```\n\n**Why drop all:**\n- Reduces attack surface\n- Most apps don't need any capabilities\n- Forces explicit thought about requirements\n\n**The dangerous ones:**\n- `SYS_ADMIN` - Basically root, avoid\n- `NET_ADMIN` - Network configuration\n- `SYS_PTRACE` - Process debugging\n\nNever use `--privileged`, which grants ALL capabilities plus more.\"\n\n### Q3: How do you implement defense in depth for containers?\n\n**Strong Answer:**\n\n\"Defense in depth means multiple security layers, so compromising one doesn't mean full compromise:\n\n**Layer 1 - Image security:**\n- Minimal base image (distroless, alpine)\n- No secrets in image\n- Regular vulnerability scanning\n- Image signing\n\n**Layer 2 - Runtime security:**\n- Non-root user\n- Read-only filesystem\n- Drop all capabilities\n- no-new-privileges\n\n**Layer 3 - System call filtering:**\n- Seccomp profiles (default or custom)\n- Block dangerous syscalls\n\n**Layer 4 - Mandatory Access Control:**\n- AppArmor or SELinux profiles\n- Restrict file access patterns\n\n**Layer 5 - Network security:**\n- Isolated networks\n- Network policies\n- No unnecessary ports\n\n**Layer 6 - Resource limits:**\n- Memory limits (prevent DoS)\n- CPU limits\n- PID limits\n\n**Example implementation:**\n```yaml\nservices:\n app:\n image: myapp # Scanned, signed\n user: '1001:1001'\n read_only: true\n cap_drop: [ALL]\n security_opt:\n - no-new-privileges:true\n - seccomp:./profile.json\n tmpfs: [/tmp]\n networks: [internal]\n deploy:\n resources:\n limits:\n memory: 256M\n```\n\nEach layer stops different attack types. Even if one fails, others provide protection.\"\n\n### Q4: What is the difference between --privileged and specific capability grants?\n\n**Strong Answer:**\n\n\"They're vastly different in security impact:\n\n**--privileged:**\n- Grants ALL capabilities (~40+)\n- Gives access to all host devices\n- Disables seccomp filtering\n- Disables AppArmor\n- Container is essentially root on host\n- Use case: Almost never in production\n\n**Specific capabilities:**\n- Grant only what's needed\n- Keep other restrictions in place\n- Seccomp still active\n- AppArmor still active\n- Principle of least privilege\n\n**Example - needing raw sockets:**\n```bash\n# ❌ Privileged - massive overkill\ndocker run --privileged myapp\n\n# ✅ Just the capability needed\ndocker run --cap-add NET_RAW myapp\n```\n\n**When people think they need privileged:**\n- Docker-in-Docker → Use Docker socket mount or DinD image properly\n- Device access → Mount specific device\n- Network tools → Add NET_ADMIN or NET_RAW\n- Debugging → Add SYS_PTRACE temporarily\n\nI've never seen a legitimate production use case for `--privileged`. It's a security red flag in any deployment.\"\n\n### Q5: How would you secure a container that needs to run as root?\n\n**Strong Answer:**\n\n\"Sometimes root is truly needed (legacy apps, port 80, specific syscalls). Here's how to minimize risk:\n\n**1. Drop to non-root after initialization:**\n```dockerfile\n# Start as root for privileged operations\nCOPY entrypoint.sh /\nENTRYPOINT ['/entrypoint.sh']\n```\n\n```bash\n#!/bin/sh\n# Privileged operations\nchown app:app /data\nsetcap 'cap_net_bind_service=+ep' /app/server\n\n# Drop to non-root\nexec gosu app /app/server\n```\n\n**2. Use capabilities instead:**\n```yaml\ncap_drop:\n - ALL\ncap_add:\n - NET_BIND_SERVICE # For port 80\n # Still running as root, but with minimal powers\n```\n\n**3. Maximum restrictions:**\n```yaml\nservices:\n app:\n # If must run as root:\n read_only: true\n security_opt:\n - no-new-privileges:true\n cap_drop:\n - ALL\n cap_add:\n - CHOWN\n - SETUID\n - SETGID\n```\n\n**4. User namespace remapping:**\n```json\n// /etc/docker/daemon.json\n{\n 'userns-remap': 'default'\n}\n```\nRoot in container maps to unprivileged user on host.\n\n**5. Additional isolation:**\n- Custom seccomp profile\n- AppArmor profile\n- Read-only mounts\n- Network isolation\n\nThe goal: If container is compromised, attacker gets minimal access even as root in container.\"\n\n---\n\n## 📝 Summary & Key Takeaways\n\n### Security Checklist\n\n| Control | Implementation |\n|---------|---------------|\n| Non-root user | `USER appuser` in Dockerfile |\n| Drop capabilities | `cap_drop: [ALL]` |\n| Read-only fs | `read_only: true` |\n| No new privileges | `no-new-privileges:true` |\n| Resource limits | `memory`, `cpus` limits |\n| Minimal base | distroless, alpine |\n\n### Security Hierarchy\n\n```\nNEVER: --privileged, host mounts, docker.sock\nAVOID: Running as root, unnecessary capabilities\nALWAYS: Non-root, dropped caps, read-only, limits\n```\n\n---\n\n## 📋 Quick Reference\n\n### Docker Run Security Flags\n\n```bash\n# Non-root\n--user 1000:1000\n\n# Drop capabilities\n--cap-drop ALL\n--cap-add NET_BIND_SERVICE\n\n# Read-only with tmpfs\n--read-only --tmpfs /tmp\n\n# No privilege escalation\n--security-opt no-new-privileges:true\n\n# Custom seccomp\n--security-opt seccomp=./profile.json\n```\n\n### Compose Security\n\n```yaml\nservices:\n app:\n user: \"1000:1000\"\n read_only: true\n cap_drop:\n - ALL\n security_opt:\n - no-new-privileges:true\n tmpfs:\n - /tmp\n```\n\n---\n\n## 📅 Review Schedule\n\n| Day | Task | Time |\n|-----|------|------|\n| Day 1 | Review security layers diagram | 10 min |\n| Day 3 | Audit a running container's security | 15 min |\n| Day 7 | Convert an image to non-root | 20 min |\n| Day 14 | Implement read-only filesystem | 25 min |\n| Day 30 | Full security hardening of production service | 45 min |\n\n---\n\n## 📚 Series Navigation\n\n| Previous | Current | Next |\n|----------|---------|------|\n| [Part 11: Logging & Observability](docker-part-11-logging) | **Part 12: Container Security** | [Part 13: Debugging Containers](docker-part-13-debugging) |\n","2026-06-10T07:19:25.503885Z","2026-06-10T07:19:25.504388Z",14,12,{"_33":57,"_99":59,"_100":101,"_102":103,"_104":-5,"_105":-5,"_106":-5,"_107":-5,"_108":-5},{"_33":65,"_74":61,"_37":96,"_77":97,"_98":-5},[122,123,124],{"_33":57,"_74":75,"_37":76,"_77":78,"_79":24},{"_33":87,"_74":88,"_37":89},{"_33":125,"_74":126,"_37":127},{"_33":84,"_74":85,"_37":86},239,"Hardening","hardening"]