[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6},"actionData","errors","routes/blog-tag",{"_7":8,"_9":10},"tag",{"_31":86,"_71":87,"_35":88},"articles",{"_11":12,"_13":14,"_15":16,"_17":16,"_18":19,"_20":21,"_22":23,"_24":23,"_25":16,"_26":27},"content",[30],"pageable",{"_28":21,"_29":19},"totalElements",1,"totalPages","size",10,"number",0,"first",true,"last","numberOfElements","empty",false,"page_number","page_size",{"_31":32,"_33":34,"_35":36,"_37":38,"_11":39,"_40":-5,"_41":23,"_42":27,"_43":44,"_45":46,"_47":48,"_49":50,"_51":52,"_53":52,"_54":55,"_56":57,"_58":59,"_60":52,"_61":55,"_62":55,"_63":64,"_65":66,"_67":68,"_69":70},"id",53,"title","Build Process Deep Dive","slug","docker-part-3-build-process","excerpt","Master BuildKit, build stages, and cache mechanics","# Docker Compendium Part 3: Build Process Deep Dive\n\nYou run `docker build` and wait. Sometimes it's fast, sometimes painfully slow. Sometimes cache works, sometimes it doesn't. This article explains exactly what happens during a build - from parsing your Dockerfile to producing the final image - so you can optimize builds with precision.\n\n## 📋 At a Glance\n\n| Aspect | Details |\n|--------|---------|\n| Topic | Build process, BuildKit, build context, cache mechanics |\n| Complexity | Advanced |\n| Prerequisites | Part 2 (Image Anatomy), Dockerfile basics |\n| Key Insight | Understanding build context and cache is 80% of optimization |\n| Time to Master | 3-4 hours |\n\n---\n\n## 🎯 What You'll Learn\n\n- **Build context** - what gets sent to Docker daemon and why it matters\n- **BuildKit internals** - the modern build engine and its advantages\n- **Cache mechanics** - exactly when cache hits or misses\n- **Build stages** - how multi-stage builds work internally\n- **Parallel building** - how BuildKit parallelizes where possible\n\n---\n\n## 🔥 Production Story: The 30-Minute Build\n\nA team's CI builds took 30 minutes. They blamed slow runners, added more resources, saw no improvement. The real problem was simpler.\n\n**Investigation:**\n\n```bash\nSending build context to Docker daemon 2.4GB\n```\n\nThat line appeared at the start of every build. They were sending 2.4GB of files to Docker daemon before build even started.\n\n**Root cause:**\n- No `.dockerignore` file\n- Build directory included:\n - `node_modules/` (800MB)\n - `.git/` (1.2GB of history)\n - Test fixtures (400MB of images/videos)\n- Context transfer: 2-3 minutes\n- Context parsing: 1-2 minutes\n- Actual build: 5 minutes\n\n**The fix:**\n\n```bash\n# .dockerignore\nnode_modules\n.git\n*.log\ntest/fixtures\ncoverage\ndist\n```\n\nBuild time: 30 minutes → 6 minutes. The build itself was always 5 minutes - they were wasting 25 minutes on context.\n\n---\n\n## 🧠 Mental Model: Build Pipeline\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│ docker build -t myapp . │\n└──────────────────────────┬──────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ STAGE 1: CONTEXT PREPARATION │\n│ │\n│ Current directory (.) │\n│ ├── Dockerfile ✓ Always included │\n│ ├── src/ ✓ Included unless in .dockerignore │\n│ ├── node_modules/ ✗ Excluded by .dockerignore │\n│ ├── .git/ ✗ Excluded by .dockerignore │\n│ └── test/ ✓ Included (maybe shouldn't be?) │\n│ │\n│ → Creates tarball │\n│ → Sends to Docker daemon (or BuildKit) │\n└──────────────────────────┬──────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ STAGE 2: DOCKERFILE PARSING │\n│ │\n│ Parse Dockerfile → Abstract Syntax Tree │\n│ Validate syntax, resolve ARGs │\n│ Plan execution (what depends on what?) │\n└──────────────────────────┬──────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ STAGE 3: LAYER EXECUTION │\n│ │\n│ For each instruction: │\n│ ┌────────────────────────────────────────────────────────────┐ │\n│ │ 1. Check cache (can we reuse existing layer?) │ │\n│ │ └─ Cache key = parent layer + instruction + context │ │\n│ │ │ │\n│ │ 2. If cache miss: │ │\n│ │ ├─ Create temporary container from parent layer │ │\n│ │ ├─ Execute instruction (RUN, COPY, etc.) │ │\n│ │ ├─ Commit container as new layer │ │\n│ │ └─ Store layer with cache key │ │\n│ │ │ │\n│ │ 3. If cache hit: │ │\n│ │ └─ Use existing layer, skip execution │ │\n│ └────────────────────────────────────────────────────────────┘ │\n└──────────────────────────┬──────────────────────────────────────┘\n │\n ▼\n┌─────────────────────────────────────────────────────────────────┐\n│ STAGE 4: IMAGE ASSEMBLY │\n│ │\n│ Stack all layers │\n│ Create image config (ENV, CMD, etc.) │\n│ Generate manifest │\n│ Tag image (if -t provided) │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 🔬 Deep Dive\n\n### Build Context: The Foundation\n\nBuild context is everything Docker has access to during build. It's critical to understand because:\n1. Large context = slow builds (network transfer)\n2. Wrong context = security risks (secrets included)\n3. Missing context = failed COPY commands\n\n```bash\n# What IS the build context?\ndocker build -t myapp .\n# ↑ This dot is the context\n\n# Context is EVERYTHING in this directory (recursively)\n# Except what's in .dockerignore\n```\n\n**Visualizing context transfer:**\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│ YOUR MACHINE │\n│ │\n│ project/ │\n│ ├── Dockerfile │\n│ ├── src/ ┐ │\n│ │ ├── app.py │ │\n│ │ └── utils.py │ │\n│ ├── requirements.txt │ Included in context │\n│ ├── README.md │ (unless .dockerignore) │\n│ ├── tests/ │ │\n│ │ └── test_app.py ┘ │\n│ ├── .git/ ┐ │\n│ ├── node_modules/ │ Should be in .dockerignore! │\n│ ├── __pycache__/ │ │\n│ └── .env ┘ │\n│ │\n│ docker build creates tarball ─────────────────────────────────┐│\n└──────────────────────────────┬────────────────────────────────┬┘│\n │ │ │\n ▼ ▼ │\n┌─────────────────────────────────────────────────────────────────┐\n│ DOCKER DAEMON │\n│ │\n│ Receives tarball │\n│ Extracts to temporary directory │\n│ COPY commands reference files from here │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n**The perfect .dockerignore:**\n\n```bash\n# Version control\n.git\n.gitignore\n.gitattributes\n\n# Dependencies (will be installed in container)\nnode_modules\nvendor\n__pycache__\n*.pyc\nvenv\n.venv\n\n# Build outputs\ndist\nbuild\n*.egg-info\n\n# IDE and editors\n.idea\n.vscode\n*.swp\n*.swo\n\n# Test and coverage\ncoverage\n.coverage\nhtmlcov\n.pytest_cache\n.tox\n\n# Documentation\ndocs\n*.md\n!README.md # Keep README if needed\n\n# Local configuration\n.env\n.env.local\n*.local\n\n# Docker\nDockerfile*\ndocker-compose*\n.docker\n\n# Secrets (NEVER in context!)\n*.pem\n*.key\ncredentials*\nsecrets*\n```\n\n**Debugging context issues:**\n\n```bash\n# See what's being sent\ndocker build --no-cache -t test . 2>&1 | head -1\n# Sending build context to Docker daemon 45.2MB\n\n# Create a test archive to see exactly what's included\ntar -cvf context-test.tar --exclude-from=.dockerignore .\nls -lh context-test.tar\ntar -tvf context-test.tar | head -50\n```\n\n### BuildKit: The Modern Build Engine\n\nBuildKit is Docker's next-generation build engine (default since Docker 23.0).\n\n**Enable BuildKit (if not default):**\n\n```bash\n# Environment variable\nexport DOCKER_BUILDKIT=1\ndocker build .\n\n# Or in daemon.json\n{\n \"features\": {\n \"buildkit\": true\n }\n}\n```\n\n**BuildKit vs Legacy Builder:**\n\n| Feature | Legacy Builder | BuildKit |\n|---------|---------------|----------|\n| Parallel builds | No | Yes |\n| Better caching | Basic | Advanced mount caches |\n| Build secrets | No | `--mount=type=secret` |\n| SSH forwarding | No | `--mount=type=ssh` |\n| Output formats | Image only | Image, tar, OCI, local |\n| Progress output | Sequential | Parallel with dependencies |\n\n**BuildKit parallel execution:**\n\n```dockerfile\n# Dockerfile with independent stages\nFROM node:20 AS frontend\nWORKDIR /frontend\nCOPY frontend/package*.json ./\nRUN npm install\nCOPY frontend/ .\nRUN npm run build\n\nFROM python:3.11 AS backend\nWORKDIR /backend\nCOPY backend/requirements.txt .\nRUN pip install -r requirements.txt\nCOPY backend/ .\nRUN python -m pytest\n\nFROM nginx:alpine AS final\nCOPY --from=frontend /frontend/dist /usr/share/nginx/html\nCOPY --from=backend /backend/static /usr/share/nginx/html/api\n```\n\n**BuildKit execution:**\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│ BUILDKIT PARALLEL EXECUTION │\n│ │\n│ Time ──────────────────────────────────────────────────────► │\n│ │\n│ frontend: [npm install]────[build]─────────┐ │\n│ │ │\n│ backend: [pip install]────[pytest]────────┼──┐ │\n│ │ │ │\n│ final: └──┴─[copy both] │\n│ │\n│ Total time = max(frontend, backend) + final │\n│ Not: frontend + backend + final │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n### Cache Mechanics: Deep Dive\n\nCache is the key to fast builds. Let's understand exactly how it works.\n\n**Cache key components:**\n\n| Instruction | Cache Key Includes |\n|-------------|-------------------|\n| `FROM` | Base image digest |\n| `RUN` | Parent layer + command string (exact match) |\n| `COPY` | Parent layer + source file contents (hash) |\n| `ADD` | Parent layer + source contents + URL content |\n| `ARG` | Parent layer + arg name (value doesn't invalidate) |\n| `ENV` | Parent layer + key=value |\n\n**Cache invalidation cascade:**\n\n```dockerfile\nFROM node:20 # Layer 1\nWORKDIR /app # Layer 2 (depends on 1)\nCOPY package*.json ./ # Layer 3 (depends on 2)\nRUN npm install # Layer 4 (depends on 3)\nCOPY . . # Layer 5 (depends on 4)\nRUN npm run build # Layer 6 (depends on 5)\n```\n\n```\nScenario: package.json changed\n\nLayer 1: HIT (base image unchanged)\nLayer 2: HIT (WORKDIR unchanged, parent hit)\nLayer 3: MISS (file content changed!)\nLayer 4: MISS (parent missed) ← npm install runs again\nLayer 5: MISS (parent missed)\nLayer 6: MISS (parent missed)\n\nScenario: Only src/app.js changed\n\nLayer 1: HIT\nLayer 2: HIT\nLayer 3: HIT (package.json unchanged)\nLayer 4: HIT (parent hit, command unchanged)\nLayer 5: MISS (file content in \".\" changed)\nLayer 6: MISS (parent missed)\n\nnpm install cached! Only final stages rebuild.\n```\n\n**Cache busting techniques:**\n\n```dockerfile\n# Force rebuild from specific point\nARG CACHE_BUST\nRUN echo \"Cache bust: $CACHE_BUST\" && npm install\n# docker build --build-arg CACHE_BUST=$(date +%s) .\n\n# Conditional cache bust (e.g., daily)\nARG BUILD_DATE\nRUN echo \"Build date: $BUILD_DATE\" && apt-get update\n# docker build --build-arg BUILD_DATE=$(date +%Y-%m-%d) .\n```\n\n**BuildKit cache mounts (game changer):**\n\n```dockerfile\n# Traditional: cache inside layer, lost on rebuild\nRUN npm install\n\n# BuildKit: persistent cache mount\nRUN --mount=type=cache,target=/root/.npm \\\n npm install\n\n# Python example\nRUN --mount=type=cache,target=/root/.cache/pip \\\n pip install -r requirements.txt\n\n# Go example\nRUN --mount=type=cache,target=/go/pkg/mod \\\n --mount=type=cache,target=/root/.cache/go-build \\\n go build -o /app .\n\n# apt-get example\nRUN --mount=type=cache,target=/var/cache/apt \\\n --mount=type=cache,target=/var/lib/apt \\\n apt-get update && apt-get install -y build-essential\n```\n\nCache mounts persist across builds but don't become part of the image layer.\n\n### Build Arguments and Variables\n\nUnderstanding ARG vs ENV:\n\n```dockerfile\n# ARG: Build-time only, can be set with --build-arg\nARG NODE_VERSION=20\nFROM node:${NODE_VERSION}\n\n# ARG with default, overridable\nARG NPM_TOKEN\nRUN --mount=type=secret,id=npm,target=/root/.npmrc \\\n npm install\n\n# ENV: Runtime, persists in image\nENV NODE_ENV=production\n\n# ARG that becomes ENV\nARG APP_VERSION\nENV APP_VERSION=${APP_VERSION}\n```\n\n**ARG scope rules:**\n\n```dockerfile\nARG BASE_IMAGE=node:20 # Available before FROM\n\nFROM ${BASE_IMAGE} # ARG used here\n\nARG BASE_IMAGE # Must redeclare after FROM!\nRUN echo ${BASE_IMAGE} # Now available again\n\n# ARGs reset after each FROM in multi-stage\nFROM alpine\nARG BASE_IMAGE # Must redeclare again\n```\n\n### Build Secrets (BuildKit)\n\nNever put secrets in build args (they're visible in history):\n\n```bash\n# ❌ BAD: Secret in build arg\ndocker build --build-arg NPM_TOKEN=secret123 .\ndocker history myimage # NPM_TOKEN visible!\n\n# ✅ GOOD: Secret mount\ndocker build --secret id=npm,src=.npmrc .\n```\n\n```dockerfile\n# Using secret mount\nRUN --mount=type=secret,id=npm,target=/root/.npmrc \\\n npm install\n# Secret available during build, not in final image\n\n# SSH agent forwarding for private repos\nRUN --mount=type=ssh \\\n git clone git@github.com:private/repo.git\n\n# docker build --ssh default .\n```\n\n### Multi-Stage Build Internals\n\nHow stages actually work:\n\n```dockerfile\n# Stage 1: builder\nFROM golang:1.21 AS builder\nWORKDIR /src\nCOPY go.* ./\nRUN go mod download\nCOPY . .\nRUN CGO_ENABLED=0 go build -o /app\n\n# Stage 2: final\nFROM scratch\nCOPY --from=builder /app /app\nENTRYPOINT [\"/app\"]\n```\n\n**What happens internally:**\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│ MULTI-STAGE BUILD │\n│ │\n│ Stage: builder │\n│ ┌────────────────────────────────────────────────────────────┐ │\n│ │ FROM golang:1.21 (800MB) │ │\n│ │ WORKDIR /src │ │\n│ │ COPY go.* ./ │ │\n│ │ RUN go mod download │ │\n│ │ COPY . . │ │\n│ │ RUN go build -o /app │ │\n│ │ │ │\n│ │ Final: ~850MB with all Go toolchain │ │\n│ │ But only /app binary needed! │ │\n│ └────────────────────────────────────────────────────────────┘ │\n│ │ │\n│ │ COPY --from=builder /app │\n│ ▼ │\n│ Stage: final │\n│ ┌────────────────────────────────────────────────────────────┐ │\n│ │ FROM scratch (0MB) │ │\n│ │ COPY --from=builder /app /app │ │\n│ │ │ │\n│ │ Final image: ~10MB (just the binary!) │ │\n│ └────────────────────────────────────────────────────────────┘ │\n│ │\n│ builder stage layers: NOT in final image │\n│ They're discarded after COPY extracts what's needed │\n└─────────────────────────────────────────────────────────────────┘\n```\n\n**Targeting specific stages:**\n\n```bash\n# Build only up to builder stage (for debugging)\ndocker build --target builder -t myapp:builder .\n\n# Build final (default - last stage)\ndocker build -t myapp:latest .\n\n# Build specific stage for testing\ndocker build --target test -t myapp:test .\n```\n\n**Copying from external images:**\n\n```dockerfile\n# Copy from any image, not just build stages\nCOPY --from=nginx:alpine /etc/nginx/nginx.conf /etc/nginx/\nCOPY --from=busybox:latest /bin/busybox /bin/\n```\n\n### Build Output Formats (BuildKit)\n\n```bash\n# Default: load into Docker\ndocker build -t myapp .\n\n# Export to tar file\ndocker build -o type=tar,dest=image.tar .\n\n# Export filesystem only (no image metadata)\ndocker build -o type=local,dest=./output .\n\n# OCI image format\ndocker build -o type=oci,dest=image-oci.tar .\n\n# Push directly to registry\ndocker build -t registry.example.com/myapp:1.0 --push .\n```\n\n### Build Performance Analysis\n\n```bash\n# See detailed timing\nDOCKER_BUILDKIT=1 docker build --progress=plain .\n\n# Example output:\n#1 [internal] load build definition from Dockerfile\n#1 DONE 0.0s\n\n#2 [internal] load .dockerignore\n#2 DONE 0.0s\n\n#3 [internal] load metadata for docker.io/library/node:20\n#3 DONE 1.2s\n\n#4 [1/6] FROM docker.io/library/node:20@sha256:abc...\n#4 CACHED\n\n#5 [2/6] WORKDIR /app\n#5 CACHED\n\n#6 [3/6] COPY package*.json ./\n#6 DONE 0.1s\n\n#7 [4/6] RUN npm install\n#7 2.543 npm WARN deprecated...\n#7 DONE 45.2s ← This is slow!\n\n#8 [5/6] COPY . .\n#8 DONE 0.3s\n\n#9 [6/6] RUN npm run build\n#9 DONE 12.1s\n```\n\n---\n\n## ⚠️ Common Mistakes\n\n### Mistake 1: Ignoring Build Context Size\n\n```bash\n# ❌ No .dockerignore, sending everything\n$ docker build .\nSending build context to Docker daemon 2.1GB\n\n# ✅ Proper .dockerignore\n$ docker build .\nSending build context to Docker daemon 15.2MB\n```\n\n**Quick check:**\n\n```bash\n# See context size without building\ntar -cf - --exclude-from=.dockerignore . | wc -c | numfmt --to=iec\n```\n\n### Mistake 2: Cache-Busting ORDER Instructions\n\n```dockerfile\n# ❌ COPY all files before install - any change busts cache\nCOPY . .\nRUN npm install\n\n# ✅ COPY dependency files first\nCOPY package*.json ./\nRUN npm install\nCOPY . .\n```\n\n### Mistake 3: Not Using BuildKit Cache Mounts\n\n```dockerfile\n# ❌ Downloads dependencies every time parent changes\nRUN pip install -r requirements.txt\n\n# ✅ Persistent cache across builds\nRUN --mount=type=cache,target=/root/.cache/pip \\\n pip install -r requirements.txt\n```\n\n---\n\n## 🐛 Debug This: The Phantom Cache Miss\n\nA developer reports: \"My build should use cache but it keeps rebuilding everything. The Dockerfile hasn't changed!\"\n\n```dockerfile\nFROM node:20\nWORKDIR /app\nCOPY package.json .\nRUN npm install\nCOPY . .\nCMD [\"node\", \"index.js\"]\n```\n\n```bash\n$ docker build -t myapp .\n# ... all steps show CACHED ...\n\n# Make a tiny change to index.js, rebuild\n$ docker build -t myapp .\n# Step 3/6: COPY package.json .\n# ---> Running in abc123\n# Step 4/6: RUN npm install\n# ---> Running in def456\n# ... everything rebuilds from step 3!\n\n# But package.json didn't change!\n```\n\n**Why does COPY package.json miss cache when only index.js changed?**\n\n---\n\n**✅ Solution:**\n\nThere are several possible causes:\n\n**Cause 1: File permissions/timestamps changed**\n\n```bash\n# Git might change file permissions\ngit checkout . # Can modify permissions\n\n# Check with docker build --no-cache and compare\ndocker build --no-cache -t test1 .\n# Make no changes\ndocker build -t test2 . # Should all cache\n```\n\n**Cause 2: Different build context**\n\n```bash\n# Building from different directories\ncd /path/to/project && docker build .\ncd /other/path && docker build -f /path/to/project/Dockerfile .\n# Context is different!\n```\n\n**Cause 3: .dockerignore not excluding generated files**\n\n```bash\n# If node_modules is in context (shouldn't be)\n# and npm install created/modified it\n# COPY . . would see changes\n```\n\n**Cause 4: Build args changing**\n\n```dockerfile\nARG NODE_ENV=development # If this changes, cache busts after\n\nFROM node:20\nARG NODE_ENV\nENV NODE_ENV=${NODE_ENV}\n# Everything after ARG usage may rebuid\n```\n\n**Cause 5: Multi-stage target confusion**\n\n```bash\n# Different targets can have different caches\ndocker build --target dev .\ndocker build --target prod . # Might not share cache\n```\n\n**Debug steps:**\n\n```bash\n# 1. Check file hashes\nsha256sum package.json\n\n# 2. Build with verbose output\nDOCKER_BUILDKIT=1 docker build --progress=plain . 2>&1 | tee build.log\n\n# 3. Compare cache keys\ndocker history myapp --no-trunc\n\n# 4. Force fresh build to establish baseline\ndocker build --no-cache .\n```\n\n---\n\n## 💻 Exercises\n\n### Exercise 1: Measure Build Context Impact\n⭐ Difficulty: Easy | ⏱️ Time: 15 minutes\n\n```bash\n# 1. Create test project\nmkdir build-context-test && cd build-context-test\n\n# Create a large file\ndd if=/dev/urandom of=large-file.bin bs=1M count=100\n\n# Create Dockerfile\ncat > Dockerfile << 'EOF'\nFROM alpine:3.18\nCOPY small-file.txt /app/\nCMD [\"cat\", \"/app/small-file.txt\"]\nEOF\n\necho \"Hello\" > small-file.txt\n\n# 2. Build without .dockerignore - time it\ntime docker build -t test1 .\n\n# 3. Add .dockerignore\necho \"large-file.bin\" > .dockerignore\n\n# 4. Build with .dockerignore - time it\ntime docker build -t test2 .\n\n# Questions:\n# - How much faster was the second build?\n# - What was the context size difference?\n```\n\n---\n\n### Exercise 2: Optimize a Slow Dockerfile\n⭐⭐ Difficulty: Medium | ⏱️ Time: 25 minutes\n\n```dockerfile\n# slow.Dockerfile - Fix this!\nFROM node:20\n\nWORKDIR /app\n\n# Everything copies first\nCOPY . .\n\n# Then install\nRUN npm install\n\n# Then build\nRUN npm run build\n\nCMD [\"npm\", \"start\"]\n```\n\n```bash\n# 1. Create test project\nmkdir optimize-test && cd optimize-test\nnpm init -y\nnpm install express\necho 'console.log(\"hello\")' > index.js\n\n# 2. Build and time with \"slow\" Dockerfile\ndocker build -f slow.Dockerfile -t slow .\n# Make a change to index.js\ndocker build -f slow.Dockerfile -t slow . # Time this\n\n# 3. Create optimized.Dockerfile with proper layer ordering\n\n# 4. Compare rebuild times after code change\n```\n\n---\n\n### Exercise 3: Use BuildKit Cache Mounts\n⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes\n\n```bash\n# Create Python project\nmkdir cache-mount-test && cd cache-mount-test\n\ncat > requirements.txt << 'EOF'\nflask==3.0.0\nrequests==2.31.0\nnumpy==1.26.0\npandas==2.1.0\nEOF\n\n# Traditional Dockerfile\ncat > Dockerfile.traditional << 'EOF'\nFROM python:3.11-slim\nWORKDIR /app\nCOPY requirements.txt .\nRUN pip install -r requirements.txt\nCOPY . .\nCMD [\"python\", \"app.py\"]\nEOF\n\n# BuildKit cache mount Dockerfile\ncat > Dockerfile.cached << 'EOF'\nFROM python:3.11-slim\nWORKDIR /app\nCOPY requirements.txt .\nRUN --mount=type=cache,target=/root/.cache/pip \\\n pip install -r requirements.txt\nCOPY . .\nCMD [\"python\", \"app.py\"]\nEOF\n\n# 1. Build both versions twice\n# 2. Add a new dependency to requirements.txt\n# 3. Rebuild both - compare times\n# 4. Why is the cached version faster on second rebuild?\n```\n\n---\n\n### Exercise 4: Multi-Stage Build Analysis\n⭐⭐⭐ Difficulty: Hard | ⏱️ Time: 30 minutes\n\n```bash\n# Create Go project\nmkdir multistage-test && cd multistage-test\n\ncat > main.go << 'EOF'\npackage main\n\nimport (\n \"fmt\"\n \"net/http\"\n)\n\nfunc main() {\n http.HandleFunc(\"/\", func(w http.ResponseWriter, r *http.Request) {\n fmt.Fprintf(w, \"Hello from Go!\")\n })\n http.ListenAndServe(\":8080\", nil)\n}\nEOF\n\ncat > go.mod << 'EOF'\nmodule myapp\ngo 1.21\nEOF\n\n# Single-stage Dockerfile\ncat > Dockerfile.single << 'EOF'\nFROM golang:1.21\nWORKDIR /app\nCOPY . .\nRUN go build -o server .\nCMD [\"./server\"]\nEOF\n\n# Multi-stage Dockerfile\ncat > Dockerfile.multi << 'EOF'\nFROM golang:1.21 AS builder\nWORKDIR /app\nCOPY . .\nRUN CGO_ENABLED=0 GOOS=linux go build -o server .\n\nFROM scratch\nCOPY --from=builder /app/server /server\nEXPOSE 8080\nENTRYPOINT [\"/server\"]\nEOF\n\n# 1. Build both\ndocker build -f Dockerfile.single -t go-single .\ndocker build -f Dockerfile.multi -t go-multi .\n\n# 2. Compare sizes\ndocker images | grep go-\n\n# 3. Analyze layers\ndocker history go-single\ndocker history go-multi\n\n# 4. Try to run go-multi - what happens? Why?\n# (Hint: scratch has no shell, no libc)\n\n# 5. Can you use `docker exec` on go-multi container?\n```\n\n---\n\n### Exercise 5: Build Secrets Deep Dive\n⭐⭐⭐⭐ Difficulty: Expert | ⏱️ Time: 30 minutes\n\n```bash\n# Setup\nmkdir secrets-test && cd secrets-test\n\n# Create a \"secret\" file\necho \"super-secret-token-123\" > .npmrc\n\n# Bad way (DON'T DO THIS IN REAL PROJECTS)\ncat > Dockerfile.bad << 'EOF'\nFROM node:20-alpine\nARG NPM_TOKEN\nRUN echo \"//registry.npmjs.org/:_authToken=${NPM_TOKEN}\" > ~/.npmrc\nRUN npm install private-package || echo \"Would install if token valid\"\nRUN rm ~/.npmrc\nCMD [\"node\"]\nEOF\n\n# Good way with BuildKit secrets\ncat > Dockerfile.good << 'EOF'\n# syntax=docker/dockerfile:1\nFROM node:20-alpine\nWORKDIR /app\nCOPY package*.json ./\nRUN --mount=type=secret,id=npmrc,target=/root/.npmrc \\\n npm install || echo \"Would install if token valid\"\nCMD [\"node\"]\nEOF\n\n# 1. Build \"bad\" way\ndocker build -f Dockerfile.bad \\\n --build-arg NPM_TOKEN=secret123 \\\n -t secrets-bad .\n\n# 2. Check if secret is visible\ndocker history secrets-bad --no-trunc\n# Can you see the token?\n\n# 3. Build \"good\" way\ndocker build -f Dockerfile.good \\\n --secret id=npmrc,src=.npmrc \\\n -t secrets-good .\n\n# 4. Check if secret is visible\ndocker history secrets-good --no-trunc\n# Is the token visible?\n\n# 5. Can you extract the secret from either image?\ndocker save secrets-bad | tar -xf - -O\ndocker save secrets-good | tar -xf - -O\n```\n\n---\n\n## 🎤 Senior-Level Interview Questions\n\n### Q1: Explain how Docker build cache works and how to optimize for it.\n\n**Strong Answer:**\n\n\"Docker's build cache stores layers keyed by parent layer hash plus instruction details. For RUN, the key is the command string. For COPY/ADD, it's the file content hashes.\n\nCache invalidation cascades: if layer N misses, all subsequent layers must rebuild. This is why instruction ordering is crucial:\n\n1. **Base image first** - rarely changes\n2. **System dependencies** - apt-get, apk add\n3. **Application dependencies** - package.json/requirements.txt, then install\n4. **Application code** - most frequently changing, last\n\nFor example:\n```dockerfile\nCOPY package.json .\nRUN npm install\nCOPY . .\n```\n\nIf only source code changes, npm install stays cached.\n\nWith BuildKit, I also use cache mounts:\n```dockerfile\nRUN --mount=type=cache,target=/root/.npm npm install\n```\n\nThis persists the npm cache across builds without including it in layers. Even if the layer rebuilds, cached packages remain.\n\nOther optimizations: proper .dockerignore to avoid context changes, avoiding build args that change frequently, and pinning base image versions.\"\n\n### Q2: What is build context and why does it matter?\n\n**Strong Answer:**\n\n\"Build context is the set of files Docker can access during build. When you run `docker build .`, the entire directory (minus .dockerignore entries) is tarred and sent to the daemon.\n\nWhy it matters:\n\n1. **Performance**: Large context = slow builds. I've seen 2GB contexts that took 5 minutes just to transfer. A proper .dockerignore got it to 15MB.\n\n2. **Security**: Context might include secrets, .env files, or credentials. Without .dockerignore, these get sent to daemon and potentially into layers via COPY.\n\n3. **Reproducibility**: Different contexts can produce different images even with the same Dockerfile.\n\nBest practices:\n- Always have a .dockerignore\n- Exclude: .git, node_modules, build outputs, test files, IDE configs\n- Use specific COPY statements, not `COPY . .` when possible\n- Consider using `docker build -f Dockerfile path/to/context` to control context location\n\nDebug with: `tar -cvf - --exclude-from=.dockerignore . | wc -c`\"\n\n### Q3: How does BuildKit differ from the legacy builder?\n\n**Strong Answer:**\n\n\"BuildKit is Docker's next-gen builder with major improvements:\n\n**Parallel execution**: BuildKit builds independent stages concurrently. If you have frontend and backend stages, they build in parallel, merging at the final stage.\n\n**Better caching**:\n- Cache mounts persist across builds: `--mount=type=cache`\n- External cache import/export for CI: `--cache-from`, `--cache-to`\n- More intelligent cache invalidation\n\n**Secrets handling**:\n- `--mount=type=secret` exposes secrets during build only\n- Never written to layers\n- `--mount=type=ssh` for SSH agent forwarding\n\n**Frontend extensibility**: BuildKit uses frontends parsed from `# syntax=` directive. Can support non-Dockerfile syntaxes.\n\n**Output flexibility**: Can output to local directory, tar, OCI format, or push directly to registry.\n\n**Progress display**: Shows parallel operations and dependencies clearly.\n\nPractical impact: CI builds that took 10 minutes can drop to 3 minutes with proper parallelization and cache mounts.\"\n\n### Q4: How would you debug a build that's not caching as expected?\n\n**Strong Answer:**\n\n\"My debugging process:\n\n1. **Enable verbose output**:\n ```bash\n DOCKER_BUILDKIT=1 docker build --progress=plain . 2>&1 | tee build.log\n ```\n This shows exactly which steps are cached and why.\n\n2. **Check for invisible changes**:\n - File permissions (git checkout can change these)\n - Timestamps (some builds are timestamp-sensitive)\n - Generated files not in .dockerignore\n\n3. **Verify context**:\n ```bash\n tar -cvf test.tar --exclude-from=.dockerignore .\n ```\n Make sure only expected files are included.\n\n4. **Check build args**: ARGs that change between builds invalidate cache from that point.\n\n5. **Compare layer hashes**:\n ```bash\n docker history myimage --no-trunc\n ```\n\n6. **Test incrementally**: Build with `--target` to isolate which stage has the problem.\n\nCommon culprits:\n- Missing .dockerignore\n- COPY before dependency install\n- Build args with timestamps/hashes\n- Different Docker versions having different cache behavior\"\n\n### Q5: Explain multi-stage builds and when you'd use them.\n\n**Strong Answer:**\n\n\"Multi-stage builds let you use multiple FROM statements, with each creating an isolated stage. Only the final stage becomes the output image, but you can COPY artifacts from previous stages.\n\nKey use cases:\n\n**Compiled languages**: Build with full toolchain, copy only binary to minimal runtime:\n```dockerfile\nFROM golang AS builder\nRUN go build -o /app\n\nFROM scratch\nCOPY --from=builder /app /app\n```\nReduces 800MB Go image to 10MB binary.\n\n**Separation of concerns**:\n```dockerfile\nFROM node AS deps\nRUN npm install\n\nFROM node AS test\nCOPY --from=deps /node_modules ./node_modules\nRUN npm test\n\nFROM node AS prod\nCOPY --from=deps /node_modules ./node_modules\n```\n\n**Build-time secrets**: Put secret-requiring steps in non-final stage.\n\n**Conditional builds**: Use `--target` to build different outputs from same Dockerfile.\n\nWhat doesn't transfer between stages:\n- Environment variables\n- Build arguments (must redeclare)\n- Layer cache state\n- Everything except explicitly COPYed files\n\nStages can be named (AS name) or numbered (0, 1, 2). Named is clearer. You can also COPY from external images: `COPY --from=nginx:alpine /etc/nginx/...`\"\n\n---\n\n## 📝 Summary & Key Takeaways\n\n### Core Concepts\n\n| Concept | Key Point |\n|---------|-----------|\n| Build context | Everything Docker can access - use .dockerignore |\n| Cache mechanics | Parent hash + instruction = cache key, cascade on miss |\n| BuildKit | Parallel builds, cache mounts, secrets, better output |\n| Multi-stage | Separate build from runtime, copy only what's needed |\n| Layer ordering | Put rarely-changing before frequently-changing |\n\n### The Build Equation\n\n```\nBuild Time = Context Transfer + Cache Misses × Build Time + Final Assembly\n\nOptimize by:\n1. Minimize context (.dockerignore)\n2. Maximize cache hits (layer ordering)\n3. Parallelize independent stages (multi-stage)\n4. Use cache mounts (BuildKit)\n```\n\n### What You Can Do Now\n\n1. **Audit your .dockerignore**: Is everything excluded that should be?\n2. **Reorder your Dockerfile**: Dependencies before code\n3. **Add cache mounts**: Use BuildKit --mount=type=cache\n4. **Profile builds**: Use --progress=plain to find slow steps\n\n---\n\n## 📋 Quick Reference\n\n### Build Commands\n\n```bash\n# Standard build\ndocker build -t myapp .\n\n# With BuildKit (explicit)\nDOCKER_BUILDKIT=1 docker build -t myapp .\n\n# Verbose progress\ndocker build --progress=plain -t myapp .\n\n# No cache\ndocker build --no-cache -t myapp .\n\n# Specific target stage\ndocker build --target builder -t myapp:builder .\n\n# With build args\ndocker build --build-arg VERSION=1.0 -t myapp .\n\n# With secrets (BuildKit)\ndocker build --secret id=mysecret,src=./secret.txt -t myapp .\n\n# Multiple platforms\ndocker buildx build --platform linux/amd64,linux/arm64 -t myapp .\n```\n\n### .dockerignore Essentials\n\n```bash\n# Version control\n.git\n.gitignore\n\n# Dependencies\nnode_modules\nvendor\n__pycache__\n\n# Build artifacts\ndist\nbuild\n*.egg-info\n\n# IDE\n.idea\n.vscode\n\n# Testing\ncoverage\n.pytest_cache\n\n# Secrets (CRITICAL!)\n.env*\n*.pem\n*.key\n```\n\n### BuildKit Features\n\n```dockerfile\n# Cache mount\nRUN --mount=type=cache,target=/root/.cache/pip pip install -r req.txt\n\n# Secret mount\nRUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm install\n\n# SSH mount\nRUN --mount=type=ssh git clone git@github.com:private/repo.git\n\n# Bind mount (access files without COPY)\nRUN --mount=type=bind,source=.,target=/src ls /src\n```\n\n---\n\n## 📅 Review Schedule\n\n| Day | Task | Time |\n|-----|------|------|\n| Day 1 | Review cache invalidation rules | 10 min |\n| Day 3 | Add/improve .dockerignore in a project | 15 min |\n| Day 7 | Do Exercise 2 (optimize slow Dockerfile) | 25 min |\n| Day 14 | Implement BuildKit cache mounts | 20 min |\n| Day 30 | Profile a real project's build, optimize | 30 min |\n\n---\n\n## 📚 Series Navigation\n\n| Previous | Current | Next |\n|----------|---------|------|\n| [Part 2: Image Anatomy](docker-part-2-image-anatomy) | **Part 3: Build Process** | [Part 4: Networking Internals](docker-part-4-networking-internals) |\n\n**Docker Compendium Series:**\n- [Part 0: How to Use This Series](docker-part-0-how-to-use-this-series)\n- [Part 1: Container Internals](docker-part-1-container-internals)\n- [Part 2: Image Anatomy](docker-part-2-image-anatomy)\n- **Part 3: Build Process Deep Dive** ← You are here\n- [Part 4: Networking Internals](docker-part-4-networking-internals)\n","featuredImage","isPublished","isFeatured","subscriptionType","BASIC","publishedAt","2026-06-10T07:19:25.398702Z","createdAt","2026-06-10T07:19:25.399422Z","updatedAt","2026-06-16T21:26:56.086200Z","readingTimeMinutes",20,"estimatedReadingTime","authorId",3,"authorUsername","tomasz_main","categoryName","Devops","readingTime","seriesId","seriesOrder","author",{"_31":55,"_93":57,"_94":95,"_96":97,"_98":-5,"_99":-5,"_100":-5,"_101":-5,"_102":-5},"category",{"_31":89,"_71":59,"_35":90,"_74":91,"_92":-5},"tags",[77,78,79],"series",{"_31":55,"_71":72,"_35":73,"_74":75,"_76":23},"name","Docker Compendium: From Practitioner to Expert","docker-compendium","description","Master Docker internals, optimization patterns, and production best practices. For developers who use Docker daily but want deeper understanding of what's happening under the hood. Not a beginner series - we assume you can write Dockerfiles and use docker-compose. Learn what namespaces and cgroups really do, why your builds are slow, how to debug containers that crash, and how to run Docker in production like a pro.\n","isActive",{"_31":86,"_71":87,"_35":88},{"_31":83,"_71":84,"_35":85},{"_31":80,"_71":81,"_35":82},224,"Optimization","optimization",213,"Docker","docker",223,"Buildkit","buildkit",7,"devops","Category for Devops","colorHex","username","firstName","Tomasz","lastName","Jarosz","profilePicture","bio","website","githubUrl","linkedinUrl"]