[{"_1":2,"_3":-5,"_4":-5},"loaderData",{"_5":6},"actionData","errors","routes/blog-tag",{"_7":8,"_9":10},"tag",{"_31":81,"_72":82,"_35":83},"articles",{"_11":12,"_13":14,"_15":16,"_17":16,"_18":19,"_20":21,"_22":23,"_24":23,"_25":16,"_26":27},"content",[30],"pageable",{"_28":21,"_29":19},"totalElements",1,"totalPages","size",10,"number",0,"first",true,"last","numberOfElements","empty",false,"page_number","page_size",{"_31":32,"_33":34,"_35":36,"_37":38,"_11":39,"_40":-5,"_41":23,"_42":27,"_43":44,"_45":46,"_47":48,"_49":50,"_51":52,"_53":52,"_54":55,"_56":57,"_58":59,"_60":52,"_61":55,"_62":63,"_64":65,"_66":67,"_68":69,"_70":71},"id",65,"title","Service Dependencies & Orchestration","slug","docker-part-15-dependencies","excerpt","depends_on, healthchecks, restart policies","## 📋 At a Glance\n\n| Aspect | Details |\n|--------|---------|\n| **Difficulty** | 🟡 Intermediate |\n| **Prerequisites** | Part 14 (Compose Deep Dive) |\n| **Key Concepts** | depends_on, health checks, startup ordering, graceful shutdown |\n| **Time Investment** | 26 minutes read + 45 minutes practice |\n| **Payoff** | Zero-downtime deployments and reliable service orchestration |\n\n---\n\n## 🎯 What You'll Learn\n\nAfter this article, you'll be able to:\n\n1. **Implement proper startup ordering** with health-based dependencies\n2. **Design health checks** that accurately reflect service readiness\n3. **Handle graceful shutdown** to prevent request loss during deployments\n4. **Manage service initialization** including migrations and seeding\n5. **Debug dependency issues** and circular dependency problems\n\n---\n\n## 🔥 Production Story: The Race Condition Deployment\n\n**The Setup**: A microservices application with API, worker, and database. Every deployment, about 10% of initial requests failed with \"Connection refused.\"\n\n**The Configuration**:\n```yaml\nservices:\n api:\n depends_on:\n - db\n - redis\n # ...\n\n worker:\n depends_on:\n - db\n - redis\n # ...\n```\n\n**The Investigation**:\n```bash\n# API logs on startup\nConnecting to database...\nError: Connection refused to db:5432\nRetrying in 1s...\nConnecting to database...\nConnected!\nServer listening on port 8080\n```\n\n**Root Cause**: `depends_on` only waits for container start, not database readiness. PostgreSQL takes 2-5 seconds to initialize. API started connecting before Postgres was ready.\n\n**The Fix**:\n```yaml\nservices:\n api:\n depends_on:\n db:\n condition: service_healthy\n redis:\n condition: service_healthy\n\n db:\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U postgres\"]\n interval: 5s\n timeout: 5s\n retries: 5\n start_period: 10s\n\n redis:\n healthcheck:\n test: [\"CMD\", \"redis-cli\", \"ping\"]\n interval: 5s\n timeout: 3s\n retries: 5\n```\n\n**Result**: Zero failed connections during deployment. API waits for databases to be fully ready before starting.\n\n---\n\n## 🧠 Mental Model: Service Lifecycle\n\n```\n┌──────────────────────────────────────────────────────────────────┐\n│ SERVICE LIFECYCLE │\n├──────────────────────────────────────────────────────────────────┤\n│ │\n│ STARTUP PHASE │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ │ │\n│ │ [Created] ──→ [Starting] ──→ [Running] ──→ [Healthy] │ │\n│ │ │ │ │ │ │ │\n│ │ │ start_period healthcheck ready │ │\n│ │ │ (grace time) begins to serve │ │\n│ │ │ │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │\n│ DEPENDENCY ORDERING │\n│ ┌─────────────────────────────────────────────────────────┐ │\n│ │ │ │\n│ │ Infrastructure ──→ Core Services ──→ App │ │\n│ │ (db, redis) (migrations) (api) │ │\n│ │ │ │\n│ │ service_healthy service_completed service_healthy│ │\n│ │ _successfully │ │\n│ └─────────────────────────────────────────────────────────┘ │\n│ │\n│ SHUTDOWN PHASE │\n│ ┌────────────────────────────────────────────────────────┐ │\n│ │ │ │\n│ │ [Running] ──→ [SIGTERM] ──→ [Draining] ──→ [Stopped] │ │\n│ │ │ │ │ │ │\n│ │ stop_signal stop_grace SIGKILL │ │\n│ │ received _period if needed │ │\n│ │ │ │\n│ └────────────────────────────────────────────────────────┘ │\n│ │\n└──────────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 🔬 Deep Dive\n\n### 1. Understanding depends_on Conditions\n\n**The Three Conditions**:\n\n```yaml\nservices:\n app:\n depends_on:\n # Condition 1: Just wait for container to start\n service1:\n condition: service_started\n\n # Condition 2: Wait for health check to pass\n service2:\n condition: service_healthy\n\n # Condition 3: Wait for container to exit with code 0\n service3:\n condition: service_completed_successfully\n```\n\n**When to Use Each**:\n\n| Condition | Use Case | Example |\n|-----------|----------|---------|\n| `service_started` | Quick-start services, app handles retry | Feature flags service |\n| `service_healthy` | Must be ready before dependent starts | Database, cache, queue |\n| `service_completed_successfully` | One-time setup tasks | Migrations, seed data |\n\n**Complete Dependency Example**:\n\n```yaml\nservices:\n # Infrastructure layer\n db:\n image: postgres:16\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U postgres\"]\n interval: 5s\n timeout: 5s\n retries: 5\n start_period: 10s\n\n redis:\n image: redis:7\n healthcheck:\n test: [\"CMD\", \"redis-cli\", \"ping\"]\n interval: 5s\n timeout: 3s\n retries: 5\n\n # Setup layer - runs once\n migrate:\n image: myapp:latest\n command: npm run migrate\n depends_on:\n db:\n condition: service_healthy\n\n seed:\n image: myapp:latest\n command: npm run seed\n depends_on:\n migrate:\n condition: service_completed_successfully\n\n # Application layer\n api:\n image: myapp:latest\n depends_on:\n db:\n condition: service_healthy\n redis:\n condition: service_healthy\n seed:\n condition: service_completed_successfully\n healthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health\"]\n interval: 10s\n timeout: 5s\n retries: 3\n start_period: 30s\n\n # Worker layer\n worker:\n image: myapp:latest\n command: npm run worker\n depends_on:\n api:\n condition: service_healthy # Wait for API to be ready\n```\n\n---\n\n### 2. Designing Effective Health Checks\n\n**Health Check Components**:\n\n```yaml\nhealthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health\"]\n interval: 30s # How often to check\n timeout: 10s # Max time for check to complete\n retries: 3 # Failures before unhealthy\n start_period: 40s # Grace period for slow starts\n```\n\n**Database Health Checks**:\n\n```yaml\n# PostgreSQL\nhealthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U ${POSTGRES_USER:-postgres} -d ${POSTGRES_DB:-postgres}\"]\n interval: 5s\n timeout: 5s\n retries: 5\n\n# MySQL\nhealthcheck:\n test: [\"CMD\", \"mysqladmin\", \"ping\", \"-h\", \"localhost\", \"-u\", \"root\", \"-p${MYSQL_ROOT_PASSWORD}\"]\n interval: 5s\n timeout: 5s\n retries: 5\n\n# MongoDB\nhealthcheck:\n test: [\"CMD\", \"mongosh\", \"--eval\", \"db.adminCommand('ping')\"]\n interval: 10s\n timeout: 5s\n retries: 5\n```\n\n**Cache/Queue Health Checks**:\n\n```yaml\n# Redis\nhealthcheck:\n test: [\"CMD\", \"redis-cli\", \"ping\"]\n interval: 5s\n timeout: 3s\n retries: 5\n\n# Redis with AUTH\nhealthcheck:\n test: [\"CMD-SHELL\", \"redis-cli -a ${REDIS_PASSWORD} ping | grep PONG\"]\n interval: 5s\n timeout: 3s\n retries: 5\n\n# RabbitMQ\nhealthcheck:\n test: [\"CMD\", \"rabbitmq-diagnostics\", \"-q\", \"ping\"]\n interval: 10s\n timeout: 10s\n retries: 3\n\n# Kafka (via kafka-topics)\nhealthcheck:\n test: [\"CMD-SHELL\", \"kafka-topics.sh --bootstrap-server localhost:9092 --list\"]\n interval: 10s\n timeout: 10s\n retries: 5\n```\n\n**Application Health Checks**:\n\n```yaml\n# Basic HTTP check\nhealthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health\"]\n interval: 30s\n timeout: 10s\n retries: 3\n start_period: 40s\n\n# Check with specific response\nhealthcheck:\n test: [\"CMD-SHELL\", \"curl -sf http://localhost:8080/health | grep -q '\\\"status\\\":\\\"UP\\\"'\"]\n interval: 30s\n timeout: 10s\n retries: 3\n\n# HTTPS with insecure (self-signed cert)\nhealthcheck:\n test: [\"CMD\", \"curl\", \"-kf\", \"https://localhost:8443/health\"]\n interval: 30s\n timeout: 10s\n retries: 3\n\n# For images without curl\nhealthcheck:\n test: [\"CMD\", \"wget\", \"--spider\", \"-q\", \"http://localhost:8080/health\"]\n interval: 30s\n timeout: 10s\n retries: 3\n```\n\n**Liveness vs Readiness** (implementing both in one endpoint):\n\n```javascript\n// Node.js health endpoint\napp.get('/health', (req, res) => {\n // Liveness: Is the process alive?\n // (If this endpoint responds, yes)\n\n // Readiness: Can we serve traffic?\n const checks = {\n database: checkDatabase(),\n redis: checkRedis(),\n disk: checkDiskSpace()\n };\n\n const allHealthy = Object.values(checks).every(c => c.status === 'UP');\n\n res.status(allHealthy ? 200 : 503).json({\n status: allHealthy ? 'UP' : 'DOWN',\n checks\n });\n});\n\n// Separate endpoints for different purposes\napp.get('/health/live', (req, res) => {\n // Just respond if process is alive\n res.json({ status: 'UP' });\n});\n\napp.get('/health/ready', (req, res) => {\n // Check all dependencies\n const ready = checkAllDependencies();\n res.status(ready ? 200 : 503).json({ ready });\n});\n```\n\n---\n\n### 3. Graceful Shutdown Patterns\n\n**Understanding Stop Signals**:\n\n```yaml\nservices:\n app:\n stop_signal: SIGTERM # Signal sent first (default)\n stop_grace_period: 30s # Time before SIGKILL\n```\n\n**Signal Handling in Application**:\n\n```javascript\n// Node.js graceful shutdown\nconst server = app.listen(8080);\n\nconst gracefulShutdown = async (signal) => {\n console.log(`${signal} received, shutting down gracefully`);\n\n // 1. Stop accepting new connections\n server.close(() => {\n console.log('HTTP server closed');\n });\n\n // 2. Wait for existing requests to complete\n // (server.close handles this)\n\n // 3. Close database connections\n await db.close();\n console.log('Database connections closed');\n\n // 4. Close other resources\n await redis.quit();\n console.log('Redis connection closed');\n\n process.exit(0);\n};\n\nprocess.on('SIGTERM', () => gracefulShutdown('SIGTERM'));\nprocess.on('SIGINT', () => gracefulShutdown('SIGINT'));\n```\n\n```java\n// Java Spring Boot graceful shutdown\n@Configuration\npublic class GracefulShutdownConfig {\n\n @Bean\n public GracefulShutdownHandler gracefulShutdownHandler() {\n return new GracefulShutdownHandler();\n }\n}\n\n// application.yml\nserver:\n shutdown: graceful\n\nspring:\n lifecycle:\n timeout-per-shutdown-phase: 30s\n```\n\n**Compose Configuration for Graceful Shutdown**:\n\n```yaml\nservices:\n api:\n image: myapp:latest\n stop_grace_period: 30s\n deploy:\n update_config:\n order: start-first # Start new before stopping old\n failure_action: rollback\n rollback_config:\n parallelism: 1\n```\n\n**Pre-Stop Hook Pattern** (wait for connections to drain):\n\n```yaml\nservices:\n api:\n image: myapp:latest\n # Custom entrypoint that handles shutdown\n entrypoint: [\"/bin/sh\", \"-c\"]\n command:\n - |\n # Trap SIGTERM\n trap 'echo \"SIGTERM received\"; kill -TERM $PID; wait $PID' TERM\n\n # Start app in background\n node server.js &\n PID=$!\n\n # Wait for app\n wait $PID\n```\n\n---\n\n### 4. Migration and Initialization Patterns\n\n**One-Time Migration Service**:\n\n```yaml\nservices:\n migrate:\n image: myapp:latest\n command: npm run migrate\n depends_on:\n db:\n condition: service_healthy\n # No restart - should only run once\n restart: \"no\"\n # No healthcheck needed - will exit\n```\n\n**Migration with Locking** (prevent concurrent runs):\n\n```yaml\nservices:\n migrate:\n image: myapp:latest\n command: >\n sh -c \"\n # Try to acquire lock\n if ! mkdir /tmp/migration.lock 2>/dev/null; then\n echo 'Migration already running, skipping'\n exit 0\n fi\n trap 'rmdir /tmp/migration.lock' EXIT\n npm run migrate\n \"\n volumes:\n - migration-lock:/tmp\n depends_on:\n db:\n condition: service_healthy\n\nvolumes:\n migration-lock:\n```\n\n**Seed Data Pattern**:\n\n```yaml\nservices:\n # Only seed in development\n seed:\n image: myapp:latest\n command: npm run seed\n profiles: [dev]\n depends_on:\n migrate:\n condition: service_completed_successfully\n restart: \"no\"\n```\n\n**Wait-For Pattern** (when depends_on isn't enough):\n\n```yaml\nservices:\n app:\n image: myapp:latest\n entrypoint: [\"/wait-for-it.sh\", \"db:5432\", \"--\", \"npm\", \"start\"]\n volumes:\n - ./wait-for-it.sh:/wait-for-it.sh:ro\n```\n\nOr build it into the image:\n\n```dockerfile\nFROM node:20-alpine\n\n# Add wait script\nADD https://github.com/vishnubob/wait-for-it/raw/master/wait-for-it.sh /wait-for-it.sh\nRUN chmod +x /wait-for-it.sh\n\n# Your app\nCOPY . /app\nWORKDIR /app\n\nCMD [\"/wait-for-it.sh\", \"db:5432\", \"-t\", \"60\", \"--\", \"npm\", \"start\"]\n```\n\n---\n\n### 5. Handling Circular Dependencies\n\n**Problem: Circular Dependency**:\n\n```yaml\n# ❌ This creates a circular dependency\nservices:\n service-a:\n depends_on:\n service-b:\n condition: service_healthy\n\n service-b:\n depends_on:\n service-a:\n condition: service_healthy\n# Neither can start because each waits for the other!\n```\n\n**Solution 1: Break the Cycle with service_started**:\n\n```yaml\nservices:\n service-a:\n depends_on:\n service-b:\n condition: service_started # Just start, don't wait for healthy\n healthcheck:\n test: curl -f http://localhost:8080/health\n interval: 10s\n retries: 10 # More retries to handle startup race\n\n service-b:\n depends_on:\n service-a:\n condition: service_started\n healthcheck:\n test: curl -f http://localhost:8080/health\n interval: 10s\n retries: 10\n```\n\n**Solution 2: Design Out the Circular Dependency**:\n\n```yaml\n# Instead of A <-> B, use:\n# A -> shared-resource <- B\n\nservices:\n shared-db:\n image: postgres:16\n healthcheck:\n test: pg_isready\n\n service-a:\n depends_on:\n shared-db:\n condition: service_healthy\n # Communicates with B through DB or queue\n\n service-b:\n depends_on:\n shared-db:\n condition: service_healthy\n # Communicates with A through DB or queue\n```\n\n**Solution 3: Event-Driven Communication**:\n\n```yaml\nservices:\n redis:\n image: redis:7\n healthcheck:\n test: redis-cli ping\n\n service-a:\n depends_on:\n redis:\n condition: service_healthy\n # Publishes events, subscribes to B's events\n\n service-b:\n depends_on:\n redis:\n condition: service_healthy\n # Publishes events, subscribes to A's events\n```\n\n---\n\n### 6. Startup and Shutdown Order\n\n**Startup Order** (compose figures this out from depends_on):\n\n```\n1. Infrastructure (db, redis, queue)\n2. Setup tasks (migrations, seeds)\n3. Core services (api, worker)\n4. Auxiliary services (monitoring, logging)\n```\n\n**Shutdown Order** (reverse of startup):\n\n```yaml\nservices:\n # Stops last (infrastructure)\n db:\n # ...\n\n # Stops second\n api:\n depends_on:\n - db\n\n # Stops first (depends on api)\n nginx:\n depends_on:\n - api\n```\n\n**Controlling Shutdown Order Explicitly**:\n\n```bash\n# Manual ordered shutdown\ndocker compose stop nginx # Stop reverse proxy first\ndocker compose stop api # Stop API\ndocker compose stop worker # Stop workers\ndocker compose stop db redis # Stop infrastructure last\n```\n\n**Automated Graceful Shutdown Script**:\n\n```bash\n#!/bin/bash\n# graceful-shutdown.sh\n\nset -e\n\necho \"Starting graceful shutdown...\"\n\n# 1. Stop accepting new traffic\ndocker compose stop nginx\nsleep 5 # Allow in-flight requests to complete\n\n# 2. Stop application services\ndocker compose stop api worker\nsleep 10 # Allow graceful shutdown\n\n# 3. Stop infrastructure\ndocker compose stop redis db\n\necho \"Shutdown complete\"\n```\n\n---\n\n### 7. Complete Production Example\n\n```yaml\n# docker-compose.yml\n\nx-healthcheck-defaults: &healthcheck-defaults\n interval: 10s\n timeout: 5s\n retries: 5\n start_period: 30s\n\nservices:\n # ═══════════════════════════════════════════════════════════════\n # INFRASTRUCTURE LAYER\n # ═══════════════════════════════════════════════════════════════\n\n db:\n image: postgres:16-alpine\n environment:\n POSTGRES_USER: app\n POSTGRES_PASSWORD: ${DB_PASSWORD:?Required}\n POSTGRES_DB: app\n volumes:\n - pgdata:/var/lib/postgresql/data\n healthcheck:\n <<: *healthcheck-defaults\n test: [\"CMD-SHELL\", \"pg_isready -U app -d app\"]\n start_period: 10s\n\n redis:\n image: redis:7-alpine\n command: redis-server --appendonly yes\n volumes:\n - redisdata:/data\n healthcheck:\n <<: *healthcheck-defaults\n test: [\"CMD\", \"redis-cli\", \"ping\"]\n start_period: 5s\n\n # ═══════════════════════════════════════════════════════════════\n # SETUP LAYER (runs once on deploy)\n # ═══════════════════════════════════════════════════════════════\n\n migrate:\n image: ${IMAGE}:${VERSION}\n command: npm run migrate\n environment:\n DATABASE_URL: postgres://app:${DB_PASSWORD}@db:5432/app\n depends_on:\n db:\n condition: service_healthy\n restart: \"no\"\n profiles: [setup]\n\n # ═══════════════════════════════════════════════════════════════\n # APPLICATION LAYER\n # ═══════════════════════════════════════════════════════════════\n\n api:\n image: ${IMAGE}:${VERSION}\n environment:\n NODE_ENV: production\n PORT: 8080\n DATABASE_URL: postgres://app:${DB_PASSWORD}@db:5432/app\n REDIS_URL: redis://redis:6379\n depends_on:\n db:\n condition: service_healthy\n redis:\n condition: service_healthy\n healthcheck:\n <<: *healthcheck-defaults\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health/ready\"]\n stop_grace_period: 30s\n deploy:\n resources:\n limits:\n memory: 1G\n\n worker:\n image: ${IMAGE}:${VERSION}\n command: npm run worker\n environment:\n NODE_ENV: production\n DATABASE_URL: postgres://app:${DB_PASSWORD}@db:5432/app\n REDIS_URL: redis://redis:6379\n depends_on:\n api:\n condition: service_healthy # Start after API is ready\n healthcheck:\n <<: *healthcheck-defaults\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:9090/health\"]\n start_period: 10s\n stop_grace_period: 60s # Longer for job completion\n deploy:\n resources:\n limits:\n memory: 512M\n\n # ═══════════════════════════════════════════════════════════════\n # PROXY LAYER\n # ═══════════════════════════════════════════════════════════════\n\n nginx:\n image: nginx:alpine\n ports:\n - \"80:80\"\n - \"443:443\"\n volumes:\n - ./nginx.conf:/etc/nginx/nginx.conf:ro\n - ./certs:/etc/nginx/certs:ro\n depends_on:\n api:\n condition: service_healthy\n healthcheck:\n <<: *healthcheck-defaults\n test: [\"CMD\", \"nginx\", \"-t\"]\n start_period: 5s\n stop_grace_period: 10s\n\nvolumes:\n pgdata:\n redisdata:\n```\n\n**Deployment Script**:\n\n```bash\n#!/bin/bash\n# deploy.sh\n\nset -e\n\nexport IMAGE=myapp\nexport VERSION=${1:-latest}\nexport DB_PASSWORD=${DB_PASSWORD:?DB_PASSWORD required}\n\necho \"Deploying version: $VERSION\"\n\n# 1. Pull new images\ndocker compose pull\n\n# 2. Run migrations\necho \"Running migrations...\"\ndocker compose --profile setup run --rm migrate\n\n# 3. Rolling update of services\necho \"Updating services...\"\ndocker compose up -d --no-deps api\ndocker compose up -d --no-deps worker\ndocker compose up -d --no-deps nginx\n\n# 4. Wait for health checks\necho \"Waiting for services to be healthy...\"\ntimeout 120 bash -c '\n while ! docker compose ps | grep -q \"(healthy)\"; do\n sleep 2\n done\n'\n\n# 5. Clean up old containers\ndocker compose down --remove-orphans\n\necho \"Deployment complete!\"\n```\n\n---\n\n## ⚠️ Common Mistakes\n\n### Mistake 1: Using depends_on Without Condition\n\n```yaml\n# ❌ Wrong: Only waits for container start\nservices:\n app:\n depends_on:\n - db # DB not ready when app starts!\n\n# ✅ Right: Wait for healthy\nservices:\n app:\n depends_on:\n db:\n condition: service_healthy\n db:\n healthcheck:\n test: pg_isready\n```\n\n### Mistake 2: Shallow Health Checks\n\n```yaml\n# ❌ Wrong: Just checks if port is open\nhealthcheck:\n test: [\"CMD\", \"nc\", \"-z\", \"localhost\", \"8080\"]\n\n# ✅ Right: Checks actual functionality\nhealthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health/ready\"]\n```\n\n### Mistake 3: Missing start_period\n\n```yaml\n# ❌ Wrong: Health check fails during slow startup\nhealthcheck:\n test: curl -f http://localhost:8080/health\n interval: 10s\n retries: 3\n # Container marked unhealthy after 30s even if startup takes 60s\n\n# ✅ Right: Allow time for initialization\nhealthcheck:\n test: curl -f http://localhost:8080/health\n interval: 10s\n retries: 3\n start_period: 90s # Don't count failures during first 90s\n```\n\n### Mistake 4: Immediate SIGKILL\n\n```yaml\n# ❌ Wrong: Only 10s to shutdown\nstop_grace_period: 10s\n# Long-running requests get killed!\n\n# ✅ Right: Allow time for graceful shutdown\nstop_grace_period: 30s # Or longer for workers\n```\n\n### Mistake 5: Not Handling SIGTERM\n\n```javascript\n// ❌ Wrong: Process killed immediately\nconst server = app.listen(8080);\n// No signal handling - connections dropped on stop\n\n// ✅ Right: Graceful shutdown\nconst server = app.listen(8080);\n\nprocess.on('SIGTERM', () => {\n console.log('SIGTERM received, closing server');\n server.close(() => {\n console.log('Server closed');\n process.exit(0);\n });\n});\n```\n\n---\n\n## 🐛 Debug This\n\nServices won't start in the right order:\n\n```yaml\nservices:\n api:\n image: myapp:latest\n depends_on:\n db:\n condition: service_healthy\n redis:\n condition: service_healthy\n healthcheck:\n test: curl -f http://localhost:8080/health\n interval: 5s\n retries: 3\n\n db:\n image: postgres:16\n healthcheck:\n test: pg_isready\n interval: 5s\n retries: 5\n\n redis:\n image: redis:7\n healthcheck:\n test: redis-cli ping\n interval: 5s\n retries: 5\n```\n\n```bash\n$ docker compose up\ndb is healthy\nredis is healthy\napi exited with code 1\n\n$ docker compose logs api\nError: ECONNREFUSED connecting to db:5432\n```\n\nAPI fails even though db shows healthy. What's wrong?\n\n

Click to reveal analysis

\n\n**Problems identified**:\n\n1. **Missing start_period on API** - API health check starts immediately, but API needs time to boot:\n```yaml\nhealthcheck:\n test: curl -f http://localhost:8080/health\n interval: 5s\n retries: 3\n # No start_period! API marked unhealthy during startup\n```\n\n2. **Missing PostgreSQL user in health check**:\n```yaml\nhealthcheck:\n test: pg_isready # Uses default user, not app user\n```\n\n3. **API needs connection retry logic** - Even with depends_on, there's a small window where connection might fail.\n\n**Fixed version**:\n```yaml\nservices:\n api:\n image: myapp:latest\n depends_on:\n db:\n condition: service_healthy\n redis:\n condition: service_healthy\n healthcheck:\n test: curl -f http://localhost:8080/health\n interval: 10s\n timeout: 5s\n retries: 3\n start_period: 30s # Allow startup time\n\n db:\n image: postgres:16\n environment:\n POSTGRES_USER: app\n POSTGRES_PASSWORD: secret\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U app\"] # Check as app user\n interval: 5s\n timeout: 5s\n retries: 5\n start_period: 10s\n\n redis:\n image: redis:7\n healthcheck:\n test: [\"CMD\", \"redis-cli\", \"ping\"]\n interval: 5s\n timeout: 3s\n retries: 5\n```\n\nAdditionally, the application should implement connection retry:\n```javascript\nconst connectWithRetry = async () => {\n for (let i = 0; i < 10; i++) {\n try {\n await db.connect();\n return;\n } catch (err) {\n console.log(`DB connection attempt ${i + 1} failed, retrying...`);\n await sleep(1000);\n }\n }\n throw new Error('Could not connect to database');\n};\n```\n

\n\n---\n\n## 💻 Exercises\n\n### Exercise 1: Basic Dependency Chain\nCreate a compose file with:\n- PostgreSQL with proper health check\n- API that depends on PostgreSQL being healthy\n- Worker that depends on API being healthy\n\n### Exercise 2: Migration Pattern\nImplement a migration service that:\n- Runs only once (doesn't restart)\n- Waits for database to be healthy\n- API waits for migration to complete\n\n### Exercise 3: Graceful Shutdown\nCreate an application that:\n- Handles SIGTERM properly\n- Completes in-flight requests before stopping\n- Has appropriate stop_grace_period\n\n### Exercise 4: Health Check Design\nDesign health checks for:\n- A service that needs 45 seconds to start\n- A service that should fail if Redis is unreachable\n- A service with both liveness and readiness requirements\n\n### Exercise 5: Circular Dependency Resolution\nYou have two services that need to communicate with each other. Design a solution that:\n- Avoids circular depends_on\n- Ensures both services can find each other\n- Handles the case where one service starts before the other\n\n---\n\n## 🎤 Interview Questions\n\n### Q1: What's the difference between `service_started` and `service_healthy` conditions?\n\n**Answer**:\n\n| Condition | Waits For | Use Case |\n|-----------|-----------|----------|\n| `service_started` | Container is running | Quick services, apps with built-in retry |\n| `service_healthy` | Health check passes | Databases, caches that need init time |\n\n```yaml\n# service_started - just waits for container\ndepends_on:\n cache:\n condition: service_started\n# Good when: app has retry logic, service starts fast\n\n# service_healthy - waits for health check\ndepends_on:\n db:\n condition: service_healthy\n# Good when: dependent can't handle connection failures\n```\n\n**Important**: Without explicit condition, `depends_on` defaults to `service_started`, which is why many deployments have race conditions - developers assume it waits for readiness.\n\n---\n\n### Q2: How would you implement a health check that verifies database connectivity?\n\n**Answer**: Two approaches - database-side and app-side:\n\n**Database-side health check**:\n```yaml\nservices:\n db:\n image: postgres:16\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}\"]\n interval: 5s\n timeout: 5s\n retries: 5\n```\n\n**Application-side health check** (more comprehensive):\n```yaml\nservices:\n api:\n healthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health/ready\"]\n```\n\n```javascript\n// /health/ready endpoint\napp.get('/health/ready', async (req, res) => {\n try {\n // Test actual query, not just connection\n await db.query('SELECT 1');\n res.json({ database: 'connected' });\n } catch (err) {\n res.status(503).json({ database: 'disconnected', error: err.message });\n }\n});\n```\n\nBest practice is both: database health check ensures it accepts connections, application health check ensures your specific user and database work.\n\n---\n\n### Q3: How do you handle graceful shutdown in a containerized application?\n\n**Answer**: Multi-layer approach:\n\n1. **Compose configuration**:\n```yaml\nservices:\n api:\n stop_signal: SIGTERM\n stop_grace_period: 30s\n```\n\n2. **Application signal handling**:\n```javascript\nlet isShuttingDown = false;\n\nprocess.on('SIGTERM', async () => {\n isShuttingDown = true;\n console.log('Shutdown initiated');\n\n // 1. Stop accepting new requests\n server.close();\n\n // 2. Wait for in-flight requests (handled by server.close)\n\n // 3. Close database connections\n await db.end();\n\n // 4. Exit\n process.exit(0);\n});\n\n// Middleware to reject new requests during shutdown\napp.use((req, res, next) => {\n if (isShuttingDown) {\n res.status(503).json({ error: 'Server shutting down' });\n } else {\n next();\n }\n});\n```\n\n3. **Load balancer coordination**: Remove container from pool before stopping:\n```bash\n# Pre-stop script\ncurl -X DELETE http://lb/backends/this-instance\nsleep 5 # Allow in-flight to complete\nkill -TERM 1\n```\n\n---\n\n### Q4: How do you prevent race conditions when multiple services need the same database?\n\n**Answer**: Several strategies:\n\n1. **Single migration service with completion dependency**:\n```yaml\nservices:\n migrate:\n command: npm run migrate\n depends_on:\n db:\n condition: service_healthy\n restart: \"no\"\n\n api:\n depends_on:\n migrate:\n condition: service_completed_successfully\n\n worker:\n depends_on:\n migrate:\n condition: service_completed_successfully\n```\n\n2. **Database-level migration locking**:\n```javascript\n// Migration script with advisory lock\nasync function migrate() {\n // Acquire lock (only one process can hold it)\n await db.query('SELECT pg_advisory_lock(12345)');\n\n try {\n await runMigrations();\n } finally {\n await db.query('SELECT pg_advisory_unlock(12345)');\n }\n}\n```\n\n3. **Idempotent migrations**: Design migrations to be safe if run multiple times:\n```sql\n-- Idempotent migration\nCREATE TABLE IF NOT EXISTS users (...);\nCREATE INDEX IF NOT EXISTS idx_users_email ON users(email);\n```\n\n---\n\n### Q5: What's the startup and shutdown order when you run `docker compose down`?\n\n**Answer**: Compose reverses the dependency order for shutdown:\n\n**Startup order** (based on depends_on):\n```\n1. db (no dependencies)\n2. redis (no dependencies)\n3. migrate (depends on db)\n4. api (depends on db, redis, migrate)\n5. worker (depends on api)\n6. nginx (depends on api)\n```\n\n**Shutdown order** (reverse):\n```\n1. nginx (most dependent)\n2. worker\n3. api\n4. migrate (if still running)\n5. db, redis (least dependent)\n```\n\n**Important nuances**:\n- `docker compose down` sends SIGTERM to all containers in reverse dependency order\n- Containers have `stop_grace_period` to finish gracefully\n- After grace period, SIGKILL is sent\n- Networks and volumes are removed last\n\n**Best practice for clean shutdown**:\n```bash\n# Explicit ordered shutdown for production\ndocker compose stop nginx # Stop accepting traffic\nsleep 5 # Drain connections\ndocker compose stop api worker # Stop application\nsleep 10 # Allow graceful shutdown\ndocker compose stop db redis # Stop infrastructure\ndocker compose down # Remove containers\n```\n\n---\n\n## 📝 Summary & Key Takeaways\n\n### Dependency Conditions\n| Condition | Wait For | Use Case |\n|-----------|----------|----------|\n| `service_started` | Container running | Quick services |\n| `service_healthy` | Health check pass | Databases, caches |\n| `service_completed_successfully` | Exit code 0 | Migrations |\n\n### Health Check Best Practices\n- Always include `start_period` for slow-starting services\n- Test actual functionality, not just port availability\n- Use appropriate intervals (5-30s depending on service)\n\n### Graceful Shutdown\n1. Handle SIGTERM in application\n2. Stop accepting new connections\n3. Complete in-flight requests\n4. Close resources (DB, cache)\n5. Exit cleanly\n\n### Startup Order\n```\nInfrastructure → Setup → Application → Proxy\n```\n\n### Shutdown Order\n```\nProxy → Application → Setup → Infrastructure\n```\n\n---\n\n## 📋 Quick Reference\n\n```yaml\n# Health-based dependency\ndepends_on:\n db:\n condition: service_healthy\n\n# Complete health check\nhealthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://localhost:8080/health\"]\n interval: 10s\n timeout: 5s\n retries: 3\n start_period: 30s\n\n# Graceful shutdown\nstop_signal: SIGTERM\nstop_grace_period: 30s\n\n# One-time service\nrestart: \"no\"\ndepends_on:\n db:\n condition: service_healthy\n```\n\n---\n\n## 📅 Review Schedule\n\n- **Day 1**: Practice depends_on with conditions\n- **Day 3**: Implement health checks for your services\n- **Day 7**: Add graceful shutdown handling\n- **Day 14**: Design complete startup/shutdown flow\n- **Day 30**: Audit production compose files\n\n---\n\n## 📚 Series Navigation\n\n- **Previous**: [Part 14 - Compose File Deep Dive](/articles/docker-part-14-compose-deep-dive)\n- **Next**: [Part 16 - Development vs Production Compose](/articles/docker-part-16-dev-vs-prod)\n- **Index**: [Docker Compendium Series](/series/docker-compendium)","featuredImage","isPublished","isFeatured","subscriptionType","BASIC","publishedAt","2026-06-10T07:19:25.535852Z","createdAt","2026-06-10T07:19:25.536469Z","updatedAt","2026-06-16T21:26:56.086200Z","readingTimeMinutes",16,"estimatedReadingTime","authorId",3,"authorUsername","tomasz_main","categoryName","Devops","readingTime","seriesId","seriesOrder",15,"author",{"_31":55,"_94":57,"_95":96,"_97":98,"_99":-5,"_100":-5,"_101":-5,"_102":-5,"_103":-5},"category",{"_31":90,"_72":59,"_35":91,"_75":92,"_93":-5},"tags",[78,79,80],"series",{"_31":55,"_72":73,"_35":74,"_75":76,"_77":23},"name","Docker Compendium: From Practitioner to Expert","docker-compendium","description","Master Docker internals, optimization patterns, and production best practices. For developers who use Docker daily but want deeper understanding of what's happening under the hood. Not a beginner series - we assume you can write Dockerfiles and use docker-compose. Learn what namespaces and cgroups really do, why your builds are slow, how to debug containers that crash, and how to run Docker in production like a pro.\n","isActive",{"_31":87,"_72":88,"_35":89},{"_31":84,"_72":85,"_35":86},{"_31":81,"_72":82,"_35":83},243,"Orchestration","orchestration",213,"Docker","docker",242,"Compose","compose",7,"devops","Category for Devops","colorHex","username","firstName","Tomasz","lastName","Jarosz","profilePicture","bio","website","githubUrl","linkedinUrl"]