Development vs Production Compose

📋 At a Glance

Aspect	Details
Difficulty	🟡 Intermediate
Prerequisites	Part 14 (Compose Deep Dive), Part 15 (Dependencies)
Key Patterns	Override files, profiles, multi-stage builds
Time Investment	25 minutes read + 60 minutes practice
Payoff	Same codebase works perfectly in dev and prod

🎯 What You'll Learn

After this article, you'll be able to:

Structure compose files for multiple environments
Configure development with hot reload, debugging, and local tools
Configure production with security, performance, and reliability
Handle secrets differently across environments
Implement environment parity while maintaining dev productivity

🔥 Production Story: The "Works on My Machine" Disaster

The Setup: A team used one compose file for everything. Development worked great. Production... not so much.

The Problem Compose File:

YAML(13 lines)
Code
Loading syntax highlighter...

What Went Wrong:

Volume mount .:/app overwrote production code with empty directory
Debug port 9229 exposed to internet
DEBUG=true leaked sensitive information in logs
npm run dev ran nodemon instead of production server
No resource limits caused OOM under load

The Fix: Separate development and production configurations that share a common base.

🧠 Mental Model: Environment Layering

┌─────────────────────────────────────────────────────────────────┐
│                    COMPOSE FILE LAYERING                        │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   BASE LAYER (docker-compose.yml)                               │
│   ┌─────────────────────────────────────────────────────────┐   │
│   │  • Service definitions                                  │   │
│   │  • Network topology                                     │   │
│   │  • Volume definitions                                   │   │
│   │  • Health checks                                        │   │
│   │  • Common environment variables                         │   │
│   └─────────────────────────────────────────────────────────┘   │
│                           ↓                                     │
│   DEV OVERRIDE (docker-compose.override.yml)                    │
│   ┌─────────────────────────────────────────────────────────┐   │
│   │  • build context (instead of image)                     │   │
│   │  • Volume mounts for hot reload                         │   │
│   │  • Debug ports exposed                                  │   │
│   │  • DEV environment variables                            │   │
│   │  • Local database ports exposed                         │   │
│   │  • Dev tools (pgadmin, redis-commander)                 │   │
│   └─────────────────────────────────────────────────────────┘   │
│                                                                 │
│   PROD OVERRIDE (docker-compose.prod.yml)                       │
│   ┌─────────────────────────────────────────────────────────┐   │
│   │  • Pre-built images                                     │   │
│   │  • No volume mounts (except data)                       │   │
│   │  • Resource limits                                      │   │
│   │  • Production logging                                   │   │
│   │  • Security hardening                                   │   │
│   │  • Replicas for scaling                                 │   │
│   └─────────────────────────────────────────────────────────┘   │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Auto-loaded: compose.yml + compose.override.yml
Production:  compose.yml + compose.prod.yml
Staging:     compose.yml + compose.staging.yml
Testing:     compose.yml + compose.test.yml

🔬 Deep Dive

1. File Structure and Naming

Recommended Directory Structure:

project/
├── docker-compose.yml           # Base configuration
├── docker-compose.override.yml  # Development (auto-loaded)
├── docker-compose.prod.yml      # Production
├── docker-compose.staging.yml   # Staging (optional)
├── docker-compose.test.yml      # Testing
├── .env                         # Default environment
├── .env.example                 # Template for .env
├── .env.production              # Production env (gitignored)
├── Dockerfile                   # Multi-stage Dockerfile
└── docker/
    ├── nginx/
    │   └── nginx.conf
    └── scripts/
        └── entrypoint.sh

Naming Conventions:

BASH(12 lines)
Code
Loading syntax highlighter...

2. Base Configuration (docker-compose.yml)

The base file contains everything common across all environments:

YAML(107 lines)
Code
Loading syntax highlighter...

3. Development Override (docker-compose.override.yml)

Development focuses on productivity: hot reload, debugging, easy database access:

YAML(90 lines)
Code
Loading syntax highlighter...

Development .env file:

BASH(7 lines)
Code
Loading syntax highlighter...

Running Development:

BASH(11 lines)
Code
Loading syntax highlighter...

4. Production Override (docker-compose.prod.yml)

Production focuses on security, reliability, and performance:

YAML(113 lines)
Code
Loading syntax highlighter...

Production .env.production file (gitignored):

BASH(8 lines)
Code
Loading syntax highlighter...

Running Production:

BASH(11 lines)
Code
Loading syntax highlighter...

5. Multi-Stage Dockerfile

Single Dockerfile that serves both development and production:

DOCKERFILE(78 lines)
Code
Loading syntax highlighter...

Usage in compose:

YAML(13 lines)
Code
Loading syntax highlighter...

6. Environment-Specific Secrets

Development: Simple, committed secrets (okay for local dev):

YAML(6 lines)
Code
Loading syntax highlighter...

Production: External secrets (never in compose files):

YAML(18 lines)
Code
Loading syntax highlighter...

Docker Secrets Pattern:

BASH(5 lines)
Code
Loading syntax highlighter...

Environment Variables from File:

YAML(5 lines)
Code
Loading syntax highlighter...

7. Testing Configuration

Testing needs isolation and speed:

YAML(60 lines)
Code
Loading syntax highlighter...

Running Tests:

BASH(7 lines)
Code
Loading syntax highlighter...

8. Complete Example: Makefile for Environment Management

MAKEFILE(51 lines)
Code
Loading syntax highlighter...

⚠️ Common Mistakes

Mistake 1: Same Dockerfile CMD for Dev and Prod

DOCKERFILE(11 lines)
Code
Loading syntax highlighter...

Mistake 2: Volume Mount in Production

YAML(13 lines)
Code
Loading syntax highlighter...

Mistake 3: Exposing Debug Ports in Production

YAML(19 lines)
Code
Loading syntax highlighter...

Mistake 4: Hardcoded Development Settings

YAML(13 lines)
Code
Loading syntax highlighter...

Mistake 5: No Resource Limits in Production

YAML(14 lines)
Code
Loading syntax highlighter...

🐛 Debug This

Development works, production fails:

YAML(13 lines)
Code
Loading syntax highlighter...

BASH(7 lines)
Code
Loading syntax highlighter...

Why does production fail?

Click to reveal analysis

Problem: Volume mount from base file persists into production!

YAML(6 lines)
Code
Loading syntax highlighter...

Compose merge rules: Arrays (like volumes) are appended, not replaced. The production override doesn't remove the volume mount.

Fix Option 1: Move volume to dev-only override:

YAML(20 lines)
Code
Loading syntax highlighter...

Fix Option 2: Explicitly clear volumes in prod:

YAML(5 lines)
Code
Loading syntax highlighter...

Note: volumes: [] doesn't work in all compose versions. Best practice is to not have dev-only settings in the base file.

💻 Exercises

Exercise 1: Basic Environment Split

Take this single compose file and split it into base + dev override:

YAML(11 lines)
Code
Loading syntax highlighter...

Exercise 2: Production Hardening

Create a docker-compose.prod.yml that:

Uses pre-built images
Adds resource limits
Enables read-only filesystem
Removes debug settings
Adds nginx as reverse proxy

Exercise 3: Multi-Stage Dockerfile

Create a multi-stage Dockerfile with:

development target with all dev tools
build target that compiles the app
production target that's minimal and secure

Exercise 4: Test Configuration

Create docker-compose.test.yml that:

Uses separate test database (tmpfs for speed)
Runs tests in isolation
Cleans up after itself

Exercise 5: Secrets Management

Implement different secret handling for:

Development: Simple env vars
Production: Docker secrets or external secrets manager

🎤 Interview Questions

Q1: How do you structure compose files for multiple environments?

Answer: Use layered compose files with automatic and explicit loading:

project/
├── docker-compose.yml           # Base (always loaded)
├── docker-compose.override.yml  # Dev (auto-loaded locally)
├── docker-compose.prod.yml      # Production
├── docker-compose.test.yml      # Testing

Loading behavior:

BASH(8 lines)
Code
Loading syntax highlighter...

Key principles:

Base file: Service definitions, networks, health checks
Dev override: Build context, volume mounts, debug ports
Prod override: Images, resource limits, security hardening

Q2: What should be in the base compose file vs environment-specific overrides?

Answer:

Base file (docker-compose.yml):

Service names and structure
Network topology
Volume definitions (without mounts)
Health checks
Dependencies
Environment variables using ${VAR:-default} pattern

Development override:

build: context (build from source)
Volume mounts for hot reload
Debug ports (9229, etc.)
Dev tools (pgadmin, etc.)
Database ports exposed
DEBUG=true, verbose logging

Production override:

image: (pre-built images)
Resource limits
Security options (read_only, no-new-privileges)
Production logging config
Replicas for scaling
Reverse proxy configuration

Q3: How do you handle secrets differently in development vs production?

Answer: Different strategies for different security needs:

Development (convenience over security):

YAML(6 lines)
Code
Loading syntax highlighter...

Production (security first):

Option 1 - Environment variables from CI/CD:

YAML(5 lines)
Code
Loading syntax highlighter...

Option 2 - Docker secrets (Swarm):

YAML(8 lines)
Code
Loading syntax highlighter...

Option 3 - External secrets manager:

YAML(6 lines)
Code
Loading syntax highlighter...

Best practice: Never commit production secrets. Use CI/CD secret injection or external secrets management.

Q4: How do you ensure development environment matches production?

Answer: Environment parity with controlled differences:

Same base services:

YAML(6 lines)
Code
Loading syntax highlighter...

Same network topology:

YAML(4 lines)
Code
Loading syntax highlighter...

Multi-stage Dockerfile:

DOCKERFILE(5 lines)
Code
Loading syntax highlighter...

Controlled differences (documented):

Dev: Volume mounts, debug ports, verbose logging
Prod: Resource limits, security hardening, monitoring

Same configurations:

YAML(5 lines)
Code
Loading syntax highlighter...

Q5: How would you debug an issue that only happens in production compose setup?

Answer: Systematic approach to reproduce and diagnose:

Run production config locally:

BASH(3 lines)
Code
Loading syntax highlighter...

Compare configurations:

BASH(4 lines)
Code
Loading syntax highlighter...

Check for common issues:

Volume mounts persisting from base (overwriting image contents)
Missing environment variables (required in prod)
Resource limits causing OOM
Network isolation blocking communication
Read-only filesystem blocking temp files

Inspect running containers:

BASH(3 lines)
Code
Loading syntax highlighter...

Add debug temporarily:

YAML(7 lines)
Code
Loading syntax highlighter...

📝 Summary & Key Takeaways

File Structure

docker-compose.yml           # Base (services, networks, volumes)
docker-compose.override.yml  # Dev (auto-loaded)
docker-compose.prod.yml      # Production (explicit)

Development Config

Build from source
Volume mounts for hot reload
Debug ports exposed
Databases accessible
Verbose logging

Production Config

Pre-built images
No volume mounts (except data)
Resource limits
Security hardening
Minimal logging

Running Commands

BASH(5 lines)
Code
Loading syntax highlighter...

📋 Quick Reference

YAML(20 lines)
Code
Loading syntax highlighter...

📅 Review Schedule

Day 1: Split existing compose into base + override
Day 3: Add production override with security
Day 7: Implement multi-stage Dockerfile
Day 14: Set up complete dev/test/prod pipeline
Day 30: Audit production configs for security

Previous: Part 15 - Service Dependencies & Orchestration
Next: Part 17 - CI/CD Pipeline Patterns
Index: Docker Compendium Series