Container Security Hardening

Containers provide isolation, but not security by default. Running as root, with all capabilities, writable filesystem - these defaults are convenient but dangerous. This article covers practical hardening: non-root users, read-only filesystems, capability dropping, and defense in depth.

📋 At a Glance

Aspect	Details
Topic	Non-root users, capabilities, read-only fs, seccomp, AppArmor
Complexity	Advanced
Prerequisites	Part 1 (Container Internals), Part 7 (Base Images)
Key Insight	Security is layers - each control reduces attack surface
Time to Master	3-4 hours

🎯 What You'll Learn

Non-root containers - why and how to run without root
Linux capabilities - granular permissions instead of all-or-nothing
Read-only filesystems - preventing runtime modifications
Seccomp profiles - limiting system calls
Defense in depth - combining multiple security controls

🔥 Production Story: The Container Escape

A security audit found a critical vulnerability: an attacker who compromised the web application could escape the container and access the host.

The vulnerable setup:

YAML(6 lines)
Code
Loading syntax highlighter...

The attack path:

BASH(4 lines)
Code
Loading syntax highlighter...

The fix:

YAML(11 lines)
Code
Loading syntax highlighter...

Lesson: Never use privileged: true or mount host root. Every unnecessary permission is an attack vector.

🧠 Mental Model: Defense in Depth

┌─────────────────────────────────────────────────────────────────────────┐
│                    CONTAINER SECURITY LAYERS                            │
│                                                                         │
│  ┌──────────────────────────────────────────────────────────────────────┐
│  │  LAYER 1: IMAGE SECURITY                                             │
│  │  • Minimal base image (distroless, alpine)                           │
│  │  • No secrets in layers                                              │
│  │  • Regular vulnerability scanning                                    │
│  │  • Signed/verified images                                            │
│  └──────────────────────────────────────────────────────────────────────┘
│                              │                                          │
│                              ▼                                          │
│  ┌──────────────────────────────────────────────────────────────────────┐
│  │  LAYER 2: USER PERMISSIONS                                           │
│  │  • Non-root user                                                     │
│  │  • no-new-privileges                                                 │
│  │  • Minimal file permissions                                          │
│  └──────────────────────────────────────────────────────────────────────┘
│                              │                                          │
│                              ▼                                          │
│  ┌──────────────────────────────────────────────────────────────────────┐
│  │  LAYER 3: CAPABILITIES                                               │
│  │  • Drop ALL capabilities                                             │
│  │  • Add only what's needed                                            │
│  │  • Never use --privileged                                            │
│  └──────────────────────────────────────────────────────────────────────┘
│                              │                                          │
│                              ▼                                          │
│  ┌──────────────────────────────────────────────────────────────────────┐
│  │  LAYER 4: FILESYSTEM                                                 │
│  │  • Read-only root filesystem                                         │
│  │  • tmpfs for writable paths                                          │
│  │  • No host mounts (or read-only)                                     │
│  └──────────────────────────────────────────────────────────────────────┘
│                              │                                          │
│                              ▼                                          │
│  ┌──────────────────────────────────────────────────────────────────────┐
│  │  LAYER 5: SYSCALL FILTERING                                          │
│  │  • Seccomp profiles                                                  │
│  │  • AppArmor/SELinux                                                  │
│  │  • Block dangerous syscalls                                          │
│  └──────────────────────────────────────────────────────────────────────┘
│                              │                                          │
│                              ▼                                          │
│  ┌──────────────────────────────────────────────────────────────────────┐
│  │  LAYER 6: NETWORK                                                    │
│  │  • Isolated networks                                                 │
│  │  • No unnecessary port exposure                                      │
│  │  • Network policies                                                  │
│  └──────────────────────────────────────────────────────────────────────┘
│                                                                         │
│  Each layer reduces attack surface. Compromise of one layer             │
│  doesn't mean full compromise.                                          │
└─────────────────────────────────────────────────────────────────────────┘

🔬 Deep Dive

Non-Root Containers

By default, containers run as root (UID 0). This is dangerous:

BASH(6 lines)
Code
Loading syntax highlighter...

Create non-root user in Dockerfile:

DOCKERFILE(14 lines)
Code
Loading syntax highlighter...

Run as non-root at runtime:

BASH(5 lines)
Code
Loading syntax highlighter...

Docker Compose:

YAML(4 lines)
Code
Loading syntax highlighter...

Verify non-root:

BASH(5 lines)
Code
Loading syntax highlighter...

no-new-privileges

Prevents processes from gaining additional privileges:

BASH(13 lines)
Code
Loading syntax highlighter...

Always enable:

YAML(4 lines)
Code
Loading syntax highlighter...

Linux Capabilities

Capabilities split root power into granular permissions:

BASH(8 lines)
Code
Loading syntax highlighter...

Common capabilities:

Capability	Purpose
NET_BIND_SERVICE	Bind to ports < 1024
NET_RAW	Use raw sockets (ping)
CHOWN	Change file ownership
SETUID/SETGID	Change user/group
SYS_ADMIN	Many admin operations (dangerous!)
SYS_PTRACE	Debug processes

Drop all, add only needed:

BASH(5 lines)
Code
Loading syntax highlighter...

YAML(7 lines)
Code
Loading syntax highlighter...

Never use --privileged:

BASH(4 lines)
Code
Loading syntax highlighter...

Read-Only Filesystem

Prevent runtime modifications to container filesystem:

BASH
Code
Loading syntax highlighter...

Handle writable requirements:

BASH(9 lines)
Code
Loading syntax highlighter...

YAML(7 lines)
Code
Loading syntax highlighter...

Identify needed writable paths:

BASH(5 lines)
Code
Loading syntax highlighter...

Seccomp Profiles

Seccomp filters system calls:

BASH(11 lines)
Code
Loading syntax highlighter...

Custom seccomp profile:

JSON(12 lines)
Code
Loading syntax highlighter...

Generate profile for your app:

BASH(6 lines)
Code
Loading syntax highlighter...

AppArmor

AppArmor provides Mandatory Access Control:

BASH(11 lines)
Code
Loading syntax highlighter...

Complete Hardened Container

DOCKERFILE(22 lines)
Code
Loading syntax highlighter...

YAML(17 lines)
Code
Loading syntax highlighter...

⚠️ Common Mistakes

Mistake 1: Running as Root

DOCKERFILE(11 lines)
Code
Loading syntax highlighter...

Mistake 2: Using --privileged

YAML(10 lines)
Code
Loading syntax highlighter...

Mistake 3: Mounting Sensitive Host Paths

YAML(12 lines)
Code
Loading syntax highlighter...

🐛 Debug This: The Permission Denied Mystery

A developer reports: "My app works with root but fails as non-root user!"

DOCKERFILE(7 lines)
Code
Loading syntax highlighter...

BASH(2 lines)
Code
Loading syntax highlighter...

Why does the non-root user fail?

✅ Solution:

The /app directory and files are owned by root because COPY and npm install ran before USER instruction.

The problem:

COPY runs as root → files owned by root
npm install runs as root → node_modules owned by root
USER appuser switches user
appuser can't write to root-owned directories

Fix 1: Set ownership explicitly:

DOCKERFILE(13 lines)
Code
Loading syntax highlighter...

Fix 2: Create writable directories:

DOCKERFILE(11 lines)
Code
Loading syntax highlighter...

Fix 3: Use tmpfs for temporary data:

YAML(5 lines)
Code
Loading syntax highlighter...

Key lesson: Order matters. COPY ownership, directory permissions, and USER instruction must be coordinated.

💻 Exercises

Exercise 1: Audit Container Security

⭐ Difficulty: Easy | ⏱️ Time: 15 minutes

BASH(19 lines)
Code
Loading syntax highlighter...

Exercise 2: Create Non-Root Image

⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes

BASH(25 lines)
Code
Loading syntax highlighter...

Exercise 3: Capability Analysis

⭐⭐ Difficulty: Medium | ⏱️ Time: 20 minutes

BASH(16 lines)
Code
Loading syntax highlighter...

Exercise 4: Read-Only Filesystem

⭐⭐⭐ Difficulty: Hard | ⏱️ Time: 25 minutes

BASH(33 lines)
Code
Loading syntax highlighter...

Exercise 5: Complete Hardened Setup

⭐⭐⭐⭐ Difficulty: Expert | ⏱️ Time: 35 minutes

Create a fully hardened containerized application:

YAML(17 lines)
Code
Loading syntax highlighter...

🎤 Senior-Level Interview Questions

Q1: Why should containers run as non-root?

Strong Answer:

"Running as root in containers is dangerous for several reasons:

1. Container escape amplification: If an attacker exploits a container escape vulnerability, they're root on the host. Non-root limits the damage.

2. Defense in depth: Even without escape, root can do more damage inside the container - modify binaries, read sensitive files, etc.

3. Shared kernel: Container root is the same UID as host root. Kernel doesn't distinguish between 'root in container' and 'root on host'.

4. Best practices: Most security frameworks (CIS Docker Benchmark, NIST) require non-root containers.

Implementation:

DOCKERFILE(4 lines)
Code
Loading syntax highlighter...

Exceptions:

Some software legitimately needs root (port 80, raw sockets)
Solution: Drop privileges after initialization
Or use capabilities instead of full root

I always start with non-root and only add specific capabilities if absolutely necessary."

Q2: Explain Linux capabilities and how they relate to container security.

Strong Answer:

"Linux capabilities break root's all-or-nothing power into 40+ granular permissions.

Traditional model:

UID 0 = can do everything
Other UIDs = restricted

Capabilities model:

Each capability grants specific power
Can have root UID without capabilities
Can have capabilities without root UID

Key capabilities:

CAP_NET_BIND_SERVICE - Bind ports < 1024
CAP_NET_RAW - Raw sockets (ping)
CAP_SYS_ADMIN - Mount, namespace ops (dangerous!)
CAP_CHOWN - Change file ownership

Docker default: Containers get ~14 capabilities by default. Not minimal, but not full root either.

Best practice:

YAML(4 lines)
Code
Loading syntax highlighter...

Why drop all:

Reduces attack surface
Most apps don't need any capabilities
Forces explicit thought about requirements

The dangerous ones:

SYS_ADMIN - Basically root, avoid
NET_ADMIN - Network configuration
SYS_PTRACE - Process debugging

Never use --privileged, which grants ALL capabilities plus more."

Q3: How do you implement defense in depth for containers?

Strong Answer:

"Defense in depth means multiple security layers, so compromising one doesn't mean full compromise:

Layer 1 - Image security:

Minimal base image (distroless, alpine)
No secrets in image
Regular vulnerability scanning
Image signing

Layer 2 - Runtime security:

Non-root user
Read-only filesystem
Drop all capabilities
no-new-privileges

Layer 3 - System call filtering:

Seccomp profiles (default or custom)
Block dangerous syscalls

Layer 4 - Mandatory Access Control:

AppArmor or SELinux profiles
Restrict file access patterns

Layer 5 - Network security:

Isolated networks
Network policies
No unnecessary ports

Layer 6 - Resource limits:

Memory limits (prevent DoS)
CPU limits
PID limits

Example implementation:

YAML(15 lines)
Code
Loading syntax highlighter...

Each layer stops different attack types. Even if one fails, others provide protection."

Q4: What is the difference between --privileged and specific capability grants?

Strong Answer:

"They're vastly different in security impact:

--privileged:

Grants ALL capabilities (~40+)
Gives access to all host devices
Disables seccomp filtering
Disables AppArmor
Container is essentially root on host
Use case: Almost never in production

Specific capabilities:

Grant only what's needed
Keep other restrictions in place
Seccomp still active
AppArmor still active
Principle of least privilege

Example - needing raw sockets:

BASH(5 lines)
Code
Loading syntax highlighter...

When people think they need privileged:

Docker-in-Docker → Use Docker socket mount or DinD image properly
Device access → Mount specific device
Network tools → Add NET_ADMIN or NET_RAW
Debugging → Add SYS_PTRACE temporarily

I've never seen a legitimate production use case for --privileged. It's a security red flag in any deployment."

Q5: How would you secure a container that needs to run as root?

Strong Answer:

"Sometimes root is truly needed (legacy apps, port 80, specific syscalls). Here's how to minimize risk:

1. Drop to non-root after initialization:

DOCKERFILE(3 lines)
Code
Loading syntax highlighter...

BASH(7 lines)
Code
Loading syntax highlighter...

2. Use capabilities instead:

YAML(5 lines)
Code
Loading syntax highlighter...

3. Maximum restrictions:

YAML(12 lines)
Code
Loading syntax highlighter...

4. User namespace remapping:

JSON(4 lines)
Code
Loading syntax highlighter...

Root in container maps to unprivileged user on host.

5. Additional isolation:

Custom seccomp profile
AppArmor profile
Read-only mounts
Network isolation

The goal: If container is compromised, attacker gets minimal access even as root in container."

📝 Summary & Key Takeaways

Security Checklist

Control	Implementation
Non-root user	`USER appuser` in Dockerfile
Drop capabilities	`cap_drop: [ALL]`
Read-only fs	`read_only: true`
No new privileges	`no-new-privileges:true`
Resource limits	`memory`, `cpus` limits
Minimal base	distroless, alpine

Security Hierarchy

NEVER: --privileged, host mounts, docker.sock
AVOID: Running as root, unnecessary capabilities
ALWAYS: Non-root, dropped caps, read-only, limits

📋 Quick Reference

Docker Run Security Flags

BASH(15 lines)
Code
Loading syntax highlighter...

Compose Security

YAML(10 lines)
Code
Loading syntax highlighter...

📅 Review Schedule

Day	Task	Time
Day 1	Review security layers diagram	10 min
Day 3	Audit a running container's security	15 min
Day 7	Convert an image to non-root	20 min
Day 14	Implement read-only filesystem	25 min
Day 30	Full security hardening of production service	45 min

Previous	Current	Next
Part 11: Logging & Observability	Part 12: Container Security	Part 13: Debugging Containers

📋 At a Glance

🎯 What You'll Learn

🔥 Production Story: The Container Escape

🧠 Mental Model: Defense in Depth

🔬 Deep Dive

Non-Root Containers

no-new-privileges

Linux Capabilities

Read-Only Filesystem

Seccomp Profiles

AppArmor

Complete Hardened Container

⚠️ Common Mistakes

Mistake 1: Running as Root

Mistake 2: Using --privileged

Mistake 3: Mounting Sensitive Host Paths

🐛 Debug This: The Permission Denied Mystery

💻 Exercises

Exercise 1: Audit Container Security

Exercise 2: Create Non-Root Image

Exercise 3: Capability Analysis

Exercise 4: Read-Only Filesystem

Exercise 5: Complete Hardened Setup

🎤 Senior-Level Interview Questions

Q1: Why should containers run as non-root?

Q2: Explain Linux capabilities and how they relate to container security.

Q3: How do you implement defense in depth for containers?

Q4: What is the difference between --privileged and specific capability grants?

Q5: How would you secure a container that needs to run as root?

📝 Summary & Key Takeaways

Security Checklist

Security Hierarchy

📋 Quick Reference

Docker Run Security Flags

Compose Security

📅 Review Schedule

📚 Series Navigation

Tags: