Image Layers & UnionFS

Note

This module explores the core principles of Image Layers & UnionFS, deriving solutions from first principles and hardware constraints to build world-class, production-ready expertise.

1. The Illusion of a Single File

When you run docker run ubuntu, it feels like you’re booting a virtual machine with a single disk image. But under the hood, Docker images are not single files—they are a stack of read-only layers united by a magical filesystem called UnionFS.

Understanding this architecture is critical for:

  1. Minimizing Image Size: Knowing what adds weight.
  2. Optimizing Build Speed: Leveraging the build cache.
  3. Debugging Storage Issues: Understanding write latency.

2. Anatomy of an Image

A Docker image is an ordered collection of root filesystem changes. Each instruction in a Dockerfile that modifies the filesystem creates a new layer.

The Layer Stack

Imagine a stack of transparencies (overhead projector sheets).

  • Base Layer: The OS files (e.g., Ubuntu rootfs).
  • Middle Layers: Added files or modifications (e.g., apt-get install python).
  • Top Layer (Container Layer): The only writable layer.

When you look down from the top, you see the combined result. If a top layer has a file /etc/app.conf, it obscures any /etc/app.conf in the layers below it.

3. Deep Dive: OverlayFS

Most modern Linux systems use OverlayFS as the storage driver. It uses specific terminology to describe how it merges directories:

  1. LowerDir (Read-Only): The image layers. There can be many of these.
  2. UpperDir (Read-Write): The container layer where your runtime changes go.
  3. Merged (Unified View): What the container process actually sees.
  4. WorkDir: An internal directory used by OverlayFS for atomic operations.

Interactive: The Copy-on-Write Mechanism

UpperDir
(Read-Write Container)
LowerDir 3
/app/config.json
LowerDir 2
Python Runtime
LowerDir 1
Base OS (Ubuntu)

Action: Edit Config

The app tries to write to /app/config.json.

Waiting for input...

4. Copy-on-Write (CoW) Performance

The CoW strategy is brilliant for efficiency (you share the base Ubuntu image across 100 containers), but it comes with a write penalty.

When you modify a file for the first time:

  1. Search: The storage driver searches through the layers.
  2. Copy: It copies the entire file from the lower layer to the upper layer.
  3. Write: It applies the change to the copy.
Warning

**Performance Hit**: If you have a 1GB log file in a lower layer and you append 1 byte to it, Docker must copy the entire 1GB file up to the container layer first. **Always store heavy-write data (databases, logs) in Volumes**, which bypass the storage driver completely.

5. Inspecting Layers with docker history

You can see exactly how your image was built and the size of each layer.

docker history my-app:latest

Output Breakdown:

IMAGE CREATED CREATED BY SIZE
a3b4c5... 2 mins ago CMD ["./app"] 0B
d1e2f3... 2 mins ago COPY . . 15MB
123456... 4 weeks ago /bin/sh -c apt-get update... 120MB
  • Size 0B: Metadata changes (like CMD, ENV, WORKDIR) do not create a new filesystem layer. They only modify the image configuration JSON.
  • Missing ID: Intermediate layers in modern Docker builds might show <missing> if they were built on a different machine or if BuildKit is used, as it handles caching differently.

6. Summary

  1. Immutability: Lower layers are read-only and shared.
  2. Efficiency: Containers only take up space for the diff (UpperDir).
  3. Performance: Write-heavy workloads belong in Volumes, not the container writable layer.