Skip to main content
DevOps Insightsguides & answers

Docker · troubleshooting

no space left on device

Docker has filled the filesystem, and almost always with things nothing is using: build cache, dangling images, stopped containers and orphaned volumes. Deleting containers does not reclaim their volumes, which is why the space does not come back.

Run this first

step 1 of 3
docker system df

Look for The largest RECLAIMABLE value. That is where the space is, and it is usually build cache or volumes.

One command that attributes the usage across images, containers, volumes and build cache, with a reclaimable figure for each.

Work out which cause you have

A few questions to narrow the list. Every answer ends in a command that confirms or rules the cause out — this cannot see your cluster, so nothing here is a certainty until you have checked.

Narrow it down

0 answered · nothing is sent anywhere

Is the full filesystem the one holding Docker's data root?

Check with `df -h $(docker info --format '{{.DockerRootDir}}')`.

Cause space

5 of 5 still possible

  • Build cache has grown without limitCommon
  • Dangling images from repeated rebuildsCommon
  • Volumes left behind by removed containersCommon
  • Container logs have grown unboundedOccasional
  • The full filesystem is not Docker'sRare

Nothing ruled out yet. Answer the question above and the branches your answer eliminates will strike through here.

Check it with a tool

Or diagnose it manually

In this order. The first command usually contains the whole answer.

  1. Step 1

    docker system df

    One command that attributes the usage across images, containers, volumes and build cache, with a reclaimable figure for each.

    Look for The largest RECLAIMABLE value. That is where the space is, and it is usually build cache or volumes.

  2. Step 2

    df -h $(docker info --format '{{.DockerRootDir}}')

    Confirms the full filesystem is Docker's before you delete anything.

    Look for Use% at or near 100 on the filesystem holding Docker's data root.

  3. Step 3

    docker system df -v | head -40

    The per-object breakdown, which names the specific volumes and images consuming the space.

    Look for Individual volumes or images large enough to explain the shortfall.

Every cause, and how to fix it

Ordered by how often each one turns out to be the answer.

Build cache has grown without limit

Common

BuildKit keeps cache for every build stage, and by default it keeps it indefinitely. On a machine that builds several times a day this is usually the single largest consumer, and it is entirely reclaimable.

Confirm

docker system df

The Build Cache row. A large TOTAL with an equally large RECLAIMABLE means this is your answer.

Fix

  • Reclaim it with `docker builder prune`. Add `-a` to remove all of it rather than just unused entries.
  • Set a ceiling so it does not recur, rather than pruning by hand when builds start failing.
docker builder prune -a

# Then cap it, in daemon.json:
# { "builder": { "gc": { "defaultKeepStorage": "20GB", "enabled": true } } }

The cap is the actual fix. Pruning is the symptom relief.

Dangling images from repeated rebuilds

Common

Rebuilding a tag moves the tag to the new image and leaves the old one untagged. Those `<none>` images keep every layer they own, and there is no limit on how many accumulate.

Confirm

docker images -f dangling=true --format 'table {{.ID}}	{{.Size}}' | head -20

A long list of untagged images. Each one is holding layers nothing references.

Fix

  • `docker image prune` removes dangling images. `docker image prune -a` also removes tagged images no container uses, which is more aggressive than most people intend.
  • On CI hosts, prune as a scheduled job rather than waiting for a build to fail.
docker image prune

Volumes left behind by removed containers

Common

This is the one that surprises people. Removing a container does not remove its named volumes, because a volume exists precisely so that data survives the container. So the space stays used, and `docker ps -a` shows nothing to explain it.

Confirm

docker volume ls -qf dangling=true | wc -l && docker system df -v | head -30

A non-zero count of dangling volumes, and their sizes in the verbose output. Database volumes are often the largest single objects on the host.

Fix

  • **Read the list before deleting anything.** `docker volume prune` removes data permanently, and a dangling volume can still be the only copy of something — a development database whose container was recreated, for example.
  • Once you have confirmed what they are, `docker volume prune` reclaims them.
# Look first.
docker volume ls -f dangling=true

# Then, if you are sure:
docker volume prune

This is the one destructive step on the page. There is no undo, and the data may not exist anywhere else.

Container logs have grown unbounded

Occasional

With the default json-file logging driver and no rotation configured, a container's log file grows for as long as the container runs. A chatty application over several months can produce tens of gigabytes in a single file.

Confirm

sudo du -sh /var/lib/docker/containers/*/*-json.log 2>/dev/null | sort -h | tail -10

Individual log files measured in gigabytes.

Fix

  • Configure log rotation in the daemon config, which applies to newly created containers.
  • Truncating an existing log frees the space immediately, but only until it grows again — the rotation setting is the fix.
# daemon.json
{ "log-driver": "json-file", "log-opts": { "max-size": "10m", "max-file": "3" } }

Applies to containers created after the daemon restarts, not to existing ones.

The full filesystem is not Docker's

Rare

Worth ruling out before pruning anything. If Docker's data root is on its own partition, a full root filesystem produces the same error while Docker's own usage is fine — and pruning will free nothing.

Confirm

df -h $(docker info --format '{{.DockerRootDir}}') && df -h /

Which of the two is actually full. If Docker's data root has space and root does not, pruning images is not the fix.

Fix

  • Find the real consumer on the full filesystem with `du -sh /* | sort -h` before touching Docker.

Understanding it properly

Skip this if you are mid-incident — the working part of the page is above. Worth reading afterwards, because understanding the mechanism is what stops the next one.

What is actually happening

Docker accumulates by design. Every build leaves cache, every rebuilt image leaves the previous layers dangling, every stopped container keeps its writable layer, and every named volume outlives the container that created it.

None of that is a leak — it is what makes rebuilds fast and data durable. It just has no automatic ceiling, so on a machine that builds regularly it grows until the disk is full.

The important asymmetry is volumes. Removing a container removes its writable layer but deliberately does not remove named volumes, because the whole point of a volume is to survive the container. So `docker rm` frees less than people expect, and the missing space is usually in volumes nothing references any more.

It is also worth checking that the full filesystem is actually Docker's. On a host with a separate partition for Docker's data root, the error can come from a different filesystem entirely.

How to tell this is your problem
WhereWhat you see
docker build / docker run`write /var/lib/docker/...: no space left on device`
docker system dfLarge RECLAIMABLE figures against Build Cache, Images or Local Volumes.
df -hThe filesystem holding Docker's data root at or near 100%.

How to know it is actually fixed

  • `docker system df` shows the reclaimable figure substantially reduced.
  • `df -h` shows the filesystem back below its threshold.
  • The build or run that failed now completes.

Stopping it happening again

  • Cap the build cache in the daemon config. This is the single highest-value setting on a build host, and it removes the most common cause entirely.
  • Configure log rotation before you need it. The default is unbounded, which is fine for a laptop and not for anything long-lived.
  • Prune on a schedule rather than in response to a failure — a weekly `docker system prune` on a build host prevents the whole class of problem.
  • Treat volume pruning as a separate, deliberate act with the list read first. Everything else here is safe; that one deletes data.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Sources

Behaviour described here is drawn from official documentation. Where a figure could not be confirmed on an official page it is attributed in the text rather than stated as canonical.

Related

Container exits immediatelyContainersLinux
no space left on device — causes, diagnosis and fix | DevOps Insights