Optimize Docker Storage

January 7, 2024 (2mo ago)

How to clear Docker cache and free up space on your system

Docker persists build cache, containers, images, and volumes to disk. Over time, these things can take up a lot of space on your system, either locally or in CI. In this post, we'll look at the different Docker artifacts that can take up space on your system, how to clear them individually, and how to use docker system prune to clear Docker cache.

How much disk space is Docker using?

The first step is knowing the disk usage of Docker. We can use the docker system df command to get a breakdown of how much disk space is being taken up by various artifacts.

docker system df

Removing containers from the Docker cache

We can use the docker container prune command to clear the disk space used by containers. This command will remove all stopped containers from the system.

We can omit the -f flag here and in subsequent examples to get a confirmation prompt before artifacts are removed.

docker container prune -f
Deleted Containers:
Total reclaimed space: 834.6kB

Which containers are unused?

We can see the IDs of unused containers by running the docker ps command with filters on the status of the container. A container is unused if it has a status of exited or dead.

docker ps --filter status=exited --filter status=dead -q

Note: If we want to know the size of the unused container, we can replace the -q flag with -s to get the size and other metadata about the container.

Removing all containers

If we want to remove all containers from the system, we can stop any running containers and then use the same prune command. Do this by feeding the output of docker ps -q into the docker stop or docker kill command if you want to kill the container forcibly.

docker stop $(docker ps -q)
docker container prune

Another option is the docker rm command, which can be used with docker ps -a -q to remove all containers.

docker rm $(docker ps -a -q)

Note: The docker rm command forces the removal of a running container via a SIGKILL signal. This is the same as the docker kill command. The docker ps -a -q command will list all containers on the system, including running containers, and feed that into the docker rmcommand.

Removing images

Docker images can take up a significant amount of disk space. We accumulate new images when base images change or build new ones via docker build, for example. We can use the docker image prune command to remove unused images from the system.

By default, it only removes dangling images, which are not associated with any container and don't have tags.

docker image prune -f
Deleted Images:
deleted: sha256:6f096c9fa1568f7566d4acaf57d20383851bcc433853df793f404375c8d975d6
Total reclaimed space: 2.751GB

We reclaimed over 2.7 GB of space by removing dangling images. But, if we recall from the output of our docker system df command, we have 34.15 GB of reclaimable images.

Where is the rest of that space coming from? These are images on our system that are tagged or associated with a container. We can run the docker image prune- a command to force the removal of these images as well, assuming they're unused images.

docker image prune -a -f
Deleted Images:
untagged: k8s.gcr.io/etcd:3.4.13-0
untagged: k8s.gcr.io/etcd@sha256:4ad90a11b55313b182afc186b9876c8e891531b8db4c9bf1541953021618d0e2
deleted: sha256:0369cf4303ffdb467dc219990960a9baa8512a54b0ad9283eaf55bd6c0adb934
deleted: sha256:f3cecccfe2bea1cbd18db5eae847c3a9c8253663bf30a41288f541dc1470b41e
Total reclaimed space: 22.66GB

In this way, we remove all unused images not associated with a container, not just the dangling ones.

Removing volumes

Volumes are never cleaned up automatically in Docker because they could contain valuable data. But, if we know that we no longer need the data in a volume, we can remove it with the docker volume prune command. This removes all anonymous volumes not used by any containers.

docker volume prune -f
Total reclaimed space: 0B

Interestingly, we see that we didn't reclaim any space. This is because we have volumes that are associated with containers. We can see these volumes by running the docker volume ls command.

local     0a44f085adc881ac9bb9cdcd659c28910b11fdf4c07aa4c38d0cca21c76d4ac4
local     0d3ee99b36edfada7834044f2caa063ac8eaf82b0dda8935ae9d8be2bffe404c

We get an output that shows the driver and the volume name. The command docker volume prune only removes anonymous volumes. These volumes are not named and don't have a specific source from outside the container. We can use the docker volume rm -a command to remove all volumes.

docker volume prune -a -f
Deleted Volumes:
Total reclaimed space: 15.31GB

Removing build cache

To remove the Docker build cache, we can run the docker buildx prune command to clear the build cache of the default builder.

docker buildx prune -f
ID                                        RECLAIMABLE   SIZE        LAST ACCESSED
pw11qgl0xs4zwy533i2x61pef*                true          54B         12 days ago
y37tt0kfwn1px9fnjqwxk7dnk                 true          0B          12 days ago
sq3f8r0qrqh4rniemd396s5gq*                true          154.1kB     12 days ago
Total:  5.806GB

If we want to remove the build cache for a specific builder, we can use the --builder flag to specify the builder name.

docker buildx prune --builder builder-name -f

Removing everything with docker system prune

The equivalent of a docker clean all is better known as Docker prune. We can remove all unused artifacts Docker has produced by running docker system prune. This will remove all unused containers, images, networks, and build cache.

docker system prune -f
Deleted Images:
deleted: sha256:93477d5bde9ef0d3d7d6d2054dc58cbce1c1ca159a7b33a7b9d23cd1fe7436a3
Deleted build cache objects:
Total reclaimed space: 419.6MB

By default, Docker prune will not remove volumes and only removes dangling Docker images. We can use the --volumes flag to remove volumes as well. We can also add the -a flag again to remove all images not associated with a container.

docker system prune --volumes -af

- Design and development by me.