SoFunction
Updated on 2025-04-14

Docker garbage cleaning method

1.docker image and container deletion

docker images    #See so mirrordocker ps      #View running containersdocker ps -a   #View all containers (whether running or not)docker ps -a -q   #plus-q only displays id
docker stop container name orid   #Stop the containerdocker rm container name orid    #Delete containerdocker rm -f container name orid    #Forced Delete Containerdocker rmi Mirror name plus tag orid   #Delete the mirrordocker rmi -f Mirror name plus tag orid  #Forced Delete the Mirrordocker rmi $(docker images | grep "<none>" )  
#Use grep matching to delete all containers that are nonedocker rmi $(docker images -q)
# To delete allimage

2. Docker disk space cleaning

1. Use docker system to clean up

The docker system df command viewes the disk space occupied by docker, similar to the linux df command

TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          18        6         2.508GB   1.892GB (75%)
Containers      19        2         275.8MB   275.8MB (99%)
Local Volumes   95        3         140.9kB   140.9kB (100%)
Build Cache     0         0         0B        0B

As you can see, the Docker image occupies 2.5GB of disks, the Docker container occupies 275.8MB of disks, and the Docker data volume occupies a little disk.

  • The docker system prune command can be used to clean disks, delete closed containers, useless data volumes and networks, and dangling images (i.e. tag-free images).
  • The docker system prune -a command cleans up more thoroughly, and can delete all Docker images without containers. This is very dangerous, usually not

2. Manually clean Docker images/containers/data volumes

For older versions of Docker (before version 1.13), there is no docker system command

For docker, there is not much container image space, and the log is a big one

So deleting logs is the best way to do it (you need to consider it clearly before deleting)

On Linux, all related files of Docker, including images, containers, etc., are generally saved in the /var/lib/docker/ directory:

Use the du command to view the occupation

du -hs /var/lib/docker/
3.4G    /var/lib/docker/

Not a lot just built, but it's still a bit big

Use the du command to continue viewing and you can locate the directory that really occupies so many disks

3.4G    /var/lib/docker/overlay2

It is found that it is occupied by overlay2. Overlay2 is one of Docker's default storage drivers, mainly used to manage the file system layer of containers (Layer)

It stores data from the mirror layer and container layer. This is generally not cleaned directly, and it is cleaned by deleting containers and mirrors.

However, last time a service on the cloud server went down. After checking, I found that there was insufficient space, so the database could not be connected. I quickly released some resources

Later I found that /var/lib/docker/containers accounted for more than 200 G

It is true that the logs of a competition platform occupy more than 190G. It's right. It's so strange that there are so many requests every day, but it's not too many logs.

Use the truncate command to "clear" the log file of the container:

truncate -s 0 /var/lib/docker/containers/containerid/*-

Of course, this command is only temporary, and the log files will rise again sooner or later. To fundamentally solve the problem, you need to limit the log file size of the container. This can be achieved by configuring the max-size of the log. You can use the container's docker-compose configuration file. I am the gzctf platform and add logging configuration for gzctf and db services:

 logging:
      driver: "json-file"
      options:
        max-size: "10g"
        max-file: "3"

After restarting the container, the size of its log file is limited to 30GB, divided into three files, so there is no need to worry anymore!

Restart docker and refresh disk status

3. Docker directory /var/lib/docker/containers file is too large

Without rebuilding the container, the log files will be added by default. Over time, it will gradually occupy the hard disk space of the server and the memory consumption will continue to increase. We need to control it.

1. Find out files that occupy larger disks

Docker's log file exists in the /var/lib/docker/containers directory. The following commands can list the log folders in ascending order.

sudo du -d1 -h /var/lib/docker/containers | sort -h

2. Clean up individual files

Clean up which container's log is too big

sudo sh -c "cat /dev/null > File Decision Address"

This is just a temporary solution, and it is best to control the size of the log when creating the container.

When starting the container, we can control the number of files in the log and the size of a single file through parameters.

# max-size Maximum value# max-file Maximum log count$ docker run -it --log-opt max-size=10g --log-opt max-file=3 redis
#existdockerfileIn the directory

One or two containers are fine, but if there are many containers that need to be managed, it will be inconvenient. It is best to manage them in a unified manner.

3. Global configuration

Create or modify the file /etc/docker/ and add the following configurations

{
    "log-driver":"json-file",
    "log-opts":{
        "max-size" :"50m","max-file":"1"
    }
}

This configuration is used to set Docker log driver and log options. The main function is to control the generation method and size limit of Docker container log files. The following explains the meaning of each part in detail:

Overall function:

This configuration will usually be placed/etc/docker/In the file, the global configuration belonging to the Docker daemon process takes effect on the newly created container. The purpose is to standardize the storage format, size and number of files of container logs to prevent excessive disk space from being occupied by log files.

Specific configuration items explanation:

1. "log-driver":"json-file"

  • log-driverRefers to the type of log driver used by Docker. The log driver determines the storage and output methods of container logs.
  • json-fileIndicates the use of JSON files as log storage format. When the container generates a log, the log content will be recorded into the file in JSON format. Each log entry is a JSON object, containing information such as timestamps, log levels, log messages, etc. This format facilitates the machine to parse and process log data.

2. "log-opts"

  • log-optsIt is a specific option for log driver to further configure log storage and management rules.

3. "max-size" :"50m"

  • max-sizeThe maximum size of a single log file is defined. Here is set as"50m", i.e. 50MB.
  • When a single log file reaches 50MB, Docker will automatically create a new log file to continue logging, thereby avoiding the single log file being too large.

4. "max-file":"1"

  • max-fileSpecifies the maximum number of log files. Here is set as"1", means that only 1 log file is retained at most.
  • When a new log file is created, the old log file will be automatically deleted to ensure that the number of log files does not exceed the set value. Combinedmax-sizefor 50MB andmax-fileWith a setting of 1, the maximum disk space occupied by container log files will not exceed 50MB.

Example summary:

This configuration enables the logs of the Docker container to be stored in JSON file format, with a maximum of 50MB per log file and a maximum of 1 log file is retained. This can effectively control the disk space occupied by container logs and prevent the disk resources from being exhausted due to the continuous growth of log files. However, be aware that this configuration will not take effect on existing containers, and only newly created containers will follow these rules.

Then restart the Docker service

sudo systemctl daemon-reload
sudo systemctl restart docker

However, existing containers will not take effect and need to be rebuilt!

besides

{
    "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
    "registry-mirrors": [
        "",
    "",
     "",
    "https://docker.",
    "",
    "",
    "",
    "",
    "",
    ""
    ],
    "log-driver":"json-file",
    "log-opts":{
        "max-size" :"20m","max-file":"3"
    }
}

This configuration is a Garbage Collection (GC) related configuration of the Docker builder (Builder), usually in the Docker configuration file (such as) is used in, and the following explains the meaning of each part in detail.

Overall function overview:

This configuration is mainly used to control temporary files generated during Docker construction and garbage collection mechanisms at the intermediate layer. The purpose is to effectively manage disk space and avoid the large amount of temporary data generated during the construction to occupy too much disk.

Specific configuration items explanation:

1."builder"

  • This is a top-level configuration item that sets the various parameters of the Docker builder.
  • The builder is responsible for performing the build tasks of the Docker image, and the contents in this configuration block affect the builder's behavior during the build process.

2."gc"

  • This isbuilderThe subconfiguration item under the name represents the settings related to Garbage Collection.
  • The garbage collection mechanism regularly cleans up temporary files and intermediate layers that are no longer needed during the build process, freeing up disk space.

3."defaultKeepStorage": "20GB"

  • defaultKeepStorageDefines the size of disk space that the builder reserves by default after performing a garbage collection operation.
  • Here is set as"20GB", meaning that when garbage collection, the builder tries to clean up temporary data that is no longer used until the build process takes up no more than 20GB of disk space. That is, even with a large number of cleanable intermediate layers and temporary files, the builder will retain at least 20GB of disk space for subsequent build operations.

4."enabled": true

  • enabledis a boolean value that enables or turns off the builder's garbage collection function.
  • Set astrueIndicates that the garbage collection mechanism is enabled, the builder will periodically check and clean up temporary data that is no longer used; if set tofalse, the garbage collection mechanism will be disabled, and temporary data generated during the build process will not be automatically cleaned, which may cause a large amount of disk space to be occupied.

Summarize

This configuration enables the garbage collection function of the Docker builder and stipulates that at least 20GB of disk space will be retained for building operations after garbage collection. By rationally configuring the garbage collection mechanism, disk space can be effectively managed to ensure the stability and efficiency of the Docker construction process.

The above is personal experience. I hope you can give you a reference and I hope you can support me more.