Skip to content
1012

Container logs filling up disk space in /var/lib/docker

Summary

In some NAbox versions, the compose.yaml file may be missing the logging directive for one or more containers. Without this directive, Docker does not apply any log rotation, and container logs can grow unbounded inside /var/lib/docker, eventually filling up the root partition.

Identifying the problem

If NAbox behaves unexpectedly or upgrades fail, first check available disk space:

df -h /

If the root partition is nearly full, the culprit is often container log files stored under /var/lib/docker. Use du to identify which directories are consuming the most space:

sudo du -hs /var/lib/docker/*

Then drill down into the containers directory where Docker stores per-container logs:

sudo du -hs /var/lib/docker/containers/*

You can also use the Docker CLI to inspect the log file path and size for each running container:

docker inspect --format='{{.LogPath}} {{.Name}}' $(docker ps -q) | \
  while read logpath name; do
    size=$(sudo du -sh "$logpath" 2>/dev/null | cut -f1)
    echo "$size  $name  $logpath"
  done

Freeing disk space

Once you have identified the container(s) with large log files, you can truncate the log without restarting the container:

sudo truncate -s 0 <log_path>

For example, if the havrest container log is too large:

sudo truncate -s 0 $(docker inspect --format='{{.LogPath}}' havrest)

To free logs for all containers at once:

for cid in $(docker ps -q); do
  sudo truncate -s 0 $(docker inspect --format='{{.LogPath}}' "$cid")
done