The Case of the Full Disk
A production server sends an alert: Disk usage above 90%. Your task is not to panic-delete files. Your task is to identify which filesystem is full, locate the largest directories and files, and choose a safe fix.
1. Observe
Run df -h. This tells you which mounted filesystem is short of space.
2. Investigate
Run du and find. These tell you which directories or files are consuming space.
3. Fix safely
Confirm ownership, retention, and application impact before compressing, rotating, archiving, or deleting files.
Learning objectives
By the end, beginners should be able to explain disk layout and run first-level free-space troubleshooting commands safely.
Partitions
Explain why disks are divided and how partitions appear as devices like /dev/sda1, /dev/nvme0n1p2, or logical volumes.
Mount points
Understand why /var, /home, or /data may live on different filesystems.
Usage commands
Use df -h, du -sh, lsblk, and find with confidence.
Troubleshooting
Follow a clean investigation path instead of running dangerous deletion commands.
Storage concepts made simple
Use the analogy first, then connect it to the real Linux mechanism.
Disk
The building. Example: physical disk, virtual disk, SAN LUN, cloud volume.
Partition
A room inside the building. Example: /dev/sda1, /dev/nvme0n1p1.
Filesystem
The shelves and index system. Example: ext4, XFS, Btrfs, ZFS dataset.
Mount point
The door/path used to enter the room. Example: /, /boot, /var, /data.
What happens during mounting?
Linux attaches a filesystem to an existing directory. After that, accessing that directory shows the mounted filesystem's contents.
/dev/sdb1 ──mounted at──▶ /dataIf /data is expected to be a separate disk but is not mounted, files written to /data may fill the root filesystem instead.
Why separate mount points exist
Production systems often isolate fast-growing areas such as logs, databases, backups, or application data.
/ operating system
/boot boot files and kernels
/var logs, caches, spools
/home user data
/data application or business dataClick a mount point
See what each mount point usually stores, how it fills, and which command you would start with.
/ root — operating system base
Contains the top-level filesystem tree. If root fills, many services can fail because temporary files, package operations, and logs may stop working.
df -h /
sudo du -xhd1 /The four commands you need first
Use this mental model: lsblk = layout, df = filesystem fullness, du = directory weight, find = exact large files.
df -h — disk free by mounted filesystem
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 50G 42G 8.0G 84% /
/dev/sdb1 100G 95G 5.0G 95% /data
tmpfs 1.9G 0 1.9G 0% /run/user/1000Use it when: you need to know which filesystem or mount point is full.
Beginner trap: df does not show which folder is large. It shows filesystem-level usage.
lsblk — block device layout
$ lsblk
NAME SIZE TYPE MOUNTPOINTS
sda 60G disk
├─sda1 1G part /boot
└─sda2 59G part /
sdb 100G disk
└─sdb1 100G part /dataUse it when: you need to see disks, partitions, logical volumes, and mountpoints in a tree.
Beginner trap: lsblk is not a free-space report. Pair it with df -h.
du -sh /var/log — size of one directory
$ du -sh /var/log
2.4G /var/logUse it when: df says a mount is full and you need to check a specific directory.
Beginner trap: Permission errors may hide some sizes. Use sudo when investigating system directories.
sudo find /var/log -type f -size +10M — find large files
$ sudo find /var/log -type f -size +10M
/var/log/syslog.1
/var/log/journal/ab12cd34/system.journal
/var/log/audit/audit.log.1Use it when: you need the exact files larger than 10 MB under /var/log.
Beginner trap: Finding a file does not mean it is safe to delete. Check the owner/application and retention policy.
| Question | Best first command | Why |
|---|---|---|
| Which filesystem is full? | df -h | Shows size, used, available, percentage, and mount point. |
| What disks and partitions exist? | lsblk | Shows block device hierarchy and mountpoints. |
| How large is this directory? | du -sh /path | Summarizes directory usage in human-readable format. |
| Which big files are under this path? | find /path -type f -size +10M | Lists regular files above the selected size threshold. |
df vs du
This is the most important beginner distinction in disk troubleshooting.
df -h: building-level meter
Like checking how full the entire room is. It answers: which mounted filesystem is full?
du -sh: cupboard-level weighing scale
Like weighing one cupboard or shelf. It answers: which directory is taking space?
df and du disagree: common causes include deleted files still held open by a process, filesystem reserved blocks, permissions, mounted sub-filesystems, snapshots, or container overlays.
Free-space troubleshooting without chaos
Follow this flow. It prevents the classic mistake: deleting random files before understanding the mount point.
Confirm the alert
df -hFind the Use% and Mounted on columns. Above 80% deserves attention; above 90% is usually urgent.
Identify the mount
df -h /var/log
lsblkVerify whether the path is on root, /var, /data, or another filesystem.
Find heavy directories
sudo du -xhd1 /var | sort -h-x stays on one filesystem; useful when multiple mounts exist below a path.
Find large files
sudo find /var/log -type f -size +10MStart with logs, caches, dumps, backups, and old archives. Confirm before removal.
Check hidden causes
df -i
sudo lsof +L1
journalctl --disk-usageInodes can run out, deleted files can still consume space, and systemd journals can grow.
Fix and verify
df -h
sudo du -sh /var/logAfter cleanup, verify with the same command that showed the problem.
rm -rf blindly. Never randomly delete from /boot, /usr, /lib, database directories, application data paths, or unknown production folders.
Real-world incident cards
Open each case and decide which commands help you prove the cause.
Incident 1: Logs filled /var
Symptoms: application stops writing logs, package updates fail, /var is 96% full.
df -h
sudo du -xhd1 /var
sudo find /var/log -type f -size +10M
journalctl --disk-usageSafe fix idea: compress/archive old logs, adjust logrotate, or vacuum journal logs according to retention policy.
Incident 2: /data was not mounted
Symptoms: application data was supposed to go to a separate disk, but root filled instead.
lsblk
df -h /data
mount | grep ' /data 'Root cause: the mount failed, so writes to /data went into the normal root filesystem directory.
Incident 3: /boot full after kernel updates
Symptoms: update fails while installing a new kernel.
df -h /boot
ls -lh /bootSafe fix idea: use the distribution package manager to remove old unused kernels. Do not manually delete random kernel files unless you know exactly what you are doing.
Incident 4: Disk has space, but files cannot be created
Symptoms: df -h shows free space, but applications say “No space left on device”.
df -i
sudo find /var -xdev -type f | wc -lLikely cause: inode exhaustion — too many small files.
Beginner lab: investigate disk usage
Run these on Ubuntu, Debian, Rocky Linux, RHEL, or most systemd-based Linux VMs. Outputs vary by VM, disk type, filesystem, and permissions.
sudo for system directories. These commands inspect files; they do not delete anything.
Lab 1 — Check filesystem usage
df -hYou should see filesystems such as root /, possibly /boot, /home, /var, /data, and memory filesystems like tmpfs.
Lab 2 — View disk and partition layout
lsblkYou should see a tree. Names vary: sda, vda, xvda, or nvme0n1 are all normal depending on virtualization/cloud/hardware.
Lab 3 — Check log directory size
du -sh /var/logYou get one summarized size. If permission warnings appear, rerun as sudo du -sh /var/log.
Lab 4 — Find log files larger than 10 MB
sudo find /var/log -type f -size +10MIf files appear, they are candidates for review — not automatic deletion. Check application ownership and retention requirements first.
Lab progress
0 of 4 lab checkpoints completed.
When the obvious commands are not enough
These are common production surprises. Keep them as “next checks,” not beginner starting points.
Inodes full
df -iA filesystem can run out of file entries even if it still has data blocks free.
Deleted files still open
sudo lsof +L1A process can keep using disk space from a file that was deleted but is still open.
Systemd journal size
journalctl --disk-usageOn systemd systems, persistent journal logs can consume space.
Interactive viewer
sudo ncdu /varncdu is excellent for exploration, but it may need installation and careful deletion choices.
Diagnose a full /var filesystem
Your server alert says: /var is 94% full. Write a short incident note using this structure:
Commands to run
df -h
lsblk
sudo du -xhd1 /var
sudo find /var/log -type f -size +10MIncident note template
Problem:
Evidence:
Largest path/files:
Risk:
Safe cleanup recommendation:
Verification command:Quick quiz
Click an answer. The correct answer will be highlighted.
df to show high usage after a file was deleted?Validation checklist
Use this to confirm that the learner did not only run commands, but understood what the outputs mean.
Learner can explain
- Difference between disk, partition, filesystem, and mount point.
- Why
df -handdu -shanswer different questions. - Why
lsblkis layout, not free-space usage. - Why large files should not be deleted blindly.
Safe mini validator
#!/usr/bin/env bash
set -euo pipefail
printf '\n== Filesystem usage ==\n'
df -h
printf '\n== Block layout ==\n'
lsblk
printf '\n== /var/log size ==\n'
du -sh /var/log || sudo du -sh /var/log
printf '\n== Large /var/log files > 10M ==\n'
sudo find /var/log -type f -size +10M -printThis validates command availability and learner observations. It does not delete or modify files.
/var/log is huge, should we delete all logs immediately?” Correct answer: no. First identify file ownership, business retention, application impact, and safer cleanup options such as rotation, compression, archiving, or approved deletion.