How to Use the Linux lsof Command to Find Deleted Files Held Open

In Linux, a common and highly frustrating scenario for system administrators occurs when a massive log file or database dump is accidentally deleted, but the disk space is not freed up. When you run df -h, the disk still shows as 99% full, yet the file is nowhere to be found when you run ls. This happens because of how the Linux virtual file system operates: if a running process still has a file open when you execute the rm command, Linux removes the file’s directory entry (making it invisible to ls) but leaves the actual data on the disk until the process releases its file descriptor. To reclaim the space, you must find and terminate the process holding the ghost file open using the lsof command.

Understanding lsof and Deleted Files

The lsof (List Open Files) command is a powerful diagnostic tool that queries the Linux kernel for every single file descriptor currently open by every running process. Because a “file” in Linux can be a text file, a network socket, or a hardware device, the output of a raw lsof command is usually overwhelming. However, lsof includes a specific flag designed entirely to hunt down the invisible “ghost” files that are causing your disk space issues.

Executing the lsof Search

Because you are searching for files held open by potentially privileged system services (like a web server or database daemon), you must run lsof with root privileges.

1. Open your terminal or connect via SSH.
2. Execute the following command: sudo lsof +L1

The +L1 flag specifically tells lsof to filter its massive output and only display files that have a link count of less than 1. In the Linux file system, a link count of 0 means the file has been deleted from the directory structure but is still being held active in memory by a process.

Reading the Output

The command will output a table that looks similar to this:

COMMAND    PID   USER   FD   TYPE DEVICE SIZE/OFF NLINK      NODE NAME
apache2  14231   root   4w   REG  252,1  5.2G     0   1234567 /var/log/apache2/access.log (deleted)

• COMMAND: The name of the program holding the file open (e.g., apache2, mysqld, java).
• PID: The Process ID. This is the crucial number you need to kill the process.
• SIZE/OFF: The actual size of the invisible file still consuming your disk space (e.g., 5.2G).
• NAME: The original path and name of the file before it was deleted, helpfully appended with the tag “(deleted)”.

Reclaiming the Disk Space

Once you have identified the process holding the massive deleted file open, you have two options to reclaim the storage space.

1. Restart the Service (Recommended): The safest and most graceful method is to simply restart the service that owns the process. In the example above, restarting the web server will force it to drop all file descriptors and start fresh. sudo systemctl restart apache2 As soon as the service restarts, the kernel will permanently delete the ghost file and the disk space will instantly return to normal.
2. Kill the Process: If you cannot restart the service through systemctl, or if it is a rogue background script, you can forcibly kill the specific Process ID identified by lsof. sudo kill -9 14231

How to Prevent This Issue

The best way to avoid the “deleted but not freed” problem is to never use the rm command on active log files. Instead of deleting an active log, you should “truncate” it. Truncating empties the contents of the file without deleting the actual file descriptor, meaning the process can continue writing to it without interruption.

To truncate a file safely while it is in use, run: > /path/to/logfile.log