Skip to content

df and du

June 24, 2005

One of the common questions that comes up is why the df (1M) command and the du(1) commands give different information. In fact it comes up so often I’m writing this.

df (1M) shows the amount of disk space used and free based on the return from the statvfs(2) system call.

du(1) walks the file system and adds up all the space used by all the files that it finds.

So the naive expectation is that the data returned from the df (1M)command and the du command when applied to a file system. However they often, often enough for me to write this, don’t.

The most common reason is that the file system contains a file that have been unlinked but are still open. (For those reading at home who are not computer nerds, when a file is removed all that really happens it that the file system no longer shows it. If you are still using the file then all the data associated with that file is still available. Indeed you can continue to grow and shrink the file. When the last process stops using the file (closes it) the file then disappears).

So here is an example. The file system contains just two files, “foo” and “bar”. Both are 2Gigabytes and to start df (1M) and du(1) agree that there is 4.0G of space used.

v4u-4000d-gmp03 239 # du -sh .  4.0G   . v4u-4000d-gmp03 240 # ls -lh total 8392752 -rw——-   1 root     root        2.0G Jun 24 11:33 bar -rw——-   1 root     root        2.0G Jun 24 11:10 foo drwx——   2 root     root        8.0K Jun 24 11:07 lost+found v4u-4000d-gmp03 241 #


Now open the file and keep it open. To do this I run the sleep(1) command with it’s input redirected from the file (the <) and put it in the background (the &):


v4u-4000d-gmp03 241 # sleep 100000 < foo & [1]     7802 v4u-4000d-gmp03 242 # 


Then delete the file with the rm(1) command and run df (1M) and du(1) and you see that they now disagree by 2.0G:


v4u-4000d-gmp03 242 # rm foo v4u-4000d-gmp03 243 # df -h . Filesystem             size   used  avail capacity  Mounted on /dev/dsk/c3t1d0s0      7.8G   4.0G   3.7G    52%    /mnt v4u-4000d-gmp03 244 # du -sh .  2.0G   . v4u-4000d-gmp03 245 # 


If you did not already know what was going on you can now use the find(1) command to search the /proc file system for objects of type “f”, a file, and with no links, which means it is not listed in any normal file system:


v4u-4000d-gmp03 245 # find /proc -type f -links 0 -ls 4589882    8 -rw——-   0 root     root         2048 Jun  4 05:41 /proc/9/fd/6 6772244    8 -rw——-   0 root     root         2048 Jun  4 05:42 /proc/9/fd/2 8     4 2098184 -r——–   0 root     root     2147483648 Jun 24 11:10 /proc/780 2/fd/0 v4u-4000d-gmp03 246 #  

There is the file in Red.


Finally kill(1) the sleep(1) command (since I’m not waiting for 100000 seconds to elapse):


v4u-4000d-gmp03 246 # kill $! v4u-4000d-gmp03 247 # [1] + Terminated               sleep 100000 < foo & v4u-4000d-gmp03 247 # df -h . Filesystem             size   used  avail capacity  Mounted on /dev/dsk/c3t1d0s0      7.8G   2.0G   5.7G    26%    /mnt v4u-4000d-gmp03 248 # 



Tags: Solaris, du, df

Advertisements

From → Solaris

df and du

June 24, 2005

One of the common questions that comes up is why the df (1M) command and the du(1) commands give different information. In fact it comes up so often I’m writing this.

df (1M) shows the amount of disk space used and free based on the return from the statvfs(2) system call.

du(1) walks the file system and adds up all the space used by all the files that it finds.

So the naive expectation is that the data returned from the df (1M)command and the du command when applied to a file system. However they often, often enough for me to write this, don’t.

The most common reason is that the file system contains a file that have been unlinked but are still open. (For those reading at home who are not computer nerds, when a file is removed all that really happens it that the file system no longer shows it. If you are still using the file then all the data associated with that file is still available. Indeed you can continue to grow and shrink the file. When the last process stops using the file (closes it) the file then disappears).

So here is an example. The file system contains just two files, “foo” and “bar”. Both are 2Gigabytes and to start df (1M) and du(1) agree that there is 4.0G of space used.

v4u-4000d-gmp03 239 # du -sh .  4.0G   . v4u-4000d-gmp03 240 # ls -lh total 8392752 -rw——-   1 root     root        2.0G Jun 24 11:33 bar -rw——-   1 root     root        2.0G Jun 24 11:10 foo drwx——   2 root     root        8.0K Jun 24 11:07 lost+found v4u-4000d-gmp03 241 #


Now open the file and keep it open. To do this I run the sleep(1) command with it’s input redirected from the file (the <) and put it in the background (the &):


v4u-4000d-gmp03 241 # sleep 100000 < foo & [1]     7802 v4u-4000d-gmp03 242 # 


Then delete the file with the rm(1) command and run df (1M) and du(1) and you see that they now disagree by 2.0G:


v4u-4000d-gmp03 242 # rm foo v4u-4000d-gmp03 243 # df -h . Filesystem             size   used  avail capacity  Mounted on /dev/dsk/c3t1d0s0      7.8G   4.0G   3.7G    52%    /mnt v4u-4000d-gmp03 244 # du -sh .  2.0G   . v4u-4000d-gmp03 245 # 


If you did not already know what was going on you can now use the find(1) command to search the /proc file system for objects of type “f”, a file, and with no links, which means it is not listed in any normal file system:


v4u-4000d-gmp03 245 # find /proc -type f -links 0 -ls 4589882    8 -rw——-   0 root     root         2048 Jun  4 05:41 /proc/9/fd/6 6772244    8 -rw——-   0 root     root         2048 Jun  4 05:42 /proc/9/fd/2 8     4 2098184 -r——–   0 root     root     2147483648 Jun 24 11:10 /proc/780 2/fd/0 v4u-4000d-gmp03 246 #  

There is the file in Red.


Finally kill(1) the sleep(1) command (since I’m not waiting for 100000 seconds to elapse):


v4u-4000d-gmp03 246 # kill $! v4u-4000d-gmp03 247 # [1] + Terminated               sleep 100000 < foo & v4u-4000d-gmp03 247 # df -h . Filesystem             size   used  avail capacity  Mounted on /dev/dsk/c3t1d0s0      7.8G   2.0G   5.7G    26%    /mnt v4u-4000d-gmp03 248 # 



Tags: Solaris, du, df

From → Solaris

3 Comments
  1. SomeOne permalink

    That’s good deal. So what is the solution for getting the correct size? I mean what must be done in order to release that space mistakenly occupied by the deleted file?

  2. You are already getting the correct size.
    To free up the space you have to close the file,
    so killing the process that has it open would do this but you have to be sure you know what that process was doing. It is perfectly legitimate to keep files that you have deleted open. Indeed it is a good thing for temporary files as it means you don’t have to worry about cleaning them up when you exit. You also reduce the danger of file name collision.

  3. The find command given in the blog entry finds the file. The problem is that their is no file in the file system anymore so you can’t delete it.
    The best you can hope to do is truncate it. However that really requires that you know what the file is being used for. Truncating random files that are in use by another program is not usually a good plan.
    However if you do know what it is being used for and know it is safe to do then you can do:
    : > /proc/7802/fd/0
    Which will truncate the file to 0 bytes.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: