Skip to content

Don’t keep opening those files

March 17, 2010

Constantin has a wonderful little script that will configure you home server to reduce it’s power consumption. Unfortunately for me he uses a loop that falls into one of my scripting pet peeves. Within Contstantin’s script there is absolutely no problem but I have in the past seen customers burnt by this such that I have been able to work magic to get some really spectacular performance increases.

The problem is here:

for i in $disks ; do
	echo "device-thresholds	$i	5m" >>$TMPFILE
done

Let me emphasize again that in this case there is no real problem but I believe that if you do the right thing where it does not matter you will get it right when it does.

The issue is that for every time around that loop the script opens the output file writes one line and closes the file. This is bad when the file system is a local one but when over NFS this is a performance disaster. With a small change the number of opens can be reduced to just one, which when over NFS all data has to be sent to the server and the server confirm it is on stable storage before the close can complete give a spectacular improvement.

Here is a test I ran over a very slow NFS v4  link (Wifi) to my home server:

brompton% cat /tmp/loop
#!/bin/ksh

i=0
out=${1:-${0##*/}.out}
: > $out
while (( i < 1000))
do
     echo $i >> $out
     let i=i+1
done
brompton% time /tmp/loop
/tmp/loop  0.15s user 0.51s system 2% cpu 26.850 total
brompton%

Compare that with a the good case:

brompton% cat /tmp/loop2
#!/bin/ksh
i=0
out=${1:-${0##*/}.out}
while (( i < 1000))
do
    echo $i
    let i=i+1
done > $out
brompton% time /tmp/loop2
/tmp/loop2  0.01s user 0.01s system 5% cpu 0.384 total
brompton%

From 26 seconds to 0.384 seconds.  Now this is a very bad case as the loop does nothing else and the network latency is particular bad but none the less the principle is the same if you have 10G ethernet, you don’t want to be leaving that performance on the table.

Advertisements

From → bsc, homeserver, Solaris

7 Comments
  1. Karl permalink

    Interesting that you mention this. I migrated 100 sunray users home directories to a new NFS server this week.

    I thought the server (1U x2100 with openSolaris B134 with dual 2TB drives) could handle the load but the devil is in the details!

    I have learned that ZFS disables the write cache on boot drives.

    I used nfsv4rwsnoop.d to figure out what was hammering the system.

    FF’s seems to really like to write to urlclassifier3.sqlite, sessionstore-2.js, cookies.sqlite a lot.

    I disabled the fishing filter(urlclassifier3.sqlite) for the time being and that helped.

    I have not taken the time yet to figure out why FF constantly writes the urlclassifier3.sqlite.

    I guess i kinda went into a rat hole. Keep writing, I enjoy your blog!

  2. Examples using a shell people care about next time…

  3. chrisgerhard permalink

    You are lucky I did not use bourne shell.

  4. Bourne shell would have been good. At least it is portable, unlike ksh.

  5. chrisgerhard permalink

    You are not sucking me into a shell discussion. ksh is portable by virtue of the source being out there.

    The problem is that same whatever shell you choose. So your comment is just irrelevant trolling.

  6. I didn’t know that you could redirect the output of a loop like that, it makes a lot more sense to do that, even on a local FS.

  7. I’d love to see a simpler example. I’ve written very few scripts but want to learn more performance details like this one. Where do you learn stuff like this? I’ve not seen it in O’Reilly books.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: