Skip to content

Making a simple script faster

August 4, 2009

Many databases get backed up by simply stopping the database copying all the data files and then restarting the database. This is fine for things that don’t require 24 hour access. However if you are concerned about the time it takes to take the back up then don’t do this:

stop_database cp /data/file1.db . gzip file1.db cp /data/file2.db . gzip file2.db start_database

Now there are many ways to improve this using ZFS and snapshots being one of the best but if you don’t want to go there then at the very least stop doing the “cp”. It is completely pointless. The above should just be:

stop_database gzip < /data/file1.db > file1.db gzip < /data/file2.db > file2.db start_database

You can continue to make it faster by backgrounding those gzips if the system has spare capacity while the back up is running but that is another point. Just stopping those extra copies will make life faster as they are completely unnecessary.


From → Solaris

  1. Brandon Bergren permalink

    Shouldn’t there be a .gz at the end of the second and third lines in the new script?

  2. In my days as a DB admin we just used to backup the transaction logs on a daily basis and do a full database backup weekly for the less busy DBs. Of course we also had expensive software that would backup a live data base without stopping it. So the full backups ended up getting done nightly and the transactions logs got shipped evry 30mins to the warm standbye server.

  3. If you are going to background the gzips, then you should probably add a "wait" before the start_databse too.
    Makes me think that I should take the talk I gave at last year’s CEC on the impact of poor shell scripting on performance, .. .maybe as a bit of a series

  4. Yes there should be a .gz at the end but it is not really the important point. The point is that the cp is completely pointless and just induces lots of extra IO.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: