Skip to content

How large should my semaphore group be?

December 5, 2007

A few weeks ago I had to go and talk to a customer to explain why using large numbers of semaphores in a semaphore group is a bad thing. Essentially if you allow large numbers of semaphores in a group then some applications simply try to make the semaphore group they use as large as possible. This is a bad plan.

In many cases semaphore groups are simply being used as a way to grab multiple “mutex” locks without having to worry about lock ordering. Since the semaphore operations are atomic on completion you either have all the locks you need or your thread either blocks or returns an error.

To achieve this Solaris has a single lock that protects the entire semaphore group. Any semaphore operation grabs the lock, then manipulates the semaphores and then drops the lock. However if you wish to lower the semaphore, but the semaphore is already zero then you are going to be blocked if calling semop(2) or return an error if calling semtimedop(2) if the timeout is reached. However to do this first the kernel has to undo any changes that you have already made before dropping the lock.

So in the worst case, where the last semaphore that you wish to lower is already zero you would lower all the other semaphores in the list then on finding the semaphore that can’t be lowered walk back up the list raising all the semaphores again before dropping the lock and going to sleep. Then when you get woken up by the process that raises the semaphore which you could not lower you have to grab the lock and walk the entire list again lowering each semaphore and if any of them can’t be lowered you go around again.

Now it does not take a rocket scientist to realise that if the number of semaphores in the group is large then the probability of one of the semaphores not being available increases while at the same time the time taken to walk the list grows. While there have been a number of bugs raised against Solaris for semop scalability, 6540859 & 6303546 they have all been closed as will not fix. In my opinion they should have been closed as “not a bug” as any bug here would be in the application.

Taking the case that was being seen. The number of semaphores had been set so high that the database in question used a single semaphore group effectively making every attempt to use any semaphore contend for the same lock.

So in summary when using semaphores make sure that your semaphore sets are as small as possible and contain only related semaphores. If you are not going to need to modify the semaphores atomically then put them in different groups.

Advertisements

From → Solaris

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: