Friday, 8 March 2013

No space left on device: Couldn’t create accept lock

Apache was not starting in the server and the error logs shows the following.
[emerg] (28)No space left on device: Couldn’t create accept lock (/usr/local/apache/logs/accept.lock.

It’s because the sempahore arrays in system memory is full. Check if there’re any processes in semaphore that are owned nobody in semaphores (or the user which apache is being run):
# ipcs -s|grep nobody

If you can see the semaphore processes for nobody, eventhough no apache processes are running, you can remove them safely. It happens when Apache dies without clearing the Semaphore arrays. Run the following command:
# for semid in `ipcs -s | grep nobody | cut -f2 -d" "`; do ipcrm -s $semid; done

Now restart apache
# service apache restart

Now, in almost all cases, Apache should start properly. If it doesn’t, you may just be completely out of available semaphores. You may want to increase your available semaphores, and you’ll need to tickle your kernel to do so. Add this to /etc/sysctl.conf:
kernel.msgmni = 1024
kernel.sem = 250 256000 32 1024

And then run sysctl -p to pick up the new changes.