Reopening this.
This is worse with a database with is intended to be used by different users (A and B):
A opens the DB and creates the semaphores with e.g. mode 0660. B opens it, A closes, B closes - and fails sem_unlink() which only A and root can do.
Next, B (or C) fails mdb_env_open() because sem_unlink() fails again.
The work-around I can think of is a "multi-uid" mode which instead resets the semaphore with sem_post() if sem_getvalue() returns 0. I don't know how ugly that is considered to be. Could ask comp.programming.unix, or check what Berkeley DB does. This mode should use mode 0666 for the semaphores (temporarily setting umask 0, yuck), or it should not sem_unlink() since next user may create the semaphores with a group which gives the wrong users access. Or root may give it root-only access, as in the original report. If not unlinking, we need a special "remove database + semaphores" API call.
Or use some other sync primitive, like file locks. The Linux manual says these work over NFS though, which sounds like they must be rather slow. flock() does not, but this time mdb would need to seek first. I don't know which sync variants BSD offers.
Other matters with the current implementation - I'll patch these:
mdb_env_excl_lock() need not retry getting a non-exclusive lock when closing. mdb_env_close() can pass *excl = -1 to tell it not to.
mdb_env_setup_locks() can sem_unlink both semaphores before doing anything else, so that reopening a database as root will clean up. Drop the error checks of sem_unlink (so both get called), instead use O_EXCL in sem_open(,O_CREAT,,). Unless I'm missing something, the error checks just work like an emulation of O_EXCL anyway.