Philip Guenther wrote:

I'm not sure if this is *the* problem for your situation, but it can 
certainly be *a* problem: if you run slapd as a non-root user or with the 
-U option to change its user id, then you should be running slapcat as 
that same user.

Why?  Because all the programs that open a Sleepycat/Berkeley DB 
environment should be run as the same user.  Otherwise, a transaction log 
file may be created by the wrong user, making it inaccessable by the other 
user, which will cause a database panic.  Yes, even a (read-only) slapcat 
process will create transaction log records.  It only happens if the 
transaction log is close to rolling over to the next file, making it a 
small window, but I saw it happen multiple times with a different project 
using BDB, so I know lightening can strike repeatedly.

If this is what happened then slapd will have died and you'll need to 
manually chown the transaction log files to the correct user.

The other thought is that the alock subsystem mentioned in the error 
messages depends on being able to hold kernel locks (fcntl() or lockf()) 
on a file in the BDB environment directory.  If the filesystem where that 
directory is located doesn't support file locks (NFS?) or the system has a 
hard limit on the number of locks allocated, then this may fail.  (But I 
would expect you to see those failures during slapd startup too...)

Philip Guenther

slapd is running as the user ldap.  the user ldap is disabled anyway, it's shell is set to /bin/false.  it's just an account that fedora uses to give ldap.ldap ownership to /var/lib/ldap.  slapd hasn't died however:

[root@roark ~]# /etc/rc.d/init.d/ldap status
slapd (pid  26873) is running...
[root@roark ~]# ps axuw|grep slapd
ldap     26873  0.1  0.4 723628 17320 ?        Ssl  May07 130:45 /usr/sbin/slapd -h ldap:/// -u ldap

the filesystem of both servers is strictly ext3, and nothing special on them (no LVM, truecrypt, NFS, etc), just /dev/sda3 mounted as /, /dev/sda2 as /boot, and /dev/sda1 is swap.  I'm not sure how to determine the hard lock limit, but its whatever fedora's default is, which should be enough, i'm not running into any other problems on the server and it also runs named, postfix, samba, http, dovecot, etc.

I could restart slapd, but I'm worried that it wouldn't start up properly, which isn't that big of a deal since I have the ldapsearch backup of it and it's trivial to restore from it, but I'd just like to fix this problem if possible instead of restoring from the ldapsearch backup.