Hi all,
I'm running OpenLDAP 2.4.11 on a debian lenny box and it seems that I'm having trouble with the log files. I had a corrupt database some days ago and needed to restore the database from the backup. The log said:
Jun 28 01:16:59 old slapd[17374]: bdb_db_open: database "dc=intern,dc=domain,dc=de": unclean shutdown detected; attempting recovery.
Jun 28 01:16:59 old slapd[17374]: bdb_db_open: database "dc=intern,dc=domain,dc=de": dbenv_open(/var/lib/ldap). Jun 28 01:16:59 old slapd[17374]: bdb(dc=intern,dc=domain,dc=de): Ignoring log file: /var/lib/ldap/log.0000000005: magic number 0, not 40988 Jun 28 01:16:59 old slapd[17374]: bdb(dc=intern,dc=domain,dc=de): Invalid log file: log.0000000005: Invalid argument Jun 28 01:16:59 old slapd[17374]: bdb(dc=intern,dc=domain,dc=de): PANIC: Invalid argument Jun 28 01:16:59 old slapd[17374]: bdb(dc=intern,dc=domain,dc=de): PANIC: DB_RUNRECOVERY: Fatal error, run database recovery Jun 28 01:16:59 old slapd[17374]: bdb_db_open: database "dc=intern,dc=domain,dc=de" cannot be recovered, err -30978. Restore from backup! Jun 28 01:16:59 old slapd[17374]: ====> bdb_cache_release_all Jun 28 01:16:59 old slapd[17374]: bdb(dc=intern,dc=domain,dc=de): txn_checkpoint interface requires an environment configured for the transaction subsystem Jun 28 01:16:59 old slapd[17374]: bdb_db_close: database "dc=intern,dc=domain,dc=de": txn_checkpoint failed: Invalid argument (22). Jun 28 01:16:59 old slapd[17374]: backend_startup_one: bi_db_open failed! (-30978)
So I installed a new OpenLDAP server with the same versions and the same configuration. The first thing what I found out was that db_archive has some trouble:
On the new box I get the following results from db_archive:
root@new:/var/lib/ldap# db4.2_archive -sa /var/lib/ldap/cn.bdb /var/lib/ldap/dn2id.bdb /var/lib/ldap/gidNumber.bdb /var/lib/ldap/givenName.bdb /var/lib/ldap/id2entry.bdb /var/lib/ldap/loginShell.bdb /var/lib/ldap/objectClass.bdb /var/lib/ldap/sn.bdb /var/lib/ldap/uid.bdb /var/lib/ldap/uidNumber.bdb root@new:/var/lib/ldap# db4.2_archive -la /var/lib/ldap/log.0000000001 root@new:/var/lib/ldap#
Looks good so far. On the old box I get the following results (there are the same databases)
root@new:/var/lib/ldap# db4.2_archive -sa root@new:/var/lib/ldap# db4.2_archive -la db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data pair found root@new:/var/lib/ldap#
I searched the archive and googled this message, but found nothing that fits my case.
Hope that someone can give me a hint, solving this issue.
Thanks and kind regards, Andreas
Andreas Krummrich wrote:
I'm running OpenLDAP 2.4.11 on a debian lenny box
1. This version is outdated. For various reasons you should use at least 2.4.16.
and it seems that I'm having trouble with the log files. I had a corrupt database some days ago and needed to restore the database from the backup.
How are you creating the backups? Note that just copying the database files will very likely result in such problems you describe after restore.
See also: http://www.openldap.org/faq/data/cache/286.html
Ciao, Michael.
Hi Michael,
-----Original Message----- From: Michael Ströder [mailto:michael@stroeder.com] Sent: Tuesday, June 30, 2009 6:16 PM To: andreas@krummrich.org Cc: openldap-technical@openldap.org Subject: Re: db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data pair found
Andreas Krummrich wrote:
I'm running OpenLDAP 2.4.11 on a debian lenny box
- This version is outdated. For various reasons you should use at least
2.4.16.
Okay, I will check if there are other packages for Debian.
and it seems that I'm having trouble with the log files. I had a corrupt database some days
ago
and needed to restore the database from the backup.
How are you creating the backups? Note that just copying the database files will very likely result in such problems you describe after restore.
No, I never just copy the database files. I do the backup with slapcat and the restore using slapadd.
So it seems to have another cause!?
See also: http://www.openldap.org/faq/data/cache/286.html
Ciao, Michael.
Kind Regards, Andreas
Hi Andreas,
On 30/06/2009 16:42, Andreas Krummrich wrote:
I'm running OpenLDAP 2.4.11 on a debian lenny box and it seems that I'm having trouble with the log files. I had a corrupt database some days ago and needed to restore the database from the backup. The log said:
Jun 28 01:16:59 old slapd[17374]: bdb_db_open: database "dc=intern,dc=domain,dc=de" cannot be recovered, err -30978. Restore from backup!
So I installed a new OpenLDAP server with the same versions and the same configuration. The first thing what I found out was that db_archive has some trouble:
On the new box I get the following results from db_archive:
root@new:/var/lib/ldap# db4.2_archive -sa /var/lib/ldap/cn.bdb /var/lib/ldap/dn2id.bdb /var/lib/ldap/gidNumber.bdb /var/lib/ldap/givenName.bdb /var/lib/ldap/id2entry.bdb /var/lib/ldap/loginShell.bdb /var/lib/ldap/objectClass.bdb /var/lib/ldap/sn.bdb /var/lib/ldap/uid.bdb /var/lib/ldap/uidNumber.bdb root@new:/var/lib/ldap# db4.2_archive -la /var/lib/ldap/log.0000000001 root@new:/var/lib/ldap#
Looks good so far. On the old box I get the following results (there are the same databases)
root@new:/var/lib/ldap# db4.2_archive -sa root@new:/var/lib/ldap# db4.2_archive -la db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data pair found root@new:/var/lib/ldap#
If I understand correctly, your new box works fine, and your old box gives you the error "No matching key/data pair found"? (should be root@old above, then?)
Have you restored the database from backup on your old box aswell? This would involve something like rm /var/lib/ldap/* (but save DB_CONFIG), then a slapadd.
Hope this helps, Jonathan
Hi Jonathan,
-----Original Message----- From: openldap-technical-bounces+andreas=krummrich.org@OpenLDAP.org [mailto:openldap-technical-bounces+andreas=krummrich.org@OpenLDAP.org] On Behalf Of Jonathan Clarke Sent: Wednesday, July 01, 2009 11:10 AM To: andreas@krummrich.org Cc: openldap-technical@openldap.org Subject: Re: db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data pair found
Hi Andreas,
On 30/06/2009 16:42, Andreas Krummrich wrote:
I'm running OpenLDAP 2.4.11 on a debian lenny box and it seems that
I'm
having trouble with the log files. I had a corrupt database some days
ago
and needed to restore the database from the backup. The log said:
Jun 28 01:16:59 old slapd[17374]: bdb_db_open: database "dc=intern,dc=domain,dc=de" cannot be recovered, err -30978. Restore
from
backup!
So I installed a new OpenLDAP server with the same versions and the
same
configuration. The first thing what I found out was that db_archive
has some
trouble:
On the new box I get the following results from db_archive:
root@new:/var/lib/ldap# db4.2_archive -sa /var/lib/ldap/cn.bdb /var/lib/ldap/dn2id.bdb /var/lib/ldap/gidNumber.bdb /var/lib/ldap/givenName.bdb /var/lib/ldap/id2entry.bdb /var/lib/ldap/loginShell.bdb /var/lib/ldap/objectClass.bdb /var/lib/ldap/sn.bdb /var/lib/ldap/uid.bdb /var/lib/ldap/uidNumber.bdb root@new:/var/lib/ldap# db4.2_archive -la /var/lib/ldap/log.0000000001 root@new:/var/lib/ldap#
Looks good so far. On the old box I get the following results (there
are the
same databases)
root@new:/var/lib/ldap# db4.2_archive -sa root@new:/var/lib/ldap# db4.2_archive -la db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data
pair
found root@new:/var/lib/ldap#
If I understand correctly, your new box works fine, and your old box gives you the error "No matching key/data pair found"? (should be root@old above, then?)
Right.
Have you restored the database from backup on your old box aswell? This would involve something like rm /var/lib/ldap/* (but save DB_CONFIG), then a slapadd.
Yes, I had to restore the db last weekend. I removed everything under /var/lib/ldap and restored the db with slapadd. But the error still exists.
This is what I can see in /var/log/syslog, for example:
Jul 1 11:54:28 old slapd[2169]: <= bdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30990)
Now one interesting point. I restored the backup from the old box to the new box this morning and it works fine there. No errors. Db42_archive works well and I can archive and remove the logs.
Could it be a config problem? Please find my DB_Config.
set_cachesize 0 2097152 0 set_lk_max_objects 1500 set_lk_max_locks 1500 set_lk_max_lockers 1500
I think these values are added there through slapd.conf. Maybe there is something wrong?
# Allow LDAPv2 binds allow bind_v2
# This is the main slapd configuration file. See slapd.conf(5) for more # info on the configuration options.
####################################################################### # Global Directives:
TLSCACertificateFile /etc/ldap/ca.crt TLSCertificateFile /etc/ldap/old.intern.domain.de.crt TLSCertificateKeyFile /etc/ldap/old.intern.domain.de.key TLSVerifyClient never
# Features to permit #allow bind_v2
# Schema and objectClass definitions include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema include /etc/ldap/schema/samba.schema include /etc/ldap/schema/scalix.schema
# Where the pid file is put. The init.d script # will not stop the server if you change this. pidfile /var/run/slapd/slapd.pid
# List of arguments that were passed to the server argsfile /var/run/slapd/slapd.args
# Read slapd.conf(5) for possible values loglevel 3
# Where the dynamically loaded modules are stored modulepath /usr/lib/ldap moduleload back_bdb
# The maximum number of entries that is returned for a search operation sizelimit -1
# The tool-threads parameter sets the actual amount of cpu's that is used # for indexing. tool-threads 1
####################################################################### # Specific Backend Directives for bdb: # Backend specific directives apply to this backend until another # 'backend' directive occurs backend bdb
####################################################################### # Specific Backend Directives for 'other': # Backend specific directives apply to this backend until another # 'backend' directive occurs #backend <other>
####################################################################### # Specific Directives for database #1, of type bdb: # Database specific directives apply to this databasse until another # 'database' directive occurs database bdb #ucdatapath /etc/ldap/ucdata
# The base of your directory in database #1 suffix "dc=intern,dc=domain,dc=de" checkpoint 512 30
# rootdn directive for specifying a superuser on the database. This is needed # for syncrepl. rootdn "cn=admin,dc=intern,dc=domain,dc=de" rootpw {SSHA}xxxx
# Where the database file are physically stored for database #1 directory "/var/lib/ldap"
# Save the time that the entry gets modified, for database #1 # lastmod on
# For the Debian package we use 2MB as default but be sure to update this # value if you have plenty of RAM dbconfig set_cachesize 0 2097152 0
# Sven Hartge reported that he had to set this value incredibly high # to get slapd running at all. See http://bugs.debian.org/303057 # for more information.
# Number of objects that can be locked at the same time. dbconfig set_lk_max_objects 1500 # Number of locks (both requested and granted) dbconfig set_lk_max_locks 1500 # Number of lockers dbconfig set_lk_max_lockers 1500
# Indexing options for database #1 #index objectClass eq index objectClass eq,pres index ou,cn,sn,mail,givenname eq,pres,sub index uidNumber,gidNumber,memberUid eq,pres index loginShell eq,pres ## required to support pdb_getsampwnam index uid pres,sub,eq ## required to support pdb_getsambapwrid() index displayName pres,sub,eq index nisMapName,nisMapEntry eq,pres,sub index sambaSID eq index sambaPrimaryGroupSID eq index sambaDomainName eq #index scalixScalixObject eq index default sub
# Where to store the replica logs for database #1 # replogfile /var/lib/ldap/replog
# The userPassword by default can be changed # by the entry owning it if they are authenticated. # Others should not be able to see it, except the # admin entry below # These access lines apply to database #1 only #access to attrs=userPassword,shadowLastChange # by dn="cn=admin,dc=intern,dc=domain,dc=de" write # by anonymous auth # by self write # by * none
# users can authenticate and change their password access to attrs=userPassword,sambaNTPassword,sambaLMPassword,sambaPwdMustChange,sambaP wdLastSet by dn="cn=admin,dc=intern,dc=domain,dc=de" write by self write by anonymous auth by * none
# those 2 parameters must be world readable for password aging to work correctly # # # (or use a priviledge account in /etc/ldap.conf to bind to the directory) access to attrs=shadowLastChange,shadowMax by dn="cn=admin,dc=intern,dc=domain,dc=de" write by self write by * read
access to attrs=l by dn="cn=admin,dc=intern,dc=domain,dc=de" write by dn="uid=digestreader,dc=intern,dc=domain,dc=de" read by anonymous auth by self write by * none
# Ensure read access to the base for things like # supportedSASLMechanisms. Without this you may # have problems with SASL not knowing what # mechanisms are available and the like. # Note that this is covered by the 'access to *' # ACL below too but if you change that as people # are wont to do you'll still need this if you # want SASL (and possible other things) to work # happily. access to dn.base="" by * read
# The admin dn has full write access, everyone else # can read everything. access to * by dn="cn=admin,dc=intern,dc=domain,dc=de" write by * read
# all others attributes are readable to everybody access to * by * read
# For Netscape Roaming support, each user gets a roaming # profile for which they have write access to #access to dn=".*,ou=Roaming,o=morsnet" # by dn="cn=admin,dc=intern,dc=domain,dc=de" write # by dnattr=owner write
####################################################################### # Specific Directives for database #2, of type 'other' (can be bdb too): # Database specific directives apply to this databasse until another # 'database' directive occurs #database <other>
# The base of your directory for database #2 #suffix "dc=debian,dc=org"
Hope this helps, Jonathan --
Kind Regards, Andreas
On Wednesday 01 July 2009 12:05:12 Andreas Krummrich wrote:
Hi Jonathan,
-----Original Message----- From: openldap-technical-bounces+andreas=krummrich.org@OpenLDAP.org [mailto:openldap-technical-bounces+andreas=krummrich.org@OpenLDAP.org] On Behalf Of Jonathan Clarke Sent: Wednesday, July 01, 2009 11:10 AM To: andreas@krummrich.org Cc: openldap-technical@openldap.org Subject: Re: db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data pair found
Hi Andreas,
On 30/06/2009 16:42, Andreas Krummrich wrote:
I'm running OpenLDAP 2.4.11 on a debian lenny box and it seems that
I'm
having trouble with the log files. I had a corrupt database some days
ago
and needed to restore the database from the backup. The log said:
Jun 28 01:16:59 old slapd[17374]: bdb_db_open: database "dc=intern,dc=domain,dc=de" cannot be recovered, err -30978. Restore
from
backup!
So I installed a new OpenLDAP server with the same versions and the
same
configuration. The first thing what I found out was that db_archive
has some
trouble:
On the new box I get the following results from db_archive:
root@new:/var/lib/ldap# db4.2_archive -sa /var/lib/ldap/cn.bdb /var/lib/ldap/dn2id.bdb /var/lib/ldap/gidNumber.bdb /var/lib/ldap/givenName.bdb /var/lib/ldap/id2entry.bdb /var/lib/ldap/loginShell.bdb /var/lib/ldap/objectClass.bdb /var/lib/ldap/sn.bdb /var/lib/ldap/uid.bdb /var/lib/ldap/uidNumber.bdb root@new:/var/lib/ldap# db4.2_archive -la /var/lib/ldap/log.0000000001 root@new:/var/lib/ldap#
Looks good so far. On the old box I get the following results (there
are the
same databases)
root@new:/var/lib/ldap# db4.2_archive -sa root@new:/var/lib/ldap# db4.2_archive -la db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data
pair
found root@new:/var/lib/ldap#
If I understand correctly, your new box works fine, and your old box gives you the error "No matching key/data pair found"? (should be root@old above, then?)
Right.
Have you restored the database from backup on your old box aswell? This would involve something like rm /var/lib/ldap/* (but save DB_CONFIG), then a slapadd.
Yes, I had to restore the db last weekend. I removed everything under /var/lib/ldap and restored the db with slapadd. But the error still exists.
This is what I can see in /var/log/syslog, for example:
Jul 1 11:54:28 old slapd[2169]: <= bdb_dn2id: get failed: DB_NOTFOUND: No matching key/data pair found (-30990)
Now one interesting point. I restored the backup from the old box to the new box this morning and it works fine there. No errors. Db42_archive works well and I can archive and remove the logs.
I am not satisfied that we know how you restored. You should have:
1)(Re)moved all the contents from /var/lib/ldap, except for the DB_CONFIG file, including the database environment (__db*), database files (*.bdb), and transaction logs (log.*)
2)Run slapadd with the ldif export you had from backup
3)Ensure permissions/ownership are correct on the directory and all the files (the user running slapd must have write access to all).
Please tell us what the exact process was that you used to restore.
Regards, Buchan
Hi Buchan,
-----Original Message----- From: Buchan Milne [mailto:bgmilne@staff.telkomsa.net] Sent: Wednesday, July 01, 2009 12:27 PM To: openldap-technical@openldap.org; andreas@krummrich.org Cc: 'Jonathan Clarke' Subject: Re: db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching key/data pair found
On Wednesday 01 July 2009 12:05:12 Andreas Krummrich wrote:
Hi Jonathan,
-----Original Message----- From: openldap-technical-bounces+andreas=krummrich.org@OpenLDAP.org [mailto:openldap-technical-
bounces+andreas=krummrich.org@OpenLDAP.org]
On Behalf Of Jonathan Clarke Sent: Wednesday, July 01, 2009 11:10 AM To: andreas@krummrich.org Cc: openldap-technical@openldap.org Subject: Re: db_archive: DB_ENV->log_archive: DB_NOTFOUND: No
matching
key/data pair found
Hi Andreas,
On 30/06/2009 16:42, Andreas Krummrich wrote:
I'm running OpenLDAP 2.4.11 on a debian lenny box and it seems
that
I'm
having trouble with the log files. I had a corrupt database some
days
ago
and needed to restore the database from the backup. The log said:
Jun 28 01:16:59 old slapd[17374]: bdb_db_open: database "dc=intern,dc=domain,dc=de" cannot be recovered, err -30978.
Restore
from
backup!
So I installed a new OpenLDAP server with the same versions and
the
same
configuration. The first thing what I found out was that
db_archive
has some
trouble:
On the new box I get the following results from db_archive:
root@new:/var/lib/ldap# db4.2_archive -sa /var/lib/ldap/cn.bdb /var/lib/ldap/dn2id.bdb /var/lib/ldap/gidNumber.bdb /var/lib/ldap/givenName.bdb /var/lib/ldap/id2entry.bdb /var/lib/ldap/loginShell.bdb /var/lib/ldap/objectClass.bdb /var/lib/ldap/sn.bdb /var/lib/ldap/uid.bdb /var/lib/ldap/uidNumber.bdb root@new:/var/lib/ldap# db4.2_archive -la /var/lib/ldap/log.0000000001 root@new:/var/lib/ldap#
Looks good so far. On the old box I get the following results
(there
are the
same databases)
root@new:/var/lib/ldap# db4.2_archive -sa root@new:/var/lib/ldap# db4.2_archive -la db_archive: DB_ENV->log_archive: DB_NOTFOUND: No matching
key/data
pair
found root@new:/var/lib/ldap#
If I understand correctly, your new box works fine, and your old
box
gives you the error "No matching key/data pair found"? (should be root@old above, then?)
Right.
Have you restored the database from backup on your old box aswell?
This
would involve something like rm /var/lib/ldap/* (but save
DB_CONFIG),
then a slapadd.
Yes, I had to restore the db last weekend. I removed everything under /var/lib/ldap and restored the db with slapadd. But the error still
exists.
This is what I can see in /var/log/syslog, for example:
Jul 1 11:54:28 old slapd[2169]: <= bdb_dn2id: get failed:
DB_NOTFOUND: No
matching key/data pair found (-30990)
Now one interesting point. I restored the backup from the old box to
the
new box this morning and it works fine there. No errors. Db42_archive
works
well and I can archive and remove the logs.
I am not satisfied that we know how you restored. You should have:
1)(Re)moved all the contents from /var/lib/ldap, except for the DB_CONFIG file, including the database environment (__db*), database files (*.bdb), and transaction logs (log.*)
2)Run slapadd with the ldif export you had from backup
3)Ensure permissions/ownership are correct on the directory and all the files (the user running slapd must have write access to all).
Please tell us what the exact process was that you used to restore.
Okay ;-)
0) sudo -s 1) /etc/init.d/slapd stop 2) cd /var/lib/ldap 3) rm * 4) slapadd -l /root/bin/old/dc=intern,dc=domain,dc=de.ldif 5) chown openldap: * 6) /etc/init.d/slapd start
This is how I restored the backup to my test box and how I did it this weekend on the productive box where the error still exists. I also removed the DB_CONFIG, but after the restore the DB_CONFIG has the same entries.
Regards, Buchan
Thanks and Regards, Andreas
openldap-technical@openldap.org
-
Andreas Krummrich -
Buchan Milne -
Jonathan Clarke -
Michael Ströder