Hi folks,
I've experienced some problems in one of my 3 OpenLDAP servers. First glance suggests a ppolicy overlay problem, but, at the end, the problem was resolved, simply, deleting the bbdd (BDB4) and restarting the system (is a consumer server, so the ddbb was automatically regenerated).
So, my questions are
* ¿How I can detect a corrupt database? * ¿Is there any tool/command to check the ddbb health (DBD4)? * ¿What can I do if the corrupted database is the provider database?
Jordi Espasa Clofent wrote:
Hi folks,
I've experienced some problems in one of my 3 OpenLDAP servers. First glance suggests a ppolicy overlay problem, but, at the end, the problem was resolved, simply, deleting the bbdd (BDB4) and restarting the system (is a consumer server, so the ddbb was automatically regenerated).
If your disks are working and haven't run out of space, database corruption pretty much never happens. You probably should describe the situation that leads you to believe there was a corruption. You should also list the versions of software in use.
So, my questions are
- ¿How I can detect a corrupt database?
slapd will detect unclean shutdowns automatically, and automatically recover from them.
- ¿Is there any tool/command to check the ddbb health (DBD4)?
slaptest will tell you if there was an unclean shutdown.
- ¿What can I do if the corrupted database is the provider database?
Restore from a backup.
If your disks are working and haven't run out of space, database corruption pretty much never happens. You probably should describe the situation that leads you to believe there was a corruption. You should also list the versions of software in use.
I've three OpenLDAP (one provider, two consumers) as central account server (implementing also sudo overlay).
xen-ldap01:/# dpkg -l | grep slapd && uname -a && cat /etc/debian_version ii slapd 2.4.11-1 OpenLDAP server (slapd) Linux xen-ldap01 2.6.18-6-xen-amd64 #1 SMP Wed Oct 15 11:49:51 UTC 2008 x86_64 GNU/Linux 5.0
Some concrete users (3 of 35) get errors when they try to login; only them, the others alwasy can login without problem. The 3 OpenLDAP server are balanced, so I can reduce the problem. After some test, it was clear that the "problem" remains in xen-ldap03 (one of the two consumers).
Really I don't know if the ddbb was corrupted or not, but when I did:
$ /etc/init.d/slapd stop $ cd /var/lib/openldap/ && rm * $ /etc/init.d/slapd start
the problem disappears. So, I tend to think the problem was some record in ddbdd. Maybe I'm wrong.
- ¿How I can detect a corrupt database?
slapd will detect unclean shutdowns automatically, and automatically recover from them.
Great. ¿It is showed with normal debug level (256)?
- ¿Is there any tool/command to check the ddbb health (DBD4)?
slaptest will tell you if there was an unclean shutdown.
;)
- ¿What can I do if the corrupted database is the provider database?
Restore from a backup.
Yep. I've done a simple cronjob based on slapcat tool.
On Wednesday, 5 August 2009 12:40:31 Jordi Espasa Clofent wrote:
If your disks are working and haven't run out of space, database corruption pretty much never happens. You probably should describe the situation that leads you to believe there was a corruption. You should also list the versions of software in use.
I've three OpenLDAP (one provider, two consumers) as central account server (implementing also sudo overlay).
xen-ldap01:/# dpkg -l | grep slapd && uname -a && cat /etc/debian_version ii slapd 2.4.11-1 OpenLDAP server (slapd) Linux xen-ldap01 2.6.18-6-xen-amd64 #1 SMP Wed Oct 15 11:49:51 UTC 2008 x86_64 GNU/Linux 5.0
Is BDB supposed to work under Debian Xen ?
Some concrete users (3 of 35) get errors when they try to login; only them, the others alwasy can login without problem. The 3 OpenLDAP server are balanced, so I can reduce the problem. After some test, it was clear that the "problem" remains in xen-ldap03 (one of the two consumers).
But, it is entirely possible, if you were using ppolicy, that the affected users were locked out on this slave. How did you verify that they weren't?
Really I don't know if the ddbb was corrupted or not,
If the database was corrupt, slapd wouldn't start. If it was slightly corrupt, slapd would have recovered it at start. So, I think you should not refer to this as corruption.
but when I did:
$ /etc/init.d/slapd stop $ cd /var/lib/openldap/ && rm * $ /etc/init.d/slapd start
But, nothing indicates that it was corrupt. DId you do an export (with slapcat) and compare it to the other servers? Note that some differences can be valid, such as pwd* attributes, as password policy state is not replicated from consumers to providers ....
the problem disappears. So, I tend to think the problem was some record in ddbdd. Maybe I'm wrong.
- ¿How I can detect a corrupt database?
slapd will detect unclean shutdowns automatically, and automatically recover from them.
Great. ¿It is showed with normal debug level (256)?
- ¿Is there any tool/command to check the ddbb health (DBD4)?
slaptest will tell you if there was an unclean shutdown.
;)
- ¿What can I do if the corrupted database is the provider database?
Restore from a backup.
Yep. I've done a simple cronjob based on slapcat tool.
I don't see that this will do anything to prevent the problem which was most likely the cause ... (users locked out on one slave, not database corruption).
Regards, Buchan
--On August 5, 2009 3:44:52 PM +0100 Buchan Milne bgmilne@staff.telkomsa.net wrote:
On Wednesday, 5 August 2009 12:40:31 Jordi Espasa Clofent wrote:
If your disks are working and haven't run out of space, database corruption pretty much never happens. You probably should describe the situation that leads you to believe there was a corruption. You should also list the versions of software in use.
I've three OpenLDAP (one provider, two consumers) as central account server (implementing also sudo overlay).
xen-ldap01:/# dpkg -l | grep slapd && uname -a && cat /etc/debian_version ii slapd 2.4.11-1 OpenLDAP server (slapd) Linux xen-ldap01 2.6.18-6-xen-amd64 #1 SMP Wed Oct 15 11:49:51 UTC 2008 x86_64 GNU/Linux 5.0
Is BDB supposed to work under Debian Xen ?
It can, depending on the flags used to build it. Since we don't know how BDB was built for this system, it's definitely a fair question.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Is BDB supposed to work under Debian Xen ?
AFAIK, yes.
But, it is entirely possible, if you were using ppolicy, that the affected users were locked out on this slave. How did you verify that they weren't?
No, I did. :( Anyway ¿Does not the consumers re-sync the ppolicy statement when they restart the service?
If the database was corrupt, slapd wouldn't start. If it was slightly corrupt, slapd would have recovered it at start. So, I think you should not refer to this as corruption.
Yes. You'ver the reason.
But, nothing indicates that it was corrupt. DId you do an export (with slapcat) and compare it to the other servers? Note that some differences can be valid, such as pwd* attributes, as password policy state is not replicated from consumers to providers ....
Yes. You and Howard point me out about this ppolicy behaviour: http://www.openldap.org/lists/openldap-technical/200906/msg00221.html
So... I think I can't implement correctly ppolicy if I want stills in my load-balanced environment.
I don't see that this will do anything to prevent the problem which was most likely the cause ... (users locked out on one slave, not database corruption).
I understand now. Great.
openldap-technical@openldap.org
-
Buchan Milne -
Howard Chu -
Jordi Espasa Clofent -
Quanah Gibson-Mount