In an email of 5 August 2009 Howard Chu says "If your disks are working and haven't run out of space, database corruption pretty much never happens. You probably should describe the situation that leads you to believe there was a corruption. You should also list the versions of software in use."
We have been plagued over the years with instances where the database appears to have been corrupted. I'm quite happy to be proved wrong, and would be pleased, very pleased to find a solution. As I said this has happened over the years, on ancient Debian systems, through RedHat's RHEL3, RHEL4, and maybe RHEL5. I'm not sure of the last. Every three or four months the application that uses OpenLDAP stops responding. We then run 'slapcat' against the LDAP datastore and if it hangs we stop slapd, run a recovery on the BDB and start everything up again. Now and then slapd refuses to start and we have to restore from a backup (also taken by slapcat).
As I said, this has happened on many versions of OpenLDAP and on different operating systems. The latest version that I am sure it has happened on is RHEL 4.2 and OpenLDAP 2.2.13-4 (I think that is the version - I'm not able to access the servers at present so I'm going by memory).
I'd be very pleased, ecstatic even, to find a solution to this. It's been a thorn in my side for many years. What information would be useful in pointing me at a solution? I'd only add that I am not an LDAP or OpenLDAP expert - the systems only get touched when an upgrade is necessary. Any advice would be appreciated.
Cheers, hoping for a solution, or at least pointers to one.
Cliff
Cliff Pratt wrote:
As I said, this has happened on many versions of OpenLDAP and on different operating systems. The latest version that I am sure it has happened on is RHEL 4.2 and OpenLDAP 2.2.13-4
Cliff, this version shipped by Red Hat is not recommended to be use at all. Issues therein were quickly fixed but Red Hat never updated their release. Additionally the 2.2.x release branch is really ancient today. Even 2.3.x release branch is onsidered rather historic today.
So please try to compile a recent OpenLDAP release and a fully patched Berkeley-DB. At the moment OpenLDAP 2.4.19 together with BDB 4.8.24 seems fine. Note that other libs used in your deployment which depend on BDB (Kerberos and cyrus-sasl) have to be also compiled to avoid a library mix during run-time linking. I'd recommend to build all these packages installing into a separate prefix to avoid conflicts with pre-installed RPMs.
Ciao, Michael.
Michael Ströder wrote:
Cliff Pratt wrote:
As I said, this has happened on many versions of OpenLDAP and on different operating systems. The latest version that I am sure it has happened on is RHEL 4.2 and OpenLDAP 2.2.13-4
Cliff, this version shipped by Red Hat is not recommended to be use at all. Issues therein were quickly fixed but Red Hat never updated their release. Additionally the 2.2.x release branch is really ancient today. Even 2.3.x release branch is onsidered rather historic today.
So please try to compile a recent OpenLDAP release and a fully patched Berkeley-DB. At the moment OpenLDAP 2.4.19 together with BDB 4.8.24 seems fine. Note that other libs used in your deployment which depend on BDB (Kerberos and cyrus-sasl) have to be also compiled to avoid a library mix during run-time linking. I'd recommend to build all these packages installing into a separate prefix to avoid conflicts with pre-installed RPMs.
Thanks for the information, Michael. Unfortunately our policy is to use packages where ever possible. This also means that the third party product which we have on top of OpenLDAP which our application fits on top of may not support the latest releases of OpenLDAP and BDB. (I'd have to research that).
Cheers,
Cliff
--On Sunday, October 18, 2009 10:28 AM +1300 Cliff Pratt enkidu@cliffp.com wrote:
Thanks for the information, Michael. Unfortunately our policy is to use packages where ever possible. This also means that the third party product which we have on top of OpenLDAP which our application fits on top of may not support the latest releases of OpenLDAP and BDB. (I'd have to research that).
Your policy is seriously flawed. Particularly when it comes to using the packages built by RedHat at that time. They have started getting better more recently. Also, Linux distributions have different objectives when they build their packages then people who run production level LDAP servers. See:
http://www.openldap.org/faq/data/cache/1456.html
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Sunday, October 18, 2009 10:28 AM +1300 Cliff Pratt enkidu@cliffp.com wrote:
Thanks for the information, Michael. Unfortunately our policy is to use packages where ever possible. This also means that the third party product which we have on top of OpenLDAP which our application fits on top of may not support the latest releases of OpenLDAP and BDB. (I'd have to research that).
Your policy is seriously flawed. Particularly when it comes to using the packages built by RedHat at that time. They have started getting better more recently. Also, Linux distributions have different objectives when they build their packages then people who run production level LDAP servers. See:
Hi Quanah,
I'm not going to start a debate on packages versus compiled from source, since this is not an appropriate forum. However that article misses a point or two. I'll just say the following and drop the topic of compiling from source.
Firstly, distro owners do NOT just freeze a package. They freeze at say version x.y.z of a package, then they backport fixes to it and produce a package x.y.z-v, where the '-v' indicates their modified version of the package. There's a good chance that by the time that v is 5 or 6 that the major problems will be fixed.
Secondly, I pay for support. If I do not use the supplied version of the software, then I do not get support. You might make the point that I should therefore go to the distro vendor for support, and not bother this list, and the point is a good one, and I will be pursuing that route.
Thirdly, if I were to listen to all the suppliers of the packages that I use I should compile every single one of them! Don't get me wrong - I totally understand that approach, and all things being equal I would take that approach myself, but it is not possible for me to do that and still have a life!
Cheers,
Cliff
Cliff Pratt enkidu@cliffp.com writes:
Firstly, distro owners do NOT just freeze a package. They freeze at say version x.y.z of a package, then they backport fixes to it and produce a package x.y.z-v, where the '-v' indicates their modified version of the package. There's a good chance that by the time that v is 5 or 6 that the major problems will be fixed.
This is generally not the case for the OpenLDAP server. I don't know of any distribution that is even remotely keeping up with the major fixes that have gone into the server since their release freeze. Red Hat certainly isn't.
Secondly, I pay for support. If I do not use the supplied version of the software, then I do not get support. You might make the point that I should therefore go to the distro vendor for support, and not bother this list, and the point is a good one, and I will be pursuing that route.
Good luck with that. I will be stunned if Red Hat is at all capable of supporting the version of the OpenLDAP server that they ship in a meaningful way.
Thirdly, if I were to listen to all the suppliers of the packages that I use I should compile every single one of them! Don't get me wrong - I totally understand that approach, and all things being equal I would take that approach myself, but it is not possible for me to do that and still have a life!
I think OpenLDAP's server is something of a special case, both due to the number of serious bugs that are fixed and the pace of development.
Full disclosure: I help out with the Debian OpenLDAP packages when I have time.
Cliff Pratt wrote:
I'm not going to start a debate on packages versus compiled from source, since this is not an appropriate forum. However that article misses a point or two. I'll just say the following and drop the topic of compiling from source.
Firstly, distro owners do NOT just freeze a package. They freeze at say version x.y.z of a package, then they backport fixes to it and produce a package x.y.z-v, where the '-v' indicates their modified version of the package. There's a good chance that by the time that v is 5 or 6 that the major problems will be fixed.
They don't.
Secondly, I pay for support. If I do not use the supplied version of the software, then I do not get support. You might make the point that I should therefore go to the distro vendor for support, and not bother this list, and the point is a good one, and I will be pursuing that route.
I also often have this discussion with my customers (especially the bigger ones) who certainly have the same policy and pay for e.g. RHEL support contracts. But practice shows that those support contracts are not worth the paper they are written on. They are only for managers who then think everything's ok. But the same managers will blame *you* if things go wrong. => you have to clarify this with your management => define a build process in an operational concept
Thirdly, if I were to listen to all the suppliers of the packages that I use I should compile every single one of them! Don't get me wrong - I totally understand that approach, and all things being equal I would take that approach myself, but it is not possible for me to do that and still have a life!
My practice (not theory) is to compile the "important" ones. So for a mission-critical OpenLDAP server I compile BDB, heimdal (if needed), cyrus-sasl (if needed) and OpenLDAP from source for a separate prefix to avoid conflicts with pre-installed packages. Most times I build on customer virtual machine matching the software installation of the production system. Once you have simple build scripts in place it's not much effort.
Ciao, Michael.
openldap-technical@openldap.org
-
Cliff Pratt -
Michael Ströder -
Quanah Gibson-Mount -
Russ Allbery