Debian Squeeze: Slapd subtree disappears, but ldapsearch finds it | unable to allocate memory for mutex; resize mutex region
hi,
on the last hour I had a very strange problem:
I have a Debian Squeeze with Slapd installed and it was working a long time, but today one subtree disappears completely from the ldap browser (Apache Directory Studio). I wasn't accessible anymore, but with ldapsearch I was able to see the tree. Also I tried to add the LDIF again to the LDAP, but slapd says: "... exists ...."
The only way to get it back working again, was to restore the plain BDB files from the backupjob yesterday.
I absolutely don't know, what happens. The only thing I changed, I reinstalled the second LDAP (n-way master) on a new host, with a config only and let the second ldap synchronize with the main LDAP. After ~20-40 minutes the job was done and the second was up to date. That was yesterday.
-
The subtree was also missing on the second LDAP ... so, the synchronizing did the same ...
So, I deleted all *dbd* *log* on ldap2 (after the restore from the main LDAP) and wanted to synchronizing again the full tree ... but I get many warnings:
dc=...... unable to allocate memory for mutex; resize mutex region
any suggestions?
cu denny
--On Tuesday, April 30, 2013 3:39 PM +0200 Denny Schierz linuxmail@4lin.net wrote:
hi, dc=...... unable to allocate memory for mutex; resize mutex region
You probably need to fix DB_CONFIG at a guess. I'd personally avoid using Debian's openldap build, there are serious problems with it, and significant problems with the way in which they build their BDB package too.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
hi,
Am 30.04.2013 um 23:42 schrieb Quanah Gibson-Mount quanah@zimbra.com:
--On Tuesday, April 30, 2013 3:39 PM +0200 Denny Schierz linuxmail@4lin.net wrote:
hi, dc=...... unable to allocate memory for mutex; resize mutex region
You probably need to fix DB_CONFIG at a guess. I'd personally avoid using Debian's openldap build, there are serious problems with it, and significant problems with the way in which they build their BDB package too.
hmm, bad. I try to build 2.4.35 with the Debian build files, but without several patches. Some packages needs libldap etc. pp.
DB_CONFIG looks like this one:
set_cachesize 0 2097152 0 set_lg_bsize 2097512 set_lk_max_objects 1500 set_lk_max_locks 1500 set_lk_max_lockers 1500 set_flags DB_LOG_AUTOREMOVE
I only added the last line, because of filling the disks ...
any idea what happens to the first problem?
cu denny
--On May 1, 2013 12:08:51 AM +0200 Denny Schierz linuxmail@4lin.net wrote:
hmm, bad. I try to build 2.4.35 with the Debian build files, but without several patches. Some packages needs libldap etc. pp.
You should not try and replace the broken system OpenLDAP. Build OpenLDAP yourself out into /usr/local or /opt, etc.
Also, as I already stated, the BDB libraries provided by Debian are built incorrectly as well, and should not be used.
As for DB_CONFIG tuning, I suggest reading over http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning#Berkeley_DB_DB_CONFIG_tuning
--Quanah
On Wed, May 01, 2013 at 10:26:58AM -0700, Quanah Gibson-Mount wrote:
--On May 1, 2013 12:08:51 AM +0200 Denny Schierz linuxmail@4lin.net wrote:
hmm, bad. I try to build 2.4.35 with the Debian build files, but without several patches. Some packages needs libldap etc. pp.
You should not try and replace the broken system OpenLDAP. Build OpenLDAP yourself out into /usr/local or /opt, etc.
Also, as I already stated, the BDB libraries provided by Debian are built incorrectly as well, and should not be used.
How do people with generic sysadmin skills figure out which packaged components are safe to use with a non-distribution-packaged OpenLDAP? Having built a production slapd on Debian all I have is "well, worked and didn't crash for me".
As for DB_CONFIG tuning, I suggest reading over http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning#Berkeley_DB_DB_CONFIG_tuning
--Quanah
-- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration
--On Wednesday, May 01, 2013 3:59 PM -0400 Christopher Wood christopher_wood@pobox.com wrote:
How do people with generic sysadmin skills figure out which packaged components are safe to use with a non-distribution-packaged OpenLDAP? Having built a production slapd on Debian all I have is "well, worked and didn't crash for me".
I never use distribution software for OpenLDAP except gcc. I.e., I build openssl, heimdal, cyrus-sasl, etc, all myself. And now with MDB, I no longer have to build BDB. ;)
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
hi,
Am 01.05.2013 um 23:08 schrieb Quanah Gibson-Mount quanah@zimbra.com:
--On Wednesday, May 01, 2013 3:59 PM -0400 Christopher Wood christopher_wood@pobox.com wrote:
How do people with generic sysadmin skills figure out which packaged components are safe to use with a non-distribution-packaged OpenLDAP? Having built a production slapd on Debian all I have is "well, worked and didn't crash for me".
I never use distribution software for OpenLDAP except gcc. I.e., I build openssl, heimdal, cyrus-sasl, etc, all myself. And now with MDB, I no longer have to build BDB. ;)
but than you have to download, patch and update security fixes by your self.
I have now build Openldap 2.4.35 with the system libs. In a few weeks Wheezy is out and I hope, that the BDB files are not "broken" as you said.
cu denny
--On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz linuxmail@4lin.net wrote:
but than you have to download, patch and update security fixes by your self.
Yep. Part of being a competent sys admin anyhow.
I have now build Openldap 2.4.35 with the system libs. In a few weeks Wheezy is out and I hope, that the BDB files are not "broken" as you said.
Debian's BDB libs will always be broken. I filed a bug report with them on it ages ago, and they declined to fix it.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On 05/02/2013 04:08 PM, Quanah Gibson-Mount wrote:
--On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz linuxmail@4lin.net wrote:
but than you have to download, patch and update security fixes by your self.
Yep. Part of being a competent sys admin anyhow.
Sorry, I disagree.
A competent sysadmin has to make choices on how he has to employ his time. When having limited resources the choice you suggest can be easily seen as an incompetent wasting of time.
For example when you have to manage > 70 small server for > 70 school, applying security upgrade by recompiling apache, bind, samba, openldap (just to cite some of the services on them) every time is plain wrong. It's a waste of the scarce sysadmin time that could not be afforded.
That's just an example, but there are lot of situations in which the solution to bad distribution packaging cannot be "recompile it by yourself and reinstall". Better to point to another distribution or to a good packaging (if they exist). Otherwise every competent sysadmin will use the packages, also if they are suboptimal.
I'm sorry to hear that Debian OpenLDAP packages are in a such bad state, but if, as it seems, there no distribution getting OpenLDAP right (I heard complaints also about RedHat), then I start thinking that something is not working fine, at least on the user end of OpenLDAP distribution.
Simone
--On Thursday, May 02, 2013 5:52 PM +0200 Simone Piccardi piccardi@truelite.it wrote:
On 05/02/2013 04:08 PM, Quanah Gibson-Mount wrote:
--On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz linuxmail@4lin.net wrote:
but than you have to download, patch and update security fixes by your self.
Yep. Part of being a competent sys admin anyhow.
Sorry, I disagree.
A competent sysadmin has to make choices on how he has to employ his time. When having limited resources the choice you suggest can be easily seen as an incompetent wasting of time.
For example when you have to manage > 70 small server for > 70 school, applying security upgrade by recompiling apache, bind, samba, openldap (just to cite some of the services on them) every time is plain wrong. It's a waste of the scarce sysadmin time that could not be afforded.
Sorry, as someone who used to maintain some 600 servers for a major university running a very wide variety of services, I disagree. If you can't figure out an easy way to build and distribute your own packages in an automated fashion, you are putting yourself at the mercy of your distribution provider's capabilities and competence, which has been shown time and again to be severely lacking in multiple areas.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On 05/02/2013 06:10 PM, Quanah Gibson-Mount wrote:
Sorry, as someone who used to maintain some 600 servers for a major university running a very wide variety of services, I disagree. If you can't figure out an easy way to build and distribute your own packages in an automated fashion, you are putting yourself at the mercy of your distribution provider's capabilities and competence, which has been shown time and again to be severely lacking in multiple areas.
I can figure it, but that was not my point. The point is that in a lot of situations people prefer to be at the mercy of a distribution provider and less dependent from a skilled sysadmin (or an external consultant) because this would cost them less on update management.
When you have strong budget limits (both in money or skills you can use) inferior solution are willingly chosen because they cost less or have less resource requirements. What I cannot agree is calling this kind of choice incompetence.
Simone
--On Thursday, May 02, 2013 8:18 PM +0200 Simone Piccardi piccardi@truelite.it wrote:
On 05/02/2013 06:10 PM, Quanah Gibson-Mount wrote:
Sorry, as someone who used to maintain some 600 servers for a major university running a very wide variety of services, I disagree. If you can't figure out an easy way to build and distribute your own packages in an automated fashion, you are putting yourself at the mercy of your distribution provider's capabilities and competence, which has been shown time and again to be severely lacking in multiple areas.
I can figure it, but that was not my point. The point is that in a lot of situations people prefer to be at the mercy of a distribution provider and less dependent from a skilled sysadmin (or an external consultant) because this would cost them less on update management.
It costs them "less" until such a time as they have a catastrophic failure because of the fact they tried to cut corners by relying on distribution software. Then they get the cost of dealing with the mess they are to blame for by trying to cut costs in the first place.
I would note that OpenLDAP is not unique with the issues caused by distribution builds. The postfix authors give the same advice. Don't use the outdated distribution packages: http://archives.neohapsis.com/archives/postfix/2013-05/0021.html
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
--On Thursday, May 02, 2013 5:52 PM +0200 Simone Piccardi piccardi@truelite.it wrote:
I'm sorry to hear that Debian OpenLDAP packages are in a such bad state, but if, as it seems, there no distribution getting OpenLDAP right (I heard complaints also about RedHat), then I start thinking that something is not working fine, at least on the user end of OpenLDAP distribution.
The OpenLDAP foundation has zero input or control into how distribution providers build their OpenLDAP packages. Thus the end users are at the mercy of the distribution provider's decisions on building OpenLDAP, which are quite often extremely flawed.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Hi Quanah-
On May 2, 2013, at 12:12 PM, Quanah Gibson-Mount quanah@zimbra.com wrote:
The OpenLDAP foundation has zero input or control into how distribution providers build their OpenLDAP packages. Thus the end users are at the mercy of the distribution provider's decisions on building OpenLDAP, which are quite often extremely flawed.
Yes, it is a big bummer. Has the OpenLDAP foundation ever considered publishing any official guidelines that could be used both by these distributions and individuals who want to do their own packages? Just two lists of "Do this:…" and "Don't do this: …" would probably go a long way.
-- dNb
--On Thursday, May 02, 2013 12:35 PM -0400 David Blank-Edelman dnb@ccs.neu.edu wrote:
Yes, it is a big bummer. Has the OpenLDAP foundation ever considered publishing any official guidelines that could be used both by these distributions and individuals who want to do their own packages? Just two lists of "Do this:…" and "Don't do this: …" would probably go a long way.
The distribution maintainers are quite aware of the objections to the way in which they build their software. Their decisions have little to do with needs of the end users.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On May 2, 2013, at 12:53 PM, Quanah Gibson-Mount quanah@zimbra.com wrote:
The distribution maintainers are quite aware of the objections to the way in which they build their software. Their decisions have little to do with needs of the end users.
Ok, then perhaps guidelines for the rest of us? I know I try to pick up tips whenever someone says "don't do it like RH, they <blah>" but it would be really swell if these things were written down someplace in a single spot. Can you see any downside to having this documented for the public?
-- dNb
--On Thursday, May 02, 2013 12:58 PM -0400 David Blank-Edelman dnb@ccs.neu.edu wrote:
On May 2, 2013, at 12:53 PM, Quanah Gibson-Mount quanah@zimbra.com wrote:
The distribution maintainers are quite aware of the objections to the way in which they build their software. Their decisions have little to do with needs of the end users.
Ok, then perhaps guidelines for the rest of us? I know I try to pick up tips whenever someone says "don't do it like RH, they <blah>" but it would be really swell if these things were written down someplace in a single spot. Can you see any downside to having this documented for the public?
There is not a whole lot to it.
a) Link to OpenSSL, not gnutls (debian/ubuntu default) or NSS (rhel default)
b) If you are going to use BDB as your underlying database software and are on Linux, make sure to pass the following flags to configure: --enable-posixmutexes --with-mutex=POSIX/pthreads
Post build:
c) Generally, I advise preloading a memory allocator such as tcmalloc from google perf tools. Particularly important for Linux to avoid using the horrid glibc allocator.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On May 2, 2013, at 4:53 PM, Quanah Gibson-Mount quanah@zimbra.com wrote:
There is not a whole lot to it.
a) Link to OpenSSL, not gnutls (debian/ubuntu default) or NSS (rhel default)
b) If you are going to use BDB as your underlying database software and are on Linux, make sure to pass the following flags to configure: --enable-posixmutexes --with-mutex=POSIX/pthreads
Post build:
c) Generally, I advise preloading a memory allocator such as tcmalloc from google perf tools. Particularly important for Linux to avoid using the horrid glibc allocator.
That's great information, thanks. Anything special if you plan to use MDB?
-- dNb
--On Thursday, May 02, 2013 5:09 PM -0400 dnb@ccs.neu.edu wrote:
That's great information, thanks. Anything special if you plan to use MDB?
I would use the current RE24 source to pick up some fixes since 2.4.35. It's finally been stable for me with that in place. You may or may not want to set two flags for mdb:
olcDbEnvFlags: writemap olcDbEnvFlags: nometasync
I set them myself.
You can get the current source via:
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Simone Piccardi wrote:
On 05/02/2013 04:08 PM, Quanah Gibson-Mount wrote:
--On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz linuxmail@4lin.net wrote:
but than you have to download, patch and update security fixes by your self.
Yep. Part of being a competent sys admin anyhow.
Sorry, I disagree.
A competent sysadmin has to make choices on how he has to employ his time. When having limited resources the choice you suggest can be easily seen as an incompetent wasting of time.
For example when you have to manage > 70 small server for > 70 school, applying security upgrade by recompiling apache, bind, samba, openldap (just to cite some of the services on them) every time is plain wrong. It's a waste of the scarce sysadmin time that could not be afforded.
A competent sysadmin knows how to leverage tools such that 7 servers or 7000 servers requires the same amount of hands-on time. One element of making this feasible is certainly to have the minimum possible variations in deployed configurations. But a frozen configuration that you built yourself with known components is just as viable for this purpose as one you obtained from a distro. And in most cases, due to distro lag times, a config you build yourself will be superior.
That's just an example, but there are lot of situations in which the solution to bad distribution packaging cannot be "recompile it by yourself and reinstall". Better to point to another distribution or to a good packaging (if they exist). Otherwise every competent sysadmin will use the packages, also if they are suboptimal.
I'm sorry to hear that Debian OpenLDAP packages are in a such bad state, but if, as it seems, there no distribution getting OpenLDAP right (I heard complaints also about RedHat), then I start thinking that something is not working fine, at least on the user end of OpenLDAP distribution.
Simone
openldap-technical@openldap.org
-
Christopher Wood -
David Blank-Edelman -
Denny Schierz -
dnb＠ccs.neu.edu
-
Howard Chu -
Quanah Gibson-Mount -
Simone Piccardi