Debian Squeeze: Slapd subtree disappears, but ldapsearch finds it | unable to allocate memory for mutex; resize mutex region
6:39 a.m.
hi,
on the last hour I had a very strange problem:
I have a Debian Squeeze with Slapd installed and it was working a long time, but today one
subtree disappears completely from the ldap browser (Apache Directory Studio). I
wasn't accessible anymore, but with ldapsearch I was able to see the tree. Also I
tried to add the LDIF again to the LDAP, but slapd says: "... exists ...."
The only way to get it back working again, was to restore the plain BDB files from the
backupjob yesterday.
I absolutely don't know, what happens. The only thing I changed, I reinstalled the
second LDAP (n-way master) on a new host, with a config only and let the second ldap
synchronize with the main LDAP. After ~20-40 minutes the job was done and the second was
up to date. That was yesterday.
-
The subtree was also missing on the second LDAP ... so, the synchronizing did the same ...
So, I deleted all *dbd* *log* on ldap2 (after the restore from the main LDAP) and wanted
to synchronizing again the full tree ... but I get many warnings:
dc=...... unable to allocate memory for mutex; resize mutex region
any suggestions?
cu denny
2:42 p.m.
--On Tuesday, April 30, 2013 3:39 PM +0200 Denny Schierz
<linuxmail(a)4lin.net> wrote:
hi,
dc=...... unable to allocate memory for mutex; resize mutex region
You probably need to fix DB_CONFIG at a guess. I'd personally avoid using
Debian's openldap build, there are serious problems with it, and
significant problems with the way in which they build their BDB package too.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
3:08 p.m.
hi,
Am 30.04.2013 um 23:42 schrieb Quanah Gibson-Mount <quanah(a)zimbra.com>:
--On Tuesday, April 30, 2013 3:39 PM +0200 Denny Schierz
<linuxmail(a)4lin.net> wrote:
> hi,
> dc=...... unable to allocate memory for mutex; resize mutex region
You probably need to fix DB_CONFIG at a guess. I'd personally avoid using
Debian's openldap build, there are serious problems with it, and significant problems
with the way in which they build their BDB package too.
hmm, bad. I try to build 2.4.35 with the Debian build files, but without several patches.
Some packages needs libldap etc. pp.
DB_CONFIG looks like this one:
set_cachesize 0 2097152 0
set_lg_bsize 2097512
set_lk_max_objects 1500
set_lk_max_locks 1500
set_lk_max_lockers 1500
set_flags DB_LOG_AUTOREMOVE
I only added the last line, because of filling the disks ...
any idea what happens to the first problem?
cu denny
10:26 a.m.
--On May 1, 2013 12:08:51 AM +0200 Denny Schierz <linuxmail(a)4lin.net> wrote:
hmm, bad. I try to build 2.4.35 with the Debian build files, but
without
several patches. Some packages needs libldap etc. pp.
You should not try and replace the broken system OpenLDAP. Build OpenLDAP
yourself out into /usr/local or /opt, etc.
Also, as I already stated, the BDB libraries provided by Debian are built
incorrectly as well, and should not be used.
As for DB_CONFIG tuning, I suggest reading over
<http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning#Berkeley_DB_DB_CO...
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
12:59 p.m.
On Wed, May 01, 2013 at 10:26:58AM -0700, Quanah Gibson-Mount wrote:
--On May 1, 2013 12:08:51 AM +0200 Denny Schierz <linuxmail(a)4lin.net> wrote:
>hmm, bad. I try to build 2.4.35 with the Debian build files, but without
>several patches. Some packages needs libldap etc. pp.
You should not try and replace the broken system OpenLDAP. Build
OpenLDAP yourself out into /usr/local or /opt, etc.
Also, as I already stated, the BDB libraries provided by Debian are
built incorrectly as well, and should not be used.
How do people with generic sysadmin skills figure out which packaged components are safe
to use with a non-distribution-packaged OpenLDAP? Having built a production slapd on
Debian all I have is "well, worked and didn't crash for me".
As for DB_CONFIG tuning, I suggest reading over
<http://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning#Berkeley_DB_DB_CO...
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
2:08 p.m.
--On Wednesday, May 01, 2013 3:59 PM -0400 Christopher Wood
<christopher_wood(a)pobox.com> wrote:
How do people with generic sysadmin skills figure out which packaged
components are safe to use with a non-distribution-packaged OpenLDAP?
Having built a production slapd on Debian all I have is "well, worked and
didn't crash for me".
I never use distribution software for OpenLDAP except gcc. I.e., I build
openssl, heimdal, cyrus-sasl, etc, all myself. And now with MDB, I no
longer have to build BDB. ;)
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11:32 p.m.
hi,
Am 01.05.2013 um 23:08 schrieb Quanah Gibson-Mount <quanah(a)zimbra.com>:
--On Wednesday, May 01, 2013 3:59 PM -0400 Christopher Wood
<christopher_wood(a)pobox.com> wrote:
> How do people with generic sysadmin skills figure out which packaged
> components are safe to use with a non-distribution-packaged OpenLDAP?
> Having built a production slapd on Debian all I have is "well, worked and
> didn't crash for me".
I never use distribution software for OpenLDAP except gcc. I.e., I build openssl,
heimdal, cyrus-sasl, etc, all myself. And now with MDB, I no longer have to build BDB. ;)
but than you have to download, patch and update security fixes by your self.
I have now build Openldap 2.4.35 with the system libs. In a few weeks Wheezy is out and I
hope, that the BDB files are not "broken" as you said.
cu denny
7:08 a.m.
--On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz
<linuxmail(a)4lin.net> wrote:
but than you have to download, patch and update security fixes by
your
self.
Yep. Part of being a competent sys admin anyhow.
I have now build Openldap 2.4.35 with the system libs. In a few
weeks
Wheezy is out and I hope, that the BDB files are not "broken" as you
said.
Debian's BDB libs will always be broken. I filed a bug report with them on
it ages ago, and they declined to fix it.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
8:52 a.m.
On 05/02/2013 04:08 PM, Quanah Gibson-Mount wrote:
--On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz
<linuxmail(a)4lin.net> wrote:
> but than you have to download, patch and update security fixes by your
> self.
Yep. Part of being a competent sys admin anyhow.
Sorry, I disagree.
A competent sysadmin has to make choices on how he has to employ his
time. When having limited resources the choice you suggest can be easily
seen as an incompetent wasting of time.
For example when you have to manage > 70 small server for > 70 school,
applying security upgrade by recompiling apache, bind, samba, openldap
(just to cite some of the services on them) every time is plain wrong.
It's a waste of the scarce sysadmin time that could not be afforded.
That's just an example, but there are lot of situations in which the
solution to bad distribution packaging cannot be "recompile it by
yourself and reinstall". Better to point to another distribution or to a
good packaging (if they exist). Otherwise every competent sysadmin will
use the packages, also if they are suboptimal.
I'm sorry to hear that Debian OpenLDAP packages are in a such bad state,
but if, as it seems, there no distribution getting OpenLDAP right (I
heard complaints also about RedHat), then I start thinking that
something is not working fine, at least on the user end of OpenLDAP
distribution.
Simone
--
Simone Piccardi Truelite Srl
piccardi(a)truelite.it (email/jabber) Via Monferrato, 6
Tel. +39-347-1032433 50142 Firenze
http://www.truelite.it Tel. +39-055-7879597 Fax. +39-055-7333336
9:10 a.m.
--On Thursday, May 02, 2013 5:52 PM +0200 Simone Piccardi
<piccardi(a)truelite.it> wrote:
On 05/02/2013 04:08 PM, Quanah Gibson-Mount wrote:
> --On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz
> <linuxmail(a)4lin.net> wrote:
>
>> but than you have to download, patch and update security fixes by your
>> self.
>
> Yep. Part of being a competent sys admin anyhow.
Sorry, I disagree.
A competent sysadmin has to make choices on how he has to employ his
time. When having limited resources the choice you suggest can be easily
seen as an incompetent wasting of time.
For example when you have to manage > 70 small server for > 70 school,
applying security upgrade by recompiling apache, bind, samba, openldap
(just to cite some of the services on them) every time is plain wrong.
It's a waste of the scarce sysadmin time that could not be afforded.
Sorry, as someone who used to maintain some 600 servers for a major
university running a very wide variety of services, I disagree. If you
can't figure out an easy way to build and distribute your own packages in
an automated fashion, you are putting yourself at the mercy of your
distribution provider's capabilities and competence, which has been shown
time and again to be severely lacking in multiple areas.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
11:18 a.m.
On 05/02/2013 06:10 PM, Quanah Gibson-Mount wrote:
Sorry, as someone who used to maintain some 600 servers for a major
university running a very wide variety of services, I disagree. If you
can't figure out an easy way to build and distribute your own packages
in an automated fashion, you are putting yourself at the mercy of your
distribution provider's capabilities and competence, which has been
shown time and again to be severely lacking in multiple areas.
I can figure it, but that was not my point. The point is that in a lot
of situations people prefer to be at the mercy of a distribution
provider and less dependent from a skilled sysadmin (or an external
consultant) because this would cost them less on update management.
When you have strong budget limits (both in money or skills you can use)
inferior solution are willingly chosen because they cost less or have
less resource requirements. What I cannot agree is calling this kind of
choice incompetence.
Simone
--
Simone Piccardi Truelite Srl
piccardi(a)truelite.it (email/jabber) Via Monferrato, 6
Tel. +39-347-1032433 50142 Firenze
http://www.truelite.it Tel. +39-055-7879597 Fax. +39-055-7333336
12:33 p.m.
--On Thursday, May 02, 2013 8:18 PM +0200 Simone Piccardi
<piccardi(a)truelite.it> wrote:
On 05/02/2013 06:10 PM, Quanah Gibson-Mount wrote:
>
> Sorry, as someone who used to maintain some 600 servers for a major
> university running a very wide variety of services, I disagree. If you
> can't figure out an easy way to build and distribute your own packages
> in an automated fashion, you are putting yourself at the mercy of your
> distribution provider's capabilities and competence, which has been
> shown time and again to be severely lacking in multiple areas.
>
I can figure it, but that was not my point. The point is that in a lot of
situations people prefer to be at the mercy of a distribution provider
and less dependent from a skilled sysadmin (or an external consultant)
because this would cost them less on update management.
It costs them "less" until such a time as they have a catastrophic failure
because of the fact they tried to cut corners by relying on distribution
software. Then they get the cost of dealing with the mess they are to
blame for by trying to cut costs in the first place.
I would note that OpenLDAP is not unique with the issues caused by
distribution builds. The postfix authors give the same advice. Don't use
the outdated distribution packages:
<http://archives.neohapsis.com/archives/postfix/2013-05/0021.html>
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
9:12 a.m.
--On Thursday, May 02, 2013 5:52 PM +0200 Simone Piccardi
<piccardi(a)truelite.it> wrote:
I'm sorry to hear that Debian OpenLDAP packages are in a such bad
state,
but if, as it seems, there no distribution getting OpenLDAP right (I
heard complaints also about RedHat), then I start thinking that something
is not working fine, at least on the user end of OpenLDAP distribution.
The OpenLDAP foundation has zero input or control into how distribution
providers build their OpenLDAP packages. Thus the end users are at the
mercy of the distribution provider's decisions on building OpenLDAP, which
are quite often extremely flawed.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
9:35 a.m.
Hi Quanah-
On May 2, 2013, at 12:12 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote:
The OpenLDAP foundation has zero input or control into how
distribution providers build their OpenLDAP packages. Thus the end users are at the mercy
of the distribution provider's decisions on building OpenLDAP, which are quite often
extremely flawed.
Yes, it is a big bummer. Has the OpenLDAP foundation ever considered publishing any
official guidelines that could be used both by these distributions and individuals who
want to do their own packages? Just two lists of "Do this:…" and "Don't
do this: …" would probably go a long way.
-- dNb
9:53 a.m.
--On Thursday, May 02, 2013 12:35 PM -0400 David Blank-Edelman
<dnb(a)ccs.neu.edu> wrote:
Yes, it is a big bummer. Has the OpenLDAP foundation ever considered
publishing any official guidelines that could be used both by these
distributions and individuals who want to do their own packages? Just two
lists of "Do this:…" and "Don't do this: …" would probably go a
long
way.
The distribution maintainers are quite aware of the objections to the way
in which they build their software. Their decisions have little to do with
needs of the end users.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
9:58 a.m.
On May 2, 2013, at 12:53 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote:
The distribution maintainers are quite aware of the objections to the
way in which they build their software. Their decisions have little to do with needs of
the end users.
Ok, then perhaps guidelines for the rest of us? I know I try to pick up tips whenever
someone says "don't do it like RH, they <blah>" but it would be really
swell if these things were written down someplace in a single spot. Can you see any
downside to having this documented for the public?
-- dNb
1:52 p.m.
--On Thursday, May 02, 2013 12:58 PM -0400 David Blank-Edelman
<dnb(a)ccs.neu.edu> wrote:
On May 2, 2013, at 12:53 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>
wrote:
> The distribution maintainers are quite aware of the objections to the
> way in which they build their software. Their decisions have little to
> do with needs of the end users.
Ok, then perhaps guidelines for the rest of us? I know I try to pick up
tips whenever someone says "don't do it like RH, they <blah>" but it
would be really swell if these things were written down someplace in a
single spot. Can you see any downside to having this documented for the
public?
There is not a whole lot to it.
a) Link to OpenSSL, not gnutls (debian/ubuntu default) or NSS (rhel default)
b) If you are going to use BDB as your underlying database software and are
on Linux, make sure to pass the following flags to configure:
--enable-posixmutexes --with-mutex=POSIX/pthreads
Post build:
c) Generally, I advise preloading a memory allocator such as tcmalloc from
google perf tools. Particularly important for Linux to avoid using the
horrid glibc allocator.
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
2:09 p.m.
On May 2, 2013, at 4:53 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote:
There is not a whole lot to it.
a) Link to OpenSSL, not gnutls (debian/ubuntu default) or NSS (rhel default)
b) If you are going to use BDB as your underlying database software and are on Linux,
make sure to pass the following flags to configure: --enable-posixmutexes
--with-mutex=POSIX/pthreads
Post build:
c) Generally, I advise preloading a memory allocator such as tcmalloc from google perf
tools. Particularly important for Linux to avoid using the horrid glibc allocator.
That's great information, thanks. Anything special if you plan to use MDB?
-- dNb
2:13 p.m.
--On Thursday, May 02, 2013 5:09 PM -0400 dnb(a)ccs.neu.edu wrote:
That's great information, thanks. Anything special if you plan to
use MDB?
I would use the current RE24 source to pick up some fixes since 2.4.35.
It's finally been stable for me with that in place. You may or may not
want to set two flags for mdb:
olcDbEnvFlags: writemap
olcDbEnvFlags: nometasync
I set them myself.
You can get the current source via:
<http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=snapshot;h=refs...
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
9:18 a.m.
Simone Piccardi wrote:
On 05/02/2013 04:08 PM, Quanah Gibson-Mount wrote:
> --On Thursday, May 02, 2013 8:32 AM +0200 Denny Schierz
> <linuxmail(a)4lin.net> wrote:
>
>> but than you have to download, patch and update security fixes by your
>> self.
>
> Yep. Part of being a competent sys admin anyhow.
Sorry, I disagree.
A competent sysadmin has to make choices on how he has to employ his
time. When having limited resources the choice you suggest can be easily
seen as an incompetent wasting of time.
For example when you have to manage > 70 small server for > 70 school,
applying security upgrade by recompiling apache, bind, samba, openldap
(just to cite some of the services on them) every time is plain wrong.
It's a waste of the scarce sysadmin time that could not be afforded.
A competent sysadmin knows how to leverage tools such that 7 servers or 7000
servers requires the same amount of hands-on time. One element of making this
feasible is certainly to have the minimum possible variations in deployed
configurations. But a frozen configuration that you built yourself with known
components is just as viable for this purpose as one you obtained from a
distro. And in most cases, due to distro lag times, a config you build
yourself will be superior.
That's just an example, but there are lot of situations in which
the
solution to bad distribution packaging cannot be "recompile it by
yourself and reinstall". Better to point to another distribution or to a
good packaging (if they exist). Otherwise every competent sysadmin will
use the packages, also if they are suboptimal.
I'm sorry to hear that Debian OpenLDAP packages are in a such bad state,
but if, as it seems, there no distribution getting OpenLDAP right (I
heard complaints also about RedHat), then I start thinking that
something is not working fine, at least on the user end of OpenLDAP
distribution.
Simone
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
3611
days inactive
3613
days old
openldap-technical@openldap.org
19 comments
7 participants
participants (7)
-
Christopher Wood -
David Blank-Edelman -
Denny Schierz -
dnb＠ccs.neu.edu
-
Howard Chu -
Quanah Gibson-Mount -
Simone Piccardi