openldap-bugs

openldap-bugs@openldap.org

1 participants
21002 discussions

(ITS#6298) ldapsearch hangs after ldapadd
by tculjaga＠gmail.com 17 Sep '09

17 Sep '09

Full_Name: Tihomir Culjaga Version: 2.4.18 OS: CentOS 5.3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (93.136.5.55) Hi, i have an issue with ldapsearch. I have a ldif file that i use to recreate the database using slapadd. When this is done, i perform a ldapsearch of an entire tree (~ 100k entries) and this is done in an acceptable time 40 - 60 seconds. After that, i add some more entries (4 of them to be precise) using ldapadd and perform the same whole-tree search. The search works (ldapsearch returning entries) until it reaches the last entry present in the initial ldif file. It hangs here for a while (1 - 10 minutes) and after that returns the remaining 4 entries i've previously added using ldapadd. When doing the 1st ldapsearch, slapd process takes about 30% of total memory. Later, when performing the search again, and when it hangs, slapd process takes 97% of the total system memory. The increased meory usage is noticed when the hang apairs. During this time i see the CPU usage reaching 90% as well. When it finishes, i try to stop & start ldap and i get "FAILED" on stop! /etc/init.d/ldap stop Stopping slapd: [FAILED] sudo /etc/init.d/ldap start Checking configuration files for slapd: bdb_db_open: database "dc=ot,dc=hr": unclean shutdown detected; attempting recovery. bdb_db_open: database "dc=ot,dc=hr": recovery skipped in read-only mode. Run manual recovery if errors are encountered. bdb_monitor_db_open: monitoring disabled; configure monitor database to enable config file testing succeeded [ OK ] Starting slapd: [ OK ] [root@l01lnp1 ~]# My enviorment is: /usr/local/sbin/slapd -V @(#) $OpenLDAP: slapd 2.4.18 (Sep 17 2009 10:00:09) $ With Backend BDB db_stat -V Berkeley DB 4.7.25: (May 15, 2008) <= patched with all 4 patches from oracle uname -a Linux l01lnp1 2.6.18-128.el5 #1 SMP Wed Jan 21 10:41:14 EST 2009 x86_64 x86_64 x86_64 GNU/Linux cat /etc/issue CentOS release 5.3 (Final) Kernel \r on an \m The machine HW CPU: 2 x Xeon at 3.0 GHz (bogomips: 6000.22) - dualcore RAM: 10 GB HDD: 60 GB SCSI my slapd.conf threads 4 sizelimit unlimited timelimit 3600 database bdb cachesize 7000000 dncachesize 0 idlcachesize 21000000 cachefree 10000 my DB_CONFIG set_cachesize 5 0 1 set_lg_regionmax 262144 set_lg_bsize 2097152 my custom schema sipdirector.schema my ldif file used to recreate the DB all.ldif my ldif file used to load remaining 4 entries test.ldif commands to recreate the DB from ldif file slapadd -f /usr/local/etc/openldap/slapd.conf -l /home/tculjaga/all.ldif commands to add entries with ldapadd ldapadd -h localhost -x -D cn=admin,dc=ot,dc=hr -w passwd -f /home/tculjaga/test.ldif -c commands to perform whole-tree ldapsearch ldapsearch -h localhost -x -b ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr -D cn=admin,dc=ot,dc=hr -w passwd DB information 3437278 entries du -c -h /var/lib/ldap/*.bdb 204K /var/lib/ldap/bestMatchPrefix.bdb 982M /var/lib/ldap/dn2id.bdb 2.4G /var/lib/ldap/id2entry.bdb 1.9M /var/lib/ldap/objectClass.bdb 1.3M /var/lib/ldap/originatorPrefixID.bdb 48M /var/lib/ldap/uniqueID.bdb 3.4G total I have strace logs when i perormed th esearch strace.tar.gz please let me know how can i transfer you the logs and config files. Tihomir.

1 0

Re: (ITS#6296) Strong bind doesn't work in slapd-ldap when used by slapd-relay or slapo-translucent
by masarati＠aero.polimi.it 17 Sep '09

17 Sep '09

> I have to use slapd-ldap in "strong bind" mode which means that user > binds using its own credentials and no identity assertion is performed. > If slapd-ldap is the only module that is processing request then > everything works fine. If slapd-ldap is processing an request forwarded > from slapd-relay (with slapo-rwm) or request to database with > slapo-translucent then the authentication problem occurs. The solution was a little less intrusive than the one you proposed, but your analysis was just fine and definitely effective in pointing me to the core of the issue. It should now be fixed in HEAD, please test. servers/slapd/back-ldap/bind.c new revision: 1.255; previous revision: 1.254 Thanks, p.

1 0

Re: (ITS#6200) slapd crashes under load w/ syncrepl
by imf＠u.washington.edu 17 Sep '09

17 Sep '09

We build with, % CPPFLAGS="-I/usr/local/krb5/include -I/usr/local/include/sasl" \ CFLAGS="-g" ./configure ... % make The 'CFLAGS' environmental will make sure the '-g' is injected into all generated makefiles. You can add -D_GNU_SOURCE there as well. We experienced the problems you described with <2.4.17. 2.4.17 fixed many of the segfault causes. We didn't repeat our stress test with multimaster and 2.4.17+ though, since we determined that deltasync was good enough for our purposes. It may very well be that 2.4.17+ would crash in multimaster mode with our stress test. The traceback you provided looks very similar to what we were seeing. If I have time I could run a test with 2.4.18 and multimaster and see what the results are... --- Tracy Stenvik University Computing Services 354843. University of Washington email: imf(a)u.washington.edu voice: (206) 685-3344 On Thu, 17 Sep 2009, Joacim Breiler wrote: > We are trying out the 2.4.18 release with a syncrepl Multi-Master setup and > are experiencing similar problems as mentioned above. > > It will shut down with a segfault after heavy processing/updating of about > 3000-17000 entries. We can reproduce this in about 10-20 minutes of heavy > load. However, in this short execution time we can't find any signs of memory > leakage. > > Unfortunatly I can't get any good tracebacks from gdb. I've tried compiling > openldap with: > export CPPFLAGS="-D_GNU_SOURCE -g". And also --enable-debug with /configure. > Any pointers? > > This is the traceback I get, but I guess it isn't very helpful without the > complete debugging information: > > (gdb) where > #0 0x00007fc7e24fb8eb in syncprov_op_mod (op=0x7fc7aedb37d0, rs=<value > optimized out>) at syncprov.c:1970 > #1 0x000000000049531a in overlay_op_walk () > #2 0x0000000000495e7a in ?? () > #3 0x000000000044a2e2 in fe_op_modify () > #4 0x000000000044ac67 in do_modify () > #5 0x00000000004321bf in ?? () > #6 0x0000000000432e6c in ?? () > #7 0x00000000004dd960 in ?? () > #8 0x00007fc7e2e9b3ea in start_thread () from /lib/libpthread.so.0 > #9 0x00007fc7e29f3cbd in clone () from /lib/libc.so.6 > #10 0x0000000000000000 in ?? () > (gdb) > > Could these problems be related? And is there something we can do to help > resolve it? > > Regards, > Joacim Breiler > > > > Configuration: > > dn: olcDatabase={1}hdb,cn=config > objectClass: olcDatabaseConfig > objectClass: olcHdbConfig > olcDatabase: {1}hdb > olcDbDirectory: /srv/ldap > olcSuffix: dc=hgo,dc=se > olcAccess: {0}to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=hgo, > dc=se" write by anonymous auth by self write by * none > olcLastMod: TRUE > olcRootDN: cn=admin,dc=hgo,dc=se > olcRootPW: xxxxxxxx > olcSizeLimit: -1 > olcSyncrepl: {0}rid=004 provider=ldap://diablo binddn="cn=admin,dc=hgo,dc=se > " bindmethod=simple credentials="xxxxxxx" searchbase="dc=hgo,dc=se" ty > pe=refreshAndPersist retry="5 5 300 5" timeout=1 > olcSyncrepl: {1}rid=003 provider=ldap://mephisto binddn="cn=admin,dc=hgo,dc= > se" bindmethod=simple credentials="xxxxxxx" searchbase="dc=hgo,dc=se" > type=refreshAndPersist retry="5 5 300 5" timeout=1 > olcMirrorMode: TRUE > olcDbCheckpoint: 2048 5 > olcDbIDLcacheSize: 90000 > olcDbIndex: objectClass,entryCSN,entryUUID,cn eq > olcDbCacheSize: 10000 > olcDbConfig: {0}set_cachesize 1 0 0 > olcDbConfig: {1}set_lk_max_objects 32000 > olcDbConfig: {2}set_lk_max_locks 32000 > olcDbConfig: {3}set_lk_max_lockers 1500 > olcDbConfig: {4}set_flags DB_LOG_AUTOREMOVE > olcDbConfig: {5}set_lg_bsize 104857600 > olcDbConfig: {6}set_lg_dir /srv/ldap > > >

1 0

Re: (ITS#6200) slapd crashes under load w/ syncrepl
by quanah＠zimbra.com 17 Sep '09

17 Sep '09

--On Thursday, September 17, 2009 1:21 PM +0000 joacim.breiler(a)hgo.se wrote: > We are trying out the 2.4.18 release with a syncrepl Multi-Master setup > and are experiencing similar problems as mentioned above. > > It will shut down with a segfault after heavy processing/updating of > about 3000-17000 entries. We can reproduce this in about 10-20 minutes > of heavy load. However, in this short execution time we can't find any > signs of memory leakage. > > Unfortunatly I can't get any good tracebacks from gdb. I've tried > compiling openldap with: > export CPPFLAGS="-D_GNU_SOURCE -g". And also --enable-debug with > /configure. Any pointers? > > This is the traceback I get, but I guess it isn't very helpful without > the complete debugging information: When you ran "make install" did you put STRIP=""? Like make install STRIP="" otherwise, the slapd binary has all its debugging information stripped out. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#6200) slapd crashes under load w/ syncrepl
by joacim.breiler＠hgo.se 17 Sep '09

17 Sep '09

We are trying out the 2.4.18 release with a syncrepl Multi-Master setup and are experiencing similar problems as mentioned above. It will shut down with a segfault after heavy processing/updating of about 3000-17000 entries. We can reproduce this in about 10-20 minutes of heavy load. However, in this short execution time we can't find any signs of memory leakage. Unfortunatly I can't get any good tracebacks from gdb. I've tried compiling openldap with: export CPPFLAGS="-D_GNU_SOURCE -g". And also --enable-debug with /configure. Any pointers? This is the traceback I get, but I guess it isn't very helpful without the complete debugging information: (gdb) where #0 0x00007fc7e24fb8eb in syncprov_op_mod (op=0x7fc7aedb37d0, rs=<value optimized out>) at syncprov.c:1970 #1 0x000000000049531a in overlay_op_walk () #2 0x0000000000495e7a in ?? () #3 0x000000000044a2e2 in fe_op_modify () #4 0x000000000044ac67 in do_modify () #5 0x00000000004321bf in ?? () #6 0x0000000000432e6c in ?? () #7 0x00000000004dd960 in ?? () #8 0x00007fc7e2e9b3ea in start_thread () from /lib/libpthread.so.0 #9 0x00007fc7e29f3cbd in clone () from /lib/libc.so.6 #10 0x0000000000000000 in ?? () (gdb) Could these problems be related? And is there something we can do to help resolve it? Regards, Joacim Breiler Configuration: dn: olcDatabase={1}hdb,cn=config objectClass: olcDatabaseConfig objectClass: olcHdbConfig olcDatabase: {1}hdb olcDbDirectory: /srv/ldap olcSuffix: dc=hgo,dc=se olcAccess: {0}to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=hgo, dc=se" write by anonymous auth by self write by * none olcLastMod: TRUE olcRootDN: cn=admin,dc=hgo,dc=se olcRootPW: xxxxxxxx olcSizeLimit: -1 olcSyncrepl: {0}rid=004 provider=ldap://diablo binddn="cn=admin,dc=hgo,dc=se " bindmethod=simple credentials="xxxxxxx" searchbase="dc=hgo,dc=se" ty pe=refreshAndPersist retry="5 5 300 5" timeout=1 olcSyncrepl: {1}rid=003 provider=ldap://mephisto binddn="cn=admin,dc=hgo,dc= se" bindmethod=simple credentials="xxxxxxx" searchbase="dc=hgo,dc=se" type=refreshAndPersist retry="5 5 300 5" timeout=1 olcMirrorMode: TRUE olcDbCheckpoint: 2048 5 olcDbIDLcacheSize: 90000 olcDbIndex: objectClass,entryCSN,entryUUID,cn eq olcDbCacheSize: 10000 olcDbConfig: {0}set_cachesize 1 0 0 olcDbConfig: {1}set_lk_max_objects 32000 olcDbConfig: {2}set_lk_max_locks 32000 olcDbConfig: {3}set_lk_max_lockers 1500 olcDbConfig: {4}set_flags DB_LOG_AUTOREMOVE olcDbConfig: {5}set_lg_bsize 104857600 olcDbConfig: {6}set_lg_dir /srv/ldap

1 0

(ITS#6297) Allow hex in serverID directive
by rein＠OpenLDAP.org 17 Sep '09

17 Sep '09

Full_Name: Rein Tollevik Version: CVS HEAD OS: Irrelevant URL: Submission from: (NULL) (2a01:600:0:1:21c:23ff:feab:61cd) Submitted by: rein The serverID directive is specified as a decimal integer value, but its hexadecimal value is used everywhere else. I.e in the syncrepl cookies and the ID field of the CSN attributes. This can be a bit confusing, so it would be nice if the hex value could be used in the configuration as well. A patch that allows this with an 0x prefix is coming. -- Rein Tollevik Basefarm AS

1 0

Re: (ITS#6292) Cannot get all attributes for slapd-ldap configuration
by masarati＠aero.polimi.it 17 Sep '09

17 Sep '09

lkhlee(a)ogcio.gov.hk wrote: > Full_Name: Law > Version: 2.2.13 ^^^ ?!? 2.2.13 was obsoleted in 2005! Even though yours seems to be a usage question, even if it were related to a bug, do you really expect anyone to fix it? Please upgrade to the latest version and see if the problem persists. And direct further requests to the most appropriate mailing list, unless you find clear evidence of a software bug. This ITS will be closed. p. > OS: red hat linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (202.128.253.141) > > > I used the default openldap server in redhat and using the slapd-ldap(5) to > redirect the ldap request from client to another ldap server. However, I found > that not all the attributes will be shown in the result when i use JXplorer to > connect to my ldap server while all attributes will be shown when I directly > connect to the destination ldap server using anonymous binding. My slapd.conf > (about slapd-ldap) is configured as follow: > > database ldap > suffix "" > #rootdn "cn=admin,dc=example,dc=com" > #rootpw "admin" > #binddn "anonymous" > #bindpw "" > uri "ldap://10.xxx.xxx.xxx" > lastmod off > > Do you have any idea about this?? > Thank you very much!! > >

1 0

(ITS#6296) Strong bind doesn't work in slapd-ldap when used by slapd-relay or slapo-translucent
by mrubas＠kerio.com 17 Sep '09

17 Sep '09

Full_Name: Martin Rubas Version: 2.4.18, 2.4.15 OS: Windows, Linux/Ubuntu URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (195.113.184.10) Hello. I have to use slapd-ldap in "strong bind" mode which means that user binds using its own credentials and no identity assertion is performed. If slapd-ldap is the only module that is processing request then everything works fine. If slapd-ldap is processing an request forwarded from slapd-relay (with slapo-rwm) or request to database with slapo-translucent then the authentication problem occurs. First detected in OpenLDAP 2.4.15 but still present in 2.4.18. === Test Setup === Let's suppose that we have remote LDAP server with disabled anonymous binding. It could be Active Directory or OpenLDAP with following configuration: # --- config for remote server (remote.host.net) --- # [cut-off] command schema, slapd files & logs definitions # Load of modules: modulepath /usr/lib/ldap moduleload back_hdb.la # VERY IMPORTANT: Disable anonymous binding disallow bind_anon # Configuration Database database config rootdn "cn=admin,cn=config" rootpw secret # realone.net - local Repository database hdb suffix "dc=realone,dc=net" rootdn "cn=admin,dc=realone,dc=net" rootpw secret directory /var/lib/slapd/realone.net index objectClass eq index cn eq index uid eq Let's say there is a common content in database: # --- Directory content (just an overview) --- dn: dc=realone,dc=net # top, domain dn: ou=people,dc=realone,dc=net # organizationalUnit dn: ou=groups,dc=realone,dc=net # organizationalUnit dn: cn=idassert,dc=realone,dc=net # simpleSecurityObject,organizationalRole dn: cn=admin,dc=realone,dc=net # same as above dn: uid=super.account,ou=people,dc=realone,dc=net # inetOrgPerson,posixAccount, shadowAccount dn: uid=power.account,ou=people,dc=realone,dc=net # same as above dn: uid=basic.account,ou=people,dc=realone,dc=net # same as above dn: cn=admins,ou=groups,dc=realone,dc=net # posixGroup dn: cn=users,ou=groups,dc=realone,dc=net # same as above I want to use another OpenLDAP server that is configured with slapo-translucent: # --- config for local server (trans.local.net) --- # [cut-off] command schema, slapd files & logs definitions # Configuration Database database config rootdn "cn=admin,cn=config" rootpw secret # realone.net - local database extending the remote one database bdb suffix "dc=realone,dc=net" directory ./database/realone.net index objectClass eq index cn eq index uid eq overlay translucent uri ldap://remote.host.net:389/ chase-referrals true rebind-as-user true ... or with slapd-relay (this is probably very very very rare use case but I used it for debugging): # --- config for local server (relay.local.net) --- # [cut-off] command schema, slapd files & logs definitions # Configuration Database database config rootdn "cn=admin,cn=config" rootpw secret # domain (forwarded to remote repository) database ldap suffix "dc=realone,dc=net" uri ldap://remote.host.net chase-referrals yes # domain alias database relay suffix "dc=virtual,dc=net" relay "dc=realone,dc=net" overlay rwm rwm-suffixmassage "dc=virtual,dc=net" "dc=realone,dc=net" === Test Cases === Now, we can do usual search: ldapsearch -x -w secret -h <host-D <bindDN-b <baseDN-s sub "(objectClass=inetOrgPerson)" dn displayName mail (in my environment I should receive 3 records below ou=people,dc=realone,dc=net). But the story is getting interesting here. The command works only in cases that slapd-ldap is used directly: =================================================================================================== # | Host | BindDN | BaseDN | Result --+-----------------+----------------------------+-------------------+----------------------------- 1 | remote.host.net | cn=admin,dc=realone,dc=net | dc=realone,dc=net | Success (3 records) --+-----------------+----------------------------+-------------------+----------------------------- 2 | trans.local.net | cn=admin,dc=realone,dc=net | dc=realone,dc=net | Failure: Anon.bind.n.allwd. --+-----------------+----------------------------+-------------------+----------------------------- 3 | relay.local.net | cn=admin,dc=realone,dc=net | dc=realone,dc=net | Success (3 records) --+-----------------+----------------------------+-------------------+----------------------------- 4 | relay.local.net | cn=admin,dc=realone,dc=net | dc=virtual,dc=net | Success (3 records) !!!! --+-----------------+----------------------------+-------------------+----------------------------- 5 | relay.local.net | cn=admin,dc=virtual,dc=net | dc=virtual,dc=net | Failure: Anon.bind.n.allwd. =================================================================================================== The failures of case #2 and #5 are not expected because the connection to remote server is handled by slapd-ldap configured the same way like in case 3. The success in case #3 is correct (although I didn't expected it) and it was guide to solution. I noticed that each time the request fail the LDAP connection is correctly opened and bind request is successful. But the search request was processed using a new connection that was created for it and anonymous bind executed with failure. === Debugging === While debugging, I found that ldap_back_getconn() is called by ldap_back_bind() as well as ldap_back_search() request handlers. For operations other then op_bind it tests whether connection associated with Operation (op->o_hdr->oh_conn) was used for authentication in previous step. If slapd-ldap is NOT using identity assertion then it checks Operation using SLAP_IS_AUTHZ_BACKEND macro whether op->o_conn->c_authz_backend is not NULL and is equal to current backend. This test works fine is the slapd-ldap backend is the only which processes the LDAP request because op->o_conn->c_authz_backend is set in the binding handler of the frontend module - fe_op_bind(). In this, case the test succeeds and connection created during the binding is used also for searching. If the slapd-ldap module is used as nested module in slapo-translucent (case #2), then the fe_op_bind() sets variable op->o_conn->c_authz_backend to the backend extended by slapo-translucent. Macro SLAP_IS_AUTHZ_BACKEND returns false and a new LDAP connection is allocated causing failure due anonymouse binding. If the slapd-ldap module is target for slapd-relay then it behaves similar way. In case #5 the bind request is forwarded to slapd-ldap but slapd-relay is associated with connection as the backend who did the bind (op->o_conn->c_authz_backend) and subsequent search request fails the same way like above. The success in case #4 is a kind of singularity ;-) - bind request is forwarded directly from frontend to slapd-ldap and therefore op->o_conn->c_authz_backend is set correctly, the subsequent search request goes first to slapd-relay which forwards it to slapd-ldap but SLAP_IS_AUTHZ_BACKEND test succeeds. === Patches uploaded to ftp.openldap.org === mrubas(a)kerio.com-090917.slapd_back-relay_op.c.2_4_15.patch mrubas(a)kerio.com-090917.slapd_back-relay_op.c.2_4_18.patch mrubas(a)kerio.com-090917.slapd_overlays_translucent.c.2_4_18.patch Note: There is probably a better way how to fix that. I guess, that fix to slapd-ldap binding handler would be more efficient but I didn't find the right place to add code assigning the auth. backend to connection.

1 0

(ITS#6292) Cannot get all attributes for slapd-ldap configuration
by lkhlee＠ogcio.gov.hk 15 Sep '09

15 Sep '09

Full_Name: Law Version: 2.2.13 OS: red hat linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (202.128.253.141) I used the default openldap server in redhat and using the slapd-ldap(5) to redirect the ldap request from client to another ldap server. However, I found that not all the attributes will be shown in the result when i use JXplorer to connect to my ldap server while all attributes will be shown when I directly connect to the destination ldap server using anonymous binding. My slapd.conf (about slapd-ldap) is configured as follow: database ldap suffix "" #rootdn "cn=admin,dc=example,dc=com" #rootpw "admin" #binddn "anonymous" #bindpw "" uri "ldap://10.xxx.xxx.xxx" lastmod off Do you have any idea about this?? Thank you very much!!

1 0

(ITS#6291) nssov getgroupbymembers is broken
by hyc＠OpenLDAP.org 14 Sep '09

14 Sep '09

Full_Name: Howard Chu Version: 2.4.18/HEAD OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (76.91.220.157) Submitted by: hyc The getgroupbymember nss function expects the list of groups to be terminated by NSLCD_RESULT_END. The NSSOV_HANDLE macro is neglecting to write this result. (Technically, every response is supposed to end with this code, but none of the other nss functions try to read it, so this bug doesn't impact any of them.)

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs