- openldap-bugs - openldap.org

(ITS#5917) slapd crashes with assertion failed and SIGABRT
by guido＠keamera.org 02 Feb '09

02 Feb '09

Full_Name: Guido Landi Version: 2.3.43 OS: linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.246.126.62) Hello, I'm using OpenLDAP 2.3.43 with ldap backend as a proxy to Active Directory for users autentication on linux(gentoo). OpenLDAP keep on crashing while testing it through the nss_ldap module using the "id" unix command. It often crashes for an assertion failed(that i can easily reproduce) but sometimes with a segmentation fault or a glibc error related to free()(heap corruption?). I found as a workaround that if I set the maximum threads number to 2 it works just fine. I'm using glibc-2.6.1 with NPTL and I have compiled OpenLDAP with gcc-4.1.2 and 3.4.6. Here is my slapd.conf, ldap.conf(used by the nss_ldap module) and some backtraces. #### slapd.conf #### pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/local.schema modulepath /usr/lib/openldap/openldap moduleload rwm overlay rwm rwm-map attribute cn sAMAccountName database ldap suffix "dc=mydomain,dc=it" rootdn "ou=CV Utenti,dc=mydomain,dc=it" lastmod off uri ldaps://mydc.mydomain.it/ rebind-as-user chase-referrals YES index objectClass eq,sub index uidNumber,gidNumber eq index memberUid eq index sAMAccountName eq,subinitial #concurrency 0 #threads 2 TLSCACertificateFile /etc/openldap/ssl/cacert.pem TLSCertificateFile /etc/openldap/ssl/server.pem TLSCertificateKeyFile /etc/openldap/ssl/server.key TLSCipherSuite ALL TLSCRLCheck none TLSReqCert none TLSVerifyClient never #### ldap.conf #### uri ldaps://ldap-proxy.mydomain.it port 636 base OU=mygroup,dc=mydomain,dc=it scope sub binddn CN=myaccount,DC=mydomain,DC=it bindpw password bind_policy soft ldap_version 3 #nss_schema rfc2307bis #debug 1000 REFERRALS no pam_login_attribute cn pam_filter objectClass=user pam_password ad pam_groupdn CN=UNIX Admins,DC=mydomain,DC=it #pam_groupdn CN=UNIX-USERS,DC=mydomain,DC=it pam_member_attribute member nss_base_passwd OU=Users,DC=mydomain,DC=it nss_base_shadow OU=Users,DC=mydomain,DC=it nss_base_group DC=mydomain,DC=it?(objectClass=group) nss_map_objectclass posixAccount user nss_map_objectclass shadowAccount user nss_map_attribute uid uid nss_map_attribute uniqueMember member #nss_map_attribute uniqueMember msSFU30PosixMemberOf nss_map_attribute userPassword unixUserPassword nss_map_attribute homeDirectory unixHomeDirectory nss_map_attribute loginShell loginShell nss_map_attribute gecos name nss_map_objectclass posixGroup group nss_reconnect_tries 4 nss_reconnect_sleeptime 1 #nss_reconnect_maxsleeptime 16 nss_reconnect_maxconntries 6 #nss_initgroups backlink #nss_max_group_depth 100 ssl yes #### backtraces #### put_simple_filter: "member=cn=certsvc_dcom_access,cn=users,dc=mydomain,dc=it" ldap_send_initial_request ldap_send_server_request ldap_back_retry: conn 0x826a628 refcnt=2 unable to retry. slapd: bind.c:157: ldap_back_conn_delete: Assertion `!(*(&((lc))->lc_lcflags) & (((0x00000020U))))' failed. Program received signal SIGABRT, Aborted. [Switching to Thread 0xb722eb90 (LWP 25435)] 0xb7b711c7 in raise () from /lib/libc.so.6 (gdb) bt #0 0xb7b711c7 in raise () from /lib/libc.so.6 #1 0xb7b72998 in abort () from /lib/libc.so.6 #2 0xb7b6a7e5 in __assert_fail () from /lib/libc.so.6 #3 0x081147d8 in ldap_back_conn_delete () #4 0x081150e1 in ?? () #5 0x08217930 in ?? () #6 0x0826a628 in ?? () #7 0xb722cde8 in ?? () #8 0x081166f5 in ldap_back_release_conn_lock () Backtrace stopped: frame did not save the PC ber_flush: 104 bytes to sd 11 ldap_result ld 0x825dfb0 msgid 5 ldap_chkResponseList ld 0x825dfb0 msgid 5 all 0 ldap_chkResponseList returns ld 0x825dfb0 NULL Program received signal SIGABRT, Aborted. [Switching to Thread 0xb692db90 (LWP 25728)] 0xb7b711c7 in raise () from /lib/libc.so.6 (gdb) bt #0 0xb7b711c7 in raise () from /lib/libc.so.6 #1 0xb7b72998 in abort () from /lib/libc.so.6 #2 0xb7b6a7e5 in __assert_fail () from /lib/libc.so.6 #3 0xb7f8da8c in ber_flush () from /usr/lib/liblber-2.3.so.0 #4 0xb7fb9005 in ldap_int_flush_request () from /usr/lib/libldap_r-2.3.so.0 #5 0xb7fb94da in ldap_send_server_request () from /usr/lib/libldap_r-2.3.so.0 #6 0xb7fb8fa9 in ldap_send_initial_request () from /usr/lib/libldap_r-2.3.so.0 #7 0xb7fa8188 in ldap_search_ext () from /usr/lib/libldap_r-2.3.so.0 #8 0x080f6eb0 in ldap_back_search () #9 0x08079a9f in fe_op_search () #10 0x080df1fa in overlay_op_walk () #11 0x080df3b1 in ?? () #12 0x082968d0 in ?? () #13 0xb692d18c in ?? () #14 0x00000002 in ?? () #15 0x08217190 in ?? () #16 0x00000000 in ?? () put_simple_filter: "member=cn=unix admins,ou=cv acl,dc=mydomain,dc=it" ldap_send_initial_request ldap_send_server_request ber_scanf fmt ({it) ber: ber_scanf fmt ({) ber: slapd: io.c:210: ber_flush: Assertion `( (sb)->sb_opts.lbo_valid == 0x3 )' failed. ** ld 0xb560f238 Response Queue: Empty ldap_chkResponseList ld 0xb560f238 msgid 5 all 0 ldap_chkResponseList returns ld 0xb560f238 NULL ldap_int_select Program received signal SIGABRT, Aborted. [Switching to Thread 0xb722eb90 (LWP 29674)] ---Type <return> to continue, or q <return> to quit--- 0xb7b711c7 in raise () from /lib/libc.so.6 (gdb) bt #0 0xb7b711c7 in raise () from /lib/libc.so.6 #1 0xb7b72998 in abort () from /lib/libc.so.6 #2 0xb7b6a7e5 in __assert_fail () from /lib/libc.so.6 #3 0xb7f8da70 in ber_flush () from /usr/lib/liblber-2.3.so.0 #4 0xb7fb9005 in ldap_int_flush_request () from /usr/lib/libldap_r-2.3.so.0 #5 0xb7fb94da in ldap_send_server_request () from /usr/lib/libldap_r-2.3.so.0 #6 0xb7fb8fa9 in ldap_send_initial_request () from /usr/lib/libldap_r-2.3.so.0 #7 0xb7fa8188 in ldap_search_ext () from /usr/lib/libldap_r-2.3.so.0 #8 0x080f6e10 in ldap_back_search () #9 0x08079a97 in fe_op_search () #10 0x080df15a in overlay_op_walk () #11 0x080df311 in over_op_func () #12 0x080df395 in over_op_search () #13 0x08079549 in do_search () #14 0x0807677a in connection_operation () #15 0xb7fa30a9 in ldap_int_thread_pool_wrapper () from /usr/lib/libldap_r-2.3.so.0 #16 0xb7e5a013 in start_thread () from /lib/libpthread.so.0 #17 0xb7c0948e in clone () from /lib/libc.so.6

1 0

Re: (ITS#5916) Allow alias dereferencing in search C API
by ando＠sys-net.it 02 Feb '09

02 Feb '09

michael(a)stroeder.com wrote: > Hmm, maybe I missed something in this discussion and about the usage of > client controls. > > But I'd vote for another function which provides a complete set of > arguments for sending search requests. So if ldap_search_ext is > considered incomplete regarding alias dereferencing I'd vote for > introducing ldap_search_ext2 (or whatever name) with an additional argument. Well, that's ldap_int_search*(); let's vote for the "public" name. As I said, I think it makes sense, since alias dereferencing has always been explicitly required in the request. OTOH, I have no objection to using a client-side control, as it wouldn't add too much complexity (well, it'd require to parse a list of strings with a worst-case of N * M, with N: number of known client controls; M: client controls passed to the request), and it would pave the way for client controls in libldap :) I see Hallvard already has a request for another one. One consideration: I'm not overlooking the value of not changing the API too often; also, I fear client-side controls could be another way to wind up with revitalizing the old API instead of designing a new one - if anyone ever seriously considered the opportunity to design it. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5916) Allow alias dereferencing in search C API
by h.b.furuseth＠usit.uio.no 01 Feb '09

01 Feb '09

michael(a)stroeder.com writes: > But I'd vote for another function which provides a complete set of > arguments for sending search requests. So if ldap_search_ext is > considered incomplete regarding alias dereferencing I'd vote for > introducing ldap_search_ext2 (or whatever name) with an additional > argument. Yes, the missing aliases argument is an API design bug. The API almost but not quite has functions that match the protocol messages. If we are going to export yet another search function, I'd like to see another feature added as well: A way to pass the BER form of a filter to the search function. And some helpers to build the BER form, and to parse it back into the string form of a filter. Filters are complex enough that it can be a pain to deal with the string form and its quoting at times. Of course, that one could be a client control too - e.g. combined with passing NULL as the filter... -- Hallvard

1 0

Re: (ITS#5916) Allow alias dereferencing in search C API
by michael＠stroeder.com 01 Feb '09

01 Feb '09

Kurt(a)OpenLDAP.org wrote: > On Jan 31, 2009, at 11:21 AM, hyc(a)symas.com wrote: >> ando(a)sys-net.it wrote: >>> h.b.furuseth(a)usit.uio.no wrote: >>>> ando(a)sys-net.it writes: >>>>> Proxy backends use ldap_set_option(3) to set alias >>>>> dereferencing when requested by a search operation (and do >>>>> not clean it up afterwards, as far as I can tell). Since >>>>> LDAP handlers are pooled and reused, this behavior is >>>>> inherently broken. >> [..] >> I recall complaining about this deficiency in the API back in 1998, >> when I >> first wrote back-ldap. Alias deref has always been a part of the >> protocol, but >> it has always been missing from the API. Perplexing... > > Yes, of course, one might use the client control mechanism to add an > ability to set deref on a call by call basis instead of introducing > yet another function. Hmm, maybe I missed something in this discussion and about the usage of client controls. But I'd vote for another function which provides a complete set of arguments for sending search requests. So if ldap_search_ext is considered incomplete regarding alias dereferencing I'd vote for introducing ldap_search_ext2 (or whatever name) with an additional argument. Ciao, Michael.

1 0

Re: (ITS#5916) Allow alias dereferencing in search C API
by Kurt＠OpenLDAP.org 31 Jan '09

31 Jan '09

On Jan 31, 2009, at 11:21 AM, hyc(a)symas.com wrote: > ando(a)sys-net.it wrote: >> h.b.furuseth(a)usit.uio.no wrote: >>> ando(a)sys-net.it writes: >>>> Proxy backends use ldap_set_option(3) to set alias dereferencing >>>> when >>>> requested by a search operation (and do not clean it up >>>> afterwards, as >>>> far as I can tell). Since LDAP handlers are pooled and reused, >>>> this >>>> behavior is inherently broken. >>> I note you committed this with an ldap_int_* interface. >> >> Yes, that's because I needed it internally. > > I think ldap_pvt would make more sense. _int_ means internal to the library. _pvt_ means private to OpenLDAP Software. > I recall complaining about this deficiency in the API back in 1998, > when I > first wrote back-ldap. Alias deref has always been a part of the > protocol, but > it has always been missing from the API. Perplexing... Yes, of course, one might use the client control mechanism to add an ability to set deref on a call by call basis instead of introducing yet another function. > > >>> But it could >>> be useful outside OpenLDAP code too. >> >> Yes, that will probably be the next step. But to make it publicly >> available I should have called it ldap_search_ext2, or >> ldap_search_really_ext or something like that. I think we need to >> make >> it public based on demand and after a ballot on the name. > > Ok. > >>> Seems to me this is what client-side controls are for. Extending >>> the >>> client-side functionality without needing a new API. >> >> Well, since alias dereferencing is part of the protocol, I think >> using a >> client-side control is sort of an overkill :) > > Agreed. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > >

1 0

Re: (ITS#5916) Allow alias dereferencing in search C API
by hyc＠symas.com 31 Jan '09

31 Jan '09

ando(a)sys-net.it wrote: > h.b.furuseth(a)usit.uio.no wrote: >> ando(a)sys-net.it writes: >>> Proxy backends use ldap_set_option(3) to set alias dereferencing when >>> requested by a search operation (and do not clean it up afterwards, as >>> far as I can tell). Since LDAP handlers are pooled and reused, this >>> behavior is inherently broken. >> I note you committed this with an ldap_int_* interface. > > Yes, that's because I needed it internally. I think ldap_pvt would make more sense. I recall complaining about this deficiency in the API back in 1998, when I first wrote back-ldap. Alias deref has always been a part of the protocol, but it has always been missing from the API. Perplexing... >> But it could >> be useful outside OpenLDAP code too. > > Yes, that will probably be the next step. But to make it publicly > available I should have called it ldap_search_ext2, or > ldap_search_really_ext or something like that. I think we need to make > it public based on demand and after a ballot on the name. Ok. >> Seems to me this is what client-side controls are for. Extending the >> client-side functionality without needing a new API. > > Well, since alias dereferencing is part of the protocol, I think using a > client-side control is sort of an overkill :) Agreed. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5916) Allow alias dereferencing in search C API
by ando＠sys-net.it 31 Jan '09

31 Jan '09

h.b.furuseth(a)usit.uio.no wrote: > ando(a)sys-net.it writes: >> Proxy backends use ldap_set_option(3) to set alias dereferencing when >> requested by a search operation (and do not clean it up afterwards, as >> far as I can tell). Since LDAP handlers are pooled and reused, this >> behavior is inherently broken. > > I note you committed this with an ldap_int_* interface. Yes, that's because I needed it internally. > But it could > be useful outside OpenLDAP code too. Yes, that will probably be the next step. But to make it publicly available I should have called it ldap_search_ext2, or ldap_search_really_ext or something like that. I think we need to make it public based on demand and after a ballot on the name. > Seems to me this is what client-side controls are for. Extending the > client-side functionality without needing a new API. Well, since alias dereferencing is part of the protocol, I think using a client-side control is sort of an overkill :) p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5916) Allow alias dereferencing in search C API
by h.b.furuseth＠usit.uio.no 31 Jan '09

31 Jan '09

ando(a)sys-net.it writes: > Proxy backends use ldap_set_option(3) to set alias dereferencing when > requested by a search operation (and do not clean it up afterwards, as > far as I can tell). Since LDAP handlers are pooled and reused, this > behavior is inherently broken. I note you committed this with an ldap_int_* interface. But it could be useful outside OpenLDAP code too. Seems to me this is what client-side controls are for. Extending the client-side functionality without needing a new API. -- Hallvard

1 0

(ITS#5916) Allow alias dereferencing in search C API
by ando＠sys-net.it 31 Jan '09

31 Jan '09

Full_Name: Pierangelo Masarati Version: HEAD OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.63.140.131) Submitted by: ando Proxy backends use ldap_set_option(3) to set alias dereferencing when requested by a search operation (and do not clean it up afterwards, as far as I can tell). Since LDAP handlers are pooled and reused, this behavior is inherently broken. The only reason for doing this is that the C API does not allow to explicitly define alias dereferencing when sending a search request. This is a feature request for an internal API extension (ldap_int_search*) that allows alias dereferencing. The current C API (essentially, ldap_search_ext*) should wrap the new one, until promoted to public. A patch is coming. p.

1 0

Re: (ITS#5889) [PATCH] Sending dereference field when retrying connection in meta backend
by ando＠sys-net.it 31 Jan '09

31 Jan '09

jorge.perez(a)adaptia.net wrote: > When we have two slapds with a established meta connection between them and the > connection is reset, for example by a router, the next search query will always > be send with never in dereferencing. > > Steps to reproduce: > > - Established a meta connection between 2 slapds > - Reset the connection, for example with cutter > - Send a search dereferencing with something different to never. > - See the results are no dereferenced. Actually, I was reviewing this fix, and it seems that the code for alias dereferencing is inherently broken, essentially because the (C) API for alias dereferencing is broken. In fact, back-ldap and back-meta reuse and pool connections, so setting this parameter using ldap_set_option() will actually affect all (search) operations occurring simultaneously in an unprotected manner. I think this needs to be fixed. The simplest (and my favorite) solution would be to have back-ldap and back-meta discontinue aliasing support. Another option, which I do not consider particularly viable, is to separately pool connections with different alias dereferencing strategies Finally (my second favorite option) is to add a new C API for ldap_search* operations that allows to explicitly set the alias dereferencing parameter. This API does not need to be public, since it is mostly useful inside the proxy backends. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: slapo-unique man page misleading (ITS#5908)
by ghenry＠suretecsystems.com 30 Jan '09

30 Jan '09

Ok, but we should mention this somewhere like in the admin guide perhaps. ------Original Message------ From: Quanah Gibson-Mount Sender: To: Gavin Henry To: openldap-its(a)openldap.org Subject: Re: slapo-unique man page misleading (ITS#5908) Sent: 30 Jan 2009 19:13 --On Friday, January 30, 2009 7:08 PM +0000 Gavin Henry <ghenry(a)suretecsystems.com> wrote: > Ok, so this should be in the man page. Where else should this be listed > that affects other overlays? Eh, I'm not sure it should be in the man page. The point is, if you ever want guaranteed atomic writes, you have to use slapo-accesslog to serialize them. That's an issue that affects other LDAP servers as well. AS Pierangelo said: There is no guarantee of DSA-wide or even database-wide atomicity in write operations, including internal ones. This was never even intended to be in place. This is a known design limitation not only of slapo-unique, but also of slapd (and, I'd say, of LDAP itself). --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie, Aberdeenshire, AB51 4FP. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html

1 0

Re: slapo-unique man page misleading (ITS#5908)
by quanah＠zimbra.com 30 Jan '09

30 Jan '09

--On Friday, January 30, 2009 7:08 PM +0000 Gavin Henry <ghenry(a)suretecsystems.com> wrote: > Ok, so this should be in the man page. Where else should this be listed > that affects other overlays? Eh, I'm not sure it should be in the man page. The point is, if you ever want guaranteed atomic writes, you have to use slapo-accesslog to serialize them. That's an issue that affects other LDAP servers as well. AS Pierangelo said: There is no guarantee of DSA-wide or even database-wide atomicity in write operations, including internal ones. This was never even intended to be in place. This is a known design limitation not only of slapo-unique, but also of slapd (and, I'd say, of LDAP itself). --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: slapo-unique man page misleading (ITS#5908)
by ghenry＠suretecsystems.com 30 Jan '09

30 Jan '09

Ok, so this should be in the man page. Where else should this be listed that affects other overlays? ------Original Message------ From: Quanah Gibson-Mount Sender: To: Gavin Henry To: openldap-its(a)openldap.org Subject: Re: slapo-unique man page misleading (ITS#5908) Sent: 30 Jan 2009 19:04 --On Thursday, January 29, 2009 3:48 PM +0000 ghenry(a)suretecsystems.com wrote: >> Huh? slapo-unique only does internal searches. The writes are all user >> ops, >> and accesslog will serialize all of them. > > So, where does this leave us? If you want guaranteed atomic writes, use slapo-accesslog on your master to serialize them. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie, Aberdeenshire, AB51 4FP. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html

1 0

Re: slapo-unique man page misleading (ITS#5908)
by quanah＠zimbra.com 30 Jan '09

30 Jan '09

--On Thursday, January 29, 2009 3:48 PM +0000 ghenry(a)suretecsystems.com wrote: >> Huh? slapo-unique only does internal searches. The writes are all user >> ops, >> and accesslog will serialize all of them. > > So, where does this leave us? If you want guaranteed atomic writes, use slapo-accesslog on your master to serialize them. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#5898) test044 abort with back-ldif
by ando＠sys-net.it 30 Jan '09

30 Jan '09

Hallvard B Furuseth wrote: > Pierangelo Masarati writes: >> I don't know how useful it would be to have back-ldif directly implement >> compare, as soon as one only uses it for back-config and not for a real >> storage with stacked overlays (however I note that back-config needs at >> least to nicely interoperate with one overlay: slapo-syncprov). > > Syncprov does use compare, and back-config doesn't provide it. > I don't know if that's a problem. In any case, the fallthru compare works fine, except for those cases that require overlay manipulation of the call before actually comparing data. This is what happens to slapo-dynlist: compare gets straight to back-ldif without prior generating the dynamic listing. > I forgot, back-config doesn't fall back to back-ldif for reads. > So I guess it's just as well to leave back-ldif as it is. :) p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5898) test044 abort with back-ldif
by h.b.furuseth＠usit.uio.no 30 Jan '09

30 Jan '09

Pierangelo Masarati writes: > I don't know how useful it would be to have back-ldif directly implement > compare, as soon as one only uses it for back-config and not for a real > storage with stacked overlays (however I note that back-config needs at > least to nicely interoperate with one overlay: slapo-syncprov). Syncprov does use compare, and back-config doesn't provide it. I don't know if that's a problem. I forgot, back-config doesn't fall back to back-ldif for reads. So I guess it's just as well to leave back-ldif as it is. -- Hallvard

1 0

Re: (ITS#5898) test044 abort with back-ldif
by ando＠sys-net.it 30 Jan '09

30 Jan '09

h.b.furuseth(a)usit.uio.no wrote: > Pierangelo Masarati writes: >> Probably related; in any case, misused in slapo-dynlist(5). Now slapd >> doesn't assert() any more. However, the test fails because back-ldif >> does not implement compare, and falls back to be_fetch(), which does not >> trigger dynlist expansion. > > Yes, the "compare without help from the backend" functionality has been > broken for years - which I presume means it's hard to fix, or needs a > different design. > > Should we patch it over by adding Compare to back-ldif and whatever else > lacks it? Or is this a "fix coming Real Soon Now" issue so it's good > to have this reminder from a backend which doesn't provide Compare? Not sure whether it's about to come sooner or later, but the issue is always about be_fetch, aka bi_entry_get_rw() whose design Is Broken (TM) (actually, it broke when overlays came into play). I don't know how useful it would be to have back-ldif directly implement compare, as soon as one only uses it for back-config and not for a real storage with stacked overlays (however I note that back-config needs at least to nicely interoperate with one overlay: slapo-syncprov). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5898) test044 abort with back-ldif
by h.b.furuseth＠usit.uio.no 30 Jan '09

30 Jan '09

Pierangelo Masarati writes: > Probably related; in any case, misused in slapo-dynlist(5). Now slapd > doesn't assert() any more. However, the test fails because back-ldif > does not implement compare, and falls back to be_fetch(), which does not > trigger dynlist expansion. Yes, the "compare without help from the backend" functionality has been broken for years - which I presume means it's hard to fix, or needs a different design. Should we patch it over by adding Compare to back-ldif and whatever else lacks it? Or is this a "fix coming Real Soon Now" issue so it's good to have this reminder from a backend which doesn't provide Compare? -- Hallvard

1 0

Re: (ITS#5915) Some man pages still only talk about slapd.conf
by h.b.furuseth＠usit.uio.no 30 Jan '09

30 Jan '09

hyc(a)symas.com writes: > I guess for this release (and probably for 2.5) we're just going to > have to document both, as cumbersome as that will be. When we get to > to 2.6/3.0 and drop support for slapd.conf we'll have to come back > thru again and delete the slapd.conf references from all the > docs. Unless anyone has a better idea... I'm not sure I see the problem, except that Someone(TM) must do the work... For each configuration keyword, use two headers in the manpages: Both the slapd.conf syntax and the cn=config syntax. Explain at the beginning of slapd.conf.5 or slapd-config.5. We'll be deleting most of the other one, since they are mostly duplicates. Other manpages need just one-line reminder for explanation. There'll be exceptions that spell things out too differently, or use slightly different parameters in slapd.conf and cn=config. Keep them in just slapd.conf-syntax, and add a separate cn=config attribute description below. Likely that'll only need to explain the difference. Don't go trough literary contortions to fit them both into the same description. Some of these will likely be things that could use some cleanup in the code anyway, bringing the two config formats in sync. Take a random sample of the slapd.conf-syntax example sections and replace with their cn=config equivalent. Ensure all example sections say either "slapd.conf example" or "cn=config example". slapd.conf.5 and slapd-config.5 themselves will need sections about how they work as interfaces to configuration, separate from the configuration keywords/attributes themselves. Since we are moving towards cn=config, I suppose it's that one which should keep the actual keyword/attribute descriptions. Or they could move to a third file. -- Hallvard

1 0

Re: (ITS#5898) test044 abort with back-ldif
by ando＠sys-net.it 30 Jan '09

30 Jan '09

h.b.furuseth(a)usit.uio.no wrote: >> When using the back-ldif backend with test044, it aborts: > > Probably ITS#5340. Probably related; in any case, misused in slapo-dynlist(5). Now slapd doesn't assert() any more. However, the test fails because back-ldif does not implement compare, and falls back to be_fetch(), which does not trigger dynlist expansion. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5913) supportedSASLMechanisms EXTERNAL not Work
by ando＠sys-net.it 30 Jan '09

30 Jan '09

jarbas.peixoto(a)gmail.com wrote: > Full_Name: Jarbas Peixoto Junior > Version: 2.4.11 > OS: Debian Lenny > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (200.152.43.195) > > > I have two servers: > > * Server A: Debian Etch - Works Fine > > * Server B: Debian Lenny - Do not Works supportedSASLMechanisms EXTERNAL > > In Server A I have: > > # ldapsearch -v -H ldap://server-Etch -b "" -LLL -s base supportedSASLMechanisms > -ZZ > ldap_initialize( ldap://server-Etch ) > SASL/EXTERNAL authentication started > SASL username: emailAddress=jarbas.peixoto(a)previdencia.gov.br,CN=jarbas.peixoto,OU=DATAPREV,O=Previdencia > Social,L=Campo Grande,ST=Mato Grosso do Sul,C=BR > SASL SSF: 0 > filter: (objectclass=*) > requesting: supportedSASLMechanisms > dn: > supportedSASLMechanisms: PLAIN > supportedSASLMechanisms: DIGEST-MD5 > supportedSASLMechanisms: LOGIN > supportedSASLMechanisms: NTLM > supportedSASLMechanisms: CRAM-MD5 > supportedSASLMechanisms: EXTERNAL > > In Server B I have: > > # ldapsearch -v -H ldap://server-Lenny -b "" -LLL -s base > supportedSASLMechanisms -ZZ > ldap_initialize( ldap://server-Lenny:389/??base ) > ldap_start_tls: Connect error (-11 ^^^ start TLS is failing. There is no evidence of a bug in the software; likely either the client or the server's TLS is misconfigured. Please direct software usage questions to the openldap-software list. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5915) Some man pages still only talk about slapd.conf
by hyc＠symas.com 30 Jan '09

30 Jan '09

ghenry(a)OpenLDAP.org wrote: > Full_Name: Gavin Henry > Version: > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (92.41.236.102) > Submitted by: ghenry > > > Some discussion will be needed on the best way to talk about config. We don't > want to list everything twice for both formats...lots of slapd.conf written > throughout most (5/8) man pages. I guess for this release (and probably for 2.5) we're just going to have to document both, as cumbersome as that will be. When we get to to 2.6/3.0 and drop support for slapd.conf we'll have to come back thru again and delete the slapd.conf references from all the docs. Unless anyone has a better idea... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#5915) Some man pages still only talk about slapd.conf
by ghenry＠OpenLDAP.org 30 Jan '09

30 Jan '09

Full_Name: Gavin Henry Version: OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (92.41.236.102) Submitted by: ghenry Some discussion will be needed on the best way to talk about config. We don't want to list everything twice for both formats...lots of slapd.conf written throughout most (5/8) man pages. Thanks.

1 0

(ITS#5914) Update all man pages that only "SEE ALSO slapd.conf(5)" to list slapd-config(5)
by ghenry＠OpenLDAP.org 30 Jan '09

30 Jan '09

Full_Name: Gavin Henry Version: HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (92.41.164.91) Submitted by: ghenry Will commit shortly.

1 0

(ITS#5913) supportedSASLMechanisms EXTERNAL not Work
by jarbas.peixoto＠gmail.com 29 Jan '09

29 Jan '09

Full_Name: Jarbas Peixoto Junior Version: 2.4.11 OS: Debian Lenny URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (200.152.43.195) I have two servers: * Server A: Debian Etch - Works Fine * Server B: Debian Lenny - Do not Works supportedSASLMechanisms EXTERNAL In Server A I have: # ldapsearch -v -H ldap://server-Etch -b "" -LLL -s base supportedSASLMechanisms -ZZ ldap_initialize( ldap://server-Etch ) SASL/EXTERNAL authentication started SASL username: emailAddress=jarbas.peixoto(a)previdencia.gov.br,CN=jarbas.peixoto,OU=DATAPREV,O=Previdencia Social,L=Campo Grande,ST=Mato Grosso do Sul,C=BR SASL SSF: 0 filter: (objectclass=*) requesting: supportedSASLMechanisms dn: supportedSASLMechanisms: PLAIN supportedSASLMechanisms: DIGEST-MD5 supportedSASLMechanisms: LOGIN supportedSASLMechanisms: NTLM supportedSASLMechanisms: CRAM-MD5 supportedSASLMechanisms: EXTERNAL In Server B I have: # ldapsearch -v -H ldap://server-Lenny -b "" -LLL -s base supportedSASLMechanisms -ZZ ldap_initialize( ldap://server-Lenny:389/??base ) ldap_start_tls: Connect error (-11 # ldapsearch -v -H ldap://server-Lenny -b "" -LLL -s base supportedSASLMechanisms -ZZ -d 1 ldap_url_parse_ext(ldap://server-Lenny) ldap_create ldap_url_parse_ext(ldap://server-Lenny:389/??base) ldap_extended_operation_s ldap_extended_operation ldap_send_initial_request ldap_new_connection 1 1 0 ldap_int_open_connection ldap_connect_to_host: TCP server-Lenny:389 ldap_new_socket: 3 ldap_prepare_socket: 3 ldap_connect_to_host: Trying 10.82.0.234:389 ldap_pvt_connect: fd: 3 tm: -1 async: 0 ldap_open_defconn: successful ldap_send_server_request ber_scanf fmt ({it) ber: ber_scanf fmt ({) ber: ber_flush2: 31 bytes to sd 3 ldap_result ld 0x9d73280 msgid 1 wait4msg ld 0x9d73280 msgid 1 (infinite timeout) wait4msg continue ld 0x9d73280 msgid 1 all 1 ** ld 0x9d73280 Connections: * host: www-mmldap port: 389 (default) refcnt: 2 status: Connected last used: Thu Jan 29 18:17:22 2009 ** ld 0x9d73280 Outstanding Requests: * msgid 1, origid 1, status InProgress outstanding referrals 0, parent count 0 ld 0x9d73280 request count 1 (abandoned 0) ** ld 0x9d73280 Response Queue: Empty ld 0x9d73280 response count 0 ldap_chkResponseList ld 0x9d73280 msgid 1 all 1 ldap_chkResponseList returns ld 0x9d73280 NULL ldap_int_select read1msg: ld 0x9d73280 msgid 1 all 1 ber_get_next ber_get_next: tag 0x30 len 12 contents: read1msg: ld 0x9d73280 msgid 1 message type extended-result ber_scanf fmt ({eAA) ber: read1msg: ld 0x9d73280 0 new referrals read1msg: mark request completed, ld 0x9d73280 msgid 1 request done: ld 0x9d73280 msgid 1 res_errno: 0, res_error: <>, res_matched: <> ldap_free_request (origid 1, msgid 1) ldap_free_connection 0 1 ldap_free_connection: refcnt 1 ldap_parse_extended_result ber_scanf fmt ({eAA) ber: ldap_parse_result ber_scanf fmt ({iAA) ber: ber_scanf fmt (}) ber: ldap_msgfree ldap_err2string ldap_start_tls: Connect error (-11) In the /var/log/syslog I have: Jan 29 18:17:22 server-Lenny slapd[12945]: conn=99 fd=21 ACCEPT from IP=10.82.0.234:50441 (IP=0.0.0.0:389) Jan 29 18:17:22 server-Lenny slapd[12945]: conn=99 op=0 EXT oid=1.3.6.1.4.1.1466.20037 Jan 29 18:17:22 server-Lenny slapd[12945]: conn=99 op=0 STARTTLS Jan 29 18:17:22 server-Lenny slapd[12945]: conn=99 op=0 RESULT oid= err=0 text= Jan 29 18:17:22 server-Lenny slapd[12945]: conn=99 fd=21 closed (TLS negotiation failure) This is very important for use openldap with user certificates. Anybody have a help for me? Tanks!

1 0

Re: (ITS#5912) Meta modify increment feature
by ando＠sys-net.it 29 Jan '09

29 Jan '09

Fixed in HEAD; please test. Thanks, p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5912) Meta modify increment feature
by ando＠sys-net.it 29 Jan '09

29 Jan '09

jorge.nevado(a)ericsson.com wrote: > Full_Name: Jorge Nevado > Version: 2.4.12 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.237.142.7) > > > We are trying to send an increment modify operation on an entry. This request > has to be proxied using the meta to a backend we have developed. When we send > the operation directly to the backend it works without any problem. But when we > have to pass through the meta backend we receive the following error: > > ldapmodify -x -H ldap://localhost:9090 -D "cn=manager,dc=operator,dc=com" -w > manager -f ldif/nam.ldif modifying entry > "appName=CsPs,subsId=10000000000000000000000000000000,ou=subscribers,dc=operator,dc=com" > ldap_modify: Server is unwilling to perform (53) > additional info: modify/increment not supported in context > > However we see that the operation has been performed and the attribute > incremented. The same problem occurs in back-ldap. The frontend is sending the error message, but it allows the operation to continue, and eventually it succeeds. I'm fixing it. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: slapo-unique man page misleading (ITS#5908)
by ghenry＠suretecsystems.com 29 Jan '09

29 Jan '09

----- hyc(a)symas.com wrote: > ando(a)sys-net.it wrote: > > quanah(a)zimbra.com wrote: > >> --On January 28, 2009 10:39:41 PM +0000 Magne.Land(a)citrix.com > wrote: > >> > >>> Hi, > >>> Please see this thread for background info: > >>> > http://www.openldap.org/lists/openldap-software/200901/msg00112.html > >>> > >>> To me, something is either guaranteed or not. And I think > "enforce" means > >>> something that is guaranteed. In this case, the uniqueness is not > >>> guaranteed, therefore it is not enforced. > >> Set up slapo-accesslog on your master. Then uniqueness is > guaranteed. > > > > Are you sure about this? I didn't look in detail to the code, so I > > can't tell whether serializing writes that way will cause the > desired > > behavior, since slapo-unique is using internal writes. > > Huh? slapo-unique only does internal searches. The writes are all user > ops, > and accesslog will serialize all of them. So, where does this leave us? -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie, Aberdeenshire, AB51 4FP. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html

1 0

Re: (ITS#5909) stack overflow in tavl_delete (libraries/liblutil/tavl.c)?
by h.b.furuseth＠usit.uio.no 29 Jan '09

29 Jan '09

ulrich.windl(a)rz.uni-regensburg.de writes: > To make a long discussion short, you cold add a comment regarding the > use of "8". The 8 should be CHAR_BIT. /* Maximum tree depth this host's address space could support */ #define MAX_TREE_DEPTH (sizeof(void *) * CHAR_BIT) -- Hallvard

1 0

(ITS#5912) Meta modify increment feature
by jorge.nevado＠ericsson.com 29 Jan '09

29 Jan '09

Full_Name: Jorge Nevado Version: 2.4.12 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.237.142.7) We are trying to send an increment modify operation on an entry. This request has to be proxied using the meta to a backend we have developed. When we send the operation directly to the backend it works without any problem. But when we have to pass through the meta backend we receive the following error: ldapmodify -x -H ldap://localhost:9090 -D "cn=manager,dc=operator,dc=com" -w manager -f ldif/nam.ldif modifying entry "appName=CsPs,subsId=10000000000000000000000000000000,ou=subscribers,dc=operator,dc=com" ldap_modify: Server is unwilling to perform (53) additional info: modify/increment not supported in context However we see that the operation has been performed and the attribute incremented. The request we are sending is the following: dn: msisdn=910000000000000,dc=msisdn,ou=identities,dc=operator,dc=com changetype:modify increment: usageCnt usageCnt:4

1 0

Re: slapo-unique man page misleading (ITS#5908)
by hyc＠symas.com 29 Jan '09

29 Jan '09

ando(a)sys-net.it wrote: > quanah(a)zimbra.com wrote: >> --On January 28, 2009 10:39:41 PM +0000 Magne.Land(a)citrix.com wrote: >> >>> Hi, >>> Please see this thread for background info: >>> http://www.openldap.org/lists/openldap-software/200901/msg00112.html >>> >>> To me, something is either guaranteed or not. And I think "enforce" means >>> something that is guaranteed. In this case, the uniqueness is not >>> guaranteed, therefore it is not enforced. >> Set up slapo-accesslog on your master. Then uniqueness is guaranteed. > > Are you sure about this? I didn't look in detail to the code, so I > can't tell whether serializing writes that way will cause the desired > behavior, since slapo-unique is using internal writes. Huh? slapo-unique only does internal searches. The writes are all user ops, and accesslog will serialize all of them. > As Gavin says,. > if this is the case then it should be at least documented, and not only > in slapo-unique, but also for any overlay/module that does internal writes. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: slapo-unique man page misleading (ITS#5908)
by ando＠sys-net.it 29 Jan '09

29 Jan '09

quanah(a)zimbra.com wrote: > --On January 28, 2009 10:39:41 PM +0000 Magne.Land(a)citrix.com wrote: > >> Hi, >> Please see this thread for background info: >> http://www.openldap.org/lists/openldap-software/200901/msg00112.html >> >> To me, something is either guaranteed or not. And I think "enforce" means >> something that is guaranteed. In this case, the uniqueness is not >> guaranteed, therefore it is not enforced. > > Set up slapo-accesslog on your master. Then uniqueness is guaranteed. Are you sure about this? I didn't look in detail to the code, so I can't tell whether serializing writes that way will cause the desired behavior, since slapo-unique is using internal writes. As Gavin says,. if this is the case then it should be at least documented, and not only in slapo-unique, but also for any overlay/module that does internal writes. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: slapo-unique man page misleading (ITS#5908)
by ghenry＠suretecsystems.com 29 Jan '09

29 Jan '09

----- quanah(a)zimbra.com wrote: > --On January 28, 2009 10:39:41 PM +0000 Magne.Land(a)citrix.com wrote: > > > Hi, > > Please see this thread for background info: > > > http://www.openldap.org/lists/openldap-software/200901/msg00112.html > > > > To me, something is either guaranteed or not. And I think "enforce" > means > > something that is guaranteed. In this case, the uniqueness is not > > guaranteed, therefore it is not enforced. > > Set up slapo-accesslog on your master. Then uniqueness is > guaranteed. We don't document this requirement in slapo-unique.5 though. -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/ Suretec Systems is a limited company registered in Scotland. Registered number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie, Aberdeenshire, AB51 4FP. Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html

1 0

Re: (ITS#5909) stack overflow in tavl_delete (libraries/liblutil/tavl.c)?
by ulrich.windl＠rz.uni-regensburg.de 29 Jan '09

29 Jan '09

On 29 Jan 2009 at 2:21, Howard Chu wrote: > Hardcoding arbitrary limits in portable code is stupid. ("No one will ever > need more than 640K...") What may seem (im)practical to you today may be > commonplace tomorrow. This code will be maintenance-free no matter what class > of machine it's built on, forever. That's *excellent* programming. Howard, my proposals were related to the virtual address space, and the specific value for 64 bit was assuming that no more than 800GB of tree nodes are allocated. Your settings relate the depth of the tree to the width of a pointer multiplied by a hardcoded value without further explanation. > > There is no bug here, this ITS is closed. To make a long discussion short, you cold add a comment regarding the use of "8". Regards, Ulrich > > Ulrich Windl wrote: > > On 28 Jan 2009 at 13:06, Howard Chu wrote: > > > >> Ulrich.Windl(a)rz.uni-regensburg.de wrote: > >>> Full_Name: Ulrich Windl > >>> Version: 2.4.11 > >>> OS: Linux > >>> URL: ftp://ftp.openldap.org/incoming/ > >>> Submission from: (NULL) (132.199.156.178) > >>> > >>> > >>> Looking for a threaded AVL tree implementation that is supposed to work, I found > >>> libraries/liblutil/tavl.c in openLDAP. I'm no expert on threaded AVL trees, but > >>> there's thing that worries me: > >>> tavl_delete uses two stacks (pptr, pdir) with a size of 8 elements on the > >>> routine' stack. While locating the element to delete, it pushes the parent node > >>> onto the stack. > >> You're misreading the code. The stack is 8*sizeof(void *) which means 32 > >> elements deep on a 32 bit machine, and 64 deep on a 64 bit machine. There is > >> no possibility of overflow now or for several years to come. > > > > OK! I was confused by the terrible programming style at that point: as the array > > elements are of the proper type (and not bytes), there is no need for a sizeof > > here: I see no reason why the depth of the tree to be allowed should depend on the > > length of a pointer, maning: If you want 32, why don't you write 32? So the worst > > case stack depth would be something like "log2(2^32/(sizeof(void *) * 5))" ("5" > > being 2 links + 2 threading marks + 1 data pointer), something near 32-5 == 27 > > (for 32-bit machines) and 64-6 == 58. Practically the latter could be safely > > reduced to a value around 42 IMHO. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5909) stack overflow in tavl_delete (libraries/liblutil/tavl.c)?
by hyc＠symas.com 29 Jan '09

29 Jan '09

Hardcoding arbitrary limits in portable code is stupid. ("No one will ever need more than 640K...") What may seem (im)practical to you today may be commonplace tomorrow. This code will be maintenance-free no matter what class of machine it's built on, forever. That's *excellent* programming. There is no bug here, this ITS is closed. Ulrich Windl wrote: > On 28 Jan 2009 at 13:06, Howard Chu wrote: > >> Ulrich.Windl(a)rz.uni-regensburg.de wrote: >>> Full_Name: Ulrich Windl >>> Version: 2.4.11 >>> OS: Linux >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (132.199.156.178) >>> >>> >>> Looking for a threaded AVL tree implementation that is supposed to work, I found >>> libraries/liblutil/tavl.c in openLDAP. I'm no expert on threaded AVL trees, but >>> there's thing that worries me: >>> tavl_delete uses two stacks (pptr, pdir) with a size of 8 elements on the >>> routine' stack. While locating the element to delete, it pushes the parent node >>> onto the stack. >> You're misreading the code. The stack is 8*sizeof(void *) which means 32 >> elements deep on a 32 bit machine, and 64 deep on a 64 bit machine. There is >> no possibility of overflow now or for several years to come. > > OK! I was confused by the terrible programming style at that point: as the array > elements are of the proper type (and not bytes), there is no need for a sizeof > here: I see no reason why the depth of the tree to be allowed should depend on the > length of a pointer, maning: If you want 32, why don't you write 32? So the worst > case stack depth would be something like "log2(2^32/(sizeof(void *) * 5))" ("5" > being 2 links + 2 threading marks + 1 data pointer), something near 32-5 == 27 > (for 32-bit machines) and 64-6 == 58. Practically the latter could be safely > reduced to a value around 42 IMHO. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5909) stack overflow in tavl_delete (libraries/liblutil/tavl.c)?
by ulrich.windl＠rz.uni-regensburg.de 29 Jan '09

29 Jan '09

On 28 Jan 2009 at 13:06, Howard Chu wrote: > Ulrich.Windl(a)rz.uni-regensburg.de wrote: > > Full_Name: Ulrich Windl > > Version: 2.4.11 > > OS: Linux > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (132.199.156.178) > > > > > > Looking for a threaded AVL tree implementation that is supposed to work, I found > > libraries/liblutil/tavl.c in openLDAP. I'm no expert on threaded AVL trees, but > > there's thing that worries me: > > tavl_delete uses two stacks (pptr, pdir) with a size of 8 elements on the > > routine' stack. While locating the element to delete, it pushes the parent node > > onto the stack. > > You're misreading the code. The stack is 8*sizeof(void *) which means 32 > elements deep on a 32 bit machine, and 64 deep on a 64 bit machine. There is > no possibility of overflow now or for several years to come. OK! I was confused by the terrible programming style at that point: as the array elements are of the proper type (and not bytes), there is no need for a sizeof here: I see no reason why the depth of the tree to be allowed should depend on the length of a pointer, maning: If you want 32, why don't you write 32? So the worst case stack depth would be something like "log2(2^32/(sizeof(void *) * 5))" ("5" being 2 links + 2 threading marks + 1 data pointer), something near 32-5 == 27 (for 32-bit machines) and 64-6 == 58. Practically the latter could be safely reduced to a value around 42 IMHO. > > > So from my knowledge this means that for a perfectly filled balanced tree, the > > stack will overflow when the tree consists of 2^8 (2^0 + 2^1 + 2^2 + ... + 2^7) > > or more nodes, and you are locating a leaf node. Maybe even with fewer nodes. > > As the algorithm also pushes an other element onto the stack, the problem could > > appear even before 128 nodes. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: slapo-unique man page misleading (ITS#5908)
by quanah＠zimbra.com 29 Jan '09

29 Jan '09

--On January 28, 2009 10:39:41 PM +0000 Magne.Land(a)citrix.com wrote: > Hi, > Please see this thread for background info: > http://www.openldap.org/lists/openldap-software/200901/msg00112.html > > To me, something is either guaranteed or not. And I think "enforce" means > something that is guaranteed. In this case, the uniqueness is not > guaranteed, therefore it is not enforced. Set up slapo-accesslog on your master. Then uniqueness is guaranteed. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#5911) password policy - alternate lockout mechanism
by aravind＠freeshell.org 28 Jan '09

28 Jan '09

Full_Name: Aravind Gottipati Version: 2.4.13 OS: Linux - RHEL5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (63.245.220.241) I'd like to propose a change to how the password lockouts work. The current system does not differentiate between multiple bind attempts with a single (or even a few) incorrect password(s) vs. multiple bind attempts with different incorrect passwords. In our case, this results in a ton of false positives when folks change their password, but don't propagate their password change to all the applications/machines that use it. This causes a bunch of un-necessary lockouts. A real crack attempt on the other hand would most likely try a bunch of passwords (none of which repeat). I have posted the same on the openldap-software mailing lists and Jeff Clowser proposed a scheme that should work to solve the problem. Record each failed bind attempt as a (hash,timestamp) pair. If there is another failed attempt, check the password against these (hash, timestamp) pairs and update the timestamp if the hash is found. If its a new password that hasn't been attempted before, then create a new (hash,timestamp) pair. Lock the account out if there are more than pwdMaxFailure hashes stored. http://www.openldap.org/lists/openldap-software/200901/msg00147.html

1 0

Re: slapo-unique man page misleading (ITS#5908)
by Magne.Land＠citrix.com 28 Jan '09

28 Jan '09

Hi, Please see this thread for background info: http://www.openldap.org/lists/openldap-software/200901/msg00112.html To me, something is either guaranteed or not. And I think "enforce" means something that is guaranteed. In this case, the uniqueness is not guaranteed, therefore it is not enforced. Regards, Magne Land On 1/28/09 2:33 PM, "Gavin Henry" <openldap-its(a)OpenLDAP.org> wrote: > Full_Name: Magne Land > Version: 2.3.43 > OS: RHEL 5.1 64-bit > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (72.37.244.20) > > > Currently the man page of slapo-unique says "enforce uniqueness", which I find > misleading. > Would it be possible to change it to say "makes the best effort to enforce > uniqueness" or something to that effect? > > It would need more words than that to comment on what exactly happens i.e. why it is a "best effort".

1 0

Re: (ITS#5909) stack overflow in tavl_delete (libraries/liblutil/tavl.c)?
by hyc＠symas.com 28 Jan '09

28 Jan '09

Ulrich.Windl(a)rz.uni-regensburg.de wrote: > Full_Name: Ulrich Windl > Version: 2.4.11 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (132.199.156.178) > > > Looking for a threaded AVL tree implementation that is supposed to work, I found > libraries/liblutil/tavl.c in openLDAP. I'm no expert on threaded AVL trees, but > there's thing that worries me: > tavl_delete uses two stacks (pptr, pdir) with a size of 8 elements on the > routine' stack. While locating the element to delete, it pushes the parent node > onto the stack. You're misreading the code. The stack is 8*sizeof(void *) which means 32 elements deep on a 32 bit machine, and 64 deep on a 64 bit machine. There is no possibility of overflow now or for several years to come. > So from my knowledge this means that for a perfectly filled balanced tree, the > stack will overflow when the tree consists of 2^8 (2^0 + 2^1 + 2^2 + ... + 2^7) > or more nodes, and you are locating a leaf node. Maybe even with fewer nodes. > As the algorithm also pushes an other element onto the stack, the problem could > appear even before 128 nodes. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#5910) Minor fixes to test scripts
by rein＠OpenLDAP.org 28 Jan '09

28 Jan '09

Full_Name: Rein Tollevik Version: CVS HEAD OS: Irrelevant URL: Submission from: (NULL) (81.93.160.250) Submitted by: rein test006 truncates its log file instead of appending to it. test048 substitutes pathnames wrong if it happens to be run from a directory which includes ".2." in the path. Fixes are coming.. Rein Tollevik Basefarm AS

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by hyc＠symas.com 28 Jan '09

28 Jan '09

h.b.furuseth(a)usit.uio.no wrote: > ando(a)sys-net.it writes: >> OK, makes sense. But perhaps creating entries on the stack could be >> deprecated now that entries are pooled and reused by calling >> entry_alloc() entry_free(). This is just food for thoughts, we could as >> well leave all those flags 'round, if we can streamline their handling. > > I think we might as well leave them. Since most are for saving memory, > might even want to expand them eventually. (E.g. separate flags for > _which_ parts of an entry are modifiable. Modifiable attribute list, > DNs, values _in_ attribute list, etc.) > > For that matter, looking at entry_free/entry_alloc, I don't see how that > pooling saves more than it costs. It saves 1 malloc for the Entry > structure, but not the mallocs for the DN and attribute list. And it > costs two global mutex lock/unlocks per entry, in entry_alloc() and > entry_free(). An entry on the stack avoids both malloc and locks. Yes, using the stack is still preferable when it's doable. The point of entry_alloc() was to avoid heap fragmentation, not to generally save on mallocs. It might make sense to turn the entry lists into per-thread lists, to avoid locks in the default cases. But if you follow this path to completion, that turns into re-implementing tcmalloc ("tc" stands for "thread-caching" which is exactly what we would be duplicating...). As for the attribute list: attr_alloc() is also pre-allocated for the same reason. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#5909) stack overflow in tavl_delete (libraries/liblutil/tavl.c)?
by Ulrich.Windl＠rz.uni-regensburg.de 28 Jan '09

28 Jan '09

Full_Name: Ulrich Windl Version: 2.4.11 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (132.199.156.178) Looking for a threaded AVL tree implementation that is supposed to work, I found libraries/liblutil/tavl.c in openLDAP. I'm no expert on threaded AVL trees, but there's thing that worries me: tavl_delete uses two stacks (pptr, pdir) with a size of 8 elements on the routine' stack. While locating the element to delete, it pushes the parent node onto the stack. So from my knowledge this means that for a perfectly filled balanced tree, the stack will overflow when the tree consists of 2^8 (2^0 + 2^1 + 2^2 + ... + 2^7) or more nodes, and you are locating a leaf node. Maybe even with fewer nodes. As the algorithm also pushes an other element onto the stack, the problem could appear even before 128 nodes. If the problem occurs, I'd suspect data corruption at least, because one stack will overwrite the other stack. I've spotted these uses (besides /libraries/liblutil/testtavl.c) of tavl_delete(): servers/slapd/overlays/translucent.c, servers/slapd/overlays/pcache.c I don't understand that code that much, but if the TAVL isn't used in a very controlled way (i.e. limiting the number of possible nodes), a problem may be triggered. I also noticed that the limitation in the code exists for some time already.

1 0

(ITS#5908) slapo-unique man page misleading
by magne.land＠citrix.com 28 Jan '09

28 Jan '09

Full_Name: Magne Land Version: 2.3.43 OS: RHEL 5.1 64-bit URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (72.37.244.20) Currently the man page of slapo-unique says "enforce uniqueness", which I find misleading. Would it be possible to change it to say "makes the best effort to enforce uniqueness" or something to that effect?

1 0

Re: (ITS#5903) memberof stacked instance member-attr delete failure
by ando＠sys-net.it 27 Jan '09

27 Jan '09

hyc(a)symas.com wrote: > mbackes(a)symas.com wrote: >> Full_Name: Matthew Backes >> Version: 2.4, head >> OS: any >> URL: >> Submission from: (NULL) (76.88.99.93) >> >> >> If slapo-memberof is instanced more than once, deleting the >> member-attr values from a group or deleting the group object will not >> remove the memberof-attr values from the members. >> >> Adds are not affected. >> >> Internally, the operation fails because it tries using the >> memberof-attribute name from the last memberof instance in the stack. > > The memberOf code is using a single static variable to hold its state. This is > obviously broken. It's also using thread keys for its temporary storage, which > is totally unnecessary; it should simply use its own callback and the > cb_private field of that callback. As such, much of this code needs to be > gutted and rewritten. See also ITS#5654. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

1 0

Re: (ITS#5903) memberof stacked instance member-attr delete failure
by hyc＠symas.com 27 Jan '09

27 Jan '09

mbackes(a)symas.com wrote: > Full_Name: Matthew Backes > Version: 2.4, head > OS: any > URL: > Submission from: (NULL) (76.88.99.93) > > > If slapo-memberof is instanced more than once, deleting the > member-attr values from a group or deleting the group object will not > remove the memberof-attr values from the members. > > Adds are not affected. > > Internally, the operation fails because it tries using the > memberof-attribute name from the last memberof instance in the stack. The memberOf code is using a single static variable to hold its state. This is obviously broken. It's also using thread keys for its temporary storage, which is totally unnecessary; it should simply use its own callback and the cb_private field of that callback. As such, much of this code needs to be gutted and rewritten. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#5907) accesslog should use overlay BackendInfo to read old entry
by rein＠OpenLDAP.org 27 Jan '09

27 Jan '09

Full_Name: Rein Tollevik Version: CVS HEAD OS: Irrelevant URL: Submission from: (NULL) (81.93.160.250) Submitted by: rein I use accesslog stacked above the glue overlay to have it log access to all of the glued databases. When accesslog reads the old entry it goes directly to the original BackendInfo, which causes it to not find entries in the subordinate DBs. A fix is coming. Rein Tollevik Basefarm AS

1 0

Re: (ITS#5661) contextCSN gets corrupted on the stand by mirror
by carl.johnstone＠gmgrd.co.uk 27 Jan '09

27 Jan '09

I'm seeing the same think on a 3-way multi-master setup here. Two servers (#001 & #002) are currently sat next to each other. The third (#003) is at a remote location. We're currently doing all amends through #001, although in the long term we'll be doing amends through all the servers. When I checked them yesterday I spotted that the remote server had a corrupt contextCSN for server #001. I dropped the DB and synced both the config and data from the main server again overnight. On checking again today the contextCSN is once again corrupt. In my case it's the first 8 bytes rather than the first 4. I'm running 2.4.13 with bdb 4.7.25 (first 3 patches applied). build: ./configure --enable-dynamic --enable-crypt --enable-modules=yes --enable-backends=mod --enable-overlays=mod --enable-sql=no --enable-ndb=no Carl

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by rlvcosta＠yahoo.com 27 Jan '09

27 Jan '09

Howard, One more comment. If you take a longer time between the ldapsearches the issue doesn't appear sometimes. So you need to start the first ldapsearh and like 15 seconds later start the second one. With this scenario the problem will for sure appear. In this way we will see large chunks of memory being allocated and after it stabilizes the query get stuck returning as the issue before 16 records per minute, or very slow and getting stuck all the time. Regards, Rodrigo. --- On Mon, 1/26/09, Howard Chu <hyc(a)symas.com> wrote: > From: Howard Chu <hyc(a)symas.com> > Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 > To: rlvcosta(a)yahoo.com > Cc: openldap-its(a)openldap.org > Date: Monday, January 26, 2009, 11:16 PM > rlvcosta(a)yahoo.com wrote: > > Quanah, > > > > In this specific test I reported in my previous e-mail > it was used glibc. > > > > Now I tested with tcmalloc and the memory again was > much more consumed and never released. > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM > TIME+ COMMAND > > 10652 ldap 18 0 2006m 1.8g 68m S 100 15.8 > 8:36.75 slapd > > > > Once it get this much memory it then stabilize but it > is much more than before that was 385m. > > We may need to have sample data from you to reproduce this > situation. I > haven't seen anything like it on my tests. Running two > searches simultaneously > still behaves normally for me. > > > > The scenario was : > > Likewise, the exact commands and arguments will be > needed... > > > > 1) Start a full ldapsearch for one specific dn(the DB > has 4 dn's); > > 2) After sometime start a new full ldapsearch to > behave as queries to DB when replication for LDIF backup is > being done; > > 3) Export one of the queries to a file so speed > verification can be done with grep and wc. > > > > After sometime the memory starts to be consumed in > large chunks until it stopped in around 2G. > > > > Since one of the queries is being redirect to a file I > can tail to have a visual rate of the entrances being > returned by slapd. As the issue before after sometime the > rate appears to get stuck and come in small 16 entrances > chunks but only after some minutes. > > > > This behavior above appears to happen after the second > ldapsearch fills its own dn cache even monitor always shows > only one information as I could see. > > > > [root@brtldp11 ~]# date; grep -e '^pnnum*' > /backup/temp.ldif |wc -l > > Mon Jan 26 21:43:31 BRST 2009 > > 482432 > > [root@brtldp11 ~]# date; grep -e '^pnnum*' > /backup/temp.ldif |wc -l > > Mon Jan 26 21:44:56 BRST 2009 > > 482448 > > > > This behavior is similar as before with the initial > issue with a single process. > > > > I just notice when slapd starts there are these > threads : > > > > [root@brtldp12 ~]# ps -mu ldap > > PID TTY TIME CMD > > 10638 pts/2 00:00:00 slapd > > - - 00:00:00 - > > - - 00:00:00 - > > > > Then after the 2 queries are started I have : > > > > [root@brtldp12 ~]# ps -mu ldap > > PID TTY TIME CMD > > 10195 pts/2 00:44:22 slapd > > - - 00:00:00 - > > - - 00:00:00 - > > - - 00:26:09 - > > - - 00:18:13 - > > - - 00:00:00 - > > > > So 3 new threads. Looks like for some reason with > multiples queries the issue with response getting stuck > repeats. > > > > Even one ldapsearch is stopped the number of threads > continues the same, memory continues allocated and the other > search get with this slow rate. > > > > These outputs are using now tcmalloc but similar as > with glibc. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP > http://www.openldap.org/project/

1 0

Re: (ITS#5860) slapd memeory leak under openldap 2.4
by rlvcosta＠yahoo.com 27 Jan '09

27 Jan '09

Howard, One more comment. If you take a longer time between the ldapsearches the issue doesn't appear sometimes. So you need to start the first ldapsearh and like 15 seconds later start the second one. With this scenario the problem will for sure appear. In this way we will see large chunks of memory being allocated and after it stabilizes the query get stuck returning as the issue before 16 records per minute, or very slow and getting stuck all the time. Regards, Rodrigo. --- On Mon, 1/26/09, Howard Chu <hyc(a)symas.com> wrote: > From: Howard Chu <hyc(a)symas.com> > Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 > To: rlvcosta(a)yahoo.com > Cc: openldap-its(a)openldap.org > Date: Monday, January 26, 2009, 11:16 PM > rlvcosta(a)yahoo.com wrote: > > Quanah, > > > > In this specific test I reported in my previous e-mail > it was used glibc. > > > > Now I tested with tcmalloc and the memory again was > much more consumed and never released. > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM > TIME+ COMMAND > > 10652 ldap 18 0 2006m 1.8g 68m S 100 15.8 > 8:36.75 slapd > > > > Once it get this much memory it then stabilize but it > is much more than before that was 385m. > > We may need to have sample data from you to reproduce this > situation. I > haven't seen anything like it on my tests. Running two > searches simultaneously > still behaves normally for me. > > > > The scenario was : > > Likewise, the exact commands and arguments will be > needed... > > > > 1) Start a full ldapsearch for one specific dn(the DB > has 4 dn's); > > 2) After sometime start a new full ldapsearch to > behave as queries to DB when replication for LDIF backup is > being done; > > 3) Export one of the queries to a file so speed > verification can be done with grep and wc. > > > > After sometime the memory starts to be consumed in > large chunks until it stopped in around 2G. > > > > Since one of the queries is being redirect to a file I > can tail to have a visual rate of the entrances being > returned by slapd. As the issue before after sometime the > rate appears to get stuck and come in small 16 entrances > chunks but only after some minutes. > > > > This behavior above appears to happen after the second > ldapsearch fills its own dn cache even monitor always shows > only one information as I could see. > > > > [root@brtldp11 ~]# date; grep -e '^pnnum*' > /backup/temp.ldif |wc -l > > Mon Jan 26 21:43:31 BRST 2009 > > 482432 > > [root@brtldp11 ~]# date; grep -e '^pnnum*' > /backup/temp.ldif |wc -l > > Mon Jan 26 21:44:56 BRST 2009 > > 482448 > > > > This behavior is similar as before with the initial > issue with a single process. > > > > I just notice when slapd starts there are these > threads : > > > > [root@brtldp12 ~]# ps -mu ldap > > PID TTY TIME CMD > > 10638 pts/2 00:00:00 slapd > > - - 00:00:00 - > > - - 00:00:00 - > > > > Then after the 2 queries are started I have : > > > > [root@brtldp12 ~]# ps -mu ldap > > PID TTY TIME CMD > > 10195 pts/2 00:44:22 slapd > > - - 00:00:00 - > > - - 00:00:00 - > > - - 00:26:09 - > > - - 00:18:13 - > > - - 00:00:00 - > > > > So 3 new threads. Looks like for some reason with > multiples queries the issue with response getting stuck > repeats. > > > > Even one ldapsearch is stopped the number of threads > continues the same, memory continues allocated and the other > search get with this slow rate. > > > > These outputs are using now tcmalloc but similar as > with glibc. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP > http://www.openldap.org/project/

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs