openldap-bugs

openldap-bugs@openldap.org

1 participants
20961 discussions

Re: (ITS#6548) Many "connection_read(): no connection!" warnings when using ldapi:/// and a bind DN (no external authentication)
by gturner＠unzane.com 08 Sep '10

08 Sep '10

Hello folks, I've never used a jitterbug, so I hope this email goes to the right place and doesn't spam up the system. I've been experiencing the same problem as Mark. I use OpenLDAP 2.4.23 packaged in Debian. I have a few processes that connect to ldapi:/// socket and do anonymous binds, there are one or two "no connection!" log messages each time these processes run. None of my remote connections produce these messages. This happens at very low volume, when slapd is idle. When slapd gets more utilization, maybe one or two requests per second, the noise stops. I've spent a few hours looking at the source of OpenLDAP slapd and libldap, also Perl Net::LDAP (the client most often generating these log entries on my system). The source code for OpenLDAP looks very clean, but I'm no C expert, and I'm lost in the asynchronous nature of the connection.c. FYI, Net::LDAP doesn't use libldap - it's all Perl, no native libraries. After hundreds of tests and various olcLogLevel settings I've finally stumbled upon something that solves the problem... usleep(1); I know this has to be the lamest bug report ever, but it works, please bear with me. The first thing that tipped me to try 'usleep' was that with olcLogLevel set to "trace packets args conns stats", I get the following output: connection_get(14) connection_get(14): got connid=1007 connection_read(14): checking for input on id=1007 * op tag 0x42, time 1283917507 * ber_get_next on fd 14 failed errno=0 (Success) connection_read(14): input error=-2 id=1007, closing. connection_closing: readying conn=1007 sd=14 for close connection_close: deferring conn=1007 sd=14 conn=1007 op=2 do_unbind conn=1007 op=2 UNBIND connection_resched: attempting closing conn=1007 sd=14 connection_close: conn=1007 sd=14 daemon: removing 14 conn=1007 fd=14 closed daemon: activity on 1 descriptor daemon: activity on: daemon: epoll: listen=8 active_threads=0 tvp=zero daemon: epoll: listen=9 active_threads=0 tvp=zero connection_get(14) connection_get(14): connection not used connection_get(14) connection_get(14): connection not used connection_read(14): no connection! connection_read(14) error connection_read(14): no connection! The two lines I marked with asterisks are the output from the connection_input function, "op tag 0x42..." should be printed after "ber_get_next on fd 14 failed...", but it's not - so evidently this function is reentrant, the input loop is processing both the UNBIND request and the client closing the socket at the same time, and unfortunately the socket close processing came first so that connection_input thread tears down the connection. Now with the perl scripts I tried adding a microsecond sleep just after they unbind and before they exit, it worked - haven't seen any "no connection!" logs from them for hours. As for the libldap-client and ldapsearch commands, everytime I executed a simple 'ldapsearch -x -H ldapi:///' I'd get two "no connection!" log messages. I tried the same braindead usleep(1) solution and it worked. The following is the diff of what I changed to libldap, ldap_free_connection function in request.c: if ( unbind ) { ldap_send_unbind( ld, lc->lconn_sb, NULL, NULL ); + Debug( LDAP_DEBUG_TRACE, + "ldap_free_connection: sleeping after unbind\n", + 0, 0, 0 ); + usleep(1); } Here's the last bits of output from ldapsearch with -d3 after applying this brutal "fix" to libldap: ldap_free_connection 1 1 ldap_send_unbind ber_flush2: 7 bytes to sd 3 ldap_write: want=7, written=7 0000: 30 05 02 01 03 42 00 0....B. ldap_free_connection: sleeping after unbind ldap_free_connection: actually freed Notice the bytes getting flushed between unbind, my dumb sleep, and "actually freed". Without the sleep it seems slapd may process the close before the unbind. -- Gerald Turner Email: gturner(a)unzane.com JID: gturner(a)jabber.unzane.com GPG: 0xFA8CD6D5 21D9 B2E8 7FE7 F19E 5F7D 4D0C 3FA0 810F FA8C D6D5

1 0

Re: (ITS#5796) back-sql intermittently failing
by frederik.bosch＠gmail.com 07 Sep '10

07 Sep '10

http://www.openldap.org/lists/openldap-technical/201009/msg00070.html While googling for reconnect in the ODBC API I found this: http://dev.mysql.com/doc/refman/5.0/en/connector-odbc-configuration-connect… option 4194304: Enables auto-reconnection functionality. It seems like it fixes my problem.

1 0

Re: (ITS#6641) Syncrepl failure with 'overlay unique'
by andrew.findlay＠skills-1st.co.uk 07 Sep '10

07 Sep '10

On Tue, Sep 07, 2010 at 05:09:07AM -0700, Howard Chu wrote: >> We've talked about doing this isolation in the first refresh upon slapd >> startup. That might still be a good idea. It would certainly help to keep the apparent promises made by things like the uniqueness overlay. Alternatively you could take the view that the data will converge eventually and that is all that the LDAP standards promise. > But that reminds me of the joys of being a Sun sysadmin back in the 1980s, > when Sun's boot scripts always started their NFS client before starting > their NFS server. If two machines cross-mounted each other's filesystems > and both were booting at the same time they would hang, each waiting for > the other's NFS server to respond to their mount request. I remember that - one of the many reasons for switching to automounters (along with their own set of problems)... The alternative was 'soft' mounts, which may be a better model for solving the mirrormode problem. > Mirrormode and multimaster bootstrapping becomes a lot harder if you > implement this type of isolation during startup refresh. I was originally going to suggest that servers should not listen for connections until the first refresh completes, but that would indeed cause the deadlock you describe. How about having master servers listen on an extra port which is used purely for replication and *is* available immediately? The main LDAP port would thus remain closed until the server is synced-up, making it much easier for load-balancers to do the right thing. [This should really be a separate discussion as it is not directly related to the bug.] >> Doing it on every refresh seems far more problematic, because without some >> type of multi-version concurrency control, that means making the server >> non-responsive until the refresh completes. That may not be a problem with refresh-and-persist, as in normal circumstances I would expect updates to arrive at the consumer in the same order they hit the supplier (so this bug could not trigger). More difficult for scheduled refresh mode though. Could the consumer server simply write-lock every entry involved in the refresh while it processes the list, and then commit the whole lot in one DB transaction? Andrew -- ----------------------------------------------------------------------- | From Andrew Findlay, Skills 1st Ltd | | Consultant in large-scale systems, networks, and directory services | | http://www.skills-1st.co.uk/ +44 1628 782565 | -----------------------------------------------------------------------

1 0

Re: (ITS#6641) Syncrepl failure with 'overlay unique'
by hyc＠symas.com 07 Sep '10

07 Sep '10

Howard Chu wrote: > andrew.findlay(a)skills-1st.co.uk wrote: >> Full_Name: Andrew Findlay >> Version: 2.4.23 >> OS: SLES 10.3 x86_64 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (88.97.25.132) >> >> >> The attribute uniqueness overlay can cause syncrepl to jam up. >> Consider this case: > >> The obvious answer is for syncrepl to bypass all data checks, as the supplier >> server should have already enforced them. The problem with that is that clients >> could see data that breaks the rules if they query while syncrepl is running >> (unless the refresh phase is handled as a single transaction and thus isolated >> from clients until it is complete). > > We've talked about doing this isolation in the first refresh upon slapd > startup. That might still be a good idea. But that reminds me of the joys of being a Sun sysadmin back in the 1980s, when Sun's boot scripts always started their NFS client before starting their NFS server. If two machines cross-mounted each other's filesystems and both were booting at the same time they would hang, each waiting for the other's NFS server to respond to their mount request. Mirrormode and multimaster bootstrapping becomes a lot harder if you implement this type of isolation during startup refresh. > Doing it on every refresh seems far more problematic, because without some > type of multi-version concurrency control, that means making the server > non-responsive until the refresh completes. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6641) Syncrepl failure with 'overlay unique'
by hyc＠symas.com 07 Sep '10

07 Sep '10

andrew.findlay(a)skills-1st.co.uk wrote: > Full_Name: Andrew Findlay > Version: 2.4.23 > OS: SLES 10.3 x86_64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (88.97.25.132) > > > The attribute uniqueness overlay can cause syncrepl to jam up. > Consider this case: > The obvious answer is for syncrepl to bypass all data checks, as the supplier > server should have already enforced them. The problem with that is that clients > could see data that breaks the rules if they query while syncrepl is running > (unless the refresh phase is handled as a single transaction and thus isolated > from clients until it is complete). We've talked about doing this isolation in the first refresh upon slapd startup. That might still be a good idea. Doing it on every refresh seems far more problematic, because without some type of multi-version concurrency control, that means making the server non-responsive until the refresh completes. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#6641) Syncrepl failure with 'overlay unique'
by andrew.findlay＠skills-1st.co.uk 07 Sep '10

07 Sep '10

Full_Name: Andrew Findlay Version: 2.4.23 OS: SLES 10.3 x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (88.97.25.132) The attribute uniqueness overlay can cause syncrepl to jam up. Consider this case: 1) Configure a mirrormode pair with identical configs, including 'unique_attributes uid' 2) Start one server and add some data, including an entry containing a UID value. 3) Start the second server, let it synchronise, then stop it again. 4) On the first server, delete the entry and add a new one with a different DN that has the same UID value. 5) Start the second server. Look in the logs, and you will see something like this: Sep 7 11:33:03 jupiter slapd[19407]: @(#) $OpenLDAP: slapd 2.4.23 (Sep 6 2010 19:55:47) $ andrew@jupiter:/usr/local/src/openldap-2.4.23/servers/slapd Sep 7 11:33:03 jupiter slapd[19408]: slapd starting Sep 7 11:33:04 jupiter slapd[19408]: conn=1000 fd=23 ACCEPT from IP=127.0.0.1:33611 (IP=0.0.0.0:389) Sep 7 11:33:04 jupiter slapd[19408]: conn=1000 op=0 BIND dn="" method=128 Sep 7 11:33:04 jupiter slapd[19408]: conn=1000 op=0 RESULT tag=97 err=0 text= Sep 7 11:33:04 jupiter slapd[19408]: conn=1000 op=1 SRCH base="" scope=0 deref=0 filter="(objectClass=*)" Sep 7 11:33:04 jupiter slapd[19408]: conn=1000 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text= Sep 7 11:33:04 jupiter slapd[19408]: conn=1000 op=2 UNBIND Sep 7 11:33:04 jupiter slapd[19408]: conn=1000 fd=23 closed Sep 7 11:33:04 jupiter slapd[19408]: null_callback : error code 0x13 Sep 7 11:33:04 jupiter slapd[19408]: syncrepl_entry: rid=001 be_add uniqueIdentifier=list.6b9432f9,dc=routing,dc=mail,dc=apps,dc=nis,dc=brunel,dc=ac,dc=uk failed (19) Sep 7 11:33:04 jupiter slapd[19408]: do_syncrepl: rid=001 rc 19 retrying (9 retries left) Sep 7 11:34:04 jupiter slapd[19408]: null_callback : error code 0x13 Sep 7 11:34:04 jupiter slapd[19408]: syncrepl_entry: rid=001 be_add uniqueIdentifier=list.6b9432f9,dc=routing,dc=mail,dc=apps,dc=nis,dc=brunel,dc=ac,dc=uk failed (19) Sep 7 11:34:04 jupiter slapd[19408]: do_syncrepl: rid=001 rc 19 retrying (9 retries left) Sep 7 11:35:05 jupiter slapd[19408]: null_callback : error code 0x13 Sep 7 11:35:05 jupiter slapd[19408]: syncrepl_entry: rid=001 be_add uniqueIdentifier=list.6b9432f9,dc=routing,dc=mail,dc=apps,dc=nis,dc=brunel,dc=ac,dc=uk failed (19) Sep 7 11:35:05 jupiter slapd[19408]: do_syncrepl: rid=001 rc 19 retrying (9 retries left) The problem is that syncrepl is trying to add the new entry before deleting the old one, and the uniqueness constraint on UID is preventing it from doing so. It is likely that other constraints on data will have the same effect. See for example this thread: http://www.openldap.org/lists/openldap-software/201001/msg00038.html The obvious answer is for syncrepl to bypass all data checks, as the supplier server should have already enforced them. The problem with that is that clients could see data that breaks the rules if they query while syncrepl is running (unless the refresh phase is handled as a single transaction and thus isolated from clients until it is complete). Andrew

1 0

(ITS#6640) pcache must always free callback
by masarati＠aero.polimi.it 07 Sep '10

07 Sep '10

Full_Name: Pierangelo Masarati Version: irrelevant OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.72.165.221) Submitted by: ando pcache_op_cleanup should always free callback malloc'ed by pcache_op_search when done; caught with valgrind (need -DSLAP_NO_SL_MALLOC); see ITS#6629

1 0

Re: (ITS#6598) EXOP to return number of children of a (sub)tree
by masarati＠aero.polimi.it 07 Sep '10

07 Sep '10

> masarati(a)aero.polimi.it wrote: >>> If you want to be able to distinguish results based on scope then it >>> makes >>> more sense to just define a Search control. Like No-op - count the >>> entries >>> that would be returned, instead of returning them. (And bypassing the >>> sizelimit too, but probably not bypassing timelimit.) >> >> I'm drafting the code to implement this no-op search. > > Maybe sizelimit should not be by-passed in all cases. > How about letting the client choose which limits are by-passed? Michael, your idea sounds interesting. In the current implementation sizelimits are technically bypassed during the internal search. Since the operation returns only a (possibly successful) searchResultDone, it seems appropriate to return success when sizelimit would be violated, because from the client's perspective no sizelimit could be exceeded since no entry is being returned. The fact that the actual search would be exceeded is reflected in the additional response code in the control response's value. Could you be more precise about what you mean? One or two use cases or a not-so-formal specification would suffice. Thanks, p.

1 0

Re: objectClass index from slapd.conf is not working
by tim stone 06 Sep '10

06 Sep '10

Hello, the line "index objectClass eq" is the first index in my config and placed there before first filling the database. But I tried to rebuild this index with slapindex too and I also set up the hole database with slapcat/slapadd on a new testsystem. Every time the same behavior. kindly regards, Tim 2010/9/2 tim stone <timstone10001(a)googlemail.com> > > Hello Michael, > > the line > > "index objectClass eq" > > is the first index in my config and placed there before first filling the database. But I tried to rebuild this index with slapindex too and I also set up the hole database with slapcat/slapadd on a new testsystem. Every time the same behavior. > > kindly regards, Tim > > 2010/9/2 Michael Ströder <michael(a)stroeder.com> >> >> tim stone wrote: >> > >> > I've a strange behavior while using index objectClass for searching. >> >> Did you change the objectClass index *after* populating your database? >> >> http://www.openldap.org/faq/data/cache/136.html >> >> Ciao, Michael. >

2 1

Re: (ITS#6598) EXOP to return number of children of a (sub)tree
by michael＠stroeder.com 06 Sep '10

06 Sep '10

masarati(a)aero.polimi.it wrote: >> If you want to be able to distinguish results based on scope then it makes >> more sense to just define a Search control. Like No-op - count the entries >> that would be returned, instead of returning them. (And bypassing the >> sizelimit too, but probably not bypassing timelimit.) > > I'm drafting the code to implement this no-op search. Maybe sizelimit should not be by-passed in all cases. How about letting the client choose which limits are by-passed? Ciao, Michael.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs