openldap-bugs

openldap-bugs@openldap.org

1 participants
20958 discussions

Re: (ITS#6104) race condition with cancel operation
by hyc＠symas.com 11 May '09

11 May '09

Hallvard B Furuseth wrote: > hyc(a)symas.com writes: >>> slapd/cancel.c sets o_abandon before o_cancel. Thus it's possible for >>> the canceled operation to obey o_abandon before o_cancel gets set. >> >> Cancel is just best-effort, why not just set o_cancel before o_abandon? > > Memory coherence. Without mutex protection, the cancelled thread might > still see the o_abandon change before the o_cancel change. And thus not > send a response, nor update info for the Cancel operation to react to. > I'd prefer to condense it all into a bitfield. We already ensure that the c_mutex is held when setting these flags. It would be a lot of overhead to grab the mutex just to read the flag, and would require additional analysis to make sure the new locking behavior doesn't introduce new deadlocks. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6104) race condition with cancel operation
by h.b.furuseth＠usit.uio.no 11 May '09

11 May '09

hyc(a)symas.com writes: >> slapd/cancel.c sets o_abandon before o_cancel. Thus it's possible for >> the canceled operation to obey o_abandon before o_cancel gets set. > > Cancel is just best-effort, why not just set o_cancel before o_abandon? Memory coherence. Without mutex protection, the cancelled thread might still see the o_abandon change before the o_cancel change. And thus not send a response, nor update info for the Cancel operation to react to. -- Hallvard

1 0

(ITS#6108) unique overlay and rootdn
by mfn＠fs-etit.de 11 May '09

11 May '09

Full_Name: Christopher Dyck Version: 2.4.11 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (130.83.183.3) I found a strange behavior in the unique overlay. unique overlay only works, when a rootdn is specified in the slapd.conf, because the unique_search function uses the rootdn for searching. (took me hours to determine why my configuration didn't do it's job) Isn't it more reasonable to use the dn with which the add or modify is performed? Or mention this circumstance at least in the manpage?!

1 0

Re: (ITS#6107) Manual pages do not mention -e option
by masarati＠aero.polimi.it 11 May '09

11 May '09

rein(a)OpenLDAP.org wrote: > Full_Name: Rein Tollevik > Version: 2.4.16 > OS: Irrelevant > URL: > Submission from: (NULL) (81.93.160.250) > Submitted by: rein > > > The manual pages for (at least) ldapmodify, ldapmodrdn and ldapdelete do not > mention the "-e control" option, nor is the private "-E" documented for > ldapmodify. > > A lot of the options accepted by the ldap* tools are common to them all. Should > they be documented in a separate man page referred to by the tools manuals, or > at least included from a common source? I favor documenting everything in each man page, to limit the amount of redirection (when reading man pages, one wants to quickly find an answer to a specific problem, not learn everything about LDAP :). Documentation should be concise enough to minimally impact the amount one needs to read to learn how to *use* those controls. To learn about the purpose and the details of those controls, references should be used. My 2c. p.

1 0

(ITS#6107) Manual pages do not mention -e option
by rein＠OpenLDAP.org 11 May '09

11 May '09

Full_Name: Rein Tollevik Version: 2.4.16 OS: Irrelevant URL: Submission from: (NULL) (81.93.160.250) Submitted by: rein The manual pages for (at least) ldapmodify, ldapmodrdn and ldapdelete do not mention the "-e control" option, nor is the private "-E" documented for ldapmodify. A lot of the options accepted by the ldap* tools are common to them all. Should they be documented in a separate man page referred to by the tools manuals, or at least included from a common source? -- Rein Tollevik Basefarm AS

1 0

Re: (ITS#6059) Abandon syncprov race condition?
by rein＠OpenLDAP.org 11 May '09

11 May '09

hyc(a)symas.com wrote: > rein(a)OpenLDAP.org wrote: >> I've had two cases where a delete operation was performed on the master without >> being replicated to its consumers, which so far appear to be cases of possible >> connection lost (abandon) race conditions. The log (level: stats) shows the >> "DEL" message of the entry, immediately followed by a "closed (connection lost)" >> message on the connection. Note: No "RESULT" message was logged. >> >> I haven't looked very much into this, but my theory so far is that syncprov >> skipped replicating of the delete op after noticing the abandon resulting from >> loosing the connection, even though the delete had already taken place in the >> local database. That it happened after a delete op might very well have been a >> coincident, this possible race could exist after any modify op for all I know. > >> Do we need some sort of o_committed flag that can be used to prevent o_abandon >> from being set or acted upon? Or handle o_abandon more like o_cancel, i.e with >> multiple values, including "too late"? > > No. What good can that do, since the connection has already been lost? > > It doesn't matter if syncprov fails to send an update to a consumer - the > consumer's cookie state will let it pick up where it left off when it reconnects. It isn't the connection to the syncprov consumer that was lost, it is the connection to the client that made the change. The abandon may cause syncprov to abandon the modify op (in syncprov_op_abandon), and it will definitely cause the entire response callback to be skipped. Which is where syncprov sends updates to its consumers. The change will take place in the local database, but not be replicated, nor will auditlog log it. Accesslog does though, probably since the cleanup callback it enables in accesslog_op_mod explicitly calls its response callback. The syncprov clients will receive updates to the csn when new modifications takes place, i.e the clients must be restarted with the "-c" option to resync their databases. Rein

1 0

(ITS#6106) Unable to start LDAP Service in Windows
by Jemmy.Sentonius＠infor.com 11 May '09

11 May '09

Full_Name: Jemmy Sentonius Version: 2.2.29 OS: Windows 2003 Server URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (211.24.230.162) 1) Start the LDAP Service in Windows (logon as Domain Administrator) 2) System will prompt error: Windows could not start the InforSecurity LDAP on local computer. For more information, review the system event log. If this is non-Microsoft Service, contact the service vendor, and refer to service-specific error code 16. 3) Go to Windows Event Viewer, System Log Notice that there's an error log with description: The InforSecurity LDAP Service terminated with service-specific error 16 (0x10)

1 0

Re: (ITS#6104) race condition with cancel operation
by hyc＠symas.com 11 May '09

11 May '09

h.b.furuseth(a)usit.uio.no wrote: > Full_Name: Hallvard B Furuseth > Version: HEAD > OS: Linux > URL: > Submission from: (NULL) (129.240.6.233) > Submitted by: hallvard > > > slapd/cancel.c sets o_abandon before o_cancel. Thus it's possible for > the canceled operation to obey o_abandon before o_cancel gets set. Cancel is just best-effort, why not just set o_cancel before o_abandon? > Though I had to insert some sleeps to achieve that. > Either the operation is abandoned and the Cancel operation receives > tooLate, or if the client unbinds/closes the connection fast enough > Cancel will hang: slapd does not close the connection, and hangs on > shutdown: "slapd shutdown: waiting for 1 operations/tasks to finish". > > Since the flags are not mutex-protected (at least not when read), it's > not enough to move the o_cancel setting after o_abandon in the Cancel > thread. The cancelled thread might still see the o_abandon change > first. A fix could be to make o_abandon a bitmask which says whether > the abandon is actually a cancel, but the Abandon and Cancel operations > will still need a mutex to coordinate so that Abandon does not reset > a Cancel bitflag. In any case, it'd be cleaner if an operation which > reacts to o_abandon grabs some mutex before checking o_cancel. > > > The problem was tested as follows: > - sleep 0.2 sec after Statslog "DEL" and before setting SLAP_CANCEL_REQ. > - log "ABANDONED" when send_ldap_response() abandons the operation. > - Client: A python socket client which sends raw BER, no libldap: > s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) > s.connect(('localhost', 3890)) > s.send(delete("cn=test")) > time.sleep(0.1) > s.send(cancel()) # cancel last operation > #sys.exit() > time.sleep(0.4) > s.send(unbind()) > --> > conn=0 fd=9 ACCEPT from IP=127.0.0.1:56945 (IP=127.0.0.1:3890) > conn=0 op=0 DEL dn="cn=test" > conn=0 op=1 EXT oid=1.3.6.1.1.8 > conn=0 op=1 CANCEL msg=1 > conn=0 op=0 ABANDONED > conn=0 op=2 UNBIND > conn=0 op=1 RESULT oid= err=120 text= > conn=0 fd=9 closed > <server closed connection, client exited> > ^C slapd > > If the client exits after send(cancel()): > conn=0 fd=9 ACCEPT from IP=127.0.0.1:48826 (IP=127.0.0.1:3890) > conn=0 op=0 DEL dn="cn=test" > conn=0 op=1 EXT oid=1.3.6.1.1.8 > conn=0 op=1 CANCEL msg=1 > conn=0 op=2 UNBIND > conn=0 op=0 ABANDONED > <not closing connection> > ^C slapd > daemon: shutdown requested and initiated. > slapd shutdown: waiting for 1 operations/tasks to finish > <slapd is hanging> > kill -KILL<slapd> > > slapd.conf: > include servers/slapd/schema/core.schema > allow update_anon > database ldif > directory "." > suffix "cn=test" > > Patches to slapd: > > Index: cancel.c > --- cancel.c 21 Jan 2009 23:40:25 -0000 1.30 > +++ cancel.c 11 May 2009 04:42:58 -0000 > @@ -92,4 +92,8 @@ > } > > + { > + struct timeval timeout = { 0, 200000 }; > + select(0, NULL, NULL, NULL,&timeout); > + } > o->o_cancel = SLAP_CANCEL_REQ; > > Index: delete.c > --- delete.c 21 Jan 2009 23:40:26 -0000 1.144 > +++ delete.c 11 May 2009 04:42:58 -0000 > @@ -75,4 +75,8 @@ > op->o_log_prefix, op->o_req_dn.bv_val, 0, 0, 0 ); > > + { > + struct timeval timeout = { 0, 200000 }; > + select(0, NULL, NULL, NULL,&timeout); > + } > if( op->o_req_ndn.bv_len == 0 ) { > Debug( LDAP_DEBUG_ANY, "%s do_delete: root dse!\n", > Index: result.c > --- result.c 11 May 2009 02:23:51 -0000 1.331 > +++ result.c 11 May 2009 04:42:58 -0000 > @@ -418,4 +418,6 @@ > if (( rs->sr_err == SLAPD_ABANDON || op->o_abandon )&& !op->o_cancel ) { > rc = SLAPD_ABANDON; > + Statslog( LDAP_DEBUG_STATS, > + "%s ABANDONED\n", op->o_log_prefix, 0, 0, 0, 0 ); > goto clean2; > } > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#6104) race condition with cancel operation
by h.b.furuseth＠usit.uio.no 11 May '09

11 May '09

Full_Name: Hallvard B Furuseth Version: HEAD OS: Linux URL: Submission from: (NULL) (129.240.6.233) Submitted by: hallvard slapd/cancel.c sets o_abandon before o_cancel. Thus it's possible for the canceled operation to obey o_abandon before o_cancel gets set. Though I had to insert some sleeps to achieve that. Either the operation is abandoned and the Cancel operation receives tooLate, or if the client unbinds/closes the connection fast enough Cancel will hang: slapd does not close the connection, and hangs on shutdown: "slapd shutdown: waiting for 1 operations/tasks to finish". Since the flags are not mutex-protected (at least not when read), it's not enough to move the o_cancel setting after o_abandon in the Cancel thread. The cancelled thread might still see the o_abandon change first. A fix could be to make o_abandon a bitmask which says whether the abandon is actually a cancel, but the Abandon and Cancel operations will still need a mutex to coordinate so that Abandon does not reset a Cancel bitflag. In any case, it'd be cleaner if an operation which reacts to o_abandon grabs some mutex before checking o_cancel. The problem was tested as follows: - sleep 0.2 sec after Statslog "DEL" and before setting SLAP_CANCEL_REQ. - log "ABANDONED" when send_ldap_response() abandons the operation. - Client: A python socket client which sends raw BER, no libldap: s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect(('localhost', 3890)) s.send(delete("cn=test")) time.sleep(0.1) s.send(cancel()) # cancel last operation #sys.exit() time.sleep(0.4) s.send(unbind()) --> conn=0 fd=9 ACCEPT from IP=127.0.0.1:56945 (IP=127.0.0.1:3890) conn=0 op=0 DEL dn="cn=test" conn=0 op=1 EXT oid=1.3.6.1.1.8 conn=0 op=1 CANCEL msg=1 conn=0 op=0 ABANDONED conn=0 op=2 UNBIND conn=0 op=1 RESULT oid= err=120 text= conn=0 fd=9 closed <server closed connection, client exited> ^C slapd If the client exits after send(cancel()): conn=0 fd=9 ACCEPT from IP=127.0.0.1:48826 (IP=127.0.0.1:3890) conn=0 op=0 DEL dn="cn=test" conn=0 op=1 EXT oid=1.3.6.1.1.8 conn=0 op=1 CANCEL msg=1 conn=0 op=2 UNBIND conn=0 op=0 ABANDONED <not closing connection> ^C slapd daemon: shutdown requested and initiated. slapd shutdown: waiting for 1 operations/tasks to finish <slapd is hanging> kill -KILL <slapd> slapd.conf: include servers/slapd/schema/core.schema allow update_anon database ldif directory "." suffix "cn=test" Patches to slapd: Index: cancel.c --- cancel.c 21 Jan 2009 23:40:25 -0000 1.30 +++ cancel.c 11 May 2009 04:42:58 -0000 @@ -92,4 +92,8 @@ } + { + struct timeval timeout = { 0, 200000 }; + select(0, NULL, NULL, NULL, &timeout); + } o->o_cancel = SLAP_CANCEL_REQ; Index: delete.c --- delete.c 21 Jan 2009 23:40:26 -0000 1.144 +++ delete.c 11 May 2009 04:42:58 -0000 @@ -75,4 +75,8 @@ op->o_log_prefix, op->o_req_dn.bv_val, 0, 0, 0 ); + { + struct timeval timeout = { 0, 200000 }; + select(0, NULL, NULL, NULL, &timeout); + } if( op->o_req_ndn.bv_len == 0 ) { Debug( LDAP_DEBUG_ANY, "%s do_delete: root dse!\n", Index: result.c --- result.c 11 May 2009 02:23:51 -0000 1.331 +++ result.c 11 May 2009 04:42:58 -0000 @@ -418,4 +418,6 @@ if (( rs->sr_err == SLAPD_ABANDON || op->o_abandon ) && !op->o_cancel ) { rc = SLAPD_ABANDON; + Statslog( LDAP_DEBUG_STATS, + "%s ABANDONED\n", op->o_log_prefix, 0, 0, 0, 0 ); goto clean2; }

1 0

Re: (ITS#6059) Abandon syncprov race condition?
by hyc＠symas.com 11 May '09

11 May '09

rein(a)OpenLDAP.org wrote: > Full_Name: Rein Tollevik > Version: 2.4.16 > OS: linux > URL: > Submission from: (NULL) (81.93.160.250) > Submitted by: rein > > > I've had two cases where a delete operation was performed on the master without > being replicated to its consumers, which so far appear to be cases of possible > connection lost (abandon) race conditions. The log (level: stats) shows the > "DEL" message of the entry, immediately followed by a "closed (connection lost)" > message on the connection. Note: No "RESULT" message was logged. > > I haven't looked very much into this, but my theory so far is that syncprov > skipped replicating of the delete op after noticing the abandon resulting from > loosing the connection, even though the delete had already taken place in the > local database. That it happened after a delete op might very well have been a > coincident, this possible race could exist after any modify op for all I know. > Do we need some sort of o_committed flag that can be used to prevent o_abandon > from being set or acted upon? Or handle o_abandon more like o_cancel, i.e with > multiple values, including "too late"? No. What good can that do, since the connection has already been lost? It doesn't matter if syncprov fails to send an update to a consumer - the consumer's cookie state will let it pick up where it left off when it reconnects. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs