openldap-bugs April 2008

openldap-bugs@openldap.org

45 participants
192 discussions

Re: (ITS#5440) back-meta opens too many connections in case of error
by ali.pouya＠free.fr 02 Apr '08

02 Apr '08

Pierangelo Masarati <ando(a)sys-net.it> wrote : > > issue I observed does not show up any longer. Probably, as an outcome > of this ITS, we should change the default to "yes". > Yes. I agree with you. Because in all my configurations I have to set : pseudoroot-bind-defer yes Thanks for the improvement. Best Regards Ali

1 0

Re: (ITS#5451) syncprov_checkpoint deadlock bugfix?
by rein＠basefarm.no 01 Apr '08

01 Apr '08

On Tue, 1 Apr 2008, rein(a)basefarm.no wrote: > Full_Name: Rein Tollevik > Version: CVS head > OS: linux and solaris > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (81.93.160.250) > > > We have seen deadlock/hang situations in our master server today, which looks > like a deadlock caused by the si_csn_rwlock lock being held while > syncprov_checkpoint() is running. The patch at the end should (I hope) fix > this. Hm, I looks as if I was a bit to quick here, it just hung got again :-(. And this time without anyone competing for this lock, only the lock on the glue suffix entry. I wonder if it can be the upgrade from db 4.6.18 to 4.6.21.1 I dit yesterday that is the real problem. I'll try to downgrade and see if that helps. Please put this case on hold. Sorry! Rein

1 0

(ITS#5451) syncprov_checkpoint deadlock bugfix?
by rein＠basefarm.no 01 Apr '08

01 Apr '08

Full_Name: Rein Tollevik Version: CVS head OS: linux and solaris URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (81.93.160.250) We have seen deadlock/hang situations in our master server today, which looks like a deadlock caused by the si_csn_rwlock lock being held while syncprov_checkpoint() is running. The patch at the end should (I hope) fix this. The scenario is a server with a the syncprov overlay on a glue database with multiple bdb backends as subordinate. One of them is a syncrepl consumer, although I don't know if that matters. A modify operation on the syncrepl consumer causes a checkpoint of the contextCSN. The resulting modify on the glue backend hangs in bdb_cache_modify() awaiting the bdb_cache_entry_db_relock() to return a write lock on the glue suffix entry. At the same time is a modify on one of the other backends hanging in syncprov_op_response() waiting for a lock on the si_csn_rwlock which the first thread holds. The remaining threads quickly blocks waiting for a read lock on the glue suffix entry (the most common search base). I recon the read locks are queued to give priority to the thread waiting for the write lock. What I haven't figured out is who that holds the lock on the glue suffix entry in the first place... Anyway, holding the si_csn_rwlock while the contextCSN is written to the database should not be necessary. But I don't know what consequences it whould have if more than one thread should end up checkpointing competing values, so I have added a new flag to the syncprov_info structure to prevent simultaneons checkpoints from occuring rather than using locks to do it. Rein Tollevik Basefarm AS Index: OpenLDAP/servers/slapd/overlays/syncprov.c diff -u OpenLDAP/servers/slapd/overlays/syncprov.c:1.1.1.13 OpenLDAP/servers/slapd/overlays/syncprov.c:1.12 --- OpenLDAP/servers/slapd/overlays/syncprov.c:1.1.1.13 Mon Mar 31 17:29:48 2008 +++ OpenLDAP/servers/slapd/overlays/syncprov.c Tue Apr 1 21:54:05 2008 @@ -128,6 +128,7 @@ int si_numops; /* number of ops since last checkpoint */ int si_nopres; /* Skip present phase */ int si_usehint; /* use reload hint */ + int si_check; /* are we checkpointing? */ time_t si_chklast; /* time of last checkpoint */ Avlnode *si_mods; /* entries being modified */ sessionlog *si_logs; @@ -1308,8 +1309,10 @@ slap_callback cb = {0}; BackendDB be; + ldap_pvt_thread_rdwr_rlock( &si->si_csn_rwlock ); mod.sml_numvals = si->si_numcsns; mod.sml_values = si->si_ctxcsn; + ldap_pvt_thread_rdwr_runlock( &si->si_csn_rwlock ); mod.sml_nvalues = NULL; mod.sml_desc = slap_schema.si_ad_contextCSN; mod.sml_op = LDAP_MOD_REPLACE; @@ -1335,6 +1338,8 @@ if ( mod.sml_next != NULL ) { slap_mods_free( mod.sml_next, 1 ); } + + si->si_check = 0; } static void @@ -1612,7 +1617,7 @@ } si->si_numops++; - if ( si->si_chkops || si->si_chktime ) { + if ( !si->si_check && (si->si_chkops || si->si_chktime) ) { if ( si->si_chkops && si->si_numops >= si->si_chkops ) { do_check = 1; si->si_numops = 0; @@ -1627,12 +1632,14 @@ } } } + + if ( do_check ) + si->si_check = 1; + ldap_pvt_thread_rdwr_wunlock( &si->si_csn_rwlock ); if ( do_check ) { - ldap_pvt_thread_rdwr_rlock( &si->si_csn_rwlock ); syncprov_checkpoint( op, rs, on ); - ldap_pvt_thread_rdwr_runlock( &si->si_csn_rwlock ); } opc->sctxcsn.bv_len = maxcsn.bv_len;

1 0

(ITS#5450) slapd: schema_init.c:2215: integerIndexer: Assertion `i > 0' failed.
by sahullameerbasha.nagore＠etrade.com 01 Apr '08

01 Apr '08

Full_Name: ameer basha Version: 2.4.8 OS: Red Hat linux As3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (198.93.33.254) I am getting the following error while adding the data into the ldap through siteminder gui. slapd: schema_init.c:2215: integerIndexer: Assertion `i > 0' failed. Do you have any idea why this error is getting

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by ando＠sys-net.it 01 Apr '08

01 Apr '08

Hallvard B Furuseth wrote: > Pierangelo Masarati writes: >> Hallvard B Furuseth wrote: >>> But I don't see how the be_release() code can work now. It sounds like >>> be->be_release() functions must check (how?) that the entry was created >>> by 'be', and otherwise pass it on to the next overlay/backend or >>> otherwise to entry_free(). Might involve mucking with op->o_bd and >>> sr_entry->e_private, I suppose. Except maybe I'm missing some existing >>> magic since slapd doesn't regularly crash... >> Yes, but that's trivial: e_private must be NULL for temporary entries, >> and copying the entry loses it (no one is supposed to muck with it >> expect the entry's creator). > > OK... > >> And the appropriate o_bd of a >> (non-modifiable) entry can be easily computed from the entry's DN. > > Hm? Can only the "outermost" backend/overlay create non-modifiable > entries? > > I think any overlay can replace a non-modifiable rs->sr_entry value, as > long as it restores it on the way out (unless mustbe<freed/released>). That's why I'm not so happy with MUSTRELEASE. >>>> Similarly, the existence of REP_ENTRY_MUSTBEFREED is not totally clear: >>>> in principle as soon as REP_ENTRY_MODIFYABLE is set, it should imply >>>> REP_ENTRY_MUSTBEFREED; the only difference in the semantics of the two >>> That's not my impression. (...) > > Some functions create entries on the stack an send them MODIFIABLE - but > obviously without MUSTBEFREED. They use entry_clean() instead. > > Others set MODIFIABLE but not MUSTBEFREED on non-stack entries, and call > entry_free after sending it. Or forget to free it - back-perl/search.c > if LDAP_SIZELIMIT_EXCEEDED. (Not patching yet - don't have time to test > it at the moment.) OK, makes sense. But perhaps creating entries on the stack could be deprecated now that entries are pooled and reused by calling entry_alloc() entry_free(). This is just food for thoughts, we could as well leave all those flags 'round, if we can streamline their handling. >> (...) A copy might be created by an overlay after receiving a >> read-only entry, but the same overlay might not actually perform the >> copy if it receives the entry from another overlay that already copied >> it, or from a proxy backend. However, after the entry is copied the >> overlay will have no means to determined who actually created the copy. > > I don't think it matters who created a MUSTBEFREED copy: right. >> This might be an issue depending on the order cleanup handlers are >> called (didn't check what order they're called). My point is that >> temporary entries need to be freed at some point; who frees them should >> not be relevant... > > Right, someone must call entry_free() and clear rs->sr_entry (so it's > not freed again), but entry_free() is not passed a backend parameter. > > OTOH a MUSTBERELEASED entry must be released by the right > backend/overlay, and it would be strange for such an entry to be > MODIFIBALE. Though I guess it could be - if the backend has no cache, > and be_release just exists to clean up some data in e_private. > >>> Others apparent problems (also not tested, I've just browsed the code): >>> Overlays that obey and reset MUSTBEFREED or MUSTRELEASE, do not >>> necessarily clear or set MODIFIABLE when setting a new entry. >>> (...) >> It seems to me that we should provide "smart" handlers to deal with >> preparing sr_entry for modification, and to take care of cleaning things >> up as appropriate. Those helpers should then be consistently used in >> the code. > > Yup. And add some aggressive asserts - at least #ifdef LDAP_DEVEL. > Don't know when I'll have time for it though. Making room in my todo list... p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5448) CONTRIB: Java slapd
by Kurt＠OpenLDAP.org 01 Apr '08

01 Apr '08

On Mar 31, 2008, at 8:01 PM, hyc(a)OpenLDAP.org wrote: > Full_Name: Howard Chu > Version: 0.0.1 > OS: > URL: ftp://ftp.openldap.org/incoming/jslapd.tgz > Submission from: (NULL) (76.91.220.157) > Submitted by: hyc > > > Given the increasing popularity of Java, I decided it was time that > the > OpenLDAP Project had its own Java implementation of an LDAP server. It > gave me a good opportunity to explore all of the benefits of rapid > development, introspection, and other advantages of the Java system. > Indeed, drawing on my experience with the existing OpenLDAP code base, > it took only a matter of hours to create my first working server, > which > I present here. Of course it's still experimental in nature. > > Invocation is similar to the regular C-based slapd, and it accepts all > of the same arguments and reads all of the same config settings. In > addition, the first commandline argument must be the name of the > directory > where slapd resides, so that the java app can locate the config files, > schema, etc. > > E.g. for a typical OpenLDAP installation in /usr/local you would run > java jslapd /usr/local/libexec -h ldap://:389 > > A small shell script has been provided to simplify this invocation. > In my initial testing, I found jslapd to have performance on par with slapd(8)! And the memory footprint doesn't appear all that much larger. Good work! -- Kurt

1 0

Re: (ITS#5440) back-meta opens too many connections in case of error
by ando＠sys-net.it 01 Apr '08

01 Apr '08

Ali Pouya wrote: > Hi Pierangelo; > > When trying to reproduce the problem with test035 material I found a > problem in my own JAVA code ! > There is no such problem in back-meta :-) > Sorry for misreporting and thanks for your help. OK, no problem. I note that the issue I fixed in HEAD is basically related to not using pseudoroot-bind-defer yes The default is "no", basically to preserve the original behavior. The original behavior consisted in always binding to all targets when bound as the rootdn. When "pseudoroot-bind-defer" is set to "yes", the bind only occurs locally at the proxy, and is propagated to the targets only when they actually need to be used. The reason is that binding as the rootdn might only be needed to operate on a subset of the targets, so there is no need to add extra traffic to those outside this set. When "pseudoroot-bind-defer" is set to "yes", this no longer occurs, and the issue I observed does not show up any longer. Probably, as an outcome of this ITS, we should change the default to "yes". p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5340) REP_ENTRY_MODIFIABLE bug in dynlist
by h.b.furuseth＠usit.uio.no 01 Apr '08

01 Apr '08

Pierangelo Masarati writes: > Hallvard B Furuseth wrote: >> But I don't see how the be_release() code can work now. It sounds like >> be->be_release() functions must check (how?) that the entry was created >> by 'be', and otherwise pass it on to the next overlay/backend or >> otherwise to entry_free(). Might involve mucking with op->o_bd and >> sr_entry->e_private, I suppose. Except maybe I'm missing some existing >> magic since slapd doesn't regularly crash... > > Yes, but that's trivial: e_private must be NULL for temporary entries, > and copying the entry loses it (no one is supposed to muck with it > expect the entry's creator). OK... > And the appropriate o_bd of a > (non-modifiable) entry can be easily computed from the entry's DN. Hm? Can only the "outermost" backend/overlay create non-modifiable entries? I think any overlay can replace a non-modifiable rs->sr_entry value, as long as it restores it on the way out (unless mustbe<freed/released>). >>> Similarly, the existence of REP_ENTRY_MUSTBEFREED is not totally clear: >>> in principle as soon as REP_ENTRY_MODIFYABLE is set, it should imply >>> REP_ENTRY_MUSTBEFREED; the only difference in the semantics of the two >> >> That's not my impression. (...) Some functions create entries on the stack an send them MODIFIABLE - but obviously without MUSTBEFREED. They use entry_clean() instead. Others set MODIFIABLE but not MUSTBEFREED on non-stack entries, and call entry_free after sending it. Or forget to free it - back-perl/search.c if LDAP_SIZELIMIT_EXCEEDED. (Not patching yet - don't have time to test it at the moment.) > (...) A copy might be created by an overlay after receiving a > read-only entry, but the same overlay might not actually perform the > copy if it receives the entry from another overlay that already copied > it, or from a proxy backend. However, after the entry is copied the > overlay will have no means to determined who actually created the copy. I don't think it matters who created a MUSTBEFREED copy: > This might be an issue depending on the order cleanup handlers are > called (didn't check what order they're called). My point is that > temporary entries need to be freed at some point; who frees them should > not be relevant... Right, someone must call entry_free() and clear rs->sr_entry (so it's not freed again), but entry_free() is not passed a backend parameter. OTOH a MUSTBERELEASED entry must be released by the right backend/overlay, and it would be strange for such an entry to be MODIFIBALE. Though I guess it could be - if the backend has no cache, and be_release just exists to clean up some data in e_private. >> Others apparent problems (also not tested, I've just browsed the code): >> Overlays that obey and reset MUSTBEFREED or MUSTRELEASE, do not >> necessarily clear or set MODIFIABLE when setting a new entry. >> (...) > > It seems to me that we should provide "smart" handlers to deal with > preparing sr_entry for modification, and to take care of cleaning things > up as appropriate. Those helpers should then be consistently used in > the code. Yup. And add some aggressive asserts - at least #ifdef LDAP_DEVEL. Don't know when I'll have time for it though. -- Hallvard

1 0

Re: (ITS#5448) CONTRIB: Java slapd
by ando＠sys-net.it 01 Apr '08

01 Apr '08

hyc(a)OpenLDAP.org wrote: > Given the increasing popularity of Java, I decided it was time that the > OpenLDAP Project had its own Java implementation of an LDAP server. It > gave me a good opportunity to explore all of the benefits of rapid > development, introspection, and other advantages of the Java system. > Indeed, drawing on my experience with the existing OpenLDAP code base, > it took only a matter of hours to create my first working server, which > I present here. Of course it's still experimental in nature. > > Invocation is similar to the regular C-based slapd, and it accepts all > of the same arguments and reads all of the same config settings. In > addition, the first commandline argument must be the name of the directory > where slapd resides, so that the java app can locate the config files, > schema, etc. > > E.g. for a typical OpenLDAP installation in /usr/local you would run > java jslapd /usr/local/libexec -h ldap://:389 > > A small shell script has been provided to simplify this invocation. Howard, many thanks for the contribution. I haven't stressed it yet intensively, but at a first glance it seems to be as performing as the latest 2.4. Will this be released in 2.4, or do we need to wait until 2.5 (what about making it 3.0?). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5440) back-meta opens too many connections in case of error
by ali.pouya＠free.fr 01 Apr '08

01 Apr '08

Hi Pierangelo; When trying to reproduce the problem with test035 material I found a problem in my own JAVA code ! There is no such problem in back-meta :-) Sorry for misreporting and thanks for your help. The issue is to be closed. Best regards Ali Pierangelo Masarati a écrit : > ali.pouya(a)free.fr wrote: > >> I precise that The behaviour is not the same if the client binds with >> rootdn. > > I've committed a fix that removes dangling connections as the rootdn; > I have been unable to reproduce the issue with regular users. Please > provide more useful info in order to track the issue (slapd.conf, LDIF > and requests). > > p. > > > > Ing. Pierangelo Masarati > OpenLDAP Core Team > > SysNet s.r.l. > via Dossi, 8 - 27100 Pavia - ITALIA > http://www.sys-net.it > --------------------------------------- > Office: +39 02 23998309 > Mobile: +39 333 4963172 > Email: pierangelo.masarati(a)sys-net.it > --------------------------------------- > > > >

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs April 2008