- openldap-bugs - openldap.org

Re: (ITS#5433) syncprov in a glue'ed environment write contextCSN to random subordinate DB
by hyc＠symas.com 20 Mar '08

20 Mar '08

rein(a)basefarm.no wrote: > Full_Name: Rein Tollevik > Version: 2.4.8 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (84.215.2.34) > > > syncprov_checkpoint() writes the contextCSN attribute to the suffix of the > backend DB that is currently being written when it is triggered. When syncprov > is stacked on top of a set of glue'ed databases this can be any of the > subordinate backend DBs. The attached patch fixes this problem. > > Rein Tollevik > Basefarm AS Thanks for the report. An alternate fix is in HEAD. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5432) syncrepl seg. faults if received a sync cookie with an empty csn= value
by hyc＠symas.com 20 Mar '08

20 Mar '08

rein(a)basefarm.no wrote: > Full_Name: Rein Tollevik > Version: 2.4.8 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (84.215.2.34) > > > The attached patch fixes a seg. fault in compare_csns() in syncrepl.c if it > receives a sync cookie with an empty "csn=" value. This has been fixed in HEAD. > syncprov_playlog() in syncprov.c generates such invalid cookies when replaying a > sessionlog where no entries has been deleted. The patch also fixes this bug. This should never happen, since the cookie is not generated if there were no deletes recorded. > diff -u OpenLDAP/servers/slapd/overlays/syncprov.c:1.5 > OpenLDAP/servers/slapd/overlays/syncprov.c:1.6 > --- OpenLDAP/servers/slapd/overlays/syncprov.c:1.5 Fri Mar 14 16:23:48 2008 > +++ OpenLDAP/servers/slapd/overlays/syncprov.c Tue Mar 18 17:22:41 2008 > @@ -1413,8 +1413,7 @@ > num * UUID_LEN, op->o_tmpmemctx ); > uuids[0].bv_val = (char *)(uuids + num + 1); > > - delcsn[0].bv_len = 0; > - delcsn[0].bv_val = cbuf; > + BER_BVZERO(&delcsn[0]); > BER_BVZERO(&delcsn[1]); > > /* Make a copy of the relevant UUIDs. Put the Deletes up front > @@ -1453,6 +1452,7 @@ > i++; > AC_MEMCPY( cbuf, se->se_csn.bv_val, se->se_csn.bv_len ); > delcsn[0].bv_len = se->se_csn.bv_len; > + delcsn[0].bv_val = cbuf; > delcsn[0].bv_val[delcsn[0].bv_len] = '\0'; > } else { > nmods++; > > > -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5431) syncrepl on a subordinate DB writes contextCSN to suffix of parent DB
by hyc＠symas.com 19 Mar '08

19 Mar '08

Rein Tollevik wrote: > On Tue, 18 Mar 2008, Howard Chu wrote: > >> rein(a)basefarm.no wrote: >>> syncrepl_updateCookie() writes the contextCSN to the suffix of si->si_wbe >>> (assigned to op->o_bd). >> Yes. >> >>> In a subordinate database this is the parent DB, it >>> should be written to the suffix of si->si_be instead as the attached patch >>> fixes. >> No. It works as designed. > > Hm, that sounds as a bit odd design to me. I find it most natural that > the contextCSN should be stored in the suffix of the backendDB where > syncrepl is used. Then it doesn't interfere with the contextCSN of > other neighboring databases replicated below the same common superior, > nor with the contextCSN a syncprov overlay on the superior glue database > maintains. But I will not claim to have understood more than a tiny > fraction of the syncrepl protocol and code, so I might be complete lost > on this one. If I'm wrong this raises a few other questions though. > Upon startup syncrepl reads (in do_syncrep1()) its initial contextCSN > value from the suffix of the backend where it is used. If it is correct > that it in a glue'ed environment should write it to the suffix of the > superior glue DB, shouldn't it be read from there as well? Yes. This code is still being reworked. > My experience from the 2.4.8 release is that syncrepl and syncprov tends > to interfere with each other when they both try to maintain the > contextCSN of the same backend DB. I.e if they are both used on the > same database or if syncrepl writes its contextCSN to the suffix of a > superior glue database where syncprov is used. I realize that using > both on the same backend may be necessary in many configurations, but > luckily we was able to avoid it. I see that the syncrepl code that uses > delete/add to update the attribute (which would fail after syncprov has > replaced all values) has been rewritten in cvs, so I guess this would > not be a problem in an upcoming release. That's ITS#5426. It's also one of the reasons that our contributing guidelines state that you should submit patches against HEAD, not the released code. These fixes will be in 2.4.9 but it's likely that the patches you've created against 2.4.8 are already obsolete. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5431) syncrepl on a subordinate DB writes contextCSN to suffix of parent DB
by rein＠basefarm.no 19 Mar '08

19 Mar '08

On Tue, 18 Mar 2008, Howard Chu wrote: > rein(a)basefarm.no wrote: >> >> syncrepl_updateCookie() writes the contextCSN to the suffix of si->si_wbe >> (assigned to op->o_bd). > > Yes. > >> In a subordinate database this is the parent DB, it >> should be written to the suffix of si->si_be instead as the attached patch >> fixes. > > No. It works as designed. Hm, that sounds as a bit odd design to me. I find it most natural that the contextCSN should be stored in the suffix of the backendDB where syncrepl is used. Then it doesn't interfere with the contextCSN of other neighboring databases replicated below the same common superior, nor with the contextCSN a syncprov overlay on the superior glue database maintains. But I will not claim to have understood more than a tiny fraction of the syncrepl protocol and code, so I might be complete lost on this one. If I'm wrong this raises a few other questions though. Upon startup syncrepl reads (in do_syncrep1()) its initial contextCSN value from the suffix of the backend where it is used. If it is correct that it in a glue'ed environment should write it to the suffix of the superior glue DB, shouldn't it be read from there as well? My experience from the 2.4.8 release is that syncrepl and syncprov tends to interfere with each other when they both try to maintain the contextCSN of the same backend DB. I.e if they are both used on the same database or if syncrepl writes its contextCSN to the suffix of a superior glue database where syncprov is used. I realize that using both on the same backend may be necessary in many configurations, but luckily we was able to avoid it. I see that the syncrepl code that uses delete/add to update the attribute (which would fail after syncprov has replaced all values) has been rewritten in cvs, so I guess this would not be a problem in an upcoming release. Rein

1 0

Re: (ITS#5436) htons() expects an unsigned short
by h.b.furuseth＠usit.uio.no 19 Mar '08

19 Mar '08

Rein Tollevik writes: >On Tue, 18 Mar 2008, Hallvard B Furuseth wrote: >> (...) >> short is signed by definition. > > More the good reason why it should be cast to an unsigned then, as > that is what htons() expects. Yup. > No, I don't have a test case. This is a fairly insignificant bugfix I > commited to our copy of the source six years ago and has never gotten > around to report back (sorry for that). Right, then I'll not bother try to figure out how it broke. Maybe it's an incomplete cleanup after some other error which has been fixed. -- Hallvard

1 0

Re: (ITS#5436) htons() expects an unsigned short
by rein＠basefarm.no 19 Mar '08

19 Mar '08

On Tue, 18 Mar 2008, Hallvard B Furuseth wrote: > Thanks. Fixed os-ip.c in CVS HEAD (but see below). > > rein(a)basefarm.no writes: >> libraries/libldap/os-ip.c casts the argument to htons() to a short, it >> should be an explicit unsigned short. This causes the use of high >> port numbers to fail on systems where shorts are signed by default. > > short is signed by definition. More the good reason why it should be cast to an unsigned then, as that is what htons() expects. > Do you have a test case? As far as I can tell it broke if the caller > had the same error: storing a port number in a short, and passing that > short to ldap_init() & co. And it broke on systems with 16-bit int, if > there are such system around where OpenLDAP compiles in the first place. > > If we care about systems with 16-bit int, the proper fix is to also make > LDAPURLDesc.lud_port and various 'port' function parameters unsigned. I > have no idea if there is such a system which can even compile OpenLDAP > though. No, I don't have a test case. This is a fairly insignificant bugfix I commited to our copy of the source six years ago and has never gotten around to report back (sorry for that). The exact details I'm afraid I have forgotten a long time ago, and I generally find that figuring too much about the gory storage details of signed vs. unsigned, and big/little-endianness only gives me a slight headache ;-) But it definitely wasn't any 16-bit systems, and I can't see any reason why anyone should worry about those anymore.. It was most likely on a 64bit sparc system using sun's compiler, but it might have been linux/gcc on a 32bit i686 for all I remember now. Rein

1 0

Re: (ITS#5435) Seg. fault due to dereference of NULL ConfigReply * arguments.
by rein＠basefarm.no 19 Mar '08

19 Mar '08

On Tue, 18 Mar 2008, Howard Chu wrote: > rein(a)basefarm.no wrote: >> Full_Name: Rein Tollevik >> Version: 2.4.8 >> OS: >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (84.215.2.34) >> >> >> Functions in servers/slapd/back-bdb/init.c dereference their ConfigReply * >> arguments without testing that they are non-NULL first, which causes seg. >> faults. > > Obviously this should never happen. Please provide the stack trace from such > an occurrence. It happens if the glue overlay is used, backend_startup_one() is called with a NULL ConfigReply * in glue_open() as the stack frame below shows. This might be the real bug, but testing that the ConfigReply * is non-NULL (as is done in other backend modules) sounds as the right ting to do anyhow. Some of the overlay modules also calls backend_startup_one() with a NULL second argument, and back-bdb/config.c calls bi_db_open() the same way. Rein (gdb) where #0 bdb_db_open (be=0x7f86a0, cr=0x0) at init.c:106 #1 0x00000000004bfe0a in over_db_func (be=0x7f86a0, cr=0x0, which=db_open) at backover.c:55 #2 0x00000000004c030a in over_db_open (be=0x7f86a0, cr=0x0) at backover.c:176 #3 0x000000000044a479 in backend_startup_one (be=0x7f86a0, cr=0x0) at backend.c:224 #4 0x00000000004beaf5 in glue_open (bi=0x7e8850) at backglue.c:653 #5 0x000000000044a62f in backend_startup (be=0x7f8100) at backend.c:255 #6 0x000000000047660c in slap_startup (be=0x7f8100) at init.c:225 #7 0x00000000004c728f in slap_tool_init (progname=0x53fca8 "slapcat", tool=2, argc=5, argv=0x7fbffffb78) at slapcommon.c:725 #8 0x00000000004c5836 in slapcat (argc=5, argv=0x7fbffffb78) at slapcat.c:51 #9 0x0000000000417641 in main (argc=5, argv=0x7fbffffb78) at main.c:640

1 0

Re: (ITS#5435) Seg. fault due to dereference of NULL ConfigReply * arguments.
by hyc＠symas.com 18 Mar '08

18 Mar '08

rein(a)basefarm.no wrote: > Full_Name: Rein Tollevik > Version: 2.4.8 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (84.215.2.34) > > > Functions in servers/slapd/back-bdb/init.c dereference their ConfigReply * > arguments without testing that they are non-NULL first, which causes seg. > faults. Obviously this should never happen. Please provide the stack trace from such an occurrence. > I have only seen this happen when I tried to slapcat a database that > hadn't had its directory created or some other stupidity, but anyhow... > > I don't include my patch to this, as it is probably easier to fix it again in > your source than to apply a patch. > > Rein Tollevik > Basefarm AS > > > -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5431) syncrepl on a subordinate DB writes contextCSN to suffix of parent DB
by hyc＠symas.com 18 Mar '08

18 Mar '08

rein(a)basefarm.no wrote: > Full_Name: Rein Tollevik > Version: 2.4.8 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (84.215.2.34) > > > syncrepl_updateCookie() writes the contextCSN to the suffix of si->si_wbe > (assigned to op->o_bd). Yes. > In a subordinate database this is the parent DB, it > should be written to the suffix of si->si_be instead as the attached patch > fixes. No. It works as designed. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5436) htons() expects an unsigned short
by h.b.furuseth＠usit.uio.no 18 Mar '08

18 Mar '08

Thanks. Fixed os-ip.c in CVS HEAD (but see below). rein(a)basefarm.no writes: > libraries/libldap/os-ip.c casts the argument to htons() to a short, it > should be an explicit unsigned short. This causes the use of high > port numbers to fail on systems where shorts are signed by default. short is signed by definition. Do you have a test case? As far as I can tell it broke if the caller had the same error: storing a port number in a short, and passing that short to ldap_init() & co. And it broke on systems with 16-bit int, if there are such system around where OpenLDAP compiles in the first place. If we care about systems with 16-bit int, the proper fix is to also make LDAPURLDesc.lud_port and various 'port' function parameters unsigned. I have no idea if there is such a system which can even compile OpenLDAP though. -- Hallvard

1 0

Re: Segfault in slapd 2.4.8 - syncrepl and glue (ITS#5430)
by Duncan.Gibb＠SiriusIT.co.uk 18 Mar '08

18 Mar '08

Gavin Henry wrote: GH> OK, thanks. I take it 2.4.8 is a self-compile? Yes it is. Debian ships a patched 2.4.7 which gives you incomplete debug info. We deliberately built from the official source with no patches and optimisation off so we/you could see what's going on before submitting a bug report. Cheers Duncan

1 0

Re: Segfault in slapd 2.4.8 - syncrepl and glue (ITS#5430)
by Duncan.Gibb＠SiriusIT.co.uk 18 Mar '08

18 Mar '08

Quanah Gibson-Mount wrote: QGM> What version of BDB? Was BDB self compiled? What options QGM> were used to compile BDB in either case? In the case from which the backtrace came, it's the system (ie Debian packaged) BDB 4.6.21. A quick glance at debian/rules suggests it at least starts with CFLAGS="-O2" and runs configure with --prefix=/usr \ --mandir=\$${prefix}/share/man \ --localstatedir=/var \ --sysconfdir=/etc \ --libexecdir=/usr/lib \ --enable-cxx \ --enable-compat185 \ --enable-rpc \ --enable-tcl \ --with-tcl=/usr/lib/tcl8.4 \ --enable-test plus --enable-java Is that what you need to know? QGM> I will note that there's known problems with using BDB QGM> under Xen given certain compile options I'm aware of that from doing Cyrus deployments under Xen. Anyway, we're not using Xen in this case precisely so we don't have to worry about that. QGM> I don't know whether or not QEMU is affected. As far as I know it isn't, but to exclude that possibility we posted the bug based on three slapds on the same - bare metal - machine. What makes you think it could be a "blame Berserkley" problem? Is it worth re-testing with a different backend and/or a private controlled build of BDB? Cheers Duncan

1 0

Re: Segfault in slapd 2.4.8 - syncrepl and glue (ITS#5430)
by quanah＠zimbra.com 18 Mar '08

18 Mar '08

--On Tuesday, March 18, 2008 7:36 PM +0000 ghenry(a)suretecsystems.com wrote: > <quote who="Duncan.Gibb(a)SiriusIT.co.uk"> >> Thanks, Gavin. My GDB foo is not up to getting useful info out of your >> core dump, but I have placed one from one of our test rigs at >> >> http://pastebin.siriusit.co.uk/slapd-2.4.8-nullpointer-bin-and-core.tar. >> gz >> >> Note that this expects to be looked at with 64-bit GDB 6.7.1 from Debian >> Lenny/Sid (GDB 6.4.90 from Etch doesn't work). > > OK, thanks. I take it 2.4.8 is a self-compile? > > True, but at least you had a version number, platform and bt. Many don't > give this ;-) It's the details. Useful but missing information: What version of BDB? Was BDB self compiled? What options were used to compile BDB in either case? I will note that there's known problems with using BDB under Xen given certain compile options, I don't know whether or not QEMU is affected. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: Segfault in slapd 2.4.8 - syncrepl and glue (ITS#5430)
by ghenry＠suretecsystems.com 18 Mar '08

18 Mar '08

<quote who="Duncan.Gibb(a)SiriusIT.co.uk"> > Thanks, Gavin. My GDB foo is not up to getting useful info out of your > core dump, but I have placed one from one of our test rigs at > > http://pastebin.siriusit.co.uk/slapd-2.4.8-nullpointer-bin-and-core.tar.gz > > Note that this expects to be looked at with 64-bit GDB 6.7.1 from Debian > Lenny/Sid (GDB 6.4.90 from Etch doesn't work). OK, thanks. I take it 2.4.8 is a self-compile? > > > Re your previous comments: > > The engineer who put together the test case did look for bug reporting > guidelines, but their location in the FAQ is, IMHO, non-obvious; perhaps > one day I'll fix that. I reviewed and edited the report before sending > it and I think it meets any reasonable person's definition of at least > attempting to be helpful whilst not unduly verbose. True, but at least you had a version number, platform and bt. Many don't give this ;-) It's the details. I wrote up a useful page in the Admin guide and would appreciate any feedback you have as it is obviously not easily found by our users: http://www.openldap.org/doc/admin24/troubleshooting.html > > Posting to the wrong list was my fault and I apologise. The moderator > was correct to reject the original message. ;-) > > We know all kinds of weird stuff happens if you restart the crashed > server(s) - that's how we come to be looking at this in the first place. Of course. > We have, however, tried very hard to come up with a simple, confined > case of this particular null pointer dereference, which is the first > thing that goes wrong if you start from a clean sheet with empty > databases each time and follow the recipe. > > In the four environments we can easily construct (RHEL vs Debian, i386 > vs amd64) the segfault of server1 happens immediately you submit any > change via LDAP to server2. > > Can anyone else reproduce this, or do we need to work on more examples? I'm going to try and reproduce it reliably shortly. Lastly, without knowing your project requirements, I would say that this deployment could be greatly simplified by having one Provider and 2 Consumers. The second server (server2 in your example) could use the chain overlay to push back changes. Again, without knowing the big picture, it's hard to comment. This discussion is best carried out on -software or -technical however, so let's concentrate on the bug. Thanks, Gavin.

1 0

Re: Segfault in slapd 2.4.8 - syncrepl and glue (ITS#5430)
by Duncan.Gibb＠SiriusIT.co.uk 18 Mar '08

18 Mar '08

Thanks, Gavin. My GDB foo is not up to getting useful info out of your core dump, but I have placed one from one of our test rigs at http://pastebin.siriusit.co.uk/slapd-2.4.8-nullpointer-bin-and-core.tar.gz Note that this expects to be looked at with 64-bit GDB 6.7.1 from Debian Lenny/Sid (GDB 6.4.90 from Etch doesn't work). Re your previous comments: The engineer who put together the test case did look for bug reporting guidelines, but their location in the FAQ is, IMHO, non-obvious; perhaps one day I'll fix that. I reviewed and edited the report before sending it and I think it meets any reasonable person's definition of at least attempting to be helpful whilst not unduly verbose. Posting to the wrong list was my fault and I apologise. The moderator was correct to reject the original message. We know all kinds of weird stuff happens if you restart the crashed server(s) - that's how we come to be looking at this in the first place. We have, however, tried very hard to come up with a simple, confined case of this particular null pointer dereference, which is the first thing that goes wrong if you start from a clean sheet with empty databases each time and follow the recipe. In the four environments we can easily construct (RHEL vs Debian, i386 vs amd64) the segfault of server1 happens immediately you submit any change via LDAP to server2. Can anyone else reproduce this, or do we need to work on more examples? Cheers Duncan

1 0

(ITS#5436) htons() expects an unsigned short
by rein＠basefarm.no 18 Mar '08

18 Mar '08

Full_Name: Rein Tollevik Version: 2.4.8 OS: Solaris URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.215.2.34) libraries/libldap/os-ip.c casts the argument to htons() to a short, it should be an explicit unsigned short. This causes the use of high port numbers to fail on systems where shorts are signed by default. Rein Tollevik Basefarm AS

1 0

(ITS#5435) Seg. fault due to dereference of NULL ConfigReply * arguments.
by rein＠basefarm.no 18 Mar '08

18 Mar '08

Full_Name: Rein Tollevik Version: 2.4.8 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.215.2.34) Functions in servers/slapd/back-bdb/init.c dereference their ConfigReply * arguments without testing that they are non-NULL first, which causes seg. faults. I have only seen this happen when I tried to slapcat a database that hadn't had its directory created or some other stupidity, but anyhow... I don't include my patch to this, as it is probably easier to fix it again in your source than to apply a patch. Rein Tollevik Basefarm AS

1 0

(ITS#5434) syncprov in a glue'ed environment must search through the glue overlay
by rein＠basefarm.no 18 Mar '08

18 Mar '08

Full_Name: Rein Tollevik Version: 2.4.8 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.215.2.34) When the syncprov overlay is stacked on top of other overlays (notably the glue overlay) it must search through the next overlay, not the original backend DB. Otherwise it will not find any entries from the subordinate databases and consumers will remove those entries when they enters the refresh phase of the syncrepl protocol. The attached patch fixes this problem. Rein Tollevik Basefarm AS Index: OpenLDAP/servers/slapd/overlays/syncprov.c diff -u OpenLDAP/servers/slapd/overlays/syncprov.c:1.6 OpenLDAP/servers/slapd/overlays/syncprov.c:1.7 --- OpenLDAP/servers/slapd/overlays/syncprov.c:1.6 Tue Mar 18 17:22:41 2008 +++ OpenLDAP/servers/slapd/overlays/syncprov.c Tue Mar 18 18:02:32 2008 @@ -696,7 +696,11 @@ break; } - fop.o_bd->bd_info = on->on_info->oi_orig; + if ( on->on_next ) { + fop.o_bd->bd_info = (BackendInfo *) on->on_next; + } else { + fop.o_bd->bd_info = on->on_info->oi_orig; + } fop.o_bd->be_search( &fop, &frs ); fop.o_bd->bd_info = (BackendInfo *)on; @@ -1522,7 +1526,11 @@ fop.ors_filter = ⁡ cb.sc_response = playlog_cb; - fop.o_bd->bd_info = on->on_info->oi_orig; + if (on->on_next) { + fop.o_bd->bd_info = (BackendInfo *) on->on_next; + } else { + fop.o_bd->bd_info = on->on_info->oi_orig; + } for ( i=ndel; i<num; i++ ) { if ( uuids[i].bv_len == 0 ) continue;

1 0

(ITS#5433) syncprov in a glue'ed environment write contextCSN to random subordinate DB
by rein＠basefarm.no 18 Mar '08

18 Mar '08

Full_Name: Rein Tollevik Version: 2.4.8 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.215.2.34) syncprov_checkpoint() writes the contextCSN attribute to the suffix of the backend DB that is currently being written when it is triggered. When syncprov is stacked on top of a set of glue'ed databases this can be any of the subordinate backend DBs. The attached patch fixes this problem. Rein Tollevik Basefarm AS Index: OpenLDAP/servers/slapd/overlays/syncprov.c diff -u OpenLDAP/servers/slapd/overlays/syncprov.c:1.1.1.11 OpenLDAP/servers/slapd/overlays/syncprov.c:1.5 --- OpenLDAP/servers/slapd/overlays/syncprov.c:1.1.1.11 Thu Feb 21 14:55:36 2008 +++ OpenLDAP/servers/slapd/overlays/syncprov.c Fri Mar 14 16:23:48 2008 @@ -1301,6 +1301,7 @@ Operation opm; SlapReply rsm = { 0 }; slap_callback cb = {0}; + BackendInfo *bi = on->on_info->oi_origdb->bd_info; mod.sml_numvals = si->si_numcsns; mod.sml_values = si->si_ctxcsn; @@ -1316,8 +1317,9 @@ opm.o_callback = &cb; opm.orm_modlist = &mod; opm.orm_no_opattrs = 1; - opm.o_req_dn = op->o_bd->be_suffix[0]; - opm.o_req_ndn = op->o_bd->be_nsuffix[0]; + opm.o_bd = on->on_info->oi_origdb; + opm.o_req_dn = opm.o_bd->be_suffix[0]; + opm.o_req_ndn = opm.o_bd->be_nsuffix[0]; opm.o_bd->bd_info = on->on_info->oi_orig; opm.o_managedsait = SLAP_CONTROL_NONCRITICAL; opm.o_no_schema_check = 1; @@ -1326,6 +1328,7 @@ slap_mods_free( mod.sml_next, 1 ); } opm.orm_no_opattrs = 0; + opm.o_bd->bd_info = bi; } static void

1 0

(ITS#5432) syncrepl seg. faults if received a sync cookie with an empty csn= value
by rein＠basefarm.no 18 Mar '08

18 Mar '08

Full_Name: Rein Tollevik Version: 2.4.8 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.215.2.34) The attached patch fixes a seg. fault in compare_csns() in syncrepl.c if it receives a sync cookie with an empty "csn=" value. syncprov_playlog() in syncprov.c generates such invalid cookies when replaying a sessionlog where no entries has been deleted. The patch also fixes this bug. Rein Tollevik Basefarm AS Index: OpenLDAP/servers/slapd/syncrepl.c diff -u OpenLDAP/servers/slapd/syncrepl.c:1.6 OpenLDAP/servers/slapd/syncrepl.c:1.7 --- OpenLDAP/servers/slapd/syncrepl.c:1.6 Fri Mar 14 16:27:41 2008 +++ OpenLDAP/servers/slapd/syncrepl.c Tue Mar 18 17:15:30 2008 @@ -669,8 +669,8 @@ return -1; } - for (i=0; !BER_BVISNULL( &sc1->ctxcsn[i] ); i++) { - for (j=0; !BER_BVISNULL( &sc2->ctxcsn[j] ); j++) { + for (i=0; !BER_BVISEMPTY( &sc1->ctxcsn[i] ); i++) { + for (j=0; !BER_BVISEMPTY( &sc2->ctxcsn[j] ); j++) { if ( sc1->sids[i] != sc2->sids[j] ) continue; value_match( &match, slap_schema.si_ad_entryCSN, Index: OpenLDAP/servers/slapd/overlays/syncprov.c diff -u OpenLDAP/servers/slapd/overlays/syncprov.c:1.5 OpenLDAP/servers/slapd/overlays/syncprov.c:1.6 --- OpenLDAP/servers/slapd/overlays/syncprov.c:1.5 Fri Mar 14 16:23:48 2008 +++ OpenLDAP/servers/slapd/overlays/syncprov.c Tue Mar 18 17:22:41 2008 @@ -1413,8 +1413,7 @@ num * UUID_LEN, op->o_tmpmemctx ); uuids[0].bv_val = (char *)(uuids + num + 1); - delcsn[0].bv_len = 0; - delcsn[0].bv_val = cbuf; + BER_BVZERO(&delcsn[0]); BER_BVZERO(&delcsn[1]); /* Make a copy of the relevant UUIDs. Put the Deletes up front @@ -1453,6 +1452,7 @@ i++; AC_MEMCPY( cbuf, se->se_csn.bv_val, se->se_csn.bv_len ); delcsn[0].bv_len = se->se_csn.bv_len; + delcsn[0].bv_val = cbuf; delcsn[0].bv_val[delcsn[0].bv_len] = '\0'; } else { nmods++;

1 0

(ITS#5431) syncrepl on a subordinate DB writes contextCSN to suffix of parent DB
by rein＠basefarm.no 18 Mar '08

18 Mar '08

Full_Name: Rein Tollevik Version: 2.4.8 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.215.2.34) syncrepl_updateCookie() writes the contextCSN to the suffix of si->si_wbe (assigned to op->o_bd). In a subordinate database this is the parent DB, it should be written to the suffix of si->si_be instead as the attached patch fixes. I first considered writing it directly to si->si_be rather than through si->si_wbe, but I'm not familiar enough with the syncrepl/syncprov code to tell whether that would be correct or not. Rein Tollevik Basefarm AS Index: OpenLDAP/servers/slapd/syncrepl.c diff -u OpenLDAP/servers/slapd/syncrepl.c:1.5 OpenLDAP/servers/slapd/syncrepl.c:1.6 --- OpenLDAP/servers/slapd/syncrepl.c:1.5 Mon Mar 3 14:04:54 2008 +++ OpenLDAP/servers/slapd/syncrepl.c Fri Mar 14 16:27:41 2008 @@ -2683,8 +2683,8 @@ cb.sc_private = si; op->o_callback = &cb; - op->o_req_dn = op->o_bd->be_suffix[0]; - op->o_req_ndn = op->o_bd->be_nsuffix[0]; + op->o_req_dn = si->si_be->be_suffix[0]; + op->o_req_ndn = si->si_be->be_nsuffix[0]; /* update contextCSN */ op->o_msgid = SLAP_SYNC_UPDATE_MSGID;

1 0

(ITS#5430) Segfault in slapd 2.4.8 - syncrepl and glue
by duncan.gibb＠siriusit.co.uk 18 Mar '08

18 Mar '08

Full_Name: Duncan Gibb Version: 2.4.8 OS: Linux (Debian Etch and RHEL 4, x86 and x86_64) URL: Submission from: (NULL) (217.207.197.142) (Previously sent by email to openldap-devel, but lost or moderated out) In the course of debugging a fairly complex OpenLDAP deployment, we've found a reproducible segfault in 2.4.8. Below is a simplified testcase. Server1 is master for the main database and syncrepl consumer for a subordinate (starting at ou=dept2,... in our example). It's syncrepl provider for both parts of the tree. Server2 is master for the subordinate database, and consumer for the main database. Server3 is 100% consumer, taking both parts of the database from Server1 (in the real environment there is no routing between Server2 and Server3). Starting with empty databases, we populate Server1 with "main.ldif", and Server2 with "dept2.ldif". Then start 1 and 2 (only). With servers 1 and 2 running, we have full replication in both directions. When we start Server3 with an empty database, it replicates the whole tree from Server1. However: - subsequent changes in the main database (on Server1) replicate only to Server2 and not to Server3 - the next change in the subordinate database (on Server2) replicates to 1 and 3, but segfaults Server1 thus: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x41001950 (LWP 5512)] 0x000000000045c95e in test_filter (op=0x41000710, e=0xa29c08, f=0x0) at filterentry.c:69 69 if ( f->f_choice & SLAPD_FILTER_UNDEFINED ) {(gdb) bt #0 0x000000000045c95e in test_filter (op=0x41000710, e=0xa29c08, f=0x0) at filterentry.c:69 #1 0x00002b9fcd9dddf2 in syncprov_matchops (op=0x41000710, opc=0xdb4c40, saveit=1) at syncprov.c:1212 #2 0x00002b9fcd9dfd4c in syncprov_op_mod (op=0x41000710, rs=0x41000290) at syncprov.c:1807 #3 0x00000000004b5dce in overlay_op_walk (op=0x41000710, rs=0x41000290, which=op_modify, oi=0x7f1ab0, on=0x7f1f40) at backover.c:643 #4 0x00000000004b6060 in over_op_func (op=0x41000710, rs=0x41000290, which=op_modify) at backover.c:705 #5 0x00000000004b613e in over_op_modify (op=0x41000710, rs=0x41000290) at backover.c:739 #6 0x00000000004abfac in syncrepl_updateCookie (si=0x7f1020, op=0x41000710, pdn=0x7f01c0, syncCookie=0x41000500) at syncrepl.c:2693 #7 0x00000000004a5987 in do_syncrep2 (op=0x41000710, si=0x7f1020) at syncrepl.c:848 #8 0x00000000004a6e66 in do_syncrepl (ctx=0x41000e00, arg=0x7f0c20) at syncrepl.c:1226 #9 0x00000000004359fd in connection_read_thread (ctx=0x41000e00, argv=0x10) at connection.c:1213 #10 0x00002b9fc9f34ae5 in ldap_int_thread_pool_wrapper (xpool=0x7ab720) at tpool.c:625 #11 0x00002b9fca8f03f7 in start_thread () from /lib/libpthread.so.0 #12 0x00002b9fcba3997d in clone () from /lib/libc.so.6 #13 0x0000000000000000 in ?? () (gdb) We've reproduced this on RHEL4/i386 both under Xen and under QEMU, and also on Debian Etch/amd64 using Xen virtuals and using three slapds listening on different ports on the same machine. Below are config files for the last of these, which should be the simplest to rig up (127.0.N.1 represents ServerN). Ideas and comments welcome. Cheers Duncan -8<-- slapd.conf.127.0.1.1 -8<---8<---8<---8<---8<---8<---8<-- # slapd.conf for 127.0.1.1 include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema loglevel any sizelimit unlimited timelimit unlimited pidfile /home/ldaptest/rel/var/run/slapd_127.0.1.1.pid argsfile /home/ldaptest/rel/var/run/slapd_127.0.1.1.args modulepath /home/ldaptest/rel/libexec/openldap moduleload back_bdb moduleload syncprov.la database bdb suffix "ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk" rootdn "cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" overlay syncprov subordinate directory /home/ldaptest/rel/var/openldap-data/127.0.1.1/dept2 index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index entryCSN eq,pres syncrepl rid=103 provider=ldap://127.0.2.1:38902 type=refreshAndPersist retry="10 +" searchbase="ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk" bindmethod="simple" binddn="cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" credentials="secret" updateref ldap://127.0.2.1:38902 database bdb suffix "dc=ldaplab,dc=siriusit,dc=co,dc=uk" rootdn "cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" overlay glue overlay syncprov directory /home/ldaptest/rel/var/openldap-data/127.0.1.1/main index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index entryCSN eq,pres -8<-- slapd.conf.127.0.2.1 -8<---8<---8<---8<---8<---8<---8<-- # slapd.conf for 127.0.2.1 include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema loglevel any sizelimit unlimited timelimit unlimited pidfile /home/ldaptest/rel/var/run/slapd_127.0.2.1.pid argsfile /home/ldaptest/rel/var/run/slapd_127.0.2.1.args modulepath /home/ldaptest/rel/libexec/openldap moduleload back_bdb moduleload syncprov.la database bdb suffix "ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk" rootdn "cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" overlay syncprov subordinate directory /home/ldaptest/rel/var/openldap-data/127.0.2.1/dept2 index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index entryCSN eq,pres database bdb suffix "dc=ldaplab,dc=siriusit,dc=co,dc=uk" rootdn "cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" overlay glue overlay syncprov directory /home/ldaptest/rel/var/openldap-data/127.0.2.1/main index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index entryCSN eq,pres syncrepl rid=100 provider=ldap://127.0.1.1:38901 type=refreshAndPersist retry="10 +" searchbase="dc=ldaplab,dc=siriusit,dc=co,dc=uk" bindmethod="simple" binddn="cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" credentials="secret" updateref ldap://127.0.1.1:38901 -8<-- slapd.conf.127.0.3.1 -8<---8<---8<---8<---8<---8<---8<-- # slapd.conf for 127.0.3.1 include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema loglevel any sizelimit unlimited timelimit unlimited pidfile /home/ldaptest/rel/var/run/slapd_127.0.3.1.pid argsfile /home/ldaptest/rel/var/run/slapd_127.0.3.1.args modulepath /home/ldaptest/rel/libexec/openldap moduleload back_bdb moduleload syncprov.la database bdb suffix "ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk" rootdn "cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" subordinate directory /home/ldaptest/rel/var/openldap-data/127.0.3.1/dept2 index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index entryCSN eq,pres syncrepl rid=104 provider=ldap://127.0.1.1:38901 type=refreshAndPersist retry="10 +" searchbase="ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk" bindmethod="simple" binddn="cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" credentials="secret" updateref ldap://127.0.2.1:38902 database bdb suffix "dc=ldaplab,dc=siriusit,dc=co,dc=uk" rootdn "cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" overlay glue directory /home/ldaptest/rel/var/openldap-data/127.0.3.1/main index objectClass eq,pres index ou,cn,mail,surname,givenname eq,pres,sub index entryCSN eq,pres syncrepl rid=105 provider=ldap://127.0.1.1:38901 type=refreshAndPersist retry="10 +" searchbase="dc=ldaplab,dc=siriusit,dc=co,dc=uk" bindmethod="simple" binddn="cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk" credentials="secret" updateref ldap://127.0.1.1:38901 -8<-- main.ldif -8<---8<---8<---8<---8<---8<---8<-- # LDIF for main tree dn: dc=ldaplab,dc=siriusit,dc=co,dc=uk dc: ldaplab o: ldaplab objectClass: dcObject objectClass: organization dn: cn=admin,dc=ldaplab,dc=siriusit,dc=co,dc=uk cn: admin objectClass: organizationalRole objectClass: simpleSecurityObject userPassword: {SSHA}HEadVOU0EenIL0NX1bMblWYLAHrVW8Jy dn: ou=dept1,dc=ldaplab,dc=siriusit,dc=co,dc=uk ou: dept1 objectClass: organizationalUnit dn: cn=person11,ou=dept1,dc=ldaplab,dc=siriusit,dc=co,dc=uk sn: person11 cn: person11 objectClass: person dn: cn=person12,ou=dept1,dc=ldaplab,dc=siriusit,dc=co,dc=uk sn: person12 cn: person12 objectClass: person -8<-- dept2.ldif -8<---8<---8<---8<---8<---8<---8<-- # LDIF for "dept2" delegated subordinate tree dn: ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk ou: dept2 objectClass: organizationalUnit dn: cn=person21,ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk sn: person21 cn: person21 objectClass: person dn: cn=person22,ou=dept2,dc=ldaplab,dc=siriusit,dc=co,dc=uk sn: person22 cn: person22 objectClass: person

1 0

(ITS#5429) component Matching
by ron＠zytrax.com 17 Mar '08

17 Mar '08

Full_Name: Ron Aitchison Version: 2.4.8 OS: FreeBSD (5.4/6.2) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (209.148.181.131) Component matching (RFC 3687) is defined to be released with 2.4.8. However it is conditionally compiled from the variable LDAP_COMP_MATCH which is only set (in servers/slapd/slap.h) if LDAP_DEVEL is set. LDAP_DEVEL is not set by default in a normal build. The net result is that component matching is not included in an OpenLDAP build.

1 0

Re: (ITS#5419) test048 fails...
by hyc＠symas.com 17 Mar '08

17 Mar '08

luca(a)OpenLDAP.org wrote: > h.b.furuseth(a)usit.uio.no wrote: >> hyc(a)symas.com writes: >> >>> op->o_bd is essentially supposed to be a constant once it is set on an >>> operation. Operations within a backend are generally not allowed to >>> access other backends so it should not be overriding the backend that >>> was given to it. >>> >> "grep -n -e '.->o_bd *=[^=]' *.c */*.c" finds 10 from 'orig'something. >> Some of the other 202 matches also restore an old o_bd. >> > Seems fixed. can we close it? Yes, fine, as long as no one is seeing any unexpected side-effects from the fixes. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5409) paged results invalid sizelimit error
by pshao＠zimbra.com 17 Mar '08

17 Mar '08

when connection pooling is off: if page size = 1000 test 1 failed at the second iteration test 2 failed at the third iteration if page size = 999 test 1 always works test 2 failed at the second iteration I have not tried other tests where expected result is other number. phoebe ----- "Howard Chu" <hyc(a)symas.com> wrote: > pshao(a)zimbra.com wrote: > > ------=_Part_3887_1760018525.1205731478084 Content-Type: > text/plain; > > charset=utf-8 Content-Transfer-Encoding: 7bit > > > > > > I did more testing and found there is more to the bug, page size is > not the > > only factor. The number of results in the next search also > matters. > > However, in all cases, if JNDI connection pooling is disabled, then > it > > always works. > > > > Test description: > > > > create 10000 account: user-1 to user-10000 search size limit = 5000 > > > > test 1: step 1. search with filter (uid=*user*): expect > > SizeLimitExceededException step 2. search with filter > (uid=*user-9999*): > > expect 1 entry > > > > test 2: step 1. search with filter (uid=*user*): expect > > SizeLimitExceededException step 2. search with filter > (uid=*user-10*): > > expect 112 entries > > Please be more descriptive, re: intermittent failure - how many > iterations of > a test a required (on average) before the problem shows up? > > -- > -- Howard Chu > Chief Architect, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5419) test048 fails...
by luca＠OpenLDAP.org 17 Mar '08

17 Mar '08

h.b.furuseth(a)usit.uio.no wrote: > hyc(a)symas.com writes: > >> op->o_bd is essentially supposed to be a constant once it is set on an >> operation. Operations within a backend are generally not allowed to >> access other backends so it should not be overriding the backend that >> was given to it. >> > > "grep -n -e '.->o_bd *=[^=]' *.c */*.c" finds 10 from 'orig'something. > Some of the other 202 matches also restore an old o_bd. > Seems fixed. can we close it?

1 0

Re: (ITS#5409) paged results invalid sizelimit error
by hyc＠symas.com 17 Mar '08

17 Mar '08

pshao(a)zimbra.com wrote: > ------=_Part_3887_1760018525.1205731478084 Content-Type: text/plain; > charset=utf-8 Content-Transfer-Encoding: 7bit > > > I did more testing and found there is more to the bug, page size is not the > only factor. The number of results in the next search also matters. > However, in all cases, if JNDI connection pooling is disabled, then it > always works. > > Test description: > > create 10000 account: user-1 to user-10000 search size limit = 5000 > > test 1: step 1. search with filter (uid=*user*): expect > SizeLimitExceededException step 2. search with filter (uid=*user-9999*): > expect 1 entry > > test 2: step 1. search with filter (uid=*user*): expect > SizeLimitExceededException step 2. search with filter (uid=*user-10*): > expect 112 entries Please be more descriptive, re: intermittent failure - how many iterations of a test a required (on average) before the problem shows up? -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5409) paged results invalid sizelimit error
by pshao＠zimbra.com 17 Mar '08

17 Mar '08

oops, looks my attachment in Followup 3 did not go thru. Let me know if you need the attachment.

1 0

Re: (ITS#5409) paged results invalid sizelimit error
by pshao＠zimbra.com 17 Mar '08

17 Mar '08

------=_Part_3887_1760018525.1205731478084 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit I did more testing and found there is more to the bug, page size is not the only factor. The number of results in the next search also matters. However, in all cases, if JNDI connection pooling is disabled, then it always works. Test description: create 10000 account: user-1 to user-10000 search size limit = 5000 test 1: step 1. search with filter (uid=*user*): expect SizeLimitExceededException step 2. search with filter (uid=*user-9999*): expect 1 entry test 2: step 1. search with filter (uid=*user*): expect SizeLimitExceededException step 2. search with filter (uid=*user-10*): expect 112 entries With Connection Pooling ======================= (A) page size = factor of 5000, e.g. 1000, 2500 test 1: fails intermittently, step 2 got SizeLimitExceededException while 1 entry is expected test 2: fails intermittently, step 2 got SizeLimitExceededException while 112 entry is expected (B) page size = not factor of 5000, e.g. 999, 2000 test 1: always works test 2: fails intermittently, step 2 got SizeLimitExceededException while 112 entry is expected Without Connection Pooling ========================== test 1, test 2 always work, regardless of page size. Apparently the page size and result size of test 1 in (B) somehow mask the real bug, which has to do with connection reuse. Updated test java program attached. Thanks, phoebe ------=_Part_3887_1760018525.1205731478084 Content-Type: application/octet_stream; name=TestPagedControl.java Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=TestPagedControl.java package com.zimbra.sandbox; import java.io.IOException; import java.util.ArrayList; import java.util.Hashtable; import java.util.List; import javax.naming.Context; import javax.naming.NameNotFoundException; import javax.naming.NamingEnumeration; import javax.naming.NamingException; import javax.naming.SizeLimitExceededException; import javax.naming.directory.Attribute; import javax.naming.directory.Attributes; import javax.naming.directory.DirContext; import javax.naming.directory.InvalidSearchFilterException; import javax.naming.directory.SearchControls; import javax.naming.directory.SearchResult; import javax.naming.ldap.Control; import javax.naming.ldap.InitialLdapContext; import javax.naming.ldap.LdapContext; import javax.naming.ldap.PagedResultsControl; import javax.naming.ldap.PagedResultsResponseControl; /* README: Intermittently if the max entries requested is exact multiple of page size, after a paged search hits SizeLimitExceededException, the next search also always throws a SizeLimitExceededException, even when it should not. e.g. if max requested is 5000, then page size of: 1000 => bad 2500 => bad 999 => good 2000 => good - create 10100 user entries in OpenLDAP: with uid user-0 to user-10100 - Perform the following steps in a loop 1. search (uid=*user*) with limit 5000 and paged control, expecting SizeLimitExceededException 2. search (uid=*user-10001*) with limit 5000 and paged control, expecting 1 entry returned - observed that if the limit is multiple of page size (e.g. 1000, 2500), the test failed at the second iteration when it performs 2 - it throws SizeLimitExceededException when it should find one entry. OpenLDAP returns err=4: conn=262 op=8 SRCH base="" scope=2 deref=3 filter="(uid=*user-10001*)" conn=262 op=8 SRCH attr=uid conn=262 op=8 SEARCH RESULT tag=101 err=4 nentries=0 text= otherwise (e.g. when page size is 999, or 123, or 2000) the test passes. Example test runs: --------------------------------------------------- >java TestPagedControl 1000 Testing page size 1000 iter 0 iter 1 javax.naming.SizeLimitExceededException: [LDAP: error code 4 - Sizelimit Exceeded]; remaining name '' at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3037) at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2931) at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2737) at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1808) at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1731) at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:368) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:338) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:321) at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:248) at TestPagedControl.testPagedControl(TestPagedControl.java:81) at TestPagedControl.doTest(TestPagedControl.java:127) at TestPagedControl.main(TestPagedControl.java:140) --------------------------------------------------- >java TestPagedControl 999 Testing page size 999 iter 0 iter 1 iter 2 iter 3 iter 4 iter 5 iter 6 iter 7 iter 8 iter 9 pass */ public class TestPagedControl { static byte[] getCookie(LdapContext lctxt) throws NamingException { Control[] controls = lctxt.getResponseControls(); if (controls != null) { for (int i = 0; i < controls.length; i++) { if (controls[i] instanceof PagedResultsResponseControl) { PagedResultsResponseControl prrc = (PagedResultsResponseControl)controls[i]; return prrc.getCookie(); } } } return null; } static List<String> testPagedControl(String query, int maxResults, int pageSize, boolean useConnPool) throws Exception { List<String> results = new ArrayList<String>(); // int total = 0; DirContext ctxt = null; try { Hashtable<String, String> env = new Hashtable<String, String>(); env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory"); env.put(Context.PROVIDER_URL, "ldap://localhost:389"); env.put(Context.SECURITY_AUTHENTICATION, "simple"); env.put(Context.SECURITY_PRINCIPAL, "uid=zimbra,cn=admins,cn=zimbra"); env.put(Context.SECURITY_CREDENTIALS, "zimbra"); env.put(Context.REFERRAL, "follow"); env.put("com.sun.jndi.ldap.connect.timeout", "30000"); env.put("com.sun.jndi.ldap.read.timeout", "30000"); if (useConnPool) env.put("com.sun.jndi.ldap.connect.pool", "true"); else env.put("com.sun.jndi.ldap.connect.pool", "false"); ctxt = new InitialLdapContext(env, null); String base = ""; String[] returnAttrs = {"uid"}; SearchControls searchControls = new SearchControls(SearchControls.SUBTREE_SCOPE, maxResults, 0, returnAttrs, false, false); byte[] cookie = null; LdapContext lctxt = (LdapContext)ctxt; NamingEnumeration ne = null; try { do { lctxt.setRequestControls(new Control[]{new PagedResultsControl(pageSize, cookie, Control.CRITICAL)}); ne = ctxt.search(base, query, searchControls); while (ne != null && ne.hasMore()) { // total++; SearchResult sr = (SearchResult) ne.nextElement(); String dn = sr.getNameInNamespace(); results.add(dn); } cookie = getCookie(lctxt); } while (cookie != null); } finally { if (ne != null) ne.close(); } } catch (InvalidSearchFilterException e) { throw e; } catch (NameNotFoundException e) { throw e; } catch (SizeLimitExceededException e) { throw e; } catch (NamingException e) { throw e; } catch (IOException e) { throw e; } finally { ctxt.close(); } return results; } static void test1(int pageSize, boolean useConnPool) throws Exception { /* * first search: expecting SizeLimitExceededException */ boolean good = false; try { testPagedControl("(uid=*user*)", 5000, pageSize, useConnPool); } catch (SizeLimitExceededException e) { good = true; } if (!good) throw new Exception("expecting SizeLimitExceededException"); List<String> results; /* * second search: expecting one result */ results = testPagedControl("(uid=*user-9999*)", 5000, pageSize, useConnPool); // for (String s : results) System.out.println(s); if (results.size() != 1) throw new Exception("expecting 1 result, got " + results.size()); } static void test2(int pageSize, boolean useConnPool) throws Exception { /* * first search: expecting SizeLimitExceededException */ boolean good = false; try { testPagedControl("(uid=*user*)", 5000, pageSize, useConnPool); } catch (SizeLimitExceededException e) { good = true; } if (!good) throw new Exception("expecting SizeLimitExceededException"); List<String> results; /* * second search: expecting 112 results */ results = testPagedControl("(uid=*user-10*)", 5000, pageSize, useConnPool); // for (String s : results) System.out.println(s); if (results.size() != 112) throw new Exception("expecting 112 result, got " + results.size()); } private static void usage() { System.out.println("usage: TestPagedControl <page size> <whcih test 1|2> <use conn pool: true|false>"); } public static void main(String[] args) { if (args.length != 3) { usage(); System.exit(1); } int pageSize = Integer.parseInt(args[0]); int test = Integer.parseInt(args[1]); boolean useConnPool = Boolean.parseBoolean(args[2]); System.out.println("Testing page size " + pageSize); try { for (int i=0; i<10; i++) { System.out.println(" iter " + i); if (test == 1) test1(pageSize, useConnPool); else if (test == 2) test2(pageSize, useConnPool); else { usage(); System.exit(1); } } System.out.println("pass"); } catch (Exception e) { e.printStackTrace(); } } } ------=_Part_3887_1760018525.1205731478084--

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by h.b.furuseth＠usit.uio.no 16 Mar '08

16 Mar '08

ando(a)sys-net.it writes: >> slapadd: tools.c:494: bdb_tool_entry_put: Assertion `text->bv_val[0] == >> '\0'' failed. > > I only tested with ldapadd, but at a glance I don't see what could be the > difference. The protocol Add operation doesn't run into that assert. Have only looked briefly at the code, but I guess entry_schema_check()'s params textbuf[0] (and textlen?) must be zeroed if they were filled with a message but entry_schema_check() returns success anyway. Don't know where (in which function) that should be done though. -- Hallvard

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by ando＠sys-net.it 16 Mar '08

16 Mar '08

> slapadd: tools.c:494: bdb_tool_entry_put: Assertion `text->bv_val[0] == > '\0'' failed. I only tested with ldapadd, but at a glance I don't see what could be the difference. Unfortunately I won't be able to work at it/connect for a while (a week or so); I ran the whole test suite and this fix shouldn't break any existing test, in the meanwhile. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by h.b.furuseth＠usit.uio.no 16 Mar '08

16 Mar '08

Pierangelo Masarati writes: > Patched in HEAD; please test. p. test.conf: include servers/slapd/schema/core.schema database bdb suffix o=foo directory /tmp/db test.ldif: dn: o=foo objectClass: organization $ servers/slapd/slapadd -f test.conf < test.ldif ... slapadd: tools.c:494: bdb_tool_entry_put: Assertion `text->bv_val[0] == '\0'' failed. -- Hallvard

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by ando＠sys-net.it 15 Mar '08

15 Mar '08

h.b.furuseth(a)usit.uio.no wrote: > slapd and slapadd should generate them if they are missing. Patched in HEAD; please test. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5412) Add operation should not require naming attrs
by ando＠sys-net.it 15 Mar '08

15 Mar '08

h.b.furuseth(a)usit.uio.no wrote: > A somewhat belated ITS: ldapadd of > dn: o=foo > objectClass: organization > complains that "naming attribute 'o' is not present in entry". > RFC 2251 required this, but RFC 4511 changed it to the opposite: > The RDN attrs need not be present in AddRequest.attributes. > > slapd and slapadd should generate them if they are missing. I'm patching HEAD code in this sense. For this purpose, I decided to hijack the add_soc parameter of entry_schema_check() that controlled adding the structural objectClass. In fact, in the current code so far I couldn't find any call to that function where the structural objectClass had to be added and the naming attributes hadn't. If this distinction becomes necessary the parameter can be used as a mask instead of a boolean. > I don't know if the change belongs in RE23. re23 claims to comply with rfc 2251-2256, and knows nothing of rfc 4510-4519, so IMHO it shouldn't be affected. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

(ITS#5428) Enhancement: allow slapo-refint(5) to implement global referential integrity
by ando＠sys-net.it 15 Mar '08

15 Mar '08

Full_Name: Pierangelo Masarati Version: HEAD OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.63.140.131) Submitted by: ando See <http://www.openldap.org/lists/openldap-technical/200803/msg00068.html> for discussion. p.

1 0

(ITS#5427) slapo-refint(5) erroneously uses smr_str field for dnSubtreeMatch
by ando＠sys-net.it 15 Mar '08

15 Mar '08

Full_Name: Pierangelo Masarati Version: HEAD/re24 OS: irrelevant URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (82.63.140.131) Submitted by: ando The right field is smr_bvoid (or smr_names[0]). p.

1 0

(ITS#5426) syncrepl and syncprov clash on contextCSN updates
by hyc＠OpenLDAP.org 14 Mar '08

14 Mar '08

Full_Name: Howard Chu Version: 2.4/HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (76.91.220.157) Submitted by: hyc If syncprov-checkpoint is configured on a database with a syncrepl consumer (either for cascaded syncrepl or for mirrormode), the syncrepl consumer may fail to update the contextCSN because syncprov updates it first. In 2.4 the syncrepl consumer tries to delete its old value and add the new value, instead of just doing a modify/Replace as it used to in 2.3. The 2.4 consumer was changed in this respect because multiple consumers are now allowed on a database, thus using multiple values in the contextCSN attribute. Replacing the entire attribute when only a single value needed to change seemed like the wrong thing to do. The syncprov overlay still just uses a modify/Replace, so it doesn't have this problem. Since both syncrepl and syncprov now currently track all of the contextCSNs in use, probably syncrepl should just be switched back to using modify/Replace, with the full list of known contextCSNs.

1 0

Re: Adding new Attribute in OpenLdap (ITS#5423)
by ando＠sys-net.it 14 Mar '08

14 Mar '08

vijayaragavanp(a)zenithsoft.com wrote: > i was't able to find the manual ,can you please send me the link Please send software usage requests to the OpenLDAP-software mailing list. The ITS is to track software bug. p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

(ITS#5425) slapdconfig.sdf and slapdconf2.sdf out of date
by ghenry＠OpenLDAP.org 14 Mar '08

14 Mar '08

Full_Name: Gavin Henry Version: HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (80.229.93.1) Submitted by: ghenry Note to self: These are missing the latest Syncrepl directives and others things. Need to sync them with slapd.conf(5)

1 0

RE: Adding new Attribute in OpenLdap (ITS#5423)
by vijayaragavanp＠zenithsoft.com 14 Mar '08

14 Mar '08

i was't able to find the manual ,can you please send me the link -----Original Message----- From: Pierangelo Masarati [mailto:openldap-its@OpenLDAP.org] Sent: Friday, March 14, 2008 2:46 PM To: vijayaragavanp(a)zenithsoft.com Subject: Re: Adding new Attribute in OpenLdap (ITS#5423) > how to add custom attributes in OpenLdap As clearly explained in the manual. p.

1 0

(ITS#5424) pwdGraceLoginLimit only mentioned, not documented in slapo-ppolicy
by ghenry＠OpenLDAP.org 14 Mar '08

14 Mar '08

Full_Name: Gavin Henry Version: HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (212.159.59.85) Submitted by: ghenry "(please refer to the pwdGraceLoginLimit password policy attribute)" Nothing else mentioned. Gavin.

1 0

(ITS#5423) Adding new Attribute in OpenLdap
by vijayaragavanp＠zenithsoft.com 14 Mar '08

14 Mar '08

Full_Name: vijayaragavanp Version: 2.0 OS: yes URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (124.30.128.168) how to add custom attributes in OpenLdap

1 0

(ITS#5422) strerror(errno) problems
by h.b.furuseth＠usit.uio.no 13 Mar '08

13 Mar '08

Full_Name: Hallvard B Furuseth Version: HEAD, RE23, RE24 OS: URL: Submission from: (NULL) (129.240.6.233) Submitted by: hallvard "grep -i strerror" shows some places <strerror/STRERROR>(errno) are passed to Debug(). However Debug() can evaluate its arguments twice: for -d output and syslog. Thus it can syslog the wrong message, since the -d stdio call can change errno. Also strerror() can return NULL, which can break with Debug(%s). STRERROR() fixes that, but not the duplicate call. And strerror too can change errno:-( So the STRERROR() variant #elif defined( HAVE_STRERROR ) #define STRERROR(e) ( strerror(e) ? strerror(e) : _AC_ERRNO_UNKNOWN ) in include/ac/errno.h isn't quite safe either. I suggest we put something like below in ac/string.h or ac/errno.h, then we can usually say if (whatever) { DEF_STRERROR(msg, errno); Debug(..., msg, ...); } ac/something.h: #define DEF_STRERROR(msg, err) \ DECL_STRERROR(msg); \ SET_STRERROR(msg, err) #ifdef HAVE_SYS_ERRLIST #define DECL_STRERROR(msg) \ unsigned int msg##_err_; \ const char *msg #define SET_STRERROR(msg, err) \ ((void)(msg = (msg##_err_ = (e)) < (unsigned) sys_nerr \ ? sys_errlist[msg##_err_] : _AC_ERRNO_UNKNOWN)) #elif <threaded> && defined( HAVE_NONPOSIX_STRERROR_R ) #define DECL_STRERROR(msg) \ char msg##_buf_[80]; \ const char *msg #define SET_STRERROR(msg, err) \ ((void) ((msg = strerror_r(err, msg##_buf_, 80)) || \ (msg = _AC_ERRNO_UNKNOWN))) #elif <threaded> && defined( HAVE_STRERROR_R ) #define DECL_STRERROR(msg) \ char msg##_buf_[80]; \ const char *msg #define SET_STRERROR(msg, err) \ ((void) (msg = strerror_r(err, msg##_buf_, 80) != -1 \ ? msg##_buf_ : _AC_ERRNO_UNKNOWN)) #elif defined( HAVE_STRERROR ) #define DECL_STRERROR(msg) \ const char *msg #define SET_STRERROR(msg, err) \ ((void) ((msg = strerror(err)) || (msg = _AC_ERRNO_UNKNOWN))) #else #define DECL_STRERROR(msg) const char *msg #define SET_STRERROR(msg, err) ((void) (msg = "Unknown error")) #endif Don't know what the #if <threaded> test should look like, see ITS#5421.

1 0

(ITS#5421) slapd vs LDAP_R_COMPILE
by h.b.furuseth＠usit.uio.no 13 Mar '08

13 Mar '08

Full_Name: Hallvard B Furuseth Version: HEAD, RE23, RE24 OS: URL: Submission from: (NULL) (129.240.6.233) Submitted by: hallvard Some slapd source files include libldap/ldap-int.h, which contains #ifdef LDAP_R_COMPILE in structs. slapd links libldap_r but does not define LDAP_R_COMPILE, so these files have access to bad struct definitions. A quick grep doesn't show any use cases though. Anyway: Either #ifdef LDAP_R_COMPILE in libldap(_r)should be something else, or we should define LDAP_R_COMPILE in slapd - maybe like this: --- slap.h 13 Mar 2008 13:37:07 +++ slap.h 13 Mar 2008 16:40:25 @@ -28,4 +28,10 @@ #define _SLAP_H_ +#if defined _LDAP_INT_H +# error include slap.h before ldap-int.h +#elif !defined NO_THREADS +# define LDAP_R_COMPILE 1 +#endif + #include "ldap_defaults.h"

1 0

Re: (ITS#5416) slapd core dump
by ando＠sys-net.it 13 Mar '08

13 Mar '08

rui.yan(a)gmail.com wrote: > Full_Name: yanr > Version: 2.4.8 > OS: Linux 2.6.22 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (128.221.197.21) > > > Verion: 2.4.8 > Desc: client runs some codes like ldap_modify_ext, sometimes slapd core dump. > The same codes works on 2.3.38. > > [root@test1 /]# uname -r > 2.6.22.9-2.smp.gcc3.4.x86_64 > > (gdb) bt > > #0 0x00002b018fa6da3d in *__GI_raise (sig=19040) > at ../nptl/sysdeps/unix/sysv/linux/raise.c:67 > 67 int res = INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); > (gdb) bt > #0 0x00002b018fa6da3d in *__GI_raise (sig=19040) > at ../nptl/sysdeps/unix/sysv/linux/raise.c:67 > #1 0x00002b018fa6ef1e in *__GI_abort () at ../sysdeps/generic/abort.c:88 > #2 0x00002b018fa67720 in *__GI___assert_fail (assertion=0x56ad43 "i > 0", > file=0x56acf8 "schema_init.c", line=366, > function=0x56ad30 "octetStringIndexer") at assert.c:83 > #3 0x00000000004664b3 in octetStringIndexer (use=163, flags=4, > syntax=0x77ea70, mr=0x77ed70, prefix=0x77ec48, values=0x2aaaaacad6b0, > keysp=0x42001598, ctx=0x83d970) at schema_init.c:366 > #4 0x000000000046ffde in objectSubClassIndexer (use=163, mask=4, > syntax=0x77ea70, mr=0x77ed70, prefix=0x77ec48, values=0x87b7e0, > keysp=0x42001598, ctx=0x83d970) at schema_prep.c:305 You should provide: - your slapd.conf - the LDIF of the modification that caused the issue - the contents of some variables from gdb: (gdb) frame 4 (gdb) print nocs (gdb) print values (gdb) print values[0] p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --------------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: pierangelo.masarati(a)sys-net.it ---------------------------------------

1 0

Re: (ITS#5420) test048-syncrepl-multiproxy: ldapsearch failed at P1 slave (32)!
by hyc＠symas.com 13 Mar '08

13 Mar '08

Dup of ITS#5419. michael(a)stroeder.com wrote: > Full_Name: Michael Ströder > Version: HEAD > OS: OpenSUSE Linux 10.3 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (84.163.68.253) > > > make test fails with current HEAD synced right now. > > michael@nb2:~/src/openldap/HEAD/openldap/tests> ./run -b bdb test048 > Cleaning up test run directory leftover from previous run. > Running ./scripts/test048-syncrepl-multiproxy... > running defines.sh > Starting master slapd on TCP/IP port 9011... > Using ldapsearch to check that master slapd is running... > Using ldapadd to create the context prefix entry in the master... > Starting P1 slave slapd on TCP/IP port 9012... > Using ldapsearch to check that P1 slave slapd is running... > Starting R1 slave slapd on TCP/IP port 9013... > Using ldapsearch to check that R1 slave slapd is running... > 1> Using ldapadd to populate the master directory... > Waiting 15 seconds for syncrepl to receive changes... > ldapsearch failed at P1 slave (32)! > > > -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#5420) test048-syncrepl-multiproxy: ldapsearch failed at P1 slave (32)!
by michael＠stroeder.com 13 Mar '08

13 Mar '08

Full_Name: Michael Ströder Version: HEAD OS: OpenSUSE Linux 10.3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.163.68.253) make test fails with current HEAD synced right now. michael@nb2:~/src/openldap/HEAD/openldap/tests> ./run -b bdb test048 Cleaning up test run directory leftover from previous run. Running ./scripts/test048-syncrepl-multiproxy... running defines.sh Starting master slapd on TCP/IP port 9011... Using ldapsearch to check that master slapd is running... Using ldapadd to create the context prefix entry in the master... Starting P1 slave slapd on TCP/IP port 9012... Using ldapsearch to check that P1 slave slapd is running... Starting R1 slave slapd on TCP/IP port 9013... Using ldapsearch to check that R1 slave slapd is running... 1 > Using ldapadd to populate the master directory... Waiting 15 seconds for syncrepl to receive changes... ldapsearch failed at P1 slave (32)!

1 0

Re: (ITS#5419) test048 fails...
by h.b.furuseth＠usit.uio.no 13 Mar '08

13 Mar '08

hyc(a)symas.com writes: > op->o_bd is essentially supposed to be a constant once it is set on an > operation. Operations within a backend are generally not allowed to > access other backends so it should not be overriding the backend that > was given to it. "grep -n -e '.->o_bd *=[^=]' *.c */*.c" finds 10 from 'orig'something. Some of the other 202 matches also restore an old o_bd. -- Hallvard

1 0

Re: (ITS#5419) test048 fails...
by hyc＠symas.com 13 Mar '08

13 Mar '08

luca(a)OpenLDAP.org wrote: > Full_Name: Luca Scamoni > Version: HEAD > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (82.63.140.131) > > > consistently > > Running ./scripts/test048-syncrepl-multiproxy... > running defines.sh > Starting master slapd on TCP/IP port 9011... > Using ldapsearch to check that master slapd is running... > Using ldapadd to create the context prefix entry in the master... > Starting P1 slave slapd on TCP/IP port 9012... > Using ldapsearch to check that P1 slave slapd is running... > Starting R1 slave slapd on TCP/IP port 9013... > Using ldapsearch to check that R1 slave slapd is running... > 1> Using ldapadd to populate the master directory... > Waiting 15 seconds for syncrepl to receive changes... > ldapsearch failed at P1 slave (32)! > > modifications to the master are not propagated to the slaves Yes, I see this as well. syncrepl is trying to use backend_attribute() to retrieve the underlying DB's contextCSN, but backend_attribute is discarding the current op->o_bd and using select_backend() to pick which backend to query. This is definitely the wrong behavior; op->o_bd is essentially supposed to be a constant once it is set on an operation. Operations within a backend are generally not allowed to access other backends so it should not be overriding the backend that was given to it. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#5391) hdb deadlock
by h.b.furuseth＠usit.uio.no 13 Mar '08

13 Mar '08

hyc(a)symas.com writes: >Hallvard B Furuseth wrote: >> How about making #bytes-based checkpoints signal or (pthread_kill?) the >> timed checkpoints thread, so that thread can handle all checkpoints? > > It's an interesting idea. But that means we have to count bytes ourselves, > rather than letting BerkeleyDB do it. Whoops, good point. > Personally I don't think byte counts are very useful here. They're made > somewhat superfluous because the data volume will tend to cause the > transaction log buffers to fill up and be flushed regardless. I don't think Berkeley DB writes a checkpoint record then, at least I don't see that in the docs. If not, #bytes do help keep recovery and checkpoint times down. Anyway, unless you are proposing to ignore the #bytes parameter: If this is a Berkeley DB bug we could hear what they say, and in the meantime put a mutex_trylock around checpoints. Unless db has some feature which defeats that, like if txn_checkpoint() merely schedules a checkpoint. #define CHECKPOINT_WORKAROUND /* hopefully an ITS#5391 workaround */ struct bdb_info { ... #ifdef CHECKPOINT_WORKAROUND # ldap_pvt_thread_mutex_t bi_checkpoint_mutex; #define CHECKPOINT_LOCK(op, bdb) op(&(bdb)->bi_checkpoint_mutex) #else #define CHECKPOINT_LOCK(op, bdb) 0 #endif ... }; #define TXN_CHECKPOINT(bdb, k, m, f) do { \ if ( CHECKPOINT_LOCK( ldap_pvt_thread_mutex_trylock, bdb ) == 0 ) { \ (bdb)->bi_dbenv->txn_checkpoint( (bdb)->bi_dbenv, k, m, f ); \ (void) CHECKPOINT_LOCK( ldap_pvt_thread_mutex_unlock, bdb ); \ } \ } while (0) -- Hallvard

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs