- openldap-bugs - openldap.org

Re: (ITS#6720) back-ldap core dump
by masarati＠aero.polimi.it 24 Nov '10

24 Nov '10

olivier.chirossel(a)sfr.com wrote: > I have to migrate from a openldap 2.3.x to openldap 2.4. During this migration > phase i want my openldap 2.3.x master replicate with slurpd to my new openldap > 2.4.x master mirror mode architecture with some rewriting rules for split my big > directory. > > > I configure a proxy ldap using back-ldap with slapo-rwm overlay, but > operational attributes related to entry creation and modification are > proxied, even when i put « lastmod off » in the conf, contrary of the man of > slapd-ldap > > > > So i put rw-map directives in the conf for strip operationnal attributes but > this kind of operation generate a core dump Your configuration contains a load of custom schema items; can you create a simple configuration, based on standard attributes, that reproduces the problem (I mean: the sigsegv), and describe which operation is causing it? Is it a client operation or replication? In your report you mention the intention to use "master mirror mode" (?), it's not clear what back-ldap has to do with it. p.

1 0

Re: (ITS#6603) EAGAIN, EWOULDBLOCK occasionally used without checking #ifdef
by h.b.furuseth＠usit.uio.no 24 Nov '10

24 Nov '10

> Here's a patch, please inspect. > > <ftp://ftp.openldap.org/incoming/pierangelo.masarati.2010-11-23.1.patch> Looks fine to me. -- Hallvard

1 0

Re: (ITS#6532) Support for common orderingMatching rules in extensible match filters
by h.b.furuseth＠usit.uio.no 24 Nov '10

24 Nov '10

Variant applied to HEAD: schema_init.c 1.473. Correct 'match' code is simpler than yours, now that you've pointed me in the right direction. All ordering rules except CSNOrderingMatch and UUIDOrderingMatch now support extensible match. Someone who knows slapd's use of CSNs and UUIDs can (tell me to) do the same there if it's OK to do so. I didn't want to meddle carelessly, also I don't know what UUIDOrderingMatch's SLAP_MR_MUTATION_NORMALIZER flag means and if it would interfere. -- Hallvard

1 0

Re: (ITS#6603) EAGAIN, EWOULDBLOCK occasionally used without checking #ifdef
by masarati＠aero.polimi.it 23 Nov '10

23 Nov '10

>> At least in libldap and liblber EAGAIN and EWOULDBLOCK are >> occasionally used without checking whether they are defined or not. >> Either the checks should be removed, counting on the fact that these >> error codes are always defined, or they should be used consistently. > > I'd say the libldap #ifdefs can safely be removed by now: "cvs annotate" > says os-ip.c has had unconditional EWOULDBLOCK since rev 1.34 (Aug > 1999), and request.c unconditional EAGAIN since rev 1.85 (Aug 2002). > No bug reports since then... > > We need to keep the '#if EWOULDBLOCK != EAGAIN's in switches, though. > > liblber always #ifdefs them, except in a test program. I suppose these > #ifdefs could stay in case someone uses the liblber source stand-alone. Here's a patch, please inspect. <ftp://ftp.openldap.org/incoming/pierangelo.masarati.2010-11-23.1.patch> p.

1 0

(ITS#6724) Feature request: support for PKCS11 pin input callback in public TLS api
by silvan＠kernelconcepts.de 23 Nov '10

23 Nov '10

Full_Name: Silvan Marco Fin Version: OS: Ubuntu Linux 10.04 URL: Submission from: (NULL) (217.146.132.69) Support for PKCS #11 devices in TLS via MozNSS in OpenLDAP currently lacks the possibility to "ask" for a PIN via callback. The methods supplied in tls_m.c are reading a PIN from a file or alternativly reading a PIN from STDIN. To add the needed flexibility to the MozNSS part, an additional callback argument to the init function or alternatively an additional set function for the callback would be needed. http://www.mozilla.org/projects/security/pki/nss/ref/ssl/pkfnc.html#1023128 provides the signature for the callback function. Since GnuTLS and OpenSSL provide PKCS #11 support by themselves in some way, I propose to add an additional set function to OpenLDAPs public TLS API to register a callback with the corresponding security library.

1 0

Re: (ITS#5421) slapd vs LDAP_R_COMPILE
by h.b.furuseth＠usit.uio.no 23 Nov '10

23 Nov '10

Taken to this thread: http://www.openldap.org/lists/openldap-devel/201011/msg00013.html librewrite & co vs. logging & ldap-int.h Since this ITS now depends on the ldap_debug hack in ldap_log.h, I'm removing some unnecessary '#include <ldap_log.h>'s for clarity. -- Hallvard

1 0

(ITS#6603) EAGAIN, EWOULDBLOCK occasionally used without checking #ifdef
by h.b.furuseth＠usit.uio.no 23 Nov '10

23 Nov '10

> At least in libldap and liblber EAGAIN and EWOULDBLOCK are > occasionally used without checking whether they are defined or not. > Either the checks should be removed, counting on the fact that these > error codes are always defined, or they should be used consistently. I'd say the libldap #ifdefs can safely be removed by now: "cvs annotate" says os-ip.c has had unconditional EWOULDBLOCK since rev 1.34 (Aug 1999), and request.c unconditional EAGAIN since rev 1.85 (Aug 2002). No bug reports since then... We need to keep the '#if EWOULDBLOCK != EAGAIN's in switches, though. liblber always #ifdefs them, except in a test program. I suppose these #ifdefs could stay in case someone uses the liblber source stand-alone. -- Hallvard

1 0

(ITS#6723) test programs break on EOF
by h.b.furuseth＠usit.uio.no 23 Nov '10

23 Nov '10

Full_Name: Hallvard B Furuseth Version: HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (193.157.206.208) Submitted by: hallvard tests/progs/slapd-tester.c and libraries/libldap/dntest.c use uninitialized buffer data on EOF from stdin. Fixing.

1 0

Re: (ITS#6721) uninitialized timeout in back-ldap
by h.b.furuseth＠usit.uio.no 23 Nov '10

23 Nov '10

Pierangelo Masarati writes: > The way I read the code (at the time of the ITS#6282) if tv_sec < 0 no > timeout was used. In any case, your fix is fine for me. Looking closer, the valgrind complaint was just for Debug() statements. So we can downgrade this ITS description to Fix back-ldap debug output of timeout -- Hallvard

1 0

Re: (ITS#6721) uninitialized timeout in back-ldap
by masarati＠aero.polimi.it 23 Nov '10

23 Nov '10

h.b.furuseth(a)usit.uio.no wrote: > Full_Name: Hallvard B Furuseth > Version: 4.2.19 - 4.2.23 > OS: > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (129.240.6.233) > Submitted by: hallvard > > > ITS#6282 introduced > struct timeval tv; > tv.tv_sec = -1; > in back-ldap: extended.c and config.c. This leaves tv_usec uninitialized > and the timeout possibly positive. Fixing to: > struct timeval tv = { -1, 0 }; > The way I read the code (at the time of the ITS#6282) if tv_sec < 0 no timeout was used. In any case, your fix is fine for me. p.

1 0

Re: (ITS#6722) Sortvals broken for some matching rules
by h.b.furuseth＠usit.uio.no 23 Nov '10

23 Nov '10

I wrote: > - The sortvals doc should say it need not sort by ORDERING. Fixing. Never mind that one, I must have looked cross-eyed on the manpage. -- Hallvard

1 0

(ITS#6722) Sortvals broken for some matching rules
by h.b.furuseth＠usit.uio.no 23 Nov '10

23 Nov '10

Full_Name: Hallvard B Furuseth Version: HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (193.157.206.208) Submitted by: hallvard slapd.conf:sortvals sorts attribute values by the EQUALITY rule's match result, which can be faster than ORDERING. Or octetStringMatch when no EQUALITY rule, I suppose. Haven't looked closely. However, (1) When an attribute has sortvals set, filtering by the ORDERING rule only tests first/last value. This breaks when the EQUALITY rule gives a different sort order. E.g. string equality sorts by length before contents of the normalized value, for speed. (2) Sortvals should only be settable for attributes whose EQUALITY rule: - imposes an absolute ordering, - always succeeds. At least, this seems cleanest to me. Currently there are no such restrictions: - At least booleanMatch() returns 0 or 1, never -1. Thus (TRUE, FALSE) sort randomly. And we cannot control third-party dynamically loaded EQUALITY rules, nor is it documented they should impose an ordering. - attr_valfind() proceeds with the binary search when value_match() fails so the 'match' to partition by is unknown. This makes little sense, attr_valfind() should fail when this happens. Thus setting sortvals could break previously working functionality, which is why I think it may be better to forbid sortvals for attributes where this can happen. Alternatively, we could allow it if the user sets an "I really mean it" mark for the attribute. (3) Also it makes me nervous to see search.c in back-ldap + back-meta react to SLAP_AT_SORTED_VAL. Are there other backends/overlays which need to do the same? Can some/most of that code be moved into slapd? Fixes for (1)-(2): - Sortvals needs some new flags: One to allow SLAP_AT_SORTED_VAL. SLAP_ATTR_SORT_IS_ORDERING to tell filterentry that ORDERING can use sortvals. And something to help set SLAP_ATTR_SORT_IS_ORDERING. The first would be set for the relevant equality matching rules. Not sure of the last: It could almost be set if (equality rule == ordering rule), but matching rules are passed as parameters so their implementation can act differently for EQUALITY and ORDERING:-( - filterentry.c:test_ava_filter() should ignore SLAP_ATTR_SORTED_VALS if (use == SLAP_MR_ORDERING) && (SLAP_ATTR_SORT_IS_ORDERING is not set). - booleanMatch() needs a fix. Don't know which others, except a marginal case: Rules that can set *matchp = (int) (value->bv_len - asserted->bv_len) need to be more careful when sizeof(ber_len_t) > sizeof(int). I assume we can safely truncate to ber_slen_t, i.e. that OpenLDAP would fail anyway for values with bv_len > max ber_slen_t. I'll fix this where I see it, but I leave the rest to someone else. - attr_valfind() should abort the binary search on failure. Possibly some of its callers must be updated to handle more result codes. In theory it shouldn't happen anyway with my other suggestions, but this looks hairy enough that it seems best to be safe. - The sortvals doc should say it need not sort by ORDERING. Fixing. ################################ Test case: #### sortvals.conf include servers/slapd/schema/core.schema attributetype ( 1.3.6.1.4.1.4203.666.777.1 NAME 'boolTest' EQUALITY booleanMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.7 ) # dnQualifier has string syntax and an ORDERING rule. sortvals boolTest dnQualifier database hdb directory sortvals.dir suffix st=top # shut up monitor warning database monitor #### sortvals.ldif dn: st=top boolTest: TRUE boolTest: FALSE dnQualifier: bbbb dnQualifier: dd dnQualifier: aaaa objectClass: locality objectClass: extensibleObject st: top dn: st=sub,st=top boolTest: FALSE boolTest: TRUE dnQualifier: aa dnQualifier: dddd objectClass: locality objectClass: extensibleObject st: sub #### sortvals.sh #!/bin/sh -e mkdir sortvals.dir touch sortvals.dir/DB_CONFIG servers/slapd/slapadd -f sortvals.conf -l sortvals.ldif (servers/slapd/slapcat -f sortvals.conf && echo === && servers/slapd/slapcat -f sortvals.conf -a '(dnQualifier<=c)' ) | egrep '^($|[bd=])' #### slapcat output # boolTest sorted differently in the two entries, # 2nd slapcat skips entry #1 since dd <= c even though it has aaaa <= c. dn: st=top boolTest: TRUE boolTest: FALSE dnQualifier: dd dnQualifier: aaaa dnQualifier: bbbb dn: st=sub,st=top boolTest: FALSE boolTest: TRUE dnQualifier: aa dnQualifier: dddd === dn: st=sub,st=top boolTest: FALSE boolTest: TRUE dnQualifier: aa dnQualifier: dddd

1 0

(ITS#6721) uninitialized timeout in back-ldap
by h.b.furuseth＠usit.uio.no 23 Nov '10

23 Nov '10

Full_Name: Hallvard B Furuseth Version: 4.2.19 - 4.2.23 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.240.6.233) Submitted by: hallvard ITS#6282 introduced struct timeval tv; tv.tv_sec = -1; in back-ldap: extended.c and config.c. This leaves tv_usec uninitialized and the timeout possibly positive. Fixing to: struct timeval tv = { -1, 0 };

1 0

Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by rein＠OpenLDAP.org 22 Nov '10

22 Nov '10

Just a note that I have switched my reply to -devel. Rein

1 0

(ITS#6720) back-ldap core dump
by olivier.chirossel＠sfr.com 22 Nov '10

22 Nov '10

Full_Name: olivier chirossel Version: 2.4.23 OS: linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (62.39.9.251) Hi I have to migrate from a openldap 2.3.x to openldap 2.4. During this migration phase i want my openldap 2.3.x master replicate with slurpd to my new openldap 2.4.x master mirror mode architecture with some rewriting rules for split my big directory. I configure a proxy ldap using back-ldap with slapo-rwm overlay, but operational attributes related to entry creation and modification are proxied, even when i put « lastmod off » in the conf, contrary of the man of slapd-ldap So i put rw-map directives in the conf for strip operationnal attributes but this kind of operation generate a core dump dn: neufDhcpIP=10.145.250.119,suffix=9,o=auth,dc=neuf,dc=fr changetype: modify delete: neufDhcpOption neufDhcpOption: 1;255.255.255.0 - replace: entryCSN entryCSN: 20100511215647Z#000024#00#000000 - replace: modifiersName modifiersName: cn=manager,dc=neuf,dc=fr - replace: modifyTimestamp modifyTimestamp: 20100511215647Z dn: neufDhcpIP=10.145.250.119,suffix=9,o=auth,dc=neuf,dc=fr changetype: modify add: neufDhcpOption neufDhcpOption: 1;255.255.255.128 - replace: entryCSN entryCSN: 20100511215647Z#000024#00#000000 - replace: modifiersName modifiersName: cn=manager,dc=neuf,dc=fr - replace: modifyTimestamp modifyTimestamp: 20100511215647Z Core was generated by `openldap-2.4.23/servers/slapd/slapd -F /etc/openldap/slapd-ldap.d -h ldap://10.'. Program terminated with signal 6, Aborted. [New process 7995] [New process 7994] [New process 7990] [New process 7989] #0 0x00007f01373f9ed5 in raise () from /lib/libc.so.6 (gdb) bt #0 0x00007f01373f9ed5 in raise () from /lib/libc.so.6 #1 0x00007f01373fb3f3 in abort () from /lib/libc.so.6 #2 0x00007f0137436408 in __libc_message () from /lib/libc.so.6 #3 0x00007f013743b9a8 in malloc_printerr () from /lib/libc.so.6 #4 0x00007f013743dab6 in free () from /lib/libc.so.6 #5 0x000000000055b8e2 in ber_bvarray_free_x (a=0x26de840, ctx=0x0) at memory.c:737 #6 0x000000000046e3a1 in slap_mod_free (mod=0x26d07f0, freeit=0) at mods.c:461 #7 0x000000000046e42d in slap_mods_free (ml=0x26d07f0, freevals=1) at mods.c:481 #8 0x0000000000437960 in do_modify (op=0x25d02e0, rs=0x41d24cb0) at modify.c:187 #9 0x000000000041f6ef in connection_operation (ctx=0x41d24e00, arg_v=<value optimized out>) at connection.c:1109 #10 0x00000000004203dd in connection_read_thread (ctx=0x41d24e00, argv=<value optimized out>) at connection.c:1245 #11 0x0000000000534080 in ldap_int_thread_pool_wrapper (xpool=<value optimized out>) at tpool.c:685 #12 0x00007f0137f21fc7 in start_thread () from /lib/libpthread.so.0 #13 0x00007f013749764d in clone () from /lib/libc.so.6 #14 0x0000000000000000 in ?? () (gdb) My conf : database ldap suffix "dc=neuf,dc=fr" rootdn "cn=manager,dc=neuf,dc=fr" rootpw secret updatedn "cn=manager,dc=neuf,dc=fr" uri "ldap://127.0.0.1/" idassert-bind bindmethod=simple binddn="cn=manager,dc=neuf,dc=fr" credentials=secret overlay rwm rwm-rewriteEngine on rwm-rewriteContext default rwm-rewriteRule "neufDhcpIP=([0-9\\.]+)([0-9]{1}),ou=People,o=auth,dc=neuf,dc=fr$" "neufDhcpIP=$1$2,suffix=$2,o=auth,dc=neuf,dc=fr" rwm-map attribute neufDhcpIP neufDhcpIP rwm-map attribute neufDhcp82 neufDhcp82 rwm-map attribute neufDhcpOption neufDhcpOption rwm-map attribute neufDhcp60 neufDhcp60 rwm-map attribute neufDhcpRelay neufDhcpRelay rwm-map attribute neufDhcpComment neufDhcpComment rwm-map attribute neufDhcpUnique neufDhcpUnique rwm-map attribute neufDhcpProvider neufDhcpProvider rwm-map attribute neufDhcpFQDN neufDhcpFQDN rwm-map attribute neufDhcp82b neufDhcp82b rwm-map attribute neufDhcpMac neufDhcpMac rwm-map attribute neufDhcpSiaddr neufDhcpSiaddr rwm-map attribute neufDhcpBootfilename neufDhcpBootfilename rwm-map attribute neufBTID neufBTID rwm-map attribute neufBTOption neufBTOption rwm-map attribute neufTWINOption neufTWINOption rwm-map attribute neufE28Option neufE28Option rwm-map attribute circuitid circuitid rwm-map attribute clientid clientid rwm-map attribute device device rwm-map attribute clientoffer clientoffer rwm-map attribute ispid ispid rwm-map attribute suspended suspended rwm-map attribute neufDhcpIPb neufDhcpIPb rwm-map attribute neufDhcpOptionb neufDhcpOptionb rwm-map attribute idworkflow idworkflow rwm-map attribute neufDhcpRelayIP neufDhcpRelayIP rwm-map attribute interceptID interceptID rwm-map attribute interceptEndpoint interceptEndpoint rwm-map attribute neufDhcpRadiusOption neufDhcpRadiusOption rwm-map attribute suffix suffix rwm-map attribute dc dc rwm-map attribute o o rwm-map attribute ou ou rwm-map attribute cn cn rwm-map objectclass neufDhcpOption1 neufDhcpOption1 rwm-map objectclass neufDhcpOption5 neufDhcpOption5 rwm-map objectclass neufDhcpOption1R neufDhcpOption1R rwm-map objectclass neufDhcpOption35 neufDhcpOption35 rwm-map objectclass neufBTOption neufBTOption rwm-map objectclass neufDhcpOptionwifi neufDhcpOptionwifi rwm-map objectclass Dhcp82 Dhcp82 rwm-map objectclass ClientDevice ClientDevice rwm-map objectclass DhcpDevice DhcpDevice rwm-map objectclass suffixobj suffixobj rwm-map objectclass OrganizationalUnit OrganizationalUnit rwm-map objectclass Organization Organization rwm-map objectclass dcObject dcObject rwm-map objectclass domain domain rwm-map objectclass device device rwm-map attribute * rwm-map objectclass * Thanks in avance for your help Regards,

1 0

Re: (ITS#6700) memberOf overlay does not handle MODRDN correctly
by masarati＠aero.polimi.it 22 Nov '10

22 Nov '10

raphael.ouazana(a)linagora.com wrote: > When using memberOf overlay, a MODRDN request on a member does not update > correctly the matching group, even if the memberof-refint option is set to > true. > > In my little understanding, memberof_value_modify calls op->o_bd->be_modify > which calls back the overlay code, and finally never reach the backend. Guess what, modrdn is the only operation test052 does not check :( WOrking at it. Thanks, p.

1 0

Re: (ITS#6625) draft-zeilenga-ldap-c-api-concurrency support
by h.b.furuseth＠usit.uio.no 22 Nov '10

22 Nov '10

Some issues with tests/progs/slapd-mtread.c: - snprintf format "%04.4d" is not standard. How about just "%04d"? - However, shouldn't 'me' be 'myidx' in those snprintfs and when computing 'thisconn = (me + j) % noconns;' ? - A thread ID can be a struct, in which case whoami() will fail. OpenLDAP (the TLS code) already fails in that case, but still, no need to add do that. Could pass 'myidx' around, or create a thread-local variable for it with ldap_pvt_thread_key_create(). - It says "initially developed by Kurt Spanier", shouldn't that be Doug Leavitt? - scripts/test060-mt-hot takes a long time and can generate 2G of logs. Maybe it should run with a lesser loglevel by default? Put : ${SLAPD_DEBUG=0x4100} -- by default, avoid 1.-2 gigabytes of logs before sourcing defines.sh. -- Hallvard

1 0

Re: (ITS#6715) Broken 'sortvals' for attr with 1 value
by h.b.furuseth＠usit.uio.no 22 Nov '10

22 Nov '10

Also attr_valfind() breaks with sortvals if #values = 0. Not sure if it can be called that way, but it doesn't hurt to handle it right. -- Hallvard

1 0

(ITS#6719) syncrepl_add_glue_ancestors() uninitialized return
by h.b.furuseth＠usit.uio.no 22 Nov '10

22 Nov '10

Full_Name: Hallvard B Furuseth Version: 2.4.23, HEAD OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.240.6.233) Submitted by: hallvard syncrepl_add_glue_ancestors() returns 'rc' uninitialized if the 'else' branch of 'if ( ndn.bv_len > be->be_nsuffix[0].bv_len )' is taken: Then the loop is not entered. This also means the 'else' branch is useless. Should it be doing something more? Found with gcc -O -Wuninitialized . 'make test' does not seem to enter the 'else'.

1 0

Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by hyc＠symas.com 21 Nov '10

21 Nov '10

rein(a)OpenLDAP.org wrote: > On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote: >> New syncprov consumers connecting to a forwarding server and presenting an >> apperently up-to-date cookie will loose any mods that have already taken place >> on the forwarding server if it itself is refreshing from its provider. This >> should not be a problem if the forwarding server have a sufficiently large >> syncprov log, but a fix for servers without is coming. > The currently committed fix for this its leaves one problem open. If a > forwarding server restarts in the middle of the refresh phase, after > making some changes but before updating the csn, new consumers > connecting with an apparently up to date csn set after the server comes > up again will not know that the context is dirty and will loose these > changes. The same problem arise if the server restarts between the time > a locally initiated delete operation is performed in the database and > the accompanying csn set is saved. > > A fix could be to always assume the context is dirty after start up, and > thereby forcing all clients to undergo the refresh phase even if they > are in sync until some operation that updates the csn set is performed. > An unnecessary refresh is probably better than loosing changes.. I haven't had the opportunity to review these patches yet, but the bug description sounds a little flaky to me. The original design is this: when a consumer requests a refresh from the provider, the provider uses a snapshot of its current contextCSN. All changes from the consumer's cookie to this snapshot (inclusive) are sent to the consumer. Any changes currently in progress that have not yet been committed will be skipped until the next refresh. Nothing is lost, it's simply delayed, and that's in accordance with syncrepl's "eventual convergence" model. Likewise, the provider only updates its contextCSN when a change is fully committed. syncprov should NOT need to defer any consumer while it has outstanding mods. There is no reason that would be needed by the sync protocol. Servers only update their contextCSN after an entire refresh has completed. If a downstream consumer connects while a forwarder is still refreshing, the consumer should receive nothing. (Or, it should only receive the changes between its cookie and the server's committed contextCSN, if any.) You may very well have found bugs in the implementation, but it sounds to me like you've changed the overall behavior to something outside of the original design. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#6718) Send new-cookie messages to all consumers
by rein＠OpenLDAP.org 21 Nov '10

21 Nov '10

Full_Name: Rein Tollevik Version: CVS head OS: Irrelevant URL: Submission from: (NULL) (84.215.59.121) Submitted by: rein When syncprov updates its csn as a result of syncrepl updating its, syncprov currently sends NEW_COOKIE messages only to those consumers that replicate the entry where the csn is stored. It should be sent to all consumers. A fix is coming. -- Rein Tollevik Basefarm AS

1 0

Re: (ITS#6710) Mods already refreshed on a forwarding server is lost by its consumer
by rein＠OpenLDAP.org 21 Nov '10

21 Nov '10

On 11/15/10 16:39 , rein(a)OpenLDAP.org wrote: > New syncprov consumers connecting to a forwarding server and presenting an > apperently up-to-date cookie will loose any mods that have already taken place > on the forwarding server if it itself is refreshing from its provider. This > should not be a problem if the forwarding server have a sufficiently large > syncprov log, but a fix for servers without is coming. The currently committed fix for this its leaves one problem open. If a forwarding server restarts in the middle of the refresh phase, after making some changes but before updating the csn, new consumers connecting with an apparently up to date csn set after the server comes up again will not know that the context is dirty and will loose these changes. The same problem arise if the server restarts between the time a locally initiated delete operation is performed in the database and the accompanying csn set is saved. A fix could be to always assume the context is dirty after start up, and thereby forcing all clients to undergo the refresh phase even if they are in sync until some operation that updates the csn set is performed. An unnecessary refresh is probably better than loosing changes.. Rein

1 0

(ITS#6717) syncprov performs unnecessary presentations/refreshes.
by cmikk＠qwest.net 19 Nov '10

19 Nov '10

Full_Name: Chris Mikkelson Version: 2.4.23 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (204.147.85.37) I have a multimaster setup, and some read-only slaves mirroring the data with syncrepl. In some circumstances, I've noticed that restarting a slave server leads to a full presentation, even though the masters have an adequately sized sessionlog. This makes refreshOnly synchronization prohibitively expensive. It appears that the problem is as follows: The syncprov overlay uses the minimum CSN from the client cookie to decide where to start synchronization, regardless of whether that CSN has changed relative to the same SID's CSN on the provider. For example: Provider state: CSN: <1 second ago>...#001#... CSN: <1 year ago>.....#002#... sessionlog mincsn: <1 hour ago>...#001#... Consumer state: CSN: <1 minute ago>...#001#... CSN: <1 year ago>.....#002#... <-- same as SID 2 CSN on provider The syncprov overly will take the mincsn from the consumer cookie (the <1 year ago> one above) and compare it to the sessionlog mincsn when deciding whether to do a full presentation. Since the SID 2 CSN hasn't changed in the above example, the consumer is up to date on changes recorded on SID 2, so the SID 1 CSN from the cookie should be used. To reproduce: * Set up slapd servers ID 1 and 2 as multimaster nodes, and a third slapd server replicating from server 1. * Insert one entry into server ID 2, and restart servers 1 and 2. * Insert entries into Server ID 1. * Perform enough modifications / deletions on Server ID 1 to cause entries to be purged from the sessionlog (see ITS #6716). * Wait for slave to catch up, then stop the slave. * Do any modification on Server ID 1 (only need to change the SID 1 contextCSN) * Restart slave. Server will present entries to the slave rather than replaying the session log.

1 0

(ITS#6716) syncprov sessionlog 'sl_mincsn' not assigned a value.
by cmikk＠qwest.net 19 Nov '10

19 Nov '10

Full_Name: Chris Mikkelson Version: 2.4.23 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (204.147.85.37) On line 2875 of servers/slapd/overlays/syncprov.c (release 2.4.23), the sessionlog mincsn is initialized to a zero-length value. if ( !sl ) { sl = ch_malloc( sizeof( sessionlog ) + LDAP_PVT_CSNSTR_BUFSIZE ); sl->sl_mincsn.bv_val = (char *)(sl+1); sl->sl_mincsn.bv_len = 0; sl->sl_num = 0; sl->sl_head = sl->sl_tail = NULL; ldap_pvt_thread_mutex_init( &sl->sl_mutex ); si->si_logs = sl; } >From there, sl->sl_mincsn isn't touched until log entries are purged from the sessionlog on line 1522: while ( sl->sl_num > sl->sl_size ) { se = sl->sl_head; sl->sl_head = se->se_next; strcpy( sl->sl_mincsn.bv_val, se->se_csn.bv_val ); sl->sl_mincsn.bv_len = se->se_csn.bv_len; ch_free( se ); sl->sl_num--; } So, after the sessionlog has its first entry recorded and before it has its first entry received, sl->sl_mincsn is a zero-length value, which means the following comparison on line 2536: if ( sl->sl_num > 0 && ber_bvcmp( &mincsn, &sl->sl_mincsn ) >= 0 ) { do_present = 0; will evaluate to true in cases where it should not. A quick fix would be to compare directly against the CSN of the sessionlog's head: --- syncprov.c.orig 2010-11-17 14:07:26.000000000 -0600 +++ syncprov.c 2010-11-19 16:50:58.000000000 -0600 @@ -2533,7 +2533,7 @@ /* Are there any log entries, and is the consumer state * present in the session log? */ - if ( sl->sl_num > 0 && ber_bvcmp( &mincsn, &sl->sl_mincsn ) >= 0 ) { + if ( sl->sl_num > 0 && ber_bvcmp( &mincsn, &sl->sl_head->se_csn ) >= 0 ) { do_present = 0; /* mutex is unlocked in playlog */ syncprov_playlog( op, rs, sl, srs, ctxcsn, numcsns, sids );

1 0

Re: ITS#6713
by masarati＠aero.polimi.it 19 Nov '10

19 Nov '10

DevMan wrote: > Please let me know the upgradation procedure. <http://www.openldap.org/doc/admin24/install.html> In general, <http://www.openldap.org/doc/admin24/index.html> p.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs