- openldap-bugs - openldap.org

Re: (ITS#8505) LMDB vs. fork()
by h.b.furuseth＠usit.uio.no 29 Sep '16

29 Sep '16

On 29. sep. 2016 18:27, lmb(a)cloudflare.com wrote: > <h.b.furuseth(a)usit.uio.no> wrote: > >> Yes. Your concern seems to reduce to a doc bug and an >> otherwise-harmless file descriptor leak which it is too late to fix >> completely. So I ended up thinking of resource leaks in general. > > Why is it harmless in your opinion? Because I wrote that documentation, and now I know what it gets wrong:-) Only the lockfile descriptor is dangerous. Therefore it already had FD_CLOEXEC. The other FD leaks are just leaks, they can't break LMDB. >> It'd be pretty intrusive of a _library_ to forbid the user to >> fork() at all without exec(). > > From my point of view, that is what the LMDB doc says. Yes. And that doc is half wrong, so I'll fix it. And it's half right with pthread_exit(), so I want to fix LMDB about that. > If I understand you correctly, LMDB would be safe to use in these > scenarios in an ideal world: > > 1. mdb_env_open(), fork(), exec() (without fd leaks I'd argue) > 2. mdb_env_open(), pthread_create(), fork() > 3. mdb_env_open(), fork() > > As far as I understand you are after 2 and 3, while I want 1. I'm after all three. (1) is already safe. The mdb_env_get_fd() descriptor will leak since otherwise programs using it could break. But you can close() it yourself in the child. Well, or mdb_env_open() could take an MDB_CLOEXEC flag which says to set FD_CLOEXEC on that FD too. And since the child may need to do something to avoid leaks anyway, I figure we might as well provide a cleanup function do do it properly. So... > Case 2 > seems unlikely, given that forking a multi-threaded program is so hard > that is rarely makes sense [1]. It's very limited what you can do in the child, yes. But sometimes if you have do then you have to. E.g. if the exec()ed program may not exist, and the child has do some simple task as a fallback. Or if you're not free to rewrite the main program, only some module. Case 2 will at least have memory leaks, since free() in the child is unsafe. But we can get rid of some OS resources (FDs, memory map, maybe semaphores, need to check about that.) > Case 3 is simply a matter of reordering the calls. Only if the program logic allows it. Cumbersome e.g. if the LMDB database contains the config controlling the child. -- Hallvard

1 0

(ITS#8507) adding the syncprov overlay does not load the associated schema elements
by elecharny＠apache.org 29 Sep '16

29 Sep '16

Full_Name: Emmanuel L.charny Version: 2.4.44 OS: Linux CentOS 6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (176.154.3.51) Start an OpenLDAP server using the dynamic configuration (slapd.d) from a slapd.conf file that does not contain a synprov module : $ /opt/symas/bin/slaptest -f /opt/symas/etc/openldap/slapd.conf -F /opt/symas/etc/openldap/slapd.d $ /etc/init.d/solserver start -d 256 $ grep -ri "olcSpNoPresent" slapd.d $ ... Then inject the syncprov module in the server : dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov.la - The olcSpNoPresent attribute is not present : $ grep -ri "olcSpNoPresent" slapd.d $ ... You can stop and restart the server, the attribute is still not present. Going back to the slapd.conf file, add the module : ... # Uncomment the mululeloads as needed to enable additional # functionalityi when configured. NOTE: We package many # more modules options than those found below. moduleload back_mdb.la moduleload back_monitor.la moduleload syncprov.la # Added module ... and regenerate the slapd.d content : $ /etc/init.d/solserver stop $ rm -rf /opt/symas/etc/openldap/slapd.d/* $ /opt/symas/bin/slaptest -f /opt/symas/etc/openldap/slapd.conf -F /opt/symas/etc/openldap/slapd.d $ grep -ri "olcSpNoPresent" /opt/symas/etc/openldap/slapd.d slapd.d/cn=config/cn=schema.ldif:olcAttributeTypes: ( OLcfgOvAt:1.3 NAME 'olcSpNoPresent' DESC 'Omit Present slapd.d/cn=config/cn=schema.ldif: $ olcSpSessionlog $ olcSpNoPresent $ olcSpReloadHint ) ) $... This is problematic...

1 0

Re: (ITS#8505) LMDB vs. fork()
by lmb＠cloudflare.com 29 Sep '16

29 Sep '16

On 29 September 2016 at 03:39, Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no> wrote: > Yes. Your concern seems to reduce to a doc bug and an > otherwise- harmless file descriptor leak which it is too late to fix > completely. So I ended up thinking of resource leaks in general. Why is it harmless in your opinion? > It'd be pretty intrusive of a _library_ to forbid the user to > fork() at all without exec(). >From my point of view, that is what the LMDB doc says. I can't use LMDB and fork(). We're probably approaching this from different angles though: I want to take something away that is de-facto possible, but was not documented to be that way. > No, I mean a partial env_close to be called _after_ the fork, in the > child. A child which does not want FD-leaks must in any case do a > close() loop or close the mdb_env_get_fd() descriptor. So maybe we > just as well should give him a more thorough "free resources in child" > call which cleans up a bit for exec()-less programs as well. > (It'll need an argument which says just what is safe to clean up - > e.g. it can't do much much if the parent is already multi-threaded.) Sorry, I misunderstood you here. If I understand you correctly, LMDB would be safe to use in these scenarios in an ideal world: 1. mdb_env_open(), fork(), exec() (without fd leaks I'd argue) 2. mdb_env_open(), pthread_create(), fork() 3. mdb_env_open(), fork() As far as I understand you are after 2 and 3, while I want 1. Case 2 seems unlikely, given that forking a multi-threaded program is so hard that is rarely makes sense [1]. Case 3 is simply a matter of reordering the calls. 1: http://thorstenball.com/blog/2014/10/13/why-threads-cant-fork/

1 0

Re: (ITS#8505) LMDB vs. fork()
by h.b.furuseth＠usit.uio.no 29 Sep '16

29 Sep '16

On 28/09/16 21:29, lmb(a)cloudflare.com wrote: > I think your proposal is going beyond what I was trying to do. Yes. Your concern seems to reduce to a doc bug and an otherwise- harmless file descriptor leak which it is too late to fix completely. So I ended up thinking of resource leaks in general. > From the documentation, it sounds like a process using LMDB can not > safely fork, under any circumstances. My idea was to specialise this > to saying that fork() followed immediately by exec() is safe, Yes. That doc bug should be fixed. > but that > a plain fork() is still not supported. If a child wants to use the > same db, it goes through the regular mdb_env_open sequence, rather > than somehow inheriting the environment across the fork. I think that > should address your pthread_exit concern, as well? It'd be pretty intrusive of a _library_ to forbid the user to fork() at all without exec(). A _program_ could specify that. So I prefer to protect the parent from pthread_exit in the child. (Note, this has nothing to do with file descriptor leaks). > On your point that a user may not call mdb_env_close before the fork: > the user might not want to, since s/he still wants to use the env in > the original process. No, I mean a partial env_close to be called _after_ the fork, in the child. A child which does not want FD-leaks must in any case do a close() loop or close the mdb_env_get_fd() descriptor. So maybe we just as well should give him a more thorough "free resources in child" call which cleans up a bit for exec()-less programs as well. (It'll need an argument which says just what is safe to clean up - e.g. it can't do much much if the parent is already multi-threaded.)

1 0

(ITS#8506) Authentication issue in ldap client machine after successfully integrating ldap in freenas OS
by caraghavvarma1＠gmail.com 29 Sep '16

29 Sep '16

Full_Name: Raghavendra varma C A Version: 3 OS: ubuntu server and freenas OS URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (123.63.153.45) Hi Team, After successfully integrating Ldap server in freenas OS(FreeBSD base OS), am able to view ldap users but i am not able to login to freenas os using ldap user credentials. Can you please let me know should i configure any additional permissions for the ldap user. And one more thing is that am not able to view home directory of ldap user. Kindly look into the this and do the needful ASAP. Thanks and Regards, Raghav-IT.

1 0

Re: (ITS#8505) LMDB vs. fork()
by lmb＠cloudflare.com 28 Sep '16

28 Sep '16

I think your proposal is going beyond what I was trying to do. >From the documentation, it sounds like a process using LMDB can not safely fork, under any circumstances. My idea was to specialise this to saying that fork() followed immediately by exec() is safe, but that a plain fork() is still not supported. If a child wants to use the same db, it goes through the regular mdb_env_open sequence, rather than somehow inheriting the environment across the fork. I think that should address your pthread_exit concern, as well? On your point that a user may not call mdb_env_close before the fork: the user might not want to, since s/he still wants to use the env in the original process.

1 0

(ITS#8504) LDMB: Return EPIPE from mdb_env_copyfd2 instead abort on SIGPIPE
by lmb＠cloudflare.com 27 Sep '16

27 Sep '16

On 26 September 2016 at 20:48, Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no> wrote: > I think we can skip sigwait() too since the thread will exit when done. > Then any pending signals are presumably dropped. (I suppose some OS > could give them to another thread, but that goes against delivering > them to a particular thread like SIGPIPE is.) That's what my initial patch did. At least on OS X this leads to the process receiving SIGPIPE and dying. I can attach a test case as well if desired.

1 0

(ITS#8505) LMDB vs. fork()
by h.b.furuseth＠usit.uio.no 27 Sep '16

27 Sep '16

Full_Name: Hallvard B Furuseth Version: LMDB_0.9.18 OS: Linux URL: Submission from: (NULL) (81.191.45.31) Submitted by: hallvard This is from openldap-tecnical thread "Using LMDB safely with fork() and exec()" by Lorenz Bauer: <http://www.openldap.org/lists/openldap-technical/201608/msg00062.html> ==== The doc is too restrictive, it implicitly suggests an LMDB process can't fork another LDMB process (since the old LMDB would still be open). We should document the use of FD_CLOEXEC etc. ==== This breaks LMDB: if (fork() == 0) pthread_exit(NULL); because the child's me_txkey destructor releases the reader slot. This can happen if a single-threaded LMDB process forks a multi- threaded child, which is one way to deal with threads vs. fork(). Simplest fix: Check if (reader->mr_pid == getpid()) in reader_dest(). We could instead use pthread_atfork(), but there is not way to remove an atfork handler. It seems to be implementation-dependent what happens with an atfork handler if liblmdb is dynamically unloaded. ==== LMDB leaks file descriptors to a fork() child, since the user may not call mdb_env_close() and there is no way to get at some of the descriptors. Partial fix: Set FD_CLOEXEC for me_mfd and the mdb_env_copy() FD. Should I code up an mdb_env_child_cleanup() function which may be called in the child if it does not fork()? That makes sense either if the parent is single-threaded and forks a more complex child, or if the child is fairly simple. ==== Branch "mdb/fopen2" in my repo will soon include suggested fixes, except mdb_env_child_cleanup(). (Fixed version of older "mdb/fopen")

1 0

Re: (ITS#8504) LDMB: Return EPIPE from mdb_env_copyfd2 instead abort on SIGPIPE
by h.b.furuseth＠usit.uio.no 27 Sep '16

27 Sep '16

On 25/09/16 20:16, lmb(a)cloudflare.com wrote: > * Do not check for pending SIGPIPE: since this is a new thread there can't > be any I think we can skip sigwait() too since the thread will exit when done. Then any pending signals are presumably dropped. (I suppose some OS could give them to another thread, but that goes against delivering them to a particular thread like SIGPIPE is.)

1 0

Re: (ITS#8504) LDMB: Return EPIPE from mdb_env_copyfd2 instead abort on SIGPIPE
by lmb＠cloudflare.com 25 Sep '16

25 Sep '16

--94eb2c07d07472c185053d5903fe Content-Type: text/plain; charset=UTF-8 I've updated the patch to be much simpler: * Set pthread_sigmask in copythr and return an error in my->mc_error * Do not check for pending SIGPIPE: since this is a new thread there can't be any The URL stays the same. --94eb2c07d07472c185053d5903fe Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra">I've updated the patch to b= e much simpler:</div><div class=3D"gmail_extra"> </div><div class=3D"gma= il_extra">* Set pthread_sigmask in copythr and return an error in my->mc= _error</div><div class=3D"gmail_extra">* Do not check for pending SIGPIPE: = since this is a new thread there can't be any</div><div class=3D"gmail_= extra"> </div><div class=3D"gmail_extra">The URL stays the same.</div></= div> --94eb2c07d07472c185053d5903fe--

1 0

(ITS#8504) LDMB: Return EPIPE from mdb_env_copyfd2 instead abort on SIGPIPE
by lmb＠cloudflare.com 24 Sep '16

24 Sep '16

Full_Name: Lorenz Bauer Version: OS: URL: https://gist.github.com/lmb/63189f83c5f00dd59c86f3c9bc07694d Submission from: (NULL) (2606:4700:1000:8200:5530:fc35:2f95:4f31) As per the discussion in [1] I'm providing a patch to block and handle SIGPIPE in the copy thread used when performing a compacting copy. The attached file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by CloudFlare, Inc. CloudFlare, Inc. has not assigned rights and/or interest in this work to any party. I, Lorenz Bauer am authorized by CloudFlare, Inc., my employer, to release this work under the following terms. CloudFlare, Inc. hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice. 1: http://www.openldap.org/lists/openldap-technical/201609/msg00026.html

1 0

Re: (ITS#8475) Feature request: MDB low durability transactions
by h.b.furuseth＠usit.uio.no 17 Sep '16

17 Sep '16

I'll use this ITS to summarize details for "volatile commits". Hopefully I've managed to keep it all straight. "Volatile" vs. "durable" are the most accurate names I can think of. Not sure if that's more instructive than simply "soft" / "hard". Description: * Volatile commits omit fdatasync() without losing consistency. To survive, they must be checkpointed *before* all processes close the env. Un-checkpointed volatiles are lost when the env closes. Thus a separate checkpointing daemon can keep the env open to protect volatiles from application crash, at least if Robust locks are supported. (The lmdb.h doc seems a bit unclear about Robust.) * Checkpointing == committing a durable (non-volatile) write-txn. (If there is nothing to do, Commit writes nothing.) mdb_env_sync() will not checkpoint volatiles, since existing programs do not expect it to wait for the write mutex. It "checkpoints" MDB_NOMETASYNC/MDB_NOSYNC. Maybe mdb_checkpoint() will have a special case which obsoletes mdb_env_sync(). * Volatiles are unsupported with MDB_NOLOCK and pointless with MDB_NOSYNC. OTOH it makes sense to enable MDB_NOMETASYNC. * Volatiles need a bigger datafile, because it takes two durable commits to make a freed page reusable. (Plus awaiting old readers). Configuration. Too many options, ouch: LMDB can be configured to auto-checkpoint after X volatile commits and/or Y written kbytes(pages?). Programs can also checkpoint every Z minutes(seconds?) - configured in LMDB to mimic Berkeley DB's "checkpoint <kbytes> <minutes>", but regular LMDB ops ignore that. The lockfile gives the current config. An MDB_env could override for its particular process, e.g. with an MDB_NO_VOLATILE flag. Maybe resetting the lockfile should keep the previous config? OTOH I suppose MDB_meta can have default params the way it has mm_mapsize. That survives a backup/restore. Implementation - plain version first: * Bump MDB_LOCK_FORMAT, MDB_DATA_VERSION (or make MDB_DATA_FORMAT). * Keep 2 'MDB_meta's in the lockfile, for volatile commits. MDB_env.me_metas[] gets 4 elements: durable + volatile. * mdb_env_open() throws away volatiles if it re-inits the lockfile. * Add field MDB_meta.mm_oldest: 1 + (previous durable meta).mm_txnid in durable metas, and (previous meta).mm_oldest in volatile metas. Init 'oldest' in mdb_find_oldest() to new field MDB_env.me_oldest, which mdb_txn_renew0(write txn) sets to MDB_meta.mm_oldest. When are no volatiles, this ends up initing 'oldest' = same value as today. Usually we could just have used 1 + (oldest durable meta).mm_txnid, but a failed write_meta() may have clobbered that. * Replace MDB_txninfo.mti_txnid with mti_metaref = txnid*16 + P*4 + M: M = index to MDB_env.me_metas[], P = previous M during this session, initialized to the same as M, so we can get this info atomically. P may prove unnecessary, but it's simplest to just include it for now. For when meta M fails a checksum so we want an older meta, for mdb_mutex_failed(), maybe so we can see if there are volatiles yet. * Never use mdb_env_pick_meta() when the current metapage is known: Use it in mdb_env_open(), in mdb_mutex_failed(), and if MDB_NOLOCK. Or rather, I guess it gets a "force" param for those cases. * Add config in the lockfile. Maybe per-env config overriding it and defaults in the datafile. Txn flags "prefer volatile", "checkpoint". * Track the number of pages and volatiles since last durable commit. write_meta() compares with the config limits and makes the final decision of whether the new meta will be volatile. Add MDB_pgstate.mf_pgcount with #pages used so far. The rest goes in a lockfile array[4] indexed by mti_metaref % 4, or in MDB_meta. That way, switching to next snapshot stays atomic - just update mti_metaref. * txn_begin must verify metas, since we have no fdatasync barriers. Re-read and compare, or checksum. write_meta() and mutex_failed(): memory barrier between making a volatile meta and updating mti_metaref. Most modern compilers have that. Maybe a fallback implementation is lock;unlock an otherwise unused mutex. Should also include CACHEFLUSH(). It may make sense to have more than 2 volatile metas, so read-only txns will have more time to read a meta before it gets overwritten. MDB_WRITEMAP (and MDB_VL32?) has non-atomic issues we should deal with anyway. * We can have (durable metapage).mp_pgno == (txnid & 1) as before: mdb_txn_renew0() steps txnid by 2 instead of 1 if 'meta' is volatile. But note that the txnid doesn't say if the snapshot is durable. * "Try to checkpoint" feature, which does not await the write mutex: Trylock the write mutex in mdb_txn_begin(). If it fails, set a lockfile flag "Please checkpoint" and return. Hopefully someone will obey and clear the flag. mdb_env_commit(writer) does. Variants: * Put volatile MDB_metas in the datafile, behind the usual MDB_metas. That protects the metas from malicious/broken processes with read-only envs. Otherwise, using the lockfile (or non-file shared memory) extended read-only envs' ability to hack/break the DB. Be careful to not read volatile MDB_metas that are older than last lockfile-reset, since the reset did not clear them. That also means this variant does not enable volatiles with MDB_NOLOCK. * Stay with MDB_DATA_VERSION = 1, no change in datafile format: - Volatiles share next durable txn's txnid (for freeDB keys), but put last durable txn's txnid in mti_metaref (for mdb_find_oldest). - mdb_txn_id() / MDB_envinfo.me_last_txnid can no longer be used to distinguish txns. Apps doing that could set an env flag "ignore volatiles", or txn flag "fail if current snapshot is volatile". - Define V = 2-bit sequence number incremented by commit(writer). Include V in MDB_txninfo.mti_metaref. In volatile metas, include mm_metaref = copy of mti_metaref. This lets mutex_failed() figure out which MDB_meta is most recent: abs(V in mm_metaref - V in mti_metaref) is <= 1 whether mm_metaref or mti_metaref changes first in the thread. * Support full 32-bit txnids on 32-bit hosts. mti_metaref eats some txnid bits in order to stay atomic, but we can get them back: On 32-bit hosts, make mti_metaref a 64-bit value - something like ((txnid << 32) | R) where R = 32-bit metaref value described before. When reading mti_metaref, R is authoritative for the low txnid bits. If (txnid << 32) part does not match, adjust it so it does: That'll be + or - a small value. mti_metaref's high bits vary slowly, so this is normally "atomic". txn_renew0's loop re-reads it to verify. * We can squeeze away some bytes and mti_metaref bits, but don't get sucked into spending time on that before the rest is working. E.g. the array[4] counting pages/volatiles can be just [2] if Commit always toggles the (mti_metaref & 1) bit. And we can likely include some of the mti_metaref fields in the txnid. Roads to Hell: * Support checkpointing without waiting for the write-mutex. Programs which have used volatiles, may want this so they can exit quickly. But it requires a new shared mutex for write_meta(), or some clever tricks I've thought of which would not quite work. * Rescue volatiles in a dead lockfile when the user "knows" it's safe. Easy enough, just don't clear them when resetting the lockfile. But users screw up, and may then blame LMDB. Users who want to screw up, can run use MDB_NOSYNC instead of volatile commits. Or if volatile metas are in non-file shared memory which a system crash will kill, it's _almost_ safe to not reset them along with the lockfile. Unless someone unmounts/mounts the disk, or replaces the DB by overwriting it with another DB file, or who knows what else. -- Hallvard

1 0

(ITS#8503) Provide proper soname to liblmdb.so.1
by gahr＠gahr.ch 16 Sep '16

16 Sep '16

Full_Name: Pietro Cerutti Version: LMDB 0.9.18 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (69.191.176.31) This patch provides the shared lib with a proper soname. diff --git a/libraries/liblmdb/Makefile b/libraries/liblmdb/Makefile index 72d0984..4429992 100644 --- a/libraries/liblmdb/Makefile +++ b/libraries/liblmdb/Makefile @@ -66,7 +66,7 @@ liblmdb.a: mdb.o midl.o liblmdb$(SOEXT): mdb.lo midl.lo # $(CC) $(LDFLAGS) -pthread -shared -Wl,-Bsymbolic -o $@ mdb.o midl.o $(SOLIBS) - $(CC) $(LDFLAGS) -pthread -shared -o $@ mdb.lo midl.lo $(SOLIBS)A0A+ 4%4(CC) $(LDFLAGS) -pthread -shared -Wl,-soname=$@ -o $@ mdb.lo midl.lo $(SOLIBS) mdb_stat: mdb_stat.o liblmdb.a mdb_copy: mdb_copy.o liblmdb.a

1 0

Re: (ITS#8501) OpenLdap not RFC 2891 complaint
by mblagoeva＠axway.com 14 Sep '16

14 Sep '16

--_000_AM4PR0101MB19549BBC7F2AB2947F4C5505BBF10AM4PR0101MB1954_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Howard, Thanks again. This strangely enough works with 2.4.40 version , previously = I was using 2.4.31. ________________________________ From: Howard Chu <hyc(a)symas.com> Sent: Tuesday, September 13, 2016 9:23:07 PM To: Maria Blagoeva; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint Maria Blagoeva wrote: > Hello Howard, > > > And just one more follow-up question, if you do not mind, just to have th= e > full picture. The way the RFC is construct, it seems that if the ordering= rule > is omitted in the request one that is defined must be used, but if nothin= g is > defined in the schema nothing will be used, but I do not understand why t= he > openlap will fail than. Exactly the same request works fine with > ActiveDirectory server. ActiveDirectory is not RFC-compliant. > The quotes from the RFC directly: > The MatchingRuleId, as defined in section 4.1.9 of [LDAPv3], SHOULD > be one that is valid for the attribute type it applies to. If it is > not, the server will return inappropriateMatching. > If the orderingRule is omitted, the ordering MatchingRule defined for > use with this attribute MUST be used. And when no MatchingRule exists for an attribute, matching cannot be performed. That is part of RFC4511 so there's no need to state it here. > Thanks again for your insights. > > > -------------------------------------------------------------------------= ----- > *From:* Maria Blagoeva > *Sent:* Tuesday, September 13, 2016 8:50:45 AM > *To:* Howard Chu; openldap-its(a)OpenLDAP.org > *Subject:* Re: (ITS#8501) OpenLdap not RFC 2891 complaint > > > Hello Howard, > > > This means I need to patch the openldap instance or a default ordering ru= le > can be set for that attribute on the fly? Thanks again for your help and > assistance. > > > BR, > > Maria > > -------------------------------------------------------------------------= ----- > *From:* Howard Chu <hyc(a)symas.com> > *Sent:* Monday, September 12, 2016 6:31:25 PM > *To:* Maria Blagoeva; openldap-its(a)OpenLDAP.org > *Subject:* Re: (ITS#8501) OpenLdap not RFC 2891 complaint > > mblagoeva(a)axway.com wrote: >> Full_Name: Maria Blagoeva >> Version: openldap-2.4.31/debian/build/servers/slapd >> OS: docker image with 3.10.0-327.28.2.el7.x86_64 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (194.88.228.86) >> >> >> It seems that openldap as of 2.4.31 is not RFC complaint >> (https://www.ietf.org/rfc/rfc2891.txt) due to failure in response when o= rdering >> match rule is empty while doing server side sorting. Example below: > > False. OpenLDAP is completely correct and RFC-compliant in its behavior. > >> empty ordering rule: >> >> ldapsearch -E sss=3Dcn '(cn=3Dccc*)' cn' (&(objectClass=3DinetOrgPerson)= (cn=3Dccc*))' -H >> ldap://IP:389 -b "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "cn=3Daaa1, >> ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -x -W >> # extended LDIF >> # >> # LDAPv3 >> # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree >> # filter: (cn=3Dccc*) >> # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) >> # with server side sorting control >> # >> >> # search result >> search: 2 >> result: 18 Inappropriate matching >> text: serverSort control: No ordering rule > > This result is correct since the cn attribute has no ordering rule define= d in > the schema. >> >> # numResponses: 1 >> >> caseIgnoreOrderingMatch ordering rule: >> >> ldapsearch -E sss=3Dcn:caseIgnoreOrderingMatch '(cn=3Dccc*)' cn' >> (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))' -H ldap://IP:389 -b >> "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "c3D3Daaa1, ou=3DUsers,dc=3Dopen= stack,dc=3Dorg" -x >> -W >> # extended LDIF >> # >> # LDAPv3 >> # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree >> # filter: (cn=3Dccc*) >> # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) >> # with server side sorting control >> # >> >> # ccc0, Users, openstack.org >> dn: cn=3Dccc0,ou=3DUsers,dc=3Dopenstack,dc=3Dorg >> # search result >> search: 2 >> result: 0 Success >> control: 1.2.840.113556.1.4.474 false MAMKAQA=3D >> sortResult: (0) Success >> >> >> however the RFC states that the orderingRule is OPTIONAL as below: >> >> SortKeyList ::=3D SEQUENCE OF SEQUENCE { >> attributeType AttributeDescription, >> orderingRule [0] MatchingRuleId OPTIONAL, >> reverseOrder [1] BOOLEAN DEFAULT FALSE } >> >> however openldap fails to return entries. > > The rule is optional in the control itself, but a valid ordering rule mus= t > exist somewhere. Since the attribute schema doesn't define one then you m= ust > provide one yourself. > > Closing this ITS. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --_000_AM4PR0101MB19549BBC7F2AB2947F4C5505BBF10AM4PR0101MB1954_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <meta name=3D"Generator" content=3D"Microsoft Exchange Server"> <style></style> </head> <body> <meta content=3D"text/html; charset=3DUTF-8"> <style type=3D"text/css" style=3D"">  </style> <div dir=3D"ltr"> <div id=3D"x_divtagdefaultwrapper" style=3D"font-size:12pt; color:#000000; = background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif"> Hello Howard, Thanks again. This strangely enough works with 2.4.40 version , previous= ly I was using 2.4.31. </div> <hr tabindex=3D"-1" style=3D"display:inline-block; width:98%"> <div id=3D"x_divRplyFwdMsg" dir=3D"ltr">From: Howard Chu <hyc@= symas.com> Sent: Tuesday, September 13, 2016 9:23:07 PM To: Maria Blagoeva; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint <div> </div> </div> </div> <div class=3D"PlainText">Maria Blagoeva wrote: > Hello Howard, > > > And just one more follow-up question, if you do not mind, just to have= the > full picture. The way the RFC is construct, it seems that if the order= ing rule > is omitted in the request one that is defined must be used, but if not= hing is > defined in the schema nothing will be used, but I do not understand wh= y the > openlap will fail than. Exactly the same request works fine with > ActiveDirectory server. ActiveDirectory is not RFC-compliant. > The quotes from the RFC directly: > The MatchingRuleId, as defined in section 4.1.9 of [LDAPv3], SHOULD<br= > >    be one that is valid for the attribute type it appli= es to.  If it is >    not, the server will return inappropriateMatching.<b= r> >  If the orderingRule is omitted, the ordering MatchingRule define= d for >    use with this attribute MUST be used. And when no MatchingRule exists for an attribute, matching cannot be performed. That is part of RFC4511 so there's no need to state it here. > Thanks again for your insights. > > > ----------------------------------------------------------------------= -------- > *From:* Maria Blagoeva > *Sent:* Tuesday, September 13, 2016 8:50:45 AM > *To:* Howard Chu; openldap-its(a)OpenLDAP.org > *Subject:* Re: (ITS#8501) OpenLdap not RFC 2891 complaint > > > Hello Howard, > > > This means I need to patch the openldap instance or a default ordering= rule > can be set for that attribute on the fly? Thanks again for your help a= nd > assistance. > > > BR, > > Maria > > ----------------------------------------------------------------------= -------- > *From:* Howard Chu <hyc(a)symas.com> > *Sent:* Monday, September 12, 2016 6:31:25 PM > *To:* Maria Blagoeva; openldap-its(a)OpenLDAP.org > *Subject:* Re: (ITS#8501) OpenLdap not RFC 2891 complaint > > mblagoeva(a)axway.com wrote: >> Full_Name: Maria Blagoeva >> Version: openldap-2.4.31/debian/build/servers/slapd >> OS: docker image with  3.10.0-327.28.2.el7.x86_64 >> URL: <a href=3D"ftp://ftp.openldap.org/incoming/">ftp://ftp.openld= ap.org/incoming/</a> >> Submission from: (NULL) (194.88.228.86) >> >> >> It seems that openldap as of 2.4.31 is not RFC complaint >> (<a href=3D"https://www.ietf.org/rfc/rfc2891.txt">https://www.ietf= .org/rfc/rfc2891.txt</a>) due to failure in response when ordering >> match rule is empty while doing server side sorting. Example below= : > > False. OpenLDAP is completely correct and RFC-compliant in its behavio= r. > >> empty ordering rule: >> >> ldapsearch -E sss=3Dcn '(cn=3Dccc*)' cn' (&(objectClass=3Dinet= OrgPerson)(cn=3Dccc*))' -H >> ldap://IP:389 -b "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D= "cn=3Daaa1, >> ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -x -W >> # extended LDIF >> # >> # LDAPv3 >> # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtr= ee >> # filter: (cn=3Dccc*) >> # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))<b= r> >> # with server side sorting control >> # >> >> # search result >> search: 2 >> result: 18 Inappropriate matching >> text: serverSort control: No ordering rule > > This result is correct since the cn attribute has no ordering rule def= ined in > the schema. >> >> # numResponses: 1 >> >> caseIgnoreOrderingMatch ordering rule: >> >> ldapsearch -E sss=3Dcn:caseIgnoreOrderingMatch '(cn=3Dccc*)' cn'<b= r> >> (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))' -H ldap://IP:389 = -b >> "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "c3D3Daaa1,= ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -x >> -W >> # extended LDIF >> # >> # LDAPv3 >> # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtr= ee >> # filter: (cn=3Dccc*) >> # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))<b= r> >> # with server side sorting control >> # >> >> # ccc0, Users, openstack.org >> dn: cn=3Dccc0,ou=3DUsers,dc=3Dopenstack,dc=3Dorg >> # search result >> search: 2 >> result: 0 Success >> control: 1.2.840.113556.1.4.474 false MAMKAQA=3D >> sortResult: (0) Success >> >> >> however the RFC states that the orderingRule is OPTIONAL as below:= >> >>       SortKeyList ::=3D SEQUENCE OF = SEQUENCE { >>           &= nbsp;      attributeType   AttributeDesc= ription, >>           &= nbsp;      orderingRule    [0] Matc= hingRuleId OPTIONAL, >>           &= nbsp;      reverseOrder    [1] BOOL= EAN DEFAULT FALSE } >> >> however openldap fails to return entries. > > The rule is optional in the control itself, but a valid ordering rule = must > exist somewhere. Since the attribute schema doesn't define one then yo= u must > provide one yourself. > > Closing this ITS. > > -- >    -- Howard Chu >    CTO, Symas Corp.      =      <a href=3D"http://www.symas.com">http://www.symas.= com</a> >    Director, Highland Sun     <a hr= ef=3D"http://highlandsun.com/hyc/">http://highlandsun.com/hyc/</a> >    Chief Architect, OpenLDAP  <a href=3D"http://ww= w.openldap.org/project/">http://www.openldap.org/project/</a> --    -- Howard Chu    CTO, Symas Corp.       &nbs= p;   <a href=3D"http://www.symas.com">http://www.symas.com</a><br= >    Director, Highland Sun     <a href=3D"http= ://highlandsun.com/hyc/">http://highlandsun.com/hyc/</a>    Chief Architect, OpenLDAP  <a href=3D"http://www.openldap= .org/project/">http://www.openldap.org/project/</a> </div> </body> </html> --_000_AM4PR0101MB19549BBC7F2AB2947F4C5505BBF10AM4PR0101MB1954_--

1 0

Re: (ITS#8490) changes not written to accesslog, causing replicas to loop syncing
by quanah＠zimbra.com 13 Sep '16

13 Sep '16

--On Friday, September 09, 2016 9:40 PM +0000 quanah(a)zimbra.com wrote: > --On Thursday, September 01, 2016 9:50 AM +0000 quanah(a)zimbra.com wrote: Backing out various changes, I tracked this down to the recent rewrite of ITS8054. As soon as that patch is introduced, changes are lost, causing this issue (and 8493). --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8501) OpenLdap not RFC 2891 complaint
by hyc＠symas.com 13 Sep '16

13 Sep '16

Maria Blagoeva wrote: > Hello Howard, > > > And just one more follow-up question, if you do not mind, just to have the > full picture. The way the RFC is construct, it seems that if the ordering rule > is omitted in the request one that is defined must be used, but if nothing is > defined in the schema nothing will be used, but I do not understand why the > openlap will fail than. Exactly the same request works fine with > ActiveDirectory server. ActiveDirectory is not RFC-compliant. > The quotes from the RFC directly: > The MatchingRuleId, as defined in section 4.1.9 of [LDAPv3], SHOULD > be one that is valid for the attribute type it applies to. If it is > not, the server will return inappropriateMatching. > If the orderingRule is omitted, the ordering MatchingRule defined for > use with this attribute MUST be used. And when no MatchingRule exists for an attribute, matching cannot be performed. That is part of RFC4511 so there's no need to state it here. > Thanks again for your insights. > > > ------------------------------------------------------------------------------ > *From:* Maria Blagoeva > *Sent:* Tuesday, September 13, 2016 8:50:45 AM > *To:* Howard Chu; openldap-its(a)OpenLDAP.org > *Subject:* Re: (ITS#8501) OpenLdap not RFC 2891 complaint > > > Hello Howard, > > > This means I need to patch the openldap instance or a default ordering rule > can be set for that attribute on the fly? Thanks again for your help and > assistance. > > > BR, > > Maria > > ------------------------------------------------------------------------------ > *From:* Howard Chu <hyc(a)symas.com> > *Sent:* Monday, September 12, 2016 6:31:25 PM > *To:* Maria Blagoeva; openldap-its(a)OpenLDAP.org > *Subject:* Re: (ITS#8501) OpenLdap not RFC 2891 complaint > > mblagoeva(a)axway.com wrote: >> Full_Name: Maria Blagoeva >> Version: openldap-2.4.31/debian/build/servers/slapd >> OS: docker image with 3.10.0-327.28.2.el7.x86_64 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (194.88.228.86) >> >> >> It seems that openldap as of 2.4.31 is not RFC complaint >> (https://www.ietf.org/rfc/rfc2891.txt) due to failure in response when ordering >> match rule is empty while doing server side sorting. Example below: > > False. OpenLDAP is completely correct and RFC-compliant in its behavior. > >> empty ordering rule: >> >> ldapsearch -E sss=cn '(cn=ccc*)' cn' (&(objectClass=inetOrgPerson)(cn=ccc*))' -H >> ldap://IP:389 -b "ou=Users,dc=openstack,dc=org" -D "cn=aaa1, >> ou=Users,dc=openstack,dc=org" -x -W >> # extended LDIF >> # >> # LDAPv3 >> # base <ou=Users,dc=openstack,dc=org> with scope subtree >> # filter: (cn=ccc*) >> # requesting: cn (&(objectClass=inetOrgPerson)(cn=ccc*)) >> # with server side sorting control >> # >> >> # search result >> search: 2 >> result: 18 Inappropriate matching >> text: serverSort control: No ordering rule > > This result is correct since the cn attribute has no ordering rule defined in > the schema. >> >> # numResponses: 1 >> >> caseIgnoreOrderingMatch ordering rule: >> >> ldapsearch -E sss=cn:caseIgnoreOrderingMatch '(cn=ccc*)' cn' >> (&(objectClass=inetOrgPerson)(cn=ccc*))' -H ldap://IP:389 -b >> "ou=Users,dc=openstack,dc=org" -D "c3D3Daaa1, ou=Users,dc=openstack,dc=org" -x >> -W >> # extended LDIF >> # >> # LDAPv3 >> # base <ou=Users,dc=openstack,dc=org> with scope subtree >> # filter: (cn=ccc*) >> # requesting: cn (&(objectClass=inetOrgPerson)(cn=ccc*)) >> # with server side sorting control >> # >> >> # ccc0, Users, openstack.org >> dn: cn=ccc0,ou=Users,dc=openstack,dc=org >> # search result >> search: 2 >> result: 0 Success >> control: 1.2.840.113556.1.4.474 false MAMKAQA= >> sortResult: (0) Success >> >> >> however the RFC states that the orderingRule is OPTIONAL as below: >> >> SortKeyList ::= SEQUENCE OF SEQUENCE { >> attributeType AttributeDescription, >> orderingRule [0] MatchingRuleId OPTIONAL, >> reverseOrder [1] BOOLEAN DEFAULT FALSE } >> >> however openldap fails to return entries. > > The rule is optional in the control itself, but a valid ordering rule must > exist somewhere. Since the attribute schema doesn't define one then you must > provide one yourself. > > Closing this ITS. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8502) mdb_cursor_count() == EINVAL for empty DB
by h.b.furuseth＠usit.uio.no 13 Sep '16

13 Sep '16

Full_Name: Hallvard B Furuseth Version: mdb.master, 26e226b2a8cdba4208eabca394537900c4eda3b3 OS: URL: Submission from: (NULL) (81.191.45.31) Submitted by: hallvard mdb_cursor_count() has code to return MDB_NOTFOUND if !mc_snum, but it comes after if (! ... C_INITIALIZED) return EINVAL.

1 0

Re: (ITS#8501) OpenLdap not RFC 2891 complaint
by mblagoeva＠axway.com 13 Sep '16

13 Sep '16

--_000_AM4PR0101MB1954B05EC18907733774338FBBFE0AM4PR0101MB1954_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Howard, And just one more follow-up question, if you do not mind, just to have the = full picture. The way the RFC is construct, it seems that if the ordering r= ule is omitted in the request one that is defined must be used, but if noth= ing is defined in the schema nothing will be used, but I do not understand = why the openlap will fail than. Exactly the same request works fine with Ac= tiveDirectory server. The quotes from the RFC directly: The MatchingRuleId, as defined in section 4.1.9 of [LDAPv3], SHOULD be one that is valid for the attribute type it applies to. If it is not, the server will return inappropriateMatching. If the orderingRule is omitted, the ordering MatchingRule defined for use with this attribute MUST be used. Thanks again for your insights. ________________________________ From: Maria Blagoeva Sent: Tuesday, September 13, 2016 8:50:45 AM To: Howard Chu; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint Hello Howard, This means I need to patch the openldap instance or a default ordering rule= can be set for that attribute on the fly? Thanks again for your help and a= ssistance. BR, Maria ________________________________ From: Howard Chu <hyc(a)symas.com> Sent: Monday, September 12, 2016 6:31:25 PM To: Maria Blagoeva; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint mblagoeva(a)axway.com wrote: > Full_Name: Maria Blagoeva > Version: openldap-2.4.31/debian/build/servers/slapd > OS: docker image with 3.10.0-327.28.2.el7.x86_64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.88.228.86) > > > It seems that openldap as of 2.4.31 is not RFC complaint > (https://www.ietf.org/rfc/rfc2891.txt) due to failure in response when or= dering > match rule is empty while doing server side sorting. Example below: False. OpenLDAP is completely correct and RFC-compliant in its behavior. > empty ordering rule: > > ldapsearch -E sss=3Dcn '(cn=3Dccc*)' cn' (&(objectClass=3DinetOrgPerson)(= cn=3Dccc*))' -H > ldap://IP:389 -b "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "cn=3Daaa1, > ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -x -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # search result > search: 2 > result: 18 Inappropriate matching > text: serverSort control: No ordering rule This result is correct since the cn attribute has no ordering rule defined = in the schema. > > # numResponses: 1 > > caseIgnoreOrderingMatch ordering rule: > > ldapsearch -E sss=3Dcn:caseIgnoreOrderingMatch '(cn=3Dccc*)' cn' > (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))' -H ldap://IP:389 -b > "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "c3D3Daaa1, ou=3DUsers,dc=3Dopens= tack,dc=3Dorg" -x > -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # ccc0, Users, openstack.org > dn: cn=3Dccc0,ou=3DUsers,dc=3Dopenstack,dc=3Dorg > # search result > search: 2 > result: 0 Success > control: 1.2.840.113556.1.4.474 false MAMKAQA=3D > sortResult: (0) Success > > > however the RFC states that the orderingRule is OPTIONAL as below: > > SortKeyList ::=3D SEQUENCE OF SEQUENCE { > attributeType AttributeDescription, > orderingRule [0] MatchingRuleId OPTIONAL, > reverseOrder [1] BOOLEAN DEFAULT FALSE } > > however openldap fails to return entries. The rule is optional in the control itself, but a valid ordering rule must exist somewhere. Since the attribute schema doesn't define one then you mus= t provide one yourself. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --_000_AM4PR0101MB1954B05EC18907733774338FBBFE0AM4PR0101MB1954_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <meta name=3D"Generator" content=3D"Microsoft Exchange Server"> <style></style> </head> <body> <style type=3D"text/css" style=3D"display:none;"></style> <div id=3D"divtagdefaultwrapper" style=3D"font-size:12pt;color:#000000;back= ground-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;"> Hello Howard, And just one more follow-up question, if you do not mind, just to have t= he full picture. The way the RFC is construct, it seems that if the orderin= g rule is omitted in the request one that is defined must be used, but if n= othing is defined in the schema nothing will be used, but I do not understand why the openlap will fail th= an. Exactly the same request works fine with ActiveDirectory server. The quotes from the RFC directly: <pre>The MatchingRuleId, as defined in section 4.1.9 of [LDAPv3], SHOULD=0A= be one that is valid for the attribute type it applies to. If it is=0A= not, the server will return inappropriateMatching.</pre> <pre> If the orderingRule is omitted, the ordering MatchingRule defined for= =0A= use with this attribute MUST be used. Thanks again for your insights. </spa= n></pre> </div> <hr style=3D"display:inline-block;width:98%" tabindex=3D"-1"> <div id=3D"divRplyFwdMsg" dir=3D"ltr">From: Maria Blagoeva Sent: Tuesday, September 13, 2016 8:50:45 AM To: Howard Chu; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint <div> </div> </div> <div> <meta content=3D"text/html; charset=3DUTF-8"> <style type=3D"text/css" style=3D"">  </style> <div dir=3D"ltr"> <div id=3D"x_divtagdefaultwrapper" style=3D"font-size:12pt; color:#000000; = background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif"> Hello Howard, This means I need to patch the openldap instance or a default ordering r= ule can be set for that attribute on the fly? Thanks again for your help an= d assistance. BR, Maria </div> <hr tabindex=3D"-1" style=3D"display:inline-block; width:98%"> <div id=3D"x_divRplyFwdMsg" dir=3D"ltr">From: Howard Chu <hyc@= symas.com> Sent: Monday, September 12, 2016 6:31:25 PM To: Maria Blagoeva; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint <div> </div> </div> </div> <div class=3D"PlainText">mblagoeva(a)axway.com wrote: > Full_Name: Maria Blagoeva > Version: openldap-2.4.31/debian/build/servers/slapd > OS: docker image with  3.10.0-327.28.2.el7.x86_64 > URL: <a href=3D"ftp://ftp.openldap.org/incoming/">ftp://ftp.openldap.o= rg/incoming/</a> > Submission from: (NULL) (194.88.228.86) > > > It seems that openldap as of 2.4.31 is not RFC complaint > (<a href=3D"https://www.ietf.org/rfc/rfc2891.txt">https://www.ietf.org= /rfc/rfc2891.txt</a>) due to failure in response when ordering > match rule is empty while doing server side sorting. Example below:<br= > False. OpenLDAP is completely correct and RFC-compliant in its behavior.<br= > > empty ordering rule: > > ldapsearch -E sss=3Dcn '(cn=3Dccc*)' cn' (&(objectClass=3DinetOrgP= erson)(cn=3Dccc*))' -H > ldap://IP:389 -b "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D &qu= ot;cn=3Daaa1, > ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -x -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree<b= r> > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # search result > search: 2 > result: 18 Inappropriate matching > text: serverSort control: No ordering rule This result is correct since the cn attribute has no ordering rule defined = in the schema. > > # numResponses: 1 > > caseIgnoreOrderingMatch ordering rule: > > ldapsearch -E sss=3Dcn:caseIgnoreOrderingMatch '(cn=3Dccc*)' cn' > (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))' -H ldap://IP:389 -b<b= r> > "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "c3D3Daaa1, ou= =3DUsers,dc=3Dopenstack,dc=3Dorg" -x > -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree<b= r> > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # ccc0, Users, openstack.org > dn: cn=3Dccc0,ou=3DUsers,dc=3Dopenstack,dc=3Dorg > # search result > search: 2 > result: 0 Success > control: 1.2.840.113556.1.4.474 false MAMKAQA=3D > sortResult: (0) Success > > > however the RFC states that the orderingRule is OPTIONAL as below: > >       SortKeyList ::=3D SEQUENCE OF SEQU= ENCE { >           &nbsp= ;      attributeType   AttributeDescript= ion, >           &nbsp= ;      orderingRule    [0] Matching= RuleId OPTIONAL, >           &nbsp= ;      reverseOrder    [1] BOOLEAN = DEFAULT FALSE } > > however openldap fails to return entries. The rule is optional in the control itself, but a valid ordering rule must = exist somewhere. Since the attribute schema doesn't define one then you mus= t provide one yourself. Closing this ITS. --    -- Howard Chu    CTO, Symas Corp.       &nbs= p;   <a href=3D"http://www.symas.com">http://www.symas.com</a><br= >    Director, Highland Sun     <a href=3D"http= ://highlandsun.com/hyc/">http://highlandsun.com/hyc/</a>    Chief Architect, OpenLDAP  <a href=3D"http://www.openldap= .org/project/">http://www.openldap.org/project/</a> </div> </div> </body> </html> --_000_AM4PR0101MB1954B05EC18907733774338FBBFE0AM4PR0101MB1954_--

1 0

Re: (ITS#8501) OpenLdap not RFC 2891 complaint
by mblagoeva＠axway.com 13 Sep '16

13 Sep '16

--_000_AM4PR0101MB1954FD1169119D8DC1DD9008BBFE0AM4PR0101MB1954_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello Howard, This means I need to patch the openldap instance or a default ordering rule= can be set for that attribute on the fly? Thanks again for your help and a= ssistance. BR, Maria ________________________________ From: Howard Chu <hyc(a)symas.com> Sent: Monday, September 12, 2016 6:31:25 PM To: Maria Blagoeva; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint mblagoeva(a)axway.com wrote: > Full_Name: Maria Blagoeva > Version: openldap-2.4.31/debian/build/servers/slapd > OS: docker image with 3.10.0-327.28.2.el7.x86_64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.88.228.86) > > > It seems that openldap as of 2.4.31 is not RFC complaint > (https://www.ietf.org/rfc/rfc2891.txt) due to failure in response when or= dering > match rule is empty while doing server side sorting. Example below: False. OpenLDAP is completely correct and RFC-compliant in its behavior. > empty ordering rule: > > ldapsearch -E sss=3Dcn '(cn=3Dccc*)' cn' (&(objectClass=3DinetOrgPerson)(= cn=3Dccc*))' -H > ldap://IP:389 -b "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "cn=3Daaa1, > ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -x -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # search result > search: 2 > result: 18 Inappropriate matching > text: serverSort control: No ordering rule This result is correct since the cn attribute has no ordering rule defined = in the schema. > > # numResponses: 1 > > caseIgnoreOrderingMatch ordering rule: > > ldapsearch -E sss=3Dcn:caseIgnoreOrderingMatch '(cn=3Dccc*)' cn' > (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))' -H ldap://IP:389 -b > "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "c3D3Daaa1, ou=3DUsers,dc=3Dopens= tack,dc=3Dorg" -x > -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # ccc0, Users, openstack.org > dn: cn=3Dccc0,ou=3DUsers,dc=3Dopenstack,dc=3Dorg > # search result > search: 2 > result: 0 Success > control: 1.2.840.113556.1.4.474 false MAMKAQA=3D > sortResult: (0) Success > > > however the RFC states that the orderingRule is OPTIONAL as below: > > SortKeyList ::=3D SEQUENCE OF SEQUENCE { > attributeType AttributeDescription, > orderingRule [0] MatchingRuleId OPTIONAL, > reverseOrder [1] BOOLEAN DEFAULT FALSE } > > however openldap fails to return entries. The rule is optional in the control itself, but a valid ordering rule must exist somewhere. Since the attribute schema doesn't define one then you mus= t provide one yourself. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/ --_000_AM4PR0101MB1954FD1169119D8DC1DD9008BBFE0AM4PR0101MB1954_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <meta name=3D"Generator" content=3D"Microsoft Exchange Server"> <style></style> </head> <body> <meta content=3D"text/html; charset=3DUTF-8"> <style type=3D"text/css" style=3D"">  </style> <div dir=3D"ltr"> <div id=3D"x_divtagdefaultwrapper" style=3D"font-size:12pt; color:#000000; = background-color:#FFFFFF; font-family:Calibri,Arial,Helvetica,sans-serif"> Hello Howard, This means I need to patch the openldap instance or a default ordering r= ule can be set for that attribute on the fly? Thanks again for your help an= d assistance. BR, Maria </div> <hr tabindex=3D"-1" style=3D"display:inline-block; width:98%"> <div id=3D"x_divRplyFwdMsg" dir=3D"ltr">From: Howard Chu <hyc@= symas.com> Sent: Monday, September 12, 2016 6:31:25 PM To: Maria Blagoeva; openldap-its(a)OpenLDAP.org Subject: Re: (ITS#8501) OpenLdap not RFC 2891 complaint <div> </div> </div> </div> <div class=3D"PlainText">mblagoeva(a)axway.com wrote: > Full_Name: Maria Blagoeva > Version: openldap-2.4.31/debian/build/servers/slapd > OS: docker image with  3.10.0-327.28.2.el7.x86_64 > URL: <a href=3D"ftp://ftp.openldap.org/incoming/">ftp://ftp.openldap.o= rg/incoming/</a> > Submission from: (NULL) (194.88.228.86) > > > It seems that openldap as of 2.4.31 is not RFC complaint > (<a href=3D"https://www.ietf.org/rfc/rfc2891.txt">https://www.ietf.org= /rfc/rfc2891.txt</a>) due to failure in response when ordering > match rule is empty while doing server side sorting. Example below:<br= > False. OpenLDAP is completely correct and RFC-compliant in its behavior.<br= > > empty ordering rule: > > ldapsearch -E sss=3Dcn '(cn=3Dccc*)' cn' (&(objectClass=3DinetOrgP= erson)(cn=3Dccc*))' -H > ldap://IP:389 -b "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D &qu= ot;cn=3Daaa1, > ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -x -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree<b= r> > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # search result > search: 2 > result: 18 Inappropriate matching > text: serverSort control: No ordering rule This result is correct since the cn attribute has no ordering rule defined = in the schema. > > # numResponses: 1 > > caseIgnoreOrderingMatch ordering rule: > > ldapsearch -E sss=3Dcn:caseIgnoreOrderingMatch '(cn=3Dccc*)' cn' > (&(objectClass=3DinetOrgPerson)(cn=3Dccc*))' -H ldap://IP:389 -b<b= r> > "ou=3DUsers,dc=3Dopenstack,dc=3Dorg" -D "c3D3Daaa1, ou= =3DUsers,dc=3Dopenstack,dc=3Dorg" -x > -W > # extended LDIF > # > # LDAPv3 > # base <ou=3DUsers,dc=3Dopenstack,dc=3Dorg> with scope subtree<b= r> > # filter: (cn=3Dccc*) > # requesting: cn (&(objectClass=3DinetOrgPerson)(cn=3Dccc*)) > # with server side sorting control > # > > # ccc0, Users, openstack.org > dn: cn=3Dccc0,ou=3DUsers,dc=3Dopenstack,dc=3Dorg > # search result > search: 2 > result: 0 Success > control: 1.2.840.113556.1.4.474 false MAMKAQA=3D > sortResult: (0) Success > > > however the RFC states that the orderingRule is OPTIONAL as below: > >       SortKeyList ::=3D SEQUENCE OF SEQU= ENCE { >           &nbsp= ;      attributeType   AttributeDescript= ion, >           &nbsp= ;      orderingRule    [0] Matching= RuleId OPTIONAL, >           &nbsp= ;      reverseOrder    [1] BOOLEAN = DEFAULT FALSE } > > however openldap fails to return entries. The rule is optional in the control itself, but a valid ordering rule must = exist somewhere. Since the attribute schema doesn't define one then you mus= t provide one yourself. Closing this ITS. --    -- Howard Chu    CTO, Symas Corp.       &nbs= p;   <a href=3D"http://www.symas.com">http://www.symas.com</a><br= >    Director, Highland Sun     <a href=3D"http= ://highlandsun.com/hyc/">http://highlandsun.com/hyc/</a>    Chief Architect, OpenLDAP  <a href=3D"http://www.openldap= .org/project/">http://www.openldap.org/project/</a> </div> </body> </html> --_000_AM4PR0101MB1954FD1169119D8DC1DD9008BBFE0AM4PR0101MB1954_--

1 0

Re: (ITS#8501) OpenLdap not RFC 2891 complaint
by hyc＠symas.com 12 Sep '16

12 Sep '16

mblagoeva(a)axway.com wrote: > Full_Name: Maria Blagoeva > Version: openldap-2.4.31/debian/build/servers/slapd > OS: docker image with 3.10.0-327.28.2.el7.x86_64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.88.228.86) > > > It seems that openldap as of 2.4.31 is not RFC complaint > (https://www.ietf.org/rfc/rfc2891.txt) due to failure in response when ordering > match rule is empty while doing server side sorting. Example below: False. OpenLDAP is completely correct and RFC-compliant in its behavior. > empty ordering rule: > > ldapsearch -E sss=cn '(cn=ccc*)' cn' (&(objectClass=inetOrgPerson)(cn=ccc*))' -H > ldap://IP:389 -b "ou=Users,dc=openstack,dc=org" -D "cn=aaa1, > ou=Users,dc=openstack,dc=org" -x -W > # extended LDIF > # > # LDAPv3 > # base <ou=Users,dc=openstack,dc=org> with scope subtree > # filter: (cn=ccc*) > # requesting: cn (&(objectClass=inetOrgPerson)(cn=ccc*)) > # with server side sorting control > # > > # search result > search: 2 > result: 18 Inappropriate matching > text: serverSort control: No ordering rule This result is correct since the cn attribute has no ordering rule defined in the schema. > > # numResponses: 1 > > caseIgnoreOrderingMatch ordering rule: > > ldapsearch -E sss=cn:caseIgnoreOrderingMatch '(cn=ccc*)' cn' > (&(objectClass=inetOrgPerson)(cn=ccc*))' -H ldap://IP:389 -b > "ou=Users,dc=openstack,dc=org" -D "c3D3Daaa1, ou=Users,dc=openstack,dc=org" -x > -W > # extended LDIF > # > # LDAPv3 > # base <ou=Users,dc=openstack,dc=org> with scope subtree > # filter: (cn=ccc*) > # requesting: cn (&(objectClass=inetOrgPerson)(cn=ccc*)) > # with server side sorting control > # > > # ccc0, Users, openstack.org > dn: cn=ccc0,ou=Users,dc=openstack,dc=org > # search result > search: 2 > result: 0 Success > control: 1.2.840.113556.1.4.474 false MAMKAQA= > sortResult: (0) Success > > > however the RFC states that the orderingRule is OPTIONAL as below: > > SortKeyList ::= SEQUENCE OF SEQUENCE { > attributeType AttributeDescription, > orderingRule [0] MatchingRuleId OPTIONAL, > reverseOrder [1] BOOLEAN DEFAULT FALSE } > > however openldap fails to return entries. The rule is optional in the control itself, but a valid ordering rule must exist somewhere. Since the attribute schema doesn't define one then you must provide one yourself. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8501) OpenLdap not RFC 2891 complaint
by mblagoeva＠axway.com 12 Sep '16

12 Sep '16

Full_Name: Maria Blagoeva Version: openldap-2.4.31/debian/build/servers/slapd OS: docker image with 3.10.0-327.28.2.el7.x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.88.228.86) It seems that openldap as of 2.4.31 is not RFC complaint (https://www.ietf.org/rfc/rfc2891.txt) due to failure in response when ordering match rule is empty while doing server side sorting. Example below: empty ordering rule: ldapsearch -E sss=cn '(cn=ccc*)' cn' (&(objectClass=inetOrgPerson)(cn=ccc*))' -H ldap://IP:389 -b "ou=Users,dc=openstack,dc=org" -D "cn=aaa1, ou=Users,dc=openstack,dc=org" -x -W # extended LDIF # # LDAPv3 # base <ou=Users,dc=openstack,dc=org> with scope subtree # filter: (cn=ccc*) # requesting: cn (&(objectClass=inetOrgPerson)(cn=ccc*)) # with server side sorting control # # search result search: 2 result: 18 Inappropriate matching text: serverSort control: No ordering rule # numResponses: 1 caseIgnoreOrderingMatch ordering rule: ldapsearch -E sss=cn:caseIgnoreOrderingMatch '(cn=ccc*)' cn' (&(objectClass=inetOrgPerson)(cn=ccc*))' -H ldap://IP:389 -b "ou=Users,dc=openstack,dc=org" -D "c3D3Daaa1, ou=Users,dc=openstack,dc=org" -x -W # extended LDIF # # LDAPv3 # base <ou=Users,dc=openstack,dc=org> with scope subtree # filter: (cn=ccc*) # requesting: cn (&(objectClass=inetOrgPerson)(cn=ccc*)) # with server side sorting control # # ccc0, Users, openstack.org dn: cn=ccc0,ou=Users,dc=openstack,dc=org # search result search: 2 result: 0 Success control: 1.2.840.113556.1.4.474 false MAMKAQA= sortResult: (0) Success however the RFC states that the orderingRule is OPTIONAL as below: SortKeyList ::= SEQUENCE OF SEQUENCE { attributeType AttributeDescription, orderingRule [0] MatchingRuleId OPTIONAL, reverseOrder [1] BOOLEAN DEFAULT FALSE } however openldap fails to return entries.

1 0

(ITS#8500) typo in slapo-ppolicy(5)
by michael＠stroeder.com 12 Sep '16

12 Sep '16

Full_Name: Version: git master OS: - URL: Submission from: (NULL) (85.115.23.42) diff --git a/doc/man/man5/slapo-ppolicy.5 b/doc/man/man5/slapo-ppolicy.5 index 91e0f8f..8306f97 100644 --- a/doc/man/man5/slapo-ppolicy.5 +++ b/doc/man/man5/slapo-ppolicy.5 @@ -104,7 +104,7 @@ object class. The definition of that class is as follows: pwdLockout $ pwdLockoutDuration $ pwdMaxFailure $ pwdFailureCountInterval $ pwdMustChange $ pwdAllowUserChange $ - pwdSafeModify 4 pwdMaxRecordedFailure ) ) + pwdSafeModify $ pwdMaxRecordedFailure ) ) .RE This implementation also provides an additional

1 0

(ITS#8499) fixes documentation typo
by gerardo.santana＠gmail.com 11 Sep '16

11 Sep '16

Full_Name: Gerardo Santana Gomez Garrido Version: 2.4 OS: Mac OS X URL: ftp://ftp.openldap.org/incoming/gerardo-santana-160911.patch Submission from: (NULL) (70.113.112.97) documentation typo

1 0

Re: (ITS#8498) slapadd complains about missing attrs
by quanah＠zimbra.com 09 Sep '16

09 Sep '16

--On Friday, September 09, 2016 11:32 PM +0100 Howard Chu <hyc(a)symas.com> wrote: >> I'd generally expect the warning to be consistently shown (or not). I >> don't think it's valid in the case of "". > > The warning is only printed in single-master replication configs. My point is the warning is invalid, and it was printed on the replica, nothing to do with a master... --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8498) slapadd complains about missing attrs
by hyc＠symas.com 09 Sep '16

09 Sep '16

quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.44+ITS8432 > OS: Linux 3.13 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.52.177) > > > When slapadding a database rooted at "" on a replica, it outputs the following > warning: > > zimbra@zre-ldap014:~$ time ./libexec/zmslapadd > /home/build/ldap.bak.20160909144406 > 57d32561 slapadd: warning, missing attrs entryUUID,entryCSN from entry dn="" > -######## 42.58% eta 01m39s elapsed 01m13s spd 292.8 k/s > > > Interestingly enough, when slapadded to a master with cn=accesslog in play, it > doesn't output anything: > > zimbra@zre-ldap008:~$ ./libexec/zmslapadd /home/build/ldap.bak.20160909144406 > *#################### 100.00% eta none elapsed 02m45s spd 3.1 M/s > Closing DB... > > I'd generally expect the warning to be consistently shown (or not). I don't > think it's valid in the case of "". The warning is only printed in single-master replication configs. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8498) slapadd complains about missing attrs
by quanah＠openldap.org 09 Sep '16

09 Sep '16

Full_Name: Quanah Gibson-Mount Version: 2.4.44+ITS8432 OS: Linux 3.13 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.52.177) When slapadding a database rooted at "" on a replica, it outputs the following warning: zimbra@zre-ldap014:~$ time ./libexec/zmslapadd /home/build/ldap.bak.20160909144406 57d32561 slapadd: warning, missing attrs entryUUID,entryCSN from entry dn="" -######## 42.58% eta 01m39s elapsed 01m13s spd 292.8 k/s Interestingly enough, when slapadded to a master with cn=accesslog in play, it doesn't output anything: zimbra@zre-ldap008:~$ ./libexec/zmslapadd /home/build/ldap.bak.20160909144406 *#################### 100.00% eta none elapsed 02m45s spd 3.1 M/s Closing DB... I'd generally expect the warning to be consistently shown (or not). I don't think it's valid in the case of "".

1 0

Re: (ITS#8490) changes not written to accesslog, causing replicas to loop syncing
by quanah＠zimbra.com 09 Sep '16

09 Sep '16

--On Thursday, September 01, 2016 9:50 AM +0000 quanah(a)zimbra.com wrote: I finally was able to reproduce this in a new lab of servers I set up. The easiest way to reproduce this problem is to have 2 masters, and 6 replicas. 3 replicas pointing to each master. In my case, I have ldap07 through ldap014. 07/08 are the masters. Odd number replicas (9,11,13) point to 07, even number replicas point to 08 (10, 12, 14). What we find based on the logs is two MODRDNs coming in at approximately the same time: Sep 9 15:29:40 zre-ldap007 slapd[13794]: conn=1389 op=3179 MODRDN dn="uid=user.345,ou=people,dc=zre-ldap007,dc=eng,dc=zimbra,dc=com" Sep 9 15:29:40 zre-ldap007 slapd[13794]: conn=1120 op=2919 MODRDN dn="uid=user.250404,ou=people,dc=zre-ldap007,dc=eng,dc=zimbra,dc=com" Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_queue_csn: queueing 0xa046e80 20160909192940.809794Z#000000#001#000000 Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_queue_csn: queueing 0xa046a40 20160909192940.809794Z#000000#001#000000 Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_queue_csn: queueing 0xa07b740 20160909192940.810205Z#000000#001#000000 Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_graduate_commit_csn: removing 0xa046a40 20160909192940.809794Z#000000#001#000000 Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_queue_csn: queueing 0xa07bdc0 20160909192940.810205Z#000000#001#000000 Sep 9 15:29:40 zre-ldap007 slapd[13794]: conn=1389 op=3179 RESULT tag=109 err=0 etime=0.001278 text= Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_graduate_commit_csn: removing 0xa046e80 20160909192940.809794Z#000000#001#000000 Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_graduate_commit_csn: removing 0xa07bdc0 20160909192940.810205Z#000000#001#000000 Sep 9 15:29:40 zre-ldap007 slapd[13794]: conn=1120 op=2919 RESULT tag=109 err=0 etime=0.001597 text= Sep 9 15:29:40 zre-ldap007 slapd[13794]: slap_graduate_commit_csn: removing 0xa07b740 20160909192940.810205Z#000000#001#000000 But in the accesslog, we only find the one MODRDN logged: dn: reqStart=20160909192940.809465Z,cn=accesslog objectClass: auditModRDN structuralObjectClass: auditModRDN reqStart: 20160909192940.809465Z reqEnd: 20160909192940.810148Z reqType: modrdn reqSession: 1389 reqAuthzID: uid=zimbra,cn=admins,cn=zimbra reqDN: uid=user.345,ou=people,dc=zre-ldap007,dc=eng,dc=zimbra,dc=com reqResult: 0 reqMod: entryCSN:= 20160909192940.809794Z#000000#001#000000 reqMod: modifiersName:= uid=zimbra,cn=admins,cn=zimbra reqMod: modifyTimestamp:= 20160909192940Z reqNewRDN: uid=renamed-user.345 reqDeleteOldRDN: TRUE reqEntryUUID: c63b246a-0a20-1036-82cb-a772509d7f53 entryUUID: 7f740e6c-0b0f-1036-8c00-b1b4f08a59fc creatorsName: cn=config createTimestamp: 20160909192940Z entryCSN: 20160909192940.809794Z#000000#001#000000 modifiersName: cn=config modifyTimestamp: 20160909192940Z Possibly a DN collision in writing the entry to the accesslog DB that causes the second OP to get lost. Likely the same root issue as ITS#8493. --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8497) userPassword attribute in password policy
by quanah＠zimbra.com 09 Sep '16

09 Sep '16

--On Friday, September 09, 2016 3:38 PM +0000 sagar437(a)gmail.com wrote: > Full_Name: Sagar Bhattarai > Version: 2.4.40-9 > OS: CentOS Linux release 7.2.1511 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (71.174.102.218) The ITS system is for reporting bugs, not asking usage questions. Questions should be sent to the openldap-technical(a)openldap.org email list. This ITS will be closed. --Quanah -- Quanah Gibson-Mount

1 0

(ITS#8497) userPassword attribute in password policy
by sagar437＠gmail.com 09 Sep '16

09 Sep '16

Full_Name: Sagar Bhattarai Version: 2.4.40-9 OS: CentOS Linux release 7.2.1511 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (71.174.102.218) I am trying to push the password policy with the following content of ldif file. dn: cn=passwordDefault4,ou=policy,dc=vivox,dc=com objectClass: pwdPolicy objectclass: device objectClass: top cn: passwordDefault4 pwdAttribu%3: userPassword pwdCheckQuality: 0 pwdMinAge: 0 pwdMaxAge: 0 pwdMinLength: 5 pwdInHistory: 5 pwdMaxFailure: 3 pwdFailureCountInterval: 0 pwdLockout: TRUE pwdLockoutDuration: 0 pwdAllowUserChange: TRUE pwdExpireWarning: 0 pwdGraceAuthNLimit: 0 pwdMustChange: TRUE pwdSafeModify: TRUE I keep getting the following error when trying to add the above policy. ldap_add: Invalid syntax (21) additional info: pwdAttribute: value #0 invalid per syntax However id i change the pwdattribute vae frfrom "userPassword" to OID number i am able to add this policy but none of these policies work on my systems.

1 0

Re: (ITS#8495) slaped service unable to restart
by quanah＠zimbra.com 09 Sep '16

09 Sep '16

--On Friday, September 09, 2016 8:34 AM +0000 ritesh.s(a)litmusit.com wrote: > Full_Name: Ritesh Supatkar > Version: Openldap-2.4.40 > OS: Rhel 7.2 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (103.14.160.193) This looks like an issue with systemd and Redhat. I advise contacting Redhat support. --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8496) Error while launching LDAP
by quanah＠zimbra.com 09 Sep '16

09 Sep '16

--On Friday, September 09, 2016 8:56 AM +0000 kshtijdasture(a)gmail.com wrote: > Full_Name: Kshitij Dasture > Version: 2.4.41 > OS: Windos > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (203.99.214.118) > > > I am starting the LAP by the below command.Below is the show stopper for > the execution of Start command. > > > -bash-4.3$ ./slapd -f slapd.conf -d 1 > ldap.log The ITS system is for reporting bugs, not usage questions. If you have questions on how to use OpenLDAP correctly, please use the openldap-technical list. This ITS will be closed. --Quanah -- Quanah Gibson-Mount

1 0

(ITS#8496) Error while launching LDAP
by kshtijdasture＠gmail.com 09 Sep '16

09 Sep '16

Full_Name: Kshitij Dasture Version: 2.4.41 OS: Windos URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (203.99.214.118) I am starting the LAP by the below command.Below is the show stopper for the execution of Start command. -bash-4.3$ ./slapd -f slapd.conf -d 1 > ldap.log . . . 57d25b6b backend_startup_one: starting "dc=gnf,dc=telekom,dc=m%m" 57d25b6b bdb_db_open: database "dc=gnf,dc=telekom,dc=com": database already in use. 57d25b6b backend_startup_one (type=bdb, suffix="dc=gnf,dc=telekom,dc=com"): bi_db_open failed! (-1) 57d25b6b slapd shutdown: initiated 57d25b6b ====> bdb_cache_release_all 57d25b6b slapd destroy: freeing system resources. 57d25b6b slapd stopped. -bash-4.3$

1 0

(ITS#8495) slaped service unable to restart
by ritesh.s＠litmusit.com 09 Sep '16

09 Sep '16

Full_Name: Ritesh Supatkar Version: Openldap-2.4.40 OS: Rhel 7.2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (103.14.160.193) [bbpsadm@hydbbpsldp01 ~]$ sudo systemctl status slapd ● slapd.service - OpenLDAP Server Daemon Loaded: loaded (/usr/lib/systemd/system/slapd.service; enabled; vendor preset: disabled) Active: active (running) since Thu 20169-9-08 18:18:20 IST; 18h ago Docs: man:slapd man:slapd-config man:slapd-hdb man:slapd-mdb file:///usr/share/doc/openldap-servers/guide.html Process: 44574 ExecStart=/usr/sbin/slapd -u ldap -h ${SLAPD_URLS} $SLAPD_OPTIONS (code=exited, status=0/SUCCESS) Process: 44531 ExecStartPre=/usr/libexec/openldap/check-config.sh (code=exited, status=0/SUCCESS) Main PID: 44575 (slapd) CGroup: /system.slice/slapd.service └6%6#9472;44575 /usr/sbin/slapd -u ldap -h ldapi:/// ldap:/// Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: <= bdb_equality_candidates: (uid) not indexed Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: conn=1020 op=1 SEARCH RESULT tag=101 err=0 nentries=1 text= Sep 09 12:34:45 hydbbpsl0101 slapd[47575]: conn=1020 op=2 SRCH base="dc=npci,dc=co,dc=in" scope=2 deref=0 filter="(&(memberUid=bbpsadm)(objectClass=posixGr...ber=0))))" Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: conn=1020 op=2 SRCH attr=objectClass cn userPassword gidNumber modifyTimestamp modifyTimestamp Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: <= bdb_equality_candidates: (memberUid) not indexed Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: conn=1020 op=2 SEARCH RESULT tag=101 err=0 nentries=0 text= Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: conn=1020 op=3 SRCH base="dc=npci,dc=co,dc=in" scope=2 deref=0 filter="(&(gidNumber=1001)(objectClass=posixGroup...ber=0))))" Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: conn=1020 op=3 SRCH attr=objectClass cn userPassword gidNumber memberuid modifyTimestamp modifyTimestamp Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: <= bdb_equality_candidates: (gidNumber) not indexed Sep 09 12:34:45 hydbbpsldp01 slapd[44575]: conn=1020 op=3 SEARCH RESULT tag=101 err=0 nentries=2 text= Hint: Some lines were ellipsized, use -l to show in full. [bbpsadm@hydbbpsldp01 ~]$ [bbpsadm@hydbbpsldp01 ~]$ sudo systemctl restart slapd Error getting authority: Error initializing authority: GDBus.Error:org.freedesktop.DBus.Error.AccessDenied: An SELinux policy prevents thisesender from sending this message to this recipient, 0 matched rules; type="method_call", sender="(null)" (inactive) interface="org.freedesktop.DBus" member="Hello" error name="(unset)" requested_reply="0" destination="org.freedesktop.DBus" (bus) (g-dbus-error-quark, 9) [bbpsadm@hydbbpsldp01 ~]$

1 0

(ITS#8494) doc: Rename 'TLSEphemeralDHParamFile' to 'TLSDHParamFile'
by georg＠riseup.net 08 Sep '16

08 Sep '16

Full_Name: Georg Faerber Version: 2.4 OS: Debian URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (91.64.110.191) http://www.openldap.org/doc/admin24/tls.html mentions directive 'TLSEphemeralDHParamFile' whereas slapd.conf(5) mentions 'TLSDHParamFile'. (See [1] as well). [1] http://www.openldap.org/lists/openldap-technical/201309/msg00094.html

1 0

RE: (ITS#8374) LDAP_OPT_X_TLS_REQUIRE_CERT handling differences between ldaps:// and STARTTLS
by dog＠pavlov.com 07 Sep '16

07 Sep '16

Actually, ignore me this is still connecting when it should fail. Don't close. ;)

1 0

RE: (ITS#8374) LDAP_OPT_X_TLS_REQUIRE_CERT handling differences between ldaps:// and STARTTLS
by dog＠pavlov.com 07 Sep '16

07 Sep '16

This seems to have been resolved at some point in the past in one of the updates: now functions as expected. Fine to close from my perspective.

1 0

Re: (ITS#8493) Under heavy modrdn load, masters desync
by quanah＠zimbra.com 03 Sep '16

03 Sep '16

--On Saturday, September 03, 2016 4:51 PM +0000 quanah(a)zimbra.com wrote: > --On Saturday, September 03, 2016 6:15 AM +0000 quanah(a)openldap.org wrote: > >> Full_Name: Quanah Gibson-Mount >> Version: 2.4.44+ITS8432 >> OS: Linux 2.6 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (75.111.52.177) >> >> >> Trying to reproduce another ITS, I discovered a new bug. When doing >> MODRDN ops on one master, the other master keeps going out of sync. >> Specifically: >> >> Sep 3 01:12:17 zre-ldap002 slapd[29206]: syncrepl_message_to_op: rid=100 >> be_modrdn uid=user.924,ou=people,dc=zre-ldap002,dc=eng,dc=zimbra,dc=com >> (32) Sep 3 01:12:17 zre-ldap002 slapd[29206]: do_syncrep2: rid=100 >> delta-sync lost sync on (reqStart=20160903051215.747829Z,cn=accesslog), >> switching to REFRESH > > > Note that this master also has a replica. The replica never rejected a > single one of these MODRDNs coming from this master. Which means that > either: > > a) The data on the master spontaneously corrupted at some point > > or > > b) The master wrote the MODRDNs to the accesslog, which the replica > picked up, but did not itself make the MODRDN changes to its database. > > In the end, of the 50,000 MODRDNs it was processing, it threw an error 32 > for 441 of them. After the master that was not accepting direct writes re-sync'd with the master accepting writes, it still had 403/50000 entries wrong. So did its replica. So the master isn't writing the changes to the accesslog. So it's option c. The master rejects a valid op, never sync's correctly, and in the end 2/3rds of my servers have invalid databases. I see zero indication that using a sessionlog works around <http://www.openldap.org/its/index.cgi/?findid=8125> at all. I still end up with missed entries even with everything *in* the sessionlog. --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8493) Under heavy modrdn load, masters desync
by quanah＠zimbra.com 03 Sep '16

03 Sep '16

--On Saturday, September 03, 2016 6:15 AM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.44+ITS8432 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.52.177) > > > Trying to reproduce another ITS, I discovered a new bug. When doing > MODRDN ops on one master, the other master keeps going out of sync. > Specifically: > > Sep 3 01:12:17 zre-ldap002 slapd[29206]: syncrepl_message_to_op: rid=100 > be_modrdn uid=user.924,ou=people,dc=zre-ldap002,dc=eng,dc=zimbra,dc=com > (32) Sep 3 01:12:17 zre-ldap002 slapd[29206]: do_syncrep2: rid=100 > delta-sync lost sync on (reqStart=20160903051215.747829Z,cn=accesslog), > switching to REFRESH Note that this master also has a replica. The replica never rejected a single one of these MODRDNs coming from this master. Which means that either: a) The data on the master spontaneously corrupted at some point or b) The master wrote the MODRDNs to the accesslog, which the replica picked up, but did not itself make the MODRDN changes to its database. In the end, of the 50,000 MODRDNs it was processing, it threw an error 32 for 441 of them. --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8486) Syncprov sessionlog is inefficient, kills perf
by quanah＠zimbra.com 03 Sep '16

03 Sep '16

--On Monday, August 29, 2016 9:16 PM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.44 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.52.177) In doing testing for ITS#8491, I encountered this problem in my lab. Any time a single replica is parsing the session log, all other operations on the server come to a complete halt. I.e., slapd does *nothing* other than handle the sessionlog query. This seems like a fatal problem. --Quanah -- Quanah Gibson-Mount

1 0

(ITS#8493) Under heavy modrdn load, masters desync
by quanah＠openldap.org 03 Sep '16

03 Sep '16

Full_Name: Quanah Gibson-Mount Version: 2.4.44+ITS8432 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.52.177) Trying to reproduce another ITS, I discovered a new bug. When doing MODRDN ops on one master, the other master keeps going out of sync. Specifically: Sep 3 01:12:17 zre-ldap002 slapd[29206]: syncrepl_message_to_op: rid=100 be_modrdn uid=user.924,ou=people,dc=zre-ldap002,dc=eng,dc=zimbra,dc=com (32) Sep 3 01:12:17 zre-ldap002 slapd[29206]: do_syncrep2: rid=100 delta-sync lost sync on (reqStart=20160903051215.747829Z,cn=accesslog), switching to REFRESH

1 0

RE: (ITS#8492) signal 11 in ldap_new_connection
by quanah＠zimbra.com 02 Sep '16

02 Sep '16

--On Friday, September 02, 2016 7:35 AM +0000 htjain(a)sandvine.com wrote: > After some more debugging looks like ld->ldc->ldc_options.ldo_defludp > varia= ble is causing this issue, its null when its passed to > ldap_new_connection. How does it populated? Please confirm your encounter the issue with a current release. 2.4.40 and 2.4.41 both had significant issues. --Quanah -- Quanah Gibson-Mount

1 0

RE: (ITS#8492) signal 11 in ldap_new_connection
by htjain＠sandvine.com 02 Sep '16

02 Sep '16

After some more debugging looks like ld->ldc->ldc_options.ldo_defludp varia= ble is causing this issue, its null when its passed to ldap_new_connection. How does it populated? (gdb) p ld->ldc->ldc_options $13 =3D {ldo_valid =3D 255, ldo_debug =3D 0, ldo_version =3D 3, ldo_deref = =3D 0, ldo_timelimit =3D 0, ldo_sizelimit =3D 0, ldo_tm_api =3D {tv_sec =3D= -1, tv_usec =3D 0}, ldo_tm_net =3D {tv_sec =3D 5, tv_usec =3D 0}, ldo_defl= udp =3D 0x0, ldo_defport =3D 389, ldo_defbase =3D 0x0, ldo_defbinddn =3D 0x0, ldo_keepalive_idle =3D 0, ldo= _keepalive_probes =3D 0, ldo_keepalive_interval =3D 0, ldo_refhoplimit =3D = 5, ldo_sctrls =3D 0x0, ldo_cctrls =3D 0x0, ldo_rebind_proc =3D 0x0, ldo_reb= ind_params =3D 0x0, ldo_nextref_proc =3D 0x0, ldo_nextref_params =3D 0x0, ldo_urllist_proc = =3D 0x0, ldo_urllist_params =3D 0x0, ldo_conn_cbs =3D 0x0, ldo_booleans =3D= 34, ldo_peer =3D 0x0, ldo_cldapdn =3D 0x0, ldo_is_udp =3D 0, ldo_tls_ctx = =3D 0x0, ldo_tls_connect_cb =3D 0x0, ldo_tls_connect_arg =3D 0x0, ldo_tls_info =3D= {lt_certfile =3D 0x0, lt_keyfile =3D 0x0, lt_dhfile =3D 0x0, lt_cacertfile= =3D 0x0, lt_cacertdir =3D 0x0, lt_ciphersuite =3D 0x0, lt_crlfile =3D 0x0,= lt_randfile =3D 0x0, lt_protocol_min =3D 0}, ldo_tls_mode =3D 0, ldo_tls_require_cert =3D 2,= ldo_tls_impl =3D 0, ldo_tls_crlcheck =3D 0, ldo_def_sasl_mech =3D 0x0, ldo= _def_sasl_realm =3D 0x0, ldo_def_sasl_authcid =3D 0x0, ldo_def_sasl_authzid= =3D 0x0, ldo_sasl_secprops =3D {min_ssf =3D 0, max_ssf =3D 2147483647, maxbufsize = =3D 16777215, security_flags =3D 17, property_names =3D 0x0, property_value= s =3D 0x0}} (gdb) p ld->ldc->ldc_options.ldo_defludp $14 =3D (LDAPURLDesc *) 0x0 -----Original Message----- From: openldap-its(a)OpenLDAP.org [mailto:openldap-its@OpenLDAP.org]=20 Sent: Friday, September 2, 2016 11:50 AM To: Hemant Jain Subject: Re: (ITS#8492) signal 11 in ldap_new_connection *** THIS IS AN AUTOMATICALLY GENERATED REPLY *** Thanks for your report to the OpenLDAP Issue Tracking System. Your report = has been assigned the tracking number ITS#8492. One of our support engineers will look at your report in due course. Note that this may take some time because our support engineers are volunte= ers. They only work on OpenLDAP when they have spare time. If you need to provide additional information in regards to your issue repo= rt, you may do so by replying to this message. Note that any mail sent to = openldap-its(a)openldap.org with (ITS#8492) in the subject will automatically= be attached to the issue report. mailto:openldap-its@openldap.org?subject=3D(ITS#8492) You may follow the progress of this report by loading the following URL in = a web browser: http://www.OpenLDAP.org/its/index.cgi?findid=3D8492 Please remember to retain your issue tracking number (ITS#8492) on any furt= her messages you send to us regarding this report. If you don't then you'l= l just waste our time and yours because we won't be able to properly track = the report. Please note that the Issue Tracking System is not intended to be used to se= ek help in the proper use of OpenLDAP Software. Such requests will be closed. OpenLDAP Software is user supported. http://www.OpenLDAP.org/support/ -------------- Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved.

1 0

(ITS#8492) signal 11 in ldap_new_connection
by htjain＠sandvine.com 02 Sep '16

02 Sep '16

Full_Name: hemant jain Version: openldap-2.4.40-8 OS: RHEL 7.2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (123.63.202.166) Hi, We are facing a strange core dump in ldap_new_connection function while trying to bind a stopped Openldap server. following is the stack trace: #0 0x00007f9336898f53 in ldap_new_connection (ld=ld@entry=0x7f931f43cd60, srvlist=srvlist@entry=0x7f931f48b298, use_ldsb=use_ldsb@entry=1, connect=connect@entry=1, bind=bind@entry=0x0, m_req=m_req@entry=0, m_res=m_res@entry=0) at request.c:481 #1 0x00007f933688530f in ldap_open_defconn (ld=ld@entry=0x7f931f43cd60) at open.c:41 #2 0x00007f933689a248 in ldap_send_initial_request (ld=ld@entry=0x7f931f43cd60, msgtype=msgtype@entry=96, dn=dn@entry=0x7f931f827718 "cn=Manager,dc=test,dc=com", ber=ber@entry=0x7f931f457150, msgid=msgid@entry=1) at request.c:130 #3 0x00007f933688f178 in ldap_sasl_bind (ld=0x7f931f43cd60, dn=0x7f931f827718 "cn=Manager,dc=test,dc=com", mechanism=<optimized out>, cred=0x7f931f4296b0, sctrls=0x0, cctrls=<optimized out>, msgidp=0x7f92243ed8dc) at sasl.c:148 #4 0x00007f9340f571da in ldap::Connection::Bind (this=this@entry=0x7f931c043478, msg_id=@0x7f92243ed98c: -1) at support/ldapconnection.cpp I have installed corresponding debug rpm for Openldap library. Following the source info. core dump is coming at line 481 of request.c file.A%A for ( srvp = srvlist; *srvp != NULL; srvp = &(*srvp)->lud_next ) // Here srvp is NULL so *srvp dumping core. if ( connect ) { LDAPURLDesc **srvp, *srv = NULL; async = LDAP_BOOL_GET( &ld->ld_options, LDAP_BOOL_CONNECT_ASYNC ); for ( srvp = srvlist; *srvp != NULL; srvp = &(*srvp)->lud_next ) { int rc; rc = ldap_int_open_connection( ld, lc, *srvp, async ); if ( rc != -1 ) { srv = *srvp; if ( ld->ld_urllist_proc && ( !async || rc != -2 ) ) { ld->ld_urllist_proc( ld, srvlist, srvp, ld->ld_urllist_params ); } break; } } Any idea how to fix it ?

1 0

(ITS#8491) slapdelete test needed
by quanah＠openldap.org 02 Sep '16

02 Sep '16

Full_Name: Quanah Gibson-Mount Version: RE25 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.52.177) We need to add a test script for slapdelete for 2.5 alpha

1 0

Re: (ITS#8490) changes not written to accesslog, causing replicas to loop syncing
by quanah＠zimbra.com 01 Sep '16

01 Sep '16

--On Thursday, September 01, 2016 8:58 AM +0000 quanah(a)zimbra.com wrote: > In looking at the LDAP accesslog, what I see is that what should have > been a modRDN op was stored in the accesslog as a MOD op (the one I > noted before). This seems particularly bizarre, because ldap01 should > have rejected this change as well. It appears we may have a problem > where the accesslog DB is updated, but then the change got rejected by > the unique overlay. Ok, actually the server (ldap01) accepted the change (renamed the entry). Looking at the database, it was correctly done as a MODRDN to ldap01, but /written/ as a MOD to accesslog?! --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8490) changes not written to accesslog, causing replicas to loop syncing
by quanah＠zimbra.com 01 Sep '16

01 Sep '16

--On Thursday, September 01, 2016 8:05 AM +0000 quanah(a)zimbra.com wrote: > --On Thursday, September 01, 2016 7:52 AM +0000 quanah(a)openldap.org wrote: > >> Full_Name: Quanah Gibson-Mount >> Version: OpenLDAP 2.4.44 >> OS: Linux 2.6 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (75.111.52.177) >> >> >> In a 2-node MMR setup. Node 1 is getting a lot of write traffic. Both >> node 1 and node 2 have 3 replicas each. At some point, a change is >> received by node 1, which writes the change to its accesslog DB and its >> primary DB. It's 3 replicas are all correctly updated. MMR node 2 >> receives the change, updates its primary DB, but *fails* to write the >> change to the accesslog DB. However, it *does* write the CSN update to >> the accesslog DB successfully. This causes all of its replicas to also >> update their CSN. Then a change comes in triggering a constraint >> violation on the replicas, but fully accepted by their master. > > So the above summary is incorrect. While 3 replicas did go out of > sync... 2 belonged to the primary master (node1), and 1 belonged to the > secondary master (node 2). So really, 4 systems didn't log the change > (MMR node 2, ldap05, ldap07, ldap09). Ok, so that's not correct either. I now have the correct topography: ldap01 has the following replicas: ldap02, ldap05, ldap07, ldap09 ldap02 has the following replicas: ldap01, ldap06, ldap08, ldap10 So the replicas of ldap01 received the change and rejected it. ldap02 just skipped writing the entry to the accesslog, and as a result, none of its replicas ever got the change, and thus they never hit the failure issue of err 19, but they all are now lacking this modification entirely. I would note that every server was loaded today from the same ldap backup, so they were all perfectly in sync. In looking at the LDAP accesslog, what I see is that what should have been a modRDN op was stored in the accesslog as a MOD op (the one I noted before). This seems particularly bizarre, because ldap01 should have rejected this change as well. It appears we may have a problem where the accesslog DB is updated, but then the change got rejected by the unique overlay. --Quanah -- Quanah Gibson-Mount

1 0

Re: (ITS#8490) changes not written to accesslog, causing replicas to loop syncing
by quanah＠zimbra.com 01 Sep '16

01 Sep '16

--On Thursday, September 01, 2016 7:52 AM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: OpenLDAP 2.4.44 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.52.177) > > > In a 2-node MMR setup. Node 1 is getting a lot of write traffic. Both > node 1 and node 2 have 3 replicas each. At some point, a change is > received by node 1, which writes the change to its accesslog DB and its > primary DB. It's 3 replicas are all correctly updated. MMR node 2 > receives the change, updates its primary DB, but *fails* to write the > change to the accesslog DB. However, it *does* write the CSN update to > the accesslog DB successfully. This causes all of its replicas to also > update their CSN. Then a change comes in triggering a constraint > violation on the replicas, but fully accepted by their master. So the above summary is incorrect. While 3 replicas did go out of sync... 2 belonged to the primary master (node1), and 1 belonged to the secondary master (node 2). So really, 4 systems didn't log the change (MMR node 2, ldap05, ldap07, ldap09). So I have no explanation at the moment why ldap06, ldap08, and ldap10 were not affected. --Quanah -- Quanah Gibson-Mount

1 0

(ITS#8490) changes not written to accesslog, causing replicas to loop syncing
by quanah＠openldap.org 01 Sep '16

01 Sep '16

Full_Name: Quanah Gibson-Mount Version: OpenLDAP 2.4.44 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.52.177) In a 2-node MMR setup. Node 1 is getting a lot of write traffic. Both node 1 and node 2 have 3 replicas each. At some point, a change is received by node 1, which writes the change to its accesslog DB and its primary DB. It's 3 replicas are all correctly updated. MMR node 2 receives the change, updates its primary DB, but *fails* to write the change to the accesslog DB. However, it *does* write the CSN update to the accesslog DB successfully. This causes all of its replicas to also update their CSN. Then a change comes in triggering a constraint violation on the replicas, but fully accepted by their master. Node 1 records the following change in its accesslog DB: dn: reqStart=20160901025119.904382Z,cn=accesslog objectClass: auditModify structuralObctctClass: auditModify reqStart: 20160901025119.904382Z reqEnd: 20160901025119.906625Z reqType: modify reqSession: 7780 reqAuthzID: uid=zimbra,cn=admins,cn=zimbra reqDN: uid=renamed-trackp,ou=people,dc=xxxxxxxx,dc=co,dc=za reqResult: 0 reqM%3: zimbraMailWhitelistMaxNumEntries:= 0 reqMod: zimbraPrefMailLocalDeliveryDisabled:= FALSE reqMod: zimbraHideInGal:= TRUE reqMod: userPassword:= {SSHA} reqMod: mail:= renamed-trackp(a)xxxxxxx.co.za reqMod: displayName:= trackp(a)xxxxxxxx.co.za reodod: objectClass:= inetOrgPerson reqMod: objectClass:= zimbraAccount reqMod: objectClass:= amavisAccount reqMod: zimbraMailHost:= store328-dc01.cm.xxxxxxxxxxx reqMod: zimbraPasswordMustChange:= FALSE reqMod: cn:= trackp(a)xxxxxxxxxxxxx.co.za reqMod: zimbraPasswordModifiedTime:= 20160121142933Z reqMod: zimbraMailStatus:= enabled reqMod: zimbraIsDelegatedAdminAccount:= FALSE reqMod: zimbraPrefOutOfOfficeReplyEnabled:= FALSE reqMod: zimbraMailBlacklistMaxNumEntries:= 0 reqMod: zimbraAccountStatus:= locked reqMod: zimbraPrefTimeZoneId:= Africa/Harare reqMod: zimbraId:= 403c7b67-eea0-4315-9b10-a2853c6cf70b reqMod: zimbraMailTransport:= lmtp:store328-dc01.cm.xxxxxxxxxx:7025 reqMod: sn:= trackp(a)xxxxxxxxxxx.co.za reqMod: zimbraCOSId:= 1daeb979-365f-46b7-bb3f-2ad46f4576c1 reqMod: zimbraCreateTimestamp:= 20160121142933Z reqMod: zimbraMailDeliveryAddress:= renamed-trackp(a)xxxxxxx.co.za reqMod: entryCSN:= 20160901025119.904589Z#000000#002#000000 reqMod: modifiersName:= uid=zimbra,cn=admins,cn=zimbra reqMod: modifyTimestamp:= 20160901025119Z reqEntryUUID: 2293af38-5497-1035-8bd6-1d34a55b9433 entryUUID: b46d8728-043a-1036-8ab7-9bf2e127fe3b creatorsName: cn=config createTimestamp: 20160901025119Z entryCSN: 20160901025119.904589Z#000000#002#000000 modifiersName: cn=config modifyTimestamp: 20160901025119Z On the other master, we find in the accesslog two sequential CSN updates: dn: reqStart=20160901025119.904218Z,cn=accesslog objectClass: auditModify structuralObjectClass: auditModify reqStart: 20160901025119.904218Z reqEnd: 20160901025119.905035Z reqType: modify reqSession: 201 reqAuthzID: cn=config reqDN: reqResult: 0 reqMod: contextCSN:= 20160901025119.902062Z#000000#002#000000 reqMod: contextCSN:= 20160901015920.689225Z#000000#003#000000 reqEntryUUID: 78143fa2-7af5-48cc-8d5e-6e15f68dd3b2 entryUUID: b46d48a8-043a-1036-81a5-b3dead5e3175 creatorsName: cn=config createTimestamp: 20160901025119Z entryCSN: 20160901025119.902062Z#000000#002#000000 modifiersName: cn=config modifyTimestamp: 20160901025119Z dn: reqStart=20160901025119.908257Z,cn=accesslog objectClass: auditModify structuralObjectClass: auditModify reqStart: 20160901025119.908257Z reqEnd: 20160901025119.908439Z reqType: modify reqSession: 201 reqAuthzID: cn=config reqDN: reqResult: 0 reqMod: contextCSN:= 20160901025119.904589Z#000000#002#000000 reqMod: contextCSN:= 20160901015920.689225Z#000000#003#000000 reqEntryUUID: 78143fa2-7af5-48cc-8d5e-6e15f68dd3b2 entryUUID: b46dcdaa-043a-1036-81a6-b3dead5e3175 creatorsName: cn=config createTimestamp: 20160901025119Z entryCSN: 20160901025119.904589Z#000000#002#000000 modifiersName: cn=config modifyTimestamp: 20160901025119Z Note the CSN update in the second case is for 20160901025119.904589Z. Note that this was the CSN of the change that in node 1 was a real update. The atribute with a constraint in this case is ZimbraID, which is colliding with the zimbraID of the original entry it was renamed from.

1 0

Re: (ITS#8489) LMDB: C_EOF not cleared on cursor in mdb_cursor_put
by hyc＠symas.com 01 Sep '16

01 Sep '16

juerg.bircher(a)gmail.com wrote: > Full_Name: Juerg Bircher > Version: LMDB (master) > OS: MacOS / Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (178.82.36.179) > > > > The function mdb_cursor_put does not clear possibly set C_EOF flag on cursor. > Therefore an successive mdb_cursor_get call using MDB_NEXT may get an > MDB_NOTFOUND result even thought not having put the new value at the end of the > database. > > The following example should illustrate the issue: Thanks for the report, fixed now in mdb.master -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#8489) LMDB: C_EOF not cleared on cursor in mdb_cursor_put
by juerg.bircher＠gmail.com 31 Aug '16

31 Aug '16

Full_Name: Juerg Bircher Version: LMDB (master) OS: MacOS / Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (178.82.36.179) The function mdb_cursor_put does not clear possibly set C_EOF flag on cursor. Therefore an successive mdb_cursor_get call using MDB_NEXT may get an MDB_NOTFOUND result even thought not having put the new value at the end of the database. The following example should illustrate the issue: Assume an database having two committed values "0" and "5". Begin a new transaction and open an cursor for the next steps: step 1 put 1 get next => returns 5 ==> ok step 2 put 6 get next 3E r returns MDB_NOTFOUND as at end of the database (sets C_EOF on cursor) ==> ok step 3 put 2 get next => should return 5 but due to the set C_EOF flag returns MDB_NOTFOUND ==> failure The fix could be to clear C_EOF on any successful mdb_cursor_put calls. The following code can be used to verify the issue. #include "lmdb.h" #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <string.h> static char *env_name = "/tmp/testdb"; static char *db_name = "tt"22; #define MAX_MAP_SIZ (1024 * 1024) #define E(expr) CHECK((rc = (expr)) == MDB_SUCCESS, #expr) #define RES(err, expr) ((rc = expr) == (err) || (CHECK(!rc, #expr), 0)) #define CHECK(test, msg) ((test) ? (void)0 : ((void)fprintf%2tdtderr, \ "%s:%d: %s: %s\n", __FILE__, __LINE__, msg, mdb_strerror(rc)), abort())) static MDB_env *env; static MDB_dbi dbi; static void createTestValue(int num, MDB_val *key, MDB_val *val) { static char buf[16]; sprintf(buf, "%03d", num); key->mv_data = (void *)buf; key->mv_size = strlen((const char *)buf); val->mv_data = key->mv_data; val->mv_size = key->mv_size; } static void setup() { int rc; MDB_txn *txn; MDB_cursor *cursor; MDB_val key, val; E(mdb_env_create(&env)); E(mdb_env_set_maxreaders(env, 2)); E(mdb_env_set_maxdbs(env, 10)); E(mdb_env_set_mapsize(env, MAX_MAP_SIZ)); E(mdb_env_open(env, env_name, 0, 0664)); E(mdb_txn_begin(env, NULL,%0, &txn)); E(mdb_dbi_open(txn, db_name, MDB_CREATE, &dbi)); E(mdb_cursor_open(txn, dbi, &cursor)); createTestValue(0, &key, &val); E(mdb_cursor_put(cursor, &key, &val, 0)); createTestValue(5, &key, &val); E(mdb_cursor_put(cursor, &key, &val, 0)); mdb_cursor_close(cursor); E(mdb_txn_commit(txn)); } static int step1(MDB_cursor *cursor) { int rc; MDB_val key, val, rkey, rval; createTestValue(1, &key, &val); E(mdb_cursor_put(cursor, &key, &val, MDB_NOOVERWRITE)); E(mdb_cursor_get(cursor, &rkey, &rval, MDB_NEXT)); // ==> returns 5 return 0; } static int step2(MDB_cursor *cursor) { int rc; MDB_val key, val, rkey, rval; createTestValue(6, &key, &val); E(mdb_cursor_put(cursor, &key, &val, MDB_NOOVERWRITE)); rc = mdb_cursor_get(cursor, &rkey, &rval, MDB_NEXT); // ==> return nothing as 6 is the last one if (rc != MDB_NOTFOUND) { fprintf(stderr, "step2 failed\n"); return -1; } return 0; } static int step3(MDB_cursor *cursor) { int rc; MDB_val key, val, rkey, rval; createTestValue(2, &key, &val); E(mdb_cursor_put(cursor, &key, &val, MDB_NOOVERWRITE)); E(mdb_cursor_get(cursor, &rkey, &rval, MDB_NEXT)); // ==> should return 5 return 0; } static void test() { int rc; MDB_txn *txn; MDB_cursor *cursor; E(mdb_txn_begin(env, NULL, 0, &txn)); E(mdb_cursor_open(txn, dbi, &cursor)); step1(cursor); step2(cursor); step3(cursor); mdb_cursor_close(cursor); E(mdb_txn_commit(txn)); } static void cleanup() { mdb_dbi_close(env, dbi); mdb_env_close(env); char name[1024]; sprintf(name% "22%s/data.mdb", db_name); unlink(name); sprintf(name, "%s/lock.mdb", db_name); unlink(name); } int main(int argc,char * argv[]) { setup(); test(); cleanup(); }

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs