- openldap-bugs - openldap.org

Re: (ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by hyc＠symas.com 16 Oct '14

16 Oct '14

leo(a)yuriev.ru wrote: > Full_Name: Leonid Yuriev > Version: 2.4.40 > OS: RHEL7 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (31.130.36.33) > > > Currently there is a couple of backtraces only. > This is the result of a stress test of replication in the presence of > sync-conflicts after a "split brain" case. > No any network troubles (just a loopback connections). Looks like it's accessing freed memory, and it's also related to Abandon processing. You might be running into ITS#7967 as well. If you can reproduce this, try running with a malloc debugger. > > ** Signal 11 (Segmentation fault), address is 0x797a from 0x50e8bd > (0) /opt/openldap.devel/libexec/slapd() [0x50e8bd]: syncprov_op_abandon > /home/ly/Projects/openldap.git/servers/slapd/overlays/syncprov.c:1134 > (1) /opt/openldap.devel/libexec/slapd() [0x48b31a]: overlay_op_walk > /home/ly/Projects/openldap.git/servers/slapd/backover.c:662 > (2) /opt/openldap.devel/libexec/slapd() [0x48b4c1]: over_op_func > /home/ly/Projects/openldap.git/servers/slapd/backover.c:724 > (3) /opt/openldap.devel/libexec/slapd() [0x4429a7]: fe_op_abandon > /home/ly/Projects/openldap.git/servers/slapd/abandon.c:134 (discriminator 2) > (4) /opt/openldap.devel/libexec/slapd() [0x42283c]: connection_abandon > /home/ly/Projects/openldap.git/servers/slapd/connection.c:740 (discriminator 3) > (5) /opt/openldap.devel/libexec/slapd() [0x424509]: connection_closing > /home/ly/Projects/openldap.git/servers/slapd/connection.c:829 > (6) /opt/openldap.devel/libexec/slapd() [0x4250ef]: connection_read > /home/ly/Projects/openldap.git/servers/slapd/connection.c:1477 > connection_read_thread > /home/ly/Projects/openldap.git/servers/slapd/connection.c:1284 > (7) /opt/openldap.devel/lib/libldap_r-2.4.so.2(+0x10cf2) [0x7f8c6de89cf2]: ?? > > *** Signal 11 (Segmentation fault), address is 0x7f7a1e1c5000 from > 0x7f7a1eebfb54 > (0) /lib/x86_64-linux-gnu/libc.so.6(+0x98b54) [0x7f7a1eebfb54]: ????:0 > (1) /opt/openldap.devel/libexec/slapd() [0x4afa7d]: mdb_search > /home/ly/Projects/openldap.git/servers/slapd/back-mdb/search.c:987 > (discriminator 3) > (2) /opt/openldap.devel/libexec/slapd() [0x48b356]: 543edda2 overlay_op_walk > /home/ly/Projects/openldap.git/servers/slapd/backover.c:674 > (3) /opt/openldap.devel/libexec/slapd() [0x48b4c1]: over_op_func > /home/ly/Projects/openldap.git/servers/slapd/backover.c:724 > (4) /opt/openldap.devel/libexec/slapd() [0x427201]: fe_op_search > /home/ly/Projects/openldap.git/servers/slapd/search.c:402 > (5) /opt/openldap.devel/libexec/slapd() [0x426c0c]: do_search > /home/ly/Projects/openldap.git/servers/slapd/search.c:247 > (6) /opt/openld.d.devel/libexec/slapd() [0x424b54]: connection_operation > /home/ly/Projects/openldap.git/servers/slapd/connection.c:1158 > (7) /opt/openldap.devel/libexec/slapd() [0x42526c]: connection_read_thread > /home/ly/Projects/openldap.git/servers/slapd/connection.c:1291 > (8) /opt/openldap.devel/lib/libldap_r-2.4.so.2(+0x10cf2) [0x7f7a1f629cf2]: ?? > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by leo＠yuriev.ru 16 Oct '14

16 Oct '14

One more case *** Signal 11 (Segmentation fault), address is 0xb from 0x442f07 (0) /opt/openldap.devel/libexec/slapd() [0x442f07]: test_filter /home/ly/Projects/openldap.git/servers/slapd/filterentry.c:69 (1) /opt/openldap.devel/libexec/slapd() [0x514721]: syncprov_matchops /home/ly/Projects/openldap.git/servers/slapd/overlays/syncprov.c:1316 (2) /opt/openldap.devel/libexec/slapd() [0x514b83]: syncprov_op_mod /home/ly/Projects/openldap.git/servers/slapd/overlays/syncprov.c:2145 (3) /opt/openldap.devel/libexec/slapd() [0x48b31a]: overlay_op_walk /home/ly/Projects/openldap.git/servers/slapd/backover.c:662 (4) /opt/openldap.devel/libexec/slapd() [0x48b4c1]: over_op_func /home/ly/Projects/openldap.git/servers/slapd/backover.c:724 (5) /opt/openldap.devel/libexec/slapd() [0x4811a6]: syncrepl_entry /home/ly/Projects/openldap.git/servers/slapd/syncrepl.c:3177 do_syncrep2 /home/ly/Projects/openldap.git/servers/slapd/syncrepl.c:1024 (6) /opt/openldap.devel/libexec/slapd() [0x4844b2]: do_syncrepl /home/ly/Projects/openldap.git/servers/slapd/syncrepl.c:1539 (7) /opt/openldap.devel/lib/libldap_r-2.4.so.2(+0x10cf2) [0x7f398891acf2]: ??

1 0

(ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by leo＠yuriev.ru 15 Oct '14

15 Oct '14

Full_Name: Leonid Yuriev Version: 2.4.40 OS: RHEL7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (31.130.36.33) Currently there is a couple of backtraces only. This is the result of a stress test of replication in the presence of sync-conflicts after a "split brain" case. No any network troubles (just a loopback connections). ** Signal 11 (Segmentation fault), address is 0x797a from 0x50e8bd (0) /opt/openldap.devel/libexec/slapd() [0x50e8bd]: syncprov_op_abandon /home/ly/Projects/openldap.git/servers/slapd/overlays/syncprov.c:1134 (1) /opt/openldap.devel/libexec/slapd() [0x48b31a]: overlay_op_walk /home/ly/Projects/openldap.git/servers/slapd/backover.c:662 (2) /opt/openldap.devel/libexec/slapd() [0x48b4c1]: over_op_func /home/ly/Projects/openldap.git/servers/slapd/backover.c:724 (3) /opt/openldap.devel/libexec/slapd() [0x4429a7]: fe_op_abandon /home/ly/Projects/openldap.git/servers/slapd/abandon.c:134 (discriminator 2) (4) /opt/openldap.devel/libexec/slapd() [0x42283c]: connection_abandon /home/ly/Projects/openldap.git/servers/slapd/connection.c:740 (discriminator 3) (5) /opt/openldap.devel/libexec/slapd() [0x424509]: connection_closing /home/ly/Projects/openldap.git/servers/slapd/connection.c:829 (6) /opt/openldap.devel/libexec/slapd() [0x4250ef]: connection_read /home/ly/Projects/openldap.git/servers/slapd/connection.c:1477 connection_read_thread /home/ly/Projects/openldap.git/servers/slapd/connection.c:1284 (7) /opt/openldap.devel/lib/libldap_r-2.4.so.2(+0x10cf2) [0x7f8c6de89cf2]: ?? *** Signal 11 (Segmentation fault), address is 0x7f7a1e1c5000 from 0x7f7a1eebfb54 (0) /lib/x86_64-linux-gnu/libc.so.6(+0x98b54) [0x7f7a1eebfb54]: ????:0 (1) /opt/openldap.devel/libexec/slapd() [0x4afa7d]: mdb_search /home/ly/Projects/openldap.git/servers/slapd/back-mdb/search.c:987 (discriminator 3) (2) /opt/openldap.devel/libexec/slapd() [0x48b356]: 543edda2 overlay_op_walk /home/ly/Projects/openldap.git/servers/slapd/backover.c:674 (3) /opt/openldap.devel/libexec/slapd() [0x48b4c1]: over_op_func /home/ly/Projects/openldap.git/servers/slapd/backover.c:724 (4) /opt/openldap.devel/libexec/slapd() [0x427201]: fe_op_search /home/ly/Projects/openldap.git/servers/slapd/search.c:402 (5) /opt/openldap.devel/libexec/slapd() [0x426c0c]: do_search /home/ly/Projects/openldap.git/servers/slapd/search.c:247 (6) /opt/openld.d.devel/libexec/slapd() [0x424b54]: connection_operation /home/ly/Projects/openldap.git/servers/slapd/connection.c:1158 (7) /opt/openldap.devel/libexec/slapd() [0x42526c]: connection_read_thread /home/ly/Projects/openldap.git/servers/slapd/connection.c:1291 (8) /opt/openldap.devel/lib/libldap_r-2.4.so.2(+0x10cf2) [0x7f7a1f629cf2]: ??

1 0

(ITS#7967) back-meta crash sending abandon after error
by hyc＠openldap.org 15 Oct '14

15 Oct '14

Full_Name: Howard Chu Version: 2.4.40 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (195.235.15.202) Submitted by: hyc The patch in ITS#7712 breaks ldap_abandon processing. In particular, releasing the req_mutex makes it possible for ldap_free_request() to be called twice on the same request. A new fix is being tested.

1 0

Re: (ITS#7966) OpenLDAP server 2.4.40 crashes with rwm and ppolicy overlay during bind
by konrad.windszus＠netcentric.biz 14 Oct '14

14 Oct '14

We use two backends 1) Is leveraging ppolicy with an hdb backend 2) Is leveraging rwm with a relay backend (relaying to 1) with a = different suffix Therefore we cannot influence the order. Modifying the loading order of = the overlays did result in exactly the same stacktrace.=

1 0

Re: (ITS#7966) OpenLDAP server 2.4.40 crashes with rwm and ppolicy overlay during bind
by michael＠stroeder.com 14 Oct '14

14 Oct '14

On Tue, 14 Oct 2014 15:31:25 +0000 konrad.windszus(a)netcentric.biz wrote > As soon as the overlay ppolicy is disabled, the crash is gone. Can you also please try whether the order of slapo-rwm and slapo-ppolicy makes any difference? Ciao, Michael.

1 0

(ITS#7966) OpenLDAP server 2.4.40 crashes with rwm and ppolicy overlay during bind
by konrad.windszus＠netcentric.biz 14 Oct '14

14 Oct '14

Full_Name: Konrad Windszus Version: 2.4.40 OS: CentOS 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (213.61.79.222) Running the overlays ppolicy and rwm on OpenLDAP 2.4.40 (RPM from http://ltb-project.org/) and executing a bind request on the mapped area leads to a segmentation fault. This is the stacktrace from GDB Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ff1f61a4700 (LWP 22660)] 0x0000000000000012 in ?? () (gdb) bt #0 0x0000000000000012 in ?? () #1 0x0000000000505dc0 in hdb_reader_get (op=op@entry=0x7ff1ec000980, env=0x11fa840, txn=txn@entry=0x7ff1f61a30e0) at cache.c:1673 #2 0x000000000050e5ea in hdb_entry_get (op=0x7ff1ec000980, ndn=0x7ff1ec0009b8, oc=0x0, at=0x0, rw=0, ent=0x7ff1f61a3398) at id2entry.c:349 #3 0x00000000004a4cba in overlay_entry_get_ov (op=0x7ff1ec000980, dn=0x7fecec0009b8, oc=0x0, ad=0x0, rw=0, e=0x7ff1f61a3398, on=<optimized out>) at backover.c:364 #4 0x00000000004a4d27 in over_entry_get_rw (op=<optimized out>, dn=<optimized out>, oc=<optimized out>, ad=<optimized out>, rw=<optimized out>, e=<optimized out>) at backover.c:396 #5 0x00000000005528a7 in ppolicy_bind_response (op=0x7ff1ec000980, rs=0x7ff1f61a39a0) at ppolicy.c:924 #6 0x000000000044edb6 in slap_response_play (op=op@entry=0x7ff1ec000980, rs=rs@entry=0x7ff1f61a39a0) at result.c:508 #7 0x000000000044f2c7 in send_ldap_response (op=op@entry=0x7ff1ec000980, rs=rs@entry=0x7ff1f61a39a0) at result.c:583 #8 0x000000000044fc62 in slap_send_ldap_result (op=0x7ff1ec000980, rs=0x7ff1f61a39a0) at result.c:861 #9 0x000000000045c239 in fe_op_bind_success (op=op@entry=0x7ff1ec000980, rs=rs@entry=0x7ff1f61a39a0) at bind.c:441 #10 0x000000000045c8fd in fe_op_bind (op=0x7ff1ec000980, rs=0x7ff1f61a39a0) at bind.c:386 #11 0x000000000045c041 in do_bind (op=0x7ff1ec000980, rs=0x7ff1f61a39a0) at bind.c:205 #12 0x000000000044032e in connection_operation (ctx=ctx@entry=0x7ff1f61a3ad0, arg_v=arg_v@entry=0x7ff1ec000980) at connection.c:1155 #13 0x000000000044060a in connection_read_thread (ctx=0x7ff1f61a3ad0, argv=0xc) at connection.c:1291 #14 0x000000000058da59 in ldap_int_thread_pool_wrapper (xpool=0x113cee0) at tpool.c:688 #15 0x00007ff1fab7ddf3 in start_thread (arg=0x7ff1f61a4700) at pthread_create.c:308 #16 0x00007ff1f99e201d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 As soon as the overlay ppolicy is disabled, the crash is gone.

1 0

(ITS#7963)
by kenel.bastoon＠gmail.com 14 Oct '14

14 Oct '14

--001a113ad9f09d6b280505632e3d Content-Type: text/plain; charset=UTF-8 Thank you for your reply. I will follow your suggestion. As my case is not urgent, I'll send comments to ITS#6664 when I'll have time to test more in depth. Regards --001a113ad9f09d6b280505632e3d Content-Type: text/html; charset=UTF-8 <div dir="ltr">Thank you for your reply.<div><br></div><div>I will follow your suggestion. As my case is not urgent, I'll send comments to ITS#6664 when I'll have time to test more in depth.</div><div><br></div><div>Regards</div></div> --001a113ad9f09d6b280505632e3d--

1 0

Re: (ITS#7948) mdb_copy insecure permissions
by hyc＠symas.com 14 Oct '14

14 Oct '14

geert(a)hendrickx.be wrote: > Full_Name: Geert Hendrickx > Version: 2.4.39 > OS: centos6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (212.123.14.2) > > > mdb_copy creates a copy using the default umask. This usually leads to insecure > (world readable) copies, as typically an LDAP databse is 600 owned by some > unprivileged ldap user. The mode has changed to 0600 as of commit 58ddb5527bd4868bb7017cfe2051bc2e24bcf5a8 -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7965) heap corruption due to returning a reference to the local variable
by hyc＠symas.com 14 Oct '14

14 Oct '14

Леонид Юрьев wrote: > The attached files is derived from OpenLDAP Software. All of the modifications > to OpenLDAP Software represented in the following patch(es) were developed by > Peter-Service LLC, Moscow, Russia. Peter-Service LLC has not assigned rights > and/or interest in this work to any party. I, Leonid Yuriev am authorized by > Peter-Service LLC, my employer, to release this work under the following terms. > > Peter-Service LLC hereby places the following modifications to OpenLDAP Software > (and only these modifications) into the public domain. Hence, these > modifications may be freely used and/or redistributed for any purpose with or > without attribution and/or other notice. Thanks, committed to git master. > https://github.com/leo-yuriev/openldap-lmdb-challenge/commit/1d29214f60300c… > > Author: Leo Yuriev <leo(a)yuriev.ru> > Date: 2014-10-14 14:49:25 +0400 > > BUGFIX - lmdb-backend: heap corruption due to returning a > reference to the local variable. > > diff --git a/servers/slapd/back-mdb/dn2id.c b/servers/slapd/back-mdb/dn2id.c > index 06e6ad3..41c4758 100644 > --- a/servers/slapd/back-mdb/dn2id.c > +++ b/servers/slapd/back-mdb/dn2id.c > @@ -346,7 +346,7 @@ mdb_dn2id( > cursor = mc; > } else { > rc = mdb_cursor_open( txn, dbi, &cursor ); > - if ( rc ) return rc; > + if ( rc ) goto done; > } > > for (;;) { > @@ -470,7 +470,7 @@ mdb_dn2sups( > key.mv_size = sizeof(ID); > > rc = mdb_cursor_open( txn, dbi, &cursor ); > - if ( rc ) return rc; > + if ( rc ) goto done; > > for (;;) { > key.mv_data = &pid; > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7636) slapd crash when multi-master replication (syncrepl) enabled
by leo＠yuriev.ru 14 Oct '14

14 Oct '14

It is one more relevant bug with patch http://www.OpenLDAP.org/its/index.cgi?findid=7965

1 0

Re: (ITS#7965) heap corruption due to returning a reference to the local variable
by leo＠yuriev.ru 14 Oct '14

14 Oct '14

The attached files is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Peter-Service LLC, Moscow, Russia. Peter-Service LLC has not assigned rights and/or interest in this work to any party. I, Leonid Yuriev am authorized by Peter-Service LLC, my employer, to release this work under the following terms. Peter-Service LLC hereby places the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice. https://github.com/leo-yuriev/openldap-lmdb-challenge/commit/1d29214f60300c… Author: Leo Yuriev <leo(a)yuriev.ru> Date: 2014-10-14 14:49:25 +0400 BUGFIX - lmdb-backend: heap corruption due to returning a reference to the local variable. diff --git a/servers/slapd/back-mdb/dn2id.c b/servers/slapd/back-mdb/dn2id.c index 06e6ad3..41c4758 100644 --- a/servers/slapd/back-mdb/dn2id.c +++ b/servers/slapd/back-mdb/dn2id.c @@ -346,7 +346,7 @@ mdb_dn2id( cursor = mc; } else { rc = mdb_cursor_open( txn, dbi, &cursor ); - if ( rc ) return rc; + if ( rc ) goto done; } for (;;) { @@ -470,7 +470,7 @@ mdb_dn2sups( key.mv_size = sizeof(ID); rc = mdb_cursor_open( txn, dbi, &cursor ); - if ( rc ) return rc; + if ( rc ) goto done; for (;;) { key.mv_data = &pid;

1 0

(ITS#7965) heap corruption due to returning a reference to the local variable
by leo＠yuriev.ru 14 Oct '14

14 Oct '14

Full_Name: Leonid Yuriev Version: 2.4.40 OS: RHEL7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (31.130.36.33) #4 0x00000030e1834105 in abort () from /lib64/libc.so.6 #5 0x00000030e1876190 in malloc_printerr () from /lib64/libc.so.6 #6 0x00000000004cc593 in mdb_dn2entry (op=0x7ff36ef32380, tid=0xeb44d0, m2=0x0, dn=0x7ff36ef323b8, e=0x7ff36ef31188, nsubs=<value optimized out>, matched=1) at dn2entry.c:75 mdb = 0x7ff3980da010 rc = -30782 rc2 = <value optimized out> id = 18446744073709551615 0x7ff36ef30db8 mbv = {bv_len = 0, bv_val = 0x7ff36ef30ebe ""} nmbv = {bv_len = 0, bv_val = 0x0} mc = 0x0 while the rsp is 0x7ff36ef30db8 #7 0x00000000004a6707 in mdb_modify (op=0x7ff36ef32380, rs=0x7ff36ef31450) at modify.c:531 #8 0x0000000000487fc7 in overlay_op_walk (op=0x7ff36ef32380, rs=0x7ff36ef31450, which=op_modify, oi=0xe0e840, on=0x0) at backover.c:671 #9 0x00000000004889b3 in over_op_func (op=0x7ff36ef32380, rs=<value optimized out>, which=<value optimized out>) at backover.c:723 #10 0x000000000047c74inin syncrepl_del_nonpresent (op=0x7ff36ef32380, si=0xdcc570, uuids=<value optimized out>, sc=0x7ff36ef314c0, m=89008752) at syncrepl.c:3452 #11 0x0000000000480e92 in do_syncrep2 (op=0x7ff36ef32380, si=0xdcc570) at syncrepl.c:1335 #12 0x00000000004843c2 in do_syncrepl (ctx=<value optimized out>, arg=0xdcc9d0) at syncrepl.c:1539 #13 0x00007ff3987e2868 in ldap_int_thread_pool_wrapper (xpool=0xd79e80) at tpool.c:688

1 0

(ITS#7964) overlay rwm escape issue with more the 9 rules / rewrite statements
by uwe.werler＠retiolum.eu 13 Oct '14

13 Oct '14

Full_Name: Uwe Werler Version: 2.4.40 OS: Linux / SLES 11 SP3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (155.56.68.214) If I have rewrite rules like this: 23 olcOverlay={1}rwm,olcDatabase={3}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcRwmConfig olcOverlay: {1}rwm olcRwmRewrite: {0}rwm-rewriteEngine on olcRwmRewrite: {1}rwm-rewriteContext searchFilter olcRwmRewrite: {2}rwm-rewriteRule "(.*\$)uid=sapr3(\$.*)" "$1uid=dlmsapr3$2" olcRwmRewrite: {3}rwm-rewriteRule "(.*\$)uid=sdb(\$.*)" "$1uid=sdb$2" olcRwmRewrite: {4}rwm-rewriteRule "(.*\$)uid=sapadm(\$.*)%2"%2$1uid=dlmsapadm$2" olcRwmRewrite: {5}rwm-rewriteRule "(.*\$)uid=sapmnt(\$.*)" "$1uid=sapmnt$2" olcRwmRewrite: {6}rwm-rewriteRule "(.*\$)uid=[a-z0-9]{3}adm(\$.*)" "$1uid=dlmsidadm$2" olcRwmRewrite: {7}rwm-rewriteRule "(.*\$)uid=sqd[a-z0-9]%3%3}(\$.*)" "$1uid=dlmsqdsid$2" olcRwmRewrite: {8}rwm-rewriteRule "(.*\$)uid=ora[a-z0-9]{3}(\$.*)" "$1uid=dlmorasid$2" olcRwmRewrite: {9}rwm-rewriteRule "(.*\$)uid=sap[a-z0-9]{3}(\$.*)" "$1uid=dlmsapr3$2" olcRwmRewrite: {10}rwm-rewriteRule "(.*\$)uid=sap[a-z0-9]{3}db(\$.*)" "$1uid=dlmsapr3db$2" olcRwmRewrite: {11}rwm-rewriteRule "(.*\$)uid=db2[a-z0-9]{3}(\$.*)" "$1uid=dlmdb2sid$2" olcRwmRewrite: {12}rwm-rewriteRule "(.*\$)uid=db2[a-z0-9]{3}ap(\$.*)" "$1uid=dlmdb2sid$2" then the ninth rule / statent f failes to escape. In this example ora*** get's not correctly rewritten to dlmora***: See loglevel trace: 543b711d ==> rewrite_rule_apply rule='(.*$)uid=sapr3($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=sdb($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=sapadm($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=sapmnt($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==E E rewrite_rule_apply rule='(.*$)uid=[a-z0-9]{3}adm($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=sqd[a-z0-9]{3}($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*\$)uid=ora[a-z0-9]{3}(\$.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=sap[a-z0-9]{3}($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=sap[a-z0-9]{3}db($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=db2[a-z0-9]{3}($.*)' string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] 543b711d ==> rewrite_rule_apply rule='(.*$)uid=db2[a-z0-9]{3}ap($.*"727 string='(&(objectClass=dynamicObject)(entryExpireTimestamp<=20141013062840Z))' [1 pass(es)] If I insert a dummy statement like this: olcRwmRewrite: {0}rwm-rewriteEngine on olcRwmRewrite: {1}rwm-rewriteContext searchFilter olcRwmRewrite: {2}rwm-rewriteRule "(.*\$)uid=sapr3(\$.*)" "$1uid=dlmsapr3$2" olcRwmRewrite: {3}rwm-rewriteRule "(.*\$)uid=sdb(\$.*)" "$1uid=sdb$2" olcRwmRewrite: {4}rwm-rewriteRule "(.*\$)uid=sapadm(\$.*)" "$1uid=dlmsapadm$2" olcRwmRewrite: {5}rwm-rewriteRule "(.*\$)uid=sapmnt(\$.*)" "$1uid=sapmnt$2" olcRwmRewrite: {6}rwm-rewriteRule "(.*\$)uid=[a-z0-9]{3}adm(\$.*)" "$1uid=dlmsidadm$2" olcRwmRewrite: {7}rwm-rewriteRule "(.*\$)uid=sqd[a-z0-9]{3}(\$.*)" "$1uid=dlmsqdsid$2" olcRwmRewrite: {8}rwm-rewriteRule "(.*\$)uid=ora[a-z0-9]{3}(\$.*)" "$1uid=dlmorasid$2" olcRwmRewrite: {9}rwm-rewriteContext placeHolder alias searchFilter olcRwmRewrite: {10}rwm-rewriteRule "(.*\$)uid=sap[a-z0-9]{3}(\$.*)" "$1uid=dlmsapr3$2" olcRwmRewrite: {11}rwm-rewriteRule "(.*\$)uid=sap[a-z0-9]{3}db(\$.*)" "$1uid=dlmsapr3db$2" olcRwmRewrite: {12}rwm-rewriteRule "(.*\$)uid=db2[a-z0-9]{3}(\$.*)" "$1uid=dlmdb2sid$2" olcRwmRewrite: {13}rwm-rewriteRule "(.*\$)uid=db2[a-z0-9]{3}ap(\$.*)" "$1uid=dlmdb2sid$2" then the escapes are working properly. Sometimes this occurs with the last rule too. I first tried with 2.4.26 (standard version in SLES11 SP3) and now with 2.4.40. Regards Uwe

1 0

Re: (ITS#7963) Client paged results directive fails
by hyc＠symas.com 09 Oct '14

09 Oct '14

kenel.bastoon(a)gmail.com wrote: > Full_Name: Bastien Bonnefon > Version: 2.4.39 > OS: CentOS 7 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (194.2.202.93) > > > Hi, > > I have installed openldap as meta directory to request multiple Active > Directory. > I have managed to install and make it work with dynamic configuration or > slapd.conf. > But one of the applications accessing the directory needs paged results due to > the large amount of entries returned. > > So I've searched and found the directive "client-pr", which seems to have been > enabled since this case : > http://www.openldap.org/its/index.cgi/Software%20Enhancements?id=6664;page=4 > > The directive is also dcribibed in the slapd-meta man page : > http://www.openldap.org/software/man.cgi?query=slapd-meta&apropos=0&sektion… Looking at the ITS history, it appears that this code was released in January 2011 but in fact, the released code is not actually enabled. (It is behind an #ifdef LDAP_DEVEL mask.) Most likely a mistake was made in releasing it at that time, since I see no actual test feedback in the ITS. If you want to test this you will have to compile back-meta yourself, and edit back-meta.h to make sure SLAPD_META_CLIENT_PR gets defined instead of being hidden. Please then send your test results as a followup to ITS#6664. > However, enabling the feature in slapd.conf (I just can't in olc format) doesn't > work. Syslog shows this : > "unknown directive <client-pr> inside backend database definition" > > I've started testing with CentOS 7 and package openldap 2.4.39 > I've then tried with Debian Wheezy and Ubuntu 14.04 (package slapd 2.4.31) > I've also tried installing openldap from the source with the version 2.4.24 > (client-pr should have been enabled in this version due to ITS#6664) => no way > :/ > > I think I've declared the directive as specified in the man page but maybe I > miss something. I have not found any other report on the web on how to use > "client-pr". > Thank you for your help. > > > Here is my slapd.conf > > # Include > include /etc/ldap/schema/core.schema > include /etc/ldap/schema/cosine.schema > include /etc/ldap/schema/inetorgperson.schema > include /etc/ldap/schema/nis.schema > > pidfile /var/run/slapd/slapd.pid > argsfile /var/run/slapd/slapd.args > > # Modules > moduleload back_ldap.la > moduleload back_meta.la > > # Database meta > database meta > suffix "dc=meta,dc=local" > > rootdn "cn=Manager,dc=meta,dc=local" > rootpw secret_password1 > > # First directory > uri "ldap://192.168.0.1/ou=test1,dc=meta,dc=local" > client-pr accept-unsolicited > lastmod off > suffixmassage "ou=test1,dc=meta,dc=local" "dc=test1,dc=local" > idassert-bind bimemethod=simple > binddn="cn=openldap,OU=users,OU=TEST,dc=test1,dc=local" > credentials="secret_password2" > mode=none > flags=non-prescriptive > idassert-authzFrom "dn.exact:cn=Manager,dc=meta,dc=local" > chase-referrals no > acl-authcDN cn=openldap,OU=users,OU=TEST,dc=test1,dc=local > acl-passwd secret_password2 > > # Second Directory > uri "ldap://192.168.0.2/ou=test2,dc=meta,dc=local" > client-pr accept-unsolicited > lastmod off > suffixmassage "ou=test2,dc=meta,dc=local" ,%c=test2,dc=local" > idassert-bind bindmethod=simple > binddn="cn=openldap,OU=users,OU=TEST,dc=test2,dc=local" > credentials="secret_password3" > mode=none > flags=non-prescriptive > idassert-authzFrom "dn.exact:cn=Manager,dc=meta,dc=local" > chase-referrals no > acl-authcDN "cn=openldap,OU=users,OU=TEST,dc=test2,dc=local" > acl-passwd secret_password3 > > > idletimeout 1800 > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7963) Client paged results directive fails
by kenel.bastoon＠gmail.com 09 Oct '14

09 Oct '14

Full_Name: Bastien Bonnefon Version: 2.4.39 OS: CentOS 7 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.2.202.93) Hi, I have installed openldap as meta directory to request multiple Active Directory. I have managed to install and make it work with dynamic configuration or slapd.conf. But one of the applications accessing the directory needs paged results due to the large amount of entries returned. So I've searched and found the directive "client-pr", which seems to have been enabled since this case : http://www.openldap.org/its/index.cgi/Software%20Enhancements?id=6664;page=4 The directive is also dcribibed in the slapd-meta man page : http://www.openldap.org/software/man.cgi?query=slapd-meta&apropos=0&sektion… However, enabling the feature in slapd.conf (I just can't in olc format) doesn't work. Syslog shows this : "unknown directive <client-pr> inside backend database definition" I've started testing with CentOS 7 and package openldap 2.4.39 I've then tried with Debian Wheezy and Ubuntu 14.04 (package slapd 2.4.31) I've also tried installing openldap from the source with the version 2.4.24 (client-pr should have been enabled in this version due to ITS#6664) => no way :/ I think I've declared the directive as specified in the man page but maybe I miss something. I have not found any other report on the web on how to use "client-pr". Thank you for your help. Here is my slapd.conf # Include include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/inetorgperson.schema include /etc/ldap/schema/nis.schema pidfile /var/run/slapd/slapd.pid argsfile /var/run/slapd/slapd.args # Modules moduleload back_ldap.la moduleload back_meta.la # Database meta database meta suffix "dc=meta,dc=local" rootdn "cn=Manager,dc=meta,dc=local" rootpw secret_password1 # First directory uri "ldap://192.168.0.1/ou=test1,dc=meta,dc=local" client-pr accept-unsolicited lastmod off suffixmassage "ou=test1,dc=meta,dc=local" "dc=test1,dc=local" idassert-bind bimemethod=simple binddn="cn=openldap,OU=users,OU=TEST,dc=test1,dc=local" credentials="secret_password2" mode=none flags=non-prescriptive idassert-authzFrom "dn.exact:cn=Manager,dc=meta,dc=local" chase-referrals no acl-authcDN cn=openldap,OU=users,OU=TEST,dc=test1,dc=local acl-passwd secret_password2 # Second Directory uri "ldap://192.168.0.2/ou=test2,dc=meta,dc=local" client-pr accept-unsolicited lastmod off suffixmassage "ou=test2,dc=meta,dc=local" ,%c=test2,dc=local" idassert-bind bindmethod=simple binddn="cn=openldap,OU=users,OU=TEST,dc=test2,dc=local" credentials="secret_password3" mode=none flags=non-prescriptive idassert-authzFrom "dn.exact:cn=Manager,dc=meta,dc=local" chase-referrals no acl-authcDN "cn=openldap,OU=users,OU=TEST,dc=test2,dc=local" acl-passwd secret_password3 idletimeout 1800

1 0

Re: (ITS#7958) LMDB: LIFO-reclaiming, write-performance improvement & bugfixes
by leo＠yuriev.ru 06 Oct '14

06 Oct '14

https://github.com/leo-yuriev/openldap-lmdb-challenge/pull/1 Best regards. Leonid Yuriev.

1 0

Re: (ITS#7961) [LMDB] remove logically dead code (2nd)
by hyc＠symas.com 04 Oct '14

04 Oct '14

jcd(a)tribudubois.net wrote: > Full_Name: Jean-Christophe Dubois > Version: 2.4.40 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (78.235.240.156) > > > In function mdb_txn_begin() at line 2672 there is a check for the parent > pointer. > > The same check is done few lines earlier (line 2665) and if the condition was > met the program would have skip the all section to ok: > > So code line 2673 can never be reached. It could be removed. Thanks. This was leftover from 4d02c741b120786df1b87ee9ed49c1d3f9bc7522. Fixed in mdb.master. > > Patch available at URL below: > > https://github.com/jcdubois/lmdb/commit/a3770a5fef56417ceea677efbbde7687551… > > JC > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7962) [LMDB] remove logically dead code (2nd)
by jcd＠tribudubois.net 04 Oct '14

04 Oct '14

Full_Name: Jean-Christophe Dubois Version: 2.4.40 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (78.235.240.156) In function mdb_txn_begin() at line 2672 there is a check for the parent pointer. The same check is done few lines earlier (line 2665) and if the condition was met the program would have skip the all section to ok: So code line 2673 can never be reached. It could be removed. Patch available at URL below: https://github.com/jcdubois/lmdb/commit/a3770a5fef56417ceea677efbbde7687551… JC

1 0

(ITS#7961) [LMDB] remove logically dead code (2nd)
by jcd＠tribudubois.net 04 Oct '14

04 Oct '14

Full_Name: Jean-Christophe Dubois Version: 2.4.40 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (78.235.240.156) In function mdb_txn_begin() at line 2672 there is a check for the parent pointer. The same check is done few lines earlier (line 2665) and if the condition was met the program would have skip the all section to ok: So code line 2673 can never be reached. It could be removed. Patch available at URL below: https://github.com/jcdubois/lmdb/commit/a3770a5fef56417ceea677efbbde7687551… JC

1 0

Re: (ITS#7960) [LMDB] wrong pointer used in mdb_cassert()
by jcd＠tribudubois.net 03 Oct '14

03 Oct '14

Le 10/03/2014 10:19 PM, Howard Chu a =E9crit : > jcd(a)tribudubois.net wrote: >> Full_Name: Jean-Christophe Dubois >> Version: 2.4.40 >> OS: Linux >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (78.235.240.156) >> >> >> In mdb_node_move() csrc is passed to mdb_cassert() at line 7396 when=20 >> it seems it >> should be cdst (as the operation is on cdst). >> >> https://gitorious.org/mdb/mdb/source/56c2c160be19c555e4c42e459c8608ffa= ae7b150:libraries/liblmdb/mdb.c#L7396=20 >> > > Irrelevant. The cursor is only passed to provide an env pointer, and=20 > both cursors point to the same env. Closing this ITS. Right. It is just that it would be nicer/more logical as it is not clear=20 beforehand what the pointer is passed for. An who knows what additional thing could be done in mdb_cassert in the=20 future. JC >> >> Patch available at URL below: >> >> https://github.com/jcdubois/lmdb/commit/41ed03c4584ac46dc233dcf60f93ad= db09962093=20 >> >> >> JC >> >> > >

1 0

Re: (ITS#7960) [LMDB] wrong pointer used in mdb_cassert()
by hyc＠symas.com 03 Oct '14

03 Oct '14

jcd(a)tribudubois.net wrote: > Full_Name: Jean-Christophe Dubois > Version: 2.4.40 > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (78.235.240.156) > > > In mdb_node_move() csrc is passed to mdb_cassert() at line 7396 when it seems it > should be cdst (as the operation is on cdst). > > https://gitorious.org/mdb/mdb/source/56c2c160be19c555e4c42e459c8608ffaae7b1… Irrelevant. The cursor is only passed to provide an env pointer, and both cursors point to the same env. Closing this ITS. > > Patch available at URL below: > > https://github.com/jcdubois/lmdb/commit/41ed03c4584ac46dc233dcf60f93addb099… > > JC > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7958) LMDB: LIFO-reclaiming, write-performance improvement & bugfixes
by leo＠yuriev.ru 03 Oct '14

03 Oct '14

2014-10-03 3:13 GMT+04:00 Howard Chu <hyc(a)symas.com>: >> commit fc409d89e0d9dde20f612e34c2a463c8a81ea000 >> Author: Leo Yuriev <leo(a)yuriev.ru> >> Date: 2014-09-20 06:51:04 +0400 >> >> EXTENSION - lmdb: more usefull info from mdb_stat tool. > > > A bit ambiguous. me_tail_txnid is actually the ID of the oldest reader, n= ot > the "last" reader. I'm not convinced of the value of this patch, since yo= u > can already view the readers list. I am agree that "tail" is NOT a best choice. But the main value of this patch is not to show a txn of oldest reader, but to show an info about pages usage. Especially the amount of pages which are "blocked" by oldest (laggard) reader, and how much pages are actually available. 2014-10-04 0:04 GMT+04:00 =D0=9B=D0=B5=D0=BE=D0=BD=D0=B8=D0=B4 =D0=AE=D1=80= =D1=8C=D0=B5=D0=B2 <leo(a)yuriev.ru>: > Fwd: (ITS#7841) high disk utilization > > 2014-10-03 3:13 GMT+04:00 Howard Chu <hyc(a)symas.com>: >>> commit 841059330fd44769e93eb4b937c3ce42654fad6f >>> Author: Leo Yuriev <leo(a)yuriev.ru> >>> Date: 2014-09-20 07:16:15 +0400 >>> >>> BUGFIX - lmdb: lock meta-pages in writemap-mode to avoid unexpect= ed >>> write, >>> before the data pages would be synchronized. >>> >>> Without locking the meta-pages may be writen by OS before other >>> data, >>> in this case database would be inconsistent. >> >> >> Seems unnecessary. Won't happen by default; could happen with MDB_NOSYNC= but >> that risk is already documented. > > We are using the combination: > envflags writemap nosync lifo > checkpoint 0 1 > > If the checkpoint is set in seconds, it gives us the assurance > consistent state database on disk. > However, without this patch meta-pages can be written by the kernel > before the data. > > In fact, for a full guarantee in case of death slapd process, > meta-page should be written explicitly. > But it requires a lot of changes and I do not do that. > >>> commit 0c168d0e63ed78d13df3fc8a42f3667335678639 >>> Author: Leo Yuriev <leo(a)yuriev.ru> >>> Date: 2014-09-20 10:13:28 +0400 >>> >>> FEATURE - lmdb: MDB_LIFORECLAIM & MDB_COALESCE modes. >>> >>> Reclaim FreeDB in LIFO order - this is a main feature. >>> Also aim to coalesce small FreeDFB records. >> >> Will spend more time looking at this closer. > > I would be suggested, but do not insist, review this patch on github. > >>> commit 8ddd63161aeb2689822d1a8d27385d62e4e341ae >>> Author: Leo Yuriev <leo(a)yuriev.ru> >>> Date: 2014-09-19 22:47:19 +0400 >>> >>> BUGFIX - lmdb: properly sync meta-pages in mdb_sync_env(). >>> >>> Meta-pages may be updated during data-syncing in mdb_sync_env(), >>> in this case database would be inconsistent. >>> >>> Check-and-retry if lead txn-id changed during flushing data in >>> mdb_sync_env(). >> >> Probably could simplify this, just obtain the write mutex unconditionall= y, >> then there's no need to loop or retry. But also, this depends on MDB_NOL= OCK >> - if that's set, then do no locking at all. > > I did so for reasons of performance and less a lock retention time. > > Retries will be if there an intensive flow of changes. > In this case it will be a lot of updated pages, the record which will > take some time. > > However, in subsequent iterations (if a transactions had committed > while there was a record), > the modified pages will be much fewer, and the sync will be quick. > > Thus (and it was seen in tests) even when a substantial amount of the > transactions, > usually only two iterations of the cycle, > without locking and flow of changes are not suspended. > >>> commit 147f41a8110f28456bc32123bde86d47183f9c0a >>> Author: Leo Yuriev <leo(a)yuriev.ru> >>> Date: 2014-09-04 16:01:15 +0400 >>> >>> FEATURE - lmdb: implementation of "checkpoint kbytes". >>> >>> Force flush when volume of the changes reached a configurable >>> threshold. >> >> >> Probably OK. Needs some typographical cleanup. Not sure "syncbytes" is a >> good name. > > Agree. > I just took the first choice and try to retaining the style. > Ideas? > >>> commit fb82a0b688f4c31313d0790415feda8aaa18651c >>> Author: Leo Yuriev <leo(a)yuriev.ru> >>> Date: 2014-09-04 15:18:16 +0400 >>> >>> CHANGE - lmdb-backend: checkpoint-interval in seconds instead of >>> minutes. >> >> >> Gratuitous change. We used minutes since the BDB backend uses minutes, a= nd >> the intention was to maintain parallel functionality. What's the >> justification for this change? > > As I had wrote above, we are using the combination: > envflags writemap nosync lifo > checkpoint 0 1 > > If the interval is specified in minutes, then it can not be set less > than one minute. > But it's too big amount of time to allow lost the updates. > > However, setting the synchronization interval of one second, > we reduce the amount of losses in the event of an accident to an > acceptable level, > while the load on the storage system is acceptable even for a large > flow of updates. > > As a result, I have not found a better solution than simply replace > the minutes by the seconds. > >>> commit fc409d89e0d9dde20f612e34c2a463c8a81ea000 >>> Author: Leo Yuriev <leo(a)yuriev.ru> >>> Date: 2014-09-20 06:51:04 +0400 >>> >>> EXTENSION - lmdb: more usefull info from mdb_stat tool. >> >> >> A bit ambiguous. me_tail_txnid is actually the ID of the oldest reader, = not >> the "last" reader. I'm not convinced of the value of this patch, since y= ou >> can already view the readers list. > > I am agree then "tail" is a best choice. > But the main value of this patch is not to show a txn of oldest > reader, but to show an info about pages usage. > Especially the amount of pages which are "blocked" by oldest (laggard) > reader, and how much pages are actually available. > >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ > > Thank you in advance. > BR. > Leonid Yuriev.

1 0

(ITS#7960) [LMDB] wrong pointer used in mdb_cassert()
by jcd＠tribudubois.net 03 Oct '14

03 Oct '14

Full_Name: Jean-Christophe Dubois Version: 2.4.40 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (78.235.240.156) In mdb_node_move() csrc is passed to mdb_cassert() at line 7396 when it seems it should be cdst (as the operation is on cdst). https://gitorious.org/mdb/mdb/source/56c2c160be19c555e4c42e459c8608ffaae7b1… Patch available at URL below: https://github.com/jcdubois/lmdb/commit/41ed03c4584ac46dc233dcf60f93addb099… JC

1 0

Re: (ITS#7841) high disk utilization
by leo＠yuriev.ru 03 Oct '14

03 Oct '14

As directed by Kurt Zeilenga (Executive Director, Kurt(a)openldap.org) I was re-submitted the new ITS#7958 with updated IPR statement. http://www.openldap.org/its/index.cgi/Incoming?id=7958;selectid=7958 Best regards, Leonid.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs