- openldap-bugs - openldap.org

Re: (ITS#7929) memberof overlay supresses accesslog olcAccessLogOps = all
by varkoly＠suse.com 06 Sep '14

06 Sep '14

Hello, I've tested it from git://git.openldap.org/openldap.git/OPENLDAP_REL_ENG_2_4 with same result. http://www.openldap.org/its/index.cgi/Incoming?id=7239 looks vaguely similar, although with a different overlay. Am Donnerstag, den 04.09.2014, 15:55 +0000 schrieb quanah(a)zimbra.com: > --On Thursday, September 04, 2014 10:11 AM +0000 varkoly(a)suse.com wrote: > > > Full_Name: Peter Varkoly > > Version: 2.4.26, 2.4.39 > > Hello, > > Can you please test using the current RE24 code? There have been > significant updates made to the overlay interactions since 2.4.39. > > Thanks! > > --Quanah > > > -- > > Quanah Gibson-Mount > Server Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration > > > > -- Peter Varkoly Sr. Developer SUSE Linux Enterprise Applications SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany

1 0

(ITS#7930) slapo-chain causes double-free with rwm
by hyc＠openldap.org 06 Sep '14

06 Sep '14

Full_Name: Howard Chu Version: RE24 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (78.155.233.73) Submitted by: hyc chaining substitutes op->o_req_dn and then restores its original value after performing the chained op. But rwm tries to cleanup op->o_req_dn when the chained op finishes, and so the original DN will be invalid after that. Partial fix: chain should only resto t the original value if the current value matches the last value that chain used. Additional fix: rwm should not cleanup at this time, because the send_ldap_result() of the chained op is not necessarily the final result of the original op. (E.g., in a search that processes multiple search refs, send_ldap_result will occur for each ref that is chained.)

1 0

Re: (ITS#7929) memberof overlay supresses accesslog olcAccessLogOps = all
by quanah＠zimbra.com 04 Sep '14

04 Sep '14

--On Thursday, September 04, 2014 10:11 AM +0000 varkoly(a)suse.com wrote: > Full_Name: Peter Varkoly > Version: 2.4.26, 2.4.39 Hello, Can you please test using the current RE24 code? There have been significant updates made to the overlay interactions since 2.4.39. Thanks! --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7929) memberof overlay supresses accesslog olcAccessLogOps = all
by varkoly＠suse.com 04 Sep '14

04 Sep '14

Full_Name: Peter Varkoly Version: 2.4.26, 2.4.39 OS: SUSE Enterprise Server URL: ftp://ftp.suse.com/pub/people/varkoly Submission from: (NULL) (2620:113:80c0:5::2222) Enabling both the "memberof" and "accesslog" overlays in openldap causes accesslog to only log "reads" and "session" operations when logops is set to all (olcAccessLogOps = all). Disabling the "memberof" overlay results in "writes", "reads" and "session" operations to be logged. Setting "olcAccessLogOps = writes" in conjunction with the "memberof" overlay results in "writes" operations to be logged. Expected result is that when "olcAccessLogOps = all" is set for the "accesslog" overlay together with the "memberof" overlay, that all operations (add, modify, delete, modrdn, search, bind, unbind, abandon, compare) are logged to cn=accesslog. The order of activating the overlays is irrelevant. We have tested it by activating first memberof and accesslog too. We are using the config-backend. The configuration will be available soon under the provided URL.

1 0

Re: (ITS#7928) Segfault when cancelling syncprov search
by hyc＠symas.com 29 Aug '14

29 Aug '14

ben(a)morrow.me.uk wrote: > Full_Name: Ben Morrow > Version: 2.4.39 and git head > OS: FreeBSD > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (86.140.232.115) > > > I believe I have found a crashing bug in the syncprov overlay. If I > start a refreshAndPersist search, wait for the persist phase to start > and at least one change to be returned, and then cancel the search, > slapd dies with SIGSEGV. (I don't know how many of these conditions are > necessary. I suspect that cancelling any syncprov search will cause the > same crash, but I haven't investigated that.) > > I also believe the following change to > servers/slapd/overlays/syncprov.c:syncprov_ab_cleanup fixes the prlelem: > > slap_callback *sc = op->o_callback; > op->o_callback = sc->sc_next; > - syncprov_drop_psearch( op->o_callback->sc_private, 0 ); > + syncprov_drop_psearch( sc->sc_private, 0 ); > > The crash I see actually occurs in libthr (FreeBSD's threading library), > because drop_psearch passes its first argument to free_syncop which > starts by trying to lock a mutex; but in fact the object being passed is > not the right type at all, and where there is supposed to be a mutex > there is instead a function pointer. Thanks, looks like you're right. Fixed in git master. > > Ben > > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7928) Segfault when cancelling syncprov search
by ben＠morrow.me.uk 29 Aug '14

29 Aug '14

Full_Name: Ben Morrow Version: 2.4.39 and git head OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (86.140.232.115) I believe I have found a crashing bug in the syncprov overlay. If I start a refreshAndPersist search, wait for the persist phase to start and at least one change to be returned, and then cancel the search, slapd dies with SIGSEGV. (I don't know how many of these conditions are necessary. I suspect that cancelling any syncprov search will cause the same crash, but I haven't investigated that.) I also believe the following change to servers/slapd/overlays/syncprov.c:syncprov_ab_cleanup fixes the prlelem: slap_callback *sc = op->o_callback; op->o_callback = sc->sc_next; - syncprov_drop_psearch( op->o_callback->sc_private, 0 ); + syncprov_drop_psearch( sc->sc_private, 0 ); The crash I see actually occurs in libthr (FreeBSD's threading library), because drop_psearch passes its first argument to free_syncop which starts by trying to lock a mutex; but in fact the object being passed is not the right type at all, and where there is supposed to be a mutex there is instead a function pointer. Ben

1 0

(ITS#7927) back-mdb SEGV when using an alias pointing to a referral
by hyc＠openldap.org 26 Aug '14

26 Aug '14

Full_Name: Howard Chu Version: 2.4 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (78.155.233.73) Submitted by: hyc Patch coming shortly

1 0

Re: (ITS#7926) modifying olcListenerThreads crashes slapd
by quanah＠zimbra.com 26 Aug '14

26 Aug '14

--On Monday, August 25, 2014 9:53 PM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.39 > OS: Linux 3.13 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.58.125) > > > After modifying olcListenerThreads for a running slapd, the process died > and I had to restart ldap. Going to see if cores were enabled. > Trivial to reproduce: (gdb) cont Continuing. Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f963ca41700 (LWP 12048)] 0x0000000000432279 in slapd_clr_read (s=13, wake=0) at daemon.c:981 981 daemon.c: No such file or directory. in daemon.c Thread 6 (Thread 0x7f963ca41700 (LWP 12048)): #0 0x0000000000432279 in slapd_clr_read (s=13, wake=0) at daemon.c:981 rc = 1 id = 1 #1 0x000000000043ab9c in connection_read_activate (s=13) at connection.c:1285 rc = 62 #2 0x0000000000436ddb in slapd_daemon_task (ptr=0x1ba1d28) at daemon.c:2769 rc = 1 fd = 13 w = 0 r = 1 ns = 1 at = 0 nfds = 18 revents = 0x1c40000 tvp = 0x0 cat = {tv_sec = 0, tv_usec = 0} i = 0 nwriters = 0 now = 1409005820 tv = {tv_sec = 0, tv_usec = 0} tdelta = 1 rtask = 0x0 l = 2 last_idle_check = 1408061664 ebadf = 0 tid = 0 #3 0x0000003e130079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x0000003e12ce8b5d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 5 (Thread 0x7f963c240700 (LWP 12052)): #0 0x0000003e1300b5bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007fa28ef29787 in ldap_pvt_thread_cond_wait (cond=0x1bdc038, mutex=0x1bdc010) at thr_posix.c:277 No locals. #2 0x00007fa28ef27fa2 in ldap_int_thread_pool_wrapper (xpool=0x1bdc000) at tpool.c:927 pq = 0x1bdc000 pool = 0x1da0180 task = 0x0 work_list = 0x1bdc070 ctx = {ltu_pq = 0x1bdc000, ltu_id = 140283230816000, ltu_key = {{ltk_key = 0x43a106, ltk_data = 0x1da4e00, ltk_free = 0x439f4a <conn_counter_destroy>}, {ltk_key = 0x4ad967, ltk_data = 0x3d9c2c0, ltk_free = 0x4ad78c <slap_sl_mem_destroy>}, {ltk_key = 0x455375, ltk_data = 0x20a03c0, ltk_free = 0x4552c8 <slap_op_q_destroy>}, { ltk_key = 0x7fa28b554533, ltk_data = 0x419e000, ltk_free = 0x7fa28b554510 <search_stack_free>}, {ltk_key = 0x7fa28b5515ab, ltk_data = 0x3e9e000, ltk_free = 0x7fa28b551563 <scope_chunk_free>}, {ltk_key = 0x1bdd6c0, ltk_data = 0x1fe2200, ltk_free = 0x7fa28b55f195 <mdb_reader_free>}, {ltk_key = 0x0, ltk_data = 0x9106400, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = 0x0 i = 32 keyslot = 380 hash = 3641033084 pool_lock = 0 freeme = 0 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0x0000003e130079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x0000003e12ce8b5d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 4 (Thread 0x7f963ba3f700 (LWP 12183)): #0 0x0000003e1300b5bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007fa28ef29787 in ldap_pvt_thread_cond_wait (cond=0x1bdc038, mutex=0x1bdc010) at thr_posix.c:277 No locals. #2 0x00007fa28ef27fa2 in ldap_int_thread_pool_wrapper (xpool=0x1bdc000) at tpool.c:927 pq = 0x1bdc000 pool = 0x1da0180 task = 0x0 work_list = 0x1bdc070 ctx = {ltu_pq = 0x1bdc000, ltu_id = 140283222423296, ltu_key = {{ltk_key = 0x43a106, ltk_data = 0x1da4700, ltk_free = 0x439f4a <conn_counter_destroy>}, {ltk_key = 0x4ad967, ltk_data = 0x3d9c9c0, ltk_free = 0x4ad78c <slap_sl_mem_destroy>}, {ltk_key = 0x1bdd6c0, ltk_data = 0x52dc000, ltk_free = 0x7fa28b55f195 <mdb_reader_free>}, { ltk_key = 0x7fa28b554533, ltk_data = 0x55e6000, ltk_free = 0x7fa28b554510 <search_stack_free>}, {ltk_key = 0x7fa28b5515ab, ltk_data = 0x52e6000, ltk_free = 0x7fa28b551563 <scope_chunk_free>}, {ltk_key = 0x455375, ltk_data = 0x237b840, ltk_free = 0x4552c8 <slap_op_q_destroy>}, {ltk_key = 0x0, ltk_data = 0x66ed600, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = 0x0 i = 32 keyslot = 138 hash = 3676775562 pool_lock = 0 freeme = 0 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0x0000003e130079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x0000003e12ce8b5d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 3 (Thread 0x7f963b23e700 (LWP 24606)): #0 0x0000003e1300b5bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007fa28ef29787 in ldap_pvt_thread_cond_wait (cond=0x1bdc038, mutex=0x1bdc010) at thr_posix.c:277 No locals. #2 0x00007fa28ef27fa2 in ldap_int_thread_pool_wrapper (xpool=0x1bdc000) at tpool.c:927 pq = 0x1bdc000 pool = 0x1da0180 task = 0x0 work_list = 0x1bdc070 ctx = {ltu_pq = 0x1bdc000, ltu_id = 140283214030592, ltu_key = {{ltk_key = 0x43a106, ltk_data = 0x2155100, ltk_free = 0x439f4a <conn_counter_destroy>}, {ltk_key = 0x4ad967, ltk_data = 0x66d6b40, ltk_free = 0x4ad78c <slap_sl_mem_destroy>}, {ltk_key = 0x455375, ltk_data = 0x2379a40, ltk_free = 0x4552c8 <slap_op_q_destroy>}, {ltk_key = 0x1bdd6c0, ltk_data = 0x6a60000, ltk_free = 0x7fa28b55f195 <mdb_reader_free>}, {ltk_key = 0x7fa28b554533, ltk_data = 0x6d6e000, ltk_free = 0x7fa28b554510 <search_stack_free>}, { ltk_key = 0x7fa28b5515ab, ltk_data = 0x6a6e000, ltk_free = 0x7fa28b551563 <scope_chunk_free>}, {ltk_key = 0x0, ltk_data = 0x9108800, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = 0x0 i = 32 keyslot = 250 hash = 907520250 pool_lock = 0 freeme = 0 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0x0000003e130079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x0000003e12ce8b5d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 2 (Thread 0x7f963aa3d700 (LWP 24607)): #0 0x0000003e1300b5bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007fa28ef29787 in ldap_pvt_thread_cond_wait (cond=0x1bdc038, mutex=0x1bdc010) at thr_posix.c:277 No locals. #2 0x00007fa28ef27fa2 in ldap_int_thread_pool_wrapper (xpool=0x1bdc000) at tpool.c:927 pq = 0x1bdc000 pool = 0x1da0180 task = 0x0 work_list = 0x1bdc070 ctx = {ltu_pq = 0x1bdc000, ltu_id = 140283205637888, ltu_key = {{ltk_key = 0x43a106, ltk_data = 0x2154f00, ltk_free = 0x439f4a <conn_counter_destroy>}, {ltk_key = 0x4ad967, ltk_data = 0x66d6b80, ltk_free = 0x4ad78c <slap_sl_mem_destroy>}, {ltk_key = 0x455375, ltk_data = 0x66c6000, ltk_free = 0x4552c8 <slap_op_q_destroy>}, {ltk_key = 0x1bdd6c0, ltk_data = 0x6a61a00, ltk_free = 0x7fa28b55f195 <mdb_reader_free>}, {ltk_key = 0x7fa28b554533, ltk_data = 0x806e000, ltk_free = 0x7fa28b554510 <search_stack_free>}, { ltk_key = 0x7fa28b5515ab, ltk_data = 0x7d6e000, ltk_free = 0x7fa28b551563 <scope_chunk_free>}, {ltk_key = 0x0, ltk_data = 0x9108800, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 25 times>}} kctx = 0x0 i = 32 keyslot = 41 hash = 1003389993 pool_lock = 0 freeme = 0 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #3 0x0000003e130079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #4 0x0000003e12ce8b5d in clone () from /lib64/libc.so.6 No symbol table info available. Thread 1 (Thread 0x7fa28e26a720 (LWP 12047)): #0 0x0000003e1300822d in pthread_join () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007fa28ef296c8 in ldap_pvt_thread_join (thread=140283239208704, thread_return=0x0) at thr_posix.c:197 No locals. #2 0x0000000000437587 in slapd_daemon () at daemon.c:2907 i = 0 rc = 0 #3 0x0000000000414afb in main (argc=9, argv=0x7fff33ea3258) at main.c:1012 i = 9 no_detach = 0 rc = 0 urls = 0x1bb0000 "ldap://zre-ldap003.eng.zimbra.com:389 ldapi:///" username = 0x1ba0010 "root" groupname = 0x0 sandbox = 0x0 syslogUser = 128 pid = 0 waitfds = {9, 10} g_argc = 9 g_argv = 0x7fff33ea3258 configfile = 0x0 configdir = 0x1ba4020 "/opt/zimbra/data/ldap/config" serverName = 0x7fff33ea4d76 "slapd" serverMode = 1 scp = 0x0 scp_entry = 0x0 debug_unknowns = 0x0 syslog_unknowns = 0x0 serverNamePrefix = 0x4f19c8 "" l = 5182640 slapd_pid_file_unlink = 1 slapd_args_file_unlink = 1 firstopt = 0 __PRETTY_FUNCTION__ = "main" (gdb) -- Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7926) modifying olcListenerThreads crashes slapd
by quanah＠openldap.org 25 Aug '14

25 Aug '14

Full_Name: Quanah Gibson-Mount Version: 2.4.39 OS: Linux 3.13 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.58.125) After modifying olcListenerThreads for a running slapd, the process died and I had to restart ldap. Going to see if cores were enabled.

1 0

Re: (ITS#7925) mdb_from_db Berkeley DB importer
by hyc＠symas.com 25 Aug '14

25 Aug '14

graehl(a)gmail.com wrote: > Full_Name: Jonathan Graehl > Version: commit 8d346721a60684aeaa7b1e3b2111c972393bfad3 > OS: linux > URL: > Submission from: (NULL) (75.85.99.117) > > > mdb_from_db Berkeley DB->LMDB import utility Interesting, thanks for the submission. > +Additionally, > +.B mdb_from_db > +may write in the > +.B -T > +plain text format understood by > +.BR mdb_load (1) > +which can only understand a single subdatabase at a time. Not true. mdb_load can load multiple DBs from a single input file. Also, I'm not sure if it's your mailer or ours, but this submission text is riddled with corruptions. > + > B2B.SH OPTION0D0D In general I don't think including this utility into LMDB is a good idea since it has an obvious binary dependency on BerkeleyDB. In particular, with the change of BerkeleyDB's license to AGPL it would not be possible for anyone to distribute any binaries for this package. Better to just use BDB db_dump piped into mdb_load. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7925) mdb_from_db Berkeley DB importer
by graehl＠gmail.com 25 Aug '14

25 Aug '14

Full_Name: Jonathan Graehl Version: commit 8d346721a60684aeaa7b1e3b2111c972393bfad3 OS: linux URL: Submission from: (NULL) (75.85.99.117) mdb_from_db Berkeley DB->LMDB import utility See also https://github.com/openldap/openldap/pull/1 >From 32f6c10570bf7ede64cdb734775e97ea2afe1011 Mon Sep 17 00:00:00 2001 From: graehl <graehl(a)gmail.com> Date: Sun, 24 Aug 2014 16:02:41 -0700 Subject: [PATCH] mdb_from_db Berkeley DB->LMDB import --- libraries/liblmdb/Makefile | 3 +- libraries/liblmdb/mdb_from_db.1 | 104 ++++++++ libraries/liblmdb/mdb_from_db.c | 548 +++++++++++++++++++++++++++++++++++++B%B++ 3 files changed, 654 insertions(+), 1 deletion(-) create mode 100644 libraries/liblmdb/mdb_from_db.1 create mode 100644 libraries/liblmdb/mdb_from_db.c diff --git a/libraries/liblmdb/Makefile b/libraries/liblmdb/Makefile index 25c1095..196ed08 100644 --- a/libraries/liblmdb/Makefile +++ b/libraries/liblmdb/Makefile @@ -29,7 +29,7 @@ prefix = /usr/local IHDRS = lmdb.h ILIBS = liblmdb.a liblmdb.so -IPROGS = mdb_stat mdb_copy mdb_dump mdb_load +IPROGS = mdb_stat mdb_copy mdb_dump mdb_load mdb_from_db IDOCS = mdb_stat.1 mdb_copy.1 mdb_dump.1 mdb_load.1 PROGS = $(IPROGS) mtest mtest2 mtest3 mtest4 mtest5 all: $(ILIBS) $(PROGS) @@ -58,6 +58,7 @@ mdb_stat: mdb_stat.o liblmdb.a mdb_copy: mdb_copy.o liblmdb.a mdb_dump: mdb_dump.o liblmdb.a mdb_load: mdb_load.o liblmdb.a +mdb_from_db: mdb_from_db.o liblmdb.a mtest: mtest.o liblmdb.a mtest2: mtest2.o liblmdb.a mtest3: mtest3.o liblmdb.a diff --git a/libraries/liblmdb/mdb_from_db.1 b/libraries/liblmdb/mdb_from_db.1 new file mode 100644 index 0000000..dbd6797 --- /dev/null +++ b/libraries/liblmdb/mdb_from_db.1 @@ -0,0 +1,104 @@ +.TH MDB_FROM_DB 1 "2014/06/20" "LMDB 0.9.14" +.\" Copyright 2014 Howard Chu, Symas Corp. All Rights Reserved. +.\" Copying restrictions apply. See COPYRIGHT/LICENSE. +.SH NAME +mdb_from_db \- LMDB environment translate from Berkeley DB environment tool +.SH SYNOPSIS +.B mdb_from_db +.BR \ berkeley.db +.BR \ envpath +[\c +.BR \-V ] +[\c +.BR \-n ] +[\c +.BI \-s \ subdb\fR] +[\c +.B%5\-b \ bahshsize\fR] +[\c +.BI \-h \ berkeley-db-homedir\fR] +[\c +.BR \-N ] +[\c +.BR \-T ] +.SH DESCRIPTION +The +.B mdb_from_db +utility reads from a Berkeley DB environment +.BR berkeley.db +and from_dbs all its subdatabases, or just the specified +.BR subdb +, into the +LMDB environment +.BR envpath . + +Additionally, +.B mdb_from_db +may write in the +.B -T +plain text format understood by +.BR mdb_load (1) +which can only understand a single subdatabase at a time. + B2B.SH OPTION0D0D +.TP +.BR \-V +Write the library version number to the standard output, and exit. +.TP +.BR \-n +From_Db an LMDB database which does not use subdirectories. +.TP +.BR \-s \ subdb +From_Db a specific subdatabase. If no database is specified, data is from_dbed into the main database. +.TP +.BR \-N +Don't overwrite existing records when from_dbing into an already existing database; just skip them. +.TP +.BR \-b \ sz +Commit LMDB records +.B sz +at a time. +.TP +.BR \-h \ db_homedir +Treat input db path as relative to this homedir (see the Berkeley DB docs). Default is '.' +.TP +.BR \-B +Perform a nonblocking Berkeley DB open. +.TP +.BR \-T +Write the key/data into a single simple text file (stderr messages +would allow segmenting the output into separate files for each +subdatabase). The input will be paired lines of text, where the first +line of the pair is the key item, and the second line of the pair is +its corresponding data item. If more than one database is read then +refer to the counts reported on stderr. + +A simple escape mechanism, where newline and backslash (\\) characters +are special, is applied to the text input. Newline characters are +interpreted as record separators. Backslash characters in the text +will be interpreted in one of two ways: If the backslash character +precedes another backslash character, the pair will be interpreted as +a literal backslash. If the backslash character precedes any other +character, the two characters following the backslash will be +interpreted as a hexadecimal specification of a single character; for +example, \\0a is a newline character in the ASCII character set. + +For this reason, any backslash or newline characters that naturally +occur in the text input must be escaped to avoid misinterpretation by +.BR mdb_load. + +.SH DIAGNOSTICS +Exit status is zero if no errors occur. +Errors result in a non-zero exit status and +a diagnostic message being written to standard error. + +Information about each subdatabase processed, and the total number of +records is also written to standard error. + +.SH "SEE ALSO" +.BR mdb_load (1) +.BR mdb_dump (1) +.BR db_dump (1) + +.SH AUTHOR +Jonathan Graehl <graehl(a)gmail.com> diff --git a/libraries/liblmdb/mdb_from_db.c b/libraries/liblmdb/mdb_from_db.c new file mode 100644 index 0000000..ee81db0 --- /dev/null +++ b/libraries/liblmdb/mdb_from_db.c @@ -0,0 +1,548 @@ +/* mdb_from_db.c - translate Berkeley DB to memory-mapped database(s) */ +/* + * Copyright 2014 Jonathan Graeh2C2C 2011-2014 Howard Chu, Symas Corp. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted only as authorized by the OpenLDAP + * Public License. + * + * A copy of this license is available in the file LICENSE in the + * top-level directory of the distribution or, alternatively, at + * <http://www.OpenLDAP.org/license.html>. + */ +#include <stdio.h> +#include <stdlib.h> +#include <errno.h> +#include <string.h> +#include <ctype.h> +#include <unistd.h> +#include "lmdb.h" +#include <db.h> +#include <stdbool.h> +#include <sys/stat.h> + +static int datasize = 64*1024; +static int batchsize = 100; + +static char *subname = NULL; + +static char *progB0D + +static MDB_val kbuf, dbuf; + +#ifdef _WIN32 +#define Z "I" +#else +#define Z "z" +#endif + + +char *usagestr = + "path.input.berkeley.db [path.output.mdb|T-txt] [-P dbpasswd] [-V] [-l] [-n] [-s subdbname] [-N] [-B] [-T] [-v] [-h homedirpath] [-b write-batchsize] [-f redirect_stdout.txt]\n" + " (-T prints to stdout key/val in mdb_load format)\n" + " (-l: only list (to stdout) database names, -N: don't overwrite existing keys; -B nonblocking db open; -n: create single mdb file instead of dir)\n" + ; + +/** + fail() and shutdown(): everything that might need to be cleaned up on error->exit + (conceptually some of these are locals, but having them global lets us call + fail() from anywhere) +*/A%A+static MDB_env *env; +static MDB_txn *txn; +static MDB_dbi dbi; + +static DB *dbp; +static DBC *dbcp; +static DB *parent_dbp; +static DBC *bdb_subdbcursor; +static char *subdbname; +static DB_TXN *dbtxn; + +void bdb_close() { + if (dbcp) + dbcp->cse%2(dbcp); + dbcp = 0; + if (dbp) + dbp->close(dbp, 0); + dbp = 0; +} +void shutdown() { + if (bdb_subdbcursor) + bdb_subdbcursor->close(bdb_subdbcursor); + bdb_subdbcursor = 0; + bdb_close(); + if (parent_dbp) + parent_dbp->close(parent_dbp, 0); + parent_dbp = 0; + if (txn) + mdb_txn_abort(txn); + txn = 0; + if (dbi) + mdb_dbi_close(env, dbi); + if (env) + mdb_env_close(env); + env = 0; + if (subdbname) + free(subdbname); + subdbname =%0; +} + +void fail() { + shutdown(); + exit(EXIT_FAILURE); +} + +static void usage(void) +{ + fprintf(stderr, "usage: %s %s", prog, usagestr); + fail(); +} + +/** + BDB env +*/ +static char *dbhome; +static DB_ENV *dbenv; +static u_int32_t dbcache = 1024*1024; +static char *dbpasswd; +static bool dbnonblocking; +void strfill(char *str, char fill) { + while(*str) + *str++ = fill; +} +void bdb_err(char *fn, int rc) { + fprintf(stderr, "%s: ", prog); + if (dbenv) + dbenv->err(dbenv, rc, fn); + else + fprintf(stderr, "%s\n", db_strerror(rc)); + fail(); +} + +DBT dbkey, dbdata; +void bdb_init_dbenv() { + int rc; + if ((rc = db_env_create(&dbenv, 0)) != 0) + bdb_err("db_env_create", rc); + dbenv->set_errfile(dbenv, stderr); + dbv-v->set_errpfx(dbenv, prog); + if (dbpasswd != NULL) { + rc = dbenv->set_encrypt(dbenv, dbpasswd, DB_ENCRYPT_AES); + strfill(dbpasswd, '\0'); + if (rc) + bdb_err("dbenv::set_encrypt", rc); + } + if (dbnonblocking) { + if ((rc = dbenv->set_flags(dbenv, DB_NOLOCKING, 1))) + bdb_err("DB_NOLOCKING", rc); + if ((rc = dbenv->set_flags(dbenv, DB_NOPANIC, 1))) + bdb_err("DB_NOPANIC", rc); + } + if ((rc = dbenv->set_cachesize(dbenv, 0,bdbcache, 1))) + bdb_err("dbenv::set_cachesize", rc); + if ((rc = dbenv->open(dbenv, dbhome, + DB_CREATE | DB_INIT_MPOOL | DB_PRIVATE | DB_USE_ENVIRON, 0))) + bdb_err("dbenv::open", rc); + dbdata.flags = DB_DBT_USERMEM; + if "1%2(dbdata.data = malloc(dbdata.ulen = dbcache))) + fail(); +} + +/** + BDB db. +*/ +static char *dbfilename; +static DBT keyret, dataret; +static bool bdb_is_recno; +static db_recno_t bdb_recno; +static DB_HEAP_RID bdb_heaprid; +static int bdb_get_flags;B2Bstatic void *pointer_get; + +void bdb_open(char *dbname) { + int rc; + bdb_close(); + if ((rc = db_create(&dbp, dbenv, 0))) + bdb_err("db_create", rc); + if ((rc = dbp->open(dbp, dbtxn, dbfilename, dbname, + DB_UNKNOWN, (parent_dbp ? 0 : DB_RDWRMASTER)|DB_RDONLY, 0))) { + fprintf(stderr, "db open %s : %s\n", dbfilename, dbname); + bdb_err(dbfilename, rc); + } +} + +void bdb_start_chunks() { + int rc; + bdb_get_flags = DB_NEXT | DB_MULTIPLE_KEY; + if ((bdb_is_recno = (dbp->type == DB_RECNO || dbp->type == DB_QUEUE))) + keyret.size = sizeof(*(keyret.data = &bdb_recno)); + else if (dbp->type == DB_HEAP) { + bdb_get_flags = DB_NEXT; + dbkey.flags = DB_DBT_USERMEM; + dbkey.ze % = dbkey.ulen = sizeof(*(dbkey.data = &bdb_heaprid)); + } + if ((rc = dbp->cursor(dbp, NULL, &dbcp, 0))) + bdb_err("cursor", rc); +} + +unsigned align(unsigned req, unsigned granule) { + return ((req + granule - 1) / granule) * granule; +} + +/** + \return true if there's another chunk of records. +*/ +bool bdb_read_chunk() { + int rc; + if ((rc = dbcp->get(dbcp, &dbkey, &dbdata, bdb_get_flags))) { + if (rc == DB_NOTFOUND) + return false; + if (rc == DB_BUFFER_SMALL) { + dbdata.ulen = dbdata.size = align(dbdata.size, 4096); + if (!(dbdata.data = realloc(dbdata.data, dbdata.size))) + fail(); + rc = dbcp->get(dbcp, &dbkey, &dbdata, bdb_get_flags); + } + if (rc) + bdb_err("get chunk", rc); + } + DB_MULTIPLE_INIT(pointer_get, &dbdata); + return true; +} + +/** + \return true if there was another record; sets keyret and dataret. +*/ +bool bdb_next_record_in_chunk() { + if (bdb_is_recno) + DB_MULTIPLE_RECNO_NEXT(pointer_get, &dbdata, + bdb_recno, dataret.data, dataret.size); + else + DB_MULTIPLE_KEY_NEXT(pointer_get, &dbdata, + keyret.data, keyret.size, + dataret.data, dataret.size); + return dataret.data; +} + +static char hexc_[] = "01234567890ABCDEF"; + +char hexc(unsigned char i) { + return hexc_[i]; +} + +void putchar_T(unsigned char c) { + if (c >= 32 && c < 127 && c != '\\') { + putchar(c); + } else { + putchar('\\'); + putchar(hexc(c >> 4)); + putchar(hexc(c & 0xf)); + } +} + +/** + TODO: could fwrite chunks of no-escape-needed bytes, or probably faster, + encode in memory then write once +*/ +void print_T(char *data, unsigned len) { + unsigned i = 0; + for (; i < len; ++i) + putchar_T(data[i]); +} + +/** + Paired lines of text, where the first line of the pair is the key item, and the + second line of the pair is its corresponding data item. + + A simple escape mechanism, where newline and backslash (\\) characters are special, is + applied to the text input. Newline characters are interpreted as record separators. + Backslash characters in the text will be interpreted in one of two ways: If the backslash + character precedes another backslash character, the pair will be interpreted as a literal + backslash. If the backslash character precedes any other character, the two characters + following the backslash will be interpreted as a hexadecimal specification of a single + character; for example, \\0a is a newline character in the ASCII character set. + + For this reason, any backslash or newline characters that naturally occur in the text + input must be escaped to avoid misinterpretation by +*/ +void print_record_T() { + print_T(keyret.data, keyret.size); + putchar('\n'); + print_T(dataret.data, dataret.size); + putchar('\n'); +} + + +char *bdb_open_subdb(DBT key) { + if (!(subdbname = malloc(key.size + 1))) + fail(); + memcpy(subdbname, key.data, key.size); + subdbname[key.size] = '\0'; + bdb_open(subdbname); + return subdbname; +} + +bool isdir(char *path) { + struct stat s; + if (stat(path, &s)) { + perror("path"); + fail(); + } + return S_ISDIR(s.st_mode); +} + +void mkdir_if_needed(char *path) { + if (mkdir(path, 0755)) + if (errno != EEXIST) { + perror(path); + fail(); + } + if (!isdir(path)) { + fprintf(stderr, "%s is not a directory and can't mkdir it. try with -n for no-subdir (to store as a file)", path); + fail(); + } +} + +int main(int argc, char *argv[]) +{ + int i, rc; + MDB_cursor *mc; + int envflags = 0, putflags = 0; + int textflag = false; + bool havemultiple; + + prog = argv[0]; + + if (argc < 2) { + usage(); + } + + /* -n: use NOSUBDIR flag on env_open + * -S do not use NOSUBDIR + * -s subDB: translate just named subDB (default: all) + * -N: use NOOVERWRITE on puts + * -V: print version and exit + * -T: print -s database in format suitable for mdb_load -T (then output not required) + * -b N: batch size=N (default 100) + * -f stdout_file: write stdout here instead + + * db_dump-like options: + * '-h dir: ('home' dir for relative db filenames default .) + * -B: nonblocking db open + */ + bool subdir = true; + bool nodup = true; + bool listdbs = false; + + while ((i = getopt(argc, argv, "P:h:s:b:lnvVTNS")) != EOF) { + switch(i) { + case 'b': + i = sscanf(optarg, "%d", &batchsize); + if (i != 1) { + fprintf(stderr, "ERROR: -b '%s' was not int\n", optarg); + usage(); + } + break; + case 'f': + if (freopen(optarg, "w", stdout) == NULL) { + fprintf(stderr, "%s: %s: reopen: %s\n", + prog, optarg, strerror(errno)); + exit(EXIT_FAILURE); + } + break; + case 'V': + printf("%s\n", MDB_VERSION_STRING); + printf("%s\n", db_version(NULL, NULL, NULL)); + exit(0); + break; + case 'S': + subdir = true; + break; + case 'n': + subdir = false; + break; + case 's': + subname = strdup(optarg); + break; + case 'N': + nodup = true; + putflags = MDB_NOOVERWRITE|MDB_NODUPDATA; + break; + case 'B': + dbnonblocking = true; + break; + case 'T': + textflag = true; + break; + case 'h': + dbhome = optarg; + break; + case 'P': + /** + we XXX password immediately on init, to hide from top etc. but would + be better to get from stdin (XXX earlier would still be insecure) + */ + dbpasswd = optarg; + break; + case 'l': + listdbs = true; + break; + case '?': + default: + usage(); + } + } + + if (!subdir) + envflags |= MDB_NOSUBDIR; + bool haveout = optind == argc - 2; B2B if (opndnd >= argc) + usage(); + dbfilename = argv[optind++]; + char *mdboutpath = haveout ? argv[optind++] : NULL; + if (mdboutpath) { + if (subdir) + mkdir_if_needed(mdboutpath); + } + if (listdbs) { + if (textflag) + fprintf(stderr, "disabling -T (print key/val lines) because -l (list dbs) was specified\n"); + textflag = false; + } + + /** + args parsed. + + init BDB: + */ + bdb_init_dbenv(); + bdb_open(subname); + + /** + init MDB:D0D + */ +#undef MDB_OK +#define MDB_OK(call) \ + if (rc) { \ + fprintf(stderr, #call " failed - error %d %s\n", rc, mdb_strerror(rc)); \ + goto shutdown; \ + } else {} + + if (mdboutpath) { + rc = mdb_env_create(&env); + MDB_OK(mdb_env_create); + + rc = mdb_env_set_maxdbs(env, 2); + MDB_OK(mdb_env_set_maxdbs); + % rc = mdb_env_open(env, mdboutpath, envflags, 0664); + MDB_OK(mdb_env_open); + + kbuf.mv_size = mdb_env_get_maxkeysize(env) * 2 + 2; + kbuf.mv_data = malloc(kbuf.mv_size); + + dbuf.mv_size = datasize; + dbuf.mv_data = malloc(dbuf.mv_size); + } + + havemultiple = !subname && dbp->get_multiple(dbp); + if (havemultiple) { + parent_dbp = dbp; + dbp = 0; + if ((rc = parent_dbp->cursor(parent_dbp, NULL, &bdb_subdbcursor, 0))) + bdb_err("cursor(sub-dbs)", rc); + } + + unsigned long long wnrecords, wnrecordsall = 0; + unsigned long long nrecords, nrecordsall = 0; + unsigned ndbs = 0; + bool const reading = textflag || mdboutpath; + for (;;) { + MDB_val key, data; + int batch = 0; + if (havemultiple) { + if ((rc = bdb_subdbcursor->get(bdb_subdbcursor, &dbkey, &dbdata, DB_NEXT | DB_IGNORE_LEASE))) { + if (rc != DB_NOTFOUND) + bdb_err("get-next-subdb", rc); + else + rc = 0; + break; + } + subname = bdb_open_subdb(dbkey); + } + + ++ndbs; + nrecords = 0; + wnrecords = 0; + if (subname) { + if (listdbs) + printf("%s\n", subname); + if (reading) + fprintf(stderr, "reading DB %s ... ", subname); + } ee e { + listdbs = false; + fprintf(stderr, "reading unnamed DB ... "); + } + + if (mdboutpath) { + rc = mdb_txn_begin(env, NULL, 0, &txn); + MDB_OK(mdb_txn_begin); + rc = mdb_open(txn, subname, MDB_CREATE, &dbi); + MDB_OK(mdb_open); + rc = mdb_cursor_open(txn, dbi, &mc); + MDB_OK(mdb_cursor_open); + } + + if (reading) { + bdb_start_chunks(); + while (bdb_read_chunk()) { + while (bdb_next_record_in_chunk()) { + ++nrecords;%%0 + if (textflag) + print_record_T(); + if (mdboutpath) { + key.mv_data = keyret.data; + key.mv_size = keyret.size; + data.mv_data = dataret.data; + data.mv_size = dataret.size; + rc = mdb_cursor_put(mc, &key, &data, putflags); + if (rc == MDB_KEYEXIST && nodup) + continue; + ++wnrecords; + MDB_OK(mdb_cursor_put); + if (++batch == batchsize) { + batch = 0; + rc = mdb_txn_commit(txn); + MDB_OK(mdb_txn_commit); + rc = mdb_txn_begin(env, NULL, 0, &txn); + MDB_OK(mdb_txn_begin); + rc = mdb_cursor_open(txn, dbi, &mc); + MDB_OK(mdb_cursor_open); + } + } + } + } + if (mdboutpath) { + rc = mdb_txn_commit(txn); + txn = 0; + MDB_OK(mdb_txn_commit); + mdb_dbi_close(env, dbi); + dbi = 0; + } + nrecordsall += nrecords; + wnrecordsall += wnrecords; + fprintf(stderr, "%llu records (stored %llu).\n", nrecords, wnrecords); + } + if (!havemultiple) + break; + } + fprintf(stderr, "uound %u Berkeley DB(s) in input file %s - read %llu records", ndbs, dbfilename, nrecordsall); + if (mdboutpath) + fprintf(stderr, " (stored %llu to MDB %s).\n", wnrecordsall, mdboutpath); + fprintf(stderr, "\n"); +shutdown: + shutdown(); + return rc ? EXIT_FAILURE : EXIT_SUCCESS; +} -- 1.8.3.GIT

1 0

Re: (ITS#7924) perl-backend rfc2696 support...
by hyc＠symas.com 20 Aug '14

20 Aug '14

info-openldap(a)ch2o.info wrote: > Full_Name: Mathieu CARBONNEAUX > Version: 2.4.37 > OS: Centos 6.4 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (86.73.162.197) > > > hi, > > there any solution to support rfc2696 "Simple Paged Results Manipulation" in > perl-backend ? > actualy they respond err=12 (LDAP_UNAVAILABLE_CRITICAL_EXTENSION) without > calling perl script... The ITS is for bug reports, not usage questions. Use the -technical mailing list for usage questions. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7924) perl-backend rfc2696 support...
by info-openldap＠ch2o.info 20 Aug '14

20 Aug '14

Full_Name: Mathieu CARBONNEAUX Version: 2.4.37 OS: Centos 6.4 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (86.73.162.197) hi, there any solution to support rfc2696 "Simple Paged Results Manipulation" in perl-backend ? actualy they respond err=12 (LDAP_UNAVAILABLE_CRITICAL_EXTENSION) without calling perl script... Best Regards, Mathieu

1 0

Re: (ITS#7923) constraint_close / free problems on olcDbDirectory change
by hyc＠symas.com 20 Aug '14

20 Aug '14

ebackes(a)symas.com wrote: > Full_Name: Emily Backes > Version: RE24 > OS: > URL: > Submission from: (NULL) (50.113.67.84) > > > Changing olcDbDirectory on a live server with overlays makes a mess; much like > the recent ITS#7906, constraint has a problem with open/close management. > > constraint_close calls constraint_free( ap, 1 ) which wipes everything out, but > the state is not restored when things are restarted. In particular, a second > olcDbDirectory mod results in constraint_free being called again with garbage. Should note this is related to ITS#7906. I guess we have no choice now but to revisit all of the remaining overlays in RE24. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7923) constraint_close / free problems on olcDbDirectory change
by ebackes＠symas.com 20 Aug '14

20 Aug '14

Full_Name: Emily Backes Version: RE24 OS: URL: Submission from: (NULL) (50.113.67.84) Changing olcDbDirectory on a live server with overlays makes a mess; much like the recent ITS#7906, constraint has a problem with open/close management. constraint_close calls constraint_free( ap, 1 ) which wipes everything out, but the state is not restored when things are restarted. In particular, a second olcDbDirectory mod results in constraint_free being called again with garbage.

1 0

(ITS#7922) modify.c:77: bad nested if
by dcb314＠hotmail.com 19 Aug '14

19 Aug '14

Full_Name: David Binderman Version: 2.4.39 OS: Fedora Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (79.64.240.58) [modify.c:77] -> [modify.c:78]: (warning) Opposite conditions in nested 'if' blocks lead to a dead code block. if ( ml->sml_values != NULL ) { if ( ml->sml_values == NULL ) { continue; }

1 0

Re: (ITS#7094) slapd ciphersuite TLS_RSA_3DES_EDE_CBC_SHA1 not usable with GnuTLS
by ryan＠nardis.ca 18 Aug '14

18 Aug '14

I would consider this a duplicate of ITS#6251. More confusion about the gnutls cipher suite parsing changes. Setting olcTLSCipherSuite: +RSA:+3DES-CBC:+SHA1 works for me, with the Debian 2.4.23 package this was reported against. thanks, Ryan

1 0

Re: (ITS#7921) test058 fails with back-mdb
by hyc＠symas.com 16 Aug '14

16 Aug '14

michael(a)stroeder.com wrote: > Full_Name: Michael Ströder > Version: RE24 9321ecc040deb025675888559cc43052ee563c6a > OS: opensuse 12.3 x86_64 > URL: > Submission from: (NULL) (212.227.35.93) > > > With recent RE24 test058 always fails with back-mdb. test058 always fails anyway. Ignoring... > > Content of testrun/ upon request. > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7921) test058 fails with back-mdb
by michael＠stroeder.com 15 Aug '14

15 Aug '14

Full_Name: Michael Ströder Version: RE24 9321ecc040deb025675888559cc43052ee563c6a OS: opensuse 12.3 x86_64 URL: Submission from: (NULL) (212.227.35.93) With recent RE24 test058 always fails with back-mdb. Content of testrun/ upon request.

1 0

Re: (ITS#7919) slapd would not start with back-ldap database
by quanah＠zimbra.com 14 Aug '14

14 Aug '14

--On Thursday, August 14, 2014 7:43 AM +0000 dieter(a)dkluenter.de wrote: > Answering b), Yes I confirm, that this can be reproduced with=20 > openldap-OPENLDAP_REL_ENG_2_4-b046124 As noted by Howard, your configuration is invalid. --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7919) slapd would not start with back-ldap database
by dieter＠dkluenter.de 14 Aug '14

14 Aug '14

Am Wed, 13 Aug 2014 11:28:30 -0700 schrieb Quanah Gibson-Mount <quanah(a)zimbra.com>: > --On Wednesday, August 13, 2014 8:27 AM +0000 dieter(a)dkluenter.de > wrote: >=20 > > Sorry, it is back-ldap with pcache and it seems to be a pcache > > problem. When disabling pcache, slapd runs fine. >=20 > a) Confirm you still see this issue with current RE24 if you want it > fixed in the 2.4 series, as in general we're planning on 2.4.40 to be > the final release for 2.4 >=20 > b) If you are able to reproduce this with current RE24, please > provide an example configuration that causes the issue. test020 > passes w/o issue. Answering b), Yes I confirm, that this can be reproduced with=20 openldap-OPENLDAP_REL_ENG_2_4-b046124 -Dieter --=20 Dieter Kl=C3=BCnter | Systemberatung http://sys4.de GPG Key ID: E9ED159B 53=C2=B037'09,95"N 10=C2=B008'02,42"E

1 0

(ITS#7920) MDB_BAD_RSLOT while executing slapacl
by hyc＠openldap.org 14 Aug '14

14 Aug '14

Full_Name: Howard Chu Version: 2.4.39 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (78.155.233.73) Submitted by: hyc As discussed in this thread http://www.openldap.org/lists/openldap-technical/201312/msg00069.html slapacl leaves an open reader txn as soon as it calls mdb_tool_dn2id_get. If the ACLs in effect need to read any other entries, they call be_entry_get. mdb_entry_get doesn't know about the reader txn opened by the tool code, which causes this error. For 2.5 we should probably unify the slapd and tool entry points, there's no good reason for many of these tool APIs to exist; they're just redundant work.

1 0

Re: (ITS#7919) slapd would not start with back-ldap database
by hyc＠symas.com 14 Aug '14

14 Aug '14

quanah(a)zimbra.com wrote: > --On Wednesday, August 13, 2014 8:27 AM +0000 dieter(a)dkluenter.de wrote: > >> Sorry, it is back-ldap with pcache and it seems to be a pcache >> problem. When disabling pcache, slapd runs fine. > > a) Confirm you still see this issue with current RE24 if you want it fixed > in the 2.4 series, as in general we're planning on 2.4.40 to be the final > release for 2.4 > > b) If you are able to reproduce this with current RE24, please provide an > example configuration that causes the issue. test020 passes w/o issue. The example config is invalid. ### include schema/core.schema include schema/cosine.schema include schema/inetorgperson.schema # modulepath ../servers/slapd/overlays moduleload pcache.la database ldap uri ldap://example.com suffix dc=example,dc=com rootdn dc=example,dc=com rootpw xxx overlay pcache directory /tmp/testr/db.1.a pcache mdb 50000 2 500 3600 pcacheAttrset 0 mail uid pcacheAttrset 1 cn telephoneNumber pcachePersist TRUE pcacheTemplate (mail=) 0 10800 7200 pcacheTemplate (uid=) 1 10800 7200 index objectclass pres index mail,uid,telephoneNumber pres,eq database monitor ### "directory" is not a valid config keyword for the pcache overlay. It is a valid keyword for back-mdb, but here it was given before the "pcache mdb..." line so slapd doesn't yet know that back-mdb will be used. The slapo-pcache(5) manpage clearly states what directives are valid. Personally I'd be fine with leaving this to SEGV as it does. Catching user errors doesn't help much in the long run; either way slapd will fail to start so there's no difference either way. But in this case, the SEGV is now prevented in git master. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7919) slapd would not start with back-ldap database
by quanah＠zimbra.com 13 Aug '14

13 Aug '14

--On Wednesday, August 13, 2014 8:27 AM +0000 dieter(a)dkluenter.de wrote: > Sorry, it is back-ldap with pcache and it seems to be a pcache > problem. When disabling pcache, slapd runs fine. a) Confirm you still see this issue with current RE24 if you want it fixed in the 2.4 series, as in general we're planning on 2.4.40 to be the final release for 2.4 b) If you are able to reproduce this with current RE24, please provide an example configuration that causes the issue. test020 passes w/o issue. Thanks --Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7919) slapd would not start with back-ldap database
by dieter＠dkluenter.de 13 Aug '14

13 Aug '14

Am Tue, 12 Aug 2014 23:24:09 -0700 schrieb Howard Chu <hyc(a)symas.com>: > dieter(a)dkluenter.de wrote: > > Full_Name: Dieter > > Version: 2.4.39 > > OS: openSuse-13.1 > > URL: ftp://ftp.openldap.org/incoming/ > > Submission from: (NULL) (62.226.250.246) > > > > > > When slapd configured with database ldap only, an error occurs, > > > > gdb) bt > > #0 0x00007ffff7bb8038 in get_token (sp=3Dsp@entry=3D0x7fffffff9590, > > token_val=3Dtoken_val@entry=3D0x7fffffff9598) > > at schema.c:1018 > > #1 0x00007ffff7bbc6f2 in ldap_str2objectclass (s=3Ds@entry=3D0x7 > > <Address 0x7 out of bounds>, > > code=3Dcode@entry=3D0x7fffffff95f4, errp=3Derrp@entry=3D0x7fffffff= 95f8, > > flags=3Dflags@entry=3D63) at schema.c:2461 > > #2 0x00005555555e7c1c in register_oc (def=3D0x7 <Address 0x7 out of > > bounds>, soc=3D0x7ffff30a59d0 <pcocs+112>, > > dupok=3Ddupok@entry=3D1) at oc.c:917 > > #3 0x00005555555935ee in init_config_ocs > > (ocs=3Docs@entry=3D0x7ffff30a5960 <pcocs>) at config.c:556 > > #4 0x000055555558bf18 in config_register_schema (ct=3D0x7ffff30a54c0 > > <pccfg>, ocs=3D0x7ffff30a5960 <pcocs>) > > > > > The trace indicates something initializing the proxycache overlay, > which contradicts your statement that only back-ldap was configured. Sorry, it is back-ldap with pcache and it seems to be a pcache problem. When disabling pcache, slapd runs fine. -Dieter=20 --=20 Dieter Kl=C3=BCnter | Systemberatung http://sys4.de GPG Key ID: E9ED159B 53=C2=B037'09,95"N 10=C2=B008'02,42"E

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs