Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
Thanks!
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
It seems something's mdb-specific is already in RE24. So no testing now?
Ciao, Michael.
----------------------------------- snip ---------------------------------- cd back-ldif; make -w depend make[3]: Entering directory `/usr/src/michael/openldap/servers/slapd/back-ldif' ../../../build/mkdep -l -d "." -c "cc" -m "-M" -I../../../include -I../../../include -I.. -I./.. -I/usr/include -I/usr/include -I/usr/include -I/usr/include ldif.c make[3]: Leaving directory `/usr/src/michael/openldap/servers/slapd/back-ldif'
cd back-mdb; make -w depend make[3]: Entering directory `/usr/src/michael/openldap/servers/slapd/back-mdb' make[3]: *** No rule to make target `Makefile.in', needed by `Makefile'. Stop. make[3]: Leaving directory `/usr/src/michael/openldap/servers/slapd/back-mdb' make[2]: *** [depend-local-srv] Error 1 make[2]: Leaving directory `/usr/src/michael/openldap/servers/slapd' make[1]: *** [depend-common] Error 1 make[1]: Leaving directory `/usr/src/michael/openldap/servers' make: *** [depend-common] Error 1
--On Friday, November 04, 2011 6:15 PM +0100 Michael Ströder michael@stroeder.com wrote:
----------------------------------- snip ---------------------------------- cd back-ldif; make -w depend make[3]: Entering directory `/usr/src/michael/openldap/servers/slapd/back-ldif' ../../../build/mkdep -l -d "." -c "cc" -m "-M" -I../../../include -I../../../include -I.. -I./.. -I/usr/include -I/usr/include -I/usr/include -I/usr/include ldif.c make[3]: Leaving directory `/usr/src/michael/openldap/servers/slapd/back-ldif'
cd back-mdb; make -w depend make[3]: Entering directory `/usr/src/michael/openldap/servers/slapd/back-mdb' make[3]: *** No rule to make target `Makefile.in', needed by `Makefile'. Stop. make[3]: Leaving directory `/usr/src/michael/openldap/servers/slapd/back-mdb' make[2]: *** [depend-local-srv] Error 1 make[2]: Leaving directory `/usr/src/michael/openldap/servers/slapd' make[1]: *** [depend-common] Error 1 make[1]: Leaving directory `/usr/src/michael/openldap/servers' make: *** [depend-common] Error 1
Seems like you have a rather odd source checkout. I get no references to back-mdb at all.
I do however find breakage in back-hdb I'm investigating.
modify.c: In function 'hdb_modify_internal': modify.c:267: error: duplicate case value modify.c:217: error: previously used here modify.c:291: error: duplicate case value modify.c:241: error: previously used here make: *** [modify.lo] Error 1
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
--On Friday, November 04, 2011 10:58 AM -0700 Quanah Gibson-Mount quanah@zimbra.com wrote:
Seems like you have a rather odd source checkout. I get no references to back-mdb at all.
I do however find breakage in back-hdb I'm investigating.
That was my own fault (had 2.4.26 patches applied to RE24). Current RE24 builds fine for me, no references to back-mdb.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
(Now I have completely separate trees for master and re24 to avoid accidential build mixes...)
Ciao, Michael.
--On Friday, November 04, 2011 9:47 PM +0100 Michael Ströder michael@stroeder.com wrote:
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
Yeah, Howard picked out a few more things for 2.4.27. Now would be a good time to run make test again, as I'm caught up on those bits and am back to working on mdb.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Friday, November 04, 2011 9:47 PM +0100 Michael Ströder michael@stroeder.com wrote:
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
Yeah, Howard picked out a few more things for 2.4.27.
Any chance for ITS#6984 and ITS#7019?
Ciao, Michael.
--On Monday, November 07, 2011 1:13 PM +0100 Michael Ströder michael@stroeder.com wrote:
Quanah Gibson-Mount wrote:
--On Friday, November 04, 2011 9:47 PM +0100 Michael Ströder michael@stroeder.com wrote:
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
Yeah, Howard picked out a few more things for 2.4.27.
Any chance for ITS#6984 and ITS#7019?
6984 done for RE24. Please test.
Someone else will need to address 7019.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Monday, November 07, 2011 1:13 PM +0100 Michael Ströder michael@stroeder.com wrote:
Any chance for ITS#6984 and ITS#7019?
6984 done for RE24. Please test.
Thanks! Works for me (with demo script in python-ldap's source distribution).
Ciao, Michael.
Michael Ströder wrote:
Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
make test and my local configuration runs fine also with back-mdb.
Ciao, Michael.
Michael Ströder wrote:
Michael Ströder wrote:
Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
make test and my local configuration runs fine also with back-mdb.
Hmm, slapaad -q seg faults. slapadd works. Maybe the new pipelining. Can't investigate in detail since I'm in a hurry...
Ciao, Michael.
Michael Ströder wrote:
Michael Ströder wrote:
Michael Ströder wrote:
Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
make test and my local configuration runs fine also with back-mdb.
Hmm, slapaad -q seg faults. slapadd works. Maybe the new pipelining. Can't investigate in detail since I'm in a hurry...
Need backtrace from SEGV, steps to reproduce. Not seeing it here.
Howard Chu wrote:
Michael Ströder wrote:
Michael Ströder wrote:
Michael Ströder wrote:
Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
make test and my local configuration runs fine also with back-mdb.
Hmm, slapaad -q seg faults. slapadd works. Maybe the new pipelining. Can't investigate in detail since I'm in a hurry...
Need backtrace from SEGV, steps to reproduce. Not seeing it here.
Note the very same action works with OpenLDAP 2.4.26.
(gdb) info threads 2 Thread 14284 0x00007f7ef725638c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 * 1 Thread 14283 0x00007f7ef62e5a4b in memset () from /lib64/libc.so.6 (gdb) thread apply all bt
Thread 2 (Thread 14284): #0 0x00007f7ef725638c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7ef828bf33 in ldap_pvt_thread_cond_wait (cond=0x7f7ef44e33e0, mutex=0x7f7ef44e33a0) at thr_posix.c:277 #2 0x00007f7ef429c215 in bdb_tool_trickle_task (ctx=0x7f7ef06deb60, ptr=0x8a34a0) at tools.c:1280 #3 0x00007f7ef828aa50 in ldap_int_thread_pool_wrapper (xpool=0x7c4cd0) at tpool.c:685 #4 0x00007f7ef7251a3f in start_thread () from /lib64/libpthread.so.0 #5 0x00007f7ef633866d in clone () from /lib64/libc.so.6 #6 0x0000000000000000 in ?? ()
Thread 1 (Thread 14283): #0 0x00007f7ef62e5a4b in memset () from /lib64/libc.so.6 #1 0x00007f7ef429a806 in bdb_tool_index_add (op=0x7fff3bdb5720, txn=0x0, e=0x8a6858) at tools.c:596 #2 0x00007f7ef429ade2 in hdb_tool_entry_put (be=0x869510, e=0x8a6858, text=0x7fff3bdb5980) at tools.c:695 #3 0x00000000004d27a4 in slapadd (argc=9, argv=0x7fff3bdb5cf8) at slapadd.c:428 #4 0x000000000041597e in main (argc=9, argv=0x7fff3bdb5cf8) at main.c:410
Ciao, Michael.
Michael Ströder wrote:
Howard Chu wrote:
Need backtrace from SEGV, steps to reproduce. Not seeing it here.
Note the very same action works with OpenLDAP 2.4.26.
(gdb) info threads 2 Thread 14284 0x00007f7ef725638c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
- 1 Thread 14283 0x00007f7ef62e5a4b in memset () from /lib64/libc.so.6
(gdb) thread apply all bt
Thread 2 (Thread 14284): #0 0x00007f7ef725638c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7ef828bf33 in ldap_pvt_thread_cond_wait (cond=0x7f7ef44e33e0, mutex=0x7f7ef44e33a0) at thr_posix.c:277 #2 0x00007f7ef429c215 in bdb_tool_trickle_task (ctx=0x7f7ef06deb60, ptr=0x8a34a0) at tools.c:1280 #3 0x00007f7ef828aa50 in ldap_int_thread_pool_wrapper (xpool=0x7c4cd0) at tpool.c:685 #4 0x00007f7ef7251a3f in start_thread () from /lib64/libpthread.so.0 #5 0x00007f7ef633866d in clone () from /lib64/libc.so.6 #6 0x0000000000000000 in ?? ()
Thread 1 (Thread 14283): #0 0x00007f7ef62e5a4b in memset () from /lib64/libc.so.6 #1 0x00007f7ef429a806 in bdb_tool_index_add (op=0x7fff3bdb5720, txn=0x0, e=0x8a6858) at tools.c:596 #2 0x00007f7ef429ade2 in hdb_tool_entry_put (be=0x869510, e=0x8a6858, text=0x7fff3bdb5980) at tools.c:695 #3 0x00000000004d27a4 in slapadd (argc=9, argv=0x7fff3bdb5cf8) at slapadd.c:428 #4 0x000000000041597e in main (argc=9, argv=0x7fff3bdb5cf8) at main.c:410
It's a simple "slapadd -q". Without -q it works fine. git master and RE24 seg faults with -q.
More from gdb:
(gdb) thread 1 [Switching to thread 1 (Thread 14659)]#3 0x00000000004d27a4 in slapadd (argc=9, argv=0x7fffaeed3468) at slapadd.c:428 428 id = be->be_entry_put( be, erec.e, &bvtext ); (gdb) bt full #0 0x00007f0c18638a4b in memset () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f0c165ed806 in bdb_tool_index_add (op=0x7fffaeed2e90, txn=0x0, e=0x8a6858) at tools.c:596 ir = 0x0 i = 0 rc = 425324080 a = 0x7fffaeed2e70 bdb = 0x8696b0 #2 0x00007f0c165edde2 in hdb_tool_entry_put (be=0x869510, e=0x8a6858, text=0x7fffaeed30f0) at tools.c:695 rc = 0 bdb = 0x8696b0 tid = 0x0 op = {o_hdr = 0x7fffaeed2d40, o_tag = 0, o_time = 0, o_tincr = 0, o_bd = 0x869510, o_req_dn = {bv_len = 0, bv_val = 0x0}, o_req_ndn = {bv_len = 0, bv_val = 0x0}, o_request = {oq_add = {rs_modlist = 0x0, rs_e = 0x0}, oq_bind = {rb_method = 0, rb_cred = {bv_len = 0, bv_val = 0x0}, rb_edn = { bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = {bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0x0}, oq_modify = {rs_mods = { rs_modlist = 0x0, rs_no_opattrs = 0 '\000'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x0, rs_no_opattrs = 0 '\000'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = 0, rs_deref = 0, rs_slimit = 0, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = {rs_msgid = 0}, oq_cancel = {rs_msgid = 0}, oq_extended = {rs_reqoid = {bv_len = 0, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended = {rs_reqoid = {bv_len = 0, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = {bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 0 '\000', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 31 times>, o_controls = 0x0, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x0, o_ctrls = 0x0, o_csn = {bv_len = 0, bv_val = 0x0}, o_private = 0x0, o_extra = {slh_first = 0x0}, o_next = { stqe_next = 0x0}} ohdr = {oh_opid = 0, oh_connid = 0, oh_conn = 0x0, oh_msgid = 0, oh_protocol = 0, oh_tid = 0, oh_threadctx = 0x0, oh_tmpmemctx = 0x0, oh_tmpmfuncs = 0x755080, oh_counters = 0x0, oh_log_prefix = '\000' <repeats 255 times>} __PRETTY_FUNCTION__ = "hdb_tool_entry_put" #3 0x00000000004d27a4 in slapadd (argc=9, argv=0x7fffaeed3468) at slapadd.c:428 textbuf = '\000' <repeats 255 times> textlen = 256 erec = {e = 0x8a6858, lineno = 1, nextline = 20} bvtext = {bv_len = 256, bv_val = 0x7fffaeed3110 ""} thr = 139689963340960 id = 4 ldifrc = 1 rc = 0 stat_buf = {st_dev = 8, st_ino = 84778, st_nlink = 1, st_mode = 4480, st_uid = 500, st_gid = 100, __pad0 = 0, st_rdev = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1320667075, tv_nsec = 601664002}, st_mtim = {tv_sec = 1320667075, tv_nsec = 625664002}, st_ctim = {tv_sec = 1320667075, tv_nsec = 625664002}, __unused = {0, 0, 0}} #4 0x000000000041597e in main (argc=9, argv=0x7fffaeed3468) at main.c:410 i = 0 no_detach = 0 rc = 1 urls = 0x0 username = 0x0 groupname = 0x0 sandbox = 0x0 syslogUser = 160 pid = 32767 waitfds = {1, 32524} g_argc = 9 g_argv = 0x7fffaeed3468 configfile = 0x0 configdir = 0x0 serverName = 0x7fffaeed4305 "slapadd" serverMode = 1 scp = 0x0 scp_entry = 0x0 debug_unknowns = 0x0 syslog_unknowns = 0x0 serverNamePrefix = 0x4f3ae8 "" l = 0 slapd_pid_file_unlink = 0 slapd_args_file_unlink = 0 firstopt = 1 __PRETTY_FUNCTION__ = "main" (gdb) thread 2 [Switching to thread 2 (Thread 14660)]#0 0x00007f0c195a938c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 (gdb) bt full #0 0x00007f0c195a938c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 No symbol table info available. #1 0x00007f0c1a5def33 in ldap_pvt_thread_cond_wait (cond=0x7f0c168363e0, mutex=0x7f0c168363a0) at thr_posix.c:277 No locals. #2 0x00007f0c165ef215 in bdb_tool_trickle_task (ctx=0x7f0c12a31b60, ptr=0x8a34a0) at tools.c:1280 env = 0x8a34a0 wrote = 32524 #3 0x00007f0c1a5dda50 in ldap_int_thread_pool_wrapper (xpool=0x7c4cd0) at tpool.c:685 pool = 0x7c4cd0 task = 0x8baad0 work_list = 0x7c4d68 ctx = {ltu_id = 139689829017344, ltu_key = {{ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, { ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x7f0c12a31c60}, {ltk_key = 0x0, ltk_data = 0x7f0c12a31c60, ltk_free = 0xe}, {ltk_key = 0x0, ltk_data = 0xa8428197, ltk_free = 0x7f0c1a8361a1 <do_lookup_x+1537>}, {ltk_key = 0x0, ltk_data = 0x17, ltk_free = 0x2a10a06}, {ltk_key = 0x0, ltk_data = 0x7f0c12a31db0, ltk_free = 0x7f0c185bbc40}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x7f0c1aa29000, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x7f0c1aa27000, ltk_free = 0x7f0c195a0d0f}, {ltk_key = 0x0, ltk_data = 0x7f0c1959ef68, ltk_free = 0x500000000}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x7f0c12a31e00, ltk_free = 0x7f0c1aa26600}, {ltk_key = 0x0, ltk_data = 0x7f0c1aa29000, ltk_free = 0xa8428197}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x7f0c1aa26600}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x1}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x7f0c12a31db0, ltk_free = 0x7f0c12a31dc8}, {ltk_key = 0x0, ltk_data = 0x7f0c195a0d0f, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, { ltk_key = 0x0, ltk_data = 0x7f0c185c15c8, ltk_free = 0x7f0c1aa27000}, {ltk_key = 0x0, ltk_data = 0xffffffff, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x218050, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x3, ltk_free = 0x7f0c1959e000}, {ltk_key = 0x0, ltk_data = 0x5, ltk_free = 0}, { ltk_key = 0x0, ltk_data = 0x7f0c185c15c8, ltk_free = 0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x7fffaeed2f30}, {ltk_key = 0x0, ltk_data = 0x7f0c1a8401e5, ltk_free = 0x7f0c12a32700}}} kctx = 0x0 i = 32 keyslot = 135 hash = 1295237255 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #4 0x00007f0c195a4a3f in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #5 0x00007f0c1868b66d in clone () from /lib64/libc.so.6 No symbol table info available. #6 0x0000000000000000 in ?? () No symbol table info available.
Michael Ströder wrote:
Howard Chu wrote:
Michael Ströder wrote:
Michael Ströder wrote:
Michael Ströder wrote:
Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
When should we test before integration of back-mdb? I ran make test without trouble but saw more committs coming after that.
make test and my local configuration runs fine also with back-mdb.
Hmm, slapaad -q seg faults. slapadd works. Maybe the new pipelining. Can't investigate in detail since I'm in a hurry...
Need backtrace from SEGV, steps to reproduce. Not seeing it here.
Note the very same action works with OpenLDAP 2.4.26.
Fixed in master.
(gdb) info threads 2 Thread 14284 0x00007f7ef725638c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
- 1 Thread 14283 0x00007f7ef62e5a4b in memset () from /lib64/libc.so.6
(gdb) thread apply all bt
Thread 1 (Thread 14283): #0 0x00007f7ef62e5a4b in memset () from /lib64/libc.so.6 #1 0x00007f7ef429a806 in bdb_tool_index_add (op=0x7fff3bdb5720, txn=0x0, e=0x8a6858) at tools.c:596 #2 0x00007f7ef429ade2 in hdb_tool_entry_put (be=0x869510, e=0x8a6858, text=0x7fff3bdb5980) at tools.c:695 #3 0x00000000004d27a4 in slapadd (argc=9, argv=0x7fff3bdb5cf8) at slapadd.c:428 #4 0x000000000041597e in main (argc=9, argv=0x7fff3bdb5cf8) at main.c:410
Ciao, Michael.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 11/03/11 17:10, Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
It seems that back-mdb really expects FDSYNC and fdatasync be available on target system. Currently FreeBSD does not have implemented these semantics (yet).
I've made a workaround in the FreeBSD port to make it work; it would be nice to have configure script test if FDSYNC and fdatasync is available and make the corresponding workaround (FSYNC and fsync) if not so building from source would just work.
Test shows test018-syncreplication-persist failed but I haven't took looked into it yet. Is this a known issue?
Cheers, - -- Xin LI delphij@delphij.net https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die
Xin LI wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 11/03/11 17:10, Quanah Gibson-Mount wrote:
Tomorrow I'll be working on merging in libmdb/back-mdb. If anyone would like to do a basic sanity test against current RE24, that would be helpful. It has all of the planned fixes for 2.4.27 except ITS#7025 which Howard is still working on.
It seems that back-mdb really expects FDSYNC and fdatasync be available on target system. Currently FreeBSD does not have implemented these semantics (yet).
I've made a workaround in the FreeBSD port to make it work; it would be nice to have configure script test if FDSYNC and fdatasync is available and make the corresponding workaround (FSYNC and fsync) if not so building from source would just work.
You're welcome to submit appropriate patches for configure.in and any affected code.
Test shows test018-syncreplication-persist failed but I haven't took looked into it yet. Is this a known issue?
Works fine for me.
Cheers,
Xin LIdelphij@delphij.net https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD)
iQEcBAEBCAAGBQJOuHxvAAoJEATO+BI/yjfBq3EIALPHhihXLsIHMEOdv/bR2u8j 9TMqMA0RAs0osOX15PRhTN+JqOSWT9q9kABHenyRoFZHxN1QKmFqaKjsNz/aECKm H8SqaLrFzJwO0exUmiO40eMXncci6ZvBhyir1p4C/CqxWxk55jY4APqvt4okTfhL pvHUv3O6Gcuxaf29+ZKc2G/qfwcSVV7nfzv6DAzk7z5IHzndVOZ4uCo3nQuXAgxY ej6qRCBaOOe5WMu0wKjg5uquoPUyWZnBbJHOIcGpdRE9PXuSZ7z72z9l1y8BTxih MrnvN4VopHmLrkZoCzPACwNwm7fB73dqtAmNNKpd0e3YssOCnZmDgquhOWGWkWY= =onQu -----END PGP SIGNATURE-----
On Mon, 7 Nov 2011, Howard Chu wrote:
It seems that back-mdb really expects FDSYNC and fdatasync be available on target system. Currently FreeBSD does not have implemented these semantics (yet).
I've made a workaround in the FreeBSD port to make it work; it would be nice to have configure script test if FDSYNC and fdatasync is available and make the corresponding workaround (FSYNC and fsync) if not so building from source would just work.
You're welcome to submit appropriate patches for configure.in and any affected code.
Do you envision issues with using msync() instead, across the board? I would think that, given that back-mdb only makes sense when _POSIX_MAPPED_FILES, msync would present a more portable way of getting to the underlying fsync()/fdatasync() calls. (Heck, on Linux, it's just a couple lines of error checking wrapping a vfs_fsync().)
Xin: is msync() available on FreeBSD? You're mentioning fsync as the proposed patch...
Aaron Richton wrote:
On Mon, 7 Nov 2011, Howard Chu wrote:
It seems that back-mdb really expects FDSYNC and fdatasync be available on target system. Currently FreeBSD does not have implemented these semantics (yet).
I've made a workaround in the FreeBSD port to make it work; it would be nice to have configure script test if FDSYNC and fdatasync is available and make the corresponding workaround (FSYNC and fsync) if not so building from source would just work.
You're welcome to submit appropriate patches for configure.in and any affected code.
Do you envision issues with using msync() instead, across the board? I would think that, given that back-mdb only makes sense when _POSIX_MAPPED_FILES, msync would present a more portable way of getting to the underlying fsync()/fdatasync() calls. (Heck, on Linux, it's just a couple lines of error checking wrapping a vfs_fsync().)
msync is not applicable here. It's only relevant if changes are made using the mmap'd region, and we don't.
Xin: is msync() available on FreeBSD? You're mentioning fsync as the proposed patch...
On Tue, 8 Nov 2011, Howard Chu wrote:
msync is not applicable here. It's only relevant if changes are made using the mmap'd region, and we don't.
Right, good point. A few one-liners that aren't really worth an ITS (yet?):
---
In libraries/libmdb/mdb.c, the compiler is warning about:
5920 rc = mdb_drop0(mc, mc->mc_db->md_flags & MDB_DUPSORT); 5921 if (rc) 5922 mdb_cursor_close(mc); 5923 return rc; 5924 5925 /* Can't delete the main DB */ 5926 if (del && dbi > MAIN_DBI) {
5926 being unreached (because of 5923). Braces on 5921?
---
#undef DEBUG, and "#define DKEY(x)"
preprocesses to:
( void ) ( "found leaf index %u [%s], rc = %i" , i, , rc);
so IMO:
#define DKEY(x) ""
or any other blank-ish concept...
---
On Solaris 9, stdint.h doesn't exist. inttypes.h does, and should (in theory) be a drop-in replacement. Or, we can try for C90-compliance (by using neither), or.......?
and
if it's C99-style types, servers/slapd/back-mdb uses u_int32_t which should probably be uint32_t. (Berkeley DB gets around this with some definitions in db.h, but I see no reason for mdb to have those hacks...)
----
Solaris 9 defines BYTE_ORDER via <resolv.h>, I don't see any danger in just #include'ing that on all platforms? (Apparently future standards might include an endian.h for this...there's no standard like no standard...)
----
PAGESIZE is part of SUSv2 <limits.h>; would you be open to a patch to change that to MDB_PAGESIZE?
Aaron Richton wrote:
On Tue, 8 Nov 2011, Howard Chu wrote:
msync is not applicable here. It's only relevant if changes are made using the mmap'd region, and we don't.
Right, good point. A few one-liners that aren't really worth an ITS (yet?):
In libraries/libmdb/mdb.c, the compiler is warning about:
5920 rc = mdb_drop0(mc, mc->mc_db->md_flags& MDB_DUPSORT); 5921 if (rc) 5922 mdb_cursor_close(mc); 5923 return rc; 5924 5925 /* Can't delete the main DB */ 5926 if (del&& dbi> MAIN_DBI) {5926 being unreached (because of 5923). Braces on 5921?
Fixed in mdb.master, thanks.
#undef DEBUG, and "#define DKEY(x)"
preprocesses to:
( void ) ( "found leaf index %u [%s], rc = %i" , i, , rc);so IMO:
#define DKEY(x) ""
or any other blank-ish concept...
DEBUG cannot (normally) be undef'd, so this is a misuse of the code.
On Solaris 9, stdint.h doesn't exist. inttypes.h does, and should (in theory) be a drop-in replacement. Or, we can try for C90-compliance (by using neither), or.......?
and
if it's C99-style types, servers/slapd/back-mdb uses u_int32_t which should probably be uint32_t. (Berkeley DB gets around this with some definitions in db.h, but I see no reason for mdb to have those hacks...)
Haven't really thought about this. If it's present on MacOSX then I guess it's safe to switch; it is present in Android/bionic.
Solaris 9 defines BYTE_ORDER via<resolv.h>, I don't see any danger in just #include'ing that on all platforms? (Apparently future standards might include an endian.h for this...there's no standard like no standard...)
Solaris uses <sys/byteorder.h> or <sys/isa_defs.h>, which is pulled in already via <sys/param.h>. I think all we need to do is add a few #ifdef _LITTLE_ENDIAN / _BIG_ENDIAN to map this over.
PAGESIZE is part of SUSv2<limits.h>; would you be open to a patch to change that to MDB_PAGESIZE?
Maybe. It might be sufficient just to surround ours with #ifndef PAGESIZE since the particular value isn't critical.
On Tue, 8 Nov 2011, Howard Chu wrote:
#undef DEBUG, and "#define DKEY(x)"
preprocesses to:
( void ) ( "found leaf index %u [%s], rc = %i" , i, , rc);so IMO:
#define DKEY(x) ""
or any other blank-ish concept...
DEBUG cannot (normally) be undef'd, so this is a misuse of the code.
Sorry, I misread the #if as #ifdef. The path in question:
225 #define DEBUG 0 287 #if DEBUG 298 #else 300 #define DKEY(x)
i.e. add the "", or any other value to be discarded, to line 300. The issue is the DKEY argument, we can't have the (... ,, ...) construct.
Aaron Richton wrote:
On Tue, 8 Nov 2011, Howard Chu wrote:
#undef DEBUG, and "#define DKEY(x)"
preprocesses to:
( void ) ( "found leaf index %u [%s], rc = %i" , i, , rc);so IMO:
#define DKEY(x) ""
or any other blank-ish concept...
DEBUG cannot (normally) be undef'd, so this is a misuse of the code.
Sorry, I misread the #if as #ifdef. The path in question:
225 #define DEBUG 0 287 #if DEBUG 298 #else 300 #define DKEY(x)i.e. add the "", or any other value to be discarded, to line 300. The issue is the DKEY argument, we can't have the (... ,, ...) construct.
OK, I see. Done.
Aaron Richton wrote:
On Tue, 8 Nov 2011, Howard Chu wrote:
msync is not applicable here. It's only relevant if changes are made using the mmap'd region, and we don't.
Right, good point. A few one-liners that aren't really worth an ITS (yet?):
All of these have been addressed, except <stdint.h> vs <inttypes.h>. Since it builds fine for me as-is on Solaris, I haven't changed that. I guess that's still open to discussion if anyone else has info on which is more commonly provided.
In libraries/libmdb/mdb.c, the compiler is warning about:
5920 rc = mdb_drop0(mc, mc->mc_db->md_flags& MDB_DUPSORT); 5921 if (rc) 5922 mdb_cursor_close(mc); 5923 return rc; 5924 5925 /* Can't delete the main DB */ 5926 if (del&& dbi> MAIN_DBI) {5926 being unreached (because of 5923). Braces on 5921?
#undef DEBUG, and "#define DKEY(x)"
preprocesses to:
( void ) ( "found leaf index %u [%s], rc = %i" , i, , rc);so IMO:
#define DKEY(x) ""
or any other blank-ish concept...
On Solaris 9, stdint.h doesn't exist. inttypes.h does, and should (in theory) be a drop-in replacement. Or, we can try for C90-compliance (by using neither), or.......?
and
if it's C99-style types, servers/slapd/back-mdb uses u_int32_t which should probably be uint32_t. (Berkeley DB gets around this with some definitions in db.h, but I see no reason for mdb to have those hacks...)
Solaris 9 defines BYTE_ORDER via<resolv.h>, I don't see any danger in just #include'ing that on all platforms? (Apparently future standards might include an endian.h for this...there's no standard like no standard...)
PAGESIZE is part of SUSv2<limits.h>; would you be open to a patch to change that to MDB_PAGESIZE?
I thought the alignment was dealt with?
t@1 (l@1) program terminated by signal BUS (invalid address alignment) current thread: t@1 =>[1] mdb_node_search(mc = 0x100620a68, key = 0xffffffff7fffe7b8, exactp = 0xffffffff7fffc56c), line 2957 in "mdb.c" [2] mdb_cursor_set(mc = 0x100620a68, key = 0xffffffff7fffe7b8, data = (nil), op = MDB_SET_RANGE, exactp = 0xffffffff7fffc56c), line 3613 in "mdb.c" [3] mdb_cursor_set(mc = 0x1006208e0, key = 0xffffffff7fffe7c8, data = 0xffffffff7fffe7b8, op = MDB_GET_BOTH, exactp = 0xffffffff7fffc688), line 3653 in "mdb.c" [4] mdb_cursor_get(mc = 0x1006208e0, key = 0xffffffff7fffe7c8, data = 0xffffffff7fffe7b8, op = MDB_GET_BOTH), line 3798 in "mdb.c" [5] mdb_dn2id(op = 0xffffffff7fffebd0, txn = 0x100a3cf10, mc = 0x1006208e0, in = 0xffffffff7fffe960, id = 0xffffffff7fffe928, matched = (nil), nmatched = 0xffffffff7fffe930), line 317 in "dn2id.c" [6] mdb_tool_next_id(op = 0xffffffff7fffebd0, tid = 0x100a3cf10, e = 0x100a25698, text = 0xffffffff7fffeed0, hole = 0), line 396 in "tools.c" [7] mdb_tool_entry_put(be = 0x10061e270, e = 0x100a25698, text = 0xffffffff7fffeed0), line 631 in "tools.c" [8] slapadd(argc = 8, argv = 0xffffffff7ffff268), line 428 in "slapadd.c" [9] main(argc = 8, argv = 0xffffffff7ffff268), line 664 in "main.c"
I'm on 2_4:
commit 3d386f0738e92dcbaf76950835ab4bc9e5e1a0b8 Author: Howard Chu hyc@openldap.org Date: Wed Nov 9 15:34:53 2011 -0800
did it not make it to the branch yet?
Aaron Richton wrote:
I thought the alignment was dealt with?
The full test suite passed for me. What invocation triggered this error?
t@1 (l@1) program terminated by signal BUS (invalid address alignment) current thread: t@1 =>[1] mdb_node_search(mc = 0x100620a68, key = 0xffffffff7fffe7b8, exactp = 0xffffffff7fffc56c), line 2957 in "mdb.c" [2] mdb_cursor_set(mc = 0x100620a68, key = 0xffffffff7fffe7b8, data = (nil), op = MDB_SET_RANGE, exactp = 0xffffffff7fffc56c), line 3613 in "mdb.c" [3] mdb_cursor_set(mc = 0x1006208e0, key = 0xffffffff7fffe7c8, data = 0xffffffff7fffe7b8, op = MDB_GET_BOTH, exactp = 0xffffffff7fffc688), line 3653 in "mdb.c" [4] mdb_cursor_get(mc = 0x1006208e0, key = 0xffffffff7fffe7c8, data = 0xffffffff7fffe7b8, op = MDB_GET_BOTH), line 3798 in "mdb.c" [5] mdb_dn2id(op = 0xffffffff7fffebd0, txn = 0x100a3cf10, mc = 0x1006208e0, in = 0xffffffff7fffe960, id = 0xffffffff7fffe928, matched = (nil), nmatched = 0xffffffff7fffe930), line 317 in "dn2id.c" [6] mdb_tool_next_id(op = 0xffffffff7fffebd0, tid = 0x100a3cf10, e = 0x100a25698, text = 0xffffffff7fffeed0, hole = 0), line 396 in "tools.c" [7] mdb_tool_entry_put(be = 0x10061e270, e = 0x100a25698, text = 0xffffffff7fffeed0), line 631 in "tools.c" [8] slapadd(argc = 8, argv = 0xffffffff7ffff268), line 428 in "slapadd.c" [9] main(argc = 8, argv = 0xffffffff7ffff268), line 664 in "main.c"
I'm on 2_4:
commit 3d386f0738e92dcbaf76950835ab4bc9e5e1a0b8 Author: Howard Chuhyc@openldap.org Date: Wed Nov 9 15:34:53 2011 -0800
did it not make it to the branch yet?
Howard Chu writes:
All of these have been addressed, except <stdint.h> vs <inttypes.h>. Since it builds fine for me as-is on Solaris, I haven't changed that. I guess that's still open to discussion if anyone else has info on which is more commonly provided.
Might as well switch to inttypes and see what happens. Google "inttypes.h missing" shows fewer resulsts than "stdint.h missing". Both are from C99. <inttypes.h> includes <stdint.h> and adds format macros PRIu32 etc, for scanf/printf of these integers without casts.
Hallvard B Furuseth wrote:
Howard Chu writes:
All of these have been addressed, except<stdint.h> vs<inttypes.h>. Since it builds fine for me as-is on Solaris, I haven't changed that. I guess that's still open to discussion if anyone else has info on which is more commonly provided.
Might as well switch to inttypes and see what happens. Google "inttypes.h missing" shows fewer resulsts than "stdint.h missing". Both are from C99.<inttypes.h> includes<stdint.h> and adds format macros PRIu32 etc, for scanf/printf of these integers without casts.
OK, done in master.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 11/08/11 13:45, Aaron Richton wrote:
On Mon, 7 Nov 2011, Howard Chu wrote:
It seems that back-mdb really expects FDSYNC and fdatasync be available on target system. Currently FreeBSD does not have implemented these semantics (yet).
I've made a workaround in the FreeBSD port to make it work; it would be nice to have configure script test if FDSYNC and fdatasync is available and make the corresponding workaround (FSYNC and fsync) if not so building from source would just work.
You're welcome to submit appropriate patches for configure.in and any affected code.
Do you envision issues with using msync() instead, across the board? I would think that, given that back-mdb only makes sense when _POSIX_MAPPED_FILES, msync would present a more portable way of getting to the underlying fsync()/fdatasync() calls. (Heck, on Linux, it's just a couple lines of error checking wrapping a vfs_fsync().)
Xin: is msync() available on FreeBSD? You're mentioning fsync as the proposed patch...
msync() is available on FreeBSD but can not be used here if I understood the code correctly (OpenLDAP don't use mmap for writes in mdb).
The right solution would be to implement fdatasync() in FreeBSD, I'm evaluating the work needs to be done but that's beyond the scope of this mailing list I think. I can live with a workaround for now.
Cheers, - -- Xin LI delphij@delphij.net https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die
-
Aaron Richton -
Hallvard B Furuseth -
Howard Chu -
Michael Ströder -
Quanah Gibson-Mount -
Xin LI