- openldap-bugs - openldap.org

Re: (ITS#7390) slapschema crashes with rwm-rewriteMap slapd
by michael＠stroeder.com 14 Sep '12

14 Sep '12

This is a multi-part message in MIME format. --------------030803020000090707060805 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit See slapd.conf attached. --------------030803020000090707060805 Content-Type: text/plain; charset=UTF-8; name="slapd.conf" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="slapd.conf" IwojIFNlZSBzbGFwZC5jb25mKDUpIGZvciBkZXRhaWxzIG9uIGNvbmZpZ3VyYXRpb24gb3B0 aW9ucy4KIyBUaGlzIGZpbGUgc2hvdWxkIE5PVCBiZSB3b3JsZCByZWFkYWJsZS4KIwoKaW5j bHVkZQkJL2hvbWUvbWljaGFlbC90ZW1wL29wZW5sZGFwLXRlc3RiZWQtaXRzNzM5MC9zY2hl bWEvaW5pdC5zY2hlbWEKCiMgRGVmaW5lIGdsb2JhbCBBQ0xzIHRvIGRpc2FibGUgZGVmYXVs dCByZWFkIGFjY2Vzcy4KCnBpZGZpbGUJCS9ob21lL21pY2hhZWwvdGVtcC9vcGVubGRhcC10 ZXN0YmVkLWl0czczOTAvcnVuL3NsYXBkLTEucGlkCmFyZ3NmaWxlCS9ob21lL21pY2hhZWwv dGVtcC9vcGVubGRhcC10ZXN0YmVkLWl0czczOTAvcnVuL3NsYXBkLTEuYXJncwoKbW9kdWxl cGF0aAkvb3B0L29wZW5sZGFwLVJFMjQvbGliZXhlYy9vcGVubGRhcAoKIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj CiMgTG9hZCBkeW5hbWljIGJhY2tlbmQgbW9kdWxlczoKIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCm1vZHVsZWxv YWQJYmFja19oZGIubGEKCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwojIExvYWQgZHluYW1pYyBvdmVybGF5IG1v ZHVsZXM6CiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIwptb2R1bGVsb2FkCXJ3bS5sYQoKIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCiMg R2xvYmFsIGFjY2VzcyBjb250cm9sCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIwoKIyBSb290IERTRTogYWxsb3cg YW55b25lIHRvIHJlYWQgaXQKYWNjZXNzIHRvIGRuLmJhc2U9IiIgYnkgKiByZWFkCiMgU3Vi c2NoZW1hIChzdWIpZW50cnkgRFNFOiBhbGxvdyBhbnlvbmUgdG8gcmVhZCBpdAphY2Nlc3Mg dG8gZG4uYmFzZT0iY249U3Vic2NoZW1hIiBieSAqIHJlYWQKCgoKIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMKIyBEYXRlbmJhbmsgZnVlciBOYW1lbnNyYXVtIG91PXNjaHVsdW5nLGRjPXN0cm9lZGVy LGRjPWxvY2FsCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCgpkYXRhYmFzZQloZGIKCnN1ZmZpeAkJIm91 PXNjaHVsdW5nLGRjPXN0cm9lZGVyLGRjPWxvY2FsIgpkaXJlY3RvcnkJL2hvbWUvbWljaGFl bC90ZW1wL29wZW5sZGFwLXRlc3RiZWQtaXRzNzM5MC9kYi9idgpyb290ZG4JCSJ1aWQ9ZGly YWRtLG91PXNjaHVsdW5nLGRjPXN0cm9lZGVyLGRjPWxvY2FsIgpyb290cHcJCXRlc3RzZWNy ZXQKbGFzdG1vZAkJb24KCiMgSW5kZXgtS29uZmlndXJhdGlvbgppbmRleCBvYmplY3RDbGFz cwkJZXEKaW5kZXggY24sc24sZGlzcGxheU5hbWUsZ2l2ZW5OYW1lLG1haWwsbyxvdSx0ZWxl cGhvbmVOdW1iZXIsaG9tZVBob25lLG1vYmlsZSBzdWIsZXEKaW5kZXggdWlkLHVpZE51bWJl cixnaWROdW1iZXIsaG9tZURpcmVjdG9yeSxtZW1iZXIsbWFuYWdlcixvd25lcixzZWVBbHNv LGRlcGFydG1lbnROdW1iZXIgZXEKCmNoZWNrcG9pbnQgMjAwIDUKCiMgRW50cmllcyB0byBj YWNoZSBpbiBtZW1vcnkKY2FjaGVzaXplIDEwMDAKIyBTZWFyY2ggcmVzdWx0cyB0byBjYWNo ZSBpbiBtZW1vcnkKaWRsY2FjaGVzaXplIDEwMAoKc2l6ZWxpbWl0CS0xCgojIERCX0NPTkZJ RyBwYXJhbWV0ZXJzCmRiY29uZmlnIHNldF9jYWNoZXNpemUgICAwICAgICAgIDIwMDAwMDAw ICAgICAgICAxCmRiY29uZmlnIHNldF9sZ19yZWdpb25tYXggICAgICAgIDI2MjE0NApkYmNv bmZpZyBzZXRfbGdfYnNpemUgICAgICAgICAgICAyNjIxNDQKZGJjb25maWcgc2V0X2xnX21h eCAgICAgICAgICAgICAgMTA0ODU3NjAKZGJjb25maWcgc2V0X3Rhc19zcGlucyAwCgojIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjCiMgQmVyZWNodGlndW5nZW4gaW0gTmFtZW5zcmF1 bSBvdT1zY2h1bHVuZyxkYz1zdHJvZWRlcixkYz1sb2NhbAojIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjCgphY2Nlc3MgdG8gZG4uc3VidHJlZT0ib3U9c2NodWx1bmcsZGM9c3Ryb2Vk ZXIsZGM9bG9jYWwiCiAgICBieSAqIHdyaXRlCgojIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjCiMgT3ZlcmxheXMgZnVlciBOYW1lbnNyYXVtIG91PXNjaHVsdW5nLGRjPXN0cm9lZGVy LGRjPWxvY2FsCiMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMKCm92ZXJsYXkgcndtCnJ3 bS1yZXdyaXRlRW5naW5lIG9uCgojIHJld3JpdGUgbWFwIGZvciAodWlkPWZvbykgd2l0aCBE TiBvZiBlbnRyeSBhcyByZXN1bHQKcndtLXJld3JpdGVNYXAgc2xhcGQgdWlkMmRuICJsZGFw Oi8vL291PXNjaHVsdW5nLGRjPXN0cm9lZGVyLGRjPWxvY2FsP2VudHJ5RE4/c3ViP3VpZD0i Cgo= --------------030803020000090707060805--

1 0

Re: (ITS#7390) slapschema crashes with rwm-rewriteMap slapd
by michael＠stroeder.com 14 Sep '12

14 Sep '12

This is a multi-part message in MIME format. --------------050607060501060102050805 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit See bt full attached. --------------050607060501060102050805 Content-Type: text/plain; charset=UTF-8; name="bt_full.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="bt_full.txt" #0 0x00007f1699909ab5 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f169990afb6 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f1699944dd3 in __libc_message () from /lib64/libc.so.6 No symbol table info available. #3 0x00007f169994a3b6 in malloc_printerr () from /lib64/libc.so.6 No symbol table info available. #4 0x00007f169ca6fbed in ber_memfree_x (p=0x820778, ctx=0x0) at memory.c:152 __PRETTY_FUNCTION__ = "ber_memfree_x" #5 0x0000000000462a23 in ch_free (ptr=0x820778) at ch_malloc.c:139 ctx = 0x0 #6 0x00000000004ab8fc in slapd_rw_destroy (private=0x820700) at sasl.c:1093 md = 0x820700 __PRETTY_FUNCTION__ = "slapd_rw_destroy" #7 0x00000000004e5532 in rewrite_builtin_map_free (tmp=0x81e110) at map.c:487 map = 0x81e110 __PRETTY_FUNCTION__ = "rewrite_builtin_map_free" #8 0x00000000004f1cb5 in avl_free (root=0x8205f0, dfree=0x4e54c7 <rewrite_builtin_map_free>) at avl.c:515 nleft = 0 nright = 0 #9 0x00000000004e4645 in rewrite_info_delete (pinfo=0x81f690) at info.c:120 info = 0x820420 __PRETTY_FUNCTION__ = "rewrite_info_delete" #10 0x00007f1696d78e3f in rwm_db_destroy (be=0x81eba0, cr=0x0) at rwm.c:2664 rwmap = 0x81f690 on = 0x820240 rc = 0 #11 0x00000000004cee6e in over_db_destroy (be=0x81eba0, cr=0x0) at backover.c:208 oi = 0x820060 on = 0x820240 next = 0x462a23 bi_orig = 0x820060 rc = 0 #12 0x000000000044e833 in backend_stopdown_one (bd=0x81eba0) at backend.c:438 No locals. #13 0x000000000044e928 in backend_destroy_one (bd=0x81eba0, dynamic=1) at backend.c:452 No locals. #14 0x000000000044ea8e in backend_destroy () at backend.c:491 bd = 0x81eba0 bi = 0x7fff07f1e7b0 #15 0x000000000047a55f in slap_destroy () at init.c:247 rc = 0 #16 0x00000000004d6bd8 in slap_tool_destroy () at slapcommon.c:932 rc = 0 #17 0x00000000004db6c4 in slapschema (argc=3, argv=0x7fff07f1ecf8) at slapschema.c:161 id = 18446744073709551615 rc = 0 progname = 0x518180 "slapschema" conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = { __lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = { bv_len = 0, bv_val = 0x4f5fd0 ""}, c_peer_name = {bv_len = 0, bv_val = 0x4f5fd0 ""}, c_listener = 0x4fdf40, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = {bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0, c_clientarg = 0x0, c_send_ldap_result = 0x453d4f <slap_send_ldap_result>, c_send_search_entry = 0x454a57 <slap_send_search_entry>, c_send_search_reference = 0x456a6e <slap_send_search_reference>, c_send_ldap_extended = 0x4545b6 <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x454834 <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7fff07f1e560, o_tag = 0, o_time = 1347638558, o_tincr = 0, o_bd = 0x81eba0, o_req_dn = {bv_len = 0, bv_val = 0x0}, o_req_ndn = { bv_len = 0, bv_val = 0x0}, o_request = {oq_add = {rs_modlist = 0x0, rs_e = 0x0}, oq_bind = {rb_method = 0, rb_cred = {bv_len = 0, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = {bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0x0}, oq_modify = {rs_mods = { rs_modlist = 0x0, rs_no_opattrs = 0 '\000'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0x0, rs_no_opattrs = 0 '\000'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = 0, rs_deref = 0, rs_slimit = 0, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = {rs_msgid = 0}, oq_cancel = {rs_msgid = 0}, oq_extended = {rs_reqoid = {bv_len = 0, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = {rs_extended = {rs_reqoid = {bv_len = 0, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = {bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 0 '\000', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 31 times>, o_controls = 0x7fff07f1e6a8, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = { bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x0, o_ctrls = 0x0, o_csn = {bv_len = 0, bv_val = 0x0}, o_private = 0x0, o_extra = {slh_first = 0x0}, o_next = { stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 18446744073709551615, oh_conn = 0x7fff07f1e7b0, oh_msgid = 0, oh_protocol = 0, oh_tid = 139735100856160, oh_threadctx = 0x7f169ced89c0, oh_tmpmemctx = 0x0, oh_tmpmfuncs = 0x75b3e0, oh_counters = 0x75e8e0, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>}, ob_controls = {0x0 <repeats 32 times>}} op = 0x7fff07f1e3f0 thrctx = 0x7f169ced89c0 requestBSF = 0 doBSF = 0 __PRETTY_FUNCTION__ = "slapschema" #18 0x0000000000415c6e in main (argc=3, argv=0x7fff07f1ecf8) at main.c:410 i = 5 no_detach = 0 rc = 1 urls = 0x0 username = 0x0 groupname = 0x0 sandbox = 0x0 syslogUser = 160 pid = 0 waitfds = {1, 32534} g_argc = 3 g_argv = 0x7fff07f1ecf8 configfile = 0x0 configdir = 0x0 serverName = 0x7fff07f20313 "slapschema" serverMode = 1 scp = 0x0 scp_entry = 0x0 debug_unknowns = 0x0 syslog_unknowns = 0x0 serverNamePrefix = 0x4f5a68 "" l = 0 slapd_pid_file_unlink = 0 slapd_args_file_unlink = 0 firstopt = 1 __PRETTY_FUNCTION__ = "main" --------------050607060501060102050805--

1 0

(ITS#7390) slapschema crashes with rwm-rewriteMap slapd
by michael＠stroeder.com 14 Sep '12

14 Sep '12

Full_Name: Version: 2.4.32 OS: Debian Squeeze URL: Submission from: (NULL) (212.227.35.94) Config with the following lines works as expected but slapschema crashes no matter whether slapd is running or not: -------------------------------------------------------------------------- overlay rwm rwm-rewriteEngine on # rewrite map for (uid=foo) with DN of entry as result rwm-rewriteMap slapd uid2dn "ldap:///ou=um-demail?entryDN?sub?uid=" -------------------------------------------------------------------------- This is self-compiled OpenLDAP 2.4.32 on Debian Squeeze linked against OpenSSL and BDB 4.8.30. Example config will follow. Maybe I even can reproduce on another system and provide gdb output.

1 0

(ITS#7389) [PATCH] MozNSS: load certificates from certdb, fallback to PEM
by jvcelak＠redhat.com 14 Sep '12

14 Sep '12

Full_Name: Jan Vcelak Version: git master OS: Linux URL: ftp://ftp.openldap.org/incoming/jvcelak-20120914-moznss-load-certs-from-cer… Submission from: (NULL) (209.132.186.34) If TLS_CACERT pointed to a PEM file and TLS_CACERTDIR was set to NSS certificate database, the backend assumed that the certificate is always located in the certificate database. This assumption might be wrong. This patch makes the library to try to load the certificate from NSS database and fallback to PEM file if unsuccessfull. The attached file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Red Hat. Red Hat has not assigned rights and/or interest in this work to any party. I, Jan Vcelak am authorized by Red Hat, my employer, to release this work under the following terms. Red Hat hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

(ITS#7388) [PATCH] MozNSS: ignore certdb 'sql:' prefix when checking directory existence
by jvcelak＠redhat.com 14 Sep '12

14 Sep '12

Full_Name: Jan Vcelak Version: git master OS: Linux URL: ftp://ftp.openldap.org/incoming/jvcelak-20120914-moznss-ignore-certdb-sql-p… Submission from: (NULL) (209.132.186.34) Mozilla NSS certificate database in newer SQL format could not be used with OpenLDAP, because the 'sql:' prefix is not removed while checking for the existence of the directory. The attached patch resolves this problem. Steps to reproduce are described in Red Hat bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=857373 The attached file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Red Hat. Red Hat has not assigned rights and/or interest in this work to any party. I, Jan Vcelak am authorized by Red Hat, my employer, to release this work under the following terms. Red Hat hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

(ITS#7387) submit a contribution
by philippe.eychart＠mail.pf 13 Sep '12

13 Sep '12

Full_Name: Philippe EYCHART Version: 2 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (123.50.70.54) My work is fully described and available here: http://homepage.gna.org/ldapcppei/miniHowto.en.html Regards. PE

1 0

Re: (ITS#7385) back_mdb ENOSPC error returned from mdb_node_add incorrectly
by marco＠schirrmeister.net 13 Sep '12

13 Sep '12

--Apple-Mail=_D316E30E-F1E5-480E-A47A-F4610B0C33C6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On Sep 12, 2012, at 7:49 PM, hyc(a)symas.com wrote: > Frank.Swasey(a)uvm.edu wrote: >> Full_Name: Francis Swasey >> Version: 2.4.32 >> OS: RHEL6.3 64-bit >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (132.198.21.181) >>=20 >>=20 >> In exploring converting to back_mdb, I have discovered a problem = which results >> in the following symptoms: >=20 > Thanks for the report, fixed now in git master. I saw it is also merged into the RE24 branch. I tried today the latest commit from the RE24 branch and I did not see = the error anymore in the slapadd tool. Where I noticed the No space left = on device error before. But I'm seeing it now when I run the slapindex. Unfortunately I can't reproduce it with a test dataset that I created. Probably because I have not enough attributes that I could add to index = like on the real server. If I run slapindex a second time, again error. If I run slapindex a third time, no error. The data.mdb grew a little bit during the slapindex runs. =46rom about 710MB to 1,2GB. If I remove a few index options, slapindex runs through without a = problem. The output on the test server looks like below. [root@ds71 ~]# slapadd2.4 -b dc=3Dogilvy,dc=3Dcom -l all_slapcat.ldif=20 _#################### 100.00% eta none elapsed 05m26s spd = 596.0 k/s=20 Closing DB... [root@ds71 ~]#=20 [root@ds71 ~]# slapindex2.4=20 5051fea8 =3D> mdb_tool_entry_reindex: txn_commit failed: No space left = on device (28) [root@ds71 ~]#=20 [root@ds71 ~]# slapindex2.4=20 50520b10 =3D> mdb_tool_entry_reindex: txn_commit failed: No space left = on device (28) [root@ds71 ~]#=20 [root@ds71 ~]# slapindex2.4=20 [root@ds71 ~]#=20 Since you wrote ENOSPC can only be happen in 2 places, I thought it's = still related somehow. I'm a little bit clueless right now what else I can provide. If there is something that I can provide, then let me know please and I = try to provide it. -- Marco --Apple-Mail=_D316E30E-F1E5-480E-A47A-F4610B0C33C6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; = "><br><div><div>On Sep 12, 2012, at 7:49 PM, <a = href=3D"mailto:hyc@symas.com">hyc(a)symas.com</a> wrote:</div><br = class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><span = class=3D"Apple-style-span" style=3D"border-collapse: separate; = font-family: Helvetica; font-style: normal; font-variant: normal; = font-weight: normal; letter-spacing: normal; line-height: normal; = orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: = none; white-space: normal; widows: 2; word-spacing: 0px; = -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: = 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: = auto; -webkit-text-stroke-width: 0px; font-size: medium; "><a = href=3D"mailto:Frank.Swasey@uvm.edu">Frank.Swasey(a)uvm.edu</a><span = class=3D"Apple-converted-space"> </span>wrote:<br><blockquote = type=3D"cite">Full_Name: Francis Swasey<br></blockquote><blockquote = type=3D"cite">Version: 2.4.32<br></blockquote><blockquote = type=3D"cite">OS: RHEL6.3 64-bit<br></blockquote><blockquote = type=3D"cite">URL:<span class=3D"Apple-converted-space"> </span><a = href=3D"ftp://ftp.openldap.org/incoming/">ftp://ftp.openldap.org/incoming/= </a><br></blockquote><blockquote type=3D"cite">Submission from: (NULL) = (132.198.21.181)<br></blockquote><blockquote = type=3D"cite"><br></blockquote><blockquote = type=3D"cite"><br></blockquote><blockquote type=3D"cite">In exploring = converting to back_mdb, I have discovered a problem which = results<br></blockquote><blockquote type=3D"cite">in the following = symptoms:<br></blockquote><br>Thanks for the report, fixed now in git = master.</span></blockquote></div><br><div>I saw it is also merged into = the RE24 branch.</div><div><br></div><div>I tried today the latest = commit from the RE24 branch and I did not see the error anymore in the = slapadd tool. Where I noticed the No space left on device = error before.</div><div>But I'm seeing it now when I run the = slapindex.</div><div><br></div><div>Unfortunately I can't reproduce it = with a test dataset that I created.</div><div>Probably because I have = not enough attributes that I could add to index like on the real = server.</div><div><br></div><div>If I run slapindex a second time, again = error.</div><div>If I run slapindex a third time, no = error.</div><div><br></div><div>The data.mdb grew a little bit during = the slapindex runs.</div><div>=46rom about 710MB to = 1,2GB.</div><div><br></div><div>If I remove a few index options, = slapindex runs through without a problem.</div><div><br></div><div>The = output on the test server looks like below.</div><div><div>[root@ds71 = ~]# slapadd2.4 -b dc=3Dogilvy,dc=3Dcom -l = all_slapcat.ldif </div><div>_#################### 100.00% eta =   none elapsed          05m26s spd 596.0 = k/s </div><div>Closing DB...</div><div>[root@ds71 = ~]# </div><div>[root@ds71 ~]# slapindex2.4 </div><div>5051fea8 = =3D> mdb_tool_entry_reindex: txn_commit failed: No space left on = device (28)</div><div>[root@ds71 ~]# </div><div>[root@ds71 ~]# = slapindex2.4 </div><div>50520b10 =3D> mdb_tool_entry_reindex: = txn_commit failed: No space left on device (28)</div><div>[root@ds71 = ~]# </div><div>[root@ds71 ~]# = slapindex2.4 </div><div>[root@ds71 = ~]# </div></div><div><br></div><div><br></div><div>Since you wrote = ENOSPC can only be happen in 2 places, I thought it's still related = somehow.</div><div>I'm a little bit clueless right now what else I can = provide.</div><div><br></div><div>If there is something that I can = provide, then let me know please and I try to provide = it.</div><div><br></div><div><br></div><div>--</div><div>Marco</div><div><= br></div><div><br></div></body></html>= --Apple-Mail=_D316E30E-F1E5-480E-A47A-F4610B0C33C6--

1 0

(ITS#7386) back-mdb slapindex grows DB without bound
by hyc＠OpenLDAP.org 12 Sep '12

12 Sep '12

Full_Name: Howard Chu Version: 2.4.32 OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (37.19.96.255) Submitted by: hyc mdb_tool_entry_next() keeps a read-only cursor open, which prevents free pages from being reclaimed. Fix coming shortly.

1 0

Re: (ITS#7385) back_mdb ENOSPC error returned from mdb_node_add incorrectly
by hyc＠symas.com 12 Sep '12

12 Sep '12

Frank.Swasey(a)uvm.edu wrote: > Full_Name: Francis Swasey > Version: 2.4.32 > OS: RHEL6.3 64-bit > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (132.198.21.181) > > > In exploring converting to back_mdb, I have discovered a problem which results > in the following symptoms: Thanks for the report, fixed now in git master. > > ldapmodify with the following ldif: > > > dn: cn=COURSE-201209-94186,ou=Groups,dc=uvm,dc=edu > changetype: modify > delete: member > member: uid=pbirnbau,ou=people,dc=uvm,dc=edu > - > > fails with the following: > > ldap_modify: Other (e.g., implementation specific) error (80) > additional info: entry update failed > > syslog records the following: > > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 fd=21 ACCEPT from > IP=132.198.100.216:57847 (IP=0.0.0.0:636) > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 fd=21 TLS established > tls_ssf=256 ssf=256 > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=0 BIND > dn="cn=manager,dc=uvm,dc=edu" method=128 > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=0 BIND > dn="cn=manager,dc=uvm,dc=edu" mech=SIMPLE ssf=0 > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=0 RESULT tag=97 err=0 text= > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=1 MOD > dn="cn=COURSE-201209-94186,ou=Groups,dc=uvm,dc=edu" > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=1 MOD attr=member > Sep 11 14:14:23 ldap6dev slapd[6456]: slap_queue_csn: queing 0x7fdb6bffd0a0 > 20120911181423.401038Z#000000#000#000000 > Sep 11 14:14:23 ldap6dev slapd[6456]: mdb_id2entry_put: mdb_put failed: No space > left on device(28) "cn=course-201209-94186,ou=groups,dc=uvm,dc=edu" > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=1 RESULT tag=103 err=80 > text=entry update failed > Sep 11 14:14:23 ldap6dev slapd[6456]: slap_graduate_commit_csn: removing > 0x7fdb54002970 20120911181423.401038Z#000000#000#000000 > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=2 UNBIND > Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 fd=21 closed > > gdb points to the return ENOSPC at line 4806 of mdb_node_add > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7383) slapd startup issue with mdb
by michael＠stroeder.com 11 Sep '12

11 Sep '12

This is a cryptographically signed message in MIME format. --------------ms030701090404000009030901 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable marco(a)schirrmeister.net wrote: > Normally I also only added an index where I saw the candidate message i= n the log. Even when there's a warning about non-indexed searches in syslog I won't = index all attributes mentioned in the message without further analyzing the fil= ter. IMHO there's no point to index additional attributes if one of the indexe= d attributes already narrows the set of search candidates to one. One examp= le are filters with (uid=3Dname) with eq-index on uid. Ciao, Michael. --------------ms030701090404000009030901 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIILHzCC BT8wggQnoAMCAQICDwCmSwABAAIAivjZQ8SBvzANBgkqhkiG9w0BAQUFADB8MQswCQYDVQQG EwJERTEcMBoGA1UEChMTVEMgVHJ1c3RDZW50ZXIgR21iSDElMCMGA1UECxMcVEMgVHJ1c3RD ZW50ZXIgQ2xhc3MgMSBMMSBDQTEoMCYGA1UEAxMfVEMgVHJ1c3RDZW50ZXIgQ2xhc3MgMSBM MSBDQSBJWDAeFw0xMjA2MDYxOTAyMTZaFw0xMzA2MDcxOTAyMTZaMCgxCzAJBgNVBAYTAkRF MRkwFwYDVQQDDBBNaWNoYWVsIFN0csO2ZGVyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB CgKCAQEAxXZGav40rnGNLxEggBW94MILWHlfC8a23Jew5U1gPlfRTXOjjzmoaZ1uCyGdgF6M VvuO9T1aTQNGH+OdeGe3P7Tfc/NsLJFJ2wtd8blvhmodUgse2eypiWjNOd4gZuhalBhgsQ0K b5D6/1foghII4E264iZlJ7AJ+UYcO+GxvFWT0YMTbLckgDkZk7c3qwTozdhYvXarvqx+8Ou/ kuxpQQhac/ebzxpu0N+RHSf2KIUS0g0tEGnPtGv6iL+9QNHc4JKo9Y9KKVw3tQy+Re+FQLxB 1fPE5F+qxuD3AUENpOwkMsqWLM94ohtx3CFqLpxfUPrnKFLAHOhHEbByYGvFPwIDAQABo4IC EDCCAgwwgaUGCCsGAQUFBwEBBIGYMIGVMFEGCCsGAQUFBzAChkVodHRwOi8vd3d3LnRydXN0 Y2VudGVyLmRlL2NlcnRzZXJ2aWNlcy9jYWNlcnRzL3RjX2NsYXNzMV9MMV9DQV9JWC5jcnQw QAYIKwYBBQUHMAGGNGh0dHA6Ly9vY3NwLml4LnRjY2xhc3MxLnRjdW5pdmVyc2FsLWkudHJ1 c3RjZW50ZXIuZGUwHwYDVR0jBBgwFoAU6bgoHUbP/M34TpvF7ktg69g7P9EwDAYDVR0TAQH/ BAIwADBKBgNVHSAEQzBBMD8GCSqCFAAsAQEBATAyMDAGCCsGAQUFBwIBFiRodHRwOi8vd3d3 LnRydXN0Y2VudGVyLmRlL2d1aWRlbGluZXMwDgYDVR0PAQH/BAQDAgTwMB0GA1UdDgQWBBS2 KAWfTfgJ/JQ63qLGwTXYLnI+LzBiBgNVHR8EWzBZMFegVaBThlFodHRwOi8vY3JsLml4LnRj Y2xhc3MxLnRjdW5pdmVyc2FsLWkudHJ1c3RjZW50ZXIuZGUvY3JsL3YyL3RjX0NsYXNzMV9M MV9DQV9JWC5jcmwwMwYDVR0lBCwwKgYIKwYBBQUHAwIGCCsGAQUFBwMEBggrBgEFBQcDBwYK KwYBBAGCNxQCAjAfBgNVHREEGDAWgRRtaWNoYWVsQHN0cm9lZGVyLmNvbTANBgkqhkiG9w0B AQUFAAOCAQEAQ3bvVUpEq+cQrLpcogyt5BJNk/WvUvOHqhzyj28M9pg9hcDl1+MYl5qqj6tR GSTLPQZyf287pcmbMwbcTGZO/gbW9v7RYcut6RauWdwKMCUmKC3J4fVfDq9ZETA2WOV68ef4 B3Gzdhghsbp3Rhp5dDmrCVKAHlafm6ZwJrEQ9P76fxnQZzRLgeKpZep5ePH5YHUB3+YaOQvJ FG0bOXvfHhRiRG7/HW2G+yDgjHSxDz8AFzMWL/RFePqZ4pn6T/SM/qU6WEpW39MWyJNoH/Kx QDYK8gGYuesn1ciMCTnjrvZQj0fonGTO4SfWekJRkuGrJ7dYSZRjYbDcWBBkdFLWzzCCBdgw ggTAoAMCAQICDgboAAEAAkqWLSQM/sXJMA0GCSqGSIb3DQEBBQUAMHkxCzAJBgNVBAYTAkRF MRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSQwIgYDVQQLExtUQyBUcnVzdENlbnRl ciBVbml2ZXJzYWwgQ0ExJjAkBgNVBAMTHVRDIFRydXN0Q2VudGVyIFVuaXZlcnNhbCBDQSBJ MB4XDTA5MTEwMzE0MDgxOVoXDTI1MTIzMTIxNTk1OVowfDELMAkGA1UEBhMCREUxHDAaBgNV BAoTE1RDIFRydXN0Q2VudGVyIEdtYkgxJTAjBgNVBAsTHFRDIFRydXN0Q2VudGVyIENsYXNz IDEgTDEgQ0ExKDAmBgNVBAMTH1RDIFRydXN0Q2VudGVyIENsYXNzIDEgTDEgQ0EgSVgwggEi MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC75pBuz2Lp6QuqthDVR+V8XSsncZpozVVt 5KLv5P7yemMRwleKyH3PjmYfZUVL64Biab1GjovFblqVGCrep/EfdRonq20yU+P7TVhiLP8Z 5cegDZotIYhZhM0d8cPIij6w5d4IJM/8QCy6QSOUu4ASiTVItoYE4AFPjLqpmPwcie0fiqHH hpgmHnJla/7PZdkMZEsaCfVDEWBmJuMzVprJPT40anjG5VBLyM2I5DlsUCaeQCy2O3w3sqf1 3dyzUcv03IICuNc63towXA31Qt0TaVNU6YAmQjMepdfMbspmCZ+G8D2+xophEPPR/1vkstst smUMqX0XrLonTUJczglPAgMBAAGjggJZMIICVTCBmgYIKwYBBQUHAQEEgY0wgYowUgYIKwYB BQUHMAKGRmh0dHA6Ly93d3cudHJ1c3RjZW50ZXIuZGUvY2VydHNlcnZpY2VzL2NhY2VydHMv dGNfdW5pdmVyc2FsX3Jvb3RfSS5jcnQwNAYIKwYBBQUHMAGGKGh0dHA6Ly9vY3NwLnRjdW5p dmVyc2FsLUkudHJ1c3RjZW50ZXIuZGUwHwYDVR0jBBgwFoAUkqR1LKSevoFE63n8isWVpesQ dXMwEgYDVR0TAQH/BAgwBgEB/wIBADBSBgNVHSAESzBJMAYGBFUdIAAwPwYJKoIUACwBAQEB MDIwMAYIKwYBBQUHAgEWJGh0dHA6Ly93d3cudHJ1c3RjZW50ZXIuZGUvZ3VpZGVsaW5lczAO BgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYEFOm4KB1Gz/zN+E6bxe5LYOvYOz/RMIH9BgNVHR8E gfUwgfIwge+ggeyggemGRmh0dHA6Ly9jcmwudGN1bml2ZXJzYWwtSS50cnVzdGNlbnRlci5k ZS9jcmwvdjIvdGNfdW5pdmVyc2FsX3Jvb3RfSS5jcmyGgZ5sZGFwOi8vd3d3LnRydXN0Y2Vu dGVyLmRlL0NOPVRDJTIwVHJ1c3RDZW50ZXIlMjBVbml2ZXJzYWwlMjBDQSUyMEksTz1UQyUy MFRydXN0Q2VudGVyJTIwR21iSCxPVT1yb290Y2VydHMsREM9dHJ1c3RjZW50ZXIsREM9ZGU/ Y2VydGlmaWNhdGVSZXZvY2F0aW9uTGlzdD9iYXNlPzANBgkqhkiG9w0BAQUFAAOCAQEAOcjE m+6+mO5Icm+N53G2DpCM07LBFSGoRpBoX0oE8TrJaIQh2KXmBHVdn9LU8kt3QzLclctgvwJV 0KwcsMUUl5tlCsMPpR3s2Ek5lbWpvvr0HqtW56blAQiINV9nBd1EJFASIkRjefGbV2nOq9Yz UU+N8HA7jq1ROhd/NZZraGhjthwKyfjfHV7PKxGlY+3M0MbTIG+q/GhIfm0euDpFqhKG88e9 ALXr/uoSn3MzeOcoOWjTpW3adtFO4VWVgKbgG7jNrFbvRVlHmFLbOm4msjE5aXWxLiTwpJ2X iF4zKca1vAdAOgw9us90jEtOeiH6GzjNxEMvb7TfeO6Zkuc6HDGCA84wggPKAgEBMIGPMHwx CzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYDVQQLExxU QyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENlbnRlciBD bGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wCQYFKw4DAhoFAKCCAhMwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTIwOTExMjAxOTU1WjAjBgkq hkiG9w0BCQQxFgQU+0E5Hxed6F+8Fdmsr2y1whu833MwbAYJKoZIhvcNAQkPMV8wXTALBglg hkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggq hkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBoAYJKwYBBAGCNxAEMYGSMIGP MHwxCzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYDVQQL ExxUQyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENlbnRl ciBDbGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wgaIGCyqGSIb3DQEJEAILMYGS oIGPMHwxCzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYD VQQLExxUQyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENl bnRlciBDbGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wDQYJKoZIhvcNAQEBBQAE ggEAPPc18RzIfVnUVnXD2xBJwbbny8UE6R9+LKbdLJlPCmiKuq7iUlKvLLTJdI15gXDrWokI NH9COp2ZbUrSjmJGYufpH4lx+FXGG75YVgnxyaFyeL3WA9v/G6reiu2UT3971v6eFEMqDEjB DaP3aJ1sme86uYuRQSTyfzC2EPjww1gS5fXgOhKlEtbbdg5ltgZP6osdKX+qr0ueOBmiEs1g LyGktHeq8LAFxnWvNh50tLuLutr/vnIdnsLlw888GQSXtUQl3ugsI0q+AUv6bSmrGcImbvXK ZqeTa6JMqBxPQk5u56dCBqujjvDSCXKG1uan7packG1NeyGVZ/anPyAxhQAAAAAAAA== --------------ms030701090404000009030901--

1 0

Re: (ITS#7383) slapd startup issue with mdb
by marco＠schirrmeister.net 11 Sep '12

11 Sep '12

On Sep 11, 2012, at 8:14 PM, hyc(a)symas.com wrote: > marco(a)schirrmeister.net wrote: >> Full_Name: Marco Schirrmeister >> Version: 2.4.32 >> OS: CentOS6 x86_64 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (84.59.6.160) >> >> >> I'm testing mdb since the last versions with our dataset. >> If I use my production config slapd does't start up. >> >> The following is in the log file. >> >> Sep 11 16:48:37 ds71 slapd2.4[8349]: @(#) $OpenLDAP: slapd 2.4.32 (Sep 6 2012 >> 13:50:54) $#012#011mschirrmeister@defrsl6bh2.ogilvy.com:/home/mschirrmeister/rpmbuild/BUILD/openldap-2.4.32/servers/slapd >> Sep 11 16:48:37 ds71 slapd2.4[8349]: mdb_attr_dbs: database "dc=ogilvy,dc=com": >> mdb_open(omgPublishDate) failed: Too many open files in system (23). >> Sep 11 16:48:37 ds71 slapd2.4[8349]: backend_startup_one (type=mdb, >> suffix="dc=ogilvy,dc=com"): bi_db_open failed! (23) >> Sep 11 16:48:37 ds71 slapd2.4[8349]: slapd stopped. >> >> My /etc/security/limits.conf file has already the following entries. >> * soft nofile 65535 >> * hard nofile 65535 >> >> No matter what I try on the OS side, the error above comes up. >> >> I have about 130 index attributes in my config. Looks to me that this is the >> problem. >> If I comment 8 attributes, slapd starts fine. > > There is a hardcoded limit of 128. This same limit exists in back-bdb/hdb. You > can just edit back-mdb.h (or back-bdb.h) to raise the limit. Change the > MDB_INDICES (or BDB_INDICES) definition. > > Congratulations, no one else has hit this limit in back-bdb in the past 12 > years. Seems likely to me that you're indexing too much, but there's not > enough information here to judge. Ok, thanks for that information. I will try to increase it. But it's weird that there is no error with bdb on startup. I looked at my back-bdb.h and it has 128. I'm definitely over this number. You are right, some are too much and I could maybe remove a few indexes. Some defaults. But our own schema extension is quite long and many of those attributes are used in searches from the various apps. Normally I also only added an index where I saw the candidate message in the log. So this is then also no bug and we can close it. Sorry for being to fast with an ITS. -- Marco

1 0

(ITS#7385) back_mdb ENOSPC error returned from mdb_node_add incorrectly
by Frank.Swasey＠uvm.edu 11 Sep '12

11 Sep '12

Full_Name: Francis Swasey Version: 2.4.32 OS: RHEL6.3 64-bit URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (132.198.21.181) In exploring converting to back_mdb, I have discovered a problem which results in the following symptoms: ldapmodify with the following ldif: dn: cn=COURSE-201209-94186,ou=Groups,dc=uvm,dc=edu changetype: modify delete: member member: uid=pbirnbau,ou=people,dc=uvm,dc=edu - fails with the following: ldap_modify: Other (e.g., implementation specific) error (80) additional info: entry update failed syslog records the following: Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 fd=21 ACCEPT from IP=132.198.100.216:57847 (IP=0.0.0.0:636) Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 fd=21 TLS established tls_ssf=256 ssf=256 Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=0 BIND dn="cn=manager,dc=uvm,dc=edu" method=128 Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=0 BIND dn="cn=manager,dc=uvm,dc=edu" mech=SIMPLE ssf=0 Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=0 RESULT tag=97 err=0 text= Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=1 MOD dn="cn=COURSE-201209-94186,ou=Groups,dc=uvm,dc=edu" Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=1 MOD attr=member Sep 11 14:14:23 ldap6dev slapd[6456]: slap_queue_csn: queing 0x7fdb6bffd0a0 20120911181423.401038Z#000000#000#000000 Sep 11 14:14:23 ldap6dev slapd[6456]: mdb_id2entry_put: mdb_put failed: No space left on device(28) "cn=course-201209-94186,ou=groups,dc=uvm,dc=edu" Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=1 RESULT tag=103 err=80 text=entry update failed Sep 11 14:14:23 ldap6dev slapd[6456]: slap_graduate_commit_csn: removing 0x7fdb54002970 20120911181423.401038Z#000000#000#000000 Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 op=2 UNBIND Sep 11 14:14:23 ldap6dev slapd[6456]: conn=1002 fd=21 closed gdb points to the return ENOSPC at line 4806 of mdb_node_add

1 0

Re: (ITS#7383) slapd startup issue with mdb
by hyc＠symas.com 11 Sep '12

11 Sep '12

marco(a)schirrmeister.net wrote: > Full_Name: Marco Schirrmeister > Version: 2.4.32 > OS: CentOS6 x86_64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (84.59.6.160) > > > I'm testing mdb since the last versions with our dataset. > If I use my production config slapd does't start up. > > The following is in the log file. > > Sep 11 16:48:37 ds71 slapd2.4[8349]: @(#) $OpenLDAP: slapd 2.4.32 (Sep 6 2012 > 13:50:54) $#012#011mschirrmeister@defrsl6bh2.ogilvy.com:/home/mschirrmeister/rpmbuild/BUILD/openldap-2.4.32/servers/slapd > Sep 11 16:48:37 ds71 slapd2.4[8349]: mdb_attr_dbs: database "dc=ogilvy,dc=com": > mdb_open(omgPublishDate) failed: Too many open files in system (23). > Sep 11 16:48:37 ds71 slapd2.4[8349]: backend_startup_one (type=mdb, > suffix="dc=ogilvy,dc=com"): bi_db_open failed! (23) > Sep 11 16:48:37 ds71 slapd2.4[8349]: slapd stopped. > > My /etc/security/limits.conf file has already the following entries. > * soft nofile 65535 > * hard nofile 65535 > > No matter what I try on the OS side, the error above comes up. > > I have about 130 index attributes in my config. Looks to me that this is the > problem. > If I comment 8 attributes, slapd starts fine. There is a hardcoded limit of 128. This same limit exists in back-bdb/hdb. You can just edit back-mdb.h (or back-bdb.h) to raise the limit. Change the MDB_INDICES (or BDB_INDICES) definition. Congratulations, no one else has hit this limit in back-bdb in the past 12 years. Seems likely to me that you're indexing too much, but there's not enough information here to judge. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7384) Assert Crash in ppolicy_ctrls_cleanup
by ghola＠rebelbase.com 11 Sep '12

11 Sep '12

Full_Name: Duncan Idaho Version: 2.4.32 OS: Centos 5.5 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (216.148.0.72) We are seeing this a few times a week in our environment. #0 0x0000003874c30265 in raise () from /lib64/libc.so.6 #1 0x0000003874c31d10 in abort () from /lib64/libc.so.6 #2 0x0000003874c296e6 in __assert_fail () from /lib64/libc.so.6 #3 0x000000000055c369 in ctrls_cleanup (op=0x11144d40, rs=0x42e3cc10, oldctrls=0x11251620) at ppolicy.c:870 #4 0x000000000055c4aa in ppolicy_ctrls_cleanup (op=0x11144d40, rs=0x42e3cc10) at ppolicy.c:895 #5 0x0000000000442a46 in slap_cleanup_play (op=0x11144d40, rs=0x42e3cc10) at result.c:541 #6 0x0000000000443229 in send_ldap_response (op=0x11144d40, rs=0x42e3cc10) at result.c:733 #7 0x0000000000443a5d in slap_send_ldap_result (op=0x11144d40, rs=0x42e3cc10) at result.c:860 #8 0x000000000052d7d7 in hdb_bind (op=0x11144d40, rs=0x42e3cc10) at bind.c:158 #9 0x00000000004bc67a in overlay_op_walk (op=0x11144d40, rs=0x42e3cc10, which=op_bind, oi=0x10af8670, on=0x0) at backover.c:671 #10 0x00000000004bc87d in over_op_func (op=0x11144d40, rs=0x42e3cc10, which=op_bind) at backover.c:723 #11 0x00000000004bc903 in over_op_bind (op=0x11144d40, rs=0x42e3cc10) at backover.c:738 #12 0x0000000000514136 in relay_back_op (op=0x11144d40, rs=0x42e3cc10, which=0) at op.c:210 #13 0x00000000005142b9 in relay_back_op_bind (op=0x11144d40, rs=0x42e3cc10) at op.c:243 #14 0x00000000004bc67a in overlay_op_walk (op=0x11144d40, rs=0x42e3cc10, which=op_bind, oi=0x10af97f0, on=0x0) at backover.c:671 #15 0x00000000004bc87d in over_op_func (op=0x11144d40, rs=0x42e3cc10, which=op_bind) at backover.c:723 #16 0x00000000004bc903 in over_op_bind (op=0x11144d40, rs=0x42e3cc10) at backover.c:738 #17 0x000000000045520d in fe_op_bind (op=0x11144d40, rs=0x42e3cc10) at bind.c:383 #18 0x000000000045493c in do_bind (op=0x11144d40, rs=0x42e3cc10) at bind.c:205 #19 0x000000000042cbd4 in connection_operation (ctx=0x42e3cd60, arg_v=0x11144d40) at connection.c:1150 #20 0x000000000042d159 in connection_read_thread (ctx=0x42e3cd60, argv=0xc7) at connection.c:1286 #21 0x000000000058c6c1 in ldap_int_thread_pool_wrapper (xpool=0x109dfe80) at tpool.c:688 #22 0x000000387540673d in start_thread () from /lib64/libpthread.so.0 #23 0x0000003874cd3d1d in clone () from /lib64/libc.so.6 #24 0x0000000000000000 in ?? ()

1 0

(ITS#7383) slapd startup issue with mdb
by marco＠schirrmeister.net 11 Sep '12

11 Sep '12

Full_Name: Marco Schirrmeister Version: 2.4.32 OS: CentOS6 x86_64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.59.6.160) I'm testing mdb since the last versions with our dataset. If I use my production config slapd does't start up. The following is in the log file. Sep 11 16:48:37 ds71 slapd2.4[8349]: @(#) $OpenLDAP: slapd 2.4.32 (Sep 6 2012 13:50:54) $#012#011mschirrmeister@defrsl6bh2.ogilvy.com:/home/mschirrmeister/rpmbuild/BUILD/openldap-2.4.32/servers/slapd Sep 11 16:48:37 ds71 slapd2.4[8349]: mdb_attr_dbs: database "dc=ogilvy,dc=com": mdb_open(omgPublishDate) failed: Too many open files in system (23). Sep 11 16:48:37 ds71 slapd2.4[8349]: backend_startup_one (type=mdb, suffix="dc=ogilvy,dc=com"): bi_db_open failed! (23) Sep 11 16:48:37 ds71 slapd2.4[8349]: slapd stopped. My /etc/security/limits.conf file has already the following entries. * soft nofile 65535 * hard nofile 65535 No matter what I try on the OS side, the error above comes up. I have about 130 index attributes in my config. Looks to me that this is the problem. If I comment 8 attributes, slapd starts fine. I did not find anything about index limits. Is this a bug in mdb? I'm not able to create stacktrace. When I run slapd via gdb, output is this. Program exited with code 01. (gdb) bt full No stack. (gdb) If you can let me know how to do it, I can provide a trace. My config looks like below. I omitted all our internal attributes. --------- include /usr/share/openldap2.4/schema/core.schema include /usr/share/openldap2.4/schema/cosine.schema include /usr/share/openldap2.4/schema/corba.schema include /usr/share/openldap2.4/schema/inetorgperson.schema include /usr/share/openldap2.4/schema/java.schema include /usr/share/openldap2.4/schema/krb5-kdc.schema include /usr/share/openldap2.4/schema/kerberosobject.schema include /usr/share/openldap2.4/schema/misc.schema include /usr/share/openldap2.4/schema/nis.schema include /usr/share/openldap2.4/schema/openldap.schema include /usr/share/openldap2.4/schema/autofs.schema include /usr/share/openldap2.4/schema/samba.schema include /usr/share/openldap2.4/schema/kolab.schema include /usr/share/openldap2.4/schema/calendar.schema include /usr/share/openldap2.4/schema/sudo.schema include /usr/share/openldap2.4/schema/dhcp.schema include /usr/share/openldap2.4/schema/ldapns.schema include /etc/openldap2.4/slapd.access.ogilvy.conf pidfile /var/run/ldap2.4/slapd.pid argsfile /var/run/ldap2.4/slapd.args modulepath /usr/lib64/oldap24/openldap2.4 moduleload back_monitor.la moduleload accesslog.la moduleload syncprov.la moduleload auditlog.la TLSCertificateFile /etc/pki/tls/certs/ogilvy.com.crt TLSCertificateKeyFile /etc/pki/tls/private/ogilvy.com_unsec.rsa TLSCACertificateFile /etc/pki/tls/certs/ca-bundle.crt loglevel stats sasl-secprops noplain,noanonymous,noactive serverID 40 ldap://ds71.ogilvy.com database mdb suffix "dc=ogilvy,dc=com" rootdn "cn=manager,dc=ogilvy,dc=com" rootpw xxx directory /var/lib/ldap2.4/ogilvy.com limits dn.exact="cn=manager,dc=ogilvy,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited limits dn.exact="uid=replicator,ou=admin,dc=ogilvy,dc=com" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited limits group/ogilvyGroup/uniqueMember="cn=svcgrp-unlimitedsearch,ou=groups,dc=ogilvy,dc=com" size.soft=unlimited size.hard=unlimited sizelimit 90000 checkpoint 256 5 conn_max_pending_auth 2000 dbnosync maxsize 10485760000 monitoring on overlay syncprov syncprov-checkpoint 256 5 syncprov-sessionlog 5000 database config rootdn "cn=admin,cn=config" rootpw xxx include /etc/openldap2.4/slapd.access.config.conf database monitor rootdn cn=monitor rootpw xxx ------- -- Marco

1 0

Re: (ITS#7382) slapadd segfaults (during freeing resources)
by hyc＠symas.com 09 Sep '12

09 Sep '12

daniel(a)pluta.biz wrote: > Full_Name: Daniel Pluta > Version: OPENLDAP_REL_ENG > OS: Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (2001:470:9feb:ff02:2958:c83c:2589:166d) > > > Hi, Looks like you're mixing different libldaps in the same process. libldap init.c:484 is not what your backtrace shows. At least, not in the RE24 branch. Closing this ITS, your environment is invalid. > root@srv:/home/tester/src/openldap# gdb > /home/tester/src/openldap/servers/slapd/slapadd > GNU gdb (GDB) 7.4.1-debian > Copyright (C) 2012 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "x86_64-linux-gnu". > For bug reporting instructions, please see: > <http://www.gnu.org/software/gdb/bugs/>... > Reading symbols from /home/tester/src/openldap/servers/slapd/slapadd...done. > (gdb) run -v -F /etc/ldap/slapd_example.d_stat -b "dc=example,dc=com" -l > /tmp/example_data.ldif > Starting program: /home/tester/src/openldap/servers/slapd/slapadd -v -F > /etc/ldap/slapd_example.d_stat -b "dc=example,dc=com" -l /tmp/example_data.ldif > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > 504cc479 mdb_id2entry_put: mdb_put failed: MDB_KEYEXIST: Key/data pair already > exists(-30799) "dc=example,dc=com" > 504cc479 => mdb_tool_entry_put: id2entry_add failed: err=-30799 > 504cc479 => mdb_tool_entry_put: txn_aborted! MDB_KEYEXIST: Key/data pair already > exists (-30799) > slapadd: could not add entry dn="dc=example,dc=com" (line=1): txn_aborted! > MDB_KEYEXIST: Key/data pair already exists (-30799) > _### 16.13% eta none elapsed none spd 501.5 k/s > Closing DB... > > Program received signal SIGSEGV, Segmentation fault. > 0x00007ffff6a34859 in free () from /lib/x86_64-linux-gnu/libc.so.6 > (gdb) bt full > #0 0x00007ffff6a34859 in free () from /lib/x86_64-linux-gnu/libc.so.6 > No symbol table info available. > #1 0x0000000000592f46 in ber_memfree_x (p=0x426f6f462f6d6f74, ctx=0x0) at > memory.c:152 > __PRETTY_FUNCTION__ = "ber_memfree_x" > #2 0x0000000000579f5f in ldap_free_urldesc (ludp=0x8fcd10) at url.c:1528 > No locals. > #3 0x0000000000579f15 in ldap_free_urllist (ludlist=0x8fcd10) at url.c:1516 > ludp = 0x8fcd10 > next = 0x7375632f6374652f > #4 0x00007ffff635b2b3 in ldap_int_destroy_global_options () at init.c:484 > gopts = 0x7ffff6599080 > #5 0x00007ffff7defef9 in ?? () from /lib64/ld-linux-x86-64.so.2 > No symbol table info available. > #6 0x00007ffff7df09ce in ?? () from /lib64/ld-linux-x86-64.so.2 > No symbol table info available. > #7 0x00007ffff7deabd6 in ?? () from /lib64/ld-linux-x86-64.so.2 > No symbol table info available. > #8 0x00007ffff79d12ec in ?? () from /lib/x86_64-linux-gnu/libdl.so.2 > No symbol table info available. > #9 0x00007ffff79d100f in dlclose () from /lib/x86_64-linux-gnu/libdl.so.2 > No symbol table info available. > #10 0x00007ffff77c8291 in ?? () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2 > No symbol table info available. > #11 0x00007ffff77bfae1 in sasl_common_done () from > /usr/lib/x86_64-linux-gnu/libsasl2.so.2 > No symbol table info available. > #12 0x00000000004b34d4 in slap_sasl_destroy () at sasl.c:1190 > No locals. > #13 0x0000000000482039 in slap_destroy () at init.c:249 > rc = 0 > #14 0x00000000004ddbc8 in slap_tool_destroy () at slapcommon.c:932 > rc = 0 > #15 0x00000000004db86f in slapadd (argc=8, argv=0x7fffffffe648) at > slapadd.c:502 > textbuf = "\000xn_aborted! MDB_KEYEXIST: Key/data pair already exists > (-30799)", '\000' <repeats 191 times> > textlen = 256 > erec = {e = 0x8db288, lineno = 1, nextline = 14} > bvtext = {bv_len = 256, bv_val = 0x7fffffffe300 ""} > thr = 4270934 > id = 18446744073709551615 > prev = 0x0 > ldifrc = 1 > rc = 1 > stat_buf = {st_dev = 2307, st_ino = 149868, st_nlink = 1, st_mode = > 33188, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 2225, > st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1347128272, > tv_nsec = 0}, st_mtim = {tv_sec = 1347128189, tv_nsec = 0}, st_ctim = { > tv_sec = 1347128189, tv_nsec = 0}, __unused = {0, 0, 0}} > #16 0x000000000041e688 in main (argc=8, argv=0x7fffffffe648) at main.c:410 > i = 0 > no_detach = 0 > ---Type <return> to continue, or q <return> to quit--- > rc = 1 > urls = 0x0 > username = 0x0 > groupname = 0x0 > sandbox = 0x0 > syslogUser = 160 > pid = 32767 > waitfds = {0, 0} > g_argc = 8 > g_argv = 0x7fffffffe648 > configfile = 0x0 > configdir = 0x0 > serverName = 0x7fffffffe8c3 "slapadd" > serverMode = 1 > scp = 0x0 > scp_entry = 0x0 > debug_unknowns = 0x0 > syslog_unknowns = 0x0 > serverNamePrefix = 0x596388 "" > l = 140733193388033 > slapd_pid_file_unlink = 0 > slapd_args_file_unlink = 0 > firstopt = 1 > __PRETTY_FUNCTION__ = "main" > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7382) slapadd segfaults (during freeing resources)
by daniel＠pluta.biz 09 Sep '12

09 Sep '12

Full_Name: Daniel Pluta Version: OPENLDAP_REL_ENG OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2001:470:9feb:ff02:2958:c83c:2589:166d) Hi, root@srv:/home/tester/src/openldap# gdb /home/tester/src/openldap/servers/slapd/slapadd GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /home/tester/src/openldap/servers/slapd/slapadd...done. (gdb) run -v -F /etc/ldap/slapd_example.d_stat -b "dc=example,dc=com" -l /tmp/example_data.ldif Starting program: /home/tester/src/openldap/servers/slapd/slapadd -v -F /etc/ldap/slapd_example.d_stat -b "dc=example,dc=com" -l /tmp/example_data.ldif [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". 504cc479 mdb_id2entry_put: mdb_put failed: MDB_KEYEXIST: Key/data pair already exists(-30799) "dc=example,dc=com" 504cc479 => mdb_tool_entry_put: id2entry_add failed: err=-30799 504cc479 => mdb_tool_entry_put: txn_aborted! MDB_KEYEXIST: Key/data pair already exists (-30799) slapadd: could not add entry dn="dc=example,dc=com" (line=1): txn_aborted! MDB_KEYEXIST: Key/data pair already exists (-30799) _### 16.13% eta none elapsed none spd 501.5 k/s Closing DB... Program received signal SIGSEGV, Segmentation fault. 0x00007ffff6a34859 in free () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt full #0 0x00007ffff6a34859 in free () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #1 0x0000000000592f46 in ber_memfree_x (p=0x426f6f462f6d6f74, ctx=0x0) at memory.c:152 __PRETTY_FUNCTION__ = "ber_memfree_x" #2 0x0000000000579f5f in ldap_free_urldesc (ludp=0x8fcd10) at url.c:1528 No locals. #3 0x0000000000579f15 in ldap_free_urllist (ludlist=0x8fcd10) at url.c:1516 ludp = 0x8fcd10 next = 0x7375632f6374652f #4 0x00007ffff635b2b3 in ldap_int_destroy_global_options () at init.c:484 gopts = 0x7ffff6599080 #5 0x00007ffff7defef9 in ?? () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #6 0x00007ffff7df09ce in ?? () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #7 0x00007ffff7deabd6 in ?? () from /lib64/ld-linux-x86-64.so.2 No symbol table info available. #8 0x00007ffff79d12ec in ?? () from /lib/x86_64-linux-gnu/libdl.so.2 No symbol table info available. #9 0x00007ffff79d100f in dlclose () from /lib/x86_64-linux-gnu/libdl.so.2 No symbol table info available. #10 0x00007ffff77c8291 in ?? () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2 No symbol table info available. #11 0x00007ffff77bfae1 in sasl_common_done () from /usr/lib/x86_64-linux-gnu/libsasl2.so.2 No symbol table info available. #12 0x00000000004b34d4 in slap_sasl_destroy () at sasl.c:1190 No locals. #13 0x0000000000482039 in slap_destroy () at init.c:249 rc = 0 #14 0x00000000004ddbc8 in slap_tool_destroy () at slapcommon.c:932 rc = 0 #15 0x00000000004db86f in slapadd (argc=8, argv=0x7fffffffe648) at slapadd.c:502 textbuf = "\000xn_aborted! MDB_KEYEXIST: Key/data pair already exists (-30799)", '\000' <repeats 191 times> textlen = 256 erec = {e = 0x8db288, lineno = 1, nextline = 14} bvtext = {bv_len = 256, bv_val = 0x7fffffffe300 ""} thr = 4270934 id = 18446744073709551615 prev = 0x0 ldifrc = 1 rc = 1 stat_buf = {st_dev = 2307, st_ino = 149868, st_nlink = 1, st_mode = 33188, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 2225, st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1347128272, tv_nsec = 0}, st_mtim = {tv_sec = 1347128189, tv_nsec = 0}, st_ctim = { tv_sec = 1347128189, tv_nsec = 0}, __unused = {0, 0, 0}} #16 0x000000000041e688 in main (argc=8, argv=0x7fffffffe648) at main.c:410 i = 0 no_detach = 0 ---Type <return> to continue, or q <return> to quit--- rc = 1 urls = 0x0 username = 0x0 groupname = 0x0 sandbox = 0x0 syslogUser = 160 pid = 32767 waitfds = {0, 0} g_argc = 8 g_argv = 0x7fffffffe648 configfile = 0x0 configdir = 0x0 serverName = 0x7fffffffe8c3 "slapadd" serverMode = 1 scp = 0x0 scp_entry = 0x0 debug_unknowns = 0x0 syslog_unknowns = 0x0 serverNamePrefix = 0x596388 "" l = 140733193388033 slapd_pid_file_unlink = 0 slapd_args_file_unlink = 0 firstopt = 1 __PRETTY_FUNCTION__ = "main"

1 0

Re: (ITS#7364) mdb: clean up POSIX semaphores on environment close.
by h.b.furuseth＠usit.uio.no 06 Sep '12

06 Sep '12

Howard Chu writes: >h.b.furuseth(a)usit.uio.no wrote: >> This is worse with a database with is intended to be used by >> different users (A and B): > (...) >> mdb_env_setup_locks() can sem_unlink both semaphores before doing >> anything else, so that reopening a database as root will clean up. >> Drop the error checks of sem_unlink (so both get called), instead >> use O_EXCL in sem_open(,O_CREAT,,). Unless I'm missing something, >> the error checks just work like an emulation of O_EXCL anyway. > > The sem_unlink() and sem_open() sequence isn't ideal, certainly. I would > prefer to just use the existing semaphore. I just realized reusing the semaphores is a security problem: Anyone who can compute/read the semaphore names, can create the semaphores owned by themselves and then muck with them to sabotage someone else's mdb. I suppose the same goes for SysV semaphores. I don't know what the accepted way to deal with that is, but I suggest to unlink + O_EXCL as above for now, document the multi-uid restriction, and punt the rest until anyone who cares shows up. > Another possibility is to just use SysV semaphores instead of POSIX > semaphores. Then you can use the ipcs(1) command to manually cleanup. > (...) [later] > I'm also reminded that there is no defined behavior for SysV > semaphores in threads, they are only speciried for inter-process > synchronization. So forget that... I expect the work-around is to use them for IPC only, and wrap them in process-local mutexes in the MDB_env for thread sync. -- Hallvard

1 0

(ITS#7381) Dynamically added chaining configuration stops working after restart
by ian＠uns.ac.rs 05 Sep '12

05 Sep '12

Full_Name: Ivan Nejgebauer Version: 2.4.32 OS: CentOS 5 URL: ftp://ftp.openldap.org/incoming/ivannejgebauer-120905.tgz Submission from: (NULL) (2001:4170:2000:2:213:d3ff:fecf:b9ea) If a minimal chaining configuration is dynamically added to a consumer server in a provider/consumer pair running delta-syncrepl, it will stop working when the consumer is restarted. The configuration is modelled after the one used in test022 in the OpenLDAP source distribution: dn: olcOverlay={0}chain,olcDatabase={-1}frontend,cn=config objectClass: olcOverlayConfig objectClass: olcChainConfig olcOverlay: {0}chain dn: olcDatabase={0}ldap,olcOverlay={0}chain,olcDatabase={-1}frontend,cn=config objectClass: olcLDAPConfig objectClass: olcChainDatabase olcDatabase: {0}ldap olcDbIDAssertBind: bindmethod=simple binddn="cn=Manager,dc=example,dc=com" credentials="secret" mode=self olcDbURI: "ldap://localhost:6389" Scripts and data for replicating the issue are in the referenced archive. Quick summary: # make clean-all master replica # ./start-master.sh # ./start-replica.sh # ./replica-apply.sh pmod-add.ldif # returns referral # ./replica-conf.sh chain.ldif # adds chaining configuration # ./replica-apply.sh pmod-add.ldif # chains to master # ./stop-replica.sh # ./start-replica.sh # ./replica-apply.sh pmod-del.ldif # returns referral, but shouldn't I have tested both 2.4.32 and git master (54a111907b48900e1ce5de12a9d87818a334423d).

1 0

(ITS#7380) Open LDAP and Oracle Backend
by iliyas.nadaf＠gmail.com 04 Sep '12

04 Sep '12

Full_Name: iliyas Version: openldap-2.4.31 OS: solaris-x86 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (59.160.74.254) Hi, I need to develope "ladap client in C" with OpenLdap source and backend as Oracle database 11gr2. If it is possible suggest some resources. I have done it properly with BDB. Thanks, Iliyas

1 0

(ITS#7379) Open LDAP and Oracle Backend
by iliyas.nadaf＠gmail.com 04 Sep '12

04 Sep '12

Full_Name: iliyas Version: openldap-2.4.31 OS: solaris-x86 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (59.160.74.254) Hi, I need to develope "ladap client in C" with OpenLdap source and backend as Oracle database 11gr2. If it is possible suggest some resources. I have done it properly with BDB. Thanks, Iliyas

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by nikolai＠net24.co.nz 04 Sep '12

04 Sep '12

Thanks guys - I think we will look at going to MDB as well now. On 4/09/2012, at 8:42 AM, Quanah Gibson-Mount wrote: > --On Monday, September 03, 2012 6:35 PM +0000 hyc(a)symas.com wrote: > >> Nikolai Schupbach wrote: >>> Hi Howard, >>> >>> Thank you very much for the explanation. What BDB version would you >> recommend. Obviously I have quite a few options and would like to use a >> version that is known to be very solid. >> >> I believe 4.7.25 + all 4 of its official patches was pretty stable. >> http://www.oracle.com/technetwork/products/berkeleydb/patch-088170.html >> >> I've done limited testing with 4.8.30, 5.1.19, and 5.3.21. At this point >> I'm no longer tracking BDB revisions since MDB has superior performance >> while using 1/4 as much RAM and requiring no tuning. > > We've been using BDB 4.7.25+all 4 patches without issue for several years as well. However, I will also note that we are now switching over to MDB as well for our production services starting with OpenLDAP 2.4.32. > > --Quanah > > > -- > > Quanah Gibson-Mount > Sr. Member of Technical Staff > Zimbra, Inc > A Division of VMware, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by quanah＠zimbra.com 03 Sep '12

03 Sep '12

--On Monday, September 03, 2012 6:35 PM +0000 hyc(a)symas.com wrote: > Nikolai Schupbach wrote: >> Hi Howard, >> >> Thank you very much for the explanation. What BDB version would you > recommend. Obviously I have quite a few options and would like to use a > version that is known to be very solid. > > I believe 4.7.25 + all 4 of its official patches was pretty stable. > http://www.oracle.com/technetwork/products/berkeleydb/patch-088170.html > > I've done limited testing with 4.8.30, 5.1.19, and 5.3.21. At this point > I'm no longer tracking BDB revisions since MDB has superior performance > while using 1/4 as much RAM and requiring no tuning. We've been using BDB 4.7.25+all 4 patches without issue for several years as well. However, I will also note that we are now switching over to MDB as well for our production services starting with OpenLDAP 2.4.32. --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by hyc＠symas.com 03 Sep '12

03 Sep '12

Nikolai Schupbach wrote: > Hi Howard, > > Thank you very much for the explanation. What BDB version would you recommend. Obviously I have quite a few options and would like to use a version that is known to be very solid. I believe 4.7.25 + all 4 of its official patches was pretty stable. http://www.oracle.com/technetwork/products/berkeleydb/patch-088170.html I've done limited testing with 4.8.30, 5.1.19, and 5.3.21. At this point I'm no longer tracking BDB revisions since MDB has superior performance while using 1/4 as much RAM and requiring no tuning. > Sincerely, > Nikolai Schupbach > > On 3/09/2012, at 9:45 PM, Howard Chu wrote: > >> nikolai(a)net24.co.nz wrote: >>> Full_Name: Nikolai Schupbach >>> Version: 2.4.31 >>> OS: FreeBSD >>> URL: ftp://ftp.openldap.org/incoming/ >>> Submission from: (NULL) (202.78.158.60) >>> >>> >>> We are experiencing frequent hangs in slapd. Once hung we can continue to >>> connect, but all searches will just hang indefinitely until we kill -9 the slapd >>> process and restart it. The directory is used for mail routing and we have been >>> migrating to it from an existing directory server over the last 3 weeks - we >>> have noted the busier the directory becomes the more often it hangs (now once >>> every 2 days). >>> >>> We have one master and 10 syncrepl read only replicas - the master is used >>> mainly for writes and has not hung yet, but most of the replicas have hung at >>> least once. The replicas receive anywhere between 50 to 300 searches/sec, while >>> the master would only get 1/sec. There are 45k entries in the directory. >>> >>> We are running: >>> >>> FreeBSD 8.3/9.0 x64 >>> OpenLDAP 2.4.31 >>> Berkeley DB 4.6.21 >>> >>> The old directory we are migrating from has the same load and is also running >>> OpenLDAP, but has been rock solid for 5 years. It is running Berkeley DB 4.3.29 >>> and OpenLDAP 2.3.27. >>> >>> We have managed to collect db_stat lock information, which indicates the same >>> issue each time - a write lock on dn2id.bdb. >> >> It's more than that. Your db_stat shows that a single thread has 3 active >> transactions. This should never happen: >> >> 8000a85e dd= 0 locks held 2 write locks 0 pid/thread 88000/34386526336 >> 8000a85e READ 1 HELD 0xb19a8 len: 9 data: 40xa800000000000000 >> 8000a85e READ 1 HELD 0xb26c8 len: 9 data: 60xa800000000000000 >> 8000a85f dd= 0 locks held 8 write locks 4 pid/thread 88000/34386526336 >> 8000a85f READ 1 WAIT dn2id.bdb page 559 >> 8000a85f READ 1 HELD dn2id.bdb page 768 >> 8000a85f WRITE 2 HELD dn2id.bdb page 1362 >> 8000a85f READ 2 HELD dn2id.bdb page 1362 >> 8000a85f WRITE 2 HELD dn2id.bdb page 1353 >> 8000a85f READ 2 HELD dn2id.bdb page 1353 >> 8000a85f WRITE 2 HELD dn2id.bdb page 933 >> 8000a85f READ 1 HELD dn2id.bdb page 933 >> 8000a85f WRITE 4 HELD dn2id.bdb page 219 >> 80001047 dd=28 locks held 1 write locks 1 pid/thread 88000/34386526336 >> 80001047 WRITE 1 HELD dn2id.bdb page 559 >> >> I would first recommend changing from BDB 4.6.21 to some other version. There >> are no code paths in back-bdb where we would ever return without either >> committing or aborting the current transactions, so this appears to be a BDB >> bug, not an OpenLDAP bug. >> >>> We have also collected the backtrace for all the threads which I have uploaded >>> to: >>> >>> ftp://ftp.openldap.org/incoming/nikolai-gdb-120902.txt >>> >>> The full db_stat output is located at: >>> >>> ftp://ftp.openldap.org/incoming/nikolai-dbstat-120902.txt >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by nikolai＠net24.co.nz 03 Sep '12

03 Sep '12

Hi Howard, Thank you very much for the explanation. What BDB version would you recommend. Obviously I have quite a few options and would like to use a version that is known to be very solid. Sincerely, Nikolai Schupbach On 3/09/2012, at 9:45 PM, Howard Chu wrote: > nikolai(a)net24.co.nz wrote: >> Full_Name: Nikolai Schupbach >> Version: 2.4.31 >> OS: FreeBSD >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (202.78.158.60) >> >> >> We are experiencing frequent hangs in slapd. Once hung we can continue to >> connect, but all searches will just hang indefinitely until we kill -9 the slapd >> process and restart it. The directory is used for mail routing and we have been >> migrating to it from an existing directory server over the last 3 weeks - we >> have noted the busier the directory becomes the more often it hangs (now once >> every 2 days). >> >> We have one master and 10 syncrepl read only replicas - the master is used >> mainly for writes and has not hung yet, but most of the replicas have hung at >> least once. The replicas receive anywhere between 50 to 300 searches/sec, while >> the master would only get 1/sec. There are 45k entries in the directory. >> >> We are running: >> >> FreeBSD 8.3/9.0 x64 >> OpenLDAP 2.4.31 >> Berkeley DB 4.6.21 >> >> The old directory we are migrating from has the same load and is also running >> OpenLDAP, but has been rock solid for 5 years. It is running Berkeley DB 4.3.29 >> and OpenLDAP 2.3.27. >> >> We have managed to collect db_stat lock information, which indicates the same >> issue each time - a write lock on dn2id.bdb. > > It's more than that. Your db_stat shows that a single thread has 3 active > transactions. This should never happen: > > 8000a85e dd= 0 locks held 2 write locks 0 pid/thread 88000/34386526336 > 8000a85e READ 1 HELD 0xb19a8 len: 9 data: 40xa800000000000000 > 8000a85e READ 1 HELD 0xb26c8 len: 9 data: 60xa800000000000000 > 8000a85f dd= 0 locks held 8 write locks 4 pid/thread 88000/34386526336 > 8000a85f READ 1 WAIT dn2id.bdb page 559 > 8000a85f READ 1 HELD dn2id.bdb page 768 > 8000a85f WRITE 2 HELD dn2id.bdb page 1362 > 8000a85f READ 2 HELD dn2id.bdb page 1362 > 8000a85f WRITE 2 HELD dn2id.bdb page 1353 > 8000a85f READ 2 HELD dn2id.bdb page 1353 > 8000a85f WRITE 2 HELD dn2id.bdb page 933 > 8000a85f READ 1 HELD dn2id.bdb page 933 > 8000a85f WRITE 4 HELD dn2id.bdb page 219 > 80001047 dd=28 locks held 1 write locks 1 pid/thread 88000/34386526336 > 80001047 WRITE 1 HELD dn2id.bdb page 559 > > I would first recommend changing from BDB 4.6.21 to some other version. There > are no code paths in back-bdb where we would ever return without either > committing or aborting the current transactions, so this appears to be a BDB > bug, not an OpenLDAP bug. > >> We have also collected the backtrace for all the threads which I have uploaded >> to: >> >> ftp://ftp.openldap.org/incoming/nikolai-gdb-120902.txt >> >> The full db_stat output is located at: >> >> ftp://ftp.openldap.org/incoming/nikolai-dbstat-120902.txt > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7364) mdb: clean up POSIX semaphores on environment close.
by h.b.furuseth＠usit.uio.no 03 Sep '12

03 Sep '12

Howard Chu writes: > h.b.furuseth(a)usit.uio.no wrote: >> Reopening this. >> >> >> This is worse with a database with is intended to be used by >> different users (A and B): > > This is pretty much never a use case we would worry about. In most > applications, a single userID creates and operates on the DB. I'm fine with just documenting that as a restriction on some systems. If not: >> (...) >> The work-around I can think of is a "multi-uid" mode which instead >> resets the semaphore with sem_post() if sem_getvalue() returns 0. >> I don't know how ugly that is considered to be. Could ask >> comp.programming.unix, or check what Berkeley DB does. > > That would be OK in general. It still leaves the question of how to remove the > semaphore if the DB is being destroyed. But it's probably not worth the > trouble to worry about this so much. Those OSes should just get their act > together and support the POSIX process-shared mutexes. > >> This mode >> should use mode 0666 for the semaphores (temporarily setting umask >> 0, yuck), > > Definitely not. The caller specifies a mode; if they want 0666 they > should configure it as such. 0666 would likely be wrong for the database file(s). But this flag could just as well consist of specifying a mode parameter for the semaphores. It's a threaded library doing umask() I dislike. And, as you say, the need to remove the semaphore afterwards. >> or it should not sem_unlink() since next user may create >> the semaphores with a group which gives the wrong users access. > > Same as today where running slapadd with the wrong uid causes trouble > for the following slapd. The answer is obvious: use the right uid when > accessing the DB. We're talking about the case where there is no single "right" uid. Not relevant for slapd, only libmdb by itself. >> Other matters with the current implementation - I'll patch these: >> >> mdb_env_excl_lock() need not retry getting a non-exclusive lock when >> closing. mdb_env_close() can pass *excl = -1 to tell it not to. >> >> mdb_env_setup_locks() can sem_unlink both semaphores before doing >> anything else, so that reopening a database as root will clean up. >> Drop the error checks of sem_unlink (so both get called), instead >> use O_EXCL in sem_open(,O_CREAT,,). Unless I'm missing something, >> the error checks just work like an emulation of O_EXCL anyway. > > The sem_unlink() and sem_open() sequence isn't ideal, certainly. I would > prefer to just use the existing semaphore. ...followed by 'if (sem_getvalue() shows 0) sem_post()' as above, then. -- Hallvard

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by hyc＠symas.com 03 Sep '12

03 Sep '12

michael(a)stroeder.com wrote: > This is a cryptographically signed message in MIME format. > > --------------ms080100030105010600070605 > Content-Type: text/plain; charset=ISO-8859-1 > Content-Transfer-Encoding: quoted-printable > > A couple of days ago I had a hang with OpenLDAP 2.4.32 / back-hdb running= > on > Debian Squeeze, self-compiled against BDB 4.8.30. It seemed Database was > locked as restarting slapd of even rebooting OS did not help. Unfortunate= > ly I > had to bring up the system as fast as possible and could not examine the = > problem. db_recover will always return the DB to a usable state and reset any DB locks. (It completely deletes the lock region, so there cannot be any stale locks after it runs.) > The system has only 200 entries and not much load yet. I had renamed entr= > ies > with web2ldap when all 4 masters (4-way MMR) locked up one after the othe= > r. > So there seem to be lockup problems in 2.4.32. The only way to know if you're seeing the same problem as the original poster is if you provide db_stat -CA and gdb trace output, like the original poster did. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7364) mdb: clean up POSIX semaphores on environment close.
by hyc＠symas.com 03 Sep '12

03 Sep '12

Maucci, Cyrille wrote: > -----Original Message----- > From: openldap-bugs-bounces(a)OpenLDAP.org [mailto:openldap-bugs-bounces@OpenLDAP.org] On Behalf Of hyc(a)symas.com >>> Another possibility is to just use SysV semaphores instead of POSIX semaphores. >>> Then you can use the ipcs(1) command to manually cleanup. BerkeleyDB uses >>> SysV shared memory when you specify a shared memory environment and it >>> appears that SysV semaphore support is actually more widespread than POSIX semaphores. > > Just to mention that at least on HPUX, Posix semaphores are more efficient than SysV ones. I'm also reminded that there is no defined behavior for SysV semaphores in threads, they are only speciried for inter-process synchronization. So forget that... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by hyc＠symas.com 03 Sep '12

03 Sep '12

nikolai(a)net24.co.nz wrote: > Full_Name: Nikolai Schupbach > Version: 2.4.31 > OS: FreeBSD > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (202.78.158.60) > > > We are experiencing frequent hangs in slapd. Once hung we can continue to > connect, but all searches will just hang indefinitely until we kill -9 the slapd > process and restart it. The directory is used for mail routing and we have been > migrating to it from an existing directory server over the last 3 weeks - we > have noted the busier the directory becomes the more often it hangs (now once > every 2 days). > > We have one master and 10 syncrepl read only replicas - the master is used > mainly for writes and has not hung yet, but most of the replicas have hung at > least once. The replicas receive anywhere between 50 to 300 searches/sec, while > the master would only get 1/sec. There are 45k entries in the directory. > > We are running: > > FreeBSD 8.3/9.0 x64 > OpenLDAP 2.4.31 > Berkeley DB 4.6.21 > > The old directory we are migrating from has the same load and is also running > OpenLDAP, but has been rock solid for 5 years. It is running Berkeley DB 4.3.29 > and OpenLDAP 2.3.27. > > We have managed to collect db_stat lock information, which indicates the same > issue each time - a write lock on dn2id.bdb. It's more than that. Your db_stat shows that a single thread has 3 active transactions. This should never happen: 8000a85e dd= 0 locks held 2 write locks 0 pid/thread 88000/34386526336 8000a85e READ 1 HELD 0xb19a8 len: 9 data: 40xa800000000000000 8000a85e READ 1 HELD 0xb26c8 len: 9 data: 60xa800000000000000 8000a85f dd= 0 locks held 8 write locks 4 pid/thread 88000/34386526336 8000a85f READ 1 WAIT dn2id.bdb page 559 8000a85f READ 1 HELD dn2id.bdb page 768 8000a85f WRITE 2 HELD dn2id.bdb page 1362 8000a85f READ 2 HELD dn2id.bdb page 1362 8000a85f WRITE 2 HELD dn2id.bdb page 1353 8000a85f READ 2 HELD dn2id.bdb page 1353 8000a85f WRITE 2 HELD dn2id.bdb page 933 8000a85f READ 1 HELD dn2id.bdb page 933 8000a85f WRITE 4 HELD dn2id.bdb page 219 80001047 dd=28 locks held 1 write locks 1 pid/thread 88000/34386526336 80001047 WRITE 1 HELD dn2id.bdb page 559 I would first recommend changing from BDB 4.6.21 to some other version. There are no code paths in back-bdb where we would ever return without either committing or aborting the current transactions, so this appears to be a BDB bug, not an OpenLDAP bug. > We have also collected the backtrace for all the threads which I have uploaded > to: > > ftp://ftp.openldap.org/incoming/nikolai-gdb-120902.txt > > The full db_stat output is located at: > > ftp://ftp.openldap.org/incoming/nikolai-dbstat-120902.txt -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

RE: (ITS#7364) mdb: clean up POSIX semaphores on environment close.
by cyrille.maucci＠hp.com 03 Sep '12

03 Sep '12

-----Original Message----- From: openldap-bugs-bounces(a)OpenLDAP.org [mailto:openldap-bugs-bounces@OpenLDAP.org] On Behalf Of hyc(a)symas.com >> Another possibility is to just use SysV semaphores instead of POSIX semaphores. >> Then you can use the ipcs(1) command to manually cleanup. BerkeleyDB uses >> SysV shared memory when you specify a shared memory environment and it >> appears that SysV semaphore support is actually more widespread than POSIX semaphores. Just to mention that at least on HPUX, Posix semaphores are more efficient than SysV ones. ++Cyrille

1 0

Re: (ITS#7364) mdb: clean up POSIX semaphores on environment close.
by hyc＠symas.com 03 Sep '12

03 Sep '12

h.b.furuseth(a)usit.uio.no wrote: > Reopening this. > > > This is worse with a database with is intended to be used by > different users (A and B): This is pretty much never a use case we would worry about. In most applications, a single userID creates and operates on the DB. > A opens the DB and creates the semaphores with e.g. mode 0660. > B opens it, A closes, B closes - and fails sem_unlink() which > only A and root can do. > > Next, B (or C) fails mdb_env_open() because sem_unlink() fails > again. > > The work-around I can think of is a "multi-uid" mode which instead > resets the semaphore with sem_post() if sem_getvalue() returns 0. > I don't know how ugly that is considered to be. Could ask > comp.programming.unix, or check what Berkeley DB does. That would be OK in general. It still leaves the question of how to remove the semaphore if the DB is being destroyed. But it's probably not worth the trouble to worry about this so much. Those OSes should just get their act together and support the POSIX process-shared mutexes. > This mode > should use mode 0666 for the semaphores (temporarily setting umask > 0, yuck), Definitely not. The caller specifies a mode; if they want 0666 they should configure it as such. > or it should not sem_unlink() since next user may create > the semaphores with a group which gives the wrong users access. Same as today where running slapadd with the wrong uid causes trouble for the following slapd. The answer is obvious: use the right uid when accessing the DB. > Other matters with the current implementation - I'll patch these: > > mdb_env_excl_lock() need not retry getting a non-exclusive lock when > closing. mdb_env_close() can pass *excl = -1 to tell it not to. > > mdb_env_setup_locks() can sem_unlink both semaphores before doing > anything else, so that reopening a database as root will clean up. > Drop the error checks of sem_unlink (so both get called), instead > use O_EXCL in sem_open(,O_CREAT,,). Unless I'm missing something, > the error checks just work like an emulation of O_EXCL anyway. The sem_unlink() and sem_open() sequence isn't ideal, certainly. I would prefer to just use the existing semaphore. Another possibility is to just use SysV semaphores instead of POSIX semaphores. Then you can use the ipcs(1) command to manually cleanup. BerkeleyDB uses SysV shared memory when you specify a shared memory environment and it appears that SysV semaphore support is actually more widespread than POSIX semaphores. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by michael＠stroeder.com 02 Sep '12

02 Sep '12

This is a cryptographically signed message in MIME format. --------------ms080100030105010600070605 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable A couple of days ago I had a hang with OpenLDAP 2.4.32 / back-hdb running= on Debian Squeeze, self-compiled against BDB 4.8.30. It seemed Database was locked as restarting slapd of even rebooting OS did not help. Unfortunate= ly I had to bring up the system as fast as possible and could not examine the = problem. The system has only 200 entries and not much load yet. I had renamed entr= ies with web2ldap when all 4 masters (4-way MMR) locked up one after the othe= r. So there seem to be lockup problems in 2.4.32. --------------ms080100030105010600070605 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIILHzCC BT8wggQnoAMCAQICDwCmSwABAAIAivjZQ8SBvzANBgkqhkiG9w0BAQUFADB8MQswCQYDVQQG EwJERTEcMBoGA1UEChMTVEMgVHJ1c3RDZW50ZXIgR21iSDElMCMGA1UECxMcVEMgVHJ1c3RD ZW50ZXIgQ2xhc3MgMSBMMSBDQTEoMCYGA1UEAxMfVEMgVHJ1c3RDZW50ZXIgQ2xhc3MgMSBM MSBDQSBJWDAeFw0xMjA2MDYxOTAyMTZaFw0xMzA2MDcxOTAyMTZaMCgxCzAJBgNVBAYTAkRF MRkwFwYDVQQDDBBNaWNoYWVsIFN0csO2ZGVyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB CgKCAQEAxXZGav40rnGNLxEggBW94MILWHlfC8a23Jew5U1gPlfRTXOjjzmoaZ1uCyGdgF6M VvuO9T1aTQNGH+OdeGe3P7Tfc/NsLJFJ2wtd8blvhmodUgse2eypiWjNOd4gZuhalBhgsQ0K b5D6/1foghII4E264iZlJ7AJ+UYcO+GxvFWT0YMTbLckgDkZk7c3qwTozdhYvXarvqx+8Ou/ kuxpQQhac/ebzxpu0N+RHSf2KIUS0g0tEGnPtGv6iL+9QNHc4JKo9Y9KKVw3tQy+Re+FQLxB 1fPE5F+qxuD3AUENpOwkMsqWLM94ohtx3CFqLpxfUPrnKFLAHOhHEbByYGvFPwIDAQABo4IC EDCCAgwwgaUGCCsGAQUFBwEBBIGYMIGVMFEGCCsGAQUFBzAChkVodHRwOi8vd3d3LnRydXN0 Y2VudGVyLmRlL2NlcnRzZXJ2aWNlcy9jYWNlcnRzL3RjX2NsYXNzMV9MMV9DQV9JWC5jcnQw QAYIKwYBBQUHMAGGNGh0dHA6Ly9vY3NwLml4LnRjY2xhc3MxLnRjdW5pdmVyc2FsLWkudHJ1 c3RjZW50ZXIuZGUwHwYDVR0jBBgwFoAU6bgoHUbP/M34TpvF7ktg69g7P9EwDAYDVR0TAQH/ BAIwADBKBgNVHSAEQzBBMD8GCSqCFAAsAQEBATAyMDAGCCsGAQUFBwIBFiRodHRwOi8vd3d3 LnRydXN0Y2VudGVyLmRlL2d1aWRlbGluZXMwDgYDVR0PAQH/BAQDAgTwMB0GA1UdDgQWBBS2 KAWfTfgJ/JQ63qLGwTXYLnI+LzBiBgNVHR8EWzBZMFegVaBThlFodHRwOi8vY3JsLml4LnRj Y2xhc3MxLnRjdW5pdmVyc2FsLWkudHJ1c3RjZW50ZXIuZGUvY3JsL3YyL3RjX0NsYXNzMV9M MV9DQV9JWC5jcmwwMwYDVR0lBCwwKgYIKwYBBQUHAwIGCCsGAQUFBwMEBggrBgEFBQcDBwYK KwYBBAGCNxQCAjAfBgNVHREEGDAWgRRtaWNoYWVsQHN0cm9lZGVyLmNvbTANBgkqhkiG9w0B AQUFAAOCAQEAQ3bvVUpEq+cQrLpcogyt5BJNk/WvUvOHqhzyj28M9pg9hcDl1+MYl5qqj6tR GSTLPQZyf287pcmbMwbcTGZO/gbW9v7RYcut6RauWdwKMCUmKC3J4fVfDq9ZETA2WOV68ef4 B3Gzdhghsbp3Rhp5dDmrCVKAHlafm6ZwJrEQ9P76fxnQZzRLgeKpZep5ePH5YHUB3+YaOQvJ FG0bOXvfHhRiRG7/HW2G+yDgjHSxDz8AFzMWL/RFePqZ4pn6T/SM/qU6WEpW39MWyJNoH/Kx QDYK8gGYuesn1ciMCTnjrvZQj0fonGTO4SfWekJRkuGrJ7dYSZRjYbDcWBBkdFLWzzCCBdgw ggTAoAMCAQICDgboAAEAAkqWLSQM/sXJMA0GCSqGSIb3DQEBBQUAMHkxCzAJBgNVBAYTAkRF MRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSQwIgYDVQQLExtUQyBUcnVzdENlbnRl ciBVbml2ZXJzYWwgQ0ExJjAkBgNVBAMTHVRDIFRydXN0Q2VudGVyIFVuaXZlcnNhbCBDQSBJ MB4XDTA5MTEwMzE0MDgxOVoXDTI1MTIzMTIxNTk1OVowfDELMAkGA1UEBhMCREUxHDAaBgNV BAoTE1RDIFRydXN0Q2VudGVyIEdtYkgxJTAjBgNVBAsTHFRDIFRydXN0Q2VudGVyIENsYXNz IDEgTDEgQ0ExKDAmBgNVBAMTH1RDIFRydXN0Q2VudGVyIENsYXNzIDEgTDEgQ0EgSVgwggEi MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC75pBuz2Lp6QuqthDVR+V8XSsncZpozVVt 5KLv5P7yemMRwleKyH3PjmYfZUVL64Biab1GjovFblqVGCrep/EfdRonq20yU+P7TVhiLP8Z 5cegDZotIYhZhM0d8cPIij6w5d4IJM/8QCy6QSOUu4ASiTVItoYE4AFPjLqpmPwcie0fiqHH hpgmHnJla/7PZdkMZEsaCfVDEWBmJuMzVprJPT40anjG5VBLyM2I5DlsUCaeQCy2O3w3sqf1 3dyzUcv03IICuNc63towXA31Qt0TaVNU6YAmQjMepdfMbspmCZ+G8D2+xophEPPR/1vkstst smUMqX0XrLonTUJczglPAgMBAAGjggJZMIICVTCBmgYIKwYBBQUHAQEEgY0wgYowUgYIKwYB BQUHMAKGRmh0dHA6Ly93d3cudHJ1c3RjZW50ZXIuZGUvY2VydHNlcnZpY2VzL2NhY2VydHMv dGNfdW5pdmVyc2FsX3Jvb3RfSS5jcnQwNAYIKwYBBQUHMAGGKGh0dHA6Ly9vY3NwLnRjdW5p dmVyc2FsLUkudHJ1c3RjZW50ZXIuZGUwHwYDVR0jBBgwFoAUkqR1LKSevoFE63n8isWVpesQ dXMwEgYDVR0TAQH/BAgwBgEB/wIBADBSBgNVHSAESzBJMAYGBFUdIAAwPwYJKoIUACwBAQEB MDIwMAYIKwYBBQUHAgEWJGh0dHA6Ly93d3cudHJ1c3RjZW50ZXIuZGUvZ3VpZGVsaW5lczAO BgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYEFOm4KB1Gz/zN+E6bxe5LYOvYOz/RMIH9BgNVHR8E gfUwgfIwge+ggeyggemGRmh0dHA6Ly9jcmwudGN1bml2ZXJzYWwtSS50cnVzdGNlbnRlci5k ZS9jcmwvdjIvdGNfdW5pdmVyc2FsX3Jvb3RfSS5jcmyGgZ5sZGFwOi8vd3d3LnRydXN0Y2Vu dGVyLmRlL0NOPVRDJTIwVHJ1c3RDZW50ZXIlMjBVbml2ZXJzYWwlMjBDQSUyMEksTz1UQyUy MFRydXN0Q2VudGVyJTIwR21iSCxPVT1yb290Y2VydHMsREM9dHJ1c3RjZW50ZXIsREM9ZGU/ Y2VydGlmaWNhdGVSZXZvY2F0aW9uTGlzdD9iYXNlPzANBgkqhkiG9w0BAQUFAAOCAQEAOcjE m+6+mO5Icm+N53G2DpCM07LBFSGoRpBoX0oE8TrJaIQh2KXmBHVdn9LU8kt3QzLclctgvwJV 0KwcsMUUl5tlCsMPpR3s2Ek5lbWpvvr0HqtW56blAQiINV9nBd1EJFASIkRjefGbV2nOq9Yz UU+N8HA7jq1ROhd/NZZraGhjthwKyfjfHV7PKxGlY+3M0MbTIG+q/GhIfm0euDpFqhKG88e9 ALXr/uoSn3MzeOcoOWjTpW3adtFO4VWVgKbgG7jNrFbvRVlHmFLbOm4msjE5aXWxLiTwpJ2X iF4zKca1vAdAOgw9us90jEtOeiH6GzjNxEMvb7TfeO6Zkuc6HDGCA84wggPKAgEBMIGPMHwx CzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYDVQQLExxU QyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENlbnRlciBD bGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wCQYFKw4DAhoFAKCCAhMwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTIwOTAyMTMxMTU0WjAjBgkq hkiG9w0BCQQxFgQUCNHgSamIXCRCd4ANj7PKCgRv3BwwbAYJKoZIhvcNAQkPMV8wXTALBglg hkgBZQMEASowCwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggq hkiG9w0DAgIBQDAHBgUrDgMCBzANBggqhkiG9w0DAgIBKDCBoAYJKwYBBAGCNxAEMYGSMIGP MHwxCzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYDVQQL ExxUQyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENlbnRl ciBDbGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wgaIGCyqGSIb3DQEJEAILMYGS oIGPMHwxCzAJBgNVBAYTAkRFMRwwGgYDVQQKExNUQyBUcnVzdENlbnRlciBHbWJIMSUwIwYD VQQLExxUQyBUcnVzdENlbnRlciBDbGFzcyAxIEwxIENBMSgwJgYDVQQDEx9UQyBUcnVzdENl bnRlciBDbGFzcyAxIEwxIENBIElYAg8ApksAAQACAIr42UPEgb8wDQYJKoZIhvcNAQEBBQAE ggEAn/eQg7NrUAmmHsFX+FOpynezJ9ocm/9InAnbWvoHFvdbqKYZIRRmu+aZ1a6q3irVg1FX YrzH+OivltVXyvoQvt7WCMeUjiQRjM0PuRd0YlUGu/8qvEVPtcv8i3K/v74MIibRUpItxNes 47NssmY88640oWdfBqkI/KQS44c5rjBADFwPMMXAbWptp4AXoit1MaMTrsPt+2+O2iXAVREB IJkffN1lb6zeizUzWc/H3iPvVMszLZXdTOh/hknsHJHZmwsEqNzNHYhRgYkPoYiiHyi+faa/ mqPSc+22TqsnWL1wYE0HhG3uVWg1fUf3UyhUJ8/VXwmNe9ucfiQI9Kk5rwAAAAAAAA== --------------ms080100030105010600070605--

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by nikolai＠net24.co.nz 01 Sep '12

01 Sep '12

I haven't yet - I wanted to collect information before making any changes. I did look at that fix and wasn't confident it would solve our problem. You're right though - I need to test it to rule it out. I will upgrade all the servers to 2.4.32 and report back. On 2/09/2012, at 7:07 AM, Quanah Gibson-Mount wrote: > --On Saturday, September 01, 2012 1:46 PM +0000 nikolai(a)net24.co.nz wrote: > >> Full_Name: Nikolai Schupbach >> Version: 2.4.31 >> OS: FreeBSD >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (202.78.158.60) > > Have you confirmed this isn't the same thing ITS#7222, fixed in OpenLDAP 2.4.32? > > --Quanah > > > > -- > > Quanah Gibson-Mount > Sr. Member of Technical Staff > Zimbra, Inc > A Division of VMware, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration

1 0

Re: (ITS#7378) Slapd hangs on bdb write lock
by quanah＠zimbra.com 01 Sep '12

01 Sep '12

--On Saturday, September 01, 2012 1:46 PM +0000 nikolai(a)net24.co.nz wrote: > Full_Name: Nikolai Schupbach > Version: 2.4.31 > OS: FreeBSD > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (202.78.158.60) Have you confirmed this isn't the same thing ITS#7222, fixed in OpenLDAP 2.4.32? --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7378) Slapd hangs on bdb write lock
by nikolai＠net24.co.nz 01 Sep '12

01 Sep '12

Full_Name: Nikolai Schupbach Version: 2.4.31 OS: FreeBSD URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (202.78.158.60) We are experiencing frequent hangs in slapd. Once hung we can continue to connect, but all searches will just hang indefinitely until we kill -9 the slapd process and restart it. The directory is used for mail routing and we have been migrating to it from an existing directory server over the last 3 weeks - we have noted the busier the directory becomes the more often it hangs (now once every 2 days). We have one master and 10 syncrepl read only replicas - the master is used mainly for writes and has not hung yet, but most of the replicas have hung at least once. The replicas receive anywhere between 50 to 300 searches/sec, while the master would only get 1/sec. There are 45k entries in the directory. We are running: FreeBSD 8.3/9.0 x64 OpenLDAP 2.4.31 Berkeley DB 4.6.21 The old directory we are migrating from has the same load and is also running OpenLDAP, but has been rock solid for 5 years. It is running Berkeley DB 4.3.29 and OpenLDAP 2.3.27. We have managed to collect db_stat lock information, which indicates the same issue each time - a write lock on dn2id.bdb. Locks grouped by object: Locker Mode Count Status ----------------- Object --------------- 8000a85e READ 1 HELD 0xb26c8 len: 9 data: 60xa800000000000000 8a READ 1 HELD id2entry.bdb handle 0 8c READ 1 HELD dn2id.bdb handle 0 96 READ 1 HELD objectClass.bdb handle 0 93 READ 1 HELD entryCSN.bdb handle 0 90 READ 1 HELD entryUUID.bdb handle 0 8000a85f WRITE 4 HELD dn2id.bdb page 219 80000782 READ 1 HELD dn2id.bdb page 768 80000a45 READ 1 HELD dn2id.bdb page 768 80000b9e READ 1 HELD dn2id.bdb page 768 800006a0 READ 1 HELD dn2id.bdb page 768 80000771 READ 1 HELD dn2id.bdb page 768 80000534 READ 1 HELD dn2id.bdb page 768 80000a44 READ 1 HELD dn2id.bdb page 768 80000641 READ 1 HELD dn2id.bdb page 768 80001049 READ 1 HELD dn2id.bdb page 768 8000104a READ 1 HELD dn2id.bdb page 768 80001048 READ 1 HELD dn2id.bdb page 768 80000783 READ 1 HELD dn2id.bdb page 768 80000535 READ 1 HELD dn2id.bdb page 768 8000066e READ 1 HELD dn2id.bdb page 768 80000697 READ 1 HELD dn2id.bdb page 768 8000a85f READ 1 HELD dn2id.bdb page 768 8000a85e READ 1 HELD 0xb19a8 len: 9 data: 40xa800000000000000 8000a85f READ 1 HELD dn2id.bdb page 933 8000a85f WRITE 2 HELD dn2id.bdb page 933 80001047 WRITE 1 HELD dn2id.bdb page 559 80000782 READ 1 WAIT dn2id.bdb page 559 80000a45 READ 1 WAIT dn2id.bdb page 559 80000b9e READ 1 WAIT dn2id.bdb page 559 800006a0 READ 1 WAIT dn2id.bdb page 559 80000771 READ 1 WAIT dn2id.bdb page 559 80000534 READ 1 WAIT dn2id.bdb page 559 80000a44 READ 1 WAIT dn2id.bdb page 559 80000641 READ 1 WAIT dn2id.bdb page 559 80001049 READ 1 WAIT dn2id.bdb page 559 8000104a READ 1 WAIT dn2id.bdb page 559 80001048 READ 1 WAIT dn2id.bdb page 559 80000783 READ 1 WAIT dn2id.bdb page 559 80000535 READ 1 WAIT dn2id.bdb page 559 8000066e READ 1 WAIT dn2id.bdb page 559 80000697 READ 1 WAIT dn2id.bdb page 559 8000a85f READ 1 WAIT dn2id.bdb page 559 8000a85f READ 2 HELD dn2id.bdb page 1362 8000a85f WRITE 2 HELD dn2id.bdb page 1362 8000a85f READ 2 HELD dn2id.bdb page 1353 8000a85f WRITE 2 HELD dn2id.bdb page 1353 b6 READ 1 HELD uid.bdb handle 0 a5 READ 1 HELD mail.bdb handle 0 af READ 1 HELD mailLocalAddress.bdb handle 0 9b READ 1 HELD miLoginid.bdb handle 0 aa READ 1 HELD mailHost.bdb handle 0 bb READ 1 HELD miDomainName.bdb handle 0 c0 READ 1 HELD mpMailHost.bdb handle 0 a0 READ 1 HELD mpMailUserType.bdb handle 0 We have also collected the backtrace for all the threads which I have uploaded to: ftp://ftp.openldap.org/incoming/nikolai-gdb-120902.txt The full db_stat output is located at: ftp://ftp.openldap.org/incoming/nikolai-dbstat-120902.txt Our DB_CONFIG: # One 512MB cache set_cachesize 0 536870912 1 # Transaction Log settings set_lg_regionmax 1048576 set_lg_max 10485760 set_lg_bsize 2097152 set_flags DB_LOG_AUTOREMOVE # Increase lock maximums set_lk_max_locks 2000 set_lk_max_lockers 2000 set_lk_max_objects 2000 Our slapd.conf on our replicas: # Load the following schema files include /usr/local/etc/openldap/schema/core.schema include /usr/local/etc/openldap/schema/cosine.schema include /usr/local/etc/openldap/schema/nis.schema include /usr/local/etc/openldap/schema/inetorgperson.schema include /usr/local/etc/openldap/schema/misc.schema include /usr/local/etc/openldap/schema/mirapoint.schema include /usr/local/etc/openldap/schema/smp.schema # Runtime settings for slapd pidfile /var/run/openldap/slapd.pid argsfile /var/run/openldap/slapd.args loglevel none # TLS security options for slapd. TLSCipherSuite HIGH TLSCACertificateFile /usr/local/etc/openldap/tls/ca-cert.pem TLSCertificateFile /usr/local/etc/openldap/tls/server-cert.pem TLSCertificateKeyFile /usr/local/etc/openldap/tls/server-key.pem # This option configures one or more hashes to be used in generation # of user passwords stored in the userPassword attribute during # processing of LDAP Password Modify Extended Operations (RFC 3062). password-hash {SSHA} # Load dynamic backend modules: modulepath /usr/local/libexec/openldap moduleload back_bdb moduleload back_monitor # Do not limit size or time of requests. sizelimit unlimited timelimit unlimited # Require authentication prior to directory operations require authc ############################################################################### # BDB Database Definitions # # The following configuration directives relate to bdb database definitions ############################################################################### # The remaining configuration directives relate to bdb database definitions database bdb suffix "o=top" rootdn "cn=root,o=top" # Cleartext passwords, especially for the rootdn, should # be avoid. See slappasswd(8) and slapd.conf(5) for details. rootpw {SSHA}********** # The database directory must exist prior to running slapd and # should only be accessible by the slapd and slap tools. directory /var/db/openldap-data # Indices to maintain index cn eq,sub,pres index entryUUID eq index entryCSN eq index mail eq,sub,pres index mailHost eq index mailLocalAddress eq,sub,pres index miDomainName eq,sub index miLoginId eq,pres index mpMailHost eq index mpMailUserType eq index mpSystemRole eq index objectClass eq,pres index uid eq,pres # Specify the number of entries which should be held in memory cachesize 200000 # Set transactional checkpoint checkpoint 512 60 ############################################################################### # LDAP Sync Replication # # A unique replica id number is required for each replication client ############################################################################### # LDAP sync replication settings syncrepl rid=36 provider=ldaps://ldapmaster/ type=refreshAndPersist retry=30,+ searchbase="o=top" filter="(objectClass=*)" scope=sub attrs="*" sizelimit=unlimited timelimit=unlimited schemachecking=off bindmethod=simple binddn="cn=replica,ou=users,ou=directory,o=top" credentials=********** # Where to refer ldap updates to updateref ldaps://ldapmaster/ ############################################################################### # LDAP Statistics # # The OpenLDAP server can be configured to provide real time performance # statistics through the monitor branch. ############################################################################### # Enable the statistics monitoring database database monitor # Allow access to monitoring user only access to dn.subtree="cn=monitor" by dn.exact="cn=monitor,ou=users,ou=directory,o=top" read by * none Sincerely, Nikolai Schupbach

1 0

Re: (ITS#7364) mdb: clean up POSIX semaphores on environment close.
by h.b.furuseth＠usit.uio.no 01 Sep '12

01 Sep '12

Reopening this. This is worse with a database with is intended to be used by different users (A and B): A opens the DB and creates the semaphores with e.g. mode 0660. B opens it, A closes, B closes - and fails sem_unlink() which only A and root can do. Next, B (or C) fails mdb_env_open() because sem_unlink() fails again. The work-around I can think of is a "multi-uid" mode which instead resets the semaphore with sem_post() if sem_getvalue() returns 0. I don't know how ugly that is considered to be. Could ask comp.programming.unix, or check what Berkeley DB does. This mode should use mode 0666 for the semaphores (temporarily setting umask 0, yuck), or it should not sem_unlink() since next user may create the semaphores with a group which gives the wrong users access. Or root may give it root-only access, as in the original report. If not unlinking, we need a special "remove database + semaphores" API call. Or use some other sync primitive, like file locks. The Linux manual says these work over NFS though, which sounds like they must be rather slow. flock() does not, but this time mdb would need to seek first. I don't know which sync variants BSD offers. Other matters with the current implementation - I'll patch these: mdb_env_excl_lock() need not retry getting a non-exclusive lock when closing. mdb_env_close() can pass *excl = -1 to tell it not to. mdb_env_setup_locks() can sem_unlink both semaphores before doing anything else, so that reopening a database as root will clean up. Drop the error checks of sem_unlink (so both get called), instead use O_EXCL in sem_open(,O_CREAT,,). Unless I'm missing something, the error checks just work like an emulation of O_EXCL anyway.

1 0

Re: (ITS#7363) libmdb should use POSIX semaphores on non-apple BSD systems too.
by h.b.furuseth＠usit.uio.no 01 Sep '12

01 Sep '12

Chris Mikkelson writes: > The *BSD systems define PTHREAD_PROCESS_SHARED but do > not implement sharable pthread mutexes. The flag name > is part of the API, but implementation of the behavior > it requests is optional. Yes, sorry about that. How about something like a "__BSD" preprocessor symbol to replace "BSD"? There surely is one. It'll start with an underscore followed by either another underscore or an uppercase letter. -- Hallvard

1 0

Re: (ITS#7377) Poor libmdb error handling
by Hallvard B Furuseth 01 Sep '12

01 Sep '12

Howard Chu wrote: >h.b.furuseth(a)usit.uio.no wrote: >> Maybe add a flag to request error paranoia, like setting >> PTHREAD_MUTEX_ERRORCHECK. Trying to catch if a current mdb >> process has lost is locks. (The fcntl F_SETLK is lost if the >> process closes another file descriptor to the same file, >> e.g. after some other module in the process opened and closed the >> mdb lock file.) Catching a change to the pid from getpid(). > > That's really not interesting to me. No other module has any business > opening and reading our lock file. I skipped a step there. The other module could also be using mdb. "other module" = "a program component which the module using this MDB_env does not know about". They may even be linking different libmdb.so's, so the 'libmdb's can't notify each other via static vars. There are a few other use cases like a monolith program which also can backup a directory - which the user could point at the mdb dir. Anyway, a CAVEATS section in the doc is the obvious fix. > We haven't bothered with this for alock either, and > it hasn't been a problem. Note that this ITS is about libmdb, not back-mdb. >> The fork() restriction needs to be documented, as does "don't open >> or close the same database twice in the same process". > > Go ahead. Hmm. I suggested a too pessimistic caveats section once, I can dig that up and try again.

1 0

Re: (ITS#7377) Poor libmdb error handling
by hyc＠symas.com 01 Sep '12

01 Sep '12

h.b.furuseth(a)usit.uio.no wrote: > Full_Name: Hallvard B Furuseth > Version: 2.4.32 > OS: Linux x86_64 > URL: > Submission from: (NULL) (193.69.163.163) > Submitted by: hallvard > > > libmdb omits necessary checks for system call errors, and does > not always clean up after the errors it does detect. > > Also mdb could catch more user errors - including "cannot happen" > mutex errors from mutex lock/unlock, since some threading user > errors are be non-obvious from the doc. > > At least sem_wait() needs an EINTR loop. POSIX says "The > sem_wait() function is interruptible by the delivery of a signal." > The Linux manpage adds that a signal handler always interrupts it > regardless of sigaction SA_RESTART. I don't know about BSD. > > Some error checks first need updates to the Windows emulation, or > will look cleaner that way. E.g. pthread_key_create(). > > I'll fix mdb_env_open()'s error cleanup, at least. > (Hyc, that moves cleanup out of mdb_env_setup_locks(). Any > preference for using ~25 exit points 'return <ErrCode() or rc>' > vs. ~25 'goto <failure label>' which does the same?) In general I prefer that functions have only one exit point, so multiple gotos is preferable. > Maybe add a flag to request error paranoia, like setting > PTHREAD_MUTEX_ERRORCHECK. Trying to catch if a current mdb > process has lost is locks. (The fcntl F_SETLK is lost if the > process closes another file descriptor to the same file, > e.g. after some other module in the process opened and closed the > mdb lock file.) Catching a change to the pid from getpid(). That's really not interesting to me. No other module has any business opening and reading our lock file. We haven't bothered with this for alock either, and it hasn't been a problem. Change of pid... not worth bothering. Just add a note to the docs. > Regarding the pid: Repeating getpid() - in mdb_txn_renew0() and > mdb_env_close() - is useless since mdb cannot survive fork() > anyway. fork drops fcntl locks. Might as well store the pid (or > just some unique "environment id") in MDB_env and use that. Agreed. I had a patch here that does just that, seems it got lost in some other edits. > OTOH > some strategically placed getpid()s can then catch the user error > of having forked, and set e.g. MDB_FATAL_ERROR. Not really interesting. > The fork() restriction needs to be documented, as does "don't open > or close the same database twice in the same process". Go ahead. > mdb_env_open() should catch being called twice. Except, should > it try to allow a 2nd call if 1st mdb_env_open() failed? It can > easily do that if 1st attempt failed before reading params from > the DB files, but fail if the 1st open got that far. Called twice on the same env handle? Again, not interesting. As a rule, we should of course check for failure return codes from normal operation. We should not add extensive checks for programmer error/misuse of API. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7377) Poor libmdb error handling
by openldap-its＠OpenLDAP.org 01 Sep '12

01 Sep '12

I messed up last reply headers somehow. Trying again. I wrote: > Also mdb could catch more user errors - including "cannot happen" > mutex errors from mutex lock/unlock, since some threading user > errors are be non-obvious from the doc. That is, "cannot happen" when there is no user error. > The fork() restriction needs to be documented, as does "don't > open or close the same database twice in the same process". Correction, "don't have the same database open twice at the same time in the same process".

1 0

Re: (ITS#7377) Poor libmdb error handling
by openldap-its＠OpenLDAP.org 01 Sep '12

01 Sep '12

I wrote: > Also mdb could catch more user errors - including "cannot happen" > mutex errors from mutex lock/unlock, since some threading user > errors are be non-obvious from the doc. That is, "cannot happen" when there is no user error. > The fork() restriction needs to be documented, as does "don't > open or close the same database twice in the same process". Correction, "don't have the same database open twice at the same time in the same process".

1 0

(ITS#7377) Poor libmdb error handling
by h.b.furuseth＠usit.uio.no 01 Sep '12

01 Sep '12

Full_Name: Hallvard B Furuseth Version: 2.4.32 OS: Linux x86_64 URL: Submission from: (NULL) (193.69.163.163) Submitted by: hallvard libmdb omits necessary checks for system call errors, and does not always clean up after the errors it does detect. Also mdb could catch more user errors - including "cannot happen" mutex errors from mutex lock/unlock, since some threading user errors are be non-obvious from the doc. At least sem_wait() needs an EINTR loop. POSIX says "The sem_wait() function is interruptible by the delivery of a signal." The Linux manpage adds that a signal handler always interrupts it regardless of sigaction SA_RESTART. I don't know about BSD. Some error checks first need updates to the Windows emulation, or will look cleaner that way. E.g. pthread_key_create(). I'll fix mdb_env_open()'s error cleanup, at least. (Hyc, that moves cleanup out of mdb_env_setup_locks(). Any preference for using ~25 exit points 'return <ErrCode() or rc>' vs. ~25 'goto <failure label>' which does the same?) Maybe add a flag to request error paranoia, like setting PTHREAD_MUTEX_ERRORCHECK. Trying to catch if a current mdb process has lost is locks. (The fcntl F_SETLK is lost if the process closes another file descriptor to the same file, e.g. after some other module in the process opened and closed the mdb lock file.) Catching a change to the pid from getpid(). Regarding the pid: Repeating getpid() - in mdb_txn_renew0() and mdb_env_close() - is useless since mdb cannot survive fork() anyway. fork drops fcntl locks. Might as well store the pid (or just some unique "environment id") in MDB_env and use that. OTOH some strategically placed getpid()s can then catch the user error of having forked, and set e.g. MDB_FATAL_ERROR. The fork() restriction needs to be documented, as does "don't open or close the same database twice in the same process". mdb_env_open() should catch being called twice. Except, should it try to allow a 2nd call if 1st mdb_env_open() failed? It can easily do that if 1st attempt failed before reading params from the DB files, but fail if the 1st open got that far.

1 0

Re: (ITS#7375) slapo-dynlist is very slow
by hyc＠symas.com 31 Aug '12

31 Aug '12

rouazana(a)linagora.com wrote: > Note that using "sortvals uniqueMember" configuration allows dynlist to > work in a correct time (about 3s for a base search on the group). > > So I don't know I you want this issue to be fixed or not. Closing this ITS. > > Regards, > Raphaël Ouazana. > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7375) slapo-dynlist is very slow
by rouazana＠linagora.com 31 Aug '12

31 Aug '12

Note that using "sortvals uniqueMember" configuration allows dynlist to work in a correct time (about 3s for a base search on the group). So I don't know I you want this issue to be fixed or not. Regards, Raphaël Ouazana.

1 0

(ITS#7249) slapd segfault with memberof overlay on frontend db
by c.phan＠active-system.com 31 Aug '12

31 Aug '12

Full_Name: Chinh Phan Version: 2.4.32 OS: Fedora 16 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (84.96.74.38) Hello, I'm still having a segfault error even with the fix suggested by Howard Chu (http://www.openldap.org/lists/openldap-bugs/201204/msg00171.html): slapd[2812]: segfault at 7f8d0b7fffd8 ip 00000000004377b0 sp 00007f8d0b7fffe0 error 6 in slapd[400000+1c6000] Does anyone know if this issue has been addressed or is planned to be fixed in the next release? Appreciate your feedback. chinh

1 0

(ITS#7375) slapo-dynlist is very slow
by raphael.ouazana＠linagora.com 31 Aug '12

31 Aug '12

Full_Name: Raphael Ouazana Version: 2.4.31 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (88.173.78.196) Hi, I'm using dynlist to generate a dynamic group with ~50k users. The LDAP request written in the memberURL attribute takes about 8 seconds to execute. But a base search request on the group takes more than 2 minutes to execute. So I think dynlist should be optimized. Regards, Raphaël Ouazana.

1 0

(ITS#7374) [PATCH] MozNSS: better file name matching for hashed CA certificate directory
by jvcelak＠redhat.com 29 Aug '12

29 Aug '12

Full_Name: Jan Vcelak Version: git master OS: Linux URL: ftp://ftp.openldap.org/incoming/jvcelak-20120829-moznss-beter-file-name-mat… Submission from: (NULL) (209.132.186.34) This change concerns Mozilla NSS crypto backend: CA certificate files in OpenSSL compatible CACERTDIR were loaded if the file extension was '.0'. However the file name should be 8 letters long certificate hash of the certificate subject name, followed by a numeric suffix which is used to differentiate between two certificates with the same subject name. With this patch, certificate file names are matched correctly (using regular expressions). The attached file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch(es) were developed by Red Hat. Red Hat has not assigned rights and/or interest in this work to any party. I, Jan Vcelak am authorized by Red Hat, my employer, to release this work under the following terms. Red Hat hereby place the following modifications to OpenLDAP Software (and only these modifications) into the public domain. Hence, these modifications may be freely used and/or redistributed for any purpose with or without attribution and/or other notice.

1 0

Re: (ITS#7363) libmdb should use POSIX semaphores on non-apple BSD systems too.
by cmikk＠qwest.net 29 Aug '12

29 Aug '12

On Wed, Aug 29, 2012 at 08:07:31AM +0200, Hallvard Breien Furuseth wrote: > We can #ifndef PTHREAD_PROCESS_SHARED instead of BSD, unless someone > #define it without supporting it. Not for MDB_FDATASYNC though. The *BSD systems define PTHREAD_PROCESS_SHARED but do not implement sharable pthread mutexes. The flag name is part of the API, but implementation of the behavior it requests is optional. -- Chris Mikkelson | Quidquid latine dictum sit, altum viditur cmikk(a)qwest.net |

1 0

Re: (ITS#7363) libmdb should use POSIX semaphores on non-apple BSD systems too.
by h.b.furuseth＠usit.uio.no 28 Aug '12

28 Aug '12

Howard Chu writes: > PTHREAD_PROCESS_SHARED is defined just about everywhere, that's not a useful > test. (You only discover at runtime that the mutex creation/init failed...) Argh. Oh well, we should only do the namespace cleanup, then. __BSD__ symbols etc. BTW, I think the lock file should contain a field indicating the sync primitive type, so different compilations of MDB won't use different sync primitives. Which maybe also means we might as well pick the primitive at runtime (try PTHREAD_PROCESS_SHARED first, then sem_open). -- Hallvard

1 0

Re: (ITS#7363) libmdb should use POSIX semaphores on non-apple BSD systems too.
by hyc＠symas.com 28 Aug '12

28 Aug '12

h.b.furuseth(a)usit.uio.no wrote: > We can #ifndef PTHREAD_PROCESS_SHARED instead of BSD, unless someone > #define it without supporting it. Not for MDB_FDATASYNC though. > PTHREAD_PROCESS_SHARED is defined just about everywhere, that's not a useful test. (You only discover at runtime that the mutex creation/init failed...) -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs