- openldap-bugs - openldap.org

Re: (ITS#7713) Segmentation fault if the pagesize of the Operating system is not equal to 4096.
by hyc＠symas.com 01 Oct '13

01 Oct '13

sumantk2(a)linux.vnet.ibm.com wrote: > Full_Name: sumanth k > Version: 2.4.35 to any recent version with mdb support > OS: Linux - ppc64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (122.248.161.59) > > > The pagesize of Linux for x86 and s390x architecture is 4096. Whereas in Powerpc > <ppc64> , the pagesize is by default 65536. So when the pagesize is not equal to > 4096, the segmentation fault occurs. So tried to compile the powerpc64 kernel > with page size of 4096 and the problem disappears and runs smoothly. But by > default the powerpc64 architecture runs with 65536 pagesize . So there is some > problem in mdb_env_open2() function in mdb.c when the pagesize is not equal to > 4096. Thanks for the report. I believe you should be able to instead change the definition of MDB_PAGESIZE to 65536, instead of forcing your machine to use 4096 byte pages. There are other problems though; we use an unsigned short for page offsets. I'm not sure the assert that you tripped will succeed in this case. > > These are my observations : > > > Compiled the source with -O0 optimization . > /home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 -f > /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d > 0x4105 > > ./scripts/test000-rootdse: line 31: 19059 Aborted (core dumped) > $SLAPD -f $CONF1 -h $URI1 -d $LVL $TIMING > $LOG1 2>&1 > > Core file : > bash-4.2# cat .gdbinit > b mdb_db_open > b mdb_env_open > b mdb_env_open2 > b mdb_txn_begin > b mdb_txn_renew0 > b mdb_dbi_open > b mdb_cursor_init > b mdb_cursor_set > b mdb_page_search > b mdb_page_get > b mdb_page_search_root > r -s0 -f /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h > ldap://localhost:9011/ -d 0x4105 > > > # Core was generated by `/home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 > -f /home/openldap-2.4.36'. > Program terminated with signal 6, Aborted. > #0 0x00001fffff7adb70 in .raise () from /lib64/libc.so.6 > Missing separate debuginfos, use: debuginfo-install > cyrus-sasl-lib-2.1.26-8.mcp8_0.2.ppc64 cyrus-sasl-md5-2.1.26-8.mcp8_0.2.ppc64 > glibc-2.17-4.mcp8_0.6.ppc64 keyutils-libs-1.5.5-4.mcp8_0.2.ppc64 > krb5-libs-1.11.3-1.mcp8_0.2.ppc64 libcom_err-1.42.7-2.mcp8_0.1.ppc64 > libdb-5.3.21-8.mcp8_0.4.ppc64 libselinux-2.1.13-15.mcp8_0.3.ppc64 > nss-softokn-freebl-3.14.3-1.mcp8_0.2.ppc64 openssl-libs-1.0.1e-4.mcp8_0.1.ppc64 > pcre-8.32-7.mcp8_0.2.ppc64 zlib-1.2.7-10.mcp8_0.1.ppc64 > (gdb) where > #0 0x00001fffff7adb70 in .raise () from /lib64/libc.so.6 > #1 0x00001fffff7afb64 in .abort () from /lib64/libc.so.6 > #2 0x00001fffff7a455c in .__assert_fail_base () from /lib64/libc.so.6 > #3 0x00001fffff7a464c in .__assert_fail () from /lib64/libc.so.6 > #4 0x00000000101498cc in mdb_node_add (mc=0x3fffe89a7d40, indx=0, > key=0x3fffe89a7d20, data=0x3fffe89a7d30, pgno=0, flags=2) at > ./../../../libraries/liblmdb/mdb.c:6160 > #5 0x000000001014882c in mdb_cursor_put (mc=0x3fffe89a7d40, key=0x3fffe89a7d20, > data=0x3fffe89a7d30, flags=2) at ./../../../libraries/liblmdb/mdb.c:5877 > #6 0x00000000101516a8 in mdb_dbi_open (txn=0x1000f675980, name=0x1027b9c8 > "ad2i", flags=262152, dbi=0x1fffff0800a0) at > ./../../../libraries/liblmdb/mdb.c:7902 > #7 0x0000000010139cd4 in mdb_db_open (be=0x1000f4f83a0, cr=0x3fffe89a81c0) at > init.c:207 > #8 0x0000000010050e34 in backend_startup_one (be=0x1000f4f83a0, > cr=0x3fffe89a81c0) at backend.c:224 > #9 0x0000000010051588 in backend_startup (be=0x1000f4f83a0) at backend.c:325 > #10 0x0000000010089a7c in slap_startup (be=0x0) at init.c:219 > #11 0x000000001000a9c8 in main (argc=8, argv=0x3fffe89a8958) at main.c:991 > > > > Here is the error message : > > #/home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 -f > /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d > 0x4105 > (...some messages...) > 524a2fe1 mdb_db_open: database "o=OpenLDAP Project,l=Internet": > dbenv_open(/home/openldap-2.4.36/tests/testrun/db.1.a). > slapd: ./../../../libraries/liblmdb/mdb.c:6160: mdb_node_add: Assertion > `mp->mp_pb.pb.pb_upper >= mp->mp_pb.pb.pb_lower' failed. < === fails in assert() > ; > Aborted (core dumped) > > > > Some of my observation : > > in this file libraries/liblmdb/mdb.c > in X86 : rc = mdb_cursor_set(&mc, &key, &data, MDB_SET, &exact); the value of > rc = MD_SUCCESS, but for ppc64 it is MDB_NOTFOUND. This is due to the fact that > md_root != 2 > > The value of md_root=2 for env->me_metas[1]->mm_dbs in X86 , but some huge value > in ppc64. The value of md_pad = 4096 in x86 and 65536 in ppc64 . > > in x86: > > Breakpoint 2, mdb_txn_begin (env=0x9c5050, parent=0x0, flags=0, > ret=0x7fffffffdf98) at ./../../../libraries/liblmdb/mdb.c:2219 > 2219 int rc, size, tsize = sizeof(MDB_txn); > (gdb) p env->me_metas[1]->mm_dbs > $1 = {{md_pad = 4096, md_flags = 8, md_depth = 0, md_branch_pages = 0, > md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = > 18446744073709551615}, {md_pad = 0, md_flags = 0, > md_depth = 1, md_branch_pages = 0, md_leaf_pages = 1, md_overflow_pages = 0, > md_entries = 4, md_root = 2}} > (gdb) p env->me_metas[0]->mm_dbs > $2 = {{md_pad = 4096, md_flags = 8, md_depth = 0, md_branch_pages = 0, > md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = > 18446744073709551615}, {md_pad = 0, md_flags = 0, > md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, > md_entries = 0, md_root = 18446744073709551615}} > > > in ppc64: > > Breakpoint 2, mdb_txn_begin (env=0x10493ef0, parent=0x0, flags=0, > ret=0x3fffffffe590) at ./../../../libraries/liblmdb/mdb.c:2219 > 2219 int rc, size, tsize = sizeof(MDB_txn); > (gdb) p env->me_metas[0]->mm_dbs > $1 = {{md_pad = 65536, md_flags = 8, md_depth = 0, md_branch_pages = 0, > md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = > 18446744073709551615}, {md_pad = 0, md_flags = 0, > md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, > md_entries = 0, md_root = 18446744073709551615}} > (gdb) p env->me_metas[1]->mm_dbs > $2 = {{md_pad = 65536, md_flags = 8, md_depth = 0, md_branch_pages = 0, > md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = > 18446744073709551615}, {md_pad = 0, md_flags = 0, > md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, > md_entries = 0, md_root = 18446744073709551615}} > > > > > # From further investigation , the value of env->me_metas[1] is initialized in > mdb_env_open2() at : > > p = (MDB_page *)env->me_map; > env->me_metas[0] = METADATA(p); > env->me_metas[1] = (MDB_meta *)((char *)env->me_metas[0] + meta.mm_psize); > > Here the meta.mm_psize in ppc64 is 65536 , hence there is some problem.. If the > value of meta.mm_psize is 4096 , then everything works fine. > > > As i dont have deep knowledge in openldap ,some help is needed. > > Thank you, > Sumanth K > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7713) Segmentation fault if the pagesize of the Operating system is not equal to 4096.
by sumantk2＠linux.vnet.ibm.com 01 Oct '13

01 Oct '13

Full_Name: sumanth k Version: 2.4.35 to any recent version with mdb support OS: Linux - ppc64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (122.248.161.59) The pagesize of Linux for x86 and s390x architecture is 4096. Whereas in Powerpc <ppc64> , the pagesize is by default 65536. So when the pagesize is not equal to 4096, the segmentation fault occurs. So tried to compile the powerpc64 kernel with page size of 4096 and the problem disappears and runs smoothly. But by default the powerpc64 architecture runs with 65536 pagesize . So there is some problem in mdb_env_open2() function in mdb.c when the pagesize is not equal to 4096. These are my observations : Compiled the source with -O0 optimization . /home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 -f /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d 0x4105 ./scripts/test000-rootdse: line 31: 19059 Aborted (core dumped) $SLAPD -f $CONF1 -h $URI1 -d $LVL $TIMING > $LOG1 2>&1 Core file : bash-4.2# cat .gdbinit b mdb_db_open b mdb_env_open b mdb_env_open2 b mdb_txn_begin b mdb_txn_renew0 b mdb_dbi_open b mdb_cursor_init b mdb_cursor_set b mdb_page_search b mdb_page_get b mdb_page_search_root r -s0 -f /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d 0x4105 # Core was generated by `/home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 -f /home/openldap-2.4.36'. Program terminated with signal 6, Aborted. #0 0x00001fffff7adb70 in .raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.26-8.mcp8_0.2.ppc64 cyrus-sasl-md5-2.1.26-8.mcp8_0.2.ppc64 glibc-2.17-4.mcp8_0.6.ppc64 keyutils-libs-1.5.5-4.mcp8_0.2.ppc64 krb5-libs-1.11.3-1.mcp8_0.2.ppc64 libcom_err-1.42.7-2.mcp8_0.1.ppc64 libdb-5.3.21-8.mcp8_0.4.ppc64 libselinux-2.1.13-15.mcp8_0.3.ppc64 nss-softokn-freebl-3.14.3-1.mcp8_0.2.ppc64 openssl-libs-1.0.1e-4.mcp8_0.1.ppc64 pcre-8.32-7.mcp8_0.2.ppc64 zlib-1.2.7-10.mcp8_0.1.ppc64 (gdb) where #0 0x00001fffff7adb70 in .raise () from /lib64/libc.so.6 #1 0x00001fffff7afb64 in .abort () from /lib64/libc.so.6 #2 0x00001fffff7a455c in .__assert_fail_base () from /lib64/libc.so.6 #3 0x00001fffff7a464c in .__assert_fail () from /lib64/libc.so.6 #4 0x00000000101498cc in mdb_node_add (mc=0x3fffe89a7d40, indx=0, key=0x3fffe89a7d20, data=0x3fffe89a7d30, pgno=0, flags=2) at ./../../../libraries/liblmdb/mdb.c:6160 #5 0x000000001014882c in mdb_cursor_put (mc=0x3fffe89a7d40, key=0x3fffe89a7d20, data=0x3fffe89a7d30, flags=2) at ./../../../libraries/liblmdb/mdb.c:5877 #6 0x00000000101516a8 in mdb_dbi_open (txn=0x1000f675980, name=0x1027b9c8 "ad2i", flags=262152, dbi=0x1fffff0800a0) at ./../../../libraries/liblmdb/mdb.c:7902 #7 0x0000000010139cd4 in mdb_db_open (be=0x1000f4f83a0, cr=0x3fffe89a81c0) at init.c:207 #8 0x0000000010050e34 in backend_startup_one (be=0x1000f4f83a0, cr=0x3fffe89a81c0) at backend.c:224 #9 0x0000000010051588 in backend_startup (be=0x1000f4f83a0) at backend.c:325 #10 0x0000000010089a7c in slap_startup (be=0x0) at init.c:219 #11 0x000000001000a9c8 in main (argc=8, argv=0x3fffe89a8958) at main.c:991 Here is the error message : #/home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 -f /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d 0x4105 (...some messages...) 524a2fe1 mdb_db_open: database "o=OpenLDAP Project,l=Internet": dbenv_open(/home/openldap-2.4.36/tests/testrun/db.1.a). slapd: ./../../../libraries/liblmdb/mdb.c:6160: mdb_node_add: Assertion `mp->mp_pb.pb.pb_upper >= mp->mp_pb.pb.pb_lower' failed. < === fails in assert() ; Aborted (core dumped) Some of my observation : in this file libraries/liblmdb/mdb.c in X86 : rc = mdb_cursor_set(&mc, &key, &data, MDB_SET, &exact); the value of rc = MD_SUCCESS, but for ppc64 it is MDB_NOTFOUND. This is due to the fact that md_root != 2 The value of md_root=2 for env->me_metas[1]->mm_dbs in X86 , but some huge value in ppc64. The value of md_pad = 4096 in x86 and 65536 in ppc64 . in x86: Breakpoint 2, mdb_txn_begin (env=0x9c5050, parent=0x0, flags=0, ret=0x7fffffffdf98) at ./../../../libraries/liblmdb/mdb.c:2219 2219 int rc, size, tsize = sizeof(MDB_txn); (gdb) p env->me_metas[1]->mm_dbs $1 = {{md_pad = 4096, md_flags = 8, md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = 18446744073709551615}, {md_pad = 0, md_flags = 0, md_depth = 1, md_branch_pages = 0, md_leaf_pages = 1, md_overflow_pages = 0, md_entries = 4, md_root = 2}} (gdb) p env->me_metas[0]->mm_dbs $2 = {{md_pad = 4096, md_flags = 8, md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = 18446744073709551615}, {md_pad = 0, md_flags = 0, md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = 18446744073709551615}} in ppc64: Breakpoint 2, mdb_txn_begin (env=0x10493ef0, parent=0x0, flags=0, ret=0x3fffffffe590) at ./../../../libraries/liblmdb/mdb.c:2219 2219 int rc, size, tsize = sizeof(MDB_txn); (gdb) p env->me_metas[0]->mm_dbs $1 = {{md_pad = 65536, md_flags = 8, md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = 18446744073709551615}, {md_pad = 0, md_flags = 0, md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = 18446744073709551615}} (gdb) p env->me_metas[1]->mm_dbs $2 = {{md_pad = 65536, md_flags = 8, md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = 18446744073709551615}, {md_pad = 0, md_flags = 0, md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = 18446744073709551615}} # From further investigation , the value of env->me_metas[1] is initialized in mdb_env_open2() at : p = (MDB_page *)env->me_map; env->me_metas[0] = METADATA(p); env->me_metas[1] = (MDB_meta *)((char *)env->me_metas[0] + meta.mm_psize); Here the meta.mm_psize in ppc64 is 65536 , hence there is some problem.. If the value of meta.mm_psize is 4096 , then everything works fine. As i dont have deep knowledge in openldap ,some help is needed. Thank you, Sumanth K

1 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by hyc＠symas.com 01 Oct '13

01 Oct '13

michael(a)stroeder.com wrote: > Full_Name: > Version: 2.4.36 > OS: > URL: > Submission from: (NULL) (2001:8d8:1fe:7:d6be:d9ff:fe09:1b7c) > > > Configuration: > - MMR with 3 nodes and normal syncrepl. > - slapo-memberof used to populate/update attribute 'memberOf' in user entries. > > If a group entry is modified and slapo-memberof updates the attribute 'memberOf' > in the member's entry the contextCSN value of the server where the LDAP request > was sent to is not updated on the other MMR replicas. I don't understand this sentence, can you rephrase? Also see ITS#6915. > > All changes are correctly done though. > > After restart contextCSN values are all the same on all MMR replicas. > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7712) Lock ordering problem in libldap/abandon.c
by ebackes＠symas.com 01 Oct '13

01 Oct '13

Full_Name: Emily Backes Version: RE24/HEAD OS: any URL: Submission from: (NULL) (98.155.30.248) libldap has a lock ordering problem in do_abandon in libldap/abandon.c; do_abandon always runs in locked req_mutex context, but one of the cases acquires conn_mutex for the ldap_free_connection() call. The function is always called with that mutex held, but has several non-local exits and other difficulties with refactoring the problem away, so the clearest solutions seems to be dropping and retrieving the req_mutex while messing with the conn_mutex. Patch incoming.

1 0

Re: (ITS#7711) Cannot include the radius.schema
by quanah＠zimbra.com 27 Sep '13

27 Sep '13

--On Friday, September 27, 2013 8:02 AM +0000 psam1219(a)hotmail.com wrote: > Full_Name: Don > Version: slapd 2.4.23 > OS: RHEL 6.3 x64 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (58.213.47.66) The ITS system is for reporting bugs. It is not for asking questions about how to use the software. Usage questions should be asked via the openldap-technical(a)openldap.org mailing list. This ITS will be closed. --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra Software, LLC -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7711) Cannot include the radius.schema
by psam1219＠hotmail.com 27 Sep '13

27 Sep '13

Full_Name: Don Version: slapd 2.4.23 OS: RHEL 6.3 x64 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (58.213.47.66) There is a slapd server ,and there is a freeradius server . I was following these steps . A.Copy example schema file (openldap.schema)from freeradius server. scp RADIUSSERVER:/usr/share/doc/freeradius-2.1.12/examples/openldap.schema ./freeradius.schema B.Include the file to slapd.conf . include /etc/openldap/schema/freeradius.schema C.restart slapd server service slapd restart D.When I tried to add the ldif file to create entries , this error come out . [root@RAC1 test]# ldapadd -f test.ldif -x -D "cn=Manager,dc=txldomain,dc=com" -w taxili888 adding new entry "cn=students,ou=profiles,ou=radius,dc=txldomain,dc=com" ldap_add: Invalid syntax (21) additional info: objectclass: value #0 invalid per syntax E.Then I checked the ObjectClasses from a Web UI £šphpLDAPadmin£©, I cann't see radiusprofile on the list table . I think the problem is failure of adding radius's scheme ,but I don't know why and how to solve this . Thank you for your time to check this issue .

1 0

(ITS#7710) contextCSN values not updated by internal non-replicated ops
by michael＠stroeder.com 26 Sep '13

26 Sep '13

Full_Name: Version: 2.4.36 OS: URL: Submission from: (NULL) (2001:8d8:1fe:7:d6be:d9ff:fe09:1b7c) Configuration: - MMR with 3 nodes and normal syncrepl. - slapo-memberof used to populate/update attribute 'memberOf' in user entries. If a group entry is modified and slapo-memberof updates the attribute 'memberOf' in the member's entry the contextCSN value of the server where the LDAP request was sent to is not updated on the other MMR replicas. All changes are correctly done though. After restart contextCSN values are all the same on all MMR replicas.

1 0

Re: (ITS#7709) mdb back-end segfaults sporadically with paged searches
by quanah＠zimbra.com 25 Sep '13

25 Sep '13

--On Tuesday, September 24, 2013 8:25 PM +0000 openldap(a)semyon.org wrote: > Full_Name: Semyon Chaichenets > Version: 2.4.36 > OS: Linux 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 > x86_64 x86_64 x86_64 GNU/Linux URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (129.128.11.113) Please stop repeatedly creating new ITSes for your issue. This is the 3rd one you've created. Thanks. --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra Software, LLC -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7709) mdb back-end segfaults sporadically with paged searches
by openldap＠semyon.org 24 Sep '13

24 Sep '13

Full_Name: Semyon Chaichenets Version: 2.4.36 OS: Linux 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.128.11.113) We have an issue whereby a paged-search operation causes slapd to segfault. At this point, I can only reproduce the issue on our production data - for which I cannot share coredumps. I am working on isolating the issue but would appreciate any help from openLDAP developers. The issue is triggered by the following search: ldapsearch -H ldap://ldaphost -b 'ou=people,dc=our,dc=org' -x -E pr=126/noprompt '(uid=m*)' We have ~59k entries starting matching uid=m*; the issue occurs unpredictably in the sense that it may or may not happen for filters matching other sets, but it can be reproduced reliably for any given filter/page size combination. On our setup, I could reproduce it for page-sizes 81,84,87,96,112,114,116,126. The issue is specific to back-mdb; backtrace points to dn2id.c: #0 0x00007f6d6a5ea6a1 in mdb_idscopes (op=0x7f695c110df0, isc=0x7f69692ca670) at ../../../../../servers/slapd/back-mdb/dn2id.c:738 #1 0x00007f6d6a5e399b in mdb_search (op=0x7f695c110df0, rs=0x7f69692dba00) at ../../../../../servers/slapd/back-mdb/search.c:747 [..] I would greatly appreciate any help you could give me with this problem. Best, Semyon

1 0

Re: (ITS#7705) mdb back-end segfaults sporadically with paged searches
by openldap＠semyon.org 24 Sep '13

24 Sep '13

On 13-09-23 10:16 PM, Howard Chu wrote: > If you backup the DB with slapcat and reload it on another server with > slapadd, can you still reproduce the fault on the copy? Unfortunately, yes. It breaks trying to retrieve the same record as before, on the same instruction in back_mdb: yesterday: [126393.233615] slapd[19244]: segfault at 7f499f38d000 ip 00007f4da0e9b6a1 sp 00007f499f1fa540 error 6 in back_mdb-2.4.so.2.9.2[7f4da0e80000+34000] today: [517860.953779] slapd[24871]: segfault at 7fdeb50a0000 ip 00007fe2b63ad6a1 sp 00007fdeb4f0d540 error 6 in back_mdb-2.4.so.2.9.2[7fe2b6392000+34000] Semyon

1 0

Re: (ITS#7705) mdb back-end segfaults sporadically with paged searches
by openldap＠semyon.org 24 Sep '13

24 Sep '13

--089e0160bd36360a0704e7231e09 Content-Type: text/plain; charset=UTF-8 I have recloned the database overnight to make sure - the problem persists. Semyon On Mon, Sep 23, 2013 at 4:32 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote: > [..] > Was this database created with OpenLDAP 2.4.36 back-mdb, or did it > originate with a previous release of OpenLDAP? > --089e0160bd36360a0704e7231e09 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra">I have recloned the database ov= ernight to make sure - the problem persists.<br><br></div><div class=3D"gma= il_extra">Semyon<br></div><div class=3D"gmail_extra"><br><div class=3D"gmai= l_quote"> On Mon, Sep 23, 2013 at 4:32 PM, Quanah Gibson-Mount <span dir=3D"ltr"><= <a href=3D"mailto:quanah@zimbra.com" target=3D"_blank">quanah(a)zimbra.com</a= >></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 = 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> [..]<br> Was this database created with OpenLDAP 2.4.36 back-mdb, or did it originat= e with a previous release of OpenLDAP?<br></blockquote></div><br></div></di= v> --089e0160bd36360a0704e7231e09--

1 0

(ITS#7708) mdb back-end segfaults sporadically with paged searches
by openldap＠semyon.org 24 Sep '13

24 Sep '13

Full_Name: Semyon Chaichenets Version: 2.4.36 OS: Linux 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.128.11.113) We have an issue whereby a paged-search operation causes slapd to segfault. At this point, I can only reproduce the issue on our production data - for which I cannot share coredumps. I am working on isolating the issue but would appreciate any help from openLDAP developers. The issue is triggered by the following search: ldapsearch -H ldap://ldaphost -b 'ou=people,dc=our,dc=org' -x -E pr=126/noprompt '(uid=m*)' We have ~59k entries starting matching uid=m*; the issue occurs unpredictably in the sense that it may or may not happen for filters matching other sets, but it can be reproduced reliably for any given filter/page size combination. On our setup, I could reproduce it for page-sizes 81,84,87,96,112,114,116,126. The issue is specific to back-mdb; backtrace points to dn2id.c: #0 0x00007f6d6a5ea6a1 in mdb_idscopes (op=0x7f695c110df0, isc=0x7f69692ca670) at ../../../../../servers/slapd/back-mdb/dn2id.c:738 #1 0x00007f6d6a5e399b in mdb_search (op=0x7f695c110df0, rs=0x7f69692dba00) at ../../../../../servers/slapd/back-mdb/search.c:747 [..] I would greatly appreciate any help you could give me with this problem. Best, Semyon

1 0

Re: (ITS#7707) HDB very slow with _lots_ of aliases
by h.b.furuseth＠usit.uio.no 24 Sep '13

24 Sep '13

Yes. Aliases are a pain. Lots of aliases are not recommended. They defeat your indexes: When you search with alias dereferencing "always" or "search", the server must follow every alias in the search scope to see if it leads to entries which should be returned. This will not be fixed, such a change would be too invasive. -- Hallvard

1 0

Re: (ITS#7707) HDB unable to find record with _lots_ of aliases
by m.innocenti＠cineca.it 24 Sep '13

24 Sep '13

Sorry for the wrong Subject. I have tested MDB and the problem is not present but the query are way to slow to be usable. The first query took 10m30s to finish, the second one 191m31s.

1 0

(ITS#7707) HDB very slow with _lots_ of aliases
by m.innocenti＠cineca.it 24 Sep '13

24 Sep '13

Full_Name: Marco Innocenti Version: 2.4.36 OS: Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (130.186.19.81) Hi, I do administer a system which hosts a slapd instance (2.4.36) which is used by one of our customer which use a tree with tens of thousand of aliases. The backend is HDB. If I do a search with alias dereferencing set to always using "ou=1004,ou=studenti,o=uni,dc=uni,dc=it" as a base I do find the record I'm looking for. vm:~# time ldapsearch -x -H ldap://localhost:8501 -D 'cn=manager,dc=uni,dc=it' -W -b ou=1004,ou=studenti,o=uni,dc=uni,dc=it uid=mario.rossi -a always | grep \# Enter LDAP Password: # extended LDIF # # LDAPv3 # base <ou=1004,ou=studenti,o=uni,dc=uni,dc=it> with scope subtree # filter: uid=mario.rossi # requesting: ALL # # mario.rossi, m, utenti, uni, uni.it # search result # numResponses: 2 # numEntries: 1 real 0m1.607s user 0m0.000s sys 0m0.000s If I do the same search on the parent of the previously used base ("ou=studenti,o=uni,dc=uni,dc=it" vs "ou=1004,ou=studenti,o=uni,dc=uni,dc=it") I do not get any record and no errors are logged. vm:~# time ldapsearch -x -H ldap://localhost:8501 -D cn=manager,dc=uni,dc=it -W -b ou=studenti,o=uni,dc=uni,dc=it uid=mario.rossi -a always Enter LDAP Password: # extended LDIF # # LDAPv3 # base <ou=studenti,o=uni,dc=uni,dc=it> with scope subtree # filter: uid=mario.rossi # requesting: ALL # # search result search: 2 result: 0 Success # numResponses: 1 real 0m5.301s user 0m0.004s sys 0m0.000s

1 0

(ITS#7706) RFE: contrib/slapo-lastbind extension
by marco.pizzoli＠gmail.com 24 Sep '13

24 Sep '13

Full_Name: Marco Pizzoli Version: until 2.4.36 OS: N/A URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (194.11.209.11) This is another RFE for slapo-lastbind I would appreciate being able to prevent some users from having their entry updated/populated with the authTimestamp attribute. I suppose this could be implemented with just the check of a specific attribute (i.e. authTimestampLogging: FALSE) on the user entry. Thanks in advance as usual

1 0

Re: (ITS#7705) mdb back-end segfaults sporadically with paged searches
by hyc＠symas.com 24 Sep '13

24 Sep '13

openldap(a)semyon.org wrote: > Full_Name: Semyon Chaichenets > Version: 2.4.36 > OS: Linux 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (129.128.11.113) > > > We have an issue whereby a paged-search operation causes slapd to segfault. At > this point, I can only reproduce the issue on our production data - for which I > cannot share coredumps. I am working on isolating the issue but would appreciate > any help from openLDAP developers. > > The issue is triggered by the following search: > > ldapsearch -H ldap://ldaphost -b 'ou=people,dc=our,dc=org' -x -E pr=126/noprompt > '(uid=m*)' > > We have ~59k entries starting matching uid=m*; the issue occurs unpredictably in > the sense that it may or may not happen for filters matching other sets, but it > can be reproduced reliably for any given filter/page size combination. On our > setup, I could reproduce it for page-sizes 81,84,87,96,112,114,116,126. > > The issue is specific to back-mdb; backtrace points to dn2id.c: > > #0 0x00007f6d6a5ea6a1 in mdb_idscopes (op=0x7f695c110df0, isc=0x7f69692ca670) > at ../../../../../servers/slapd/back-mdb/dn2id.c:738 > #1 0x00007f6d6a5e399b in mdb_search (op=0x7f695c110df0, rs=0x7f69692dba00) > at ../../../../../servers/slapd/back-mdb/search.c:747 > > [..] > > I would greatly appreciate any help you could give me with this problem. If you backup the DB with slapcat and reload it on another server with slapadd, can you still reproduce the fault on the copy? > > > Best, > Semyon > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7705) mdb back-end segfaults sporadically with paged searches
by quanah＠zimbra.com 24 Sep '13

24 Sep '13

--On Monday, September 23, 2013 10:09 PM +0000 openldap(a)semyon.org wrote: > Full_Name: Semyon Chaichenets > Version: 2.4.36 > OS: Linux 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 > x86_64 x86_64 x86_64 GNU/Linux URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (129.128.11.113) > > > We have an issue whereby a paged-search operation causes slapd to > segfault. At this point, I can only reproduce the issue on our production > data - for which I cannot share coredumps. I am working on isolating the > issue but would appreciate any help from openLDAP developers. Was this database created with OpenLDAP 2.4.36 back-mdb, or did it originate with a previous release of OpenLDAP? --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra Software, LLC -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7705) mdb back-end segfaults sporadically with paged searches
by openldap＠semyon.org 24 Sep '13

24 Sep '13

Full_Name: Semyon Chaichenets Version: 2.4.36 OS: Linux 3.2.0-51-generic #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (129.128.11.113) We have an issue whereby a paged-search operation causes slapd to segfault. At this point, I can only reproduce the issue on our production data - for which I cannot share coredumps. I am working on isolating the issue but would appreciate any help from openLDAP developers. The issue is triggered by the following search: ldapsearch -H ldap://ldaphost -b 'ou=people,dc=our,dc=org' -x -E pr=126/noprompt '(uid=m*)' We have ~59k entries starting matching uid=m*; the issue occurs unpredictably in the sense that it may or may not happen for filters matching other sets, but it can be reproduced reliably for any given filter/page size combination. On our setup, I could reproduce it for page-sizes 81,84,87,96,112,114,116,126. The issue is specific to back-mdb; backtrace points to dn2id.c: #0 0x00007f6d6a5ea6a1 in mdb_idscopes (op=0x7f695c110df0, isc=0x7f69692ca670) at ../../../../../servers/slapd/back-mdb/dn2id.c:738 #1 0x00007f6d6a5e399b in mdb_search (op=0x7f695c110df0, rs=0x7f69692dba00) at ../../../../../servers/slapd/back-mdb/search.c:747 [..] I would greatly appreciate any help you could give me with this problem. Best, Semyon

1 0

Re: (ITS#7704) OpenLDAP-Max connection retry over issue
by quanah＠zimbra.com 23 Sep '13

23 Sep '13

--On Monday, September 23, 2013 7:37 AM +0000 saurabh.4091(a)gmail.com wrote: > Full_Name: Saurabh Sudhir Bade > Version: 0.9.8a > OS: Windows server 2003 > URL: > Submission from: (NULL) (213.42.50.245) > > > Hi, > > We are using OpenLDAP to authenticate users accessing our web service. > The LDAP is configured in TIBCO Activematrix Administrator where the > services are deployed. > We get the below mentioned error quite frequently. > Error in authenticating credentials.Ldap Max connection retry over (3): > localhost:9430: Connection refused: connect > This happens because the connection pool is full and a new connection to > LDAP is unavailable . > Is there any way to increase the connection pool in LDAP? Please help me > out. Please let me know if you need any further information on this issue. Usage and configuration questions should be sent to the openldap-technical(a)openldap.org email list. The ITS system is for reporting bugs. This ITS will be closed. --Quanah -- Quanah Gibson-Mount Lead Engineer Zimbra Software, LLC -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

(ITS#7704) OpenLDAP-Max connection retry over issue
by saurabh.4091＠gmail.com 23 Sep '13

23 Sep '13

Full_Name: Saurabh Sudhir Bade Version: 0.9.8a OS: Windows server 2003 URL: Submission from: (NULL) (213.42.50.245) Hi, We are using OpenLDAP to authenticate users accessing our web service. The LDAP is configured in TIBCO Activematrix Administrator where the services are deployed. We get the below mentioned error quite frequently. Error in authenticating credentials.Ldap Max connection retry over (3): localhost:9430: Connection refused: connect This happens because the connection pool is full and a new connection to LDAP is unavailable . Is there any way to increase the connection pool in LDAP? Please help me out. Please let me know if you need any further information on this issue. Thanks Regards Saurabh

1 0

Re: (ITS#7703) mdb sync() issues vs. ACID
by hyc＠symas.com 22 Sep '13

22 Sep '13

h.b.furuseth(a)usit.uio.no wrote: > Full_Name: Hallvard B Furuseth > Version: LMDB_0.9.8 > OS: > URL: > Submission from: (NULL) (81.191.45.35) > Submitted by: hallvard > > > mdb_env_sync() uses the wrong sync method when syncing a commit > written with a different MDB_WRITEMAP setting in another MDB_env. > > Two processes with MDB_NOMETASYNC, each process doing every 2nd > write txn, will sync each other's meta pages. If they have > different MDB_WRITEMAPs, every meta page gets synced wrongly. > This breaks durability of ACID. Sounds like a doc issue. This can only arise if two separate processes are using different configurations to access the same MDB environment. Most applications will always use identical configurations to access their databases, so this won't occur. > There is a similar problem if a process crashes after writing > the meta page but before sync succeeds, and mdb_env_open() then > resets the lockfile to refer to the unsynced commit. Robust > mutexes will introduce a similar problem without mdb_env_open. > > > I'm not volunteering to figure out how to do this right, e.g. how > do fsync/msync/FlushFileBuffers work on various OSes if the file > descriptor or memory map is read-only, do we need to set a "need > to sync" flag in the lockfile in this case for the first writer > or write txn to obey? > > Another fix: Disable this scenario. Store the MDB_WRITEMAP > setting in the lockfile when resetting it, even with MDB_RDONLY. > Obey that flag rather than the writemap flag to mdb_env_open() > when not resetting the lockfile. However, now a small program > like mdb_stat can have disproportionate effect on another process > which opens the env at the same time. Also nested txns need to > work with MDB_WRITEMAP. > > > For the crash case above and robust mutexes: > > Maybe mdb_env_open() should not modify me_txns->mti_txnid if it > refers to the oldest meta page. That way the possibly unsynced > commit will never be exposed unless the lockfile is removed. > But next write txn must then reset the "hidden" metapage and sync > before proceeding, similar to how mdb_env_write_meta() does at > failure. Otherwise removing the lockfile would expose a meta > page referring to data which may have been overwritten, e.g. by > an mdb_abort()ed commit. > > Another variant would be to sync in mdb_env_open() when resetting > the lockfile, or maybe an MDB_RDONLY env must set a "sync needed" > flag. Nothing to fix in code. Doc this as "don't do this." Nobody currently does it anyway so it will have no impact. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7703) mdb sync() issues vs. ACID
by h.b.furuseth＠usit.uio.no 21 Sep '13

21 Sep '13

Full_Name: Hallvard B Furuseth Version: LMDB_0.9.8 OS: URL: Submission from: (NULL) (81.191.45.35) Submitted by: hallvard mdb_env_sync() uses the wrong sync method when syncing a commit written with a different MDB_WRITEMAP setting in another MDB_env. Two processes with MDB_NOMETASYNC, each process doing every 2nd write txn, will sync each other's meta pages. If they have different MDB_WRITEMAPs, every meta page gets synced wrongly. This breaks durability of ACID. There is a similar problem if a process crashes after writing the meta page but before sync succeeds, and mdb_env_open() then resets the lockfile to refer to the unsynced commit. Robust mutexes will introduce a similar problem without mdb_env_open. I'm not volunteering to figure out how to do this right, e.g. how do fsync/msync/FlushFileBuffers work on various OSes if the file descriptor or memory map is read-only, do we need to set a "need to sync" flag in the lockfile in this case for the first writer or write txn to obey? Another fix: Disable this scenario. Store the MDB_WRITEMAP setting in the lockfile when resetting it, even with MDB_RDONLY. Obey that flag rather than the writemap flag to mdb_env_open() when not resetting the lockfile. However, now a small program like mdb_stat can have disproportionate effect on another process which opens the env at the same time. Also nested txns need to work with MDB_WRITEMAP. For the crash case above and robust mutexes: Maybe mdb_env_open() should not modify me_txns->mti_txnid if it refers to the oldest meta page. That way the possibly unsynced commit will never be exposed unless the lockfile is removed. But next write txn must then reset the "hidden" metapage and sync before proceeding, similar to how mdb_env_write_meta() does at failure. Otherwise removing the lockfile would expose a meta page referring to data which may have been overwritten, e.g. by an mdb_abort()ed commit. Another variant would be to sync in mdb_env_open() when resetting the lockfile, or maybe an MDB_RDONLY env must set a "sync needed" flag.

1 0

(ITS#7702) hdb and mdb derefere aliases differently
by julien.combes＠i-carre.net 20 Sep '13

20 Sep '13

Full_Name: Julien COMBES Version: 2.4.36 OS: debian squeeze URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (212.23.175.188) Hello, With openldap 2.4.36, i found a case where the aliases are deref differently between hdb and mdb. A search with deref aliases on an attribut not indexed (or on "*"), mdb backend returns the entry twice where hdb backend returns the entry once. For example : With a directory like that : --------------------------------------------------------------------- dn: dc=test,dc=com objectClass: top objectClass: dcObject objectClass: organization dc: test o: test dn: ou=a,dc=test,dc=com objectClass: top objectClass: organizationalUnit ou: a dn: ou=b,ou=a,dc=test,dc=com objectClass: top objectClass: organizationalUnit ou: b dn: ou=c,ou=a,dc=test,dc=com objectClass: top objectClass: organizationalUnit ou: c dn: cn=foo,ou=b,ou=a,dc=test,dc=com objectClass: top objectClass: person cn: foo sn: foo dn: cn=bar,ou=c,ou=a,dc=test,dc=com objectClass: top objectClass: alias objectClass: extensibleObject aliasedObjectName: cn=foo,ou=b,ou=a,dc=test,dc=com cn: bar --------------------------------------------------------------------- I have that kind of result : -> Search on mdb, cn not indexed : $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=foo dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=com dn: cn=foo,ou=b,ou=a,dc=test,dc=com $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=* dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=com dn: cn=foo,ou=b,ou=a,dc=test,dc=com -> Search on mdb, cn indexed eq,sub : $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=foo dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=com $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=* dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=com dn: cn=foo,ou=b,ou=a,dc=test,dc=com -> Search on hdb, cn not indexed : $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=foo dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=com $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=* dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=com -> Search on hdb $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=foo dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=commits $ ldapsearch -x -LLL -b "ou=a,dc=test,dc=com" cn=* dn -a always dn: cn=foo,ou=b,ou=a,dc=test,dc=com Regards, Julien COMBES P.S: I have first posted this message as a comment in ITS7577 the 25 Jul 2013. But as the ITS7577 is tagged closed and has no answers since this date, I decide to repost as a new report.

1 0

(ITS#7701) Deletion triggers SEGV on next access of existing cursor
by dw＠botanicus.net 19 Sep '13

19 Sep '13

Full_Name: David Wilson Version: 919a0f5b54dad3acf2f84e7993bb00e7aa098037 OS: Linux URL: http://www.h1.botanicus.net/20130919-lmdb.trace.bz2 Submission from: (NULL) (178.238.153.20) In some circumstances LMDB will segfault in mdb_cursor_get() on a cursor that existed prior to a deletion that just occurred. It is apparently related to the page the cursor points to becoming mutated by the deletion. In LMDB 0.9.8, the issue may manifest as a NULL ptr dereference: #0 mdb_xcursor_init1 (mc=mc@entry=0x1013da0, node=node@entry=0x7ffd76031ac2) at lib/mdb.c:6525 #1 0x00007ffff5209b73 in mdb_cursor_next (mc=0x1013da0, key=0xce39a0, data=data@entry=0xce3660, op=<optimized out>) at lib/mdb.c:5042 #2 0x00007ffff5208a6e in mdb_cursor_get (mc=0x1013da0, key=0xce39a0, data=0xce3660, op=op@entry=MDB_NEXT) at lib/mdb.c:5526 #3 0x00007ffff5202a83 in _cffi_f_mdb_cursor_get (self=<optimized out>, args=<optimized out>) at lmdb/__pycache__/lmdb_cffi.c:714 #4 0x0000000000471a8b in call_function (oparg=<optimized out>, pp_stack=0x7fffffffcf30) at ../Python/ceval.c:4021 #5 PyEval_EvalFrameEx ( Due to F_DUPDATA bit being set as tested on line 5041. Examining the leaf structure reveals a large chunk of ASCII text (which should have been written to the DB), interspersed with some integer values. Versions of LMDB <= 0.9.7 exhibit similar crashes, although in a different place: "assert(mc)" at the start of mdb_cursor_get(). Various MDB versions going back 6 months were tested, and none exhibited any better behaviour. In the attached trace/replay, version c0575825730dd2aab4031100b25d632a3d052447 from April exhibited the bug after only 1993 operations executed from the trace file. Any more recent version requires the entire trace file to execute before triggering the assert or SEGV. http://www.h1.botanicus.net/20130919-lmdb.trace.bz2 is a 33MiB trace file that can be loaded and executed using https://github.com/dw/acid/blob/master/misc/lmdb-replay.c The apparent workaround is to reinitialize any existing cursors should a deletion occur within a transaction, as demonstrated in https://github.com/dw/acid/commit/56f183c71a668e540df58d5137e946616a446c49

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs