March 2023 - openldap-bugs

[Issue 10029] New: slapd crashes when run with unlimited open files

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10029 Issue ID: 10029 Summary: slapd crashes when run with unlimited open files Product: OpenLDAP Version: 2.6.4 Hardware: All OS: Mac OS Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: slapd Assignee: bugs(a)openldap.org Reporter: gray(a)nxg.name Target Milestone: --- To reproduce: % ulimit -n unlimited % $T/openldap-2.6.4/libexec/slapd -d-1 641ee8bc.32f05820 0x1dc760140 @(#) $OpenLDAP: slapd 2.6.4 (Mar 25 2023 12:25:49) $ openldap 641ee8bc.32f39ff8 0x1dc760140 daemon_init: <null> 641ee8bc.32f40588 0x1dc760140 daemon: SLAP_SOCK_INIT: dtblsize=-1 641ee8bc.32f43080 0x1dc760140 ch_calloc of 1 elems of 18446744073709551615 bytes failed Assertion failed: (0), function ch_calloc, file ch_malloc.c, line 107. zsh: abort $T/openldap-2.6.4/libexec/slapd -d-1 This is because `daemon.c` (line 1867) uses the maximum number of open files to set `dtblsize`, which is subsequently used to size an array: 1867 #ifdef HAVE_SYSCONF 1868 dtblsize = sysconf( _SC_OPEN_MAX ); 1869 #elif defined(HAVE_GETDTABLESIZE) 1870 dtblsize = getdtablesize(); 1871 #else /* ! HAVE_SYSCONF && ! HAVE_GETDTABLESIZE */ 1872 dtblsize = FD_SETSIZE; 1873 #endif /* ! HAVE_SYSCONF && ! HAVE_GETDTABLESIZE */ If the maximum number of FDs is unlimited, then sysconf(_SC_OPEN_MAX) returns -1, and the program crashes when it tries to malloc that much memory. I've marked this as OS=macOS because that's what I'm illustrating this on, but the same thing would happen on any OS where the sysconf call returns a negative number for the 'unlimited' case. -- You are receiving this mail because: You are on the CC list for the issue.

1 day, 6 hours

1
2
0 / 0

[Issue 10028] New: crash with pwdMinDelay

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10028 Issue ID: 10028 Summary: crash with pwdMinDelay Product: OpenLDAP Version: 2.5.14 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: overlays Assignee: bugs(a)openldap.org Reporter: hamano(a)osstech.co.jp Target Milestone: --- slapd crash when using pwdMinDelay of ppolicy. The cause is that slap_timestamp() writes to undefined area. It has already been fixed. backtrace: ``` (gdb) bt #0 0x00007f2c70bdbaff in raise () from /lib64/libc.so.6 #1 0x00007f2c70baeea5 in abort () from /lib64/libc.so.6 #2 0x00007f2c70baed79 in __assert_fail_base.cold.0 () from /lib64/libc.so.6 #3 0x00007f2c70bd4456 in __assert_fail () from /lib64/libc.so.6 #4 0x0000000000499fdf in entry_schema_check (op=<optimized out>, op@entry=0x7f2c2a9a9ff0, e=<optimized out>, e@entry=0x7f2c20001d30, oldattrs=<optimized out>, oldattrs@entry=0x7f2c20001d30, manage=0, add=add@entry=0, socp=socp@entry=0x7f2c2a9a99d8, text=0x7f2c2a9a9f30, textbuf=0x7f2c2a9a9b40 "", textlen=256) at ../../../servers/slapd/schema_check.c:89 #5 0x00000000004f5dc1 in mdb_modify_internal (op=<optimized out>, op@entry=0x7f2c2a9a9ff0, tid=tid@entry=0x11dc8c0, modlist=<optimized out>, e=<optimized out>, e@entry=0x7f2c2a9a9ac0, text=text@entry=0x7f2c2a9a9f30, textbuf=textbuf@entry=0x7f2c2a9a9b40 "", textlen=256) at ../../../../servers/slapd/back-mdb/modify.c:419 #6 0x00000000004f6cfa in mdb_modify (op=0x7f2c2a9a9ff0, rs=0x7f2c2a9a9f10) at ../../../../servers/slapd/back-mdb/modify.c:714 #7 0x00000000004d5833 in overlay_op_walk (op=0x7f2c2a9a9ff0, rs=0x7f2c2a9a9f10, which=<optimized out>, oi=0xfc1a00, on=0x0) at ../../../servers/slapd/backover.c:706 #8 over_op_func (op=0x7f2c2a9a9ff0, rs=0x7f2c2a9a9f10, which=op_modify) at ../../../servers/slapd/backover.c:766 #9 0x00007f2c6be56a95 in ppolicy_bind_response (op=<optimized out>, rs=0x7f2c2a9aa730) at ../../../../servers/slapd/overlays/ppolicy.c:1827 #10 0x0000000000475878 in slap_response_play (op=0x7f2c201039a0, rs=0x7f2c2a9aa730) at ../../../servers/slapd/result.c:567 #11 send_ldap_response (op=op@entry=0x7f2c201039a0, rs=rs@entry=0x7f2c2a9aa730) at ../../../servers/slapd/result.c:642 #12 0x0000000000475f2c in slap_send_ldap_result (op=0x7f2c201039a0, rs=0x7f2c2a9aa730) at ../../../servers/slapd/result.c:918 #13 0x000000000052b6cf in mdb_bind (op=0x7f2c201039a0, rs=0x7f2c2a9aa730) at ../../../../servers/slapd/back-mdb/bind.c:148 #14 0x00000000004d5833 in overlay_op_walk (op=0x7f2c201039a0, rs=0x7f2c2a9aa730, which=<optimized out>, oi=0xfc1a00, on=0x0) at ../../../servers/slapd/backover.c:706 #15 over_op_func (op=0x7f2c201039a0, rs=0x7f2c2a9aa730, which=op_bind) at ../../../servers/slapd/backover.c:766 #16 0x00000000004824f2 in fe_op_bind (op=0x7f2c201039a0, rs=0x7f2c2a9aa730) at ../../../servers/slapd/bind.c:383 #17 0x00000000004822ea in do_bind (op=0x7f2c201039a0, rs=0x7f2c2a9aa730) at ../../../servers/slapd/bind.c:206 #18 0x0000000000466847 in connection_operation (ctx=<optimized out>, ctx@entry=0x7f2c2a9aa9b8, arg_v=arg_v@entry=0x7f2c201039a0) at ../../../servers/slapd/connection.c:1113 #19 0x0000000000465ec1 in connection_read_thread (ctx=<optimized out>, argv=0x11) at ../../../servers/slapd/connection.c:1265 #20 0x00007f2c72d5ea22 in ldap_int_thread_pool_wrapper (xpool=<optimized out>) at ../../../libraries/libldap/tpool.c:1053 #21 0x00007f2c70f5b1cf in start_thread () from /lib64/libpthread.so.0 #22 0x00007f2c70bc6e73 in clone () from /lib64/libc.so.6 ``` -- You are receiving this mail because: You are on the CC list for the issue.

2 days, 19 hours

1
1
0 / 0

[Issue 10021] New: Cannot insert data into wiredtiger backend

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10021 Issue ID: 10021 Summary: Cannot insert data into wiredtiger backend Product: OpenLDAP Version: 2.6.4 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: backends Assignee: bugs(a)openldap.org Reporter: jailbird(a)fdf.net Target Milestone: --- I have a test system running OpenLDAP 2.6.4 linked against WiredTiger 11.1.0 running on a RHEL9.1-based system. Running kernel is 6.1.16, filesystem is XFS. back_wt.la was added to cn=module and a simple olcDatabase=wt was created like: dn: olcDatabase=wt objectClass: olcDatabaseConfig objectClass: olcWtConfig olcDatabase: wt olcDbDirectory: /var/lib/ldap olcSuffix: dc=fdf,dc=net olcLimits: {0}dn.base="cn=root,dc=fdf,dc=net" time.soft=unlimited time.hard=u nlimited size.soft=unlimited size.hard=unlimited olcRootDN: cn=root,dc=fdf,dc=net olcWtConfig: create olcDbIndex: objectClass,uid,gidNumber,uidNumber pres,eq olcDbIndex: ou,cn,mail pres,eq,sub structuralObjectClass: olcWtConfig I start slapd and it creates the database files correctly. I then go and try to create the container with a simple .ldif and ldapadd: dn: dc=fdf,dc=net objectClass: dcObject objectClass: organization o: FDF dc: fdf That generates: [1677801597:758327][83158:0x55b4158fb640], file:dn2id.wt, WT_CURSOR.insert: [WT_VERB_DEFAULT][ERROR]: __wt_txn_id_check, 1339: write operations are not supported in read-committed or read-uncommitted transactions.: Operation not supported Mar 2 15:59:57 slapd[83158]: wt_dn2id_add: insert failed: Operation not supported (95) That comes from WiredTiger @ https://github.com/wiredtiger/wiredtiger/blob/5a032be765b1ebd9bb789e837cd... but I don't seem to understand why it's happening on a simple add? Am I missing something obvious? -- You are receiving this mail because: You are on the CC list for the issue.

2 days, 19 hours

1
5
0 / 0

[Issue 10027] New: MDB_TXN_FULL on large write transactions

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10027 Issue ID: 10027 Summary: MDB_TXN_FULL on large write transactions Product: LMDB Version: unspecified Hardware: All OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs(a)openldap.org Reporter: renault.cle(a)gmail.com Target Milestone: --- Hello, Our users ([1], [2]) encountered MDB_TXN_FULL errors when our Meilisearch engine processed a large write transaction. We did read the documentation about this error in the codebase of LMDB: Spill pages from the dirty list back to disk. This is intended to prevent running into #MDB_TXN_FULL situations, but note that they may still occur in a few cases: 1) our estimate of the txn size could be too small. Currently this seems unlikely, except with a large number of #MDB_MULTIPLE items. 2) child txns may run out of space if their parents dirtied a lot of pages and never spilled them. TODO: we probably should do a preemptive spill during #mdb_txn_begin() of a child txn, if the parent's dirty_room is below a given threshold. Otherwise, if not using nested txns, it is expected that apps will not run into #MDB_TXN_FULL any more. The pages are flushed to disk the same way as for a txn commit, e.g. their P_DIRTY flag is cleared. If the txn never references them again, they can be left alone. If the txn only reads them, they can be used without any fuss. If the txn writes them again, they can be dirtied immediately without going thru all of the work of #mdb_page_touch(). Such references are handled by #mdb_page_unspill(). However, It looks like we are not in those scenarios, we are not using MDB_DUPFIXED, and we are not using sub-transactions. We don't use the MDB_VL32 flag either, so this is not related to [3]. Thank you for your time, Have a nice day 💡 [1]: https://github.com/meilisearch/meilisearch/issues/3603 [2]: https://github.com/meilisearch/meilisearch/issues/3349 [3]: https://bugs.openldap.org/show_bug.cgi?id=8813 -- You are receiving this mail because: You are on the CC list for the issue.

3 days, 4 hours

1
1
0 / 0

[Issue 10026] New: Refresh handling can skip entries (si_dirty not managed properly)

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10026 Issue ID: 10026 Summary: Refresh handling can skip entries (si_dirty not managed properly) Product: OpenLDAP Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: overlays Assignee: bugs(a)openldap.org Reporter: ondra(a)mistotebe.net Target Milestone: --- Take MPR plain syncrepl with 3+ providers. When a provider's own syncrepl session transitions to persist and a it starts a new parallel session towards another host, that session always has to start as a refresh. If that refresh serves entries to us, our handling of si_dirty is not consistent: - if the existing persist session serves some of these entries to us, we can "forget" to pass the others to a newly connected consumer - same if the refresh is abandoned and we start refreshing from a different provider that might be behind what we were being served (again our consumers could suffer) - if we restart, si_dirty is forgotten and our consumers suffer even worse We might need to be told (at the beginning of the refresh?) what the end state we're going for is, so we can keep si_dirty on until then. And somehow persist that knowledge in the DB... -- You are receiving this mail because: You are on the CC list for the issue.

3 days, 9 hours

1
1
0 / 0

[Issue 10025] New: Add option to disable filtered searches for memberURL groups

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10025 Issue ID: 10025 Summary: Add option to disable filtered searches for memberURL groups Product: OpenLDAP Version: 2.5.14 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: overlays Assignee: bugs(a)openldap.org Reporter: subbarao(a)computer.org Target Milestone: --- One of the changes from 2.4 to 2.5 is that dynlist groups are now returned with (member=memberDN) searches. This is potentially appealing, but even with the ITS#9929 performance improvements, given the number of dynlist groups we have, search times are significantly impacted. We'd like to be able to cleanly disable this feature and exclude dynlist groups from (member=memberDN) filter consideration. The only way I've found so far is to patch the dynlist code itself. What I'm currently doing is adding a continue statement right above this line in dynlist_search(): https://git.openldap.org/openldap/openldap/-/blob/OPENLDAP_REL_ENG_2_5_14... That way the member searches are excluded, but dynlists otherwise work as expected. Here is the dynlist config we're using, just basic support for groupOfURLs/memberURL: overlay dynlist dynlist-attrset groupOfURLs memberURL member I'd like to request a configurable option to exclude dynlists from (member=memberDN) searches. -- You are receiving this mail because: You are on the CC list for the issue.

5 days, 3 hours

1
3
0 / 0

[Issue 10023] New: Asynchronous connects are broken

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10023 Issue ID: 10023 Summary: Asynchronous connects are broken Product: OpenLDAP Version: 2.5.14 Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: libraries Assignee: bugs(a)openldap.org Reporter: ipuleston(a)sonicwall.com Target Milestone: --- We have a port of OpenLDAP client running in an embedded system, which is using asynchronous connects to the LDAP server. We have been using OpenLDAP 2.4.40 for a long time, and I just upgraded it to use 2.5.14 (as the current LTS release). After doing this, async connects to the LDAP server no longer work. You can see this in the following debug output: A successful async connect with 2.4.40: ldap_send_initial_request ldap_new_connection 1 1 0 ldap_int_open_connection ldap_connect_to_host: TCP Ian-DC1.sd80.com:389 ldap_pvt_gethostbyname_a: host=Ian-DC1.sd80.com, r=0 ldap_new_socket: 251 ldap_prepare_socket: 251 ldap_connect_to_host: Trying 192.168.168.3:389 ldap_pvt_connect: fd: 251 tm: 10 async: -1 ldap_ndelay_on: 251 attempting to connect: connect errno: 115 ldap_int_poll: fd: -1 tm: 0 A failed async connect with 2.5.14: ldap_send_initial_request ldap_new_connection 1 1 0 ldap_int_open_connection ldap_connect_to_host: TCP Ian-DC1.sd80.com:389 ldap_pvt_gethostbyname_a: host=Ian-DC1.sd80.com, r=0 ldap_new_socket: 247 ldap_prepare_socket: 247 ldap_connect_to_host: Trying 10.21.61.3:389 ldap_pvt_connect: fd: 247 tm: 10 async: -1 ldap_ndelay_on: 247 attempting to connect: connect errno: 115 ldap_open_defconn: successful ldap_send_server_request Sending Bind Request, len=0x6ca10c1f ldap_write: want=63 error=Resource temporarily unavailable Note that in both cases the connect attempt returns errno 115, EINPROGRESS, meaning that it has not completed. But after that: ● 2.4.40 calls ldap_int_poll (via ldap_send_initial_request -> ldap_int_check_async_open) to begin the wait for async completion. ● 2.5.14 instead reports a successful connect, and tries to send the bind which fails since thre socket is not yet connected. I tracked the problem down to a change made for ITS #8022 "an async connect may still succeed immediately" in this commit: https://git.openldap.org/openldap/openldap/-/commit/ae6347bac12bbf843678a... That change in ldap_new_connection makes it set lconn_status for an async connect to LDAP_CONNST_CONNECTED rather than LDAP_CONNST_CONNECTING if ldap_int_open_connection returns 0. The problem is that ldap_int_open_connection returns 0 after getting the EINPROGRESS. ldap_connect_to_host returns -2 for the latter, but ldap_int_open_connection doesn't check for that, returning 0 for any return code other than -1. I think that the bug is actually in ldap_int_open_connection rather than in the above commit. It should probably return -2 when ldap_connect_to_host returns that. -- You are receiving this mail because: You are on the CC list for the issue.

1 week, 2 days

1
11
0 / 0

[Issue 10024] New: MDB_PREVSNAPSHOT broken

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10024 Issue ID: 10024 Summary: MDB_PREVSNAPSHOT broken Product: LMDB Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: liblmdb Assignee: bugs(a)openldap.org Reporter: markus(a)objectbox.io Target Milestone: --- It seems that the patch #9496 had a negative side effect on MDB_PREVSNAPSHOT. In certain cases, when opening the DB using MDB_PREVSNAPSHOT, the previous (2nd latest) commit is not selected. Instead, reads show that the latest commit was selected voiding the effect of MDB_PREVSNAPSHOT. I observed this in our test cases a while back. Today, I was finally able to reproduce it and debug into it. When creating the transaction to read the data, I debugged into mdb_txn_renew0. Here, ti (MDB_txninfo; env->me_txns) was non-NULL. However, ti->mti_txnid was 0 (!) and thus txn->mt_txnid was set to 0. That's the reason for always selecting the first (index 0) meta page inside mdb_txn_renew0: meta = env->me_metas[txn->mt_txnid & 1]; This line occurs twice (once for read txn and once for write txn; it affects both txn types). Thus, the chances of MDB_PREVSNAPSHOT selecting the correct meta page is 50-50. It's only correct if the first meta page (index 0) is the older one. I believe that this is related to #9496 because the patch, that was provided there, removed the initialization of "env->me_txns->mti_txnid" in mdb_env_open2. This would explain why txn->mt_txnid inside mdb_txn_renew0 was set to 0. I can confirm that adding back the following two lines back in fixes MDB_PREVSNAPSHOT: if (env->me_txns) env->me_txns->mti_txnid = meta.mm_txnid; The said patch including the removal of these two lines was applied in the commit(s) "ITS#9496 fix mdb_env_open bug from #8704" (Howard Chu on 09.04.21). I hope this information is useful to find a suitable fix. Please let me know if you have questions. Also, I'd be happy to help confirming a potential fix with our test suite. -- You are receiving this mail because: You are on the CC list for the issue.

1 week, 3 days

1
2
0 / 0

[Issue 9496] New: Some writes missing from database

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=9496 Issue ID: 9496 Summary: Some writes missing from database Product: LMDB Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: liblmdb Assignee: bugs(a)openldap.org Reporter: igfoo(a)github.com Target Milestone: --- With the attached test program, some of my database writes appear not to actually be written to the database. For example, a run may look like this: $ ./run.sh All done. All finished 1802 test.txt foo_200 is missing bar_200 is missing foo_404 is missing bar_404 is missing foo_407 is missing bar_407 is missing The script that I am using to run the program is below. This is using mdb.master 52bc29ee2efccf09c650598635cd42a50b6ecffe on Linux, with an ext4 filesystem. Is this an LMDB bug, or is there a bug in my code? Thanks Ian #!/bin/sh set -e if ! [ -d lmdb ] then rm -rf lmdb git clone https://github.com/LMDB/lmdb.git INSTALL_DIR="`pwd`/inst" cd lmdb/libraries/liblmdb make install prefix="$INSTALL_DIR" cd ../../.. fi gcc -Wall -Werror -Iinst/include loop.c inst/lib/liblmdb.a -o loop -pthread rm -f test.db test.db-lock ./loop echo "All finished" mdb_dump -np test.db > test.txt wc -l test.txt for i in `seq 100 999` do if ! grep -q "foo_$i" test.txt then echo "foo_$i is missing" fi if ! grep -q "bar_$i" test.txt then echo "bar_$i is missing" fi done -- You are receiving this mail because: You are on the CC list for the issue.

1 week, 4 days

1
16
0 / 0

[Issue 10022] New: OlcAccess (META)

by openldap-its＠openldap.org

https://bugs.openldap.org/show_bug.cgi?id=10022 Issue ID: 10022 Summary: OlcAccess (META) Product: OpenLDAP Version: 2.5.7 Hardware: x86_64 OS: Linux Status: UNCONFIRMED Keywords: needs_review Severity: normal Priority: --- Component: backends Assignee: bugs(a)openldap.org Reporter: bourguijl(a)gmail.com Target Milestone: --- Dears, I've configured a META ldap instance pointing to a LDAP backend. In this backend, there are a few ACLs but which ones don't restrict ldapsearch that I perform from the META frontend. I just have an issue when I set some ACLs on the META frontend and more specially when I insert attrs=xxx in the ACL. ACL = OK {0}to dn.one="ou=staff,o=mobistar.be" by dn="uid=a0621004,ou=ObeExternalITOnGcp,ou=partners,o=mobistar.be" read ACL NOT OK {0}to dn.one="ou=staff,o=mobistar.be" attrs=uid by dn="uid=a0621004,ou=ObeExternalITOnGcp,ou=partners,o=mobistar.be" read Can you explain why when I restrict to an attribute, my ldapsearch didn't provide me any response as expected ? Is it a bug ? Thx in advance, J-L. -- You are receiving this mail because: You are on the CC list for the issue.

1 week, 5 days

1
4
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs March 2023