https://bugs.openldap.org/show_bug.cgi?id=8977
OndÅ™ej KuznÃk <ondra(a)mistotebe.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
See Also| |https://bugs.openldap.org/s
| |how_bug.cgi?id=10031
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10027
Issue ID: 10027
Summary: MDB_TXN_FULL on large write transactions
Product: LMDB
Version: unspecified
Hardware: All
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: renault.cle(a)gmail.com
Target Milestone: ---
Hello,
Our users ([1], [2]) encountered MDB_TXN_FULL errors when our Meilisearch
engine processed a large write transaction. We did read the documentation about
this error in the codebase of LMDB:
Spill pages from the dirty list back to disk.
This is intended to prevent running into #MDB_TXN_FULL situations,
but note that they may still occur in a few cases:
1) our estimate of the txn size could be too small. Currently this
seems unlikely, except with a large number of #MDB_MULTIPLE items.
2) child txns may run out of space if their parents dirtied a
lot of pages and never spilled them. TODO: we probably should do
a preemptive spill during #mdb_txn_begin() of a child txn, if
the parent's dirty_room is below a given threshold.
Otherwise, if not using nested txns, it is expected that apps will
not run into #MDB_TXN_FULL any more. The pages are flushed to disk
the same way as for a txn commit, e.g. their P_DIRTY flag is cleared.
If the txn never references them again, they can be left alone.
If the txn only reads them, they can be used without any fuss.
If the txn writes them again, they can be dirtied immediately without
going thru all of the work of #mdb_page_touch(). Such references are
handled by #mdb_page_unspill().
However, It looks like we are not in those scenarios, we are not using
MDB_DUPFIXED, and we are not using sub-transactions. We don't use the MDB_VL32
flag either, so this is not related to [3].
Thank you for your time,
Have a nice day 💡
[1]: https://github.com/meilisearch/meilisearch/issues/3603
[2]: https://github.com/meilisearch/meilisearch/issues/3349
[3]: https://bugs.openldap.org/show_bug.cgi?id=8813
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10029
Issue ID: 10029
Summary: slapd crashes when run with unlimited open files
Product: OpenLDAP
Version: 2.6.4
Hardware: All
OS: Mac OS
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: gray(a)nxg.name
Target Milestone: ---
To reproduce:
% ulimit -n
unlimited
% $T/openldap-2.6.4/libexec/slapd -d-1
641ee8bc.32f05820 0x1dc760140 @(#) $OpenLDAP: slapd 2.6.4 (Mar 25 2023
12:25:49) $
openldap
641ee8bc.32f39ff8 0x1dc760140 daemon_init: <null>
641ee8bc.32f40588 0x1dc760140 daemon: SLAP_SOCK_INIT: dtblsize=-1
641ee8bc.32f43080 0x1dc760140 ch_calloc of 1 elems of 18446744073709551615
bytes failed
Assertion failed: (0), function ch_calloc, file ch_malloc.c, line 107.
zsh: abort $T/openldap-2.6.4/libexec/slapd -d-1
This is because `daemon.c` (line 1867) uses the maximum number of open files to
set `dtblsize`, which is subsequently used to size an array:
1867 #ifdef HAVE_SYSCONF
1868 dtblsize = sysconf( _SC_OPEN_MAX );
1869 #elif defined(HAVE_GETDTABLESIZE)
1870 dtblsize = getdtablesize();
1871 #else /* ! HAVE_SYSCONF && ! HAVE_GETDTABLESIZE */
1872 dtblsize = FD_SETSIZE;
1873 #endif /* ! HAVE_SYSCONF && ! HAVE_GETDTABLESIZE */
If the maximum number of FDs is unlimited, then sysconf(_SC_OPEN_MAX) returns
-1, and the program crashes when it tries to malloc that much memory.
I've marked this as OS=macOS because that's what I'm illustrating this on, but
the same thing would happen on any OS where the sysconf call returns a negative
number for the 'unlimited' case.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10022
Issue ID: 10022
Summary: OlcAccess (META)
Product: OpenLDAP
Version: 2.5.7
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: backends
Assignee: bugs(a)openldap.org
Reporter: bourguijl(a)gmail.com
Target Milestone: ---
Dears,
I've configured a META ldap instance pointing to a LDAP backend.
In this backend, there are a few ACLs but which ones don't restrict ldapsearch
that I perform from the META frontend.
I just have an issue when I set some ACLs on the META frontend and more
specially when I insert attrs=xxx in the ACL.
ACL = OK
{0}to dn.one="ou=staff,o=mobistar.be" by
dn="uid=a0621004,ou=ObeExternalITOnGcp,ou=partners,o=mobistar.be" read
ACL NOT OK
{0}to dn.one="ou=staff,o=mobistar.be" attrs=uid by
dn="uid=a0621004,ou=ObeExternalITOnGcp,ou=partners,o=mobistar.be" read
Can you explain why when I restrict to an attribute, my ldapsearch didn't
provide me any response as expected ?
Is it a bug ?
Thx in advance,
J-L.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10024
Issue ID: 10024
Summary: MDB_PREVSNAPSHOT broken
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: markus(a)objectbox.io
Target Milestone: ---
It seems that the patch #9496 had a negative side effect on MDB_PREVSNAPSHOT.
In certain cases, when opening the DB using MDB_PREVSNAPSHOT, the previous (2nd
latest) commit is not selected. Instead, reads show that the latest commit was
selected voiding the effect of MDB_PREVSNAPSHOT.
I observed this in our test cases a while back. Today, I was finally able to
reproduce it and debug into it.
When creating the transaction to read the data, I debugged into mdb_txn_renew0.
Here, ti (MDB_txninfo; env->me_txns) was non-NULL. However, ti->mti_txnid was 0
(!) and thus txn->mt_txnid was set to 0. That's the reason for always selecting
the first (index 0) meta page inside mdb_txn_renew0:
meta = env->me_metas[txn->mt_txnid & 1];
This line occurs twice (once for read txn and once for write txn; it affects
both txn types).
Thus, the chances of MDB_PREVSNAPSHOT selecting the correct meta page is 50-50.
It's only correct if the first meta page (index 0) is the older one.
I believe that this is related to #9496 because the patch, that was provided
there, removed the initialization of "env->me_txns->mti_txnid" in
mdb_env_open2. This would explain why txn->mt_txnid inside mdb_txn_renew0 was
set to 0.
I can confirm that adding back the following two lines back in fixes
MDB_PREVSNAPSHOT:
if (env->me_txns)
env->me_txns->mti_txnid = meta.mm_txnid;
The said patch including the removal of these two lines was applied in the
commit(s) "ITS#9496 fix mdb_env_open bug from #8704" (Howard Chu on 09.04.21).
I hope this information is useful to find a suitable fix. Please let me know if
you have questions. Also, I'd be happy to help confirming a potential fix with
our test suite.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8447
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|--- |0.9.31
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9496
Issue ID: 9496
Summary: Some writes missing from database
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: igfoo(a)github.com
Target Milestone: ---
With the attached test program, some of my database writes appear not to
actually be written to the database. For example, a run may look like this:
$ ./run.sh
All done.
All finished
1802 test.txt
foo_200 is missing
bar_200 is missing
foo_404 is missing
bar_404 is missing
foo_407 is missing
bar_407 is missing
The script that I am using to run the program is below. This is using
mdb.master 52bc29ee2efccf09c650598635cd42a50b6ecffe on Linux, with an ext4
filesystem.
Is this an LMDB bug, or is there a bug in my code?
Thanks
Ian
#!/bin/sh
set -e
if ! [ -d lmdb ]
then
rm -rf lmdb
git clone https://github.com/LMDB/lmdb.git
INSTALL_DIR="`pwd`/inst"
cd lmdb/libraries/liblmdb
make install prefix="$INSTALL_DIR"
cd ../../..
fi
gcc -Wall -Werror -Iinst/include loop.c inst/lib/liblmdb.a -o loop -pthread
rm -f test.db test.db-lock
./loop
echo "All finished"
mdb_dump -np test.db > test.txt
wc -l test.txt
for i in `seq 100 999`
do
if ! grep -q "foo_$i" test.txt
then
echo "foo_$i is missing"
fi
if ! grep -q "bar_$i" test.txt
then
echo "bar_$i is missing"
fi
done
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10017
Issue ID: 10017
Summary: ldap.conf setting "BINDDN" has no associated
LDAP_OPT_XXX constant for ldap_get_opt ldap_set_opt
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: sean(a)teletech.com.au
Target Milestone: ---
The Configuration file setting "BINDDN" has no associated LDAP_OPT_XXX constant
and is not exposed to the ldap_get_opt / ldap_set_opt API. This is the only
option that is not so accessible and this seems like an oversight.
Option "PORT" is also not exposed but that is deprecated. You could make the
case it shouldn't be.
This setting could obviously be of interest to the Application and I see no
reason for it to be hidden.
Simple applications / tools may not have their own configuration files and
instead rely solely of the ldap.conf file to configure openldap. Such an
application could not easy supply a DN to the "bind" calls but may still wish
to know the value specified in the configuration file.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10019
Issue ID: 10019
Summary: dynlist's +memberOf attribute not searchable/fetchable
with anonymous binds
Product: OpenLDAP
Version: 2.5.13
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: msl(a)touk.pl
Target Milestone: ---
Hi,
This is and issue we discovered (as a side effect of testing) after switching
from memberof overlay to dynlist with our confluence directory setup - which
previously worked fine, but not anymore.
Side effect of testing - as judging from the logs it seems that confluence is
doing normal binds (which is even stranger as non-anonymous bind ldapsearch
from commandline works correctly).
Anyway, consider the following setup:
groupOfURLs labeledURI uniqueMember+memberOf@groupOfUniqueNames
We only use static groups, so the following group with one of members:
DN: cn=TouK,ou=TouK,ou=Group,dc=touk,dc=pl
objectClass: groupOfUniqueNames
...
uniqueMember: cn=Michał Sołtys,ou=Touki,ou=People,dc=touk,dc=pl
Correlates to:
DN: cn=Michał Sołtys,ou=Touki,ou=People,dc=touk,dc=pl
...
memberOf: cn=touk,ou=touk,ou=group,dc=touk,dc=pl
The initial ACLs are set as follows:
{0}to * by dn=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by
* break
{1}to dn.subtree=ou=People,dc=touk,dc=pl
attrs=entry,entryUUID,memberOf,@toukAnonAccess by anonymous =scr by * break
{2}to dn.subtree=ou=Group,dc=touk,dc=pl
attrs=entry,@groupOfUniqueNames,@groupOfNames by anonymous =scr by * break
... later ACLs handling specific accesses and stuff, terminated with:
{14}to * by users =scr
Now if we do search doing non-anonymous binds, everything works correctly:
ldapsearch -x -H ldaps://ldap.touk.pl -D "cn=Michał
Sołtys,ou=Touki,ou=People,dc=touk,dc=pl" -s sub -b
'ou=Touki,ou=People,dc=touk,dc=pl' -o ldif-wrap=no -y ./pass -LLL -v
'(uid=ast)' memberOf entryUUID
ldap_initialize( ldaps://ldap.touk.pl:636/??base )
filter: (uid=ast)
requesting: memberOf entryUUID
dn: cn=Adam Stus,ou=Touki,ou=People,dc=touk,dc=pl
entryUUID: 6c1adf48-a800-103a-8044-3100241d53c2
memberOf: cn=touk,ou=touk,ou=group,dc=touk,dc=pl
But if we do an anonymous search - with ACLs as above explicitly allowing
access to all relevant parts as in rule {1}, memberOf is not returned (it can't
be used in filtering either):
ldapsearch -x -H ldaps://ldap.touk.pl -s sub -b
'ou=Touki,ou=People,dc=touk,dc=pl' -o ldif-wrap=no -LLL -v '(uid=ast)'
memberOf entryUUID
ldap_initialize( ldaps://ldap.touk.pl:636/??base )
filter: (uid=ast)
requesting: memberOf entryUUID
dn: cn=Adam Stus,ou=Touki,ou=People,dc=touk,dc=pl
entryUUID: 6c1adf48-a800-103a-8044-3100241d53c2
This - unless I missed something - looks like a bug.
As mentioned above - our local confluence install is using dedicated user, but
for some reason it is also unable filter using memberOf (though surprisingly it
does work from command line for non-anonymous bind). Relevant parts of the
slapd.log of such query:
Mar 6 19:21:24 lipa1 slapd[1206591]: conn=1009 op=0 BIND
dn="cn=confluence,ou=Apps,dc=touk,dc=pl" method=128
Mar 6 19:21:24 lipa1 slapd[1206591]: conn=1009 op=0 BIND
dn="cn=confluence,ou=Apps,dc=touk,dc=pl" mech=SIMPLE bind_ssf=0 ssf=256
Mar 6 19:21:24 lipa1 slapd[1206591]: conn=1009 op=0 RESULT tag=97 err=0
qtime=0.000016 etime=0.000230 text=
... other operations
Mar 6 19:26:58 lipa1 slapd[1206591]: conn=1009 op=28 SRCH
base="ou=Touki,ou=People,dc=touk,dc=pl" scope=2 deref=3
filter="(&(toukAccountActive=TRUE)(memberOf=cn=finanse,ou=touk,ou=group,dc=touk,dc=pl))"
Mar 6 19:26:58 lipa1 slapd[1206591]: conn=1009 op=28 SRCH attr=1.1
Mar 6 19:26:58 lipa1 slapd[1206591]: conn=1009 op=28 SEARCH RESULT tag=101
err=0 qtime=0.000019 etime=0.000541 nentries=0 text=
See (nentries=0) above - but identical search performed from command line,
e.g.:
ldapsearch -x -H ldaps://ldap.touk.pl -D "cn=confluence,ou=Apps,dc=touk,dc=pl"
-y ./b -a always -s sub -b 'ou=Touki,ou=People,dc=touk,dc=pl' -o ldif-wrap=no
-LLL -v
'(&(toukAccountActive=TRUE)(memberOf=cn=finanse,ou=touk,ou=group,dc=touk,dc=pl))'
1.1
correctly returns 6 people:
Mar 7 15:21:31 lipa1 slapd[1220021]: conn=26424 op=0 BIND
dn="cn=confluence,ou=Apps,dc=touk,dc=pl" method=128
Mar 7 15:21:31 lipa1 slapd[1220021]: conn=26424 op=0 BIND
dn="cn=confluence,ou=Apps,dc=touk,dc=pl" mech=SIMPLE bind_ssf=0 ssf=256
Mar 7 15:21:31 lipa1 slapd[1220021]: conn=26424 op=0 RESULT tag=97 err=0
qtime=0.000025 etime=0.000269 text=
Mar 7 15:21:31 lipa1 slapd[1220021]: conn=26424 op=1 SRCH
base="ou=Touki,ou=People,dc=touk,dc=pl" scope=2 deref=3
filter="(&(toukAccountActive=TRUE)(memberOf=cn=finanse,ou=touk,ou=group,dc=touk,dc=pl))"
Mar 7 15:21:31 lipa1 slapd[1220021]: conn=26424 op=1 SRCH attr=1.1
Mar 7 15:21:31 lipa1 slapd[1220021]: conn=26424 op=2 UNBIND
Mar 7 15:21:31 lipa1 slapd[1220021]: conn=26424 op=1 SEARCH RESULT tag=101
err=0 qtime=0.000019 etime=0.002765 nentries=6 text=
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=4501
Quanah Gibson-Mount <quanah(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords|needs_review |
--
You are receiving this mail because:
You are on the CC list for the issue.