[Issue 9735] New: [PATCH] try hard to find free space if database cannot grow
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9735
Issue ID: 9735
Summary: [PATCH] try hard to find free space if database cannot
grow
Product: LMDB
Version: 0.9.24
Hardware: All
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: libor.peltan(a)nic.cz
Target Milestone: ---
Created attachment 851
--> https://bugs.openldap.org/attachment.cgi?id=851&action=edit
Patch fixing the issue "try hard to find free space if database cannot grow"
Note:
- the issue is the same in version 0.9.70 (git)
Situation:
- the database had already grown to its limit (mapsize) in the past
- overflow pages are used heavily as stored values are usually several pages
long
- free space got fragmented
Problem:
- attempt to insert new value results in MDB_MAP_FULL despite there is free
space available
Cause: there is a heursitic in mdb_page_alloc() that gives up searching for
free space chunk if this would take too much time. This is useful when the
database can still grow, as it balances performance with space usage. However,
if the database can no longer grow, it prevents inserting new values.
Solution: detect early on in mdb_page_alloc() if the database can grow, and if
not, let it try hard to search for free space.
Patch: attached
--
You are receiving this mail because:
You are on the CC list for the issue.
3 days, 16 hours
[Issue 9278] New: liblmdb: robust mutexes should not be unmapped
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9278
Issue ID: 9278
Summary: liblmdb: robust mutexes should not be unmapped
Product: LMDB
Version: unspecified
Hardware: All
OS: FreeBSD
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: delphij(a)freebsd.org
Target Milestone: ---
Created attachment 736
--> https://bugs.openldap.org/attachment.cgi?id=736&action=edit
A possible workaround
We recently noticed that lmdb would have the memory region containing the
robust mutex unmapped on mdb_env_close0():
munmap((void *)env->me_txns,
(env->me_maxreaders-1)*sizeof(MDB_reader)+sizeof(MDB_txninfo));
Note that if this is the last unmap for a robust mutex, the FreeBSD
implementation would garbage-collect the mutex, making it no longer visible to
other processes. As the result, a second instance of the attached test.c (from
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244493 with minor changes)
would trigger the assertion at mdb_txn_begin() because the acquisition of the
mutex would return 22 (EINVAL), because the mutex appeared to be a robust
mutex, but was invalid.
The attached lmdb.diff is a possible workaround for this (it would skip
unmapping when setting up the robust mutex for the first time).
--
You are receiving this mail because:
You are on the CC list for the issue.
4 weeks
[Issue 9506] New: dynlist: member expansion when member attribute not requested
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9506
Issue ID: 9506
Summary: dynlist: member expansion when member attribute not
requested
Product: OpenLDAP
Version: 2.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
When configured to do dynamic "member" expansion, i.e.:
overlay dynlist
dynlist-attrset groupOfURLs memberURL member
Any query against an object that would trigger this expansion will incur a
penalty while dynlist does the expansion work even if there was no request for
the member attribute.
Currently that can be worked around by specifying the manageDSAit control when
doing a search on the object, but this may not be feasible for some client
applications and additionally other directory servers do not do this expansion
for their dynamic group implementations unless the underlying configured
attribute is explicitly requested.
We've already implemented this in dynlist for the memberOfAD case, we should do
it here as well.
--
You are receiving this mail because:
You are on the CC list for the issue.
2 months, 2 weeks
[Bug 9244] New: API calls blocking after async connect
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9244
Bug ID: 9244
Summary: API calls blocking after async connect
Product: OpenLDAP
Version: 2.4.49
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: ryan(a)openldap.org
Target Milestone: ---
Created attachment 721
--> https://bugs.openldap.org/attachment.cgi?id=721&action=edit
async connect test without TLS
My understanding of LDAP_OPT_CONNECT_ASYNC is that the attached program should
not block. If the connection does not establish fast enough, the bind call is
supposed to return LDAP_X_CONNECTING.
(At least that's how I understand it, based on the original behaviour (circa
2.4.23 up to 2.4.40) as well as the bind loop in back-meta. On the other hand,
the man page does "Subsequent calls to library routines will poll for
completion of the connect before performing further operations" which might be
interpreted as meaning they would block...)
In current releases it does block, as demonstrated by strace on Linux (latency
added using 'tc qdisc'):
[...]
connect(3, {sa_family=AF_INET, sin_port=htons(389),
sin_addr=inet_addr("192.168.1.204")}, 16) = -1 EINPROGRESS (Operation now in
progress)
write(3, "0\f\2\1\1`\7\2\1\3\4\0\200\0", 14) = -1 EAGAIN (Resource temporarily
unavailable)
poll([{fd=3, events=POLLOUT|POLLERR|POLLHUP}], 1, -1) = 1 ([{fd=3,
revents=POLLOUT}])
write(3, "0\f\2\1\1`\7\2\1\3\4\0\200\0", 14) = 14
poll([{fd=3, events=POLLIN|POLLPRI}], 1, -1) = 1 ([{fd=3, revents=POLLIN}])
read(3, "0\f\2\1\1a\7\n", 8) = 8
read(3, "\1\0\4\0\4\0", 6) = 6
write(2, "OK: ldap_simple_bind_returned 0 "..., 42OK: ldap_simple_bind_returned
0 (Success)
) = 42
[...]
As discussed in IRC, I believe I bisected this down to commit ae6347bac, from
bug 8022. The reasoning is sound, but ldap_int_open_connection does not
actually return -2, only -1 or 0.
The patch is simple enough, but I'm also looking at some later commits that
were probably done to work around this, and might not be needed now (bug 8957,
bug 8968, bug 8980). Also need to test all setups thoroughly (ldap, ldaps,
STARTTLS, not to mention back-meta/asyncmeta).
I also notice that LDAP_OPT_CONNECT_ASYNC is not effective unless
LDAP_OPT_NETWORK_TIMEOUT is also set. It might be intentional, but the man page
doesn't mention this specifically, and I don't see why it would be necessary...
--
You are receiving this mail because:
You are on the CC list for the bug.
4 months
[Issue 9736] New: pwrite bug in OSX breaking LMDB promise about the maximum value size
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9736
Issue ID: 9736
Summary: pwrite bug in OSX breaking LMDB promise about the
maximum value size
Product: LMDB
Version: unspecified
Hardware: All
OS: Mac OS
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: renault.cle(a)gmail.com
Target Milestone: ---
Hi,
I was working with LMDB and found an issue when trying to write a value of
approximately 3.3GiB in the database, I dive into the LMDB source code of the
mdb_put method using the lldb debugger and found out that it was not related to
an issue in LMDB itself but rather a bug in the pwrite function of the Mac OS
libc implementation.
The pwrite function is given four parameters, the file descriptor, the buffer,
the count of bytes to write from the buffer and, the offset of where to write
it in the file. On Mac OS the count of bytes is a size_t that must be a 64bits
unsigned integer but when you call pwrite with a number bigger or equal to 2^31
it returns an error 22 (invalid argument). LMDB was returning a 22 error from
the mdb_put call and not an EINVAL because the error was cause by an internal
issue and not something catchable by LMDB.
I am not sure about what we can do, can we implement this single pwrite [1] as
multiple pwrite with counts smaller than 2^31 in a loop, just for Mac OS? Like
for Windows where we do specific things for this operating system too?
I also found this issue on the RocksDB repository [2] about a similar problem
they have with pwrite and write on Mac OS it seems. I understand that this is
not a real promise that LMDB is specifying but rather an "in theory" rule [3].
Thank you for your time,
kero
[1]:
https://github.com/LMDB/lmdb/blob/01b1b7dc204abdf3849536979205dc9e3a0e3ec...
[2]: https://github.com/facebook/rocksdb/issues/5169
[3]: http://www.lmdb.tech/doc/group__mdb.html#structMDB__val
--
You are receiving this mail because:
You are on the CC list for the issue.
4 months
[Bug 9229] New: Make liblutil usable by libldap
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9229
Bug ID: 9229
Summary: Make liblutil usable by libldap
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: build
Assignee: bugs(a)openldap.org
Reporter: ryan(a)openldap.org
Target Milestone: ---
liblutil is a static library (non-PIC) and so cannot be linked into shared
objects, however we have several use cases for reusing its code in libldap.
Some options:
- moving more code from liblutil to libldap
- just merge the whole thing?
- are there components that link liblutil but _not_ libldap?
- build liblutil as PIC (take a minor performance hit when linked into
programs?)
- build liblutil twice (liblutil.a and liblutil_pic.a)
- symlink liblutil sources into libldap build dir, like libldap_r does with
libldap
- both of these last options require checking whether executables can call
the PIC symbols safely (if some symbols are used by both library and program
code)
Nice-to-have for 2.5, I'd say more likely for 2.6 at this point.
--
You are receiving this mail because:
You are on the CC list for the bug.
4 months, 1 week
[Issue 9436] New: OpenSSL 3.0: libldap uses depreciated functions
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9436
Issue ID: 9436
Summary: OpenSSL 3.0: libldap uses depreciated functions
Product: OpenLDAP
Version: 2.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
OpenLDAP master fails to build against OpenSSL 3.0 alpha when "no-deprecated"
is specified.
Currently hitting these errors:
./.libs/libldap.so: undefined reference to `SSL_get_peer_certificate'
./.libs/libldap.so: undefined reference to `PEM_read_bio_DHparams'
./.libs/libldap.so: undefined reference to `ERR_get_error_line'
./.libs/libldap.so: undefined reference to `DH_free'
./.libs/libldap.so: undefined reference to `SSL_CTX_set_tmp_dh'
Notes:
SSL_get_peer_certificate is SSL_get1_peer_certificate in 3.0.0
SSL_CTX_set_tmp_dh should be replaced as follows:
# define SSL_CTX_set_tmp_dh(ctx,dh) \
SSL_CTX_ctrl(ctx,SSL_CTRL_SET_TMP_DH,0,(char *)(dh))
Have to dig deeper for:
PEM_read_bio_DHparams
ERR_get_error_line
DH_free
--
You are receiving this mail because:
You are on the CC list for the issue.
5 months
[Issue 9460] New: Drop support for OpenSSL older than 1.1.1
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9460
Issue ID: 9460
Summary: Drop support for OpenSSL older than 1.1.1
Product: OpenLDAP
Version: 2.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
OpenSSL no longer supports the 1.0.2 series and specifically notes it should
not be used:
"All older versions (including 1.1.0, 1.0.2, 1.0.0 and 0.9.8) are now out of
support and should not be used."
Currently configure checks for 1.0.2 or higher.
OpenSSL 1.1.1 is supported through 11 Sep 2023.
--
You are receiving this mail because:
You are on the CC list for the issue.
5 months, 2 weeks
[Issue 9496] New: Some writes missing from database
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9496
Issue ID: 9496
Summary: Some writes missing from database
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: igfoo(a)github.com
Target Milestone: ---
With the attached test program, some of my database writes appear not to
actually be written to the database. For example, a run may look like this:
$ ./run.sh
All done.
All finished
1802 test.txt
foo_200 is missing
bar_200 is missing
foo_404 is missing
bar_404 is missing
foo_407 is missing
bar_407 is missing
The script that I am using to run the program is below. This is using
mdb.master 52bc29ee2efccf09c650598635cd42a50b6ecffe on Linux, with an ext4
filesystem.
Is this an LMDB bug, or is there a bug in my code?
Thanks
Ian
#!/bin/sh
set -e
if ! [ -d lmdb ]
then
rm -rf lmdb
git clone https://github.com/LMDB/lmdb.git
INSTALL_DIR="`pwd`/inst"
cd lmdb/libraries/liblmdb
make install prefix="$INSTALL_DIR"
cd ../../..
fi
gcc -Wall -Werror -Iinst/include loop.c inst/lib/liblmdb.a -o loop -pthread
rm -f test.db test.db-lock
./loop
echo "All finished"
mdb_dump -np test.db > test.txt
wc -l test.txt
for i in `seq 100 999`
do
if ! grep -q "foo_$i" test.txt
then
echo "foo_$i is missing"
fi
if ! grep -q "bar_$i" test.txt
then
echo "bar_$i is missing"
fi
done
--
You are receiving this mail because:
You are on the CC list for the issue.
6 months, 1 week
[Issue 9718] New: test022 can fail on expiry
by openldap-its＠openldap.org
https://bugs.openldap.org/show_bug.cgi?id=9718
Issue ID: 9718
Summary: test022 can fail on expiry
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: build
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
>>>>> Starting test022-ppolicy for mdb...
running defines.sh
Starting slapd on TCP/IP port 9011...
Using ldapsearch to check that slapd is running...
Testing redundant ppolicy instance...
Using ldapadd to populate the database...
Testing account lockout...
Waiting 13 seconds for lockout to reset...
Testing password expiration
Waiting seconds for password to expire...
sleep: missing operand
Try 'sleep --help' for more information.
Password expiration test failed
>>>>> test022-ppolicy failed for mdb after 43 seconds
(exit 1)
The issue here is apparently that line 122-123 failed to populate the DELAY
variable.
121
122 DELAY=`$LDAPSEARCH -D "$MANAGERDN" -H $URI1 -w $PASSWD \
123 -b "$USER" -E accountUsability 1.1 | sed -n -e
's/.*expire=\(\d*\)/\1/p'`
124
125 echo "Testing password expiration"
126 echo "Waiting $DELAY seconds for password to expire..."
127 sleep $DELAY
128 sleep 1
--
You are receiving this mail because:
You are on the CC list for the issue.
7 months, 3 weeks