https://bugs.openldap.org/show_bug.cgi?id=10293
Issue ID: 10293
Summary: Log operations generated by syncrepl at STATS level
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: nivanova(a)symas.com
Target Milestone: ---
Similarly to how incoming operations are logged, operations created by syncrepl
should be logged as well.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10395
Issue ID: 10395
Summary: Support multiple readers on uncommitted changes
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: renault.cle(a)gmail.com
Target Milestone: ---
Created attachment 1086
--> https://bugs.openldap.org/attachment.cgi?id=1086&action=edit
A patch to support multiple readers on uncommitted changes
Hello,
The attached patch is not meant to be merged immediately into LMDB. Still, it
demonstrates how I added a helpful feature to the key-value store: reading
uncommitted changes from multiple transactions. I am conscious that the patch
still requires some work and uses non-C99 features, i.e., atomics are C11,
which could be a blocker for it to be merged upstream. I would also be
delighted to merge these changes under a flag/define to ensure we don't impact
other users with non-C99 stuff.
The main feature we need at Meilisearch is to read uncommitted changes from
multiple threads, compute parallel post-processing of different data structures
[1], and speed up the search requests. We could have done the post-processing
in a following transaction by opening multiple read transactions, but this
would mean that the post-processed data structure would not include newly
inserted or modified document IDs. Both data structures would be desync.
Regarding the design choice, I decided to follow the same design as the nested
write transactions: use the parent argument of the mdb_txn_begin [2], and allow
the MDB_RDONLY flag, which was disallowed when the parent argument was non-NULL
[3]. I find it clear enough that, by calling the mdb_txn_begin function with
these arguments, you can call it multiple times (I need to update the doc) to
obtain nested read-only transactions from the parent write transaction.
ret = mdb_txn_begin(env, parent_txn, MDB_RDONLY, new_nested_rtxn);
Note that this early proposal lacks security and error handling. The generated
transactions are fake-read-only and actually write transactions that share the
underlying parent allocations and data structures. This is unsafe and must be
changed or reviewed carefully, but most importantly, we need to add read-only
capabilities to these transactions to disallow writes. Using a Rust wrapper on
top of LMDB, I wrapped the fake read-only transactions into ReadTxn, which
disallows any writes at compile time. However, I haven't checked the conflict
database creations or openings.
The main issues I encountered were concurrent free of the main shared data
structures when the different threads owning the transactions were dropping the
transactions simultaneously. So, I decided to implement the equivalent of an
ARC to free resources only when the last nested transaction was freed.
I can share numbers about how this feature improves the post-processing by
4x-9x or from 1200s to 120s [1]. You can look at this PR, which I would be
happy to merge once an improved version of this patch lands on LMDB upstream.
I would be very happy if you could guide me a bit on how I could improve this
patch to make it mergeable into LMDB. We want to contribute useful features
like this to LMDB and not keep a deviant fork. LMDB works great; we are happy
about it, and its performance is predictable.
Have a lovely week,
kero
[1]: https://github.com/meilisearch/meilisearch/pull/5307
[2]:
https://github.com/LMDB/lmdb/blob/14d6629bc8a9fe40d8a6bee1bf71c45afe7576b6/…
[3]:
https://github.com/LMDB/lmdb/blob/14d6629bc8a9fe40d8a6bee1bf71c45afe7576b6/…
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10406
Issue ID: 10406
Summary: Write regression since 0.9.19
Product: LMDB
Version: 0.9.33
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: ben.alex(a)acegi.com.au
Target Milestone: ---
I maintain LmdbJava, a Java wrapper for LMDB. We have an extensive benchmark
suite covering read, cursor, and write pathways, with isolated testing of LMDB
versions from 0.9.17 through 0.9.33.
Results: https://lmdb-benchmark.lmdbjava.org/
==============================================
SUMMARY
==============================================
Read performance: 0.9.33 consistently outperforms 0.9.17 across all read
benchmarks.
Write performance: Consistent regression since 0.9.19:
----------------------------------
Tag ms/op vs 0.9.17
----------------------------------
LMDB_0.9.17 75.790 baseline
LMDB_0.9.18 74.909 -1.2%
LMDB_0.9.19 81.404 +7.4%
LMDB_0.9.20 83.459 +10.1%
LMDB_0.9.21 83.237 +9.8%
LMDB_0.9.22 83.027 +9.5%
LMDB_0.9.23 82.870 +9.3%
LMDB_0.9.24 82.952 +9.4%
LMDB_0.9.27 82.989 +9.5%
LMDB_0.9.28 83.025 +9.5%
LMDB_0.9.29 83.063 +9.6%
LMDB_0.9.30 83.237 +9.8%
LMDB_0.9.31 88.419 +16.7%
LMDB_0.9.33 83.304 +9.9%
Excluding 0.9.31 as an outlier, write performance degraded by approximately 9%
from 0.9.20 and has remained at that level through 0.9.33.
==============================================
BENCHMARK DETAILS
==============================================
The write benchmark sequentially inserts 1,000,000 entries (100-byte values,
4-byte integer keys) into a database opened with MDB_INTEGERKEY, using a single
cursor with MDB_APPEND flag. The benchmark includes 2 warmup iterations and 3
measurement iterations of 120 seconds each.
==============================================
ANALYSIS
==============================================
Reviewing the commit history, the regressions appear linked to correctness
fixes:
- 0.9.19: ITS#8406 (xcursor fixup after cursor_del)
- 0.9.20: ITS#8557 (cursor_last and C_EOF handling)
Both issues added cursor state management overhead in write-heavy code paths.
==============================================
REQUEST
==============================================
Could you please reproduce this regression using a native C benchmark to
confirm it's not JNI-specific? If confirmed, this may be a necessary
performance/correctness tradeoff given that most LMDB workloads are read-heavy.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9739
Issue ID: 9739
Summary: Undefined reference to ber_sockbuf_io_udp in 2.6.0
Product: OpenLDAP
Version: 2.6.0
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: build
Assignee: bugs(a)openldap.org
Reporter: simon.pichugin(a)gmail.com
Target Milestone: ---
While I was trying to build OpenLDAP 2.6 on Fedora Rawhide I've got the error
message:
/usr/bin/ld: ./.libs/libldap.so: undefined reference to
`ber_sockbuf_io_udp'
I've checked commits from https://bugs.openldap.org/show_bug.cgi?id=9673 and
found that 'ber_sockbuf_io_udp' was not added to
https://git.openldap.org/openldap/openldap/-/blob/master/libraries/liblber/…
I've asked on the project's mailing list and got a reply:
"That symbol only exists if OpenLDAP is built with LDAP_CONNECTIONLESS
defined, which is not a supported feature. Feel free to file a bug report
at https://bugs.openldap.org/"
https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.org/…
Hence, creating the bug.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10407
Issue ID: 10407
Summary: unable to load TLS module
Product: OpenLDAP
Version: 2.6.10
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: build
Assignee: bugs(a)openldap.org
Reporter: 953361205(a)qq.com
Target Milestone: ---
Created attachment 1094
--> https://bugs.openldap.org/attachment.cgi?id=1094&action=edit
detail log
The version of openssl is 3.0.2 in my server.
I wanted to install openldap with TLS support. So I entered the command
"./configure --with-tls=openssl" under the path of openldap project folder,
then I got the following error:
...
checking for hstrerror... yes
checking for getaddrinfo... yes
checking for getnameinfo... yes
checking for gai_strerror... yes
checking for inet_ntop... yes
checking INET6_ADDRSTRLEN... yes
checking struct sockaddr_storage... yes
checking for sys/un.h... yes
checking for openssl/ssl.h... yes
checking for SSL_CTX_set_ciphersuites in -lssl... no
configure: error: Could not locate TLS/SSL package
---------------------------------------------------------------
I also attached the detail log. Please tell me how to deal with the problem.
Looking forward to replying.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10026
Issue ID: 10026
Summary: Refresh handling can skip entries (si_dirty not
managed properly)
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
Take MPR plain syncrepl with 3+ providers.
When a provider's own syncrepl session transitions to persist and a it starts a
new parallel session towards another host, that session always has to start as
a refresh. If that refresh serves entries to us, our handling of si_dirty is
not consistent:
- if the existing persist session serves some of these entries to us, we can
"forget" to pass the others to a newly connected consumer
- same if the refresh is abandoned and we start refreshing from a different
provider that might be behind what we were being served (again our consumers
could suffer)
- if we restart, si_dirty is forgotten and our consumers suffer even worse
We might need to be told (at the beginning of the refresh?) what the end state
we're going for is, so we can keep si_dirty on until then. And somehow persist
that knowledge in the DB...
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9884
Issue ID: 9884
Summary: Document "set" patterns in ACLs
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: documentation
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
The slapd.access(5) man page has this exceptionally unhelpful line where sets
are concerned:
"The statement set=<pattern> is undocumented yet."
Even if it is an experimental feature, it should still be documented on how it
is meant to operate.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9726
Issue ID: 9726
Summary: Admin guide and man pages need better documentation on
disabling syslog
Product: OpenLDAP
Version: 2.6.0
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: documentation
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
2.6.0 added the new feature allowing using a logfile for all debug/loglevel
messages and bypassing syslog entirely. However, there is no documentation on
the new settings or examples of how to do this in the admin guide, and the man
page sections on the new parameters for the logfile side do not note at
when/how they enable bypassing syslog.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10405
Issue ID: 10405
Summary: freebsd package missing mdb module in openldap server
2.6.10
Product: OpenLDAP
Version: 2.6.10
Hardware: All
OS: FreeBSD
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: enda(a)cronnolly.com
Target Milestone: ---
When installing openldap server on FreeBSD 14.3, the package:
openldap26-server-2.6.10 Open source LDAP server implementation
Has the following shared objects in the module folder:
ls /usr/local/libexec/openldap/back_*.so
/usr/local/libexec/openldap/back_asyncmeta.so
/usr/local/libexec/openldap/back_null.so
/usr/local/libexec/openldap/back_dnssrv.so
/usr/local/libexec/openldap/back_passwd.so
/usr/local/libexec/openldap/back_ldap.so
/usr/local/libexec/openldap/back_sock.so
/usr/local/libexec/openldap/back_meta.so
slapd.conf references these modules:
# Load dynamic backend modules:
modulepath /usr/local/libexec/openldap
moduleload back_mdb
# moduleload back_ldap
but back_mdb is missing from the installed modules.
There appears to be no other way to install this module through the package
system and the only way to run openldap server 2.6.10 is to build it from ports
instead of using the broken package.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10313
Issue ID: 10313
Summary: 3-way multimaster oathHOTPCounter attribute update
code missing
Product: OpenLDAP
Version: 2.6.6
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: agrru01(a)gmail.com
Target Milestone: ---
I posted on openldap technical mail list and got a response saying I should
file a feature request.
I am using a 3-way multimaster syncrepl setup with the slapo-otp module. My
problem is that when authenticating with a user using HOTP, the attribute
oathHOTPCounter only updates the value on the target ldap instance. This means
the other two ldap instances do not get the updated HOTP-counter value and
therefore will allow authentication using the same HOTP code.
Interestingly enough, if I manually edit the oathHOTPCounter value it
synchronizes with the other masters.
Please see the technical mail list discussion:
https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.org/…
--
You are receiving this mail because:
You are on the CC list for the issue.