https://bugs.openldap.org/show_bug.cgi?id=10241
Issue ID: 10241
Summary: Crash in mdb_page_search_root()
Product: LMDB
Version: 0.9.24
Hardware: x86_64
OS: Linux
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: david.komarek(a)whalebone.io
Target Milestone: ---
Hello,
The LMDB is crashing in mdb_page_search_root() on following instruction
`0x7f85221479d2 movzwl 0xa(%rdx),%eax`. This instruction corresponds to
following line in source code -
https://github.com/LMDB/lmdb/blob/LMDB_0.9.24/libraries/liblmdb/mdb.c#L5485
(while access mp_flags in MDB_page structure)
Here is callstack as generated by core dump (without symbols as we're using
package from Ubuntu repositories)
```
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f85221479d2 in ?? () from /usr/lib/x86_64-linux-gnu/liblmdb.so.0
No symbol table info available.
#1 0x00007f8522147d15 in ?? () from /usr/lib/x86_64-linux-gnu/liblmdb.so.0
No symbol table info available.
#2 0x00007f8522148432 in ?? () from /usr/lib/x86_64-linux-gnu/liblmdb.so.0
No symbol table info available.
#3 0x00007f8522148a70 in mdb_get () from
/usr/lib/x86_64-linux-gnu/liblmdb.so.0
No symbol table info available.
```
Translation
```
#0 mdb_page_search_root()
#1 mdb_page_search()
#2 mdb_cursor_set()
#3 mdb_get()
```
Registers in time of crash:
```
rax 0x2 2
rbx 0x7ffcd8d2a100 140723946168576
rcx 0x3 3
rdx 0x7f825baf7000 140197860700160
rsi 0x1000 4096
rdi 0x7f851c00f4d0 140209677202640
rbp 0x0 0x0
rsp 0x7ffcd8d29e40 0x7ffcd8d29e40
r8 0x7ffcd8d29e50 140723946167888
r9 0x7f851c00f5b8 140209677202872
r10 0x7f851c003090 140209677152400
r11 0x2ce33e6c02ce33e7 3234497591006606311
r12 0x0 0
r13 0x7ffcd8d2a510 140723946169616
r14 0x79 121
r15 0x7f851c003098 140209677152408
rip 0x7f85221479d2 0x7f85221479d2
eflags 0x10246 [ PF ZF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
k0 0x40 64
k1 0xfffff0f0 4294963440
k2 0xff01 65281
k3 0xffffffff 4294967295
k4 0xffffffff 4294967295
k5 0xffffffff 4294967295
k6 0xffffffff 4294967295
k7 0x0 0
```
Our setup is following:
We have single process (running in separate container) which reads and writes
to LMDB. It opens environment with following flags
MDB_NORDAHEAD|MDB_WRITEMAP|MDB_NOTLS|MDB_NOSYNC. The environment contain
several DBs. All DBIs are open with MDB_CREATE flag. Transactions are open
without flags.
On the other hand we have several processes running in the single container,
which are only allowed read (these processes crash). The environment is open
with following flags MDB_NORDAHEAD|MDB_WRITEMAP|MDB_NOTLS|MDB_RDONLY. All DBIs
are open without flags. Transactions are open with MDB_RDONLY flag.
Could you please investigate it? If you will need some other artifacts or
comments, please let me know.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10240
Issue ID: 10240
Summary: Information required about the end of support date for
OpenLDAP ver 2.6.3
Product: OpenLDAP
Version: 2.6.3
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: documentation
Assignee: bugs(a)openldap.org
Reporter: bluesoulprince(a)gmail.com
Target Milestone: ---
Hi Team,
As we have integrated OpenLDAP 2.6.3 version in our application, we would like
to know the community support availability end of date for this version, to
plan our application maintenance accordingly.
Could you please help us with this information.
Thanks,
Vivek S
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10236
Issue ID: 10236
Summary: fragmentation makes mdb_page_alloc slow
Product: LMDB
Version: 0.9.31
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: aalekseyev(a)janestreet.com
Target Milestone: ---
Created attachment 1022
--> https://bugs.openldap.org/attachment.cgi?id=1022&action=edit
patch is relative to LMDB_0.9.31
It's a known problem that mdb_page_alloc can be slow
when the free list is large and fragmented. [1] [2] [3]
I'm not sure it's known *how* slow it can be.
In our workload we saw a fragmented freelist leading
to a pathological O(n^2) behavior.
To handle a multi-page allocation we iterate loading chunks of the
free list one by one, and at every iteration we do O(n) work to check
if the allocation can succeed.
Even small-ish allocations (tens of pages) are repeatedly hitting
this edge case, with free list growing to ~1000000, and the outer loop
taking ~2000 iterations (10^9 worth of work in total, just to allocate a
few pages).
Even though I'm sure there are ways to avoid hitting this pathological
scenario so much (avoid values larger than 4k, or fix whatever causes
fragmentation), it seems unacceptable to have a performance cliff this bad.
I made a patch to make the allocation take ~O(n*log(n)), by loading
and merging multiple chunks at once instead of doing it one-by-one.
I'd appreciate it if someone could review the patch (attached), improve it,
and/or come up with an alternative fix.
The code in `midl.c` is kinda meme-y, including a contribution from GPT-4o, but
it performs well enough to speed up our pathological workload by ~20x (which is
still ~3x away from the non-fragmented case).
Anyway, the main thing that warrants scrutiny is the change in `mdb.c`:
I understand very little about lmdb internals and I worry that loading
multiple pages at once instead of one-by-one might break something.
[1] issue #8664
[2]
https://lists.openldap.org/hyperkitty/list/openldap-bugs@openldap.org/threa…
[3]
https://lists.openldap.org/hyperkitty/list/openldap-technical@openldap.org/…
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10239
Issue ID: 10239
Summary: The slapd domain name is inconsistent, synchronization
cannot be modified by the rewrite dn parameter dc.
Product: OpenLDAP
Version: 2.4.44
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: 2458943823(a)qq.com
Target Milestone: ---
/etc/openldap/slapd:
syncrepl rid=002
provider=ldap://172.18.1.1
bindmethod=simple
binddn="cn=Manager,dc=wsc,dc=ls,dc=com"
credentials=lnewnews1tter
searchbase="dc=wsc,dc=ls,dc=com"
schemachecking=off
type=refreshAndPersist
retry="60 +"
rewrite dn "dc=wsc,dc=ls,dc=com" "dc=wsc,dc=slave1,dc=ls,dc=com"
The error is as follows:
66912211 /etc/openldap/slapd.conf: line 281: Error: parse_syncrepl_line: unable
to parse "rewrite"
.
66912211 failed to add syncinfo
slaptest: bad configuration directory!
--
You are receiving this mail because:
You are on the CC list for the issue.