https://bugs.openldap.org/show_bug.cgi?id=9278
Issue ID: 9278
Summary: liblmdb: robust mutexes should not be unmapped
Product: LMDB
Version: unspecified
Hardware: All
OS: FreeBSD
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: delphij(a)freebsd.org
Target Milestone: ---
Created attachment 736
--> https://bugs.openldap.org/attachment.cgi?id=736&action=edit
A possible workaround
We recently noticed that lmdb would have the memory region containing the
robust mutex unmapped on mdb_env_close0():
munmap((void *)env->me_txns,
(env->me_maxreaders-1)*sizeof(MDB_reader)+sizeof(MDB_txninfo));
Note that if this is the last unmap for a robust mutex, the FreeBSD
implementation would garbage-collect the mutex, making it no longer visible to
other processes. As the result, a second instance of the attached test.c (from
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244493 with minor changes)
would trigger the assertion at mdb_txn_begin() because the acquisition of the
mutex would return 22 (EINVAL), because the mutex appeared to be a robust
mutex, but was invalid.
The attached lmdb.diff is a possible workaround for this (it would skip
unmapping when setting up the robust mutex for the first time).
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9528
Issue ID: 9528
Summary: liblmdb throws away environment CFLAGS
Product: LMDB
Version: 0.9.29
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
When building LMDB, specified hardening flags are thrown away due to the
Makefile overriding CFLAGS:
Makefile:CFLAGS = $(THREADS) $(OPT) $(W) $(XCFLAGS)
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9378
Issue ID: 9378
Summary: Crash in mdb_put() / mdb_page_dirty()
Product: LMDB
Version: 0.9.26
Hardware: All
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: nate(a)kde.org
Target Milestone: ---
The KDE Baloo file indexer uses lmdb as its database (source code available at
https://invent.kde.org/frameworks/baloo). Our most common crash, with over 100
duplicate bug reports, is in lmdb. Here's the bug report tracking it:
https://bugs.kde.org/show_bug.cgi?id=389848.
The version of lmdb does not seem to matter much. We have bug reports from Arch
users with lmdb 0.9.26 as well as bug reports from people using many earlier
versions.
Here's an example backtrace, taken from
https://bugs.kde.org/show_bug.cgi?id=426195:
#6 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#7 0x00007f3c0bbb9859 in __GI_abort () at abort.c:79
#8 0x00007f3c0b23ba83 in mdb_assert_fail (env=0x55e2ad710600,
expr_txt=expr_txt@entry=0x7f3c0b23e02f "rc == 0",
func=func@entry=0x7f3c0b23e978 <__func__.7221> "mdb_page_dirty",
line=line@entry=2127, file=0x7f3c0b23e010 "mdb.c") at mdb.c:1542
#9 0x00007f3c0b2306d5 in mdb_page_dirty (mp=<optimized out>,
txn=0x55e2ad7109f0) at mdb.c:2114
#10 mdb_page_dirty (txn=0x55e2ad7109f0, mp=<optimized out>) at mdb.c:2114
#11 0x00007f3c0b231966 in mdb_page_alloc (num=num@entry=1,
mp=mp@entry=0x7f3c0727aee8, mc=<optimized out>) at mdb.c:2308
#12 0x00007f3c0b231ba3 in mdb_page_touch (mc=mc@entry=0x7f3c0727b420) at
mdb.c:2495
#13 0x00007f3c0b2337c7 in mdb_cursor_touch (mc=mc@entry=0x7f3c0727b420) at
mdb.c:6523
#14 0x00007f3c0b2368f9 in mdb_cursor_put (mc=mc@entry=0x7f3c0727b420,
key=key@entry=0x7f3c0727b810, data=data@entry=0x7f3c0727b820,
flags=flags@entry=0) at mdb.c:6657
#15 0x00007f3c0b23976b in mdb_put (txn=0x55e2ad7109f0, dbi=5,
key=key@entry=0x7f3c0727b810, data=data@entry=0x7f3c0727b820,
flags=flags@entry=0) at mdb.c:9022
#16 0x00007f3c0c7124c5 in Baloo::DocumentDB::put
(this=this@entry=0x7f3c0727b960, docId=<optimized out>,
docId@entry=27041423333263366, list=...) at ./src/engine/documentdb.cpp:79
#17 0x00007f3c0c743da7 in Baloo::WriteTransaction::replaceDocument
(this=0x55e2ad7ea340, doc=..., operations=operations@entry=...) at
./src/engine/writetransaction.cpp:232
#18 0x00007f3c0c736b16 in Baloo::Transaction::replaceDocument
(this=this@entry=0x7f3c0727bc10, doc=..., operations=operations@entry=...) at
./src/engine/transaction.cpp:295
#19 0x000055e2ac5d6cbc in Baloo::UnindexedFileIndexer::run
(this=0x55e2ad79ca20) at
/usr/include/x86_64-linux-gnu/qt5/QtCore/qrefcount.h:60
#20 0x00007f3c0c177f82 in QThreadPoolThread::run (this=0x55e2ad717f20) at
thread/qthreadpool.cpp:99
#21 0x00007f3c0c1749d2 in QThreadPrivate::start (arg=0x55e2ad717f20) at
thread/qthread_unix.cpp:361
#22 0x00007f3c0b29d609 in start_thread (arg=<optimized out>) at
pthread_create.c:477
#23 0x00007f3c0bcb6103 in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9513
Issue ID: 9513
Summary: Enhanced debug output
Product: OpenLDAP
Version: 2.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: enhancement
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: hyc(a)openldap.org
Target Milestone: ---
While debugging ITS#9479 I needed more information on which threads were doing
what actions. Also, when looking for delays/hangs/race conditions, the 1-second
granularity of the debug msg timestamp was inadequate. And also, some of the
debug output for the Search operation was being done in separate Debug calls
for a single line of output. This gets jumbled up pretty badly if other threads
are printing output as well.
New patch set will extend the debug timestamp to have fractional seconds too;
it will use clock_gettime for nanosecond resolution if available, otherwise use
gettimeofday for microsecond resolution. Also tweak the Search debug output to
only use one Debug invocation per line of output.
This is MR!279.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9584
Issue ID: 9584
Summary: cn=config replication ops/refresh should pause server
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
Looking into this crash: https://git.openldap.org/openldap/openldap/-/jobs/7286
The thread in question is running a plain syncrepl refresh while another thread
seems to have done the same. This thread fetched the entryUUID attribute of the
'cn=config' entry as 'a' and in the meantime, that entry has been rewritten,
with 'a' presumably cleaned up and returned to the pool, so addressing
a->a_nvals[0] is a NULL-dereference now.
This might or might not be related to the fix in ITS#8102.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9652
Issue ID: 9652
Summary: Add "tee" capability to load balancer
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: lloadd
Assignee: bugs(a)openldap.org
Reporter: mhardin(a)symas.com
Target Milestone: ---
This is a request for an enhancement that would add a "tee" or "fan-out"
capability to load balancer, where received operations are sent to two or more
destinations simultaneously.
The primary goal or the enhancement is to make it possible to keep multiple
independent and likely dissimilar directory systems in lock-step with each
other over hours, days, or possibly even weeks.
The enhancement would not necessarily need to include a mechanism for
converging the target systems should they become out of sync.
This is not intended to be a replication solution, rather it is viewed more as
a "copy" solution intended to be used for specific short-term tasks that need
multiple directory systems to be exactly synchronized but where replication is
not desirable or even possible.
At least two uses come to mind:
1. Test harnesses, evaluating side-by-side operation of separate directory
systems over time
2. Directory system transition validation harnesses
3. (maybe) Part of a test harness to record or replay LDAP workloads
* Other uses?
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9303
Issue ID: 9303
Summary: Add support for WolfSSL as an alternative to OpenSSL
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: libraries
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
For OpenLDAP 2.6, we should investigate adding support for WolfSSL as an
alternative to OpenSSL.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9220
Bug ID: 9220
Summary: Rewrite Bind and Exop result handling
Product: OpenLDAP
Version: 2.5
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
Bind and Exop result handling needs a rewrite so it is no longer a special case
for overlays.
--
You are receiving this mail because:
You are on the CC list for the bug.
https://bugs.openldap.org/show_bug.cgi?id=9564
Issue ID: 9564
Summary: Race condition with freeing the spilled pages from
transaction
Product: LMDB
Version: 0.9.29
Hardware: Other
OS: Mac OS
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: kriszyp(a)gmail.com
Target Milestone: ---
Created attachment 825
--> https://bugs.openldap.org/attachment.cgi?id=825&action=edit
Free the spilled pages and dirty overflows before unlocking the write mutex
The spilled pages (a transaction's mt_spill_pgs) is freed *after* a write
transaction's mutex is unlocked (in mdb.master3). This can result in a race
condition where a second transaction can start and subsequently assign a new
mt_spill_pgs list to the transaction structure that it reuses. If this occurs
after the first transaction unlocks the mutex, but before it performs the free
operation on mt_spill_pgs, then the first transaction will end up calling a
free on the same spilled page list as the second transaction, resulting in a
double free (and crash).
It would seem to be an extremely unlikely scenario to actually happen, since
the free call is literally the next operation after the mutex is unlocked, and
the second transaction would need to make it all the way to the point of saving
the freelist before a page spill list is likely to be allocated. Consequently,
this probably has rarely happened. However, one of our users (see
https://github.com/DoctorEvidence/lmdb-store/issues/48 for the discussion) has
noticed this occurring, and it seems that it may be particularly likely to
happen on MacOS on the new M1 silicon. Perhaps there is some peculiarity to how
the threads are more likely to yield execution after a mutex unlock, I am not
sure. I was able to reproduce the issue by intentionally manipulating the
timing (sleeping before the free) to verify that the race condition is
technically feasible, and apparently this can happen "in the wild" on MacOS, at
least with an M1.
It is also worth noting that this is due to (or a regression from) the fix for
ITS#9155
(https://github.com/LMDB/lmdb/commit/cb256f409bb53efeda4ac69ee8165a0b4fc1a277)
where the free call was moved outside the conditional (for having a parent)
that had previously never been executed after the mutex is unlocked, but now is
called after the unlock.
Anyway, the solution is relatively simple, the free call simply has to be moved
above the unlock. In my patch, I also moved the free call for mt_dirty_ovs. I
am not sure what OVERFLOW_NOTYET/mt_dirty_ovs is for, but presumably it should
be handled the same. This could alternately be solved by saving the reference
to these lists before unlocking, and freeing after unlocking, which would
slightly decrease the amount of processing within the mutex guarded code. Let
me know if you prefer a patch that does that.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9245
Bug ID: 9245
Summary: devel/programming needs rewrite
Product: website
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Severity: normal
Priority: ---
Component: website
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
The information on the devel/programming web page is at least a decade out of
date and needs a complete overhaul.
--
You are receiving this mail because:
You are on the CC list for the bug.