https://bugs.openldap.org/show_bug.cgi?id=10395
Issue ID: 10395
Summary: Support multiple readers on uncommitted changes
Product: LMDB
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: liblmdb
Assignee: bugs(a)openldap.org
Reporter: renault.cle(a)gmail.com
Target Milestone: ---
Created attachment 1086
--> https://bugs.openldap.org/attachment.cgi?id=1086&action=edit
A patch to support multiple readers on uncommitted changes
Hello,
The attached patch is not meant to be merged immediately into LMDB. Still, it
demonstrates how I added a helpful feature to the key-value store: reading
uncommitted changes from multiple transactions. I am conscious that the patch
still requires some work and uses non-C99 features, i.e., atomics are C11,
which could be a blocker for it to be merged upstream. I would also be
delighted to merge these changes under a flag/define to ensure we don't impact
other users with non-C99 stuff.
The main feature we need at Meilisearch is to read uncommitted changes from
multiple threads, compute parallel post-processing of different data structures
[1], and speed up the search requests. We could have done the post-processing
in a following transaction by opening multiple read transactions, but this
would mean that the post-processed data structure would not include newly
inserted or modified document IDs. Both data structures would be desync.
Regarding the design choice, I decided to follow the same design as the nested
write transactions: use the parent argument of the mdb_txn_begin [2], and allow
the MDB_RDONLY flag, which was disallowed when the parent argument was non-NULL
[3]. I find it clear enough that, by calling the mdb_txn_begin function with
these arguments, you can call it multiple times (I need to update the doc) to
obtain nested read-only transactions from the parent write transaction.
ret = mdb_txn_begin(env, parent_txn, MDB_RDONLY, new_nested_rtxn);
Note that this early proposal lacks security and error handling. The generated
transactions are fake-read-only and actually write transactions that share the
underlying parent allocations and data structures. This is unsafe and must be
changed or reviewed carefully, but most importantly, we need to add read-only
capabilities to these transactions to disallow writes. Using a Rust wrapper on
top of LMDB, I wrapped the fake read-only transactions into ReadTxn, which
disallows any writes at compile time. However, I haven't checked the conflict
database creations or openings.
The main issues I encountered were concurrent free of the main shared data
structures when the different threads owning the transactions were dropping the
transactions simultaneously. So, I decided to implement the equivalent of an
ARC to free resources only when the last nested transaction was freed.
I can share numbers about how this feature improves the post-processing by
4x-9x or from 1200s to 120s [1]. You can look at this PR, which I would be
happy to merge once an improved version of this patch lands on LMDB upstream.
I would be very happy if you could guide me a bit on how I could improve this
patch to make it mergeable into LMDB. We want to contribute useful features
like this to LMDB and not keep a deviant fork. LMDB works great; we are happy
about it, and its performance is predictable.
Have a lovely week,
kero
[1]: https://github.com/meilisearch/meilisearch/pull/5307
[2]:
https://github.com/LMDB/lmdb/blob/14d6629bc8a9fe40d8a6bee1bf71c45afe7576b6/…
[3]:
https://github.com/LMDB/lmdb/blob/14d6629bc8a9fe40d8a6bee1bf71c45afe7576b6/…
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10474
Issue ID: 10474
Summary: Update contributing page to note no AI generated code
Product: website
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: website
Assignee: bugs(a)openldap.org
Reporter: quanah(a)openldap.org
Target Milestone: ---
Need to update the contributing page to note that we do not accept AI generated
code in the project.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10394
Issue ID: 10394
Summary: remove stroeder.com from support page
Product: website
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: website
Assignee: bugs(a)openldap.org
Reporter: michael(a)stroeder.com
Target Milestone: ---
HI!
for health reasons I gave up my business and I have retired . So it does no
longer make sense to be listed on the support page.
Ciao, Michael.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9967
Issue ID: 9967
Summary: Please register my company on the support page, again
(www.openldap.org)
Product: website
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: website
Assignee: bugs(a)openldap.org
Reporter: sjsong(a)aboutdap.kr
Target Milestone: ---
Howard Chu <hyc(a)openldap.org>
송상준 <sjsong(a)aboutdap.kr>
Hi,
you can submit a ticket against "website" in the ITS for this. Thanks.
송상준 wrote:
>
> Hello. openldap page administrater.
> My name is sang jun song.
>
> I want to register my company on the support page.
> I registered before, but it seems to have disappeared.
> Our company has been attending ldapcon since 2015.
>
> The sysmas employees who attended the LDAPCON in 2019 may remember me.
>
> Please register my company on the support page. Please contact me if you need additional information..
>
> thank you.
>
> Song.
>
> Registration phrase
>
> Seojindsa Co., Ltd. (Aboutdap Co., Ltd.)- Korea
>
> Provides consultancy, development, training and user support for OpenLDAP software in Korea.
>
> URL : seojindsa : www.seojindsa.kr
>
> ------------------------------------------------------
> (주) 어바웃답 기술영업팀 송상준 부장
> TEL. 010-9780-6746
> email: sjsong(a)aboutdap.kr
> homepage: www.aboutdap.kr
> -------------------------------------------------------
>
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10403
Issue ID: 10403
Summary: Add a configuration directive that uses the OpenSSL
CONF API to allow openldap config files to set any
configuration supported by that API, and to get new
OpenSSL configuration capabilities through that API
with no changes in openldap.
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: slapd
Assignee: bugs(a)openldap.org
Reporter: stephen.wall(a)redcom.com
Target Milestone: ---
Created attachment 1090
--> https://bugs.openldap.org/attachment.cgi?id=1090&action=edit
Add a configuration directive that uses the OpenSSL CONF API to allow openldap
config files to set any configuration supported by that API, and to get new
OpenSSL configuration capabilities through th
I am submitting a patch to create a new directive for OpenLDAP config files
that uses the OpenSSL SSL_CONF API to allow configuration of any aspect of
OpenSSL that the API supports without adding specific directives to OpenLDAP
for them. When OpenSSL extends that API, all versions of OpenLDAP with thie
patch will also support those extensions with no additional code.
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9717
Issue ID: 9717
Summary: The RADIUSOV overlay can be incorporated into OpenLDAP
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: contrib
Assignee: bugs(a)openldap.org
Reporter: rdubner(a)symas.com
Target Milestone: ---
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=8617
Ondřej Kuzník <ondra(a)mistotebe.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|nivanova(a)symas.com |bugs(a)openldap.org
Target Milestone|2.7.0 |2.7.1
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=9009
Howard Chu <hyc(a)openldap.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Assignee|bugs(a)openldap.org |hyc(a)openldap.org
--
You are receiving this mail because:
You are on the CC list for the issue.
https://bugs.openldap.org/show_bug.cgi?id=10473
Issue ID: 10473
Summary: syncprov leaks response controls
Product: OpenLDAP
Version: unspecified
Hardware: All
OS: All
Status: UNCONFIRMED
Keywords: needs_review
Severity: normal
Priority: ---
Component: overlays
Assignee: bugs(a)openldap.org
Reporter: ondra(a)mistotebe.net
Target Milestone: ---
syncprov_state_ctrl and syncprov_done_ctrl add controls to the response, but
noone cares to free them afterwards assuming they are allocated from the slab.
If there are many messages sent in a single run, this takes up fresh memory
that's leaked eventually.
See its8800 regression test for an example where this happens.
--
You are receiving this mail because:
You are on the CC list for the issue.