openldap-bugs June 2021

openldap-bugs@openldap.org

1 participants
315 discussions

[Issue 9027] LMDB: API to retrieve map address
by openldap-its＠openldap.org 05 Jun '21

05 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9027 --- Comment #3 from doug(a)hcsw.org --- Hi! I would like to voice my support for this feature, and explain how this API would help me. First of all, a bit of background: I'm maintaining C++ bindings for LMDB (https://github.com/hoytech/lmdbxx), and have been a contributor to the perl bindings and others. I have published two interface layers that use LMDB, https://github.com/hoytech/quadrable and https://github.com/hoytech/rasgueadb and as my colleagues would attest, am a relentless LMDB advocate. Application 1: Testing zero-copy That previous link, RasgueaDB, is an indexing and query layer for LMDB. It uses flatbuffers for its data format, which allows zero-copy access to fields. Since there are several layers involved, I wanted to make sure that no copying was happening, and also to add a test to the test-suite to make sure this remains the case. To do this, I have a library called assert_zerocopy: https://github.com/hoytech/hoytech-cpp/blob/master/hoytech/assert_zerocopy.h (and also a perl equivalent here: https://metacpan.org/pod/Test::ZeroCopy ) However, in order to verify the returned value is actually a zero-copy reference, I need the pointer to the memory map (and the map's size). I have added a terrible hack to the lmdbxx bindings to retrieve this: https://github.com/hoytech/lmdbxx/blob/08eddafcc4613c7fc8ebd88f5db87c7d7bfb… But this is obviously not a good approach (as mentioned by Nic in this thread) and it would be much better if there was an API to retrieve this value (like there is for me_mapsize). Application 2: Virtual memory controls In general I agree that mlock()ing large files like databases is counter-productive. However, there are other controls that can be applied to the virtual memory of an application. For example, rsync attempts to preserve the filesystem cache state so that rsync invocations have as small an impact as possible on the cache (since it's a shared resource): https://insights.oetiker.ch/linux/fadvise/ Another example, before restarting a server, Instagram uses my utility vmtouch to snapshot the virtual memory state (basically the set of "hot" files). After rebooting, the state is restored before the server is added back to the active pool of servers. I have generalised this with my application/library vmprobe: https://vmprobe.com/filesystem-cache If I had access to me_map within my application, I would be able to integrate libvmprobe more easily and portably. For instance, this would allow me to take a snapshot of which pages are resident in memory, perform some large read query that touches many pages, then restore the original VM page residency set after the query completes. Another variant of this: If you have multiple customers who use the same server (at different times) then you can restore a customer's previous VM state when they login. If you know approximately the pages that will be accessed in advance, sequentially pre-paging them in can have significant performance benefits. I did some experiments with Postgres pre-paging you can see here: https://vmprobe.com/database-speedup Conclusion Although certainly a user could abuse me_map if it were exposed, one of the reasons why I enjoy LMDB so much is that it generally assumes its users know what they are doing and doesn't compromise on flexibility or performance for the sake of users who don't. For example, although it is not recommended to use MDB_WRITEMAP in most cases, it nevertheless exists. As explained above, I have several use-cases for me_map that I feel are legitimate. I am already accessing it in a hacky way but naturally would prefer a more portable and future-proof API. If this API is added, the documentation should of course make clear that accessing or modifying (if MDB_WRITEMAP) data through me_map is not supported in any way, and regular use-cases have no need for this value and should stick to the supported APIs. Thank you! -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 9570] New: Roadmap needs updating now that OpenLDAP 2.5 is released
by openldap-its＠openldap.org 04 Jun '21

04 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9570 Issue ID: 9570 Summary: Roadmap needs updating now that OpenLDAP 2.5 is released Product: website Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: website Assignee: bugs(a)openldap.org Reporter: quanah(a)openldap.org Target Milestone: --- Need to update the roadmap. -- You are receiving this mail because: You are on the CC list for the issue.

1 3

[Issue 9557] New: syncrepl refreshonly sometimes finishes too early
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9557 Issue ID: 9557 Summary: syncrepl refreshonly sometimes finishes too early Product: OpenLDAP Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Keywords: replication Severity: normal Priority: --- Component: slapd Assignee: bugs(a)openldap.org Reporter: ondra(a)mistotebe.net Target Milestone: --- In a delta-MMR test environment where conflicts are intentionally introduced, the fallback sessions sometimes cut off, not sending all entries they ought to. Instead LDAP_SUCCESS is sent with the snapshot cookie at some point. That cookie represents the correct contextCSN at the time of the search, so the missing changes are skipped over forever. As of now, the cause of the cutoff is under investigation. Currently, this was only run with 2.5 + persistent sessionlog enabled but there are indications this might not be limited to 2.5, tests with other versions/configs are pending. -- You are receiving this mail because: You are on the CC list for the issue.

1 7

[Issue 9551] New: dnSubtreeMatch and others do not handle empty DN as asserted value
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9551 Issue ID: 9551 Summary: dnSubtreeMatch and others do not handle empty DN as asserted value Product: OpenLDAP Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: slapd Assignee: bugs(a)openldap.org Reporter: ondra(a)mistotebe.net Target Milestone: --- AFAIK '(reqDN:dnSubtreeMatch:=)' should be equivalent to 'reqDN=*', however it only seems to match the empty dn right now. -- You are receiving this mail because: You are on the CC list for the issue.

1 4

[Issue 9537] New: slap_timestamp() can give a duplicated timestamp across restarts
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9537 Issue ID: 9537 Summary: slap_timestamp() can give a duplicated timestamp across restarts Product: OpenLDAP Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: slapd Assignee: bugs(a)openldap.org Reporter: ondra(a)mistotebe.net Target Milestone: --- On busy sites, when a slapd restart takes <1s, accesslog can fail to log changes with LDAP_ALREADY_EXISTS. This is because slap_timestamp() only logs timestamps with a 1s precision, disambiguating the rest with a counter that's forgotten across restarts. It is possible my analysis in ITS#9487 is partially invalidated because of this. -- You are receiving this mail because: You are on the CC list for the issue.

1 7

[Issue 9526] New: slapadd -w crashes
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9526 Issue ID: 9526 Summary: slapadd -w crashes Product: OpenLDAP Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: slapd Assignee: bugs(a)openldap.org Reporter: grapvar(a)gmail.com Target Milestone: --- Let slapd.conf is: > database mdb > suffix "o=Foo" > sync_use_subentry database is blank and we are adding this foo.ldif: > dn: o=FOO > objectClass:organization Let's load: > slapd -T add -v -l foo.ldif -w then on Solaris: > added: "o=FOO" (00000001) > Segmentation Fault (core dumped) ... on Linux: > added: "o=FOO" (00000001) > => mdb_next_id: get failed: Invalid argument (22) > => mdb_tool_next_id: next_id failed: Invalid argument (22) > => mdb_tool_entry_put: txn_aborted! Invalid argument (22) > slapadd: couldn't create context entry > Closing DB... This is because: * mdb_tool_next_id() takes dead global [tools.c`static MDB_cursor *mcp] for further operations * cursor is dead because mdb_tool_entry_put() didn't initialized it * mdb_tool_entry_put() didn't initialized cursor because it thinks it is initialized, because there is an active global [tools.c`MDB_txn *mdb_tool_txn] * transaction was initialized by mdb_tool_dn2id_get(), which doesn't care about cursors. Long story short: the global state in tools.c is not managed consistently and needs rethinking. -- You are receiving this mail because: You are on the CC list for the issue.

1 9

[Issue 6467] syncrepl enhancements
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=6467 Quanah Gibson-Mount <quanah(a)openldap.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |VERIFIED -- You are receiving this mail because: You are on the CC list for the issue.

1 0

[Issue 9552] New: slapo-accesslog should record new DN after rename
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9552 Issue ID: 9552 Summary: slapo-accesslog should record new DN after rename Product: OpenLDAP Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: overlays Assignee: bugs(a)openldap.org Reporter: ondra(a)mistotebe.net Target Milestone: --- To use accesslog as a sessionlog source, one needs to be able to resolve whether a modrdn moved an entry in/out of the search scope. Syncprov would either have to examine every modrdn request or, if accesslog were to log the final DN, it could check for entries which crossed the scope. A new attribute 'reqNewDN' should be tracked for modrdn ops. -- You are receiving this mail because: You are on the CC list for the issue.

1 4

[Issue 9529] New: pcache locking issue
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9529 Issue ID: 9529 Summary: pcache locking issue Product: OpenLDAP Version: 2.4.58 Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: overlays Assignee: bugs(a)openldap.org Reporter: hyc(a)openldap.org Target Milestone: --- Since ITS#6954 commit ea228495148 the consistency_check function was changed to hold the template t_rwlock for the entire duration of a query expiration. There doesn't appear to be any valid reason for this change, and it causes the cache to be unresponsive to new searches while expiration is removing cached entries. -- You are receiving this mail because: You are on the CC list for the issue.

1 6

[Issue 9295] New: ppolicy and replication: pwdLockedTime replication fails to replicate
by openldap-its＠openldap.org 03 Jun '21

03 Jun '21

https://bugs.openldap.org/show_bug.cgi?id=9295 Issue ID: 9295 Summary: ppolicy and replication: pwdLockedTime replication fails to replicate Product: OpenLDAP Version: 2.4.50 Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: --- Component: overlays Assignee: bugs(a)openldap.org Reporter: quanah(a)openldap.org Target Milestone: --- If you have the following setup, a replica will hit an error during replication. a) ppolicy is configured on provider(s) and replicas. Replica has schemachecking=on in its syncrepl configuration b) account gets locked on the replica, so pwdAccountLockedTime is set on the replica but not on the provider(s) c) admin does a MOD/ADD op against a provider for the user entry to add a value to pwdAccountLockedTime dn: ... changetype: modify add: pwdAccountLockedTime pwdAccountLockedTime: ... d) provider accepts this modification. e) replica rejects this modification because the resulting change means that there would be two pwdAccountLockedTime values on the account in question Generally I believe that in this scenario, the MOD/ADD on the provider should be treated as a replace OP instead of an ADD op -- You are receiving this mail because: You are on the CC list for the issue.

1 18

← Newer
1
...
22
23
24
25
26
27
28
...
32
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs June 2021