openldap-devel

openldap-devel@openldap.org

4 participants
1416 discussions

ldap_get_values() called on binary data - does this return an error, or garbage data?
by Graham Leggett 03 Apr '24

03 Apr '24

Hi all, Looking back in time to the definitions of ldap_get_values() and ldap_get_values_len(), we are told that "If the attribute values are binary in nature, and thus not suitable to be returned as an array of char *'s, the ldap_get_values_len() routine can be used instead." This has been historically vague - first off, what happens if an attempt is made to call ldap_get_values() on binary data, do you get an error, or garbage data? The source isn't giving me a clear answer. Second question is how do you know which of ldap_get_values() or ldap_get_values_len() to call? Obviously you can manually know this, but I'm interested in automated behaviour. What is the canonical way to discover that if you queried a jpegPhoto (for example) that the result would be binary? To be clear, the end goal it to patch the following documentation to make this clear: https://www.openldap.org/software/man.cgi?query=ldap_get_values&apropos=0&s… Regards, Graham --

4 6

Dynamically managed groups, etc
by Howard Chu 26 Feb '24

26 Feb '24

The recent work on expanding dynamic group functionality in the dynlist overlay seems to have been a bad idea. It makes an already fairly complex overlay even more complicated, and it puts a lot more work into the read side of operations, which adds up to quite noticeable slowdowns in search performance on deployments that make heavy use of dynamic groups with large memberships. It appears that in these situations, the approach used in the autogroup overlay (which has been in contrib since 2007) is better. It moves all of the membership management into the write side of operations, updating memberships whenever dynamic group definitions are modified, or when member entries are written/updated/etc. As such, it allows dynamically defined group memberships to be read/searched at full speed, as if they were static groups. The search performance difference between autogroup and the dynlist approach is pretty drastic when large groups and large numbers of groups are in use. As such I believe autogroup should be recommended, going forward, and we should move it from contrib into the core code. For similar reasons, dynamically managing memberOf attributes in dynlist also has a major impact on search performance. And as alluded to before, adding that functionality to dynlist has made the code quite a bit more complex. Again for performance reasons, it's better to manage this attribute on the write side, updating a real attribute in the DB when groups and memberships are modified, instead of doing the lookup work on the read/search side. As such, we should reverse the decision to deprecate the memberof overlay. There were a couple problems that the overlay presented in replication environments before, that prompted us to deprecate it, but I believe those problems have now been resolved. The first one, referenced in ITS#7400, had to do with the actual memberof attributes getting replicated even though they weren't meant to be. The solution for that should have been to specify that memberof should be excluded from replication using "exattr" in a syncrepl consumer config. The exattr option wasn't behaving as intended in the past, and using it would cause data desyncs, but that problem was fixed long ago. The other problem was simply due to the lack of ordering guarantees in syncrepl refreshes, which prevented the memberof overlay from updating an member's memberof attribute if the group entry got replicated before the member entry. That problem has been solved in ITS#10167, simply adding a check whenever new entries are added, to see if they're already members of any existing groups. As such, the memberof overlay should be perfectly fine for use in replication scenarios now. More testing of those scenarios is welcome. With statically managed member and memberof attributes, the major hits to search performance have been removed. We're still trying dynamically managed nesting of groups though, which is what the new nestgroup overlay (ITS#10161) is for. Again, we need testing to see how much performance impact there is from this much-reduced overhead in the read/search side of things. The config is also a lot easier to understand in its own overlay, instead of shoe-horned into the dynlist config. Testing and feedback appreciated. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 2

LMDB and fsync failures
by Howard Chu 09 Feb '24

09 Feb '24

If anyone remembers fsync-gate https://danluu.com/fsyncgate/ which showed a lot of vulnerabilities in other popular DBMSs, some other research was published on the topic as well https://www.usenix.org/conference/atc20/presentation/rebello I originally discussed this on twitter back in 2020 but wanted to summarize again here. As usual with these types of reports, there are a lot of flaws in their test methodology, which invalidates some of their conclusions. In particular, I question the validity of the failure scenarios their CuttleFS simulator produces. Specifically, they claim that multiple systems exhibit False Failures after fsync reports a failure, but actually (partially) succeeded. In the case of LMDB, where a 1-page synchronous write is involved, this is just an invalid test. They assume that the relevant sector that LMDB cares about is successfully written, but an I/O error occurs on some other sector in the page. And so while LMDB invalidates the commit in memory, a cache flush and subsequent page-in will read the updated sector. But in the real world, if there are hard I/O errors on these other sectors, they will most likely also be unreadable, and a subsequent page-in will also fail. So at least for LMDB, there would be no false failure. The failure modes they're modeling don't reflect reality. Leaving that issue aside, there's also the point that modern storage devices are now using 4KB sectors, and still guarantee atomic sector writes, so the partial success scenario they describe can't even happen. This is a bunch of academic speculation, with a total absence of real world modeling to validate the failure scenarios they presented. The other failures they report, on ext4fs with journaled data, are certainly disturbing. But we always recommend turning that journaling off with LMDB; it's redundant with LMDB's own COW strategy and harms perf for no benefit. Of course, you don't even need to trust the filesystem, you can just use LMDB on a raw block device. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

OpenSSL provider support in openldap: OSSL_STORE_open()
by Graham Leggett 14 Jan '24

14 Jan '24

Hi all, A search in the openldap source shows we don’t yet support the OpenSSL3 provider OSSL_STORE_open() call, which takes a URL as a parameter. I’m happy to patch the openldap client to support this, would it make sense to add a LDAP_OPT_X_TLS_URL option to ldap_option_set()? Regards, Graham —

2 5

SASL Binds and meaning of "users"
by Ondřej Kuzník 19 Apr '23

19 Apr '23

Recently seen a few people assume that authz-regexp search-based mappings enforce that an entry is found or the Bind is failed, which is not the case. Obviously the admin guide[0] should be adjusted not to cause more confusion but the question remains: Should we be able to decide whether an identity should be considered a "user" (Bind succeeds)? Right now, trusting certificates issued by a CA means *all* of them will always be accepted if valid for Binding against the server. Similar with other SASL mechanisms (kerberos, ...) but those tend to have another way of setting up a policy that can be decided for a specific combination of identity+server. It mostly matters for internal ACL purposes, those clients have passed a Bind and so are considered "users", complicating ACL design somewhat. Accepting a SASL Bind for authorisation in external clients is of less value, it can only give a yes/no answer where a more detailed answer is usually needed anyway. It might be possible to make the search-based mappings a policy point, letting a mapping that triggered but failed to map to an entry become an indication that the bind should be denied. This would be considerable change for some deployments out there, while it might protect some that (wrongly) assumed this to be the case already. Obviously "Direct Mappings" would stay unaffected because they should be well understood already[1]. [0]. https://www.openldap.org/doc/admin26/sasl.html#Search-based%20mappings [1]. Admin guide for direct mappings already says "it allows mapping to DNs which refer to entries not held by this server" in the first paragraph -- Ondřej Kuzník Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP

3 2

pcache LDAP_MATCHING_RULE_IN_CHAIN support
by Johan 27 Mar '23

27 Mar '23

Hello all, We have an OpenLDAP instance proxying an active directory with back_meta and mr_passthru. We also have pcache on top, and as it do not support LDAP_MATCHING_RULE_IN_CHAIN, I looked about implementing it. I found that we could retrieve cached results with pcacheQueryID, but I can't get why this is not used in the current code? I dit it this way (querying pcacheQueryID for previsouly cached results) and tests are successful I can now declare templates like this : pcacheAttrset 0 * pcacheTemplate (&(objectClass=)(member:1.2.840.113556.1.4.1941:=)) 0 900 Plesae see attached patch Did I miss something about the use of pcacheQueryID ? or anything else ? Do you think this could be added upstream? P.S.: Is there a reason mr_passthru is not included to OpenLDAP ? not even in contrib ? Thanks for reading

3 4

multi-modify
by Howard Chu 06 Dec '22

06 Dec '22

One of the features Symas built into its Connexitor software (back in 2000, based on OpenLDAP 2.0) was the ability to perform modifications inline with a search request. Back then we used back-perl tweaked heavily to accomplish it. Having an ability to do batch modifies would be pretty useful in general. It's a feature that's commonly used in SQL as well. One way to do this in modern LDAP would be to define a batchModify control, to be sent with a Search request. The controlValue would contain a sequence of modOps, like a Modify request, but without the rest of the request body. For each entry returned by the Search request, the modOps would be applied to the entry before returning it. A response control would be attached to each entry, giving the result code for the modification attempt on that entry. Anyone interested in implementing this as an overlay? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

make test fail on riscv64 (was: RE26 testing call #1 (2.6.4))
by Michael Ströder 20 Nov '22

20 Nov '22

On 10/4/22 18:49, Quanah Gibson-Mount wrote: > This is the first testing call for OpenLDAP 2.6.4. Depending on the > results, this may be the only testing call. make test seems to fail for openSUSE on riscv64 already for test000-rootdse. Not sure whether that's an issue with build options in the .spec file or whatever. Those package builds run in temporary VMs [1]. Do you have a command at hand how to dump all relevant files in case any of the tests aborts? Can I lookup how the OpenLDAP project runs automated builds and tests in gitlab and borrow from that? Ciao, Michael. [1] https://build.opensuse.org/package/show/home:stroeder:branches:home:stroede…

3 12

FOSDEM 2023
by Howard Chu 12 Nov '22

12 Nov '22

Anyone interested in setting up at FOSDEM next year? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 6

[OldapWS] -> Proposal of a REST Web Service for CRUD Operations
by Olivier CHATOR 16 Sep '22

16 Sep '22

Dear all, I am a "long time user" of OpenLDAP core parts. If the core is very stable, I often received some request from application developers, complaining about the lack of REST Web Service API to manipulate their Directory objects. Of course, as far as I could see, there is commercial solution available to do this, or even a project using Spring. But I could not see a simple proposal based on a simple Apache Server and "fully free" of rights. I also saw (https://www.openldap.org/devel/contributing.html) that "The OpenLDAP Project welcomes contributions of independently-developed stand-alone LDAP-related software packages suitable for distribution separately from existing packages (e.g., OpenLDAP Software, JLDAP, JDBC-LDAP)". Then, I would like to propose a full Open Source first realease of a CRUD REST Web Service to manipulate OpenLDAP's Directory Objects. I know that this first release is very limited in term of features, but I think it does the "core job", and may be a common base to be enriched to build a real OpenLDAP REST API ? Package available here: https://drive.google.com/drive/folders/1s4zlTleJ1JWhQWP2kLGHtTRofpWNevS7?us… Thanks for your time reading this proposal, do not hesitate to ask if you have any question. Kind regards, Olivier Chator

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel