openldap-devel

openldap-devel@openldap.org

1431 discussions

No symlinks in Git please (was: openldap.git branch mdb created.) 227e6976db20f424d4f6abda2b73bfa53034a714
by Hallvard B Furuseth 17 Aug '11

17 Aug '11

I think we should not have symlinks in Git. I does strange things with them. In particular on a system without symlinks, but also in my checkout now: $ git checkout mdb $ cd servers/slapd/back-mdb/ $ ls -l mdb.c libmdb/mdb.c ls: cannot access libmdb/mdb.c: No such file or directory lrwxrwxrwx 1 hbf usit 12 2011-08-17 13:26 mdb.c -> libmdb/mdb.c $ gitk --all ...but this does show the mdb source. I suggest: - treat the 'mdb' branch as a throw-away branch, instead of merging it into master, - let Makefile make the symlinks, as in libraries/liblunicode, - make some Git hook which rejects commits with symlinks. -- Hallvard

2 1

B-tree code
by Howard Chu 11 Aug '11

11 Aug '11

Hi Martin, Just thought you'd like to know about a project I've been working on for a couple months. My current code started with your append-only B-tree source. It's just about in usable shape now https://gitorious.org/mdb . Also I'll be presenting details at the LDAPCon in Hedelberg this October. http://www.daasi.de/ldapcon2011/index.php?site=program I started with your code, and removed the page cache. Instead the entire DB is accessed thru a read-only mmap region. As such, there is no longer any cache management at the DB level (it's all done by the OS/VM). I also removed the prefix-compression logic, because it made rebalancing/merging unreliable. The mmap approach avoids a ton of malloc/memcpy overhead. It also makes overflow pages quite cheap to manage. Instead of writing a new meta-page at the tail of the file, I ping-pong between two meta pages at the head of the file. (Double-buffering.) This provides most of the MVCC benefits of the append-only approach, but without the wasted space or the need to search for the most recent meta page. I also added tracking of outstanding read transactions, and tracking of free pages. Reader tracking is done without locks; readers are never blocked when accessing the DB (unless the OS itself is busy servicing page fauits). This way it can quickly check when a copied page is no longer referenced, and re-use the pages, so the DB no longer grows without bounds. This completely removes the need for the compaction logic. Since active data is never overwritten, the DB can never be corrupted, so no write-ahead logging is needed, nor any recovery procedures. I also added several ideas from BerkeleyDB, so that I can drop it into OpenLDAP more easily. The DB is now a "DB environment" with support for multiple databases within an environment. This was necessary because I didn't want to have to manage multiple separate mmap's for multiple little index databases and other misc. usages. Also the free list is itself a sub-DB in the environment. I also added support for sorted-duplicate data items for a given key, which OpenLDAP's back-hdb relies on. I'm just now getting started adapting our back-hdb code to this mdb library. It looks like the new backend will be vastly simplified, both in real code and in configuration, so it will be much friendlier to sysadmins, while at the same time giving superior performance to BerkeleyDB and excellent reliability. Of course the code is still pretty raw, and I haven't done any heavy load testing on it yet, so it remains to be seen how much of the promise is realized. I was originally targeting a design where the mmap resides at a fixed memory address. That way slapd can store its entries as-is, instead of flattening them into a storable structure. There's a hook for a relocation function, which would be used to relocate an entry if it gets shifted around during adds/deletes/rebalances. I haven't implemented this yet because I'm not sure it will actually work well in real use. For slapd it might be OK if all entries wind up in overflow pages, since those pages aren't touched by tree balancing activity. But if average entry sizes are small, it would become a serious hassle. I'd be interested to hear your comments on this. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Attribute 'auditContext' replicated
by Michael Ströder 08 Aug '11

08 Aug '11

HI! Is it intentional that attribute 'auditContext' is replicated? Ciao, Michael.

1 0

cn=config
by Ondrej Kuznik 04 Jul '11

04 Jul '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'm modifying an overlay for proper cn=config behaviour and have come across some things I'm not sure about. The overlay keeps a mapping derived from entries under its config entry, if the overlay's dn is "olcoverlay={1}bla, olcdatabase={2}null, cn=config" there are entries like: "olcMappingRuleName={x}, olcoverlay={1}bla, olcdatabase={2}null, cn=config" The rules are then stored in an avl in the overlay's info struct. However to maintain the avl I think I need to know the boundaries of the request. When adding the entry online, it is easy, one registers the cleanup callback. When processing the entry during startup, the only way to have the callback run is mucking with ConfigArgs' lineno variable. However a modify operation is harder. It can transform the entry in almost whatever way it pleases. Updating the entry in the avl each time a subpart of the modify is sent to a ConfigDriver makes providing the atomicity required by LDAP very hard. For instance, the mapping defined by the entry may at one time be equal to another one in the avl. At that time, the overlay does not know, whether this is supposed to be the final version or there is a part of the modify that makes it a distinct one again. Hopefully this paragraph makes sense. If there is no such way, I propose that the cleanup be aplicable to a modify operation too. No patches so far though. - -- Ondrej Kuznik -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4MUyUACgkQ9GWxeeH+cXuGtwCfXrKV7D5NRo6xqmIl94pYW3xA lVsAniZFXMCDQOpv4dbj+r8kiPktp6+p =nr22 -----END PGP SIGNATURE----- This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.

2 2

OID for Relax Rules Control (was: (ITS#6830) slapo-ppolicy.5 has incorrect schema fragments)
by Michael Ströder 01 Jul '11

01 Jul '11

andrew.findlay(a)skills-1st.co.uk wrote: > On Thu, Jun 09, 2011 at 01:45:17AM -0700, Howard Chu wrote: > >> I note that in ppolicy.c we have: >> >> { "( 1.3.6.1.4.1.42.2.27.8.1.17 " >> "NAME ( 'pwdAccountLockedTime' ) " >> "DESC 'The time an user account was locked' " >> "EQUALITY generalizedTimeMatch " >> "ORDERING generalizedTimeOrderingMatch " >> "SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 " >> "SINGLE-VALUE " >> #if 0 >> /* Not until Relax control is released */ >> "NO-USER-MODIFICATION " >> #endif >> "USAGE directoryOperation )", >> >> We have in fact released support for the Relax control, so it's >> probably time to unifdef these bits and go back to the documented >> behavior. What does "released support" really mean? $ grep -i relax openldap-2.4.26/include/ldap.h #define LDAP_CONTROL_RELAX "1.3.6.1.4.1.4203.666.5.12" #define LDAP_CONTROL_MANAGEDIT LDAP_CONTROL_RELAX "No released software should use an OID under this arc." See http://www.openldap.org/faq/data/cache/200.html I'd really love to see an officially assigned OID (especially given the fact that web2ldap supports it). Ciao, Michael.

1 0

RE24 (2.4.26) final testing call
by Quanah Gibson-Mount 29 Jun '11

29 Jun '11

2.4.26 has been tagged (OPENLDAP_REL_ENG_2_4_26). There were a few minor commits since the last testing call. Please test and report back any problems. If nothing is found in the next 24 hours, we will release as-is. Thanks! --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

4 3

ldap_bind_st() call
by Thomas Egerer 27 Jun '11

27 Jun '11

Hello *, I am currently dealing with a problem in a third-party application using openldap to retrieve CRLs. We are evaluating this piece of software for possible vulnerabilities. One of the risk arises from a possible DoS-attack involving the aforementioned CRL-retrieval. Our scenario involves an attacker being able to hijack the LDAP- server delivering the CRLs and replacing the actual LDAP-application by a hand-crafted one which accepts connection attempts on port 389 and then does nothing. As a result the application being evaluated is rendered unusable stalling forever while trying to download the CRL. The reason for this behavior is the function call ldap_simple_bind_s() being used to connect to the server. While we are able to control the actual tcp connect and the retrieval of the CRL using LDAP_OPT_NETWORK_TIMEOUT and LDAP_OPT_TIMEOUT, there seems no option to use something like a ldap_bind_st()- function. A mailinglist-entry back from 2002 suggested the implementation of a custom ldap_bind_st()-function, which I did using ldap_bind() and ldap_result(). Yet, me efforts were in vain as I could not retrieve the ld_error-member from LDAP-structure since it's an opaque struct. I finally ended up with a custom function which essentially does what I want, but I cannot figure out if it's due to a bind-timeout or some other reason since ldap_result() returns -1 on failure and 0 otherwise, not the actual error code. Bottom line is, I'm back where I started: wondering why there is no such a function in openldap *and* if is there any chance getting the one that I wrote -- it's added below -- upstream. I know that ldap_bind_s is deprecated in favour of ldap_sasl_bind_s which I could also extend by a ldap_sasl_bind_st function, but I do not want to put any more effort in something being deemed to fail from the start. To sum it up: any chance to get this upstream? Cheers Thomas diff --git a/include/ldap.h b/include/ldap.h index 3f0c1c4..dc3a48c 100644 --- a/include/ldap.h +++ b/include/ldap.h @@ -1270,6 +1270,15 @@ ldap_bind_s LDAP_P(( /* deprecated, use ldap_sasl_bind_s */ LDAP_CONST char *cred, int authmethod )); +LDAP_F( int ) +ldap_bind_st LDAP_P(( + LDAP *ld, + LDAP_CONST char *who, + LDAP_CONST char *cred, + int authmethod, + struct timeval *timeout, + LDAPMessage **result )); + /* * in sbind.c: */ diff --git a/libraries/libldap/bind.c b/libraries/libldap/bind.c index 875bd69..303ead1 100644 --- a/libraries/libldap/bind.c +++ b/libraries/libldap/bind.c @@ -125,3 +125,32 @@ ldap_bind_s( return( ld->ld_errno = LDAP_AUTH_UNKNOWN ); } } + +int ldap_bind_st( + LDAP *ld, + LDAP_CONST char *dn, + LDAP_CONST char *passwd, + int authmethod, + struct timeval *timeout, + LDAPMessage **res ) +{ + int msgid; + + *res = NULL; + + Debug( LDAP_DEBUG_TRACE, "ldap_bind_st\n", 0, 0, 0 ); + + if ( (msgid = ldap_bind( ld, dn, passwd, authmethod )) == -1 ) + return( ld->ld_errno ); + + if ( ldap_result( ld, msgid, LDAP_MSG_ALL, timeout, res ) == -1 || !*res ) + return( ld->ld_errno ); + + if ( ld->ld_errno == LDAP_TIMEOUT ) { + (void) ldap_abandon( ld, msgid ); + ld->ld_errno = LDAP_TIMEOUT; + return( ld->ld_errno ); + } + + return( ldap_result2error( ld, *res, 0 ) ); +}

2 6

RE24 testing call #3 (2.4.26)
by Quanah Gibson-Mount 27 Jun '11

27 Jun '11

Please test RE24, it has a few new updates. This is a release candidate. --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

5 8

RE24 testing call #2 (2.4.26)
by Quanah Gibson-Mount 24 Jun '11

24 Jun '11

Please test RE24. Thanks! --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

4 8

mmap'd DB
by Howard Chu 22 Jun '11

22 Jun '11

The main goal of mdb is to eliminate extra copies of data in various caches. A secondary goal is simplifying config (by eliminating extra caches, there's nothing left to tune/configure). One of the key requirements for this to work is to make the on-disk Entry format identical to the in-memory format. Since the in-memory format is based on pointers, we seem to need a way to nail down the memory map to a fixed address that is constant from run to run. (Just re-capping...) The tricky part is that Entries contain AttributeDescriptions and these obviously depend on whatever schema is loaded on a particular run. If you take no special steps, then adding or deleting schema elements from run to run will invalidate their addresses, and make the on-disk pointers useless. I had a notion to replace AttributeDescription pointers with indices into an array. The array would be persisted into a file, and would only ever grow; items would never be deleted from it. Then Entries could reference the array indices instead of the pointers. A scheme like this could work, but the sheer volume of code that directly (de-)references AD pointers is pretty huge. Not sure if it's the right path to take. Another option was to setup a dedicated ad_alloc() function that carves up its own mmap'd file. Then AD pointers will always be constant from run to run. The trouble here is how to configure the path to this file; all of the hard-coded schema is generated before we ever get to reading the config. A possible approach is to first allocate the AD pointers in an anonymous mmap'd region, then once the config file is read we can open the configured file and pull from there. For this to work we have to reserve a fixed size chunk of memory for the hard-coded schema, so that if we add/delete other hard-coded schema elements down the road, the map remains useful. All in all it seems extremely fragile and a major pain for compatibility between revisions. Another possibility is to collect all the schema elements at build time, and generate a .c file containing a seed AD map which is hard-coded (and again, is only allowed to grow, no elements modified in place or deleted) so that the core AD map is always the same from run to run. The more I think of it, the more I think this will work. It's essentially the same solution we used to get rid of the ucdata-path item... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel