openldap-devel

openldap-devel@openldap.org

1443 discussions

Dereference Control (Was: How to implement Extended DNs for Samba4?)
by Pierangelo Masarati 22 Oct '08

22 Oct '08

Howard, this is where I got so far with the specification (and the code :) /* * request: * * controlValue ::= SEQUENCE OF DerefSpec * * DerefSpec ::= SEQUENCE { * derefAttr AttributeDescription, * attributes AttributeList } * * AttributeList ::= SEQUENCE OF attr AttributeDescription * * * response: * * controlValue ::= SEQUENCE OF DerefRes * * DerefRes ::= SEQUENCE { * spec DerefSpec, * vals DerefVals } * * DerefVals ::= SEQUENCE OF DerefVal * * DerefVal ::= SEQUENCE { * derefSpecVal AttributeValue, * attrSets AttrSets } * * AttrSets ::= SEQUENCE of attrSet AttributeValues * * AttributeValues ::= SET OF value AttributeValue * * * example request: * * { { member, { GUID, SID } }, { memberOf, { GUID, SID } } } * * example response: * * { { { memberOf, { GUID, SID } }, * { { "cn=abartlet,cn=users,dc=abartlet,dc=net", * { { "0bc11d00-e431-40a0-8767-344a320142fa" }, * { "S-1-2-3-2345" } } }, * { "cn=ando,cn=users,dc=sys-net,dc=it", * { { "0bc11d00-e431-40a0-8767-344a320142fb" }, * { "S-1-2-3-2346" } } } } } } */ I designed the response this way, because it'd allow to make it as compact as possible while allowing for multiple as well as missing attribute values in each attrSet. However I'm facing two issues, which I can't solve directly since I don't have X.690 handy right now: - does BER allow to have an empty set/sequence? Apparently, I'm unable to set an empty set using ber_printf("[]"). - does it make sense to have a structured response like this, where the attribute types are listed first, and aggregate values follow, so that the order of the value sets (including empty sets, if allowed) matches the order of the attributes? I mean: { member, { cn, uid, drink } } is supposed to match { "cn=Ando,dc=sys-net,dc=it", { [ "Ando", "Pierangelo Masarati" ], [ "ando" ], [] } } such that set #0 ([ "Ando", "Pierangelo Masarati" ]) refers to attr #0 (cn), set #1 ([ "ando" ]) refers to attr #1 (uid), and set #2 ([]) refers to attr #2 (drink). In fact, one could argue that the order in attrSets needs not necessarily be the same of the attributes in spec. This would require to use an alternative definition like /* * response: * * controlValue ::= SEQUENCE OF DerefRes * * DerefRes ::= SEQUENCE { * derefAttrVal DerefAttrVal, * attrsVals SEQUENCE OF attrVals DerefAttrsVals } * * DerefAttrVal ::= SEQUENCE { * attr AttributeDescription, * val AttributeValue } * * DerefAttrVals ::= SEQUENCE { * attr AttributeDescription, * vals SET OF val AttributeValue } */ which implies a repetition of all the attribute descriptions for each derefAttrVal. Comments? p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it ----------------------------------- Office: +39 02 23998309 Mobile: +39 333 4963172 Fax: +39 0382 476497 Email: ando(a)sys-net.it -----------------------------------

3 5

Re: Dereference Control
by Raphaël Ouazana-Sustowski 22 Oct '08

22 Oct '08

Hi, Le Mer 22 octobre 2008 16:36, Pierangelo Masarati a écrit : > * 2.1 Example > * > * 2.1.1 Request > * > * { { member, { GUID, SID } }, { memberOf, { GUID, SID } } } > * > * 2.1.2 Response > * > * { { memberOf, "cn=abartlet,cn=users,dc=abartlet,dc=net", > * { GUID, [ "0bc11d00-e431-40a0-8767-344a320142fa" ], > * SID, [ "S-1-2-3-2345" ] } }, > * { memberOf, "cn=ando,cn=users,dc=sys-net,dc=it", > * { GUID, [ "0bc11d00-e431-40a0-8767-344a320142fb" ], > * SID, [ "S-1-2-3-2346" ] } } } > * > * 2.2 Example > * > * 2.2.1 Request > * > * { { member, { cn, uid, drink } } } > * > * 2.2.2 Response > * > * { { member, "cn=ando,cn=users,dc=sys-net,dc=it", > * { cn, [ "ando", "Pierangelo Masarati" ], > * uid, [ "ando" ] } } } Sorry if I'm completly misunderstanding, but would not it be more simple and user friendly if the attributes were directly returned to the user, like do many overlays ? Ideally this could be : member/cn: ando member/cn: Pierangelo Masarati member/uid: ando But to be more LDAP-friendly, we could see something like: cn;member: ando cn;member: Pierangelo Masarati uid;member: ando Or maybe I'm totally missing a point :) Regards, Raphael Ouazana.

2 1

Re: commit: ldap/servers/slapd/back-null null.c
by Hallvard B Furuseth 20 Oct '08

20 Oct '08

ando(a)OpenLDAP.org writes: > null.c 1.25 -> 1.26 > declare support for read controls (ITS#5757) Hold on. LDAP_CONTROL_PAGEDRESULTS requires to be sent LDAP_CONTROL_PAGEDRESULTS in the search response. Does the frontend do that? back-bdb does it "by hand". Clients may use LDAP_CONTROL_ASSERT to check something they want to know, not just to ensure they don't update entries with unexpected contents. I suppose we could evaluate the filter against an empty entry with just the DN filled in - as if the rest were hidden by access controls. Though since back-null is a liar - it cannot present a consistent view of whether data are present or not - it may be better to be conservative and always return Assertion Failed. I don't know the requirements of the other controls/extensions. -- Hallvard

2 1

Managing multiple regex matches
by manu＠netbsd.org 20 Oct '08

20 Oct '08

Let us imagine an ACL like this: access to dn.regex="^uid=.*,ou=(.*)" attrs=foo val.regex="^(.*)$/" ... We would like to use $1 (ou's value) and $2 (foo's value) in the <who> field. For now this is not possible, because slap_access_allowed() collect a single set of regmatch_t. If the <what> field of an ACL has multiple regex matches, the last one only will be retained. I suggest the following change: 1) In slap_access_allowed(), we would keep track of multiple set of regmatch_t. matches would become something such as: typedef struct AclRegexMatches { regmatch_t dn[MAXREMATCHES]; regmatch_t val[MAXREMATCHES]; } AclRegexMatches; In slap_acl_get(), the two regexec() calls would be done with matches.dn or matches.val, depending on the situation. 2) In acl_string_exapand(), we would replace $1, $2, $3 by values from matches.dn, therefore providing backward compatibility. And we would replace ${v1}, ${v2}, ${v3}... by values from matches.val and ${d1}, ${d2}, ${d3}... by values from matches.dn There is a problem with my proposal, on dynaic ACL. We cannot provide them values from attribute value without changing the API. I suggest we stick with the current API for now and improve that later if needed. Opinions? Did I miss something? -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu(a)netbsd.org

2 2

back-ndb design notes
by Howard Chu 19 Oct '08

19 Oct '08

While looking over the Admin Guide I was re-reading section 1.8 (LDAP vs RDBMS) and thought it would be worth describing the approach we took and tradeoffs we made. (Disclaimer: some of these decisions need to be changed.) (Also for more background: NDB is a cluster-based storage engine for MySQL. It runs as its own set of processes across the nodes of a cluster; all accesses to it are through remote procedure calls.) The basic approach starts with a rough adaptation of back-bdb to NDB. That is, we use one table to map DNs to numeric entryIDs, and use this numeric ID as the primary key to access the main entry data. But instead of using a monolithic id2entry table (and treating all the LDAP data as blobs) we start from the notion of a table per objectclass. For objectclasses that inherit from others, only the unique attributes of the subordinate class reside in that class's table. This is one of the underlying requirements - an LDAP attribute must only appear in one table. For attributes that appear in unrelated LDAP objectclasses, you must configure them into "attribute sets" (which are yet another set of tables) to ensure that their column definition is used uniformly by all the objectclasses. Note that this means to retrieve the complete information for a particular entry, we have to pull from many tables at once, and each requires a separate NDB operation. The NDB API allows many operations to be batched together and executed in parallel, so we can do all of these fetches in a single roundtrip. (Otherwise, performance for this approach would be pretty abominable.) The Admin Guide notes that this approach can be very poor because accessing many tables at once requires a lot of disk seeks. This consideration doesn't apply to NDB because each node of the cluster always caches everything completely in RAM. However, we first need to know which tables to fetch from, i.e., we need to consult a table to get a list of objectclasses, before we can spit out the batched request to all the relevant tables. To avoid having to visit so many different tables sequentially, we store the objectclass attribute in the dn2id table. (We obviously have to visit the dn2id table anyway. Just as with back-bdb, where accessing any entry by name requires first a dn2id lookup followed by an id2entry lookup, here we must do the dn2id lookup followed by the parallel lookups across all of the class/set tables.) For as much functionality as that covers, things are pretty straightforward. The dn2id table has some interesting issues as well. I think it's a poor design, but it was the best we could do under our initial constraints. Basically it has a 16 column primary key, where each rdn occupies a column. This approach makes it fairly easy to obtain base, subtree, and children scoping, but it suffers from the same problems as back-bdb and back-ldbm. (Bloated, no subtree rename... Plus, it imposes a hard limit on tree depth at 16 levels. It's probably an acceptable limit, but worth noting that none of our other backends have such a limit.) The entryID is just an integer column, and the objectclass attribute is a single column containing all of the classes concatenated as a space-delimited string. E.g., for an entry with objectClass: person objectClass: inetOrgPerson the column will contain " person inetOrgPerson " This format allows us to perform equality searches on objectclass using an SQL LIKE filter. (objectclass=foo) -> objectclass like "% foo %" (Filters aren't implemented using actual SQL, but the NDBapi filter mechanism obviously implements SQL filtering functionality.) As mentioned, the numeric entryID is used as the primary key of the main data tables. In fact, that was just our initial approach which only supported single-valued attributes. Now we support multivalued attributes by using a two-column primary key, <entryID,value#>. So there's a "value" column with values from 0 to N-1 for an attribute with N values, and each value occupies a separate row. The NDBapi allows us to do partially-specified primary key lookups, so we can do a lookup on just the entryID and automatically get all of the values/rows associated with that entryID in a single operation. Search indexing is currently a mess, and will need to be rewritten. We arrived at the current approach based on a couple main constraints: 1) this backend will do no local caching 2) we must always retrieve the desired data in only 2 API roundtrips #1 was based on the requirement that multiple slapds be able to operate concurrently on the database, along with multiple mysqlds. We didn't have time to design a distributed cache synchronization mechanism, therefore, no local caching. #2 was based on the simple fact that taking more than 2 roundtrips would destroy performance. The current solution uses the regular SQL table indexing features, but the indices are stored in the dn2id table, not in the main data tables. This means we can do an index lookup and obtain an entry's DN and objectclasses in a single roundtrip, then a single roundtrip for the main data, so it meets our #2 constraint. One problem with this is that a single table can only accommodate rows up to 8192 bytes wide, and we can easily max that out when multiple indices are needed. Another problem is that updating indexed attribute values requires writes to both the main data tables and the dn2id table. While back-ndb handles this itself, mysql daemons writing to the data tables won't know about this requirement. (We need to use a trigger/stored procedure to make sure the SQL side keeps the indices up to date.) The big problem here is that if you want to be able to match multiple columns in an indexed filter lookup, they must all belong to a single index. A 3 column index <a,b,c> is not the same as 3 individual indices <a>, <b>, and <c>... With the multi-column index, you can only match filters that use those columns in that specific order. E.g., (&(a=foo)(b=bar)(c=baz)) will work but not (&(c=baz)(b=bar)(a=foo)). You can omit the trailing columns (&(a=foo)(b=bar)) but you can't omit the leading columns (&(b=foo)(c=bar)). And, while the SQL indexers support equality and inequality indexing, they don't support substring indexing at all. Moving forward, I think the only feasible approach is to stop using multi-column indices. We'll use individual indices, and have to perform boolean evaluations within slapd, instead of letting the NDB engine evaluate them. This is unfortunate because it will use a lot more network bandwidth, but we don't have much choice. Likewise, we can't do anything about substring indexing, it's up to MySQL to provide a solution for that. Other big changes going forward include replacing the current dn2id scheme with a back-hdb style layout so that we don't waste so much space, and so we can support subtree rename. We'll also take a stab at local caching of the dn2id table. (That may be rather tricky; NDB has an event API that slapd can use to listen for add/delete/modify events, but it's totally asynchronous. If we can't work out a way to keep the cache transactionally consistent then this just isn't going to happen.) Mapping LDAP onto an RDBMS is still a hard problem. back-ndb implements some pretty good solutions to a lot of the problems, but there's still things that can be improved. The current backend works for a limited set of LDAP use cases, and operates well even with concurrent SQL access. With the changes I described, it will support a broader range of LDAP use cases... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: commit: ldap/servers/slapd schema_init.c
by Hallvard B Furuseth 18 Oct '08

18 Oct '08

ando(a)OpenLDAP.org writes: > schema_init.c 1.452 -> 1.453 > apply Luca Tamburo's patch for Attribute Certificate and X.509 PMI > support (with modifications, ITS#5695) Any reason to make these ber_int_t instead of ber_len_t? > + ber_int_t src, dst; > + (...) > + for ( src = 0, dst = 0; src < is->bv_len; src++, dst++ ) { I had just changed them to ber_len_t to kill a warning for the comparison with is->bv_len. -- Hallvard

2 3

Replication notes
by Howard Chu 17 Oct '08

17 Oct '08

Slurpd and syncrepl really took opposite approaches to replication. While syncrepl allows consumers to be configured without any special configuration of the provider, slurpd allowed slaves to be configured without any special knowledge. This feature allowed slurpd to be used to replicate to pretty much anything that understood some flavor of LDAP. With syncrepl, the consumer must minimally understand entryCSNs, entryUUIDs, and a contextCSN. This requirement would appear to limit our reach. But with delta-syncrepl, we actually need nothing more than a contextCSN to keep track of where we are in the changelog. And when using delta-syncrepl thru an LDAP proxy, there's actually no requirement that we store the contextCSN on the remote server. We could simply store the contextCSN as a DB-specific attribute of the proxy DB's entry under cn=config. Whether we do this by explicitly modifying the syncrepl code, or by using an overlay to accomplish the sleight-of-hand doesn't seem to matter much. (Though there may be more uses for a customizing/mapping overlay here; e.g. translating LDAPv3 schema to MS AD.) Just wanted to mention this here before I forgot it again. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: commit: ldap/servers/slapd/overlays collect.c
by Howard Chu 17 Oct '08

17 Oct '08

hallvard(a)OpenLDAP.org wrote: > Update of /repo/OpenLDAP/pkg/ldap/servers/slapd/overlays > > Modified Files: > collect.c 1.11 -> 1.12 > > Log Message: > ITS#5747: Only use C99 flexible array member when supported > > CVS Web URLs: > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/overlays/ > http://www.openldap.org/devel/cvsweb.cgi/servers/slapd/overlays/collect.c > > Changes are generally available on cvs.openldap.org (and CVSweb) > within 30 minutes of being committed. > Just set it to *ci_ad[1] and leave it at that. Ifdefing this situation is unnecessary. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 1

sorted values vs. compare errors
by Hallvard B Furuseth 17 Oct '08

17 Oct '08

I don't know if this is a false alarm or not, but anyway: Sorted values, as with SLAP_ATTR_SORTED_VALS in attr_valfind(), needs an unambiguous ordering. However value_match() can return error, and does not say which of the compared values caused the error. Are there sorted value examples where it can return error both because of the stored value and because of the assertion value? Or overlays like rwm which can cause this? If so the sort order of (normal value, value causing error) will depend on the order of arguments to the compare function. Then if we manage to insert an "ambiguously-sorted" attribute value into the sorted list of values, then a binary search for an existing normal value can fail if it hits the naughty value as a pivot entry. Or it can cause a new value to be inserted at the wrong place. Possibly there are also some other strange matching rules where the ordering is ambiguous, I don't know. In any case, I think this whould mean that: - value_match() and ordering-compare functions should try to define unambiguous orderings of (normal value, truble value). I think sorting of (trouble value 1, truble value 2) does not matter. - maybe Sorted values should only be supported for attributes whose compare functions do define such an ordering. However there are also inequality filters vs. sorted attribute values. This code in slapd/filterentry.c:test_ava_filter() is similarly broken if there is an error-generating value at either end of the array of values: if (( a->a_flags & SLAP_ATTR_SORTED_VALS ) && type != LDAP_FILTER_APPROX ) { unsigned slot; int ret; /* For Ordering matches, we just need to do one comparison with * either the first (least) or last (greatest) value. */ if ( use == SLAP_MR_ORDERING ) { const char *text; int match, which; which = (type == LDAP_FILTER_LE) ? 0 : a->a_numvals-1; ret = value_match( &match, a->a_desc, mr, use, &a->a_nvals[which], &ava->aa_value, &text ); if ( ret != LDAP_SUCCESS ) return ret; For such a value, and with a normal assertion value, the function ought to search for the nearest non-error-generating value and compare with that. -- Hallvard

1 0

Re: commit: ldap/doc/man/man5 slapd.conf.5
by Hallvard B Furuseth 17 Oct '08

17 Oct '08

ando(a)OpenLDAP.org writes: > slapd.conf.5 1.291 -> 1.292 > minor clarifications Remember slapd-config.5. -- Hallvard

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel