openldap-devel

openldap-devel@openldap.org

1420 discussions

Indexing revisited
by Howard Chu 18 Mar '14

18 Mar '14

A few thoughts occurred to me today about our indexing code: 1) we compute a hash preset for each invocation, crunching the syntax and matching rule's OID, among other things. (It used to be worse, we used to recompute this for each individual value, even though it's a constant.) There's no need to always recompute this on each invocation, we can compute it once at first usage and reuse that result. It should speed up index generation, particularly on smaller attribute values. I'm preparing a patch to test this now. 2) using this precomputed hash, we can drop the syntax, mr, and prefix arguments from the indexer function signature. That will also speed things up. 3) I note that the 'use' argument is also never used in our indexer functions. Will drop this as well. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

2 3

Compact log format for faster searching?
by Hallvard Breien Furuseth 13 Mar '14

13 Mar '14

Have anyone come up with a searchable, compact slapd-log format, so the logs can be searched quicker than the current verbose format? That is, the search tool would understand the compact format without expanding it (fully) first, and only expand the final output. Our syslog from loglevel "stats" compresses to 1/20 of the original, but that's no help when we must search the entire uncompressed log. It's easy to halve the log and still keep it mostly readable: Remove syslog cruft and write it only when it changes, replace "conn= <op/fd>=" with base-32 "conn.<o/f>op", join up multiple SRCH attr= lines, etc. Or down to 1/3 of the original in our case by replacing a few common operations (filters, suffixes, etc), but that quickly makes the result unreadable without a tool to translate back and forth. So beyond that something is needed to translate back and forth, or an entirely new, human-readable, compact format. BTW, "perl -lne '/uid=xyzzy/i && print' log" is 10-15 times faster than GNU "grep -i 'uid=xyzzy' log" on my Linux box. And the initial zcat of a compressed log is faster than the perl/grep. -- Hallvard

1 0

Performance improvements for set-based ACLs?
by Michael Ströder 10 Feb '14

10 Feb '14

Yes, I know: Set-based ACLs are slow even if one properly sets index configuration. But I'd like to raise a discussion here whether it would be possible to improve the performance within slapd. Any thoughts? Ciao, Michael.

4 6

RE24 Testing call #1 (OpenLDAP 2.4.39)
by Quanah Gibson-Mount 16 Jan '14

16 Jan '14

Current RE24 is ready for testing for the 2.4.39 release. Thanks! --Quanah -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

2 1

commit review for 2.4.39
by Quanah Gibson-Mount 13 Jan '14

13 Jan '14

Pending commits I think look good for RE24: 7759 avoid assert in parse_passwdpolicy_control 7753 (fix test suite to use $MAKE 7761 (bail out of search if config is pausing) 7762 shortcut null rdns 6758 rewrite code for contrib:wrap_slap_ops (Hallvard says good for RE24). Pierangelo did a fix to deref, looks minor and good for RE24 7773 - Pierangelo also did one for constraint that looks good for RE24 and I assume I should sync mdb.master to main/re25/re24 -- Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

3 7

re: commit review for 2.4.39
by Howard Chu 07 Jan '14

07 Jan '14

> Pending commits I think look good for RE24: > > 7759 avoid assert in parse_passwdpolicy_control > 7753 (fix test suite to use $MAKE > 7761 (bail out of search if config is pausing) > 7762 shortcut null rdns > 6758 rewrite code for contrib:wrap_slap_ops (Hallvard says good for RE24). > Pierangelo did a fix to deref, looks minor and good for RE24 > 7773 - Pierangelo also did one for constraint that looks good for RE24 > > and I assume I should sync mdb.master to main/re25/re24 Sounds good to me. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: sets and ldap:// searches
by Michael Ströder 15 Dec '13

15 Dec '13

Christian Kratzer wrote: > Hi Michael, > > On Sat, 14 Dec 2013, Michael Ströder wrote: > >> HI! >> >> I used this FAQ entry to use set-based ACLs for my current task: >> >> http://www.openldap.org/faq/data/cache/1133.html >> >> There's written: >> >> "We could make this more powerful (and more complex and costly to compute) >> by allowing base sets to be built from LDAP filters. This is something to >> consider, because the combination of filters and sets (which have little >> overlap in what they can express) is very powerful. This is partially provided >> by the URI expansion capability." >> >> Was this ever implemented? It would be helpful. IMO using 'this' and/or 'user' >> in [ldap://] sets would make evaluating the set-based <who> clause faster if >> attributes used in LDAP URLs are properly indexed. >> >> Background: >> My aim is to configure Linux clients all the same (search base, filter etc.) >> but let each *machine* individually authenticate against OpenLDAP server. >> Then only the users who are members of groups with login rights on this >> particular server should be visible to the Linux client. >> So the goal is to filter user/groups at the OpenLDAP server and *not* in the >> Linux client (e.g. by sssd configuration or similar). >> For now it seems only possible to do this with set-based ACLs given all the >> requirements I have. > > I needed to restrict users seen be a spefic application to users that are > member in the respective applications group. > > I cheated by not checking the group but the memberOf: attribute in the entries > I intended to filter. > > olcAccess: > to dn.children="dc=example,dc=org" > filter="(memberOf=cn=application1-group,ou=groups,dc=example,dc=org)" > by dn.base="cn=application1-user,ou=system,dc=example,dc=org" read > olcAccess: > to dn.children="dc=example,dc=org" > filter="(memberOf=cn=application2-group,ou=groups,dc=example,dc=org)" > by dn.base="cn=application2-user,ou=system,dc=example,dc=org" read > olcAccess: > to dn.children="dc=example,dc=org" > filter="(memberOf=cn=application3-group,ou=groups,dc=example,dc=org)" > by dn.base="cn=application3-user,ou=system,dc=example,dc=org" read > > The performance implictations of these kind of acl should be ok from my > understanding as they only apply to the specific applications user. Thanks for sharing your ACLs. But in my case it's a strong requirement that ACLs are generic and do not contain any names. So you in your case one could add more applications just by adding entries without having to touch any ACLs at all. AFAICT something like this can only be done with set-based ACLs. Ciao, Michael.

1 0

sets and ldap:// searches
by Michael Ströder 14 Dec '13

14 Dec '13

HI! I used this FAQ entry to use set-based ACLs for my current task: http://www.openldap.org/faq/data/cache/1133.html There's written: "We could make this more powerful (and more complex and costly to compute) by allowing base sets to be built from LDAP filters. This is something to consider, because the combination of filters and sets (which have little overlap in what they can express) is very powerful. This is partially provided by the URI expansion capability." Was this ever implemented? It would be helpful. IMO using 'this' and/or 'user' in [ldap://] sets would make evaluating the set-based <who> clause faster if attributes used in LDAP URLs are properly indexed. Background: My aim is to configure Linux clients all the same (search base, filter etc.) but let each *machine* individually authenticate against OpenLDAP server. Then only the users who are members of groups with login rights on this particular server should be visible to the Linux client. So the goal is to filter user/groups at the OpenLDAP server and *not* in the Linux client (e.g. by sssd configuration or similar). For now it seems only possible to do this with set-based ACLs given all the requirements I have. Ciao, Michael.

1 0

Linux kernel performance regressions
by Howard Chu 11 Dec '13

11 Dec '13

We upgraded from kernel 3.5 to 3.12.3 to update some of our benchmark numbers and hit some major performance regressions, mainly because the kernel is throttling processes that use too much CPU. This is definitely a kernel bug, as the throttle mechanism belongs to the realtime scheduler and none of the processes being affected had realtime priority when the throttle kicked in. I've posted a query to the linux kernel mailing list but haven't gotten any satisfactory answers yet. The same throttling behavior also occurs with 3.11.10, but there are no corresponding messages in the kernel log. The email thread is here http://lkml.indiana.edu/hypermail/linux/kernel/1312.1/02313.html A patch that may be related is described here http://lkml.org/lkml/2013/5/29/640 which explains part of the observed behavior, but not all of it (and indeed may be a red herring, unless it has some interaction with the realtime scheduler). There appear to be other serious networking related regressions in 3.12 as well. http://lkml.indiana.edu/hypermail/linux/kernel/1312.1/02588.html I recommend staying on 3.10 for production servers until this is sorted out. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: Read lock without a write lock
by Howard Chu 21 Nov '13

21 Nov '13

Milosz Tanski wrote: > I saw that LMDB recently added a new flag MDB_NOLOCK. Is there a way > to keep the reader lock file and but do away with the write lock? > > My motivation for doing this to be able to use LMDB from a process > that forks to share the database. And the application can implement > it's own write lock (mutex in shm or futex on linux). What advantage do you gain from using your own write lock? Note that forking to share the database is explicitly *not* supported. > This would be preferable to having to do doing something like a > MDB_NOLOCK with a pthread_rwlock since we could still have parallel > readers with one writer. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-devel