openldap-technical

openldap-technical@openldap.org

5 participants
9899 discussions

LMDB: comparison contexts, lifetimes, composite keys
by Lauri Alanko 19 Feb '15

19 Feb '15

Hello. I have some non-OpenLDAP-related LMDB questions and suggestions. I hope this is the correct list for them. If not, please point to a better venue. It seems that MDB_cmp_func is a plain binary function without a context parameter. This makes it impossible to tune a comparison operation at runtime (without libffi magic, anyway). In particular, it makes it difficult to create bindings for the set_compare operation in other languages. Funnily enough MDB_rel_func _does_ have a context pointer, even though it is not used yet. So would it be feasible to add a similarly contextful version of the comparison function? (Arguably these two could share the context pointer, it could just be mdb_dbi_set_userctx.) Although the documentation in lmdb.h is generally good, it's a bit wishy-washy when it comes to lifetimes of data. The mdb_get documentation only says: "The memory pointed to by the returned values is owned by the database". My understanding is that the pointers to the key/data pair retrieved from the database are valid for the duration of the current transaction. After the transaction is over, the page in which the key/data pair was located may be reclaimed for other uses. If this is correct, could it be reflected in the documentation a bit more explicitly? The current behavior is a bit inconsistent, though, in that depending on the operation, mdb_cursor_get may or may not update the `key` argument to point to the key in the database. It would be much clearer if after the call it _always_ pointed to the key in the database, even for operations (like MDB_SET_KEY) where the value of the key would be the same as the input argument. Finally, what is the recommended way of dealing with composite keys? Suppose I want to look up a value associated to a pair of keys (say, a directory id and a filename). There are several ways of doing this: * Encode the pair (key1, key2) into a single key, use that normally. * Use MDB_DUPSORT, encode each (key2, value) as data for a duplicate key key1. Make sure the encoding is such that the data are sorted by key2. Then use MDB_GET_BOTH_RANGE to look up the nearest entry for a given (key1, key2) pair and check it actually has the correct key2. * Create a named database for each key1, store its name as the data for key1. Store normal (key2, value) pairs in the named database, Each of these has its problems. The first is somewhat wasteful, especially if, say, key1 is long and often repeated. The second is what I use, but it feels a bit kludgy: I'm supposed to use a key-value store yet I still have to manually decode and encode key-value pairs to and from their constituents. The third has lots of overhead if there are very few entries for a given key1, and it requires us to manually manage the lifetimes of the named databases. My understanding is that MDB_DUPSORT internally does something close to the third option (but with optimizations to avoid wasting pages for just a few entries), but just using empty data items in the key-specific subdatabase. Would it then be feasible to add direct support for composite keys by allowing non-empty data there? Thanks, Lauri Alanko la(a)iki.fi

2 2

2.4.40 RPMs for openSUSE
by Michael Ströder 17 Feb '15

17 Feb '15

HI! After some struggle and help by some opensuse-packaging list members (thanks!) I achieved almost what I wanted. I'd appreciate if someone could work on SLES support. Build status and files: https://build.opensuse.org/package/show/home:stroeder:branches:network:ldap… If you're using the openldap packages shipped with openSUSE make sure to read the .changes files to see whether something could break your current setup. Up to now I've tested openSUSE 13.2 and Factory on x86_64 and Factory_ARM on rasperry pi. Note: openSUSE 12.3 was announced to be EOL so I've disabled the repo in my home project. I can easily re-enable it if needed (or branch my repo in your own home project). The usual disclaimer: Don't play with it in production! The download repos are here: http://download.opensuse.org/repositories/home:/stroeder:/branches:/network… Make yourself familiar with zypper commands. Example: Add the repo in openSUSE 13.1 (command in one line): zypper addrepo --refresh http://download.opensuse.org/repositories/home:/stroeder:/branches:/network… Please test. Your feedback is appreciated. Especially have a look at whether the RPMs behave well regarding config files, package descriptions etc. There's a new package openldap2-contrib with a selection of overlays from, you might have guessed, the contrib/ source directory. slapo-noopsrch is still missing because of http://www.openldap.org/its/index.cgi?findid=7998 Since web2ldap makes heavy use of noop search control I'd be glad if someone could look into ITS#7998. ;-) Ciao, Michael.

1 2

[SOLVED] Re: Inconsistent answer from LDAP Server
by Yoann Gini 16 Feb '15

16 Feb '15

Hello, Just for information, I’ve found the mechanism in OpenLDAP allowing integrators to do stupid things: overlays. Apple has created a customer overlay for their own services and has recently added hardcoded value for specific request with specific attributes. http://www.opensource.apple.com/source/OpenLDAP/OpenLDAP-499.27/OpenLDAP/se… I didn’t know that overlay was able to have deep control like that over LDAP request. I can’t thanks you for this end since no one has ever mentioned that OpenLDAP has official hooking API on LDAP request but here is the solution for my problem and now I know how to fix it by my own, without waiting for a fix from Apple, just patching slapd, changing a condition in odusers_search function. Best regards, Yoann

5 4

Indexing in openldap.
by Mangesh Sawant 16 Feb '15

16 Feb '15

Hi, I want information regarding adding index on an ldap attribute. My directory structure is like: ObjectClass Hierarchy: Company=>Department (In dc=example,dc=com) Inside ObjectClass"Company", One of the attributes "UserID" to be indexed. I have added index on the same as per following in slapd.conf: index UserID eq Kindly let me know whether it is the correct syntax of defining an index. Thanks And Regards , Mangesh Sawant .

1 0

Re: syncrepl multiple providers?
by Michael Ströder 16 Feb '15

16 Feb '15

Mark R Bannister wrote: > Is it possible to configure syncrepl on the consumer side with a failover > policy so that if one provider goes down it can automatically retry against > another? On each consumer you should use separate syncrepl directives for each provider you want to get data from. Two providers -> two syncrepl statements. Note the hint about RID values for each syncrepl statement in the docs. > That would remove the need for separate load balancers. No need for any kind of load balancers serving syncrepl. I'd rather consider load balancers for replication harmful. You may want to have a look in the admin guide: http://www.openldap.org/doc/admin24/replication.html Ciao, Michael.

2 2

How to debug single object syncrepl failure
by Geoff Crompton 16 Feb '15

16 Feb '15

Hi, I have a particular object in my LDAP database that is failing to replicate (using syncrepl between two slapd's running 2.4.31-1+nmu2 on Debian Wheezy), despite other objects succeeding to replicate. I'm not using a 'filter' configuration in my olcSyncrepl configuration that might exclude this particular object, and I've checked that the binddn I'm using has permission to see this object all the attributes of the object that isn't replicating. The (sanitised) configuration on the consumer is: dn: olcDatabase={1}hdb,cn=config olcSyncrepl: {0}rid=104 provider=ldap://producer.example.com bindmethod=simple binddn="uid=replicator,ou=pseudoaccounts,dc=example,dc=com" credentials="..." searchbase="dc=example,dc=com" logbase="cn=accesslog" logfilter="(& (objectClass=auditWriteObject)(reqResult=0))" schemachecking=off type=refreshAndPersist retry="60 +" syncdata=accesslog starttls=critical tls_reqcert=demand On the producer the overlay configuration for the database being replicated is: dn: olcOverlay={1}syncprov,olcDatabase={1}hdb,cn=config objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: {1}syncprov olcSpCheckpoint: 100 600 olcSpSessionlog: 100 olcSpNoPresent: TRUE If I follow the sanitising I did in the above, then the failing object would be uid=replicationcheck,ou=pseudoaccounts,dc=example,dc=com, and a successfully replicated object would be uid=geoffc,ou=People,dc=example,dc=com. I've stopped slapd on the consumer and deleted all the /var/lib/ldap/ database files, to force re-replication. I get the same symptoms, this one object doesn't replicate, but lots of other objects do replicate. Any tips on how to further debug this? Many thanks, -- Geoff Crompton, System Administrator T: +61 (0)3 9348 7138 Trinity College | University of Melbourne | Royal Parade, Parkville | Victoria 3052, Australia www.trinity.unimelb.edu.au

4 4

ldap_result() timeout zero returns only one result at a time
by Mark R Bannister 15 Feb '15

15 Feb '15

Hi, I've discovered this with the python-ldap module, but having discussed it with the author and done some further testing I'm pretty sure this is down to the ldap_result() C API. I have written a daemon that has a number of worker threads. A worker thread can launch an async LDAP search operation, and then it goes off to do other work while the search is running. It adds the FD of the socket used for the search operation to its select() call, and if there is data to read on that FD it wakes up and runs ldap_result() to grab those results. If the results are incomplete, it goes back around the loop again (check for more work to do, call select, wake up, ldap_result etc.) When I call ldap_result(), the timeout parameter is 0. It turns out, however, this is not what I want. When I run an LDAP search operation with say 80,000 entries, examining the program with truss (on Solaris) on strace (on RHEL) reveals that I end up going around that loop 80,000 times, I call select() 80,000 times, I call ldap_result() 80,000 times ... one per entry. The behaviour I actually want is for ldap_result() to return all the items currently available in kernel buffers but without waiting for any new network packets to arrive. The best approximation I can get for this is to use a timeout value of 0.0001. Having tried various values, this is best for the hardware I'm testing with when OpenLDAP is listening on the loopback interface. It reduces the number of times I called select() from 80,000 times to about 3,000 times, and shaved 5 seconds off the total execution time. However, I'm acutely aware that results will be hardware dependent, and will differ when I'm using a physical network interface. So what I'm asking is, what is the best way of achieving the following workflow: - start LDAP search - start loop - select() on FD, wake me up when there is data to read - ldap.result() gives me ALL available data, and without waiting for anything new - Attend to other unrelated events - Go round loop again Thanks & regards, Mark Bannister. http://dbis.sf.net DBIS, a replacement for RFC2307

2 2

RE: Is OpenLDAP PAE Aware?
by Jack Kielsmeier 14 Feb '15

14 Feb '15

I suppose I should also mention the older version of OpenLDAP we are running. It's openldap-2.4.17. From: Jack Kielsmeier Sent: Friday, February 13, 2015 9:40 AM To: openldap-technical(a)openldap.org Subject: Is OpenLDAP PAE Aware? Hello, We have an older LDAP server running on a 32-bit Linux OS. We've had slapd crash 2 times now, running out of memory. The box only has 2GB of RAM, but it's using PAE capable hardware and PAE support is enabled in Linux. We are debating if we should simply just add more RAM to the box, or rebuild the server. Obviously, rebuilding the server would be the best solution, but adding more RAM would be the quickest. Is 32-bit OpenLDAP able to address 4GB of memory? Thanks, - [cid:image003.png@01CFB713.F712CDA0]<http://www.lightedge.com/> Jack Kielsmeier Systems Engineer Phone: 515-471-1712 E-mail: jkielsmeier(a)lightedge.com<mailto:jkielsmeier@lightedge.com> Follow Us: [cid:image012.png@01CFB715.39A84900] <http://twitter.com/lightedge> [cid:image013.png@01CFB715.39A84900] <http://www.linkedin.com/groups?gid=1846184&trk=hb_side_g> [cid:image014.png@01CFB715.39A84900] <http://www.facebook.com/pages/Des-Moines-IA/LightEdge-Solutions/105451056954> [cid:image010.png@01CFB713.F712CDA0] <http://blog.lightedge.com/> "This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation."

3 3

Is OpenLDAP PAE Aware?
by Jack Kielsmeier 14 Feb '15

14 Feb '15

Hello, We have an older LDAP server running on a 32-bit Linux OS. We've had slapd crash 2 times now, running out of memory. The box only has 2GB of RAM, but it's using PAE capable hardware and PAE support is enabled in Linux. We are debating if we should simply just add more RAM to the box, or rebuild the server. Obviously, rebuilding the server would be the best solution, but adding more RAM would be the quickest. Is 32-bit OpenLDAP able to address 4GB of memory? Thanks, - [cid:image003.png@01CFB713.F712CDA0]<http://www.lightedge.com/> Jack Kielsmeier Systems Engineer Phone: 515-471-1712 E-mail: jkielsmeier(a)lightedge.com Follow Us: [cid:image012.png@01CFB715.39A84900] <http://twitter.com/lightedge> [cid:image013.png@01CFB715.39A84900] <http://www.linkedin.com/groups?gid=1846184&trk=hb_side_g> [cid:image014.png@01CFB715.39A84900] <http://www.facebook.com/pages/Des-Moines-IA/LightEdge-Solutions/105451056954> [cid:image010.png@01CFB713.F712CDA0] <http://blog.lightedge.com/> "This message may contain confidential and/or privileged information. If you are not the addressee or authorized to receive this for the addressee, you must not use, copy, disclose, or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by reply e-mail and delete this message. Thank you for your cooperation."

2 1

Inconsistent answer from LDAP Server
by Yoann Gini 13 Feb '15

13 Feb '15

Hello, I’m working with OS X Server 10.9 who’s using OpenLDAP to provide directory services. I’ve a problem with a simple LDAP request who is not consistent depending of the attribute filter requested. Here are the two commands I’ve used in my CLI: ldapsearch -x -H ldap://192.168.42.10 -b "cn=passwordserver,cn=config,dc=example,dc=com" "(objectclass=*)" and ldapsearch -x -H ldap://192.168.42.10 -b "cn=passwordserver,cn=config,dc=example,dc=com" "(objectclass=*)" apple-xmlplist For each request, the content of apple-xmlplist is different. The problem is the same with ldap_search_ext_s in C. I’ve looked the request with wireshark and the only difference I see is the presence or not of the attribute request. The rest of the request is exactly the same. I don’t understand how is it possible. Do you know this bug? Do you have a way to edit the stored record used for the request with the filter (it’s the one used by Apple internal tools, so it’s not really possible for me to change it). Here are version info: @(#) $OpenLDAP: slapd 2.4.28 (Nov 12 2013 12:02:47) $ root@hikkaduwa.apple.com:/private/var/tmp/OpenLDAP/OpenLDAP-491.1~1/servers/slapd ldapsearch: @(#) $OpenLDAP: ldapsearch 2.4.28 (Nov 12 2013 12:04:43) $ root@hikkaduwa.apple.com:/private/var/tmp/OpenLDAP/OpenLDAP-491.1~1/clients/tools (LDAP library: OpenLDAP 20428) Best regards, Yoann Gini

5 18

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical