openldap-technical

openldap-technical@openldap.org

2 participants
9943 discussions

Virtual list view problem
by Venish Khant 31 May '16

31 May '16

Hi all I am using cpan Net::LDAP module to access LDAP entries. I want to search LDAP entries using Net::LDAP search method. When I do search, I want some limited number of entries from search result, for this(searching) process I am using Net::LDAP::Control::VLV module. But I get error on VLV response control. Please, any one have idea about this error. * Error:* Died at vlv.pl line 50, This is my example. I changed the font style of line 50 #!/usr/bin/perl -w use Net::LDAP; use Net::LDAP::Control::VLV; use Net::LDAP::Constant qw( LDAP_CONTROL_VLVRESPONSE ); use Net::LDAP::Control::Sort; sub procentry { my ( $mesg, $entry) = @_; # Return if there is no entry to process if ( !defined($entry) ) { return; } print "dn: " . $entry->dn() . "\n"; @attrs = $entry->attributes(); foreach $attr (@attrs) { #printf("\t%s: %s\n", $attr, $entry->get_value($attr)); $attrvalue = $entry->get_value($attr,asref=>1); #print $attr.":". $entry->get_value($attr)."\n"; foreach $value(@$attrvalue) { print "$attr: $value\n"; } } $mesg->pop_entry; print "\n"; } $ldap = Net::LDAP->new( "localhost" ); # Get the first 20 entries $vlv = Net::LDAP::Control::VLV->new( before => 0, # No entries from before target entry after => 19, # 19 entries after target entry content => 0, # List size unknown offset => 1, # Target entry is the first ); my $sort = Net::LDAP::Control::Sort->new( order => 'cn' ); @args = ( base => "dc=example,dc=co,dc=in", scope => "subtree", filter => "(objectClass=inetOrgPerson)", callback => \&procentry, # Call this sub for each entry control => [ $sort, $vlv ], ); $mesg = $ldap->search( @args ); # Get VLV response control *($resp) = $mesg->control( LDAP_CONTROL_VLVRESPONSE ) or die;* $vlv->response( $resp ); # Set the control to get the last 20 entries $vlv->end; $mesg = $ldap->search( @args ); # Get VLV response control ($resp) = $mesg->control( LDAP_CONTROL_VLVRESPONSE ) or die; $vlv->response( $resp ); # Now get the previous page $vlv->scroll_page( -1 ); $mesg = $ldap->search( @args ); # Get VLV response control ($resp) = $mes # Now page with first entry starting with "B" in the middle $vlv->before(9); # Change page to show 9 before $vlv->after(10); # Change page to show 10 after $vlv->assert("B"); # assert "B" $mesg = $ldap->search( @args );g->control( LDAP_CONTROL_VLVRESPONSE ) or die; $vlv->response( $resp ); -- Venish Khant www.deeproot.co.in

4 11

Re: Documentation
by Daniel Howard 29 May '16

29 May '16

My recent experience is OpenLDAP on Ubuntu. I thought I would go with OpenLDAP's guide because they should know better, but the quick start was for older versions or something and hadn't been updated. I like documentation systems that allow for user feedback, comments, or patches via github. If you want community improvements then maybe low hanging fruit is a link in the doc template on how to interact with the documentation.... If the current docs platform has no mechanism for feedback and comments on a specific document, the source code/ build process is obscure ... Maybe consider porting even a "fork " to github and solicit community feedback (you can file issues in a familiar mechanism, and take in patches from dudes like me...) For what it's worth, the Ubuntu OpenLDAP quick start guide and only a few minor hiccups, and I would recommend that resource for anyone looking to implement or update a quickstart guide. The Faq-o-Matic ... Man a modern stack overflow site might be nice but a question in the back of my mind is whether the advice is still useful. I don't know if the FoM does this but if the answers showed dates then I could evaluate an answer that's say 2-4 years old with greater confidence than say something 12 years old ... One of the challenges of a venerable, old and mature project like OpenLDAP. I guess what I'm trying to say, from my mobile phone, is if the documentation were more "agile" in engaging the community for identifying issues and taking in corrections, we may all be happier. I would be open to volunteering to port docs... Thanks, -Danny -- Sent from my Western Electric Model 500 On Apr 29, 2016 5:01 PM, "Gavin Henry" <ghenry(a)openldap.org> wrote: Hi all, List what you love about our docs: List what you hate: List what you'd love to see: List what you can help with: Thanks. -- Kind Regards, Gavin Henry.

6 9

slapadd hang in bdb_tool_entry_close / ldap_pvt_thread_cond_wait
by Bannister, Mark 27 May '16

27 May '16

Hi, Can anyone shed light on why my slapadd process on Solaris 11 was hanging (slapd 2.4.30)? I was piping 'gunzip -c' through to slapadd (from an overnight cron job). This usually works fine. I'm guessing perhaps the gunzip process encountered an error, maybe an EPIPE was sent, but the stack trace many days later from slapadd doesn't look right. Only 2 threads, and both of them in ldap_pvt_thread_cond_wait(), but no other threads so what are they waiting for exactly? # pstack /var/tmp/cores/core.slapd.0.16189.1463492837 core '/var/tmp/cores/core.slapd.0.16189.1463492837' of 16189: /usr/sbin/slapadd -q -f /etc/openldap/slapd.conf ------------ lwp# 1 / thread# 1 --------------- 07ff7af9 lwp_park (0, 0, 0) 07ff1193 cond_wait_queue (82ed048, 82ed020, 0, 7ff16b0) + 63 07ff1728 __cond_wait (82ed048, 82ed020, f5a685c8, 7ff1771) + 89 07ff177f cond_wait (82ed048, 82ed020, f5a685f8, 7ff17b7) + 27 07ff17cc pthread_cond_wait (82ed048, 82ed020, f5a68618, 7e56137) + 24 07e5614c ldap_pvt_thread_cond_wait (82ed048, 82ed020, f5a68648, 81b7e29) + 24 081b7e7a bdb_tool_entry_close (8e9d398, ffffffff, f5a68698, 82ecc38) + 62 081a38ec slapadd (4, f5a68bbc, f5a68bbc, 80dccb1) + cb8 080dcd2c main (4, f5a68bbc, f5a68bd0, ef60f968) + ac 080dc86d _start (4, f5a68c8e, f5a68ca0, f5a68ca3, f5a68ca6, 0) + 7d ------------ lwp# 2 / thread# 2 --------------- 07ff7af9 lwp_park (0, 0, 0) 07ff1193 cond_wait_queue (8e6b0e4, 8e6b0cc, 0, 7ff16b0) + 63 07ff1728 __cond_wait (8e6b0e4, 8e6b0cc, d7dffdb8, 7ff1771) + 89 07ff177f cond_wait (8e6b0e4, 8e6b0cc, 8e6b0c8, 7ff17b7) + 27 07ff17cc pthread_cond_wait (8e6b0e4, 8e6b0cc, 0, 7e56137) + 24 07e5614c ldap_pvt_thread_cond_wait (8e6b0e4, 8e6b0cc, d7dfffc8, 7e54bfc) + 24 07e54d66 ldap_int_thread_pool_wrapper (8e6b0c8, 803a000, d7dfffe8, 7ff7769) + 242 07ff77bc _thrp_setup (ee1e0240) + 9d 07ff7aa0 _lwp_start (ee1e0240, 0, 0, 0, 0, 0) Thanks, Mark. ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies; do not disclose, use or act upon the information; and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By messaging with Morgan Stanley you consent to the foregoing.

2 7

Improved handling for large number of databases / Access newly opened database from another transaction
by Jürg Bircher 27 May '16

27 May '16

Hello Improved handling for large number of databases =============================================== There is a increased performance penalty the more databases are created within the same environment. I was looking for a way the improve that by keeping the simplicity of tracking databases within a list with direct access by index (MDB_dbi). mdb_dbi_open() is however not improved with the assumption that the database handle (dbi) is cached in the application. So mdb_dbi_open() should happen only once for each database during the life time of an application. One issue is that mdb_txn_begin() (for read-only transactions) calloc the sizeof(MDB_txn) + me_maxdbs * sizeof(MDB_db + 1). The plus 1 for the dbflags. However it is sufficient only to malloc that size and clear the sizeof(MDB_txn) memset(txn, 0, sizeof(MDB_txn) After that the data beyond the MDB_txn is not initialized which is ok for the moment. The next improvement happens in mdb_txn_renew0() where the dbflags are only set to DB_UNUSED (a new flag) for each database currently opened in the environment. memset(txn->mt_dbflags, DB_UNUSED, txn->mt_numdbs); The former code used to to loop through each database to calculate the dbflags. This is still done but lazily for each accessed database with the assumption that a read only transaction rarely uses all databases of the environment. The lazy initialization of the dbflag happens in the macro TXN_DBI_EXIST which is always used when a database handle (dbi) is passed to an function. The flags are updated in mdb_setup_db_info() once a database is access which is marked as unused (DB_UNUSED). static int mdb_setup_db_info(MDB_txn *txn, MDB_dbi dbi) { /* Setup db info */ uint16_t x = txn->mt_env->me_dbflags[dbi]; txn->mt_dbs[dbi].md_flags = x & PERSISTENT_FLAGS; txn->mt_dbflags[dbi] = (x & MDB_VALID) ? DB_VALID|DB_USRVALID|DB_STALE : 0; return (txn->mt_dbflags[dbi] & validity); } /** Check \b txn and \b dbi arguments to a function and initialize db info if needed */ #define TXN_DBI_EXIST(txn, dbi, validity) \ ((txn) && (dbi)<(txn)->mt_numdbs && (((txn)->mt_dbflags[dbi] & (validity)) || (((txn)->mt_dbflags[dbi] & DB_UNUSED) && mdb_setup_db_info((txn), (dbi), (validity))))) The next improvement is done in any function which needs to loop through the databases for example in mdb_cursors_close(). Again the more databases in the environment the longer the execution time. It should be best if looping only through dbflags and searching for those databases which are used (!DB_UNUSED). This could be done byte wise or more efficient in 8/4 byte steps comparing with an extended mask DB_UNUSED_LONG instead of DB_UNUSED. So we can skip 8 or 4 (32 bit) unused databases in one step (still with the assumption that a transaction rarely uses all databases of the environment). So the loop looks as follows always starting at the lower index to avoid alignment issues with ARM prior v6. #define DB_UNUSED 0x20 /**< DB not used in this txn */ #ifdef MDB_VL32 #define DB_UNUSED_LONG 0x20202020 /* DB_UNUSED long mask for fast tracking */ #else #define DB_UNUSED_LONG 0x2020202020202020 /* DB_UNUSED long mask for fast tracking */ #endif #ifdef MDB_VL32 #define MDB_WORD unsigned int #else #define MDB_WORD unsigned long long #endif MDB_dbi n = src->mt_numdbs; MDB_dbi i = 0; while (1) { unsigned int upper = i + sizeof(MDB_WORD); if (upper < n) { // skip unused if ((*(MDB_WORD *)(tdbflags + i)) == DB_UNUSED_LONG) { i = upper; continue; } } else { upper = n; } for (; i < upper; i++) { // any other filter criteria appropriate to the function .... } if (i >= n) { break; } } Access newly opened database from another transaction ======================================================= A transaction tracks newly opened databases and if the transaction is committed the newly opened databases are propagated to the list of open databases of the environment. However if the read only transaction is aborted the databases are not propagated. If database handles (MDB_dbi) are cached in the application to avoid calling mdb_dbi_open() there might be the situation of two threads running a read only transaction concurrently. First threads opens the database and commits the read-only transaction. The database is added to the list of open databases in the environment. The returned dbi is globally cached in the application. The second thread also wants to access the same database and finds the database handle in the global application cache. The database handle however is not valid as the transaction only uses a snapshot of open databases. So the second thread gets an EINVAL when using that database handle. This should not happen as the database is open and added to the environment. The following should fix this issue by updating the mt_numdbs and marking the delta with DB_UNUSED static int mdb_update_db_info(MDB_txn *txn, MDB_dbi dbi) { /* propagate newly opened db from env to current txn. Mark them as unused */ if (txn->mt_numdbs < txn->mt_env->me_numdbs) { memset(txn->mt_dbflags + txn->mt_numdbs, DB_UNUSED, txn->mt_env->me_numdbs - txn->mt_numdbs); } txn->mt_numdbs = txn->mt_env->me_numdbs; return dbi < txn->mt_numdbs; } /** Check \b txn and \b dbi arguments to a function and initialize db info if needed */ #define TXN_DBI_EXIST(txn, dbi, validity) \ ((txn) && ((dbi)<(txn)->mt_numdbs || mdb_update_db_info((txn), (dbi))) && (((txn)->mt_dbflags[dbi] & (validity)) || (((txn)->mt_dbflags[dbi] & DB_UNUSED) && mdb_setup_db_info((txn), (dbi), (validity))))) If interested let me know how to contribute. Hope it is useful! Regards Juerg

2 5

memberOf group searches very slow
by Jason Whitener 27 May '16

27 May '16

We have some groups that contain over 250,000 members. Are there any general tips for improving performance when applications/code attempts to lookup memberOf values in the ldap? For example, I have something like ou=SomeGroup,ou=Groups,o=ourOrg. ou=SomeGroup has over 250,000 memberOf: values. Attempting to modify, search, insert, etc.. into that memberOf multivalue field takes over 20 seconds per user.

2 1

problems connecting to ldaps:// under high load with ppc64 client
by Matthias Leopold 27 May '16

27 May '16

hi, i'm operating an owncloud server that connects to an IBM Tivoli Directory Server as LDAP backend. the ldap admin tells me he is seeing "null binds" from my owncloud server in his logs: 2016-05-24T14:32:56.349452+2:00 srvr_ssl_read: EIO in handshake. EWOULDBLOCK timeout. Read: -2 of 0 2016-05-24T14:32:56.350445+2:00 GLPSSL019E The SSL layer has reported an unidentified internal error, SSL extended error code:406. 2016-05-24T14:32:56.351813+2:00 GLPSRV022E Failed to initialize secure connection from client (connection ID: 61786, IP address: x.x.x.x, Port: 59921). 2016-05-24T14:32:56.357220+2:00 GLPSRV044W Client connection from x.x.x.x bound as NULL closed by server. i investigated on my server and noticed that it has problems connecting to the ldaps://ldap.example.com uri (which is the ITDS server) under high client system load, whereas connection to ldap://ldap.example.com is ok. $ ldapsearch -v -x -z 0 -H ldaps://ldap.example.com -b "ou=groups,dc=example,dc=com" -v "objectClass=posixGroup" ldap_initialize( ldaps://ldap.example.com:636/??base ) ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) my server (RHEL 7 on a ppc64 LPAR) is using the openldap clients/libraries. the high load that is causing the problems is on _my_ server. is there any specific tuning (besides increasing RAM/CPU) i can do to optimize ldaps client queries? i'm thinking of tuning the tcp stack or something similar, but i'm not an expert on this. where can i look for debug info? i have strace and tcpdump output thx matthias

3 2

Use OpenLDAP for some users and as a proxy for another set of users?
by Siebrand Mazeland 26 May '16

26 May '16

Hi. I'm a first time poster, new to OpenLDAP, and I have identified this list as the (hopefully) best place for my question. I have an Active Directory that contains accounts and groups for employees. Besides that, there is a group of around 1000 people that also need to authenticated and authorized (based on group membership). I'm trying to assess if OpenLDAP can be used for a scenario to avoid Windows CAL license costs. Is it possible to administer and authenticate the non-employees in OpenLDAP, and proxy requests about users that are not found in OpenLDAP to an AD? The information needed by the applications using OpenLDAP would be UPN, sAMAccountName, email address and group membership of the authenticated users. If this can be accomplished with OpenLDAP, that would a) be very nice, and b) I would like you to explain this in brief here, and approach me off-list to help me accomplish this. If there's no ready-made recipe for this, and it can be done, I'm willing to publish the configuration so others can benefit from the work, too. Thanks. -- Siebrand Mazeland

2 1

ABI report for LMDB
by Ponomarenko Andrey 26 May '16

26 May '16

Hello, The LMDB library is now tracked by the ABI tracker project: http://abi-laboratory.pro/tracker/timeline/lmdb/ One can look at the recent changes in the API/ABI and analysis of backward compatibility and navigate over the history of API/ABI changes in the library. The reports are generated daily by the abi-compliance-checker and abi-tracker tools: https://github.com/lvc/abi-tracker Thank you.

1 0

Re: entryCSN is larger (later) in consumer node than in provider node
by Frank Luo 25 May '16

25 May '16

same error if I try to update this ppolicy attribute "changetype: modify replace: pwdMinLength pwdMinLength: 9" ldap_modify: Server is unwilling to perform (53) additional info: shadow context; no update referral On Wed, May 25, 2016 at 4:26 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > --On Wednesday, May 25, 2016 5:21 PM -0400 Frank Luo <frank.luoy(a)gmail.com> > wrote: > >> yes >> >> overlay ppolicy > > > So ppolicy maintains various data internally to the server. That's likely > what is giving you #000# items. > > --Quanah > > > -- > > Quanah Gibson-Mount > Platform Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration > A division of Synacor, Inc

1 0

Re: entryCSN is larger (later) in consumer node than in provider node
by Frank Luo 25 May '16

25 May '16

yes overlay ppolicy and overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100 So it looks like on node 3 I have both syncprov and syncrepl defined? "overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100 syncrepl rid=003 ..... " But actully when I try to write to node 3 directly, I have a error like this, "ldap_modify: Server is unwilling to perform (53) additional info: shadow context; no update referral " Thanks Frank On Tue, May 24, 2016 at 7:31 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote: > Do you have any overlays on the replicas? > > --Quanah > > --On Tuesday, May 24, 2016 4:55 PM -0400 Frank Luo <frank.luoy(a)gmail.com> > wrote: > >> but we always have two masters - and you can tell that this is pretty >> current time tag. >> >> contextCSN: 20160521215740.310825Z#000000#000#000000 >> >> Does this mean the slave gets updated from some other source in >> additional to the two masters. >> >> I am attache the replication portion of the slaver sever 3, >> server2.u.com is one of the two masters >> >> >> syncrepl rid=003 >> provider=ldap://server2.u.com >> bindmethod=simple >> binddn="cn=Manager,dc=u,dc=com" >> credentials=password >> searchbase="dc=u,dc=com" >> schemachecking=on >> type=refreshAndPersist >> retry="60 +" >> >> Thanks >> >> Frank >> >> >> >> On Tue, May 24, 2016 at 12:54 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> >> wrote: >>> >>> Sure -- 000 is from when things were single master. >>> >>> --Quanah >>> >>> --On Tuesday, May 24, 2016 12:37 PM -0400 Frank Luo >>> <frank.luoy(a)gmail.com> wrote: >>> >>>> Thanks. The two servers were off by 15 seconds >>>> >>>> I also checked contextCSN and see there were more than 1 values - any >>>> idea there are multiple? >>>> >>>> To help you understand we have a multiple(2) masters and three slaves. >>>> >>>> on two masters (node1 and node 2), each has two contextCSN like this, >>>> I can understand since they are in mirror mode, it get updates from >>>> each other >>>> >>>> node 1: >>>> contextCSN: 20160524152519.525094Z#000000#001#000000 >>>> contextCSN: 20160524152340.159717Z#000000#002#000000 >>>> >>>> node 2: >>>> contextCSN: 20160524152519.525094Z#000000#001#000000 >>>> contextCSN: 20160524152340.159717Z#000000#002#000000 >>>> >>>> >>>> on three slavers (nodes3,4 and5), each has 3 contextCSN. Where does >>>> the third contextCSN come from - you can see the third one is out of >>>> sync with each other. >>>> >>>> node 3: >>>> contextCSN: 20160524152519.525094Z#000000#001#000000 >>>> contextCSN: 20160521215740.310825Z#000000#000#000000 >>>> contextCSN: 20160524152340.159717Z#000000#002#000000 >>>> >>>> node 4: >>>> contextCSN: 20160524152519.525094Z#000000#001#000000 >>>> contextCSN: 20160524152558.806951Z#000000#000#000000 >>>> contextCSN: 20160524152340.159717Z#000000#002#000000 >>>> >>>> node 5: >>>> contextCSN: 20150723095205.352520Z#000000#000#000000 >>>> contextCSN: 20160524152519.525094Z#000000#001#000000 >>>> contextCSN: 20160524152340.159717Z#000000#002#000000 >>>> >>>> THanks >>>> >>>> Frank >>>> >>>> >>>> On Mon, May 23, 2016 at 5:11 PM, Quanah Gibson-Mount <quanah(a)zimbra.com> >>>> wrote: >>>>> >>>>> >>>>> --On Monday, May 23, 2016 5:56 PM -0400 Frank Luo >>>>> <frank.luoy(a)gmail.com> wrote: >>>>> >>>>>> All, >>>>>> >>>>>> I am having a out-of sync situation now with some entries' entryCSN >>>>>> is larger (later) in consumer node than in provider node. So the >>>>>> consumer never gets updated since it thinks it has the latest value. >>>>> >>>>> >>>>> >>>>> >>>>> Check the clocks on each server. It is required that your clocks be >>>>> tightly sync'd. >>>>> >>>>> --Quanah >>>>> >>>>> >>>>> -- >>>>> >>>>> Quanah Gibson-Mount >>>>> Platform Architect >>>>> Zimbra, Inc. >>>>> -------------------- >>>>> Zimbra :: the leader in open source messaging and collaboration >>>>> A division of Synacor, Inc >>> >>> >>> >>> >>> >>> -- >>> >>> Quanah Gibson-Mount >>> Platform Architect >>> Zimbra, Inc. >>> -------------------- >>> Zimbra :: the leader in open source messaging and collaboration >>> A division of Synacor, Inc > > > > > -- > > Quanah Gibson-Mount > Platform Architect > Zimbra, Inc. > -------------------- > Zimbra :: the leader in open source messaging and collaboration > A division of Synacor, Inc

2 1

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical