openldap-technical

openldap-technical@openldap.org

9 participants
9866 discussions

Re: AW:ldap server stops responding periodically?
by John Jasen 23 Jul '18

23 Jul '18

On 07/20/2018 04:41 AM, Ulrich.Windl(a)rz.uni-regensburg.de wrote: > Hi! > > Stupid question: could it be your load-balancer that had a problem? > How does the netstat look like (sockets opened, queued data, etc.?) I do not believe it to be the load-balancer. They log loss of contact with the LDAP servers and drop them from the relay group shortly after one of these events start; and when it gets cleaned up, they're added back in. I also do not suspect network between the load balancers and the LDAP servers. During such an event, ps -efT will usually show slapd running at full thread capacity. Comparing that to threads in cn=monitor is not possible, as those ldap searches fail. Open sockets does not substantially change until after the event subsides. The servers will show 1200-2000 open sockets before an event, and drop lower when it clears up -- to quickly scale back up to pre-event levels. The queues will show data being held until the socket(s) time out. Thanks for the feedback!

2 1

Re: ldap server stops responding periodically? WAS: openldap-technical Digest, Vol 128, Issue 12
by John Jasen 21 Jul '18

21 Jul '18

On 07/21/2018 08:00 AM, openldap-technical-request(a)openldap.org wrote: > From: Ryan Tandy <ryan(a)nardis.ca> To: openldap-technical(a)openldap.org > Subject: Re: ldap server stops responding periodically? Message-ID: > <20180720185004.po4pckupxyjrrih4(a)comet.nardis.ca> Content-Type: > text/plain; charset=iso-8859-1; format=flowed On Fri, Jul 20, 2018 at > 08:39:55PM +0200, Dieter Kl?nter wrote: >> If that really is 2.4.4 > Given the mention of "CentOS7 stock RPM" I would expect this was a typo > and John is actually running 2.4.44. Yes, my apologies. I meant 2.4.44. -- John Jasen

1 0

back-ldap redirect RootDSE? RWM re-encode attributes?
by B Galliart 21 Jul '18

21 Jul '18

I am trying to use OpenLDAP slapd to proxy to multiple LDAP servers but run into some issues with how the LDAP client application is written. I wondered if anyone has any suggestions for doing either of the following with slapd: (1) Can the back-ldap suffix override the RootDSE of the OpenLDAP server so that all RootDSE requests get proxied? (2) Is there a way with the RWM or a different overlay to convert specific attributes to hex or MIME before passing it back to the LDAP client? For example, if an attribute is given to back-ldap as a BLOB, can it convert the BLOB to hex before passing it along? It would be nice to have the option to do both of the above but for my current situation being able to do just one or the other would work. Thanks

1 0

ldap server stops responding periodically?
by John Jasen 20 Jul '18

20 Jul '18

Summary: an openldap 2.4.4 (CentOS7 stock RPM) replication consumer slapd server stops responding to requests for a period of up to fifteen minutes. Environment: Two centos7 ldap servers in mirror mode, providers to 4 openldap syncrepl consumers. The systems are 2 CPU, 12 core Intel Xeon E5-2420s, and have 48GB of RAM. The four consumers are load-balanced through a FreeBSD "relayd" redirector, facing approximately six thousand clients. Problem: Periodically, one or more (or all) of the consumers will stop responding, including localhost cn=monitoring traffic and anything over the network. Note, only slapd stops responding. email out, logging in, etc, all remain unaffected. Analysis after the event starts doesn't show anything unusual in CPU usage or memory. Analysis of the ldap logs doesn't show anything unusual in number of requests, number of connects, etc until the system stops responding -- at which point, they drop to zero. I'm stumped as to a) what's causing it, and b) how to address it on the slapd side so my servers stop dozing off. Any suggestions? -- John Jasen (jjasen(a)gmail.com)

4 3

Re: ldap server stops responding periodically? WAS: openldap-technical Digest, Vol 128, Issue 11
by John Jasen 20 Jul '18

20 Jul '18

On 07/20/2018 08:00 AM, openldap-technical-request(a)openldap.org wrote: > Date: Thu, 19 Jul 2018 13:50:47 -0500 From: Andy Dorman > <adorman(a)ironicdesign.com> To: openldap-technical(a)openldap.org > Subject: Re: ldap server stops responding periodically? Message-ID: > <6ca235e9-10e6-df27-f06c-359875407a6b(a)ironicdesign.com> Content-Type: > text/plain; charset=utf-8; format=flowed > <snip> > So when this happens you can still log into the consumer and run typical > slapdump/slapcat, ldapsearch, etc. commands with no problem? or are you > only losing access via the network? I've not tried slap* commands on an afflicted consumer, nor have I tried locally through ldapi:/// Both are good ideas. Thank you. local lapsearch, etc with a URL to the listening network socket fail from localhost as well.

1 0

Where does the debugging message come from?
by Ray Lauff 16 Jul '18

16 Jul '18

Hello- I'm testing out some local mods I've made to the ldapsearch command and I have the -d 2 flag to enable debugging set. /usr/local/src/openldap/openldap-2.4.46/clients/tools/ldapsearch -h ldap-test1.test.org -p 11389 -ZZ -D cn=somecn -w somepw -b 'dc=test,dc=org' '(uid=husk)' -d 2 When I run this, I get a lot of what I need, but I noticed the following is emitted at the top. *Using CA cert /etc/openldap/cacerts/ca-bundle.crt as stipulated by environment variable $LDAPTLS_CACERT.* I was looking to see where in the openldap code this string comes from, but I can't seem to find it. (I thought a thorough 'grepping' would locate it, but it did not, nor does strings /usr/local/src/openldap/openldap-2.4.46/clients/tools/ldapsearch |grep "stipulated" ). So it's from the ssl library I assumed. Nope, didn't find it there either. Can someone point me to where it comes from please? Sorry if this is a dumb question that I should already know.

2 1

Is checkpoint required any more?
by Geoff Swan 11 Jul '18

11 Jul '18

I am running openldap-2.4.44 (source build) using lmdb backend with configuration set in slapd.conf on a Linux machine. The configuration includes dbnosync and also checkpoint settings. The kernel vm settings are flushing dirty cache pages to disc periodically, so my question is what is the reason for the checkpoint setting if the kernel is already taking care of page flushing?

2 1

OpenSLP anyone?
by Michael Ströder 09 Jul '18

09 Jul '18

HI! Is anybody here using OpenSLP with OpenLDAP libs to locate LDAP directory servers? I never used it and I only vaguely remember that Novell eDirectory instances was announced via SLP. But I don't know of any clients supporting that and therefore I wonder whether Linux distros should package libldap with SLP support. Ciao, Michael.

2 1

back-mdb and alias dereferencing
by Henrik Bohnenkamp 07 Jul '18

07 Jul '18

Hi, I would like to bring up the problem with back-mdb and alias-dereferening again. There is Bug ITS#7657 for this already, and in http://www.openldap.org/lists/openldap-technical/201504/msg00008.html the problem was also addressed. Probably in other places as well, but I did not find anything else in the archives. I would like to reconfigure all my LDAP clusters to use back-mdb, however, I can't: my LDAP tree has about 2 Mio entries, 600000 of which are alias entries. I observe that when using back-mdb and making search request with search scope "sub" and deref=always, than this search request will take seconds before a response... in contrast, with deref=never, the same search request takes only millisecs. It does not matter whether there are aliases in the search scope or not... with deref=always it always takes seconds, each request keeping one CPU thread very busy. So if many requests come in with sub/always (which unfortunately happens a lot because many LDAP clients just set deref=always as a default) it is a sure way to bring down an LDAP cluster configured with back-mdb. The problem has haunted me for a while (incidents, incidents :-), and I have tried to find out where it comes from. I would like to share my conjectures here. A search request with sub/always eventually ends up in the function search_aliases(..), located in back-mdb/search.c (code/linenumbers is from v2.4.46 ): 157 /* Find all aliases in database */ 158 MDB_IDL_ZERO( aliases ); 159 rs->sr_err = mdb_filter_candidates( op, isc->mt, &af, aliases, 160 curscop, visited ); 161 if (rs->sr_err != LDAP_SUCCESS || MDB_IDL_IS_ZERO( aliases )) { 162 return rs->sr_err; 163 } 164 oldsubs[0] = 1; 165 oldsubs[1] = e_id; 166 167 MDB_IDL_ZERO( visited ); 168 MDB_IDL_ZERO( newsubs ); 169 170 cursoro = 0; 171 ido = mdb_idl_first( oldsubs, &cursoro ); 172 173 for (;;) { 174 /* Set curscop to only the aliases in the current scope. Start with 175 * all the aliases, then get the intersection with the scope. 176 */ 177 rs->sr_err = mdb_idscope( op, isc->mt, e_id, aliases, curscop ); After adding a bit of extra debug code, I could see in the logs that the time is actually spent in line 177, i.e. the mdb_idscope function. What I believe is happening is the following: in line 159 the index list of all aliases in the DB is constructed and put into the "aliases" list. The list comes from the objectClass index, but since there are so many alias entries, the index contains not an explicit list, but a range... a range that covers basically *all* 2 Mio entries in the DB. (this happens also with the hdb backend, so the fact that "aliases" is a range is perhaps part, but not cause of the problem). Then the for loop is entered in 173, and mdb_idscope in 177 is called. The idea there is to compute the intersection of alias list "aliases" and search scope, i.e. get all aliases that are in the actual search scope. This is done by walking through the 2 millions indexes represented by the range in "aliases", and deciding whether the id is in the scope or not. This, if I understand correctly, by walking up the LDAP tree from leaf towards root (actually walking up the structure of the dn2id DB) ... if we encounter the search base on the way, the alias is part of the search scope, if we reach the root, then not. Computing this intersection is what costs time, and it does not depend on whether the intersection is empty or not. Apparently it is done differently in back_hdb and by extension, back_bdb. I have not analysed that code there in detail but I believe the intersection is computed by actually looking at 2 different ID lists, one containing the aliases (range) again, the other one the actual IDs of the search scope. Which, it appears, is more efficient. So the problem seems to be to compute the relevant aliases to start expanding from. The way it is done now the complexity seems to be linear in the number of aliases, if not number of entries on the LDAP tree. For my own use cases at least it would be much more efficient to just do a tree traversal of the search scope, pick up the relevant aliases and put them in the list. I believe even for large trees this would work quite well... if I search my whole tree for entries with objectClass=alias (deref=never, of course), this is amazingly fast. I am currently looking into how something like that could be implemented, but I need to dive much deeper in the code before I could come up with something worth considering. What are the views of the developers on this? Does the analysis sound plausible? Could a fix be easily obtained? I'm willing to try out any patch thrown my way. Somewhere is written that mdb will replace hdb/bdb eventually ... I believe this problem needs a fix before that can happen. Henrik -- Henrik Bohnenkamp Senior Sysadmin Linux & Applications Operation United Internet Corporate Services GmbH | Brauerstraße 50 | 76135 Karlsruhe | Germany Phone: +49 721 91374 - 4159 E-mail: hbohnenkamp atatatat united-internet.de | Web: www.united-internet.de Amtsgericht Montabaur / HRB 23031 Geschäftsführer: Verena Amann, Markus Kadelke, Frank Krause, Guido Mannshausen Member of United Internet This e-mail may contain confidential and/or privileged information. If you are not the intended recipient of this e-mail, you are hereby notified that saving, distribution or use of the content of this e-mail in any way is prohibited. If you have received this e-mail in error, please notify the sender and delete the e-mail.

2 2

Fixing slapo-nops (ITS#8759)
by manu＠netbsd.org 06 Jul '18

06 Jul '18

Hello Following ITS#8759, I am looking for ideas about how to fix slapo-nops. This overlays suppress nilpotent operations that replace an attribute values by the same values. The code is in contrib/slapd-modules/nops/nops.c The problem is that slapo-nops assumes it can free the nilpotent struct Modifications after removing them from the list, but for instande slapo-memberof uses struct Modifications allocated on the stack. Not freeing in slapo-nops is not a fix, since unlinked struct Modifications will not be freed anywhere else, and we will get a memory leak. Would it make sense to add a SLAP_MOD_ONSTACK flag in struct Modification's sm_flags so that we can tell the difference? Any other idea? -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu(a)netbsd.org

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical