March 2012 - openldap-technical

OpenLDAP high CPU usage when performing mass changes

by Jeffrey Crawford

We are using openldap 2.4.26 with BDB 4.8 and have replication set up in mirror mode for our main ldap database. There are a couple of other replicas that have a subset of the data that the main cluster has but we are seeing the following behavior on all of them. When performing mass updates via LDAP, lets say on the order of 30,000 entries being added to existing entries. We've noticed that the CPU use of the slapd instances goes through the roof (between 65% and 95% continuously), and seems to stay there until it is restarted. The Problem is that this system has to be highly available, even for writing and when these updates "shock" the system, the response time goes way down when the process are turning like that. I don't think they are trying to catch up to the data changes because if I let them run a while after the updates are done. (Talking like 1hr) and then restart the instances, they go back to their normal state. So far the only way I've been able to mitigate the issues is to reconfigure our ldap proxy instances to a machine that is having less trouble, restart the instances that are chugging along, then repoint the proxies back to the one just started, and start the others. Not exactly a quick operation. I've played with cache settings for both OpenLDAP and BDB and have gotten the frequency of this issue reduced but I can't seem to get rid of it completely and it shows up quite often after large data manipulations. I'm at a loss of how to debug since nothing is crashing. Any suggestions on how to find out what's causing this would be very helpful. The logs are not throwing any warnings or posting messages that would seem out of the ordinary and I have played with the log settings but nothing seems to relate to anything that might explain why we are seeing CPU usage to go so high. Thanks in advance -- I fly because it releases my mind from the tyranny of petty things . . . — Antoine de Saint-Exupéry Jeffrey E. Crawford ITS Application Administrator (IDM) 831-459-4365 jeffreyc(a)ucsc.edu

11 years, 8 months

3
7
0 / 0

ldapi vs ldap

by Nick Milas

Hi, For applications accessing a directory (mainly Postfix, Dovecot, PowerDNS), we are replicating locally and applications connect using: ldap://localhost. We are considering using ldapi://localhost instead. Would there be any performance / reliability pros/cons? Thanks, Nick

11 years, 8 months

3
2
0 / 0

Re: URI="(null)" DN="" (Was: ldap && rwm && pcache && transparent)

by Brian Thompson

Hello, We're running into almost exactly the same issue that Turbo described a few months ago and I'm not exactly sure what to try next... Basically there are two remote directories that we only have read-only access to via fixed set of bind credentials. Both directories have the same naming contexts and same user entry DNs but a slightly different non-conflicting set of user attributes. What we would like to do is create a combined merged view that contains the user entry attribute responses from both remote directories displayed as a single entry and allow anonymous access via the local lan. Only one of the two directories has the specific attribute being searched for but again we would like the response to include the full set of attributes from both directories and the entry DNs do match. So, I'm attempting to use slapd-ldap for the first remote directory and slapo-translucent for the second remote directory but can't seem to make the two mechanisms cooperate with each other. The errors are the same as what Turbo described. If I change the database from slapd-ldap to slapd-shell or slapd-hdb (empty db), the slapo-translucent portion works correctly and I can see the attributes in the searched entry retrieved from the second remote directory. If I leave the slapd-ldap configuration in place and comment out the overlay translucent portion, the slapd-ldap portion works correctly and I can see the attributes in the searched entry retrieved from the first remote directory. Still no luck on getting both to work at the same time though so that it combines the attributes from both entries into a single response. Here's the pertinent portion of the config: backend ldap database ldap suffix "dc=example,dc=com" uri "ldap://172.16.16.16/" idassert-bind mode=none bindmethod=simple binddn="uid=readonly1,cn=admins,dc=example,dc=com" credentials="secret1" flags=override,non-prescriptive overlay translucent uri "ldap://172.17.17.17/" idassert-bind mode=none bindmethod=simple binddn="uid=readonly2,cn=admins,dc=example,dc=com" credentials="secret2" flags=override,non-prescriptive If anyone has any ideas why the above isn't working or knows of a cleaner way to do it, please let me know. It would be much appreciated. Thanks, Brian ----- Original Message ----- > On Wed, 14 Sep 2011 20:53:03 +0200, Turbo Fredriksson wrote: > > I'm trying to proxy an AD and an OpenLDAP server on a > > separate machine to get a 'combined' view. > > Some more testing and finding a loglevel that doesn't drown me > in information, I see this: > > ----- s n i p ----- > [...] > slapd starting > conn=1000 fd=13 ACCEPT from IP=127.0.0.1:54815 (IP=0.0.0.0:389) > conn=1000 op=0 BIND dn="" method=128 > conn=1000 op=0 RESULT tag=97 err=0 text= > conn=1000 op=1 SRCH base="dc=company,dc=tld" scope=2 deref=0 > filter="(uid=kpxb140)" > conn=1000 op=1 SRCH attr=uid > [...] > conn=1015 fd=43 ACCEPT from IP=127.0.0.1:54830 (IP=0.0.0.0:389) > conn=1015 op=0 BIND dn="" method=128 > conn=1015 op=0 RESULT tag=97 err=0 text= > conn=1015 op=1 SRCH base="dc=company,dc=tld" scope=2 deref=0 > filter="(uid=kpxb140)" > conn=1015 op=1 SRCH attr=* + > daemon: shutdown requested and initiated. > conn=1015 op=1 SEARCH RESULT tag=101 err=52 nentries=0 text=Proxy > operation retry failed > slapd shutdown: waiting for 18 operations/tasks to finish > conn=1015 fd=43 closed (slapd shutdown) > daemon: accept(-1) failed errno=9 (Bad file descriptor) > [...] > conn=1000 op=1 ldap_back_retry: retrying URI="(null)" DN="" > conn=1000 fd=13 closed (slapd shutdown) > RESETTING CACHED QUERY URLS > slapd stopped. > ----- s n i p ----- > > > So on the last connection, I get 'Proxy operation retry failed' and on > all > the rest 'retrying URI="(null)" DN=""'. So I'll asume that there's a > URI > on conn=1000. > > > But why do I get these problems? Isn't slapo-translucent supposed to > work > with slapd-{ldap,meta}? > > > > ------------------------------

11 years, 8 months

2
1
0 / 0

segfault in slapd

by A Malaviya

Hi, I've a provider-consumer setup using syncrepl on which I'm experiencing segfaults on the provider. Daily there are about 4 - 5 segfaults. Weirdly it seems as if the segfaults happen when we do ldapsearch on the entire database within short intervals. The segfault line gives 'error 4' at the end. My openldap version is 2.4.23. Pls suggest some resolution for the prb. A Malaviya.

11 years, 9 months

2
1
0 / 0

ACL in dynamic configuration

by Nick Milas

Hello, I have converted from static (slapd.conf) to dynamic (cn=config) configuration using auto file conversion. I would like to ask a couple of questions regarding ACL conversion. Here follows one of the rules we have in initial form (a), and after conversion (b): (a) access to dn.subtree="dc=xxx.xxx.xxx.in-addr.arpa,ou=dns1,dc=example,dc=gr" attrs="children,entry" by group.exact="cn=TechAdmins,ou=Groups,dc=example,dc=gr" write by group.exact="cn=Dept1Admins,ou=Groups,dc=example,dc=gr" read by group.exact="cn=Dept2Admins,ou=Groups,dc=example,dc=gr" write by group.exact="cn=Dept3Admins,ou=Groups,dc=example,dc=gr" read by group.exact="cn=Dept4Admins,ou=Groups,dc=example,dc=gr" read by group.exact="cn=Dept5Admins,ou=Groups,dc=example,dc=gr" read by group.exact="cn=GuestAdmins,ou=Groups,dc=example,dc=gr" read by dn.exact="uid=dnsauthusr,ou=System,dc=example,dc=gr" read by * break (b) as an olcAccess attribute value: {10}to dn.subtree="dc=xxx.xxx.xxx.in-addr.arpa,ou=dns1,dc=example,dc=gr" attrs=children,entry by group/groupOfNames/member.exact="cn=techadmins,ou=groups,dc=example,dc=gr" write by group/groupOfNames/member.exact="cn=Dept1Admins,ou=groups,dc=example,dc=gr" read by group/groupOfNames/member.exact="cn=Dept2Admins,ou=groups,dc=example,dc=gr" write by group/groupOfNames/member.exact="cn=Dept3Admins,ou=groups,dc=example,dc=gr" read by group/groupOfNames/member.exact="cn=Dept4Admins,ou=groups,dc=example,dc=gr" read by group/groupOfNames/member.exact="cn=Dept5Admins,ou=groups,dc=example,dc=gr" read by group/groupOfNames/member.exact="cn=guestadmins,ou=groups,dc=example,dc=gr" read by dn.base="uid=dnsauthusr,ou=system,dc=example,dc=gr" read by * +0 break Question 1. Why "group.exact" was changed to "group/groupOfNames/member.exact" ? Yes, groups are defined as entries of groupOfNames objectClass, with members defined as values of attribute "member". But should it be like that? Should we change (manually) "group/groupOfNames/member.exact" back to "group.exact" again or not (and why)? Question 2. Is there a way we can add (manually, since conversion removed the ones which existed in initial configuration files) line breaks in olcAccess attribute value so it can be more legible (for administrative purposes)? Question 3. What is the "+0" added before "break" and why is needed? Thanks in advance, Nick

11 years, 9 months

3
3
0 / 0

Is this a sensible architecture?

by Brett Watson

Folks, I'm currently planning a shift in our use of LDAP to incorporate mirror mode masters for the sake of high availability. The plan is to hide a mirror mode master pair behind a virtual IP using "sorry server" fail-over, such that the primary mirror server takes 100% of the load when it's up and responsive, with fall-back to the secondary mirror server when the primary is down. In this way, the virtual IP presents a "virtual master" to the outside world, and the plan is for *all* outside LDAP interaction with the masters to happen over this single virtual IP. There will be other "slave" servers which replicate the master (to distribute read-only load), and the plan is for them to syncrepl from the master virtual IP. I gather that a possible alternative to this arrangement is to have the "slave" servers act as syncrepl consumers to *both* masters simultaneously, via their real IP addresses. If this is indeed a valid configuration, does it convey any advantages? The single "virtual master" approach seems architecturally simpler, but is it considered robust by those in the know? Thanks in advance.

11 years, 9 months

5
5
0 / 0

Fwd: OpenLdap Startup Failure

by Swapnesh sharma

Hi All, I have installed OpenLdap-2.4.26 on Solaris 10 (SunOS xxx 5.10 Generic_142900-09 sun4v sparc SUNW,SPARC-Enterprise-T5220) using the packages available from sunfreeware ( http://www.sunfreeware.com/indexsparc10.html) Installation was successful without any error. But when starting slapd, I am getting below error. Can anyone please help/redirect me ? I am trying to find a solution using the documents and archive mails. Let me know if you require more information. bash-3.00#/usr/local/libexec/slapd -d 255 slapd startup: initiated. backend_startup_one: starting "cn=config" config_back_db_open Backend ACL: access to * by * none config_back_db_open: line 0: warning: cannot assess the validity of the ACL scope within backend naming context config_build_entry: "cn=config" config_build_entry: "cn=schema" >>> dnNormalize: <cn={0}core> <<< dnNormalize: <cn={0}core> config_build_entry: "cn={0}core" config_build_entry: "olcDatabase={-1}frontend" config_build_entry: "olcDatabase={0}config" config_build_entry: "olcDatabase={1}bdb" backend_startup_one: starting "dc=my-domain,dc=com" bdb_db_open: "dc=my-domain,dc=com" *bdb_db_open: database "dc=my-domain,dc=com": dbenv_open(/usr/local/var/openldap-data).* *bdb(dc=my-domain,dc=com): Ignoring log file: /usr/local/var/openldap-data/log.0000000001: magic number 88090400, not 40988* *bdb(dc=my-domain,dc=com): Invalid log file: log.0000000001: Invalid argument* *bdb(dc=my-domain,dc=com): PANIC: Invalid argument* *bdb(dc=my-domain,dc=com): unable to join the environment* *bdb_db_open: database "dc=my-domain,dc=com" cannot be opened, err -30974. Restore from backup!* ====> bdb_cache_release_all bdb(dc=my-domain,dc=com): txn_checkpoint interface requires an environment configured for the transaction subsystem bdb_db_close: database "dc=my-domain,dc=com": txn_checkpoint failed: Invalid argument (22). backend_startup_one (type=bdb, suffix="dc=my-domain,dc=com"): bi_db_open failed! (-30974) slapd shutdown: initiated ====> bdb_cache_release_all bdb_db_close: database "dc=my-domain,dc=com": alock_close failed slapd destroy: freeing system resources. slapd stopped. Regards, Swapnesh

11 years, 9 months

2
1
0 / 0

FW: Partial LDAP replication with syncrepl

by Patrick Krauer

-----Original message----- To:+openldap+pkrauer+4a901aeb3d.openldap-technical#openldap.org@spamgourmet.com; From:Patrick Krauer <patrick(a)krauer.org> Sent:Thu 15-03-2012 08:05 Subject:Partial LDAP replication with syncrepl Hi all, I'm using OpenLDAP for different purposes (e.g. zarafa, ssh authentication and HTTP authentication). Each Service (zarafa, apache) is running on a different server. SSH for sure runs on every server. For stability reasons, I replicate my LDAP database to each slave server (zarafa, apache and others). I'm using syncrepl which works fine by know. Now I want to use LDAP for another service (typo3). For this I need to apply another schema. If I do this on the master LDAP server only, all slave servers will claim. If I set schemachecking off in the slave configuration, the LDAP server does not stop, but it does not load entities which use the new objectClass. I tried to select only particular attributes on the slave server (like cn, uid but not objectClass) but that doesn't help. I don't want to apply the new schema to each and every server which doesn't use it at all. How can I solve this issue? What is the common way for such a setup? Best regards, Patrick -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

11 years, 9 months

1
0
0 / 0

RE: OPENLDAP SYNCREPL

by Quanah Gibson-Mount

--On Thursday, March 15, 2012 1:08 PM -0400 "Borresen, John - 0442 - MITLL" <john.borresen(a)ll.mit.edu> wrote: > Quanah; > > A question about the "entryCSN" attributes, the three zeros (entryCSN:= > 20120308173545.066786Z#000000#000#000000) between the #-marks correspond > to the Server ID, correct? If they do, why are some "000" and others > "001" when I only have one server? Some entries were created before it considered itself a provider. I wouldn't worry about 000 vs 001 at this time. ;) > Thank you for your assistance on getting me over the syncrepl wall. > > I noticed in the 2.4 Admin Guide that configuring the monitor database for > cn=config has not been published yet; and, there isn't a lot online, > period, about it. The manpages are not overly helpful either from > cn=config is concerned. > > I am seeing this: "bdb_monitor_db_open: monitoring disabled; configure > monitor database to enable" > > on both the Provider and Consumer. > > Can you recommend something, relevant to cn=config monitor setup? Add the monitor DB to your cn=config, and add any acl you want applied to it. That's all there is to it. For example, mine is: dn: olcDatabase={1}monitor,cn=config objectClass: olcDatabaseConfig olcDatabase: {1}monitor olcAccess: {0}to dn.children="cn=monitor" by dn.children="cn=admins,cn=zimbra " read olcLastMod: TRUE olcMaxDerefDepth: 15 olcReadOnly: FALSE olcRootDN: cn=config olcMonitoring: FALSE --Quanah -- Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration

11 years, 9 months

1
0
0 / 0

WINDOWS AD CLIENTS & LINUX LDAP CLIENTS UNABLE TO OPEN X-WINDOWS EMULATORS

by Borresen, John - 0442 - MITLL

All, Sorry for the long subject line. 1) We are migrating away from NIS to LDAP for our Linux/Unix clients and servers. 2) All our Windows clients authenticate with the main Active Directory server. Now the issue: Many of our end-users utilize various X-Windows Emulators (Exceed, XWin32, Xming, etc) to open a connection to a Linux server. When we used NIS, the End-users had no problem opening a X-Win session to a server. Now under LDAP, the users get authenticated properly but the X-Win session gets blocked and never opens. In the messages file, and secure file, you can see the users getting authenticated and granted access but the X-win session never opens on the client machine. Has anyone else seen this issue? Thanks, Dave Borresen Solaris/Linux Systems Administrator Surveillance Systems Group MIT Lincoln Laboratory 244 Wood Street Lexington, MA 02420 john.borresen(a)ll.mit.edu

11 years, 9 months

1
0
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical March 2012