openldap-technical

openldap-technical@openldap.org

9897 discussions

Re: ldapsearch hangs
by Tihomir Culjaga 10 Sep '09

10 Sep '09

nice link... thnaks! Will let you know the results. T. On Thu, Sep 10, 2009 at 10:10 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote: > --On Thursday, September 10, 2009 10:07 PM +0200 Tihomir Culjaga < > tculjaga(a)gmail.com> wrote: > > wow, Quanah, >> >> that's the best answer i ever got... >> >> I'm gong to move to 2.4.18 and try to tune up the machine... regarding >> more RAM ... i can get it but not tha fast. >> >> what can i expect with a total of 10 GB ?... >> >> I plan to set set_cachesize 7 0 1 >> >> what is the relation between cachesize (in slapd.conf) and set_cachesize >> (in DB_CONFIG)? >> > > They aren't related. cachesize (in slapd.conf) is for the entry cache. > set_cachesize is for the BDB backing cache. > > set_cachesize =>how much memory i wiill cache >> cachesize => number of entries i will keep within the cache >> >> how much one entry take ? >> >> Lets say if i use 7 GB of cache, what cachesize should i put? >> > > Hard to say. ;) A bit of the BDB cache will likely be swapped out. > > You may also want to read: > > <http://wiki.zimbra.com/index.php?title=OpenLDAP_Performance_Tuning_6.0> > > It has some zimbra specific commands in it, but it should be fairly trivial > to see how it maps to your openldap install. I'd also particularly read > over the bit about using a shared memory key for the BDB cache once you go > beyond 8GB cachesize (which you will do after you get more RAM. ;) ). > > > --Quanah > > -- > > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration >

1 0

Re: ldapsearch hangs
by Tihomir Culjaga 10 Sep '09

10 Sep '09

wow, Quanah, thats the best answer i ever got... I'm gong to move to 2.4.18 and try to tune up the machine... regarding more RAM ... i can get it but not tha fast. what can i expect with a total of 10 GB ?... I plan to set set_cachesize 7 0 1 what is the relation between cachesize (in slapd.conf) and set_cachesize (in DB_CONFIG)? set_cachesize =>how much memory i wiill cache cachesize => number of entries i will keep within the cache how much one entry take ? Lets say if i use 7 GB of cache, what cachesize should i put? T. On Thu, Sep 10, 2009 at 9:37 PM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote: > --On Thursday, September 10, 2009 8:56 PM +0200 Tihomir Culjaga < > tculjaga(a)gmail.com> wrote: > > So, the situation is that i have 2 ldif files i'm recreating the database >> from. >> >> /usr/local/libexec/slapadd -l /home/tculjaga/file2.ldif -f >> /usr/local/etc/openldap/slapd.conf >> /usr/local/libexec/slapadd -l /home/tculjaga/file2.ldif -f >> /usr/local/etc/openldap/slapd.conf >> > > I would suggest you just make these a single file, so all the work can be > done at one time. > > I tried to re-index with /usr/local/libexec/slapindex -f >> /usr/local/etc/openldap/slapd.conf -v >> restart slapd process, restart the machine ... it is always the same >> issue. >> > > Nothing here indicates a problem with your indices. Running slapindex > repeatedly is a waste of your time. > > [root@l01lnp2 traces]# /usr/local/libexec/slapd -V >> @(#) $OpenLDAP: slapd 2.4.16 (Sep 9 2009 14:39:44) $ >> root@l01lnp2:/home/tculjaga/openldap-2.4.16/servers/slapd >> > > I would strongly urge you to upgrade to 2.4.18 (for reasons I will note > further down) > > > [root@l01lnp2 traces]# /usr/local/BerkeleyDB.4.7/bin/db_stat -V >> Berkeley DB 4.7.25: (May 15, 2008) - unpached! >> > > You need to rebuild BDB 4.7.25 with the 4 patches from Oracle. There are > known issues when running BDB 4.7 without them. > > [root@l01lnp2 traces]# du -c -h /var/lib/ldap/*.bdb >> 200K /var/lib/ldap/bestMatchPrefix.bdb >> 3.8G /var/lib/ldap/dn2id.bdb >> 6.2G /var/lib/ldap/id2entry.bdb >> 1.8M /var/lib/ldap/objectClass.bdb >> 1.2M /var/lib/ldap/originatorPrefixID.bdb >> 48M /var/lib/ldap/uniqueID.bdb >> 10G total >> > > Since your database is a total of 10 GB in size, for slapadd to work at > optimum efficiency, you need at least 10GB of cache for your DB_CONFIG file. > Unfortunately, you only have 10GB of RAM. Essentially, your system is > under powered for your database size. > > > > [tculjaga@l01lnp2 ~]$ cat ot.ldif | grep -c "dn: " >> 101588 >> [tculjaga@l01lnp2 ~]$ cat l01sipdir1.ldif | grep -c "dn: " >> 9994864 >> [tculjaga@l01lnp2 ~]$ >> > > So you have 10,096,452 entries total. > > [root@l01lnp2 traces]# cat /var/lib/ldap/DB_CONFIG | grep -v "#" >> >> set_cachesize 0 3221225472 1 >> set_lg_regionmax 262144 >> set_lg_bsize 2097152 >> > > You only have a 3GB DB cachesize configured here. Expect things to perform > sub optimally. It would have been easier to set this by going > > set_cachesize 3 0 1 > > Which would have the same effect, since the first number is the number of > gigabytes to allocate. > > Please find attached slapd.conf >> > > Ok, so the relevant bits from here are: > > cachesize 2500000 > idlcachesize 7500000 > cachefree 1000 > > Which means you have a cachesize of 2.5 million, an idlcachesize of 7.5 > million, and (with OL 2.4.16) a dncachesize of 5 million. > > I would highly advise you upgrade to OpenLDAP 2.4.18, and change the > slapd.conf settings to: > > dncachesize 0 (which means unlimited). > > And setting no cache or idlcachesize, and fixing your DB_CONFIG. But you > also need to buy a substantial amount of RAM for a DB of this size. :P I > would advise you upgrade to at least 32GB total. Then you can more > optimally tune the system. > > > --Quanah > > -- > > Quanah Gibson-Mount > Principal Software Engineer > Zimbra, Inc > -------------------- > Zimbra :: the leader in open source messaging and collaboration >

2 1

Re: ldapsearch hangs
by Quanah Gibson-Mount 10 Sep '09

10 Sep '09

--On Thursday, September 10, 2009 8:56 PM +0200 Tihomir Culjaga <tculjaga(a)gmail.com> wrote: > So, the situation is that i have 2 ldif files i'm recreating the database > from. > > /usr/local/libexec/slapadd -l /home/tculjaga/file2.ldif -f > /usr/local/etc/openldap/slapd.conf > /usr/local/libexec/slapadd -l /home/tculjaga/file2.ldif -f > /usr/local/etc/openldap/slapd.conf I would suggest you just make these a single file, so all the work can be done at one time. > I tried to re-index with /usr/local/libexec/slapindex -f > /usr/local/etc/openldap/slapd.conf -v > restart slapd process, restart the machine ... it is always the same > issue. Nothing here indicates a problem with your indices. Running slapindex repeatedly is a waste of your time. > [root@l01lnp2 traces]# /usr/local/libexec/slapd -V > @(#) $OpenLDAP: slapd 2.4.16 (Sep 9 2009 14:39:44) $ > root@l01lnp2:/home/tculjaga/openldap-2.4.16/servers/slapd I would strongly urge you to upgrade to 2.4.18 (for reasons I will note further down) > [root@l01lnp2 traces]# /usr/local/BerkeleyDB.4.7/bin/db_stat -V > Berkeley DB 4.7.25: (May 15, 2008) - unpached! You need to rebuild BDB 4.7.25 with the 4 patches from Oracle. There are known issues when running BDB 4.7 without them. > [root@l01lnp2 traces]# du -c -h /var/lib/ldap/*.bdb > 200K /var/lib/ldap/bestMatchPrefix.bdb > 3.8G /var/lib/ldap/dn2id.bdb > 6.2G /var/lib/ldap/id2entry.bdb > 1.8M /var/lib/ldap/objectClass.bdb > 1.2M /var/lib/ldap/originatorPrefixID.bdb > 48M /var/lib/ldap/uniqueID.bdb > 10G total Since your database is a total of 10 GB in size, for slapadd to work at optimum efficiency, you need at least 10GB of cache for your DB_CONFIG file. Unfortunately, you only have 10GB of RAM. Essentially, your system is under powered for your database size. > [tculjaga@l01lnp2 ~]$ cat ot.ldif | grep -c "dn: " > 101588 > [tculjaga@l01lnp2 ~]$ cat l01sipdir1.ldif | grep -c "dn: " > 9994864 > [tculjaga@l01lnp2 ~]$ So you have 10,096,452 entries total. > [root@l01lnp2 traces]# cat /var/lib/ldap/DB_CONFIG | grep -v "#" > > set_cachesize 0 3221225472 1 > set_lg_regionmax 262144 > set_lg_bsize 2097152 You only have a 3GB DB cachesize configured here. Expect things to perform sub optimally. It would have been easier to set this by going set_cachesize 3 0 1 Which would have the same effect, since the first number is the number of gigabytes to allocate. > Please find attached slapd.conf Ok, so the relevant bits from here are: cachesize 2500000 idlcachesize 7500000 cachefree 1000 Which means you have a cachesize of 2.5 million, an idlcachesize of 7.5 million, and (with OL 2.4.16) a dncachesize of 5 million. I would highly advise you upgrade to OpenLDAP 2.4.18, and change the slapd.conf settings to: dncachesize 0 (which means unlimited). And setting no cache or idlcachesize, and fixing your DB_CONFIG. But you also need to buy a substantial amount of RAM for a DB of this size. :P I would advise you upgrade to at least 32GB total. Then you can more optimally tune the system. --Quanah -- Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

1 0

Reg Adding New Attributes
by Asimananda Mohanty 10 Sep '09

10 Sep '09

Hi All, I have installed OpenLdap and PHPLdapAdmin. I have added a user which objectclass is : objectClass inetOrgPerson posixAccount shadowAccount I need to add a new attribute which will be application specific (application which will need LDAP authentication and based on the attribute value, it will provide service to the user). When I tried to do the same by adding attribute and value to ldif files and then adding the ldif file, it showed error: ldap_add: Undefined attribute type (17) additional info: AppAttribute1: attribute type undefined. Do I need to modify related schema for the same? Please suggest some suitable way to do the same. Is there any way that I can do it from the PhpLDAPAdmin GUI? Thanks in advance. -Asimananda

2 2

ldapsearch hangs
by Tihomir Culjaga 10 Sep '09

10 Sep '09

hello, I got an issue with OpenLDAP.... BDB 4.7 and OpenLDAP 2.4.18 is runnong on CentOS 5.3 installed on an HP DL380 G5 machine with 10 GB RAM. The database is quite big...ldif file is ~2GB. I manage to load the ldif file into the DB with ldapadd. Well, so far so good, i'm able to search entries. My application on a different server connects to LDAP and requests entire subtree (ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr) and within a minute and half, it gets everything. Now, the problem comes when you add some few more entries to the DB with ldapadd... in my case 2 entries dn: carrierPrefixID=043010.100.10000.100,bestMatchPrefix=385,ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr qos: 100 priority: 10000 carrierPrefixID: 043010.100.10000.100 carrierPrefix: 043010 weight: 100 carrier: Telekom Austria objectClass: top objectClass: carrierPrefixID dn: originatorPrefixID=000010,carrierPrefixID=043010.100.10000.100,bestMatchPrefix=385,ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr originatorPrefix: 000010 priority: 100 originator: T-COM/HT originatorPrefixID: 000010 objectClass: top objectClass: originatorPrefixID The application doesn't get all entries ... as LDAP server stops respnding. In brief, i run ldapsearch manually and i get it stuk after a while: [root@l01lnp2 ldap]# [root@l01lnp2 ldap]# time ldapsearch -h localhost -x -b ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr -D cn=admin,dc=ot,dc=hr -w password # extended LDIF # # LDAPv3 # base <ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr> with scope subtree # filter: (objectclass=*) # requesting: ALL # # bestMatchPrefixList, sipDirektor, ot.hr dn: ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr ou: bestMatchPrefixList objectClass: top objectClass: organizationalUnit # 7, bestMatchPrefixList, sipDirektor, ot.hr dn: bestMatchPrefix=7,ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr destination: Russian Federation bestMatchPrefix: 7 objectClass: top objectClass: bestMatchPrefix <-------------------------snip-------------------------> # 043010, 046010.100.8000.100, 99893, bestMatchPrefixList, sipDirektor, ot.hr dn: originatorPrefixID=043010,carrierPrefixID=046010.100.8000.100,bestMatchPre fix=99893,ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr originatorPrefix: 043010 priority: 100 originator: 043010 originatorPrefixID: 043010 objectClass: top objectClass: originatorPrefixID # 039010, 046010.100.8000.100, 99893, bestMatchPrefixList, sipDirektor, ot.hr dn: originatorPrefixID=039010,carrierPrefixID=046010.100.8000.100,bestMatchPre fix=99893,ou=bestMatchPrefixList,ou=sipDirektor,dc=ot,dc=hr originatorPrefix: 039010 priority: 100 originator: 039010 originatorPrefixID: 039010 objectClass: top objectClass: originatorPrefixID It hangs here ... and i have to stop ldapsearch (CTRL + C) real 12m2.592s user 0m2.448s sys 0m2.150s [root@l01lnp2 ldap]# I have the same issue on the BDB4.3 and OpenLDAP 2.3 that comes with CentOD 5.3 distribution as well. What m'I missing? What can i do ? I already tried to: reindex (slapindex -f slapd.conf) db_recover everytihng

2 3

crafting acls
by fmgre-liste01＠yahoo.fr 10 Sep '09

10 Sep '09

Hello, I was wondering if there is a way to let specific users access (write ) given fields : • all my users are in a single branch • all my users are inetOrgPerson + posixAccount • users are distributed in several groups according to their gidNumbers • some users are priviledged and also belong to a given posixGroup I would like to know if it is possible to write an acl so that : a priviledge user can modify some fields ( shell, homeDirectory ) of users whose gidNumber matches the gidNumber of the priviledged user Thanks Fred

2 1

ldapsearch hang on OpenLDAP 2.3
by William Jojo 09 Sep '09

09 Sep '09

Now I know that 2.4 is the recommended level, however, this system is not ready to be converted (yet). It's running 2.3.39 on AIX 5.3 (BDB 4.4.20 with all patches) and in the master in a master/slave (slurpd) environment. I could upgrade to 2.3.43 (if you think it will help), but we are planning moving it to 2.4 soon. We had an issue with an ldapsearch that would not finish - it connected, but failed to return information. When I "truss"-ed the pid, it was switching between threads but acted like The slapd daemon responded to kill -HUP and stopped successfully. db_recover was: [svc1:/ldap/db/database] # /opt/pware/bin/db_recover -v -h . Finding last valid log LSN: file: 545 offset 8672731 Recovery starting from [545][8672586] Recovery complete at Wed Sep 9 15:22:45 2009 Maximum transaction ID 8005cffa Recovery checkpoint [545][8672731] Then we ran a script that stops the DB and does an export (which looks good) and tars up the db files and log file. Shortly thereafter searching for the same entry cause a hang, but slapd stopped successfully. Then ran db_recover again: [svc1:/ldap/db/database] # db_recover -v -h . Finding last valid log LSN: file: 545 offset 8900926 Recovery starting from [545][8900781] Recovery complete at Wed Sep 9 15:29:37 2009 Maximum transaction ID 800000ad Recovery checkpoint [545][8900926] Unfortunately we had the log level set to 0, so we never saw the details in syslog. Is 256 the correct value (which logs quite a bit of stuff) for a busy server? Is this a corrupt environment? Corrupt index? Should we remove the environment and reindex the database? We are trying to avoid reloading from the export, if possible. Again, we are just trying to breathe some life into this server for the short term as moving to 2.4 right this minute has other implications we just can't change at the moment. Appreciate any ideas on this. Cheers, Bill

1 0

Sudden drop in cache efficiency
by Bjørn Ruberg 09 Sep '09

09 Sep '09

Hi list, I'm currently monitoring the cache efficiency in a BerkeleyDB backend, as examined with db_stat -m. About 24 hours after starting slapd I noticed a significant drop in several indexes, while the general activity did not change much at that time. Also, the systems using the service did not experience any reduction in availability or response. The cache efficiency in percentage remains at 99%. Its MirrorMode peer experienced a similar drop at the exact same time, though for fewer indexes. The affected nodes are in a refreshAndPersist, retry="60 +" MirrorMode setup. Backend database is configured as hdb. The binaries in use are the latest RPMs from Buchan Milne (openldap2.4-servers-2.4.17-3.rhel5, which comes packed with its own BerkeleyDB 4.7 libraries), running on a RHEL5.3 system. The graphs are available at http://www.ruberg.no/tmp/slapd.html. The drops occured just before 15:00. The nodes are in the same IP network without any routers or firewalls in-between. Replication between the nodes works flawlessly. Is this kind of drop normal behaviour? Has slapd stopped asking its backend (or peer) for data and started serving everything from its own buffer? Since the observations correlate between the active and the passive node I currently suspect this involves the replication mechanism(s). (The reason some indexes are not currently graphed is that the numbers read from db_stat are suffixed with SI units when above 10 million, and the monitoring tool doesn't account for that yet.) Thanks for any pointers and ideas, -- Bjørn

1 0

OpenLdap/Active Directory/Trusts
by tom mahy 08 Sep '09

08 Sep '09

Hi, Our current setup is 2 active directories, with one openLdap. Our applications syncs with openLdap, openLdap acts as a proxy to the active directories, meaning we can see both AD's in OpenLdap. This is great, but when we setup trusts in Active Directory, openLdap detects them as simple folders, and does not apply the trust. Therefore when we sync we only get the top folder, the trust inside it is ignored. meaning we are missing a level. Is there anyway of making openLdap detect these trusts and apply them ? Sincerely, Tom Mahy

2 1

How to get certain log messages to filter through to a user connecting to OpenLDAP via a JSP page?
by Akke Bengtsson 08 Sep '09

08 Sep '09

I realise this borders on a Java development question but I presume there is an ldap-specific side of it as well so - here we go. We use an OpenLDAP service to authenticate and authorize users of server based web applications written as JSP/EJB pages. We have a reasonably working solution to let assorted exception messages filter back to the user. Much more difficult getting the warning messages back telling for instance that the password is about to expire or that it already has expired and so and so many grace logins remain, due to ppolicy. I can see them in the ldap server application log but how to access them programmatically boggles my mind. Best Regards Akke Bengtsson Systems Developer

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical