openldap-software December 2006

openldap-software@openldap.org

60 participants
65 discussions

syncrepl - when master restarted, slaves loose connection forever (and never reconnects)?
by Tomasz Chmielewski 15 Dec '06

15 Dec '06

I decided to use syncrepl (refreshAndPersist) instead of slurpd to synchronize. I made simple tests but something doesn't work right for me. I started a totally empty OpenLDAP slave - it got replicated from the master. Any change made on the master is also replicated fine. Unfortunately, when the connection between the slave and the master is broken (i.e., master restarted), the slave never reconnects. According to the fine slapd.conf manual, reconnection should be made quite fast: "If the connection is lost, the consumer will attempt to reconnect at an interval time (specified by interval parameter; 60 seconds by default) until the session is re-established." I waited for about 12 hours, and it didn't happen. Restarting slave helped, and all pending changes were transferred from the master. Am I missing a setting or something? The slave is running slapd from OpenLDAP 2.3.37, and here is its slapd.conf part concerning syncrepl: syncrepl rid=0 provider=ldap://master:389 # interval=00:00:00:30 <- also didn't help if it's 30 secs type=refreshAndPersist searchbase="dc=some,dc=domain" bindmethod=simple binddn="cn=replicationuser,dc=some,dc=domain" credentials=secret schemachecking=off -- Tomasz Chmielewski http://wpkg.org

5 7

listening on localhost only
by Alex Girchenko 15 Dec '06

15 Dec '06

Hello. I want to make slapd listen on localhost only. Is there such a configuration option? Thanks.

3 2

slapd grows during SASL Binds
by Paul Turgyan 14 Dec '06

14 Dec '06

We've seen the slapd's on our mail slaves trying to grow greater than the 2 gigabyte resident set size. I spent two days w/ valgrind looking for memory leaks, but I failed to find anything. I finally discovered that repeatedly binding and unbinding using a SASL/ GSSAPI bind would cause slapd's resident set size and vm size to grow. So I wrote a test program to SASL Bind, issue a search, and unbind. The test program would do this sequence 10000 times before it stopped. The test program always issues the same exact search trying to eliminate any interferance from a growing entry cache and/or idlcache. I also tried this test w/ OpenLDAP 2.3.30 and got similar results. Initial program size for slapd RSS = 8392 VMSIZE = 180180 "Bind, search, unbind" memory usage. Searches Anonymous Bind SASL/GSSAPI Bind Completed RSS VMSIZE RSS VMSIZE 10000 9320 182760 12560 184916 20000 9320 182760 15404 187988 30000 9376 187888 18268 196188 40000 9376 187888 21080 199260 50000 9380 187888 23896 201308 60000 9380 187888 26708 204380 70000 9380 187888 29524 207452 80000 9380 187888 32336 210524 90000 9380 187888 35148 212572 100000 9380 187888 37972 215644 And when I only bind once for every 10,00 searches, I get a slapd memory usage like: Searches Anonymous Bind SASL/GSSAPI Bind Completed RSS VMSIZE RSS VMSIZE 10000 9308 181736 9788 181856 20000 9308 181736 9788 181856 30000 9308 181736 9788 181856 40000 9316 181736 9788 181856 50000 9316 181736 9788 181856 60000 9316 181736 9796 181856 70000 9316 181736 9796 181856 80000 9316 181736 9800 181856 90000 9316 181736 9800 181856 100000 9316 181736 9800 181856 Current Software Versions Linux kernal 2.6.17.6 OpenLDAP 2.3.27 also tried 2.3.30 w/ similar results cyrus SASL 2.1.21 Heimdal 0.7.2 Berkeley 4.2 slapd.conf loglevel 768 include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/local.schema include /etc/openldap/slapd.acl.prod # default settings sizelimit 350 timelimit 60 idletimeout 300 defaultsearchbase "dc=umich,dc=edu" pidfile /etc/openldap/slapd.slave.pid argsfile /etc/openldap/slapd.slave.args # Allow these features allow bind_v2 bind_anon_cred bind_anon_dn threads 32 include /etc/openldap/slapd.tls # SASL / GSSAPI / Kerberos sasl-realm UMICH.EDU saslRegexp uid=simta/mx.umich.edu,cn=umich.edu,cn=gssapi,cn=auth cn=simta,ou=mail agents,ou=security,dc=umich,dc=edu saslRegexp uid=(.*),cn=umich.edu,cn=gssapi,cn=auth uid=$1,ou=people,dc=umich,dc=edu saslRegexp dc=edu,dc=umich,ou=security,ou=(.*),cn=(.*) cn=$2,ou=$1,ou=security,dc=umich,dc=edu # Proxy authorization checks saslAuthzFrom in destination entry sasl-authz-policy from include /etc/openldap/slapd.sasl.hostname TLSCertificateFile /etc/openssl/private/ldap-dev_itd_umich_edu.crt TLSCertificateKeyFile /etc/openssl/private/ldap-dev.itd.umich.edu.key # database settings database bdb suffix "dc=umich,dc=edu" rootdn xxxxxxxx rootpw xxxxxxxx limits dn.exact="cn=batch update,ou=security,dc=umich,dc=edu" size.soft=unlimited size.hard=soft time.soft=unlimited time.hard=soft limits users size.soft=350 size.hard=soft size.unchecked=1000 time.soft=60 time.hard=soft limits anonymous size.soft=50 size.hard=soft size.unchecked=1000 time.soft=5 time.hard=soft directory /var/slapd/tmp/db/db.slave cachesize 100000 checkpoint 2048 5 idlcachesize 20000 include /etc/openldap/slapd.index # Monitor Backend database monitor BerkeleyDB DB_CONFIG file set_lk_max_locks 25000 set_lk_max_objects 25000 set_cachesize 0 128000000 1 # Set transaction log buffer size to 2 megs set_lg_bsize 2097152 Any ideas? Thanks, Paul Turgyan University of Michigan

3 4

howto create base dn??? postgres(8.1.4)+slapd(2.2.23)
by Axel Christiansen 14 Dec '06

14 Dec '06

Hello List, i can not get the base dn into the postgres db. The phpLDAPadmin tool complains about the missing base dn. When i try to set one, the slapd segfalts. The w32 LDAP Admin V0.9.9.2 does not complain about the base dn. When i try to add an item, the slapd allso segfalts. Without the sql backend the LDAP where wokring fine. The slapd seems to talk to postgres just fine. I read a hole lot of howtos. Waht i really missed was clear instructions in howto init the sql database and brand it with the base dn. What can i do next? Thx a lot, Axel The LDAP host: Linux fn 2.6.7-1-686-smp #1 SMP Wed Jul 28 13:02:18 CEST 2004 i686 ii libiodbc2 3.52.2-3 iODBC Driver Manager ii odbc-postgresq 07.03.0200-5 ODBC driver for PostgreSQL ii odbcinst1 2.2.4-11 Support library and helper ii odbc-postgresq 07.03.0200-5 ODBC driver for PostgreSQL ii postgresql-8.1 8.1.4-6~bpo.1 object-relational SQL database ii postgresql-cli 8.1.4-6~bpo.1 front-end programs for PostgreSQL 8.1 ii postgresql-cli 59~bpo.1 manager for multiple PostgreSQL ii postgresql-com 59~bpo.1 manager for PostgreSQL database ii postgresql-doc 8.1.4-6~bpo.1 documentation for the PostgreSL da ii postgresql-ser 8.1.4-6~bpo.1 development files for PostgreSQL ii ldap-utils 2.2.23-8 OpenLDAP utilities ii libldap-2.2-7 2.2.23-8 OpenLDAP libraries ii libldap2 2.1.30-8 OpenLDAP libraries Down here you find: 1. slapd debug to stdout/err 2. postgres statements ###################################################################### 1. fn:/etc/ldap# slapd -d 160000 -f slapd.conf @(#) $OpenLDAP: slapd 2.2.23 (May 30 2005 08:52:42) $ @pulsar:/home/torsten/packages/openldap/openldap2.2-2.2.23/debian/build/servers/slapd slapd starting conn=0 fd=10 ACCEPT from IP=217.86.159.169:4084 (IP=0.0.0.0:389) conn=0 op=0 BIND dn="cn=admin,dc=c111,dc=org" method=128 conn=0 op=0 BIND dn="cn=admin,dc=c111,dc=org" mech=SIMPLE ssf=0 conn=0 op=0 RESULT tag=97 err=0 text= conn=0 op=1 SRCH base="dc=c111,dc=org" scope=1 deref=0 filter="(objectClass=*)" conn=0 op=1 SRCH attr=objectclass conn=0 op=1 SEARCH RESULT tag=101 err=12 nentries=0 text=control unavailable in context conn=0 op=2 SRCH base="dc=c111,dc=org" scope=1 deref=0 filter="(objectClass=*)" conn=0 op=2 SRCH attr=objectclass conn=0 op=2 SEARCH RESULT tag=101 err=0 nentries=0 text= conn=0 op=3 SRCH base="dc=c111,dc=org" scope=0 deref=0 filter="(objectClass=*)" conn=0 op=3 SEARCH RESULT tag=101 err=0 nentries=0 text= conn=0 op=4 ADD dn="ou=test,dc=c111,dc=org" Segmentation fault ########################################################################## 2. conn = 135496696, PGAPI_Connect(DSN='ldap', UID='ldap', PWD='xxxxx') Global Options: Version='07.03.0200', fetch=100, socket=4096, unknown_sizes=0, max_varchar_size=254, max_longvarchar_size=8190 disable_optimizer=1, ksqo=1, unique_index=1, use_declarefetch=0 text_as_longvarchar=1, unknowns_as_longvarchar=0, bools_as_char=1 NAMEDATALEN=64 extra_systable_prefixes='dd_;', conn_settings='' conn_encoding='OTHER' conn=135496696, query=' ' conn=135496696, query='select version()' [ fetched 1 rows ] [ PostgreSQL version string = 'PostgreSQL 8.1.4 on i386-pc-linux-gnu, compiled by GCC cc (GCC) 3.3.5 (Debian 1:3.3.5-13)' ] [ PostgreSQL version number = '8.1' ] conn=135496696, query='set DateStyle to 'ISO'' conn=135496696, query='set geqo to 'OFF'' conn=135496696, query='set extra_float_digits to 2' conn=135496696, query='select oid from pg_type where typname='lo'' [ fetched 0 rows ] conn=135496696, query='select pg_client_encoding()' [ fetched 1 rows ] [ Client encoding = 'SQL_ASCII' (code = 0) ] conn=135496696, query='SELECT id,name,keytbl,keycol,create_proc,delete_proc,expect_return FROM ldap_oc_mappings' [ fetched 0 rows ] conn=135496696, query='ROLLBACK' conn=135496696, PGAPI_Disconnect conn = 135495944, PGAPI_Connect(DSN='ldap', UID='ldap', PWD='xxxxx') Global Options: Version='07.03.0200', fetch=100, socket=4096, unknown_sizes=0, max_varchar_size=254, max_longvarchar_size=8190 disable_optimizer=1, ksqo=1, unique_index=1, use_declarefetch=0 text_as_longvarchar=1, unknowns_as_longvarchar=0, bools_as_char=1 NAMEDATALEN=64 extra_systable_prefixes='dd_;', conn_settings='' conn_encoding='OTHER' conn=135495944, query=' ' conn=135495944, query='select version()' [ fetched 1 rows ] [ PostgreSQL version string = 'PostgreSQL 8.1.4 on i386-pc-linux-gnu, compiled by GCC cc (GCC) 3.3.5 (Debian 1:3.3.5-13)' ] [ PostgreSQL version number = '8.1' ] conn=135495944, query='set DateStyle to 'ISO'' conn=135495944, query='set geqo to 'OFF'' conn=135495944, query='set extra_float_digits to 2' conn=135495944, query='select oid from pg_type where typname='lo'' [ fetched 0 rows ] conn=135495944, query='select pg_client_encoding()' [ fetched 1 rows ] [ Client encoding = 'SQL_ASCII' (code = 0) ]

3 5

RE: Disaster recovery question wrt replication
by Steven Harms (stharms) 14 Dec '06

14 Dec '06

Judging by the number of other comments about the miracles of 2.30, maybe I'll try to talk to my admins and see if I can get that. Steven -----Original Message----- From: openldap-software-bounces+stharms=cisco.com(a)openldap.org [mailto:openldap-software-bounces+stharms=cisco.com@openldap.org] On Behalf Of Tony Earnshaw Sent: Thursday, December 14, 2006 2:04 AM Cc: openldap-software(a)openldap.org Subject: Re: Disaster recovery question wrt replication Steven Harms (stharms) wrote: [...] > I suppose, more generally, I'm asking: How do a start replication all > over. What files can / should I delete. Which should I not under any > circumstance touch? Others (Quanah) have mentioned dropping implementation of slurpd in favor of i(delta-)syncrepl - requires upgrading to 2.3(.30 at the moment). This site used to use slurpd with OpenLDAP 2.2. Before we switched to 2.3, we tested syncrepl slave DB rebuilds from scratch (empty DB, valid slapd.conf and DB_CONFIG in the DB directory). We simply started slapd on the slave machines and the small (about 40MB) DB was automatically rebuilt within seconds (100Mb Ethernet and fast SCSI RAID5), high-quality iron. Having previously had to deal with the same thing with slurpd earlier, I was completely gob-smacked. --Tonni -- Tonni Earnshaw tonni @ barlaeus.nl

1 0

OpenLDAP Proxy Cache question
by Gessy 14 Dec '06

14 Dec '06

Hi , On OpenLDAP Proxy Cache, the pointer to backend directory server (uri) can be multi-value? Like: suffix "dc=example,dc=com" uri ldap://host1/dc=example,dc=com uri ldap://host2/dc=example,dc=com #setting cache parameters cacheparams 10000 15000 4 50 1000 Or suffix "dc=example,dc=com" uri ldap://host1/dc=example,dc=com ldap://host2/dc=example,dc=com #setting cache parameters cacheparams 10000 15000 4 50 1000 thanks a lot. Gessy Jr. -- (o< Avoid the Gates of hell! //\ Use GNU Linux. V_/_

1 0

Re: slapd responds very slowly when cpu has 100% usage (but actually low load)
by Antonis Christofides 14 Dec '06

14 Dec '06

(My original message, which presents the problem, is at the bottom.) Thank you for your responses, here is some more information: > You didn't mention what version of slapd you're running. That's right, sorry. I'm running Debian-packaged slapd 2.2.23-8, using bdb 4.2.52-18 (everything on my system is the Debian sarge's packages, except for the kernel, which is a recompiled Ubuntu 6.06 kernel, 2.6.12 SMP). > I would expect the call to wait after forking to exec true to be treated > as blocking IO, but, perhaps on your system, true is an sh builtin. That's right, true is a shell builtin. So the two processes don't fork or anything, they just run and run. Some tests I made right now show that a certain ldapsearch completes in 4 seconds if the processes have nice 19, 35-40 seconds if they have nice 10, and longer if nice 0. And, yes, if I replace "while true" with "while /bin/true", slapd responds instantly. > nicing a process does not affect its time slice, just where it sits > in the run queue when it is ready to run. Your description of how the scheduler works might explain the cause of the problem if slapd makes a huge number of blocking I/O requests: each time it makes such a request, it goes to Sleep, and the next process on the run queue (the niced shell in our case) is set to run, and exhausts its time slice. Then, supposing slapd is ready again, it is set to run, it makes a request, it sleeps again, etc. If it needs to make hundreds of I/O requests, could it explain the delay? Here is what ps shows while I'm waiting for slapd to respond: anthony@acheloos:~$ ps u -m 16204 USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 16204 0.0 0.5 30140 5596 ? - 11:45 0:00 /usr/sbin/slapd root - 0.0 - - - - Ss 11:45 0:00 - root - 0.0 - - - - Ss 11:45 0:00 - root - 0.0 - - - - Rs 11:46 0:00 - One of the threads is "Rs"; after slapd delivers its response, it goes back to "Ss". > Maybe you have the memory to let everything rest in memory. I don't know what > your two looping shells do to your memory... If you had some control to never > swap out ldap, this theory could be tested. I think I have lots of spare memory: top - 12:53:32 up 60 days, 1:35, 5 users, load average: 1.38, 0.77, 0.63 Tasks: 262 total, 3 running, 258 sleeping, 1 stopped, 0 zombie Cpu0 : 0.7% us, 0.7% sy, 98.7% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si Cpu1 : 0.0% us, 0.0% sy, 100.0% ni, 0.0% id, 0.0% wa, 0.0% hi, 0.0% si Mem: 1035680k total, 1016936k used, 18744k free, 39004k buffers Swap: 2097144k total, 99888k used, 1997256k free, 577360k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8551 anthony 35 10 4032 1204 916 R 99.9 0.1 0:10.96 sh 8550 anthony 35 10 4032 1204 916 R 98.8 0.1 0:10.72 sh 16204 root 21 0 30140 5600 3528 S 0.0 0.5 0:00.22 slapd I tried to play with cachesize and idlcachesize (set them to 10 thousand), but didn't see any difference, which hardly surprises me given that my ldap database has only 257 records. Finally, here is my DB_CONFIG: set_cachesize 0 2097152 0 set_lg_bsize 524288 set_lk_max_objects 5000 set_lk_max_locks 5000 set_lk_max_lockers 5000 (My slapd.conf does not contain any db-related parameters). My original message: > Hi, > > At the almost idle Dual Core machine which runs slapd, I run: > > nice sh -c 'while true; do true; done' & > nice sh -c 'while true; do true; done' & > > (i.e. I'm running this twice). Then each of the two CPUs always has > some job to do, so both CPUs have 100% usage, but this is "nice". > > Then, slapd takes too long to respond to queries. It may take 10 or > 20 seconds. If I kill or stop one of the two dummy processes, it > replies instantly. If I continue both dummy processes, it's back to > 10 or 20 seconds. Needless to say all machine resources seem ok; low > disk usage, lots of spare memory; and slapd is not niced. > > If it's not something immediately obvious, could you help me debug it? > I've run slapd with various "-d" options but it gives me results that > I have trouble understanding. > > The OS is Debian 3.1 (Sarge), with a 2.6.12 SMP Linux kernel.

3 3

dnsDomain2.schema and aRecord
by Jürgen Magin 13 Dec '06

13 Dec '06

Hello I'm using dnsDomain2 schema with openldap 2.2.27 for dns server (pdns). When I look into the logfiles i saw that some questions are not answered by slapd. When I ask: ldapsearch -x -LLL "dc=hostname" i get the follwing answer: dn: dc=hostnanme,dc=example.local,ou=DNS,ou=Services,dc=example,dc=local objectClass: top objectClass: dNSDomain2 objectClass: domainRelatedObject dc: hostname aRecord: 192.168.1.1 pTRRecord: hostname.example.local associatedDomain: hostname.example.local ok, but ldapsearch -x -LLL "aRecord=192.168.1.1" or ldapsearch -x -LLL "aRecord=192*" returns nothing and ldapsearch -x -LLL "aRecord=*" returns all entries. The entry 'aRecord' is of type caseIgnoreIA5Match. What's wrong there? Any ideas? -- Mit freundlichen Grüßen Jürgen Magin ************************************************************** # # # OCTOGON Software Development GmbH # # http://www.octo-soft.de # # # # Jürgen Magin, Einsteinstr. 11, D 68519 Viernheim # # # # Tel : +49 6204/738353 # # Fax : +49 6204/914875 # # EMail : gaston(a)octo-soft.de # # # **************************************************************

4 11

syncrepl and slowness during updates
by Paul Main 12 Dec '06

12 Dec '06

Hello, Here at UCI we have just deployed syncrepl to replicate our LDAP directory. Unfortunately, we use an old directory structure (PH/QI) for our master database, and ldap gets updates from that system in batch. The batch process updates our master LDAP server, and syncrepl is used to push the changes out to all the other LDAP servers. This causes somewhere around 100 to 1000 entries to be updated all at once (out of 80k + entries in the LDAP directory). The problem we are experiencing is that when a syncrepl slave receives a bunch of updates, queries to these slaves slow down tremendously. We are talking going from a sub second query time for a single LDAP entry, to, in some cases, over 20 seconds response time for simple queries. This is causing all sorts of problems for us. One thing to note: the master does basically the same updates, but through an normal ldap client, rather than through syncrepl -- and it does not experience this slowness. We are using BDB, Openldap ver. 2.3.28 and our syncrepl entry looks something like this: syncrepl rid=11 provider=ldap://ldap3.nac.uci.edu:389 type=refreshAndPersist interval=0:00:00:05 searchbase="OU=University of California Irvine, O=University of California, C=US" filter="(objectClass=*)" scope=sub schemachecking=off sizelimit=0 timelimit=0 updatedn="cn=root,OU=University of California Irvine, O=Universi ty of California, C=US" bindmethod=simple binddn="uid=nsp,OU=University of California Irvine, O=University of California, C=US" credentials="REMOVED" starttls=yes retry=1,2,3,4,5,+ Any help would be appreciated, -Paul Main Network and Academic Computing Services University of California, Irvine

3 3

misc.schema bundled with OpenLDAP - recommended?
by Gavin Henry 12 Dec '06

12 Dec '06

Hi all, I notice that misc.schema says at the top: 20 # Not recommended for production use! 21 # Use with extreme caution! This definitely the case?, as I wanted to use 'nisMailAlias' and 'rfc822MailMember' from it, but not sure now. I've done our own schema ages ago with suretecMailAlias/suretecMailAliasMember, so might stick with that, but wanted to use a bundled schema for most things. Should we stick with our own for Mail Aliases, or use bundled? Advice? Thanks. -- Kind Regards, Gavin Henry. Managing Director. T +44 (0) 1224 279484 M +44 (0) 7930 323266 F +44 (0) 1224 824887 E ghenry(a)suretecsystems.com Open Source. Open Solutions(tm). http://www.suretecsystems.com/

3 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-software December 2006