openldap-software November 2009

openldap-software@openldap.org

38 participants
30 discussions

syncrepl replication on 2.4.19 (stable)
by Brett ＠Google 12 Nov '09

12 Nov '09

Hello, I am having a very odd problem after upgrading from openldap 2.4.16 (stable) I have a syncrepl provider/ consumer setup using openldap 2.4.19 (stable) and when i start an empty consumer, in the provider logs i am getting: Nov 4 17:07:51 producer slapd[7250]: [ID 702911 local4.debug] @(#) $OpenLDAP: slapd 2.4.19 (Nov 4 2009 12:53:47) $ Nov 4 17:07:51 producer @qgdevpro:/home/govops/build.local/openldap-2.4.19/servers/slapd Nov 4 17:07:51 producer slapd[7286]: [ID 100111 local4.debug] slapd starting Nov 4 17:08:04 producer slapd[7286]: [ID 848112 local4.debug] conn=0 fd=16 ACCEPT from IP=10.0.0.2:53951 (IP=10.0.0.1:389) Nov 4 17:08:04 producer slapd[7286]: [ID 215403 local4.debug] conn=0 op=0 BIND dn="cn=replicator,dc=example,dc=org" method=128 Nov 4 17:08:04 producer slapd[7286]: [ID 600343 local4.debug] conn=0 op=0 BIND dn="cn=replicator,dc=example,dc=org" mech=SIMPLE ssf=0 Nov 4 17:08:04 producer slapd[7286]: [ID 588225 local4.debug] conn=0 op=0 RESULT tag=97 err=0 text= Nov 4 17:08:04 producer slapd[7286]: [ID 469902 local4.debug] conn=0 op=1 SRCH base="dc=example,dc=org" scope=2 deref=0 filter="(objectClass=*)" Nov 4 17:08:04 producer slapd[7286]: [ID 744844 local4.debug] conn=0 op=1 SRCH attr=* + Nov 4 17:08:04 producer slapd[7286]: [ID 832699 local4.debug] conn=0 op=1 SEARCH RESULT tag=101 err=0 nentries=0 text= Nov 4 17:08:04 producer slapd[7286]: [ID 218904 local4.debug] conn=0 op=2 UNBIND Nov 4 17:08:04 producer slapd[7286]: [ID 952275 local4.debug] conn=0 fd=16 closed on the consumer i get a lot of (one set after each refresh attempt) : Nov 4 17:41:04 consumer slapd[7660]: [ID 365351 local4.debug] do_syncrep2: rid=001 LDAP_RES_SEARCH_RESULT Nov 4 17:41:04 consumer slapd[7660]: [ID 664938 local4.debug] do_syncrepl: rid=001 rc -2 retrying Important part being "nentries=0", i run the equivalent command at the command propmt of the consumer, ie: ldapsearch -b dc=example,dc=org -D 'cn=replicator,dc=example,dc=org' -w <password> -s sub -x '(objectclass=*) ' '* +' I get the result i would expect above, ie: Nov 4 17:20:14 producer slapd[7286]: [ID 848112 local4.debug] conn=16 fd=16 ACCEPT from IP=10.0.0.2:54049 (IP=10.0.0.1:389) Nov 4 17:20:14 producer slapd[7286]: [ID 215403 local4.debug] conn=16 op=0 BIND dn="cn=replicator,dc=example,dc=org" method=128 Nov 4 17:20:14 producer slapd[7286]: [ID 600343 local4.debug] conn=16 op=0 BIND dn="cn=replicator,dc=example,dc=org" mech=SIMPLE ssf=0 Nov 4 17:20:14 producer slapd[7286]: [ID 588225 local4.debug] conn=16 op=0 RESULT tag=97 err=0 text= Nov 4 17:20:14 producer slapd[7286]: [ID 469902 local4.debug] conn=16 op=1 SRCH base="dc=example,dc=org" scope=2 deref=0 filter="(objectClass=*)" Nov 4 17:20:14 producer slapd[7286]: [ID 744844 local4.debug] conn=16 op=1 SRCH attr=* + Nov 4 17:21:03 producer slapd[7286]: [ID 832699 local4.debug] conn=16 op=1 SEARCH RESULT tag=101 err=0 nentries=85611 text= Nov 4 17:21:03 producer slapd[7286]: [ID 218904 local4.debug] conn=16 op=2 UNBIND Nov 4 17:21:03 producer slapd[7286]: [ID 952275 local4.debug] conn=16 fd=16 closed Note here i get nentries=85611 (with a phole bunch of results) for what is essentialy the same query. I'd appreciate any feedback, surely i must be missing something really obvious? My config is below. Cheers Brett << begin of provider slapd >> ###################################################################### # global options ###################################################################### include /usr/local/openldap/etc/openldap/schema/core.schema include /usr/local/openldap/etc/openldap/schema/cosine.schema include /usr/local/openldap/etc/openldap/schema/inetorgperson.schema modulepath /usr/local/openldap/libexec/openldap #moduleload back_ldbm.la #moduleload back_monitor.la pidfile /var/openldap/run/slapd.pid argsfile /var/openldap/run/slapd.args # threads for faster concurrent slapadd tool-threads 4 ###################################################################### # global database ACLs ###################################################################### # allow replicator to read all access to * by dn.exact="cn=replicator,dc=example,dc=org" read by * break [ ..etc.. ] # default rules access to * by self write by * read ###################################################################### # logging configuration ###################################################################### # testing loglevel stats sync ###################################################################### # primary database ###################################################################### database hdb suffix "dc=example,dc=org" directory /var/openldap/data rootdn "cn=Manager, dc=example,dc=org" rootpw <password> checkpoint 2000 15 cachesize 20000 idlcachesize 60000 cachefree 4000 # unlimited dn cache (openldap 2.4.16 and above) dncachesize 0 # General Indexes (there is more than this - but they are all the same form) index default pres,eq index objectClass,uid,mail pres,eq index cn,sn,ou,streetAddress,givenName,title,telephoneNumber eq,sub # Indices for Syncrepl index entryCSN,entryUUID eq # allow replicator DN have unlimited searches (per-database) limits dn.exact="cn=replicator,dc=example,dc=org" time=unlimited size=unlimited ###################################################################### # replication information - monitor backend ###################################################################### database monitor << end of provider slapd >> << below snipit added to above on the consumer only, just before "database monitor", but after the rest of the config >> ###################################################################### # replication information - only for consumer ###################################################################### # Where we pull data from syncrepl rid=001 provider=ldap://provider.example.org:389 bindmethod=simple binddn="cn=replicator,dc=example,dc=org" credentials=<password> searchbase="dc=example,dc=org" filter=(objectclass=*) attrs="*,+" schemachecking=off scope=sub type=refreshAndPersist retry="60 +" # not using accesslog atm - debugging initial refresh # logbase="cn=accesslog" # logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" # syncdata=accesslog # Refer all rights to master updateref ldap://provider.example.org:389

3 5

Troubleshooting synchronization
by Torsten Schlabach (Tascel eG) 12 Nov '09

12 Nov '09

Hi all! I am currently trying to chase some problems in an n-way multi-master setup with three servers. We have used the instructions at http://www.openldap.org/doc/admin24/replication.html#N-Way%20Multi-Master as our guidance and we are using OpenLDAP version 2.4.11. The result we see currently is that replication works only partially, with some strange errors here and there. As I believe it will be pointless to post all our cn=config LDIF here and explain scenarios which work and those which don't, I thought it would be more productive to double-check that I have correctly understand what I *should* be seeing happen on my systems and how I can properly monitor this. My problem may be that I still need to learn how to properly monitor my slapd. To begin with, I would just ask for confirmation of my proper understanding of the documentation: 1. A master server is a server which is using the syncprov overlay (servers/slapd/overlays/syncprov.c). This overlay will do little more than just provide a synchronization cookie (CSN) which consumers may ask for to find out what needs to get replicated and what not. 2. A consumer server is a server in which an additional thread is running which will query the master(s) in a given interval to ask for updated and if any, get them over the wire and into the local copy of the database. This synchronization thread is servers/slapd/syncrepl.c I guess? 3. An N-Way Multi-Master setup is a setup in which N servers are each a master and any of the others is a consumer of all other masters? I am I right up to here? So what I fail to understand is: 1. What is the difference between Mirror Mode and N-Way Multi-Master? Especially given that in N-Way Multi-Master, have to set olcMirrorMode to TRUE. 2. Given that I have added a 'Sync' value to the olcLogLevel attribute, what would be the "health check" information I should be watching in the log for to see that replication is attempted as expected. 3. What problems should I be watching for in the logs? 4. Could I for example manually ask a master (using some ldapsearch statement, pretending I was the consumer) what the master thinks which entries I would have to update? Regards, Torsten

5 18

TLS renotiation
by Dieter Kluenter 12 Nov '09

12 Nov '09

Hi, I just wonder weather LDAP and in particular OpenLDAP is affected by TLS client auth renegotiation, as described http://extendedsubset.com/?p=8 and the fix http://isc.sans.org/diary.html?storyid=7543 -Dieter -- Dieter Klünter | Systemberatung http://dkluenter.de GPG Key ID:8EF7B6C6 53°37'09,95"N 10°08'02,42"E

5 7

Anonymous Syncrepl?
by Eric B. 11 Nov '09

11 Nov '09

Hi, I'm relatively new to OpenLDAP and am trying to set up a slave server. I figured the easiest way would be to use the anonymous user to perform the synchronization given that my master allows for full anonymous reads: access to * by self write by users read by anonymous read I have tried to specify the following in my slave slapd.conf: syncrepl rid=8 provider=ldap://snoopy.domain.com:389 type=refreshAndPersist retry="60 +" searchbase="dc=domain,dc=com" schemachecking=off bindmethod=simple However, my slave seems to be unable to connect properly to the master. It seems to be trying to write something, and am not quite sure what. My master has the following log: Nov 9 16:37:52 snoopy slapd[1481]: conn=6270 fd=72 ACCEPT from IP=10.1.1.8:39558 (IP=0.0.0.0:389) Nov 9 16:37:52 snoopy slapd[1481]: conn=6270 op=0 BIND dn="" method=128 Nov 9 16:37:52 snoopy slapd[1481]: conn=6270 op=0 RESULT tag=97 err=0 text= Nov 9 16:37:52 snoopy slapd[1481]: conn=6270 op=1 SRCH base="dc=domain,dc=com" scope=2 deref=0 filter="(objectClass=*)" Nov 9 16:37:52 snoopy slapd[1481]: conn=6270 op=1 SRCH attr=* + Nov 9 16:37:52 snoopy slapd[1481]: send_search_entry: conn 6270 ber write failed. Nov 9 16:37:52 snoopy slapd[1481]: conn=6270 fd=72 closed (connection lost on write) My slave logs display the following: Nov 9 16:45:36 spike slapd[32415]: do_syncrep2: rid 008got search entry without control Nov 9 16:45:36 spike slapd[32415]: do_syncrepl: rid 008 retrying I thought it might have something to do with the type (in that refreshAndPersist may require some form of write privileges), so I switched to type refreshOnly, however, it made no difference. My log outputs remain the same. Can anyone steer me in the correct direction? Thanks, Eric _________________________________________________________________ Windows Live: Keep your friends up to date with what you do online. http://go.microsoft.com/?linkid=9691815

3 2

hdb_search: does not match filter
by Antonini Gabriele 11 Nov '09

11 Nov '09

Hello, I'm running openldap 2.3.43 on CentOS. I'm tring to setup a working master-slave sync system. I populate master server using slapadd -q -l from a ~160k entry ldif file. Every time I (re)start master server it takes long time to become available (about 20 minutes), in the logs I read these entries: Nov 10 13:56:17 ldap01 slapd[2191]: hdb_search: 1 does not match filter Nov 10 13:56:17 ldap01 slapd[2191]: entry_decode: "" Nov 10 13:56:17 ldap01 slapd[2191]: <= entry_decode() Nov 10 13:56:17 ldap01 slapd[2191]: hdb_search: 2 does not match filter Nov 10 13:56:17 ldap01 slapd[2191]: entry_decode: "" Nov 10 13:56:17 ldap01 slapd[2191]: <= entry_decode() Nov 10 13:56:17 ldap01 slapd[2191]: hdb_search: 3 does not match filter Nov 10 13:56:17 ldap01 slapd[2191]: entry_decode: "" ... ... ... Nov 10 14:17:58 ldap01 slapd[32246]: hdb_search: 162959 does not match filter Nov 10 14:17:58 ldap01 slapd[32246]: entry_decode: "" Nov 10 14:17:58 ldap01 slapd[32246]: <= entry_decode() Nov 10 14:17:58 ldap01 slapd[32246]: send_ldap_result: conn=-1 op=0 p=0 Nov 10 14:17:58 ldap01 slapd[32246]: slapd starting If I try to comment the syncprov section: overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100 in slapd.conf the server startup is very quick (2-3 seconds). Is there something wrong in what I'm doing? Is this very long startup normal? Thanks, G.

2 2

Bug when converting syncprov-checkpoint to olcSpCheckpoint?
by Kyle Blaney 11 Nov '09

11 Nov '09

I am experiencing unexpected results when converting the syncprov-checkpoint option in file-based configuration to the olcSpCheckpoint attribute in online configuration. To reproduce the unexpected results: 1. Configure OpenLDAP using file-based configuration that contains the following line: syncprov-checkpoint 100 10 2. Convert file-based configuration to online configuration using the following command: slapd -f slapd.conf -F slapd.d My file-based configuration is converted to the cn=config database, but after conversion the value of the olcSpCheckpoint attribute in the olcOverlay={1}syncprov,olcDatabase={1}bdb,cn=config entry is "100 600" when I expect it to be the same as in my file-based configuration ("100 10"). Is this a bug or does the olcSpCheckpoint attribute use different units (seconds) than the syncprov-checkpoint option (minutes)? Kyle Blaney

5 8

Syncrepl: Two step forward, one step back
by Peter Mogensen 10 Nov '09

10 Nov '09

Hi, I still can't figure out what to think of this: I have a mirrormode setup. One server has the database, the other starts empty. The data is slowly propagating from server-1 to server-2, but when I monitor the process by counting how many objects are created on server-2 below the root object, it is not strictly increasing. All object below the root has DN: o=... Is see output like this: # slapcat | grep 'dn: o=' | wc -l 601 # slapcat | grep 'dn: o=' | wc -l 622 # slapcat | grep 'dn: o=' | wc -l 620 # slapcat | grep 'dn: o=' | wc -l 628 Why does server-2 "regret" already replicated objects? Is this expected behaviour? /Peter

3 8

2.4.19 (stable) - sync replication issue
by Ken Ko 10 Nov '09

10 Nov '09

Hello I have TWO openldap servers (ServerA & ServerB) and recently i upgraded to 2.4.19. Now when I add a record into ServerA, the new record appear inside both ServerA & ServerB, then 20 minutes later, the record will be deleted in both servers. Here is the log that i discover: Nov 4 23:33:48 srr200-001 slapd[28100]: nonpresent_callback: rid=002 nonpresent UUID e73bcd9c-5da0-102e-96f1-8d1c45da731c, dn uid=amyzjkang,ou=users,dc=ygmt,dc=com Nov 4 23:33:48 srr200-001 slapd[28100]: syncrepl_del_nonpresent: rid=002 be_delete uid=amyzjkang,ou=users,dc=ygmt,dc=com (0) Here is my conf file: serverID 1 include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/rfc2307bis.schema include /etc/openldap/schema/yast.schema include /etc/openldap/schema/samba3.schema include /etc/openldap/schema/dnszone.schema include /etc/openldap/schema/ygmt.schema pidfile /var/run/slapd/slapd.pid argsfile /var/run/slapd/slapd.args # Load dynamic backend modules: modulepath /usr/lib/openldap/modules access to attrs=SambaLMPassword,SambaNTPassword by dn="uid=administrator,ou=users,dc=ygmt,dc=com" write by * none access to dn.base="" by * read access to dn.base="cn=Subschema" by * read access to attrs=userPassword,userPKCS12 by self write by * auth access to attrs=shadowLastChange by self write by * read access to * by * read loglevel 16384 TLSCertificateFile /etc/ssl/servercerts/servercert.pem TLSCACertificatePath /etc/ssl/certs/ TLSCertificateKeyFile /etc/ssl/servercerts/serverkey.pem database bdb suffix "dc=ygmt,dc=com" rootdn "uid=administrator,ou=users,dc=ygmt,dc=com" rootpw "12345678pass" directory /var/lib/ldap checkpoint 1024 5 cachesize 10000 index objectClass,uidNumber,gidNumber eq index member,mail eq,pres index cn,displayname,uid,sn,givenname sub,eq,pres index sambaSID eq index sambaPrimaryGroupSID eq index sambaDomainName eq index entryCSN,entryUUID eq index memberUid eq index uniqueMember eq,pres index sambaSIDList eq index sambaGroupType eq overlay memberof syncrepl rid=001 provider=ldap://172.16.2.1 searchbase="dc=ygmt,dc=com" bindmethod=simple binddn="uid=administrator,ou=users,dc=ygmt,dc=com" credentials=12345678pass type=refreshOnly interval=00:00:05:00 retry="20 5 300 +" schemachecking=off sizelimit=unlimited timelimit=unlimited mirrormode on overlay syncprov Both of my server slapd.conf is identical except serverID and the provider=ldap ip address The ldap syncrepl was working previously. Since the upgrade, i can't add any users if the syncrepl is turn on. Now if i want to add a record, I have to manually turn off syncrepl. Then add the user into ServerA, Stop ServerB ldap and remove ldap DB, turn on syncrepl, restart & re sync. How can i fix this issue? Thanks~ Ken Windows Live: Keep your friends up to date with what you do online. _________________________________________________________________ Eligible CDN College & University students can upgrade to Windows 7 before Jan 3 for only $39.99. Upgrade now! http://go.microsoft.com/?linkid=9691819

3 3

Syncrepl: 3 simple questions
by Torsten Schlabach (Tascel eG) 10 Nov '09

10 Nov '09

Hi! What I would like to understand: 1. How do I query a master for the cookie? 2. How do I query a slave for the coolie? 3. How do I query the master in a human readable format for all changes based on a cookie which I present? My apologies if these questions should have been answered by reading the documentation; I did not find it there, unfortunately. I am sure someone who is a bit more into this can just reply with three simple ldadpsearch statemenets. Regards, Torsten

4 11

/var/lib/ldap/log.000*
by Ed Greenberg 09 Nov '09

09 Nov '09

I have a setup with a master and four slaves, all of which have a number of files in the format /var/lib/ldap/log.000* The replication is syncrepl. Are all these files needed? How are they pruned/maintained? I am running out of disk space :) What can I read about this? I couldn't find anything. Thanks, </edg>

3 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-software November 2009