Hello list,
SunOS ldapmaster01.unix 5.10 Generic_118844-26 i86pc i386 i86pc openldap-2.3.19 db-4.2.52.NC-sol10-intel-local ./configure LDFLAGS=-L/usr/local/lib --enable-crypt
Generally, Openldap is working well, and replication is working. I get no .rej files, and the replog file itself is frequently drained.
But, ldapmaster and ldapslave frequently get out of sync. I thought it was initially just on the modrdn operations, but we have had "simpler" operations vanish.
I realise this isn't much to go on, but perhaps I can get suggestions which may lead to more clues.
The important looking sections in slapd.conf:
loglevel none database bdb suffix "dc=company,dc=jp" rootdn "cn=admin,dc=company,dc=jp"
access to attr=userPassword by self write by anonymous auth by * none
access to * by self write by dn="cn=admin,dc=company,dc=jp" write by * read
password-hash {CRYPT}
checkpoint 128 15
directory /usr/local/var/openldap-data index objectClass eq index uid eq index uidNumber eq index mail eq index mailAlternateAddress pres,eq index deliveryMode eq index accountStatus eq index gecos eq index radiusGroupName eq index o pres,eq
replogfile /usr/local/var/openldap-slurp/replica/slurpd.replog
replica host=172.20.12.23:389 binddn="cn=admin,dc=company,dc=jp" bindmethod=simple credentials="<secret>"
ldapslave slapd.conf is identical (it really is scp'ed) but with the last two lines removed (replogfile, replica).
# /usr/local/bin/ldapsearch -h ldapmaster01 -D cn=admin,dc=company,dc =jp -x -w secret -b ou=mail,dc=company,dc=jp mail=test01@cs-sd01.com # extended LDIF # # LDAPv3 # base <ou=mail,dc=company,dc=jp> with scope subtree # filter: mail=test01@cs-sd01.com # requesting: ALL #
# test01, cs-sd01.com, mail, company.jp dn: uid=test01,o=cs-sd01.com,ou=mail,dc=company,dc=jp objectClass: top objectClass: account objectClass: posixAccount objectClass: shadowAccount objectClass: qmailUser objectClass: amavisAccount uid: test01 cn: test01 [snip]
# search result search: 2 result: 0 Success
# numResponses: 2 # numEntries: 1
# /usr/local/bin/ldapsearch -h ldapslave01 -D cn=admin,dc=company,dc=jp -x -w <secret> -b ou=mail,dc=company,dc=jp mail=test01@cs-sd01.com # extended LDIF # # LDAPv3 # base <ou=mail,dc=secret,dc=jp> with scope subtree # filter: mail=test01@cs-sd01.com # requesting: ALL #
# search result search: 2 result: 0 Success
# numResponses: 1
The various slapdlogs appear to contain little of interest, but perhaps "loglevel none" is the wrong setting?
ldapmaster usually gives lines like:
Mar 25 09:51:23 ldapmaster01.unix slapd[1319]: [ID 458966 local4.debug] do_search: invalid dn (uid=a.user, o=, ou=mail, dc=company, dc=jp) Mar 25 09:51:25 ldapmaster01.unix slapd[1319]: [ID 458966 local4.debug] do_search: invalid dn (uid=user4447, o=, ou=mail, dc=company, dc=jp) Mar 25 09:51:29 ldapmaster01.unix slapd[1319]: [ID 458966 local4.debug] do_search: invalid dn (uid=6201535, o=, ou=mail, dc=company, dc=jp)
Which is just users entering their usernames wrong.
ldapslave01 has no real log entries:
Mar 24 22:42:41 ldapslave01.unix slapd[25495]: [ID 854361 local4.debug] bdb_db_open: unclean shutdown detected; attempting recovery. Mar 24 22:42:41 ldapslave01.unix slapd[25495]: [ID 100111 local4.debug] slapd starting
When things are out of sync, I stop ldapslave01, stop provisioning (the only thing writing to ldap), then rsync everything from ldapmaster, and start it again.
We are considering running without ldapslave until it can be resolved, it will just a bit slower. As I said, it is not all replication that fails, just some.
There was a period where ldapslave was experiencing "too many files open error", but we increased file descriptors since then (6 weeks ago).
I do also have a DBCONFIG file.
Lund
--On Tuesday, March 25, 2008 9:59 AM +0900 Jorgen Lundman lundman@lundman.net wrote:
Hello list,
SunOS ldapmaster01.unix 5.10 Generic_118844-26 i86pc i386 i86pc openldap-2.3.19 db-4.2.52.NC-sol10-intel-local ./configure LDFLAGS=-L/usr/local/lib --enable-crypt
Here's a list of problems with your setup:
You are using an ancient version of OpenLDAP. You are using BDB 4.2.52 without the required patches. You are using slurpd.
I realise this isn't much to go on, but perhaps I can get suggestions which may lead to more clues.
Use a current version of OpenLDAP (I'd recommend 2.3.41 for now) Patch your BDB version Use syncrepl instead of slurpd
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Thank you for your fast reply:
openldap-2.3.19
You are using an ancient version of OpenLDAP.
The newest available when this project started, which took longer than expected. I can probably upgrade to newer as long as it is backward compatible.
You are using BDB 4.2.52 without the required patches.
Used latest Solaris package at the time. But will re-fetch latest stable sources, and build on test system.
You are using slurpd. Use syncrepl instead of slurpd
I have no idea what syncrepl is. The old system used slurpd successfully, but I take it there is a new feature in town. I shall see how difficult it is to set up syncrepl.
--On Tuesday, March 25, 2008 3:08 PM +0900 Jorgen Lundman lundman@lundman.net wrote:
Thank you for your fast reply:
openldap-2.3.19
You are using an ancient version of OpenLDAP.
The newest available when this project started, which took longer than expected. I can probably upgrade to newer as long as it is backward compatible.
2.3.41 is compatible DB wise with 2.3.19, assuming you keep the BDB version the same as well.
You are using BDB 4.2.52 without the required patches.
Used latest Solaris package at the time. But will re-fetch latest stable sources, and build on test system.
BDB 4.6 is the latest stable release. OpenLDAP 2.3 doesn't support BDB 4.6. I highly advise getting BDB 4.2.52 plus all relevant patches.
You are using slurpd. Use syncrepl instead of slurpd
I have no idea what syncrepl is. The old system used slurpd successfully, but I take it there is a new feature in town. I shall see how difficult it is to set up syncrepl.
syncrepl is the replacement for slurpd, as slurpd had a number of limitations. It has been completely removed from OL 2.4.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Here's a list of problems with your setup:
You are using an ancient version of OpenLDAP.
Installed 2.3.41
You are using BDB 4.2.52 without the required patches.
Installed 4.2.52 5 patches. (patch 3 failed, but appears java related)
You are using slurpd. Use syncrepl instead of slurpd
Changed to syncrepl. But I do feel that the documentation could do with a better example, one simpler. I would have thought that simply replicating everything is a common desire, when the example is one much more complicated.
These are the changes I made to master-slapd.conf (plus ./configure --enable-lastmod) :
lastmod on
index entryCSN,entryUUID eq
overlay syncprov
syncprov-checkpoint 100 10 syncprov-sessionlog 100
These are the changes I made to slave-slapd.conf:
lastmod on
index entryCSN,entryUUID eq
syncrepl rid=1 provider=ldap://172.20.12.113 type=refreshAndPersist interval=00:00:00:30 searchbase="dc=company,dc=jp" filter="(objectClass=*)" attrs="*" scope=sub schemachecking=off updatedn="cn=admin,dc=company,dc=jp" bindmethod=simple binddn="cn=admin,dc=company,dc=jp" credentials="<secret>"
updateref ldap://172.20.12.113
My replication test of one worked.
Updating of production servers went without a hitch. (knock on wood).
Cheers,
Lund
--On Wednesday, March 26, 2008 4:11 PM +0100 Michael Ströder michael@stroeder.com wrote:
Jorgen Lundman wrote:
Installed 4.2.52 5 patches. (patch 3 failed, but appears java related)
I vaguely remember there were six patches. And all should apply to the source just fine.
Ciao, Michael.
All six can be found at:
http://www.stanford.edu/services/directory/openldap/configuration/bdb-build-notes.html
The one that's missing from the Oracle BDB website is the one for fixing region size calculations.
And all of them have always applied cleanly to BDB 4.2.52's source for me...
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Jorgen Lundman wrote:
Here's a list of problems with your setup:
You are using an ancient version of OpenLDAP.
Installed 2.3.41
You are using BDB 4.2.52 without the required patches.
Installed 4.2.52 5 patches. (patch 3 failed, but appears java related)
You are using slurpd. Use syncrepl instead of slurpd
Changed to syncrepl. But I do feel that the documentation could do with a better example, one simpler. I would have thought that simply replicating everything is a common desire, when the example is one much more complicated.
These are the changes I made to master-slapd.conf (plus ./configure --enable-lastmod) :
That --enable option isn't needed. It has nothing to do with the lastmod directive in slapd.conf.
lastmod on
As documented in slapd.conf(5), lastmod is already on by default.
index entryCSN,entryUUID eq
overlay syncprov
syncprov-checkpoint 100 10 syncprov-sessionlog 100
These are the changes I made to slave-slapd.conf:
lastmod on
index entryCSN,entryUUID eq
syncrepl rid=1 provider=ldap://172.20.12.113 type=refreshAndPersist interval=00:00:00:30
"interval" is not valid for refreshAndPersist.
searchbase="dc=company,dc=jp" filter="(objectClass=*)"
That's already the default.
attrs="*"
The default is "*,+" and you probably should leave it as such.
scope=sub
That's already the default.
schemachecking=off
You don't need that.
updatedn="cn=admin,dc=company,dc=jp"
That's not valid for 2.3 syncrepl. If you've actually been reading the documentation, it seems you're reading very old docs. You should make sure you're actually reading the docs that match your code.
bindmethod=simple binddn="cn=admin,dc=company,dc=jp" credentials="<secret>"updateref ldap://172.20.12.113
My replication test of one worked.
Updating of production servers went without a hitch. (knock on wood).
openldap-software@openldap.org
-
Howard Chu -
Jorgen Lundman -
Michael Ströder -
Quanah Gibson-Mount