Hello,
[I sent this message once but somehow it didn't get through so I resend it -- my sincere apology if anyone received this twice]
I am trying to troubleshot a problem with entries being deleted on consumer, exactly as described in this thread: http://www.openldap.org/lists/openldap-software/200704/msg00248.html
I am running both provider and consumer on debian etch. After some searching I see some hints that openldap as provided by debian etch (2.3.30) is out of date and this bug might be fixed in 2.3.34, I installed openldap from sources (2.3.43) on both provider and consumer. But the problem is not gone.
Perhaps it has something to do with the fact that the consumer is not online all the time; it is switched on/off daily. I am thinking of upgrading to openldap 2.4, but I am still a bit reluctant since the configuration syntax has changed.
Can someone please give me a hint how to proceed further? I appended the config file of both provider/consumer below. They were derived from debian config files with little modification.
thanks, Tony
Update: it seems that the deletion happens on consumer if data on provider have been changed during the period the consumer was off. At least I can reproduce this consistently.
slapd.conf on provider: # This is the main slapd configuration file. See slapd.conf(5) for more # info on the configuration options.
####################################################################### # Global Directives:
# Features to permit #allow bind_v2
# Schema and objectClass definitions include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema
## <added for gosa minimal> include /etc/ldap/schema/gosa/samba3.schema include /etc/ldap/schema/gosa/trust.schema include /etc/ldap/schema/gosa/gosystem.schema include /etc/ldap/schema/gosa/gofon.schema include /etc/ldap/schema/gosa/goto.schema include /etc/ldap/schema/gosa/gosa-samba3.schema include /etc/ldap/schema/gosa/gofax.schema include /etc/ldap/schema/gosa/goserver.schema include /etc/ldap/schema/gosa/goto-mime.schema ## </added for gosa minimal>
## <added for gosa additional plugins> include /etc/ldap/schema/gosa/fai.schema ## </added for gosa additional plugins>
# Where the pid file is put. The init.d script # will not stop the server if you change this. pidfile /var/run/slapd/slapd.pid
# List of arguments that were passed to the server argsfile /var/run/slapd/slapd.args
# Read slapd.conf(5) for possible values loglevel 0
# Where the dynamically loaded modules are stored modulepath /usr/lib/ldap moduleload back_bdb moduleload syncprov.la
# The maximum number of entries that is returned for a search operation sizelimit 500
# The tool-threads parameter sets the actual amount of cpu's that is used # for indexing. tool-threads 1
####################################################################### # Specific Backend Directives for bdb: # Backend specific directives apply to this backend until another # 'backend' directive occurs backend bdb checkpoint 512 30
####################################################################### # Specific Backend Directives for 'other': # Backend specific directives apply to this backend until another # 'backend' directive occurs #backend <other>
####################################################################### # Specific Directives for database #1, of type bdb: # Database specific directives apply to this databasse until another # 'database' directive occurs database bdb
# The base of your directory in database #1 suffix "dc=example,dc=com"
# rootdn directive for specifying a superuser on the database. This is needed # for syncrepl. # rootdn "cn=admin,dc=example,dc=com"
# Where the database file are physically stored for database #1 directory "/var/lib/ldap"
# For the Debian package we use 2MB as default but be sure to update this # value if you have plenty of RAM dbconfig set_cachesize 0 2097152 0
# Sven Hartge reported that he had to set this value incredibly high # to get slapd running at all. See http://bugs.debian.org/303057 # for more information.
# Number of objects that can be locked at the same time. dbconfig set_lk_max_objects 1500 # Number of locks (both requested and granted) dbconfig set_lk_max_locks 1500 # Number of lockers dbconfig set_lk_max_lockers 1500
# Indexing options for database #1 #index objectClass eq
# Save the time that the entry gets modified, for database #1 lastmod on
# Where to store the replica logs for database #1 # replogfile /var/lib/ldap/replog
## <setting for replica using syncreplrefreshAndPersist; master config> overlay syncprov syncprov-checkpoint 100 10 # syncprov-sessionlog 100
index objectclass,entryCSN,entryUUID eq
TLSCertificateFile /etc/ssl/certs/cert.pem TLSCertificateKeyFile /etc/ssl/certs/key.pem TLSCACertificateFile /etc/ssl/certs/cacert.pem
TLSCipherSuite HIGH:MEDIUM:+SSLv2 ## </setting for replica using syncreplrefreshAndPersist; master config>
# The userPassword by default can be changed # by the entry owning it if they are authenticated. # Others should not be able to see it, except the # admin entry below # These access lines apply to database #1 only
# ldapSync can read anything access to * by dn.base="cn=ldapSync,dc=example,dc=com" read by * none break
# allow readUserInfo can read users and groups access to dn.regex=".*ou=(people|groups),dc=example,dc=com" by dn.base="cn=readUserInfo,ou=admin,dc=example,dc=com" read by * none break
# admin can change anything access to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=example,dc=com" write by anonymous auth by self write by * none
# Ensure read access to the base for things like # supportedSASLMechanisms. Without this you may # have problems with SASL not knowing what # mechanisms are available and the like. # Note that this is covered by the 'access to *' # ACL below too but if you change that as people # are wont to do you'll still need this if you # want SASL (and possible other things) to work # happily. access to dn.base="" by * read
# The admin dn has full write access, everyone else # can read everything. access to * by dn="cn=admin,dc=example,dc=com" write by * read
# For Netscape Roaming support, each user gets a roaming # profile for which they have write access to #access to dn=".*,ou=Roaming,o=morsnet" # by dn="cn=admin,dc=example,dc=com" write # by dnattr=owner write
####################################################################### # Specific Directives for database #2, of type 'other' (can be bdb too): # Database specific directives apply to this databasse until another # 'database' directive occurs #database <other>
# The base of your directory for database #2 #suffix "dc=debian,dc=org"
slapd.conf on consumer:
# This is the main slapd configuration file. See slapd.conf(5) for more # info on the configuration options.
####################################################################### # Global Directives:
# Features to permit #allow bind_v2
# Schema and objectClass definitions include /etc/ldap/schema/core.schema include /etc/ldap/schema/cosine.schema include /etc/ldap/schema/nis.schema include /etc/ldap/schema/inetorgperson.schema
## <added for gosa minimal> include /etc/ldap/schema/gosa/samba3.schema include /etc/ldap/schema/gosa/trust.schema include /etc/ldap/schema/gosa/gosystem.schema include /etc/ldap/schema/gosa/gofon.schema include /etc/ldap/schema/gosa/goto.schema include /etc/ldap/schema/gosa/gosa-samba3.schema include /etc/ldap/schema/gosa/gofax.schema include /etc/ldap/schema/gosa/goserver.schema include /etc/ldap/schema/gosa/goto-mime.schema ## </added for gosa minimal>
## <added for gosa additional plugins> include /etc/ldap/schema/gosa/fai.schema ## </added for gosa additional plugins>
# Where the pid file is put. The init.d script # will not stop the server if you change this. pidfile /var/run/slapd/slapd.pid
# List of arguments that were passed to the server argsfile /var/run/slapd/slapd.args
# Read slapd.conf(5) for possible values # loglevel 0 loglevel 16384
# Where the dynamically loaded modules are stored modulepath /usr/lib/ldap moduleload back_bdb moduleload ppolicy.la
# The maximum number of entries that is returned for a search operation sizelimit 500
# The tool-threads parameter sets the actual amount of cpu's that is used # for indexing. tool-threads 1
####################################################################### # Specific Backend Directives for bdb: # Backend specific directives apply to this backend until another # 'backend' directive occurs backend bdb checkpoint 512 30
####################################################################### # Specific Backend Directives for 'other': # Backend specific directives apply to this backend until another # 'backend' directive occurs #backend <other>
####################################################################### # Specific Directives for database #1, of type bdb: # Database specific directives apply to this databasse until another # 'database' directive occurs database bdb
# The base of your directory in database #1 suffix "dc=example,dc=com"
# rootdn directive for specifying a superuser on the database. This is needed # for syncrepl. # rootdn "cn=admin,dc=example,dc=com" rootdn "cn=admin,dc=example,dc=com"
# Where the database file are physically stored for database #1 directory "/var/lib/ldap"
# For the Debian package we use 2MB as default but be sure to update this # value if you have plenty of RAM dbconfig set_cachesize 0 2097152 0
# Sven Hartge reported that he had to set this value incredibly high # to get slapd running at all. See http://bugs.debian.org/303057 # for more information.
# Number of objects that can be locked at the same time. dbconfig set_lk_max_objects 1500 # Number of locks (both requested and granted) dbconfig set_lk_max_locks 1500 # Number of lockers dbconfig set_lk_max_lockers 1500
# Indexing options for database #1 index objectClass eq
# Save the time that the entry gets modified, for database #1 lastmod on
## read-only is good for slave readonly on
# Where to store the replica logs for database #1 # replogfile /var/lib/ldap/replog
syncrepl rid=001 provider=ldaps://ldap-provider.example.com:636 type=refreshAndPersist retry="5 5 300 +" searchbase="dc=example,dc=com" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldapSync,dc=example,dc=com" credentials=SECRET
# allow readUserInfo can read users and groups access to dn.regex=".*ou=(people|groups),dc=example,dc=com" by dn.base="cn=readUserInfo,dc=example,dc=com" read by * none break
# The userPassword by default can be changed # by the entry owning it if they are authenticated. # Others should not be able to see it, except the # admin entry below # These access lines apply to database #1 only access to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=example,dc=com" write by anonymous auth by self write by * none
# Ensure read access to the base for things like # supportedSASLMechanisms. Without this you may # have problems with SASL not knowing what # mechanisms are available and the like. # Note that this is covered by the 'access to *' # ACL below too but if you change that as people # are wont to do you'll still need this if you # want SASL (and possible other things) to work # happily. access to dn.base="" by * read
# default access to * by self read by users read by * none
--On Friday, September 18, 2009 7:37 PM +0100 Tony Smith tony.smith.124@googlemail.com wrote:
Hello,
Hi Tony,
This is due to a common mistake of using "attrs=*", which removes the operational attributes that syncrepl uses to track changes. I really wish I knew where people got this from, because 99.999999% of the time you do not want to use this value. You should not specify the attrs= line at all in a syncrepl configuration unless you really really need to limit replication in some way. If you want to keep the attrs line, change it to attrs="*,+" so that all attributes are replicated.
syncrepl rid=001 provider=ldaps://ldap-provider.example.com:636 type=refreshAndPersist retry="5 5 300 +" searchbase="dc=example,dc=com" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldapSync,dc=example,dc=com" credentials=SECRET
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
This is due to a common mistake of using "attrs=*", which removes the operational attributes that syncrepl uses to track changes. I really wish I knew where people got this from, because 99.999999% of the time you do not want to use this value. You should not specify the attrs= line at all in a syncrepl configuration unless you really really need to limit replication in some way. If you want to keep the attrs line, change it to attrs="*,+" so that all attributes are replicated.
The unfortunate reality is that even that doesn't always help. If operational attributes are set by an overlay without updating the entrycsn, etc, they still won't be replicated. Worse, if something else in an entry changes that causes replication for that entry, these op attrs *may* replicate along with the other change (I think it depends on whether or not you use delta sync), making operational attribute replication entirely unreliable and unpredictable. Learned this the hard way with password policy attributes (supposedly this has been fixed for ppolicy, but nothing preventing this from happening elsewhere/in other plugins). Until and unless core code always updates this with a write to the db, without any way for an overlay to write to the db without updating the entrycsn/etc, this will always be unreliable.
Clowser, Jeff wrote:
The unfortunate reality is that even that doesn't always help. If operational attributes are set by an overlay without updating the entrycsn, etc, they still won't be replicated. Worse, if something else in an entry changes that causes replication for that entry, these op attrs *may* replicate along with the other change (I think it depends on whether or not you use delta sync), making operational attribute replication entirely unreliable and unpredictable. Learned this the hard way with password policy attributes (supposedly this has been fixed for ppolicy, but nothing preventing this from happening elsewhere/in other plugins).
I think you're barking up the wrong tree. The ppolicy overlay was originally coded to not replicate any state attributes, by design.
openldap-software@openldap.org
-
Clowser, Jeff -
Howard Chu -
Quanah Gibson-Mount -
Tony Smith