Hello,
We are running openldap 2.4.21 configured as a consumer, using a refreshAndPersist replication:
syncrepl rid=202 provider=ldaps://ldap.hashed.net type=refreshAndPersist interval=00:00:05:00 retry="10 3 60 +" searchbase="ou=emailservice,o=hashed" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldap-hashed-service,o=hashed" credentials=xx-xx-xx tls_reqcert=never
In order to check if the replication works OK we compare (periodically) the provider's contextCSN versus the consumer's contextCSN
Most of the time both are in sync but from time to time we see that these parameters are different (provider contextCSN > consumer contextCSN) and we thought that the consumer was not "in sync" with the provider.
Continuing our investigations we found that in the consumer there are objects with entryCSN greater than the consumer contextCSN.
Is this a normal situation or a bug?
Bellow you could find an extract from the ldapsearch we ran on the consumer side.
Best regards, Ioan Indreias
### Provider SID: 001 Local SID: 015
test199.entryCSN (20100910085049.959625) less than 001.contextCSN (20100910085055.650259) less than johndoe.entryCSN (20100910092955.693053)
--- CONSUMER --- # extended LDIF # # LDAPv3 # base <o=hashed> with scope subtree # filter: (objectclass=*) # requesting: + # # hashed dn: o=hashed structuralObjectClass: organization entryUUID: 541ccdcb-8d82-4160-9fc7-1892765cbdac creatorsName: cn=superuser,o=hashed createTimestamp: 20100714124242Z entryCSN: 20100714124242.721169Z#000000#015#000000 modifiersName: cn=superuser,o=hashed modifyTimestamp: 20100714124242Z contextCSN: 20100909124709.457916Z#000000#015#000000 contextCSN: 20100910085055.650259Z#000000#001#000000 entryDN: o=hashed subschemaSubentry: cn=Subschema auditContext: cn=auditlog
# emailservice, hashed dn: ou=emailservice,o=hashed structuralObjectClass: organizationalUnit entryUUID: b8950978-248b-4b77-bcf9-6455f9c273c0 creatorsName: cn=superuser,o=hashed createTimestamp: 20100714124242Z modifiersName: cn=superuser,o=hashed modifyTimestamp: 20100714124242Z entryCSN: 20100730103008.669976Z#000000#001#000000 entryDN: ou=emailservice,o=hashed subschemaSubentry: cn=Subschema hasSubordinates: TRUE
# org, emailservice, hashed dn: dc=org,ou=emailservice,o=hashed structuralObjectClass: domain entryUUID: 32e805e8-669c-471e-b9d3-1d22938bdce3 creatorsName: cn=superuser,o=hashed createTimestamp: 20100714124242Z modifiersName: cn=superuser,o=hashed modifyTimestamp: 20100714124242Z entryCSN: 20100730103013.472630Z#000000#001#000000 entryDN: dc=org,ou=emailservice,o=hashed subschemaSubentry: cn=Subschema hasSubordinates: TRUE
# hashed, org, emailservice, hashed dn: dc=hashed,dc=org,ou=emailservice,o=hashed structuralObjectClass: domain entryUUID: 3bd366ba-5213-4bf2-8935-3ddbb26bf220 creatorsName: cn=superuser,o=hashed createTimestamp: 20100714124242Z modifiersName: cn=superuser,o=hashed modifyTimestamp: 20100714124242Z entryCSN: 20100730103018.690150Z#000000#001#000000 entryDN: dc=hashed,dc=org,ou=emailservice,o=hashed subschemaSubentry: cn=Subschema hasSubordinates: TRUE
# users, hashed, org, emailservice, hashed dn: ou=users,dc=hashed,dc=org,ou=emailservice,o=hashed structuralObjectClass: organizationalUnit entryUUID: 87390f34-5698-4653-a700-5045843813db creatorsName: cn=superuser,o=hashed createTimestamp: 20100714124242Z modifiersName: cn=superuser,o=hashed modifyTimestamp: 20100714124242Z entryCSN: 20100802144805.677791Z#000000#001#000000 entryDN: ou=users,dc=hashed,dc=org,ou=emailservice,o=hashed subschemaSubentry: cn=Subschema hasSubordinates: TRUE ... # test199, users, hashed, org, emailservice, hashed dn: uid=test199,ou=users,dc=hashed,dc=org,ou=emailservice,o=hashed structuralObjectClass: inetOrgPerson entryCSN: 20100910085049.959625Z#000000#001#000000 entryUUID: 416a2a78-5104-102f-8592-6d824ed7581a creatorsName: cn=superuser,o=hashed createTimestamp: 20100910085049Z modifiersName: cn=superuser,o=hashed modifyTimestamp: 20100910085049Z entryDN: uid=test199,ou=users,dc=hashed,dc=org,ou=emailservice,o=hashed subschemaSubentry: cn=Subschema hasSubordinates: FALSE
# johndoe, users, hashed, org, emailservice, hashed dn: uid=johndoe,ou=users,dc=hashed,dc=org,ou=emailservice,o=hashed structuralObjectClass: inetOrgPerson entryCSN: 20100910092955.693053Z#000000#001#000000 entryUUID: b794ae8a-5109-102f-8593-6d824ed7581a creatorsName: cn=superuser,o=hashed createTimestamp: 20100910092955Z modifiersName: cn=superuser,o=hashed modifyTimestamp: 20100910092955Z entryDN: uid=johndoe,ou=users,dc=hashed,dc=org,ou=emailservice,o=hashed subschemaSubentry: cn=Subschema hasSubordinates: FALSE
# search result search: 2 result: 0 Success # numResponses: 197 # numEntries: 196 ###
On 14/09/2010 12:30, Ioan Indreias wrote:
Hello,
We are running openldap 2.4.21 configured as a consumer, using a refreshAndPersist replication:
syncrepl rid=202 provider=ldaps://ldap.hashed.net type=refreshAndPersist interval=00:00:05:00 retry="10 3 60 +" searchbase="ou=emailservice,o=hashed" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldap-hashed-service,o=hashed" credentials=xx-xx-xx tls_reqcert=never
In order to check if the replication works OK we compare (periodically) the provider's contextCSN versus the consumer's contextCSN
Most of the time both are in sync but from time to time we see that these parameters are different (provider contextCSN> consumer contextCSN) and we thought that the consumer was not "in sync" with the provider.
Continuing our investigations we found that in the consumer there are objects with entryCSN greater than the consumer contextCSN.
Is this a normal situation or a bug?
What method do you use to consult the contextCSN?
As stated in slapo-syncrepl(5), "the contextCSN is only updated in memory". So if you're using slapcat to check the value, this is pretty normal. See the slapo-checkpoint option to control the frequency it is written to disk.
If you're checking via LDAP, then this is a whole different matter...
Hello Jonathan,
For obtaining the contextCSN we are using ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file -s base '(contextCSN=*)' contextCSN
For the extract I have added to my original post we have used ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file +
So I conclude we are using the in-memory information.
Best regards, Ioan
On Tue, Sep 14, 2010 at 3:11 PM, Jonathan Clarke jonathan@phillipoux.net wrote:
On 14/09/2010 12:30, Ioan Indreias wrote:
Hello,
We are running openldap 2.4.21 configured as a consumer, using a refreshAndPersist replication:
syncrepl rid=202 provider=ldaps://ldap.hashed.net type=refreshAndPersist interval=00:00:05:00 retry="10 3 60 +" searchbase="ou=emailservice,o=hashed" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldap-hashed-service,o=hashed" credentials=xx-xx-xx tls_reqcert=never
In order to check if the replication works OK we compare (periodically) the provider's contextCSN versus the consumer's contextCSN
Most of the time both are in sync but from time to time we see that these parameters are different (provider contextCSN> consumer contextCSN) and we thought that the consumer was not "in sync" with the provider.
Continuing our investigations we found that in the consumer there are objects with entryCSN greater than the consumer contextCSN.
Is this a normal situation or a bug?
What method do you use to consult the contextCSN?
As stated in slapo-syncrepl(5), "the contextCSN is only updated in memory". So if you're using slapcat to check the value, this is pretty normal. See the slapo-checkpoint option to control the frequency it is written to disk.
If you're checking via LDAP, then this is a whole different matter...
--
Jonathan Clarke - jonathan@phillipoux.net
Ldap Synchronization Connector (LSC) - http://lsc-project.org
well - I understand that nobody meet such a case, with entryCSN grater than the contextCSN.
we are planning to compile and test the latest stable version, hoping that included syncrepl fixes will solve our problem.
Best regards, Ioan
On Tue, Sep 14, 2010 at 4:35 PM, Ioan Indreias indreias@gmail.com wrote:
Hello Jonathan,
For obtaining the contextCSN we are using ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file -s base '(contextCSN=*)' contextCSN
For the extract I have added to my original post we have used ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file +
So I conclude we are using the in-memory information.
Best regards, Ioan
On Tue, Sep 14, 2010 at 3:11 PM, Jonathan Clarke jonathan@phillipoux.net wrote:
On 14/09/2010 12:30, Ioan Indreias wrote:
Hello,
We are running openldap 2.4.21 configured as a consumer, using a refreshAndPersist replication:
syncrepl rid=202 provider=ldaps://ldap.hashed.net type=refreshAndPersist interval=00:00:05:00 retry="10 3 60 +" searchbase="ou=emailservice,o=hashed" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldap-hashed-service,o=hashed" credentials=xx-xx-xx tls_reqcert=never
In order to check if the replication works OK we compare (periodically) the provider's contextCSN versus the consumer's contextCSN
Most of the time both are in sync but from time to time we see that these parameters are different (provider contextCSN> consumer contextCSN) and we thought that the consumer was not "in sync" with the provider.
Continuing our investigations we found that in the consumer there are objects with entryCSN greater than the consumer contextCSN.
Is this a normal situation or a bug?
What method do you use to consult the contextCSN?
As stated in slapo-syncrepl(5), "the contextCSN is only updated in memory". So if you're using slapcat to check the value, this is pretty normal. See the slapo-checkpoint option to control the frequency it is written to disk.
If you're checking via LDAP, then this is a whole different matter...
--
Jonathan Clarke - jonathan@phillipoux.net
Ldap Synchronization Connector (LSC) - http://lsc-project.org
Ioan Indreias wrote:
well - I understand that nobody meet such a case, with entryCSN grater than the contextCSN.
There are plenty of situations where an entryCSN will be greater than the contextCSN. The most obvious is during a refresh phase, since entries are not sent in any particular order. In that case, the contextCSN is not updated until the refresh is completed. The fact that you see this occurring does not indicate that there is any bug or problem.
we are planning to compile and test the latest stable version, hoping that included syncrepl fixes will solve our problem.
Best regards, Ioan
On Tue, Sep 14, 2010 at 4:35 PM, Ioan Indreiasindreias@gmail.com wrote:
Hello Jonathan,
For obtaining the contextCSN we are using ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file -s base '(contextCSN=*)' contextCSN
For the extract I have added to my original post we have used ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file +
So I conclude we are using the in-memory information.
Best regards, Ioan
On Tue, Sep 14, 2010 at 3:11 PM, Jonathan Clarke jonathan@phillipoux.net wrote:
On 14/09/2010 12:30, Ioan Indreias wrote:
Hello,
We are running openldap 2.4.21 configured as a consumer, using a refreshAndPersist replication:
syncrepl rid=202 provider=ldaps://ldap.hashed.net type=refreshAndPersist interval=00:00:05:00 retry="10 3 60 +" searchbase="ou=emailservice,o=hashed" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldap-hashed-service,o=hashed" credentials=xx-xx-xx tls_reqcert=never
In order to check if the replication works OK we compare (periodically) the provider's contextCSN versus the consumer's contextCSN
Most of the time both are in sync but from time to time we see that these parameters are different (provider contextCSN> consumer contextCSN) and we thought that the consumer was not "in sync" with the provider.
Continuing our investigations we found that in the consumer there are objects with entryCSN greater than the consumer contextCSN.
Is this a normal situation or a bug?
What method do you use to consult the contextCSN?
As stated in slapo-syncrepl(5), "the contextCSN is only updated in memory". So if you're using slapcat to check the value, this is pretty normal. See the slapo-checkpoint option to control the frequency it is written to disk.
If you're checking via LDAP, then this is a whole different matter...
--
Jonathan Clarke - jonathan@phillipoux.net
Ldap Synchronization Connector (LSC) - http://lsc-project.org
Hello Howard,
I understand your point but I am 100% sure that we were not in such a situation (refresh phase) as the "problem" was there for more minutes (sometimes more hours).
My personal feeling is that the contextCSN is not updated correctly and this is why I was thinking it is a bug.
Do you know other reasons why an entryCSN from the LDAP could be greater than the contextCSN?
Best regards, Ioan
On Wed, Sep 15, 2010 at 12:36 PM, Howard Chu hyc@symas.com wrote:
Ioan Indreias wrote:
well - I understand that nobody meet such a case, with entryCSN grater than the contextCSN.
There are plenty of situations where an entryCSN will be greater than the contextCSN. The most obvious is during a refresh phase, since entries are not sent in any particular order. In that case, the contextCSN is not updated until the refresh is completed. The fact that you see this occurring does not indicate that there is any bug or problem.
we are planning to compile and test the latest stable version, hoping that included syncrepl fixes will solve our problem.
Best regards, Ioan
On Tue, Sep 14, 2010 at 4:35 PM, Ioan Indreiasindreias@gmail.com wrote:
Hello Jonathan,
For obtaining the contextCSN we are using ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file -s base '(contextCSN=*)' contextCSN
For the extract I have added to my original post we have used ldapsearch like:
ldapsearch -H ... -x -D ... -b o=hashed -y pass_file +
So I conclude we are using the in-memory information.
Best regards, Ioan
On Tue, Sep 14, 2010 at 3:11 PM, Jonathan Clarke jonathan@phillipoux.net wrote:
On 14/09/2010 12:30, Ioan Indreias wrote:
Hello,
We are running openldap 2.4.21 configured as a consumer, using a refreshAndPersist replication:
syncrepl rid=202 provider=ldaps://ldap.hashed.net type=refreshAndPersist interval=00:00:05:00 retry="10 3 60 +" searchbase="ou=emailservice,o=hashed" filter="(objectClass=*)" scope=sub attrs="*" schemachecking=off bindmethod=simple binddn="cn=ldap-hashed-service,o=hashed" credentials=xx-xx-xx tls_reqcert=never
In order to check if the replication works OK we compare (periodically) the provider's contextCSN versus the consumer's contextCSN
Most of the time both are in sync but from time to time we see that these parameters are different (provider contextCSN> consumer contextCSN) and we thought that the consumer was not "in sync" with the provider.
Continuing our investigations we found that in the consumer there are objects with entryCSN greater than the consumer contextCSN.
Is this a normal situation or a bug?
What method do you use to consult the contextCSN?
As stated in slapo-syncrepl(5), "the contextCSN is only updated in memory". So if you're using slapcat to check the value, this is pretty normal. See the slapo-checkpoint option to control the frequency it is written to disk.
If you're checking via LDAP, then this is a whole different matter...
--
Jonathan Clarke - jonathan@phillipoux.net
Ldap Synchronization Connector (LSC) - http://lsc-project.org
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
openldap-technical@openldap.org
-
Howard Chu -
Ioan Indreias -
Jonathan Clarke