Hi All,
I am testing delta-syncrepl using OpenLDAP 2.3.43.7(actually as ZimbraLDAP on ZCS 5.0.16).
It spends a lot of time to complete replication when adding 20,000 or 200,000 objects(accounts) to the master at a time.
[Time for replication] User time(updates to master) replica1 replica2 ------------------------------------------- 20,000 0:19:03 0:58:51 1:19:29 200,000 2:38:03 4:33:39 13:33:38
[Setting value in LDAP] *checkpoint = 10240 *Thread numbers = 8 *Security = 0(zimbra_require_interprocess_security)
*The following operation is actually executed by each user in a loop.
1.Add account 2.Add settings(filters,contacts,signatures) for an account 3.Update the setting of the created account
There seems no performance problem with writing accesslog to the database, thus, updates to the master server was finished successfully in reasonable time.
Also, it seems that there is no bottleneck such as resource of the servers, and we still have extra spaces on CPU, memory and network. I've attached the servers spec.
I assume there will be a problem with the process of pushing data from LDAP master to multiple replicas, but is this process serialized ?
How can we reduce the replication time on replica servers?
I think updates could not be delayed on the replicas themselves because only the secondary replica took a lot of time to update data. And we don't have plan to execute export/import method.
Here are the settings of the provider and the replicas:
######################################################## # BDB database definitions for provider # excluding Zimbra settings ######################################################## database bdb suffix "" rootdn "cn=config" # number of entries to keep in memory cachesize 10000 # number of search results to keep in memory idlcachesize 10000 # check point whenever 64k data bytes written or # 5 minutes has elapsed whichever occurs first checkpoint 64 5 # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory "/opt/zimbra/openldap-data" # recommended for replication index entryUUID eq index entryCSN eq sizelimit unlimited timelimit unlimited overlay syncprov syncprov-checkpoint 20 10 syncprov-sessionlog 500 include /opt/zimbra/conf/master-accesslog-overlay.conf
################################################## # BDB database definitions for consumer ##################################################
database bdb suffix "" rootdn "cn=config" # number of entries to keep in memory cachesize 10000 # number of search results to keep in memory idlcachesize 10000 # check point whenever 64k data bytes written or # 5 minutes has elapsed whichever occurs first checkpoint 64 5 # The database directory MUST exist prior to running slapd AND # should only be accessible by the slapd and slap tools. # Mode 700 recommended. directory "/opt/zimbra/openldap-data" index uid pres,eq # white pages index mail pres,eq,sub index cn pres,eq,sub index displayName pres,eq,sub index sn pres,eq,sub index gn pres,eq,sub # recommended for replication index entryUUID eq index entryCSN eq sizelimit unlimited timelimit unlimited ###include /opt/zimbra/conf/master-accesslog-overlay.conf syncrepl rid=100 provider=ldap://ldapm.XXX1.XXX-test.ne.jp:389 retry="60 +" type=refreshAndPersist schemachecking=off searchbase="" bindmethod=simple binddn=uid=zmreplica,cn=admins,cn=zimbra credentials=@@ldap_replication_password@@ starttls=critical logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" syncdata="accesslog"
updateref ldap://ldapm.ngm1.XXX-test.ne.jp:389 overlay syncprov
Please let us know if you need more information.
Best Regards, Soichiro Miki
===== Soichiro Miki Hitachi Software Engineering Co,Ltd. s-miki@hitachisoft.jp