I will be highly thankful to you, if you can help me out in below issue.
We have three servers on rhel 6.3, 16 vcpu and 32G RAM , openldap 2.4.33 with mdb db of 200G
We are facing replication issue on our servers, otherwise servers are good with login and user registration from website.(doing on one server only as of now)
We imported data on one server mmam01 and copied it to other 2 servers. After some time we got big diff b/w db size of mmam01 and other two servers. Then exported data and restored it on other two servers.
i tried adding a user and i got replicated to other two servers, but after some time, new users stop getting replicated to other servers.
initial replication status after some 30 min
Even when i tried with blank db
it initally started and then stopped.
i got errors like
dn_callback : entries have identical CSN
syncrepl_entry: rid=111 entry unchanged, ignored
Sat Jan 12 12:40:41 EST 2013
DR-SJ contextCSN: 20130101132757.303803Z#000000#000#000000 contextCSN: 20130111144013.926562Z#000000#001#000000 contextCSN: 20130112174023.266193Z#000000#002#000000 DC-mmam01 contextCSN: 20130101132757.303803Z#000000#000#000000 contextCSN: 20130112174006.314483Z#000000#001#000000 contextCSN: 20130112174023.266193Z#000000#002#000000 DC-mmam04 contextCSN: 20130101132757.303803Z#000000#000#000000 contextCSN: 20130111144013.926562Z#000000#001#000000 contextCSN: 20130112174023.266193Z#000000#002#000000
After 2 hours
DR-SJ contextCSN: 20130101132757.303803Z#000000#000#000000 contextCSN: 20130111144013.926562Z#000000#001#000000 contextCSN: 20130112174023.266193Z#000000#002#000000 contextCSN: 20130112175710.938307Z#000000#003#000000 DC-mmam01 contextCSN: 20130101132757.303803Z#000000#000#000000 contextCSN: 20130112193219.242546Z#000000#001#000000 contextCSN: 20130112174023.266193Z#000000#002#000000 contextCSN: 20130112175710.938307Z#000000#003#000000 DC-mmam04 contextCSN: 20130101132757.303803Z#000000#000#000000 contextCSN: 20130111144013.926562Z#000000#001#000000 contextCSN: 20130112174023.266193Z#000000#002#000000 contextCSN: 20130112175710.938307Z#000000#003#000000
My ldap.conf file is(same on all server) , we have host-ip mapping in /etc/hosts file
BASE dc=example, dc=com URI ldap://mmam01.com ldaps://mmam01.com ldap://mmam04.com ldaps:// mmam04.com ldap://sjam01.com ldaps://sjam01.com TLS_REQCERT demand TLS_CACERT /etc/openldap/cacerts/cacert.pem
slapd.conf file(from mmam01)
include /etc/openldap/schema/core.schema include /etc/openldap/schema/cosine.schema include /etc/openldap/schema/nis.schema include /etc/openldap/schema/inetorgperson.schema include /etc/openldap/schema/openldap.schema include /etc/openldap/schema/dyngroup.schema include /etc/openldap/schema/ppolicy.schema include /etc/openldap/schema2/channelIdentifier.schema include /etc/openldap/schema2/platform.schema include /etc/openldap/schema2/extendedProfileKey.schema include /etc/openldap/schema2/extendedProfileValue.schema include /etc/openldap/schema2/behaviorKey.schema include /etc/openldap/schema2/behaviorValue.schema include /etc/openldap/schema2/questionAnswer.schema include /etc/openldap/schema2/extendedTop.schema include /etc/openldap/schema2/counter.schema serverid 1 TLSCipherSuite HIGH:MEDIUM:+SSLv3 TLSCACertificateFile /etc/openldap/cacerts/cacert.pem TLSCertificateFile /etc/openldap/cacerts/mmam01.crt TLSCertificateKeyFile /etc/openldap/cacerts/mmam01.key TLSVerifyClient never pidfile /var/symas/run/slapd.pid argsfile /var/symas/run/slapd.args loglevel sync stats idletimeout 30 writetimeout 30 modulepath /etc/openldap/lib64/openldap moduleload back_mdb.la moduleload ppolicy.la moduleload unique.la moduleload syncprov.la database mdb suffix "dc=example,dc=com" directory /openldap/var/data access to attrs=userPassword by self write by anonymous auth by * break
access to * by group/groupOfUniqueNames/uniqueMember.exact="cn=PWrite,ou=bGroup,dc=example,dc=com" manage by group/groupOfUniqueNames/uniqueMember.exact="cn=PRead,ou=bGroup,dc=example,dc=com" read by * break access to * by self write by anonymous auth by * read rootdn "cn=Manager,dc=example,dc=com" rootpw {SSHA}dXDESQeFjSoa/A1HfJ2TAzYf4DrSYWY index mail,uid,postalCode,smail,channelType,channelValue,answer,behavName,objectclass,type eq index givenName,sn,city,cn,extName sub index displayName approx index entryCSN,entryUUID eq checkpoint 128 15 maxsize 274877906944 syncrepl rid=111 provider=ldap://sjam01.com binddn="cn=Manager,dc=example,dc=com" bindmethod=simple credentials=0m2013 tls_cacert=/etc/openldap/cacerts/cacert.pem searchbase="dc=example,dc=com" type=refreshAndPersist retry="5 5 60 +" network-timeout=10 timeout=10 syncrepl rid=222 provider=ldap://mmam04.com binddn="cn=Manager,dc=example,dc=com" bindmethod=simple credentials=0m2013 tls_cacert=/etc/openldap/cacerts/cacert.pem searchbase="dc=example,dc=com" type=refreshAndPersist retry="5 5 60 +" network-timeout=10 timeout=10 overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100 mirrormode true overlay unique unique_attributes mail overlay ppolicy ppolicy_default "cn=default,ou=pwdPolicy,dc=example,dc=com" ppolicy_use_lockout
logs
DR-sj
Jan 12 14:51:28 sjprodam01 slapd[25165]: do_syncrep2: rid=111 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Jan 12 14:51:28 sjprodam01 slapd[25165]: do_syncrep2: rid=111 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Jan 12 14:51:28 sjprodam01 slapd[25165]: do_syncrep2: rid=111 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Jan 12 14:51:28 sjprodam01 slapd[25165]: do_syncrep2: rid=111 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Jan 12 14:51:28 sjprodam01 slapd[25165]: do_syncrep2: rid=111 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
mmam04
Jan 12 14:53:24 mmprodam04 slapd[14108]: do_syncrep2: rid=222 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Jan 12 14:53:24 mmprodam04 slapd[14108]: do_syncrep2: rid=222 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Jan 12 14:53:25 mmprodam04 slapd[14108]: do_syncrep2: rid=222 LDAP_RES_INTERMEDIATE - SYNC_ID_SET Jan 12 14:53:25 mmprodam04 slapd[14108]: do_syncrep2: rid=222 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
mmam01 Users are add on this server
Jan 12 14:53:26 mmprodam01 slapd[24380]: slap_queue_csn: queing 0x7f55dd4fa120 20130112195326.941804Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=291 RESULT tag=105 err=0 text= Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_graduate_commit_csn: removing 0x7f55d010ee30 20130112195326.941804Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=292 MOD dn="extName=PIT,cn=entitlements,cn=extendedProfile,uid=6a9ddf85-1072-48b4-9f09-10f032c8f05e,ou=endUsers,dc=example,dc=com" Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=292 MOD attr=extValue Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_queue_csn: queing 0x7f55deffc210 20130112195327.100182Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=292 RESULT tag=103 err=0 text= Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_graduate_commit_csn: removing 0x7f55d8392770 20130112195327.100182Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=293 MOD dn="extName=RVW,cn=entitlements,cn=extendedProfile,uid=6a9ddf85-1072-48b4-9f09-10f032c8f05e,ou=endUsers,dc=example,dc=com" Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=293 MOD attr=extValue Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_queue_csn: queing 0x7f560a22e210 20130112195327.103686Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=293 RESULT tag=103 err=0 text= Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_graduate_commit_csn: removing 0x7f55fc6f35e0 20130112195327.103686Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=294 MOD dn="extName=ICA,cn=entitlements,cn=extendedProfile,uid=6a9ddf85-1072-48b4-9f09-10f032c8f05e,ou=endUsers,dc=example,dc=com" Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=294 MOD attr=extValue Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_queue_csn: queing 0x7f55f3ffd210 20130112195327.107815Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=294 MOD attr=extValue Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_queue_csn: queing 0x7f55f3ffd210 20130112195327.107815Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=294 RESULT tag=103 err=0 text= Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_graduate_commit_csn: removing 0x7f55e87ae550 20130112195327.107815Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=295 MOD dn="extName=RVP,cn=entitlements,cn=extendedProfile,uid=6a9ddf85-1072-48b4-9f09-10f032c8f05e,ou=endUsers,dc=example,dc=com" Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=295 MOD attr=extValue Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_queue_csn: queing 0x7f55f0cf8210 20130112195327.112994Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=295 RESULT tag=103 err=0 text= Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_graduate_commit_csn: removing 0x7f55e4618780 20130112195327.112994Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=296 MOD dn="extName=RAD,cn=entitlements,cn=extendedProfile,uid=6a9ddf85-1072-48b4-9f09-10f032c8f05e,ou=endUsers,dc=example,dc=com" Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=296 MOD attr=extValue Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_queue_csn: queing 0x7f55dd4f9210 20130112195327.117321Z#000000#001#000000 Jan 12 14:53:27 mmprodam01 slapd[24380]: conn=8516 op=296 RESULT tag=103 err=0 text= Jan 12 14:53:27 mmprodam01 slapd[24380]: slap_graduate_commit_csn: removing 0x7f55d0001490 20130112195327.117321Z#000000#001#000000
I have tried adding users and changing passwords one by one, it doesn't work. I can search old entries from one server to other server.