Hello openldap-technical,
new on the list, Arjan Filius is my Name.
Having setup openldap 2.4.21, with one master, and six slaves/consumers in delta syncrepl configuration and testing an upgrade from an older openldap version.
exporting (slapcat > export-file), importing (slapadd -l `export-file`) on the (empty/pristine) master, and attaching empty/pristine slaves works just fine except for taking more than one hour to complete.
To speed up migration i decided to import the export on master _and_ 6 slaves which finishes under 15 minutes. and now my problem starts. Starting the master and slaves results in a situation which doesn't "go away" and causes load on master, unneeded network traffic, all slaves/consumers complain with: 2010-07-06 07:00:58.797739500 do_syncrep2: rid=000 (4096) Content Sync Refresh Required 2010-07-06 07:00:58.801662500 do_syncrep2: rid=000 (4096) Content Sync Refresh Required 2010-07-06 07:00:58.804550500 do_syncrep2: rid=000 (4096) Content Sync Refresh Required at a fast rate. and the master complains about stale cookies (all slave connections ): 2010-07-06 07:08:44.470077500 conn=1003 op=40499239 SRCH attr=reqDN reqType reqMod reqNewRDN reqDeleteOldRDN reqNewSuperior entryCSN 2010-07-06 07:08:44.470079500 conn=1003 op=40499239 SEARCH RESULT tag=101 err=4096 nentries=0 text=sync cookie is stale 2010-07-06 07:08:44.470760500 conn=1003 op=40499240 SRCH base="dc=com" scope=2 deref=0 filter="(objectClass=*)" 2010-07-06 07:08:44.470762500 conn=1003 op=40499240 SRCH attr=* + 2010-07-06 07:08:44.470764500 conn=1003 op=40499240 SEARCH RESULT tag=101 err=0 nentries=0 text= 2010-07-06 07:08:44.471582500 conn=1003 op=40499241 SRCH base="cn=accesslog" scope=2 deref=0 filter="(&(objectClass=auditWriteObject)(reqResult=0))" 2010-07-06 07:08:44.471585500 conn=1003 op=40499241 SRCH attr=reqDN reqType reqMod reqNewRDN reqDeleteOldRDN reqNewSuperior entryCSN 2010-07-06 07:08:44.471587500 conn=1003 op=40499241 SEARCH RESULT tag=101 err=4096 nentries=0 text=sync cookie is stale 2010-07-06 07:08:44.472298500 conn=1003 op=40499242 SRCH base="dc=com" scope=2 deref=0 filter="(objectClass=*)" 2010-07-06 07:08:44.472301500 conn=1003 op=40499242 SRCH attr=* + 2010-07-06 07:08:44.472302500 conn=1003 op=40499242 SEARCH RESULT tag=101 err=0 nentries=0 text= 2010-07-06 07:08:44.473006500 conn=1003 op=40499243 SRCH base="cn=accesslog" scope=2 deref=0 filter="(&(objectClass=auditWriteObject)(reqResult=0))" 2010-07-06 07:08:44.473008500 conn=1003 op=40499243 SRCH attr=reqDN reqType reqMod reqNewRDN reqDeleteOldRDN reqNewSuperior entryCSN 2010-07-06 07:08:44.473010500 conn=1003 op=40499243 SEARCH RESULT tag=101 err=4096 nentries=0 text=sync cookie is stale
The slave has a rid config of: syncrepl rid=4 provider=ldap://X.X.X.X:389 bindmethod=simple binddn="cn=XXXXX,dc=XXX,dc=com" credentials=XXXX searchbase="dc=com" logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" schemachecking=on type=refreshAndPersist retry="1 10 2 +" sizelimit=unlimited timelimit=unlimited syncdata=accesslog
Has anyone an idea what is/might be going on, and how to fix/prevent this? or how to migrate/replicate in a fast way?
Regards, and thanks in advance.
On Tuesday, 6 July 2010 07:08:05 Arjan Filius wrote:
Hello openldap-technical,
new on the list, Arjan Filius is my Name.
Having setup openldap 2.4.21, with one master, and six slaves/consumers in delta syncrepl configuration and testing an upgrade from an older openldap version.
Please specify the version you are upgrading from, it *is* relevant.
exporting (slapcat > export-file), importing (slapadd -l `export-file`) on the (empty/pristine) master, and attaching empty/pristine slaves works just fine except for taking more than one hour to complete.
Well, maybe you should consider appropriate tuning/changes to your import process to speed things up, rather than risk data integrity. You don't specify how large your database is, or any tuning etc., or other slapadd flags, so it is difficult to know if 1 hour is good or bad.
E.g., using the -q flag to slapadd can speed things up significantly, setting 'tool-threads' in slapd.conf appropriately can too, and you should have database tuning (e.g. DB_CONFIG) in place.
[...]
Starting with different data on providers and consumers is sure to result in broken replication.
Has anyone an idea what is/might be going on, and how to fix/prevent this? or how to migrate/replicate in a fast way?
See above, but you provide no detail on what you have done to speed up your import, and this seems the original problem.
Regards, Buchan
Hello Buchan, ea.
On Tue, 6 Jul 2010, Buchan Milne wrote:
On Tuesday, 6 July 2010 07:08:05 Arjan Filius wrote:
Hello openldap-technical,
new on the list, Arjan Filius is my Name.
Having setup openldap 2.4.21, with one master, and six slaves/consumers in delta syncrepl configuration and testing an upgrade from an older openldap version.
Please specify the version you are upgrading from, it *is* relevant.
Excuse, didn't came up it's 2.3.38 for 32bits i386 the older ldap version is on an other machine (and is using slurpd for replication) , and upgrade is by doing an export on the old (slapcat > export-file.ldif) and on the new machien (2.4.21) : slapadd -l /tmp/export-file.ldif -F ./slapd.d/
exporting (slapcat > export-file), importing (slapadd -l `export-file`) on the (empty/pristine) master, and attaching empty/pristine slaves works just fine except for taking more than one hour to complete.
Well, maybe you should consider appropriate tuning/changes to your import process to speed things up, rather than risk data integrity. You don't specify how large your database is, or any tuning etc., or other slapadd flags, so it is difficult to know if 1 hour is good or bad.
slapadd (on master) can be done in 12 minutes , with a tuned config, major ingredients: #checkpoint 10 3 checkpoint 2000 60 #dbnosync dbnosync # syncprov-checkpoint 100 10 syncprov-checkpoint 1000 100
'#' is the regular value without the special import tunables.
after import start master just with regular parameters (without nodbsync and checkpointing more strict 100 10)
machines all have 8G RAM, 1 CPU.
I'm also not use what is good or bad in terms of performance. Just looking for the quickest migration path, but not interested by doing exotic "tricks" . I thought just importing the same data (slapadd) on master and all slaves in paralel is not exotic, and would shorten the migration time.
E.g., using the -q flag to slapadd can speed things up significantly, setting 'tool-threads' in slapd.conf appropriately can too, and you should have database tuning (e.g. DB_CONFIG) in place.
Never looked at the -g option, but i will look at that Thanks.
the DB_CONFIG i use: # grep -v ^$ DB_CONFIG |grep -v ^# set_cachesize 2 0 1 set_flags DB_LOG_AUTOREMOVE set_lg_regionmax 1048576 set_lg_max 10485760 set_lg_bsize 2097152 set_lg_dir /var/openldap-logs set_lk_max_locks 20000 set_lk_max_objects 30000 set_lk_max_lockers 30000
The tool-threads is also something i never looked into, not having it in mij config, is uses the default now. Having also a single core cpu, i'm not sure settings above 1 wil speed things up.
But, the import of the created export isn't realy a problem, it's the replication to the slaves which takes some time, and it's a lot faster "just" to import the same import as i did on the master (only if possible and works fine)
[...]
Starting with different data on providers and consumers is sure to result in broken replication.
I used exactly the same import file master ans slaves (source 2.3.39) , the imported ldif is about 580MB in the imported ldif is no contextCSN, so i think (re)generating it may lead to the situation (cookie issue) encountered.
Has anyone an idea what is/might be going on, and how to fix/prevent this? or how to migrate/replicate in a fast way?
See above, but you provide no detail on what you have done to speed up your import, and this seems the original problem.
It's the replication with the slaves which takes a much longer time dan just an import. preventing that by doing a import of the same file, leads to the cooky issue. But from what i understand from you that might not a good idea?
Thanks for your reaction
Regards,
Regards, Buchan
On Wednesday, 7 July 2010 11:25:26 Arjan Filius wrote:
Hello Buchan, ea.
On Tue, 6 Jul 2010, Buchan Milne wrote:
On Tuesday, 6 July 2010 07:08:05 Arjan Filius wrote:
Hello openldap-technical,
new on the list, Arjan Filius is my Name.
Having setup openldap 2.4.21, with one master, and six slaves/consumers in delta syncrepl configuration and testing an upgrade from an older openldap version.
Please specify the version you are upgrading from, it *is* relevant.
Excuse, didn't came up it's 2.3.38 for 32bits i386 the older ldap version is on an other machine (and is using slurpd for replication) , and upgrade is by doing an export on the old (slapcat > export-file.ldif) and on the new machien (2.4.21) : slapadd -l /tmp/export-file.ldif -F ./slapd.d/
exporting (slapcat > export-file), importing (slapadd -l `export-file`) on the (empty/pristine) master, and attaching empty/pristine slaves works just fine except for taking more than one hour to complete.
Well, maybe you should consider appropriate tuning/changes to your import process to speed things up, rather than risk data integrity. You don't specify how large your database is, or any tuning etc., or other slapadd flags, so it is difficult to know if 1 hour is good or bad.
slapadd (on master) can be done in 12 minutes , with a tuned config, major ingredients: #checkpoint 10 3 checkpoint 2000 60 #dbnosync dbnosync # syncprov-checkpoint 100 10 syncprov-checkpoint 1000 100
'#' is the regular value without the special import tunables.
after import start master just with regular parameters (without nodbsync and checkpointing more strict 100 10)
The -q flag can be used in place of dbnosync.
machines all have 8G RAM, 1 CPU.
I'm also not use what is good or bad in terms of performance. Just looking for the quickest migration path, but not interested by doing exotic "tricks" . I thought just importing the same data (slapadd) on master and all slaves in paralel is not exotic, and would shorten the migration time.
Well, you weren't explicit in how you were loading the slaves.
You should probably do as follows:
1)slapcat on old master (let's say to old.ldif) 2)slapadd old.ldif on new master 3)slapcat on new master (let's say to new.ldif) 4)slapadd new.ldif on consumers
Starting with different data on providers and consumers is sure to result in broken replication.
I used exactly the same import file master ans slaves (source 2.3.39) , the imported ldif is about 580MB in the imported ldif is no contextCSN, so i think (re)generating it may lead to the situation (cookie issue) encountered.
Only contextCSN difference would not lead to replication failures, but entryCSN differences would ...
If the entryCSN after import on the new master is different to the ldif you imported (which is likely when doing migration over major versions), then your data isn't consistent, even if you are using the same original ldif ...
Regards, Buchan
Buchen,
Many thanks you got me on the right track.
main issue was the importing the same data of the old slapd on all new master + slaves. As you mentioned, i should do another export on the new master (which i didn't), and import that one on all new slaves. That export _does_ contain the contextCSN, solves cookie issue, and is in sync on all slaves.
That takes me, included exprting old ldap, scping data, importing on master, exporting on master, scping new export to slaves, importing new export on slaves starting slapd (1 master + 6 slaves) 18 minutes for the complete path.
Also skipped the special import tunables in the conf file, which i have no need for anymore.
This seems perfectly clean, and fast way to do so.
Many Thanks again!
Regards,
Arjan Filius
On Tue, 6 Jul 2010, Buchan Milne wrote:
On Tuesday, 6 July 2010 07:08:05 Arjan Filius wrote:
Hello openldap-technical,
new on the list, Arjan Filius is my Name.
Having setup openldap 2.4.21, with one master, and six slaves/consumers in delta syncrepl configuration and testing an upgrade from an older openldap version.
Please specify the version you are upgrading from, it *is* relevant.
exporting (slapcat > export-file), importing (slapadd -l `export-file`) on the (empty/pristine) master, and attaching empty/pristine slaves works just fine except for taking more than one hour to complete.
Well, maybe you should consider appropriate tuning/changes to your import process to speed things up, rather than risk data integrity. You don't specify how large your database is, or any tuning etc., or other slapadd flags, so it is difficult to know if 1 hour is good or bad.
E.g., using the -q flag to slapadd can speed things up significantly, setting 'tool-threads' in slapd.conf appropriately can too, and you should have database tuning (e.g. DB_CONFIG) in place.
[...]
Starting with different data on providers and consumers is sure to result in broken replication.
Has anyone an idea what is/might be going on, and how to fix/prevent this? or how to migrate/replicate in a fast way?
See above, but you provide no detail on what you have done to speed up your import, and this seems the original problem.
Regards, Buchan
openldap-technical@openldap.org
-
Arjan Filius -
Buchan Milne