Buchan,
I made exactly what you said even using the -q flag in the slapadd command. So in summary I did :
1) Load the master DB using LDIF file through slapdd(-q flag and DB_CACHEZIZE to 1GB); 2) Load the slave DB using LDIF(same) file through slapdd(-q flag and DB_CACHEZIZE to 1GB); 3)Then have the slapd.conf files appropriately configured 4)Start master and then after some minutes start slave.
So the slave is not starting blank but exactly loaded and in terms of data there isn't any difference between provider(master) and consumer(slave). I just see when the consumer slapd process start at slave machine(slapd started with -d 256 flag) the connection from the consumer to the provider slapd process.
Then all behavior explained start to happen. Please see the provider and consumer configuration file attached.
Something appears not being following the expect behavior. Also the memory consumed in the consumer is growing too fast and doesn't appear to really follow the caches directives.
Regards,
Rodrigo.
Buchan Milne wrote:
On Sunday 03 May 2009 04:15:47 Rodrigo Costa wrote:
openldap software,
Sometime ago I open the ITS#5860 about some memory cache limitations not being respected by config files. Even this issue was solved when I tried to configured openldap to use replication(syncrepl) the system never enter into sync and the behavior appears similar to the ITS#5860 bug.
The system start to sync and in the provider(master) I see the query for the DB sync. But the consumer(slave) memory consumption start to grow very fast making me to constrain much more the dncachesize to a 1/10 of the size of the provider(master) where at least system doesn't crash at consumer.
Since changes were done in the openldap 2.4.16 I download and made tests with this version. I get into the same behavior with consumer(slave) never getting in sync with provider(master).
The behaviors are :
- Consumer(slave) start query to the provider(master) DB;
- Memory allocation and number of threads in the provider(master) start
to increase as expected; 3) dncachesize directive into provider(master) controls as expected the maximum memory to be allocated by slapd process in provider(master); 4) Consumer(slave) consumer memory in a much faster pace. dncachesize configured to 1/10 of provider(master) to avoid memory allocation problems; 5) After sometime the consumer(slave) CPU usage maintains in 200%. Provider(master) stays with low CPU usage, around 1 to 3 %; 6) A new provisioning in provider(master) isn't propagated to consumer(slave); 7) Bases never get in sync and CPU usage in consumer still high. Queries to provider(master) are answer very fast and even multiple individual queries to consumer(slave) are also answer in reasonable time.
It looks like could exist certain issue in the replication logic where some processing dead loop could be found by the replication consumer(slave) logic.
The newest openldap version and Berkeley DB 4.7 with all patches were compiled in the platform running the code.
Any idea about this behavior?
I have seen behaviour like this when there was something preventing synchronisation, and the comsumer would spawn more consumer threads, until the box ran out of memory. I fixed the real issue, and haven't seen it since (and haven't had time to try and reproduce it).
However, on a large database, you may have better success by initialising the consumer via 'slapadd' with an export from a provider, instead of using syncrepl to do it. Since slapadd can run multiple threads (syncrepl only runs one thread), and doesn't need to bother serving client requests, and can run without transactions (see -q flag), it is much more efficient.
Note that you could consider different tuning for import vs run-time, e.g. I usually increase the BDB cache_size for imports with slapadd, and decrease it for runtime.
Regards, Buchan
--On Tuesday, May 05, 2009 6:50 PM -0700 Rodrigo Costa rlvcosta@yahoo.com wrote:
Buchan,
I made exactly what you said even using the -q flag in the slapadd command. So in summary I did :
- Load the master DB using LDIF file through slapdd(-q flag and
DB_CACHEZIZE to 1GB); 2) Load the slave DB using LDIF(same) file through slapdd(-q flag and DB_CACHEZIZE to 1GB); 3)Then have the slapd.conf files appropriately configured 4)Start master and then after some minutes start slave.
Unless the LDIF was a valid export from a master, you should have slapcat'd the master after step 1, and used that LDIF to load the replica in step 2.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Rodrigo Costa wrote:
Buchan,
I made exactly what you said even using the -q flag in the slapadd command. So in summary I did :
- Load the master DB using LDIF file through slapdd(-q flag and
DB_CACHEZIZE to 1GB); 2) Load the slave DB using LDIF(same) file through slapdd(-q flag and DB_CACHEZIZE to 1GB); 3)Then have the slapd.conf files appropriately configured 4)Start master and then after some minutes start slave.
So the slave is not starting blank but exactly loaded and in terms of data there isn't any difference between provider(master) and consumer(slave). I just see when the consumer slapd process start at slave machine(slapd started with -d 256 flag) the connection from the consumer to the provider slapd process.
Then all behavior explained start to happen. Please see the provider and consumer configuration file attached.
Something appears not being following the expect behavior. Also the memory consumed in the consumer is growing too fast and doesn't appear to really follow the caches directives.
Regards,
Rodrigo.
I do not know what attrs="*,+" is supposed to mean. But with OpenLDAP2.4.11, if you do a search with attrs="*,+" as the attributes to search, the search will not return any attributes. Your slave database will never be synchronized with the master. I'd comment out that line and try a sync again to determine if this is the cause.
I used an LDIF file to load 4 OpenLDAP servers and the synchronization works perfectly among the 4 servers after the LDIF file was loaded to all of them.
Buchan Milne wrote:
On Sunday 03 May 2009 04:15:47 Rodrigo Costa wrote:
openldap software,
Sometime ago I open the ITS#5860 about some memory cache limitations not being respected by config files. Even this issue was solved when I tried to configured openldap to use replication(syncrepl) the system never enter into sync and the behavior appears similar to the ITS#5860 bug.
The system start to sync and in the provider(master) I see the query for the DB sync. But the consumer(slave) memory consumption start to grow very fast making me to constrain much more the dncachesize to a 1/10 of the size of the provider(master) where at least system doesn't crash at consumer.
Since changes were done in the openldap 2.4.16 I download and made tests with this version. I get into the same behavior with consumer(slave) never getting in sync with provider(master).
The behaviors are :
- Consumer(slave) start query to the provider(master) DB;
- Memory allocation and number of threads in the provider(master) start
to increase as expected; 3) dncachesize directive into provider(master) controls as expected the maximum memory to be allocated by slapd process in provider(master); 4) Consumer(slave) consumer memory in a much faster pace. dncachesize configured to 1/10 of provider(master) to avoid memory allocation problems; 5) After sometime the consumer(slave) CPU usage maintains in 200%. Provider(master) stays with low CPU usage, around 1 to 3 %; 6) A new provisioning in provider(master) isn't propagated to consumer(slave); 7) Bases never get in sync and CPU usage in consumer still high. Queries to provider(master) are answer very fast and even multiple individual queries to consumer(slave) are also answer in reasonable time.
It looks like could exist certain issue in the replication logic where some processing dead loop could be found by the replication consumer(slave) logic.
The newest openldap version and Berkeley DB 4.7 with all patches were compiled in the platform running the code.
Any idea about this behavior?
I have seen behaviour like this when there was something preventing synchronisation, and the comsumer would spawn more consumer threads, until the box ran out of memory. I fixed the real issue, and haven't seen it since (and haven't had time to try and reproduce it).
However, on a large database, you may have better success by initialising the consumer via 'slapadd' with an export from a provider, instead of using syncrepl to do it. Since slapadd can run multiple threads (syncrepl only runs one thread), and doesn't need to bother serving client requests, and can run without transactions (see -q flag), it is much more efficient.
Note that you could consider different tuning for import vs run-time, e.g. I usually increase the BDB cache_size for imports with slapadd, and decrease it for runtime.
Regards, Buchan
John Du wrote:
Rodrigo Costa wrote:
Buchan,
I made exactly what you said even using the -q flag in the slapadd command. So in summary I did :
- Load the master DB using LDIF file through slapdd(-q flag and
DB_CACHEZIZE to 1GB); 2) Load the slave DB using LDIF(same) file through slapdd(-q flag and DB_CACHEZIZE to 1GB); 3)Then have the slapd.conf files appropriately configured 4)Start master and then after some minutes start slave.
So the slave is not starting blank but exactly loaded and in terms of data there isn't any difference between provider(master) and consumer(slave). I just see when the consumer slapd process start at slave machine(slapd started with -d 256 flag) the connection from the consumer to the provider slapd process.
Then all behavior explained start to happen. Please see the provider and consumer configuration file attached.
Something appears not being following the expect behavior. Also the memory consumed in the consumer is growing too fast and doesn't appear to really follow the caches directives.
Regards,
Rodrigo.
I do not know what attrs="*,+" is supposed to mean. But with OpenLDAP2.4.11, if you do a search with attrs="*,+" as the attributes to search, the search will not return any attributes. Your slave database will never be synchronized with the master. I'd comment out that line and try a sync again to determine if this is the cause.
I used an LDIF file to load 4 OpenLDAP servers and the synchronization works perfectly among the 4 servers after the LDIF file was loaded to all of them.
Run the following on the command line and you will see what you are trying to replicate.
ldapsearch -x -h 10.142.15.41 -b "ou=INDEXES,o=domain,c=fr" -D ="cn=admin,ou=INDEXES,o=domain,c=fr" -w secret * "*,+"
The above query is what your specified in your slapd_slave file.
Buchan Milne wrote:
On Sunday 03 May 2009 04:15:47 Rodrigo Costa wrote:
openldap software,
Sometime ago I open the ITS#5860 about some memory cache limitations not being respected by config files. Even this issue was solved when I tried to configured openldap to use replication(syncrepl) the system never enter into sync and the behavior appears similar to the ITS#5860 bug.
The system start to sync and in the provider(master) I see the query for the DB sync. But the consumer(slave) memory consumption start to grow very fast making me to constrain much more the dncachesize to a 1/10 of the size of the provider(master) where at least system doesn't crash at consumer.
Since changes were done in the openldap 2.4.16 I download and made tests with this version. I get into the same behavior with consumer(slave) never getting in sync with provider(master).
The behaviors are :
- Consumer(slave) start query to the provider(master) DB;
- Memory allocation and number of threads in the provider(master) start
to increase as expected; 3) dncachesize directive into provider(master) controls as expected the maximum memory to be allocated by slapd process in provider(master); 4) Consumer(slave) consumer memory in a much faster pace. dncachesize configured to 1/10 of provider(master) to avoid memory allocation problems; 5) After sometime the consumer(slave) CPU usage maintains in 200%. Provider(master) stays with low CPU usage, around 1 to 3 %; 6) A new provisioning in provider(master) isn't propagated to consumer(slave); 7) Bases never get in sync and CPU usage in consumer still high. Queries to provider(master) are answer very fast and even multiple individual queries to consumer(slave) are also answer in reasonable time.
It looks like could exist certain issue in the replication logic where some processing dead loop could be found by the replication consumer(slave) logic.
The newest openldap version and Berkeley DB 4.7 with all patches were compiled in the platform running the code.
Any idea about this behavior?
I have seen behaviour like this when there was something preventing synchronisation, and the comsumer would spawn more consumer threads, until the box ran out of memory. I fixed the real issue, and haven't seen it since (and haven't had time to try and reproduce it).
However, on a large database, you may have better success by initialising the consumer via 'slapadd' with an export from a provider, instead of using syncrepl to do it. Since slapadd can run multiple threads (syncrepl only runs one thread), and doesn't need to bother serving client requests, and can run without transactions (see -q flag), it is much more efficient.
Note that you could consider different tuning for import vs run-time, e.g. I usually increase the BDB cache_size for imports with slapadd, and decrease it for runtime.
Regards, Buchan
John Du wrote:
John Du wrote:
Rodrigo Costa wrote:
Buchan,
I made exactly what you said even using the -q flag in the slapadd command. So in summary I did :
- Load the master DB using LDIF file through slapdd(-q flag and
DB_CACHEZIZE to 1GB); 2) Load the slave DB using LDIF(same) file through slapdd(-q flag and DB_CACHEZIZE to 1GB); 3)Then have the slapd.conf files appropriately configured 4)Start master and then after some minutes start slave.
So the slave is not starting blank but exactly loaded and in terms of data there isn't any difference between provider(master) and consumer(slave). I just see when the consumer slapd process start at slave machine(slapd started with -d 256 flag) the connection from the consumer to the provider slapd process.
Then all behavior explained start to happen. Please see the provider and consumer configuration file attached.
Something appears not being following the expect behavior. Also the memory consumed in the consumer is growing too fast and doesn't appear to really follow the caches directives.
Regards,
Rodrigo.
I do not know what attrs="*,+" is supposed to mean. But with OpenLDAP2.4.11, if you do a search with attrs="*,+" as the attributes to search, the search will not return any attributes. Your slave database will never be synchronized with the master. I'd comment out that line and try a sync again to determine if this is the cause.
I used an LDIF file to load 4 OpenLDAP servers and the synchronization works perfectly among the 4 servers after the LDIF file was loaded to all of them.
Run the following on the command line and you will see what you are trying to replicate.
ldapsearch -x -h 10.142.15.41 -b "ou=INDEXES,o=domain,c=fr" -D ="cn=admin,ou=INDEXES,o=domain,c=fr" -w secret * "*,+"
The above query is what your specified in your slapd_slave file.
Not exactly. On the command line, attributes must be separated by spaces, not commas. And there's no "=" in front of the argument to -D. So:
ldapsearch -x -h 10.142.15.41 -b "ou=INDEXES,o=domain,c=fr" -D "cn=admin,ou=INDEXES,o=domain,c=fr" -w secret * +
Howard Chu wrote:
John Du wrote:
John Du wrote:
Rodrigo Costa wrote:
Buchan,
I made exactly what you said even using the -q flag in the slapadd command. So in summary I did :
- Load the master DB using LDIF file through slapdd(-q flag and
DB_CACHEZIZE to 1GB); 2) Load the slave DB using LDIF(same) file through slapdd(-q flag and DB_CACHEZIZE to 1GB); 3)Then have the slapd.conf files appropriately configured 4)Start master and then after some minutes start slave.
So the slave is not starting blank but exactly loaded and in terms of data there isn't any difference between provider(master) and consumer(slave). I just see when the consumer slapd process start at slave machine(slapd started with -d 256 flag) the connection from the consumer to the provider slapd process.
Then all behavior explained start to happen. Please see the provider and consumer configuration file attached.
Something appears not being following the expect behavior. Also the memory consumed in the consumer is growing too fast and doesn't appear to really follow the caches directives.
Regards,
Rodrigo.
I do not know what attrs="*,+" is supposed to mean. But with OpenLDAP2.4.11, if you do a search with attrs="*,+" as the attributes to search, the search will not return any attributes. Your slave database will never be synchronized with the master. I'd comment out that line and try a sync again to determine if this is the cause.
I used an LDIF file to load 4 OpenLDAP servers and the synchronization works perfectly among the 4 servers after the LDIF file was loaded to all of them.
Run the following on the command line and you will see what you are trying to replicate.
ldapsearch -x -h 10.142.15.41 -b "ou=INDEXES,o=domain,c=fr" -D ="cn=admin,ou=INDEXES,o=domain,c=fr" -w secret * "*,+"
The above query is what your specified in your slapd_slave file.
Not exactly. On the command line, attributes must be separated by spaces, not commas. And there's no "=" in front of the argument to -D. So:
ldapsearch -x -h 10.142.15.41 -b "ou=INDEXES,o=domain,c=fr" -D "cn=admin,ou=INDEXES,o=domain,c=fr" -w secret * +
My mistake. I thought the command line and slapd.conf use the same syntax for attribute list. The '=' after -D was a cut and paste error.
dapsearch -x -h 10.142.15.41 -b "ou=INDEXES,o=domain,c=fr" -D "cn=admin,ou=INDEXES,o=domain,c=fr" -w secret '*' '+' works too.
Is it necessary to explicitly specify the '+' in slapd.conf to replicate operational attributes?
--On Wednesday, May 06, 2009 1:55 PM -0700 John Du jjohndu@gmail.com wrote:
Is it necessary to explicitly specify the '+' in slapd.conf to replicate operational attributes?
You shouldn't specify the "attrs=" line in slapd.conf at all, unless you only want to replicate a subset of attributes. The default is usually all you want. But yes, if you plan on doing replication, and want it to work, and you are going to specify the attrs= line, then you better include "+" in there.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
John Du wrote:
I do not know what attrs="*,+" is supposed to mean. But with OpenLDAP2.4.11, if you do a search with attrs="*,+" as the attributes to search, the search will not return any attributes.
My slave replica configuration contains attrs="* +". Without having read the postings before IMHO probably the comma is the configuration error.
Ciao, Michael.
openldap-software@openldap.org
-
Howard Chu -
John Du -
Michael Ströder -
Quanah Gibson-Mount -
Rodrigo Costa