Hi,
I want to use delta-syncrepl replication with 2 masters. But each slapd-process permanently needs about 25 % CPU usage without any traffic on it.
The log looks endless like this:
** ld 01e43698 Outstanding Requests: * msgid 55, origid 55, status InProgress outstanding referrals 0, parent count 0 ld 01e43698 request count 1 (abandoned 18) ** ld 01e43698 Response Queue: Empty ld 01e43698 response count 0 ldap_chkResponseList ld 01e43698 msgid 55 all 0 ldap_chkResponseList returns ld 01e43698 NULL ldap_int_select read1msg: ld 01e43698 msgid 55 all 0 ber_get_next ber_get_next: tag 0x30 len 1187 contents: abandoned/discarded ld 01e43698 msgid 53 message type search-entry wait4msg continue ld 01e43698 msgid 55 all 0 ** ld 01e43698 Connections: * host: secondmaster.mydomain.local port: 389 (default) refcnt: 2 status: Connected last used: Mon Feb 13 11:26:53 2012
** ld 01e43698 Outstanding Requests: * msgid 55, origid 55, status InProgress outstanding referrals 0, parent count 0 ld 01e43698 request count 1 (abandoned 18) ** ld 01e43698 Response Queue: Empty ld 01e43698 response count 0 ldap_chkResponseList ld 01e43698 msgid 55 all 0 ldap_chkResponseList returns ld 01e43698 NULL ldap_int_select read1msg: ld 01e43698 msgid 55 all 0 ber_get_next ber_get_next: tag 0x30 len 1187 contents: abandoned/discarded ld 01e43698 msgid 53 message type search-entry wait4msg continue ld 01e43698 msgid 55 all 0 ** ld 01e43698 Connections: * host: secondmaster.mydomain.local port: 389 (default) refcnt: 2 status: Connected last used: Mon Feb 13 11:26:53 2012
here is my configuration (completely the same for both masters): ----------------------------------------------------------------------------------------------------- ucdata-path ./ucdata include ./schema/core.schema include ./schema/cosine.schema include ./schema/Personcaesar.schema include ./schema/ConfigObjects.schema
loglevel 0 logfile "C:/test/slapd.log"
pidfile ./run/slapd.pid argsfile ./run/slapd.args
access to * by dn.one="ou=Admins,o=caesar" write by anonymous auth
ServerID 1 "ldap://firstmaster.mydomain.local" ServerID 2 "ldap://secondmaster.mydomain.local"
###################################################################### database config rootdn cn=config rootpw {SHA}secret
####################################################################### # BDB database definitions ####################################################################### # Accesslog database definitions database hdb suffix cn=accesslog checkpoint 1024 5 cachesize 10000 directory "C:/test/accessdata" dbconfig set_cachesize 0 30000000 1 dbconfig set_flags DB_LOG_AUTOREMOVE dbconfig set_lg_regionmax 1048576 dbconfig set_lg_max 10485760 dbconfig set_lg_bsize 2097152 rootdn cn=accesslog index objectClass,entryCSN,entryUUID eq # I even tried removing reqMod, reading your docs I am not sure if this is needed here index reqEnd,reqResult,reqMod,reqStart eq
overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE # Let the replica DN have limitless searches limits dn.exact="cn=Replicator,ou=admins,o=caesar" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
# Primary database definitions database hdb suffix "o=caesar" checkpoint 1024 5 cachesize 10000 idlcachesize 30000 rootdn "cn=Administrator,o=caesar" rootpw {SHA}secret directory "C:/test/data" dbconfig set_cachesize 0 100000000 1 dbconfig set_flags DB_LOG_AUTOREMOVE dbconfig set_lg_regionmax 1048576 dbconfig set_lg_max 10485760 dbconfig set_lg_bsize 2097152
# syncprov specific indexing index sn pres,eq index cn pres,eq,sub ... index entryUUID eq index entryCSN eq index objectClass eq
# syncrepl Provider for primary db overlay syncprov syncprov-checkpoint 1000 60 syncprov-sessionlog 10000
# accesslog overlay definitions for primary db overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE # scan the accesslog DB every day, and purge entries older than 7 days logpurge 07+00:00 01+00:00
sizelimit size.soft=100 size.hard=1000 size.prtotal=unlimited # Let the replica DN have limitless searches limits dn.exact="cn=Replicator,ou=admins,o=caesar" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
syncrepl rid=001 provider="ldap://firstmaster.mydomain.local" searchbase="o=caesar" type=refreshAndPersist retry="5 3 15 +" binddn="cn=Replicator,ou=admins,o=caesar" bindmethod=simple credentials="secret" logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" schemachecking=on syncdata=accesslog
syncrepl rid=002 provider="ldap://secondmaster.mydomain.local" searchbase="o=caesar" type=refreshAndPersist retry="5 3 15 +" binddn="cn=Replicator,ou=admins,o=caesar" bindmethod=simple credentials="secret" logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" schemachecking=on syncdata=accesslog
MirrorMode On -----------------------------------------------------------------------------------------------------
I did my test on 2 Windows PCs and OpenLDAP 2.4.29 with Berkeley 5.1 .
Thanks for any hints, FO
I wanted to add, that one database is empty when I start and the other has only some data. But with master/slave this is no problem, so I think it should also work for master/master, doesn't it?
I retried with a patched version with these patches:
< http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=patch;h=8e7af6331e...
< http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=d4b3c...
< http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=eace2...
Unfortunately the problem still persists.
Could at least anyone confirm that the configuration is correct? Then I can at last stop searching for a error I possibly did and accept that OpenLDAP with Master/Master (delta-syncrepl) for Windows simply does not work.
Best regards, Frank --
openldap-technical-bounces@OpenLDAP.org schrieb am 13.02.2012 11:53:16:
Von: frank.offermanns@caseris.de An: openldap-technical@openldap.org Datum: 13.02.2012 12:07 Betreff: delta-syncrepl and mirrormode problem (2.4.29) Gesendet von: openldap-technical-bounces@OpenLDAP.org
Hi,
I want to use delta-syncrepl replication with 2 masters. But each slapd-process permanently needs about 25 % CPU usage without
any
traffic on it.
The log looks endless like this:
** ld 01e43698 Outstanding Requests:
- msgid 55, origid 55, status InProgress outstanding referrals 0, parent count 0
ld 01e43698 request count 1 (abandoned 18) ** ld 01e43698 Response Queue: Empty ld 01e43698 response count 0 ldap_chkResponseList ld 01e43698 msgid 55 all 0 ldap_chkResponseList returns ld 01e43698 NULL ldap_int_select read1msg: ld 01e43698 msgid 55 all 0 ber_get_next ber_get_next: tag 0x30 len 1187 contents: abandoned/discarded ld 01e43698 msgid 53 message type search-entry wait4msg continue ld 01e43698 msgid 55 all 0 ** ld 01e43698 Connections:
- host: secondmaster.mydomain.local port: 389 (default) refcnt: 2 status: Connected last used: Mon Feb 13 11:26:53 2012
** ld 01e43698 Outstanding Requests:
- msgid 55, origid 55, status InProgress outstanding referrals 0, parent count 0
ld 01e43698 request count 1 (abandoned 18) ** ld 01e43698 Response Queue: Empty ld 01e43698 response count 0 ldap_chkResponseList ld 01e43698 msgid 55 all 0 ldap_chkResponseList returns ld 01e43698 NULL ldap_int_select read1msg: ld 01e43698 msgid 55 all 0 ber_get_next ber_get_next: tag 0x30 len 1187 contents: abandoned/discarded ld 01e43698 msgid 53 message type search-entry wait4msg continue ld 01e43698 msgid 55 all 0 ** ld 01e43698 Connections:
- host: secondmaster.mydomain.local port: 389 (default) refcnt: 2 status: Connected last used: Mon Feb 13 11:26:53 2012
here is my configuration (completely the same for both masters):
-----------------------------------------------------------------------------------------------------
ucdata-path ./ucdata include ./schema/core.schema include ./schema/cosine.schema include ./schema/Personcaesar.schema include ./schema/ConfigObjects.schema
loglevel 0 logfile "C:/test/slapd.log"
pidfile ./run/slapd.pid argsfile ./run/slapd.args
access to * by dn.one="ou=Admins,o=caesar" write by anonymous auth
ServerID 1 "ldap://firstmaster.mydomain.local" ServerID 2 "ldap://secondmaster.mydomain.local"
###################################################################### database config rootdn cn=config rootpw {SHA}secret
####################################################################### # BDB database definitions ####################################################################### # Accesslog database definitions database hdb suffix cn=accesslog checkpoint 1024 5 cachesize 10000 directory "C:/test/accessdata" dbconfig set_cachesize 0 30000000 1 dbconfig set_flags DB_LOG_AUTOREMOVE dbconfig set_lg_regionmax 1048576 dbconfig set_lg_max 10485760 dbconfig set_lg_bsize 2097152 rootdn cn=accesslog index objectClass,entryCSN,entryUUID eq # I even tried removing reqMod, reading your docs I am not sure if this
is
needed here index reqEnd,reqResult,reqMod,reqStart eq
overlay syncprov syncprov-nopresent TRUE syncprov-reloadhint TRUE # Let the replica DN have limitless searches limits dn.exact="cn=Replicator,ou=admins,o=caesar" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
# Primary database definitions database hdb suffix "o=caesar" checkpoint 1024 5 cachesize 10000 idlcachesize 30000 rootdn "cn=Administrator,o=caesar" rootpw {SHA}secret directory "C:/test/data" dbconfig set_cachesize 0 100000000 1 dbconfig set_flags DB_LOG_AUTOREMOVE dbconfig set_lg_regionmax 1048576 dbconfig set_lg_max 10485760 dbconfig set_lg_bsize 2097152
# syncprov specific indexing index sn pres,eq index cn pres,eq,sub ... index entryUUID eq index entryCSN eq index objectClass eq
# syncrepl Provider for primary db overlay syncprov syncprov-checkpoint 1000 60 syncprov-sessionlog 10000
# accesslog overlay definitions for primary db overlay accesslog logdb cn=accesslog logops writes logsuccess TRUE # scan the accesslog DB every day, and purge entries older than 7 days logpurge 07+00:00 01+00:00
sizelimit size.soft=100 size.hard=1000 size.prtotal=unlimited # Let the replica DN have limitless searches limits dn.exact="cn=Replicator,ou=admins,o=caesar" time.soft=unlimited time.hard=unlimited size.soft=unlimited size.hard=unlimited
syncrepl rid=001 provider="ldap://firstmaster.mydomain.local" searchbase="o=caesar" type=refreshAndPersist retry="5 3 15 +" binddn="cn=Replicator,ou=admins,o=caesar" bindmethod=simple credentials="secret" logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" schemachecking=on syncdata=accesslog
syncrepl rid=002 provider="ldap://secondmaster.mydomain.local" searchbase="o=caesar" type=refreshAndPersist retry="5 3 15 +" binddn="cn=Replicator,ou=admins,o=caesar" bindmethod=simple credentials="secret" logbase="cn=accesslog" logfilter="(&(objectClass=auditWriteObject)(reqResult=0))" schemachecking=on syncdata=accesslog
MirrorMode On
-----------------------------------------------------------------------------------------------------
I did my test on 2 Windows PCs and OpenLDAP 2.4.29 with Berkeley 5.1 .
Thanks for any hints, FO
--On February 17, 2012 11:35:41 AM +0100 frank.offermanns@caseris.de wrote:
I wanted to add, that one database is empty when I start and the other has only some data. But with master/slave this is no problem, so I think it should also work for master/master, doesn't it?
Hi Frank,
I notice you've configured via slapd.conf, but set up your replication as if you were replicating cn=config, which doesn't really make much sense. For example, listing the olcServerID twice with URIs. You should only need to have a single olcServerID that is unique to each master, and with no URI when using slapd.conf. In any case, I've never used OpenLDAP on Windows, so I don't know specifically if it works, but if test063 passes for you, then that would indicate it works.
--Quanah
Hello Quanah,
I am double checking my configuration right now.
I notice you've configured via slapd.conf, but set up your replication
as
if you were replicating cn=config, which doesn't really make much sense.
I have no syncrepl - directives in my config-database configured: ######################################################################
database config
rootdn cn=config
rootpw {SHA}secret
####################################################################### As far as I understood, the configuration only gets replicated when it is configured to do so.
For example, listing the olcServerID twice with URIs. You should only
need
to have a single olcServerID that is unique to each master, and with no
URI
when using slapd.conf.
According to the man pages on openldap.org it should be right: serverID <integer> [<URL>] ... If the URL is provided, this directive may be specified multiple times, providing a complete list of participating servers and their IDs. The fully qualified hostname of each server should be used in the supplied URLs.
Nevertheless I tried with only one server-entry for each master and had the same problems.
so I don't know specifically if it works, but if test063 passes for you,
then that would indicate it works.
It is not so easy to get the tests running with windows. So I can't check if test063 passes yet.
Thanks for your response and best regards, Frank
Hello Quanah,
I did change my configuration, unfortunately without a change in behaviour.
I posted this Email at 21.2.2012 to inform you about this :
Hello Quanah,
I am double checking my configuration right now.
I notice you've configured via slapd.conf, but set up your replication
as
if you were replicating cn=config, which doesn't really make much
sense.
I have no syncrepl - directives in my config-database configured: ######################################################################
database config
rootdn cn=config
rootpw {SHA}secret
####################################################################### As far as I understood, the configuration only gets replicated when it is
configured to do so.
For example, listing the olcServerID twice with URIs. You should only
need
to have a single olcServerID that is unique to each master, and with no
URI
when using slapd.conf.
According to the man pages on openldap.org it should be right: serverID <integer> [<URL>]
...
If the URL is provided, this directive may bespecified multiple times, providing a complete list of participating servers and their IDs. The fully qualified hostname of each server should be used in the supplied URLs.
Nevertheless I tried with only one server-entry for each master and had the same problems.
so I don't know specifically if it works, but if test063 passes for
you,
then that would indicate it works.
It is not so easy to get the tests running with windows. So I can't check
if test063 passes yet.
Thanks for your response and best regards, Frank
--
Von: Quanah Gibson-Mount quanah@zimbra.com An: frank.offermanns@caseris.de, openldap-technical@openldap.org Datum: 21.02.2012 07:05 Betreff: Re: delta-syncrepl and mirrormode problem (2.4.29) Gesendet von: openldap-technical-bounces@OpenLDAP.org
--On February 17, 2012 11:35:41 AM +0100 frank.offermanns@caseris.de wrote:
I wanted to add, that one database is empty when I start and the other has only some data. But with master/slave this is no problem, so I
think
it should also work for master/master, doesn't it?
Hi Frank,
I notice you've configured via slapd.conf, but set up your replication as if you were replicating cn=config, which doesn't really make much sense. For example, listing the olcServerID twice with URIs. You should only need to have a single olcServerID that is unique to each master, and with no URI when using slapd.conf. In any case, I've never used OpenLDAP on Windows, so I don't know specifically if it works, but if test063 passes for you, then that would indicate it works.
--Quanah
openldap-technical@openldap.org
-
frank.offermanns＠caseris.de -
Quanah Gibson-Mount