9:31 a.m.
Hi,
I am facing a problem with OpenLDAP-2.4.23. That is I am trying to setup
a delta replication based on accesslog. Although the accesslog database
contains new entries and the log database is successfully read, no
replication ever takes place, while a setup without accesslog replicates successfully.
An excerpt from the systemlogs:
slapd[19170]: conn=1050 fd=20 ACCEPT from IP=192.168.0.8:41487 (IP=0.0.0.0:389)
slapd[19170]: conn=1050 op=0 EXT oid=1.3.6.1.4.1.1466.20037
slapd[19170]: conn=1050 op=0 STARTTLS
slapd[19170]: conn=1050 op=0 RESULT oid= err=0 text=
slapd[19170]: conn=1050 fd=20 TLS established tls_ssf=256 ssf=256
slapd[19170]: conn=1050 op=1 BIND dn="" method=163
slapd[19170]: conn=1050 op=1 BIND authcid="cn=replicator,o=avci,c=de"
authzid="cn=replicator,o=avci,c=de"
slapd[19170]: conn=1050 op=1 BIND dn="cn=replicator,o=avci,c=de" mech=EXTERNAL
sasl_ssf=0 ssf=256
slapd[19170]: conn=1050 op=1 RESULT tag=97 err=0 text=
slapd[19170]: conn=1050 op=2 SRCH base="cn=log" scope=2 deref=0
filter="(&(objectClass=auditWriteObject)(reqResult=0))"
slapd[19170]: conn=1050 op=2 SRCH attr=reqDN reqType reqMod reqNewRDN reqDeleteOldRDN
reqNewSuperior entryCSN
slapd[19170]: conn=1050 op=2 SEARCH RESULT tag=101 err=0 nentries=7 text=
slapd[19170]: conn=1050 op=3 UNBIND
slapd[19170]: conn=1050 fd=20 closed
A ldapsearch -Y EXTERNAL -ZZ -H ldap://localhost -b cn=log -s one
"(&(objectClass=auditWriteObject)(reqResult=0))"
provides the expected results, an example
# 20101129144212.000003Z, log
dn: reqStart=20101129144212.000003Z,cn=log
objectClass: auditModify
reqStart: 20101129144212.000003Z
reqEnd: 20101129144212.000004Z
reqType: modify
reqSession: 1007
reqAuthzID: cn=admanager,o=avci,c=de
reqControls: {0}{2.16.840.1.113730.3.4.18 controlValue "646O3K636O3N61646N616O
616765722M6P3N617663692M633N6465"}
reqDN:: Y249U2Now6RmZXIgV29sZmdhbmcsb3U9YWRyZXNzYnVjaCxvPWF2Y2ksYz1kZQ==
reqResult: 0
reqMod: sn:- Wolfgang
reqMod: sn:+ Schaefer
reqMod: entryCSN:= 20101129144212.353910Z#000000#000#000000
reqMod: modifiersName:= cn=admanager,o=avci,c=de
reqMod: modifyTimestamp:= 20101129144212Z
The relevant config files can be found here:
Provider slapd.conf: http://pastebin.de/12273
Consumer slapd.conf: http://pastebin.de/12275
-Dieter
--
Dieter Kl�nter | Systemberatung
sip: 7770535(a)sipgate.de
http://www.dpunkt.de/buecher/2104.html
GPG Key ID:8EF7B6C6
10:38 a.m.
--On Monday, November 29, 2010 6:31 PM +0100 Dieter Klünter
<dieter(a)dkluenter.de> wrote:
Hi,
I am facing a problem with OpenLDAP-2.4.23. That is I am trying to setup
a delta replication based on accesslog. Although the accesslog database
contains new entries and the log database is successfully read, no
replication ever takes place, while a setup without accesslog replicates
successfully. An excerpt from the systemlogs:
delta-sync replication works great for me with 2.4.23. I would suggest you
define fewer things in your replica configuration. In specific, I would
drop:
logfilter
syncdata
scope
I would also add a new line between the syncrepl and updateref lines.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
12:31 p.m.
Hi Quanah,
On Mon, Nov 29, 2010 at 10:38:16AM -0800, Quanah Gibson-Mount wrote:
--On Monday, November 29, 2010 6:31 PM +0100 Dieter Klünter
<dieter(a)dkluenter.de> wrote:
>Hi,
>I am facing a problem with OpenLDAP-2.4.23. That is I am trying to setup
>a delta replication based on accesslog. Although the accesslog database
>contains new entries and the log database is successfully read, no
>replication ever takes place, while a setup without accesslog replicates
>successfully. An excerpt from the systemlogs:
delta-sync replication works great for me with 2.4.23. I would
suggest you define fewer things in your replica configuration. In
specific, I would drop:
logfilter
syncdata
scope
After removing this paramaters, replication took place:
slapd[19170]: conn=3853 op=15 ADD dn="cn=xxxxxxxxxxxxxxx"
slapd[19170]: slap_queue_csn: queing 0xb0b5789e 20101129201509.698643Z#000000#000#000000
slapd[19170]: syncprov_sendresp:
cookie=rid=042,csn=20101129201509.698643Z#000000#000#000000
slapd[19170]: slap_queue_csn: queing 0xb0657370 20101129201509.698643Z#000000#000#000000
slapd[19170]: slap_graduate_commit_csn: removing 0xb7b024e0
20101129201509.698643Z#000000#000#000000
but I'm not sure, whether the consumer had read the accesslog database and replicated
based on this
information, because no access to cn=log has been logged.
-Dieter
--
Dieter Kl�nter | Systemberatung
sip: 7770535(a)sipgate.de
http://www.dpunkt.de/buecher/2104.html
GPG Key ID:8EF7B6C6
12:49 p.m.
--On Monday, November 29, 2010 9:31 PM +0100 Dieter Klünter
<dieter(a)dkluenter.de> wrote:
Hi Quanah,
but I'm not sure, whether the consumer had read the accesslog database
and replicated based on this information, because no access to cn=log
has been logged.
If the replica has no data when it is started, then it will do a full
refresh from the primary DB on the master, just like with normal syncrepl,
and skip the log database entirely. Now that it is in sync, I would make a
change and see if that gets replicated.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
9:04 a.m.
Dieter Klünter <dieter(a)dkluenter.de> writes:
Hi Quanah,
On Mon, Nov 29, 2010 at 10:38:16AM -0800, Quanah Gibson-Mount wrote:
> --On Monday, November 29, 2010 6:31 PM +0100 Dieter Klünter
> <dieter(a)dkluenter.de> wrote:
>
> >Hi,
> >I am facing a problem with OpenLDAP-2.4.23. That is I am trying to
> > setup
> >a delta replication based on accesslog. Although the accesslog
> > database
> >contains new entries and the log database is successfully read, no
> >replication ever takes place, while a setup without accesslog
> > replicates
> >successfully. An excerpt from the systemlogs:
> delta-sync replication works great for me with 2.4.23. I would
> suggest you define fewer things in your replica configuration. In
> specific, I would drop:
>
> logfilter syncdata scope
>
After removing this paramaters, replication took place:
slapd[19170]: conn=3853 op=15 ADD dn="cn=xxxxxxxxxxxxxxx"
slapd[19170]: slap_queue_csn: queing 0xb0b5789e
20101129201509.698643Z#000000#000#000000 slapd[19170]:
syncprov_sendresp:
cookie=rid=042,csn=20101129201509.698643Z#000000#000#000000
slapd[19170]: slap_queue_csn: queing 0xb0657370
20101129201509.698643Z#000000#000#000000 slapd[19170]:
slap_graduate_commit_csn: removing 0xb7b024e0
20101129201509.698643Z#000000#000#000000
but I'm not sure, whether the consumer had read the accesslog database
and replicated based on this information, because no access to cn=log
has been logged.
It seems the consumer has not read the accesslog database, so without
acesslog replication takes place, but with accesslog enabled, no
replication.
After adding the removed parameters, again no replication. The
consumer logs (loglevel sync) are pested with:
slapd[2656]: do_syncrep2: rid=042 got search entry without Sync State control
slapd[2656]: do_syncrepl: rid=042 rc -1 retrying (4 retries left)
slapd[2656]: do_syncrep2: rid=042 got search entry without Sync State control
slapd[2656]: do_syncrepl: rid=042 rc -1 retrying (4 retries left)
slapd[2656]: do_syncrep2: rid=042 got search entry without Sync State control
slapd[2656]: do_syncrepl: rid=042 rc -1 retrying (4 retries left)
Any idea what is going on?
-Dieter
--
Dieter Klünter | Systemberatung
sip: 7770535(a)sipgate.de
http://www.dpunkt.de/buecher/2104.html
GPG Key ID:8EF7B6C6
9:53 a.m.
--On Tuesday, November 30, 2010 6:04 PM +0100 Dieter Kluenter
<dieter(a)dkluenter.de> wrote:
Any idea what is going on?
On the master, in the accesslog overlay section for the main DB, you're
missing the following parameter:
logsuccess TRUE
On the master, in the accesslog database section, you are missing the
syncprov overlay:
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
11:33 a.m.
Quanah Gibson-Mount <quanah(a)zimbra.com> writes:
--On Tuesday, November 30, 2010 6:04 PM +0100 Dieter Kluenter
<dieter(a)dkluenter.de> wrote:
> Any idea what is going on?
On the master, in the accesslog overlay section for the main DB,
you're missing the following parameter:
logsuccess TRUE
I want to log positiv and negativ write operations
On the master, in the accesslog database section, you are missing
the
syncprov overlay:
overlay syncprov
syncprov-nopresent TRUE
syncprov-reloadhint TRUE
This I realised after running test043 too, but I wonder whether this
is required at all, or is this just lack of documentation?
I am sorry for all this noise, mea culpa. As I mentioned, the OS on
both systems is suse and as such, an administrator should take care of
/etc/sysconfig/openldap, in particular the option
OPENLDAP_CONFIG_BACKEND='ldap', that is, slapd is not reading
slapd.conf but slapd.d
The system is working now as desired. Thank you Quanah.
-Dieter
--
Dieter Klünter | Systemberatung
sip: 7770535(a)sipgate.de
http://www.dpunkt.de/buecher/2104.html
GPG Key ID:8EF7B6C6
11:41 a.m.
--On Tuesday, November 30, 2010 8:33 PM +0100 Dieter Kluenter
<dieter(a)dkluenter.de> wrote:
> On the master, in the accesslog database section, you are missing
the
> syncprov overlay:
>
> overlay syncprov
> syncprov-nopresent TRUE
> syncprov-reloadhint TRUE
This I realised after running test043 too, but I wonder whether this
is required at all, or is this just lack of documentation?
It is absolutely required, as the accesslog database is providing
information to the syncrepl client, thus it must have the syncprov overlay
in place.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
12:23 p.m.
Quanah Gibson-Mount <quanah(a)zimbra.com> writes:
--On Tuesday, November 30, 2010 8:33 PM +0100 Dieter Kluenter
<dieter(a)dkluenter.de> wrote:
>> On the master, in the accesslog database section, you are missing the
>> syncprov overlay:
>>
>> overlay syncprov
>> syncprov-nopresent TRUE
>> syncprov-reloadhint TRUE
>
> This I realised after running test043 too, but I wonder whether this
> is required at all, or is this just lack of documentation?
It is absolutely required, as the accesslog database is providing
information to the syncrepl client, thus it must have the syncprov
overlay in place.
I am not convinced, as the consumer conducts a search operation on the
log database and not a sync operation, unless the consumer maintains a
local logdatabase. But others may clarify this.
-Dieter
--
Dieter Klünter | Systemberatung
http://dkluenter.de
GPG Key ID:8EF7B6C6
53°37'09,95"N
10°08'02,42"E
12:35 p.m.
--On Tuesday, November 30, 2010 9:23 PM +0100 Dieter Kluenter
<dieter(a)dkluenter.de> wrote:
I am not convinced, as the consumer conducts a search operation on
the
log database and not a sync operation, unless the consumer maintains a
local logdatabase. But others may clarify this.
There is replication information stored in the log database that is
maintained by the syncprov overlay. Thus, it is required for it to be
present. And that's why you were getting a very explicit error about the
overlay not being present when it was searching the log database. But you
can ignore your errors if you want.
Note the contextCSN stored in the accesslog DB:
# accesslog
dn: cn=accesslog
structuralObjectClass: auditContainer
entryCSN: 20101123115530.669034Z#000000#000#000000
contextCSN: 20101130203425.474878Z#000000#000#000000
entryDN: cn=accesslog
subschemaSubentry: cn=Subschema
hasSubordinates: TRUE
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
1:05 p.m.
Hi Quanah,
Quanah Gibson-Mount <quanah(a)zimbra.com> writes:
--On Monday, November 29, 2010 6:31 PM +0100 Dieter Klünter
<dieter(a)dkluenter.de> wrote:
> Hi,
> I am facing a problem with OpenLDAP-2.4.23. That is I am trying to setup
> a delta replication based on accesslog. Although the accesslog database
> contains new entries and the log database is successfully read, no
> replication ever takes place, while a setup without accesslog replicates
> successfully. An excerpt from the systemlogs:
delta-sync replication works great for me with 2.4.23. I would
suggest you define fewer things in your replica configuration. In
specific, I would drop:
logfilter
syncdata
scope
Are you sure, that only a logbase statement as part of syncrepl
statements as reference to delta synchronization is sufficient?
-Dieter
--
Dieter Klünter | Systemberatung
http://dkluenter.de
GPG Key ID:8EF7B6C6
53°37'09,95"N
10°08'02,42"E
1:09 p.m.
--On Monday, November 29, 2010 10:05 PM +0100 Dieter Kluenter
<dieter(a)dkluenter.de> wrote:
Hi Quanah,
Are you sure, that only a logbase statement as part of syncrepl
statements as reference to delta synchronization is sufficient?
Egh, no, you're right. I have:
syncrepl rid=${ldap_replica_rid}
provider=${ldap_master_url}
retry="60 +"
type=refreshAndPersist
schemachecking=off
searchbase=""
bindmethod=simple
binddn=uid=zmreplica,cn=admins,cn=zimbra
credentials=@@ldap_replication_password@@
logbase="cn=accesslog"
logfilter="(&(objectClass=auditWriteObject)(reqResult=0))"
syncdata="accesslog"
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
4501
days inactive
4502
days old
openldap-technical@openldap.org
11 comments
3 participants
participants (3)
-
Dieter Kluenter -
Dieter Klünter
-
Quanah Gibson-Mount