5:51 p.m.
I'm a little puzzled by the openldap replication docs; in particular
rid, entryuuid, entrycsn, contextcsn fields that I see referenced a lot.
I'm guessing the rid is a random chosen id number for the secondary
server (consumer?) that is used to compare the master db (entryuuid?)
for context? information that indicates sync state?
Basically, I'd like to understand the replication process at a slightly
higher level than http://www.openldap.org/doc/admin24/syncrepl.html
describes not quite completely enough...
The specific task at hand is trying to understand what I'm seeing in the
logs, having setup a master, loaded the db, configured replication,
setup a fresh secondary mirror and turned on replication there. I let
it run over the weekend, and see debug logs of what I assume is polling
on the secondary, but 'slapcat | grep -c "dn:"' returns (and has been
for a while) a suspiciously round number of "500" (there are around
38,000 records in the master), though I don't see any such limit
anywhere in the configs.
master config:
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
secondary config:
syncrepl rid=314
provider=ldap://master-server.peak.org:389
type=refreshOnly
interval=00:00:05:00
retry="60 10 300 +"
searchbase="dc=peak,dc=org"
schemachecking=off
bindmethod=simple
binddn="cn=replicator,dc=peak,dc=org"
credentials="somepassword"
syncdata="accesslog"
Mar 1 17:43:33 ldap04 slapd[6242]: >>> dnPrettyNormal:
<uid=someuser,dc=peak,dc=org>
Mar 1 17:43:33 ldap04 slapd[6242]: <<< dnPrettyNormal:
<uid=someuser,dc=peak,dc=org>, <uid=someuser,dc=peak,dc=org>
Mar 1 17:43:33 ldap04 slapd[6242]: >>> dnPretty: <cn=Directory
Manager,dc=peak,dc=org>
Mar 1 17:43:33 ldap04 slapd[6242]: <<< dnPretty: <cn=Directory
Manager,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize: <cn=Directory
Manager,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize: <cn=directory
manager,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnPretty: <cn=Directory
Manager,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnPretty: <cn=Directory
Manager,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize: <cn=Directory
Manager,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize: <cn=directory
manager,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnPretty:
<uid=someuser,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnPretty:
<uid=someuser,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize:
<uid=someuser,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize:
<uid=someuser,dc=peak,dc=org>
Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnPretty: <cn=Subschema>
Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnPretty: <cn=Subschema>
Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize: <cn=Subschema>
Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize: <cn=subschema>
Mar 1 17:43:34 ldap04 slapd[6242]: syncrepl_entry: rid 314
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
Mar 1 17:43:34 ldap04 slapd[6242]: => bdb_search
Mar 1 17:43:34 ldap04 slapd[6242]: bdb_dn2entry("dc=peak,dc=org")
Mar 1 17:43:34 ldap04 slapd[6242]: search_candidates: base="dc=peak,dc=org"
(0x00000001) scope=2
Mar 1 17:43:34 ldap04 slapd[6242]: => bdb_dn2idl("dc=peak,dc=org")
Mar 1 17:43:34 ldap04 slapd[6242]: => bdb_filter_candidates
Mar 1 17:43:34 ldap04 slapd[6242]: AND
Mar 1 17:43:35 ldap04 slapd[6242]: => bdb_list_candidates 0xa0
Mar 1 17:43:35 ldap04 slapd[6242]: => bdb_filter_candidates
Mar 1 17:43:35 ldap04 slapd[6242]: EQUALITY
Mar 1 17:43:35 ldap04 slapd[6242]: => bdb_equality_candidates (entryUUID)
Mar 1 17:43:35 ldap04 slapd[6242]: => key_read
Mar 1 17:43:35 ldap04 slapd[6242]: bdb_idl_fetch_key: [15cb4950]
Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_index_read 1 candidates
Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_equality_candidates: id=1, first=169,
last=169
Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_filter_candidates: id=1 first=169 last=169
Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_list_candidates: id=1 first=169 last=169
Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_filter_candidates: id=1 first=169 last=169
Mar 1 17:43:35 ldap04 slapd[6242]: bdb_search_candidates: id=1 first=169 last=169
Mar 1 17:43:35 ldap04 slapd[6242]: => test_filter
Mar 1 17:43:35 ldap04 slapd[6242]: EQUALITY
Mar 1 17:43:35 ldap04 slapd[6242]: => access_allowed: search access to
"uid=someuser,dc=peak,dc=org" "entryUUID" requested
Mar 1 17:43:35 ldap04 slapd[6242]: <= root access granted
Mar 1 17:43:35 ldap04 slapd[6242]: <= test_filter 6
Mar 1 17:43:35 ldap04 slapd[6242]: send_ldap_result: conn=-1 op=0 p=3
Mar 1 17:43:35 ldap04 slapd[6242]: send_ldap_result: err=0 matched=""
text=""
Mar 1 17:43:35 ldap04 slapd[6242]: syncrepl_entry: rid 314 be_search (0)
Mar 1 17:43:35 ldap04 slapd[6242]: syncrepl_entry: rid 314 uid=someuser,dc=peak,dc=org
10:49 a.m.
On Mon, 1 Mar 2010, Alan Batie wrote:
I'm a little puzzled by the openldap replication docs; in
particular
rid, entryuuid, entrycsn, contextcsn fields that I see referenced a lot.
[...]
Basically, I'd like to understand the replication process at a
slightly
higher level than http://www.openldap.org/doc/admin24/syncrepl.html
describes not quite completely enough...
In addition to the Admin Guide, I suggest reading RFC4533 and the other
material in the doc/ directory of the distribution. Also, even if you're
not particularly adept at C, you may wish to read the comments in
syncprov.c source file. The terms you're asking about are the names that
OpenLDAP uses in its implementation; hopefully with the RFC in hand it
will become clear how the concepts map to the names, and you can always
ask if anything is still unclear.
I'm guessing the rid is a random chosen id number for the
secondary
server (consumer?) that is used to compare the master db (entryuuid?)
for context? information that indicates sync state?
rid is chosen by administrator (i.e. manually configured). I honestly
forget what it's used for nowadays...I vaguely remember "nothing" and/or
"it's only important for multimaster." I'm certain this is in the
archives
of openldap-software and/or openldap-devel, but I'll leave the search to
you.
The specific task at hand is trying to understand what I'm seeing
in the
logs, having setup a master, loaded the db, configured replication,
setup a fresh secondary mirror and turned on replication there. I let
it run over the weekend, and see debug logs of what I assume is polling
on the secondary, but 'slapcat | grep -c "dn:"' returns (and has been
for a while) a suspiciously round number of "500" (there are around
38,000 records in the master), though I don't see any such limit
anywhere in the configs.
My take on this would be that, if you turned up debugging and/or created a
packet capture, you will find LDAP_SIZELIMIT_EXCEEDED returned by the
provider in response to the consumer search request. As for not seeing the
limit, please read slapd.conf(5) man page thoroughly.
master config:
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
That take above would be more precise if you gave the unabridged provider
config...
12:03 p.m.
--On Tuesday, March 02, 2010 1:49 PM -0500 Aaron Richton
<richton(a)nbcs.rutgers.edu> wrote:
> I'm guessing the rid is a random chosen id number for the
secondary
> server (consumer?) that is used to compare the master db (entryuuid?)
> for context? information that indicates sync state?
rid is chosen by administrator (i.e. manually configured). I honestly
forget what it's used for nowadays...I vaguely remember "nothing" and/or
"it's only important for multimaster." I'm certain this is in the
archives of openldap-software and/or openldap-devel, but I'll leave the
search to you.
The RID value is a unique number in a given replica used to specifically
identify that particular syncrepl statement. It really only matters if you
have multiple syncrepl statements in a given slapd configuration for a
replica. For me, since I only replicate from a single master, I use the
same RID value on all my replicas.
With MMR, there is also the SID, which may be the other part you are
thinking of, but if you aren't using MMR, it doesn't apply.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
12:27 p.m.
Aaron Richton wrote:
On Mon, 1 Mar 2010, Alan Batie wrote:
> I'm a little puzzled by the openldap replication docs; in particular
> rid, entryuuid, entrycsn, contextcsn fields that I see referenced a lot.
[...]
> Basically, I'd like to understand the replication process at a slightly
> higher level than http://www.openldap.org/doc/admin24/syncrepl.html
> describes not quite completely enough...
In addition to the Admin Guide, I suggest reading RFC4533 and the other
material in the doc/ directory of the distribution. Also, even if you're
not particularly adept at C, you may wish to read the comments in
syncprov.c source file. The terms you're asking about are the names that
OpenLDAP uses in its implementation; hopefully with the RFC in hand it
will become clear how the concepts map to the names, and you can always
ask if anything is still unclear.
> I'm guessing the rid is a random chosen id number for the secondary
> server (consumer?) that is used to compare the master db (entryuuid?)
> for context? information that indicates sync state?
rid is chosen by administrator (i.e. manually configured). I honestly
forget what it's used for nowadays...I vaguely remember "nothing" and/or
"it's only important for multimaster." I'm certain this is in the
archives
of openldap-software and/or openldap-devel, but I'll leave the search to
you.
Like the manpage says:
>>
rid identifies the current syncrepl directive within
the replication consumer
site.
<<<
It's only use is to give the slapd -c option something to reference. Nothing else.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
4510
days inactive
4510
days old
openldap-software@openldap.org
3 comments
4 participants
participants (4)
-
Aaron Richton -
Alan Batie -
Howard Chu -
Quanah Gibson-Mount