I'm a little puzzled by the openldap replication docs; in particular rid, entryuuid, entrycsn, contextcsn fields that I see referenced a lot. I'm guessing the rid is a random chosen id number for the secondary server (consumer?) that is used to compare the master db (entryuuid?) for context? information that indicates sync state?
Basically, I'd like to understand the replication process at a slightly higher level than http://www.openldap.org/doc/admin24/syncrepl.html describes not quite completely enough...
The specific task at hand is trying to understand what I'm seeing in the logs, having setup a master, loaded the db, configured replication, setup a fresh secondary mirror and turned on replication there. I let it run over the weekend, and see debug logs of what I assume is polling on the secondary, but 'slapcat | grep -c "dn:"' returns (and has been for a while) a suspiciously round number of "500" (there are around 38,000 records in the master), though I don't see any such limit anywhere in the configs.
master config:
overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
secondary config:
syncrepl rid=314 provider=ldap://master-server.peak.org:389 type=refreshOnly interval=00:00:05:00 retry="60 10 300 +" searchbase="dc=peak,dc=org" schemachecking=off bindmethod=simple binddn="cn=replicator,dc=peak,dc=org" credentials="somepassword" syncdata="accesslog"
Mar 1 17:43:33 ldap04 slapd[6242]: >>> dnPrettyNormal: <uid=someuser,dc=peak,dc=org> Mar 1 17:43:33 ldap04 slapd[6242]: <<< dnPrettyNormal: <uid=someuser,dc=peak,dc=org>, <uid=someuser,dc=peak,dc=org> Mar 1 17:43:33 ldap04 slapd[6242]: >>> dnPretty: <cn=Directory Manager,dc=peak,dc=org> Mar 1 17:43:33 ldap04 slapd[6242]: <<< dnPretty: <cn=Directory Manager,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize: <cn=Directory Manager,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize: <cn=directory manager,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnPretty: <cn=Directory Manager,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnPretty: <cn=Directory Manager,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize: <cn=Directory Manager,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize: <cn=directory manager,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnPretty: <uid=someuser,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnPretty: <uid=someuser,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize: <uid=someuser,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize: <uid=someuser,dc=peak,dc=org> Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnPretty: <cn=Subschema> Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnPretty: <cn=Subschema> Mar 1 17:43:34 ldap04 slapd[6242]: >>> dnNormalize: <cn=Subschema> Mar 1 17:43:34 ldap04 slapd[6242]: <<< dnNormalize: <cn=subschema> Mar 1 17:43:34 ldap04 slapd[6242]: syncrepl_entry: rid 314 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD) Mar 1 17:43:34 ldap04 slapd[6242]: => bdb_search Mar 1 17:43:34 ldap04 slapd[6242]: bdb_dn2entry("dc=peak,dc=org") Mar 1 17:43:34 ldap04 slapd[6242]: search_candidates: base="dc=peak,dc=org" (0x00000001) scope=2 Mar 1 17:43:34 ldap04 slapd[6242]: => bdb_dn2idl("dc=peak,dc=org") Mar 1 17:43:34 ldap04 slapd[6242]: => bdb_filter_candidates Mar 1 17:43:34 ldap04 slapd[6242]: AND Mar 1 17:43:35 ldap04 slapd[6242]: => bdb_list_candidates 0xa0 Mar 1 17:43:35 ldap04 slapd[6242]: => bdb_filter_candidates Mar 1 17:43:35 ldap04 slapd[6242]: EQUALITY Mar 1 17:43:35 ldap04 slapd[6242]: => bdb_equality_candidates (entryUUID) Mar 1 17:43:35 ldap04 slapd[6242]: => key_read Mar 1 17:43:35 ldap04 slapd[6242]: bdb_idl_fetch_key: [15cb4950] Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_index_read 1 candidates Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_equality_candidates: id=1, first=169, last=169 Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_filter_candidates: id=1 first=169 last=169 Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_list_candidates: id=1 first=169 last=169 Mar 1 17:43:35 ldap04 slapd[6242]: <= bdb_filter_candidates: id=1 first=169 last=169 Mar 1 17:43:35 ldap04 slapd[6242]: bdb_search_candidates: id=1 first=169 last=169 Mar 1 17:43:35 ldap04 slapd[6242]: => test_filter Mar 1 17:43:35 ldap04 slapd[6242]: EQUALITY Mar 1 17:43:35 ldap04 slapd[6242]: => access_allowed: search access to "uid=someuser,dc=peak,dc=org" "entryUUID" requested Mar 1 17:43:35 ldap04 slapd[6242]: <= root access granted Mar 1 17:43:35 ldap04 slapd[6242]: <= test_filter 6 Mar 1 17:43:35 ldap04 slapd[6242]: send_ldap_result: conn=-1 op=0 p=3 Mar 1 17:43:35 ldap04 slapd[6242]: send_ldap_result: err=0 matched="" text="" Mar 1 17:43:35 ldap04 slapd[6242]: syncrepl_entry: rid 314 be_search (0) Mar 1 17:43:35 ldap04 slapd[6242]: syncrepl_entry: rid 314 uid=someuser,dc=peak,dc=org
On Mon, 1 Mar 2010, Alan Batie wrote:
I'm a little puzzled by the openldap replication docs; in particular rid, entryuuid, entrycsn, contextcsn fields that I see referenced a lot.
[...]
Basically, I'd like to understand the replication process at a slightly higher level than http://www.openldap.org/doc/admin24/syncrepl.html describes not quite completely enough...
In addition to the Admin Guide, I suggest reading RFC4533 and the other material in the doc/ directory of the distribution. Also, even if you're not particularly adept at C, you may wish to read the comments in syncprov.c source file. The terms you're asking about are the names that OpenLDAP uses in its implementation; hopefully with the RFC in hand it will become clear how the concepts map to the names, and you can always ask if anything is still unclear.
I'm guessing the rid is a random chosen id number for the secondary server (consumer?) that is used to compare the master db (entryuuid?) for context? information that indicates sync state?
rid is chosen by administrator (i.e. manually configured). I honestly forget what it's used for nowadays...I vaguely remember "nothing" and/or "it's only important for multimaster." I'm certain this is in the archives of openldap-software and/or openldap-devel, but I'll leave the search to you.
The specific task at hand is trying to understand what I'm seeing in the logs, having setup a master, loaded the db, configured replication, setup a fresh secondary mirror and turned on replication there. I let it run over the weekend, and see debug logs of what I assume is polling on the secondary, but 'slapcat | grep -c "dn:"' returns (and has been for a while) a suspiciously round number of "500" (there are around 38,000 records in the master), though I don't see any such limit anywhere in the configs.
My take on this would be that, if you turned up debugging and/or created a packet capture, you will find LDAP_SIZELIMIT_EXCEEDED returned by the provider in response to the consumer search request. As for not seeing the limit, please read slapd.conf(5) man page thoroughly.
master config:
overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
That take above would be more precise if you gave the unabridged provider config...
--On Tuesday, March 02, 2010 1:49 PM -0500 Aaron Richton richton@nbcs.rutgers.edu wrote:
I'm guessing the rid is a random chosen id number for the secondary server (consumer?) that is used to compare the master db (entryuuid?) for context? information that indicates sync state?
rid is chosen by administrator (i.e. manually configured). I honestly forget what it's used for nowadays...I vaguely remember "nothing" and/or "it's only important for multimaster." I'm certain this is in the archives of openldap-software and/or openldap-devel, but I'll leave the search to you.
The RID value is a unique number in a given replica used to specifically identify that particular syncrepl statement. It really only matters if you have multiple syncrepl statements in a given slapd configuration for a replica. For me, since I only replicate from a single master, I use the same RID value on all my replicas.
With MMR, there is also the SID, which may be the other part you are thinking of, but if you aren't using MMR, it doesn't apply.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Aaron Richton wrote:
On Mon, 1 Mar 2010, Alan Batie wrote:
I'm a little puzzled by the openldap replication docs; in particular rid, entryuuid, entrycsn, contextcsn fields that I see referenced a lot.
[...]
Basically, I'd like to understand the replication process at a slightly higher level than http://www.openldap.org/doc/admin24/syncrepl.html describes not quite completely enough...
In addition to the Admin Guide, I suggest reading RFC4533 and the other material in the doc/ directory of the distribution. Also, even if you're not particularly adept at C, you may wish to read the comments in syncprov.c source file. The terms you're asking about are the names that OpenLDAP uses in its implementation; hopefully with the RFC in hand it will become clear how the concepts map to the names, and you can always ask if anything is still unclear.
I'm guessing the rid is a random chosen id number for the secondary server (consumer?) that is used to compare the master db (entryuuid?) for context? information that indicates sync state?
rid is chosen by administrator (i.e. manually configured). I honestly forget what it's used for nowadays...I vaguely remember "nothing" and/or "it's only important for multimaster." I'm certain this is in the archives of openldap-software and/or openldap-devel, but I'll leave the search to you.
Like the manpage says:
rid identifies the current syncrepl directive within the replication consumer site. <<<
It's only use is to give the slapd -c option something to reference. Nothing else.
openldap-software@openldap.org
-
Aaron Richton -
Alan Batie -
Howard Chu -
Quanah Gibson-Mount