Hi Quanah,
Thanks again for your answer.
From what you have written, we understand now that we should not aim for four identical timestamps in contextCSN attributes on each node. As contextCSN is updated (as you said) only when a server receives a direct write. (and NOT for writes received through replication)
So, as long as the contextCSN output across all four MM-ldap nodes is identical, replication is doing it's thing.
Specifically, from my original post:
contextCSN: 20230622151102.137085Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230627112536.529432Z#000000#0dd#000000 contextCSN: 20230627112536.529512Z#000000#0de#000000
The first two servers (001/002) are just as up-to-date as the last two, only: more recent writes have come into the cluster through the 0de/0dd servers.
I have written a script to compare actual ldap served content, and that confirms that they all serve the same data.
If any of the above is wrong, I'd appreciate to be corrected :-)
On Wed, 5 Jul 2023 at 17:09, Quanah Gibson-Mount quanah@fast-mail.org wrote:
--On Wednesday, July 5, 2023 11:17 AM +0200 cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
serverID 222 ldaps://ldapm01.company.com serverID 221 ldaps://ldapm02.company.com serverID 212 ldaps://ldapm03.company.com serverID 211 ldaps://ldapm04.company.com
And when I (quick and dirty) check contextCSN using slapcat (I know that ldapsearch is the better way) I get:
[root@ldapm04 ~]# slapcat | grep contextcsn -i contextCSN: 20180917142109.765066Z#000000#000#000000 contextCSN: 20230622151102.137085Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230627112536.529432Z#000000#0dd#000000 contextCSN: 20230627112536.529512Z#000000#0de#000000
The first 2018 contextCSN is irrelevant (it has alwas been there, and I should probably try to get rid of it) but the last four seem to be the "actual" configured replication 'lines' on each node to the other three, like this:
Yet: all info I can find tells me that the third field of contextCSN is "the SID".
Can anyone explain? Are they perhaps HEX? If why the large gap between to consequtive pairs..?
Hi!
So #000# is as you noted an old SID from when the system used single provider.
The SID values are indeed in hex. Your current non-zero SIDS indicate:
You have servers with SID values 001, 002 that last received a direct write operation on 2023/06/22, about 3 seconds apart.
You have servers with SID values 221 (0dd) and 222 (0de) that last received a direct write operation on 2023/06/27, during the same second.
Any server that has a different SID value that has never received a direct write operation will not appear in the list.
Regards, Quanah
--On Thursday, July 6, 2023 3:44 PM +0200 cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Hi Quanah,
Thanks again for your answer.
From what you have written, we understand now that we should not aim for four identical timestamps in contextCSN attributes on each node. As contextCSN is updated (as you said) only when a server receives a direct write. (and NOT for writes received through replication)
So, as long as the contextCSN output across all four MM-ldap nodes is identical, replication is doing it's thing.
Specifically, from my original post:
contextCSN: 20230622151102.137085Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230627112536.529432Z#000000#0dd#000000 contextCSN: 20230627112536.529512Z#000000#0de#000000
The first two servers (001/002) are just as up-to-date as the last two, only: more recent writes have come into the cluster through the 0de/0dd servers.
I have written a script to compare actual ldap served content, and that confirms that they all serve the same data.
If any of the above is wrong, I'd appreciate to be corrected :-)
If you were using delta-syncrepl (I see from your configs you are not) you'd also need to do comparisons on the CSNs in the accesslog DB.
If you use the Symas OpenLDAP packages, there's also a nice utility called slapd-watcher that shows you the status of your servers included in their builds.
--Quanah
Hi Quanah!
Did NOT know that tool!
Thanks again! Great info.
MJ
Op 06-07-2023 om 22:06 schreef Quanah Gibson-Mount:
--On Thursday, July 6, 2023 3:44 PM +0200 cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Hi Quanah,
Thanks again for your answer.
From what you have written, we understand now that we should not aim for four identical timestamps in contextCSN attributes on each node. As contextCSN is updated (as you said) only when a server receives a direct write. (and NOT for writes received through replication)
So, as long as the contextCSN output across all four MM-ldap nodes is identical, replication is doing it's thing.
Specifically, from my original post:
contextCSN: 20230622151102.137085Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230627112536.529432Z#000000#0dd#000000 contextCSN: 20230627112536.529512Z#000000#0de#000000
The first two servers (001/002) are just as up-to-date as the last two, only: more recent writes have come into the cluster through the 0de/0dd servers.
I have written a script to compare actual ldap served content, and that confirms that they all serve the same data.
If any of the above is wrong, I'd appreciate to be corrected :-)
If you were using delta-syncrepl (I see from your configs you are not) you'd also need to do comparisons on the CSNs in the accesslog DB.
If you use the Symas OpenLDAP packages, there's also a nice utility called slapd-watcher that shows you the status of your servers included in their builds.
--Quanah
Follow-up question on slapd-watcher.
In a youtube video (https://www.youtube.com/watch?v=rYW2jQnS_PM) I can see that slapd-watcher can display contextCSN replication info, real time as it happens, but I'm not getting that info.
Starting it like:
slapd-watcher -i 1 \ -b o=company,c=com \ -D cn=monitoring,cn=Monitor \ -w abc...xyz \ ldaps://ldap01.company.com \ ldaps://ldap02.company.com \ ldaps://ldap03.company.com \ ldaps://ldap04.company.com \
I also tried adding "-s 222 221 212 211" (and also (learning from this list) :-) in HEX format) but it only gives: "Number of sids doesn't equal number of server URLs" (same result with four separate -s options) (for the record: the number of SIDs DID match the number of server URLs: four)
This is openldap 2.5, symas packages, RHEL9, in a 4-host multimaster setup
Can anyone explain what is required to also get the real-time replication info?
(in the youtube video I see the command is issued even more basic, with just -b -i and 4 server URI)
And should I file an issue on the "Number of sids doesn't equal number of server URLs"? (as the number DID match)
On Thu, 6 Jul 2023 at 22:17, sacawulu cyusedfzfb@gmail.com wrote:
Hi Quanah!
Did NOT know that tool!
Thanks again! Great info.
MJ
Op 06-07-2023 om 22:06 schreef Quanah Gibson-Mount:
--On Thursday, July 6, 2023 3:44 PM +0200 cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Hi Quanah,
Thanks again for your answer.
From what you have written, we understand now that we should not aim for four identical timestamps in contextCSN attributes on each node. As contextCSN is updated (as you said) only when a server receives a direct write. (and NOT for writes received through replication)
So, as long as the contextCSN output across all four MM-ldap nodes is identical, replication is doing it's thing.
Specifically, from my original post:
contextCSN: 20230622151102.137085Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230627112536.529432Z#000000#0dd#000000 contextCSN: 20230627112536.529512Z#000000#0de#000000
The first two servers (001/002) are just as up-to-date as the last two, only: more recent writes have come into the cluster through the 0de/0dd servers.
I have written a script to compare actual ldap served content, and that confirms that they all serve the same data.
If any of the above is wrong, I'd appreciate to be corrected :-)
If you were using delta-syncrepl (I see from your configs you are not) you'd also need to do comparisons on the CSNs in the accesslog DB.
If you use the Symas OpenLDAP packages, there's also a nice utility called slapd-watcher that shows you the status of your servers included in their builds.
--Quanah
Follow-up, some progress.
Using our o=company,c=com rootDN I do get some replication info, but only two lines of them, and not the four that seem to be the current *actual* configured RIDs.
The four (current) contextCSN's are displayed with slapcat and ldapsearch:
dn: o=company,c=com contextCSN: 20180917142109.725361Z#000000#000#000000 contextCSN: 20230622022112.071635Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230706131809.608140Z#000000#0d3#000000 contextCSN: 20230706131825.932137Z#000000#0d4#000000 contextCSN: 20230707083130.119957Z#000000#0dd#000000 contextCSN: 20230706132303.902949Z#000000#0de#000000
Tips?
On Fri, 7 Jul 2023 at 10:19, cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Follow-up question on slapd-watcher.
In a youtube video (https://www.youtube.com/watch?v=rYW2jQnS_PM) I can see that slapd-watcher can display contextCSN replication info, real time as it happens, but I'm not getting that info.
Starting it like:
slapd-watcher -i 1 \ -b o=company,c=com \ -D cn=monitoring,cn=Monitor \ -w abc...xyz \ ldaps://ldap01.company.com \ ldaps://ldap02.company.com \ ldaps://ldap03.company.com \ ldaps://ldap04.company.com \
I also tried adding "-s 222 221 212 211" (and also (learning from this list) :-) in HEX format) but it only gives: "Number of sids doesn't equal number of server URLs" (same result with four separate -s options) (for the record: the number of SIDs DID match the number of server URLs: four)
This is openldap 2.5, symas packages, RHEL9, in a 4-host multimaster setup
Can anyone explain what is required to also get the real-time replication info?
(in the youtube video I see the command is issued even more basic, with just -b -i and 4 server URI)
And should I file an issue on the "Number of sids doesn't equal number of server URLs"? (as the number DID match)
On Thu, 6 Jul 2023 at 22:17, sacawulu cyusedfzfb@gmail.com wrote:
Hi Quanah!
Did NOT know that tool!
Thanks again! Great info.
MJ
Op 06-07-2023 om 22:06 schreef Quanah Gibson-Mount:
--On Thursday, July 6, 2023 3:44 PM +0200 cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Hi Quanah,
Thanks again for your answer.
From what you have written, we understand now that we should not aim
for
four identical timestamps in contextCSN attributes on each node. As contextCSN is updated (as you said) only when a server receives a
direct
write. (and NOT for writes received through replication)
So, as long as the contextCSN output across all four MM-ldap nodes is identical, replication is doing it's thing.
Specifically, from my original post:
contextCSN: 20230622151102.137085Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230627112536.529432Z#000000#0dd#000000 contextCSN: 20230627112536.529512Z#000000#0de#000000
The first two servers (001/002) are just as up-to-date as the last two, only: more recent writes have come into the cluster through the 0de/0dd servers.
I have written a script to compare actual ldap served content, and that confirms that they all serve the same data.
If any of the above is wrong, I'd appreciate to be corrected :-)
If you were using delta-syncrepl (I see from your configs you are not) you'd also need to do comparisons on the CSNs in the accesslog DB.
If you use the Symas OpenLDAP packages, there's also a nice utility called slapd-watcher that shows you the status of your servers included in their builds.
--Quanah
Solution:
the -s SID has to be comma separated for multiple values. Space separation gives "Number of sids doesn't equal number of server URLs"
Comma separation also displays the expected contextCSN replication lines.
So... all solved :-)
On Fri, 7 Jul 2023 at 10:38, cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Follow-up, some progress.
Using our o=company,c=com rootDN I do get some replication info, but only two lines of them, and not the four that seem to be the current *actual* configured RIDs.
The four (current) contextCSN's are displayed with slapcat and ldapsearch:
dn: o=company,c=com contextCSN: 20180917142109.725361Z#000000#000#000000 contextCSN: 20230622022112.071635Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230706131809.608140Z#000000#0d3#000000 contextCSN: 20230706131825.932137Z#000000#0d4#000000 contextCSN: 20230707083130.119957Z#000000#0dd#000000 contextCSN: 20230706132303.902949Z#000000#0de#000000
Tips?
On Fri, 7 Jul 2023 at 10:19, cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Follow-up question on slapd-watcher.
In a youtube video (https://www.youtube.com/watch?v=rYW2jQnS_PM) I can see that slapd-watcher can display contextCSN replication info, real time as it happens, but I'm not getting that info.
Starting it like:
slapd-watcher -i 1 \ -b o=company,c=com \ -D cn=monitoring,cn=Monitor \ -w abc...xyz \ ldaps://ldap01.company.com \ ldaps://ldap02.company.com \ ldaps://ldap03.company.com \ ldaps://ldap04.company.com \
I also tried adding "-s 222 221 212 211" (and also (learning from this list) :-) in HEX format) but it only gives: "Number of sids doesn't equal number of server URLs" (same result with four separate -s options) (for the record: the number of SIDs DID match the number of server URLs: four)
This is openldap 2.5, symas packages, RHEL9, in a 4-host multimaster setup
Can anyone explain what is required to also get the real-time replication info?
(in the youtube video I see the command is issued even more basic, with just -b -i and 4 server URI)
And should I file an issue on the "Number of sids doesn't equal number of server URLs"? (as the number DID match)
On Thu, 6 Jul 2023 at 22:17, sacawulu cyusedfzfb@gmail.com wrote:
Hi Quanah!
Did NOT know that tool!
Thanks again! Great info.
MJ
Op 06-07-2023 om 22:06 schreef Quanah Gibson-Mount:
--On Thursday, July 6, 2023 3:44 PM +0200 cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Hi Quanah,
Thanks again for your answer.
From what you have written, we understand now that we should not aim
for
four identical timestamps in contextCSN attributes on each node. As contextCSN is updated (as you said) only when a server receives a
direct
write. (and NOT for writes received through replication)
So, as long as the contextCSN output across all four MM-ldap nodes is identical, replication is doing it's thing.
Specifically, from my original post:
contextCSN: 20230622151102.137085Z#000000#001#000000 contextCSN: 20230622151105.174343Z#000000#002#000000 contextCSN: 20230627112536.529432Z#000000#0dd#000000 contextCSN: 20230627112536.529512Z#000000#0de#000000
The first two servers (001/002) are just as up-to-date as the last
two,
only: more recent writes have come into the cluster through the
0de/0dd
servers.
I have written a script to compare actual ldap served content, and
that
confirms that they all serve the same data.
If any of the above is wrong, I'd appreciate to be corrected :-)
If you were using delta-syncrepl (I see from your configs you are not) you'd also need to do comparisons on the CSNs in the accesslog DB.
If you use the Symas OpenLDAP packages, there's also a nice utility called slapd-watcher that shows you the status of your servers
included
in their builds.
--Quanah
--On Monday, July 10, 2023 1:02 PM +0200 cYuSeDfZfb cYuSeDfZfb cyusedfzfb@gmail.com wrote:
Solution:
the -s SID has to be comma separated for multiple values. Space separation gives "Number of sids doesn't equal number of server URLs"
Comma separation also displays the expected contextCSN replication lines.
So... all solved :-)
Excellent! I've been busy working on the releases plus my usual work so didn't get a chance to follow up. Glad you resolved it. :)
--Quanah
openldap-technical@openldap.org
-
cYuSeDfZfb cYuSeDfZfb -
Quanah Gibson-Mount -
sacawulu