Thank you Quanah. This helps a lot. Let me make the changes.
Thanks Rahul
On Mon, Jul 6, 2020 at 5:34 PM Quanah Gibson-Mount quanah@symas.com wrote:
--On Monday, July 6, 2020 6:21 PM -0400 kumar rahul rahul2002mit@gmail.com wrote:
Hi Quanah
I have some questions from update_config.ldif file
Please don't attach screenshots of text. If you have questions, use text email.
In general, however:
One database is used as the main database for the system. Lets call this database A. One database is used to store the accesslog data (the change deltas). Lets call this database B. There may be other databases on the system (such as the monitor database). Lets call this database C.
Both the syncprov and accesslog *overlays* must be configured on database A with the appropriate configuration options.
The syncprov overlay must *also* be configured on database B with the appropriate (but different) configuration options to indicate it's an accesslog DB.
In cn=config, every database has a weight that indicates the order in which it loads. So:
olcDatabase={1}... is the first DB to get loaded by slapd (Ignoring {-1} and {0} which always exist, and are the frontend and config DBs respectively).
olcDatabase={2}... would be the second DB to get loaded by slapd
olcDatabase={3}... would be the third DB to get loaded by slapd
Additionally, overlays have weights associated with them to indicate the order in which they get loaded. So:
olcOverlay={1}...,olcDatabase={2}... means that this overlay is the first one that gets loaded for database #2.
In the EXAMPLE I provided on my blog:
olcDatabase={1} is the monitoring database (database C above)
olcDatabase={2} is the ACCESSLOG database (database B above)
olcDatabase={3} is the MAIN database (database A above)
So which overlays need to apply to which databases depends ENTIRELY ON THE ORDER in which they are defined in *YOUR* configuration. You need to make adjustments accordingly.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Hi Quanah Based on your blog and information you provided I was able to set up delta syncrepl in my environment. I am not seeing the sync issue as well. Thank you so much for all the help.
I do have couple of questions
1) What is the recommended value for olcSpCheckpoint? As per your blog value is olcSpCheckpoint: 20 10 2) What is the recommended value of olcAccessLogPurge? As per your blog value is olcAccessLogPurge: 01+00:00 00+04:00
Thanks Rahul
On Mon, Jul 6, 2020 at 5:39 PM kumar rahul rahul2002mit@gmail.com wrote:
Thank you Quanah. This helps a lot. Let me make the changes.
Thanks Rahul
On Mon, Jul 6, 2020 at 5:34 PM Quanah Gibson-Mount quanah@symas.com wrote:
--On Monday, July 6, 2020 6:21 PM -0400 kumar rahul rahul2002mit@gmail.com wrote:
Hi Quanah
I have some questions from update_config.ldif file
Please don't attach screenshots of text. If you have questions, use text email.
In general, however:
One database is used as the main database for the system. Lets call this database A. One database is used to store the accesslog data (the change deltas). Lets call this database B. There may be other databases on the system (such as the monitor database). Lets call this database C.
Both the syncprov and accesslog *overlays* must be configured on database A with the appropriate configuration options.
The syncprov overlay must *also* be configured on database B with the appropriate (but different) configuration options to indicate it's an accesslog DB.
In cn=config, every database has a weight that indicates the order in which it loads. So:
olcDatabase={1}... is the first DB to get loaded by slapd (Ignoring {-1} and {0} which always exist, and are the frontend and config DBs respectively).
olcDatabase={2}... would be the second DB to get loaded by slapd
olcDatabase={3}... would be the third DB to get loaded by slapd
Additionally, overlays have weights associated with them to indicate the order in which they get loaded. So:
olcOverlay={1}...,olcDatabase={2}... means that this overlay is the first one that gets loaded for database #2.
In the EXAMPLE I provided on my blog:
olcDatabase={1} is the monitoring database (database C above)
olcDatabase={2} is the ACCESSLOG database (database B above)
olcDatabase={3} is the MAIN database (database A above)
So which overlays need to apply to which databases depends ENTIRELY ON THE ORDER in which they are defined in *YOUR* configuration. You need to make adjustments accordingly.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Friday, July 10, 2020 11:04 AM -0400 kumar rahul rahul2002mit@gmail.com wrote:
- What is the recommended value for olcSpCheckpoint?
As per your blog value is olcSpCheckpoint: 20 10
I prefer to have it checkpoint fairly frequently, which is why I chose those values.
- What is the recommended value of olcAccessLogPurge?
As per your blog value is olcAccessLogPurge: 01+00:00 00+04:00
So this depends specifically on the environment in question. The more data retained, the further back a replica can "catch up" to the provider(s) by using delta-syncrepl. However, the more data retained, the larger the database will be. Whether or not one wants a replica to be able to "catch up" to a provider if it is several days behind vs reloading it with a fresh copy of the db from the provider is the real question.
Regardless of the data retention period, I always suggest a frequent purge interval. This is because slapd essentially pauses while a purge is ongoing. For most environments, a 4 hour purge interval is not noticable. I've gone to as frequent as every 2 hours for extremely active sites.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Quanah Gibson-Mount quanah@symas.com schrieb am 14.07.2020 um 17:28 in
Nachricht <13BD7019D5690162BCFE9056@[192.168.0.156]>:
‑‑On Friday, July 10, 2020 11:04 AM ‑0400 kumar rahul rahul2002mit@gmail.com wrote:
- What is the recommended value for olcSpCheckpoint?
As per your blog value is olcSpCheckpoint: 20 10
I prefer to have it checkpoint fairly frequently, which is why I chose those values.
- What is the recommended value of olcAccessLogPurge?
As per your blog value is olcAccessLogPurge: 01+00:00 00+04:00
So this depends specifically on the environment in question. The more data
retained, the further back a replica can "catch up" to the provider(s) by using delta‑syncrepl. However, the more data retained, the larger the database will be. Whether or not one wants a replica to be able to "catch up" to a provider if it is several days behind vs reloading it with a fresh
copy of the db from the provider is the real question.
Regardless of the data retention period, I always suggest a frequent purge interval. This is because slapd essentially pauses while a purge is ongoing. For most environments, a 4 hour purge interval is not noticable. I've gone to as frequent as every 2 hours for extremely active sites.
Is there a specific log token that lets slapd log the duration (and occurrence) of purge events without flooding the logs with all other types of messages?
Regards, Quanah
‑‑
Quanah Gibson‑Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Wednesday, July 15, 2020 10:15 AM +0200 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
Is there a specific log token that lets slapd log the duration (and occurrence) of purge events without flooding the logs with all other types of messages?
No, there is no logging around the purge activity at all. As long as the purge interval is frequent enough, it's not even noticeable.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Quanah Gibson-Mount quanah@symas.com schrieb am 15.07.2020 um 17:35 in
Nachricht <B48696AB8E5B21D1AD933492@[192.168.0.156]>:
‑‑On Wednesday, July 15, 2020 10:15 AM +0200 Ulrich Windl <Ulrich.Windl@rz.uni‑regensburg.de> wrote:
Is there a specific log token that lets slapd log the duration (and occurrence) of purge events without flooding the logs with all other types of messages?
No, there is no logging around the purge activity at all. As long as the purge interval is frequent enough, it's not even noticeable.
I think such a performance number could be interesting, especially if slapd refuses to process other requests meanwhile (as you said). In monitoring a "time of last purge" and "duration of last purge" could be useful. Maybe "number of objects/attributes/values purged" could be interesting as well.
Regards, Quanah
‑‑
Quanah Gibson‑Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
kumar rahul -
Quanah Gibson-Mount -
Ulrich Windl