Hi Quanah,
Thanks for your answer. Here are the details asked:
OpenLDAP version 2.4.46.
Mirror configuration on node A:
overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
# Global section serverID 1 # database section
# syncrepl directive syncrepl rid=001 provider=ldaps://02.host.com bindmethod=simple binddn="cn=root,dc=xxx" credentials=xxx searchbase="dc=xxx" schemachecking=on type=refreshAndPersist retry="10 +" tls_cacert=/usr/local/etc/openldap/cert/cert02.crt
mirrormode on
Mirror configuration on node B: overlay syncprov syncprov-checkpoint 100 10 syncprov-sessionlog 100
# Global section serverID 2 # database section
# syncrepl directive syncrepl rid=001 provider=ldaps://01.host.com bindmethod=simple binddn="cn=root,dc=xxx" credentials=xxx searchbase="dc=xxx" schemachecking=on type=refreshAndPersist retry="10 +" tls_cacert=/usr/local/etc/openldap/cert/cert01.crt
mirrormode on
Thank you very much.
On Tue, Apr 2, 2019 at 7:02 PM Quanah Gibson-Mount quanah@symas.com wrote:
--On Tuesday, April 02, 2019 4:59 PM -0300 Alex Hebra hebraalex@gmail.com
wrote:
Hi there,
I have a very critical OpenLDAP environment running on mirror mode configuration. Is has about 800.000 users.
Hello,
You failed to provide any information on the release of OpenLDAP you are using and did not provide any information on the replication mechanism in use. By the sound of it, you are using standard syncrepl, where this sort of problem is not uncommon, particularly with older versions of OpenLDAP. I would suggest you start by noting the version of OpenLDAP in use and providing information on the setup and configuration of the multimaster nodes.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Wednesday, April 03, 2019 11:14 AM -0300 Alex Hebra hebraalex@gmail.com wrote:
Thanks for your answer. Here are the details asked:
OpenLDAP version 2.4.46.
syncprov-sessionlog 100 syncprov-sessionlog 100
Hi Alex,
Your sessionlog value is way too small. It needs to be something a little larger than the entire size of your database (so > 800,000 in your case). I also see you're using standard syncrepl rather than delta-syncrepl.
Generally, having a "glued" object on a server indicates that at some point, the server was in a REFRESH mode (either you populated the server using syncrepl instead of slapcat, or it had some reason to fall back to REFRESH at a later date).
In the REFRESH mode, the server may get entries "out of order". In this case, a stub (glue) object is created to "hold" the place for tha entry until it gets fully replicated. Unfortunately, this doesn't always happen, and you end up being stuck with the glued object on the server.
You may be able to force replication of the object onto the server where it's missing by doing a MOD op against the affected entry on the master where it exists. If one master is "complete" (I.e., no glued objects), you could also slapcat that master and import the database into the othe rmaster via slapadd to resolve the problem for now.
Overally, I strongly advise using delta-syncrepl as it largely avoids these issues (as long as it never has to fall back to standard syncrepl REFRESH mode).
I would note that there was a fix in OpenLDAP 2.4.47 specific to delta-syncrepl when ppolicy is in use, so if using delta-sync you likely want that fix.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On 4/3/19 6:09 PM, Quanah Gibson-Mount wrote:
--On Wednesday, April 03, 2019 11:14 AM -0300 Alex Hebra hebraalex@gmail.com wrote:
syncprov-sessionlog 100 syncprov-sessionlog 100
Your sessionlog value is way too small. It needs to be something a little larger than the entire size of your database (so > 800,000 in your case).
Size of database means number of entries?
Ciao, Michael.
--On Wednesday, April 03, 2019 9:19 PM +0200 Michael Ströder michael@stroeder.com wrote:
On 4/3/19 6:09 PM, Quanah Gibson-Mount wrote:
--On Wednesday, April 03, 2019 11:14 AM -0300 Alex Hebra hebraalex@gmail.com wrote:
syncprov-sessionlog 100 syncprov-sessionlog 100
Your sessionlog value is way too small. It needs to be something a little larger than the entire size of your database (so > 800,000 in your case).
Size of database means number of entries?
Yes.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Quanah Gibson-Mount quanah@symas.com schrieb am 03.04.2019 um 21:00 in
Nachricht <5C23F01B4C24C3A8ACBCCE7C@[192.168.1.39]>:
--On Wednesday, April 03, 2019 9:19 PM +0200 Michael Ströder michael@stroeder.com wrote:
On 4/3/19 6:09 PM, Quanah Gibson-Mount wrote:
--On Wednesday, April 03, 2019 11:14 AM -0300 Alex Hebra hebraalex@gmail.com wrote:
syncprov-sessionlog 100 syncprov-sessionlog 100
Your sessionlog value is way too small. It needs to be something a little larger than the entire size of your database (so > 800,000 in your case).
Size of database means number of entries?
Yes.
Hi!
May I ask what the problems are if the value is less? Do changes get lost if more changes were done than session log can hold while sync can't keep pace? Or does the server getting more changes simply block until there's spece in the sessionlog?
Regards, Ulrich
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
--On Thursday, April 04, 2019 9:15 AM +0200 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
May I ask what the problems are if the value is less? Do changes get lost if more changes were done than session log can hold while sync can't keep pace? Or does the server getting more changes simply block until there's spece in the sessionlog?
The server doesn't block anything. The session log is just a list of the IDs of the entries that changed. When a new one is added to the end of the list, the oldest one is dropped from the start of the list.
The reason it needs to be so large is to avoid the issues described in https://www.openldap.org/its/index.cgi/?findid=8125
Unfortunately, the session log (currently) does not persist across restarts of slapd, so it is still possible to hit ITS#8125 with "standard" syncrepl configurations. Thus one of the reasons I always advise to use delta-syncrepl.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On Wed, Apr 03, 2019 at 09:09:47 -0700, Quanah Gibson-Mount wrote:
Your sessionlog value is way too small. It needs to be something a little larger than the entire size of your database (so > 800,000 in your case).
Hi Quanah
This means https://bugzilla.zimbra.com/show_bug.cgi?id=106848 no longer applies? We were hit by this bug at the time, so we still have ldap_overlay_syncprov_sessionlog=15000 here.
Geert
--On Wednesday, April 03, 2019 10:54 PM +0200 Geert Hendrickx geert@hendrickx.be wrote:
On Wed, Apr 03, 2019 at 09:09:47 -0700, Quanah Gibson-Mount wrote:
Your sessionlog value is way too small. It needs to be something a little larger than the entire size of your database (so > 800,000 in your case).
Hi Quanah
This means https://bugzilla.zimbra.com/show_bug.cgi?id=106848 no longer applies? We were hit by this bug at the time, so we still have ldap_overlay_syncprov_sessionlog=15000 here.
Hi Geert,
I don't know what version of OpenLDAP ships with Zimbra these days. The underlying issue was fixed in OpenLDAP 2.4.46 (ITS#8486).
Fixed slapo-syncprov session log parsing to not block other operations (ITS#8486)
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
On Wed, Apr 03, 2019 at 12:57:51 -0700, Quanah Gibson-Mount wrote:
I don't know what version of OpenLDAP ships with Zimbra these days.
I think you know it's 2.4.46 :-)
The underlying issue was fixed in OpenLDAP 2.4.46 (ITS#8486).
Thanks for confirmation, we'll revert to a larger sessionlog size.
Cheers!
Geert
--On Wednesday, April 03, 2019 11:03 PM +0200 Geert Hendrickx geert@hendrickx.be wrote:
On Wed, Apr 03, 2019 at 12:57:51 -0700, Quanah Gibson-Mount wrote:
I don't know what version of OpenLDAP ships with Zimbra these days.
I think you know it's 2.4.46 :-)
I honestly haven't tracked it. ;)
Hopefully you don't hit ITS#8843 (For any MMR deployment with > 2 nodes). Maybe Zimbra has patched this one in?
Hopefully you don't hit ITS#8990. This issue is primarily limited to users of current versions of python-ldap where they execute a MOD/REPLACE operation because of the way in which newer versions of python-ldap re-write the operation into a del+add instead of a replace op. I throw this one in because I know you have OpenLDAP servers that are not Zimbra based. ;) Unfortunately this issue existed all the way back to 2011, but didn't become an issue really until python-ldap changed how it handles these types of changes.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
Alex Hebra -
Geert Hendrickx -
Michael Ströder -
Quanah Gibson-Mount -
Ulrich Windl