2:46 a.m.
Hi,
I have an OpenLDAP server I'm upgrading from 2.4.58 to 2.6.0. The server is configured
with the syncprov plugin, with "olcSpNoPresent: TRUE".
Previously, on 2.4.58, we would never see any Present phase data originating from the
server. However, post upgrade, we are seeing this, and it's causing problems with the
consumers. I've tried looking at the code, but I must admit I got lost trying to trace
through the propagation of this flag.
Are there undocumented circumstances where olcSpNoPresent will not be obeyed, or is this a
regression?
Thanks!
Lisa
10:19 a.m.
--On Tuesday, March 8, 2022 10:46 AM +0000 lisa.parratt(a)bjss.com wrote:
Hi,
I have an OpenLDAP server I'm upgrading from 2.4.58 to 2.6.0. The server
is configured with the syncprov plugin, with "olcSpNoPresent: TRUE".
Hello,
You'd have to better describe your replication setup. The *only* time
no-present should be set to TRUE is when it's on top of an accesslog
database that's being used by delta-syncrepl, as explicitly stated in the
man page.
Regards,
Quanah
1:04 a.m.
Hi Quanah,
It’s not a replication setup. The consumer is a relatively ancient proprietary
implementation of the syncrepl protocol, over which I have very influence.
I believe the problem is syncprov.c lines 3335 and 3341, which appear to be setting
`do_present = SS_PRESENT;` while simultaneously ignoring the value of si->si_nopres.
Replacing the lines above with `if ( !si->si_nopres ) do_present = SS_PRESENT;` appears
to fix the problem for my use case, and is closer to how the logic worked in previous
versions of slapd.
Thanks!
Lisa
From: Quanah Gibson-Mount <quanah(a)fast-mail.org>
Date: Tuesday, 8 March 2022 at 18:19
To: Lisa Parratt <Lisa.Parratt(a)bjss.com>, openldap-technical(a)openldap.org
<openldap-technical(a)openldap.org>
Subject: Re: syncprov olcSpNoPresent not being honoured
--On Tuesday, March 8, 2022 10:46 AM +0000 lisa.parratt(a)bjss.com wrote:
Hi,
I have an OpenLDAP server I'm upgrading from 2.4.58 to 2.6.0. The server
is configured with the syncprov plugin, with "olcSpNoPresent: TRUE".
Hello,
You'd have to better describe your replication setup. The *only* time
no-present should be set to TRUE is when it's on top of an accesslog
database that's being used by delta-syncrepl, as explicitly stated in the
man page.
Regards,
Quanah
The information included in this email and any files transmitted with it may contain
information that is confidential and it must not be used by, or its contents or
attachments copied or disclosed to, persons other than the intended addressee. If you have
received this email in error, please notify BJSS. In the absence of written agreement to
the contrary BJSS' relevant standard terms of contract for any work to be undertaken
will apply. Please carry out virus or such other checks as you consider appropriate in
respect of this email. BJSS does not accept responsibility for any adverse effect upon
your system or data in relation to this email or any files transmitted with it. BJSS
Limited, a company registered in England and Wales (Company Number 2777575), VAT
Registration Number 613295452, Registered Office Address, 1 Whitehall Quay, Leeds, LS1
4HR
8:31 a.m.
On Wed, Mar 09, 2022 at 09:04:49AM +0000, Lisa Parratt wrote:
Hi Quanah,
It’s not a replication setup. The consumer is a relatively ancient
proprietary implementation of the syncrepl protocol, over which I have
very influence.
I believe the problem is syncprov.c lines 3335 and 3341, which appear
to be setting `do_present = SS_PRESENT;` while simultaneously ignoring
the value of si->si_nopres. Replacing the lines above with `if (
!si->si_nopres ) do_present = SS_PRESENT;` appears to fix the problem
for my use case, and is closer to how the logic worked in previous
versions of slapd.
Hi Lisa,
I find it strange that you maintain a sessionlog (or use accesslog to host
it) yet don't want to host a present/delete phase which is where they
are useful. Could you explain what you're trying to achieve with your
set up?
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
8:42 a.m.
Hi Ondřej,
A byzantine java application connects to the LDAP server, and somehow uses the information
presented to trigger API requests to business logic, eventually resulting in the data
being replicated to a relational database. If olcSpNoPresent is honoured, this works. It
it’s ignored, it dies with an error. I have, as yet, been unable to find an engineer who
can explain this behaviour to me, and this code is largely forgotten.
It seems reasonable to me that if I configure slapd to not do a Present phase, then it
shouldn’t do a Present phase? Am I wrong?
Thanks!
Lisa
From: Ondřej Kuzník <ondra(a)mistotebe.net>
Date: Wednesday, 9 March 2022 at 16:32
To: Lisa Parratt <Lisa.Parratt(a)bjss.com>
Cc: Quanah Gibson-Mount <quanah(a)fast-mail.org>, openldap-technical(a)openldap.org
<openldap-technical(a)openldap.org>
Subject: Re: syncprov olcSpNoPresent not being honoured
On Wed, Mar 09, 2022 at 09:04:49AM +0000, Lisa Parratt wrote:
Hi Quanah,
It’s not a replication setup. The consumer is a relatively ancient
proprietary implementation of the syncrepl protocol, over which I have
very influence.
I believe the problem is syncprov.c lines 3335 and 3341, which appear
to be setting `do_present = SS_PRESENT;` while simultaneously ignoring
the value of si->si_nopres. Replacing the lines above with `if (
!si->si_nopres ) do_present = SS_PRESENT;` appears to fix the problem
for my use case, and is closer to how the logic worked in previous
versions of slapd.
Hi Lisa,
I find it strange that you maintain a sessionlog (or use accesslog to host
it) yet don't want to host a present/delete phase which is where they
are useful. Could you explain what you're trying to achieve with your
set up?
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
The information included in this email and any files transmitted with it may contain
information that is confidential and it must not be used by, or its contents or
attachments copied or disclosed to, persons other than the intended addressee. If you have
received this email in error, please notify BJSS. In the absence of written agreement to
the contrary BJSS' relevant standard terms of contract for any work to be undertaken
will apply. Please carry out virus or such other checks as you consider appropriate in
respect of this email. BJSS does not accept responsibility for any adverse effect upon
your system or data in relation to this email or any files transmitted with it. BJSS
Limited, a company registered in England and Wales (Company Number 2777575), VAT
Registration Number 613295452, Registered Office Address, 1 Whitehall Quay, Leeds, LS1
4HR
11:23 a.m.
On Wed, Mar 09, 2022 at 04:42:17PM +0000, Lisa Parratt wrote:
Hi Ondřej,
A byzantine java application connects to the LDAP server, and somehow
uses the information presented to trigger API requests to business
logic, eventually resulting in the data being replicated to a
relational database. If olcSpNoPresent is honoured, this works. It
it’s ignored, it dies with an error. I have, as yet, been unable to
find an engineer who can explain this behaviour to me, and this code
is largely forgotten.
It seems reasonable to me that if I configure slapd to not do a
Present phase, then it shouldn’t do a Present phase? Am I wrong?
The question was, is there a reason why you also configure a sessionlog
when to me it seems you're saying you'd be perfectly happy without
(ignoring any deletes)?
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
1:48 a.m.
Hi Ondřej,
My understanding was a sessionlog was required to record delete events, to efficiently
generate a Delete phase? The server has been configured with `olcSpSessionlog: 10000`
since long before I inherited it, and the consumer seems to be processing the transmitted
Delete phase as expected.
Lisa
From: Ondřej Kuzník <ondra(a)mistotebe.net>
Date: Wednesday, 9 March 2022 at 19:23
To: Lisa Parratt <Lisa.Parratt(a)bjss.com>
Cc: Quanah Gibson-Mount <quanah(a)fast-mail.org>, openldap-technical(a)openldap.org
<openldap-technical(a)openldap.org>
Subject: Re: syncprov olcSpNoPresent not being honoured
On Wed, Mar 09, 2022 at 04:42:17PM +0000, Lisa Parratt wrote:
Hi Ondřej,
A byzantine java application connects to the LDAP server, and somehow
uses the information presented to trigger API requests to business
logic, eventually resulting in the data being replicated to a
relational database. If olcSpNoPresent is honoured, this works. It
it’s ignored, it dies with an error. I have, as yet, been unable to
find an engineer who can explain this behaviour to me, and this code
is largely forgotten.
It seems reasonable to me that if I configure slapd to not do a
Present phase, then it shouldn’t do a Present phase? Am I wrong?
The question was, is there a reason why you also configure a sessionlog
when to me it seems you're saying you'd be perfectly happy without
(ignoring any deletes)?
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
The information included in this email and any files transmitted with it may contain
information that is confidential and it must not be used by, or its contents or
attachments copied or disclosed to, persons other than the intended addressee. If you have
received this email in error, please notify BJSS. In the absence of written agreement to
the contrary BJSS' relevant standard terms of contract for any work to be undertaken
will apply. Please carry out virus or such other checks as you consider appropriate in
respect of this email. BJSS does not accept responsibility for any adverse effect upon
your system or data in relation to this email or any files transmitted with it. BJSS
Limited, a company registered in England and Wales (Company Number 2777575), VAT
Registration Number 613295452, Registered Office Address, 1 Whitehall Quay, Leeds, LS1
4HR
2:09 a.m.
On Thu, Mar 10, 2022 at 09:48:40AM +0000, Lisa Parratt wrote:
Hi Ondřej,
My understanding was a sessionlog was required to record delete
events, to efficiently generate a Delete phase? The server has been
configured with `olcSpSessionlog: 10000` since long before I inherited
it, and the consumer seems to be processing the transmitted Delete
phase as expected.
Hi Lisa,
it is required for that, but the fact your sessionlog isn't enough means
you had already been missing deletes and setting olcSpNoPresent just hid
that from view.
It could be argued that olcSpNoPresent and sessionlog are mutually
exclusive and we should reject that (or at least warn about it being
very suspicious), feel free to open an issue for this.
You might want to use olcSpSessionlogSource instead instead of the
in-memory implementation to make sure you don't get in this situation
even after a server restart. Maybe also keep olcSpNoPresent: FALSE to
alert you when you'd be losing information already and need to handle
that manually.
Regards,
--
Ondřej Kuzník
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
639
days inactive
641
days old
openldap-technical@openldap.org
7 comments
4 participants
participants (4)
-
Lisa Parratt -
lisa.parratt＠bjss.com
-
Ondřej Kuzník -
Quanah Gibson-Mount