Sorry, I refreshed my browser after creating #8717 and it re-posted and cre=
ated this one. Please close it as a duplicate of #8717.
> -----Original Message-----
> From: openldap-its(a)OpenLDAP.org [mailto:openldap-its@OpenLDAP.org]
> Sent: Friday, August 25, 2017 1:30 PM
> To: Ian Puleston
> Subject: Re: (ITS#8718) Connection delete callback registreed with
> LDAP_OPT_CONNECT_CB is not called on TLS failure
>=20
>=20
> *** THIS IS AN AUTOMATICALLY GENERATED REPLY ***
>=20
> Thanks for your report to the OpenLDAP Issue Tracking System. Your repor=
t
> has been assigned the tracking number ITS#8718.
>=20
> One of our support engineers will look at your report in due course.
> Note that this may take some time because our support engineers are
> volunteers. They only work on OpenLDAP when they have spare time.
>=20
> If you need to provide additional information in regards to your issue
> report, you may do so by replying to this message. Note that any mail se=
nt
> to openldap-its(a)openldap.org with (ITS#8718) in the subject will
> automatically be attached to the issue report.
>=20
> mailto:openldap-its@openldap.org?subject=3D(ITS#8718)
>=20
> You may follow the progress of this report by loading the following URL i=
n
> a web browser:
> http://www.OpenLDAP.org/its/index.cgi?findid=3D8718
>=20
> Please remember to retain your issue tracking number (ITS#8718) on any
> further messages you send to us regarding this report. If you don't then
> you'll just waste our time and yours because we won't be able to properly
> track the report.
>=20
> Please note that the Issue Tracking System is not intended to be used to
> seek help in the proper use of OpenLDAP Software.
> Such requests will be closed.
>=20
> OpenLDAP Software is user supported.
> http://www.OpenLDAP.org/support/
>=20
> --------------
> Copyright 1998-2007 The OpenLDAP Foundation, All Rights Reserved.
Full_Name: Ian Puleston
Version: 2.4.40
OS: VxWorks
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (204.118.31.9)
If connect and delete callbacks are registered via LDAP_OPT_CONNECT_CB, then the
lc_add callback is called when the TCP connects, but if TLS then fails to
connect the lc_del does not get called. If these callbacks are doing something
like allocating/freeing resources this can lead to a resource leak.
When I am seeing this, the sequence of calls leading to the lc_add callback is
as follows:
ldap_new_connection -> ldap_int_open_connection -> ldap_connect_to_host ->
ldap_int_connect_cbs()
Then if ldap_connect_to_host returned 0 for synchronous success this call
happens:
ldap_new_connection -> ldap_int_open_connection -> ldap_int_tls_start
And if that fails and returns something other than LDAP_SUCCESS then
ldap_int_open_connection exits. So we got a call to lc_add but no corresponding
call to lc_del.
And note that there is not call to ldap_free_connection, presumably because
ldap-new-connection did not succeed, hence the lc_del callback does not get
called from there.
In my case when I see this it is on a referral with the above calls made from
ldap_chase_v3referrals, hence it is happening synchronously. Whether the same
would happen with an asynchronous connect I don't know.
I do have a working fix which is to make the lc_del callbacks directly from
ldap_int_open_connection if ldap_int_tls_start fails. I will supply a patch for
that shortly.
This configuration should have been rejected by back-config just like it
is rejected by slapd.conf. Patch to have this happen is at:
ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170825-ITS8637.patch
This doesn't reject online modifications just touching olcDBURI, but I
think back ldap has no idea when it's configured as a slave to the chain
overlay, so it can't do much.
--=20
Ond=C5=99ej Kuzn=C3=ADk
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
On Thu, Aug 24, 2017 at 03:55:32PM +0100, Ond=C5=99ej Kuzn=C3=ADk wrote:
> I think it looks like this: syncproc_checkpoint modifies the suffix
> entry, that calls slap_graduate_commit_csn and the csn is removed from
> be_pending_csn_list. accesslog_response then can't find the CSN there
> and has nothing to insert into its own pending csn list. Strange that
> changing the overlay order (accesslog vs. syncprov) doesn't change this
> behaviour, something I'd expect if the above is the reason this happens=
.
In any case, having the checkpoint modification get its own CSN entry
lets the main operation CSN entry persist, which fixes this issue and
all operations that need a CSN seem to get it.
Patch is available at:
ftp://ftp.openldap.org/incoming/Ondrej-Kuznik-20170825-ITS8444.patch
--=20
Ond=C5=99ej Kuzn=C3=ADk
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
Full_Name: Ian Puleston
Version: 2.4.40
OS: VxWorks
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (204.118.31.9)
If connect and delete callbacks are registered via LDAP_OPT_CONNECT_CB, then the
lc_add callback is called when the TCP connects, but if TLS then fails to
connect the lc_del does not get called. If these callbacks are doing something
like allocating/freeing resources this can lead to a resource leak.
When I am seeing this, the sequence of calls leading to the lc_add callback is
as follows:
ldap_new_connection -> ldap_int_open_connection -> ldap_connect_to_host ->
ldap_int_connect_cbs()
Then if ldap_connect_to_host returned 0 for synchronous success this call
happens:
ldap_new_connection -> ldap_int_open_connection -> ldap_int_tls_start
And if that fails and returns something other than LDAP_SUCCESS then
ldap_int_open_connection exits. So we got a call to lc_add but no corresponding
call to lc_del.
And note that there is not call to ldap_free_connection, presumably because
ldap-new-connection did not succeed, hence the lc_del callback does not get
called from there.
In my case when I see this it is on a referral with the above calls made from
ldap_chase_v3referrals, hence it is happening synchronously. Whether the same
would happen with an asynchronous connect I don't know.
I do have a working fix which is to make the lc_del callbacks directly from
ldap_int_open_connection if ldap_int_tls_start fails. I will supply a patch for
that shortly.
On Wed, Aug 23, 2017 at 02:42:29PM +0100, Ond=C5=99ej Kuzn=C3=ADk wrote:
> It is caused by the cookie not containing CSN and a race between the
> syncCookie check in do_syncrep2 and syncrepl_message_to_op.
>=20
> This race is probably fine with plain syncrepl which is idempotent, but
> deltasync changes get their own dn in each accesslog instance and some
> can be applied twice unless we know how to find out we've already seen
> them - they need to mention the CSN.
>=20
> The CSN itself gets lost on at least one occasion - when there's a
> checkpoint triggered. Not 100 % sure why the cookie gets eaten because
> of it, the op pointer is different between the syncprov_op_response tha=
t
> calls syncprov_checkpoint and the one that decides CSN hasn't changed.
Yes, whenever a checkpoint happens, the syncCookie in cn=3Daccesslog only
contains rid=3DXXX,sid=3DYYY. I thought that was because the checkpoint
results in a new accesslog entry and that would be transmitted first,
but that's not the case, there is no accesslog entry nor anything sent
to the client (as observed by ldapsearch -E sync=3Drp).
I think it looks like this: syncproc_checkpoint modifies the suffix
entry, that calls slap_graduate_commit_csn and the csn is removed from
be_pending_csn_list. accesslog_response then can't find the CSN there
and has nothing to insert into its own pending csn list. Strange that
changing the overlay order (accesslog vs. syncprov) doesn't change this
behaviour, something I'd expect if the above is the reason this happens.
--=20
Ond=C5=99ej Kuzn=C3=ADk
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
It is caused by the cookie not containing CSN and a race between the
syncCookie check in do_syncrep2 and syncrepl_message_to_op.
This race is probably fine with plain syncrepl which is idempotent, but
deltasync changes get their own dn in each accesslog instance and some
can be applied twice unless we know how to find out we've already seen
them - they need to mention the CSN.
The CSN itself gets lost on at least one occasion - when there's a
checkpoint triggered. Not 100 % sure why the cookie gets eaten because
of it, the op pointer is different between the syncprov_op_response that
calls syncprov_checkpoint and the one that decides CSN hasn't changed.
--=20
Ond=C5=99ej Kuzn=C3=ADk
Senior Software Engineer
Symas Corporation http://www.symas.com
Packaged, certified, and supported LDAP solutions powered by OpenLDAP
--On Monday, August 21, 2017 4:45 PM +0000 germinouz(a)gmail.com wrote:
Hello,
The ITS system is for reporting bugs in the software, not for asking
question on how to do things such as build software. Please redirect your
question to openldap-technical(a)openldap.org if you need help with how to
set up a proper environment for building software. I would note the error
is rather obvious from the config.log you supplied:
> configure:5786: cc -I /usr/local/BerkeleyDB/6.2.32/include
> -L/usr/local/lib -L/usr/local/BerkeleyDB/6.2.32/lib
> -R/data/BerkeleyDB/6.2.32/lib conftest.c
>> &5
> cc: error: unrecognized command line option '-R'
Hope that helps!
This ITS will be closed.
Regards,
Quanah
--
Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
<http://www.symas.com>