Re: (ITS#5916) Allow alias dereferencing in search C API
by Kurt＠OpenLDAP.org
On Jan 31, 2009, at 11:21 AM, hyc(a)symas.com wrote:
> ando(a)sys-net.it wrote:
>> h.b.furuseth(a)usit.uio.no wrote:
>>> ando(a)sys-net.it writes:
>>>> Proxy backends use ldap_set_option(3) to set alias dereferencing
>>>> when
>>>> requested by a search operation (and do not clean it up
>>>> afterwards, as
>>>> far as I can tell). Since LDAP handlers are pooled and reused,
>>>> this
>>>> behavior is inherently broken.
>>> I note you committed this with an ldap_int_* interface.
>>
>> Yes, that's because I needed it internally.
>
> I think ldap_pvt would make more sense.
_int_ means internal to the library. _pvt_ means private to OpenLDAP
Software.
> I recall complaining about this deficiency in the API back in 1998,
> when I
> first wrote back-ldap. Alias deref has always been a part of the
> protocol, but
> it has always been missing from the API. Perplexing...
Yes, of course, one might use the client control mechanism to add an
ability to set deref on a call by call basis instead of introducing
yet another function.
>
>
>>> But it could
>>> be useful outside OpenLDAP code too.
>>
>> Yes, that will probably be the next step. But to make it publicly
>> available I should have called it ldap_search_ext2, or
>> ldap_search_really_ext or something like that. I think we need to
>> make
>> it public based on demand and after a ballot on the name.
>
> Ok.
>
>>> Seems to me this is what client-side controls are for. Extending
>>> the
>>> client-side functionality without needing a new API.
>>
>> Well, since alias dereferencing is part of the protocol, I think
>> using a
>> client-side control is sort of an overkill :)
>
> Agreed.
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
>
>
14 years, 10 months
Re: (ITS#5916) Allow alias dereferencing in search C API
by hyc＠symas.com
ando(a)sys-net.it wrote:
> h.b.furuseth(a)usit.uio.no wrote:
>> ando(a)sys-net.it writes:
>>> Proxy backends use ldap_set_option(3) to set alias dereferencing when
>>> requested by a search operation (and do not clean it up afterwards, as
>>> far as I can tell). Since LDAP handlers are pooled and reused, this
>>> behavior is inherently broken.
>> I note you committed this with an ldap_int_* interface.
>
> Yes, that's because I needed it internally.
I think ldap_pvt would make more sense.
I recall complaining about this deficiency in the API back in 1998, when I
first wrote back-ldap. Alias deref has always been a part of the protocol, but
it has always been missing from the API. Perplexing...
>> But it could
>> be useful outside OpenLDAP code too.
>
> Yes, that will probably be the next step. But to make it publicly
> available I should have called it ldap_search_ext2, or
> ldap_search_really_ext or something like that. I think we need to make
> it public based on demand and after a ballot on the name.
Ok.
>> Seems to me this is what client-side controls are for. Extending the
>> client-side functionality without needing a new API.
>
> Well, since alias dereferencing is part of the protocol, I think using a
> client-side control is sort of an overkill :)
Agreed.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
14 years, 10 months
Re: (ITS#5916) Allow alias dereferencing in search C API
by ando＠sys-net.it
h.b.furuseth(a)usit.uio.no wrote:
> ando(a)sys-net.it writes:
>> Proxy backends use ldap_set_option(3) to set alias dereferencing when
>> requested by a search operation (and do not clean it up afterwards, as
>> far as I can tell). Since LDAP handlers are pooled and reused, this
>> behavior is inherently broken.
>
> I note you committed this with an ldap_int_* interface.
Yes, that's because I needed it internally.
> But it could
> be useful outside OpenLDAP code too.
Yes, that will probably be the next step. But to make it publicly
available I should have called it ldap_search_ext2, or
ldap_search_really_ext or something like that. I think we need to make
it public based on demand and after a ballot on the name.
> Seems to me this is what client-side controls are for. Extending the
> client-side functionality without needing a new API.
Well, since alias dereferencing is part of the protocol, I think using a
client-side control is sort of an overkill :)
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
14 years, 10 months
Re: (ITS#5916) Allow alias dereferencing in search C API
by h.b.furuseth＠usit.uio.no
ando(a)sys-net.it writes:
> Proxy backends use ldap_set_option(3) to set alias dereferencing when
> requested by a search operation (and do not clean it up afterwards, as
> far as I can tell). Since LDAP handlers are pooled and reused, this
> behavior is inherently broken.
I note you committed this with an ldap_int_* interface. But it could
be useful outside OpenLDAP code too.
Seems to me this is what client-side controls are for. Extending the
client-side functionality without needing a new API.
--
Hallvard
14 years, 10 months
(ITS#5916) Allow alias dereferencing in search C API
by ando＠sys-net.it
Full_Name: Pierangelo Masarati
Version: HEAD
OS: irrelevant
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (82.63.140.131)
Submitted by: ando
Proxy backends use ldap_set_option(3) to set alias dereferencing when requested
by a search operation (and do not clean it up afterwards, as far as I can tell).
Since LDAP handlers are pooled and reused, this behavior is inherently broken.
The only reason for doing this is that the C API does not allow to explicitly
define alias dereferencing when sending a search request.
This is a feature request for an internal API extension (ldap_int_search*) that
allows alias dereferencing. The current C API (essentially, ldap_search_ext*)
should wrap the new one, until promoted to public.
A patch is coming.
p.
14 years, 10 months
Re: (ITS#5889) [PATCH] Sending dereference field when retrying connection in meta backend
by ando＠sys-net.it
jorge.perez(a)adaptia.net wrote:
> When we have two slapds with a established meta connection between them and the
> connection is reset, for example by a router, the next search query will always
> be send with never in dereferencing.
>
> Steps to reproduce:
>
> - Established a meta connection between 2 slapds
> - Reset the connection, for example with cutter
> - Send a search dereferencing with something different to never.
> - See the results are no dereferenced.
Actually, I was reviewing this fix, and it seems that the code for alias
dereferencing is inherently broken, essentially because the (C) API for
alias dereferencing is broken. In fact, back-ldap and back-meta reuse
and pool connections, so setting this parameter using ldap_set_option()
will actually affect all (search) operations occurring simultaneously in
an unprotected manner. I think this needs to be fixed.
The simplest (and my favorite) solution would be to have back-ldap and
back-meta discontinue aliasing support.
Another option, which I do not consider particularly viable, is to
separately pool connections with different alias dereferencing strategies
Finally (my second favorite option) is to add a new C API for
ldap_search* operations that allows to explicitly set the alias
dereferencing parameter. This API does not need to be public, since it
is mostly useful inside the proxy backends.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
14 years, 10 months
Re: slapo-unique man page misleading (ITS#5908)
by ghenry＠suretecsystems.com
Ok, but we should mention this somewhere like in the admin guide perhaps.
------Original Message------
From: Quanah Gibson-Mount
Sender:
To: Gavin Henry
To: openldap-its(a)openldap.org
Subject: Re: slapo-unique man page misleading (ITS#5908)
Sent: 30 Jan 2009 19:13
--On Friday, January 30, 2009 7:08 PM +0000 Gavin Henry
<ghenry(a)suretecsystems.com> wrote:
> Ok, so this should be in the man page. Where else should this be listed
> that affects other overlays?
Eh, I'm not sure it should be in the man page. The point is, if you ever
want guaranteed atomic writes, you have to use slapo-accesslog to serialize
them. That's an issue that affects other LDAP servers as well.
AS Pierangelo said:
There is no guarantee of DSA-wide or even database-wide atomicity in write
operations, including internal ones. This was never even intended to be in
place. This is a known design limitation not only of slapo-unique, but
also of slapd (and, I'd say, of LDAP itself).
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
--
Kind Regards,
Gavin Henry.
Managing Director.
T +44 (0) 1224 279484
M +44 (0) 7930 323266
F +44 (0) 1224 824887
E ghenry(a)suretecsystems.com
Open Source. Open Solutions(tm).
http://www.suretecsystems.com/
Suretec Systems is a limited company registered in Scotland. Registered
number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie,
Aberdeenshire, AB51 4FP.
Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html
14 years, 10 months
Re: slapo-unique man page misleading (ITS#5908)
by quanah＠zimbra.com
--On Friday, January 30, 2009 7:08 PM +0000 Gavin Henry
<ghenry(a)suretecsystems.com> wrote:
> Ok, so this should be in the man page. Where else should this be listed
> that affects other overlays?
Eh, I'm not sure it should be in the man page. The point is, if you ever
want guaranteed atomic writes, you have to use slapo-accesslog to serialize
them. That's an issue that affects other LDAP servers as well.
AS Pierangelo said:
There is no guarantee of DSA-wide or even database-wide atomicity in write
operations, including internal ones. This was never even intended to be in
place. This is a known design limitation not only of slapo-unique, but
also of slapd (and, I'd say, of LDAP itself).
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
14 years, 10 months
Re: slapo-unique man page misleading (ITS#5908)
by ghenry＠suretecsystems.com
Ok, so this should be in the man page. Where else should this be listed that affects other overlays?
------Original Message------
From: Quanah Gibson-Mount
Sender:
To: Gavin Henry
To: openldap-its(a)openldap.org
Subject: Re: slapo-unique man page misleading (ITS#5908)
Sent: 30 Jan 2009 19:04
--On Thursday, January 29, 2009 3:48 PM +0000 ghenry(a)suretecsystems.com
wrote:
>> Huh? slapo-unique only does internal searches. The writes are all user
>> ops,
>> and accesslog will serialize all of them.
>
> So, where does this leave us?
If you want guaranteed atomic writes, use slapo-accesslog on your master to
serialize them.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
--
Kind Regards,
Gavin Henry.
Managing Director.
T +44 (0) 1224 279484
M +44 (0) 7930 323266
F +44 (0) 1224 824887
E ghenry(a)suretecsystems.com
Open Source. Open Solutions(tm).
http://www.suretecsystems.com/
Suretec Systems is a limited company registered in Scotland. Registered
number: SC258005. Registered office: 13 Whiteley Well Place, Inverurie,
Aberdeenshire, AB51 4FP.
Subject to disclaimer at http://www.suretecgroup.com/disclaimer.html
14 years, 10 months
Re: slapo-unique man page misleading (ITS#5908)
by quanah＠zimbra.com
--On Thursday, January 29, 2009 3:48 PM +0000 ghenry(a)suretecsystems.com
wrote:
>> Huh? slapo-unique only does internal searches. The writes are all user
>> ops,
>> and accesslog will serialize all of them.
>
> So, where does this leave us?
If you want guaranteed atomic writes, use slapo-accesslog on your master to
serialize them.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
14 years, 10 months
