Hi, this FAQ http://www.openldap.org/faq/data/cache/649.html says that openldap 2.3 doesn't support Server Side Sorting (RFC 2891). Looking at openldap 2.4.23, it seems that Server Side Sorting is now supported - is this documented somewhere? Chris
--On Friday, June 10, 2011 2:24 PM +0000 Chris Card ctcard@hotmail.com wrote:
Hi,
this FAQ http://www.openldap.org/faq/data/cache/649.html says that openldap 2.3 doesn't support Server Side Sorting (RFC 2891).
Looking at openldap 2.4.23, it seems that Server Side Sorting is now supported - is this documented somewhere?
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Looking at openldap 2.4.23, it seems that Server Side Sorting is now supported - is this documented somewhere?
--Quanah
Thanks! Chris
I'm trying to use the sssvlv overlay with ldapsearch, by supplying these options to ldapsearch (openldap-2.4.23 on Centos 5.4): -E !sss=mail:caseIgnoreIA5Match -E !vlv=0/10/0/10 The server returns 10 entries as expected and then ldapsearch prompts: Press [before/after(/offset/count|:value)] Enter for the next window. If I then enter 0/10/0/10, I get the response from the server: # extended LDIF## LDAPv3# base <.........> with scope subtree# filter: (objectclass=*)# requesting: ALL# with server side sorting critical control# with virtual list view critical control: 0/10/0/10# # search resultsearch: 3result: 4 Size limit exceeded This appears to be because of these lines in result.c, in slap_send_search_entry(): if ( op->ors_slimit >= 0 && rs->sr_nentries >= op->ors_slimit ) { return LDAP_SIZELIMIT_EXCEEDED; } and because op->ors_slimit is now 0 instead of SLAP_NO_LIMIT (-1). Any ideas? Chris
From: ctcard@hotmail.com To: quanah@zimbra.com; openldap-technical@openldap.org Subject: RE: server side sorting Date: Sat, 11 Jun 2011 08:36:33 +0000
Looking at openldap 2.4.23, it seems that Server Side Sorting is now supported - is this documented somewhere?
--Quanah
Thanks! Chris
--On Monday, June 13, 2011 11:03 AM +0000 Chris Card ctcard@hotmail.com wrote:
and because op->ors_slimit is now 0 instead of SLAP_NO_LIMIT (-1).
Any ideas?
File a bug?
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
When using the sssvlv overlay how does the client tell the server that the sort request can be thrown away, so that sort requests don't hang around until sssvlv-max or sssvlv-maxpercon are exceeded? Chris
I installed openldap from source code, but I do not find a startup script after "make install". I wonder where the startup scirpt like /etc/init.d/ldap is, or where I can find one.
Thanks.
Hai T.
You have to create it
From: openldap-technical-bounces@openldap.org [mailto:openldap-technical-bounces@openldap.org] On Behalf Of Hai Tao Sent: Monday, June 27, 2011 5:46 PM To: openldap-technical@openldap.org Subject: where is startup script /etc/init.d/ldap
I installed openldap from source code, but I do not find a startup script after "make install". I wonder where the startup scirpt like /etc/init.d/ldap is, or where I can find one.
Thanks.
Hai T.
2011/6/28 Darouichi, Aziz adarouic@post03.curry.edu:
You have to create it
Hi,
you can find such a script here: http://ltb-project.org/wiki/documentation/openldap-initscript http://tools.ltb-project.org/attachments/download/182/ltb-project-openldap-i...
Clément.
I have multiple ldap servers using replication: one provider and multiple consumers.
In my multiple consumers' slapd.conf, the replication rid is the same:
syncrepl rid=123.
I wonder if this is a problem. Should the rid for each consumer be unique? why?
Thanks.
Hai Tao
--On Tuesday, June 28, 2011 5:01 PM -0700 Hai Tao taoh666@hotmail.com wrote:
I have multiple ldap servers using replication: one provider and multiple consumers.
In my multiple consumers' slapd.conf, the replication rid is the same:
syncrepl rid=123.
I wonder if this is a problem. Should the rid for each consumer be unique? why?
RID's only need to be unique inside a given consumer. I.e., if the consumer has MULTIPLE syncrepl statements, then each RID must be unique. Different consumers have no concept of what RIDs other consumers use, so they do not have to be unique across consumers.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Chris Card wrote:
When using the sssvlv overlay how does the client tell the server that the
sort request can be thrown away, so that sort requests don't hang around until sssvlv-max or sssvlv-maxpercon are exceeded?
There is no official mechanism for this. This is a flaw in the Sorting/VLV specification. For Paged Results you simply send a followup request with pagesize set to zero, but that doesn't apply for VLV.
I believe in our current implementation, if you send a new Paged Results request on a connection that has VLVs outstanding, one of the VLVs will be aborted. Unfortunately, if you have multiple VLVs outstanding, there's no way to determine which one gets aborted. (Apparently this was overlooked when patching the overlay to allow multiple VLVs on a connection...)
Howard Chu wrote:
Chris Card wrote:
When using the sssvlv overlay how does the client tell the server that the
sort request can be thrown away, so that sort requests don't hang around until sssvlv-max or sssvlv-maxpercon are exceeded?
There is no official mechanism for this. This is a flaw in the Sorting/VLV specification. For Paged Results you simply send a followup request with pagesize set to zero, but that doesn't apply for VLV.
Btw, I noted this flaw to the IETF a long time ago but nobody seemed to notice or care.
http://www.openldap.org/lists/ietf-ldapext/200907/msg00004.html
When using the sssvlv overlay how does the client tell the server that the
sort request can be thrown away, so that sort requests don't hang around until sssvlv-max or sssvlv-maxpercon are exceeded?
There is no official mechanism for this. This is a flaw in the Sorting/VLV specification. For Paged Results you simply send a followup request with pagesize set to zero, but that doesn't apply for VLV.
I believe in our current implementation, if you send a new Paged Results request on a connection that has VLVs outstanding, one of the VLVs will be aborted. Unfortunately, if you have multiple VLVs outstanding, there's no way to determine which one gets aborted. (Apparently this was overlooked when patching the overlay to allow multiple VLVs on a connection...)
Thanks for the info. Is it the case that sort requests are associated with a particular connection?
From my testing it seems that a sort request will remain active in the server evenif the client disconnects, which doesn't seem right.
Chris
Chris Card wrote:
When using the sssvlv overlay how does the client tell the server that the
sort request can be thrown away, so that sort requests don't hang around until sssvlv-max or sssvlv-maxpercon are exceeded?
There is no official mechanism for this. This is a flaw in the Sorting/VLV specification. For Paged Results you simply send a followup request with pagesize set to zero, but that doesn't apply for VLV.
I believe in our current implementation, if you send a new Paged Results request on a connection that has VLVs outstanding, one of the VLVs will be aborted. Unfortunately, if you have multiple VLVs outstanding, there's no way to determine which one gets aborted. (Apparently this was overlooked when patching the overlay to allow multiple VLVs on a connection...)
Thanks for the info. Is it the case that sort requests are associated with a particular connection?
What does RFC2891 say?
From my testing it seems that a sort request will remain active in the server evenif the client disconnects, which doesn't seem right.
How are you determining that this is the case?
When using the sssvlv overlay how does the client tell the server that the
sort request can be thrown away, so that sort requests don't hang around until sssvlv-max or sssvlv-maxpercon are exceeded?
There is no official mechanism for this. This is a flaw in the Sorting/VLV specification. For Paged Results you simply send a followup request with pagesize set to zero, but that doesn't apply for VLV.
I believe in our current implementation, if you send a new Paged Results request on a connection that has VLVs outstanding, one of the VLVs will be aborted. Unfortunately, if you have multiple VLVs outstanding, there's no way to determine which one gets aborted. (Apparently this was overlooked when patching the overlay to allow multiple VLVs on a connection...)
Thanks for the info. Is it the case that sort requests are associated with a particular connection?
What does RFC2891 say?
As far as I can see it doesn't mention connections.
However, looking at the code in sssvlv.c, it appears that a sort operation is associated with a session, and a session is associated with a connection, e.g. on line 900 of sssvlv.c in sssvlv_op_search():
sort_conns[op->o_conn->c_conn_idx][sess_id] = so;
From my testing it seems that a sort request will remain active in the server evenif the client disconnects, which doesn't seem right.
How are you determining that this is the case?
I'm using ldapsearch to test requests with sss and vlv controls. After running several such ldapsearch commands I get an error from the server :
# search result search: 2 result: 51 Server is busy text: Other sort requests already in progress
I am assuming that the connection to the server is dropped when the ldapsearch command terminates; that must certainly be the case at the client end since the process no longer exists.
Chris
Hi,
As co-contributor of the latest piece of code about SSS/VLV, I will try and give some explanations : VLV does not give a clean way to destroy the server side sorted entries list.
The SSS/VLV overlay implementation is using a limit in order to manage memory limits : a connection will be limited to a specified number of paged searches. The only way to reset this limit is to close and reopen a connection.
Maybe a timeout will be a good idea also but this is not implemented now.
Regards,
2011/7/1 Chris Card ctcard@hotmail.com
When using the sssvlv overlay how does the client tell the server
that the
sort request can be thrown away, so that sort requests don't hang
around until
sssvlv-max or sssvlv-maxpercon are exceeded?
There is no official mechanism for this. This is a flaw in the
Sorting/VLV
specification. For Paged Results you simply send a followup request
with
pagesize set to zero, but that doesn't apply for VLV.
I believe in our current implementation, if you send a new Paged
Results
request on a connection that has VLVs outstanding, one of the VLVs
will be
aborted. Unfortunately, if you have multiple VLVs outstanding, there's
no way
to determine which one gets aborted. (Apparently this was overlooked
when
patching the overlay to allow multiple VLVs on a connection...)
Thanks for the info. Is it the case that sort requests are associated with a particular
connection?
What does RFC2891 say?
As far as I can see it doesn't mention connections.
However, looking at the code in sssvlv.c, it appears that a sort operation is associated with a session, and a session is associated with a connection, e.g. on line 900 of sssvlv.c in sssvlv_op_search():
sort_conns[op->o_conn->c_conn_idx][sess_id] = so;From my testing it seems that a sort request will remain active in the
server evenif the client disconnects, which doesn't seem right.
How are you determining that this is the case?
I'm using ldapsearch to test requests with sss and vlv controls. After running several such ldapsearch commands I get an error from the server :
# search result search: 2 result: 51 Server is busy text: Other sort requests already in progress
I am assuming that the connection to the server is dropped when the ldapsearch command terminates; that must certainly be the case at the client end since the process no longer exists.
Chris
Chris Card wrote:
When using the sssvlv overlay how does the client tell the server that the
sort request can be thrown away, so that sort requests don't hang around until sssvlv-max or sssvlv-maxpercon are exceeded?
There is no official mechanism for this. This is a flaw in the Sorting/VLV specification. For Paged Results you simply send a followup request with pagesize set to zero, but that doesn't apply for VLV.
I believe in our current implementation, if you send a new Paged Results request on a connection that has VLVs outstanding, one of the VLVs will be aborted. Unfortunately, if you have multiple VLVs outstanding, there's no way to determine which one gets aborted. (Apparently this was overlooked when patching the overlay to allow multiple VLVs on a connection...)
Thanks for the info. Is it the case that sort requests are associated with a particular connection?
What does RFC2891 say?
As far as I can see it doesn't mention connections.
Then probably the answer is no.
However, looking at the code in sssvlv.c, it appears that a sort operation is associated with a session, and a session is associated with a connection, e.g. on line 900 of sssvlv.c in sssvlv_op_search():
sort_conns[op->o_conn->c_conn_idx][sess_id] = so;
That's actually limiting VLV requests. You didn't ask about VLV requests, you asked about sort requests. Ask the wrong question, get a useless answer. So now, what does draft-ietf-ldapv3-vlv say?
From my testing it seems that a sort request will remain active in the server evenif the client disconnects, which doesn't seem right.
How are you determining that this is the case?
I'm using ldapsearch to test requests with sss and vlv controls. After running several such ldapsearch commands I get an error from the server :
# search result search: 2 result: 51 Server is busy text: Other sort requests already in progress
I am assuming that the connection to the server is dropped when the ldapsearch command terminates; that must certainly be the case at the client end since the process no longer exists.
I see no such error here. I can run ldapsearch and send VLV requests ad nauseam. Most likely you have misconfigured something again.
Thanks for the info. Is it the case that sort requests are associated with a particular connection?
What does RFC2891 say?
As far as I can see it doesn't mention connections.
Then probably the answer is no.
However, looking at the code in sssvlv.c, it appears that a sort operation is associated with a session, and a session is associated with a connection, e.g. on line 900 of sssvlv.c in sssvlv_op_search():
sort_conns[op->o_conn->c_conn_idx][sess_id] = so;
That's actually limiting VLV requests. You didn't ask about VLV requests, you asked about sort requests. Ask the wrong question, get a useless answer. So now, what does draft-ietf-ldapv3-vlv say?
I guess this is a relevant section (the only occurrences of "connection" in the spec):
"contextID values have no validity outside the connection and query with which they were received. A client MUST NOT submit a contextID which it received from a different connection, a different query, or a different server."
which does partly answer my question about the VLV request, but says nothing about any related sort operation.
From my testing it seems that a sort request will remain active in the server evenif the client disconnects, which doesn't seem right.
How are you determining that this is the case?
I'm using ldapsearch to test requests with sss and vlv controls. After running several such ldapsearch commands I get an error from the server :
# search result search: 2 result: 51 Server is busy text: Other sort requests already in progress
I am assuming that the connection to the server is dropped when the ldapsearch command terminates; that must certainly be the case at the client end since the process no longer exists.
I see no such error here. I can run ldapsearch and send VLV requests ad nauseam. Most likely you have misconfigured something again.
Always possible, but I would expect that ldapsearch terminating would cause the server to drop the connection and clean up any sort request associated with a vlv request on that connection. I'll do some more investigation.
Chris
Thanks for the info. Is it the case that sort requests are associated with a particular connection?
What does RFC2891 say?
As far as I can see it doesn't mention connections.
Then probably the answer is no.
However, looking at the code in sssvlv.c, it appears that a sort operation is associated with a session, and a session is associated with a connection, e.g. on line 900 of sssvlv.c in sssvlv_op_search():
sort_conns[op->o_conn->c_conn_idx][sess_id] = so;
That's actually limiting VLV requests. You didn't ask about VLV requests, you asked about sort requests. Ask the wrong question, get a useless answer. So now, what does draft-ietf-ldapv3-vlv say?
I guess this is a relevant section (the only occurrences of "connection" in the spec):
"contextID values have no validity outside the connection and query with which they were received. A client MUST NOT submit a contextID which it received from a different connection, a different query, or a different server."
which does partly answer my question about the VLV request, but says nothing about any related sort operation.
From my testing it seems that a sort request will remain active in the server evenif the client disconnects, which doesn't seem right.
How are you determining that this is the case?
I'm using ldapsearch to test requests with sss and vlv controls. After running several such ldapsearch commands I get an error from the server :
# search result search: 2 result: 51 Server is busy text: Other sort requests already in progress
I am assuming that the connection to the server is dropped when the ldapsearch command terminates; that must certainly be the case at the client end since the process no longer exists.
I see no such error here. I can run ldapsearch and send VLV requests ad nauseam. Most likely you have misconfigured something again.
Always possible, but I would expect that ldapsearch terminating would cause the server to drop the connection and clean up any sort request associated with a vlv request on that connection. I'll do some more investigation.
More investigation done, and the problem appears to be another difference between specifying the sssvlv overlay as global or local to the database config.
If I specify the sssvlv overlay in the database config then I can also send VLV requests ad nauseam, I only see the error when I specify the sssvlv overlay as global.
Chris
I see no such error here. I can run ldapsearch and send VLV requests ad nauseam. Most likely you have misconfigured something again.
Always possible, but I would expect that ldapsearch terminating would cause the server to drop the connection and clean up any sort request associated with a vlv request on that connection. I'll do some more investigation.
More investigation done, and the problem appears to be another difference between specifying the sssvlv overlay as global or local to the database config.
If I specify the sssvlv overlay in the database config then I can also send VLV requests ad nauseam, I only see the error when I specify the sssvlv overlay as global.
I'm seeing another issue with sssvlv now.
If I use the AssertionValue in the VLV control, some values work and the expected entries are returned, but others values which should find entries do not work.
I am specifying cn:caseIgnoreOrderingMatch in the sort control, and if I specify 'd' in the AssertionValue of the VLV control I get entries with cns
Dave Smith Fiona Smith Glynis Smith Jane Smith Jennifer Smith
If I specify 'g', I get
Glynis Smith Jane Smith Jennifer Smith Jim Smith Mark Smith
but if I specify 'f' I get nothing.
I am running slapd built from HEAD on June 30th.
This feels like a bug in the sssvlv overlay, so I'll raise it in ITS too.
Chris
openldap-technical@openldap.org
-
Chris Card -
Clément OUDOT -
Darouichi, Aziz -
Hai Tao -
Howard Chu -
Quanah Gibson-Mount -
Sébastien Bahloul