--On Tuesday, August 02, 2011 10:08:32 AM -0700 Bill MacAllister <whm(a)stanford.edu> wrote:
>
>
> --On Monday, August 01, 2011 02:46:50 PM -0700 Howard Chu <hyc(a)symas.com> wrote:
>
>> whm(a)stanford.edu wrote:
>>> Full_Name: Bill MacAllister
>>> Version: 2.4.26
>>> OS: Debian 6
>>> URL: ftp://ftp.openldap.org/incoming/
>>> Submission from: (NULL) (171.64.19.165)
>>>
>>>
>>> We typically setup local proxy servers to support applications that cannot
>>> support a GSSAPI bind to the directory server. The proxy server allows
>>> anonymous access to the directory for connections from the localhost and
>>> connects to the master using GSSAPI. We are experiencing a failures when
>>> we attempt to use the paged results control on the proxy. For example:
>>>
>>> ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h localhost
>>> "(&(objectClass=suPerson)(suVisibIdentity=world))" ou telephonenumber title
>>>
>>> ends with the error:
>>>
>>> # search result
>>> search: 5
>>> result: 0 Success
>>> control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
>>> pagedresults: cookie=KIMAAAAAAAA=
>>> # extended LDIF
>>> #
>>> # LDAPv3
>>> # base<cn=people,dc=stanford,dc=edu> with scope subtree
>>> # filter: (&(objectClass=suPerson)(suVisibIdentity=world))
>>> # requesting: ou telephonenumber title
>>> # with pagedResults control: size=1000
>>> #
>>>
>>> # search result
>>> search: 6
>>> result: 2 Protocol error
>>> text: paged results cookie is invalid
>>>
>>> # numResponses: 4005
>>> # numEntries: 4000
>>>
>>> This result is not consistent. We have seen examples where 2000 and 3000
>>> entries being returned and then the error. Another test that we performed with
>>> a slightly more complex filter, i.e.
>>>
>>> "(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=world)))"
>>>
>>> returned usually returned 1000 entries before erroring.
>>>
>>> Issuing a similar search directly against the backend ldap server completes
>>> without
>>> error.
>>>
>>> We have seen the same behavior on OpenLDAP 2.4.23 as well.
>>>
>>> Logs generated running slapd standalone with '-d stats,packets' are available at
>>> http://www.stanford.edu/~whm/files/ldap-debugging/.
>>
>> Your log shows that the subsequent search request initiates a new
>> Bind to the remote server, which implies that it's not re-using the
>> same connection as the first request. Since a paged results cookie
>> is only valid within the context of a single connection, you get
>> this error result.
>
> Not sure which log you are looking at. When I look at the log:
>
> http://www.stanford.edu/~whm/files/ldap-debugging/slapd-trace-paged-results…
>
> The only connection I see in the log is conn=1000 and it ends with:
>
> conn=1000 op=5 SEARCH RESULT tag=101 err=2 nentries=0 text=paged results cookie is invalid
> ldap_read: want=8, got=7
> 0000: 30 05 02 01 07 42 00 0....B.
> ldap_read: want=8, got=0
>
> conn=1000 op=6 UNBIND
> conn=1000 fd=11 closed
>
> These tests where made with a single ldapsearch request. The ldapsearch
> tests fail when using the proxy and succeed when connecting directly to
> the LDAP server with the database on it.
>
> A side node: the test case I submitted used ldapsearch, but the
> problem was uncovered using a python application that is used for
> syncing Gmail account data.
>
> Bill
I have copied the backend server configuration to
http://www.stanford.edu/~whm/files/ldap-debugging/. I dumped an
copy of cn=config and there is a files based version the in ldap
subdirectory as well.
Bill
--
Bill MacAllister
Infrastructure Delivery Group, Stanford University
--On Monday, August 01, 2011 02:46:50 PM -0700 Howard Chu <hyc(a)symas.com> wrote:
> whm(a)stanford.edu wrote:
>> Full_Name: Bill MacAllister
>> Version: 2.4.26
>> OS: Debian 6
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (171.64.19.165)
>>
>>
>> We typically setup local proxy servers to support applications that cannot
>> support a GSSAPI bind to the directory server. The proxy server allows
>> anonymous access to the directory for connections from the localhost and
>> connects to the master using GSSAPI. We are experiencing a failures when
>> we attempt to use the paged results control on the proxy. For example:
>>
>> ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h localhost
>> "(&(objectClass=suPerson)(suVisibIdentity=world))" ou telephonenumber title
>>
>> ends with the error:
>>
>> # search result
>> search: 5
>> result: 0 Success
>> control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
>> pagedresults: cookie=KIMAAAAAAAA=
>> # extended LDIF
>> #
>> # LDAPv3
>> # base<cn=people,dc=stanford,dc=edu> with scope subtree
>> # filter: (&(objectClass=suPerson)(suVisibIdentity=world))
>> # requesting: ou telephonenumber title
>> # with pagedResults control: size=1000
>> #
>>
>> # search result
>> search: 6
>> result: 2 Protocol error
>> text: paged results cookie is invalid
>>
>> # numResponses: 4005
>> # numEntries: 4000
>>
>> This result is not consistent. We have seen examples where 2000 and 3000
>> entries being returned and then the error. Another test that we performed with
>> a slightly more complex filter, i.e.
>>
>> "(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=world)))"
>>
>> returned usually returned 1000 entries before erroring.
>>
>> Issuing a similar search directly against the backend ldap server completes
>> without
>> error.
>>
>> We have seen the same behavior on OpenLDAP 2.4.23 as well.
>>
>> Logs generated running slapd standalone with '-d stats,packets' are available at
>> http://www.stanford.edu/~whm/files/ldap-debugging/.
>
> Your log shows that the subsequent search request initiates a new
> Bind to the remote server, which implies that it's not re-using the
> same connection as the first request. Since a paged results cookie
> is only valid within the context of a single connection, you get
> this error result.
Not sure which log you are looking at. When I look at the log:
http://www.stanford.edu/~whm/files/ldap-debugging/slapd-trace-paged-results…
The only connection I see in the log is conn=1000 and it ends with:
conn=1000 op=5 SEARCH RESULT tag=101 err=2 nentries=0 text=paged results cookie is invalid
ldap_read: want=8, got=7
0000: 30 05 02 01 07 42 00 0....B.
ldap_read: want=8, got=0
conn=1000 op=6 UNBIND
conn=1000 fd=11 closed
These tests where made with a single ldapsearch request. The ldapsearch
tests fail when using the proxy and succeed when connecting directly to
the LDAP server with the database on it.
A side node: the test case I submitted used ldapsearch, but the
problem was uncovered using a python application that is used for
syncing Gmail account data.
Bill
--
Bill MacAllister
Infrastructure Delivery Group, Stanford University
Full_Name: Bjoern Jacke
Version: 2.4.23
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (193.159.216.39)
When TIMEOUT is set and happend to occur there is nothing that shows that we
timed out. For example if you do a large/slow LDAP search using ldapsearch and
then stop the conversation between slapd and ldapsearch by "killall -STOP slapd"
then ldapsearch will terminate after TIMEOUT seconds and it will return the
number of records that it got so far and exit with return code 0 and not error
message. I'd expect that after TIMEOUT occuring that we get a warning and a
return code != 0
Quanah Gibson-Mount wrote:
> --On Monday, August 01, 2011 9:47 PM +0000 hyc(a)symas.com wrote:
>> Your log shows that the subsequent search request initiates a new Bind to
>> the remote server, which implies that it's not re-using the same
>> connection as the first request. Since a paged results cookie is only
>> valid within the context of a single connection, you get this error
>> result.
>
> Given that the client making the request is ldapsearch, which is not going
> to rebind mid search, and you can see it is done with noprompt, so there's
> no human interaction here, it seems like back-ldap is buggy here. Why is
> *back-ldap* closing its connection to the upstream server and initiating a
> new bind?
You haven't provided any slapd configuration info. Without seeing that it's
way premature to claim there's any bug here.
Look at the log yourself. back-ldap *doesn't* close the connection. Don't spew
garbage about what it does when it didn't actually do such a thing.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--On Monday, August 01, 2011 9:47 PM +0000 hyc(a)symas.com wrote:
> whm(a)stanford.edu wrote:
>> Full_Name: Bill MacAllister
>> Version: 2.4.26
>> OS: Debian 6
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (171.64.19.165)
>>
>>
>> We typically setup local proxy servers to support applications that
>> cannot support a GSSAPI bind to the directory server. The proxy server
>> allows anonymous access to the directory for connections from the
>> localhost and connects to the master using GSSAPI. We are experiencing
>> a failures when we attempt to use the paged results control on the
>> proxy. For example:
>>
>> ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h
>> localhost "(&(objectClass=suPerson)(suVisibIdentity=world))" ou
>> telephonenumber title
>>
>> ends with the error:
>>
>> # search result
>> search: 5
>> result: 0 Success
>> control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
>> pagedresults: cookie=KIMAAAAAAAA=
>> # extended LDIF
>> #
>> # LDAPv3
>> # base<cn=people,dc=stanford,dc=edu> with scope subtree
>> # filter: (&(objectClass=suPerson)(suVisibIdentity=world))
>> # requesting: ou telephonenumber title
>> # with pagedResults control: size=1000
>> #
>>
>> # search result
>> search: 6
>> result: 2 Protocol error
>> text: paged results cookie is invalid
>>
>> # numResponses: 4005
>> # numEntries: 4000
>>
>> This result is not consistent. We have seen examples where 2000 and 3000
>> entries being returned and then the error. Another test that we
>> performed with a slightly more complex filter, i.e.
>>
>> "(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=wo
>> rld)))"
>>
>> returned usually returned 1000 entries before erroring.
>>
>> Issuing a similar search directly against the backend ldap server
>> completes without
>> error.
>>
>> We have seen the same behavior on OpenLDAP 2.4.23 as well.
>>
>> Logs generated running slapd standalone with '-d stats,packets' are
>> available at http://www.stanford.edu/~whm/files/ldap-debugging/.
>
> Your log shows that the subsequent search request initiates a new Bind to
> the remote server, which implies that it's not re-using the same
> connection as the first request. Since a paged results cookie is only
> valid within the context of a single connection, you get this error
> result.
Given that the client making the request is ldapsearch, which is not going
to rebind mid search, and you can see it is done with noprompt, so there's
no human interaction here, it seems like back-ldap is buggy here. Why is
*back-ldap* closing its connection to the upstream server and initiating a
new bind?
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
whm(a)stanford.edu wrote:
> Full_Name: Bill MacAllister
> Version: 2.4.26
> OS: Debian 6
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (171.64.19.165)
>
>
> We typically setup local proxy servers to support applications that cannot
> support a GSSAPI bind to the directory server. The proxy server allows
> anonymous access to the directory for connections from the localhost and
> connects to the master using GSSAPI. We are experiencing a failures when
> we attempt to use the paged results control on the proxy. For example:
>
> ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h localhost
> "(&(objectClass=suPerson)(suVisibIdentity=world))" ou telephonenumber title
>
> ends with the error:
>
> # search result
> search: 5
> result: 0 Success
> control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
> pagedresults: cookie=KIMAAAAAAAA=
> # extended LDIF
> #
> # LDAPv3
> # base<cn=people,dc=stanford,dc=edu> with scope subtree
> # filter: (&(objectClass=suPerson)(suVisibIdentity=world))
> # requesting: ou telephonenumber title
> # with pagedResults control: size=1000
> #
>
> # search result
> search: 6
> result: 2 Protocol error
> text: paged results cookie is invalid
>
> # numResponses: 4005
> # numEntries: 4000
>
> This result is not consistent. We have seen examples where 2000 and 3000
> entries being returned and then the error. Another test that we performed with
> a slightly more complex filter, i.e.
>
> "(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=world)))"
>
> returned usually returned 1000 entries before erroring.
>
> Issuing a similar search directly against the backend ldap server completes
> without
> error.
>
> We have seen the same behavior on OpenLDAP 2.4.23 as well.
>
> Logs generated running slapd standalone with '-d stats,packets' are available at
> http://www.stanford.edu/~whm/files/ldap-debugging/.
Your log shows that the subsequent search request initiates a new Bind to the
remote server, which implies that it's not re-using the same connection as the
first request. Since a paged results cookie is only valid within the context
of a single connection, you get this error result.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Bill MacAllister
Version: 2.4.26
OS: Debian 6
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (171.64.19.165)
We typically setup local proxy servers to support applications that cannot
support a GSSAPI bind to the directory server. The proxy server allows
anonymous access to the directory for connections from the localhost and
connects to the master using GSSAPI. We are experiencing a failures when
we attempt to use the paged results control on the proxy. For example:
ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h localhost
"(&(objectClass=suPerson)(suVisibIdentity=world))" ou telephonenumber title
ends with the error:
# search result
search: 5
result: 0 Success
control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
pagedresults: cookie=KIMAAAAAAAA=
# extended LDIF
#
# LDAPv3
# base <cn=people,dc=stanford,dc=edu> with scope subtree
# filter: (&(objectClass=suPerson)(suVisibIdentity=world))
# requesting: ou telephonenumber title
# with pagedResults control: size=1000
#
# search result
search: 6
result: 2 Protocol error
text: paged results cookie is invalid
# numResponses: 4005
# numEntries: 4000
This result is not consistent. We have seen examples where 2000 and 3000
entries being returned and then the error. Another test that we performed with
a slightly more complex filter, i.e.
"(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=world)))"
returned usually returned 1000 entries before erroring.
Issuing a similar search directly against the backend ldap server completes
without
error.
We have seen the same behavior on OpenLDAP 2.4.23 as well.
Logs generated running slapd standalone with '-d stats,packets' are available at
http://www.stanford.edu/~whm/files/ldap-debugging/.
Bill
Full_Name: Stephan
Version: 2.4.26
OS: Debian 5.0.8
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (134.100.101.1)
Hi everybody,
I have an odd problem with synchronisation. I have 5 nodes running multi master
synchronisation. I recently updated one of them to 2.4.26 (they were all running
2.4.23 before) and started it with an empty database (the database directory
contained nothing more than the DB_CONFIG file). Afterwards I noted that some
attributes were missing, especially in the root of the directory tree, which
looks like this:
dn: o=test
objectClass: organization
o: test
After the sync was complete, I could search for the o=test entry but couldn't
see it:
ldapsearch -H "ldaps://filled.test.xyz" -D cn=superman,o=test -W -x -b o=test
o=test
# extended LDIF
#
# LDAPv3
# base <o=test> with scope subtree
# filter: o=test
# requesting: ALL
#
# search result
search: 2
result: 0 Success
# numResponses: 1
I played around a bit and am able to reproduce the problem. I tried combinations
of 2.4.23 and 2.4.26, bdb and hdb backends, bdb versions 4.5.20-13 and 4.6.21
but the problem persists. If I take the output of slapcat on one of the nodes
and feed it to the empty node before its startup everything seems to be ok.
I increased the loglevel and can see the following (rid 1 being the node with
the filled database and rid 2 the one with the empty database):
On the empty node (rid2) I get lots of 'syncrepl_entry: rid=001 inserted UUID
xxx'
'syncprov_matchops: skipping original sid 001' and 'syncrepl_entry: rid=001
be_add'
Once the sync is complete the log on the formerly empty node says:
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE -
REFRESH_DELETE
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001
cookie=rid=001,sid=001,csn=20110310162449.396663Z#000000#000#000000;20110801135804.526931Z#000000#001#000000;20110614092017.772141Z#000000#003#000000;20110722131433.793594Z#000000#004#000000;20110225131904.168266Z#000000#005#000000
Aug 1 17:24:55 empty slapd[7796]: slap_queue_csn: queing 0x9f901da8
20110801135804.526931Z#000000#001#000000
Aug 1 17:24:55 empty slapd[7796]: syncprov_matchops: skipping original sid 001
Aug 1 17:24:55 empty slapd[7796]: slap_graduate_commit_csn: removing 0x9f91f5f8
20110801135804.526931Z#000000#001#000000
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 cookie=
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY)
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001 be_search (0)
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001 o=test
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001 entry unchanged,
ignored (o=test)
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE -
NEW_COOKIE
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 NEW_COOKIE:
rid=001,sid=001,csn=20110801152321.168164Z#000000#000#000000;20110801135804.526931Z#000000#001#000000;20110614092017.772141Z#000000#003#000000;20110722131433.793594Z#000000#004#000000;20110225131904.168266Z#000000#005#000000
On the node from which it took the database (rid 1):
Aug 1 17:24:54 filled slapd[17862]: do_syncrep2: rid=002 LDAP_RES_INTERMEDIATE
- SYNC_ID_SET
Aug 1 17:24:54 filled last message repeated 7 times
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 be_search (0)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 o=test
Aug 1 17:24:54 filled slapd[17862]: syncprov_sendresp: cookie=
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 be_modify o=test
(0)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
Aug 1 17:24:54 filled slapd[17862]: dn_callback : entries have identical CSN
sambaDomainName=XYZ,ou=abc,o=test 20110722131010.624524Z#000000#004#000000
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 be_search (0)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002
sambaDomainName=XYZ,ou=abc,o=test
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 entry unchanged,
ignored (sambaDomainName=XYZ,ou=abc,o=test)
Aug 1 17:24:54 filled slapd[17862]: do_syncrep2: rid=002 LDAP_RES_INTERMEDIATE
- REFRESH_PRESENT
Aug 1 17:24:54 filled slapd[17862]: do_syncrep2: rid=002
cookie=rid=002,csn=20110801152321.168164Z#000000#000#000000
Aug 1 17:24:54 filled slapd[17862]: slap_queue_csn: queing 0x7eff600022c0
20110801152321.168164Z#000000#000#000000
Aug 1 17:24:54 filled slapd[17862]: slap_graduate_commit_csn: removing
0x7eff60007040 20110801152321.168164Z#000000#000#000000
It seems to me that the formerly empty node overwrites the entry on the node
from which it took all the entries.
This is what my config looks like:
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
syncprov-reloadhint TRUE
serverID 1 "ldaps://filled.test.xyz"
serverID 2 "ldaps://empty.test.xyz"
#filled
syncrepl rid=2
provider=ldaps://filled.test.xyz:636
type=refreshAndPersist
searchbase="o=test"
retry="120 20 300 5 1800 5 3600 +"
scope=sub
bindmethod=simple
binddn="cn=superman,o=test"
credentials="hello"
tls_reqcert=never
mirrormode on
And the other node:
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
syncprov-reloadhint TRUE
serverID 1 "ldaps://filled.test.xyz"
serverID 2 "ldaps://empty.test.xyz"
#filled
syncrepl rid=2
provider=ldaps://empty.test.xyz:636
type=refreshAndPersist
searchbase="o=test"
retry="120 20 300 5 1800 5 3600 +"
scope=sub
bindmethod=simple
binddn="cn=superman,o=test"
credentials="hello"
tls_reqcert=never
mirrormode on
It seems to me that the formerly empty node overwrites the other one after the
sync.
In some cases the entries don't go missing on the source node, they just don't
get replicated to the formerly emtpy node.
Unless I made some terrible mistake in the config this is seems to be a bug
since in the administrator's guide
http://www.openldap.org/doc/admin24/replication.html#N-Way%20Multi-Master%2…
it says: "Syncrepl is self-synchronizing; you can start with a consumer database
in any state from totally empty to fully synced and it will automatically do the
right thing to achieve and maintain synchronization"
All nodes are configured to use the same ntp source.
Full_Name: Mahesh P
Version: 2.4.23
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (206.126.170.20)
Hi,
We have one application which will use the Oracle DB, OpenLDAP and Curl
libraries.
We have started the LDAP server. And we want to perform search/add/update
operations using LDAP calls ldap_search_ext_s(), ldap_add_ext_s() and
ldap_modify_ext_s().
While performing the search Operation, getting the "Bad Search Filter" error.
In the application code we have used the ldap_initialize() for getting the LDAP
handle.I am thinking this problem is result of the collision of LDAP calls
between Oracle Library and OpenLDAP,Curl libraries.
When we did nm on libclntsh.so.11.1, got the references for
0000000001055732 T ldap_search_ext
0000000001055594 T ldap_search_ext_s
0000000001051714 T ora_ldap_search_ext
000000000105145a T ora_ldap_search_ext_s
If we try to have a standalone program without any database connectivity same
LDAP code is working fine for the search/add/modify operations.
Kindly please advise how we can avoid the problem if we use Oracle DB, OpenLDAP
and CURL functionality in out application code.
Thanks
Mahesh