Full_Name: Bjoern Jacke
Version: 2.4.23
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (193.159.216.39)
When TIMEOUT is set and happend to occur there is nothing that shows that we
timed out. For example if you do a large/slow LDAP search using ldapsearch and
then stop the conversation between slapd and ldapsearch by "killall -STOP slapd"
then ldapsearch will terminate after TIMEOUT seconds and it will return the
number of records that it got so far and exit with return code 0 and not error
message. I'd expect that after TIMEOUT occuring that we get a warning and a
return code != 0
Quanah Gibson-Mount wrote:
> --On Monday, August 01, 2011 9:47 PM +0000 hyc(a)symas.com wrote:
>> Your log shows that the subsequent search request initiates a new Bind to
>> the remote server, which implies that it's not re-using the same
>> connection as the first request. Since a paged results cookie is only
>> valid within the context of a single connection, you get this error
>> result.
>
> Given that the client making the request is ldapsearch, which is not going
> to rebind mid search, and you can see it is done with noprompt, so there's
> no human interaction here, it seems like back-ldap is buggy here. Why is
> *back-ldap* closing its connection to the upstream server and initiating a
> new bind?
You haven't provided any slapd configuration info. Without seeing that it's
way premature to claim there's any bug here.
Look at the log yourself. back-ldap *doesn't* close the connection. Don't spew
garbage about what it does when it didn't actually do such a thing.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
--On Monday, August 01, 2011 9:47 PM +0000 hyc(a)symas.com wrote:
> whm(a)stanford.edu wrote:
>> Full_Name: Bill MacAllister
>> Version: 2.4.26
>> OS: Debian 6
>> URL: ftp://ftp.openldap.org/incoming/
>> Submission from: (NULL) (171.64.19.165)
>>
>>
>> We typically setup local proxy servers to support applications that
>> cannot support a GSSAPI bind to the directory server. The proxy server
>> allows anonymous access to the directory for connections from the
>> localhost and connects to the master using GSSAPI. We are experiencing
>> a failures when we attempt to use the paged results control on the
>> proxy. For example:
>>
>> ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h
>> localhost "(&(objectClass=suPerson)(suVisibIdentity=world))" ou
>> telephonenumber title
>>
>> ends with the error:
>>
>> # search result
>> search: 5
>> result: 0 Success
>> control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
>> pagedresults: cookie=KIMAAAAAAAA=
>> # extended LDIF
>> #
>> # LDAPv3
>> # base<cn=people,dc=stanford,dc=edu> with scope subtree
>> # filter: (&(objectClass=suPerson)(suVisibIdentity=world))
>> # requesting: ou telephonenumber title
>> # with pagedResults control: size=1000
>> #
>>
>> # search result
>> search: 6
>> result: 2 Protocol error
>> text: paged results cookie is invalid
>>
>> # numResponses: 4005
>> # numEntries: 4000
>>
>> This result is not consistent. We have seen examples where 2000 and 3000
>> entries being returned and then the error. Another test that we
>> performed with a slightly more complex filter, i.e.
>>
>> "(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=wo
>> rld)))"
>>
>> returned usually returned 1000 entries before erroring.
>>
>> Issuing a similar search directly against the backend ldap server
>> completes without
>> error.
>>
>> We have seen the same behavior on OpenLDAP 2.4.23 as well.
>>
>> Logs generated running slapd standalone with '-d stats,packets' are
>> available at http://www.stanford.edu/~whm/files/ldap-debugging/.
>
> Your log shows that the subsequent search request initiates a new Bind to
> the remote server, which implies that it's not re-using the same
> connection as the first request. Since a paged results cookie is only
> valid within the context of a single connection, you get this error
> result.
Given that the client making the request is ldapsearch, which is not going
to rebind mid search, and you can see it is done with noprompt, so there's
no human interaction here, it seems like back-ldap is buggy here. Why is
*back-ldap* closing its connection to the upstream server and initiating a
new bind?
--Quanah
--
Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
whm(a)stanford.edu wrote:
> Full_Name: Bill MacAllister
> Version: 2.4.26
> OS: Debian 6
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (171.64.19.165)
>
>
> We typically setup local proxy servers to support applications that cannot
> support a GSSAPI bind to the directory server. The proxy server allows
> anonymous access to the directory for connections from the localhost and
> connects to the master using GSSAPI. We are experiencing a failures when
> we attempt to use the paged results control on the proxy. For example:
>
> ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h localhost
> "(&(objectClass=suPerson)(suVisibIdentity=world))" ou telephonenumber title
>
> ends with the error:
>
> # search result
> search: 5
> result: 0 Success
> control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
> pagedresults: cookie=KIMAAAAAAAA=
> # extended LDIF
> #
> # LDAPv3
> # base<cn=people,dc=stanford,dc=edu> with scope subtree
> # filter: (&(objectClass=suPerson)(suVisibIdentity=world))
> # requesting: ou telephonenumber title
> # with pagedResults control: size=1000
> #
>
> # search result
> search: 6
> result: 2 Protocol error
> text: paged results cookie is invalid
>
> # numResponses: 4005
> # numEntries: 4000
>
> This result is not consistent. We have seen examples where 2000 and 3000
> entries being returned and then the error. Another test that we performed with
> a slightly more complex filter, i.e.
>
> "(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=world)))"
>
> returned usually returned 1000 entries before erroring.
>
> Issuing a similar search directly against the backend ldap server completes
> without
> error.
>
> We have seen the same behavior on OpenLDAP 2.4.23 as well.
>
> Logs generated running slapd standalone with '-d stats,packets' are available at
> http://www.stanford.edu/~whm/files/ldap-debugging/.
Your log shows that the subsequent search request initiates a new Bind to the
remote server, which implies that it's not re-using the same connection as the
first request. Since a paged results cookie is only valid within the context
of a single connection, you get this error result.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Full_Name: Bill MacAllister
Version: 2.4.26
OS: Debian 6
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (171.64.19.165)
We typically setup local proxy servers to support applications that cannot
support a GSSAPI bind to the directory server. The proxy server allows
anonymous access to the directory for connections from the localhost and
connects to the master using GSSAPI. We are experiencing a failures when
we attempt to use the paged results control on the proxy. For example:
ldapsearch -E pr=1000/noprompt -x -b "cn=people,dc=stanford,dc=edu" -h localhost
"(&(objectClass=suPerson)(suVisibIdentity=world))" ou telephonenumber title
ends with the error:
# search result
search: 5
result: 0 Success
control: 1.2.840.113556.1.4.319 false MA0CAQAECCiDAAAAAAAA
pagedresults: cookie=KIMAAAAAAAA=
# extended LDIF
#
# LDAPv3
# base <cn=people,dc=stanford,dc=edu> with scope subtree
# filter: (&(objectClass=suPerson)(suVisibIdentity=world))
# requesting: ou telephonenumber title
# with pagedResults control: size=1000
#
# search result
search: 6
result: 2 Protocol error
text: paged results cookie is invalid
# numResponses: 4005
# numEntries: 4000
This result is not consistent. We have seen examples where 2000 and 3000
entries being returned and then the error. Another test that we performed with
a slightly more complex filter, i.e.
"(&(objectClass=suPerson)(|(suVisibIdentity=world)(suVisibIdentity=world)))"
returned usually returned 1000 entries before erroring.
Issuing a similar search directly against the backend ldap server completes
without
error.
We have seen the same behavior on OpenLDAP 2.4.23 as well.
Logs generated running slapd standalone with '-d stats,packets' are available at
http://www.stanford.edu/~whm/files/ldap-debugging/.
Bill
Full_Name: Stephan
Version: 2.4.26
OS: Debian 5.0.8
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (134.100.101.1)
Hi everybody,
I have an odd problem with synchronisation. I have 5 nodes running multi master
synchronisation. I recently updated one of them to 2.4.26 (they were all running
2.4.23 before) and started it with an empty database (the database directory
contained nothing more than the DB_CONFIG file). Afterwards I noted that some
attributes were missing, especially in the root of the directory tree, which
looks like this:
dn: o=test
objectClass: organization
o: test
After the sync was complete, I could search for the o=test entry but couldn't
see it:
ldapsearch -H "ldaps://filled.test.xyz" -D cn=superman,o=test -W -x -b o=test
o=test
# extended LDIF
#
# LDAPv3
# base <o=test> with scope subtree
# filter: o=test
# requesting: ALL
#
# search result
search: 2
result: 0 Success
# numResponses: 1
I played around a bit and am able to reproduce the problem. I tried combinations
of 2.4.23 and 2.4.26, bdb and hdb backends, bdb versions 4.5.20-13 and 4.6.21
but the problem persists. If I take the output of slapcat on one of the nodes
and feed it to the empty node before its startup everything seems to be ok.
I increased the loglevel and can see the following (rid 1 being the node with
the filled database and rid 2 the one with the empty database):
On the empty node (rid2) I get lots of 'syncrepl_entry: rid=001 inserted UUID
xxx'
'syncprov_matchops: skipping original sid 001' and 'syncrepl_entry: rid=001
be_add'
Once the sync is complete the log on the formerly empty node says:
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE -
REFRESH_DELETE
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001
cookie=rid=001,sid=001,csn=20110310162449.396663Z#000000#000#000000;20110801135804.526931Z#000000#001#000000;20110614092017.772141Z#000000#003#000000;20110722131433.793594Z#000000#004#000000;20110225131904.168266Z#000000#005#000000
Aug 1 17:24:55 empty slapd[7796]: slap_queue_csn: queing 0x9f901da8
20110801135804.526931Z#000000#001#000000
Aug 1 17:24:55 empty slapd[7796]: syncprov_matchops: skipping original sid 001
Aug 1 17:24:55 empty slapd[7796]: slap_graduate_commit_csn: removing 0x9f91f5f8
20110801135804.526931Z#000000#001#000000
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 cookie=
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_MODIFY)
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001 be_search (0)
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001 o=test
Aug 1 17:24:55 empty slapd[7796]: syncrepl_entry: rid=001 entry unchanged,
ignored (o=test)
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 LDAP_RES_INTERMEDIATE -
NEW_COOKIE
Aug 1 17:24:55 empty slapd[7796]: do_syncrep2: rid=001 NEW_COOKIE:
rid=001,sid=001,csn=20110801152321.168164Z#000000#000#000000;20110801135804.526931Z#000000#001#000000;20110614092017.772141Z#000000#003#000000;20110722131433.793594Z#000000#004#000000;20110225131904.168266Z#000000#005#000000
On the node from which it took the database (rid 1):
Aug 1 17:24:54 filled slapd[17862]: do_syncrep2: rid=002 LDAP_RES_INTERMEDIATE
- SYNC_ID_SET
Aug 1 17:24:54 filled last message repeated 7 times
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 be_search (0)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 o=test
Aug 1 17:24:54 filled slapd[17862]: syncprov_sendresp: cookie=
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 be_modify o=test
(0)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
Aug 1 17:24:54 filled slapd[17862]: dn_callback : entries have identical CSN
sambaDomainName=XYZ,ou=abc,o=test 20110722131010.624524Z#000000#004#000000
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 be_search (0)
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002
sambaDomainName=XYZ,ou=abc,o=test
Aug 1 17:24:54 filled slapd[17862]: syncrepl_entry: rid=002 entry unchanged,
ignored (sambaDomainName=XYZ,ou=abc,o=test)
Aug 1 17:24:54 filled slapd[17862]: do_syncrep2: rid=002 LDAP_RES_INTERMEDIATE
- REFRESH_PRESENT
Aug 1 17:24:54 filled slapd[17862]: do_syncrep2: rid=002
cookie=rid=002,csn=20110801152321.168164Z#000000#000#000000
Aug 1 17:24:54 filled slapd[17862]: slap_queue_csn: queing 0x7eff600022c0
20110801152321.168164Z#000000#000#000000
Aug 1 17:24:54 filled slapd[17862]: slap_graduate_commit_csn: removing
0x7eff60007040 20110801152321.168164Z#000000#000#000000
It seems to me that the formerly empty node overwrites the entry on the node
from which it took all the entries.
This is what my config looks like:
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
syncprov-reloadhint TRUE
serverID 1 "ldaps://filled.test.xyz"
serverID 2 "ldaps://empty.test.xyz"
#filled
syncrepl rid=2
provider=ldaps://filled.test.xyz:636
type=refreshAndPersist
searchbase="o=test"
retry="120 20 300 5 1800 5 3600 +"
scope=sub
bindmethod=simple
binddn="cn=superman,o=test"
credentials="hello"
tls_reqcert=never
mirrormode on
And the other node:
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 100
syncprov-reloadhint TRUE
serverID 1 "ldaps://filled.test.xyz"
serverID 2 "ldaps://empty.test.xyz"
#filled
syncrepl rid=2
provider=ldaps://empty.test.xyz:636
type=refreshAndPersist
searchbase="o=test"
retry="120 20 300 5 1800 5 3600 +"
scope=sub
bindmethod=simple
binddn="cn=superman,o=test"
credentials="hello"
tls_reqcert=never
mirrormode on
It seems to me that the formerly empty node overwrites the other one after the
sync.
In some cases the entries don't go missing on the source node, they just don't
get replicated to the formerly emtpy node.
Unless I made some terrible mistake in the config this is seems to be a bug
since in the administrator's guide
http://www.openldap.org/doc/admin24/replication.html#N-Way%20Multi-Master%2…
it says: "Syncrepl is self-synchronizing; you can start with a consumer database
in any state from totally empty to fully synced and it will automatically do the
right thing to achieve and maintain synchronization"
All nodes are configured to use the same ntp source.
Full_Name: Mahesh P
Version: 2.4.23
OS: Linux
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (206.126.170.20)
Hi,
We have one application which will use the Oracle DB, OpenLDAP and Curl
libraries.
We have started the LDAP server. And we want to perform search/add/update
operations using LDAP calls ldap_search_ext_s(), ldap_add_ext_s() and
ldap_modify_ext_s().
While performing the search Operation, getting the "Bad Search Filter" error.
In the application code we have used the ldap_initialize() for getting the LDAP
handle.I am thinking this problem is result of the collision of LDAP calls
between Oracle Library and OpenLDAP,Curl libraries.
When we did nm on libclntsh.so.11.1, got the references for
0000000001055732 T ldap_search_ext
0000000001055594 T ldap_search_ext_s
0000000001051714 T ora_ldap_search_ext
000000000105145a T ora_ldap_search_ext_s
If we try to have a standalone program without any database connectivity same
LDAP code is working fine for the search/add/modify operations.
Kindly please advise how we can avoid the problem if we use Oracle DB, OpenLDAP
and CURL functionality in out application code.
Thanks
Mahesh
hyc(a)symas.com wrote:
> I guess we
> need to keep using the NULL timeout while the connection is actually in its
> Refresh phase, before it enters its Persist phase.
Fixed now in master, please test.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
ml+openldap(a)esmtp.org wrote:
> On Fri, Jul 29, 2011, Howard Chu wrote:
>
>> The ITS mailer only supports plaintext, no attachments.
>
> Sorry about that. Here's the log inline (5 attempts by syncrepl):
I was able to reproduce this behavior. The culprit is the ldap_result() call
in do_syncrep2() returning 0 due to a timeout. The actual sequence of events
is that do_syncrep1() successfully binds to the master and sends a syncsearch
request, then do_syncrep2() reads the search response. The first message is a
Present UUID list which it parses successfully. It then loops back to look for
the next message from the provider, but due to whatever timing vagaries, that
message is not available yet. But anyway, do_syncrep2() returns with rc=0 and
this resets the retry counter, as I said before.
At the moment I don't have any good ideas how this could be fixed in the code.
Timeouts of this sort are normal behavior for refreshAndPersist. I guess we
need to keep using the NULL timeout while the connection is actually in its
Refresh phase, before it enters its Persist phase.
I think you could workaround it by using delta-syncrepl, which doesn't use a
Present UUID list.
This will require some more thought.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/