Re: (ITS#4715) proxy retries anonymously
by hyc＠symas.com
The proxycache has nothing to do with the connections to the remote
server. That's handled by the back-ldap or whatever backend you're
using. Since you haven't provided any of the relevant config info from
your slapd.conf there's no way to tell what you're doing there.
In fact your log shows that a Bind was performed on the retry, but it
was done anonymously. So it may be something as simple as a missing
config keyword (like rebind-as-user in back-ldap). Or it may possibly be
related to a bug in back-ldap's retry handling as Pierangelo mentioned
in his reply to you.
There is no proxycache bug here. At the moment, without further details,
it doesn't seem there's any OpenLDAP software bug here at all.
Ashish Gawarikar wrote:
>
> ------------------------------------------------------------------------
>
> The problem is that the connection to the backend server drops and the proxy
> cache has to reestablish it. Doing this, it doesn't do a BIND again and thus
> the query is executed with anonymous privileges and fails. See log file from
> the backend server:
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/
16 years, 11 months
Re: (ITS#4715) Proxy cache TTL does not seem to be working as documented
by hyc＠symas.com
Your log actually indicates that it attempted to re-query the remote
server but got an error from the remote server. Sounds like you have
other problems, unrelated to the proxycache. (Note that the pcache
overlay never returns a NO_SUCH_OBJECT error code itself, that can only
happen because the remote server returned that code.)
ashish(a)ratboy.net wrote:
> Full_Name: Ashish Gawarikar
> Version: 2.3.27
> OS: Linux - 2.6 based
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (63.211.143.38)
>
>
> Openldap 2.3.27 configured as proxy cache is behaving erratically.
>
> Entries being answered from the proxy cache sometimes result with a negative
> result, even though the LDAP server has the information.
>
> It appears like the proxycache "forgets" that it has gotten the answer, but
> doesn't re-query the target server to fetch the information again.
>
> Proxy cache returns 'no such object' for a previously found entry if the ttl
> has passed between queries:
>
> conn=0 op=2 SRCH base="dc=mail,dc=example,dc=com" scope=2 deref=0
> filter="(mailLocalAddress=account1(a)example.com)"
> conn=0 op=2 SRCH attr=maillocaladdress
> request done: ld 0x83dd3b8 msgid 3
> conn=0 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text=Search operation
> succeeded
>
> <-- wait "some" time here -->
>
> conn=0 op=3 SRCH base="dc=mail,dc=example,dc=com" scope=2 deref=0
> filter="(mailLocalAddress=account1(a)example.com)"
> conn=0 op=3 SRCH attr=maillocaladdress
> conn=0 op=3 ldap_back_retry: retrying URI="ldap://ldap.example.com:389"
> DN="cn=admin,dc=example,dc=com"
> request done: ld 0x83dd3b8 msgid 1
> request done: ld 0x83dd3b8 msgid 2
> conn=0 op=3 SEARCH RESULT tag=101 err=32 nentries=0 text=No such object:
> dc=mail,dc=example,dc=com
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/
16 years, 11 months
(ITS#4716) syncrepl - consumer missed some delete
by Paolo.Rossi.con＠h3g.it
Full_Name: Paolo Rossi
Version: 2.3.27
OS: solaris 8
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (88.149.168.124)
Hi,
Doing stress test on syncrepl, 1 producer --> 1 consumer, I've found a strange
issue:
Afer load the openLDAP cluster (1m entry whit random 1,2 or 3 sub dn), I've used
a script to make constant random changes on the producer.
Meanwhile I've do some test & check on consumer.
I've found a issue when, stopping the consumer for some minutes, whit a
continuous load versus producer, restarting the consumer it resync itself but
some DELETE operation of entire subDN went lost.
Doing a search I've noticed on producer:
dn: testid=106087406087,ou=profiles,dc=foo,dc=bar
structuralObjectClass: profile
entryUUID: e2543b6a-559a-1029-82d7-f1c24b7d324b
creatorsName: cn=Manager,dc=foo,dc=bar
modifiersName: cn=Manager,dc=foo,dc=bar
createTimestamp: 20050510122923Z
modifyTimestamp: 20050510122923Z
entryCSN: 20050510122923Z#000017#00#000000
entryDN: testid=106087406087,ou=profiles,dc=foo,dc=bar
subschemaSubentry: cn=Subschema
hasSubordinates: FALSE
but on consumer:
dn: testid=106087406087,ou=profiles,dc=foo,dc=bar
structuralObjectClass: profile
entryUUID: e2543b6a-559a-1029-82d7-f1c24b7d324b
creatorsName: cn=Manager,dc=foo,dc=bar
modifiersName: cn=Manager,dc=foo,dc=bar
createTimestamp: 20050510122923Z
modifyTimestamp: 20050510122923Z
entryCSN: 20050510122923Z#000017#00#000000
entryDN: testid=106087406087,ou=profiles,dc=foo,dc=bar
subschemaSubentry: cn=Subschema
hasSubordinates: TRUE
dn: subtestID=106087406087,testid=106087406087,ou=profiles,dc=foo,dc=bar
structuralObjectClass: profileAccount
entryUUID: e25462d4-559a-1029-82d8-f1c24b7d324b
creatorsName: cn=Manager,dc=foo,dc=bar
modifiersName: cn=Manager,dc=foo,dc=bar
createTimestamp: 20050510122923Z
modifyTimestamp: 20050510122923Z
entryCSN: 20050510122923Z#000018#00#000000
entryDN: subtestID=106087406087,testid=106087406087,ou=profiles,dc=foo,dc=bar
subschemaSubentry: cn=Subschema
hasSubordinates: FALSE
-----
only on producer:
hasSubordinates: FALSE
I've "loglevel config acl sync" on consumer, but there is no log of sync
operation on that dn.
Also restarting the cluster, replica remain desynched on that dn.
Solaris 8 sparc
openLDAP 2.3.27 (64 bit build)
BDB 4.2.52.5
hdb backend
Paolo
16 years, 11 months
(ITS#4715) Proxy cache TTL does not seem to be working as documented
by ashish＠ratboy.net
Full_Name: Ashish Gawarikar
Version: 2.3.27
OS: Linux - 2.6 based
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (63.211.143.38)
Openldap 2.3.27 configured as proxy cache is behaving erratically.
Entries being answered from the proxy cache sometimes result with a negative
result, even though the LDAP server has the information.
It appears like the proxycache "forgets" that it has gotten the answer, but
doesn't re-query the target server to fetch the information again.
Proxy cache returns 'no such object' for a previously found entry if the ttl
has passed between queries:
conn=0 op=2 SRCH base="dc=mail,dc=example,dc=com" scope=2 deref=0
filter="(mailLocalAddress=account1(a)example.com)"
conn=0 op=2 SRCH attr=maillocaladdress
request done: ld 0x83dd3b8 msgid 3
conn=0 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text=Search operation
succeeded
<-- wait "some" time here -->
conn=0 op=3 SRCH base="dc=mail,dc=example,dc=com" scope=2 deref=0
filter="(mailLocalAddress=account1(a)example.com)"
conn=0 op=3 SRCH attr=maillocaladdress
conn=0 op=3 ldap_back_retry: retrying URI="ldap://ldap.example.com:389"
DN="cn=admin,dc=example,dc=com"
request done: ld 0x83dd3b8 msgid 1
request done: ld 0x83dd3b8 msgid 2
conn=0 op=3 SEARCH RESULT tag=101 err=32 nentries=0 text=No such object:
dc=mail,dc=example,dc=com
16 years, 11 months
(ITS#4714) 'Connection' object not throwing LDAPException upon sudden server connection loss
by alejandro.robles＠tre.se
Full_Name: Alejandro Robles
Version: n/a
OS: Solaris 8
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (82.182.141.229)
SUMMARY
This issue relates to JLDAP software package
(novell-jldap-devel-2006.06.22-1unix.zip) whereby during a sudden server
connection loss the Connection object doesn't through an LDAPException upon an
LDAP Search operation. If this situation isn't detectec it causes the running
thread to hang where retrieving results by first invoking hasMoreElements()
method.
DETAILS
A client program of the form:
1{
2 Connect;
3 Bind;
4
5 <being loop>
6 try {Search;}
7 catch (LDAPException) {<do something>}
8 while(hasMoreElements()) {
9 retrieve;
10 }
11 Sleep 10secs;
12 <end loop>
13
14 Unbind;
15 Disconnect;
16}
If a connection loss is simulated while the program is sleeping (line 11), when
the thread wakes up and invokes the Search method (line 6) is doesn't through an
exception and thereafter the thread gets locked when running hasMoreElements()
on the results (line 8).
SOLUTION
The cause of the problem is belived to be within the Connection object in the
method 'writeMessage' (below extract from /com/novell/ldap/Connection.java
object version 1.88):
/*
* IOException could be due to a server shutdown notification which
* caused our Connection to quit. If so we send back a slightly
* different error message. We could have checked this a little
* earlier in the method but that would be an expensive check each
* time we send out a message. Since this shutdown request is
* going to be an infrequent occurence we check for it only when
* we get an IOException. shutdown() will do the cleanup.
*/
if( clientActive) { // We beliefe the connection was alive
if (unsolSvrShutDnNotification) { // got server shutdown
throw new LDAPException(
ExceptionMessages.SERVER_SHUTDOWN_REQ,
new Object[] { host, new Integer(port)},
LDAPException.CONNECT_ERROR, null,
ioe);
}
// Other I/O Exceptions on host:port are reported as is
throw new LDAPException(ExceptionMessages.IO_EXCEPTION,
new Object[] {host, new Integer(port)},
LDAPException.CONNECT_ERROR, null, ioe);
}
The fact that the Connection has been lost causes prior to issuing a search to
the server to set the value of clietActive to false. The latter in turn causes
that any IOExceptions within the method not be handled properly (not thrown at
all) causing the mentioned problem.
PROPOSED FIX
To add a default Exception throwing code for the cases where 'clientActive' is
false.
/*
* IOException could be due to a server shutdown notification which
* caused our Connection to quit. If so we send back a slightly
* different error message. We could have checked this a little
* earlier in the method but that would be an expensive check each
* time we send out a message. Since this shutdown request is
* going to be an infrequent occurence we check for it only when
* we get an IOException. shutdown() will do the cleanup.
*/
if( clientActive) { // We beliefe the connection was alive
if (unsolSvrShutDnNotification) { // got server shutdown
throw new LDAPException(
ExceptionMessages.SERVER_SHUTDOWN_REQ,
new Object[] { host, new Integer(port)},
LDAPException.CONNECT_ERROR, null,
ioe);
}
// Other I/O Exceptions on host:port are reported as is
throw new LDAPException(ExceptionMessages.IO_EXCEPTION,
new Object[] {host, new Integer(port)},
LDAPException.CONNECT_ERROR, null, ioe);
} else
// Other I/O Exceptions on host:port are reported as is
throw new LDAPException(ExceptionMessages.IO_EXCEPTION,
new Object[] {host, new Integer(port)},
LDAPException.CONNECT_ERROR, null, ioe);
}
16 years, 11 months
Re: (ITS#4702) out-of-memory on huge DB ?
by paolo.rossi.con＠h3g.it
Another day, another try.
Today I've tried the slapcat (32Bit) a 8 GB RAM solaris/sparc machine.
also this time I had:
core dumped
ch_malloc of 16392 bytes failed
ch_malloc.c:57: failed assertion `0'
allocation at coredump was
PID RSS VSZ %MEM TIME STIME COMMAND
29941 4164952 4168968 51.8 01:15:27 16:03:43 slapcat
It seems the famous 4GB allocation limit on 32Bit build ?
I will try the last shot, reloading a BD on a 64 Bit build;
but seems the 32Bit build, had no hope to finish the task :(
Paolo
16 years, 11 months
Re: (ITS#4713) test030-relay broken in HEAD
by ando＠sys-net.it
quanah(a)stanford.edu wrote:
> --On Monday, October 16, 2006 5:41 AM +0000 quanah(a)stanford.edu wrote:
>
>
>
>
>> I'll see if I can get a core from it.
>>
>
> Never mind, I found the problem in slapd.1.out:
>
> /usr/local/build/ldap-head-20061015/servers/slapd/.libs/lt-slapd:
> relocation error: ../servers/slapd/back-meta/.libs/back_meta-2-devel.so.0:
> undefined symbol: ldap_back_proxy_authz_ctrl
>
>
> Sounds like something isn't being linked right, or something isn't declared
> right in the back-meta build.
>
Right. Back-meta occasionally reuses stuff from back-ldap; apparently I
pulled et another function from there, so it seems you need to load
back_ldap.la as well to use back_meta.la. I wonder if there's any
better solution (e.g. statically link back_meta.la with back_ldap.la,
but then there would be duplicates when one loads both?). Too bad I
don't routinely check HEAD with all-modules builds, sorry.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
16 years, 11 months
Re: (ITS#4713) test030-relay broken in HEAD
by quanah＠stanford.edu
--On Monday, October 16, 2006 5:41 AM +0000 quanah(a)stanford.edu wrote:
> I'll see if I can get a core from it.
Never mind, I found the problem in slapd.1.out:
/usr/local/build/ldap-head-20061015/servers/slapd/.libs/lt-slapd:
relocation error: ../servers/slapd/back-meta/.libs/back_meta-2-devel.so.0:
undefined symbol: ldap_back_proxy_authz_ctrl
Sounds like something isn't being linked right, or something isn't declared
right in the back-meta build.
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
16 years, 11 months
Re: (ITS#4713) test030-relay broken in HEAD
by quanah＠stanford.edu
--On Monday, October 16, 2006 5:38 AM +0000 ando(a)sys-net.it wrote:
> quanah(a)stanford.edu wrote:
>> When running test030-relay, I get a failure.
>>
> works just fine here, on Linux 32 bit and Solaris 64 bit
>> Basic output shows:
>>
>> Using meta backend...
>>
>> Starting slapd on TCP/IP port 9011...
>> Using ldapsearch to check that slapd is running...
>> Using ldapadd to populate the database...
>> Searching base="dc=example,dc=com"...
>> Searching base="o=Example,c=US"...
>> Search failed (255)!
>> ./scripts/test030-relay: line 79: kill: (20796) - No such process
>>
> Sounds like slapd either stopped or crashed; anything relevant from
> slapd's logs (or any core dump around)?
>> The ldapsearch.out file shows:
>>
> snip...
>> # searching base="o=Example,c=US"...
>> ldap_result: Can't contact LDAP server (-1)
>>
> This is not much indicative of the reason of the failure; see above.
> Note that Saturday 14 I made sort of a mess in HEAD with daemon.c, which
> I only noticed and fixed late Sunday 15; please make sure you're using
> the correct files from HEAD.
It is HEAD as of now, i.e., I retrieved it this evening. ;)
I'll see if I can get a core from it.
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
16 years, 11 months