Re: (ITS#5112) memory leak in pcache overlay
by rhafer＠suse.de
On Wednesday 29 August 2007 17:36, Howard Chu wrote:
> rhafer(a)suse.de wrote:
> > Full_Name: Ralf Haferkamp
> > Version: RE23, HEAD
> >
> > Valgrind gives me the log pasted below when I abort an ldapsearch command
> > (CTRL-C) that is running against a back-ldap database that uses the
> > pcache overlay.
>
> Makes sense. We need to use the cleanup handler and free the saved query
> info if op->o_abandon is set. Do you want to code this up?
I am still working on that, but checking for op->o_abandon doesn't seem to be
enough. E.g. if I abort the ldapsearch while slapd is updating its cache (all
entries have been returned, but the final LDAP_RESULT to the client is still
missing) a filter_free in the cleanup handler will invalidly free the filter.
We somehow need to make sure that the query is only free'd if nothing has been
written to the cache database.
--
Ralf
16 years, 3 months
Re: (ITS#5112) memory leak in pcache overlay
by hyc＠symas.com
rhafer(a)suse.de wrote:
> Full_Name: Ralf Haferkamp
> Version: RE23, HEAD
> Valgrind gives me the log pasted below when I abort an ldapsearch command
> (CTRL-C) that is running against a back-ldap database that uses the pcache
> overlay.
Makes sense. We need to use the cleanup handler and free the saved query info
if op->o_abandon is set. Do you want to code this up?
> ==20850== 53,092 (35,830 direct, 17,262 indirect) bytes in 883 blocks are
> definitely lost in loss record 17 of 18
> ==20850== at 0x4C22AC6: malloc (in
> /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
> ==20850== by 0x508A940: ber_memalloc_x (memory.c:226)
> ==20850== by 0x4ABEEE: slap_sl_malloc (sl_malloc.c:273)
> ==20850== by 0x4487C0: filter_dup (filter.c:801)
> ==20850== by 0x5AB151: pcache_op_search (pcache.c:2237)
> ==20850== by 0x4C3F41: overlay_op_walk (backover.c:642)
> ==20850== by 0x4C41D3: over_op_func (backover.c:704)
> ==20850== by 0x4C4269: over_op_search (backover.c:726)
> ==20850== by 0x446210: fe_op_search (search.c:369)
> ==20850== by 0x445B2E: do_search (search.c:217)
> ==20850== by 0x4427DB: connection_operation (connection.c:1145)
> ==20850== by 0x442CA6: connection_read_thread (connection.c:1271)
>
> The database configuration looks like this:
> -----------------------------
> database ldap
> suffix o=test
> uri ldap://xxxxxxxxxxxxxxxxx
> readonly on
> lastmod off
>
> overlay pcache
> proxycache bdb 100000 1 10000 180
> proxyattrset 0 givenname uid ou o cn sn mail objectclass
>
> ###
> proxytemplate (|(cn=)(mail=)(uid=)) 0 86400
>
> directory /var/lib/ldap/pcache
> cachesize 10000
> index objectclass,queryid eq
> index sn,cn,givenname,uid,mail pres,eq,sub
> -----------------------------
>
>
>
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
16 years, 3 months
(ITS#5113) pcache returns incomplete results
by rhafer＠suse.de
Full_Name: Ralf Haferkamp
Version: HEAD
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (89.166.180.39)
While the results of a query are being cached, slapo-pcache will answer queries
that match the same template from the cache that is currently being populated.
This means that subsequent queries will get incomplete results until the
original query is completely cached.
This doesn't happen with current RE23 code.
16 years, 3 months
(ITS#5112) memory leak in pcache overlay
by rhafer＠suse.de
Full_Name: Ralf Haferkamp
Version: RE23, HEAD
OS: -
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (89.166.180.39)
Valgrind gives me the log pasted below when I abort an ldapsearch command
(CTRL-C) that is running against a back-ldap database that uses the pcache
overlay.
==20850== 53,092 (35,830 direct, 17,262 indirect) bytes in 883 blocks are
definitely lost in loss record 17 of 18
==20850== at 0x4C22AC6: malloc (in
/usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)
==20850== by 0x508A940: ber_memalloc_x (memory.c:226)
==20850== by 0x4ABEEE: slap_sl_malloc (sl_malloc.c:273)
==20850== by 0x4487C0: filter_dup (filter.c:801)
==20850== by 0x5AB151: pcache_op_search (pcache.c:2237)
==20850== by 0x4C3F41: overlay_op_walk (backover.c:642)
==20850== by 0x4C41D3: over_op_func (backover.c:704)
==20850== by 0x4C4269: over_op_search (backover.c:726)
==20850== by 0x446210: fe_op_search (search.c:369)
==20850== by 0x445B2E: do_search (search.c:217)
==20850== by 0x4427DB: connection_operation (connection.c:1145)
==20850== by 0x442CA6: connection_read_thread (connection.c:1271)
The database configuration looks like this:
-----------------------------
database ldap
suffix o=test
uri ldap://xxxxxxxxxxxxxxxxx
readonly on
lastmod off
overlay pcache
proxycache bdb 100000 1 10000 180
proxyattrset 0 givenname uid ou o cn sn mail objectclass
###
proxytemplate (|(cn=)(mail=)(uid=)) 0 86400
directory /var/lib/ldap/pcache
cachesize 10000
index objectclass,queryid eq
index sn,cn,givenname,uid,mail pres,eq,sub
-----------------------------
16 years, 3 months
Re: (ITS#5106) LDAPI problem on amd64
by ando＠sys-net.it
> I updated to 2.3.38 and the problem seems to have vanished.
> Thanks for the hint.
Thanks for the prompt feedback. I've closed this ITS.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
---------------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Email: pierangelo.masarati(a)sys-net.it
---------------------------------------
16 years, 3 months
Re: (ITS#5106) LDAPI problem on amd64
by svgmuc＠bluebottle.com
Hi,
Quoting Pierangelo Masarati <ando(a)sys-net.it>:
> You should be using the latest 2.3 release; there was quite a bit
> of > work on ldapi between 2.3.32 and 2.3.38 (ITS#4893 and
> other possibly related fixes). Having said this, I don't know if it
> will solve your problem, but at least it would make sure it's not
> fixed yet. No need to say that things work fine here on
> Linux/amd64.
I updated to 2.3.38 and the problem seems to have vanished.
Thanks for the hint.
Best,
Sven
----------------------------------------------------------------------
Finally - A spam blocker that actually works.
http://www.bluebottle.com/tag/4
16 years, 3 months
Re: (ITS#5065) out of order ctxcsn from old entryCSN
by ando＠sys-net.it
> Old entryCSN values imported into the data from another server, can crash
> replicas.
The ITS reports that this is now fixed in HEAD. I would appreciate if you
feedback using a fresh HEAD build, since the CSN syntax is now more
thoroughly checked, although it is supposed to allow the old one in input.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
---------------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Email: pierangelo.masarati(a)sys-net.it
---------------------------------------
16 years, 3 months
slapd makes very strange troubles in VServer
by Niki Hammler
Hi,
Today I began to move my LDAP-Server to a Linux VServer jail. On the
host and on the VServer I run Debian etch 4.0. Installing slapd was no
problem (aptitude install slapd). I just copied all configuration and
data files from the host which worked perfectly before and copied them
1:1 into the VServer.
slapd starts without any warnings. But connecting with a client is not
possible, neither inside the VServer nor outside.
For security reasons, I just allow access with SSL or TLS-required (the
very same file worked greatly on the host!):
pidfile /var/run/slapd/slapd.pid
argsfile /var/run/slapd/slapd.args
loglevel 0
disallow bind_anon
require authc
security tls=1
TLSCACertificateFile /etc/ldap/ssl/cacert.crt
TLSCertificateFile /etc/ldap/ssl/ldap.crt
TLSCertificateKeyFile /etc/ldap/ssl/ldap.key
TLSVerifyClient never
All key files and database files are accessible by slapd (user=openldap,
group=openldap).
Connecting to ldap and ldaps works greatly from inside the VServer and
also from outside:
$ nc 192.168.0.2 ldap
^c
$ nc 192.168.0.2 ldaps
^c
$
There are no packet filters installed, debugging with tcpdump (local
interface) shows the complete TCP traffic and handshake with no errors.
The strange error is always the same: The clients do not want to
connect, sending SSL stuff is always possible but after that, nothing is
received.
Let's begin with plain SSL:
$ openssl s_client -connect 192.168.0.2:636 -debug
CONNECTED(00000003)
write to 0x80bee90 [0x80bf4e8] (118 bytes => 118 (0x76))
0000 - 80 74 01 03 01 00 4b 00-00 00 20 00 00 39 00 00 .t....K... ..9..
0010 - 38 00 00 35 00 00 16 00-00 13 00 00 0a 07 00 c0 8..5............
0020 - 00 00 33 00 00 32 00 00-2f 03 00 80 00 00 05 00 ..3..2../.......
0030 - 00 04 01 00 80 00 00 15-00 00 12 00 00 09 06 00 ................
0040 - 40 00 00 14 00 00 11 00-00 08 00 00 06 04 00 80 @...............
0050 - 00 00 03 02 00 80 f5 5d-10 d2 64 21 24 0c b1 c3 .......]..d!$...
0060 - 92 37 72 7a 4a 9c 09 88-46 b2 f8 39 ef f6 c4 c8 .7rzJ...F..9....
0070 - 84 74 8e a6 79 e1 .t..y.
read from 0x80bee90 [0x80c4a48] (7 bytes => 0 (0x0))
20500:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake
failure:s23_lib.c:188:
$
I did lots of tests with many LDAP clients (ldapwhoami, ldapsearch
commandline tools, PHP, Softerra LDAP Administrator and Clients in
exim4, Courier etc). In this case I show connecting with SSL using
ldapwhoami:
$ ldapwhoami -x -H ldaps://nobaq.intern.stiftingtal.net -D
"uid=nobaq,ou=users,dc=intern,dc=stiftingtal,dc=net" -w secret -v -d 999
ldap_initialize( ldaps://nobaq.intern.stiftingtal.net )
ldap_create
ldap_url_parse_ext(ldaps://nobaq.intern.stiftingtal.net)
ldap_extended_operation_s
ldap_extended_operation
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP nobaq.intern.stiftingtal.net:636
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 192.168.0.2:636
ldap_connect_timeout: fd: 3 tm: -1 async: 0
TLS trace: SSL_connect:before/connect initialization
tls_write: want=118, written=118
0000: 80 74 01 03 01 00 4b 00 00 00 20 00 00 39 00 00 .t....K... ..9..
0010: 38 00 00 35 00 00 16 00 00 13 00 00 0a 07 00 c0 8..5............
0020: 00 00 33 00 00 32 00 00 2f 03 00 80 00 00 05 00 ..3..2../.......
0030: 00 04 01 00 80 00 00 15 00 00 12 00 00 09 06 00 ................
0040: 40 00 00 14 00 00 11 00 00 08 00 00 06 04 00 80 @...............
0050: 00 00 03 02 00 80 20 25 c9 bb fe 68 e8 52 6c 87 ...... %...h.Rl.
0060: 04 ae 25 d1 8b f5 73 c8 3d ca 73 ab e3 92 3f 9f ..%...s.=.s...?.
0070: c9 9d 79 4e aa fb ..yN..
TLS trace: SSL_connect:SSLv2/v3 write client hello A
tls_read: want=7, got=0
TLS: can't connect.
ldap_perror
ldap_start_tls: Can't contact LDAP server (-1)
$
As you can see, reading fails. And now using a normal connection but
with TLS:
ldapwhoami -x -H ldap://192.168.0.2 -D
"uid=nobaq,ou=users,dc=intern,dc=stiftingtal,dc=net" -w secret -v -d 999 -ZZ
ldap_initialize( ldap://192.168.0.2 )
ldap_create
ldap_url_parse_ext(ldap://192.168.0.2)
ldap_extended_operation_s
ldap_extended_operation
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 192.168.0.2:389
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 192.168.0.2:389
ldap_connect_timeout: fd: 3 tm: -1 async: 0
ldap_open_defconn: successful
ldap_send_server_request
ber_scanf fmt ({it) ber:
ber_scanf fmt ({) ber:
ber_flush: 31 bytes to sd 3
0000: 30 1d 02 01 01 77 18 80 16 31 2e 33 2e 36 2e 31 0....w...1.3.6.1
0010: 2e 34 2e 31 2e 31 34 36 36 2e 32 30 30 33 37 .4.1.1466.20037
ldap_write: want=31, written=31
0000: 30 1d 02 01 01 77 18 80 16 31 2e 33 2e 36 2e 31 0....w...1.3.6.1
0010: 2e 34 2e 31 2e 31 34 36 36 2e 32 30 30 33 37 .4.1.1466.20037
ldap_result ld 0x8051088 msgid 1
ldap_chkResponseList ld 0x8051088 msgid 1 all 1
ldap_chkResponseList returns ld 0x8051088 NULL
wait4msg ld 0x8051088 msgid 1 (infinite timeout)
wait4msg continue ld 0x8051088 msgid 1 all 1
** ld 0x8051088 Connections:
* host: 192.168.0.2 port: 389 (default)
refcnt: 2 status: Connected
last used: Wed Aug 29 00:04:19 2007
** ld 0x8051088 Outstanding Requests:
* msgid 1, origid 1, status InProgress
outstanding referrals 0, parent count 0
** ld 0x8051088 Response Queue:
Empty
ldap_chkResponseList ld 0x8051088 msgid 1 all 1
ldap_chkResponseList returns ld 0x8051088 NULL
ldap_int_select
read1msg: ld 0x8051088 msgid 1 all 1
ber_get_next
ldap_read: want=8, got=8
0000: 30 0c 02 01 01 78 07 0a 0....x..
ldap_read: want=6, got=6
0000: 01 00 04 00 04 00 ......
ber_get_next: tag 0x30 len 12 contents:
read1msg: ld 0x8051088 msgid 1 message type extended-result
ber_scanf fmt ({eaa) ber:
read1msg: ld 0x8051088 0 new referrals
read1msg: mark request completed, ld 0x8051088 msgid 1
request done: ld 0x8051088 msgid 1
res_errno: 0, res_error: <>, res_matched: <>
ldap_free_request (origid 1, msgid 1)
ldap_free_connection 0 1
ldap_free_connection: refcnt 1
ldap_parse_extended_result
ber_scanf fmt ({eaa) ber:
ldap_parse_result
ber_scanf fmt ({iaa) ber:
ber_scanf fmt (}) ber:
ldap_msgfree
TLS trace: SSL_connect:before/connect initialization
tls_write: want=118, written=118
0000: 80 74 01 03 01 00 4b 00 00 00 20 00 00 39 00 00 .t....K... ..9..
0010: 38 00 00 35 00 00 16 00 00 13 00 00 0a 07 00 c0 8..5............
0020: 00 00 33 00 00 32 00 00 2f 03 00 80 00 00 05 00 ..3..2../.......
0030: 00 04 01 00 80 00 00 15 00 00 12 00 00 09 06 00 ................
0040: 40 00 00 14 00 00 11 00 00 08 00 00 06 04 00 80 @...............
0050: 00 00 03 02 00 80 83 33 38 7c c9 0a f9 0b 42 67 .......38|....Bg
0060: 76 97 03 8b e3 47 55 fe 74 ed 86 08 83 fe ed 88 v....GU.t.......
0070: 0e 40 53 97 e5 3e .@S..>
TLS trace: SSL_connect:SSLv2/v3 write client hello A
tls_read: want=7, got=0
TLS: can't connect.
ldap_perror
ldap_start_tls: Connect error (-11)
$
Connecting without the '-ZZ' seems to work, I get the "confidentiality
required" message:
$ ldapwhoami -x -H ldap://192.168.0.2 -D
"uid=nobaq,ou=users,dc=intern,dc=stiftingtal,dc=net" -w secret -v
ldap_initialize( ldap://192.168.0.2 )
ldap_bind: Confidentiality required (13)
additional info: TLS confidentiality required
$
So I guess the problem is SSL. As said, all SSL files are accessible and
there are no errors in the logs or when starting slapd with '-d'. The
certificates CN is exactly the hostname.
I'm very frustrated because I can't find any errors. I repeat: the very
same configuration worked perfectly on the host system.
Is this a bug?
Thank you very much in advance for any advice,
Niki
16 years, 3 months
