Quanah,
In this specific test I reported in my previous e-mail it was used glibc.
Now I tested with tcmalloc and the memory again was much more consumed and never released.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10652 ldap 18 0 2006m 1.8g 68m S 100 15.8 8:36.75 slapd
Once it get this much memory it then stabilize but it is much more than before that was 385m.
The scenario was :
1) Start a full ldapsearch for one specific dn(the DB has 4 dn's); 2) After sometime start a new full ldapsearch to behave as queries to DB when replication for LDIF backup is being done; 3) Export one of the queries to a file so speed verification can be done with grep and wc.
After sometime the memory starts to be consumed in large chunks until it stopped in around 2G.
Since one of the queries is being redirect to a file I can tail to have a visual rate of the entrances being returned by slapd. As the issue before after sometime the rate appears to get stuck and come in small 16 entrances chunks but only after some minutes.
This behavior above appears to happen after the second ldapsearch fills its own dn cache even monitor always shows only one information as I could see.
[root@brtldp11 ~]# date; grep -e '^pnnum*' /backup/temp.ldif |wc -l Mon Jan 26 21:43:31 BRST 2009 482432 [root@brtldp11 ~]# date; grep -e '^pnnum*' /backup/temp.ldif |wc -l Mon Jan 26 21:44:56 BRST 2009 482448
This behavior is similar as before with the initial issue with a single process.
I just notice when slapd starts there are these threads :
[root@brtldp12 ~]# ps -mu ldap PID TTY TIME CMD 10638 pts/2 00:00:00 slapd - - 00:00:00 - - - 00:00:00 -
Then after the 2 queries are started I have :
[root@brtldp12 ~]# ps -mu ldap PID TTY TIME CMD 10195 pts/2 00:44:22 slapd - - 00:00:00 - - - 00:00:00 - - - 00:26:09 - - - 00:18:13 - - - 00:00:00 -
So 3 new threads. Looks like for some reason with multiples queries the issue with response getting stuck repeats.
Even one ldapsearch is stopped the number of threads continues the same, memory continues allocated and the other search get with this slow rate.
These outputs are using now tcmalloc but similar as with glibc.
Regards,
Rodrigo.
--- On Mon, 1/26/09, Quanah Gibson-Mount quanah@zimbra.com wrote:
From: Quanah Gibson-Mount quanah@zimbra.com Subject: Re: (ITS#5860) slapd memeory leak under openldap 2.4 To: rlvcosta@yahoo.com, openldap-its@openldap.org Date: Monday, January 26, 2009, 9:25 PM --On Monday, January 26, 2009 3:18 PM -0800 Rodrigo Costa rlvcosta@yahoo.com wrote:
Quanah and Howard,
Sorry to come back.
I tried to make 2 ldapsearch to the same DB
simultaneously, doing some
stress test like dump the LDIF and queries coming.
The slapd after sometime start to get more memory. By
what I could
observe looks like new thread was started consuming
again the same memory
as before so the slad now is consuming around the
double of memory as
before.
Are you also using an alternate malloc() instead of glibc? I know you were investigating this, just want to confirm you are using tcmalloc or hoard and not glibc.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc
Zimbra :: the leader in open source messaging and collaboration