ryan@nardis.ca wrote:
bisect identifies c365ac359e9c9b483b934c2a1f0bc552645c32fa as the commit that introduced this behaviour.
003dfbda574f37bbf1a2240f530ff9fa35ab0801 on RE24 (2.4.20)
commit c365ac359e9c9b483b934c2a1f0bc552645c32fa Author: Howard Chu hyc@openldap.org Date: Sun Nov 22 04:42:00 2009 +0000
ITS#6368 use dup'd entries in response queue
I've run your reproducer and see no memory leak. The response queue will indeed grow without bound if the consumer runs slower than the provider, and doesn't read responses fast enough. But in the case of this test script, eventually the client finishes and the consumer catches up.
The provider's process size may not decrease, but that just means the malloc implementation isn't returning freed memory to the kernel - it's not a leak.
This can be verified using mleak, and using SIGPROF to snapshot the memory usage of the provider. The simplest way to force the memory use to grow is to first suspend the consumer with SIGSTOP. Let the modify client run to completion. mleak / SIGPROF will show a large amount of memory in use. Resume the consumer with SIGCONT, let it run to completion, and then check with SIGPROF on the provider again - all of the response queue memory is freed.
So, conclusively, there is no actual leak. But there's a problem with sustained client modifications when the consumer is too slow. Our options here are to configure a size limit on the response queue, and hang the client when the limit is hit, or to return LDAP_BUSY to the client. Neither of these are very attractive options.
Doing batched commits will speed up the consumer, but that feature is only in 2.5.