--On Thursday, September 30, 2010 4:59 PM +0000 quanah@zimbra.com wrote:
--On Thursday, September 30, 2010 11:14 AM +0000 masarati@aero.polimi.it wrote:
There's no mention of overlays; I was wondering whether or not it could be related to some adverse interaction, e.g. with sssvlv, which has some paged-results specific code. Can you also post on overlays either global or specific to "olcDatabase={2}hdb"?
There are no overlays being used. ;) As I said, a very vanilla config.
Howard and I continue to examine this problem, but as yet, have found no solution nor insight into why the problem exists. It has nothing to do with paged results, and you can see the behavior relatively easily with a smaller DB than I was working with. Right now, I'm using a 1 million entry DB. If I execute two concurrent searches of the database with a 25,000 entry cache limit, that frees 1,000 entries on purge, at the end of those searches, we end up with:
counted 1000087 entryInfos and 114476 entries, 0 notcached
What this shows is that after the search completed, there are 114,476 entries in the entry cache, even though it only reports between 24,000 and 25,000. Repeated additional searches will continue to increase the number of cached entries until all memory is exhausted.
The following code is used to count up the cached entries, and it is called from inside gdb via: call bdb_lru_count(&bdb->bi_cache)
--- openldap-2.4.23/servers/slapd/back-bdb/cache.c.orig 2010-10-05 16:20:27.085259568 -0700 +++ openldap-2.4.23/servers/slapd/back-bdb/cache.c 2010-10-05 16:21:03.655255057 -0700 @@ -1530,6 +1530,35 @@ }
#ifdef LDAP_DEBUG +static void +bdb_lru_count( Cache *cache ) +{ + EntryInfo *e; + int ei = 0, ent = 0, nc = 0; + + for ( e = cache->c_lrutail; ; ) { + ei++; + if ( e->bei_e ) { + ent++; + if ( e->bei_state & CACHE_ENTRY_NOT_CACHED ) + nc++; + } + e = e->bei_lrunext; + if ( e == cache->c_lrutail ) + break; + } + fprintf( stderr, "counted %d entryInfos and %d entries, %d notcached\n", + ei, ent, nc ); + ei = 0; + for ( e = cache->c_lrutail; ; ) { + ei++; + e = e->bei_lruprev; + if ( e == cache->c_lrutail ) + break; + } + fprintf( stderr, "counted %d entryInfos (on lruprev)\n", ei ); +} + #ifdef SLAPD_UNUSED static void bdb_lru_print( Cache *cache )
If anyone else has some thoughts on why the entry cache is growing unbounded, much appreciated.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration