On Friday 08 February 2008 08:24:44 Tony Earnshaw wrote:

Kick, Claus skrev, on 07-02-2008 15:40:

...

how can I check whether caching queries and so on works?

Works on what? In recent versions of OpenLDAP query caching isn't possible, only connection caching.

???

SLAPO-PCACHE(5) SLAPO-PCACHE(5)

NAME slapo-pcache - proxycache overlay

SYNOPSIS /etc/openldap/slapd.conf

DESCRIPTION The pcache overlay to slapd(8) allows caching of LDAP search requests (queries) in a local database. For an incoming query, the proxy cache determines its corresponding template. If the template was specified as cacheable using the proxytemplate directive and the request is con‐ tained in a cached request, it is answered from the proxy cache. Oth‐ erwise, the search is performed as usual and cacheable search results are saved in the cache for use in future queries.

[...]

Or, are you implying that pcache is broken in 2.3.3x (I don't use it myself).

Regards, Buchan

Tony Earnshaw wrote:

Buchan Milne skrev, on 08-02-2008 16:29:

[...]

...

DESCRIPTION The pcache overlay to slapd(8) allows caching of LDAP search requests (queries) in a local database. For an incoming query, the proxy cache determines its corresponding template. If the template was specified as cacheable using the proxytemplate directive and the request is con‐ tained in a cached request, it is answered from the proxy cache. Oth‐ erwise, the search is performed as usual and cacheable search results are saved in the cache for use in future queries.

[...]

Or, are you implying that pcache is broken in 2.3.3x (I don't use it myself).

"Well", he answered cautiously, "it (at least the 2.4 man page) also goes on to say:

CAVEATS Caching data is prone to inconsistencies because updates on the remote server will not be reflected in the response of the cache at least (and at most) for the duration of the proxytemplate TTL."

The general acceptance of any-brand query caching by LDAP "experts" such as Victor Duchovny of Postfix fame is that ... leave it alone, we don't support it, if you try it and it doesn't work, then I don't want to know.

And that's a safe and reasonable answer from someone who doesn't understand the material and doesn't want to learn about it.

My original answer to OP still stands: he should get equivalent functionality from connection caching.

No. Connection caching is helpful but it's nowhere near as effective at boosting performance. Still, you need to have many factors working in your favor before you can safely do query caching. E.g., all of the querying clients must have equivalent privileges on the remote server, otherwise some clients may get incomplete results from the cache, while others may get excess results (info they would not otherwise have access to). You need to understand how frequently the remote data changes, and how frequently it's accessed, so that you can set a useful TTL on the cached data. It is certainly error prone if you don't take the time to study your environment before deploying it.

"Kick, Claus" claus.kick@siemens.com writes:

Hello everyone,

Okay, I am confused now: if I specify something like this:

overlay proxycache proxycache bdb 100000 1 1000 100 proxyAttrset 0 memberUid teamsiteuserrole uniqueMember uidNumber gidNumber objectClass proxyTemplate (cn=*,ou=Role,o=*********) 0 3600 proxyTemplate (cn=*,ou=Group,o=***********) 0 3600

That should cache queries? How can I see that the settings are coresponding to the queries? How can I see whether something comes from the cache at all?

The only thing I see is:

Could you be a bit more specific about your requirements? Although pcache can be attached to any backend, it only makes sense when connecting to a remote server, that is either back-ldap, back-meta and back-mysql.

On a side note: Is there some hint book out there what indeces to use depending on what queries are being received?

You may run debugging mode pcache, but if you insist on a book: http://www.dpunkt.de/buecher/2104.html

-Dieter

Hi,

"Kick, Claus" claus.kick@siemens.com writes:

Hello everyone,

...

Could you be a bit more specific about your requirements?

You are right, I should have been. We are using OpenLDAP to manage a lot of users for a huge CMS. The ldap server runs on localhost. After an OS + hardware + ldap server upgrade (we used Netscape Directory Server 4.16 before), the whole CMS was very slow and from what we have deduced, there is no ldap query caching going on. So this might in part be responsible (there are *a lot of* ldap queries).

In fact you may configure the backend (back-bdb) and the frontend caches.

...

Although pcache can be attached to any backend, it only makes sense when connecting to a remote server, that is either back-ldap, back-meta and back-mysql.

Okay, so pcache does not really help in our case for we do not have an ldap server behind the ldap server, right?

Yes

So, what else can we do? Tony Earnshaw posted about connection caching, how is that enabled? I must be too blind to find it. :(

man slapd-bdb(5), configuration parameters cachesize, idlcachesize, shm_key, dncachesize (openldap-2.4 only) and proper DB_CONFIG cachesize settings. http://www.openldap.org/faq/data/cache/1072.html http://www.openldap.org/faq/data/cache/1075.html

-Dieter

On Monday 11 February 2008 11:32:58 Kick, Claus wrote:

Hello everyone,

...

Could you be a bit more specific about your requirements?

You are right, I should have been. We are using OpenLDAP to manage a lot of users for a huge CMS. The ldap server runs on localhost. After an OS + hardware + ldap server upgrade (we used Netscape Directory Server 4.16 before), the whole CMS was very slow and from what we have deduced, there is no ldap query caching going on.

So, you want to use a cache on the same server to cache queries that can already be answered by that server?

So this might in part be responsible (there are *a lot of* ldap queries).

How many is *a lot of* ?

My production servers peak at over 400 operations/second, with a load average peaking at 2 (CPUs still 80% idle) on 3-year old hardware. I have benchmarked (very simple ones) over 10 000 searches/sec, and Howard has been benchmarking over 50 000 authentications/second.

So ... I doubt you need a separate cache, you probably need better tuning.

Regards, Buchan