i am trying to setup the pcache overlay to cache routinely used entries in the DIT. using the below, i am able to load the pcache module:
ldapmodify -QY EXTERNAL -H ldapi:/// dn: cn=module,cn=config changetype:add objectClass: olcModuleList cn: module olcModulePath: /usr/lib64/openldap olcModuleLoad: pcache.la
then i try to add some caching directives, using the below:
ldapmodify -QY EXTERNAL -H ldapi:/// dn: olcOverlay={0}pcache,olcDatabase={2}hdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcPcacheConfig objectClass: olcPcacheDatabase olcOverlay: {0}pcache olcPcache: hdb 100000 50 1000 100
olcPcacheAttrset: 0 nSRecord sOARecord pTRRecord aRecord cNAMERecord sRVRecord tXTRecord olcPcacheTemplate: "(zoneName=)" 0 3600 olcPcacheTemplate: "(&(zoneName=)(relativeDomainName=))" 0 3600
olcPcacheAttrset: 1 krbprincipalname krbcanonicalname objectclass krbprincipalkey krbmaxrenewableage krbmaxticketlife krbticketflags krbprincipalexpiration krbticketpolicyreference krbUpEnabled krbpwdpolicyreference krbpasswordexpiration krbLastFailedAuth krbLoginFailedCount krbLastSuccessfulAuth nsaccountlock krbLastPwdChange krbLastAdminUnlock krbExtraData krbObjectReferences krbAllowedToDelegateTo olcPcacheTemplate: "(&(|(objectClass=)(objectClass=))(krbPrincipalName=))" 1 3600
olcPcacheAttrset: 2 userPassword cn gidNumber uidNumber loginShell objectClass gecos uid homeDirectory olcPcacheTemplate: "(&(objectClass=)(uid=))" 2 3600
olcPcacheAttrset: 3 cn userPassword memberUid gidNumber uniqueMember olcPcacheTemplate: "(&(objectClass=)(cn=))" 3 3600 olcPcacheTemplate: "(&(objectClass=)(memberUid=))" 3 3600 olcPcacheTemplate: "(&(objectClass=)(gidNumber=))" 3 3600 olcPcacheTemplate: "(&(objectClass=)(|(memberUid=)(uniqueMember=)))" 3 3600
i no longer have logs around the failure, but the error message indicated that the olcDbDirectory could not be found. the DN dn: olcDatabase={2}hdb,cn=config has that specified (olcDbDirectory: /var/lib/ldap), but i am missing something about pcache and the backend sharing this directive. can anyone point me in the right direction about how to get the pcache overlay setup correctly?
Hey,
On 19.03.2013 15:17, brendan kearney wrote:
i no longer have logs around the failure, but the error message indicated that the olcDbDirectory could not be found. the DN dn: olcDatabase={2}hdb,cn=config has that specified (olcDbDirectory: /var/lib/ldap), but i am missing something about pcache and the backend sharing this directive.
AFAIK you are missing a database to store the caching data in. While the manpages and documentation do make mention of this implicitly, neither of them states it in an obvious way. Unfortunately even the examples in the admin guide are a bit misleading because the DNs there do not reflect the object hierarchy in the config database. So to solve your problem you will need a second database, that stores the caching data, e.g.
# your database backend you want to cache for dn: olcDatabase={2}ldap,cn=config ... # the pcache overlay dn: olcOverlay={0}pcache,olcDatabase={2}ldap,cn=config ... # the pcache overlay db dn: olcDatabase=hdb,olcOverlay={0}pcache,olcDatabase={2}ldap,cn=config ... objectClass: olcPcacheDatabase olcDbDirectory: /var/lib/ldap/pcache olcDbIndex: pcacheQueryID eq ...
Note also that pcache is meant to increase the performance of ldap and meta backends. I do not know whether it makes sense or actually works to cache entries for a mundane hdb database.
hth
bernd,
thank you for the info. i think i have a better picture now, but still don't grasp some of the details.
first, i see what you are saying about caching applying to ldap, meta and sql backends. in reading the man page for slapo-pcache, it states that the proxy cache may be used with any backend but is intended for use with what you specify. does that mean caching will be ineffective or will not occur? because i have things like dns zone info and the kerberos database in my DIT, it would be a great benefit to get caching working. i am not opposed to changing the backend if needed, but dont know what pros and cons are to each backend type. i simply used hdb to learn with.
next, with the examples you give below, it looks like you build at least one level / layer more than i do with my attempt. i am not sure what i would need to do to create this structure. your example creates an ldap backend, adds the overlay to the ldap backend, and adds the cache database to the overlay. because i have an hdb backend, would i want / need to create a separate ldap backend? could i use the existing hdb backend? this is really just for learning right now...
On Wed, Mar 20, 2013 at 2:02 PM, Bernd May bernd@net.t-labs.tu-berlin.dewrote:
Hey,
On 19.03.2013 15:17, brendan kearney wrote:
i no longer have logs around the failure, but the error message indicated that the olcDbDirectory could not be found. the DN dn: olcDatabase={2}hdb,cn=config has that specified (olcDbDirectory: /var/lib/ldap), but i am missing something about pcache and the backend sharing this directive.
AFAIK you are missing a database to store the caching data in. While the manpages and documentation do make mention of this implicitly, neither of them states it in an obvious way. Unfortunately even the examples in the admin guide are a bit misleading because the DNs there do not reflect the object hierarchy in the config database. So to solve your problem you will need a second database, that stores the caching data, e.g.
# your database backend you want to cache for dn: olcDatabase={2}ldap,cn=config ... # the pcache overlay dn: olcOverlay={0}pcache,olcDatabase={2}ldap,cn=config ... # the pcache overlay db dn: olcDatabase=hdb,olcOverlay={0}pcache,olcDatabase={2}ldap,cn=config ... objectClass: olcPcacheDatabase olcDbDirectory: /var/lib/ldap/pcache olcDbIndex: pcacheQueryID eq ...
Note also that pcache is meant to increase the performance of ldap and meta backends. I do not know whether it makes sense or actually works to cache entries for a mundane hdb database.
hth
-- Technische Universität Berlin - FGINET
Bernd May
System Administration An-Institut Deutsche Telekom Laboratories Sekr. TEL 16 Ernst-Reuter-Platz 7 10587 BERLIN GERMANY
Mobile: 0160/90257737 E-Mail: bernd@net.t-labs.tu-berlin.de (T-Labs work) WWW: net.t-labs.tu-berlin.de
Hey,
On 21.03.2013 01:25, brendan kearney wrote:
does that mean caching will be ineffective or will not occur? because i have things like dns zone info and the kerberos database in my DIT, it would be a great benefit to get caching working. i am not opposed to changing the backend if needed, but dont know what pros and cons are to each backend type. i simply used hdb to learn with.
I have no experience with using a pcache for a hdb or any other backend than ldap so far. I assume that using a db to cache entries from another db of the same or similiar type is somewhat ineffective though - maybe one of the devs can share insight into this.
For DNS entries I would recommend setting up a slapd with ldap backend (your ldap server) and pcache overlay with db of your choice on the dns server. That way you can cache the more common requests on the dns server in the pcache and only incur a small delay for the uncached ones. I assume you do not have the master slapd on your dns server. Another solution might be to partially replicate the dit. I guess its a matter of your liking in the end.
I have no experience with using ldap as a kerberos db but I guess replicating that data or even caching it at the client side might be a bit insecure depending on what is actually stored in the ldap. Otoh the kerberos db is usually only queried on tgt creation and for service tickets so I would assume the slapd will be able to handle the traffic without caching. I assume you have setup proper indexing for the db ;)
next, with the examples you give below, it looks like you build at least one level / layer more than i do with my attempt. i am not sure what i would need to do to create this structure. your example creates an ldap backend, adds the overlay to the ldap backend, and adds the cache database to the overlay. because i have an hdb backend, would i want / need to create a separate ldap backend? could i use the existing hdb backend? this is really just for learning right now...
As the manpage says, the pcache overlay is mainly desgined for ldap and meta backends... I assume you can use it with a hdb, i.e. replace my example ldap backend database with a hdb. Not sure as said above if this will result in a performance boost. Otoh you could setup your slapd to store its data in a hdb as usual and then add an additional ldap backend db with caching that uses the same server. Not sure if that makes sense though.
E.g.:
# the actual database to store data in dn: olcDatabase={2}hdb,cn=config ...
# your ldap backend you want to cache for, pointing to this server dn: olcDatabase={3}ldap,cn=config ... # the pcache overlay dn: olcOverlay={0}pcache,olcDatabase={3}ldap,cn=config ... # the pcache overlay db dn: olcDatabase=hdb,olcOverlay={0}pcache,olcDatabase={3}ldap,cn=config ... objectClass: olcPcacheDatabase olcDbDirectory: /var/lib/ldap/pcache olcDbIndex: pcacheQueryID eq
hth :)
openldap-technical@openldap.org
-
Bernd May -
brendan kearney