Hi all,
When I started at this job I acquired OpenLDAP based authentication infrastructure, which I've pretty much been able to leave to its own devices without looking under the hood at what's going on. Recently I've been noticing that a few minor performance problems I'm having seem to be linked to LDAP so I've started looking at how it's set up and it's got some quirks, e.g.
# Indices to maintain index entryUUID eq index cn,sn,uid,displayName pres,sub,eq index uidNumber eq index gidNumber eq index memberUid eq index sambaSID eq index sambaPrimaryGroupSID eq index sambaDomainName eq index sambaGroupType eq index sambaSIDList eq index sudoUser eq index uniqueMember,member eq index objectClass pres,eq index mail,mailAlternateAddress eq,sub index accountStatus eq index mailHost,deliveryMode eq index Status,HostGroup eq index default sub
There are only around 600 users in the ldap tree, hardly anything. It seems a bit excessive to have that many indexes set up for the tree. Is there any way to identify which indexes are being used and how frequently? I'm pretty certain some of those entries probably get checked once in a blue moon.
I'm also seeing odd entries in the ldap logs:
Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (gidNumber) index_param failed (18)
Which seems bizarre given those entries are in the list of indexes. I've tried running slapindex and restarting to no avail. Those items still seem to throw up those alerts, as does the sudoUser entry.
On a separate note I've found the slapcat doesn't produce an LDIF file that slapadd will use, even keeping to the same version of software (2.3.37), or importing into newer (2.3.43). It always errors after the first entry because it seems to be trying to add a user to an ou that doesn't exist yet, which disturbs me slightly because the backup infrastructure has been relying on that mechanism for recovery. To create another ldap slave I ended up having to use Apache Directory Studio to create the LDIF for import, which slapadd was perfectly happy to work with. I'm not sure if that may be a known bug that I'm running up against. I see a few entries in archives relating to it but not much by way of response other than suggests it should work.
Paul
--On Monday, December 20, 2010 12:44 PM -1000 Paul paul@ehawaii.gov wrote:
I'm also seeing odd entries in the ldap logs:
Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (gidNumber) index_param failed (18)
This means those entries are being searched on with a search other than the way they are indexed. No amount of re-running slapindex is going to change those messages.
On a separate note I've found the slapcat doesn't produce an LDIF file that slapadd will use, even keeping to the same version of software (2.3.37), or importing into newer (2.3.43). It always errors after the first entry because it seems to be trying to add a user to an ou that doesn't exist yet, which disturbs me slightly because the backup infrastructure has been relying on that mechanism for recovery. To create another ldap slave I ended up having to use Apache Directory Studio to create the LDIF for import, which slapadd was perfectly happy to work with. I'm not sure if that may be a known bug that I'm running up against. I see a few entries in archives relating to it but not much by way of response other than suggests it should work.
It may be producing an out-of-order LDIF file. You probably need to find out why and permanently re-order it. OpenLDAP 2.4 handles that situation correctly.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On 12/21/2010 07:26 AM, Quanah Gibson-Mount wrote:
--On Monday, December 20, 2010 12:44 PM -1000 Paul paul@ehawaii.gov wrote:
I'm also seeing odd entries in the ldap logs:
Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (gidNumber) index_param failed (18)
This means those entries are being searched on with a search other than the way they are indexed. No amount of re-running slapindex is going to change those messages.
Sorry, I may be being completely dense here. So those errors are because the wrong type of index is being specified in these lines in my config file?
index objectClass pres,eq index gidNumber eq
I assumed that "bdb_equality_candidates" would indicate "eq" indexing as being the type of search it's trying to do, is my assumption off the mark?
On the general subject of indexes, is there a way to identify in OpenLDAP/slapd which ones are actually being used? Beyond just basic pam/e-mail stuff I've got a number of java apps and the like not all done in house that could be querying the LDAP server in numerous ways.
On a separate note I've found the slapcat doesn't produce an LDIF file that slapadd will use, even keeping to the same version of software (2.3.37), or importing into newer (2.3.43). It always errors after the first entry because it seems to be trying to add a user to an ou that doesn't exist yet, which disturbs me slightly because the backup infrastructure has been relying on that mechanism for recovery. To create another ldap slave I ended up having to use Apache Directory Studio to create the LDIF for import, which slapadd was perfectly happy to work with. I'm not sure if that may be a known bug that I'm running up against. I see a few entries in archives relating to it but not much by way of response other than suggests it should work.
It may be producing an out-of-order LDIF file. You probably need to find out why and permanently re-order it. OpenLDAP 2.4 handles that situation correctly.
I haven't even the faintest clue how to begin there. Googling is returning nothing useful (but I may be searching for the wrong things), nor is searching through the OpenLDAP documentation. What could be the root cause of such antics? Should I be particularly concerned by it?
Paul
On 12/21/2010 07:26 AM, Quanah Gibson-Mount wrote:
--On Monday, December 20, 2010 12:44 PM -1000 Paul paul@ehawaii.gov wrote:
I'm also seeing odd entries in the ldap logs:
Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (objectClass) index_param failed (18) Dec 20 11:21:17 <server> slapd[11171]: <= bdb_equality_candidates: (gidNumber) index_param failed (18)
This means those entries are being searched on with a search other than the way they are indexed. No amount of re-running slapindex is going to change those messages.
Sorry, I may be being completely dense here. So those errors are because the wrong type of index is being specified in these lines in my config file?
index objectClass pres,eq index gidNumber eq
I assumed that "bdb_equality_candidates" would indicate "eq" indexing as being the type of search it's trying to do, is my assumption off the mark?
Your assumption is correct. You don't need to index "objectClass" for "pres", as the objectClass attribute is always present, "eq" suffices. If you changed the "index" directive in slapd.conf you need to run slapindex while slapd is not running. Make sure you run slapindex with the correct identity, as all index files will be generated with that identity, and then slapd needs to be able to read them.
On the general subject of indexes, is there a way to identify in OpenLDAP/slapd which ones are actually being used? Beyond just basic pam/e-mail stuff I've got a number of java apps and the like not all done in house that could be querying the LDAP server in numerous ways.
You can enable monitoring (configure the monitor database in slapd.conf), and check the contents of olmBDBNotIndexed in the database entry (e.g. "cn=Database 1,cn=databases,cn=monitor" if the BDB database is the first one); to use this feature, you need to #define BDB_MONITOR_IDX and recompile slapd. You'll get data like
olmBDBNotIndexed: description#present=1#substr=1
indicating that "description" was searched once with "present" and once with "substring" and they were not indexed.
p.
--On Monday, December 27, 2010 9:41 AM -1000 Paul paul@ehawaii.gov wrote:
I assumed that "bdb_equality_candidates" would indicate "eq" indexing as being the type of search it's trying to do, is my assumption off the mark?
"eq" means you are indexing it for equality searches. It says zero about what type of searches the client may be doing. For example, it may be doing a substr search on objectClass. If it is, you'll get the warning. It is not necessarily a bad thing. It is just letting you know that some client is trying to do searches on that attribute in ways other than you've indexed it for.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
--On Monday, December 27, 2010 9:41 AM -1000 Paul paul@ehawaii.gov wrote:
It may be producing an out-of-order LDIF file. You probably need to find out why and permanently re-order it. OpenLDAP 2.4 handles that situation correctly.
I haven't even the faintest clue how to begin there. Googling is returning nothing useful (but I may be searching for the wrong things), nor is searching through the OpenLDAP documentation. What could be the root cause of such antics? Should I be particularly concerned by it?
Root cause is hard to say. You can look at fixing it permanently (with a master reload) with a tool like:
http://search.cpan.org/~gbarr/perl-ldap-0.33/contrib/ldifsort.pl
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org
-
masaratiļ¼ aero.polimi.it -
Paul -
Quanah Gibson-Mount