cachesize 500000
###### DB_CONFIG
set_cachesize 2 0 1 set_flags DB_TXN_NOSYNC set_lg_bsize 5097152 set_lg_max 50485760
You never state the size of your database (how many dns), or the disk size of your database (du -c -h *.bdb), so there is no way to tell if these settings are in any way valid.
Thanks for getting back to me, I'm new here :) - here is some more data:
Approximately 450,000 dns - about 200+/- are groups, the rest are users. du output:
163M cn.bdb 92M dn2id.bdb 415M id2entry.bdb 3.7M objectClass.bdb 16M uid.bdb 16M uniqueMember.bdb 703M total
I don't see a checkpoint setting for slapd.conf/cn=config, and I don't see an idlcachesize setting.
I added those as well, no noticeable change in performance:
cachesize 500000 idlcachesize 500000 checkpoint 10000 15
Also, you are definitely not using "dynamic" groups in the OpenLDAP sense of the word, although they would probably perform significantly better for you.
Yes, I understand that - thats why I put it in quotes. I looked into using OpenLDAP dynamic lists, but I think I'm limited by the fact that some of our systems requiring these groups need to do searchs off of it based on the dynamic membership (and from what I can tell, its not possible to use it that way), ie they need to search for (uniquemember=cn=xxxx,cn=users,...) on my group section of the tree.
I'll openly admit some of the values I have been picking for caching and checkpoint are somewhat arbitrary. I've been trying many different values and have yet to settle on any that work well. I'll gladly try any recommendations.
Thanks again, I appreciate your response.
Al
--On Monday, April 11, 2011 1:55 PM -0400 Al afrunning@gmail.com wrote:
I added those as well, no noticeable change in performance:
cachesize 500000 idlcachesize 500000 checkpoint 10000 15
In a heavy write environment, I've found that smaller more frequent checkpoints are better. Yours is set rather high.
Also, you are definitely not using "dynamic" groups in the OpenLDAP sense of the word, although they would probably perform significantly better for you.
Yes, I understand that - thats why I put it in quotes. I looked into using OpenLDAP dynamic lists, but I think I'm limited by the fact that some of our systems requiring these groups need to do searchs off of it based on the dynamic membership (and from what I can tell, its not possible to use it that way), ie they need to search for (uniquemember=cn=xxxx,cn=users,...) on my group section of the tree.
I'll openly admit some of the values I have been picking for caching and checkpoint are somewhat arbitrary. I've been trying many different values and have yet to settle on any that work well. I'll gladly try any recommendations.
Make sure you read over the dynlist overlay, I think you can do what you want. A group with 50,000+ members is probably going to take a long time to update. So OpenLDAP dynamic groups may well do better for you.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Monday, April 11, 2011 1:55 PM -0400 Alafrunning@gmail.com wrote:
I added those as well, no noticeable change in performance:
cachesize 500000 idlcachesize 500000 checkpoint 10000 15
In a heavy write environment, I've found that smaller more frequent checkpoints are better. Yours is set rather high.
Also, you are definitely not using "dynamic" groups in the OpenLDAP sense of the word, although they would probably perform significantly better for you.
Yes, I understand that - thats why I put it in quotes. I looked into using OpenLDAP dynamic lists, but I think I'm limited by the fact that some of our systems requiring these groups need to do searchs off of it based on the dynamic membership (and from what I can tell, its not possible to use it that way), ie they need to search for (uniquemember=cn=xxxx,cn=users,...) on my group section of the tree.
I'll openly admit some of the values I have been picking for caching and checkpoint are somewhat arbitrary. I've been trying many different values and have yet to settle on any that work well. I'll gladly try any recommendations.
Make sure you read over the dynlist overlay, I think you can do what you want. A group with 50,000+ members is probably going to take a long time to update. So OpenLDAP dynamic groups may well do better for you.
The dynlist overlay does not support searches of the form (member=foo). While updates of such large groups will involve a lot of I/O (and thus will always be slow) you can still improve things a little using the sortvals config option.
openldap-technical@openldap.org
-
Al -
Howard Chu -
Quanah Gibson-Mount