Hi All,
First of all thks for helping me out with the issues on openLDAP. Well today, my query is pretty generic and lot many people working on LDAP would face such an issues.
We are using openldap 2.4.26 with BDB as backend. We've installed the setup on linux machine of 64 bit with 4GB ram. Now, currently we have some 10K records in it and its working perfectly fine. Our systems is running under replication - syncrepl.
However, in near future we can foresee some million of records to turn up. So, Can any one please advise - What are the different Scaling options available with LDAP? I need to do some research and test our cases.
Please suggest.
Thanks and Regards, Gaurav Gugnani
--On Wednesday, March 14, 2012 11:08 AM +0530 Gaurav Gugnani gugnanigaurav@gmail.com wrote:
Hi All,
First of all thks for helping me out with the issues on openLDAP. Well today, my query is pretty generic and lot many people working on LDAP would face such an issues.
We are using openldap 2.4.26 with BDB as backend. We've installed the setup on linux machine of 64 bit with 4GB ram. Now, currently we have some 10K records in it and its working perfectly fine. Our systems is running under replication - syncrepl.
However, in near future we can foresee some million of records to turn up. So, Can any one please advise - What are the different Scaling options available with LDAP?
I'm not sure what you mean by scaling options. OpenLDAP scales, and that has been shown numerous times. How well/far it scales depends entirely on your hardware and operating system and the size of the DB in relation to those things.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
How much write activity do you expect? How big entries? Will your entire DB fit in RAM, so disk read speed will mostly be an issue during startup (if back-bdb is properly tuned)?
Current stats for our slapd machines, which are currently vastly overpowered for their task:
- 3G database size from 0.5G LDIF data with 0.9 million entries.
- Average 60 connections and 280 requests per second the last two days, but just 6 per minute were write requests.
- 8G RAM - the entire databases can be cached in RAM.
- 4 2GHz cores in 2 processors, Linux x86_64.
- The 'unchecked' limit is set so that only the rootdn can trawl an entire DB with one search. Others get adminSizeExceeded if the number of candidate result entries after consulting the database indexes exceed the unchecked limit.
- RAID disks not specially tuned for BDB. Not recommended for speed nor database durability. But it's fast enough anyway and we can regenerate the DBs from LDIF, so we're happy to leave the disks to the site admins and the site's standard disk setup.
- The traffic gives 100K syslog output/second from loglevel 'stats'.
Some years ago, the disks on our aging machines got too slow for all the syslog output. Many requests took more than one second. So we had to turn off slapd logging. The issue went away when we put '-' in front of the logfile name in /etc/syslog.conf and upgraded to more modern hardware.
It would surely also have helped to play around more with the disks - like logging to separate disks if we did not already. Don't remember how much we did with that. We do like leaving such concerns to others at the site.
I wrote:
Current stats for our slapd machines, which are currently vastly overpowered for their task:
- 3G database size from 0.5G LDIF data with 0.9 million entries.
Whoops, halve that. 1.5G DB, 0.25G LDIF data, 0.45 mill entires. I did wonder why our stats had grown so much lately:-)
- Average 60 connections and 280 requests per second the last two days, but just 6 per minute were write requests.
(...)
openldap-technical@openldap.org
-
Gaurav Gugnani -
Hallvard B Furuseth -
Quanah Gibson-Mount