Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
### snip ### dn: cn=newtest0,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none
dn: cn=newtest1,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none ### snip ###
Then add it with ldapadd: ldapadd -x -D cn=admin,dc=test,dc=com -w test -f ./test
My configuration (for mdb in this instance) used with 2.4.39 is this: ### config ### include /usr/local/etc/openldap/schema/core.schema include /usr/local/etc/openldap/schema/cosine.schema include /usr/local/etc/openldap/schema/inetorgperson.schema
database mdb directory /usr/local/var/openldap-data suffix "dc=test,dc=com" rootdn "cn=admin,dc=test,dc=com" rootpw test maxsize 1980732000
#Indexes index cn,telephoneNumber eq index objectClass eq ### config ###
Any ideas or things I could try?
Thank you! Andrew
________________________________
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
### snip ### dn: cn=newtest0,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none
dn: cn=newtest1,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none ### snip ###
Then add it with ldapadd: ldapadd -x -D cn=admin,dc=test,dc=com -w test -f ./test
<snipped>
Any ideas or things I could try?
slapadd will be much faster
Chris
Thanks, Chris.
Yeah, I hear you on that, but sorry, to be more specific, I was running this test to get an idea of performance for regular LDAP use, and slapadd is a purely offline solution.
It would be helpful for a restore, of course, but not equivalent to when our application will be adding/modifying records (which is what I'm really trying to simulate).
Running the same test of inserting 1M rows into postgres with the same type of data record on the same machine goes about 3x faster, which just doesn't sound right, since LDAP should be way faster than Postgres, right?
Cheers, Andrew
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 8:25 AM, Chris Card ctcard@hotmail.com wrote:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
### snip ### dn: cn=newtest0,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none
dn: cn=newtest1,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none ### snip ###
Then add it with ldapadd: ldapadd -x -D cn=admin,dc=test,dc=com -w test -f ./test
<snipped>
Any ideas or things I could try?
slapadd will be much faster
Chris
Am Tue, 4 Feb 2014 08:45:47 -0200 schrieb Andrew Eross eross@locatrix.com:
Thanks, Chris.
Yeah, I hear you on that, but sorry, to be more specific, I was running this test to get an idea of performance for regular LDAP use, and slapadd is a purely offline solution.
It would be helpful for a restore, of course, but not equivalent to when our application will be adding/modifying records (which is what I'm really trying to simulate).
Running the same test of inserting 1M rows into postgres with the same type of data record on the same machine goes about 3x faster, which just doesn't sound right, since LDAP should be way faster than Postgres, right?
Cheers, Andrew
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 8:25 AM, Chris Card ctcard@hotmail.com wrote:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
You may have a look at this paper, helt at LDAPcon 2013
http://fr.slideshare.net/ldapcon/benchmarks-on-ldap-directories
-Dieter
Thanks, Dieter, Quanah.
I've been doing some experimenting with those mdb options.
I ran a few tests with inserting 10,000 records, wiping the DB in between, and changing just the one option at a time:
Base-line, no extra options: 4m8s With "writemap" enabled: 8m55s With "writemap+mapasync" enabled: 5m12s With "dbnosync+checkpoint 0kb/1min": 0m14s
Oddly I didn't have much luck with writemap, which by default actually slowed things down for me.
The magic bullet of the bunch is dbnosync, which completed so quickly because it didn't even hit the 1 minute mark where it would've written to disk.
That seems a bit risky to me, though. You'd have to be willing to have 1 minute worth of data be theoretically expendable with that option.
Cheers, Andrew
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 2:04 PM, Dieter Klünter dieter@dkluenter.de wrote:
Am Tue, 4 Feb 2014 08:45:47 -0200 schrieb Andrew Eross eross@locatrix.com:
Thanks, Chris.
Yeah, I hear you on that, but sorry, to be more specific, I was running this test to get an idea of performance for regular LDAP use, and slapadd is a purely offline solution.
It would be helpful for a restore, of course, but not equivalent to when our application will be adding/modifying records (which is what I'm really trying to simulate).
Running the same test of inserting 1M rows into postgres with the same type of data record on the same machine goes about 3x faster, which just doesn't sound right, since LDAP should be way faster than Postgres, right?
Cheers, Andrew
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 8:25 AM, Chris Card ctcard@hotmail.com wrote:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
You may have a look at this paper, helt at LDAPcon 2013
http://fr.slideshare.net/ldapcon/benchmarks-on-ldap-directories
-Dieter
-- Dieter Klünter | Systemberatung http://dkluenter.de GPG Key ID: E9ED159B 53°37'09,95"N 10°08'02,42"E
Hi!
It's complete nonsense, but did you try to put a database in a RAM disk? This would show the delay that actual secondary storage access costs in addition to primary storage access.
Regards, Ulrich
Andrew Eross eross@locatrix.com schrieb am 04.02.2014 um 21:52 in
Nachricht CAL_tfFdPYK-vePsyPqSya1xsjDAOqHOcS_SEjOaf-noUTXWMsw@mail.gmail.com:
Thanks, Dieter, Quanah.
I've been doing some experimenting with those mdb options.
I ran a few tests with inserting 10,000 records, wiping the DB in between, and changing just the one option at a time:
Base-line, no extra options: 4m8s With "writemap" enabled: 8m55s With "writemap+mapasync" enabled: 5m12s With "dbnosync+checkpoint 0kb/1min": 0m14s
Oddly I didn't have much luck with writemap, which by default actually slowed things down for me.
The magic bullet of the bunch is dbnosync, which completed so quickly because it didn't even hit the 1 minute mark where it would've written to disk.
That seems a bit risky to me, though. You'd have to be willing to have 1 minute worth of data be theoretically expendable with that option.
Cheers, Andrew
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 2:04 PM, Dieter Klünter dieter@dkluenter.de wrote:
Am Tue, 4 Feb 2014 08:45:47 -0200 schrieb Andrew Eross eross@locatrix.com:
Thanks, Chris.
Yeah, I hear you on that, but sorry, to be more specific, I was running this test to get an idea of performance for regular LDAP use, and slapadd is a purely offline solution.
It would be helpful for a restore, of course, but not equivalent to when our application will be adding/modifying records (which is what I'm really trying to simulate).
Running the same test of inserting 1M rows into postgres with the same type of data record on the same machine goes about 3x faster, which just doesn't sound right, since LDAP should be way faster than Postgres, right?
Cheers, Andrew
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 8:25 AM, Chris Card ctcard@hotmail.com wrote:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
You may have a look at this paper, helt at LDAPcon 2013
http://fr.slideshare.net/ldapcon/benchmarks-on-ldap-directories
-Dieter
-- Dieter Klünter | Systemberatung http://dkluenter.de GPG Key ID: E9ED159B 53°37'09,95"N 10°08'02,42"E
--On Tuesday, February 04, 2014 6:52 PM -0200 Andrew Eross eross@locatrix.com wrote:
Thanks, Dieter, Quanah.
I've been doing some experimenting with those mdb options.
I ran a few tests with inserting 10,000 records, wiping the DB in between, and changing just the one option at a time:
Base-line, no extra options: 4m8sWith "writemap" enabled: 8m55s With "writemap+mapasync" enabled: 5m12s With "dbnosync+checkpoint 0kb/1min": 0m14s
I know you answered some of this before, but please update with:
What kernel? What OpenLDAP version? What Ubuntu release? What filesystem for the LDAP DB?
Thanks, Quanah
--
Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Am Tue, 4 Feb 2014 10:25:15 +0000 schrieb Chris Card ctcard@hotmail.com:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
have you read slapd-hdb(5) and slapd-mdb(5) on proper database configuration? a slapadd of 1 mio entries takes about 30 min, depending on hardware and filesystem, I haven't used ldapadd for some time, but it shouldn't take much longer.
-Dieter
Hi Dieter,
I definitely read the man entries - for hdb it's more complicated, of course, but with mdb, my simple config came directly from the online docs.
My understanding is that there's really nothing needed/suggested for tuning parameters for mdb.
I've seen there's a 'checkpoint' option along with 'dbnosync' which can perhaps increase performance at the expense of reliability, but I wouldn't want to use that in production.
Using single inserts with auto-commit after each insert on Postgres on the same machine completed the same task in just under 2 hours.
I'd very much like to get my LDAP setup to add the entries in ~30 min, that'd be much more in-line with what I was shooting for on performance.
Any ideas to try very welcome.
Thank you, Andrew
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 9:54 AM, Dieter Klünter dieter@dkluenter.de wrote:
Am Tue, 4 Feb 2014 10:25:15 +0000 schrieb Chris Card ctcard@hotmail.com:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
have you read slapd-hdb(5) and slapd-mdb(5) on proper database configuration? a slapadd of 1 mio entries takes about 30 min, depending on hardware and filesystem, I haven't used ldapadd for some time, but it shouldn't take much longer.
-Dieter
-- Dieter Klünter | Systemberatung http://dkluenter.de GPG Key ID: E9ED159B 53°37'09,95"N 10°08'02,42"E
Am Tue, 4 Feb 2014 10:26:40 -0200 schrieb Andrew Eross eross@locatrix.com:
Hi Dieter,
I definitely read the man entries - for hdb it's more complicated, of course, but with mdb, my simple config came directly from the online docs.
My understanding is that there's really nothing needed/suggested for tuning parameters for mdb.
I've seen there's a 'checkpoint' option along with 'dbnosync' which can perhaps increase performance at the expense of reliability, but I wouldn't want to use that in production.
Using single inserts with auto-commit after each insert on Postgres on the same machine completed the same task in just under 2 hours.
I'd very much like to get my LDAP setup to add the entries in ~30 min, that'd be much more in-line with what I was shooting for on performance.
configure pool-threads, dbconfig, cachesize, idlcachsize, dncachsize, checkpoint and dbnosync for back-bdb and for back-mdb pool-threads, envflags, dbnosync and checkpoint.
-Dieter
Andrew Eross CTO Locatrix Communications Office: +61 7 3123 1469 Mobile: +55 37 9858 9815 eross@locatrix.com
On Tue, Feb 4, 2014 at 9:54 AM, Dieter Klünter dieter@dkluenter.de wrote:
Am Tue, 4 Feb 2014 10:25:15 +0000 schrieb Chris Card ctcard@hotmail.com:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
have you read slapd-hdb(5) and slapd-mdb(5) on proper database configuration? a slapadd of 1 mio entries takes about 30 min, depending on hardware and filesystem, I haven't used ldapadd for some time, but it shouldn't take much longer.
-Dieter
-- Dieter Klünter | Systemberatung http://dkluenter.de GPG Key ID: E9ED159B 53°37'09,95"N 10°08'02,42"E
--On Tuesday, February 04, 2014 10:26 AM -0200 Andrew Eross eross@locatrix.com wrote:
I've seen there's a 'checkpoint' option along with 'dbnosync' which can perhaps increase performance at the expense of reliability, but I wouldn't want to use that in production.
The db env flags for mdb can be important. I.e., I use writemap myself, and ldapadd is quite fast.
--Quanah
--
Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
On 02/04/2014 02:58 AM, Andrew Eross wrote:
Hello all,
I've been Google'ing around and searching the archives, but I haven't quite been able to find an answer, so I wanted to ask the list.
I've been experimenting with OpenLDAP adds to see how quickly we can get data inserted into the DB.
I've got a few OpenLDAP instances that I use for writing log data to, so write performance is critical, but since I'm building it from log data, absitively, posolutely, guaranteed perfect DB consistency isn't. I can always replay log data to rebuild the DB if, say, I had a power outage, the UPS failed, the RAID write-cache failed, the planets aligned, and I lost data. :-)
First off, disable logging to syslog. The first bottleneck I've run into (reads or writes) was not OpenLDAP but syslogd trying to write everything OpenLDAP told it. Set "loglevel 0" in your slapd.conf. 'Might also consider setting "threads" in slapd.conf if you've got multiple threads doing writes. (see the man pages for this)
Then, to optimize write performance, I added this to slapd.conf (in the database section): checkpoint 1024000 60 dbnosync envflags writemap
And read the man pages so you understand these options and what you're getting into. Some would consider this risky but it's perfect for my particular situation, where the data could always be rebuilt if needed (though I've never needed to).
With this setup, on a single-CPU-core VM, I got around 5400 writes per second with several threads. If memory serves, I got over 4000 writes per second with one thread. This was using the new MDB database back-end which I highly recommend. VERY fast. :-)
Brent
I'm using Ubuntu 10.04, and I've tried both the packaged OpenLDAP 2.4.21 using hdb, and just recently the latest OpenLDAP 2.4.39 using lmdb, both with relatively similar results.
The short version is: to insert 1 million records, it's taking about 8 hours on a machine with 2GB RAM / 3Ghz / SSD, which seems like a long time to me.
The insert method is to use a single big ldiff file like this:
### snip ### dn: cn=newtest0,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none
dn: cn=newtest1,ou=customer,dc=test,dc=com objectclass: inetOrgPerson userPassword: 90sac90sudasdjcao0sdjtest telephoneNumber: 61400000000 sn: none ### snip ###
Then add it with ldapadd: ldapadd -x -D cn=admin,dc=test,dc=com -w test -f ./test
My configuration (for mdb in this instance) used with 2.4.39 is this: ### config ### include /usr/local/etc/openldap/schema/core.schema include /usr/local/etc/openldap/schema/cosine.schema include /usr/local/etc/openldap/schema/inetorgperson.schema
database mdb directory /usr/local/var/openldap-data suffix "dc=test,dc=com" rootdn "cn=admin,dc=test,dc=com" rootpw test maxsize 1980732000
#Indexes index cn,telephoneNumber eq index objectClass eq ### config ###
Any ideas or things I could try?
Thank you! Andrew
On Fri, Feb 07, 2014 at 10:49:29AM -0700, Brent Bice wrote:
(SNIP)
I've got a few OpenLDAP instances that I use for writing log data to, so write performance is critical, but since I'm building it from log data, absitively, posolutely, guaranteed perfect DB consistency isn't. I can always replay log data to rebuild the DB if, say, I had a power outage, the UPS failed, the RAID write-cache failed, the planets aligned, and I lost data. :-)
Out of interest, what are you using this log data for, and have you tested how many reads you are getting?
On 02/07/2014 02:40 PM, Christopher Wood wrote:
On Fri, Feb 07, 2014 at 10:49:29AM -0700, Brent Bice wrote:
(SNIP)
I've got a few OpenLDAP instances that I use for writing log datato, so write performance is critical, but since I'm building it from log data, absitively, posolutely, guaranteed perfect DB consistency isn't. I can always replay log data to rebuild the DB if, say, I had a power outage, the UPS failed, the RAID write-cache failed, the planets aligned, and I lost data. :-)
Out of interest, what are you using this log data for, and have you tested how many reads you are getting?
In the recent past, I've setup a java script to log postfix/sendmail/cuda logs to OpenLDAP and some simple php scripts to query it. 'Makes it easier for junior admins and managerial types to be able to track how an email got from point A to point B. Say, an Exchange user sent an email to an internal list server - so it went from exchange to a postfix relay to the list server, then back to the postfix relays then to some recips on Exchange, some recips on other lists, some on departmental mail servers using sendmail, etc. I can search by to/from and/or date/time, find the email, then click on the message-ID to search by that and show the email every hop along the way as well as all the recipients who got it. Makes it faster to sort out those "I sent an email to list ABC and user XYZ didn't get it! Why not!" problems. The answer usually is "user XYZ did get it and here's the log showing it". :-)
I also recently started logging DHCP client hostnames, IPs, MAC Addresses, and (if the dhcp request came from our VPN hardware) username. That way when I'm sifting through snort/FireEye/PaloAlto logs and I see some IP with a dhcp hostname of "MyPC" I can quickly tell which user's home machine is infected with malware-du-jour. I can see who was on which IPs when.
Yeah, I coulda used MySQL or Postgres or something else. The first one (the relay logs) started off as a weekend project to edjimicate myself on the LDAP API in Java (or one of 'em). It proved useful enough we just kept it. And since I had that in place, adding on the vpn/dhcp stuff later was easy. I use the dds overlay to automagically throw away records older than X days.
For both of those, the number of writes per second we do is low - around 4 or 5 per second last I checked.
However, we have a lot of DNS servers in a lot of different geographies and I've thought about trying to centralize their logs. But the query logs can be substantial - a terabyte per region per day - more than I really want to shove over the WAN to a central spot. So it occurred to me one morning that I could leave the log data distributed, but centralize how I query it. I could have one LDAP server that had referrals to other LDAP servers, one per region, and have all the DNS servers in a given region log their queries to their local LDAP server. Then a simple php script can do one query against the root server and find any query handled by any DNS server in any region. (useful when handling an intrusion event, for instance, and you want to know every DNS query made by some system between certain dates/times).
But SGI sells HPC equipment (big storage too, btw - grin). So it's not unheard of for someone to spin up a big cluster in one location and generate thousands of DNS queries per second. So any sort of logging I do has to scale well or it just won't work. There's likely a better way, but this gave me a good excuse to try out OpenLDAP + mdb on xfs and to see if PHP's LDAP API would chase referrals. :-) I'll probably wind up using some tool to index the textual query logs and some way to search all the indexes on all the regional log servers with regex patterns instead or somethin'...
(scrolls back) Oh yeah... Reads... I haven't been paying close attention to the number of reads per second I've been getting as writes and deletes were the bottleneck I was curious about. But the last time I checked, I was getting something like 30k+ queries per second with 8 threads on one client. But this is with zero tuning of the filesystem options and with a really simple-minded bit of java - this shouldn't be taken as any sort of serious benchmark. I've learned that proper benchmarking is HARD and I only use the java tool for rough guesstimates (and comparing how different config options may improve performance - or not - in a relative sort of way).
Brent
openldap-technical@openldap.org
-
Andrew Eross -
Brent Bice -
Chris Card -
Christopher Wood -
Dieter Klünter -
Quanah Gibson-Mount -
Ulrich Windl