How do you measure or compare speed?
I have a Python program which add a predefined number of users (1mil and each user consists of 8 DNs and 3 alias DNs) and for Modify test I have also Python program which modify 1 attribute with random value under one DN changed sequentially from 1 to 1mil)
Now I can add those 1 million of users in 5508 sec what amounts to 181,55 users per second.
Modify program can do 2141,33 modifications per sec.
Do you find those numbers reasonable or are they nonsense?
Is there a way to get 10000 modifications per second or I am asking a stupid question?
Sasa
On Mon, Apr 8, 2013 at 4:40 PM, Quanah Gibson-Mount quanah@zimbra.comwrote:
--On Monday, April 08, 2013 4:49 AM -0700 Howard Chu hyc@symas.com wrote:
Saša-Stjepan Bakša wrote:
Hi,
Maybe this is not the best way to ask but I would like to get some performance expectations or maybe suggestions how to improve performance. I do have relatively long experience with OpenLDAP as a precompiled package and with much less users so performance was not an issue for those installations.
Now I need to put few million users (now one million for test), custom tailored schema and search performance is crucial but also modify performance is big issue. Before MDB I have used HDB as backend.
What to do to improve write part of performance or performance in general (when adding data – from time to time – OpenLDAP stalls in a way)? Just around 5 add/mod operations during that time then it continue with much higher speed.
I have used many different sources to find out other peoples experience and I didn't choose to write to list lightly but I really need some help/hints.
There's nothing particular to LMDB to tune. But if you're seeing pauses due to disk I/O, as your iotop output seems to indicate, you might want to look into using a different I/O scheduler.
Not quite true. On Linux, I suggest setting the olcDbFlags as follows:
olcDbEnvFlags: writemap olcDbEnvFlags: nometasync
With those flags set, writes are 65x faster for me with back-mdb than they are with back-hdb/back-bdb.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc.
Zimbra :: the leader in open source messaging and collaboration
Saša-Stjepan Bakša wrote:
Now I can add those 1 million of users in 5508 sec what amounts to 181,55 users per second.
Modify program can do 2141,33 modifications per sec.
Do you find those numbers reasonable or are they nonsense?
Depends very much on your hardware and configuration.
Ciao, Michael.
Well, hardware is described in my first post. Now I am waiting for HP blades and SSD array :-) so if you need/want some tests to be done it can be arranged (in idle time).
Br
Sasa
On Mon, Apr 15, 2013 at 7:09 PM, Michael Ströder michael@stroeder.comwrote:
Saša-Stjepan Bakša wrote:
Now I can add those 1 million of users in 5508 sec what amounts to 181,55 users per second.
Modify program can do 2141,33 modifications per sec.
Do you find those numbers reasonable or are they nonsense?
Depends very much on your hardware and configuration.
Ciao, Michael.
I am also interested in this thread. I've been doing some comparison tests between MDB and BDB using openldap 2.4.35, and find that the BDB backend is consistently much faster for writes. For example, modifying 10000 DNs and changing one attribute using ldapmodify takes about 80 seconds with an MDB backend and about 18 seconds with a BDB backend. Is there any chance that MDB write performance will catch-up (or even overtake) BDB write performance?
Chris
--On Thursday, April 18, 2013 2:27 PM +0000 Chris Card ctcard@hotmail.com wrote:
I am also interested in this thread. I've been doing some comparison tests between MDB and BDB using openldap 2.4.35, and find that the BDB backend is consistently much faster for writes. For example, modifying 10000 DNs and changing one attribute using ldapmodify takes about 80 seconds with an MDB backend and about 18 seconds with a BDB backend. Is there any chance that MDB write performance will catch-up (or even overtake) BDB write performance?
Did you make the changes to the DB flags for MDB that I noted the other day? For me, MDB writes are a minimum of 65 times faster than writes with BDB/HDB, even when BDB/HDB use an SHM key.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
----------------------------------------
Date: Thu, 18 Apr 2013 08:16:03 -0700 From: quanah@zimbra.com To: ctcard@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend?
--On Thursday, April 18, 2013 2:27 PM +0000 Chris Card ctcard@hotmail.com wrote:
I am also interested in this thread. I've been doing some comparison tests between MDB and BDB using openldap 2.4.35, and find that the BDB backend is consistently much faster for writes. For example, modifying 10000 DNs and changing one attribute using ldapmodify takes about 80 seconds with an MDB backend and about 18 seconds with a BDB backend. Is there any chance that MDB write performance will catch-up (or even overtake) BDB write performance?
Did you make the changes to the DB flags for MDB that I noted the other day? For me, MDB writes are a minimum of 65 times faster than writes with BDB/HDB, even when BDB/HDB use an SHM key.
Yes. I have these values in my mdb configuration: olcDbEnvFlags: writemap olcDbEnvFlags: nometasync
I think these setting may have helped to speed up slapadd, but not ldapmodify.
Chris
--On Thursday, April 18, 2013 3:29 PM +0000 Chris Card ctcard@hotmail.com wrote:
Did you make the changes to the DB flags for MDB that I noted the other day? For me, MDB writes are a minimum of 65 times faster than writes with BDB/HDB, even when BDB/HDB use an SHM key.
Yes. I have these values in my mdb configuration: olcDbEnvFlags: writemap olcDbEnvFlags: nometasync
I think these setting may have helped to speed up slapadd, but not ldapmodify.
I'm specifically talking about ldapmodify actually. ;)
http://wiki.zimbra.com/wiki/OpenLDAP_MDB_vs_HDB_performance
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
----------------------------------------
Date: Thu, 18 Apr 2013 08:50:23 -0700 From: quanah@zimbra.com To: ctcard@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend?
--On Thursday, April 18, 2013 3:29 PM +0000 Chris Card ctcard@hotmail.com wrote:
Did you make the changes to the DB flags for MDB that I noted the other day? For me, MDB writes are a minimum of 65 times faster than writes with BDB/HDB, even when BDB/HDB use an SHM key.
Yes. I have these values in my mdb configuration: olcDbEnvFlags: writemap olcDbEnvFlags: nometasync
I think these setting may have helped to speed up slapadd, but not ldapmodify.
I'm specifically talking about ldapmodify actually. ;)
Maybe MDB performance relative to BDB degrades as the database get bigger.
From your wiki page: "This particular client has 25,208 entries in their LDAP database".
My test database has over 3 million entries (production has nearly 7 million), which take about 20 minutes to slapadd into MDB initially. My machine has 128 Gb RAM and the MDB db size is 429496729600, so slapd can't map the whole db file into RAM.
Chris
--On Thursday, April 18, 2013 3:59 PM +0000 Chris Card ctcard@hotmail.com wrote:
Maybe MDB performance relative to BDB degrades as the database get bigger. From your wiki page: "This particular client has 25,208 entries in their LDAP database". My test database has over 3 million entries (production has nearly 7 million), which take about 20 minutes to slapadd into MDB initially. My machine has 128 Gb RAM and the MDB db size is 429496729600, so slapd can't map the whole db file into RAM.
How big is your DB when loaded vs maxsize? If you are using writemap, the maxsize *must* be larger than the DB size. Also, what OpenLDAP version are you using? That's always important to note.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
----------------------------------------
Date: Thu, 18 Apr 2013 09:13:42 -0700 From: quanah@zimbra.com To: ctcard@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend?
--On Thursday, April 18, 2013 3:59 PM +0000 Chris Card ctcard@hotmail.com wrote:
Maybe MDB performance relative to BDB degrades as the database get bigger. From your wiki page: "This particular client has 25,208 entries in their LDAP database". My test database has over 3 million entries (production has nearly 7 million), which take about 20 minutes to slapadd into MDB initially. My machine has 128 Gb RAM and the MDB db size is 429496729600, so slapd can't map the whole db file into RAM.
How big is your DB when loaded vs maxsize? If you are using writemap, the maxsize *must* be larger than the DB size. Also, what OpenLDAP version are you using? That's always important to note.
I'm using openldap 2.4.35.
olcDbMaxSize: 429496729600 and that's the size of the file on the disk too:
-rw------- 1 ldap ldap 429496729600 Apr 18 16:52 data.mdb -rw------- 1 ldap ldap 8192 Apr 18 17:52 lock.mdb
The size of the db files for BDB is about 22 Gb.
Chris
--On Thursday, April 18, 2013 4:57 PM +0000 Chris Card ctcard@hotmail.com wrote:
I'm using openldap 2.4.35.
olcDbMaxSize: 429496729600 and that's the size of the file on the disk too:
-rw------- 1 ldap ldap 429496729600 Apr 18 16:52 data.mdb -rw------- 1 ldap ldap 8192 Apr 18 17:52 lock.mdb
What's the *actual* size of the mdb db? Remember it is a sparse file.
https://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning_8.0#Notes_on_MDB
I routinely test with DBs in the range of 3 million to 6 million entries, and it is always substantially faster than bdb/hdb.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
----------------------------------------
Date: Thu, 18 Apr 2013 10:13:10 -0700 From: quanah@zimbra.com To: ctcard@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend?
--On Thursday, April 18, 2013 4:57 PM +0000 Chris Card ctcard@hotmail.com wrote:
I'm using openldap 2.4.35.
olcDbMaxSize: 429496729600 and that's the size of the file on the disk too:
-rw------- 1 ldap ldap 429496729600 Apr 18 16:52 data.mdb -rw------- 1 ldap ldap 8192 Apr 18 17:52 lock.mdb
What's the *actual* size of the mdb db? Remember it is a sparse file.
https://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning_8.0#Notes_on_MDB
I routinely test with DBs in the range of 3 million to 6 million entries, and it is always substantially faster than bdb/hdb.
du -c -h data.mdb returns 16G
Chris
--On Thursday, April 18, 2013 5:21 PM +0000 Chris Card ctcard@hotmail.com wrote:
I routinely test with DBs in the range of 3 million to 6 million entries, and it is always substantially faster than bdb/hdb.
du -c -h data.mdb returns 16G
So drop your maxsize to 32GB from 400GB. Clearly you have no need to exceed your total RAM.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Date: Thu, 18 Apr 2013 10:41:40 -0700 From: quanah@zimbra.com To: ctcard@hotmail.com; openldap-technical@openldap.org Subject: RE: How to improve performance with MDB backend?
--On Thursday, April 18, 2013 5:21 PM +0000 Chris Card ctcard@hotmail.com wrote:
I routinely test with DBs in the range of 3 million to 6 million entries, and it is always substantially faster than bdb/hdb.
du -c -h data.mdb returns 16G
So drop your maxsize to 32GB from 400GB. Clearly you have no need to exceed your total RAM.
I'll give that a try. I had perhaps been taking the advice given in the slapd-mdb man page too seriously:"Note: It is important to set this to as large a value as possible, (relative to anticipated growth of the actual data over time) since growing the size later may not be practical when the system is under heavy load." Chris
--On Thursday, April 18, 2013 8:18 PM +0000 Chris Card ctcard@hotmail.com wrote:
I'll give that a try. I had perhaps been taking the advice given in the slapd-mdb man page too seriously:
Heh, yes, it seems so. ;) With writemap, you definitely do not want to exceed actual RAM.
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
Quanah Gibson-Mount wrote:
--On Thursday, April 18, 2013 8:18 PM +0000 Chris Card ctcard@hotmail.com wrote:
I'll give that a try. I had perhaps been taking the advice given in the slapd-mdb man page too seriously:
Heh, yes, it seems so. ;) With writemap, you definitely do not want to exceed actual RAM.
Nonsense. Set the size to as large as you need to allow the DB to grow. Period. Writemap has no bearing on this.
----------------------------------------
Date: Fri, 19 Apr 2013 00:03:30 -0700 From: hyc@symas.com To: quanah@zimbra.com; ctcard@hotmail.com; openldap-technical@openldap.org Subject: Re: How to improve performance with MDB backend?
Quanah Gibson-Mount wrote:
--On Thursday, April 18, 2013 8:18 PM +0000 Chris Card ctcard@hotmail.com wrote:
I'll give that a try. I had perhaps been taking the advice given in the slapd-mdb man page too seriously:
Heh, yes, it seems so. ;) With writemap, you definitely do not want to exceed actual RAM.
Nonsense. Set the size to as large as you need to allow the DB to grow. Period. Writemap has no bearing on this.
I tried reducing the maxsize, but it made no difference to the performance. So I'm still at the point where writes to BDB are roughly 4 times faster than writes to MDB. Any more suggestions?
Chris
On 19/4/2013 2:00 μμ, Chris Card wrote:
I tried reducing the maxsize, but it made no difference to the performance. So I'm still at the point where writes to BDB are roughly 4 times faster than writes to MDB. Any more suggestions?
Could it be possibly related to the OS / filesystem used? Please provide details. System IO load?
Is OpenLDAP compiled by you or you are using a standard package? Which?
(I am not an expert on this, just trying to possibly consider other factors.)
Nick
On 19/4/2013 2:00 μμ, Chris Card wrote:
I tried reducing the maxsize, but it made no difference to the performance. So I'm still at the point where writes to BDB are roughly 4 times faster than writes to MDB. Any more suggestions?
Could it be possibly related to the OS / filesystem used? Please provide details. System IO load?
Is OpenLDAP compiled by you or you are using a standard package? Which?
(I am not an expert on this, just trying to possibly consider other factors.)
The MDB and BDB tests are done on the same machine, and the same data, the only difference is the openldap configuration. The MDB and BDB files are on the same filesystem (xfs). OpenLDAP is built into an rpm for CentOS 6.3 on our build system from the latest 2.4.35 source downloaded from openldap.org, using a spec file based on the one from the LTB project.
Chris
First test with your sugestions. I am using Phyton program writen by me to add data to server. Server is Centos 6.2 based (hardware described in my first post) Python runs on separeate dual core PC with 1Gb connection to servers. Servers are configured as N-way Multymaster
Test start Test stop Test duration Num users User/sec
19.4.2013 19.4.2013 sec 7789,00 1000000 128,39 11:53:45 14:03:34 min 129,8166667
Database location mounted as: UUID=616c291a-7fe4-47a1-87d1-c221a8e1c4f8 /opt ext4 noatime,auto 1 2
vm.dirty_ratio = 90 vm.dirty_expire_centisecs = 60000
Scheduler is: [root@spr1 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq]
memory manager libhoard.so (latest from hoard site)
Database size 36GB
-rw-------. 1 openldap openldap 14471663616 Apr 19 12:01 data.mdb -rw-------. 1 openldap openldap 8192 Apr 19 12:02 lock.mdb
[root@spr1 openldap-data]# du -hs data.mdb 14G data.mdb
This is user data (one user). I can add 128,39 complete users in 1 second.
dn: uid=1,ds=USERS,o=STANDARD,dc=SPR objectClass: sprUser uid: 1
dn: subdata=userData,uid=1,ds=USERS,o=STANDARD,dc=SPR roamingAllowed: TRUE objectClass: sprUserData subdata: userData prepaidFlag: TRUE sn: TestProv pfUsername: user1 givenName: pcrfUser loginPassword: 123 pfPassword: 123
dn: subdata=applicationData,uid=1,ds=USERS,o=STANDARD,dc=SPR objectClass: sprUserData subdata: applicationData
dn: ds=aaa,subdata=applicationData,uid=1,ds=USERS,o=STANDARD,dc=SPR objectClass: sprDs ds: aaa
dn: ds=2g3g,ds=aaa,subdata=applicationData,uid=1,ds=USERS,o=STANDARD,dc=SPR objectClass: sprDs ds: 2g3g
dn: ds=pcrf,ds=aaa,subdata=applicationData,uid=1,ds=USERS,o=STANDARD,dc=SPR objectClass: sprDs ds: pcrf
dn: subdata=2g3gProf,ds=2g3g,ds=aaa,subdata=applicationData,uid=1,ds=USERS,o=STA NDARD,dc=SPR service2g3g: defaultService profile2g3g: profile1 productCode2g3g: prodCode barringStatus2g3g: barringStatus objectClass: spr2g3gUser uniqueId2g3g: user1 apnGroup2g3g: apnGG provider2g3g: memyselveandi framedIPAddress2g3g: 1.1.1.1 accountGroup2g3g: acctGroup subdata: 2g3gProf msisdn: 1234560001 flagStatus2g3g: flagStatus class2g3g: classs type2g3g: type creationDate2g3g: 01.10.2010. classification2g3g: 111 poolGroup2g3g: poolGroup status2g3g: ACTIVE
dn: subdata=pcrfProf,ds=pcrf,ds=aaa,subdata=applicationData,uid=1,ds=USERS,o=STA NDARD,dc=SPR profile: 3100 totalUsedVolume: 3 msisdn: 1234560001 objectClass: sprPcrfProf totalMaximumVolume: 5 imsi: 1991234560001 subdata: pcrfProf subscriberServiceProfile: TM00003100 featureList: Faeture1 paymentTypexy: NeverPaid pfUsername: user1 tariffId: 1 pcrfServiceId: 3100
dn: pfUsername=user1,dc=USERNAME,dc=SPR objectClass: alias objectClass: extensibleObject pfUsername: user1 aliasedObjectName: uid=1,ds=USERS,o=STANDARD,dc=spr
dn: msisdn=1234560001,dc=MSISDN,dc=SPR objectClass: alias objectClass: extensibleObject msisdn: 1234560001 aliasedObjectName: uid=1,ds=USERS,o=STANDARD,dc=spr
dn: imsi=1991234560001,dc=IMSI,dc=SPR objectClass: alias objectClass: extensibleObject aliasedObjectName: uid=1,ds=USERS,o=STANDARD,dc=spr imsi: 1991234560001
On Fri, Apr 19, 2013 at 1:54 PM, Nick Milas nick@eurobjects.com wrote:
On 19/4/2013 2:00 μμ, Chris Card wrote:
I tried reducing the maxsize, but it made no difference to the
performance. So I'm still at the point where writes to BDB are roughly 4 times faster than writes to MDB. Any more suggestions?
Could it be possibly related to the OS / filesystem used? Please provide details. System IO load?
Is OpenLDAP compiled by you or you are using a standard package? Which?
(I am not an expert on this, just trying to possibly consider other factors.)
Nick
Saša-Stjepan Bakša wrote:
First test with your sugestions. I am using Phyton program writen by me to add data to server. Server is Centos 6.2 based (hardware described in my first post) Python runs on separeate dual core PC with 1Gb connection to servers. Servers are configured as N-way Multymaster
Test start Test stop Test duration Num users User/sec
19.4.2013 19.4.2013 sec 7789,00 1000000 128,39 11:53:45 14:03:34 min 129,8166667
Database location mounted as: UUID=616c291a-7fe4-47a1-87d1-c221a8e1c4f8 /opt ext4 noatime,auto 1 2
vm.dirty_ratio = 90 vm.dirty_expire_centisecs = 60000
Scheduler is: [root@spr1 ~]# cat /sys/block/sda/queue/scheduler noop anticipatory deadline [cfq]
memory manager libhoard.so (latest from hoard site)
You really need to learn something more about system administration; you clearly don't know what to investigate but this is all fundamental sysadmin knowledge.
First things first - when something is "slow" - what exactly is slow? Is it using excessive CPU time? Is it waiting for disk I/O? Every sysadmin should automatically ask this question first of all, and every sysadmin should know how to tell the difference. If you don't know these things then you are not qualified to be a sysadmin and need to go get training. This is not the forum for teaching you these things.
Copy/pasting someone else's VM tuning settings without understanding what they mean or why they are being set is "cargo cult sysadmin". It is wrong and nobody on this list / in this community should be encouraging it. Quick easy spoonfed answers don't actually help understanding, and understanding is the only real way forward.
In particular, VM tuning settings are highly OS dependent, and probably kernel version dependent too. Good settings depend on exactly what your own system contains; settings that work for someone else may be useless or worse on your own setup.
Simple answers have narrow relevance that gets obsolete quickly. Learning how to think and investigate problems is knowledge that serves you the rest of your life.
As a starting point - what does vmstat tell you? Don't just paste its output here, learn what it means.
Ok. I see your point and lesson is learned. Will do as suggested.
On Fri, Apr 19, 2013 at 6:07 PM, Howard Chu hyc@symas.com wrote:
You really need to learn something more about system administration; you clearly don't know what to investigate but this is all fundamental sysadmin knowledge.
First things first - when something is "slow" - what exactly is slow? Is it using excessive CPU time? Is it waiting for disk I/O? Every sysadmin should automatically ask this question first of all, and every sysadmin should know how to tell the difference. If you don't know these things then you are not qualified to be a sysadmin and need to go get training. This is not the forum for teaching you these things.
Copy/pasting someone else's VM tuning settings without understanding what they mean or why they are being set is "cargo cult sysadmin". It is wrong and nobody on this list / in this community should be encouraging it. Quick easy spoonfed answers don't actually help understanding, and understanding is the only real way forward.
In particular, VM tuning settings are highly OS dependent, and probably kernel version dependent too. Good settings depend on exactly what your own system contains; settings that work for someone else may be useless or worse on your own setup.
Simple answers have narrow relevance that gets obsolete quickly. Learning how to think and investigate problems is knowledge that serves you the rest of your life.
As a starting point - what does vmstat tell you? Don't just paste its output here, learn what it means.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/**project/http://www.openldap.org/project/
Just for Info, you should take a look to the "tuned" daemon and setting a consistent profile for your workload. It can help a lot.
Best
2013/4/19, Saša-Stjepan Bakša ssbaksa@gmail.com:
Ok. I see your point and lesson is learned. Will do as suggested.
On Fri, Apr 19, 2013 at 6:07 PM, Howard Chu hyc@symas.com wrote:
You really need to learn something more about system administration; you clearly don't know what to investigate but this is all fundamental sysadmin knowledge.
First things first - when something is "slow" - what exactly is slow? Is it using excessive CPU time? Is it waiting for disk I/O? Every sysadmin should automatically ask this question first of all, and every sysadmin should know how to tell the difference. If you don't know these things then you are not qualified to be a sysadmin and need to go get training. This is not the forum for teaching you these things.
Copy/pasting someone else's VM tuning settings without understanding what they mean or why they are being set is "cargo cult sysadmin". It is wrong and nobody on this list / in this community should be encouraging it. Quick easy spoonfed answers don't actually help understanding, and understanding is the only real way forward.
In particular, VM tuning settings are highly OS dependent, and probably kernel version dependent too. Good settings depend on exactly what your own system contains; settings that work for someone else may be useless or worse on your own setup.
Simple answers have narrow relevance that gets obsolete quickly. Learning how to think and investigate problems is knowledge that serves you the rest of your life.
As a starting point - what does vmstat tell you? Don't just paste its output here, learn what it means.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/**project/http://www.openldap.org/project/
Ok. tnx for your suggestion.
br Sasa .
On Sat, Apr 20, 2013 at 10:16 AM, devzero2000 pinto.elia@gmail.com wrote:
Just for Info, you should take a look to the "tuned" daemon and setting a consistent profile for your workload. It can help a lot.
Best
2013/4/19, Saša-Stjepan Bakša ssbaksa@gmail.com:
Ok. I see your point and lesson is learned. Will do as suggested.
On Fri, Apr 19, 2013 at 6:07 PM, Howard Chu hyc@symas.com wrote:
You really need to learn something more about system administration; you clearly don't know what to investigate but this is all fundamental sysadmin knowledge.
First things first - when something is "slow" - what exactly is slow? Is it using excessive CPU time? Is it waiting for disk I/O? Every sysadmin should automatically ask this question first of all, and every sysadmin should know how to tell the difference. If you don't know these things then you are not qualified to be a sysadmin and need to go get training. This is not the forum for teaching you these things.
Copy/pasting someone else's VM tuning settings without understanding
what
they mean or why they are being set is "cargo cult sysadmin". It is
wrong
and nobody on this list / in this community should be encouraging it. Quick easy spoonfed answers don't actually help understanding, and understanding is the only real way forward.
In particular, VM tuning settings are highly OS dependent, and probably kernel version dependent too. Good settings depend on exactly what your own system contains; settings that work for someone else may be useless or worse on your own setup.
Simple answers have narrow relevance that gets obsolete quickly.
Learning
how to think and investigate problems is knowledge that serves you the rest of your life.
As a starting point - what does vmstat tell you? Don't just paste its output here, learn what it means.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/**project/http://www.openldap.org/project/
-- Inviato dal mio dispositivo mobile
Quanah Gibson-Mount wrote:
--On Thursday, April 18, 2013 3:59 PM +0000 Chris Card ctcard@hotmail.com wrote:
Maybe MDB performance relative to BDB degrades as the database get bigger. From your wiki page: "This particular client has 25,208 entries in their LDAP database". My test database has over 3 million entries (production has nearly 7 million), which take about 20 minutes to slapadd into MDB initially. My machine has 128 Gb RAM and the MDB db size is 429496729600, so slapd can't map the whole db file into RAM.
How big is your DB when loaded vs maxsize? If you are using writemap, the maxsize *must* be larger than the DB size. Also, what OpenLDAP version are you using? That's always important to note.
The maxsize should always be larger than the DB, regardless of writemap. That is, if you expect the DB to grow over time.
On 18/4/2013 6:16 μμ, Quanah Gibson-Mount wrote:
For me, MDB writes are a minimum of 65 times faster than writes with BDB/HDB, even when BDB/HDB use an SHM key.
Can you please share your compilation options (or spec file, if applicable) and test setup complete configuration so interested people can compare to their setups?
Thanks, Nick
For second test I have changed I/O scheduler only Nothing changed in terms of adding speed!
Test start Test stop Test duration Num users User/sec
19.4.2013 19.4.2013 min 5759,00 742837 128,99 14:37:45 16:13:34 sec 95,98333
[root@spr1 /]# cat /sys/block/sda/queue/scheduler noop [anticipatory] deadline cfq
Total DISK READ: 0.00 B/s | Total DISK WRITE: 7.91 M/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 4165 be/4 root 0.00 B/s 1307.21 K/s 0.00 % 99.99 % [flush-8:0] 2058 be/3 root 0.00 B/s 0.00 B/s 0.00 % 99.99 % [jbd2/sda3-8] 4144 be/4 openldap 0.00 B/s 2.23 M/s 0.00 % 0.00 % slapd -h ldap:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap 4145 be/4 openldap 0.00 B/s 2.18 M/s 0.00 % 0.00 % slapd -h ldap:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap 4146 be/4 openldap 0.00 B/s 2.28 M/s 0.00 % 0.00 % slapd -h ldap:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap 4157 be/4 openldap 0.00 B/s 2.19 M/s 0.00 % 0.00 % slapd -h ldap:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap 1 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % init
On second server I/O is different
Total DISK READ: 0.00 B/s | Total DISK WRITE: 9.92 M/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 2141 be/3 root 0.00 B/s 0.00 B/s 0.00 % 99.99 % [jbd2/sda3-8] 4284 be/4 root 0.00 B/s 1580.46 K/s 0.00 % 73.88 % [flush-8:0] 4201 be/4 openldap 0.00 B/s 56.76 M/s 0.00 % 0.00 % slapd -h ldap:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap
I/O difference?
Saša-Stjepan Bakša wrote:
How do you measure or compare speed?
I have a Python program which add a predefined number of users (1mil and each user consists of 8 DNs and 3 alias DNs) and for Modify test I have also Python program which modify 1 attribute with random value under one DN changed sequentially from 1 to 1mil)
Now I can add those 1 million of users in 5508 sec what amounts to 181,55 users per second.
Modify program can do 2141,33 modifications per sec.
Do you find those numbers reasonable or are they nonsense?
Is there a way to get 10000 modifications per second or I am asking a stupid question?
Sasa
On Mon, Apr 8, 2013 at 4:40 PM, Quanah Gibson-Mount <quanah@zimbra.com mailto:quanah@zimbra.com> wrote:
--On Monday, April 08, 2013 4:49 AM -0700 Howard Chu <hyc@symas.com <mailto:hyc@symas.com>> wrote: There's nothing particular to LMDB to tune. But if you're seeing pauses due to disk I/O, as your iotop output seems to indicate, you might want to look into using a different I/O scheduler. Not quite true. On Linux, I suggest setting the olcDbFlags as follows: olcDbEnvFlags: writemap olcDbEnvFlags: nometasync With those flags set, writes are 65x faster for me with back-mdb than they are with back-hdb/back-bdb.
Since he already had DbNosync set, nometasync is irrelevant. One thing to note is that ext3 and ext4 filesystems have their own writeback parameters, which are independent of the settings in /proc/sys/vm, so settings that work with other FSs like jfs or xfs will have no effect.
openldap-technical@openldap.org
-
Chris Card -
devzero2000 -
Howard Chu -
Michael Ströder -
Nick Milas -
Quanah Gibson-Mount -
Saša-Stjepan Bakša