Hi,
Year ago we have tested openldap with back_mdb and it was fantastic. Search worked as a charm. Database was filled with 20 mil. users and serach returned some 20 k results per sec (my colegue did the test).
Now we need that setup for some tests and we encountered very slow response - 1 search for user data with some aliased data need 8 to 20 seconds to be retrieved.
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w test -s sub -a always -b num=1234563123,dc=num,dc=SPR ObjectClass=*
num=1234563123,dc=num,dc=SPR is alias to uid aliasedObjectName: uid=1234563123,ds=USERS,o=STANDARD,dc=spr
We build our openldap from git source. We have tried new as older versions as well and no change is seen.
Hardware: SuperMicro, 2xQuad core, 32 GB RAM, RAID 10 storage. HP blade 2xQuad core, 64 GB RAM sorage 2 disks in mirror. Results are the same and not depending on hardware.
Openldap ver:
root@centdevel openldap# git log commit 68d9aa207f51b4d1ef29bb9876e7da8c7eaf0eee Author: Quanah Gibson-Mount quanah@openldap.org Date: Tue Apr 8 21:16:52 2014 -0500
ITS#7430, ITS#6359
OS is Centos 6.4 (also tryed on Centos 6.6)numx
mdb config part is:
[root@spr2 cn=config]# cat olcDatabase={1}mdb.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 2c245069 dn: olcDatabase={1}mdb objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /opt/openldap/var/openldap-data olcSuffix: dc=spr olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymou s auth by dn="cn=admin,dc=spr" write by * none olcAccess: {1}to attrs=shadowLastChange by self write by * read olcAccess: {2}to dn.base="" by * read olcAccess: {3}to * by self write by dn="cn=admin,dc=spr" write by * read olcLastMod: TRUE olcRootDN: cn=admin,dc=spr olcRootPW:: xyzdgsdsadeew olcDbCheckpoint: 4096 10 olcDbNoSync: TRUE olcDbIndex: objectClass eq olcDbIndex: uid eq olcDbIndex: num eq olcDbIndex: numx eq olcDbIndex: Username eq olcDbIndex: entryCSN eq olcDbIndex: entryUUID eq olcDbIndex: contextCSN eq olcDbMaxSize: 16106127360 structuralObjectClass: olcMdbConfig entryUUID: 21ac150c-6b30-1034-9009-81396a683c5e creatorsName: cn=admin,cn=config createTimestamp: 20150330135513Z entryCSN: 20150330135513.544218Z#000000#000#000000 modifiersName: cn=admin,cn=config modifyTimestamp: 20150330135513Z
MDB database stat:
[root@spr2 openldap]# /opt/openldap/sbin/mdb_stat /opt/openldap/var/openldap-data/ -e -rr -a Environment Info Map address: (nil) Map size: 16106127360 Page size: 4096 Max pages: 3932160 Number of pages used: 1523336 Last transaction ID: 16058165 Max readers: 126 Number of readers used: 0 Reader Table Status (no active readers) 0 stale readers cleared. (no active readers) Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 11 Status of ad2i Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 38 Status of contextCSN Tree depth: 0 Branch pages: 0 Leaf pages: 0 Overflow pages: 0 Entries: 0 Status of dn2i Tree depth: 4 Branch pages: 2937 Leaf pages: 333338 Overflow pages: 0 Entries: 16000069 Status of entryCSN Tree depth: 3 Branch pages: 3 Leaf pages: 307 Overflow pages: 0 Entries: 8000034 Status of entryUUID Tree depth: 3 Branch pages: 259 Leaf pages: 62932 Overflow pages: 0 Entries: 8000034 Status of id2e Tree depth: 4 Branch pages: 4446 Leaf pages: 1000005 Overflow pages: 0 Entries: 8000034 Status of numx Tree depth: 3 Branch pages: 128 Leaf pages: 22295 Overflow pages: 0 Entries: 2000004 Status of num Tree depth: 3 Branch pages: 129 Leaf pages: 22325 Overflow pages: 0 Entries: 2000004 Status of objectClass Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 29 Status of Username Tree depth: 0 Branch pages: 0 Leaf pages: 0 Overflow pages: 0 Entries: 0 Status of uid Tree depth: 3 Branch pages: 34 Leaf pages: 7883 Overflow pages: 0 Entries: 1000004
Build config: make clean ./configure --enable-hdb=no \ --enable-bdb=no \ --enable-monitor=yes \ --prefix=/opt/openldap \ --enable-local=yes \ --enable-accesslog=yes \ --enable-syncprov=yes \ --enable-debug=yes make depend make #STRIP='' rm -r /opt/openldap/etc/openldap/schema make install #STRIP=''
removing debug has no efect
Do you have any hint for us?
Br
Sasa
Does the server have access to a nameserver? I've seen dns timeouts cause this kind of thing.
On 1/04/2015 12:09 AM, Saša-Stjepan Bakša wrote:
Hi,
Year ago we have tested openldap with back_mdb and it was fantastic. Search worked as a charm. Database was filled with 20 mil. users and serach returned some 20 k results per sec (my colegue did the test).
Now we need that setup for some tests and we encountered very slow response - 1 search for user data with some aliased data need 8 to 20 seconds to be retrieved.
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w test -s sub -a always -b num=1234563123,dc=num,dc=SPR ObjectClass=*
num=1234563123,dc=num,dc=SPR is alias to uid aliasedObjectName: uid=1234563123,ds=USERS,o=STANDARD,dc=spr
We build our openldap from git source. We have tried new as older versions as well and no change is seen.
Hardware: SuperMicro, 2xQuad core, 32 GB RAM, RAID 10 storage. HP blade 2xQuad core, 64 GB RAM sorage 2 disks in mirror. Results are the same and not depending on hardware.
Openldap ver:
root@centdevel openldap# git log commit 68d9aa207f51b4d1ef29bb9876e7da8c7eaf0eee Author: Quanah Gibson-Mount <quanah@openldap.org mailto:quanah@openldap.org> Date: Tue Apr 8 21:16:52 2014 -0500
ITS#7430, ITS#6359OS is Centos 6.4 (also tryed on Centos 6.6)numx
mdb config part is:
[root@spr2 cn=config]# cat olcDatabase={1}mdb.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 2c245069 dn: olcDatabase={1}mdb objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /opt/openldap/var/openldap-data olcSuffix: dc=spr olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymou s auth by dn="cn=admin,dc=spr" write by * none olcAccess: {1}to attrs=shadowLastChange by self write by * read olcAccess: {2}to dn.base="" by * read olcAccess: {3}to * by self write by dn="cn=admin,dc=spr" write by * read olcLastMod: TRUE olcRootDN: cn=admin,dc=spr olcRootPW:: xyzdgsdsadeew olcDbCheckpoint: 4096 10 olcDbNoSync: TRUE olcDbIndex: objectClass eq olcDbIndex: uid eq olcDbIndex: num eq olcDbIndex: numx eq olcDbIndex: Username eq olcDbIndex: entryCSN eq olcDbIndex: entryUUID eq olcDbIndex: contextCSN eq olcDbMaxSize: 16106127360 structuralObjectClass: olcMdbConfig entryUUID: 21ac150c-6b30-1034-9009-81396a683c5e creatorsName: cn=admin,cn=config createTimestamp: 20150330135513Z entryCSN: 20150330135513.544218Z#000000#000#000000 modifiersName: cn=admin,cn=config modifyTimestamp: 20150330135513Z
MDB database stat:
[root@spr2 openldap]# /opt/openldap/sbin/mdb_stat /opt/openldap/var/openldap-data/ -e -rr -a Environment Info Map address: (nil) Map size: 16106127360 Page size: 4096 Max pages: 3932160 Number of pages used: 1523336 Last transaction ID: 16058165 Max readers: 126 Number of readers used: 0 Reader Table Status (no active readers) 0 stale readers cleared. (no active readers) Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 11 Status of ad2i Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 38 Status of contextCSN Tree depth: 0 Branch pages: 0 Leaf pages: 0 Overflow pages: 0 Entries: 0 Status of dn2i Tree depth: 4 Branch pages: 2937 Leaf pages: 333338 Overflow pages: 0 Entries: 16000069 Status of entryCSN Tree depth: 3 Branch pages: 3 Leaf pages: 307 Overflow pages: 0 Entries: 8000034 Status of entryUUID Tree depth: 3 Branch pages: 259 Leaf pages: 62932 Overflow pages: 0 Entries: 8000034 Status of id2e Tree depth: 4 Branch pages: 4446 Leaf pages: 1000005 Overflow pages: 0 Entries: 8000034 Status of numx Tree depth: 3 Branch pages: 128 Leaf pages: 22295 Overflow pages: 0 Entries: 2000004 Status of num Tree depth: 3 Branch pages: 129 Leaf pages: 22325 Overflow pages: 0 Entries: 2000004 Status of objectClass Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 29 Status of Username Tree depth: 0 Branch pages: 0 Leaf pages: 0 Overflow pages: 0 Entries: 0 Status of uid Tree depth: 3 Branch pages: 34 Leaf pages: 7883 Overflow pages: 0 Entries: 1000004
Build config: make clean ./configure --enable-hdb=no \ --enable-bdb=no \ --enable-monitor=yes \ --prefix=/opt/openldap \ --enable-local=yes \ --enable-accesslog=yes \ --enable-syncprov=yes \ --enable-debug=yes make depend make #STRIP='' rm -r /opt/openldap/etc/openldap/schema make install #STRIP=''
removing debug has no efect
Do you have any hint for us?
Br
Sasa
On 31 March 2015 at 23:25, Geoff Swan gswan3@bigpond.net.au wrote:
Does the server have access to a nameserver? I've seen dns timeouts cause this kind of thing.
Yes it has. Also, I have created distinct entry's in hosts file to make name resolving quicker and to make sure that name resolving is not the main culprit for this slow down.
When I start slapd in debug mode, normally all answers are so quick written on the screen that I can't read without stopping slapd. With our setup, I can read line by line.
On 1/04/2015 12:09 AM, Saša-Stjepan Bakša wrote:
Hi,
Year ago we have tested openldap with back_mdb and it was fantastic. Search worked as a charm. Database was filled with 20 mil. users and serach returned some 20 k results per sec (my colegue did the test).
Now we need that setup for some tests and we encountered very slow response - 1 search for user data with some aliased data need 8 to 20 seconds to be retrieved.
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w test -s sub -a always -b num=1234563123,dc=num,dc=SPR ObjectClass=*
num=1234563123,dc=num,dc=SPR is alias to uid aliasedObjectName: uid=1234563123,ds=USERS,o=STANDARD,dc=spr
We build our openldap from git source. We have tried new as older versions as well and no change is seen.
Hardware: SuperMicro, 2xQuad core, 32 GB RAM, RAID 10 storage. HP blade 2xQuad core, 64 GB RAM sorage 2 disks in mirror. Results are the same and not depending on hardware.
Openldap ver:
root@centdevel openldap# git log commit 68d9aa207f51b4d1ef29bb9876e7da8c7eaf0eee Author: Quanah Gibson-Mount quanah@openldap.org Date: Tue Apr 8 21:16:52 2014 -0500
ITS#7430, ITS#6359OS is Centos 6.4 (also tryed on Centos 6.6)numx
mdb config part is:
[root@spr2 cn=config]# cat olcDatabase={1}mdb.ldif # AUTO-GENERATED FILE - DO NOT EDIT!! Use ldapmodify. # CRC32 2c245069 dn: olcDatabase={1}mdb objectClass: olcDatabaseConfig objectClass: olcMdbConfig olcDatabase: {1}mdb olcDbDirectory: /opt/openldap/var/openldap-data olcSuffix: dc=spr olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymou s auth by dn="cn=admin,dc=spr" write by * none olcAccess: {1}to attrs=shadowLastChange by self write by * read olcAccess: {2}to dn.base="" by * read olcAccess: {3}to * by self write by dn="cn=admin,dc=spr" write by * read olcLastMod: TRUE olcRootDN: cn=admin,dc=spr olcRootPW:: xyzdgsdsadeew olcDbCheckpoint: 4096 10 olcDbNoSync: TRUE olcDbIndex: objectClass eq olcDbIndex: uid eq olcDbIndex: num eq olcDbIndex: numx eq olcDbIndex: Username eq olcDbIndex: entryCSN eq olcDbIndex: entryUUID eq olcDbIndex: contextCSN eq olcDbMaxSize: 16106127360 structuralObjectClass: olcMdbConfig entryUUID: 21ac150c-6b30-1034-9009-81396a683c5e creatorsName: cn=admin,cn=config createTimestamp: 20150330135513Z entryCSN: 20150330135513.544218Z#000000#000#000000 modifiersName: cn=admin,cn=config modifyTimestamp: 20150330135513Z
MDB database stat:
[root@spr2 openldap]# /opt/openldap/sbin/mdb_stat /opt/openldap/var/openldap-data/ -e -rr -a Environment Info Map address: (nil) Map size: 16106127360 Page size: 4096 Max pages: 3932160 Number of pages used: 1523336 Last transaction ID: 16058165 Max readers: 126 Number of readers used: 0 Reader Table Status (no active readers) 0 stale readers cleared. (no active readers) Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 11 Status of ad2i Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 38 Status of contextCSN Tree depth: 0 Branch pages: 0 Leaf pages: 0 Overflow pages: 0 Entries: 0 Status of dn2i Tree depth: 4 Branch pages: 2937 Leaf pages: 333338 Overflow pages: 0 Entries: 16000069 Status of entryCSN Tree depth: 3 Branch pages: 3 Leaf pages: 307 Overflow pages: 0 Entries: 8000034 Status of entryUUID Tree depth: 3 Branch pages: 259 Leaf pages: 62932 Overflow pages: 0 Entries: 8000034 Status of id2e Tree depth: 4 Branch pages: 4446 Leaf pages: 1000005 Overflow pages: 0 Entries: 8000034 Status of numx Tree depth: 3 Branch pages: 128 Leaf pages: 22295 Overflow pages: 0 Entries: 2000004 Status of num Tree depth: 3 Branch pages: 129 Leaf pages: 22325 Overflow pages: 0 Entries: 2000004 Status of objectClass Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 29 Status of Username Tree depth: 0 Branch pages: 0 Leaf pages: 0 Overflow pages: 0 Entries: 0 Status of uid Tree depth: 3 Branch pages: 34 Leaf pages: 7883 Overflow pages: 0 Entries: 1000004
Build config: make clean ./configure --enable-hdb=no \ --enable-bdb=no \ --enable-monitor=yes \ --prefix=/opt/openldap \ --enable-local=yes \ --enable-accesslog=yes \ --enable-syncprov=yes \ --enable-debug=yes make depend make #STRIP='' rm -r /opt/openldap/etc/openldap/schema make install #STRIP=''
removing debug has no efect
Do you have any hint for us?
Br
Sasa
Am Donnerstag, 02. April 2015 13:40 CEST, Saša-Stjepan Bakša ssbaksa@gmail.com schrieb:
On 31 March 2015 at 23:25, Geoff Swan gswan3@bigpond.net.au wrote:
Does the server have access to a nameserver? I've seen dns timeouts cause this kind of thing.
Yes it has. Also, I have created distinct entry's in hosts file to make name resolving quicker and to make sure that name resolving is not the main culprit for this slow down.
When I start slapd in debug mode, normally all answers are so quick written on the screen that I can't read without stopping slapd. With our setup, I can read line by line.
One shure performance killer I encountered was activating gccess logging. If this isn0t the case you might need to set up some profiling. Since you are running under Linux perf might be a good tool. It might be a good idea to recompile your binary with "-fno-omit-frame-pointer" to get meaningful/correct call graph reports. You then can do:
perf record -g -p <pid-of-ldapd>
and then later on
perf report --stdio perf report --stdio --sort=dso -g none perf report --stdio -g none perf report --stdio -g
(have a look at https://perf.wiki.kernel.org/index.php/Tutorial)
HTH Ralf Mattes
Hi,
It was long weekend with Easter holiday so...
To answer Mattes, and Ulrich first, I will do as suggested by you.
But something new first. If I have tried to add less data to database and then I have conducted the search. Till some point it works and then fails. It looks like when database is filed with more than some amount of data it stops to send data back.
To be sure that my build is not the culprit I have installed in parallel on similar server Ubuntu 14.04 with their slapd package (I know, it is not good but just for this test...) and this happened when I have tried this:
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w password -s sub -a always -b dc=SPR objectClass=*
My build on centos 6.6 chrashed with this message:
5523ac18 >>> slap_listener(ldap:///) 5523ac18 connection_get(11): got connid=1001 5523ac18 connection_read(11): checking for input on id=1001 ber_get_next ber_get_next: tag 0x30 len 34 contents: 5523ac18 op tag 0x60, time 1428401176 ber_get_next 5523ac18 conn=1001 op=0 do_bind ber_scanf fmt ({imt) ber: ber_scanf fmt (m}) ber: 5523ac18 >>> dnPrettyNormal: <cn=admin,dc=spr> 5523ac18 <<< dnPrettyNormal: <cn=admin,dc=spr>, <cn=admin,dc=spr> 5523ac18 do_bind: version=3 dn="cn=admin,dc=spr" method=128 5523ac18 do_bind: v3 bind: "cn=admin,dc=spr" to "cn=admin,dc=spr" 5523ac18 send_ldap_result: conn=1001 op=0 p=3 5523ac18 send_ldap_response: msgid=1 tag=97 err=0 ber_flush2: 14 bytes to sd 11 5523ac18 connection_get(11): got connid=1001 5523ac18 connection_read(11): checking for input on id=1001 ber_get_next ber_get_next: tag 0x30 len 43 contents: 5523ac18 op tag 0x63, time 1428401176 ber_get_next 5523ac18 conn=1001 op=1 do_search ber_scanf fmt ({miiiib) ber: 5523ac18 >>> dnPrettyNormal: <dc=SPR> 5523ac18 <<< dnPrettyNormal: <dc=SPR>, <dc=spr> ber_scanf fmt (m) ber: ber_scanf fmt ({M}}) ber: 5523ac18 => mdb_search 5523ac18 mdb_dn2entry("dc=spr") 5523ac18 => mdb_dn2id("dc=spr") 5523ac18 <= mdb_dn2id: got id=0x1 5523ac18 => mdb_entry_decode: 5523ac18 <= mdb_entry_decode 5523ac18 search_candidates: base="dc=spr" (0x00000001) scope=2 5523ac18 => mdb_equality_candidates (objectClass) 5523ac18 => key_read 5523ac18 <= mdb_index_read 10000000 candidates 5523ac18 <= mdb_equality_candidates: id=-1, first=16, last=10000015 Segmentation fault
On Ubuntu 14.04 passed through but no result:
Apr 7 12:51:36 spr2 slapd[8432]: conn=1001 op=1 SRCH base="dc=SPR" scope=2 deref=3 filter="(objectClass=*)" Apr 7 12:51:36 spr2 slapd[8432]: => mdb_search Apr 7 12:51:36 spr2 slapd[8432]: mdb_dn2entry("dc=spr") Apr 7 12:51:36 spr2 slapd[8432]: => mdb_dn2id("dc=spr") Apr 7 12:51:36 spr2 slapd[8432]: <= mdb_dn2id: got id=0x1 Apr 7 12:51:36 spr2 slapd[8432]: => mdb_entry_decode: Apr 7 12:51:36 spr2 slapd[8432]: <= mdb_entry_decode Apr 7 12:51:36 spr2 slapd[8432]: => access_allowed: search access to "dc=SPR" "entry" requested Apr 7 12:51:36 spr2 slapd[8432]: <= root access granted Apr 7 12:51:36 spr2 slapd[8432]: => access_allowed: search access granted by manage(=mwrscxd) Apr 7 12:51:36 spr2 slapd[8432]: search_candidates: base="dc=spr" (0x00000001) scope=2 Apr 7 12:51:36 spr2 slapd[8432]: => mdb_filter_candidates Apr 7 12:51:36 spr2 slapd[8432]: #011EQUALITY Apr 7 12:51:36 spr2 slapd[8432]: => mdb_equality_candidates (objectClass) Apr 7 12:51:36 spr2 slapd[8432]: => key_read Apr 7 12:51:36 spr2 slapd[8432]: mdb_idl_fetch_key: [01872a84] Apr 7 12:51:36 spr2 slapd[8432]: <= mdb_index_read 294110 candidates Apr 7 12:51:36 spr2 slapd[8432]: <= mdb_equality_candidates: id=-1, first=16, last=294125 Apr 7 12:51:36 spr2 slapd[8432]: <= mdb_filter_candidates: id=-1 first=16 last=294125 Apr 7 12:51:36 spr2 slapd[8432]: mdb_search_candidates: failed (rc=-30798) Apr 7 12:51:36 spr2 slapd[8432]: mdb_search: no candidates Apr 7 12:51:36 spr2 slapd[8432]: send_ldap_result: conn=1001 op=1 p=3 Apr 7 12:51:36 spr2 slapd[8432]: send_ldap_result: err=0 matched="" text="" Apr 7 12:51:36 spr2 slapd[8432]: send_ldap_response: msgid=2 tag=101 err=0 Apr 7 12:51:36 spr2 slapd[8432]: conn=1001 op=1 SEARCH RESULT tag=101 err=0 nentries=0 text= Apr 7 12:51:36 spr2 slapd[8432]: daemon: activity on 1 descriptor Apr 7 12:51:36 spr2 slapd[8432]: daemon: activity on: Apr 7 12:51:36 spr2 slapd[8432]: 13r
For test I have cut data from 2 mil lines to 300000 lines and database is now just 841 M in size (was 2,1 G)
root@spr2:/var/log# du -hs /var/lib/ldap/* 841M /var/lib/ldap/data.mdb 4.0K /var/lib/ldap/lock.mdb
And data is accessible:
----cut----- # search result search: 2 result: 0 Success
# numResponses: 22076 # numEntries: 22075
It looks like this is happening again:
http://www.openldap.org/lists/openldap-technical/201304/msg00198.html
Br
Sasa
On 7 April 2015 at 11:36, Saša-Stjepan Bakša ssbaksa@gmail.com wrote:
Hi,
It was long weekend with Easter holiday so...
To answer Mattes, and Ulrich first, I will do as suggested by you.
But something new first. If I have tried to add less data to database and then I have conducted the search. Till some point it works and then fails. It looks like when database is filed with more than some amount of data it stops to send data back.
To be sure that my build is not the culprit I have installed in parallel on similar server Ubuntu 14.04 with their slapd package (I know, it is not good but just for this test...) and this happened when I have tried this:
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w password -s sub -a always -b dc=SPR objectClass=*
My build on centos 6.6 chrashed with this message:
----cut----
For test I have cut data from 2 mil lines to 300000 lines and database is now just 841 M in size (was 2,1 G)
root@spr2:/var/log# du -hs /var/lib/ldap/* 841M /var/lib/ldap/data.mdb 4.0K /var/lib/ldap/lock.mdb
And data is accessible:
----cut----- # search result search: 2 result: 0 Success
# numResponses: 22076 # numEntries: 22075
It looks like this is happening again:
http://www.openldap.org/lists/openldap-technical/201304/msg00198.html
Just to add, after deleting big database from centos build openldap server, ldapsearch for all objects works as expected - no crashes.
----cut---- # search result search: 2 result: 0 Success
# numResponses: 22076 # numEntries: 22075
Br
Sasa
--On Tuesday, April 07, 2015 12:36 PM +0200 Saša-Stjepan Bakša ssbaksa@gmail.com wrote:
Hi,
It was long weekend with Easter holiday so...
To answer Mattes, and Ulrich first, I will do as suggested by you.
But something new first. If I have tried to add less data to database and then I have conducted the search. Till some point it works and then fails. It looks like when database is filed with more than some amount of data it stops to send data back.
To be sure that my build is not the culprit I have installed in parallel on similar server Ubuntu 14.04 with their slapd package (I know, it is not good but just for this test...) and this happened when I have tried this:
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w password -s sub -a always -b dc=SPR objectClass=*
Sounds like there's something wrong with your openldap build or server. Zimbra has clients with DBs of various sizes and entry counts, often in the multi-GBs, and we're not seeing any such issues.
--Quanah
--
Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
--On Tuesday, April 07, 2015 1:10 PM -0700 Quanah Gibson-Mount quanah@zimbra.com wrote:
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w password -s sub -a always -b dc=SPR objectClass=*
Sounds like there's something wrong with your openldap build or server. Zimbra has clients with DBs of various sizes and entry counts, often in the multi-GBs, and we're not seeing any such issues.
Actually, you are doing alias deref. I believe there's a known issue with back-mdb and alias deref. Do you see the same behavior if you select -a never instead?
--Quanah
--
Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
--On Tuesday, March 31, 2015 4:09 PM +0200 Saša-Stjepan Bakša ssbaksa@gmail.com wrote:
Hi,
Year ago we have tested openldap with back_mdb and it was fantastic. Search worked as a charm. Database was filled with 20 mil. users and serach returned some 20 k results per sec (my colegue did the test).
Now we need that setup for some tests and we encountered very slow response - 1 search for user data with some aliased data need 8 to 20 seconds to be retrieved.
ldapsearch -h 10.14.252.104 -p 389 -D cn=admin,dc=spr -w test -s sub -a always -b num=1234563123,dc=num,dc=SPR ObjectClass=*
num=1234563123,dc=num,dc=SPR is alias to uid aliasedObjectName: uid=1234563123,ds=USERS,o=STANDARD,dc=spr
Have you tried using writemap for the lmdb db?
--Quanah
--
Quanah Gibson-Mount Platform Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org
-
Geoff Swan -
Mattes -
Quanah Gibson-Mount -
Saša-Stjepan Bakša