oh, yes... forgot to say ... i have approx 2 milion entries within the LDAP.

T.

On Mon, Jun 15, 2009 at 3:53 PM, Tihomir Culjaga tculjaga@gmail.com wrote:

Same issue:... one request takes 93 ms the other 7 ms...

~$ time ldapsearch -h localhost -x -b ou=redirecting,ou=sipDirektor,dc=ot,dc=hr -D cn=admin,dc=ot,dc=hr -w **** uniqueID=38512303736 1.1 # extended LDIF # # LDAPv3 # base <ou=redirecting,ou=sipDirektor,dc=ot,dc=hr> with scope subtree # filter: uniqueID=38512303736 # requesting: 1.1 #

# 38512303736, redirecting, sipDirektor, ot.hr dn: uniqueID=38512303736,ou=redirecting,ou=sipDirektor,dc=ot,dc=hr

# search result search: 2 result: 0 Success

# numResponses: 2 # numEntries: 1

real 0m0.093s user 0m0.004s sys 0m0.004s

~$ time ldapsearch -h localhost -x -b ou=redirecting,ou=sipDirektor,dc=ot,dc=hr -D cn=admin,dc=ot,dc=hr -w **** uniqueID=38515000400 1.1 # extended LDIF # # LDAPv3 # base <ou=redirecting,ou=sipDirektor,dc=ot,dc=hr> with scope subtree # filter: uniqueID=38515000400 # requesting: 1.1 #

# 38515000400, redirecting, sipDirektor, ot.hr dn: uniqueID=38515000400,ou=redirecting,ou=sipDirektor,dc=ot,dc=hr

# search result search: 2 result: 0 Success

# numResponses: 2 # numEntries: 1

real 0m0.007s user 0m0.000s sys 0m0.000s

this is my DB_CONFIG

# $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.3.2.4 2007/12/18 11:53:27 ghenry Exp $ # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases. # # See the Oracle Berkeley DB documentation # < http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_con...

...

# for detail description of DB_CONFIG syntax and semantics. # # Hints can also be found in the OpenLDAP Software FAQ # http://www.openldap.org/faq/index.cgi?file=2 # in particular: # http://www.openldap.org/faq/index.cgi?file=1075

# Note: most DB_CONFIG settings will take effect only upon rebuilding # the DB environment.

# one 0.25 GB cache # set_cachesize 0 268435456 1 # one 768 MB cache #set_cachesize 0 805306368 1 # one 1GB cache set_cachesize 0 1073741824 1 # Data Directory #set_data_dir db

# Transaction Log settings set_lg_regionmax 262144 set_lg_bsize 2097152 #set_lg_dir logs

# Note: special DB_CONFIG flags are no longer needed for "quick" # slapadd(8) or slapindex(8) access (see their -q option).

and this is my slapd.conf

~$ cat /etc/ldap/slapd.conf

include /usr/local/etc/openldap/schema/core.schema include /usr/local/etc/openldap/schema/sipdirector.schema

pidfile /usr/local/var/run/slapd.pid argsfile /usr/local/var/run/slapd.args

loglevel 0

access to dn.base="" by * read

access to * by dn="cn=admin,dc=ot,dc=hr" write by * read

threads 8 ####################################################################### # BDB database definitions ####################################################################### backend bdb database bdb

cachesize 600000 idlcachesize 1800000 cachefree 1000

suffix "dc=ot,dc=hr" rootdn "cn=admin,dc=ot,dc=hr" // rootpw **** :) directory /usr/local/var/openldap-data

index objectClass eq index uniqueID eq

On Mon, Jun 15, 2009 at 3:01 PM, jakjr joao.alfredo@gmail.com wrote:

...

Are the attrs list returned in both entries euqal ?

Try to use the 1.1 attrs list.

man ldapsearch: If no attrs are listed, all user attributes are returned. If only 1.1 is listed, no attributes will be returned.

Regards

On Mon, Jun 15, 2009 at 8:29 AM, Tihomir Culjaga tculjaga@gmail.comwrote:

...

Hello,

maybe it is a dummy question but i'd like to know why i have so big discrepancy in execution between two apparently identical ldapsearch ?

The 1st search takes 94 ms while the 2nd one only 7 ms. It doesn't matter how many times i execute the 1st search (meaning everytihng should be already cached) .. it is always the same.

Does anyone know why?

~$ time ldapsearch -h localhost -x -b ou=redirecting,ou=Dir,dc=ot,dc=hr -D cn=admin,dc=ot,dc=hr -w **** uniqueID=38512303736

# extended LDIF # # LDAPv3 # base <ou=redirecting,ou=Dir,dc=ot,dc=hr> with scope subtree # filter: uniqueID=38512303736 # requesting: ALL #

# 38512303736, redirecting, Dir, ot.hr dn: uniqueID=38512303736,ou=redirecting,ou=Dir,dc=ot,dc=hr objectClass: top objectClass: uniqueID Prefix: 68A10 uniqueID: 38512303736

# search result search: 2 result: 0 Success

# numResponses: 2 # numEntries: 1

*real 0m0.094s* user 0m0.004s sys 0m0.000s

~$ time ldapsearch -h localhost -x -b ou=redirecting,ou=Dir,dc=ot,dc=hr -D cn=admin,dc=ot,dc=hr -w **** uniqueID=38515000400

# extended LDIF # # LDAPv3 # base <ou=redirecting,ou=Direktor,dc=ot,dc=hr> with scope subtree # filter: uniqueID=38515000400 # requesting: ALL #

# 38515000400, redirecting, Dir, ot.hr dn: uniqueID=38515000400,ou=redirecting,ou=Dir,dc=ot,dc=hr objectClass: top objectClass: uniqueID Prefix: 68B99 uniqueID: 38515000400

# search result search: 2 result: 0 Success

# numResponses: 2 # numEntries: 1

*real 0m0.007s* user 0m0.000s sys 0m0.004s tculjaga@l01sipindir2:~$

On 15/06/2009 20:38, Tihomir Culjaga wrote:

can anyone help ?

On Mon, Jun 15, 2009 at 3:58 PM, Tihomir Culjaga <tculjaga@gmail.com mailto:tculjaga@gmail.com> wrote:

oh, yes... forgot to say ... i have approx 2 milion entries within
the LDAP.

T.


On Mon, Jun 15, 2009 at 3:53 PM, Tihomir Culjaga <tculjaga@gmail.com
<mailto:tculjaga@gmail.com>> wrote:


    Same issue:... one request takes 93 ms the other 7 ms...

Hi,

What version of OpenLDAP are you using? I recall there being some issues recently with freeing cache on large databases. You may be running into this since you have 2 million entries and cachesize set to 600000 - thus not all entries fit in the cache. Whichever, I recommend using the latest, 2.4.16.

If performance is very important to you, you might consider optimising your cache parameters some more. Ideally, you could store all the BDB indexes (all *.bdb files in the BDB directory) in memory, and all entries.

If you have enough memory, and all your entries are likely to be requested, set cachesize higher than 2 million. See this FAQ entry for some more details: http://www.openldap.org/faq/data/cache/1075.html.

Regards, Jonathan

    ~$ time ldapsearch  -h localhost -x -b
    ou=redirecting,ou=sipDirektor,dc=ot,dc=hr  -D
    cn=admin,dc=ot,dc=hr -w **** uniqueID=38512303736 1.1
    # extended LDIF
    #
    # LDAPv3
    # base <ou=redirecting,ou=sipDirektor,dc=ot,dc=hr> with scope
    subtree
    # filter: uniqueID=38512303736
    # requesting: 1.1
    #

    # 38512303736, redirecting, sipDirektor, ot.hr <http://ot.hr>
    dn: uniqueID=38512303736,ou=redirecting,ou=sipDirektor,dc=ot,dc=hr


    # search result
    search: 2
    result: 0 Success

    # numResponses: 2
    # numEntries: 1

    real    0m0.093s
    user    0m0.004s
    sys     0m0.004s



    ~$ time ldapsearch  -h localhost -x -b
    ou=redirecting,ou=sipDirektor,dc=ot,dc=hr  -D
    cn=admin,dc=ot,dc=hr -w **** uniqueID=38515000400 1.1
    # extended LDIF
    #
    # LDAPv3
    # base <ou=redirecting,ou=sipDirektor,dc=ot,dc=hr> with scope
    subtree
    # filter: uniqueID=38515000400
    # requesting: 1.1
    #

    # 38515000400, redirecting, sipDirektor, ot.hr <http://ot.hr>
    dn: uniqueID=38515000400,ou=redirecting,ou=sipDirektor,dc=ot,dc=hr


    # search result
    search: 2
    result: 0 Success

    # numResponses: 2
    # numEntries: 1

    real    0m0.007s
    user    0m0.000s
    sys     0m0.000s



    this is my DB_CONFIG

    # $OpenLDAP: pkg/ldap/servers/slapd/DB_CONFIG,v 1.3.2.4
    2007/12/18 11:53:27 ghenry Exp $
    # Example DB_CONFIG file for use with slapd(8) BDB/HDB databases.
    #
    # See the Oracle Berkeley DB documentation
    #
    <http://www.oracle.com/technology/documentation/berkeley-db/db/ref/env/db_config.html>
    # for detail description of DB_CONFIG syntax and semantics.
    #
    # Hints can also be found in the OpenLDAP Software FAQ
    # <http://www.openldap.org/faq/index.cgi?file=2>
    # in particular:
    # <http://www.openldap.org/faq/index.cgi?file=1075>

    # Note: most DB_CONFIG settings will take effect only upon
    rebuilding
    # the DB environment.

    # one 0.25 GB cache
    # set_cachesize 0 268435456 1
    # one 768  MB cache
    #set_cachesize 0 805306368 1
    # one 1GB cache
    set_cachesize 0 1073741824 1
    # Data Directory
    #set_data_dir db

    # Transaction Log settings
    set_lg_regionmax 262144
    set_lg_bsize 2097152
    #set_lg_dir logs

    # Note: special DB_CONFIG flags are no longer needed for "quick"
    # slapadd(8) or slapindex(8) access (see their -q option).



    and this is my slapd.conf


    ~$ cat /etc/ldap/slapd.conf

    include         /usr/local/etc/openldap/schema/core.schema
    include         /usr/local/etc/openldap/schema/sipdirector.schema

    pidfile         /usr/local/var/run/slapd.pid
    argsfile        /usr/local/var/run/slapd.args

    loglevel        0

    access to dn.base="" by * read

    access to *
             by dn="cn=admin,dc=ot,dc=hr" write
             by * read


    threads 8
    #######################################################################
    # BDB database definitions
    #######################################################################
    backend         bdb
    database        bdb

    cachesize 600000
    idlcachesize 1800000
    cachefree 1000

    suffix "dc=ot,dc=hr"
    rootdn "cn=admin,dc=ot,dc=hr"
    // rootpw          ****   :)
    directory       /usr/local/var/openldap-data

    index           objectClass     eq
    index           uniqueID eq





    On Mon, Jun 15, 2009 at 3:01 PM, jakjr <joao.alfredo@gmail.com
    <mailto:joao.alfredo@gmail.com>> wrote:

        Are the attrs list returned in both entries euqal ?

        Try to use the 1.1 attrs list.

        man ldapsearch:
        If no attrs are listed, all user attributes are returned.
        If only 1.1 is listed, no attributes will be returned.

        Regards


        On Mon, Jun 15, 2009 at 8:29 AM, Tihomir Culjaga
        <tculjaga@gmail.com <mailto:tculjaga@gmail.com>> wrote:

            Hello,

            maybe it is a dummy question but i'd like to know why i
            have so big discrepancy in execution between two
            apparently identical ldapsearch ?

            The 1st search takes 94 ms while the 2nd one only 7 ms.
            It doesn't matter how many times i execute the 1st
            search (meaning everytihng should be already cached) ..
            it is always the same.

            Does anyone know why?




            ~$ time ldapsearch  -h localhost -x -b
            ou=redirecting,ou=Dir,dc=ot,dc=hr  -D
            cn=admin,dc=ot,dc=hr -w **** uniqueID=38512303736

            # extended LDIF
            #
            # LDAPv3
            # base <ou=redirecting,ou=Dir,dc=ot,dc=hr> with scope
            subtree
            # filter: uniqueID=38512303736
            # requesting: ALL
            #

            # 38512303736, redirecting, Dir, ot.hr <http://ot.hr>
            dn: uniqueID=38512303736,ou=redirecting,ou=Dir,dc=ot,dc=hr
            objectClass: top
            objectClass: uniqueID
            Prefix: 68A10
            uniqueID: 38512303736

            # search result
            search: 2
            result: 0 Success

            # numResponses: 2
            # numEntries: 1

            *real    0m0.094s*
            user    0m0.004s
            sys     0m0.000s



            ~$ time ldapsearch  -h localhost -x -b
            ou=redirecting,ou=Dir,dc=ot,dc=hr  -D
            cn=admin,dc=ot,dc=hr -w **** uniqueID=38515000400

            # extended LDIF
            #
            # LDAPv3
            # base <ou=redirecting,ou=Direktor,dc=ot,dc=hr> with
            scope subtree
            # filter: uniqueID=38515000400
            # requesting: ALL
            #

            # 38515000400, redirecting, Dir, ot.hr <http://ot.hr>
            dn: uniqueID=38515000400,ou=redirecting,ou=Dir,dc=ot,dc=hr
            objectClass: top
            objectClass: uniqueID
            Prefix: 68B99
            uniqueID: 38515000400

            # search result
            search: 2
            result: 0 Success

            # numResponses: 2
            # numEntries: 1

            *real    0m0.007s*
            user    0m0.000s
            sys     0m0.004s
            tculjaga@l01sipindir2:~$