Hi. Thanks for sharing your ideas and experience!
On Mon, 12 Oct 2009, Emmanuel Lecharny wrote:
Two ideas :
- Check the code that send requests to the prod server. It might receive
many more requests than expected
Why? I don't know how to check the code and already asked the web dev to do so without clue. I guess ldap commandline output can exclude potential issue from code writer. By the way the same code run on prod and dev server.
- Check the network latency. This is most certainly what kills your
performances : firewalls, load balancers, etc are adding some latency you don't see on your dev server because you are not using them on dev.
Couldn't be. As a new prod, no firewall nor load balancing thing running. I just double checked iptables kernel module is not loaded.
This information might be helpful. I can manaully track down the slow-down happens because of two pauses.
the small pause (around 0.1 second) is marked here:
# extended LDIF # # LDAPv3 # base <ou=contacts,ou=realss.com,dc=eoa,dc=cn> with scope # subtree # filter: (uidNumber=7) # requesting: dn # -> 0.1s PAUSE # zhangweiwu, contacts, realss.com, eoa.cn dn: uid=zhangweiwu,ou=contacts,ou=realss.com,dc=eoa,dc=cn
# search result search: 2 result: 0 Success
# numResponses: 2 # numEntries: 1
the big pause (around 0.3 second) is marked here:
-0.3s PAUSE # extended LDIF # # LDAPv3 # base <ou=contacts,ou=realss.com,dc=eoa,dc=cn> with scope # subtree # filter: (uidNumber=7) # requesting: dn #
# zhangweiwu, contacts, realss.com, eoa.cn dn: uid=zhangweiwu,ou=contacts,ou=realss.com,dc=eoa,dc=cn
# search result search: 2 result: 0 Success
# numResponses: 2 # numEntries: 1
To get these pauses I had to run ldapsearch(1) multiple times and try capture the pausing with my staring eyes!
Since there is a pause before each ldapsearch command I first think of speed of hostname resolve and time needed to establish a socket. I tested domain name resolve using code snipplet offered by http://paulschreiber.com/blog/2005/10/28/simple-gethostbyname-example/ and found prod is not slow at resolving its own name as given by hostname(1) or 'localhost', excluding resolving issue. I do not know how to test time needed to establish tcp/ip socket connection but a manual 'telnet localhost ldap' does not feel slow.
This is really frustrating...