--On Saturday, April 06, 2013 1:07 PM -0600 - lidutu lm5@ualberta.ca wrote:

So what is happening on the slapd server while things are paused?

Also, if you use an ldapi:/// socket rather than a tcp/ip socket, do you see the same issue?

A have changed to use the ldapi:/// socket and there is no improvement.

It seems that one thread of slapd is repeatedly waiting on epoll_wait. Strace prints epoll_wait(6,

and then pauses for couple of seconds.

Iotop shows to slapd threads writing to disk. Right after I start ldapadd, they write to disk at approximately 60MB/s each. Over time, they go down to 2-3 MB/s each.

TID    PRIO    USER    DISK READ    DISK WRITE    SWAPIN    IO    COMMAND 15264 be/4 ldap        0.00 B/s    2.81 M/s  0.00 % 99.99 % slapd -u ldap -g ldap -f /etc/openldap/slapd.conf -h ldapi://%2fvar%2frun%2fopenldap%2fslapd.sock 15255 be/4 ldap        0.00 B/s    3.00 M/s  0.00 % 19.48 % slapd -u ldap -g ldap -f /etc/openldap/slapd.conf -h ldapi://%2fvar%2frun%2fopenldap%2fslapd.sock

I noticed something similar on ubuntu10 as well. I upgraded my server to ubuntu12, and I also set /proc/sys/vm/dirty_ratio to 90 and /proc/sys/vm/dirty_expire_centisecs to 60000, and that helped significantly with back-mdb using current OpenLDAP master. An ldapadd of almost 3.5 million users takes approximately 4 hours now for my dataset vs 16. so 4x faster.

--Quanah

--

Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration