--089e0112c86ef0022b04e292c264 Content-Type: text/plain; charset=ISO-8859-1
After the new configure with -D_LARGE_THREADS still same status: root@x094:[/data/prj/openldap-2.4.35/tests]./run test060 Cleaning up test run directory leftover from previous run. Running ./scripts/test060-mt-hot for bdb... running defines.sh Running slapadd to build slapd database... Running slapindex to index slapd database... Starting slapd on TCP/IP port 9011... /data/prj/openldap-2.4.35/tests/../servers/slapd/slapd -s0 -f /data/prj/openldap-2.4.35/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d stats Testing basic monitor search... Monitor searches Testing basic mt-hot search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000 Testing basic mt-hot search: 100 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 100 -L 5 -l 100 slapd-mtread failed (1)! root@x094:[/data/prj/openldap-2.4.35/tests]echo $? 1
On Sun, Jul 28, 2013 at 2:23 PM, Michael Felt mamfelt@gmail.com wrote:
Note: tried nofiles=5000 and nofiles=10000 - no change.
I am beginning to think it could be related to this:
*Limits and Default Values* The threads library has some implementation-dependent limits and default values. These limits and default values can be retrieved by symbolic constants to enhance the portability of programs:
- The maximum number of threads per process is 512. The maximum number
of threads can be retrieved at compilation time using the * PTHREAD_THREADS_MAX* symbolic constant defined in the *pthread.h*header file. If an application is compiled with the *-D_LARGE_THREADS* flag, the maximum number of threads per process is 32767.
===
Made this small change to the test:
# Perform a basic multi-threaded search on a single connection echo Perform a basic multi-threaded search on a single connection THR=100 OUTER=5 INNER=`expr $TESTLOOPS * 2` for THR in 60 62 64 65 66 67 68 69 70 80 90 100 do echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..." echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ -e "$MONITORDN" \ -m $THR -L $OUTER -l $INNER $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ -e "$MONITORDN" -f "(objectclass=*)" \ -m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1 RC=$? if test $RC != 0 ; then echo "slapd-mtread failed ($RC)!" test $KILLSERVERS != no && kill -HUP $KILLPIDS exit $RC fi done
Got this as output:
Testing basic monitor search... Monitor searches Testing basic mt-hot search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000 Perform a basic multi-threaded search on a single connection Testing basic mt-hot search: 60 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 60 -L 5 -l 100 Testing basic mt-hot search: 62 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 62 -L 5 -l 100 Testing basic mt-hot search: 64 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 64 -L 5 -l 100 slapd-mtread failed (1)! root@x054:[/data/prj/openldap-2.4.35/tests]
====
slapd-mtread PID=6750246: MT Test Start: conns: 1 (ldap://localhost:9011/) slapd-mtread PID=6750246: Threads: RO: 64 RW: 0 slapd-mtread PID=6750246: RO thread 0 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 1 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 2 pass=500 fail=0
...
slapd-mtread PID=6750246: RO thread 59 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 60 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 61 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 62 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 63 pass=0 fail=0 slapd-mtread PID=6750246: FAIL RO thread 63 slapd-mtread PID=6750246: MT Test complete
===
Back to original test, mtread.out tail is:
slapd-mtread PID=5242956: RO thread 60 pass=500 fail=0 slapd-mtread PID=5242956: RO thread 61 pass=500 fail=0 slapd-mtread PID=5242956: RO thread 62 pass=500 fail=0 slapd-mtread PID=5242956: RO thread 63 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 63 slapd-mtread PID=5242956: RO thread 64 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 64 slapd-mtread PID=5242956: RO thread 65 pass=0 fail=0 ....
slapd-mtread PID=5242956: FAIL RO thread 97 slapd-mtread PID=5242956: RO thread 98 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 98 slapd-mtread PID=5242956: RO thread 99 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 99 slapd-mtread PID=5242956: MT Test complete
===
Shall test after new build with CFLAGS=-D_LARGE_THREADS
On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu hyc@symas.com wrote:
Michael Felt wrote:
The log file did not show any errors.
If there's no server side errors then I suppose it's a problem in mtread. Check the testrun/mtread.out file.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/**project/http://www.openldap.org/project/
--089e0112c86ef0022b04e292c264 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
After the new configure with -D_LARGE_THREADS still same status:<br><span s= tyle=3D"font-family:courier new,monospace">root@x094:[/data/prj/openldap-2.= 4.35/tests]./run test060<br>Cleaning up test run directory leftover from pr= evious run.<br> Running ./scripts/test060-mt-hot for bdb...<br>running defines.sh<br>Runnin= g slapadd to build slapd database...<br>Running slapindex to index slapd da= tabase...<br>Starting slapd on TCP/IP port 9011...<br>/data/prj/openldap-2.= 4.35/tests/../servers/slapd/slapd -s0 -f /data/prj/openldap-2.4.35/tests/te= strun/slapd.1.conf -h ldap://localhost:9011/ -d stats<br> Testing basic monitor search...<br>Monitor searches<br>Testing basic mt-hot= search: 1 threads (1 x 50000) loops...<br>./progs/slapd-mtread -H ldap://l= ocalhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMoni= tor -m 1 -L 1 -l 50000<br> Testing basic mt-hot search: 5 threads (1 x 10000) loops...<br>./progs/slap= d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w= secret -e cn=3DMonitor -m 5 -L 1 -l 10000<br>Testing basic mt-hot search: = 100 threads (5 x 100) loops...<br> ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample= ,dc=3Dcom -w secret -e cn=3DMonitor -m 100 -L 5 -l 100<br>slapd-mtread fail= ed (1)!<br>root@x094:[/data/prj/openldap-2.4.35/tests]echo $?<br>1<br></spa= n><br> <br><div class=3D"gmail_quote">On Sun, Jul 28, 2013 at 2:23 PM, Michael Fel= t <span dir=3D"ltr"><<a href=3D"mailto:mamfelt@gmail.com" target=3D"_bla= nk">mamfelt@gmail.com</a>></span> wrote:<br><blockquote class=3D"gmail_q= uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e= x"> Note: tried nofiles=3D5000 and nofiles=3D10000 - no change.<br><br>I am beg= inning to think it could be related to this:<br><p><b>Limits and Default Va= lues</b></p> The threads library has some implementation-dependent limits and default values. These limits and default values can be retrieved by symbolic constants to enhance the portability of programs: <ul><li>The maxi= mum number of threads per process is 512. The maximum number of threads can be retrieved at compilation time using the <b>PTHREAD= _THREADS_MAX</b> symbolic constant defined in the <b>pthread.h</b> header file. If an application is compiled with the <b>-D_LARGE_THREADS</b> flag, the maximum number of threads per process is 32767.</li></ul><p>=3D=3D=3D</p><p>Made this smal= l change to the test:</p><p><font face=3D"courier new,monospace"># Perform = a basic multi-threaded search on a single connection<br>echo Perform a basi= c multi-threaded search on a single connection<br>
THR=3D100<br>OUTER=3D5<br>INNER=3D`expr $TESTLOOPS * 2`<br>for THR in 60 6= 2 64 65 66 67 68 69 70 80 90 100<br>do<br>echo "Testing basic mt-hot s= earch: $THR threads ($OUTER x $INNER) loops..."<br>echo $SLAPDMTREAD -= H $URI1 -D "$MANAGERDN" -w $PASSWD <br>
=A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" <br>=A0=A0=A0=A0=A0=A0=A0 = -m $THR -L $OUTER -l $INNER<br>$SLAPDMTREAD -H $URI1 -D "$MANAGERDN&qu= ot; -w $PASSWD <br>=A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" -f &quo= t;(objectclass=3D*)" <br>=A0=A0=A0=A0=A0=A0=A0 -m $THR -L $OUTER -l $= INNER >> $MTREADOUT 2>&1<br>
RC=3D$?<br>if test $RC !=3D 0 ; then<br>=A0=A0=A0=A0=A0=A0=A0 echo "sl= apd-mtread failed ($RC)!"<br>=A0=A0=A0=A0=A0=A0=A0 test $KILLSERVERS != =3D no && kill -HUP $KILLPIDS<br>=A0=A0=A0=A0=A0=A0=A0 exit $RC<br>= fi<br>done<br></font></p><p>Got this as output:</p>
<p></p><div class=3D"im">Testing basic monitor search...<br>Monitor searche= s<br>Testing basic mt-hot search: 1 threads (1 x 50000) loops...<br>./progs= /slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dc= om -w secret -e cn=3DMonitor -m 1 -L 1 -l 50000<br>
Testing basic mt-hot search: 5 threads (1 x 10000) loops...<br>./progs/slap= d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w= secret -e cn=3DMonitor -m 5 -L 1 -l 10000<br></div>Perform a basic multi-t= hreaded search on a single connection<br>
Testing basic mt-hot search: 60 threads (5 x 100) loops...<br>./progs/slapd= -mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w = secret -e cn=3DMonitor -m 60 -L 5 -l 100<br>Testing basic mt-hot search: 62= threads (5 x 100) loops...<br>
./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample= ,dc=3Dcom -w secret -e cn=3DMonitor -m 62 -L 5 -l 100<br>Testing basic mt-h= ot search: 64 threads (5 x 100) loops...<br>./progs/slapd-mtread -H ldap://= localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMon= itor -m 64 -L 5 -l 100<br>
slapd-mtread failed (1)!<br>root@x054:[/data/prj/openldap-2.4.35/tests]<br>= <p></p><p>=3D=3D=3D=3D</p><p>slapd-mtread PID=3D6750246: MT Test Start: con= ns: 1 (ldap://localhost:9011/)<br>slapd-mtread PID=3D6750246: Threads: RO: = 64 RW: 0<br>
slapd-mtread PID=3D6750246: RO thread 0 pass=3D500 fail=3D0<br>slapd-mtread= PID=3D6750246: RO thread 1 pass=3D500 fail=3D0<br>slapd-mtread PID=3D67502= 46: RO thread 2 pass=3D500 fail=3D0<br></p><p>...</p><p>slapd-mtread PID=3D= 6750246: RO thread 59 pass=3D500 fail=3D0<br>
slapd-mtread PID=3D6750246: RO thread 60 pass=3D500 fail=3D0<br>slapd-mtrea= d PID=3D6750246: RO thread 61 pass=3D500 fail=3D0<br>slapd-mtread PID=3D675= 0246: RO thread 62 pass=3D500 fail=3D0<br>slapd-mtread PID=3D6750246: RO th= read 63 pass=3D0 fail=3D0<br>
slapd-mtread PID=3D6750246: FAIL RO thread 63<br>slapd-mtread PID=3D6750246= : MT Test complete<br></p><p>=3D=3D=3D</p><p>Back to original test, mtread.= out tail is:</p><p>slapd-mtread PID=3D5242956: RO thread 60 pass=3D500 fail= =3D0<br>slapd-mtread PID=3D5242956: RO thread 61 pass=3D500 fail=3D0<br>
slapd-mtread PID=3D5242956: RO thread 62 pass=3D500 fail=3D0<br>slapd-mtrea= d PID=3D5242956: RO thread 63 pass=3D0 fail=3D0<br>slapd-mtread PID=3D52429= 56: FAIL RO thread 63<br>slapd-mtread PID=3D5242956: RO thread 64 pass=3D0 = fail=3D0<br>slapd-mtread PID=3D5242956: FAIL RO thread 64<br>
slapd-mtread PID=3D5242956: RO thread 65 pass=3D0 fail=3D0<br>....</p><p>sl= apd-mtread PID=3D5242956: FAIL RO thread 97<br>slapd-mtread PID=3D5242956: = RO thread 98 pass=3D0 fail=3D0<br>slapd-mtread PID=3D5242956: FAIL RO threa= d 98<br>slapd-mtread PID=3D5242956: RO thread 99 pass=3D0 fail=3D0<br>
slapd-mtread PID=3D5242956: FAIL RO thread 99<br>slapd-mtread PID=3D5242956= : MT Test complete<br></p><p>=3D=3D=3D</p><p>Shall test after new build wit= h CFLAGS=3D-D_LARGE_THREADS<br></p><div class=3D"HOEnZb"><div class=3D"h5">= <br><br><div class=3D"gmail_quote"> On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <span dir=3D"ltr"><<a href= =3D"mailto:hyc@symas.com" target=3D"_blank">hyc@symas.com</a>></span> wr= ote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div>Michael Felt wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> The log file did not show any errors.<br> </blockquote> <br></div> If there's no server side errors then I suppose it's a problem in m= tread. Check the testrun/mtread.out file.<span><font color=3D"#888888"><br> <br> <br> <br> -- <br></font></span><div><div> =A0 -- Howard Chu<br> =A0 CTO, Symas Corp. =A0 =A0 =A0 =A0 =A0 <a href=3D"http://www.symas.com" t= arget=3D"_blank">http://www.symas.com</a><br> =A0 Director, Highland Sun =A0 =A0 <a href=3D"http://highlandsun.com/hyc/" = target=3D"_blank">http://highlandsun.com/hyc/</a><br> =A0 Chief Architect, OpenLDAP =A0<a href=3D"http://www.openldap.org/project= /" target=3D"_blank">http://www.openldap.org/<u></u>project/</a><br> </div></div></blockquote></div><br> </div></div></blockquote></div><br>
--089e0112c86ef0022b04e292c264--