openldap-bugs

openldap-bugs@openldap.org

1 participants
20961 discussions

(ITS#7651) LMDB: Uncontrolled database when opened from multiple processes
by dimitrij.denissenko＠blacksquaremedia.com 30 Jul '13

30 Jul '13

Full_Name: Dimitrij Denissenko Version: OS: Ubuntu 12.04 URL: Submission from: (NULL) (62.30.100.0) Hi, I found an interesting issue with LMDB. I have populated the DB with a bunch of records and it uses ~30M on disk (after sync). Then I added a background process to my app and populated the database again with the same record set. Surprisingly. the resulting size on disk was >70M. The background process is forked periodically to perform some maintenance tasks, here is my (simplified) code: /* Close env before forking */ mdb_env_close(env); if ((childpid = fork()) == 0) { /* Child */ rc = mdb_env_open(env, ".", MDB_NOSYNC, 0644); ... } else { /* Parent */ rc = mdb_env_open(env, ".", MDB_NOSYNC, 0644); ... } I could narrow it down to the mdb_env_open call in the child. If I add exit(0) before the mdb_env_open line, the DB size remains consistently at ~30M. The data size seems to grow proportionally to the number of forks performed during data load. What could be causing the growth? What can I do to prevent it? Thanks in advance PS: I tried it with MDB_FIXMAP and without, same result.

1 0

Re: (ITS#7631) Overlay to rewrite value of attribute using regex
by bart＠icm.edu.pl 29 Jul '13

29 Jul '13

On Mon, 29 Jul 2013 08:30:23 -0700, Howard Chu wrote: > > I presume we still need a formal IPR notice in the ITS? Is it enough: The attached patch file is derived from OpenLDAP Software. All of the modifications to OpenLDAP Software represented in the following patch were developed by Dominik Bartkiewicz bart(a)icm.edu.pl. I have not assigned rights and/or interest in this work to any party. DB

1 0

Re: (ITS#6856) OpenLDAP 2.4.23 / db 4.7.25 lockup 100% CPU
by hyc＠symas.com 29 Jul '13

29 Jul '13

requate(a)univention.de wrote: > Looks like the patch for ITS#7222 fixed this. > Thanks for the followup. Closing this ITS. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#6856) OpenLDAP 2.4.23 / db 4.7.25 lockup 100% CPU
by requate＠univention.de 29 Jul '13

29 Jul '13

Looks like the patch for ITS#7222 fixed this. -- Arvid Requate Open Source Software Engineer Univention GmbH be open. Mary-Somerville-Str.1 28359 Bremen Tel. : +49 421 22232-52 Fax : +49 421 22232-99 requate(a)univention.de http://www.univention.de Geschäftsführer: Peter H. Ganten HRB 20755 Amtsgericht Bremen Steuer-Nr.: 71-597-02876

1 0

Re: (ITS#7631) Overlay to rewrite value of attribute using regex
by hyc＠symas.com 29 Jul '13

29 Jul '13

bart(a)icm.edu.pl wrote: > --=_c864110a076fc513331e0370bcb9adbe > Content-Transfer-Encoding: 7bit > Content-Type: text/plain; charset=UTF-8; > format=flowed > > > I put patch here because the ftp server is full. > DB Kurt, the source file in the patch contains this header: @@ -0,0 +1,467 @@ +/* rewatt.c - Overlay to rewrite some attribute */ +/* Redistribution and use in source and binary forms, with or without + * modification, are permitted only as authorized by the OpenLDAP + * Public License. + * + * A copy of this license is available in the file LICENSE in the + * top-level directory of the distribution or, alternatively, at + * <http://www.OpenLDAP.org/license.html>. + */ +/* ACKNOWLEDGEMENTS: + * This work was originally developed by the Dominik Bartkiewicz bart(a)icm.edu.pl for + * inclusion in OpenLDAP Software. + */ I presume we still need a formal IPR notice in the ITS? -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7642) make test for AIX fails on test060-mt-hot
by fumiyas＠osstech.jp 29 Jul '13

29 Jul '13

At Sat, 27 Jul 2013 16:59:07 GMT, mamfelt(a)gmail.com wrote: > 1) also did .35 version, same message > 2) these, except for filesize, are the defaults for at least the last 10 > years: > time(seconds) unlimited > file(blocks) unlimited > data(kbytes) 131072 > stack(kbytes) 32768 > memory(kbytes) 32768 > coredump(blocks) 2097151 > nofiles(descriptors) 2000 Please do `export LDR_CNTRL=MAXDATA=0x20000000` and retry test060. > Now, it is perhaps possible that the data/memory/ and/or stack size is too > small. I shall check on the test system to see if the defaults are the same > as above, or if all have been reset. > I'll even look for a coredump to see if it says anything more about what > has happened. -- -- Name: SATOH Fumiyasu @ OSS Technology Corp. (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- GitHub Home: https://GitHub.com/fumiyas/ -- PGP Fingerprint: BBE1 A1C9 525A 292E 6729 CDEC ADC2 9DCA 5E1C CBCA

1 0

Re: (ITS#7642) make test for AIX fails on test060-mt-hot
by mamfelt＠gmail.com 28 Jul '13

28 Jul '13

--001a11c37b0a23a7cd04e2943a0d Content-Type: text/plain; charset=ISO-8859-1 I tried recompiling the Berkeley db (6.0.20) with -D_LARGE_THREADS, no difference. I modified the test to use THR=50 and INNER ...*4 so it looks like this: ===== # Perform a basic multi-threaded search on a single connection THR=100 THR=50 OUTER=5 INNER=`expr $TESTLOOPS \* 4` echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..." echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ -e "$MONITORDN" \ -m $THR -L $OUTER -l $INNER $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ -e "$MONITORDN" -f "(objectclass=*)" \ -m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1 RC=$? if test $RC != 0 ; then echo "slapd-mtread failed ($RC)!" test $KILLSERVERS != no && kill -HUP $KILLPIDS exit $RC fi ======= and the tests complete. Not sure how arbitrary the THR=100 is, but the 50k transactions is maintained (50 * 5 * 50 * 4) versus (100 * 5 * 50 * 2) ==== In short, using THR=50, and doubled INNER factor test060 succeeds. Now - dies this break the test, or just show a different issue? Michael On Sun, Jul 28, 2013 at 3:57 PM, Michael Felt <mamfelt(a)gmail.com> wrote: > After the new configure with -D_LARGE_THREADS still same status: > root@x094:[/data/prj/openldap-2.4.35/tests]./run test060 > Cleaning up test run directory leftover from previous run. > Running ./scripts/test060-mt-hot for bdb... > > running defines.sh > Running slapadd to build slapd database... > Running slapindex to index slapd database... > Starting slapd on TCP/IP port 9011... > /data/prj/openldap-2.4.35/tests/../servers/slapd/slapd -s0 -f > /data/prj/openldap-2.4.35/tests/testrun/slapd.1.conf -h > ldap://localhost:9011/ -d stats > Testing basic monitor search... > Monitor searches > Testing basic mt-hot search: 1 threads (1 x 50000) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000 > Testing basic mt-hot search: 5 threads (1 x 10000) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000 > Testing basic mt-hot search: 100 threads (5 x 100) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 100 -L 5 -l 100 > slapd-mtread failed (1)! > root@x094:[/data/prj/openldap-2.4.35/tests]echo $? > 1 > > > On Sun, Jul 28, 2013 at 2:23 PM, Michael Felt <mamfelt(a)gmail.com> wrote: > >> Note: tried nofiles=5000 and nofiles=10000 - no change. >> >> I am beginning to think it could be related to this: >> >> *Limits and Default Values* >> The threads library has some implementation-dependent limits and default >> values. These limits and default values can be retrieved by symbolic >> constants to enhance the portability of programs: >> >> - The maximum number of threads per process is 512. The maximum >> number of threads can be retrieved at compilation time using the * >> PTHREAD_THREADS_MAX* symbolic constant defined in the *pthread.h*header file. If an application is compiled with the >> *-D_LARGE_THREADS* flag, the maximum number of threads per process is >> 32767. >> >> === >> >> Made this small change to the test: >> >> # Perform a basic multi-threaded search on a single connection >> echo Perform a basic multi-threaded search on a single connection >> THR=100 >> OUTER=5 >> INNER=`expr $TESTLOOPS \* 2` >> for THR in 60 62 64 65 66 67 68 69 70 80 90 100 >> do >> echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) >> loops..." >> echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ >> -e "$MONITORDN" \ >> -m $THR -L $OUTER -l $INNER >> $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ >> -e "$MONITORDN" -f "(objectclass=*)" \ >> -m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1 >> RC=$? >> if test $RC != 0 ; then >> echo "slapd-mtread failed ($RC)!" >> test $KILLSERVERS != no && kill -HUP $KILLPIDS >> exit $RC >> fi >> done >> >> Got this as output: >> >> Testing basic monitor search... >> Monitor searches >> Testing basic mt-hot search: 1 threads (1 x 50000) loops... >> ./progs/slapd-mtread -H ldap://localhost:9011/ -D >> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000 >> Testing basic mt-hot search: 5 threads (1 x 10000) loops... >> ./progs/slapd-mtread -H ldap://localhost:9011/ -D >> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000 >> Perform a basic multi-threaded search on a single connection >> Testing basic mt-hot search: 60 threads (5 x 100) loops... >> ./progs/slapd-mtread -H ldap://localhost:9011/ -D >> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 60 -L 5 -l 100 >> Testing basic mt-hot search: 62 threads (5 x 100) loops... >> ./progs/slapd-mtread -H ldap://localhost:9011/ -D >> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 62 -L 5 -l 100 >> Testing basic mt-hot search: 64 threads (5 x 100) loops... >> ./progs/slapd-mtread -H ldap://localhost:9011/ -D >> cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 64 -L 5 -l 100 >> slapd-mtread failed (1)! >> root@x054:[/data/prj/openldap-2.4.35/tests] >> >> ==== >> >> slapd-mtread PID=6750246: MT Test Start: conns: 1 (ldap://localhost:9011/) >> slapd-mtread PID=6750246: Threads: RO: 64 RW: 0 >> slapd-mtread PID=6750246: RO thread 0 pass=500 fail=0 >> slapd-mtread PID=6750246: RO thread 1 pass=500 fail=0 >> slapd-mtread PID=6750246: RO thread 2 pass=500 fail=0 >> >> ... >> >> slapd-mtread PID=6750246: RO thread 59 pass=500 fail=0 >> slapd-mtread PID=6750246: RO thread 60 pass=500 fail=0 >> slapd-mtread PID=6750246: RO thread 61 pass=500 fail=0 >> slapd-mtread PID=6750246: RO thread 62 pass=500 fail=0 >> slapd-mtread PID=6750246: RO thread 63 pass=0 fail=0 >> slapd-mtread PID=6750246: FAIL RO thread 63 >> slapd-mtread PID=6750246: MT Test complete >> >> === >> >> Back to original test, mtread.out tail is: >> >> slapd-mtread PID=5242956: RO thread 60 pass=500 fail=0 >> slapd-mtread PID=5242956: RO thread 61 pass=500 fail=0 >> slapd-mtread PID=5242956: RO thread 62 pass=500 fail=0 >> slapd-mtread PID=5242956: RO thread 63 pass=0 fail=0 >> slapd-mtread PID=5242956: FAIL RO thread 63 >> slapd-mtread PID=5242956: RO thread 64 pass=0 fail=0 >> slapd-mtread PID=5242956: FAIL RO thread 64 >> slapd-mtread PID=5242956: RO thread 65 pass=0 fail=0 >> .... >> >> slapd-mtread PID=5242956: FAIL RO thread 97 >> slapd-mtread PID=5242956: RO thread 98 pass=0 fail=0 >> slapd-mtread PID=5242956: FAIL RO thread 98 >> slapd-mtread PID=5242956: RO thread 99 pass=0 fail=0 >> slapd-mtread PID=5242956: FAIL RO thread 99 >> slapd-mtread PID=5242956: MT Test complete >> >> === >> >> Shall test after new build with CFLAGS=-D_LARGE_THREADS >> >> >> On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <hyc(a)symas.com> wrote: >> >>> Michael Felt wrote: >>> >>>> The log file did not show any errors. >>>> >>> >>> If there's no server side errors then I suppose it's a problem in >>> mtread. Check the testrun/mtread.out file. >>> >>> >>> >>> -- >>> -- Howard Chu >>> CTO, Symas Corp. http://www.symas.com >>> Director, Highland Sun http://highlandsun.com/hyc/ >>> Chief Architect, OpenLDAP http://www.openldap.org/**project/<http://www.openldap.org/project/> >>> >> >> > --001a11c37b0a23a7cd04e2943a0d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable I tried recompiling the Berkeley db (6.0.20) with -D_LARGE_THREADS, no diff= erence. I modified the test to use THR=3D50 and INNER ...*4 so it lo= oks like this: =3D=3D=3D=3D=3D # Perform a basic multi-threaded searc= h on a single connection THR=3D100 THR=3D50 OUTER=3D5 INNER=3D`expr $TESTLOOPS \* 4` ech= o "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..= ." echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD = \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" \ =A0=A0=A0=A0=A0=A0=A0 = -m $THR -L $OUTER -l $INNER $SLAPDMTREAD -H $URI1 -D "$MANAGERDN&qu= ot; -w $PASSWD \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" -f &quo= t;(objectclass=3D*)" \ =A0=A0=A0=A0=A0=A0=A0 -m $THR -L $OUTER -l $= INNER >> $MTREADOUT 2>&1 RC=3D$? if test $RC !=3D 0 ; then =A0=A0=A0=A0=A0=A0=A0 echo "sl= apd-mtread failed ($RC)!" =A0=A0=A0=A0=A0=A0=A0 test $KILLSERVERS != =3D no && kill -HUP $KILLPIDS =A0=A0=A0=A0=A0=A0=A0 exit $RC = fi =3D=3D=3D=3D=3D=3D=3D and the tests complete. Not sure how arbitra= ry the THR=3D100 is, but the 50k transactions is maintained (50 * 5 * 50 * 4) versus (100 * 5 * 50 * 2) =3D=3D=3D=3D In short, us= ing THR=3D50, and doubled INNER factor test060 succeeds. Now - dies = this break the test, or just show a different issue? Michael <div= class=3D"gmail_quote"> On Sun, Jul 28, 2013 at 3:57 PM, Michael Felt <<a href= =3D"mailto:mamfelt@gmail.com" target=3D"_blank">mamfelt(a)gmail.com</a>></= span> wrote: <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8e= x;border-left:1px #ccc solid;padding-left:1ex"> After the new configure with -D_LARGE_THREADS still same status: root@x094:[/data/prj/openldap-2.= 4.35/tests]./run test060 Cleaning up test run directory leftover from pr= evious run. Running ./scripts/test060-mt-hot for bdb...<div class=3D"im"> running de= fines.sh Running slapadd to build slapd database... Running slapindex= to index slapd database... Starting slapd on TCP/IP port 9011... /data/prj/openldap-2.4.35/tests/../servers/slapd/slapd -s0 -f /data/prj/ope= nldap-2.4.35/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d stats<= br></div><div class=3D"im"> Testing basic monitor search... Monitor searches Testing basic mt-hot= search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread -H ldap://l= ocalhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMoni= tor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slap= d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w= secret -e cn=3DMonitor -m 5 -L 1 -l 10000 </div>Testing basic mt-hot se= arch: 100 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample= ,dc=3Dcom -w secret -e cn=3DMonitor -m 100 -L 5 -l 100 slapd-mtread fail= ed (1)! root@x094:[/data/prj/openldap-2.4.35/tests]echo $? 1 </spa= n><div class=3D"HOEnZb"> <div class=3D"h5"> <div class=3D"gmail_quote">On Sun, Jul 28, 2013 at 2:23 PM, Michael Fel= t <<a href=3D"mailto:mamfelt@gmail.com" target=3D"_bla= nk">mamfelt(a)gmail.com</a>> wrote: <blockquote class=3D"gmail_q= uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e= x"> Note: tried nofiles=3D5000 and nofiles=3D10000 - no change. I am beg= inning to think it could be related to this: Limits and Default Va= lues The threads library has some implementation-dependent limits and default values. These limits and default values can be retrieved by symbolic constants to enhance the portability of programs: <ul><li>The maxi= mum number of threads per process is 512. The maximum number of threads can be retrieved at compilation time using the PTHREAD= _THREADS_MAX symbolic constant defined in the pthread.h header file. If an application is compiled with the -D_LARGE_THREADS flag, the maximum number of threads per process is 32767.</li></ul>=3D=3D=3DMade this smal= l change to the test:# Perform = a basic multi-threaded search on a single connection echo Perform a basi= c multi-threaded search on a single connection THR=3D100 OUTER=3D5 INNER=3D`expr $TESTLOOPS \* 2` for THR in 60 6= 2 64 65 66 67 68 69 70 80 90 100 do echo "Testing basic mt-hot s= earch: $THR threads ($OUTER x $INNER) loops..." echo $SLAPDMTREAD -= H $URI1 -D "$MANAGERDN" -w $PASSWD \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" \ =A0=A0=A0=A0=A0=A0=A0 = -m $THR -L $OUTER -l $INNER $SLAPDMTREAD -H $URI1 -D "$MANAGERDN&qu= ot; -w $PASSWD \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" -f &quo= t;(objectclass=3D*)" \ =A0=A0=A0=A0=A0=A0=A0 -m $THR -L $OUTER -l $= INNER >> $MTREADOUT 2>&1 RC=3D$? if test $RC !=3D 0 ; then =A0=A0=A0=A0=A0=A0=A0 echo "sl= apd-mtread failed ($RC)!" =A0=A0=A0=A0=A0=A0=A0 test $KILLSERVERS != =3D no && kill -HUP $KILLPIDS =A0=A0=A0=A0=A0=A0=A0 exit $RC = fi done Got this as output: <div>Testing basic monitor search... Monitor searches Testing = basic mt-hot search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread= -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret = -e cn=3DMonitor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slap= d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w= secret -e cn=3DMonitor -m 5 -L 1 -l 10000 </div>Perform a basic multi-t= hreaded search on a single connection Testing basic mt-hot search: 60 threads (5 x 100) loops... ./progs/slapd= -mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w = secret -e cn=3DMonitor -m 60 -L 5 -l 100 Testing basic mt-hot search: 62= threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample= ,dc=3Dcom -w secret -e cn=3DMonitor -m 62 -L 5 -l 100 Testing basic mt-h= ot search: 64 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://= localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMon= itor -m 64 -L 5 -l 100 slapd-mtread failed (1)! root@x054:[/data/prj/openldap-2.4.35/tests] = =3D=3D=3D=3Dslapd-mtread PID=3D6750246: MT Test Start: con= ns: 1 (ldap://localhost:9011/) slapd-mtread PID=3D6750246: Threads: RO: = 64 RW: 0 slapd-mtread PID=3D6750246: RO thread 0 pass=3D500 fail=3D0 slapd-mtread= PID=3D6750246: RO thread 1 pass=3D500 fail=3D0 slapd-mtread PID=3D67502= 46: RO thread 2 pass=3D500 fail=3D0 ...slapd-mtread PID=3D= 6750246: RO thread 59 pass=3D500 fail=3D0 slapd-mtread PID=3D6750246: RO thread 60 pass=3D500 fail=3D0 slapd-mtrea= d PID=3D6750246: RO thread 61 pass=3D500 fail=3D0 slapd-mtread PID=3D675= 0246: RO thread 62 pass=3D500 fail=3D0 slapd-mtread PID=3D6750246: RO th= read 63 pass=3D0 fail=3D0 slapd-mtread PID=3D6750246: FAIL RO thread 63 slapd-mtread PID=3D6750246= : MT Test complete =3D=3D=3DBack to original test, mtread.= out tail is:slapd-mtread PID=3D5242956: RO thread 60 pass=3D500 fail= =3D0 slapd-mtread PID=3D5242956: RO thread 61 pass=3D500 fail=3D0 slapd-mtread PID=3D5242956: RO thread 62 pass=3D500 fail=3D0 slapd-mtrea= d PID=3D5242956: RO thread 63 pass=3D0 fail=3D0 slapd-mtread PID=3D52429= 56: FAIL RO thread 63 slapd-mtread PID=3D5242956: RO thread 64 pass=3D0 = fail=3D0 slapd-mtread PID=3D5242956: FAIL RO thread 64 slapd-mtread PID=3D5242956: RO thread 65 pass=3D0 fail=3D0 ....sl= apd-mtread PID=3D5242956: FAIL RO thread 97 slapd-mtread PID=3D5242956: = RO thread 98 pass=3D0 fail=3D0 slapd-mtread PID=3D5242956: FAIL RO threa= d 98 slapd-mtread PID=3D5242956: RO thread 99 pass=3D0 fail=3D0 slapd-mtread PID=3D5242956: FAIL RO thread 99 slapd-mtread PID=3D5242956= : MT Test complete =3D=3D=3DShall test after new build wit= h CFLAGS=3D-D_LARGE_THREADS <div><div> <div class=3D"gmail_qu= ote"> On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <<a href= =3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>> wr= ote: <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div>Michael Felt wrote: <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> The log file did not show any errors. </blockquote> </div> If there's no server side errors then I suppose it's a problem in m= tread. Check the testrun/mtread.out file. -- <div><div> =A0 -- Howard Chu =A0 CTO, Symas Corp. =A0 =A0 =A0 =A0 =A0 <a href=3D"http://www.symas.com" t= arget=3D"_blank">http://www.symas.com</a> =A0 Director, Highland Sun =A0 =A0 <a href=3D"http://highlandsun.com/hyc/" = target=3D"_blank">http://highlandsun.com/hyc/</a> =A0 Chief Architect, OpenLDAP =A0<a href=3D"http://www.openldap.org/project= /" target=3D"_blank">http://www.openldap.org/project/</a> </div></div></blockquote></div> </div></div></blockquote></div> </div></div></blockquote></div> --001a11c37b0a23a7cd04e2943a0d--

1 0

Re: (ITS#7642) make test for AIX fails on test060-mt-hot
by hyc＠symas.com 28 Jul '13

28 Jul '13

Michael Felt wrote: > Note: tried nofiles=5000 and nofiles=10000 - no change. > > I am beginning to think it could be related to this: > > *Limits and Default Values* Nah. If the limit on threads is 512 and this test is only spawning 100 threads, that's clearly not the issue. More likely it's the datasize or memorysize limit, since we allocate 8MB per thread for its stack. Change your memory limit to unlimited. In any case, I wouldn't worry about it if you can't get this test to run; clearly the server has no issues if it got that far. > > The threads library has some implementation-dependent limits and default > values. These limits and default values can be retrieved by symbolic constants > to enhance the portability of programs: > > * The maximum number of threads per process is 512. The maximum number of > threads can be retrieved at compilation time using the > *PTHREAD_THREADS_MAX* symbolic constant defined in the *pthread.h* header > file. If an application is compiled with the *-D_LARGE_THREADS* flag, the > maximum number of threads per process is 32767. > > === > > Made this small change to the test: > > # Perform a basic multi-threaded search on a single connection > echo Perform a basic multi-threaded search on a single connection > THR=100 > OUTER=5 > INNER=`expr $TESTLOOPS \* 2` > for THR in 60 62 64 65 66 67 68 69 70 80 90 100 > do > echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..." > echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ > -e "$MONITORDN" \ > -m $THR -L $OUTER -l $INNER > $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ > -e "$MONITORDN" -f "(objectclass=*)" \ > -m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1 > RC=$? > if test $RC != 0 ; then > echo "slapd-mtread failed ($RC)!" > test $KILLSERVERS != no && kill -HUP $KILLPIDS > exit $RC > fi > done > > Got this as output: > > Testing basic monitor search... > Monitor searches > Testing basic mt-hot search: 1 threads (1 x 50000) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com > -w secret -e cn=Monitor -m 1 -L 1 -l 50000 > Testing basic mt-hot search: 5 threads (1 x 10000) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com > -w secret -e cn=Monitor -m 5 -L 1 -l 10000 > Perform a basic multi-threaded search on a single connection > Testing basic mt-hot search: 60 threads (5 x 100) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com > -w secret -e cn=Monitor -m 60 -L 5 -l 100 > Testing basic mt-hot search: 62 threads (5 x 100) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com > -w secret -e cn=Monitor -m 62 -L 5 -l 100 > Testing basic mt-hot search: 64 threads (5 x 100) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com > -w secret -e cn=Monitor -m 64 -L 5 -l 100 > slapd-mtread failed (1)! > root@x054:[/data/prj/openldap-2.4.35/tests] > > ==== > > slapd-mtread PID=6750246: MT Test Start: conns: 1 (ldap://localhost:9011/) > slapd-mtread PID=6750246: Threads: RO: 64 RW: 0 > slapd-mtread PID=6750246: RO thread 0 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 1 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 2 pass=500 fail=0 > > ... > > slapd-mtread PID=6750246: RO thread 59 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 60 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 61 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 62 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 63 pass=0 fail=0 > slapd-mtread PID=6750246: FAIL RO thread 63 > slapd-mtread PID=6750246: MT Test complete > > === > > Back to original test, mtread.out tail is: > > slapd-mtread PID=5242956: RO thread 60 pass=500 fail=0 > slapd-mtread PID=5242956: RO thread 61 pass=500 fail=0 > slapd-mtread PID=5242956: RO thread 62 pass=500 fail=0 > slapd-mtread PID=5242956: RO thread 63 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 63 > slapd-mtread PID=5242956: RO thread 64 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 64 > slapd-mtread PID=5242956: RO thread 65 pass=0 fail=0 > .... > > slapd-mtread PID=5242956: FAIL RO thread 97 > slapd-mtread PID=5242956: RO thread 98 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 98 > slapd-mtread PID=5242956: RO thread 99 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 99 > slapd-mtread PID=5242956: MT Test complete > > === > > Shall test after new build with CFLAGS=-D_LARGE_THREADS > > > > On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <hyc(a)symas.com > <mailto:hyc@symas.com>> wrote: > > Michael Felt wrote: > > The log file did not show any errors. > > > If there's no server side errors then I suppose it's a problem in mtread. > Check the testrun/mtread.out file. > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/__project/ > <http://www.openldap.org/project/> > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7642) make test for AIX fails on test060-mt-hot
by mamfelt＠gmail.com 28 Jul '13

28 Jul '13

--089e0112c86ef0022b04e292c264 Content-Type: text/plain; charset=ISO-8859-1 After the new configure with -D_LARGE_THREADS still same status: root@x094:[/data/prj/openldap-2.4.35/tests]./run test060 Cleaning up test run directory leftover from previous run. Running ./scripts/test060-mt-hot for bdb... running defines.sh Running slapadd to build slapd database... Running slapindex to index slapd database... Starting slapd on TCP/IP port 9011... /data/prj/openldap-2.4.35/tests/../servers/slapd/slapd -s0 -f /data/prj/openldap-2.4.35/tests/testrun/slapd.1.conf -h ldap://localhost:9011/ -d stats Testing basic monitor search... Monitor searches Testing basic mt-hot search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000 Testing basic mt-hot search: 100 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 100 -L 5 -l 100 slapd-mtread failed (1)! root@x094:[/data/prj/openldap-2.4.35/tests]echo $? 1 On Sun, Jul 28, 2013 at 2:23 PM, Michael Felt <mamfelt(a)gmail.com> wrote: > Note: tried nofiles=5000 and nofiles=10000 - no change. > > I am beginning to think it could be related to this: > > *Limits and Default Values* > The threads library has some implementation-dependent limits and default > values. These limits and default values can be retrieved by symbolic > constants to enhance the portability of programs: > > - The maximum number of threads per process is 512. The maximum number > of threads can be retrieved at compilation time using the * > PTHREAD_THREADS_MAX* symbolic constant defined in the *pthread.h*header file. If an application is compiled with the > *-D_LARGE_THREADS* flag, the maximum number of threads per process is > 32767. > > === > > Made this small change to the test: > > # Perform a basic multi-threaded search on a single connection > echo Perform a basic multi-threaded search on a single connection > THR=100 > OUTER=5 > INNER=`expr $TESTLOOPS \* 2` > for THR in 60 62 64 65 66 67 68 69 70 80 90 100 > do > echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..." > echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ > -e "$MONITORDN" \ > -m $THR -L $OUTER -l $INNER > $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ > -e "$MONITORDN" -f "(objectclass=*)" \ > -m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1 > RC=$? > if test $RC != 0 ; then > echo "slapd-mtread failed ($RC)!" > test $KILLSERVERS != no && kill -HUP $KILLPIDS > exit $RC > fi > done > > Got this as output: > > Testing basic monitor search... > Monitor searches > Testing basic mt-hot search: 1 threads (1 x 50000) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000 > Testing basic mt-hot search: 5 threads (1 x 10000) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000 > Perform a basic multi-threaded search on a single connection > Testing basic mt-hot search: 60 threads (5 x 100) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 60 -L 5 -l 100 > Testing basic mt-hot search: 62 threads (5 x 100) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 62 -L 5 -l 100 > Testing basic mt-hot search: 64 threads (5 x 100) loops... > ./progs/slapd-mtread -H ldap://localhost:9011/ -D > cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 64 -L 5 -l 100 > slapd-mtread failed (1)! > root@x054:[/data/prj/openldap-2.4.35/tests] > > ==== > > slapd-mtread PID=6750246: MT Test Start: conns: 1 (ldap://localhost:9011/) > slapd-mtread PID=6750246: Threads: RO: 64 RW: 0 > slapd-mtread PID=6750246: RO thread 0 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 1 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 2 pass=500 fail=0 > > ... > > slapd-mtread PID=6750246: RO thread 59 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 60 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 61 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 62 pass=500 fail=0 > slapd-mtread PID=6750246: RO thread 63 pass=0 fail=0 > slapd-mtread PID=6750246: FAIL RO thread 63 > slapd-mtread PID=6750246: MT Test complete > > === > > Back to original test, mtread.out tail is: > > slapd-mtread PID=5242956: RO thread 60 pass=500 fail=0 > slapd-mtread PID=5242956: RO thread 61 pass=500 fail=0 > slapd-mtread PID=5242956: RO thread 62 pass=500 fail=0 > slapd-mtread PID=5242956: RO thread 63 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 63 > slapd-mtread PID=5242956: RO thread 64 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 64 > slapd-mtread PID=5242956: RO thread 65 pass=0 fail=0 > .... > > slapd-mtread PID=5242956: FAIL RO thread 97 > slapd-mtread PID=5242956: RO thread 98 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 98 > slapd-mtread PID=5242956: RO thread 99 pass=0 fail=0 > slapd-mtread PID=5242956: FAIL RO thread 99 > slapd-mtread PID=5242956: MT Test complete > > === > > Shall test after new build with CFLAGS=-D_LARGE_THREADS > > > On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <hyc(a)symas.com> wrote: > >> Michael Felt wrote: >> >>> The log file did not show any errors. >>> >> >> If there's no server side errors then I suppose it's a problem in mtread. >> Check the testrun/mtread.out file. >> >> >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/**project/<http://www.openldap.org/project/> >> > > --089e0112c86ef0022b04e292c264 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable After the new configure with -D_LARGE_THREADS still same status: root@x094:[/data/prj/openldap-2.= 4.35/tests]./run test060 Cleaning up test run directory leftover from pr= evious run. Running ./scripts/test060-mt-hot for bdb... running defines.sh Runnin= g slapadd to build slapd database... Running slapindex to index slapd da= tabase... Starting slapd on TCP/IP port 9011... /data/prj/openldap-2.= 4.35/tests/../servers/slapd/slapd -s0 -f /data/prj/openldap-2.4.35/tests/te= strun/slapd.1.conf -h ldap://localhost:9011/ -d stats Testing basic monitor search... Monitor searches Testing basic mt-hot= search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread -H ldap://l= ocalhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMoni= tor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slap= d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w= secret -e cn=3DMonitor -m 5 -L 1 -l 10000 Testing basic mt-hot search: = 100 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample= ,dc=3Dcom -w secret -e cn=3DMonitor -m 100 -L 5 -l 100 slapd-mtread fail= ed (1)! root@x094:[/data/prj/openldap-2.4.35/tests]echo $? 1 </spa= n> <div class=3D"gmail_quote">On Sun, Jul 28, 2013 at 2:23 PM, Michael Fel= t <<a href=3D"mailto:mamfelt@gmail.com" target=3D"_bla= nk">mamfelt(a)gmail.com</a>> wrote: <blockquote class=3D"gmail_q= uote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1e= x"> Note: tried nofiles=3D5000 and nofiles=3D10000 - no change. I am beg= inning to think it could be related to this: Limits and Default Va= lues The threads library has some implementation-dependent limits and default values. These limits and default values can be retrieved by symbolic constants to enhance the portability of programs: <ul><li>The maxi= mum number of threads per process is 512. The maximum number of threads can be retrieved at compilation time using the PTHREAD= _THREADS_MAX symbolic constant defined in the pthread.h header file. If an application is compiled with the -D_LARGE_THREADS flag, the maximum number of threads per process is 32767.</li></ul>=3D=3D=3DMade this smal= l change to the test:# Perform = a basic multi-threaded search on a single connection echo Perform a basi= c multi-threaded search on a single connection THR=3D100 OUTER=3D5 INNER=3D`expr $TESTLOOPS \* 2` for THR in 60 6= 2 64 65 66 67 68 69 70 80 90 100 do echo "Testing basic mt-hot s= earch: $THR threads ($OUTER x $INNER) loops..." echo $SLAPDMTREAD -= H $URI1 -D "$MANAGERDN" -w $PASSWD \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" \ =A0=A0=A0=A0=A0=A0=A0 = -m $THR -L $OUTER -l $INNER $SLAPDMTREAD -H $URI1 -D "$MANAGERDN&qu= ot; -w $PASSWD \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" -f &quo= t;(objectclass=3D*)" \ =A0=A0=A0=A0=A0=A0=A0 -m $THR -L $OUTER -l $= INNER >> $MTREADOUT 2>&1 RC=3D$? if test $RC !=3D 0 ; then =A0=A0=A0=A0=A0=A0=A0 echo "sl= apd-mtread failed ($RC)!" =A0=A0=A0=A0=A0=A0=A0 test $KILLSERVERS != =3D no && kill -HUP $KILLPIDS =A0=A0=A0=A0=A0=A0=A0 exit $RC = fi done Got this as output: <div class=3D"im">Testing basic monitor search... Monitor searche= s Testing basic mt-hot search: 1 threads (1 x 50000) loops... ./progs= /slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dc= om -w secret -e cn=3DMonitor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slap= d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w= secret -e cn=3DMonitor -m 5 -L 1 -l 10000 </div>Perform a basic multi-t= hreaded search on a single connection Testing basic mt-hot search: 60 threads (5 x 100) loops... ./progs/slapd= -mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w = secret -e cn=3DMonitor -m 60 -L 5 -l 100 Testing basic mt-hot search: 62= threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample= ,dc=3Dcom -w secret -e cn=3DMonitor -m 62 -L 5 -l 100 Testing basic mt-h= ot search: 64 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://= localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMon= itor -m 64 -L 5 -l 100 slapd-mtread failed (1)! root@x054:[/data/prj/openldap-2.4.35/tests] = =3D=3D=3D=3Dslapd-mtread PID=3D6750246: MT Test Start: con= ns: 1 (ldap://localhost:9011/) slapd-mtread PID=3D6750246: Threads: RO: = 64 RW: 0 slapd-mtread PID=3D6750246: RO thread 0 pass=3D500 fail=3D0 slapd-mtread= PID=3D6750246: RO thread 1 pass=3D500 fail=3D0 slapd-mtread PID=3D67502= 46: RO thread 2 pass=3D500 fail=3D0 ...slapd-mtread PID=3D= 6750246: RO thread 59 pass=3D500 fail=3D0 slapd-mtread PID=3D6750246: RO thread 60 pass=3D500 fail=3D0 slapd-mtrea= d PID=3D6750246: RO thread 61 pass=3D500 fail=3D0 slapd-mtread PID=3D675= 0246: RO thread 62 pass=3D500 fail=3D0 slapd-mtread PID=3D6750246: RO th= read 63 pass=3D0 fail=3D0 slapd-mtread PID=3D6750246: FAIL RO thread 63 slapd-mtread PID=3D6750246= : MT Test complete =3D=3D=3DBack to original test, mtread.= out tail is:slapd-mtread PID=3D5242956: RO thread 60 pass=3D500 fail= =3D0 slapd-mtread PID=3D5242956: RO thread 61 pass=3D500 fail=3D0 slapd-mtread PID=3D5242956: RO thread 62 pass=3D500 fail=3D0 slapd-mtrea= d PID=3D5242956: RO thread 63 pass=3D0 fail=3D0 slapd-mtread PID=3D52429= 56: FAIL RO thread 63 slapd-mtread PID=3D5242956: RO thread 64 pass=3D0 = fail=3D0 slapd-mtread PID=3D5242956: FAIL RO thread 64 slapd-mtread PID=3D5242956: RO thread 65 pass=3D0 fail=3D0 ....sl= apd-mtread PID=3D5242956: FAIL RO thread 97 slapd-mtread PID=3D5242956: = RO thread 98 pass=3D0 fail=3D0 slapd-mtread PID=3D5242956: FAIL RO threa= d 98 slapd-mtread PID=3D5242956: RO thread 99 pass=3D0 fail=3D0 slapd-mtread PID=3D5242956: FAIL RO thread 99 slapd-mtread PID=3D5242956= : MT Test complete =3D=3D=3DShall test after new build wit= h CFLAGS=3D-D_LARGE_THREADS <div class=3D"HOEnZb"><div class=3D"h5">= <div class=3D"gmail_quote"> On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <<a href= =3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>> wr= ote: <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div>Michael Felt wrote: <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> The log file did not show any errors. </blockquote> </div> If there's no server side errors then I suppose it's a problem in m= tread. Check the testrun/mtread.out file. -- <div><div> =A0 -- Howard Chu =A0 CTO, Symas Corp. =A0 =A0 =A0 =A0 =A0 <a href=3D"http://www.symas.com" t= arget=3D"_blank">http://www.symas.com</a> =A0 Director, Highland Sun =A0 =A0 <a href=3D"http://highlandsun.com/hyc/" = target=3D"_blank">http://highlandsun.com/hyc/</a> =A0 Chief Architect, OpenLDAP =A0<a href=3D"http://www.openldap.org/project= /" target=3D"_blank">http://www.openldap.org/project/</a> </div></div></blockquote></div> </div></div></blockquote></div> --089e0112c86ef0022b04e292c264--

1 0

Re: (ITS#7642) make test for AIX fails on test060-mt-hot
by mamfelt＠gmail.com 28 Jul '13

28 Jul '13

--001a11334f8a1e9c1f04e29172ba Content-Type: text/plain; charset=ISO-8859-1 Note: tried nofiles=5000 and nofiles=10000 - no change. I am beginning to think it could be related to this: *Limits and Default Values* The threads library has some implementation-dependent limits and default values. These limits and default values can be retrieved by symbolic constants to enhance the portability of programs: - The maximum number of threads per process is 512. The maximum number of threads can be retrieved at compilation time using the * PTHREAD_THREADS_MAX* symbolic constant defined in the *pthread.h* header file. If an application is compiled with the *-D_LARGE_THREADS* flag, the maximum number of threads per process is 32767. === Made this small change to the test: # Perform a basic multi-threaded search on a single connection echo Perform a basic multi-threaded search on a single connection THR=100 OUTER=5 INNER=`expr $TESTLOOPS \* 2` for THR in 60 62 64 65 66 67 68 69 70 80 90 100 do echo "Testing basic mt-hot search: $THR threads ($OUTER x $INNER) loops..." echo $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ -e "$MONITORDN" \ -m $THR -L $OUTER -l $INNER $SLAPDMTREAD -H $URI1 -D "$MANAGERDN" -w $PASSWD \ -e "$MONITORDN" -f "(objectclass=*)" \ -m $THR -L $OUTER -l $INNER >> $MTREADOUT 2>&1 RC=$? if test $RC != 0 ; then echo "slapd-mtread failed ($RC)!" test $KILLSERVERS != no && kill -HUP $KILLPIDS exit $RC fi done Got this as output: Testing basic monitor search... Monitor searches Testing basic mt-hot search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 5 -L 1 -l 10000 Perform a basic multi-threaded search on a single connection Testing basic mt-hot search: 60 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 60 -L 5 -l 100 Testing basic mt-hot search: 62 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 62 -L 5 -l 100 Testing basic mt-hot search: 64 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=Manager,dc=example,dc=com -w secret -e cn=Monitor -m 64 -L 5 -l 100 slapd-mtread failed (1)! root@x054:[/data/prj/openldap-2.4.35/tests] ==== slapd-mtread PID=6750246: MT Test Start: conns: 1 (ldap://localhost:9011/) slapd-mtread PID=6750246: Threads: RO: 64 RW: 0 slapd-mtread PID=6750246: RO thread 0 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 1 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 2 pass=500 fail=0 ... slapd-mtread PID=6750246: RO thread 59 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 60 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 61 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 62 pass=500 fail=0 slapd-mtread PID=6750246: RO thread 63 pass=0 fail=0 slapd-mtread PID=6750246: FAIL RO thread 63 slapd-mtread PID=6750246: MT Test complete === Back to original test, mtread.out tail is: slapd-mtread PID=5242956: RO thread 60 pass=500 fail=0 slapd-mtread PID=5242956: RO thread 61 pass=500 fail=0 slapd-mtread PID=5242956: RO thread 62 pass=500 fail=0 slapd-mtread PID=5242956: RO thread 63 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 63 slapd-mtread PID=5242956: RO thread 64 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 64 slapd-mtread PID=5242956: RO thread 65 pass=0 fail=0 .... slapd-mtread PID=5242956: FAIL RO thread 97 slapd-mtread PID=5242956: RO thread 98 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 98 slapd-mtread PID=5242956: RO thread 99 pass=0 fail=0 slapd-mtread PID=5242956: FAIL RO thread 99 slapd-mtread PID=5242956: MT Test complete === Shall test after new build with CFLAGS=-D_LARGE_THREADS On Sun, Jul 28, 2013 at 1:19 PM, Howard Chu <hyc(a)symas.com> wrote: > Michael Felt wrote: > >> The log file did not show any errors. >> > > If there's no server side errors then I suppose it's a problem in mtread. > Check the testrun/mtread.out file. > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/**project/<http://www.openldap.org/project/> > --001a11334f8a1e9c1f04e29172ba Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Note: tried nofiles=3D5000 and nofiles=3D10000 - no change. I am beg= inning to think it could be related to this: Limits and Default Values The threads library has some implementation-dependent limits and default values. These limits and default values can be retrieved by symbolic constants to enhance the portability of programs: <ul class=3D"ul"= ><li class=3D"li">The maximum number of threads per process is 512. The max= imum number of threads can be retrieved at compilation time using the PTHREAD_THREADS_MAX symbolic constant defined in the pthread.h header fi= le. If an application is compiled with the -D_LARGE_THREADS flag,= the maximum number of threads per process is 32767.</li></ul>=3D=3D=3DMade this smal= l change to the test:# Perform = a basic multi-threaded search on a single connection echo Perform a basi= c multi-threaded search on a single connection THR=3D100 OUTER=3D5 INNER=3D`expr $TESTLOOPS \* 2` for THR in 60 6= 2 64 65 66 67 68 69 70 80 90 100 do echo "Testing basic mt-hot s= earch: $THR threads ($OUTER x $INNER) loops..." echo $SLAPDMTREAD -= H $URI1 -D "$MANAGERDN" -w $PASSWD \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" \ =A0=A0=A0=A0=A0=A0=A0 = -m $THR -L $OUTER -l $INNER $SLAPDMTREAD -H $URI1 -D "$MANAGERDN&qu= ot; -w $PASSWD \ =A0=A0=A0=A0=A0=A0=A0 -e "$MONITORDN" -f &quo= t;(objectclass=3D*)" \ =A0=A0=A0=A0=A0=A0=A0 -m $THR -L $OUTER -l $= INNER >> $MTREADOUT 2>&1 RC=3D$? if test $RC !=3D 0 ; then =A0=A0=A0=A0=A0=A0=A0 echo "sl= apd-mtread failed ($RC)!" =A0=A0=A0=A0=A0=A0=A0 test $KILLSERVERS != =3D no && kill -HUP $KILLPIDS =A0=A0=A0=A0=A0=A0=A0 exit $RC = fi done Got this as output: Testing basic monitor search... Monitor searches Testing basic mt-= hot search: 1 threads (1 x 50000) loops... ./progs/slapd-mtread -H ldap:= //localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DM= onitor -m 1 -L 1 -l 50000 Testing basic mt-hot search: 5 threads (1 x 10000) loops... ./progs/slap= d-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w= secret -e cn=3DMonitor -m 5 -L 1 -l 10000 Perform a basic multi-threade= d search on a single connection Testing basic mt-hot search: 60 threads (5 x 100) loops... ./progs/slapd= -mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w = secret -e cn=3DMonitor -m 60 -L 5 -l 100 Testing basic mt-hot search: 62= threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://localhost:9011/ -D cn=3DManager,dc=3Dexample= ,dc=3Dcom -w secret -e cn=3DMonitor -m 62 -L 5 -l 100 Testing basic mt-h= ot search: 64 threads (5 x 100) loops... ./progs/slapd-mtread -H ldap://= localhost:9011/ -D cn=3DManager,dc=3Dexample,dc=3Dcom -w secret -e cn=3DMon= itor -m 64 -L 5 -l 100 slapd-mtread failed (1)! root@x054:[/data/prj/openldap-2.4.35/tests] = =3D=3D=3D=3Dslapd-mtread PID=3D6750246: MT Test Start: conns:= 1 (ldap://localhost:9011/) slapd-mtread PID=3D6750246: Threads: RO: 64 = RW: 0 slapd-mtread PID=3D6750246: RO thread 0 pass=3D500 fail=3D0 slapd-mtread= PID=3D6750246: RO thread 1 pass=3D500 fail=3D0 slapd-mtread PID=3D67502= 46: RO thread 2 pass=3D500 fail=3D0 ...slapd-mtread PID=3D= 6750246: RO thread 59 pass=3D500 fail=3D0 slapd-mtread PID=3D6750246: RO thread 60 pass=3D500 fail=3D0 slapd-mtrea= d PID=3D6750246: RO thread 61 pass=3D500 fail=3D0 slapd-mtread PID=3D675= 0246: RO thread 62 pass=3D500 fail=3D0 slapd-mtread PID=3D6750246: RO th= read 63 pass=3D0 fail=3D0 slapd-mtread PID=3D6750246: FAIL RO thread 63 slapd-mtread PID=3D6750246= : MT Test complete =3D=3D=3DBack to original test, mtread.= out tail is:slapd-mtread PID=3D5242956: RO thread 60 pass=3D500 fail= =3D0 slapd-mtread PID=3D5242956: RO thread 61 pass=3D500 fail=3D0 slapd-mtread PID=3D5242956: RO thread 62 pass=3D500 fail=3D0 slapd-mtrea= d PID=3D5242956: RO thread 63 pass=3D0 fail=3D0 slapd-mtread PID=3D52429= 56: FAIL RO thread 63 slapd-mtread PID=3D5242956: RO thread 64 pass=3D0 = fail=3D0 slapd-mtread PID=3D5242956: FAIL RO thread 64 slapd-mtread PID=3D5242956: RO thread 65 pass=3D0 fail=3D0 ....sl= apd-mtread PID=3D5242956: FAIL RO thread 97 slapd-mtread PID=3D5242956: = RO thread 98 pass=3D0 fail=3D0 slapd-mtread PID=3D5242956: FAIL RO threa= d 98 slapd-mtread PID=3D5242956: RO thread 99 pass=3D0 fail=3D0 slapd-mtread PID=3D5242956: FAIL RO thread 99 slapd-mtread PID=3D5242956= : MT Test complete =3D=3D=3DShall test after new build wit= h CFLAGS=3D-D_LARGE_THREADS <div class=3D"gmail_quote">On Su= n, Jul 28, 2013 at 1:19 PM, Howard Chu <<a href=3D"mai= lto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>> wrote:<br= > <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"><div class=3D"im">Michael Felt wrote: <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> The log file did not show any errors. </blockquote> </div> If there's no server side errors then I suppose it's a problem in m= tread. Check the testrun/mtread.out file. -- <div class=3D"HOEnZb"><div class=3D"h5"> =A0 -- Howard Chu =A0 CTO, Symas Corp. =A0 =A0 =A0 =A0 =A0 <a href=3D"http://www.symas.com" t= arget=3D"_blank">http://www.symas.com</a> =A0 Director, Highland Sun =A0 =A0 <a href=3D"http://highlandsun.com/hyc/" = target=3D"_blank">http://highlandsun.com/hyc/</a> =A0 Chief Architect, OpenLDAP =A0<a href=3D"http://www.openldap.org/project= /" target=3D"_blank">http://www.openldap.org/project/</a> </div></div></blockquote></div> --001a11334f8a1e9c1f04e29172ba--

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs