On 26/11/2014 18:23, Ulrich Windl wrote: > What you could try is (assuming you have debug symbols in your > binary) attaching gdb to the running slapd to see where it is > spending CPU time. I can't remember the commands precisely, but "bt > (backtrace)" and "info threads" should point you in the right > direction. Note that slapd will likely stop responding while gdb is > attached, so you could also try to force a core dump for offline > examination. I'm not sure I follow here. How would this work? I attach gdb to the running slapd I get, but if it stops how does that help me? I've only had a little bit of experience with gdb... How would I get a core dump, as well? That sounds like it might be more useful.

>> Da Rock <openldap-technical(a)herveybayaustralia.com.au&gt; schrieb am 26.11.2014 um

>> Da Rock <openldap-technical(a)herveybayaustralia.com.au&gt; schrieb am 26.11.2014 um

Hi! "Error accessing memory address 0x7ffff73ed000: Bad address." looks like either corrupted stack, or compiler optimization to me. Maybe you could try to compile with -O0, just for debugging purposes. Also as there a many threads active, you could try for each thread: "thread n" where n is the thread ID of gdb "bt"

Regards, Ulrich >>> Da Rock <openldap-technical(a)herveybayaustralia.com.au&gt; schrieb am 29.11.2014 um 13:49 in Nachricht <5479C0C5.4040506(a)herveybayaustralia.com.au&gt;: > On 27/11/2014 01:39, Ulrich Windl wrote: >> If gdb is attached to the process ("attach <PID>"), you can examine the CPU > registers and the memory. Specifically the call stack. If a specific routine > uses all the CPU, it's very likely that gdb will find that active routine and > where it was called from. Together with local variables (info locals), a > developer could find out what's wrong. Maybe you'll have to combine "-g" with > "-O0" for more reliable results. >> You'll get a coredump by something like kill -BUS <pid> (check ulimit -c also > before starting the process), and you can detach gdb from the process after > inspection. > Apologies for the delay - had some technical difficulties to take care of. > > So here's what I have so far: > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"...(no debugging > symbols found)... > Attaching to program: /usr/local/libexec/slapd, process 1326 > Reading symbols from /usr/local/lib/libldap_r-2.4.so.2...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libldap_r-2.4.so.2 > Reading symbols from /usr/local/lib/liblber-2.4.so.2...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/liblber-2.4.so.2 > Reading symbols from /usr/local/lib/libltdl.so.7...(no debugging symbols > found)...done. > Loaded symbols for /usr/local/lib/libltdl.so.7 > Reading symbols from /usr/local/lib/libdb-5.3.so.0...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libdb-5.3.so.0 > Reading symbols from /usr/local/lib/libsasl2.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libsasl2.so.3 > Reading symbols from /lib/libcrypt.so.5...(no debugging symbols > found)...done. > Loaded symbols for /lib/libcrypt.so.5 > Reading symbols from /usr/lib/libwrap.so.6...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libwrap.so.6 > Reading symbols from /usr/local/lib/libssl.so.8...(no debugging symbols > found)...done. > Loaded symbols for /usr/local/lib/libssl.so.8 > Reading symbols from /usr/local/lib/libcrypto.so.8...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libcrypto.so.8 > Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done. > [New Thread 802ce9800 (LWP 100233/slapd)] > [New Thread 802ce9400 (LWP 100232/slapd)] > [New Thread 802ce9000 (LWP 100231/slapd)] > [New Thread 802ce8c00 (LWP 100230/slapd)] > [New Thread 802ce8800 (LWP 100229/slapd)] > [New Thread 802ce8400 (LWP 100228/slapd)] > [New Thread 802ce8000 (LWP 100227/slapd)] > [New Thread 802ce7c00 (LWP 100226/slapd)] > [New Thread 8093a3400 (LWP 100210/slapd)] > [New Thread 802ce7800 (LWP 100209/slapd)] > [New Thread 802ce7400 (LWP 100197/slapd)] > [New Thread 802ce7000 (LWP 100196/slapd)] > [New Thread 802ce6c00 (LWP 100194/slapd)] > [New Thread 802ce6800 (LWP 100090/slapd)] > [New Thread 802ce6400 (LWP 100089/slapd)] > [New Thread 802c0dc00 (LWP 100088/slapd)] > [New Thread 802c07400 (LWP 100081/slapd)] > Loaded symbols for /lib/libthr.so.3 > Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. > Loaded symbols for /lib/libc.so.7 > Reading symbols from /usr/lib/libssl.so.6...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libssl.so.6 > Reading symbols from /lib/libcrypto.so.6...(no debugging symbols > found)...done. > Loaded symbols for /lib/libcrypto.so.6 > Reading symbols from /usr/local/lib/sasl2/libanonymous.so.3...(no > debugging symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libanonymous.so.3 > Reading symbols from /usr/local/lib/sasl2/libcrammd5.so.3...(no > debugging symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libcrammd5.so.3 > Reading symbols from /usr/local/lib/sasl2/libdigestmd5.so.3...(no > debugging symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libdigestmd5.so.3 > Reading symbols from /usr/local/lib/sasl2/liblogin.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/liblogin.so.3 > Reading symbols from /usr/local/lib/sasl2/libscram.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libscram.so.3 > Reading symbols from /usr/local/lib/sasl2/libntlm.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libntlm.so.3 > Reading symbols from /usr/local/lib/sasl2/libotp.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libotp.so.3 > Reading symbols from /usr/lib/libopie.so.7...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libopie.so.7 > Reading symbols from /lib/libmd.so.5...(no debugging symbols found)...done. > Loaded symbols for /lib/libmd.so.5 > Reading symbols from /usr/local/lib/sasl2/libplain.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libplain.so.3 > Reading symbols from /usr/local/lib/sasl2/libsasldb.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libsasldb.so.3 > Reading symbols from /usr/local/lib/sasl2/libgssapiv2.so...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/sasl2/libgssapiv2.so > Reading symbols from /usr/lib/libgssapi.so.10...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libgssapi.so.10 > Reading symbols from /usr/lib/libheimntlm.so.10...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libheimntlm.so.10 > Reading symbols from /usr/local/lib/libkrb5.so...(no debugging symbols > found)...done. > Loaded symbols for /usr/local/lib/libkrb5.so > Reading symbols from /usr/lib/libhx509.so.10...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libhx509.so.10 > Reading symbols from /usr/local/lib/libcom_err.so...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libcom_err.so > Reading symbols from /usr/lib/libasn1.so.10...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libasn1.so.10 > Reading symbols from /usr/lib/libroken.so.10...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libroken.so.10 > Reading symbols from /usr/lib/libkrb5.so.10...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libkrb5.so.10 > Reading symbols from /usr/local/lib/libk5crypto.so...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libk5crypto.so > Reading symbols from /usr/local/lib/libkrb5support.so...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libkrb5support.so > Reading symbols from /usr/local/lib/libintl.so.9...(no debugging symbols > found)...done. > Loaded symbols for /usr/local/lib/libintl.so.9 > Reading symbols from /usr/lib/libcom_err.so.5...(no debugging symbols > found)...done. > Loaded symbols for /usr/lib/libcom_err.so.5 > Reading symbols from /usr/local/lib/libiconv.so.3...(no debugging > symbols found)...done. > Loaded symbols for /usr/local/lib/libiconv.so.3 > Reading symbols from /usr/local/libexec/openldap/back_mdb-2.4.so.2...(no > debugging symbols found)...done. > Loaded symbols for /usr/local/libexec/openldap/back_mdb-2.4.so.2 > Reading symbols from /usr/local/libexec/openldap/back_bdb-2.4.so.2...(no > debugging symbols found)...done. > Loaded symbols for /usr/local/libexec/openldap/back_bdb-2.4.so.2 > Reading symbols from /usr/local/libexec/openldap/back_hdb-2.4.so.2...(no > debugging symbols found)...done. > Loaded symbols for /usr/local/libexec/openldap/back_hdb-2.4.so.2 > Reading symbols from > /usr/local/libexec/openldap/back_ldap-2.4.so.2...(no debugging symbols > found)...done. > Loaded symbols for /usr/local/libexec/openldap/back_ldap-2.4.so.2 > Reading symbols from /usr/lib/libz.so...(no debugging symbols found)...done. > Loaded symbols for /usr/lib/libz.so > Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols > found)...done. > Loaded symbols for /libexec/ld-elf.so.1 > [Switching to Thread 802ce9800 (LWP 100233/slapd)] > 0x000000080200187c in pthread_kill () from /lib/libthr.so.3 > (gdb) bt > #0 0x000000080200187c in pthread_kill () from /lib/libthr.so.3 > #1 0x0000000801ffbb25 in pthread_getschedparam () from /lib/libthr.so.3 > #2 0x0000000802003c8d in pthread_cond_signal () from /lib/libthr.so.3 > #3 0x000000080097c868 in ldap_pvt_thread_pool_submit () > from /usr/local/lib/libldap_r-2.4.so.2 > #4 0x0000000801ffa0a4 in pthread_getprio () from /lib/libthr.so.3 > #5 0x0000000000000000 in ?? () > Error accessing memory address 0x7ffff73ed000: Bad address. > > That occurs before and after adding the new data. > > I'm assuming now that I have to rebuild with debug, but I'm not sure > exactly how I do that with ports yet and that will take more time. I > also can't seem to remember how to access threads... or at least > determine which one to follow. > > At any rate, is any of this looking familiar at all? > >>>>> Da Rock <openldap-technical(a)herveybayaustralia.com.au&gt; schrieb am 26.11.2014 um >> 10:41 in Nachricht <5475A053.7060209(a)herveybayaustralia.com.au&gt;: >>> On 26/11/2014 18:23, Ulrich Windl wrote: >>>> What you could try is (assuming you have debug symbols in your binary) >>> attaching gdb to the running slapd to see where it is spending CPU time. I >>> can't remember the commands precisely, but "bt (backtrace)" and "info >>> threads" should point you in the right direction. >>>> Note that slapd will likely stop responding while gdb is attached, so you >>> could also try to force a core dump for offline examination. >>> I'm not sure I follow here. How would this work? I attach gdb to the >>> running slapd I get, but if it stops how does that help me? I've only >>> had a little bit of experience with gdb... >>> >>> How would I get a core dump, as well? That sounds like it might be more >>> useful. >>>> Maybe the gurus get a clue what may be wrong then... >>>> >>>> Regards, >>>> Ulrich >>>> >>>>>>> Da Rock <openldap-technical(a)herveybayaustralia.com.au&gt; schrieb am 26.11.2014 um >>>> 01:31 in Nachricht <54751F73.3080902(a)herveybayaustralia.com.au&gt;: >>>>> I'm trying to get openldap to play nice with mdb given that it is the >>>>> "recommended" database backend for it now- although the conf wasn't an >>>>> issue excepting I'm playing with the new cn=config setup we're expected >>>>> to use now as well (even though it is mainly broken). >>>>> >>>>> My issue is that it seems to not respond like the older bdb/hdb >>>>> databases. And when I say respond, I mean it hangs the ldapadd and makes >>>>> slapd go into conniptions. I see slapd go to 100% WCPU and not come down >>>>> as well as going into a uwait state. I've left it going for 10 minutes >>>>> or more with no change, and I'm only adding 1 small entry of less the 10 >>>>> lines. Strangely, I can still view other entries in the specific db as >>>>> well access the rest of the server, which I won't complain about (aren't >>>>> threads a wonderful invention? ). >>>>> >>>>> So coming to the experts - got a fix at all? Or should I just go back to >>>>> ye olde db backends? At this point I have a db I can't add anything to. >>>>> >>>>> And before anyone asks, there is practically nothing in the logs that I >>>>> can see; and I set the logging to everything (-1). I see recognition of >>>>> the user in the acl and then nothing. The only possible curious entry is >>>>> some blank lines and a number (that changes each time), so nothing >>>>> informative. >>>>> >>>>> I set it up using the cn=config (and I'm still not entirely convinced >>>>> that I will keep cn=config, but apparently it could be gone next version >>>>> according to the grapevine, so the consensus is to suck it up and get >>>>> used to it or your panties will get in a bunch and around your ankles >>>>> when the upgrade comes along), and I've got only olcDBMaxSize. >>>>> olcSizeLimit (not sure exactly which of these 2 can go just yet), >>>>> olcDBMode, olcDBDirectory, and olcDatabase and the obviously root attrs. >>>>> My max size I've set larger than 50M (so 7 digits) which is larger than >>>>> what I have in another db so far, and I figure I can add more if needed >>>>> - currently it is sitting at 64k. >>>>> >>>>> I'm using FreeBSD 9.1, ports Openldap version is 2.4.40_1 with bdb/hdb >>>>> and mdb set in config. But I notice lmdb is not installed as a >>>>> dependency - is this right? >>>>> >>>>> I've been on this for near a week now with no further advancement so any >>>>> help would be very welcome at this point. No googling seems to find >>>>> anything remotely similar either. >>>>> >>>>> TIA >>>> >>