Re: (ITS#4712) test007: slapd crashes
by ando＠sys-net.it
michael(a)stroeder.com wrote:
> Full_Name: Michael Ströder
> Version: HEAD
> OS: SuSE Linux 10.0
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (83.124.12.70)
>
>
> In test007 slapd seg faults. The backtrace:
>
> #0 0x080dd880 in lutil_strncopy (a=0xbfde5a08 "", b=0x111 <Address 0x111 out of
> bounds>, n=0)
> at utils.c:303
> #1 0x08066916 in slap_cf_aux_table_unparse (src=0x819e010, bv=0xbfde6218,
> tab0=0x8142180)
> at config.c:1175
> #2 0x08066b68 in bindconf_free (bc=0x3) at config.c:1243
> #3 0x0805fb84 in config_replica (c=0xbfde634c) at bconfig.c:2734
> #4 0x08065cfc in config_get_vals (cf=0x81419b0, c=0xbfde634c) at config.c:449
> #5 0x08062611 in config_build_entry (op=0x8141280, rs=0xbfde634c, parent=0x0,
> c=0x0, rdn=0x0,
> main=0xbfde6373, extra=0x81cade0) at bconfig.c:4565
> #6 0xbfde634c in ?? ()
> #7 0x081794a8 in ?? ()
> #8 0x081794a8 in ?? ()
> #9 0x00000000 in ?? ()
>
Michael,
this backtrace looks weird, because:
- bconfig.c:2734 is not in config_replica() in HEAD right now, and,
according to cvsweb, the last commit to bconfig.c occurred 4 weeks ago
(unless you modified that file yourself)
- there's nothing at config.c:449 that can call config_replica();
- bindconf_free() cannot really call slap_cf_aux_table_unparse ()
- there's more inconsistencies...
You should really cross-check what you're debugging, or something else
is going berserk. Maybe try without optimization?
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
17 years, 1 month
Re: (ITS#4708) refreshAndPersist retry parameter isn't meaningful
by ando＠sys-net.it
To me it makes a lot of sense; I've applied your patch, with minor
changes, to HEAD code. Please test.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
17 years, 1 month
(ITS#4712) test007: slapd crashes
by michael＠stroeder.com
Full_Name: Michael Ströder
Version: HEAD
OS: SuSE Linux 10.0
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (83.124.12.70)
In test007 slapd seg faults. The backtrace:
#0 0x080dd880 in lutil_strncopy (a=0xbfde5a08 "", b=0x111 <Address 0x111 out of
bounds>, n=0)
at utils.c:303
#1 0x08066916 in slap_cf_aux_table_unparse (src=0x819e010, bv=0xbfde6218,
tab0=0x8142180)
at config.c:1175
#2 0x08066b68 in bindconf_free (bc=0x3) at config.c:1243
#3 0x0805fb84 in config_replica (c=0xbfde634c) at bconfig.c:2734
#4 0x08065cfc in config_get_vals (cf=0x81419b0, c=0xbfde634c) at config.c:449
#5 0x08062611 in config_build_entry (op=0x8141280, rs=0xbfde634c, parent=0x0,
c=0x0, rdn=0x0,
main=0xbfde6373, extra=0x81cade0) at bconfig.c:4565
#6 0xbfde634c in ?? ()
#7 0x081794a8 in ?? ()
#8 0x081794a8 in ?? ()
#9 0x00000000 in ?? ()
17 years, 1 month
Re: (ITS#4711) back-meta search fails to stop on "@" action
by ando＠sys-net.it
rici(a)ricilake.net wrote:
> Full_Name: Rici Lake
> Version: 2.3.21
> OS: various
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (200.106.32.142)
>
>
> In servers/slapd/back-meta/search.c the function ldap_back_dn_massage is called
> at line 394 (in HEAD) in the function meta_back_search_start. It then checks the
> return code using a switch statement, where the default branch is to accept the
> rewrite.
>
> However, it is checking for return codes REWRITE_REGEXEC_{UNWILLING, ERR} while
> ldap_back_dn_massage is returning actual LDAP error codes
> (LDAP_UNWILLING_TO_PERFORM, LDAP_OTHER).
>
> Consequently, the "@" action is not honoured, and neither is any other return
> code inserted with a "U" action.
>
Good spot. I'll fix this in minutes.
Thanks, p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.n.c.
Via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
------------------------------------------
Office: +39.02.23998309
Mobile: +39.333.4963172
Email: pierangelo.masarati(a)sys-net.it
------------------------------------------
17 years, 1 month
Re: (ITS#4702) out-of-memory on huge DB ?
by quanah＠stanford.edu
--On Friday, October 13, 2006 1:57 PM +0000 Paolo.Rossi.con(a)h3g.it wrote:
>> No. As I already mentioned, there is no reason for slapcat to use
>> large amounts of RAM, it should use whatever size your BDB cache is
>> configured for and that's about it. It's possible there is a memory
>> leak, but since no such leak appears under Linux that would point
>> to either the Solaris C library or some other library specific to
>> your platform.
>
> Today I've finished a cross test on a suse 10.0 linux
>
> Athlon XP 2200+ 768MB RAM + 1GB swap
>
>
>
> loaded huge db with same config, shared mem and set_cachesize 0
> 384000000 1
>
> slapcat -f slapd.conf
So you understand that without the -l flag to slapcat, you are writing to
stdout instead of a file? Is it possible that is what is causing slapcat
to crash?
On my linux 2.6 system (64 bit), with a 2.5 million entry db, which is:
ldap00:/var/lib/ldap/slamd/64-2.5mil-db42# du -h *.bdb
84M cn.bdb
618M dn2id.bdb
85M employeeNumber.bdb
3.2G id2entry.bdb
3.7M objectClass.bdb
85M uid.bdb
DB_CONFIG is:
set_cachesize 0 384000000 1
set_lg_regionmax 262144
set_lg_bsize 2097152
set_lg_dir /var/log/bdb/slamd/64-2.5mil
set_lk_max_locks 3000
#
# Automatically remove log files that are no longer needed.
set_flags DB_LOG_AUTOREMOVE
#set_flags DB_DIRECT_DB
#
# Setting set_tas_spins reduces resource contention from multiple clients
on systems with multiple CPU's.
set_tas_spins 1
Using slapcat -l ldif_file, I then ran a while (1) loop to check the OSS
and RSS sizes of slapcat. It reached the following size fairly quickly:
RSS VSZ
1159740 1191916
and then never grew past that. This means the resident memory maxed out at
1.1 GB, and the virtual memory maxed out at 1.1GB as well.
So, that does mean one thing -- It definitely outgrew the 366MB of BDB
cache defined. On the other hand, once it reached its max size, it stayed
steady there until completion was reached.
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
17 years, 1 month
(ITS#4711) back-meta search fails to stop on "@" action
by rici＠ricilake.net
Full_Name: Rici Lake
Version: 2.3.21
OS: various
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (200.106.32.142)
In servers/slapd/back-meta/search.c the function ldap_back_dn_massage is called
at line 394 (in HEAD) in the function meta_back_search_start. It then checks the
return code using a switch statement, where the default branch is to accept the
rewrite.
However, it is checking for return codes REWRITE_REGEXEC_{UNWILLING, ERR} while
ldap_back_dn_massage is returning actual LDAP error codes
(LDAP_UNWILLING_TO_PERFORM, LDAP_OTHER).
Consequently, the "@" action is not honoured, and neither is any other return
code inserted with a "U" action.
17 years, 1 month
Re: (ITS#4702) out-of-memory on huge DB ?
by Paolo.Rossi.con＠h3g.it
> No. As I already mentioned, there is no reason for slapcat to use
> large amounts of RAM, it should use whatever size your BDB cache is
> configured for and that's about it. It's possible there is a memory
> leak, but since no such leak appears under Linux that would point
> to either the Solaris C library or some other library specific to
> your platform.
Today I've finished a cross test on a suse 10.0 linux
Athlon XP 2200+ 768MB RAM + 1GB swap
loaded huge db with same config, shared mem and set_cachesize 0
384000000 1
slapcat -f slapd.conf
and some hours later..
kernel: Out of Memory: Killed process 8915 (slapcat).
kernel: oom-killer: gfp_mask=0xd2, order=0
kernel: Mem-info:
kernel: DMA per-cpu:
kernel: cpu 0 hot: low 2, high 6, batch 1 used:3
kernel: cpu 0 cold: low 0, high 2, batch 1 used:1
kernel: Normal per-cpu:
kernel: cpu 0 hot: low 62, high 186, batch 31 used:92
kernel: cpu 0 cold: low 0, high 62, batch 31 used:26
kernel: HighMem per-cpu: empty
kernel: Free pages: 6464kB (0kB HighMem)
kernel: Active:75306 inactive:111212 dirty:0 writeback:0 unstable:0
free:1616 slab:2595 mapped:134308 pagetables:589
kernel: DMA free:3076kB min:72kB low:88kB high:108kB active:5132kB
inactive:4348kB present:16384kB pages_scanned:10531
all_unreclaimable? yes
kernel: lowmem_reserve[]: 0 751 751
kernel: Normal free:3388kB min:3468kB low:4332kB high:5200kB active:
296092kB inactive:440500kB present:769984kB pages_scanned:737960
all_unreclaimable? yes
kernel: lowmem_reserve[]: 0 0 0
kernel: HighMem free:0kB min:128kB low:160kB high:192kB active:0kB
inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
kernel: lowmem_reserve[]: 0 0 0
kernel: DMA: 1*4kB 0*8kB 2*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB
0*1024kB 1*2048kB 0*4096kB = 3076kB
kernel: Normal: 1*4kB 3*8kB 2*16kB 0*32kB 2*64kB 1*128kB 0*256kB
0*512kB 1*1024kB 1*2048kB 0*4096kB = 3388kB
kernel: HighMem: empty
kernel: Swap cache: add 19238177, delete 19238170, find
2371332/4902129, race 0+0
kernel: Free swap = 0kB
kernel: Total swap = 1052216kB
kernel: Free swap: 0kB
kernel: 196592 pages of RAM
kernel: 0 pages of HIGHMEM
kernel: 2823 reserved pages
kernel: 811 pages shared
kernel: 7 pages swap cached
kernel: 0 pages dirty
kernel: 0 pages writeback
kernel: 134308 pages mapped
kernel: 2595 pages slab
kernel: 589 pages pagetables
free system swap was 0k and free mem about 6000K
the out file about 40% of DB.
I've tried also with
echo 1 > /proc/sys/vm/overcommit_memory
trying to avoid oom-killer to kill the fat slapd,
same result.
out-of-memory but, not malloc error or core dump this time.
"Dura Murphy lex, sed lex" :)
Any program will expand to fill available memory.
Paolo
17 years, 1 month
Re: (ITS#4704) accesslog database fails to store contextCSN at shutdown
by quanah＠stanford.edu
--On Monday, October 09, 2006 6:42 PM +0000 quanah(a)stanford.edu wrote:
> Then, I did the following:
>
> (a) stop slapd
> (b) remove the current accesslog db (since all slaves are in sync)
> (c) get the contextCSN:
>
> dn: cn=accesslog
> contextCSN: 20061009183830Z#000000#00#000000
>
> (d) stopped slapd
>
> (e) started slapd
>
> (f) queried, the contextCSN is the same(?!!)
>
> ldap-test0:~> ldapsearch -LLL -Q -h localhost -b cn=accesslog -s base
> contextCSN
> dn: cn=accesslog
> contextCSN: 20061009183830Z#000000#00#000000
>
> (g) Made a modification
>
> (h) got the current contectCSN:
>
> dn: cn=accesslog
> contextCSN: 20061009184042Z#000000#00#000000
>
>
> (i) Stopped slapd
>
> (j) restarted slapd
>
> (k) got the contectCSN:
>
> dn: cn=accesslog
> contextCSN: 20061009184042Z#000000#00#000000
>
>
> It is the same again. So at best guess, something "corrupted" the
> accesslog DB somehow? I'm not really sure. I will monitor to see if I
> get back into the same situation again.
I've recreated the accesslog DB in all my environments, and now this
problem is gone. Either some earlier release of OpenLDAP 2.3 created a
funky accesslog DB that it wasn't happy with, or there's a state that can
get triggered that causes it to always recreate the contextCSN, I'm not
sure what that would be. In any case, I think this ITS can be closed for
now. If my servers get back into this state, then we'll know there's some
underlying bug that causes it that needs to be found.
--Quanah
--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
17 years, 1 month
Re: (ITS#4705) 11866 Segmentation fault (core dumped)
by michael＠stroeder.com
Even with a simpler build script (see below) it does not work.
Ciao, Michael.
--------------------------------------------------------------------------
. /home/michael/src/build-env-openldap
#export CFLAGS="-g -O4 -march=pentium4"
export CFLAGS="-g -DLDAP_DEVEL"
export CPPFLAGS="-I/usr/include/heimdal -I/opt/openldap-snacc/include
-I/usr/include/sasl -I${BDB}/include -I${SASL}/include -I${HEIMDAL}/include"
export LDFLAGS="-L${BDB}/lib -L${SASL}/lib -L${HEIMDAL}/lib -R${BDB}/lib
-R${SASL}/lib -R${HEIMDAL}/lib"
export LD_LIBRARY_PATH="${BDB}/lib:${SASL}/lib:${HEIMDAL}/lib"
# Disable test036-meta-concurrency since it always hangs
export TEST_META=no
PWD=`pwd`
DIRNAME=`dirname ${PWD}`
PREFIX=/opt/`basename ${DIRNAME}`
echo "PREFIX="${PREFIX}
make distclean
./configure \
--enable-modules \
--enable-dynamic \
--enable-backends=mod \
--enable-overlays=mod \
--enable-perl=no \
--enable-slapi=yes \
--enable-slp=yes \
--enable-crypt=yes
make && make test
17 years, 1 month
(ITS#4709) Insufficient permissions on PF_LOCAL sockets
by ian＠uns.ns.ac.yu
Full_Name: Ivan Nejgebauer
Version: 2.3.27
OS: Linux 2.6 (Ubuntu)
URL: ftp://ftp.openldap.org/incoming/ivan-nejgebauer-061012.patch
Submission from: (NULL) (147.91.172.229)
Linux, as opposed to other *NIXes, honors PF_LOCAL socket file mode bits, so a
user must have the write permission to use the socket. OpenLDAP bind()s its
PF_LOCAL sockets without any special arrangements, so the resulting socket's
permissions are governed by the current umask. Since the umask is usually 022 or
002, the socket ends up not being world-writable, making it unusable for users
other than root.
Earlier OpenLDAP releases recognized a non-standard "x-mod" URL extension for
manipulating socket permissions, and the parsing code is still there, but its
results are unused.
With the attached patch, PF_LOCAL sockets are always created world-writable by
setting the umask to zero before bind(). The previous umask is restored
immediately afterwards. Umask manipulation shouldn't affect PF_UNIX bind()s, so
I haven't surrounded it with #ifdef LDAP_PF_LOCAL.
i.
17 years, 1 month
