Re: (ITS#5804) attribute value regex expantion
by ando＠sys-net.it
quanah(a)zimbra.com wrote:
> --On Saturday, November 29, 2008 5:32 AM +0000 manu(a)netbsd.org wrote:
>
>> Quanah Gibson-Mount <quanah(a)zimbra.com> wrote:
>>
>>> IIRC, this also broke ACI's in RE2.4, and that still needs fixing as
>>> well.
>> Did it? Where is the report about that?
>
> Can't find it atm, but I remember someone writing in about ACI's being
> broken in HEAD right now, and I believe it was related to this change.
> RE2.4 was a typo on my part, since this isn't in RE24 right now.
Yes (I think it was Hallvard). ACIs broke since acl_string_expand()
changed prototype, but the number of args was the same, and compiling
succeeded with a warning (which I didn't notice, btw). To make it
worse, test041 did not fail (apparently, it makes no use of string
expansion). Now it's fixed; I'd say hacked, since I did not want to
change the API of dynacl, yet. Changing the API should not be a big
deal, given the presumably limited number of custom dynacl 'round.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
15 years
Re: (ITS#5804) attribute value regex expantion
by quanah＠zimbra.com
--On Saturday, November 29, 2008 5:32 AM +0000 manu(a)netbsd.org wrote:
> Quanah Gibson-Mount <quanah(a)zimbra.com> wrote:
>
>> IIRC, this also broke ACI's in RE2.4, and that still needs fixing as
>> well.
>
> Did it? Where is the report about that?
Can't find it atm, but I remember someone writing in about ACI's being
broken in HEAD right now, and I believe it was related to this change.
RE2.4 was a typo on my part, since this isn't in RE24 right now.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
15 years
Re: (ITS#5836) hung writers
by quanah＠OpenLDAP.org
--On November 29, 2008 1:33:51 AM +0000 quanah(a)OpenLDAP.org wrote:
>
>
> --On November 28, 2008 11:37:25 PM +0000 quanah(a)OpenLDAP.org wrote:
>
>
>> Patch from Howard needs to be expanded so that a new keyword is used for
>> the idle timer here, rather than the value from idletimeout. This will
>> require some reworking of the daemon to wake up for that interval as
>> well.
>
> After this patch, slapd can stay up, but seems to not be able to shut
> down. The threads show:
Ok, it did eventually finish. After starting slapd again, however, I
noticed:
Nov 29 07:06:41 new slapd[26537]: conn=8 fd=31 ACCEPT from
IP=192.168.61.21:36322 (`�)
--Quanah
15 years
Re: (ITS#5836) hung writers
by quanah＠OpenLDAP.org
--On November 28, 2008 11:37:25 PM +0000 quanah(a)OpenLDAP.org wrote:
> Patch from Howard needs to be expanded so that a new keyword is used for
> the idle timer here, rather than the value from idletimeout. This will
> require some reworking of the daemon to wake up for that interval as well.
After this patch, slapd can stay up, but seems to not be able to shut down.
The threads show:
(gdb) thr apply all bt
Thread 4 (Thread 1082132832 (LWP 2205)):
#0 0x0000003fca9089ba in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/tls/libpthread.so.0
#1 0x0000002a956c8f58 in ldap_pvt_thread_cond_wait (cond=0x8a2f30,
mutex=0x8a2f08) at thr_posix.c:299
#2 0x0000002a956c78e8 in ldap_pvt_thread_pool_destroy (tpool=0x617648,
run_pending=1) at tpool.c:381
#3 0x0000000000429704 in slapd_daemon_task (ptr=0x0) at daemon.c:2516
#4 0x0000003fca90610a in start_thread () from /lib64/tls/libpthread.so.0
#5 0x0000003fca0c68c3 in clone () from /lib64/tls/libc.so.6
#6 0x0000000000000000 in ?? ()
Thread 3 (Thread 1115703648 (LWP 2209)):
#0 0x0000003fca90bd9c in __pread_nocancel () from
/lib64/tls/libpthread.so.0
#1 0x0000002a97348edc in __os_io (dbenv=0xdc2100, op=1, fhp=0xf8d4d8,
pgno=399474, pagesize=16384, buf=0x2ae8a1cc20 "Y,", niop=0x42681c30) at
../dist/../os/os_rw.c:55
#2 0x0000002a973405ea in __memp_pgread (dbmfp=0xeab3c0,
mutexp=0x2bb963f008, bhp=0x2ae8a1cb88, can_create=0) at
../dist/../mp/mp_bh.c:219
#3 0x0000002a973413f7 in __memp_fget (dbmfp=0xeab3c0, pgnoaddr=0x42681d84,
flags=0, addrp=0x42681d88) at ../dist/../mp/mp_fget.c:580
#4 0x0000002a972d18b3 in __bam_search (dbc=0xd2dc380, root_pgno=Variable
"root_pgno" is not available.
) at ../dist/../btree/bt_search.c:307
#5 0x0000002a972c6e27 in __bam_c_search (dbc=0xd2dc380, root_pgno=0,
key=0x42682050, flags=28, exactp=0x42681eb4) at
../dist/../btree/bt_cursor.c:2547
#6 0x0000002a972c7b7e in __bam_c_get (dbc=0xd2dc380, key=0x42682050,
data=0x42682030, flags=28, pgnop=0x42681f24) at
../dist/../btree/bt_cursor.c:962
#7 0x0000002a9730f51b in __db_c_get (dbc_arg=0xd2dc1c0, key=0x42682050,
data=0x42682030, flags=28) at ../dist/../db/db_cam.c:643
#8 0x0000002a973177d7 in __db_c_get_pp (dbc=0xd2dc1c0, key=0x42682050,
data=0x42682030, flags=28) at ../dist/../db/db_iface.c:1836
#9 0x0000002a97184a16 in bdb_id2entry (be=0x42802a40, tid=0x0, locker=23,
id=3443171, e=0x42682148) at id2entry.c:130
#10 0x0000002a97189e18 in bdb_cache_find_id (op=0x428025e0, tid=0x0,
id=3443171, eip=0x42682310, islocked=0, locker=23, lock=0x42682250) at
cache.c:714
#11 0x0000002a9717b1c7 in bdb_search (op=0x428025e0, rs=0x42802570) at
search.c:696
#12 0x0000002a975921eb in syncprov_findcsn (op=0x5c1bc00,
mode=FIND_PRESENT) at syncprov.c:681
#13 0x0000002a9759690a in syncprov_op_search (op=0x5c1bc00, rs=0x42803d60)
at syncprov.c:2074
#14 0x000000000049b999 in overlay_op_walk (op=0x5c1bc00, rs=0x42803d60,
which=op_search, oi=0xdec8c0, on=0xdec700) at backover.c:640
#15 0x000000000049bbf5 in over_op_func (op=0x5c1bc00, rs=0x42803d60,
which=op_search) at backover.c:702
#16 0x000000000049bc8b in over_op_search (op=0x5c1bc00, rs=0x42803d60) at
backover.c:724
#17 0x000000000042f23c in fe_op_search (op=0x5c1bc00, rs=0x42803d60) at
search.c:355
#18 0x000000000042ed10 in do_search (op=0x5c1bc00, rs=0x42803d60) at
search.c:217
#19 0x000000000042bdff in connection_operation (ctx=0x42803e90,
arg_v=0x5c1bc00) at connection.c:1133
#20 0x000000000042c2ef in connection_read_thread (ctx=0x42803e90,
argv=0x1e) at connection.c:1261
#21 0x0000002a956c7c77 in ldap_int_thread_pool_wrapper (xpool=0x8a2f00) at
tpool.c:478
#22 0x0000003fca90610a in start_thread () from /lib64/tls/libpthread.so.0
#23 0x0000003fca0c68c3 in clone () from /lib64/tls/libc.so.6
#24 0x0000000000000000 in ?? ()
Thread 2 (Thread 1124096352 (LWP 2210)):
#0 bdb_idl_union (a=0xb3ff000, b=0xb1ff000) at idl.c:1151
#1 0x0000002a971829fc in inequality_candidates (op=0x5b11c00,
ava=0x5a06518, ids=0xb3ff000, tmp=0xb1ff000, gtorlt=165) at
filterindex.c:1063
#2 0x0000002a9717ff83 in bdb_filter_candidates (op=0x5b11c00, f=0x5a064f8,
ids=0xb3ff000, tmp=0xb1ff000, stack=0xb4ff000) at filterindex.c:158
#3 0x0000002a97180595 in list_candidates (op=0x5b11c00, flist=0x5a064f8,
ftype=160, ids=0xb2ff000, tmp=0xb1ff000, save=0xb3ff000) at
filterindex.c:503
#4 0x0000002a97180213 in bdb_filter_candidates (op=0x5b11c00, f=0x5a064d8,
ids=0xb2ff000, tmp=0xb1ff000, stack=0xb3ff000) at filterindex.c:183
#5 0x0000002a97180595 in list_candidates (op=0x5b11c00, flist=0x42e83670,
ftype=160, ids=0x42f03970, tmp=0xb1ff000, save=0xb2ff000) at
filterindex.c:503
#6 0x0000002a97180213 in bdb_filter_candidates (op=0x5b11c00,
f=0x42e836d0, ids=0x42f03970, tmp=0xb1ff000, stack=0xb2ff000) at
filterindex.c:183
#7 0x0000002a9717c2f1 in search_candidates (op=0x5b11c00, rs=0x43004d60,
e=0x42e83910, locker=57, ids=0x42f03970, scopes=0x42e83970) at search.c:1124
#8 0x0000002a9717ad57 in bdb_search (op=0x5b11c00, rs=0x43004d60) at
search.c:595
#9 0x000000000049ba2d in overlay_op_walk (op=0x5b11c00, rs=0x43004d60,
which=op_search, oi=0xdece00, on=0x0) at backover.c:650
#10 0x000000000049bbf5 in over_op_func (op=0x5b11c00, rs=0x43004d60,
which=op_search) at backover.c:702
#11 0x000000000049bc8b in over_op_search (op=0x5b11c00, rs=0x43004d60) at
backover.c:724
#12 0x000000000042f23c in fe_op_search (op=0x5b11c00, rs=0x43004d60) at
search.c:355
#13 0x000000000042ed10 in do_search (op=0x5b11c00, rs=0x43004d60) at
search.c:217
#14 0x000000000042bdff in connection_operation (ctx=0x43004e90,
arg_v=0x5b11c00) at connection.c:1133
#15 0x000000000042c2ef in connection_read_thread (ctx=0x43004e90,
argv=0x19) at connection.c:1261
#16 0x0000002a956c7c77 in ldap_int_thread_pool_wrapper (xpool=0x8a2f00) at
tpool.c:478
#17 0x0000003fca90610a in start_thread () from /lib64/tls/libpthread.so.0
#18 0x0000003fca0c68c3 in clone () from /lib64/tls/libc.so.6
#19 0x0000000000000000 in ?? ()
Thread 1 (Thread 182904263232 (LWP 2201)):
#0 0x0000003fca906ffb in pthread_join () from /lib64/tls/libpthread.so.0
#1 0x0000002a956c8eb0 in ldap_pvt_thread_join (thread=1082132832,
thread_return=0x0) at thr_posix.c:193
#2 0x00000000004297d1 in slapd_daemon () at daemon.c:2581
#3 0x00000000004129d9 in main (argc=10, argv=0x7fbfffb918) at main.c:859
--Quanah
15 years
Re: (ITS#5836) hung writers
by quanah＠OpenLDAP.org
--On November 28, 2008 10:49:36 PM +0000 quanah(a)OpenLDAP.org wrote:
> slapd ends up hanging indefinitely. This happened with 8 replicas
> connected to the server, where for four of those replicas, the connection
> between the master and the replica wouldn't connect (but you could go
> from the replica to the master) at some point after the initial
> connection was made.
>
> Patch to HEAD from Howard is committed.
Patch from Howard needs to be expanded so that a new keyword is used for
the idle timer here, rather than the value from idletimeout. This will
require some reworking of the daemon to wake up for that interval as well.
--Quanah
15 years
(ITS#5836) hung writers
by quanah＠OpenLDAP.org
Full_Name: Quanah Gibson-Mount
Version: HEAD/RE24/RE23
OS: Linux 2.6
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (75.111.29.239)
In a scenario where the network is broken (or other reasons, but that's where I
hit it), slapd can get locked up waiting on writers to finish. When that
happens, everything ends up in a pthread_cond_wait, like this:
Thread 8 (Thread 1098918240 (LWP 18884)):
#0 0x0000003fca9089ba in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/tls/libpthread.so.0
#1 0x0000002a956c8f58 in ldap_pvt_thread_cond_wait (cond=0x385bdf0,
mutex=0x385bca0) at thr_posix.c:299
#2 0x000000000043dc91 in send_ldap_ber (conn=0x385bc88, ber=0x41680610) at
result.c:198
#3 0x0000000000441450 in slap_send_search_entry (op=0x625bc00, rs=0x41801d60)
at result.c:1137
#4 0x0000002a9717b9e1 in bdb_search (op=0x625bc00, rs=0x41801d60) at
search.c:879
#5 0x000000000049b9c9 in overlay_op_walk (op=0x625bc00, rs=0x41801d60,
which=op_search, oi=0xdece00, on=0x0) at backover.c:650
#6 0x000000000049bb91 in over_op_func (op=0x625bc00, rs=0x41801d60,
which=op_search) at backover.c:702
#7 0x000000000049bc27 in over_op_search (op=0x625bc00, rs=0x41801d60) at
backover.c:724
#8 0x000000000042f1d8 in fe_op_search (op=0x625bc00, rs=0x41801d60) at
search.c:355
#9 0x000000000042ecac in do_search (op=0x625bc00, rs=0x41801d60) at
search.c:217
#10 0x000000000042bd9c in connection_operation (ctx=0x41801e90, arg_v=0x625bc00)
at connection.c:1133
#11 0x000000000042c28c in connection_read_thread (ctx=0x41801e90, argv=0x1f) at
connection.c:1261
#12 0x0000002a956c7c77 in ldap_int_thread_pool_wrapper (xpool=0x8a2f00) at
tpool.c:478
#13 0x0000003fca90610a in start_thread () from /lib64/tls/libpthread.so.0
#14 0x0000003fca0c68c3 in clone () from /lib64/tls/libc.so.6
#15 0x0000000000000000 in ?? ()
slapd ends up hanging indefinitely. This happened with 8 replicas connected to
the server, where for four of those replicas, the connection between the master
and the replica wouldn't connect (but you could go from the replica to the
master) at some point after the initial connection was made.
Patch to HEAD from Howard is committed.
--Quanah
15 years
Re: (ITS#5804) attribute value regex expantion
by ando＠sys-net.it
quanah(a)zimbra.com wrote:
> --On November 28, 2008 9:07:49 PM +0000 quanah(a)zimbra.com wrote:
>
>>
>> --On November 28, 2008 9:02:27 PM +0000 manu(a)netbsd.org wrote:
>>
>>> Konstantinos Koukopoulos <kouk(a)noc.edunet.gr> wrote:
>>>
>>>> Turns out 'val' is nil and it doesn't seem like there's any check for
>>>> that. Maybe naive fix:
>>> I see Ando committed it. But there are many other locations in the file
>>> where acl_string_expand is called with an unchecked val->bv_val. It
>>> seems these also need to be fixed, but I miaght be missing a point.
>> IIRC, this also broke ACI's in RE2.4, and that still needs fixing as well.
>
> Egh, in HEAD only, I mean, as this wasn't applied yet to RE24.
:) I've reworked the use of string expansion (and lined up regex
expansion in "by" dn). In order to fix ACIs, the dynacl API needs to be
enhanced to support AclRegexMatches.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
15 years
Re: (ITS#5804) attribute value regex expantion
by quanah＠zimbra.com
--On November 28, 2008 9:07:49 PM +0000 quanah(a)zimbra.com wrote:
>
>
> --On November 28, 2008 9:02:27 PM +0000 manu(a)netbsd.org wrote:
>
>> Konstantinos Koukopoulos <kouk(a)noc.edunet.gr> wrote:
>>
>>> Turns out 'val' is nil and it doesn't seem like there's any check for
>>> that. Maybe naive fix:
>>
>> I see Ando committed it. But there are many other locations in the file
>> where acl_string_expand is called with an unchecked val->bv_val. It
>> seems these also need to be fixed, but I miaght be missing a point.
>
> IIRC, this also broke ACI's in RE2.4, and that still needs fixing as well.
Egh, in HEAD only, I mean, as this wasn't applied yet to RE24.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
15 years
Re: (ITS#5804) attribute value regex expantion
by quanah＠zimbra.com
--On November 28, 2008 9:02:27 PM +0000 manu(a)netbsd.org wrote:
> Konstantinos Koukopoulos <kouk(a)noc.edunet.gr> wrote:
>
>> Turns out 'val' is nil and it doesn't seem like there's any check for
>> that. Maybe naive fix:
>
> I see Ando committed it. But there are many other locations in the file
> where acl_string_expand is called with an unchecked val->bv_val. It
> seems these also need to be fixed, but I miaght be missing a point.
IIRC, this also broke ACI's in RE2.4, and that still needs fixing as well.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
15 years
