Re: (ITS#7987) SIGSEGV in LMDB while adding ldap-entries
by leo＠yuriev.ru
Previously noted (2) and (3) was caused by following patch.
It is a trial from Howard Chu for workout on ITS#7968 ;)
Under a heavy load of add/delete it may induce "DN index delete failed"
in the mdb backend.
> Message-ID: <5440D299.60208(a)symas.com>
> Date: Fri, 17 Oct 2014 09:26:01 +0100
> From: Howard Chu <hyc(a)symas.com>
>
> Can you try this patch and followup again?
>
>> diff --git a/servers/slapd/overlays/syncprov.c
>> b/servers/slapd/overlays/syncprov.c
>> index e15020e..b54c83f 100644
>> --- a/servers/slapd/overlays/syncprov.c
>> +++ b/servers/slapd/overlays/syncprov.c
>> @@ -1306,11 +1306,12 @@ syncprov_matchops( Operation *op, opcookie
>> *opc, int saveit )
>> op2.o_hdr = &oh;
>> op2.o_extra = op->o_extra;
>> op2.o_callback = NULL;
>> - if (ss->s_flags & PS_FIX_FILTER) {
>> + if ((ss->s_flags & PS_FIX_FILTER)
>> + && op2.ors_filter->f_choice ==
>> LDAP_FILTER_AND) {
>> /* Skip the AND/GE clause that we
>> stuck on in front. We
>> would lose deletes/mods that
>> happen during the refresh
>> phase otherwise (ITS#6555) */
>> - op2.ors_filter =
>> ss->s_op->ors_filter->f_and->f_next;
>> + op2.ors_filter =
>> op2.ors_filter->f_and->f_next;
>> }
>> ldap_pvt_thread_mutex_unlock( &ss->s_mutex );
>> rc = test_filter( &op2, e, op2.ors_filter );
8 years, 3 months
Re: (ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by leo＠yuriev.ru
This is a multi-part message in MIME format.
--------------010409020705090508090802
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Some of Valgrind's DRD output, seems be enough to crash around
connect/disconnect and walking on overlays.
--------------010409020705090508090802
Content-Type: application/gzip;
name="valgrind-drd.log.gz"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="valgrind-drd.log.gz"
H4sICJ8XelQAA3ZhbGdyaW5kLWRyZC5sb2cA7V3bcuO4EX3PV/At3lRkgSB4U2pqQ922tmqS
naq9VB5SxaJJ2GZGIrUk5bHz9WmQ1IUayNY6HluSjx9kXdgkgO7TOH0kgB8+uMz1P3wwkiL5
qxEZ1W0ho8SQRZEXRiIrGVd58acPq6NG+eKhSG9uK+Ni9J3BGXN6nJkWWWaJ8cM/fzV++PTx
z3SiqwdjGBWV8VuUGUFZxrfycnOSX8s0u6GPZjdFmiU969Jkl6w+w8f06rfJv/5mFLJYZsaX
tLo1erfGNTUlXl84za7z7QbN52Q5MPq3+Vz2Zw/9T0X+H2p12f/4j/GwV8myiqNSXt6kVV+9
6N1FSTrv38pC9spZtEj6s/RK3su4X78yejPj40+j4KNt9BKDGb1SPQjVjFkSLQb9PjWWX7JL
c2DSH+//e/0+d+nt+gNm+bxv9K6f1agkqqJ+licy5H1Ztc26jPPsetPrT1Ehs8r49ON4YDi+
zzafbJ790nhSDLYHK7uepXGlhr8kv0rlp9bjwogqg90zh8vYkrFRpv+VhtjY0l99gDCnLHAG
hmpWOEvLSmayMC6SSM7z7DIemJ7wvuuY0TVqM2/CdszC9toba848X2c9sb0xNwf1WIdpVrWW
4SLPZ+GXIlosVCMq9ZLO4njaJows0w2GA+P77783LtLM6C/LQnm/f9eGYv/uZlHIWU4nJjj0
onniiN4szZb3l2WuOaPNh4wFgnpVUbCve7NoGxfTYyWpPRbTdcq2R8wTo4ERz/JMGhf1v8uf
VVxtHR0kSSFLisL7lWfSUnkiv74uZWUIYVwX+Xz1ccwujWA2y+OoSvOMUJNV8r4aaNw44uPp
mBwyj9ThLzQeLDDJ/wPjinw7l82pw3vjgp7nxYPyL9d6xppMxwENxG24as76KRnZQhtQk0BY
bUDlC5lpg9H2mM6Wm97UbGypa/XRFFVptW3quNpIZAFvho0G7EI90rGu2PbYTxXh2CjlzVxh
tA4NCs0mNvh3OldYYjqm5qziZr4kp4XLjHr/+WUcIzgLht5Oh6uo/LwNPXFGqOn6QNLU8pgH
1Dkn3Gr7WD6UMYUeZen5Yqaa8vPAM/WDyqbTJwbVtpwzHdTDAtsy2W5gv2BYm1NKHE+FtWWd
qQeeCuvXSSyM8ac8YJ9PYvmKX7l7+JVq9ha9clf0yvKkbbGGXnmaVGQyPqRGlA9ZvCjyu/D3
ZjRXr9XMxJh2QgRHIhLUDm+HI7E1RWo+PZgiWYyNKH3FL02RphuKFGs4kqdNWNaUuHnNkWIN
RzL1MUHeHBFZv4riz5QtwuSqpTntG4pc+doJisLQplaqsiO9CW8Ur0rpklfNGzW14lxrGDgj
vjak4Q9pqEpq7MrQsrXdM8eW5Q6MJiga4+t0JrcMXV87C5vOyJ52DLebKbiljXOicfZol8b5
/MDZznqjbDvmPqe+zsmP1KjiISxktSyoB2nC6zdqyqHtMJk6oms6k1SDdm19Lef1iPPSNJvf
yWIWPXTtw/yuCSf1qRpwSxuInjkcW01aK+Ri1pyjSWvqdR32rta/rWWSh+3BvGNGgaq/oOWr
iWljNuua2Za2q9xmE7cO34zqdUoQYe3LVdravK/OwX1tKL/fRLyfn+xDzLflh95w6K5qxN+X
cinDuCS0xNU9/Vce1KcUd+RzthWsy0VCYzPK88+p7ESR5Tra4HOnTCWzR2JW63PE7OvH7Lcm
dBThNLabKXjLS7tTsW9qp2JOM+a4497GsONX29WLJFMxEfvFOk7Nh1gHIgoiCiL6hybWie1O
VGdriC/uuhAvl1dz5dsVwi2hc5DNhiPH3a50CTqLnUrX18m3tmmORt6W5Tyq4tt8UXaNLS0T
tU0eTPiWcb4I1YXzrJRde18/21rW0GsVj7VhuCBebFzQy+WsUrHM9DK3YBOfTBUC6pHbXHhj
ugd0IvBaxTrs2JPVlrXnaENz5E0mXkP+o4SSHj3UVF0LHC8Qwt6wfRqfL9Hsc4fkO3uoemAP
WWOpzK6XWdwxc7m2bx5zV5LH3tpAz7NaywNrgz/OTr+5ejYa2jTHRVVVhG0uVc+VI7UBxMc2
c9vj57K4kevjhWtriajgPnVhnifp9YPyvUqCS0lIobfK+msOrScD1/SDJmBaW5rHZZFFs9qS
3qhttY0M3MnY2bbdMnH2RMBrhxwx/OGBDF+Px5Nh+Ec4e6j8zZyn87epC2nk73ebv48GRW8u
bBB3ClRX0ixuzk3ZuRP8rqsj0yuzRymXrSVroFyA7OtB9mUVGM/31mO0Bf6vxsrT1ilKwHEm
Bwo4LgQcCDgQcCDgQMCBgAM2AQEHAg4EHAg4yN/I38eEIgg4oFyA7CkJOGP7IAFHW4FCwIGA
AwEHAg4EHAg4YBMQcCDgHEHIQcCBgIP8fWL5+2hQBAEHlAuQfRUBp10WZR24j5B1uKgzUes6
63GL4piK9pqAfZFJZ/gsrt3BRYihS4PQNSRMqLXyUaw6JGy9IR8O6bJq16WQEE9hFUtVXlZK
EWn+r1bDelzr8O0z6A1NLRIUglfLCh+ddLWJAwh+rwiGpAVJ61UlrR0N/NVqcXM85WK1l0wN
+LXQ7ekLXSXM208L83pXQphfw/ZNtwZ4q3Cj4BHu08GjVXUQPG8YPMhWCDhkK2SrYw2er2pn
57AtRZyDtxSxAodI4bUMo+W92vy0qNm++mpgQ31NX2iDwWJ1FlqR5r3W+hxm2fa4U9CVMiqI
qrfFz7qiMxnTl7HMdfytMlad4ysJzPP1uvOO7e9N+bpl6Au9+rVrqNlTz9SqBe83iFF1oup8
Dz+kwFdhUOXwVRi+CgNkAVk9ZDFpAQHvGwGYtDBpAbInBdmXFZxojETw2BiZ2qG1rMCzD9Cp
tItuoFNBp4JOBZ0KOhUoP/gDKD8oPyALyEKnAgKAAExamLQA2XOA7IvrVK54VKfS/yQKOhV0
KuhUm8kDOhV0KlB+8Ifj5w+g/KD8gOxJQRaTFhDwvhGASQuTFiB7UpDdPGvX9HmHrenzDpWu
KBc4rNagaNCiqyhLqPC/aJ/UkayVrjgXbNjJIGvbTvZw9eIVWY+9jnk8y0vVj665p9+Fh4vR
cPJV/tpNXEIL/T3G+5Kfp3Ui9CToSdCTzllPwkbHYBmnxzKw0TE2Ol6Ny7vZ6PhlCfEz1xxQ
B1znUR6tJ6Pg0eDR4NHg0eDR4NHg0eDR4NHg0efCo5/zm0jwaPBo8GjwaPBo8GjwaPBo8Gjw
6JPl0e1vNvhhv9ngK2rNZWxJbz+1NqdTx1ltAd9siB02S2E2u2ML7RKak+d47dB0OJ7YkDz1
cXw4yePj6ZgpJvDCJM/bkDwNx9PHqTWZjoOa4801HM/W3+R7EgirnaLyhcx0u6ybtqevWUxv
au4EUUMQN6aOq8UGEahm2LYJlDj0xgn2G2V+zxra7iOJwBFYOnbgzQj2uNCeWJONCwl4CU0q
afX3v7wYBfbGLj/YFfr5+QxcATydgROBp6NxxdM39qnPappfndX7/27e8k1/Gi+4JTz/yVu3
CL0GhFu3HEs8ijfK78obpnO4N7RxdAbe2AayzhfUImdsj9s+lg+l0mh7lZwvZqopO0liCyps
ZO9HGNUQWowAYUDYuXkDCDs/nwJhx+QNIOwcfPoMFTluVGShgR9nzOVPqMh7vug9ydHbVZHj
HRVZQEWGigzVC6rXu1S9gKeTdCLwdDSugIoMFfl44xEV+PEkalTg5+FTIOyYvAGEnZ9PgbBj
8gYQdn4+BcKOyRtA2Dn4dPPsl+b0zmELAA6+34jFzSDYLO2Lb2X8Wa1Mqoq0rsjLzi1AmNAG
hTCH7mi10vYqVZGnHusVGPqluaZte7VU1zmaM/0OkcLlVkc3WN+iZHf5BJ0XAYRFrVjU2hi+
o0WtWI3XfguC1XjHsWH5MyR6IAlIApLOBkm4Jwe2OuhY4p4cuCcHIAvIauSdF7p37HP2G7a4
qXYNO1gC0noWEhAkIEhAkIBQuKJwReF64oUrkAQkAUmQgFBPnlg9+frAhAQEyAKyxyQBPWer
fEhAkIAgAUECggSEwhWFKwpXIAlIApIgAaGePLl68vWBCQkIkAVkj0YCEubQs/UyjH7jNcgw
kGEgw0CGQfGI4hHFI5AEJAFJ54ckyDCo6SDDQIYBZAHZI16MJUxCyx71RjuoUG+g3kC9gXqD
mhM1J2pOIAlIApLOD0lQb1AKQr2BegPIArJHvI4K6g3UG6g3UG9Qc6LmRM154jUnkAQkAUlQ
b1AKnlgp+PrAhHoDyAKyfxyy/wMt47cXvE0BAA==
--------------010409020705090508090802--
8 years, 3 months
Re: (ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by leo＠yuriev.ru
Program terminated with signal 11, Segmentation fault.
#0 0x00007ffda90d607b in ?? () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffda90d607b in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x000000000047fb7c in do_syncrep2 (op=0x7ffd68cf5740, si=0x1681d00)
at syncrepl.c:893
#2 0x0000000000483903 in do_syncrepl (ctx=<optimised out>,
arg=0x1682090) at syncrepl.c:1551
#3 0x00000000004250e8 in connection_input (cri=<optimised out>,
conn=<optimised out>) at connection.c:1732
#4 connection_read (cri=<optimised out>, s=<optimised out>) at
connection.c:1460
#5 connection_read_thread (ctx=0x7ffd68cf5bd0, argv=0x25) at
connection.c:1284
#6 0x00007ffda9777d22 in ldap_int_thread_pool_wrapper (xpool=0x1635090)
at tpool.c:688
#7 0x00007ffda93420a5 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#8 0x00007ffda906f84d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) frame 1
#1 0x000000000047fb7c in do_syncrep2 (op=0x7ffd68cf5740, si=0x1681d00)
at syncrepl.c:893
893 Debug( LDAP_DEBUG_ANY, "do_syncrep2: %s malformed
message (%s)\n",
(gdb) info locals
syncUUID = {{bv_len = 16, bv_val = 0x7ffd543a98b7
"\177]\201N\f'\020\064\235\255Չ\v\247\331", <incomplete sequence \351>},
{bv_len = 128849018894, bv_val = 0x7ffd68cf4ec4 "\375\177"}}
cookie = {bv_len = 60, bv_val = 0x7ffd543a98c9
"rid=002,sid=002,csn=20141129152404.971057Z#000000#001#000000"}
rctrls = 0x7ffd543f7130
bdn = {bv_len = 33, bv_val = 0x7ffd543706f7
"cn=modem,uid=4711,dc=ngdr,dc=ldap"}
si_tag = 1
syncstate = 3
retdata = 0x7ffd68cf57c0
retoid = 0xb <Address 0xb out of bounds>
syncUUIDs = 0x7ffd68cf57c0
len = 60
berbuf = {
buffer = "\002\000\001", '\000' <repeats 29 times>,
"\260\230:T\375\177\000\000\005\231:T\375\177\000\000\005\231:T\375\177", '\000'
<repeats 34 times>,
"i\215\064\251\375\177\000\000\000\000\000\000\000\000\000\000x\021V\251\375\177\000\000\340\022h\001",
'\000' <repeats 36 times>, "\060S\317h\375\177\000\000Q\020", '\000'
<repeats 14 times>, "X", '\000' <repeats 15 times>,
"\300U\317h\375\177\000\000"..., ialign = 65538, lalign = 65538, falign
= 9.18382988e-41, dalign = 3.2380074297143616e-319, palign = 0x10002
<Address 0x10002 out of bounds>}
msg = 0x7ffd543f3c50
syncCookie = {ctxcsn = 0x7ffd543f3f10, sids = 0x7ffd542eb890, numcsns =
1, rid = 2, octet_str = {bv_len = 60, bv_val = 0x7ffd5440f6d0
"rid=002,sid=002,csn=20141129152404.971057Z#000000#001#000000"},
sid = 2, sc_next = {stqe_next = 0x0}}
syncCookie_req = {ctxcsn = 0x7ffd5440f380, sids = 0x7ffd542ebae0,
numcsns = 5, rid = 2, octet_str = {bv_len = 224,
bv_val = 0x7ffd543714e0
"rid=002,sid=004,csn=20141129152404.970764Z#000000#001#000000;20141129151341.491595Z#000000#002#000000;20141129151341.507685Z#000000#003#000000;20141129151341.523508Z#000000#004#000000;20141129151341.5"...},
sid = 4, sc_next = {stqe_next = 0x0}}
rc = 100
err = 0
modlist = 0x0
m = 32765
tout = {tv_sec = 0, tv_usec = 0}
refreshDeletes = 0
empty = "empty"
(gdb) p *si
$1 = {si_next = 0x1682200, si_be = 0x1680940, si_wbe = 0x1680940, si_re
= 0x1682090, si_rid = 2, si_ridtxt = "rid=002", si_bindconf = {sb_uri =
{bv_len = 22, bv_val = 0x16816b0 "ldap://10.2.0.1:11113/"},
sb_version = 3, sb_tls = 0, sb_method = 128, sb_timeout_api = 10,
sb_timeout_net = 0, sb_binddn = {bv_len = 19, bv_val = 0x1681690
"uid=replica,dc=ldap"}, sb_cred = {bv_len = 3,
bv_val = 0x1682110 "xyz"}, sb_saslmech = {bv_len = 0, bv_val =
0x0}, sb_secprops = 0x0, sb_realm = {bv_len = 0, bv_val = 0x0},
sb_authcId = {bv_len = 0, bv_val = 0x0}, sb_authzId = {bv_len = 0,
bv_val = 0x0}, sb_keepalive = {sk_idle = 1, sk_probes = 1,
sk_interval = 1}, sb_tls_ctx = 0x0, sb_tls_cert = 0x0, sb_tls_key = 0x0,
sb_tls_cacert = 0x0, sb_tls_cacertdir = 0x0,
sb_tls_reqcert = 0x0, sb_tls_cipher_suite = 0x0,
sb_tls_protocol_min = 0x0, sb_tls_crlcheck = 0x0, sb_tls_do_init = 0},
si_base = {bv_len = 15, bv_val = 0x1682030 "dc=ngdr,dc=ldap"},
si_logbase = {
bv_len = 0, bv_val = 0x0}, si_filterstr = {bv_len = 15, bv_val =
0x1681200 "(objectclass=*)"}, si_filter = 0x1682010, si_logfilterstr =
{bv_len = 0, bv_val = 0x0}, si_contextdn = {bv_len = 7,
bv_val = 0x16811e0 "dc=ldap"}, si_scope = 2, si_attrsonly = 0,
si_anfile = 0x0, si_anlist = 0x1682130, si_exanlist = 0x1681fc0,
si_attrs = 0x169f0f0, si_exattrs = 0x0, si_allattrs = 1,
si_allopattrs = 1, si_schemachecking = 0, si_type = 3, si_ctype = 3,
si_interval = 60, si_retryinterval = 0x1681ff0, si_retrynum_init =
0x1682070, si_retrynum = 0x1682050, si_syncCookie = {
ctxcsn = 0x7ffd7c1d87d0, sids = 0x7ffd54411150, numcsns = 5, rid =
2, octet_str = {bv_len = 224,
bv_val = 0x7ffd543f57f0
"rid=002,sid=004,csn=20141129152404.970905Z#000000#001#000000;20141129151341.491595Z#000000#002#000000;20141129151341.507685Z#000000#003#000000;20141129151341.523508Z#000000#004#000000;20141129151341.5"...},
sid = 4, sc_next = {stqe_next = 0x0}}, si_cookieState = 0x1681c70,
si_cookieAge = 1539004, si_manageDSAit = 0, si_slimit = 0, si_tlimit =
0, si_refreshDelete = 0,
si_refreshPresent = 1, si_refreshDone = 1, si_syncdata = 0,
si_logstate = 0, si_got = 269715, si_strict_refresh = 0, si_too_old = 0,
si_msgid = 2, si_presentlist = 0x7ffd7c58e050,
si_ld = 0x7ffd7c718210, si_conn = 0x7ffda9a365d0, si_nonpresentlist =
{lh_first = 0x0}, si_rewrite = 0x0, si_suffixm = {bv_len = 0, bv_val =
0x0}, si_mutex = {__data = {__lock = 1, __count = 0,
__owner = 13107, __nusers = 1, __kind = 0, __spins = 0, __elision
= 0, __list = {__prev = 0x0, __next = 0x0}},
__size = "\001\000\000\000\000\000\000\000\063\063\000\000\001",
'\000' <repeats 26 times>, __align = 1}}
(gdb) p si->si_ridtxt
$2 = "rid=002"
(gdb) p (void*)si->si_ridtxt
$3 = (void *) 0x1681d24
8 years, 3 months
Re: (ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by leo＠yuriev.ru
Program terminated with signal 11, Segmentation fault.
#0 0x000000000047fb64 in do_syncrep2 (op=0x7f8c494e7740, si=0x19967e0)
at syncrepl.c:892
892 bdn.bv_val[bdn.bv_len] = '\0';
(gdb) bt
#0 0x000000000047fb64 in do_syncrep2 (op=0x7f8c494e7740, si=0x19967e0)
at syncrepl.c:892
#1 0x0000000000483903 in do_syncrepl (ctx=<optimised out>,
arg=0x1996410) at syncrepl.c:1551
#2 0x00000000004250e8 in connection_input (cri=<optimised out>,
conn=<optimised out>) at connection.c:1732
#3 connection_read (cri=<optimised out>, s=<optimised out>) at
connection.c:1460
#4 connection_read_thread (ctx=0x7f8c494e7bd0, argv=0x21) at
connection.c:1284
#5 0x00007f8c6bb5bd22 in ldap_int_thread_pool_wrapper (xpool=0x194a090)
at tpool.c:688
#6 0x00007f8c6b7260a5 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#7 0x00007f8c6b45384d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) info locals
syncUUID = {{bv_len = 16, bv_val = 0x7f8c40106ad7
"\023Z,P\f!\020\064\223e\177\031\357Ζ:"}, {bv_len = 0, bv_val = 0x0}}
cookie = {bv_len = 60, bv_val = 0x7f8c40106ae9
"rid=001,sid=001,csn=20141129143806.208485Z#000000#001#000000"}
rctrls = 0x7f8c40104ad0
bdn = {bv_len = 34, bv_val = 0x7f8c40105209
"cn=tablet,uid=1756,dc=ngdr,dc=ldap"}
si_tag = 1
syncstate = 1
retdata = 0x19eb878
retoid = 0x0
syncUUIDs = 0x0
len = 60
berbuf = {
buffer = "\002\000\001", '\000' <repeats 29 times>,
"\320j\020@\214\177\000\000%k\020@\214\177\000\000%k\020@\214\177",
'\000' <repeats 34 times>,
"i\315rk\214\177\000\000\000\000\000\000\000\000\000\000xQ\224k\214\177\000\000\340b\231\001",
'\000' <repeats 36 times>,
"\a\000\000\000\000\000\000\000\020c\231\001\000\000\000\000\a\000\000\000\000\000\000\000@b\231\001\000\000\000\000\006\000\000\000\000\000\000\000\300b\231\001\000\000\000\000"...,
ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign =
3.2380074297143616e-319, palign = 0x10002 <Address 0x10002 out of bounds>}
msg = 0x7f8c40102be0
syncCookie = {ctxcsn = 0x7f8c401038f0, sids = 0x7f8c40105890, numcsns =
1, rid = 1, octet_str = {bv_len = 60, bv_val = 0x7f8c40104e10
"rid=001,sid=001,csn=20141129143806.208485Z#000000#001#000000"},
sid = 1, sc_next = {stqe_next = 0x0}}
syncCookie_req = {ctxcsn = 0x7f8c40105e80, sids = 0x7f8c40105020,
numcsns = 1, rid = 1, octet_str = {bv_len = 60, bv_val = 0x7f8c40105760
"rid=001,sid=001,csn=20141129143806.208232Z#000000#001#000000"},
sid = 1, sc_next = {stqe_next = 0x0}}
rc = 100
err = 0
modlist = 0x7f8c40106a70
m = 32652
tout = {tv_sec = 0, tv_usec = 0}
refreshDeletes = 0
empty = "empty"
(gdb)
8 years, 3 months
Re: (ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by leo＠yuriev.ru
This is a multi-part message in MIME format.
--------------050709090905040108060403
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
A simple testcase is attached.
All activity (add/delete/read) come via first node of 4x cluster.
Unfortunately a lot of time may be required to reproduce a bug
(coredump), from 10 minutes up to 2-3 hours.
Leonid.
--------------050709090905040108060403
Content-Type: application/gzip;
name="testcase-its7968-1.tar.gz"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="testcase-its7968-1.tar.gz"
H4sIAFrZeVQAA+1abXPbNhLOV/FXbGm7phKREiXLvipVL6mtXj3n2h7LuZuOz+OhSEhizRcV
oGK7sf/77QKkSEqy7OSS9DojZGIRwGKxWDz7ApB+5CdW4PnDF1+uNLDs7u7Qr73XbhR/Zdlt
N1/YzZ2m3bbtdsN+0bAbbbvxAhpfUKZZmYrE4QAvgrvVdE/1/0WLF3XAc7uB50y0ePAbc5P9
wBGiAzEfOZH/h5P4cVTuSeI5Us89kVXNczugOHXgnWD8wEmcMzaJhZ/E/E7TaLKp73UdL/Sj
WjYttnRANj0xj+O68TRKcj6cTQLfdWpLF5BRr+Yp/HASsD5zp9xP7tJ1SIlS5toUF3KKtDcx
x9bbuz/U/DhnNPL48rlLytv4CO0Ry0e092X2P4w9Zrlx9Gfav72zQ/bfarfabfxd2/9XLBoC
7T3jhwfw5vDgjaaFPucxJ1TA+dm7nhZjZ+DcgbiL3AmP32vZg+mOmXs9if0oAbvRADvv4SyI
HW9MPZLHn73GdXm8pF7ui7qAp+y/tdeexf/W3h7av91c2//XKdJqCQTAMaBKH1AhI/Y9xrtv
3h4cnGFDcjdhGGyHnInx28g7ZVz4ItEqnCX8rqu34ZUOiR+yeJp07QZcMzZxAv896+p2B//p
WkUwh7vjgSOwbS5yYq+TJFx09Ze1V1gZ+JEXsmQce10VnFWTF3X1JTEfB6D4HosS3wlEl8Jz
hUQJ/NBPutNI/jIPJfD/WGj8s5X/f1D4NAqZ6Sdi77vdv5m2Jcaffw6y8T208eX2j8G+uSvj
/26rTX/I/vca7bX9f42y8U0drauOljnWtJ97Z73upjG58araj4fHB4dn3c0P1PhQHzPOTBE4
E087eHv+dqHLw2RV2z961z/vYY8h2O9gQ7uqbWRttrbRP3p7enXQ+/HdP7r2butvO1qhoaFp
134QOEEA5u8gJ4KQhTLJMJ3Q290xAz+a3sK332IvOhhMOHgI5hBeWt5ghH+T2wTcmDNNS5hI
wLwFqx5PWERewhTxlLusTlHOH005g/9olft7MEY+DgriiAE+dep1/GtlgyzM4mccLKKcY0dM
KiiP6y30YKtkTeKjV4ST097x0QGu9qx3dNU7/sdV82onG06E6FkZymyOHY7MuD/CPXlqTMj4
iOEYNw5DEk6NCr2BFToiYRzMEPSXL1/CL5KQszBOmJlwx732oxEMuBO5Y9heGLcNmLnFS0XW
q6ni2C3OuPn3XNebHxRgHuqC8EQb6HgemM7yjVBZp5B0Xt0K/EH6DLQrSxSK02Y6DZgTEVfc
e9NLO6zC1prmBGOVf9udyYRNLHIGAcL0LnLcIK/TFHktTXYFtni+kE3oHRLfLTMIZUNGMfAG
hdoYa0qk0Llm4DFchwdZNfv1I+QbBI9qc1FhqCHucJ8Jegpws2jDrki6lIfka+4/Z+j8rOG1
5yNaJriJqW0/kKRkXnxYaKy/nB9phHTEhnsYYVguDkciMfViUP1mAkk4GQo4e/tLgWph+cOY
Q4TKQZrUbTy8Bi/WKscnBz3lcmayRHhEudqMMAnJxEyJSEzSc76oWcd7h9cx4M03ke8qtrHE
haJYmL14oIv7dO77nPR+9FrcK5Ddz9B2P9Lhe8i8o0S1TK/hh7lJin2l+fwhXMCmcYMwu4Hv
v/8e9FwhaILmCI9ccPkakjFDBVRIbXyZ2iopJw7fdGEzKgypVCi16+qEE3R7dsPa5FbDsjub
hmFTgVc4rFqt65I4VQClh/ebXK5a5Yb3m/RTXnHxSAE/PHPROMnQx78e+mJtbsKozH92ZfF8
5shaqxweH57Pgpaf3XtqlfpUcBkEKW9EU0YnMIgFPsWALszcNChtRatnuZOrmpuRDDiPub3h
CsHMADtJGPRLfdoW3OTmDyunoghXXM/RYf/8CjX/qms8JWIV8+F4io6+IBBmzA7mv5ZUNv15
/Ym2R/GhBKJoAUQRgggyiuYe9mcUjdZ3jYxCV3Y8RAGG8cr1kDmSZ/GHyuNDEHdQh7PJIWIJ
xrFraLbbVuE/TCc4lLnjWAVFSvsS5NnRQcaPj3C4y9yHyVYj6cl1kS/M0YRzsluWwoYQc3Sy
//aoTQFv80OeNiGCBDTA3AFzTE6C9uNBXw2/5g83Mb9ejbVvFcQQXk/KjfjK8jEJJ1VpatpT
ypjc4RkvWs5bKkQRAMVEF0kKh5TJnaapnex+7qI/bghlsK+Cco4yshQJMq1CA9UxGMyfYZO4
Yag/Ojo6yC7gwRzkjzcgGNppQjtMGoL9OErYbbLfP1aK/mIK0Ix0QxuvIcvJJX5ep5l3O29f
lqJXET5SODMC/d8OuixSKWapeJanC3HLsqTN3VAX/qqFgAm0LP0TPdEKtLlO8tFOHMc86cTT
vTVnoaToouUWaTkQFJiVs5HJEu5uHHniebagnf56/vPJ8VW/t39yfNDHA5bkgUb/UWwwUXPx
MIJOZLuzTetrYsuEkn5EmfcKKwO3mgoNacHEoTz5p+6QTEVM8QxPmKcpSw2potbOQP/JDxiI
O5Q/BDxlTaaJ0KmjpJSVwaRS+TyeX8nkzkx0ERwyCZGoIPd9sGIrTRPnIfd78eby4XlbhqnA
aoZ1j72vR9MgSFlT8vB89guIKC2hBlnagz0FSRAoKXJOe2f7vWNMvgzFaQ5R8FK+QaiDMd/x
am6qqp5KpFoX+JaIP4GtcltMQW5TgQUPLeeHJx1UXHk1D1unyFtF44O8sf+fSP8LXSwuja6f
eY7V9/92c7e1k93/tfbazRfYtLO+//86xQ8nMU/UW/v0mSJoDT0wHhI8PxrVMMMejeQDdyIv
DtHeE471jB4dcA1ioWkbzyraBvwzim8iOSekF0EUTLIX7jJRQl9yoVUu9DzPoiTL1mtwjD70
sqZtLHQ2V3W2VnXu5J2Xz12FNuon8QTF/MkJBNNC5/aUx0MMSCR6G3GtYSUotBUp6pgg43g8
hGHHhwdN5YV9F0MRNsj7xv7+yWnvqv/ux/OzXk8LYvcae2abYh1hg1HVNJc+IoD+dCBS5ob6
uKDa0SpeRNwr21603QH58uRD46E2//qlRmfvbTxUC6S6qBh64esE1MuFXvyYwQno5U8cYYc+
mT2hIugHZ1CfMuiXVckVi0Gtkg3OXWp3I9lslxpFoREusfmBjnsDu7gQN+rSFUBY++QVYTD0
XZZJvkQoyX9OhuacDAnd9yVfUAg1QUEKrfJGbrd6O4b7y4agjvKGYMGwBlcoDP6l+Wn/K6Mg
HjgBSKTWQAEOuVQITZbj/j71OUMQyWsiGktWqKhotCLjLGAYHyQZvfCbckrMFNUFDbpEvdja
AjW20N0Er0k4C/6+inRSNOxJ+J2cQSKUZLe8SNUvlMouL2ziW0aPhXln6CQGzVq9lAKlvC2c
6UoYNBz357JEWIOcqxQrA5Scl54XGVHrUlaqY45Zs8CsuZRZ8zFmzTKzObU20VeRYqX7tSib
NXTkyTBb2xLbkMSwhQnvFjy2cNJ/VXJmty6bKDdv0cmzR195gCOAcZ7utZqEUYeh/+SgK/Fo
Bsrr5GyIjS08ijLXmeK5ipqemloGCgM5VqvVdHXSY57zKVsGZvyPgP8cYJZSYA6e5IiuAi7Y
KCn4G9Rw9SOR3vhEpK/CnFr247B7GmhFDsuwpjjMW9vC6KV7mY5lARRVsQSZ9ILpfYbNIY/D
r4BOJfpKgD5qy09D9HmpwIZGjnjEkiuVH8mtNwhYKWSlnrUKd5Hxsbzf9un1nAQTZgVpBc8q
WuVmTEdaAi53OwrLqlvmD/Qo9cEdH5fYk2rDqGzoxzHICyU88EUYZbANhvE0kh59y5M8nCSO
hEUqoZo0FOL8Snlvzkg6tQALT0EYmgwpd2pR2I8uWU0uKZmrpUaComqoK6DUBBmqHMUCTBT8
2MOELgjuZJ+Am3GMqzs4PFfwULOlKaBUomIhdYdzNmg6effVhSVfjRSUJbdNClewRDLDhV0h
mswmKxsZrjw2mI4MvS/TMHo5WgCUvK5CHJEgM9BWUvOxVOaGtqO6C4lc9TVRpdCebVUR1Y/D
Or0nXAXrXJwyhil7t+Q9nZHuJv34UWLYjRrYzUYVs0/EmtWo0qZROMn37IyNpoHD76hZ1JVx
CRBxyGCi0svHdk7ykRu3ZFdobI6umTy1Yk5MOi2ksVaa2NDQ1LFeSBlw7QGLUmzC5WddhKL9
rMtIQ9ozlkEC0JFBzl72rD9hM6ESLTldgXKsxRMG7T3dy0nmiJlblG/ESLoSVedT1Twn0uwo
M0xlk5e36TJSL1T0gWnYTl0hfVOdGnVqzJP0C2tqVRffuqYW5NPtnq+OiyL3iWHmjujSzY9k
RlEZxyJBDrK7cVkIv3OGfs7vSGa0NPq0DDC57HywH958aOZJJolYg0ysGhBrZfhBdkSjuz3f
Cfw/mJG9FiimqYURFqo0id04uKLDLrmBlMW/emf9w5PjlqIS6K1i5dFl78np+dVx7/zfJ2f/
vDo//KV38u68Bi2yXEUuP4+7oiXMiatidnnH+lPXZUIMp+SPaYxyNSgx+SRcfknwGhBHJX6q
bZL5fwrWUtelKctubXH++fC8UY7PhLXQ8aMFoOVoRTfpu/vyTaERsPcs6GY98gVaDdSc3e0t
wxEuOc+q/Z0A2FLUdG1cVfUQleeMWFVs44kCbWYY0qhfza3QRLvc+rmz9Utnq7+Ncs7gT3e+
ZNAlr9zG6ga9JjDIA1dUyLNLh/tz+WQkDsf4JYMt0VQz4ubziKVXXM1YkmSkq9mWSFvPJ915
FmnqeVdLmxLl5KslXiBfLfUC+WrJZ+SzLbTkK2Uj36a8QW3FXH2+vzVX38nrmXoWWpoLLa2F
lpwPAg+Ofu2AAwHGNXTy8mWZLxAz8vjklT1uHkJ2G/ASmk11Wipg2U6zw1hYEycZWwhrkQhD
p+/wdHW0KlnsglMiQm8aTr7RU055JjdALV6/lolgKQrPMZxzctg3wape3JffYnIR+bZk9XRX
StW53la5ujOrznZkrqE539Cab8h5zAf5yBdj5qHoyqf9hd4irMu6rMu6rMu6rMu6rMu6rMu6
rMu6rMu6rMu6rMu6rMu6rMv/U/kvfCGhtABQAAA=
--------------050709090905040108060403--
8 years, 3 months
Re: (ITS#7968) SIGSEGV shortly after reconnection performed by syncrepl due to synchronization conflicts
by leo＠yuriev.ru
current OPENLDAP_REL_ENG_2_4 (6b26910 Silence compiler warning...) with
merge-in current mdb.master (9a72292 ITS#7961,#7987 Re-fix txn init)
- cluster of 4 node, but on single machine.
- only localback network, no any failures.
- multi-master by config, but all writes come only to first node.
Testcase will be available shortly (config + script).
Core was generated by `/opt/openldap.devel/libexec/slapd -l LOCAL5 -d 0
-s 0 -4 -h ldap://10.4.0.1:1114'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007ff68564d07b in ?? () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ff68564d07b in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1
0x000000000047fb3c in do_syncrep2 (op=0x7ff62f7fd740, si=0x1c74190) at
syncrepl.c:934
#2 0x00000000004838c3 in do_syncrepl (ctx=<optimised out>,
arg=0x1c746d0) at syncrepl.c:1539
#3 0x00000000004250a8 in connection_read_thread (ctx=0x7ff62f7fdbd0,
argv=0x35) at connection.c:1293
#4 0x00007ff685ceecf2 in ldap_int_thread_pool_wrapper (xpool=0x1c27090)
at tpool.c:688
#5 0x00007ff6858b90a5 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#6 0x00007ff6855e684d in clone () from /lib/x86_64-linux-gnu/libc.so.6
> if ( !BER_BVISNULL( &syncCookie.octet_str ) )
> {
> slap_parse_sync_cookie( &syncCookie, NULL );
> if ( syncCookie.ctxcsn ) {
> int i, sid = slap_parse_csn_sid( syncCookie.ctxcsn );
> check_syncprov( op, si );
> for ( i =0; i<si->si_cookieState->cs_num; i++ ) {
> /* new SID */
> if ( sid < si->si_cookieState->cs_sids[i] )
> break;
> if ( si->si_cookieState->cs_sids[i] == sid ) {
syncrepl.c:934 > if ( ber_bvcmp( syncCookie.ctxcsn,
&si->si_cookieState->cs_vals[i] ) <= 0 ) {
> bdn.bv_val[bdn.bv_len] = '\0';
> Debug( LDAP_DEBUG_SYNC, "do_syncrep2: %s CSN too old, ignoring %s (%s)\n",
> si->si_ridtxt, syncCookie.ctxcsn->bv_val, bdn.bv_val );
> ldap_controls_free( rctrls );
> rc = 0;
> si->si_too_old = 1;
> goto done;
> }
> si->si_too_old = 0;
> break;
> }
> }
8 years, 3 months
Re: (ITS#7987) SIGSEGV in LMDB while adding ldap-entries
by leo＠yuriev.ru
(1) ITS#7968 not fixed
- still crash in a few 'favorite' locations
- last time SIGSEGV in do_syncrep2() at "if ( ber_bvcmp(
syncCookie.ctxcsn, &si->si_cookieState->cs_vals[i] ) <= 0 )"
(2) mdb.master seems to be have another bug(s), that also was fixed in
the 'mdb/fixes' branch on git://git.uio.no/u/hbf/openldap.git
- just a cherry-pick of ead57604 fixes this SIGSEGV, but I got a
'implementation specific' error in heavy add/delete test (same as in
the attached testcase).
- but merge 'mdb/fixes' passed my test.
(3) OPENLDAP_REL_ENG_2_4
without merge current mdb.master
and after revert all 4d02c741-related changes (single write txn)
also passed my test.
Leonid.
2014-11-28 22:06 GMT+03:00 Howard Chu <hyc(a)symas.com>:
> leo(a)yuriev.ru wrote:
>>
>> Sure this bug is introduced by the 4d02c741 (Use a single write txn).
>>
>> Could be fixed by:
>> cherry-pick ead57604 (ITS#7961 Re-fix txn init) from
>> git://git.uio.no/u/hbf/openldap.git
>> or
>> revert: d72b2f5d (ITS#7961 fix txn init), 62e4eeb7 (ITS#7943 reinit
>> txn flags), 891e6627 (Plug leak in 4d02c741...) and 4d02c741 (Use a
>> single write txn).
>
>
> Totally fixed? Or are you still seeing the ITS#7968 crash now? So far I've
> run your testcase on the patched source (with ead57604) and gotten no
> crashes.
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
8 years, 3 months
Re: (ITS#7987) SIGSEGV in LMDB while adding ldap-entries
by hyc＠symas.com
leo(a)yuriev.ru wrote:
> Sure this bug is introduced by the 4d02c741 (Use a single write txn).
>
> Could be fixed by:
> cherry-pick ead57604 (ITS#7961 Re-fix txn init) from
> git://git.uio.no/u/hbf/openldap.git
> or
> revert: d72b2f5d (ITS#7961 fix txn init), 62e4eeb7 (ITS#7943 reinit
> txn flags), 891e6627 (Plug leak in 4d02c741...) and 4d02c741 (Use a
> single write txn).
Totally fixed? Or are you still seeing the ITS#7968 crash now? So far
I've run your testcase on the patched source (with ead57604) and gotten
no crashes.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
8 years, 3 months
Re: (ITS#7987) SIGSEGV in LMDB while adding ldap-entries
by leo＠yuriev.ru
Sure this bug is introduced by the 4d02c741 (Use a single write txn).
Could be fixed by:
cherry-pick ead57604 (ITS#7961 Re-fix txn init) from
git://git.uio.no/u/hbf/openldap.git
or
revert: d72b2f5d (ITS#7961 fix txn init), 62e4eeb7 (ITS#7943 reinit
txn flags), 891e6627 (Plug leak in 4d02c741...) and 4d02c741 (Use a
single write txn).
Leonid.
8 years, 3 months
Re: (ITS#7987) SIGSEGV in LMDB while adding ldap-entries
by leo＠yuriev.ru
confirm - this bug seems to be fixed (just merge with 'mdb/fixes' from
git://git.uio.no/u/hbf/openldap.git).
but now is another, like http://www.openldap.org/its/index.cgi/Incoming?id=7968
#3 0x00007f5061836c82 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x00007f5061df4553 in ber_int_sb_write
(sb=sb@entry=0x7f5024105870, buf=0x7f4ff80019d0, len=len@entry=152) at
sockbuf.c:441
#5 0x00007f5061df0b8b in ber_flush2 (sb=0x7f5024105870,
ber=ber@entry=0x7f500cffb330, freeit=freeit@entry=0) at io.c:246
#6 0x0000000000433dce in send_ldap_ber (op=0x7f500cffb7c0,
ber=0x7f500cffb330) at result.c:339
#7 0x00000000004367a6 in slap_send_search_entry (op=0x7f500cffb7c0,
rs=0x7f500cffb5c0) at result.c:1430
#8 0x000000000051077a in syncprov_sendresp (mode=<optimised out>,
so=<optimised out>, opc=<optimised out>, op=<optimised out>) at
syncprov.c:895
#9 syncprov_qplay (so=<optimised out>, op=<optimised out>) at syncprov.c:944
#10 syncprov_qtask (ctx=0x7f4ff81086c8, arg=0x7f4ff8108670) at syncprov.c:1010
#11 0x00007f5062009d12 in ldap_int_thread_pool_wrapper
(xpool=0xd3b070) at tpool.c:688
#12 0x00007f5061bd40a5 in start_thread () from
/lib/x86_64-linux-gnu/libpthread.so.0
#13 0x00007f506190184d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) frame 4
#4 0x00007f5061df4553 in ber_int_sb_write
(sb=sb@entry=0x7f5024105870, buf=0x7f4ff80019d0, len=len@entry=152) at
sockbuf.c:441
441 assert( sb->sb_iod != NULL );
(gdb) info local
ret = <optimised out>
__PRETTY_FUNCTION__ = "ber_int_sb_write"
(gdb) p *sb
$1 = {sb_opts = {lbo_valid = 3, lbo_options = 0, lbo_debug = 16384},
sb_iod = 0x0, sb_fd = 75, sb_max_incoming = 262143,
sb_trans_needs_read = 0, sb_trans_needs_write = 0}
tail xxx.log
54775e18 syncrepl_entry: rid=001 cn=tablet,uid=1040,dc=ngdr,dc=ldap
54775e18 slap_queue_csn: queueing 0x7f5020107890
20141127172334.589180Z#000000#001#000000
54775e18 syncprov_matchops: skipping original sid 001
54775e18 slap_graduate_commit_csn: removing 0x7f502012d740
20141127172334.589180Z#000000#001#000000
54775e18 syncrepl_entry: rid=001 be_delete
cn=tablet,uid=1040,dc=ngdr,dc=ldap (0)
54775e18 slap_queue_csn: queueing 0x7f5020107890
20141127172334.589180Z#000000#001#000000
54775e18 slap_graduate_commit_csn: removing 0x7f5020108020
20141127172334.589180Z#000000#001#000000
54775e18 do_syncrep2: rid=001
cookie=rid=001,sid=001,csn=20141127172334.589328Z#000000#001#000000
54775e18 syncrepl_message_to_entry: rid=001 DN:
cn=tablet,uid=1793,dc=ngdr,dc=ldap, UUID:
dc4d5d30-0aa5-1034-8ff5-a9ee5a485f53
slapd: sockbuf.c:441: ber_int_sb_write: Assertion `sb->sb_iod !=
((void *)0)' failed.
2014-11-27 13:07 GMT+03:00 Hallvard Breien Furuseth <h.b.furuseth(a)usit.uio.no>:
> On 11/27/2014 10:26 AM, leo(a)yuriev.ru wrote:
>>
>> # possible first bad commit:
>> [4d02c741b120786df1b87ee9ed49c1d3f9bc7522] Use a single write txn
>
>
> Partly fixed in mdb.master, but I think branch "mdb/fixes"
> in <git://git.uio.no/u/hbf/openldap.git> is needed too.
> That is, the "[Untested] ITS#7961 Re-fix txn init." commit.
> Both branches have some remaining problems, however.
>
> --
> Hallvard
8 years, 4 months
