openldap-bugs

openldap-bugs@openldap.org

20960 discussions

Re: (ITS#8436) slapadd hang in bdb_tool_entry_close / ldap_pvt_thread_cond_wait
by Mark.Bannister＠morganstanley.com 06 Jun '16

06 Jun '16

--_000_814F4E458AA9FF4E89CF1A9EDA0DE2A9337A72BEOZWEX0209N2msad_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable The full details of the bug seem to have been truncated. See: http://www.openldap.org/lists/openldap-technical/201605/msg00103.html and: http://www.openldap.org/lists/openldap-technical/201605/msg00104.html ... for the conclusion. Best regards, Mark. ________________________________ NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinion= s or views contained herein are not intended to be, and do not constitute, = advice within the meaning of Section 975 of the Dodd-Frank Wall Street Refo= rm and Consumer Protection Act. If you have received this communication in = error, please destroy all electronic and paper copies; do not disclose, use= or act upon the information; and notify the sender immediately. Mistransmi= ssion is not intended to waive confidentiality or privilege. Morgan Stanley= reserves the right, to the extent permitted under applicable law, to monit= or electronic communications. This message is subject to terms available at= the following link: http://www.morganstanley.com/disclaimers If you cannot= access these links, please notify us by reply message and we will send the= contents to you. By messaging with Morgan Stanley you consent to the foreg= oing. --_000_814F4E458AA9FF4E89CF1A9EDA0DE2A9337A72BEOZWEX0209N2msad_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <HTML xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr= osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:= //www.w3.org/TR/REC-html40"> <HEAD> <STYLE type=3Dtext/css>P.2c7647fc-579c-42c7-9ec3-f99ab79291c4 { MARGIN: 0cm 0cm 0pt } LI.2c7647fc-579c-42c7-9ec3-f99ab79291c4 { MARGIN: 0cm 0cm 0pt } DIV.2c7647fc-579c-42c7-9ec3-f99ab79291c4 { MARGIN: 0cm 0cm 0pt } TABLE.2c7647fc-579c-42c7-9ec3-f99ab79291c4Table { MARGIN: 0cm 0cm 0pt } DIV.Section1 { page: Section1 } </STYLE> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"= /> <meta name=3D"Generator" content=3D"Microsoft Word 14 (filtered medium)" /> <style></style> </HEAD> <BODY lang=3D"EN-GB" link=3D"blue" vlink=3D"purple"> <div class=3D"WordSection1"> The = full details of the bug seem to have been truncated.<o:p></o:p> <o:p= > </o:p> See:= <o:p></o:p> <a h= ref=3D"http://www.openldap.org/lists/openldap-technical/201605/msg00103.htm= l">http://www.openldap.org/lists/openldap-technical/201605/msg00103.html</a= ><o:p></o:p> <o:p= > </o:p> and:= <o:p></o:p> <a h= ref=3D"http://www.openldap.org/lists/openldap-technical/201605/msg00104.htm= l">http://www.openldap.org/lists/openldap-technical/201605/msg00104.html</a= ><o:p></o:p> <o:p= > </o:p> ... = for the conclusion.<o:p></o:p> <o:p= > </o:p> Best= regards,<o:p></o:p> Mark= .<o:p></o:p> </div> <HR id=3DHR1 /> = NOTICE:=20 Morgan Stanley is not acting as a municipal advisor and the opinions or vie= ws=20 contained herein are not intended to be, and do not constitute, advice with= in=20 the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consume= r=20 Protection Act. If you have received this communication in error, please de= stroy=20 all electronic and paper copies; do not disclose, use or act upon the=20 information; and notify the sender immediately. Mistransmission is not inte= nded=20 to waive confidentiality or privilege. Morgan Stanley reserves the right, t= o the=20 extent permitted under applicable law, to monitor electronic communications= .=20 This message is subject to terms available at the following link: <A style= =3D"FONT-FAMILY: Arial; COLOR: #808080; FONT-SIZE: 7.5pt" href=3D"http://ww= w.morganstanley.com/disclaimers">http://www.morganstanley.com/disclaimers</= A>=20 If you cannot access these links, please notify us by reply message and we = will=20 send the contents to you. By messaging with Morgan Stanley you consent to t= he=20 foregoing. </BODY> </HTML> --_000_814F4E458AA9FF4E89CF1A9EDA0DE2A9337A72BEOZWEX0209N2msad_--

1 0

(ITS#8436) slapadd hang in bdb_tool_entry_close / ldap_pvt_thread_cond_wait
by Mark.Bannister＠morganstanley.com 06 Jun '16

06 Jun '16

Full_Name: Mark Bannister Version: 2.4.30 OS: Solaris 11.3 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (205.228.82.172) Summary of this thread: http://www.openldap.org/lists/openldap-technical/201605/msg00079.html slapadd process on Solaris 11 was hanging for many days (slapd 2.4.30) with the following stack trace: # pstack /var/tmp/cores/core.slapd.0.16189.1463492837 core '/var/tmp/cores/core.slapd.0.16189.1463492837' of 16189: /usr/sbin/slapadd -q -f /etc/openldap/slapd.conf ------------ lwp# 1 / thread# 1 --------------- 07ff7af9 lwp_park (0, 0, 0) 07ff1193 cond_wait_queue (82ed048, 82ed020, 0, 7ff16b0) + 63 07ff1728 __cond_wait (82ed048, 82ed020, f5a685c8, 7ff1771) + 89 07ff177f cond_wait (82ed048, 82ed020, f5a685f8, 7ff17b7) + 27 07ff17cc pthread_cond_wait (82ed048, 82ed020, f5a68618, 7e56137) + 24 07e5614c ldap_pvt_thread_cond_wait (82ed048, 82ed020, f5a68648, 81b7e29) + 24 081b7e7a bdb_tool_entry_close (8e9d398, ffffffff, f5a68698, 82ecc38) + 62 081a38ec slapadd (4, f5a68bbc, f5a68bbc, 80dccb1) + cb8 080dcd2c main (4, f5a68bbc, f5a68bd0, ef60f968) + ac 080dc86d _start (4, f5a68c8e, f5a68ca0, f5a68ca3, f5a68ca6, 0) + 7d ------------ lwp# 2 / thread# 2 --------------- 07ff7af9 lwp_park (0, 0, 0) 07ff1193 cond_wait_queue (8e6b0e4, 8e6b0cc, 0, 7ff16b0) + 63 07ff1728 __cond_wait (8e6b0e4, 8e6b0cc, d7dffdb8, 7ff1771) + 89 07ff177f cond_wait (8e6b0e4, 8e6b0cc, 8e6b0c8, 7ff17b7) + 27 07ff17cc pthread_cond_wait (8e6b0e4, 8e6b0cc, 0, 7e56137) + 24 07e5614c ldap_pvt_thread_cond_wait (8e6b0e4, 8e6b0cc, d7dfffc8, 7e54bfc) + 24 07e54d66 ldap_int_thread_pool_wrapper (8e6b0c8, 803a000, d7dfffe8, 7ff7769) + 242 07ff77bc _thrp_setup (ee1e0240) + 9d 07ff7aa0 _lwp_start (ee1e0240, 0, 0, 0, 0, 0) Had to examine disassembly output as Oracle stripped the debug symbols: > ::walk thread | ::findstack -v stack pointer for thread 1: f5a68538 [ f5a68538 libc_hwcap1.so.1`__lwp_park+0x19() ] f5a68568 libc_hwcap1.so.1`cond_wait_queue+0x63(82ed048, 82ed020, 0, 7ff16b0) f5a685a8 libc_hwcap1.so.1`__cond_wait+0x89(82ed048, 82ed020, f5a685c8, 7ff1771) f5a685c8 libc_hwcap1.so.1`cond_wait+0x27(82ed048, 82ed020, f5a685f8, 7ff17b7) f5a685e8 libc_hwcap1.so.1`pthread_cond_wait+0x24(82ed048, 82ed020, f5a68618, 7e56137) f5a68608 libldap_r-2.4.so.2.8.3`ldap_pvt_thread_cond_wait+0x24(82ed048, 82ed020, f5a68648, 81b7e29) f5a686 b bdb_tool_entry_close+0x62(8e9d398, ffffffff, f5a68698, 82ecc38) f5a68ae8 slapadd+0xcb8(4, f5a68bbc, f5a68bbc, 80dccb1) f5a68b88 main+0xac(4, f5a68bbc, f5a68bd0, ef60f968) f5a68bb0 _start+0x7d(4, f5a68c8e, f5a68ca0, f5a68ca3, f5a68ca6, 0) The offending ldap_pvt_thread_cond_wait(), which will never be unblocked because no other thread will do it, is in bdb_tool_entry_close+0x62. Checking the disassembly: > bdb_tool_entry_close::dis bdb_tool_entry_close: pushl %ebp bdb_tool_entry_cle%e+1: movl %esp,%ebp bdb_tool_entry_close+3: pushl %ebx bdb_tool_entry_close+4: pushl %esi bdb_tool_entry_close+5: pushl %edi bdb_tool_entry_close+6: subl $0x1c,%p%p bdb_tool_entry_close+9: andl $0xfffffff0,%esp bdb_tool_entry_close+0xc: call +0x0 <bdb_tool_entry_close+0x11> bdb_tool_entry_close+0x11: popl %ebx bdb_tool_entry_close+0x12: addl $0x1171d7,%ebx bdb_tool_entry_close+0x18: cmpl $0x0,0x1dfa4(%ebx) bdb_tool_entry_close+0x1f: je +0x17e <bdb_tool_entry_close+0x1a3> ... we were at 0x62, so we didn't jump to 0x17e, checking source code: int bdb_tool_entry_close( BackendDB *be ) { if ( bdb_tool_info ) { ... so bdb_tool_info was not a NULL pointer. bdb_tool_entry_close+0x25: movl 0x2dc(%ebx),%eax bdb_tool_entry_close+0x2b: movl $0x1,(%eax) bdb_tool_entry_close+0x31: subl $0xc,%esp bdb_tool_entry_close+0x34: leal 0x1e020(%ebx),%ea0D0D bdb_tool_entry_close+0x3a: pushl %eax bdb_tool_entry_close+0x3b: call -0xdcac0 <PLT=libldap_r-2.4.so.2.8.3`ldap_pvt_thread_mutex_lock> bdb_tool_entry_close+0x40: addl $0x10,%esp bdb_tool_entry_close+0x43: cmpl $0x0,0x1dfb8(%ebx) bdb_tool_entry_close+0x4a: jne +0x22 <bdb_tool_entry_close+0x6e> bdb_tool_entry_close+0x4c: leal 0x1e048(%ebx),%esi bdb_tool_entry_close+0x52: leal 0x1e020(%ebx),%edi bdb_tool_entry_close+0x58: subl $0x8,%esp bdb_tool_entry_close+0x5b: pushl %edi bdb_tool_entry_close+0x5c: pushl %esi bdb_tool_entry_close+0x5d: call -0xdc882 <PLT=libldap_r-2.4.so.2.8.3`ldap_pvt_thread_cond_wait> bdb_tool_entry_close+0x62: addl $0x10,%esp This section of disassembly seems to match up quite nicely with the next section of code: slapd_shutdown = 1; #ifdef USE_TRICKLE ldap_pvt_thread_mutex_lock( &bdb_tool_trickle_mutex ); /* trickle thread may not have started yet */ while ( !bdb_tool_trickle_active ) ldap_pvt_thread_cond_wait( &bdb_tool_trickle_cond_end, &bdb_tool_trickle_mutex ); So bdb_tool_trickle_actives 0%0, and we are blocking on the condition variable bdb_tool_trickle_cond_end. The corresponding call to ldap_pvt_thread_cond_signal occurs in bdb_tool_trickle_task(). The trickle task function looks like this: ldap_pvt_thread_mutex_lock( &bdb_tool_trickle_mutex ); bdb_tool_trickle_active = 1; ldap_pvt_thread_cond_signal( &bdb_tool_trickle_cond_end ); while ( 1 ) { ldap_pvt_thread_cond_wait( &bdb_tool_trickle_cond, &bdb_tool_trickle_mutex ); if ( slapd_shutdown ) break; env->memp_trickle( env, 30, &wrote ); } bdb_tool_trickle_active = 0; ldap_pvt_thread_cond_signal( &bdb_tool_trickle_cond_end %%3; ldap_pvt_thread_mutex_unlock( &bdb_tool_trickle_mutex ); The cond_signal calls are all contained within one big mutex_lock. So could our thread have reached its mutex_lock *after* the very last invocation of the trickle task function, then blocked on its mutex_lock until the trickle task function returned, allowing our thread to reach a cond_wait which will never get a signal? There seems to be a race condition here. The trickle task is a thread in the pool, submitted by bdb_tool_entry_open():

1 0

Re: (ITS#8339) LMDB robust mutex fix
by openldap＠gulag.ch 06 Jun '16

06 Jun '16

On 06/02/16 10:15 PM, Howard Chu wrote: >> >> There is a slight inconsistency if subsequent calls after a successfull >> pthread_mutexattr_init() fail (in the ORed if statement). This might >> result in a memory leak as the mutex attributes are not destroyed. But >> this is not related to the topic of this ITS. > > Feel free to submit another ITS for that, thanks. Has just been fixed by Hallvard: http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=c4c7… Thanks!

1 0

(ITS#8435) Uninitialized field slap_callback.sc_writewait
by h.b.furuseth＠usit.uio.no 06 Jun '16

06 Jun '16

Full_Name: Hallvard B Furuseth Version: 2.4.40 and later OS: URL: Submission from: (NULL) (81.191.45.31) Submitted by: hallvard OpenLDAP 2.4.40 (2014-09-20) introduced added field sc_writewait to slap_callback. It is not always initialized. That breaks sending LDAP responses in slapd/result.c. Typically a crash. Third-party modules written before 2.4.40 break if they use slap_callback variables without initializers (instead assigning to each field), or if the initializer sets sc_private = non-NULL. It gets put in sc_writewait instead, leaving sc_private=NULL. (Initializers in C90-compatible code like OpenLDAP normally set sc_private=NULL: sc_private is rarely a compile-time constant.) Fixes: If we move sc_writewait behind sc_private, that at least fixes non-NULL sc_private initializers in third-party modules. But instead that can break such modules written after 2.4.40. And we end up with 3 different ways to initialize slap_callback: Before 2.4.40, after, and after 2.4.45. I have not found a way to code it so gcc compilation will fail if it is done wrong. Draft branch "writewait" in my repo initializes the cases I found: Syncrepl, backover, slapi, some overlays. I did not check if this is the correct fix. ITS#8428 fixed another case. However... For the sake of third-party modules, I suppose the best fix in OpenLDAP 2.4 is to remove slap_callback.sc_writewait if that is feasible. Create a separate variable field at the end of some struct, which back-mdb callbacks should maintain and which result.c calls. Somewhere which has an existing bitflag variable which can get a "the writewait field is valid" bit. But I haven't tried to figure out the details of that either. Whatever we do, this time document the mess in the release notes of this and some future releases. And add a few macros to decide which way to initialize. I suggest to document that code which does not set sc_writewait, should normally use an initializer with at most 4 fields where sc_private is NULL, and set sc_private "by hand" later if it should not be NULL. Which is one reason not to use my branch, let OpenLDAP show what code should look like instead.

1 0

(ITS#8434) Config entry cleanup on Add should be run unconditionally
by okuznik＠symas.com 04 Jun '16

04 Jun '16

Full_Name: Ondrej Kuznik Version: master OS: URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (2a02:c7f:221f:c00:a3e:8eff:fe52:dac5) With cn=config the ConfigLDAPadd function might have to allocate resources that are needed by the time the attributes/config directives are being processed since there is no indication which directives there are and what order they come in. If any of the config values fail validation, the configuration code has to react and clean up - which is possible using ca->cleanup. While Modify handling (config_modify_internal) already calls ca->cleanup in each case, this is not true for config_add_internal, there it is only run on success.

1 0

Re: (ITS#8432) Never ending operation modifications with 3+ MMR nodes
by quanah＠zimbra.com 04 Jun '16

04 Jun '16

--On Saturday, June 04, 2016 12:06 AM +0000 quanah(a)openldap.org wrote: > Full_Name: Quanah Gibson-Mount > Version: 2.4.44 > OS: Linux 2.6 > URL: ftp://ftp.openldap.org/incoming/ > Submission from: (NULL) (75.111.52.177) The operation in question shows up in the accesslog DB as of the time I wrote the ITS nearly 40,000 times: zimbra@ldap02e:~/data/q$ grep 'entryCSN: 20160603194926.427963Z#000000#001#000000' ldap-accesslog.bak | wc -l 38601 It is simply a replace op: dn: reqStart=20160603225519.561597Z,cn=accesslog objectClass: auditModify structuralObjectClass: auditModify reqStart: 20160603225519.561597Z reqEnd: 20160603225520.995030Z reqType: modify reqSession: 100 reqAuthzID: cn=config reqDN: cn=edge02e.zimbra.com,cn=servers,cn=zimbra reqResult: 0 reqMod: zimbraReverseProxySSLToUpstreamEnabled:= TRUE reqMod: entryCSN:= 20160603194926.427963Z#000000#001#000000 reqMod: modifiersName:= uid=zimbra,cn=admins,cn=zimbra reqMod: modifyTimestamp:= 20160603194926Z reqEntryUUID: f4eca57a-be0f-1035-9026-e742d2d7a3d7 entryUUID: fe4b54ee-be29-1035-9f38-ef67f10345d6 creatorsName: cn=config createTimestamp: 20160603194926Z entryCSN: 20160603194926.427963Z#000000#001#000000 modifiersName: cn=config modifyTimestamp: 20160603194926Z --Quanah -- Quanah Gibson-Mount Platform Architect Manager, Systems Team Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration A division of Synacor, Inc

1 0

(ITS#8433) slapd deadlocks on shutdown in 3+ MMR configuration with active writes
by quanah＠openldap.org 04 Jun '16

04 Jun '16

Full_Name: Quanah Gibson-Mount Version: 2.4.44 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.52.177) Due to ITS#8432 causing there to be endless write activity on all MMR nodes, slapd deadlocks on shutdown: Jun 3 22:55:16 ldap02e zimbramon[8434]: 8434:info: Stopping antispam via zmcontrol Jun 3 22:55:17 ldap02e slapd[16804]: slap_queue_csn: queueing 0x57eb440 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:17 ldap02e slapd[16804]: slap_graduate_commit_csn: removing 0x57eb440 20160603194926.427963Z#000000#001#000000 Jun 3 22:55: 7 ldap02e slapd[16804]: slap_graduate_commit_csn: removing 0xcbdc480 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:17 ldap02e slapd[16804]: syncrepl_message_to_op: rid=100 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 22:55:17 ldap02e slapd[16804]: do_syncrep2: rid=100 cookie=rid=100,sid=003 Jun 3 22:55:17 ldap02e slapd[16804]: slap_queue_csn: queueing 0xcbdc480 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:19 ldap02e slapd[16804]: slap_queue_csn: queueing 0xcbdc700 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:19 ldap02e slapd[16804]: slap_graduate_commit_csn: removing 0xcbdc700 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:19 ldap02e slapd[16804]: slap_graduate_commit_csn: removing 0xcbdc480 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:19 ldap02e slapd[16804]: syncrepl_message_to_op: rid=100 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 22:55:19 ldap02e slapd[16804]: do_syncrep2: rid=100 cookie=rid=100,sid=003 Jun 3 22:55:19 ldap02e slapd[16804SA3A slap_queue_csn: queueing 0xcbdc480 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:19 ldap02e slapd[16804]: conn=1072 fd=20 ACCEPT from IP=10.43.1.112:35783 (IP=10.43.1.112:389) Jun 3 22:55:19 ldap02e slapd[16804]: conn=1072 op=0 EXT oid=1.3.6.1.4.1.1466.20037 Jun 3 22:55:19 ldap02e slapd[16804]: conn=1072 op=0 STARTTLS Jun 3 22:55:19 ldap02e slapd[16804]: conn=1072 op=0 RESULT oid= err=0 etime=0.000104 text= Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 fd=20 TLS established tls_ssf=256 ssf=256 tls_proto=TLSv1.2 tls_cipher=AES256-SHA256 Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 op=1 BIND dn="uid=zimbra,cn=admins,cn=zimbra" method=128 Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 op=1 BIND dn="uid=zimbra,cn=admins,cn=zimbra" mech=SIMPLE ssf=0 Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 op=1 RESULT tag=97 err=0 etime=0.000166 text= Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 op=2 SRCH base="cn=ldap02e.zimbra.com,cn=servers,cn=zimbra" scope=0 deref=0 filter="(objectClass=*)" Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 op=2 SEARCH RESULT tag=101 err=0 etime=0.000415 nentries=1 text= Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 op=3 SRCH base="cn=config,cn=zimbra" scope=0 deref=0 filter="(jejectClass=*)" Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 op=3 SEARCH RESULT tag=101 err=0 etime=0.000987 nentries=1 text= Jun 3 22:55:20 ldap02e slapd[16804]: conn=1072 fd=20 closed (connection lost) Jun 3 22:55:20 ldap02e zimbramon[8434]: 8434:fnfo: Stopping proxy via zmcontrol Jun 3 22:55:20 ldap02e zimbramon[8434]: 8434:info: Stopping memcached via zmcontrol Jun 3 22:55:20 ldap02e zimbramon[8434]: 8434:info: Stopping mailbox via zmcontrol Jun 3 22:55:20 ldap02e zimbramon[8434]: 8434:info: Stopping convertd via zmcontrol Jun 3 22:55:20 ldap02e zimbramon[8434]: 8434:info: Stopping logger via zmcontrol Jun 3 22:55:20 ldap02e zimbramon[8434]: 8434:info: Stopping dnscache via zmcontrol Jun 3 22:55:20 ldap02e zimbramon[8434]: 8434:info: Stopping ldap via zmcontrol Jun 3 22:55:20 ldap02e slapd[16804]: daemon: shutdown requested and initiated. Jun 3 22:55:20 ldap02e slapd[16804]: conn=1009 fd=16 closed (writetimeout) Jun 3 22:55:20 ldap02e slapd[16804]: conn=1001 fd=17 closed (writetimeout) Jun 3 22:55:20 ldap02e slapd[16804]: conn=1002 fd=18 closed (writetimeout) Jun 3 22:55:20 ldap02e slapd[16804]: slapd shutdown: waiting for 3 operations/tasks to finish Jun 3 22:55:20 ldap02e slapd[16804]: slap_queue_csn: queueing 0x57ea9c0 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:20 ldap02e slapd[16804]: slap_graduate_commit_csn: removing 0x57ea9c0 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:20 ldap02e slapd[16804]: slap_graduate_commit_csn: removing 0xcbdc480 20160603194926.427963Z#000000#001#000000 Jun 3 22:55:20 ldap02e slapd[16804]: syncrepl_message_to_op: rid=100 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 22:55:20 ldap02e slapd[16804]: do_syncrepl: rid=100 rc -2 retrying zimbra@ldap02e:~$ date Fri Jun 3 23:16: U UTC 2016 (so we can see about 20 minutes has passed) Full backtrace of all threads: Thread 4 (Thread 0x7f146ac9d700 (LWP 16805)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185 No locals. #1 0x00007f3c70fe8171 in ldap_pvt_thread_cond_wait (cond=0x1fa0038, mutex=0x1fa0010) at thr_posix.c:277 No locals. #2 0x00007f3c70fe63c2 in ldap_pvt_thread_pool_destroy (tpool=0x7618c0 <connection_pool>, run_pending=1) at tpool.c:817 pool = 0x1f763c0 pptr = 0x1f763c0 pq = 0x1fa0000 task = 0x7f3c716a61c8 i = 0 #3 0x0000000000438967 in slapd_daemon_task (ptr=0x1d7bce8) at daemon.c:2829 l = 3 last_idle_check = 1464372736 ebadf = 0 tid = 0 #4 0x00007f3c70552184 in start_thread (arg=0x7f146ac9d700) at pthread_create.c:312 __res = <optimized out> pd = 0x7f146ac9d700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139725667686144, -1093867468031317215, 0, 0, 139725667686848, 139725667686144, 1078920827726146337, 1056426161274956577}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> __PRETTY_FUNCTION__ = "start_thread" #5 0x00007f3c7027f37d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals. Thread 3 (Thread 0x7f1468498700 (LWP 16810)): #0 0x00007f3c705593ad in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f3c70dd2435 in sb_stream_read (sbiod=0x5d36630, buf=0x5a3c057, len=433) at sockbuf.c:490D0D __PRETTY_FUNCTION__ = "sb_stream_read" #2 0x00007f3c70dd2e56 in sb_debug_read (sbiod=0x5d36d20, buf=0x5a3c057, len=433) at sockbuf.c:829 ret = 79 ebuf = "PjIh\024\177\000\000\017\000\000\000\000\000\000\000`\016\000\000\000\000\000\000'\000\000\000\000\000\000\000\250\062H\004\000\000\000\000'\000\000\000\000\000\000\000\240\361H\004\000\000\000\000P\026u\016\000\000\000\000\000\001\000\000\000\000\000\000\001\350#p<\177\000\000\000\000\000\000\000\000\000\0006364\037\000\000\000\000\000\000\006\000\000\000\000\000\000\000hxIh\t\000\000\000\300lIh\024\177\000\000\220jIh\024\177\000" #3 0x00007f3c7101ebd1 in tlso_bio_read (b=0x5a93110, buf=0x5a3c057 "\004\areqType1\b\004\006modify02\004\005reqDN1)\004'uid=gtillman,ou=people,dc=zimbra,dc=com0\201\317\004\006reqMod1\201\304\004:zimbraAuthTokens:- 1988626989|1464372357970|8.7.0_RC1_1601\004\063entryCSN:= 20160527182235.762156Z#000000#003#000000\004."..., len=433) at tls_o.c:721 p = 0x5a6caa0 ret = 79 #4 0x00007f3c6f19988b in BIO_read () from /opt/zimbra/common/lib/libcrypto.so.1.0.0 No symbol table info available. #5 0x00007f3c6f501ffc in ssl3_read_n () from /opt/zimbra/common/lib/libssl.so.1.0.0 No symbol table info available. #6 0x00007f3c6f503ebf in ssl3_read_bytes () from /opt/zimbra/common/lib/libssl.so.1.0.0 No symbol table info available. #7 0x00007f3c6f50033b in ssl3_read () from /opt/zimbra/common/lib/libssl.so.1.0.0 No symbol table info available. #8 0x00007f3c7101f093 in tlso_sb_read (sbiod=0x5d382e0, buf=0xcb4f93f, len=8) at tls_o.c:881 p = 0x5a6caa0 ret = 635655159814 err = 28 __PRETTY_FUNCTION__ = "tlso_sb_read" #9 0x00007f3c70dd2e56 in sb_debug_read (sbiod=0x5d37800, buf=0xcb4f93f, len=8) at sockbuf.c:829 ret = 0 ebuf = "\000\000\000\000\000\000\000\000\300\211Ih\024\177\000\000\000\207Ih\024\177\000\000\000\000\000\000\000\000\000\000pmIh\024\177\000\000v\202\376p<\177\000\000\000\000\000\000\000\000\000\000\240h%h\024\177\000\000]-K\000\000\000\000\000\300{Ih\024\177\000\000\240mIh\000\000\000\000v\202\376p<\177\000\000\000\000\000\000\000\000\000\000\320mIh\024\177\000\000]-K\000\000\000\000\000\300{Ih\024\177\000" #10 0x00007f3c70dd2291 in ber_int_sb_read (sb=0x5d363c0, buf=0xcb4f93f, len=8) at sockbuf.c:423 ret = 0 __PRETTY_FUNCTION__ = "ber_int_sb_read" #11 0x00007f3c70dcebbf in ber_get_next (sb=0x5d363c0, len=0x7f1468496f00, ber=0xcb4f930) at io.c:532 sblen = 8 buf = "\000\000\000\000\000\000" tlen = 0 __PRETTY_FUNCTION__ = "ber_get_next" #12 0x00007f3c70fea22f in try_read1msg (ld=0x5d36d50, msgid=3, all=0, lc=0x3d9b320, result=0x7f1468497190) at result.c:491 ber = 0xcb4f930 newmsg = 0x0 l = 0x0 prev = 0x0 id = 0 tag = 0 len = 0 foundit = 0 lr = 0x0 tmplr = 0x0 dummy_lr = {lr_msgid = 0, lr_status = 0, lr_refcnt = 0, lr_outrefcnt = 0, lr_abandoned = 0, lr_origid = 0, lr_parentcnt = 0, lr_res_msgtype = 0, lr_res_errno = 0, lr_res_error = 0x0, lr_res_matched = 0x0, lr_ber = 0x0, lr_conn = 0x0, lr_dn = {bv_len = 0, bv_val = 0x0}, lr_parent = 0x0, lr_child = 0x0, lr_refnext = 0x0, lr_prev =x0x0, lr_next = 0x0} tmpber = {ber_opts = {lbo_valid = -12608, lbo_options = 474, lbo_debug = 0}, ber_tag = 5341050, ber_len = 0, ber_usertag = 5014619, ber_buf = 0x0, ber_ptr = 0x0, ber_end = 0x0, ber_sos_ptr = 0x574963a4 <error: Cannot access memory at address 0x574963a4>, ber_rwptr = 0x0, ber_memctx = 0x67} rc = 0 refer_cnt = 0 hadref = 0 simple_request = 0 err = 0 lderr = 0 __PRETTY_FUNCTION__ = "try_read1msg"A%A#13 0x00007f3c70fe9e53 in wait4msg (ld=0x5d36d50, msgid=3, all=0, timeout=0x7f14684971f0, result=0x7f1468497190) at result.c:362 lnext = 0x0 serviced = 1 lc_ready = 1 rc = -2 tv = {tv_sec = 0, tv_usec = 0} tv0 = {tv_sec = 0, tv_usec = 0} start_time_tv = {tv_sec = 1464427439, tv_usec = 944352} tvp = 0x7f14684970a0 lc = 0x3d9b320 __PRETTY_FUNCTION__ = "wait4msg" #14 0x00007f3c70fe9574 in ldap_result (ld=0x5d36d50, msgid=3, all=0, timeout=0x7f14684971f0, result=0x7f1468497190) at result.c:117 rc = 0 __PRETTY_FUNCTION__ = "ldap_result" #15 0x00000000004bd6b2 in do_syncrep2 (op=0x7f1468497720, si=0x1dacec0) at syncrepl.c:841 berbuf = { buffer = "\002\000\001", '\000' <repeats 29 times>, "\340\220\234\r\000\000\000\000\b\221\234\r\000\000\000\000\b\221\234\r", '\000' <repeats 29 times>, "\240<\004", '\000' <repeats 20 times>, "\370\201\376p<\177\000\000\300\211Ih\024\177\000\000`\003v\000\000\000\000\000\200sIh\024\175C5C000\000\370\201\376p<\177\000\000\070wIh\024\177\000\000ȓ\237\005\000\000\000\000\360sIh\024\177\000\000W\331\000q<\177\000\000\300sIh\024\177\000\000\034tIh\024\177\000\000\001\000\000\000\001\000\000\000P"..., ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign = 3.2380074297143616e-319, palign = 0x10002 <error: Cannot access memory at address 0x10002>} ber = 0x7f14684972d0 msg = 0x0 syncCookie = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 0, octet_str = {bv_len = 0, bv_val = 0x0}, sid = 0, sc_next = {stqe_next = 0x0}} syncCookie_req = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 101, octet_str = {bv_len = 15, bv_val = 0xcaa5460 "rid=101,sid=004"}, sid = 4, sc_next = {stqe_next D D 0x0}} rc = 0 err = 0 modlist = 0x0 m = 1 tout_p = 0x7f14684971f0 tout = {tv_sec = 0, tv_usec = 0} refreshDeletes = 0 empty = "empty" __PRETTY_FUNCTION__ = "do_syncrep2" #16 0x00000000004bdebc in do_syncrepl (ctx=0x7f1468497bc0, arg=0x1d8b220) at syncrepl.c:1561 rtask = 0x1d8b220 si = 0x1dacec0 conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x4f9b70 ""}, c_peer_name = {bv_len = 0, bv_val = 0x4f9b70 ""}, c_listener = 0x501d00 <dummy_list>, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = {bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data %3{_B__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = { __data = {__lock = 0, __futex = 0,_t_total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber = 0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c__t_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0x0, c_clientarg = 0x0, c_send_ldap_result = 0x452d9a <slap_send_ldap_result>, c_send_search_entry = 0x453c11 <slap_send_search_entry>, c_send_search_reference = 0x455ec7 <slap_send_search_reference>, c_send_ldap_extended = 0x4536fa <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x4539fc <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7f1468497890, o_tag = 102, o_time = 1464427428, o_tincr = 740798, o_bd = 0x1fa7740, o_req_dn = {bv_len = 39, bv_val = 0xdd2ab50 ""}, o_req_ndn = { bv_len = 39, bv_val = 0xbfb1600 "\260a\301\f"}, o_request = {oq_add = {rs_modlist = 0xfc28980, rs_e = 0x0}, oq_bind = {rb_method = 264407424, rb_cred = {bv_len = 0, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val3D3D 0x0}, rb_ssf = 0, rb_mech = {bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0xfc28980}, oq_modify = {rs_mods = { rs_modlist = 0xfc28980, rs_no_opattrs = 0 '\000'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0xfc28980, rs_no_opattrs = 0 '\000'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = 264407424, rs_deref = 0, rs_slimit = 0, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = { rs_msgid = 264407424}, oq_cancel = {rs_msgid = 264407424}, oq_extended = {rs_reqoid = {bv_len = 264407424, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = { rs_extended = {rs_reqoid = {bv_len = 264407424, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = {bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>, "\002", '\000' <repeats 16 times>, o_controls = 0x7f14684979d8, o_authz = {sai_method D D 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 9, bv_val = 0x23e8a60 "cn=config"}, sai_ndn = {bv_len = 9, bv_val = 0x23e8a80 "cn=config"}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7f1468496f70, o_ctrls = 0x0, o_csn = {bv_len = 0, bv_val = 0x0}, o_private = 0x0, o_extra = { slh_first = 0x0}, o_next = {stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 101,hoh_conn = 0x7f1468497460, oh_msgid = 0, oh_protocol = 0, oh_tid = 139725625722624, oh_threadctx = 0x7f1468497bc0, oh_tmpmemctx = 0x43ca000, oh_tmpmfuncs = 0x75e3c0 <slap_sl_mfuncs>, oh_counters = 0x7618e0 <slap_counters>, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>}, ob_controls = {0x0 <repeats 17 times>, 0x7f1468497240, 0x0 <repeats 14 times>}} op = 0x7f1468497720 rc = 0 dostop = 0 s = 19 i = 1 defer = 1 fail = 0 freeinfo = 0 be = 0x1fa7740 #17 0x000000000043c3ae in connection_read_thread (ctx=0x7f1468497bc0, argv=0x13) at connection.c:1273 rc = 0 cri = {op = 0x0, func = 0x4bd8b4 <do_syncrepl>, arg = 0x1d8b220, ctx = 0x7f1468497bc0, nullop = 0} %3= 19 #18 0x00007f3c70fe6a75 in ldap_int_thread_pool_wrapper (xpool=0x1fa0000) at tpool.c:956 pq = 0x1fa0000 pool = 0x1f763c0 task = 0x3db4520 work_list = 0x1fa0070 ctx = {utu_pq = 0x1fa0000, ltu_id = 139725625722624, ltu_key = {{ltk_key = 0x4b2d5d <slap_sl_mem_init>, ltk_data = 0x43ca000, ltk_free = 0x4b2b82 <slap_sl_mem_destroy>}, { ltk_key = 0x1f83300, ltk_data = 0x44ce000, ltk_free = 0x7f3c6bb23f12 <mdb_reader_free>}, {ltk_key = 0x7f3c6bb18ec6 <search_stack>, ltk_data = 0x47d8000, ltk_free = 0x7f3c6bb18ea3 <search_stack_free>}, {ltk_key = 0x7f3c6bb1589f <scope_chunk_get>, ltk_data = 0x44d8000, ltk_free = 0x7f3c6bb15857 <scope_chunk_free>}, {A0A ltk_key = 0x1f82c00, ltk_data = 0x44cfa00, ltk_free = 0x7f3c6bb23f12 <mdb_reader_free>}, {ltk_key = 0x43b82b <conn_counter_init>, ltk_data = 0x1f83c00, ltk_free = 0x43b67d <conn_counter_destroy>}, {ltk_key = 0x457375 <slap_op_free>, ltk_data = 0x2562d00, ltk_free = 0x4572c8 <slap_op_q_destroy>}, {ltk_key = 0x0, ltk_data = 0x602d600, ltk_free = 0x0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x0} <repeats 24 times>}} kctx = 0x0 i = 32 keyslot = 875 hash = 521230187 pool_lock = 0 freeme = 0 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #19 0x00007f3c70552184 in start_thread (arg=0x7f1468498700) at pthread_create.c:312 __res = <optimized out> pd = 0x7f1468498700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139725625722624, -1093867468031317215, 0, 0, 139725625723328, 139725625722624, 1078926329042381601, 1056426161274956577}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> __PRETTY_FUNCTION__ = "start_thread" #20 0x00007f3c7027f37d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals. Thread 2 (Thread 0x7f1467c97700 (LWP 16812)): #0 0x00007f3c705593ad in read () at ../sysdeps/unix/syscall-template.S:81 No locals. #1 0x00007f3c70dd2435 in sb_stream_read (sbiod=0x3dac8b0, buf=0x23cc1a8, len=96) at sockbuf.c:493 __PRETTY_FUNCTION__ = "sb_stream_read" #2 0x00007f3c70dd2e56 in sb_debug_read (sbiod=0x3dac850, buf=0x23cc1a8, len=96) at sockbuf.c:829 ret = 416 ebuf = "PZ\311g\024\177\000\000\017\000\000\000\000\000\000\000`\016\000\000\000\000\000\000'\000\000\000\000\000\000\000\230\264\343\005\000\000\000\000'\000\000\000\000\000\000\000P~\344\005\000\000\000\000@C5Cru\016\000\000\000\000\000\001\000\000\000\000\000\000\001\350#p<\177\000\000\000\000\000\000\000\000\000\000\364\037\000\000\000\000\000\000\006\000\000\000\000\000\000\000hh\311g\n\000\000\000\300\\\311g\024\177\000\000\220Z\311g\024\177\000" #3 0x00007f3c7101ebd1 in tlso_bio_read (b=0x244b2d0, buf=0x23cc1a8 "\240F0D\004\030\061.3.6.1.4.1.4203.1.9.1.2\004(0&\n\001\001\004\020\004@\320&\270\206\020\065\223\244\347B\322ף\327\004\017rid=102,sid=001췫\272\005\324a`\346\343\373\335\062\065\223\a\363\363Sv\003\003\003\003\063.1.9.1.2\004(0&\n\001\001\004\020\003\362\377\004\270\206\020\065\223\236\347B\322ף\327\004\017rid=102,sid=001\027ל{\026g\264\262\210\331U]\330#\020u\241ʂ\206\b\b\b\b\b\b\b\b\bntryCSN:= 20160527182329"..., len=96% a at tls_o.c:721 p = 0x3da51e0 ret = 416 #4 0x00007f3c6f19988b in BIO_read () from /opt/zimbra/common/lib/libcrypto.so.1.0.0 No symbol table info available. #5 0x00007f3c6f501ffc in ssl3_read_n () from /opt/zimbra/common/lib/libssl.so.0.0.0 No symbol table info available. #6 0x00007f3c6f503ebf in ssl3_read_bytes () from /opt/zimbra/common/lib/libssl.so.1.0.0 No symbol table info available. #7 0x00007f3c6f50033b in ssl3_read () from /opt/zimbra/common/lib/libssl.so.1.0.0 No symbol table info available. #8 0x00007f3c7101f093 in tlso_sb_read (sbiod=0x3dac820, buf=0xd93bc7f, len=8) at tls_o.c:881 p = 0x3da51e0 ret = 631360192517 err = 27 __PRETTY_FUNCTION__ = "tlso_sb_read" #9 0x00007f3c70dd2e56 in sb_debug_read (sbiod=0x3dac880, buf=0xd93bc7f, len=8) at sockbuf.c:829 ret = 0 ebuf = "\000\000\000\000\000\000\000\000\300y\311g\024\177\000\000\000w\311g\024\177\000\000\000\000\000\000\000\000\000\000p]\311g\024\177\0%0\000v\202\376p<\177\000\000\000\000\000\000\000\000\000\000\240]\311g\024\177\000\000]-K\000\000\000\000\000\300k\311g\024\177\000\000\240]\311g\000\000\000\000v\202\376p<\177\000\000\000\000\000\000\000\000\000\000\320]\311g\024\177\000\000]-K\000\000\0%0\000\000\300k\311g\024\177\000" #10 0x00007f3c70dd2291 in ber_int_sb_read (sb=0x3dabc80, buf=0xd93bc7f, len=8) at sockbuf.c:423 ret = 0 __PRETTY_FUNCTION__ = "ber_int_sb_read" #11 0x00007f3c70dcebbf in ber_get_next (sb=0x3dabc80, len=7x7f1467c95f00, ber=0xd93bc70) at io.c:532 sblen = 8 buf = "\000\000\000\000\000\000" tlen = 0 __PRETTY_FUNCTION__ = "ber_get_next" #12 0x00007f3c70fea22f in try_read1msg (ld=0x3dabc50, msgid=3, all=0, lc=0x1f7d960, result=0x7f1467c96190) at result.c:491 ber = 0xd93bc70 newmsg = 0x0 l = 0x0 prev = 0x0 id = 0 tag = 0 len = 0 foundit = 0 lr = 0x0 tmplr = 0x0 dummy_lr = {lr_msgid = 0, lr_status = 0, lr_refcnt = 0, lr_outrefcnt = 0, lr_abandoned = 0, lr_origid = 0, lr_parentcnt = 0, lr_res_msgtype = 0, lr_res_errno = 0, lr_res_error = 0x0, lr_res_matched = 0x0, lr_ber = 0x0, lr_conn = 0x0, lr_dn = {bv_len = 0, bv_val = 0x0}, lr_parent = 0x0, lr_child = 0x0, lr_refnext = 0x0, lr_prev = 0x0, lr_next = 0x0} tmpber = {ber_opts = {lbo_valid = -11904, lbo_options = 474, lbo_debug = 0}, ber_tag = 5341050, ber_len = 0, ber_usertag = 5014619, ber_buf = 0x0, ber_ptr = 0x0, ber_end = 0%0, ber_sos_ptr = 0x5748c3eb <error: Cannot access memory at address 0x5748c3eb>, ber_rwptr = 0x0, ber_memctx = 0x67} rc = 0 refer_cnt = 0 hadref = 0 simple_request = 0 err = 0 lderr = 0% __PRETTY_FUNCTION__ = "try_read1msg" #13 0x00007f3c70fe9e53 in wait4msg (ld=0x3dabc50, msgid=3, all=0, timeout=0x7f1467c961f0, result=0x7f1467c96190) at result.c:362 lnext = 0x0 serviced = 1 lc_ready = 1 rc = -2 tv = {tv_sec = 0, tv_usec = 0} tv0 = {tv_sec = 0, tv_usec = 0} start_time_tv = {tv_sec = 1464386543, tv_usec = 695070} tvp = 0x7f1467c960a0 lc = 0x1f7d960 __PRETTY_FUNCTION__ = "wait4msg" #14 0x007f7f3c70fe9574 in ldap_result (ld=0x3dabc50, msgid=3, all=0, timeout=0x7f1467c961f0, result=0x7f1467c96190) at result.c:117 rc = 0 __PRETTY_FUNCTION__ = "ldap_result" #15 0x00000000004bd6b2 in do_syncrep2 (op=0x7f1467c96720, si=0x1dad180) at syncrepl.c:841 berbuf = { buffer = "\002\000\001", '\000' <repeats 29 times>, "\200\060\207\r\000\000\000\000\250\060\207\r\000\000\000\000\250\060\207\r", '\000' <repeats 28 times>, "\200\267<\004", '\000' <repeats 20 times>, "\370\201\376p<\177\000\000\300y\311g\024\177\000\000`\003v\000\000\000\000\000\200c\311g\024\177\000\000\370\201\376p<\177\000\000\070g\311g\024\177\000\000\310y\370\001\000\000\000\000\360c\311g\024\177\000\000W\331\000q<\177\000\000\300c\311g\024\177\000\000\034d\311g\024\177\000\000\001\000\000\000\001\000\000\000"..., ialign = 65538, lalign = 65538, falign = 9.18382988e-41, dalign = 3.2380074297143616e-319, palign = 0x10002 <error: Cannot access memory at address 0x10002>} ber = 0x7f1467c962d0D%0 msg = 0x0 syncCookie = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 0, octet_str = {bv_len = 0, bv_val = 0x0}, sid = 0, sc_next = {stqe_next = 0x0}} syncCookie_req = {ctxcsn = 0x0, sids = 0x0, numcsns = 0, rid = 102, octet_str = {bv_len = 15, bv_val = 0xc2faf30 "rid=102,sid=001"}, sid = 1, sc_next = {stqe_next = 0x0}} rc = 0 err = 0 modlist = 0x0 m = 1 tout_p = 0x7f1467c961f0 tout = {tv_sec = 0,v_v_usec = 0} refreshDeletes = 0 empty = "empty" __PRETTY_FUNCTION__ = "do_syncrep2" #16 0x00000000004bdebc in do_syncrepl (ctx=0x7f1467c96bc0, arg=0x1d8b130) at syncrepl.c:1561 rtask = 0x1d8b130 si = 0x1dad180A0A conn = {c_struct_state = SLAP_C_UNINITIALIZED, c_conn_state = SLAP_C_INVALID, c_conn_idx = -1, c_sd = 0, c_close_reason = 0x0, c_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_sb = 0x0, c_starttime = 0, c_activitytime = 0, c_connid = 18446744073709551615, c_peer_domain = {bv_len = 0, bv_val = 0x4f9b70 ""}, c_peer_name = {bv_len = 0, bv_val = 0x4f9b70 ""}, c_listener = 0x501d00 <dummy_list>, c_sasl_bind_mech = {bv_len = 0, bv_val = 0x0}, c_sasl_dn = {bv_len = 0, bv_val = 0x0}, c_sasl_authz_dn = {bv_len = 0, bv_val = 0x0}, c_authz_backend = 0x0, c_authz_cookie = 0x0, c_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 0, bv_val = 0x0}, sai_ndn = {bv_len = 0, bv_val = 0x0}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, c_protocol = 0, c_ops = {stqh_first = 0x0, stqh_last = 0x0}, c_pending_ops = { stqh_first = 0x0, stqh_last = 0x0}, c_write1_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = { __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write1_cv = {__data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq =%0}, __size = '\000' <repeats 47 times>, __align = 0}, c_write2_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, c_write2_cv = { __data = {__lock = 0, __futex = 0, __total_seq = 0, __wakeup_seq = 0, __woken_seq = 0, __mutex = 0x0, __nwaiters = 0, __broadcast_seq = 0}, __size = '\000' <repeats 47 times>, __align = 0}, c_currentber 0x0x0, c_writers = 0, c_writing = 0 '\000', c_sasl_bind_in_progress = 0 '\000', c_writewaiter = 0 '\000', c_is_tls = 0 '\000', c_needs_tls_accept = 0 '\000', c_sasl_layers = 0 '\000', c_sasl_done = 0 '\000', c_sasl_authctx = 0x0, c_sasl_sockctx = 0x0, c_sasl_extra = 0x0, c_sasl_bindop = 0x0, c_pagedresults_state = {ps_be = 0x0, ps_size = 0, ps_count = 0, ps_cookie = 0, ps_cookieval = {bv_len = 0, bv_val = 0x0}}, c_n_ops_received = 0, c_n_ops_executing = 0, c_n_ops_pending = 0, c_n_ops_completed = 0, c_n_get = 0, c_n_read = 0, c_n_write = 0, c_extensions = 0x0, c_clientfunc = 0x0, c_clientarg = 0x0, c_send_ldap_result = 0x452d9a <slap_send_ldap_result>, c_send_search_entry = 0x453c11 <slap_send_search_entry>, c_send_search_reference = 0x455ec7 <slap_send_search_reference>, c_send_ldap_extended = 0x4536fa <slap_send_ldap_extended>, c_send_ldap_intermediate = 0x4539fc <slap_send_ldap_intermediate>} opbuf = {ob_op = {o_hdr = 0x7f1467c96890, o_tag = 102, o_time = 1464386539, o_tincr = 697802, o_bd = 0x1fa7740, o_req_dn = {bv_len = 39, bv_val = 0xfad64e0 ""}, o_req_ndn = { bv_len = 39, bv_val = 0xfb76000 "`\246\030\f"}, o_request = {oq_add = {rs_modlist = 0xfb25200, rs_e = 0x0}, oq_bind = {rb_method = 263344640, rb_cred = {bv_len = 0, bv_val = 0x0}, rb_edn = {bv_len = 0, bv_val = 0x0}, rb_ssf = 0, rb_mech = {bv_len = 0, bv_val = 0x0}}, oq_compare = {rs_ava = 0xfb25200}, oq_modify = {rs_mods = { rs_modlist = 0xfb25200, rs_no_opattrs = 0 '\000'}, rs_increment = 0}, oq_modrdn = {rs_mods = {rs_modlist = 0xfb25200, rs_no_opattrs = 0 '\000'}, rs_deleteoldrdn = 0, rs_newrdn = {bv_len = 0, bv_val = 0x0}, rs_nnewrdn = {bv_len = 0, bv_val = 0x0}, rs_newSup = 0x0, rs_nnewSup = 0x0}, oq_search = {rs_scope = 263344640, rs_deref = 0, rs_slimit = 0, rs_tlimit = 0, rs_limit = 0x0, rs_attrsonly = 0, rs_attrs = 0x0, rs_filter = 0x0, rs_filterstr = {bv_len = 0, bv_val = 0x0}}, oq_abandon = { rs_msgid = 263344640}, oq_cancel = {rs_msgid = 263344640}, oq_extended = {rs_reqoid = {bv_len = 263344640, bv_val = 0x0}, rs_flags = 0, rs_reqdata = 0x0}, oq_pwdexop = { rs_extended = {rs_reqoid = {bv_len = 263344640, bv_val = 0x0}C C rs_flags = 0, rs_reqdata = 0x0}, rs_old = {bv_len = 0, bv_val = 0x0}, rs_new = {bv_len = 0, bv_val = 0x0}, rs_mods = 0x0, rs_modtail = 0x0}}, o_abandon = 0, o_cancel = 0, o_groups = 0x0, o_do_not_cache = 0 '\000', o_is_auth_check = 0 '\000', o_dont_replicate = 0 '\000', o_acl_priv = ACL_NONE, o_nocaching = 0 '\000', o_delete_glue_parent = 0 '\000', o_no_schema_check = 1 '\001', o_no_subordinate_glue = 0 '\000', o_ctrlflag = '\000' <repeats 14 times>, "\002", '\000' <repeats 16 times>, o_controls = 0x7f1467c969d8, o_authz = {sai_method = 0, sai_mech = {bv_len = 0, bv_val = 0x0}, sai_dn = {bv_len = 9, bv_val = 0x23e8a60 "cn=config"}, sai_ndn = {bv_len = 9, bv_val = 0x23e8a80 "cn=config"}, sai_ssf = 0, sai_transport_ssf = 0, sai_tls_ssf = 0, sai_sasl_ssf = 0}, o_ber = 0x0, o_res_ber = 0x0, o_callback = 0x7f1467c95f70, o_ctrls = 0x0, o_csn = {bv_len = 0, bv_val = 0x0}, o_private = 0x0, o_extra = { slh_first = 0x0}, o_next = {stqe_next = 0x0}}, ob_hdr = {oh_opid = 0, oh_connid = 102, oh_conn = 0x7f1467c96460, oh_msgid = 0, oh_protocol = 0, oh_tid = 139725617329920, oh_threadctx = 0x7f1467c96bc0, oh_tmpmemctx = 0x43cb780, oh_tmpmfuncs = 0x75e3c0 <slap_sl_mfuncs>, oh_counters = 0x7618e0 <slap_counters>, oh_log_prefix = "conn=-1 op=0", '\000' <repeats 243 times>}, ob_controls = {0x0 <repeats 17 times>, 0x7f1467c96240, 0x0 <repeats 14 times>}} op = 0x7f1467c96720 rc = 0 dostop = 0 s = 15 i = 1 defer = 1 fail = 0 freeinfo = 0 be = 0x1fa7740 #17 0x000000000043c3ae in connection_read_thread (ctx=0x7f1467c96bc0, argv=0xf) at connection.c:1273 rc = 0 cri = {op = 0x0, func = 0x4bd8b4 <do_syncrepl>, arg = 0x1d8b130, ctx = 0x7f1467c96bc0, nullop = 0} s = 15 #18 0x00007f3c70fe6a75 in ldap_int_thread_pool_wrapper (xpool=0x1fa0000) at tpool.c:956 pq = 0x1fa0000 pool = 0x1f763c0 task = 0x3db4500 work_list = 0x1fa0070 ctx = {ltu_pq = 0x1fa0000, ltu_id = 139725617329920, ltu_key = {{ltk_key = 0x4b2d5d <slap_sl_mem_init>, ltk_data = 0x43cb780, ltk_free = 0x4b2b82 <slap_sl_mem_destroy>}, { ltk_key = 0x1f82c00, ltk_data = 0x5ee4000, ltk_free = 0x7f3c6bb23f12 <mdb_reader_free>}, {ltk_key = 0x43b82b <conn_counter_init>, ltk_data = 0x25ec700, ltk_free = 0x43b67d <conn_counter_destroy>}, {ltk_key = 0x457375 <slap_op_free>, ltk_data = 0x25621c0, ltfreeee = 0x4572c8 <slap_op_q_destroy>}, {ltk_key = 0x1f83300, ltk_data = 0x5eea800, ltk_free = 0x7f3c6bb23f12 <mdb_reader_free>}, {ltk_key = 0x7f3c6bb18ec6 <search_stack>, ltk_data = 0xafae000, ltk_free = 0x7f3c6bb18ea3 <search_stack_free>}, {ltk_key = 0x7f3c6bb1589f <scope_chunk_get>, ltk_data = 0xacae000, ltk_free = 0x7f3c6bb15857 <scope_chunk_free>}, { ltk_key = 0x0, ltk_data = 0x5a17200, ltk_free = 0x0}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0x0} <repeats 24 times>}} kctx = 0x0 i = 32 keyslot = 410 hash = 616821146 pool_lock = 0 freeme = 0 __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #19 0x00007f3c70552184 in start_thread (arg=0x7f1467c97700) at pthread_create.c:312 __res = <optimized out> pd = 0x7f1467c97700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139725617329920, -1093867468031317215, 0, 0, 139725617330624, 139725617329920, 1078896641691560737, 1056426161274956577}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> __PRETTY_FUNCTION__ = "start_thread" #20 0x00007f3c7027f37d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals. Thread 1 (Thread 0x7f3c71694780 (LWP 16804)): #0 0x00007f3c7055365b in pthread_join (threadid=139725667686144, thread_return=0x0) at pthread_join.c:92 _tid = 16805 _buffer = {__routine = 0x7f3c70553590 <cleanup>, __arg = 0x7f146ac9dd28, __canceltype = 1791612672, __prev = 0x0} oldtype = 0 pd = 0x7f146ac9d700 self = 0x7f3c71694780 result = 0 #1 0x00007f3c70fe80b2 in ldap_pvt_thread_join (thread=139725667686144, thread_return=0x0) at thr_posix.c:197 No locals. #2 0x0000000000438cc0 in slapd_daemon () at daemon.c:2910 i = 0 rc = 0 #3 0x0000000000414ce1 in main (argc=9, argv=0x7ffe003b02b8) at main.c:1017 i = 9 no_detach = 0 rc = 0 urls = 0x1d8a000 "ldap://ldap02e.zimbra.com:389 ldaps://ldap02e.zimbra.com:636 ldapi:///" username = 0x1d7a010 "root" groupname = 0x0 sandbox = 0x0 syslogUser = 128 pid = 0 waitfds = {10, 11} g_argc = 9 g_argv = 0x7ffe003b02b8 configfile = 0x0 configdir = 0x1d82020 "/opt/zimbra/data/ldap/config" serverName = 0x7ffe003b0d78 "slapd" serverMode = 1 scp = 0x0 scp_entry = 0x0 debug_unknowns = 0x0 syslog_unknowns = 0x0 serverNamePrefix = 0x4f9608 "" l = 739235890461816576 slapd_pid_file_unlink = 1 slapd_args_file_unlink = 1 firstopt = 0 __PRETTY_FUNCTION__ = "main" (gdb)

1 0

(ITS#8432) Never ending operation modifications with 3+ MMR nodes
by quanah＠openldap.org 04 Jun '16

04 Jun '16

Full_Name: Quanah Gibson-Mount Version: 2.4.44 OS: Linux 2.6 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (75.111.52.177) In MMR node, when there is > 2 nodes, operations can get sent out endlessly. For example, we see this modification occur at 20160603194926.427963Z Jun 3 14:49:22 ldap01 slapd[14014]: syncrepl_message_to_op: rid=100 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:22 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:22 ldap01 slapd[14014]: slap_queue_csn: queueing 0x121c8680 20160603194926.417898Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x9403600 20160527111854.508810Z#000000#004#000000 Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004,csn=20160603194926.417898Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: dsysyncrep2: rid=102 CSN too old, ignoring 20160603194926.417898Z#000000#001#000000 (reqStart=20160603194922.988081Z,cn=accesslog) Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002,csn=20160603194926.417898Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=101 CSN too old, ignoring 20160603194926.417898Z#000000#001#000000 (reqStart=20160603194931.992576Z,cn=accesslog) Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=100 cookie=rid=100,sid=001,csn=20160603194926.427963Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: slap_queue_csn: queueing 0x120c1c80 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:22 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:22 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:22 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x93aae80 20160527095853.061949Z#000000#004#000000 Jun 3 14:49:22 ldap01 slapd[14014]: syncrepl_message_to_op: rid=100 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:22 ldap01 slapd[14014]: slap_queue_csn: queueing 0x1104d080 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0xac25300 20160527084152.674596Z#000000#004#000000 Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004,nsn=20160603194926.427963Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=102 CSN too old, ignoring 20160603194926.427963Z#000000#001#000000 (reqStart=20160603194922.996275Z,cn=accesslog) Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=100 cookie=rid=100,sid=001,csn=20160603194926.433237Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: slap_queue_csn: queueing 0x1116e600 20160603194926.433237Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: syncprov_matchops: skipping origil l sid 001 Jun 3 14:49:22 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:22 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:22 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:22 ldap01 slapd[14014]: slap_queue_csn: queueing 0xf0cf380 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x122e2cc0 20160527095853.613262Z#000000#004#000000 Jun 3 14:49:22 ldap01 slapd[14014]: syncrepl_message_to_op: rid=100 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:22 ldap01 slapd[14014]: slap_queue_csn: queueing 0x1104f280 20160603194926.433237Z#000000#001#000000 Jun 3 14:49:22 ldap01 slapdB1B14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004,csn=20160603194926.433237Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10aad340 20160527095854.490344Z#000000#004#000000 Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=102 CSN too old, ignoring 20160603194926.433237Z#000000#001#000000 (reqStart=20160603194923.000888Z,cn=accesslog) Jun 3 14:49:23 ldap01 slapd[14014]: conn=183480 op=6644 SRCH base="" scope=0 deref=0 filter="(objectClass=*)" Jun 3 14:49:23 ldap01 slapd[14014]: conn=183480 op=6644 SRCH attr=1.1 Jun 3 14:49:23 ldap01 slapd[14014]: conn=183480 op=6644 SEARCH RESULT tag=101 err=0 etime=0.000307 nentries=1 text= Jun 3 14:49:23 ldap01 slapd[14014]: conn=183479 op=6618 SRCH base="" scope=0 deref=0 filter="(objectClass=*)" Jun 3 14:49:23 ldap01 slapd[14014]: conn=183479 op=6618 SRCH attr=1.1 Jun 3 14:49:23 ldap01 slapd[14014]: conn=183479 op=6618 SEARCH RESULT tag=101 err=0 etime=0.000237 nentries=1 text= Jun 3 14A4A49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x120d7940 20160602173545.246579Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 34%4:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002,csn=20160603194926.433237Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 CSN too old, ignoring 20160603194926.433237Z#000000#001#000000 (reqStart=20160603194932.005634Z,cn=accesslog) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10c11fc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x120d6580 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x16c6aec0 20160602173612.801299Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10dad800 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x12058c40 20160602173638.625104Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x12058c40 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x120191c0 20160602174110.510583Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x120191c0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 spdpd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 lda1 1 slapd[14014]: slap_graduate_commit_csn: removing 0x120c0c00 20160603002555.477495Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 sld[5B14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x120c0c00 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0xf0cf380 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0xf0cf380 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x115d1d00 20160603002555.477495Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]% s syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x115d1d00 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: conn=198588 op=331 SEARCH RESULT tag=101 err=0 etime=0.000154 nentries=1 text= Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10c11fc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10c11fc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0xf217bc0 20160603194914.391952Z#000000#001#000000 n n 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0xf217bc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10dad800 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10dad800 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,did=003 Jun 3 14:49:23 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10dad4c0 20160603194914.391952Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:23 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:23 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10dad4c0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:23 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cooe%e=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x12058c40 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x12058c40 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10c11fc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10c11fc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x120c0c00 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:28 ldap01 apapd[14014]: slap_queue_csn: queueing 0x120c0c00 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10dad800 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10dad800 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x115d1d00 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=server2C2Ccn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x115d1d00 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: cn%n=26584 op=49161 SRCH base="" scope=2 deref=0 filter="(&(|(zimbraMailDeliveryAddress=office7@domesticuniform.com)(zimbraMailAlias=office7@domesticuniform.com))(objectClass=zimbraAccount))" Jun 3 14:49:28 ldap01 slapd[14014]: conn=26584 op=49161 SEARCH RESULT tag=101 err=0 etime=0.000212 nentries=0 text= Jun 3 14:49:28 ldap01 slapd[14014]: conn=26591 op=49432 SRCH base="" scope=2 deref=0 filter="(&(zimbraDomainName=domesticuniform.com)(objectClass=zimbraDomain))" Jun 3 14:49: 8 ldap01 slapd[14014]: conn=26591 op=49432 SEARCH RESULT tag=101 err=0 etime=0.000113 nentries=0 text= Jun 3 14:49:28 ldap01 slapd[14014]: conn=26602 op=49397 SRCH base="" scope=2 deref=0 filter="(&(zimbraForeignPrincipal=zmgproxyacct:office7@domesticuniform.com)(objectClass=zimbraAccount))" Jun 3 14:49:28 ldap01 slapd[14014]: conn=26602 op=49397 SEARCH RESULT tag=101 err=0 etime=0.000104 nentries=0 text= Jun 3 14:49:28 ldap01 slapd[14014]: conn=26655 op=36838 MOD dn="uid=shagenbrock,ou=people,dc=zimbra,dc=com" Jun 3 14:49:28 ldap01 slapd[14014]: conn=26655 op=36838 MOD attr=zimbraAuthTokens Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x120191c0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x120191c0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x115da900 20160603194928.195282Z#000000#003#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0xf217bc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0xf217bc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x11257980 20160603194928.195282Z#000000#003#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003,csn=20160603194928.195282Z#000000#003#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003,csn=20160603194928.195282Z#000000#003#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=001, cookie=rid=100,sid=003,csn=20160603194928.195282Z#000000#003#000000 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x11257980 20160603194928.195282Z#000000#003#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0xf0cf380 20160603194926.427963Z#000000#001#000000 Jun 3 14:43A2828 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0xf0cf380 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10dad4c0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10dad4c0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003%%0 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10c11fc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10c11fc0 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x120d6580 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,d%d=004 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x120d6580 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: conn=184860 op=4552 SRCH base="cn=config,cn=zimbra" scope=0 deref=0 filter="(objectClass=*)" Jun 3 14:49:28 ldap01 slapd[14014]: conn=184860 op=4552 SEARCH RESULT tag=101 err=0 etime=0.001120 nentries=1 text= Jun 3 14:49:28 ldap01 slapd[14014]: conn=184858 op=4439 SRCH base="cn=mbs01.zimbra.com,cn=servers,cn=zimbra" scope=0 deref=0 filr%r="(objectClass=*)" Jun 3 14:49:28 ldap01 slapd[14014]: conn=184858 op=4439 SEARCH RESULT tag=101 err=0 etime=0.000231 nentries=1 text= Jun 3 14:49:28 ldap01 slapd[14014]: conn=184895 op=4430 SRCH base="cn=config,cn=zimbra" scope=0 deref=0 filter="(objectClass=*)" Jun 3 14:49:28 ldap01 slapd[14014]: conn=184895 op=4430 SEARCH RESULT tag=101 err=0 etime=0.001063 nentries=1 text= Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=004, cookie=rid=102,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: syncprov_sendresp: to=002, cookie=rid=100,sid=003 Jun 3 14:49:28 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x10dad800 20160603194926.427963Z#000000#001#000000 Jun 3 14:49:28 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 14:49:28 ldap01 slapd[14014]: do_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 14:49:28 ldap01 slapd[14014]: slap_queue_csn: queueing 0x10dad800 20160603194926.427963Z#000000#001#000000 this literally goes on for HOURS: s.com)(zimbraDomainName=@.synacordomains.com)(zimbraDomainName=synacordomains.com)(zimbraDomainName=@.com)(zimbraDomainName=com)(mbmbraDomainName=@.))))" Jun 3 18:03:17 ldap01 slapd[14014]: conn=202259 op=4 SEARCH RESULT tag=101 err=0 etime=0.000169 nentries=0 text= Jun 3 18:03:17 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 18:03:17 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x18eab480 20160603194926.427963Z#000000#001#000000 Jun 3 18:03:17 ldap01 slapd[14014]: syncrepl_message_to_op: rid=101 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 18:03:17 ldap01 slapd[14014]:o_o_syncrep2: rid=101 cookie=rid=101,sid=002 Jun 3 18:03:17 ldap01 slapd[14014]: slap_queue_csn: queueing 0x18eab480 20160603194926.427963Z#000000#001#000000 Jun 3 18:03:17 ldap01 slapd[14014]: syncprov_matchops: skipping original sid 001 Jun 3 18:03: 7 ldap01 slapd[14014]: slap_graduate_commit_csn: removing 0x18e659c0 20160603194926.427963Z#000000#001#000000 Jun 3 18:03:17 ldap01 slapd[14014]: syncrepl_message_to_op: rid=102 be_modify cn=edge02e.zimbra.com,cn=servers,cn=zimbra (0) Jun 3 18:03:17 ldap01 slapd[14014]: do_syncrep2: rid=102 cookie=rid=102,sid=004 Jun 3 18:03:17 ldap01 slapd[14014]: slap_queue_csn: queueing 0x18e659c0 20160603194926.427963Z#000000#001#000000 Jun 3 18:03:17 ldap01 slapd[14014]: conn=26580 op=75173 SRCH base="" scope=0 deref=0 filr%r="(objectClass=*)" Jun 3 18:03:17 ldap01 slapd[14014]: conn=26580 op=75173 SRCH attr=1.1

1 0

Re: (ITS#8339) LMDB robust mutex fix
by openldap＠gulag.ch 03 Jun '16

03 Jun '16

On 06/02/16 10:15 PM, Howard Chu wrote: > I've pushed a tweaked version of Hallvard's patch. While the Studio > compiler defines __SunOS_5_10 it appears that gcc doesn't, so we took a > different detection route. Perfect and the detection clearly works. Thanks for reviewing this ITS. --- pthread.h S11 --- /* * Mutex robust attribute values. * Keep these in synch with sys/synch.h lock types. */ #define PTHREAD_MUTEX_STALLED 0x0 #define PTHREAD_MUTEX_ROBUST 0x40 /* * Historical solaris-specific names, * from before pthread_mutexattr_getrobust() became standardized */ #define PTHREAD_MUTEX_STALL_NP PTHREAD_MUTEX_STALLED #define PTHREAD_MUTEX_ROBUST_NP PTHREAD_MUTEX_ROBUST --------------------- > uname -r 5.11 > nm liblmdb.so |grep setrobust [161] | 0| 0|FUNC |GLOB |0 |UNDEF |pthread_mutexattr_setrobust --- pthread.h S10 --- /* * Mutex robustness attribute values. The robustness attribute is a * Solaris specific extension to support robust mutexes. Note the _NP suffix * to indicate these are not part of the current POSIX spec (POSIX 1003.1 1996), * but are platform specific non-portable extensions. Keep these in synch * with sys/synch.h lock types. */ #define PTHREAD_MUTEX_STALL_NP 0x0 #define PTHREAD_MUTEX_ROBUST_NP 0x40 --------------------- > uname -r 5.10 > nm liblmdb.so |grep setrobust [245] | 0| 0|FUNC |GLOB |0 |UNDEF |pthread_mutexattr_setrobust_np

1 0

Re: (ITS#8339) LMDB robust mutex fix
by hyc＠symas.com 02 Jun '16

02 Jun '16

openldap(a)gulag.ch wrote: > On 06/02/16 01:48 AM, Howard Chu wrote: >> My last SPARC/Solaris dev machine was decommissioned sometime last year >> so I haven't had any system available to test this on. > > Hi Howard > > Thanks for the reply. Maybe I can help out and setup two systems for you > (accessible via Internet): > > -Solaris 11.3 Intel with Solaris Studio 12.3/4 > -Solaris 10 Intel with Solaris Studio 12.3/4 > > We already use this patch (applied to LMDB 0.9.18) with OpenLDAP 2.4.44 and as > part of our application in production on 11.3/Intel and 10/Intel+Sparc. So I > could do some further testing for you on 10/Sparc (Solaris Studio 12.3/4 as > well) if required. > > Does this sound feasible? Thanks for the offer, perhaps in the future. I just now acquired access to a Solaris VPS so we can handle this issue for now. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs