Full_Name:
Version: RE24 from git
OS: openSUSE 13.2 x86-64
URL:
Submission from: (NULL) (79.227.179.192)
>>>>> Starting test058-syncrepl-asymmetric for bdb...
running defines.sh
Initializing master configurations...
Initializing search configurations...
Starting central master slapd on TCP/IP port 9011...
Using ldapsrcrch to check that central master slapd is running...
Starting site1 master slapd on TCP/IP port 9012...
Using ldapsearch to check that site1 master is running...
Starting site2 master slapd on TCP/IP port 9013...
Using ldapsearch to check that site2 master is running...
Starting central search slapd on TCP/IP port 9014...
Using ldapsearch to check that central search slapd is running...
Starting site1 search slapd on TCP/IP port 9015...
Using ldapsearch to check that site1 search slapd is running...
Starting site2 search slapd on TCP/IP port 9016...
Using ldapsearch to check that site2 search slapd is running...
Adding schema on ldap://localhost:9011/...
Adding backend module on ldap://localhost:9011/...
Adding schema on ldap://localhost:9012/...
Adding backend module on ldap://localhost:9012/...
Adding schema on ldap://localhost:9013/...
Adding backend module on ldap://localhost:9013/...
Adding schema on ldap://localhost:9014/...
Adding backend module on ldap:/FlFlocalhost:9014/...
Adding schema on ldap://localhost:9015/...
Adding backend module on ldap://localhost:9015/...
Adding schema on ldap://localhost:9016/...
Adding backend module on ldap://localhost:9016/...
Adding database config on central master...
Adding database config on site1 master...
Adding database config on site2 master...
Adding access rules on central master...
Adding access rules on site1 master...
Adding access rules on site2 master...
Adding database config on central search...
Adding database config on sit seaearch...
Adding database config on site2 search...
Populating central master...
Adding syncrepl on site1 master...
Adding syncrepl on site2 master...
Using ldapsearch to check that site1 master received changes...
Using ldapsearch to check that site2 master received changes...
Populating site1 master...
Populating site2 master...
Stopping site1 master...
Adding syncrepl on central master...
Using ldapsearch to check that central master received site2 entries...
Using ldapmodify to modify central master...%ReRestarting site1 master slapd on
TCP/IP port 9012...
./scripts/test058-syncrepl-asymmetric: line 1020: 22046 Segmentation fault
(core dumped) $SLAPD -F slapd.d -h $URI1 -d $LVL $TIMING > $LOG1 2>&1
(wd:2/usr/src/michael/openldap-git/re24/openldap/tests/testrun/smc)
Using ldapsearch to check that site1 master is running...
Using ldapsearch to check that central master received site1 entries...
Waiting 1 seconds for syncrepl to receive changes...
Waiting 2 seconds for syncrepl to receive changes...
Waiting 3 seconds for syncrepl to receive changes...
Waiting 4 seconds for syncrepl to receive changes...
Waiting 5 seconds for syncrepl to receive changes...
ldapsearch failed (32)!
./scripts/test058-syncrepl-asymmetric: line 1051: kill: (22046) - No such
process
IMHO you or someone else should write a short I-D defining another OID and
semantics for that and post that to ldapext. I think it's worth the effort.
Ciao, Michael.
Full_Name: Howard Chu
Version: 2.4.40
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (110.49.207.42)
Submitted by: hyc
In a 3-way MMR setup I'm seeing a consistent leak of queued Entries in syncprov.
I'm not entirely sure of the exact sequence to reproduce it, but this is what I
have so far:
Test DB is 2.5GB on disk. One of the nodes (server #3) is configured with an mdb
maxsize of 1GB, so it will always fail to sync up. The other two have sufficient
maxsize configured.
While server #3 is running, stop server #1 and #2 and completely reinit their
DBs. rm the old files, rerun slapadd -w on server #1 (all new entryUUIDs and
entryCSNs). Start server #1. Server #3 will reconnect and start trying to update
itself and most of the updates will fail (as it runs out of space). Start server
#2 (with an empty DB). It will start syncing from server #1. syncprov leaks will
occur on both #2 and #3 during this time.
Part of the trigger here appears to be from slapadd'ing the DB on server #1
without using the -S <sid> argument, so the DB contents all have SID #0.
In this case updates get sent from #1 to #2 and #3 and syncprov on #2 and
#3ueueues them up for transmission to other nodes, respectively. Syncprov
doesn't know that it doesn't need to do this, because the SID #0 in the updates
doesn't match any of the servers' SID #1, 2, or 3.
The point in the code where the leak occurs is not obvious; the queued entries
are all refcounted and the refcount is incremented whenever an entry is matched
to an outbound queue. The count is decremented when a queue finishes sending the
entry. In this particular case, I don't believe the entry is matched to any
queue, so it should simply be freed again at the end of syncprov_matchops().
Full_Name: Howard Chu
Version: 2.4
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (180.180.122.131)
Submitted by: hyc
During a refresh the consumer maintains a presentlist containing the UUIDs of
all entries that exist on the provider. At the end of the refresh this list is
used to delete any entries on the consumer that don't exist on the provider.
In refreshAndPersist, at the end of the refresh phase, the presentlist is not
being freed as it should. It remains intact and unused until the persist session
breaks and is restarted.
I have now made progress in narrowing down the cause further.=0A=0AI have=
noticed that it is a regression between FreeBSD 9.x -> FreeBSD 10.x. For=
this reason, I will move any updates on this to the FreeBSD bug tracker,=
rather than the OpenLDAP one, as the bug is platform specific.=0A=0AFutu=
re news will be posted here: https://bugs.freebsd.org/bugzilla/show_bug.c=
gi?id=3D197004=0A=0AThank you for your time,=0A=0A-Jonathan=0A=0AJanuary =
22 2015 2:25 PM, freebsd(a)jonathanprice.org wrote: =0A> Sorry for the slow=
response, but I have made some progress with the issue.=0A> =0A> (as an =
aside, I installed a build from LTB, and unfortunately it does not=0A> co=
ntain this overlay)=0A> =0A> I have detailed my findings (including some =
trawling through the source)=0A> over on the FreeBSD bug tracker, as I su=
spect it could well be a platform=0A> related issue. Nonetheless, it migh=
t be worth reading:=0A> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=
=3D197004=0A> =0A> January 14 2015 4:31 PM, "Quanah Gibson-Mount" <quanah=
@zimbra.com> wrote:=0A> =0A>> --On Wednesday, January 14, 2015 11:00 AM +=
0000 freebsd(a)jonathanprice.org=0A>> wrote:=0A>> =0A>>> Hi,=0A>>> =0A>>> I=
tried 2.4.39 under FreeBSD and still had the same issue.=0A>>> =0A>>> I =
have also tried the packages for both CentOS 7 and Debian Wheezy, but=0A>=
>> unfortunately neither of them include the SHA2 overlay by default.=0A>=
>> =0A>>> Finally, I tried installing zimbra-core and zimbra-ldap under C=
entOS.=0A>>> When I used this installation, it worked successfully.=0A>>>=
=0A>>> I ran slapd -V on the zimbra installation, and it's 2.4.39. Howev=
er,=0A>>> based on it still not working on 2.4.39 on FreeBSD it appears t=
o have=0A>>> narrowed it down to two reasons: - An issue with the packagi=
ng under=0A>>> FreeBSD=0A>>> - The functionality is specific to Zimbra=0A=
>>> =0A>>> The next step in the process to narrow this down is to do a ma=
nual=0A>>> compilation on CentOS, including the SHA2 overlay. If this wor=
ks, then it=0A>>> would confirm it to be a FreeBSD issue, and if it doesn=
't work that would=0A>>> strongly suggest that Zimbra has modified someth=
ing.=0A>> =0A>> You could simply grab the LTB project builds. I'm pretty =
sure they build=0A>> out the contrib modules.=0A>> =0A>> In any case, I a=
lready noted that Zimbra doesn't patch anything in OpenLDAP=0A>> that wou=
ld affect this area.=0A>> =0A>> --Quanah=0A>> =0A>> --=0A>> =0A>> Quanah =
Gibson-Mount=0A>> Platform Architect=0A>> Zimbra, Inc. =0A>> ____________=
___________________=0A>> =0A>> Zimbra :: the leader in open source messag=
ing and collaboration
Full_Name: Ivan Nejgebauer
Version: 2.4.41 Engineering
OS: Linux
URL: ftp://ftp.openldap.org/incoming/ivannejgebauer-150128.tgz
Submission from: (NULL) (2001:4170:2000:2:11e5:197a:fff8:8042)
If an ldapmodify which changes an entry's structural object class using the
Relax Rules control is successfully performed on the provider in a
provider/consumer pair running delta-syncrepl, the modification will fail on the
consumer because relax is not in effect when the consumer attempts to modify its
copy of the entry.
The attached archive, which should be extracted in the root of the OpenLDAP
source tree, contains scripts and data to replicate the issue. Steps to
reproduce:
$ sh relax-syncrel-test/conf-ldap-mdb && make depend && make
$ cd relax-syncrepl-test
$ make clean-all master replica
$ ./start-master.sh
$ ./mod-l-master.sh here # modifies an entry to prime accesslog
$ ./start-replica.sh # writes SYNC debugging to replica.log
$ ./mod-relax-master.sh # ldapmodify w/relax
$ tail replica.log # "entry failed schema check: ..."
$ ./stop-replica.sh
$ ./stop-master.sh
A trivial but indiscriminate fix is to activate Relax Rules for every modify op
on the consumer:
--- servers/slapd/syncrepl.c.orig 2015-01-22 03:02:09.000000000 +0100
+++ servers/slapd/syncrepl.c 2015-01-28 10:31:22.225060880 +0100
@@ -2349,6 +2349,7 @@
oes.oe_si = si;
LDAP_SLIST_INSERT_HEAD( &op->o_extra,
&oes.oe, oe_next );
}
+ op->o_relax = SLAP_CONTROL_CRITICAL;
rc = op->o_bd->be_modify( op, &rs );
if ( SLAP_MULTIMASTER( op->o_bd )) {
LDAP_SLIST_REMOVE( &op->o_extra,
&oes.oe, OpExtra, oe_next );
A real fix would involve modifying the persistent search to include reqControls
in its attribute list and activating Relax Rules on the consumer only if it had
been active on the provider when the modification occurred.
Full_Name: Howard Chu
Version: 2.4.40
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (180.180.122.131)
Submitted by: hyc
The scopes array points to dn2id records from the previous read txn, some or all
of these may be invalid after the txn_reset/txn_renew in mdb_writewait. They
need to be refreshed in mdb_waitfixup. Patch coming shortly.