> I see the issue, and I've isolated it to the case where you're removing a
> value. In that case, in attr_cmp(), nn == 0 and no < o; the original test
> for modifying an attribute without matching rule fails. I've fixed it
> according to this patch, not sure whether it's appropriate for other cases
>
> <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-syncrepl-no-…>
>
> Please test and review. Thanks, p.
Please disregard, this patch is breaking other things (basically,
producer's and consumer's entries are equivalent, but attrs are listed in
a different order; this conforms to the LDAP model but breaks the
original, preferable behavior). Working at a better fix.
p.
> Full_Name: Patrick Cernko
> Version: 2.4.21
> OS: Linux
> URL:
> http://www.mpi-inf.mpg.de/~pcernko/syncrepl_fails_if_attribute_has_no_equal…
> Submission from: (NULL) (139.19.3.67)
>
>
> Hi OpenLDAP-Programmers,
>
> although some related bug reports were posted in the past years, this
> issue
> still seems not to be fixed:
>
> OpenLDAP's syncrepl replication fails if it comes to removing one of a
> multi-value attribes of an object if this attribute does not have a
> equality
> matching rule defined in the schema. Modification works on the master
> server but
> fails to replicate to the slave(s) leaving them with the dit version just
> before
> and not replicating any other changes any more.
>
> I compiled a small example how to reproduce the problem with the (maybe
> "well-known") nisNetgroupTriple attribute from nis.schema as shipped with
> openldap-2.4.21:
>
> - extract the files from
> http://www.mpi-inf.mpg.de/~pcernko/syncrepl_fails_if_attribute_has_no_equal…
> into an empty working directory
> - download the tarball to that working dir and configure similar to the
> just
> extracted configure.sh
> - build and install it to get an install subdir in the working dir
> - start the master using sh master.sh in the working dir
> This will create the "master" database in your working dir, slapadd the
> dit.ldif and start a master slapd with a unix socket /tmp/master.
> - start the slave using sh slave.sh
> This will create "slave" database in your working dir and start the
> slave
> slapd with a unix socket /tmp/slave which will immediately start syncing
> from
> the running master
> - if you want you can search the 2 servers with
> install/bin/ldapsearch -H ldapi://%2ftmp%2fmaster/ -b ou=openldap -LLL
> resp.
> install/bin/ldapsearch -H ldapi://%2ftmp%2fslave/ -b ou=openldap -LLL
> - use "python mod.py" to remove one of the nisNetgroupTriple attributes
> from the
> netgroup object
> - you will see a debug message from the slave server like this:
>
> bdb_modify_internal: delete nisNetgroupTriple
> bdb_modify_internal: 18 modify/delete: nisNetgroupTriple: no equality
> matching
> rule
> send_ldap_result: err=18 matched="" text="modify/delete:
> nisNetgroupTriple: no
> equality matching rule"
> null_callback : error code 0x12
> slap_graduate_commit_csn: removing 0x29931c0
> 20100120124712.028626Z#000000#000#000000
> syncrepl_entry: rid=001 be_modify cn=netgroup,ou=openldap (18)
> syncrepl_entry: rid=001 be_modify failed (18)
>
> if not, use "python mod.py add" to re-add the removed nisNetgroupTriple
> and
> try again. Sometimes, it seems to work (race condition?).
> - you can also verify the difference with:
> install/bin/ldapsearch -H ldapi://%2ftmp%2fmaster/ -b ou=openldap -LLL
> cn=netgroup
> install/bin/ldapsearch -H ldapi://%2ftmp%2fslave/ -b ou=openldap -LLL
> cn=netgroup
> - from now on, the slave will fail to sync any succeeding changes, as it
> continuously tries to sync the modification above first. :-(
>
> Now, obviously having an attribute without equality matching rule is not a
> good
> idea in the first place. However, the (official?) nis.schema comes with
> such an
> attribute which we must use like this (e.g. we would have to patch the
> Solaris
> kernel to change it for the Solaris NFS server).
>
> It is possible to modify a nisNetgroup object like this: according to the
> debugging output, I guess the master removes simply ALL nisNetgroupTriple
> attributes and adds only the attributes that should remain. When the
> corresponding syncrepl_entry is handled by the slave, it seems the it in
> contrast tries to really only remove the 2-be-removed attribute which does
> not
> work as it cannot identify the right one due to a missing eq-match
> operator.
>
> I suggest the best way to solve this issue is to try to get syncrepl do
> exactly
> the same actions (not resulting changes) as the server does.
I see the issue, and I've isolated it to the case where you're removing a
value. In that case, in attr_cmp(), nn == 0 and no < o; the original test
for modifying an attribute without matching rule fails. I've fixed it
according to this patch, not sure whether it's appropriate for other cases
<ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-syncrepl-no-…>
Please test and review. Thanks, p.
> Full_Name: Jochen Keutel
> Version: 2.4.21
> OS: Solaris 10
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (87.159.206.14)
>
>
> dnssrv_back_referrals() always returns ldap URLs - even when the original
> request was a ldaps query.
This is a (logical) duplicate of ITS#6462; see related answers.
> Additionally only protocol and hostname are put into the URL - the search
> base
> doesn't appear any more.
>
> Problematic code:
>
> url.bv_len = STRLENOF( "ldap://" ) + strlen( hosts[i] );
> url.bv_val = ch_malloc( url.bv_len + 1 );
>
> strcpy( url.bv_val, "ldap://" );
> strcpy( &url.bv_val[STRLENOF( "ldap://" )], hosts[i] );
>
> I'm not sure whether you want me to deliver a complete patch or not ...
This behavior conforms to RFC4511, section 4.1.10:
- If the <dn> part is present, the client uses this name in its next
request to progress the operation, and if it is not present the
client uses the same name as in the original request.
I note that few lines above the same RFC states:
- It is RECOMMENDED that the <dn> part be present to avoid ambiguity.
However, slapd-dnssrv(5) computes the referral from DNS SRV records,
rather than returning data it knows about. So returning the <dn> part
here would be incorrect, in my opinion, as the client could legitimately
rely on the fact that the returned DN is appropriate, while it may not.
p.
> --On Wednesday, January 27, 2010 1:28 AM +0000 jochen(a)keutel.de wrote:
>
>> Full_Name: Jochen Keutel
>> Version: 2.4.21
>> OS: Solaris 10
>> URL: ftp://ftp.openldap.org/incoming/
>
>> protocol is either "ldap" oder "ldaps".
>
> LDAPS is not a defined protocol. I would suggest using LDAP with
> startTLS,
> which is RFC Defined.
One could argue that OpenLDAP supports ldaps in many places. I agree in
general with your reply, but I'd treat this ITS as a feature request,
although I'm not going to give it any special priority.
p.
> Full_Name: Robert Hanson
> Version: 2.4.21
> OS: RedHat 4.7
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (74.203.208.250)
>
>
> 1) start with an empty database; use slapadd to add a large (3 MB)
> database. 2)
> Use ldapbrowser to delete nearly all the nodes. After this process, I get
> to a
> point where a subnode can't be deleted; I see the message "12:00:27 PM:
> Failed
> to delete entry ou=Agents, lcc=Call Center 1, ou=Company, o=Spanlink
> Communications
> Reason: [LDAP: error code 80 - DN index delete failed]
> 12:15:19 PM: Failed to delete entry ou=Agents, lcc=Call Center 1,
> ou=Company,
> o=Spanlink Communications
> Reason: [LDAP: error code 80 - DN index delete failed]" in the ldapbrowser
> window.
>
> Steps to reproduce: (on redhat linux; I think any linux will do)
> 1) build bdb 4.8.26 from the distro; build openldap 2.4.21 from the
> distro.
> 2) copy built slapd to an appropriate place. link it to slapadd.
> 3) FTP: ftp.calabrio.com, user openldap, pass *a68pcJH (Account is
> scheduled to
> close 3/23/2010; let me know if you need the file uploaded again). unzip
> the
> file, it contains the slapcat.out; a blank database (in a subfolder), and
> a
> slapd.conf file.
> 4) Edit the slapd.conf file as needed to point to the directory where you
> put
> the database.
>
> That was the setup, now here are the steps to reproduce the problem:
>
> 5) use slapadd -c -f slapd.conf -l slapcat.out to create the database.
> 6) start slapd (using slapd -d 1 -f slapd.conf)
> 7) use ldapbrowser to delete the "lcc=call center 1" node.
>
> It takes a long time, but eventually you'll see it fail while trying to
> delete
> the "agents" node.
Can you reproduce with -dtrace? It'll generate a large log file, please
only post the part related to the failure.
p.
masarati(a)aero.polimi.it wrote:
>
>> It's not clear to me where the issue is. What is the "right" sequence the
>> add of the new superior and the mordrdn should be transmitted? Should the
>> provider operate differently, or should the consumer check all syncrepl
>> messages and try to rebuild the final state, instead of giving up when the
>> internal lookup for the newsuperior fails? Probably, a workaround could
>> be to perform the modrdn by crating the new superior as a glue object,
>> which eventually will be replaced by the actual add.
>
> I've quickly hacked things this way, and it seems to work fine.
>
> <ftp://ftp.openldap.org/incoming/pierangelo-masarati-2010-04-17-sync-rename.…>
>
> Please let me know if this approach is sound enough, I might have
> overlooked some implications.
Patch looks good, solution makes sense. This is one of the reasons we would
expect glue entries to be used.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Kurt Zeilenga wrote:
> On Apr 17, 2010, at 3:15 AM, matthijs(a)cacholong.nl wrote:
>
>> On Apr 16, 2010, at 6:01 PM, Howard Chu wrote:
>>> If Peter wants his patch considered for inclusion in OpenLDAP he =
>> should write to the ITS himself, we cannot accept 3rd party =
>> contributions.
>>
>> What's the reasoning behind this?
>
> 3rd parties (middle men) not only hinder good communication between the authors(s) of the contribution and the Project/Foundation, they introduce various risks to both the author and the Project/Foundation.
>
> Regard, Kurt (Executive Director, The OpenLDAP Foundation)
>
Hi,
Thanks for the explanation, I'll keep it in mind when forwarding patches
and / or bug reports from the Debian bugtracker.
Regards,
Matthijs Mohlmann
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkvKDnUACgkQ2n1ROIkXqbDFtwCgjOrjYmDPXtqYpQrQTNzq7gqL
RcsAmgKRRJB28cke29WMHhKIIcB/+q7L
=QTpi
-----END PGP SIGNATURE-----