Re: (ITS#5534) Samba4 needs internal transactions/consistancy
by abartlet＠samba.org
--=-8f0GY55a/Qq8n0F+gHZS
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable
On Wed, 2008-05-28 at 08:14 -0700, Howard Chu wrote:
> Andrew Bartlett wrote:
> > On Tue, 2008-05-27 at 18:43 -0700, Howard Chu wrote:
> >> Andrew Bartlett wrote:
> >>> On Tue, 2008-05-27 at 18:22 -0700, Howard Chu wrote:
> >>>>> This needs to occur even between databases on the server, but I won=
't ask that
> >>>>> it occur outside the known trees.
> >>>> It's already possible for operations in one database to reference en=
tries in a
> >>>> different database, so that aspect of validation should be fine. How=
ever, as
> >>>> noted before, "validation" is generally bogus to begin with. In part=
icular,
> >>>> how do you create entries with circular references? If you disallow =
references
> >>>> to nonexistent entries, you can't set the references until after all=
of the
> >>>> entries have been created. This means that you cannot backup a datab=
ase that
> >>>> has these references and then later reload it in a single pass.
> >>> An interesting point, but I need to match the windows runtime
> >>> behaviour.
> >> Only when it has a visible impact on other clients. What software will=
break
> >> if the directory allows you to add new entries that contain dangling
> >> references? What will break if the directory allows you to modify a re=
ference
> >> attribute to point to a nonexistent entry?
> >
> > Sure, I'm not asking for a change to default behaviours. I'm listing
> > the things that our testsuite finds are differences, and looking for
> > solutions.
>=20
> I don't believe your proposed solution will ever be satisfactory. Entries=
with=20
> circular references will also break syncrepl Refresh if the constraint yo=
u're=20
> asking for is enforced.=20
Only if you don't consider them in replication. If the backlinks are
added on each node, and not replicated, then surely you just need to
import a set of replicated data, and then in the same transaction update
the links.=20
Is there perhaps another way to implement this - say using a
search-based virtual attribute for one half of the problem?
I'm in no position to set your priorities, and my wishlist remains only
that I hope to someday be able to make this work with OpenLDAP, but
these issues remain.
Andrew Bartlett
--=20
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Red Hat Inc.
--=-8f0GY55a/Qq8n0F+gHZS
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iD8DBQBIPpKIz4A8Wyi0NrsRAhu+AJ9fRq1o5INcGiX1ZYJTAmjmBUMBogCfQ8gC
zrbftn69NpgTvb546qKvGKA=
=kiKt
-----END PGP SIGNATURE-----
--=-8f0GY55a/Qq8n0F+gHZS--
14 years, 9 months
Re: (ITS#5532) incorrect timestamp of slapd replica log
by quanah＠zimbra.com
--On Thursday, May 29, 2008 11:16 AM +0800 Hai <hai.zhao(a)gmail.com> wrote:
> Hi, Quanah:
>
> I'm considering upgrading from 2.3 to 2.4.x. I haven't test 2.4.x yet.
> Does this bug exist in 2.4.x too?
Since slurpd was removed because it is completely unreliable and buggy, no,
it does not exist. Sane people use syncrepl replication.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
14 years, 9 months
Re: (ITS#5532) incorrect timestamp of slapd replica log
by hai.zhao＠gmail.com
------=_Part_7580_3624331.1212031000971
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Hi, Quanah:
I'm considering upgrading from 2.3 to 2.4.x. I haven't test 2.4.x yet. Does
this bug exist in 2.4.x too?
Regards,
Hai
On Thu, May 29, 2008 at 1:31 AM, Quanah Gibson-Mount <quanah(a)zimbra.com>
wrote:
> --On Tuesday, May 27, 2008 10:10 AM +0000 hai.zhao(a)gmail.com wrote:
>
> Full_Name: Zhao Hai
>> Version: 2.3.41
>> OS: Linux 2.4.21 arm
>> URL: ftp://ftp.openldap.org/incoming/zhaohai-080527.patch
>> Submission from: (NULL) (205.209.140.4)
>>
>>
>> Problem:
>> race condition makes incorrect timestamp in replogfile, cause certain
>> modification of entries not replicate to slurp slaves.
>>
>> replica: 180.0.10.2:1234
>> replica: 180.0.10.3:1234
>> time: 1211855467
>> ^^^^^^^^^^ this timestamp
>>
>> How to reproduce the problem:
>> 1) run under very slow machines (my environ: arm 266MHz)
>> 2) slapd is configed to generate replogfile
>> 3) ldapadd about 5 entries, then ldapmodify 2 entries without delay.
>>
>
> This is fixed in RE23. If there is ever a 2.3.43 release, it will be in
> that. In the meantime, I'd advise using 2.3.42 + your patch.
>
> Regards,
> Quanah
>
>
>
> --
>
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
>
------=_Part_7580_3624331.1212031000971
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Hi, Quanah:<br><br>I'm considering upgrading from 2.3 to 2.4.x. I haven't test 2.4.x yet. Does this bug exist in 2.4.x too?<br><br>Regards,<br>Hai<br><br><div class="gmail_quote">On Thu, May 29, 2008 at 1:31 AM, Quanah Gibson-Mount <<a href="mailto:quanah@zimbra.com">quanah(a)zimbra.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="Wj3C7c">--On Tuesday, May 27, 2008 10:10 AM +0000 <a href="mailto:hai.zhao@gmail.com" target="_blank">hai.zhao(a)gmail.com</a> wrote:<br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Full_Name: Zhao Hai<br>
Version: 2.3.41<br>
OS: Linux 2.4.21 arm<br>
URL: <a href="ftp://ftp.openldap.org/incoming/zhaohai-080527.patch" target="_blank">ftp://ftp.openldap.org/incoming/zhaohai-080527.patch</a><br>
Submission from: (NULL) (<a href="http://205.209.140.4" target="_blank">205.209.140.4</a>)<br>
<br>
<br>
Problem:<br>
race condition makes incorrect timestamp in replogfile, cause certain<br>
modification of entries not replicate to slurp slaves.<br>
<br>
replica: <a href="http://180.0.10.2:1234" target="_blank">180.0.10.2:1234</a><br>
replica: <a href="http://180.0.10.3:1234" target="_blank">180.0.10.3:1234</a><br>
time: 1211855467<br>
^^^^^^^^^^ this timestamp<br>
<br>
How to reproduce the problem:<br>
1) run under very slow machines (my environ: arm 266MHz)<br>
2) slapd is configed to generate replogfile<br>
3) ldapadd about 5 entries, then ldapmodify 2 entries without delay.<br>
</blockquote>
<br></div></div>
This is fixed in RE23. If there is ever a 2.3.43 release, it will be in that. In the meantime, I'd advise using 2.3.42 + your patch.<br>
<br>
Regards,<br>
Quanah<br><font color="#888888">
<br>
<br>
<br>
--<br>
<br>
Quanah Gibson-Mount<br>
Principal Software Engineer<br>
Zimbra, Inc<br>
--------------------<br>
Zimbra :: the leader in open source messaging and collaboration<br>
</font></blockquote></div><br>
------=_Part_7580_3624331.1212031000971--
14 years, 9 months
Re: (ITS#5539) openLDAP
by quanah＠zimbra.com
--On Wednesday, May 28, 2008 10:39 PM +0000 hoshahrokhi(a)gmail.com wrote:
> Full_Name: Homa Shahrokhi
> Version: 2.2.13-8.e14-6.4
> OS: Red hat 4
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (216.18.65.57)
>
>
> It is the first time that I am configuring an openLDAP.
> I download these four rpms :
> 1)openldap-2.3.30-3.fc6.i386.rpm
> 2)openldap-clients-2.3.30-3.fc6.i386.rpm
> 3)openldap-devel-2.3.30-3.fc6.i386.rpm
> 4)openldap-servers-2.3.30-3.fc6.i386.rpm
This is not provided by the OpenLDAP foundation. If you have support
issues with things provided by Redhat, you need to contact them.
Also, your issue seems to be basic knowledge questions. I suggest you read
the online documentation, and if you have further questions, send them to
openldap-software(a)openldap.org. The issue tracking system is for reporting
bugs only. This ITS will be closed.
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
14 years, 9 months
(ITS#5539) openLDAP
by hoshahrokhi＠gmail.com
Full_Name: Homa Shahrokhi
Version: 2.2.13-8.e14-6.4
OS: Red hat 4
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (216.18.65.57)
It is the first time that I am configuring an openLDAP.
I download these four rpms :
1)openldap-2.3.30-3.fc6.i386.rpm
2)openldap-clients-2.3.30-3.fc6.i386.rpm
3)openldap-devel-2.3.30-3.fc6.i386.rpm
4)openldap-servers-2.3.30-3.fc6.i386.rpm
and used "yum" to install all of them individual.
And change the conf file.
The ldap status is running.
here is my sldap.conf:
database bdb
suffix "dc=example,dc=com"
rootdn "cn=Manager,dc=example,dc=com"
rootpw password
directory /var/lib/ldap
index objectClass eq,pres
index ou,cn,mail,surname,givenname eq,pres,sub
index uidNumber,gidNumber,loginShell eq,pres
index uid,memberUid eq,pres,sub
index nisMapName,nisMapEntry eq,pres,sub
and here is my ldif file:
dn: dc=example,dc=com
objectClass:dcobject
objectClass:organization
objectClass:top
dc: example
and ran this command:
ldappadd -x -D "cn=Manager,dc=example,dc=com" -w password -f test.ldif
and here is the result:
ldap_add:Server is unwilling to perform (53)
additional info: no global superior knowledge
I add this part to the ldif file:
dn: cn=Manager,dc=example,dc=com
objectClass: organizatiolRole
cn: Manager
and ran the same command and git the same error.
I tried to use LDAP browser and I am able to connect but can not add any entry.
Could you please let me know what to do and why it happens?
I would really appreciate if someone can help.
Thanks....Homa
14 years, 9 months
Re: (ITS#5536) Always use a configured serverID in the syncrepl cookie
by rein＠OpenLDAP.org
On Wed, 28 May 2008, Howard Chu wrote:
>> A case I have is where a consumer replicates a glue'ed database, with the
>> exception of one subordinate backend where the consumer is the master. The
>> subordinate backend is replicated back to the master of the glue'ed
>> database.
>> With the current code the master would send the content of the subordinate
>> db back to its master.
>
> Understood. In fact, having multiple sources of data in a glued tree is
> really a form of multi-master. (The separate glued branches cannot cause
> inconsistencies with each other, but still their contextCSNs must be managed
> individually.)
Yes, this is a sort of multi-master configuration, but not what I think
of when I hear it (and read the doc). The doc. could be clearer that
different serverID values are always required when there are multiple
master servers, even if the configuration ensures that there will never be
more than one master for any backend.
Btw, if I have understood things correctly there cannot safely be more
then one subordinate db (in a glued environment) that replicates from any
single master, but it is OK as long as all the subordinates replicates
from different masters. That's why the syncrepl consumer in this case is
used on the glue db itself, not the subordinates. And it definitely tries
to interfere with what goes on in the subordinates (as it should).
>> A patch that fixes this is at the referenced URL. As I am not sure of the
>> consequences if a defaulted serverID=0 value was included in the syncCookie
>> the
>> patch changes the internal default slap_serverID value to -1 to make it
>> possible
>> to differentiate between a configured and defaulted serverID=0.
>
>> Btw, there are potential problems with using serverID=0, so it would be
>> best if
>> that value was reserved for the default unconfigured case. I.e, a default
>> serverID=0 value could be chosen be slapadd when the two-argument form of
>> serverID is used in the config, as resolving the URL needs the listener
>> argument
>> to slapd to succeed.
>
> You mean the three-argument form? The two-argument form only allows a single
> serverID to be configured anyway, so there is no ambiguity there. But you're
> right, in tool mode when multiple serverIDs are configured, there's no way
> for it to choose the right serverID. That's a problem regardless of whether
> the default is 0 or -1 though.
Yes, I didn't count the serverID itself as an argument.
If it was clear that different serverID values are always required if
there are more than one master server in a replication setup then it
should be safe to always include the sid in the syncrepl cookie. And the
internal distinction between a configured and defaulted serverID=0 value
this patch introduces would probably not be needed.
> For now I think this is a doc issue; we could simply recommend that slapadd
> always be performed on the node with ID 0, and you manually change the
> serverID config if you need to slapadd on some other node.
My thought was that with serverID=0 reserved for the true single-master
configuration and tool mode we could safely ignore a contextCSN with 0 in
the sid field when serverID>0 is in use (to get rid of values already
in the databases). At startup syncprov could update its own contextCSN
value with newer entryCSN values that has 0 in the sid field. Ref
ITS#5537. Slapadd'ing on more than one master without synchronizing
them between each run could still be a potential problem though..
Rein
14 years, 9 months
(ITS#5537) syncprov updates incorrect contextCSN at startup
by rein＠OpenLDAP.org
Full_Name: Rein Tollevik
Version: CVS head + rein-serverID.patch
OS: linux and solaris
URL: ftp://ftp.openldap.org/incoming/rein-syncprov.patch
Submission from: (NULL) (81.93.160.250)
Submitted by: rein
At startup syncprov searches for any entries with an entryCSN value newer than
the newest contextCSN it read from the db and replaces the newest contextCSN
value with the newest it finds. But the newest entryCSN could have a different
sid field, which would result in the server loosing one valid contextCSN and
instead introduce two contexCSNs with the same sid field. It could also
introduce a defaulted contextCSN with sid=0, which would never be updated unless
there exist a server with that sid, ref my previous ITS.
A patch that fixes this is at the referenced URL. It only updates the
contextCSN from entryCSN values matching the serverID of this server.
My initial thought was that all the contextCSNs that has newer entryCSN values
with the same sid field should be updated. But I'm afraid that could cause
syncrepl to miss some entries if it picks up the updated contextCSN values, as
there may be entries from remote servers with entryCSN values newer than the
contextCSN received from that server. The exception is delta-syncrepl where a
similar update of all the contextCSN values should probably be done at startup.
But that belongs in syncrepl.c if needed, as it is requiered whether syncprov is
used or not.
Rein Tollevik
Basefarm AS
14 years, 9 months
Re: (ITS#5536) Always use a configured serverID in the syncrepl cookie
by hyc＠symas.com
rein(a)OpenLDAP.org wrote:
> Full_Name: Rein Tollevik
> Version: CVS head
> OS: linux and solaris
> URL: ftp://ftp.openldap.org/incoming/rein-serverID.patch
> Submission from: (NULL) (81.93.160.250)
> Submitted by: rein
>
>
> Syncrepl includes the serverID in the syncCookie in multi-master mode only, but
> there are other configuration that would benefit from it as well.
>
> A case I have is where a consumer replicates a glue'ed database, with the
> exception of one subordinate backend where the consumer is the master. The
> subordinate backend is replicated back to the master of the glue'ed database.
> With the current code the master would send the content of the subordinate db
> back to its master.
Understood. In fact, having multiple sources of data in a glued tree is really
a form of multi-master. (The separate glued branches cannot cause
inconsistencies with each other, but still their contextCSNs must be managed
individually.)
> A patch that fixes this is at the referenced URL. As I am not sure of the
> consequences if a defaulted serverID=0 value was included in the syncCookie the
> patch changes the internal default slap_serverID value to -1 to make it possible
> to differentiate between a configured and defaulted serverID=0.
> Btw, there are potential problems with using serverID=0, so it would be best if
> that value was reserved for the default unconfigured case. I.e, a default
> serverID=0 value could be chosen be slapadd when the two-argument form of
> serverID is used in the config, as resolving the URL needs the listener argument
> to slapd to succeed.
You mean the three-argument form? The two-argument form only allows a single
serverID to be configured anyway, so there is no ambiguity there. But you're
right, in tool mode when multiple serverIDs are configured, there's no way for
it to choose the right serverID. That's a problem regardless of whether the
default is 0 or -1 though.
For now I think this is a doc issue; we could simply recommend that slapadd
always be performed on the node with ID 0, and you manually change the
serverID config if you need to slapadd on some other node.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
14 years, 9 months
