Re: (ITS#5065) out of order ctxcsn from old entryCSN
by hyc＠symas.com
donn(a)u.washington.edu wrote:
> On Aug 10, 2007, at 1:53 PM, Howard Chu wrote:
>> You didn't mention how your syncrepl is configured but it seems
>> that this can only occur for refreshAndPersist mode, with old
>> entries being ldapadd'd to the running master.
>
> It is indeed refreshAndPersist ...
>
>> In fact, since entryCSN is NO-USER-MODIFICATION it shouldn't even
>> be possible to add entries this way to a running server. Before we
>> can "catch the problem in the master" we need more information on
>> how the problem was caused.
>
> The master server had updatedn defined to itself. Changes, logged from
> another server, were applied with ldapmodify.
"Don't do this" ... You're essentially creating the same inconsistency that a
naive multimaster setup introduces. Since 2.4.4 has real multimaster support,
you should not be doing things this way.
Note that a proper configuration works with either refreshOnly or
refreshAndPersist mode, and cascaded to an arbitrary depth. Your current setup
only works with refreshAndPersist, and cannot be cascaded reliably. I.e. your
setup is inherently broken.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 7 months
Re: (ITS#5077) syncrepl.add_cmp() infinite loop on swapped values
by hyc＠symas.com
donn(a)u.washington.edu wrote:
> Full_Name: Donn Cave
> Version: 2.4.4
> OS: Red Hat Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (128.95.135.150)
>
>
> In "while" loop at 2656, I find old->a_vals are like [..., a, b, ...] and
> new->a_vals are like [..., b, a, ...].
>
> Matches are found at old [i] == new [k = j+1] and old [k = i+1] == new [j], but
> this just shows that neither an add nor a delete is called for, and the loop end
> is reached without having incremented i or j.
>
> Actual attribute values I'm looking at right now in the debugger are
> eduPersonAffiliation: old [ member, alum, employee, faculty ], new [ member,
> alum, faculty, employee ]. I built the replica from a slapcat dump on the idle
> master, started the slapd syncrepl client and applied a lot of normal
> modifications to the master, until it hit the infinite loop. It has been a
> recurrent problem.
Now fixed in HEAD, please test.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 7 months
Re: (ITS#5065) out of order ctxcsn from old entryCSN
by donn＠u.washington.edu
On Aug 10, 2007, at 1:53 PM, Howard Chu wrote:
>
> You didn't mention how your syncrepl is configured but it seems
> that this can only occur for refreshAndPersist mode, with old
> entries being ldapadd'd to the running master.
It is indeed refreshAndPersist ...
>
> In fact, since entryCSN is NO-USER-MODIFICATION it shouldn't even
> be possible to add entries this way to a running server. Before we
> can "catch the problem in the master" we need more information on
> how the problem was caused.
The master server had updatedn defined to itself. Changes, logged from
another server, were applied with ldapmodify.
Donn Cave, donn(a)u.washington.edu
15 years, 7 months
Re: (ITS#5065) out of order ctxcsn from old entryCSN
by hyc＠symas.com
donn(a)u.washington.edu wrote:
> Full_Name: Donn Cave
> Version: 2.4.4
> OS: RH Linux
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (128.95.135.150)
>
>
> Old entryCSN values imported into the data from another server, can crash
> replicas.
>
> In loop at top of syncrepl_updateCookie, replica encounters a syncCookie whose
> value is less than its matching si_cookieState->cs_val. This breaks out of the
> inner loop, and the outer loop, without copying anything into `first', so
> slap_queue_csn crashes on the null csn. Both are element 0 of their respective
> arrays. I assume it is no surprise that syncCookie takes its value from an
> entryCSN attribute.
>
> To duplicate, add an entry with an explicit entryCSN, with value less than the
> current contextCSN. In my case, the entryCSN is of the format without the
> `decimal fraction' field, but I doubt that matters.
>
> I don't want to say OpenLDAP needs to support this, but maybe it would be better
> to catch the problem in the master, than crash the replicas.
You didn't mention how your syncrepl is configured but it seems that this can
only occur for refreshAndPersist mode, with old entries being ldapadd'd to the
running master. As you say, this is not a supported mode of operation. New data
in the server should always have a new entryCSN as well. In fact, since
entryCSN is NO-USER-MODIFICATION it shouldn't even be possible to add entries
this way to a running server. Before we can "catch the problem in the master"
we need more information on how the problem was caused.
As for crashing the replicas - the only sure solution is to patch the code in
the replicas - "catching the problem in the master" isn't really a proper
solution anyway.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
15 years, 7 months
Re: (ITS#5085) slapo-translucent has wrong Debug message
by ghenry＠OpenLDAP.org
ghenry(a)OpenLDAP.org wrote:
> Full_Name: Gavin Henry
> Version: HEAD
> OS:
> URL: ftp://ftp.openldap.org/incoming/
> Submission from: (NULL) (212.159.59.85)
> Submitted by: ghenry
>
>
> Inside:
>
> translucent_db_destroy
>
> Debug:
>
> Debug(LDAP_DEBUG_TRACE, "==> translucent_db_close\n", 0, 0, 0);
>
> Should be:
>
> Debug(LDAP_DEBUG_TRACE, "==> translucent_db_destroy\n", 0, 0, 0);
>
> Probably copy and paste error.
>
> Correcting.
>
> Gavin.
>
>
This is missing from RE_23, but obviously not going to be added. There
is another error in RE_23 though:
Debug(LDAP_DEBUG_TRACE, "==> translucent_init\n", 0, 0, 0);
should be translucent_db_init.
Will change under this ITS.
Gavin.
--
Kind Regards,
Gavin Henry.
OpenLDAP Engineering Team.
E ghenry(a)OpenLDAP.org
Community developed LDAP software.
http://www.openldap.org/project/
15 years, 7 months
(ITS#5086) back-bdb bug when add is used with snapshot isolation
by stelios.xx.grigoriadis＠ericsson.com
Full_Name: Stelios Grigoriadis
Version: 2.3.32
OS: Linux, (SLES 10)
URL: http://ninikos.freehostia.com/bdb_idl_cache_bug.tar.gz
Submission from: (NULL) (194.237.142.7)
When using slapd with the snapshot isolation level in DB (version 4.5.20)
it is possible that added entries don't appear in subsequent searches.
Scenario: A client is adding new information while at the same time another
is initiating an LDAP search with the same searchbase as the "parent" of
the added information.
If the search is executed after the add operation has cleared the IDL-cache
but before the changes has been committed, the added entries will not appear
in any subsequent searches unless slapd is restarted.
The problem is that, in the bdb_idl_insert_key-function, the key is deleted
from the IDL-cache but the newly added keys in the database are not yet
committed, so clients that re-read the information will not have those updated
keys.
The solution would be to clear the relevant IDL-cache entries after the
changes has been committed.
This behaviour is demonstrated by the program cache_test_fork.c which is
supplied
in bdb_idl_cache_bug.tar.gz along with the slapd configuration.
I am currently working on the solution but would greatly appreciate any help
from you as I am not nearly as familiar with the code as you are.
15 years, 7 months
(ITS#5085) slapo-translucent has wrong Debug message
by ghenry＠OpenLDAP.org
Full_Name: Gavin Henry
Version: HEAD
OS:
URL: ftp://ftp.openldap.org/incoming/
Submission from: (NULL) (212.159.59.85)
Submitted by: ghenry
Inside:
translucent_db_destroy
Debug:
Debug(LDAP_DEBUG_TRACE, "==> translucent_db_close\n", 0, 0, 0);
Should be:
Debug(LDAP_DEBUG_TRACE, "==> translucent_db_destroy\n", 0, 0, 0);
Probably copy and paste error.
Correcting.
Gavin.
15 years, 7 months
Re: (ITS#5074) slapo-pcache's proxyattrset does not recognize '1.1' to describe an empty attribute set
by ando＠sys-net.it
> On Aug 6, 2007, at 6:43 AM, Pierangelo Masarati wrote:
>> I believe there's something odd in pcache's current behavior. In
>> fact, right now "proxyattrset 0" is already a valid directive, but
>> it implies "proxyattrset 0 *", which means all attributes.
>> Unfortunately, this seems to only work if no attribute is
>> requested, while it fails if a "*" is explicitly requested.
>
> Ah, I was not aware of this behaviour!
I've noticed it from the sources, I don't think it was noted anywhere.
>> I've modified the propxycache code in HEAD so that a "1.1" can only
>> explicitly appear in a list if it's the only string. An empty
>> attribute set is no longer valid, to avoid confusion about the
>> meaning of non-explicit attribute sets. A "*" or a "+" can
>> explicitly appear in a proxyattrset statement, resulting in the
>> expected behavior. Use both to indicate that all attributes are in
>> the set.
>
> Let me confirm, that I should specify "proxyattrset 0 1.1" to match
> an empty attribute list, and only an empty attribute list. Will a
> '*' than match emptiness as well, or does it only match where there
> is some attribute specified by the client?
No: you need to specify
proxyattrset <#idx> "*"
in order to have either nothing or "*" return all attributes. In short,
an empty attribute set and "*" are now recognized as equivalent.
You need to specify
proxyattrset <#idx> "1.1"
in order to have searches requesting "1.1", namely no attributes, be cached.
> I will work on this as soon as I am able to, although it seems this
> won't be for a couple of weeks. I will be in touch. Thanks again
> for your work and for your very timely response.
Right now, I'm not planning to backport this fix to re23 because it seems
to be an enhancement rather than a bugfix, and 2.4 __should be__ just
'round the corner.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
---------------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Email: pierangelo.masarati(a)sys-net.it
---------------------------------------
15 years, 7 months
Re: (ITS#5074) slapo-pcache's proxyattrset does not recognize "1.1" to describe an empty attribute set
by zep＠ni.enate.org
On Aug 6, 2007, at 6:43 AM, Pierangelo Masarati wrote:
> I believe there's something odd in pcache's current behavior. In
> fact, right now "proxyattrset 0" is already a valid directive, but
> it implies "proxyattrset 0 *", which means all attributes.
> Unfortunately, this seems to only work if no attribute is
> requested, while it fails if a "*" is explicitly requested.
Ah, I was not aware of this behaviour!
> I've modified the propxycache code in HEAD so that a "1.1" can only
> explicitly appear in a list if it's the only string. An empty
> attribute set is no longer valid, to avoid confusion about the
> meaning of non-explicit attribute sets. A "*" or a "+" can
> explicitly appear in a proxyattrset statement, resulting in the
> expected behavior. Use both to indicate that all attributes are in
> the set.
Let me confirm, that I should specify "proxyattrset 0 1.1" to match
an empty attribute list, and only an empty attribute list. Will a
'*' than match emptiness as well, or does it only match where there
is some attribute specified by the client?
> Please test.
I will work on this as soon as I am able to, although it seems this
won't be for a couple of weeks. I will be in touch. Thanks again
for your work and for your very timely response.
-Mike
15 years, 7 months
