contextCSN of subordinate syncrepl DBs
by Rein Tollevik
I've been trying to figure out why syncrepl used on a backend that is
subordinate to a glue database with the syncprov overlay should save the
contextCSN in the suffix of the glue database rather than the suffix of
the backend where syncrepl is used. But all I come up with are reasons
why this should not be the case. So, unless anyone can enlighten me as
to what I'm missing, I suggest that this be changed.
The problem with the current design is that it makes it impossible to
reliably replicate more than one subordinate db from the same remote
server, as there are now race conditions where one of the subordinate
backends could save an updated contextCSN value that is picked up by the
other before it has finished its synchronization. An example of a
configuration where more than one subordinate db replicated from the
same server might be necessary is the central master described in my
previous posting in
http://www.openldap.org/lists/openldap-devel/200806/msg00041.html
My idea as to how this race condition could be verified was to add
enough entries to one of the backends (while the consumer was stopped)
to make it possible to restart the consumer after the first backend had
saved the updated contextCSN but before the second has finished its
synchronization. But I was able to produce it by simply add or delete
of an entry in one of the backends before starting the consumer. Far to
often was the backend without any changes able to pick up and save the
updated contextCSN from the producer before syncrepl on the second
backend fetched its initial value. I.e it started with an updated
contextCSN and didn't receive the changes that had taken place on the
producer. If syncrepl stored the values in the suffix of their own
database then they wouldn't interfere with each other like this.
There is a similar problem in syncprov, as it must use the lowest
contextCSN value (with a given sid) saved by the syncrepl backends
configured within the subtree where syncprov is used. But to do that it
also needs to distinguish the contextCSN values of each syncrepl
backend, which it can't do when they all save them in the glue suffix.
This also implies that syncprov must ignore contextCSN updates from
syncrepl until all syncrepl backends has saved a value, and that
syncprov on the provider must send newCookie sync info messages when it
updates its contextCSN value when the changed entry isn't being
replicated to a consumer. I.e as outlined in the message referred to above.
Neither of these changes should interfere with ordinary multi-master
configurations where syncrepl and syncprov are both use on the same
(glue) database.
I'll volunteer to implement and test the necessary changes if this is
the right solution. But to know whether my analysis is correct or not I
need feedback. So, comments please?
--
Rein Tollevik
Basefarm AS
13 years, 6 months
dITStructureRules/nameForms in subschema subentry for informational purpose
by Michael Ströder
HI!
Discussed this very briefly with Howard at LDAPcon 2007 based on an idea
of Steve:
Support for dITStructureRules and nameForms is still in OpenLDAP's TODO.
In the meanwhile slapd could accept definitions for both in slapd.conf
and simply pass them on to a schema-aware LDAP client for informational
purpose without enforcing them. Same function like rootDSE <file> in
slapd.conf.
Opinions?
Ciao, Michael.
--
Michael Ströder
E-Mail: michael(a)stroeder.com
http://www.stroeder.com
14 years, 2 months
Enforcing attribute ACL on add operations
by manu＠netbsd.org
Hello
Right now, slapd ignore attribute ACL when performing an add operation.
If you have privilegied users that can add entries, it means that you can
prevent them from modifying attributes in existing entries, but you cannot
prevent them from adding an entry with a read-only attribute.
The problem can be interesting with an attribute such as authzTo, where the
whole access control can be circumvented by any user that can create an
entry in the tree. IMO this behavior was not intended, but if it was, then
it should be clearly documented.
Below is a patch that cause attribute ACL to be checked for add operations.
It is done in the backend, so if it is acceptable, then I will have to do it
for other backends. I wonder if the modrdn operation shoulnd't be subject to
the same sanity checks.
Any thought? Does it look right?
diff -U2 -r1.174 add.c
--- servers/slapd/back-bdb/add.c 26 Aug 2008 23:45:35 -0000 1.174
+++ servers/slapd/back-bdb/add.c 27 Sep 2008 15:54:58 -0000
@@ -300,4 +300,22 @@
}
+ /*
+ * Check ACL for attribute write access
+ */
+ if (!acl_check_modlist(op, oe, op->ora_modlist)) {
+ switch( opinfo.boi_err ) {
+ case DB_LOCK_DEADLOCK:
+ case DB_LOCK_NOTGRANTED:
+ goto retry;
+ }
+
+ Debug( LDAP_DEBUG_TRACE,
+ LDAP_XSTRING(bdb_add) ": no write access to
attribute\n",
+ 0, 0, 0 );
+ rs->sr_err = LDAP_INSUFFICIENT_ACCESS;
+ rs->sr_text = "no write access to attribute";
+ goto return_results;;
+ }
+
if ( eid == NOID ) {
rs->sr_err = bdb_next_id( op->o_bd, &eid );
--
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu(a)netbsd.org
14 years, 8 months
Fwd: RE24 testing
by Brett ＠Google
On Thu, Sep 11, 2008 at 2:33 AM, Quanah Gibson-Mount <quanah(a)zimbra.com> wrote:
> Please test RE24.
>
> Thanks!
>
> --Quanah
on suse 10.2 SMP / xen kernel, ie :
Linux suse 2.6.22.18-0.2-xen #1 SMP 2008-06-09 13:53:20 +0200 x86_64
x86_64 x86_64 GNU/Linux
test failed - producer and consumer2 databases differ
>>>>> ./scripts/test050-syncrepl-multimaster failed (exit 1)
make[2]: *** [bdb-yes] Error 1
make[2]: Leaving directory `/home/maxfieb/keep-ldap/openldap-src-RE24/tests'
make[1]: *** [test] Error 2
make[1]: Leaving directory `/home/maxfieb/keep-ldap/openldap-src-RE24/tests'
make: *** [test] Error 2
compiling against bdb 4.7.25.NC (no patched applied)
bdb options :
../dist/configure --disable-java --enable-posixmutexes
--enable-largefile --enable-cxx --disable-compat185
make
make install
openldap configure :
CFLAGS="-g -g3 -ggdb -ggdb3" CPPFLAGS="-I/usr/local/BerkeleyDB.4.7/include" \
LDFLAGS="-L/usr/local/BerkeleyDB.4.7/lib -R/usr/local/BerkeleyDB.4.7/lib" \
./configure --enable-slapd --enable-backends --enable-shared \
--enable-overlays --disable-ipv6 --disable-sql --disable-ndb --with-threads \
--enable-ssl --enable-collect --enable-debug
building HEAD with same openldap configure options and same bdb
binaries above, all tests succeed (albeit much more slowly.. perhaps
DB_CONFIG needs tweaking in head ?)..
Cheers
Brett
14 years, 8 months
Please test RE24
by Quanah Gibson-Mount
Please test RE24. Issues remain with test050, but are not considered
release blocking as they are not a regression.
Thanks,
Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
14 years, 8 months
Re: commit: ldap/servers/slapd/overlays ppolicy.c
by Pierangelo Masarati
ando(a)OpenLDAP.org wrote:
> Update of /repo/OpenLDAP/pkg/ldap/servers/slapd/overlays
>
> Modified Files:
> ppolicy.c 1.116 -> 1.117
>
> Log Message:
> allow controls deletion to be deferred; pass response controls back to frontend in extops (ITS#5711)
Please disregard, this patch is broken. A fix will appear shortly.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
14 years, 8 months
Re: Please test RE24
by ghenry＠OpenLDAP.org
----- "Quanah Gibson-Mount" <quanah(a)zimbra.com> wrote:
> Please test RE24. Issues remain with test050, but are not considered
>
> release blocking as they are not a regression.
Only test50 fails for me on dual core 386 with 4.7.25 bdb.
--
Kind Regards,
Gavin Henry.
OpenLDAP Engineering Team.
E ghenry(a)OpenLDAP.org
Community developed LDAP software.
http://www.openldap.org/project/
14 years, 8 months
Use of oidValidate() in schema-related attrs?
by Pierangelo Masarati
We currently use oidValidate() in the hardcoded definition of attributes
like dITContentRules, matchingRules, attributeTypes, objectClasses,
nameForms, matchingRuleUse. However, this only looks correct as a
validator for an assertion value of the related syntaxes, while it fails
for valid attribute values. I understand this is a moot point, since we
do not allow direct writing of cn=subschema and subschema subentries in
general. However this causes, for example, an incorrect error to be
returned when one attempts to write, for example, cn=subschema.
p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
-----------------------------------
Office: +39 02 23998309
Mobile: +39 333 4963172
Fax: +39 0382 476497
Email: ando(a)sys-net.it
-----------------------------------
14 years, 8 months
New test054-syncrepl-asymmetric
by Rein Tollevik
I have created a test script that shows the contextCSN propagation
problems we have, as well as a patch that fixes most of them. But
before I commit anything or files more ITSs I'd like to receive
comments. The test script and patch can be found at:
ftp://ftp.openldap.org/incoming/test054-syncrepl-asymmetric
ftp://ftp.openldap.org/incoming/test054-syncrepl-asymmetric.patch
The comments in the scripts should hopefully explain the configuration.
The patch adds support for the newCookie sync info protocol messages,
and sends them when the contextCSN is being updated without any changes
being sent from the syncprov provider (i.e when the entries that changed
didn't match the syncrepl filter and/or was forbidden by ACL rules). The
queuing of the CSNs done in syncrepl_updateCookie should not be
necessary with this patch, it should be removed after the patch has been
committed. The bug reported in ITS#5710 is also worked around by this
patch, but defining SLAP_SYNC_UPDATE_MSGID with an unique value would be
better.
This test requires the proposed patch to fix ITS#5572 to be applied.
Dynamically creating the configuration doesn't work as expected without
it, and the number of errors reported by the script increases from 10 to
25. Mostly due to the replication starting in an unexpected state, and
several restarts and writes are needed before an expected state is
reached. This patch can be found at:
ftp://ftp.openldap.org/incoming/ITS5572.patch
The test script finds 10 errors (assuming the race condition discussed
below is actually hit) after the ITS#5572 patch has been applied With
both patches I'm left with 3 errors, all related to the race condition.
Howard, you'll probably object to the rootdn usage in this test. But as
of now there is no way I know of that allows me to create the layout I
need without it. Extending syncrepl with the ability to exclude a list
of dn subtrees from its control could be the solution. But in that case
I assume it would be better to have a list of URIs (evaluated locally)
that defines the entries to exclude. Which again opens a can of new
questions as to when the URIs should be evaluated. For add and delete
it should be pretty obvious, but what about modify? Test against the
entry as it is before the change, would be after or both?
Hmm, the ability to exclude a list of URIs from syncrepl control could
be (ab)used to merge some type of entries from one server, other types
from other servers, if anyone should wish to do such weired things..
The errors remaining after these two patches have been applied are due
to the race condition that arise if more then one subordinate database
replicates from the same provider. I have a clear idea as to how this
race can be eliminated, it must be addressed in both syncrepl and syncprov:
In syncrepl it should be fairly easy, as storing the contextCSN values
in the suffix of the database where syncrepl is configured rather than
the suffix of the glue entry should be sufficient.
Syncprov needs a bit more work. Whenever it detects that syncrepl
updates the contextCSN value it must use the *lowest* contextCSN value
for any given sid stored in all the databases within its context. And
no updates must be accepted from syncrepl until a contextCSN value has
been stored in the suffix dn of all databases (within its context) where
syncrepl has been enabled. Assuming syncrepl cannot be used on both a
superior and a subordinate database (which could no be true if the
ability to exclude something from its control is introduced) it should
not be required to make these extra tests when syncrepl updates the
contextCSN in the suffix of the glue database itself (i.e when syncrepl
and syncprov are configured in the same database). The sending of
newCookie messages should cause the contextCSN values to be updated to
the newest values fairly quickly.
Actually, there could be another problem buried here as well, as the
consumers really need a full resynchronization after new subordinate
syncrepl consumers are added on their provider (at least if the new
consumer replicates from the same server as an existing). The easiest
fix would be to require the clients to do a full resync whenever the
config of the provider is changed, which I find a bit drastic. Noting
it in the doc. should hopefully be sufficient.
Rein Tollevik
Basefarm AS
14 years, 8 months
Re: commit: ldap/servers/slapd/back-bdb add.c back-bdb.h bind.c cache.c compare.c config.c delete.c dn2entry.c dn2id.c filterindex.c id2entry.c idl.c init.c key.c modify.c modrdn.c proto-bdb.h referral.c search.c tools.c
by Howard Chu
hyc(a)OpenLDAP.org wrote:
> Update of /repo/OpenLDAP/pkg/ldap/servers/slapd/back-bdb
>
> Modified Files:
> add.c 1.173 -> 1.174
> back-bdb.h 1.173 -> 1.174
> bind.c 1.54 -> 1.55
> cache.c 1.180 -> 1.181
> compare.c 1.57 -> 1.58
> config.c 1.107 -> 1.108
> delete.c 1.172 -> 1.173
> dn2entry.c 1.35 -> 1.36
> dn2id.c 1.156 -> 1.157
> filterindex.c 1.75 -> 1.76
> id2entry.c 1.86 -> 1.87
> idl.c 1.134 -> 1.135
> init.c 1.291 -> 1.292
> key.c 1.24 -> 1.25
> modify.c 1.176 -> 1.177
> modrdn.c 1.200 -> 1.201
> proto-bdb.h 1.159 -> 1.160
> referral.c 1.50 -> 1.51
> search.c 1.265 -> 1.266
> tools.c 1.127 -> 1.128
>
> Log Message:
> Use read-only txn's instead of read lockers. Support BDB 4.4-4.7
Please test... At the moment I've only run it with BDB 4.7.25. I'll be testing
with older releases over the next couple of days. In particular, take note of
any LDAP_OTHER/Internal Error results in the test suite. Right now it passes
all tests for me on bdb and hdb.
If push comes to shove we can probably create a new BerkeleyDB4.2.patch file
to get 4.2 working again, but at this point most people should already be on
something newer, and people stuck on 4.2 probably won't compile their own anyway.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
14 years, 8 months
