contextCSN propagation problems
by Rein Tollevik
The worst problems we had after upgrading to 2.4.x seems to be over now,
and the replication appear to work (at least mostly) as it should. One
problem that still remains is that the contextCSN attributes themselves
don't propagate as I wish they would. Before I start coding or filing
more ITSes I would like to have comments as to whether my analysis of
the problem seems correct or not.
First, a description of our configuration is needed. I think a test
script for this configuration would be nice to have, so I'm going to
create that before I do anything else. I currently think of it as
syncrepl-asymmetric, but suggestions for better names are welcome.
Now the configuration, which is centered around a central master server.
It has a glued database with a set of subordinates. The central master
is the (sole) master for most of these subordinates, but there are also
a set of remote site-masters that each has one subordinate database that
they are the master for and that is replicated back to the central master.
The site-masters has a similar glued configuration, and they replicates
the glue entry from the central master. Different rootdn values on the
subordinates managed by syncrepl and the one the site-master is the
master for prevents syncrepl from wiping out the content of that
database during the present phase. None of the site-masters receives all
the subordinate databases the central master has, and which they receive
varies. This is controlled by acl rules on the central master.
On each of the sites (including where the central master is) there are
search-only servers that replicates the glue suffix from their
site-master (or the central master). The search servers has a single
database (for historical reasons), but their layout shouldn't matter
very much.
All of the master servers uses syncprov on the glue database, and
everyone except the central master uses syncrepl on the glue database.
The central master cannot use it there, as that would have caused it to
wipe out those subordinates that aren't on the site-master it replicates
from during the refresh phase. Different serverIDs are used on the
master servers, so an updated contextCSN set should include as many
values as there are master servers.
Now to the problem. If a modification is made (on the central master) to
a subordinate database that isn't replicated to one of its consumers it
will not receive the updated contextCSN value from the central master.
Which means that I cannot monitor the contextCSN values to verify that
the replication is working as it should. And the consumers will (after
a restart) present an outdated contextCSN set during the refresh phase,
even though their database content is up to date. But this will be
corrected when/if an update is made to a subordinate db that is
replicated to the consumer.
A worse problem is when the modifications is made to a subordinate db
the central master replicates from one of the site-masters. In this
case there will never be any updates from that site-master that should
be propagated. So the consumers will, until it restarts, be stuck with
the contextCSN value matching that remote site-master which it received
after the present phase.
Or, oh well, that is not quite true... Due to a bug in syncprov it
fails (on the central master) to detect and filter out the contextCSN
update from syncrepl on a subordinate db. Which means that the central
master will send an update of the glue entry to its consumers, so that
the value is propagated from the central master to its immediate
consumers. But it will not propagate further from the site-masters to
the search servers on their site. As this bug makes things work at
least partly as I wish I'm a bit reluctant to have it fixed yet...
So far my proposed solution to this problem is that syncprov_matchops()
should, when a modification fails to match the test_filter() nor is a
deleted entry, send a sync info protocol message with the updated
contextCSN in the newCookie field to its consumers. Does this sound
like a valid solution? There seem to be support in syncrepl.c for
receiving these messages, and in syncprov.c for sending them. But it
never actually does it as far as I can see.
--
Rein Tollevik
Basefarm AS
13 years, 4 months
demand slapindex after slapd.conf += index
by Hallvard B Furuseth
A suggestion for a vain attempt to kill a FAQ:
To teach people to run sbin/slapindex after adding an index to
slapd.conf, slapd can refuse to start if an index file is missing but
dn2id.bdb exists. Same with other slap tools that need to open/create
the index file. (At least slapadd, I don't know if there are others.)
slapindex and slapcat, at least, will need to keep working.
If database recovery can sabotage this feature or vice versa, recovery
will of course have to take precedence.
--
Hallvard
13 years, 4 months
RE24 testing (again)
by Quanah Gibson-Mount
Pulled in a few more things, please re-sync and test. These were the last
outstanding changes known for 2.4.10.
Thanks!
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
13 years, 4 months
Re: RE24 testing (again)
by Raphaël Ouazana-Sustowski
Hi,
Le Mar 3 juin 2008 10:37, Jonathan Clarke a écrit :
> Gavin Henry a écrit :
>> <quote who="Howard Chu">
>>> Quanah Gibson-Mount wrote:
>>>> Pulled in a few more things, please re-sync and test. These were the
>>>> last
>>>> outstanding changes known for 2.4.10.
>>> All tests OK here. OpenSuSE 10.2, x86-64
>>
>> Likewise. Fedora 8/9 i386/x86_64, Centos 5.2 i386/x86_64 and Debian Etch
>> (4.0) i386/x86_64
>
> Also OK for me on Ubuntu i386 and Debian x86_64.
Idem here on Ubuntu/i386/dual core. Note that this time test050 (on hdb)
ran more then 200 times without any problem.
Regards,
Raphaël Ouazana.
13 years, 4 months
AVL -> T-trees
by Howard Chu
Taking a cue from our MySQL friends - MySQL uses T-trees for their in-memory
structures. These are balanced trees, like AVL trees. But instead of just one
data item per tree node, they have N items per node. (Presumably N is a
compile-time constant.) The advantage to using T-trees is that inserts and
deletes have less impact on the overall tree, thus minimizing the need for
rebalancing.
I would expect that they perform about as well as AVL trees for lookups.
Anyone interested in experimenting here and reporting on the relative performance?
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
13 years, 4 months
Re: AVL -> T-trees
by Howard Chu
Simon Riggs wrote:
> Hi Gavin,
>
> PostgreSQL uses B-trees as the main index type, though also supports
> GIST, GIN and Hash index types. We avoid re-balancing the trees at
> delete time, which is a high source of contention.
>
> MySQL, to my knowledge, doesn't support T-Trees?
> http://dev.mysql.com/doc/refman/5.0/en/create-index.html
As I noted to Gavin, that's a different thing entirely. Our on-disk indexing
uses B-trees as well, but our in-memory caches use AVL trees. I would love to
try a B-link tree implementation for our on-disk data, but that's yet another
story...
> T-Trees are a class of index known to be cache conscious, but they also
> perform poorly when I/O is involved. Selecting that would mean you'd
> need to know in advance the memory allocation given to your server,
> which may not be as much as you want and can change over time. You can
> have non-persistent indexes, though rebuilding indexes at startup can be
> annoying also.
I/O is not a consideration for this purpose.
> Does OpenLDAP experience a high insert/delete rate? I would have
> expected most uses to be mostly read-only, percentage-wise, but I guess
> you'll set me straight.
Yes, there is a high rate of insert/deletes into the caches when the cache is
smaller than the working set. Since T-trees allow more data items to be stored
per tree node, that also means we could have less overhead in tree link pointers.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
13 years, 4 months
Please retest RE24
by Quanah Gibson-Mount
Two new bug fixes for RE24 committed today. Please retest. Hopefully
this'll be all for 2.4.10.
Thanks!!
--Quanah
--
Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration
13 years, 4 months
Re: commit: ldap/contrib/slapd-modules/nssov - New directory
by Howard Chu
hyc(a)OpenLDAP.org wrote:
> Update of /repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/nssov
> Log Message:
> Directory /repo/OpenLDAP/pkg/ldap/contrib/slapd-modules/nssov added to the repository
I've imported a complete copy of Arthur de Jong's nss-ldapd-0.6.2 with this
overlay. The overlay implements a listener inside slapd that speaks the same
protocol as nss-ldapd. As such, it replaces the server side of his package
(nslcd). You still need to build and install his client side though (nss).
It passes the majority of the "make check" tests in the nss-ldapd/tests
directory on my OpenSuSE system. The ones that fail appear to be incorrectly
written tests. Unfortunately those tests are heavily dependent on your
system's nsswitch.conf and the other databases; they ought to be cleaned up to
be completely self-contained.
The point of all this: the nss-ldapd approach avoids the issue of polluting
the user space with libldap's symbols, by sending all requests thru a small
nss stub. This stub sends requests (using a very simple protocol) over a Unix
Domain socket to some other server which actually processes the requests. With
the original nss-ldapd, an nslcd daemon listening on that socket then uses
libldap to contact whatever LDAP server was configured.
With this overlay, slapd itself answers the nss requests. On a host with the
master database, this avoids an unnecessary context switch if nothing else.
Even on hosts without the full database, this approach opens up the
possibility of using pcache to perform intelligent caching of nss data, as
well as using syncrepl to keep information current.
I've also discussed with Arthur some directions for improvement in the base
nss code. I may push some of those into here later.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
13 years, 4 months
