Rex Roof wrote:

On Jul 9, 2009, at 8:53 AM, "Michael Stro"der" michael@stroeder.com michael@stroeder.com wrote:

...

Bad Guy wrote:

...

I have a 4 way Multi-master LDAP synchronization version 2.4.11 [..] From time to time, there is some records deleted in one of the LDAP server will not be replicated to the other servers occasionaly.

If using MMR you should really update to at least 2.4.16 or even better wait for 2.4.17 to be released really soon now.

as someone who is thinking about implementing MMR, can you explain why?

There have been numerous bug fixes to MMR and mirror-mode after release 2.4.11. Please check the change log of branch OPENLDAP_REL_ENG_2_4:

http://www.openldap.org/devel/cvsweb.cgi//Attic/CHANGES

The bug fixes contain the ITS numbers which you can lookup here:

http://www.openldap.org/its/

Ciao, Michael.

--On Thursday, July 09, 2009 1:09 PM -0400 Rex Roof rex@wccnet.edu wrote:

How are we supposed to be setting up OpenLDAP servers so that we can have failover? I want to have a half-dozen OpenLDAP machines and have any one of them accept writes and not be relying on any one of them to be up for writes to occur. I've asked this question numerous times and haven't ever gotten a reliable answer. It is starting to get really annoying.

Well, welcome to the world of distributed systems. You probably want back-ndb once a few of its indexing quirks have been worked out, since that uses multiple slapd instances to frontend a single mysql cluster database. You could of course, use mirror mode and chaining as an alternative. But in any sort of distributed system like MMR, changes get delayed.

--Quanah

--

Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

--On Thursday, July 09, 2009 1:21 PM -0400 Rex Roof rex@wccnet.edu wrote:

back-ndb is for mysql?

so you're suggesting that I mirror the data using a mysql cluster and just build multiple frontend servers to run slapd, all connecting to the same mysql backend database?

That is the point of back-ndb. It gets rid of all the issues of replication delays, and mysql takes care of the database failover. And please don't top post. :)

--Quanah

--

Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

--On Thursday, July 09, 2009 1:36 PM -0400 Rex Roof rex@wccnet.edu wrote:

sorry, it's how I prefer my email. I realize it isn't best for mailing list archives.

doesn't that just say that openldap can't properly handle failover, then? if you're relying on mysql to handle the failover? I feel like this is way overcomplicating my LDAP setup. I'm dealing with fewer than 200,000 objects in my LDAP tree, I feel like it should be a simple thing to provide an adequate failover setup with multiple servers, all accepting writes, and not relying on one single server to be up for writes.

I'd even be happy with two servers mirroring and other servers slaving off of them via sync-repl.

I rely on the experts because I don't understand the internals of OpenLDAP. would you really not suggest using MMR? if not, why is it an option in the stable release of the code?

The issue you are trying to solve is not unique to OpenLDAP. Any LDAP implementation has them. Now, like I suggested in a previous email in this thread, what you probably want is mirror mode and chaining. Mirror-Mode sets up a single master with a mirrored failover master that will take over after a heartbeat monitor failure. Setting up chaining on the replicas causes any modifications made to them to be forwarded to the master. Or, you could do MMR and chaining, but like any replicated service, updates will take an unknown amount of time to propogate from any write-accepting server to the rest of the servers in the pool. That's not OpenLDAP or even LDAP specific. The *only* way to avoid replication delays is to not use replication -- which implies back-ndb or a single server (which lacks failover).

--Quanah

--

Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration

Rex Roof wrote:

Gleaning any information out of that list is an arduous process. A short description of each of those ITS bugs in the CVS comments would serve that changes list well.

$ grep -i "mmr" CHANGES Fixed slapd syncrepl too many MMR messages (ITS#6020) Fixed slapd syncrepl skipped entries with MMR (ITS#5988) Fixed slapd-syncprov too many MMR messages (ITS#6020) Fixed slapo-syncprov skipped entries with MMR (ITS#5988) admin24 clarified MMR URI requirements (ITS#5942,ITS#5987) Fixed slapd glue with MMR (ITS#5925) Fixed slapd syncrepl MMR when adding new server (ITS#5850) Fixed slapd syncrepl MMR with deleted entries (ITS#5843) Fixed slapd syncrepl MMR partial refresh (ITS#5470) Fixed slapd-bdb MMR (ITS#5332)

Now this line seems relevant to the issue of the original poster:

Fixed slapd syncrepl MMR with deleted entries (ITS#5843)

So what's wrong with the CHANGES file?

but as an administrator of OpenLDAP it is really frustrating to read about Multi-Master Replication and to have people make random comments on the mailing lists about MMR being dangerous and to read 5 year old papers about how harmful it is ( http://www.watersprings.org/pub/id/draft-zeilenga-ldup-harmful-02.txt ) but yet the OpenLDAP 2.4 docs don't really warn against using it.

That's a completely different issue and it has nothing to do with which LDAP server implementation you're using. The I-D above is worth reading. Hope you did and understood it.

How are we supposed to be setting up OpenLDAP servers so that we can have failover?

That depends entirely on your deployment. Note that in general not your OpenLDAP server does the fail-over. Your LDAP clients have to do the fail-over or a component in front of your servers. Whether that really works depends on the characteristics of read/write operations sent by your LDAP client applications.

Examples:

If you're just doing single write operations and all your LDAP clients are doing simple logins with some password changes here and there you don't have to worry about doing multi-master replication. You might loose a write operation when one server is going down. But you won't loose data integrity.

If you have LDAP client applications which sends a bunch of interrelated writes or it's immediately reading the written content after writing you will have a lot of weird issues even in cases where everything's fine. Not to speak of some of the writeable servers going down without the application really noticing and acting upon it.

=> As always you have to carefully think about *your* deployment. There's no simple rule-of-thumb which fits every scenario. Being able to design something like this in a robust manner distinguishs good IT architects from bad IT architects.

I want to have a half-dozen OpenLDAP machines and have any one of them accept writes and not be relying on any one of them to be up for writes to occur. I've asked this question numerous times and haven't ever gotten a reliable answer.

It would be grossly negligent to give a simple answer to such a question. You should appreciate when people give you a hint that things might be more complicated than you thought.

It is starting to get really annoying.

Well, if you're not willing to think yourself you have to buy skilled people for analyzing/designing your architecture.

Ciao, Michael.