Hello,
So here's my situation, I need to setup two openldap instances that will be setup behind a load balancers and be redundant for each other. But i don't want any referrels' to another "master" instance to make the writes. I want both instances to be able to write and do the update without a referral and also have the same data across both instances. I am having trouble figuring out how to setup openldap to do that but from what I have read so far, syncrepl seems to be what I should use to get close to what I want. But I still need some assistance.
If a new entry 'A' gets added to the provider master (ligit entry) and then a new entry B gets added to the consumer database (ligit entry), when they replicate, will both provider and master have the A and B entries? Or will the consumer database be screwed and whatever the provider has is it?
It doesn't haven't to be Multi-master but I do what the two instances to have the same data and be able to update requests to modify/add/delete entries without a refferal..
Hugo
It doesn't haven't to be Multi-master but I do what the two instances to have the same data and be able to update requests to modify/add/delete entries without a refferal..
Lots of FAQ and list archive on this. A syncrepl slave won't allow writes, so that's out of the picture.
If you really need to load balance OpenLDAP 2.3, I submit that something is wrong with what you're doing with the directory -- on decent hardware, you could easily serve tens of thousands of requests/sec.
If all you're really trying to accomplish is HA, there are valid ways of doing it.
John
John Madden wrote:
It doesn't haven't to be Multi-master but I do what the two instances to have the same data and be able to update requests to modify/add/delete entries without a refferal..
Lots of FAQ and list archive on this. A syncrepl slave won't allow writes, so that's out of the picture.
As already suggested in another post, syncrepl + the chain overlay will handle this.
If you really need to load balance OpenLDAP 2.3, I submit that something is wrong with what you're doing with the directory -- on decent hardware, you could easily serve tens of thousands of requests/sec.
Agreed. Or perhaps they're just accustomed to using a slower LDAP server from some other vendor... I've talked to engineers at a couple of other directory server projects, and they flatly admit that they can't get anywhere near our performance.
If all you're really trying to accomplish is HA, there are valid ways of doing it.
E.g. MirrorMode in OpenLDAP 2.4 (also available in Symas CDS).
On Thursday 01 February 2007 17:22, John Madden wrote:
It doesn't haven't to be Multi-master but I do what the two instances to have the same data and be able to update requests to modify/add/delete entries without a refferal..
Lots of FAQ and list archive on this. A syncrepl slave won't allow writes, so that's out of the picture.
If you really need to load balance OpenLDAP 2.3, I submit that something is wrong with what you're doing with the directory -- on decent hardware, you could easily serve tens of thousands of requests/sec.
Hmm, I load balance OpenLDAP 2.3. Why ? Because, on the occasions we have heavy writes (averaging 30-40 modifications per second over a few hours), one slaves isn't guaranteed to respond to our radius servers within the time we have to send a response back to the radius proxy. Under the high write load, 1 slave would not handle the full read load (about 1000 operations per second), but each slave can normally still handle about 250 operations per second without too much delay.
Sure, pure read performance is much higher than I need (easily 10000 searches/sec on boxes with no write load), but you need to consider worst-case load over the entire lifetime of the installation, and write load can have a big impact.
Regards, Buchan
--On Friday, February 02, 2007 11:29 AM +0200 Buchan Milne bgmilne@staff.telkomsa.net wrote:
Hmm, I load balance OpenLDAP 2.3. Why ? Because, on the occasions we have heavy writes (averaging 30-40 modifications per second over a few hours), one slaves isn't guaranteed to respond to our radius servers within the time we have to send a response back to the radius proxy. Under the high write load, 1 slave would not handle the full read load (about 1000 operations per second), but each slave can normally still handle about 250 operations per second without too much delay.
Sure, pure read performance is much higher than I need (easily 10000 searches/sec on boxes with no write load), but you need to consider worst-case load over the entire lifetime of the installation, and write load can have a big impact.
There are other reasons to load balance, as well. Consider 24/7 uptime requirements. If I have only 1 server, that is a SPOF. Even if I have 2 servers, the risk of not meeting the 24/7 needs rather great. At Stanford, we have one master, isolated for nearly pure write purposes, 1 HA Standby master (waiting on 2.4), and 4 replicas that are geographically diverse. This also means I can upgrade the entire replica cluster with no outage. And of course, as Buchan notes, if there was a high volume of writes to hit the servers, this helps mitigate the impact.
And although OpenLDAP is blazingly fast, when you start throwing things in like SASL/GSSAPI binds, your performance can take a major hit. Although my hardware can sustain 15,000 authorizations/second with simple binds (30,000 searches/second), it drops to 100 searches/second with SASL/GSSAPI in play if the application connects, binds, searches, disconnects. If it uses persistent connections, then it is again blazingly fast, since the massively slow bind step is removed.
--Quanah
-- Quanah Gibson-Mount Principal Software Developer ITS/Shared Application Services Stanford University GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
On Thursday 01 February 2007 09:30, Aviles, Hugo wrote:
Hello,
So here's my situation, I need to setup two openldap instances that will be setup behind a load balancers and be redundant for each other. But i don't want any referrels' to another "master" instance to make the writes. I want both instances to be able to write and do the update without a referral and also have the same data across both instances. I am having trouble figuring out how to setup openldap to do that but from what I have read so far, syncrepl seems to be what I should use to get close to what I want. But I still need some assistance.
[...]
It doesn't haven't to be Multi-master but I do what the two instances to have the same data and be able to update requests to modify/add/delete entries without a refferal..
Sounds like sync-repl (for replication) and chain overlay (see 'man slap-chain') will do what you want.
Regards, Buchan
On Thu, Feb 01, 2007 at 05:28:34PM +0200, Buchan Milne wrote:
On Thursday 01 February 2007 09:30, Aviles, Hugo wrote:
Hello,
So here's my situation, I need to setup two openldap instances that will be setup behind a load balancers and be redundant for each other. But i don't want any referrels' to another "master" instance to make the writes. I want both instances to be able to write and do the update without a referral and also have the same data across both instances. I am having trouble figuring out how to setup openldap to do that but from what I have read so far, syncrepl seems to be what I should use to get close to what I want. But I still need some assistance.
[...]
It doesn't haven't to be Multi-master but I do what the two instances to have the same data and be able to update requests to modify/add/delete entries without a refferal..
Sounds like sync-repl (for replication) and chain overlay (see 'man slap-chain') will do what you want.
But what happens when one of the servers is down, does slapo-chain store the values ?
Regards, Buchan
-- Buchan Milne ISP Systems Specialist - Monitoring/Authentication Team Leader B.Eng,RHCE(803004789010797),LPIC-2(LPI000074592)
Sounds like sync-repl (for replication) and chain overlay (see 'man slap-chain') will do what you want.
But what happens when one of the servers is down, does slapo-chain store the values ?
chain + sync cant handle any request if master is down.
I made some modifications before so that a client will change into master and other clients will connect to it in case master is down, but it is only academic.
the architecture "HA + LB cluster" has been rised long before, but it seems openldap team doesnt implement it gracefully yet :-(
Zhang William wrote:
chain + sync cant handle any request if master is down.
I made some modifications before so that a client will change into master and other clients will connect to it in case master is down, but it is only academic.
the architecture "HA + LB cluster" has been rised long before, but it seems openldap team doesnt implement it gracefully yet :-(
As noted in many previous messages, mirrormode addresses this situation automatically.
On Thu, Feb 01, 2007 at 10:57:46PM -0800, Howard Chu wrote:
Zhang William wrote:
chain + sync cant handle any request if master is down.
I made some modifications before so that a client will change into master and other clients will connect to it in case master is down, but it is only academic.
the architecture "HA + LB cluster" has been rised long before, but it seems openldap team doesnt implement it gracefully yet :-(
As noted in many previous messages, mirrormode addresses this situation automatically.
But with one exception you can't write to both database at the same time
-- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc Chief Architect, OpenLDAP http://www.openldap.org/project/
Alex Samad wrote:
On Thu, Feb 01, 2007 at 10:57:46PM -0800, Howard Chu wrote:
Zhang William wrote:
chain + sync cant handle any request if master is down.
I made some modifications before so that a client will change into master and other clients will connect to it in case master is down, but it is only academic.
the architecture "HA + LB cluster" has been rised long before, but it seems openldap team doesnt implement it gracefully yet :-(
As noted in many previous messages, mirrormode addresses this situation automatically.
But with one exception you can't write to both database at the same time
Right, nor do you need such a capability for a high-availability solution.
On Fri, Feb 02, 2007 at 02:52:21AM -0800, Howard Chu wrote:
Alex Samad wrote:
On Thu, Feb 01, 2007 at 10:57:46PM -0800, Howard Chu wrote:
Zhang William wrote:
chain + sync cant handle any request if master is down.
I made some modifications before so that a client will change into master and other clients will connect to it in case master is down, but it is only academic.
the architecture "HA + LB cluster" has been rised long before, but it seems openldap team doesnt implement it gracefully yet :-(
As noted in many previous messages, mirrormode addresses this situation automatically.
But with one exception you can't write to both database at the same time
Right, nor do you need such a capability for a high-availability solution.
Sorry I am lost, if I set up 2 nodes (as a side question can you have more than 2 nodes in mirrormode ?) in mirror mode. server A and server b.
and then setup dns round robin to balance the load, or should I use a load balancer in front of them. But how do I guarantee that I don't write to both at the same time. The only way I can think of do that is to point all the clients to server A and then fail back to server b.
When I think of a multi master solution, I think of something like oracle rac's solution or novell's NDS (form memory it is a ldap tree, configured in multi master mode, I might be wrong). with Oracle rac you can write to any of the nodes and it handles it properly (albeit the oracle rac setup uses shared storage)
-- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc Chief Architect, OpenLDAP http://www.openldap.org/project/
Alex Samad wrote:
On Fri, Feb 02, 2007 at 02:52:21AM -0800, Howard Chu wrote:
Alex Samad wrote:
On Thu, Feb 01, 2007 at 10:57:46PM -0800, Howard Chu wrote:
Zhang William wrote:
chain + sync cant handle any request if master is down.
I made some modifications before so that a client will change into master and other clients will connect to it in case master is down, but it is only academic.
the architecture "HA + LB cluster" has been rised long before, but it seems openldap team doesnt implement it gracefully yet :-(
As noted in many previous messages, mirrormode addresses this situation automatically.
But with one exception you can't write to both database at the same time
Right, nor do you need such a capability for a high-availability solution.
Sorry I am lost, if I set up 2 nodes (as a side question can you have more than 2 nodes in mirrormode ?) in mirror mode. server A and server b.
2 nodes only, and they mirror each other. There's a reason we called this mirror mode...
and then setup dns round robin to balance the load, or should I use a load balancer in front of them. But how do I guarantee that I don't write to both at the same time. The only way I can think of do that is to point all the clients to server A and then fail back to server b.
This has nothing to do with load balancing. With mirrormode you must make sure your frontend only passes writes to a single server.
When I think of a multi master solution, I think of something like oracle rac's solution or novell's NDS (form memory it is a ldap tree, configured in multi master mode, I might be wrong). with Oracle rac you can write to any of the nodes and it handles it properly (albeit the oracle rac setup uses shared storage)
Nobody said this is a multimaster solution, that's why it's not called "multimaster." This is a solution for high availability, to avoid single point of failure.
"Load balancing" in the context of writes is an oxymoron, and anyone who claims multimaster helps in load balancing is either deluded or lying and trying to sell you something.
Think about it, use your brain. In a working multimaster setup, when any server receives a write operation, it *must* propagate that write to all the other servers. That means the network traffic and write load is constant and spreads across all of the servers, no matter which server you write to. Spreading out the writes among the clients does *nothing* to distribute the load on the servers because they *all* have to handle *every* write anyway.
Let that sink in. Any vendor who sells you multimaster as a load-balancing solution is *lying to you*. It is a high-availability solution, it avoids a single point of failure, and that's a good thing, but it has NOTHING to do with load balancing. Multimaster comes with a cost in terms of data consistency, and that's a bad thing. Mirrormode is an approach for avoiding single-point-of-failure without losing data consistency guarantees.
The OpenLDAP Project doesn't try to sell you anything. When we see something is stupid or wrong, we say "that's stupid and wrong." When we identify a problem, we solve the problem. There's no sugar-coating and there's no BS. When you're thinking for yourself, you don't need sugar-coating, and the BS just gets in the way.
So here's my situation, I need to setup two openldap instances that will be setup behind a load balancers and be redundant for each other.
What goal are you trying to accomplish? i.e. do you need two instances to handle the load, or for reliability? These are two separate problems and really should be addressed as such.
But i don't want any referrels' to another "master" instance to make the writes. I want both instances to be able to write and do the update without a referral and also have the same data across both instances. I am having trouble figuring out how to setup openldap to do that but from what I have read so far, syncrepl seems to be what I should use to get close to what I want. But I still need some assistance.
Mirrormode, which is available in CVS or Symas's CDS build of OpenLDAP lets you do something resembling what you seek.
In particular, you can have two servers that are masters and accepting changes from each other. Unfortunately you absolutely must ensure that only one of them receives writes at a time, generally via a load balancer. All replicas and clients would then point at the load balancer.
If a new entry 'A' gets added to the provider master (ligit entry) and then a new entry B gets added to the consumer database (ligit entry), when they replicate, will both provider and master have the A and B entries? Or will the consumer database be screwed and whatever the provider has is it?
In mirrormode, both masters are peers with neither one in charge. If you violate the design assumptions and write to both, they can trade objects, have disjoint changes, and all sorts of other interesting and generally undesirable things.
As Dr. Spangler said, "Don't cross the streams."
It doesn't haven't to be Multi-master but I do what the two instances to have the same data and be able to update requests to modify/add/delete entries without a refferal..
As long as you only write to one at a time, sure. Otherwise you need the full elaborate multi-master conflict resolution system, which has not yet been written. And no, the other directory servers don't generally have a strict-reliable multi-master either.
Matthew Backes Symas Corporation mbackes@symas.com lucca@accela.net
On 2/1/07, Matthew Backes mbackes@symas.com wrote: [snip]
Mirrormode, which is available in CVS or Symas's CDS build of OpenLDAP lets you do something resembling what you seek.
[snip]
Is this feature stable enough for production use? Which patch(es) should I apply to the current OL 2.3.x if I want to use it?
Thanks, Sam
-----Original Message----- From: openldap-software-bounces+mhardin=symas.com@OpenLDAP.org [mailto:openldap-software-bounces+mhardin=symas.com@OpenLDAP.org] On Behalf Of Sam Tran Sent: Friday, February 02, 2007 1:20 PM To: Matthew Backes Cc: openldap-software@openldap.org Subject: Re: Multi Master Enviornment for Openldap 2.3
On 2/1/07, Matthew Backes mbackes@symas.com wrote: [snip]
Mirrormode, which is available in CVS or Symas's CDS build of OpenLDAP lets you do something resembling what you seek.
[snip]
Is this feature stable enough for production use?
The Mirrormode implementation in CDS has been in production use at several of our customer sites since we introduced it in February of last year. There have been no reported problems since April.
Which patch(es) should I apply to the current OL 2.3.x if I want to use it?
I don't have the patch set at my fingertips, and I doubt anyone else does either. You can always identify the patch set from the OpenLDAP CVS tree.
I think your best bet would be to use one of our free distributions (CDS Silver 3.3.1 or later). Those releases contain the MirrorMode code with subsequent fixes and you could get a look at the feature with a minimum investment of time.
Alternatively, you could pull OpenLDAP HEAD from CVS and build that. Of course, a build of HEAD may have other problems, but at least you might be able to get a good look at MirrorMode.
Matthew Hardin Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Thanks, Sam
openldap-software@openldap.org
-
Alex Samad -
Aviles, Hugo -
Buchan Milne -
Howard Chu -
John Madden -
Matthew Backes -
Matthew Hardin -
Quanah Gibson-Mount -
Sam Tran -
Zhang William