Multimaster assumptions
by Ondrej Kuznik
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
we are preparing to set up a more or less extensive test suite for
deltasync multimaster replication. Since the resolution of conflicts can
be arbitrary (as long as it is consistent), I would like to have a way
to compare the outcome to a single authoritative DIT, the "observer" below.
Preconditions:
- - all servers start with an identical database
- - each server has unique serverID assigned
A thought experiment:
There are N servers acting as delta-syncrepl providers, arbitrary
changes flow to some/all of the servers. The "replication topology" is
arbitrary and volatile, but from some moment on it is guaranteed that it
will stay strongly connected.
Let us add another server and call it an "observer". An observer acts as
a delta-syncrepl consumer to a (possibly changing) set of providers and
it never acts as a provider to any of the servers.
Given the above and barring any software bugs, I guess it is intended
that if all servers are running recent OpenLDAP the following should
hold for the experiment:
- - given that the flow of changes stops eventually, the (actual)
replication traffic will eventually stop too with all N servers holding
the exact same DIT
- - given the above, the (actual) replication traffic to the observer too
will stop eventually with the observer holding the exact same DIT as the
servers.
Are my expectations reasonable or should the experiment be restricted to
achieve that outcome of having a reliable observer node? How about
generalizing the experiment by allowing that some of the replication
connections be plain syncrepl?
Even though this experiment might be exaggerated, I still feel like
stating what is a tautology according to the design from my point of
view, but I would like to hear whether it is supposed to work even under
these conditions.
Cheers,
- --
Ondrej Kuznik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk9p0z0ACgkQ9GWxeeH+cXuUYgCgiy0RS7wC4H6Vw67Mjv673Gm1
V20AnjL/qz8e0vw7HcJdUwA9rJaBM05a
=ygxQ
-----END PGP SIGNATURE-----
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.
11 years, 2 months
Re: Scaling LDAP
by Gaurav Gugnani
Hello Gibson,
Thks for replying. However, my concern is:
In our production system we have 8GB of ram with 100GB hard disk. So till
what limit it the Read/Write operation to LDAP goes smoothly.
Lets say - 1 row of data is 10K bytes.
Please don't mind - but we are planning for the afterwards approach. (What
to do - when LDAP goes slow with this configuration)
For Example - In mysql we could do first level partitioning the tables and
afterwards at last resort we could do its sharding.
So, my query is - Can we do anything other than upgrading H/W or OS if the
I/O operation to LDAP gets slow?
Thanks and Regards,
Gaurav Gugnani
On Thu, Mar 15, 2012 at 12:20 AM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote:
> --On Wednesday, March 14, 2012 11:08 AM +0530 Gaurav Gugnani <
> gugnanigaurav(a)gmail.com> wrote:
>
> Hi All,
>>
>> First of all thks for helping me out with the issues on openLDAP.
>> Well today, my query is pretty generic and lot many people working on
>> LDAP would face such an issues.
>>
>>
>>
>> We are using openldap 2.4.26 with BDB as backend. We've installed the
>> setup on linux machine of 64 bit with 4GB ram. Now, currently we have
>> some 10K records in it and its working perfectly fine. Our systems is
>> running under replication - syncrepl.
>>
>>
>> However, in near future we can foresee some million of records to turn
>> up. So, Can any one please advise - What are the different Scaling
>> options available with LDAP?
>>
>
> I'm not sure what you mean by scaling options. OpenLDAP scales, and that
> has been shown numerous times. How well/far it scales depends entirely on
> your hardware and operating system and the size of the DB in relation to
> those things.
>
> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Sr. Member of Technical Staff
> Zimbra, Inc
> A Division of VMware, Inc.
> --------------------
> Zimbra :: the leader in open source messaging and collaboration
>
11 years, 2 months
openLDAP as a proxy for AD
by Chris O'Kelly
Hi guys,
I have been trying for a few weeks to integrate 2 directories. One is an AD directory which holds internal employees and is used for windows domain logins/policy etc. The other is an openLDAP directory I set up myself last week which contains external users (not employed by the company but need access to various web applications we serve). As some of our web applications do not support chaining multiple authentication sources we are trying to get all the AD content available in OpenLDAP, so we can use that for web app authentication and AD for Windows Domain stuff.
Before this I have basically no experience with LDAP (if I don't count adding and removing the odd user or group from AD). As a result I went into this project rather optimistically, having read great stuff about syncrepl and how easy it is to set up replication to openLDAP. By now I am guessing most people have guessed the realisation I came to when I tried to go down that path, being that AD doesn't play nicely with others and syncrepl is not going to help here.
Since then I have been doing a good deal of RTFMing and JFGIing and I believe what I need to do is to use the back-ldap database type to set up a proxy. Unfortunately that's where I hit a dead end. Every tutorial I can find seems to relate to slapd.conf, whereas I am setup with RTC in the slapd.d directory. In attempting to get around this I have been doing things like adding olc to the beginning of pretty much everything I would put into slapd.conf, saving as a .ldif and ldapadding it. After a while of trial and error runs I discovered what modules needed to be loaded and what not in order to complete the ldapadd without error, but still saw no change, searching found the results in OpenLDAP but never the ones in AD.
In desperation, today I apt-get purged OpenLDAP and its dependencies and reinstalled it. I deleted the basic configuration loaded into slapd.d and set up a slapd.conf file as best I could with the setup I needed plus the back-ldap stuff I had found in tutorials. I successfully slaptested it but I am hitting the exact same problem, and the changes I have made since then seem to have only succeeded in breaking OpenLDAP to the point where I was no longer able to connect to it with Apache Directory Studio.
So now I have purged the lot again, and I suppose I am looking for some help as to where to go from here. OpenLDAP is running on Ubuntu and the ldif I have been trying to add for the proxy is-
olcDatabase: ldap
olcSuffix: dc=companyname,dc=local
olcSubordinate: yes
olcRebind-as-user: yes
olcUri: "ldap://companyname.local/"
olcChase-referrals: yes
Thanks in advance to anyone who can help!
11 years, 2 months
cannot get base DN / suffix from ldap browsers
by Jehan Procaccia
Hello,
I cannot figure out why on one of my replicas, I cannot browse the DIT .
Apache Directory Studio for example, only show the "root DSE(2)", but
the base DN (namingContext or directory suffix, whatever you call it
...) isn't visible !?
on my others replicas and the master, everything is fine, I do browse
the DIT, the browser shows "root DSE(3)" with the suffix visible.
I might be missing something obvious, but cannot figure out what.
I checked ACL:
access to dn.base="" by * read
access to dn.base="cn=Subschema" by * read
access to dn.subtree="dc=int-evry,dc=fr"
by dn="cn=admin,dc=int-evry,dc=fr" write
by users read
but still, the suffix dc=int-evry,dc=f doesn't shows up on that
particular replica !?
I run openldap-servers-2.4.23-20.el6.i686 with cn=config created from a
slapd.conf transformed with slaptest -f .
Any help greatly appreciated .
11 years, 2 months
replication issues
by Marvin Mundry
Hello everybody,
i am experiencing 2 issues while using syncrepl. i am running slapd
2.4.20 shipping with SLES 11.2.
1. replication of an accesslog database.
=============================
the accesslog overlay does not assign an entryUUID to its logdb
database. therefore replication of this database is not possible.
dn: cn=accesslog
objectClass: auditContainer
cn: accesslog
entryCSN: 20120317121350.967741Z#000000#000#000000
structuralObjectClass: auditContainer
contextCSN: 20120320154322.642649Z#000000#000#000000
To be able to replicate cn=accesslog anyways, i initialize the
accesslog database via slapadd from an ldif file that contains a made
up entryUUID for dn:cn=accesslog, afterwards i fire up the slapd.
Replication now seems to work fine. My question is, does this method
have any unwanted side effects?
2. replication of cn=config
===================
when replicating cn=config all objects below cn=schema,cn=config get
replicated when the consumer is initially started.
for
dn:olcDatabase={-1}frontend,cn=config and
dn:cn=config (olcGlobal)
initially no synchronization happens. Both objects get replicated only
after they have been modified on the on the provider ie. when starting
a new consumer i have to carry out meaningless modifications to both
objects on the provider to initiate the replication. is this behavior
to be expected?
my syncrepl config is:
syncrepl rid=15
provider=ldap://192.168.2.100
type=refreshAndPersist
retry="1 1 1 1"
searchbase="cn=config"
filter="(|(objectClass=olcGlobal)(objectClass=olcSchemaConfig)(objectClass=olcFrontendConfig))"
attrs="*,+"
bindmethod=simple
binddn="cn=admin,cn=config"
credentials="secret"
Best regards,
Marvin
11 years, 2 months
mdb hmmm....
by Francis Swasey
I thought I'd give mdb a try to see if it will solve a performance problem I'm approaching with
my current ldap servers. But, I have obviously missed some key step.
I have been running 2.4.28 on RHEL5 64-bit, using an hdb database.
The steps I followed were:
- slapcat the existing database
- stop slapd
- build 2.4.30 and install it
- update my slapd.conf to use mdb, remove the cachesize, cachefree, and idlcachesize options
(which are not valid with mdb), and add the maxsize option set to 100GB (107374182400).
Then, I ran slapadd with -q and it blew up on me:
slapadd -q -b dc=uvm,dc=edu -l ../cheese.ldif
4f68cc21 => mdb_idl_insert_keys: c_get failed: Invalid argument (22)
4f68cc21 => mdb_tool_entry_put: index_entry_add failed: err=80
4f68cc21 => mdb_tool_entry_put: txn_aborted! Internal error (80)
slapadd: could not add entry dn="cn=XXobfuscatedXX,ou=People,dc=uvm,dc=edu" (line=771):
txn_aborted! Internal error (80)
_ 0.01% eta 02m06s elapsed none spd 3.4 M/s
Closing DB...
Now, just to convince myself that I wasn't bonkers, I updated the slapd.conf file to go back to
hdb and the slapadd command runs just fine.
I've read the slapd-mdb man page, scoured the openldap-technical mailing list, even the
faq-o-matic and the admin guide. I'm not finding any clues to what is causing slapadd to fail
here.
--
Frank Swasey | http://www.uvm.edu/~fcs
Sr Systems Administrator | Always remember: You are UNIQUE,
University of Vermont | just like everyone else.
"I am not young enough to know everything." - Oscar Wilde (1854-1900)
11 years, 2 months
error entry store failed
by Wen, Nancy
Hi folks,
Can anyone help me on this problem?
Note we were trying to know what happen on a previous error - Sizelimit
exceeded. However, after we turned the log level up, and we saw this:
=> id2entry_add( 322, "uid=jason,ou=people,ou=0019,l=taiwan,dc=039,dc=com"
)
=> ldbm_cache_open( "id2entry.dbb", 73, 600 )
<= ldbm_cache_open (cache 1)
<= id2entry_add 30988
id2entry_add failed
=> dn2id_delete( "uid=jason,ou=people,ou=0019,l=taiwan,dc=039,dc=com", 322
)
=> ldbm_cache_open( "dn2id.dbb", 73, 600 )
<= ldbm_cache_open (cache 0)
<= dn2id_delete 0
send_ldap_result: conn=0 op=40 p=3
send_ldap_result: err=80 matched="" text="entry store failed"
send_ldap_response: msgid=41 tag=105 err=80
conn=0 op=40 RESULT tag=105 err=80 text=entry store failed
====> cache_return_entry_w( 206 ): returned (0)
====> cache_return_entry_w( 322 ): deleted (0)
@(#) $OpenLDAP: slapd 2.2.13 (Aug 13 2006 01:27:00) $ buildcentos@build-i386
:/home/buildcentos/rpmbuild/BUILD/openldap-2.2.13/openldap-2.2.13/build-servers/servers/slapd
daemon: bind(6) failed errno=98 (Address already in use)
daemon: bind(6) failed errno=98 (Address already in use)
slapd stopped.
Thanks million,
Nancy Wen
11 years, 2 months
translucent plugin
by Alex Samad - Yieldbroker
Hi
I was wondering if the translucent plugin will allow me to add whole objects
So if I had
Openldap translucent -> MS AD
Suffix dc=test,dc-com
Could I add who objects and not just attributes into the tree ?
Alex
11 years, 2 months
mirrormode and N-way multimaster replication
by mallapadi niranjan
Hi all
I am trying to understand the differences between mirrormode and N-way
multimaster replication.
I am using openldap-2.4.23-20.el6 , From the admin guide from
www.openldap.org , I would like to clarify the below point
In Mirromode . if i have two openldap servers(m1-m2), I can do add/del/mod
operations only on m1, but cannot do add/del/mod operations simultaneously
on both m1 and m2 , I can do add/del/mod operations on m2 only when m1 is
down.
Where as In N-Way multimaster replication, (if m1 and m2 are two openldap
servers), i can do add/mod/del operations on both m1 and m2 simultaneously
,
Can some body clarify if the above understanding is correct.
Regards
Niranjan
11 years, 2 months
src rpms for 2.4.30 / RHEL 6
by Aaron Bennett
Hello,
For a number of reasons, I wanted to package OpenLDAP 2.4.30 for RHEL 6. If anyone's interested, you can get my package here...
http://wordpress.clarku.edu/abennett/2012/03/16/openldap-src-rpms-for-rhe...
It depends on DB5 so you'll find a src rpm for that as well. All the binaries go into /usr/local; configuration is in /etc/openldap and /etc/sysconfig; bdb files go in /var/lib/ldap. The init script is the regular RHEL 5 patched to run the binary from a different location. It conflicts with openldap-servers and openldap-clients but NOT with the base system openldap.
Let me know if it works for you or if you find issues with it.
Best,
Aaron
---
Aaron Bennett
Manager, Systems Administration
Clark University ITS
11 years, 2 months
