-----BEGIN PGP SIGNED MESSAGE-----
we are preparing to set up a more or less extensive test suite for
deltasync multimaster replication. Since the resolution of conflicts can
be arbitrary (as long as it is consistent), I would like to have a way
to compare the outcome to a single authoritative DIT, the "observer" below.
- - all servers start with an identical database
- - each server has unique serverID assigned
A thought experiment:
There are N servers acting as delta-syncrepl providers, arbitrary
changes flow to some/all of the servers. The "replication topology" is
arbitrary and volatile, but from some moment on it is guaranteed that it
will stay strongly connected.
Let us add another server and call it an "observer". An observer acts as
a delta-syncrepl consumer to a (possibly changing) set of providers and
it never acts as a provider to any of the servers.
Given the above and barring any software bugs, I guess it is intended
that if all servers are running recent OpenLDAP the following should
hold for the experiment:
- - given that the flow of changes stops eventually, the (actual)
replication traffic will eventually stop too with all N servers holding
the exact same DIT
- - given the above, the (actual) replication traffic to the observer too
will stop eventually with the observer holding the exact same DIT as the
Are my expectations reasonable or should the experiment be restricted to
achieve that outcome of having a reliable observer node? How about
generalizing the experiment by allowing that some of the replication
connections be plain syncrepl?
Even though this experiment might be exaggerated, I still feel like
stating what is a tautology according to the design from my point of
view, but I would like to hear whether it is supposed to work even under
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
-----END PGP SIGNATURE-----
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you for understanding.
Thks for replying. However, my concern is:
In our production system we have 8GB of ram with 100GB hard disk. So till
what limit it the Read/Write operation to LDAP goes smoothly.
Lets say - 1 row of data is 10K bytes.
Please don't mind - but we are planning for the afterwards approach. (What
to do - when LDAP goes slow with this configuration)
For Example - In mysql we could do first level partitioning the tables and
afterwards at last resort we could do its sharding.
So, my query is - Can we do anything other than upgrading H/W or OS if the
I/O operation to LDAP gets slow?
Thanks and Regards,
On Thu, Mar 15, 2012 at 12:20 AM, Quanah Gibson-Mount <quanah(a)zimbra.com>wrote:
> --On Wednesday, March 14, 2012 11:08 AM +0530 Gaurav Gugnani <
> gugnanigaurav(a)gmail.com> wrote:
> Hi All,
>> First of all thks for helping me out with the issues on openLDAP.
>> Well today, my query is pretty generic and lot many people working on
>> LDAP would face such an issues.
>> We are using openldap 2.4.26 with BDB as backend. We've installed the
>> setup on linux machine of 64 bit with 4GB ram. Now, currently we have
>> some 10K records in it and its working perfectly fine. Our systems is
>> running under replication - syncrepl.
>> However, in near future we can foresee some million of records to turn
>> up. So, Can any one please advise - What are the different Scaling
>> options available with LDAP?
> I'm not sure what you mean by scaling options. OpenLDAP scales, and that
> has been shown numerous times. How well/far it scales depends entirely on
> your hardware and operating system and the size of the DB in relation to
> those things.
> Quanah Gibson-Mount
> Sr. Member of Technical Staff
> Zimbra, Inc
> A Division of VMware, Inc.
> Zimbra :: the leader in open source messaging and collaboration
I have been trying for a few weeks to integrate 2 directories. One is an AD directory which holds internal employees and is used for windows domain logins/policy etc. The other is an openLDAP directory I set up myself last week which contains external users (not employed by the company but need access to various web applications we serve). As some of our web applications do not support chaining multiple authentication sources we are trying to get all the AD content available in OpenLDAP, so we can use that for web app authentication and AD for Windows Domain stuff.
Before this I have basically no experience with LDAP (if I don't count adding and removing the odd user or group from AD). As a result I went into this project rather optimistically, having read great stuff about syncrepl and how easy it is to set up replication to openLDAP. By now I am guessing most people have guessed the realisation I came to when I tried to go down that path, being that AD doesn't play nicely with others and syncrepl is not going to help here.
Since then I have been doing a good deal of RTFMing and JFGIing and I believe what I need to do is to use the back-ldap database type to set up a proxy. Unfortunately that's where I hit a dead end. Every tutorial I can find seems to relate to slapd.conf, whereas I am setup with RTC in the slapd.d directory. In attempting to get around this I have been doing things like adding olc to the beginning of pretty much everything I would put into slapd.conf, saving as a .ldif and ldapadding it. After a while of trial and error runs I discovered what modules needed to be loaded and what not in order to complete the ldapadd without error, but still saw no change, searching found the results in OpenLDAP but never the ones in AD.
In desperation, today I apt-get purged OpenLDAP and its dependencies and reinstalled it. I deleted the basic configuration loaded into slapd.d and set up a slapd.conf file as best I could with the setup I needed plus the back-ldap stuff I had found in tutorials. I successfully slaptested it but I am hitting the exact same problem, and the changes I have made since then seem to have only succeeded in breaking OpenLDAP to the point where I was no longer able to connect to it with Apache Directory Studio.
So now I have purged the lot again, and I suppose I am looking for some help as to where to go from here. OpenLDAP is running on Ubuntu and the ldif I have been trying to add for the proxy is-
Thanks in advance to anyone who can help!
I cannot figure out why on one of my replicas, I cannot browse the DIT .
Apache Directory Studio for example, only show the "root DSE(2)", but
the base DN (namingContext or directory suffix, whatever you call it
...) isn't visible !?
on my others replicas and the master, everything is fine, I do browse
the DIT, the browser shows "root DSE(3)" with the suffix visible.
I might be missing something obvious, but cannot figure out what.
I checked ACL:
access to dn.base="" by * read
access to dn.base="cn=Subschema" by * read
access to dn.subtree="dc=int-evry,dc=fr"
by dn="cn=admin,dc=int-evry,dc=fr" write
by users read
but still, the suffix dc=int-evry,dc=f doesn't shows up on that
particular replica !?
I run openldap-servers-2.4.23-20.el6.i686 with cn=config created from a
slapd.conf transformed with slaptest -f .
Any help greatly appreciated .
i am experiencing 2 issues while using syncrepl. i am running slapd
2.4.20 shipping with SLES 11.2.
1. replication of an accesslog database.
the accesslog overlay does not assign an entryUUID to its logdb
database. therefore replication of this database is not possible.
To be able to replicate cn=accesslog anyways, i initialize the
accesslog database via slapadd from an ldif file that contains a made
up entryUUID for dn:cn=accesslog, afterwards i fire up the slapd.
Replication now seems to work fine. My question is, does this method
have any unwanted side effects?
2. replication of cn=config
when replicating cn=config all objects below cn=schema,cn=config get
replicated when the consumer is initially started.
initially no synchronization happens. Both objects get replicated only
after they have been modified on the on the provider ie. when starting
a new consumer i have to carry out meaningless modifications to both
objects on the provider to initiate the replication. is this behavior
to be expected?
my syncrepl config is:
retry="1 1 1 1"
I thought I'd give mdb a try to see if it will solve a performance problem I'm approaching with
my current ldap servers. But, I have obviously missed some key step.
I have been running 2.4.28 on RHEL5 64-bit, using an hdb database.
The steps I followed were:
- slapcat the existing database
- stop slapd
- build 2.4.30 and install it
- update my slapd.conf to use mdb, remove the cachesize, cachefree, and idlcachesize options
(which are not valid with mdb), and add the maxsize option set to 100GB (107374182400).
Then, I ran slapadd with -q and it blew up on me:
slapadd -q -b dc=uvm,dc=edu -l ../cheese.ldif
4f68cc21 => mdb_idl_insert_keys: c_get failed: Invalid argument (22)
4f68cc21 => mdb_tool_entry_put: index_entry_add failed: err=80
4f68cc21 => mdb_tool_entry_put: txn_aborted! Internal error (80)
slapadd: could not add entry dn="cn=XXobfuscatedXX,ou=People,dc=uvm,dc=edu" (line=771):
txn_aborted! Internal error (80)
_ 0.01% eta 02m06s elapsed none spd 3.4 M/s
Now, just to convince myself that I wasn't bonkers, I updated the slapd.conf file to go back to
hdb and the slapadd command runs just fine.
I've read the slapd-mdb man page, scoured the openldap-technical mailing list, even the
faq-o-matic and the admin guide. I'm not finding any clues to what is causing slapadd to fail
Frank Swasey | http://www.uvm.edu/~fcs
Sr Systems Administrator | Always remember: You are UNIQUE,
University of Vermont | just like everyone else.
"I am not young enough to know everything." - Oscar Wilde (1854-1900)
I was wondering if the translucent plugin will allow me to add whole objects
So if I had
Openldap translucent -> MS AD
Could I add who objects and not just attributes into the tree ?
I am trying to understand the differences between mirrormode and N-way
I am using openldap-2.4.23-20.el6 , From the admin guide from
www.openldap.org , I would like to clarify the below point
In Mirromode . if i have two openldap servers(m1-m2), I can do add/del/mod
operations only on m1, but cannot do add/del/mod operations simultaneously
on both m1 and m2 , I can do add/del/mod operations on m2 only when m1 is
Where as In N-Way multimaster replication, (if m1 and m2 are two openldap
servers), i can do add/mod/del operations on both m1 and m2 simultaneously
Can some body clarify if the above understanding is correct.
For a number of reasons, I wanted to package OpenLDAP 2.4.30 for RHEL 6. If anyone's interested, you can get my package here...
It depends on DB5 so you'll find a src rpm for that as well. All the binaries go into /usr/local; configuration is in /etc/openldap and /etc/sysconfig; bdb files go in /var/lib/ldap. The init script is the regular RHEL 5 patched to run the binary from a different location. It conflicts with openldap-servers and openldap-clients but NOT with the base system openldap.
Let me know if it works for you or if you find issues with it.
Manager, Systems Administration
Clark University ITS