Dear gentleman/madam,
as you may be aware i am having hard times trying to get qmail+ldap working on my OpenBSD 5.2 amd64. While after installation OLDAP works 100% (I am using BDB as backend). When i start qmail, every query sent by qmail to the OLDAP server increases memory up to a point OLDAP needs to be restarted.
I emailed misc@openbsd.org. I got responses. Here i pass them to the OLDAP developer: After some advice from Mr. Quanah, i gave up BDB and would like to give MDB a try. Here is what i got from the misc@openbsd.org
I believe the OLDAP developer should know:
---------- Forwarded message ---------- From: Stuart Henderson stu@spacehopper.org Date: Mon, Jan 14, 2013 at 8:47 PM Subject: Re: adding support to mdb To: Friedrich Locke friedrich.locke@gmail.com Cc: ports@openbsd.org
On 2013/01/14 19:18, Friedrich Locke wrote:
Hi folks,
this is my first message to this list. I am writing because i am in need
to
use openldap from ports. Initially i tried openldap with support to BDB, but it is buggy. I was suggest to try mdb, but openldap's Makefile have the --disable-mdb instruction.
Since i do need to get a directory service and have no knownledge on openldap internals, i come to you in order to suggest how could i help in order to get mdb supported by OpenBSD Ports OpenLDAP.
Thank you a lot for your time and cooperation.
Best regards,
Fried.
So this is a memory-mapped database; there have been problems with these in the past (e.g. in Cyrus imapd and Dovecot) but I am not sure of all the details. The port diff is easy enough but I would really like some advice from someone who is knowledgeable about the mmap issues as to whether or not it's safe to use this on OpenBSD.
A bit more info from slides at http://www.openldap.org/pub/hyc/mdm-slides.pdf :-
-- -- -- -- -- * Uses a read-only memory map
* Protects the database structure from corruption due to stray writes in memory
* Any attempts to write to the map will cause a SEGV, allowing immediate identification of software bugs
* There's no point in making the pages writable anyway, since only existing pages may be written. Growing the database requires file ops (write, ftruncate) so for uniformity, file ops are also used for updates. -- -- -- -- --
I *think* the problems were only with mmap writes, if this is correct then they seem to be sidestepping the problems by doing this.
BTW I'm using OpenLDAP 2.4 with BDB here and haven't noticed issues however it is not under extreme load..
--On Tuesday, January 15, 2013 3:21 PM -0200 Friedrich Locke friedrich.locke@gmail.com wrote:
Dear gentleman/madam,
as you may be aware i am having hard times trying to get qmail+ldap working on my OpenBSD 5.2 amd64. While after installation OLDAP works 100% (I am using BDB as backend). When i start qmail, every query sent by qmail to the OLDAP server increases memory up to a point OLDAP needs to be restarted.
I emailed misc@openbsd.org. I got responses. Here i pass them to the OLDAP developer: After some advice from Mr. Quanah, i gave up BDB and would like to give MDB a try. Here is what i got from the misc@openbsd.org
Hi Friedrich,
As per Howard Chu (Author of MDB, Primary OpenLDAP Developer):
------------------------------------------------------------- Full details are in the paper. http://www.openldap.org/pub/hyc/mdm-paper.pdf
MDB assumes a unified buffer cache. See section 3.1, references 17, 18, and 19.
Note that this requirement can be relaxed in the current version of the library. If you create the environment with the MDB_WRITEMAP option then all reads and writes are performed using mmap, so the file buffer cache is irrelevant. Of course then you lose the protection that the read-only map offers. -------------------------------------------------------------
I will note that I personally always set MDB_WRITEMAP in my deployments: zimbra@zre-ldap002:~$ ldapsearch -LLL -x -H ldapi:/// -D cn=config -W -b "olcDatabase={2}mdb,cn=config" -s base olcDbEnvFlags
dn: olcDatabase={2}mdb,cn=config olcDbEnvFlags: writemap olcDbEnvFlags: nometasync
I do this as the protections you lose are still better than any guarantee one ever had with using BDB (possible loss of the most recent transaction).
--Quanah
--
Quanah Gibson-Mount Sr. Member of Technical Staff Zimbra, Inc A Division of VMware, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org