Hello list,
I want to migrate some OpenLDAP servers from 3.5" disks to CF-disks. The data in the OpenLDAP is only updated once a month or so. It is just an "99%-read-only" LDAP implementation.
However, with a standard Debian install, some files in the /var/lib/ldap directory are updated upon each query:
# ls -altr -rw-r--r-- 1 openldap openldap 96 2008-11-19 11:45 DB_CONFIG drwxr-xr-x 28 root root 4096 2008-12-03 15:03 .. -rw------- 1 openldap openldap 8192 2013-04-08 10:50 cn.bdb -rw------- 1 openldap openldap 24576 2013-09-29 13:49 objectClass.bdb -rw------- 1 openldap openldap 180224 2013-09-29 13:49 id2entry.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryUUID.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryCSN.bdb -rw------- 1 openldap openldap 36864 2013-09-29 13:49 dn2id.bdb -rw------- 1 openldap openldap 1168654 2013-10-17 09:40 log.0000000001 -rw------- 1 openldap openldap 24576 2013-11-07 05:45 __db.005 -rw------- 1 openldap openldap 98304 2013-11-07 05:45 __db.003 -rw-r--r-- 1 openldap openldap 4096 2013-11-07 05:45 alock drwx------ 2 openldap openldap 4096 2013-11-07 05:45 accesslog drwx------ 3 openldap openldap 4096 2013-11-07 05:45 . -rw------- 1 openldap openldap 565248 2013-11-07 11:30 __db.004 -rw------- 1 openldap openldap 2629632 2013-11-07 11:30 __db.002 -rw------- 1 openldap openldap 8192 2013-11-07 11:30 __db.001
Apparently the cluster is doing some synchronizing at 05:45 in the morning, but that's once a day. My concern is the files called
__db.001 __db.002 __db.004
Is there a simple way to prevent OpenLDAP from updating these files at each query?
R.
You should be specifying shm-key to benefit from shared mem vs memory mapped files
++Cyrille
Le 7 nov. 2013 à 12:04, "richard lucassen" mailinglists@lucassen.org a écrit :
Hello list,
I want to migrate some OpenLDAP servers from 3.5" disks to CF-disks. The data in the OpenLDAP is only updated once a month or so. It is just an "99%-read-only" LDAP implementation.
However, with a standard Debian install, some files in the /var/lib/ldap directory are updated upon each query:
# ls -altr -rw-r--r-- 1 openldap openldap 96 2008-11-19 11:45 DB_CONFIG drwxr-xr-x 28 root root 4096 2008-12-03 15:03 .. -rw------- 1 openldap openldap 8192 2013-04-08 10:50 cn.bdb -rw------- 1 openldap openldap 24576 2013-09-29 13:49 objectClass.bdb -rw------- 1 openldap openldap 180224 2013-09-29 13:49 id2entry.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryUUID.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryCSN.bdb -rw------- 1 openldap openldap 36864 2013-09-29 13:49 dn2id.bdb -rw------- 1 openldap openldap 1168654 2013-10-17 09:40 log.0000000001 -rw------- 1 openldap openldap 24576 2013-11-07 05:45 __db.005 -rw------- 1 openldap openldap 98304 2013-11-07 05:45 __db.003 -rw-r--r-- 1 openldap openldap 4096 2013-11-07 05:45 alock drwx------ 2 openldap openldap 4096 2013-11-07 05:45 accesslog drwx------ 3 openldap openldap 4096 2013-11-07 05:45 . -rw------- 1 openldap openldap 565248 2013-11-07 11:30 __db.004 -rw------- 1 openldap openldap 2629632 2013-11-07 11:30 __db.002 -rw------- 1 openldap openldap 8192 2013-11-07 11:30 __db.001
Apparently the cluster is doing some synchronizing at 05:45 in the morning, but that's once a day. My concern is the files called
__db.001 __db.002 __db.004
Is there a simple way to prevent OpenLDAP from updating these files at each query?
R.
-- ___________________________________________________________________ It is better to remain silent and be thought a fool, than to speak aloud and remove all doubt.
+------------------------------------------------------------------+ | Richard Lucassen, Utrecht | +------------------------------------------------------------------+
On Thu, 7 Nov 2013 11:08:28 +0000 "Maucci, Cyrille" cyrille.maucci@hp.com wrote:
You should be specifying shm-key to benefit from shared mem vs memory mapped files
Ok, thanks for your answer Cyrille. Unfortunately, I'm not an OpenLDAP guru, so I googled around a bit and found a lot of Zimbra answers. But I searched for shm in the admin guide and found this:
http://www.openldap.org/doc/admin24/guide.html
Example: olcDbShmKey: 42
As I'm not quite sure what I'm doing, I backed up my database (on a single OpenLDAP, not replicated) and I entered
olcDbShmKey: 42
into:
/etc/ldap/slapd.d/cn=config/olcDatabase={1}hdb.ldif
It seems to work:
drwxr-x--- 2 openldap openldap 4096 2013-11-07 13:32 . drwxr-xr-x 61 root root 4096 2013-10-26 10:35 .. -rw-r----- 1 openldap openldap 4096 2013-11-07 13:32 alock -rw------- 1 openldap openldap 16 2013-11-07 13:32 __db.001 -rw-r----- 1 openldap openldap 96 2013-11-07 13:03 DB_CONFIG -rw-r----- 1 openldap openldap 167936 2013-11-04 19:49 dn2id.bdb -rw-r----- 1 openldap openldap 917504 2013-11-07 12:15 id2entry.bdb -rw-r----- 1 openldap openldap 10485760 2013-11-07 13:32 log.0000000001 -rw-r----- 1 openldap openldap 98304 2013-10-19 21:42 objectClass.bdb
But as I'm not knowing exactly what I'm doing, just a simple question: Is this correct or not?
R.
On Thu, 7 Nov 2013 13:41:27 +0100 richard lucassen mailinglists@lucassen.org wrote:
Oops:
Note: Although the slapd-config(5) system stores its configuration as (text-based) LDIF files, you should never edit any of the LDIF files directly. Configuration changes should be performed via LDAP operations, e.g. ldapadd(1), ldapdelete(1), or ldapmodify(1).
"Maucci, Cyrille" cyrille.maucci@hp.com wrote:
You should be specifying shm-key to benefit from shared mem vs memory mapped files
I wonder whether switching to back-mdb would be a better solution.
Ciao, Michael.
Le 7 nov. 2013 à 12:04, "richard lucassen" mailinglists@lucassen.org a écrit : Hello list,
I want to migrate some OpenLDAP servers from 3.5" disks to CF-disks. The data in the OpenLDAP is only updated once a month or so. It is just an "99%-read-only" LDAP implementation.
However, with a standard Debian install, some files in the /var/lib/ldap directory are updated upon each query:
# ls -altr -rw-r--r-- 1 openldap openldap 96 2008-11-19 11:45 DB_CONFIG drwxr-xr-x 28 root root 4096 2008-12-03 15:03 .. -rw------- 1 openldap openldap 8192 2013-04-08 10:50 cn.bdb -rw------- 1 openldap openldap 24576 2013-09-29 13:49 objectClass.bdb -rw------- 1 openldap openldap 180224 2013-09-29 13:49 id2entry.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryUUID.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryCSN.bdb -rw------- 1 openldap openldap 36864 2013-09-29 13:49 dn2id.bdb -rw------- 1 openldap openldap 1168654 2013-10-17 09:40 log.0000000001 -rw------- 1 openldap openldap 24576 2013-11-07 05:45 __db.005 -rw------- 1 openldap openldap 98304 2013-11-07 05:45 __db.003 -rw-r--r-- 1 openldap openldap 4096 2013-11-07 05:45 alock drwx------ 2 openldap openldap 4096 2013-11-07 05:45 accesslog drwx------ 3 openldap openldap 4096 2013-11-07 05:45 . -rw------- 1 openldap openldap 565248 2013-11-07 11:30 __db.004 -rw------- 1 openldap openldap 2629632 2013-11-07 11:30 __db.002 -rw------- 1 openldap openldap 8192 2013-11-07 11:30 __db.001
Apparently the cluster is doing some synchronizing at 05:45 in the morning, but that's once a day. My concern is the files called
__db.001 __db.002 __db.004
Is there a simple way to prevent OpenLDAP from updating these files at each query?
R.
Michael Ströder wrote:
"Maucci, Cyrille" cyrille.maucci@hp.com wrote:
You should be specifying shm-key to benefit from shared mem vs memory mapped files
I wonder whether switching to back-mdb would be a better solution.
If the machine is 64 bit, yes absolutely. If the machine is 32 bit, depends on how large the DB can grow, since there's a 2-3GB limit on the address space.
Also while the main DB file would be 99% read-only, the lock.mdb file would not. In an application like this it's a good idea to symlink the lock.mdb file to a RAM filesystem/tmpfs.
Ciao, Michael.
Le 7 nov. 2013 à 12:04, "richard lucassen" mailinglists@lucassen.org a écrit : Hello list,
I want to migrate some OpenLDAP servers from 3.5" disks to CF-disks. The data in the OpenLDAP is only updated once a month or so. It is just an "99%-read-only" LDAP implementation.
However, with a standard Debian install, some files in the /var/lib/ldap directory are updated upon each query:
# ls -altr -rw-r--r-- 1 openldap openldap 96 2008-11-19 11:45 DB_CONFIG drwxr-xr-x 28 root root 4096 2008-12-03 15:03 .. -rw------- 1 openldap openldap 8192 2013-04-08 10:50 cn.bdb -rw------- 1 openldap openldap 24576 2013-09-29 13:49 objectClass.bdb -rw------- 1 openldap openldap 180224 2013-09-29 13:49 id2entry.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryUUID.bdb -rw------- 1 openldap openldap 8192 2013-09-29 13:49 entryCSN.bdb -rw------- 1 openldap openldap 36864 2013-09-29 13:49 dn2id.bdb -rw------- 1 openldap openldap 1168654 2013-10-17 09:40 log.0000000001 -rw------- 1 openldap openldap 24576 2013-11-07 05:45 __db.005 -rw------- 1 openldap openldap 98304 2013-11-07 05:45 __db.003 -rw-r--r-- 1 openldap openldap 4096 2013-11-07 05:45 alock drwx------ 2 openldap openldap 4096 2013-11-07 05:45 accesslog drwx------ 3 openldap openldap 4096 2013-11-07 05:45 . -rw------- 1 openldap openldap 565248 2013-11-07 11:30 __db.004 -rw------- 1 openldap openldap 2629632 2013-11-07 11:30 __db.002 -rw------- 1 openldap openldap 8192 2013-11-07 11:30 __db.001
Apparently the cluster is doing some synchronizing at 05:45 in the morning, but that's once a day. My concern is the files called
__db.001 __db.002 __db.004
Is there a simple way to prevent OpenLDAP from updating these files at each query?
R.
On Thu, 07 Nov 2013 06:51:40 -0800 Howard Chu hyc@symas.com wrote:
I wonder whether switching to back-mdb would be a better solution.
If the machine is 64 bit, yes absolutely. If the machine is 32 bit, depends on how large the DB can grow, since there's a 2-3GB limit on the address space.
Also while the main DB file would be 99% read-only, the lock.mdb file would not. In an application like this it's a good idea to symlink the lock.mdb file to a RAM filesystem/tmpfs.
Let me put things more clear: it is a 32 bit Ubuntu-8.04 (yes I know, very old) with this version:
2.4.9-0ubuntu0.8.04.5
There is a master and two slaves. I prefer not to upgrade for some non-ldap reasons. All config is AFAIK in slapd.conf, there is no slapd.d directory. The database contains *tens* of entries, when I open the database with ldapvi I find 93 entries. IOW: this is a very very small database.
Let me put it this way:
The only files that are regularly altered are these files:
__db.001 __db.002 __db.004
What happens if I symlink these files to a RAMDISK? And what happens after a reboot or power outage? Do I get a corrupted database?
R.
Hi,
On Fri, 8 Nov 2013, richard lucassen wrote:
On Thu, 07 Nov 2013 06:51:40 -0800 Howard Chu hyc@symas.com wrote:
I wonder whether switching to back-mdb would be a better solution.
If the machine is 64 bit, yes absolutely. If the machine is 32 bit, depends on how large the DB can grow, since there's a 2-3GB limit on the address space.
Also while the main DB file would be 99% read-only, the lock.mdb file would not. In an application like this it's a good idea to symlink the lock.mdb file to a RAM filesystem/tmpfs.
Let me put things more clear: it is a 32 bit Ubuntu-8.04 (yes I know, very old) with this version:
2.4.9-0ubuntu0.8.04.5
There is a master and two slaves. I prefer not to upgrade for some non-ldap reasons. All config is AFAIK in slapd.conf, there is no slapd.d directory. The database contains *tens* of entries, when I open the database with ldapvi I find 93 entries. IOW: this is a very very small database.
Let me put it this way:
The only files that are regularly altered are these files:
__db.001 __db.002 __db.004
What happens if I symlink these files to a RAMDISK? And what happens after a reboot or power outage? Do I get a corrupted database?
as others have said before configuring a shm_id would prevent those files from being generated in the first place as berkeley db locks, cache and such would then be in sys v shared memory and not in file backed shared memory.
Greetings Christian
On Fri, 8 Nov 2013 10:14:14 +0100 (CET) Christian Kratzer ck-lists@cksoft.de wrote:
What happens if I symlink these files to a RAMDISK? And what happens after a reboot or power outage? Do I get a corrupted database?
as others have said before configuring a shm_id would prevent those files from being generated in the first place as berkeley db locks, cache and such would then be in sys v shared memory and not in file backed shared memory.
Yes, but this is an old version (2008), there is no cn=config directory. AFAIK there is no way to set the olcDbShmKey. Or am I missing something somewhere?
R.
Hi,
On Fri, 8 Nov 2013, richard lucassen wrote:
On Fri, 8 Nov 2013 10:14:14 +0100 (CET) Christian Kratzer ck-lists@cksoft.de wrote:
What happens if I symlink these files to a RAMDISK? And what happens after a reboot or power outage? Do I get a corrupted database?
as others have said before configuring a shm_id would prevent those files from being generated in the first place as berkeley db locks, cache and such would then be in sys v shared memory and not in file backed shared memory.
Yes, but this is an old version (2008), there is no cn=config directory. AFAIK there is no way to set the olcDbShmKey. Or am I missing something somewhere?
set following in slapd.conf
shm_key 100
Greetings Christian
On Fri, 8 Nov 2013 10:41:00 +0100 (CET) Christian Kratzer ck-lists@cksoft.de wrote:
set following in slapd.conf
shm_key 100
Ok, now I see, I looked at "man slapd.conf", but apparently it's mentioned in "man slapd-bdb". And I did not find this because I googled for "shm-key" instead of "shm_key". Minor detail, one should say.
"A computer does what you ask, not what you want" :(
Thanks a lot, Christian, this did the job!
R.
On Fri, 8 Nov 2013 10:41:00 +0100 (CET) Christian Kratzer ck-lists@cksoft.de wrote:
set following in slapd.conf
shm_key 100
Everything works well (thanks Christian and others!). I just want to know if these logs are just warnings or messages after a reboot (Note: shm_key is set and this is an old Ubuntu-8.04 version):
192.168.69.243: local4.debug: slapd[1782]: bdb(dc=xaq,dc=nl): shmat: id 0: unable to attach to shared system memory region: Invalid argument
192.168.69.243: local4.debug: slapd[1782]: hdb_db_open: database "dc=xaq,dc=nl": shared memory env open failed, assuming stale env.
I assume these logs are just messages. Is that assumption correct?
R.
I was talking about the slapd.conf shm-key param.
-----Original Message----- From: openldap-technical-bounces@OpenLDAP.org [mailto:openldap-technical-bounces@OpenLDAP.org] On Behalf Of richard lucassen Sent: Friday, November 08, 2013 10:32 AM To: openldap-technical@openldap.org Cc: Christian Kratzer Subject: Re: OpenLDAP on CF disk
On Fri, 8 Nov 2013 10:14:14 +0100 (CET) Christian Kratzer ck-lists@cksoft.de wrote:
What happens if I symlink these files to a RAMDISK? And what happens after a reboot or power outage? Do I get a corrupted database?
as others have said before configuring a shm_id would prevent those files from being generated in the first place as berkeley db locks, cache and such would then be in sys v shared memory and not in file backed shared memory.
Yes, but this is an old version (2008), there is no cn=config directory. AFAIK there is no way to set the olcDbShmKey. Or am I missing something somewhere?
R.
-- ___________________________________________________________________ It is better to remain silent and be thought a fool, than to speak aloud and remove all doubt.
+------------------------------------------------------------------+ | Richard Lucassen, Utrecht | +------------------------------------------------------------------+
--On Thursday, November 07, 2013 3:22 PM +0100 Michael Ströder michael@stroeder.com wrote:
"Maucci, Cyrille" cyrille.maucci@hp.com wrote:
You should be specifying shm-key to benefit from shared mem vs memory mapped files
I wonder whether switching to back-mdb would be a better solution.
It is unwise to suggest back-mdb to someone using the stock debian installation as they stated they are doing. They would need to run a fairly current version of OpenLDAP first.
--Quanah
--
Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
openldap-technical@openldap.org
-
Christian Kratzer -
Howard Chu -
Maucci, Cyrille -
Michael Ströder -
Quanah Gibson-Mount -
richard lucassen