I try to modify a hdb periodically (often). After a while the underlying database seems to be swamped down.
If I look in the directory log files are piling up. I run db_archive periodically to clean up the log files and I also run db_deadlock in parallel.
I do not run db_checkpoint as this as far as I understand is done by the backend.
Strangely the problem seems to start after about 2 h if I start from a clean database. When I try to lower the frequency of the updates it still seems to happen after 2 h.
I use openLDAP v2.3.24 with Sleepycat Software: Berkeley DB 4.4.20
Johan Jönemo
--On Monday, November 13, 2006 12:32 PM +0100 johan.jonemo@hep.lu.se wrote:
I try to modify a hdb periodically (often). After a while the underlying database seems to be swamped down.
If I look in the directory log files are piling up. I run db_archive periodically to clean up the log files and I also run db_deadlock in parallel.
I do not run db_checkpoint as this as far as I understand is done by the backend.
Strangely the problem seems to start after about 2 h if I start from a clean database. When I try to lower the frequency of the updates it still seems to happen after 2 h.
I use openLDAP v2.3.24 with Sleepycat Software: Berkeley DB 4.4.20
If you set the right BDB options in DB_CONFIG, I can't see any reason to run any of the db_* commands.
In any case, I've used hdb since OpenLDAP 2.3.11, and I've not had any problems with it swamping down from modifications. I'd check your DB_CONFIG settings.
--Quanah
-- Quanah Gibson-Mount Principal Software Developer ITS/Shared Application Services Stanford University GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
--On Monday, November 13, 2006 12:32 PM +0100 johan.jonemo@hep.lu.se wrote:
I try to modify a hdb periodically (often). After a while the underlying database seems to be swamped down.
If I look in the directory log files are piling up. I run db_archive periodically to clean up the log files and I also run db_deadlock in parallel.
I do not run db_checkpoint as this as far as I understand is done by the backend.
Strangely the problem seems to start after about 2 h if I start from a clean database. When I try to lower the frequency of the updates it still seems to happen after 2 h.
I use openLDAP v2.3.24 with Sleepycat Software: Berkeley DB 4.4.20
If you set the right BDB options in DB_CONFIG, I can't see any reason to run any of the db_* commands.
Then there must be some serious bug in back-hdb because I haven't seen it delete a single log-file. Wether the db is responsive or not no (berkley-db-)log files seems to be erased.
My starting point was not to run any "db_"-commands. But as the directory seems to be acting up I have to try every possibility.
The db-commands seem to delay the breakdown slightly, but I can't tell because I'm also testing other alternatives. The latest breakdown took 3 hours. The Breakdown times might of course be random. I haven't had time to statistically establish that the breakdown times tend to any one value.
here is my DB_CONFIG: set_cachesize 0 16777216 8 set_lg_regionmax 262144 set_lg_bsize 2097152 set_lg_max 16777216
It should be noted that the directory is at no time expected to take on very large sizes. Otherwise some memory sizes might have been set slightly higher.
Johan "the one who'll lose his job if he doesn't get this to work" Jönemo
johan.jonemo@hep.lu.se wrote:
--On Monday, November 13, 2006 12:32 PM +0100 johan.jonemo@hep.lu.se wrote:
I try to modify a hdb periodically (often). After a while the underlying database seems to be swamped down.
If I look in the directory log files are piling up. I run db_archive periodically to clean up the log files and I also run db_deadlock in parallel.
I do not run db_checkpoint as this as far as I understand is done by the backend.
Yes, checkpointing is handled by the backend if you have that configured.
Strangely the problem seems to start after about 2 h if I start from a clean database. When I try to lower the frequency of the updates it still seems to happen after 2 h.
I use openLDAP v2.3.24 with Sleepycat Software: Berkeley DB 4.4.20
If you set the right BDB options in DB_CONFIG, I can't see any reason to run any of the db_* commands.
Then there must be some serious bug in back-hdb because I haven't seen it delete a single log-file. Wether the db is responsive or not no (berkley-db-)log files seems to be erased.
back-bdb/back-hdb never delete log files. Log files are owned by the BDB library and we know nothing about them.
My starting point was not to run any "db_"-commands. But as the directory seems to be acting up I have to try every possibility.
The db-commands seem to delay the breakdown slightly, but I can't tell because I'm also testing other alternatives. The latest breakdown took 3 hours. The Breakdown times might of course be random. I haven't had time to statistically establish that the breakdown times tend to any one value.
here is my DB_CONFIG: set_cachesize 0 16777216 8 set_lg_regionmax 262144 set_lg_bsize 2097152 set_lg_max 16777216
If you want the BDB library to remove the log files you need to use the DB_LOG_AUTOREMOVE option, as described in the BerkeleyDB documentation.
It should be noted that the directory is at no time expected to take on very large sizes. Otherwise some memory sizes might have been set slightly higher.
Johan "the one who'll lose his job if he doesn't get this to work" Jönemo
...
openldap-software@openldap.org
-
Howard Chu -
johan.jonemo＠hep.lu.se -
Quanah Gibson-Mount