Frank Bonnet wrote:

Hello

Yes maybe safier to use slapcat ...

I haven't any SSDs for now so I tested ramdisk did you perform tests with SSD ?

A good SSD can work, but it will cost quite a bit and RAM is still an order of magnitude faster. Also, most of the SSDs available now are still poor for random writes, which is the main performance bottleneck. (All of my PCs are on SSDs now; they're great for fast boot times but without a big RAM cache they're still dog slow.)

thanks

On 10/19/2010 11:00 AM, Benjamin Griese wrote:

...

Hi Frank,

by rsync to hdd, do you mean copying the db files or slapcat the output to a file? The first one seems to be a db killer if the db is in use while copying it. :) I think the only safe way is by doing a slapcat after every write to the db (accesslog?).

Anyway, sounds like a nice idea to me to get a big performance boost. Have you considered SSDs instead of ramdisks? Imo its the more safe and nearly as fast as ramdisk alternative. :)

Bye, Benjamin.

On Tue, Oct 19, 2010 at 10:40, Frank Bonnetf.bonnet@esiee.fr wrote:

...

Hello

I'm thinking to put db files in RAM instead of hard disk for performance reasons.

FreeBSD ( and Linux thought ) provide some utility to build RAMDISK ( md )

I did some tests in rebuilding db files with the slapcat/slapadd commands

with RAMDISK

ldap3# time slapadd -l test.ldif /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted unlimited privileges. #################### 100.00% eta none elapsed 20s spd 633.9 k/s

WITH HARDDISK

ldap3# time slapadd -l test.ldif /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted unlimited privileges. #################### 100.00% eta none elapsed 04m22s spd 48.4 k/s

Does anyone runs production servers with ramdisk ?

I know it is risky but running rsyncd between ramdisk and a hardisk depot would be safe huh ?

Thanks for any advices

Perhaps running syncrpl between ramdisk and hardisk ?

On 10/19/2010 11:00 AM, Benjamin Griese wrote:

Hi Frank,

by rsync to hdd, do you mean copying the db files or slapcat the output to a file? The first one seems to be a db killer if the db is in use while copying it. :) I think the only safe way is by doing a slapcat after every write to the db (accesslog?).

Anyway, sounds like a nice idea to me to get a big performance boost. Have you considered SSDs instead of ramdisks? Imo its the more safe and nearly as fast as ramdisk alternative. :)

Bye, Benjamin.

On Tue, Oct 19, 2010 at 10:40, Frank Bonnetf.bonnet@esiee.fr wrote:

...

Hello

I'm thinking to put db files in RAM instead of hard disk for performance reasons.

FreeBSD ( and Linux thought ) provide some utility to build RAMDISK ( md )

I did some tests in rebuilding db files with the slapcat/slapadd commands

with RAMDISK

ldap3# time slapadd -l test.ldif /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted unlimited privileges. #################### 100.00% eta none elapsed 20s spd 633.9 k/s

WITH HARDDISK

ldap3# time slapadd -l test.ldif /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted unlimited privileges. #################### 100.00% eta none elapsed 04m22s spd 48.4 k/s

Does anyone runs production servers with ramdisk ?

I know it is risky but running rsyncd between ramdisk and a hardisk depot would be safe huh ?

Thanks for any advices

Hello, I've tried this ramdisk concept a few months ago - speed increase is huge, anyway, at least with slapd version from that time, there are some unexpected data integrity problems ( sometimes for unknown reason some data are likely to become corrupted ).

Regarding SSD disks - for now devices with bigger transfer rate are quite expensive, especially bigger capacities, IMO that should work. Note bdb checkpoint cache (depending on your configuration ), may contain more data, than data itself, e.g. with ~128MB of data one may have 300-500MB of log000.* files. Checkpointing configuration and amount of data should be taken into consideration with planning ramdisk storage capacity.

Regarding local syncrepl - slave-initiated gives performance impact, while syncrepl master-initiated scenario (master + syncrepl + ldap-backend on one side ) with current version has some problem with proxying/detectking dsa read-only attributes like contextDSN etc.

Note - ramdisk storage also brings power failure problem - if server goes down unexpectedly, after power back there's no datadir at all, and before you get everything up, you need to restore it (automatically ? ), or use external source ( backup ldap server ).

The only thing which works fine with such ideas is to use ramdisk as temporary-in-the-middle while restoring slapcat backup. Restoring full backup with several thousands of entries (including reindexing), in to-disk scenario takes at least 15-20 minutes, while restoring to ramdisk, and copying the whole data dir from ramdisk to storage media ( with dedicated script ), takes about 2-3 minutes ( also including reindexing, before moving restored backup from ramdisk to HDD ).

Regards, DT

On Tue, 19 Oct 2010, Frank Bonnet wrote:

Perhaps running syncrpl between ramdisk and hardisk ?

On 10/19/2010 11:00 AM, Benjamin Griese wrote:

...

Hi Frank,

by rsync to hdd, do you mean copying the db files or slapcat the output to a file? The first one seems to be a db killer if the db is in use while copying it. :) I think the only safe way is by doing a slapcat after every write to the db (accesslog?).

Anyway, sounds like a nice idea to me to get a big performance boost. Have you considered SSDs instead of ramdisks? Imo its the more safe and nearly as fast as ramdisk alternative. :)

Bye, Benjamin.

On Tue, Oct 19, 2010 at 10:40, Frank Bonnetf.bonnet@esiee.fr wrote:

...

Hello

I'm thinking to put db files in RAM instead of hard disk for performance reasons.

FreeBSD ( and Linux thought ) provide some utility to build RAMDISK ( md )

I did some tests in rebuilding db files with the slapcat/slapadd commands

with RAMDISK

ldap3# time slapadd -l test.ldif /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted unlimited privileges. #################### 100.00% eta none elapsed 20s spd 633.9 k/s

WITH HARDDISK

ldap3# time slapadd -l test.ldif /usr/local/etc/openldap/slapd.conf: line 110: rootdn is always granted unlimited privileges. #################### 100.00% eta none elapsed 04m22s spd 48.4 k/s

Does anyone runs production servers with ramdisk ?

I know it is risky but running rsyncd between ramdisk and a hardisk depot would be safe huh ?

Thanks for any advices