--On June 27, 2013 1:40:16 AM -0700 Howard Chu hyc@symas.com wrote:
I tried a load on an ext4 system with options 'rw,noatime, user_xattr, barrier=1, data=writeback' and got a load time of 01h40m06s. This is the best time I have gotten so far loading on ext4.
Did you try commit=60 barrier=0 ?
Here are the details of a test using commit=60 barrier=0.
mkfs -t ext4 \ -O "^flex_bg ^huge_file ^uninit_bg ^dir_nlink ^extra_isize ^extent" mount -t ext4 -o rw,noatime,barrier=0,commit=60,data=writeback Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file mount optiions: rw, noatime, user_xattr, commit=60, barrier=0, data=writeback elapsed 01h40m55s spd 211.0 k/s
I ended up writing a script that creates an ext2 file systems, loads the backend, umounts the partition, converts it to ext4 journaling, and then mounts the partition again. This will allow me to continue with the server rebuilds, but it is a pretty ugly hack.
I am now getting close to ext2 performance using ext3, but ext4 is consistently too slow in all of my tests. Here are the results of the fastest ext3 and fastest ext4 tests.
* mkfs -t ext3 -O has_journal mount -t ext3 -o rw,noatime,data=writeback Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file mount options: rw, noatime, errors=continue, user_xattr, acl, barrier=1, data=writeback elapsed 22m03s spd 965.6 k/s
* mkfs -t ext4 -O "^flex_bg ^huge_file ^uninit_bg ^dir_nlink ^extra_isize" mount -t ext4 -o rw,noatime,data=writeback Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent sparse_super large_file mount options: rw, noatime, user_xattr, barrier=1, data=writeback elapsed 01h32m19s spd 230.6 k/s
During a load the status display would stall periodically. The worse the load time the more frequently the display stalled and the longer it stalled for. I guessing that this is flushing data to the disk. I am also guessing that since mdb is using memory mapped files that some tuning of memory management might help improve performance. I am not familiar with the tuning knobs there so any pointers would be appreciated.
Bill