Jürgen Baier wrote:
Hi,
I'm using LMDB for mapping MD5 hash codes to some data. I noticed that a virtualized environment (Xen/Windows on our own servers and AWS/Windows) slows down LMDB significantly (e.g. a certain workload is executed in 30 minutes on a local machine vs. 15 hours on AWS).
Sounds like whatever filesystem is running on AWS has a really poor implementation of mmap.
My application also has an implementation in SQLite which is slower than LMDB on a local machine. But its performance is similar on AWS. The rough numbers are something like
Local:
- LMDB: 30 minutes
- SQLite: 45 minutes
AWS:
- LMDB: 15 hours
- SQLite: 50 minutes
I wanted to ask if there is something I can do to speed up LMDB on AWS? Maybe I'm missing something obvious.
Are reads slow, or writes? Or both? If writes are slow, try setting MDB_NOSYNC. If reads are slow,\ use a different filesystem.
Thanks,
Jürgen