Jürgen Baier baier@semedy.com schrieb am 07.10.2021 um 08:07 in
Nachricht a83eb812-351f-b332-45d0-3ef74b9c5c30@semedy.com:
Hi,
I'm using LMDB for mapping MD5 hash codes to some data. I noticed that a virtualized environment (Xen/Windows on our own servers and AWS/Windows) slows down LMDB significantly (e.g. a certain workload is executed in 30 minutes on a local machine vs. 15 hours on AWS).
I don't know whether Winows uses huge pages at all, but AFAIK huge pages are unavaiable under Xen. Maybe that makes a difference for large memory configurations (in addition to the other things mentioned). The other thing is whether your VM has a dedicated CPU or an "overcommitted" shared CPU.
However 15 hours vs. 30 minutes sounds drastic.
My application also has an implementation in SQLite which is slower than LMDB on a local machine. But its performance is similar on AWS. The rough numbers are something like
Local:
- LMDB: 30 minutes
- SQLite: 45 minutes
AWS:
- LMDB: 15 hours
- SQLite: 50 minutes
I wanted to ask if there is something I can do to speed up LMDB on AWS? Maybe I'm missing something obvious.
Thanks,
Jürgen
-- Juergen Baier