Markus Doppelbauer wrote:
Hello, We are using BerkeleyDB since a decade - but we are searching for something more lightweight. LMDB is what we are looking for ... ^^ The small code and the benchmark-results are very impressive! A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has about 20 GB.This particular database is a kind of archive - hardly read and hardly written (maybe once per minute). Is it possible to "mmap()" a 20 GB database into the RAM?
Yes.
If yes, will the server start "swapping"?
No.
Thanks a lot Markus
Howard Chu hyc@symas.com schrieb am 25.01.2014 um 19:07 in Nachricht
Markus Doppelbauer wrote:
Hello, We are using BerkeleyDB since a decade - but we are searching for something more lightweight. LMDB is what we are looking for ... ^^ The small code and the benchmark-results are very impressive! A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has about 20 GB.This particular database is a kind of archive - hardly read and
hardly
written (maybe once per minute). Is it possible to "mmap()" a 20 GB database into the RAM?
Yes.
If yes, will the server start "swapping"?
No.
This is only true (AFAIK) if the applications uses huge pages or memory locking. In other cases it depends on the OS and the other applications. In general mmap files "swap" onto themselves, and you would not call it "swapping" in the usual sense...
Ulrich
Thanks a lot Markus
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Ulrich Windl wrote:
Howard Chu hyc@symas.com schrieb am 25.01.2014 um 19:07 in Nachricht
Markus Doppelbauer wrote:
Hello, We are using BerkeleyDB since a decade - but we are searching for something more lightweight. LMDB is what we are looking for ... ^^ The small code and the benchmark-results are very impressive! A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has about 20 GB.This particular database is a kind of archive - hardly read and
hardly
written (maybe once per minute). Is it possible to "mmap()" a 20 GB database into the RAM?
Yes.
If yes, will the server start "swapping"?
No.
This is only true (AFAIK)
You've already demonstrated multiple times that "as far as you know" is not far at all.
if the applications uses huge pages or memory
locking. In other cases it depends on the OS and the other applications.
In general mmap files "swap" onto themselves, and you would not call it
"swapping" in the usual sense...
Thus, the answer is "No", as I already said, and you have added no value to the discussion.
Ulrich
"Markus Doppelbauer" doppelbauer@gmx.net schrieb am 25.01.2014 um 16:04 in
Nachricht trinity-2ed9e021-7691-4933-90cc-5ecf9e3509d0-1390662285748@3capp-gmx-bs07:
Hello,
We are using BerkeleyDB since a decade - but we are searching for something more lightweight. LMDB is what we are looking for ... ^^
??? More lightweight? Last time I used Sleepycat DB, the extra size for the binary was about 1MB...
The small code and the benchmark-results are very impressive!
A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has about 20 GB.This particular database is a kind of archive - hardly read and hardly written (maybe once per minute). Is it possible to "mmap()" a 20 GB database into the RAM? If yes, will the server start "swapping"?
Thanks a lot Markus
--On Monday, January 27, 2014 8:50 AM +0100 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
"Markus Doppelbauer" doppelbauer@gmx.net schrieb am 25.01.2014 um 16:04 in
Nachricht <trinity-2ed9e021-7691-4933-90cc-5ecf9e3509d0-1390662285748@3capp-gmx-bs0 7>:
Hello,
We are using BerkeleyDB since a decade - but we are searching for something more lightweight. LMDB is what we are looking for ... ^^
??? More lightweight? Last time I used Sleepycat DB, the extra size for the binary was about 1MB...
The overhead being discussed is not the size of the binary, but the "heavy" weight of the BDB code (locks, etc) that one has to deal with with BDB databases. If you paid much attention to the list and published benchmarks, however, you'd actually already be aware of this. BDB imposes a substantial overhead penalty compared to LMDB.
--Quanah
--
Quanah Gibson-Mount Architect - Server Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration
* Markus Doppelbauer:
Is it possible to "mmap()" a 20 GB database into the RAM?
As far as I understand it, LMDB has no paging layer on its own, so it needs 20 GB of address space. So you need a 64-bit architecture and a system which has not got artificial limits on virtual address space (as some hosting environments have).
openldap-technical@openldap.org
-
Florian Weimer -
Howard Chu -
Markus Doppelbauer -
Quanah Gibson-Mount -
Ulrich Windl