10:07 a.m.
Markus Doppelbauer wrote:
Hello,
We are using BerkeleyDB since a decade - but we are searching for something
more lightweight. LMDB is what we are looking for ... ^^
The small code and the benchmark-results are very impressive!
A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has
about 20 GB.This particular database is a kind of archive - hardly read and hardly
written (maybe once per minute).
Is it possible to "mmap()" a 20 GB database into the RAM?
Yes.
If yes, will the server
start "swapping"?
No.
Thanks a lot
Markus
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
12:03 a.m.
New subject: Antw: Re: LMDB database size
>> Howard Chu <hyc(a)symas.com> schrieb am 25.01.2014 um
19:07 in Nachricht
<52E3FD6E.9070906(a)symas.com>:
Markus Doppelbauer wrote:
> Hello,
> We are using BerkeleyDB since a decade - but we are searching for something
> more lightweight. LMDB is what we are looking for ... ^^
> The small code and the benchmark-results are very impressive!
> A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has
> about 20 GB.This particular database is a kind of archive - hardly read and
hardly
> written (maybe once per minute).
> Is it possible to "mmap()" a 20 GB database into the RAM?
Yes.
> If yes, will the server
> start "swapping"?
No.
This is only true (AFAIK) if the applications uses huge pages or memory locking. In other
cases it depends on the OS and the other applications.
In general mmap files "swap" onto themselves, and you would not call it
"swapping" in the usual sense...
Ulrich
> Thanks a lot
> Markus
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
2:20 a.m.
New subject: Antw: Re: LMDB database size
Ulrich Windl wrote:
>>> Howard Chu <hyc(a)symas.com> schrieb am 25.01.2014 um
19:07 in Nachricht
<52E3FD6E.9070906(a)symas.com>:
> Markus Doppelbauer wrote:
>> Hello,
>> We are using BerkeleyDB since a decade - but we are searching for something
>> more lightweight. LMDB is what we are looking for ... ^^
>> The small code and the benchmark-results are very impressive!
>> A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has
>> about 20 GB.This particular database is a kind of archive - hardly read and
> hardly
>> written (maybe once per minute).
>> Is it possible to "mmap()" a 20 GB database into the RAM?
>
> Yes.
>
>> If yes, will the server
>> start "swapping"?
>
> No.
This is only true (AFAIK)
You've already demonstrated multiple times that "as far as you know" is not
far at all.
if the applications uses huge pages or memory
locking. In other
cases it depends on the OS and the other applications.
In general mmap files "swap" onto themselves, and you would
not call it
"swapping" in the usual sense...
Thus, the answer is "No", as I already said, and you have added no value to
the discussion.
Ulrich
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
11:50 p.m.
New subject: Antw: LMDB database size
>> "Markus Doppelbauer" <doppelbauer(a)gmx.net>
schrieb am 25.01.2014 um 16:04 in
Nachricht
<trinity-2ed9e021-7691-4933-90cc-5ecf9e3509d0-1390662285748@3capp-gmx-bs07>:
Hello,
We are using BerkeleyDB since a decade - but we are searching for something
more lightweight. LMDB is what we are looking for ... ^^
??? More lightweight? Last time I used Sleepycat DB, the extra size for the binary was
about 1MB...
The small code and the benchmark-results are very impressive!
A question: We have a server with 4 GB RAM. One of the BerkeleyDB-files has
about 20 GB.This particular database is a kind of archive - hardly read and
hardly
written (maybe once per minute).
Is it possible to "mmap()" a 20 GB database into the RAM? If yes, will the
server
start "swapping"?
Thanks a lot
Markus
12:21 a.m.
New subject: Antw: LMDB database size
--On Monday, January 27, 2014 8:50 AM +0100 Ulrich Windl
<Ulrich.Windl(a)rz.uni-regensburg.de> wrote:
>>> "Markus Doppelbauer"
<doppelbauer(a)gmx.net> schrieb am 25.01.2014 um
>>> 16:04 in
Nachricht
<trinity-2ed9e021-7691-4933-90cc-5ecf9e3509d0-1390662285748@3capp-gmx-bs0
7>:
> Hello,
>
> We are using BerkeleyDB since a decade - but we are searching for
> something more lightweight. LMDB is what we are looking for ... ^^
??? More lightweight? Last time I used Sleepycat DB, the extra size for
the binary was about 1MB...
The overhead being discussed is not the size of the binary, but the "heavy"
weight of the BDB code (locks, etc) that one has to deal with with BDB
databases. If you paid much attention to the list and published
benchmarks, however, you'd actually already be aware of this. BDB imposes
a substantial overhead penalty compared to LMDB.
--Quanah
--
Quanah Gibson-Mount
Architect - Server
Zimbra, Inc.
--------------------
Zimbra :: the leader in open source messaging and collaboration
10:24 a.m.
* Markus Doppelbauer:
Is it possible to "mmap()" a 20 GB database into the RAM?
As far as I understand it, LMDB has no paging layer on its own, so it
needs 20 GB of address space. So you need a 64-bit architecture and a
system which has not got artificial limits on virtual address space
(as some hosting environments have).
2825
days inactive
2832
days old
openldap-technical@openldap.org
6 comments
5 participants
participants (5)
-
Florian Weimer -
Howard Chu -
Markus Doppelbauer -
Quanah Gibson-Mount -
Ulrich Windl