Hi
Our application do lots of caching using vmtouch, up to a point where there isn't a lot of memory left on the machine. We would like to use LMDB on the same machine to store around 40GiB data of a few hundred million entries.
How can we best understand the interaction and behavior of the OS cache and sharing of memory between processes? Is LMDB doing something to help the OS?
Cheers, -Kristoffer
Kristoffer Sjögren wrote:
Hi
Our application do lots of caching using vmtouch, up to a point where there isn't a lot of memory left on the machine. We would like to use LMDB on the same machine to store around 40GiB data of a few hundred million entries.
How can we best understand the interaction and behavior of the OS cache and sharing of memory between processes? Is LMDB doing something to help the OS?
Nope, LMDB does nothing special with its mmap.
Thanks Howard. What's the best documentation on mmap you know, aside from read kernel source code?
On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Hi
Our application do lots of caching using vmtouch, up to a point where there isn't a lot of memory left on the machine. We would like to use LMDB on the same machine to store around 40GiB data of a few hundred million entries.
How can we best understand the interaction and behavior of the OS cache and sharing of memory between processes? Is LMDB doing something to help the OS?
Nope, LMDB does nothing special with its mmap.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Kristoffer Sjögren wrote:
Thanks Howard. What's the best documentation on mmap you know, aside from read kernel source code?
Not sure what you're asking. Read the mmap(2) manpage and its related pages. If you're thinking there's something fancy hidden in there for you to take advantage of, I don't think you should be using mmap in the first place.
On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Hi
Our application do lots of caching using vmtouch, up to a point where there isn't a lot of memory left on the machine. We would like to use LMDB on the same machine to store around 40GiB data of a few hundred million entries.
How can we best understand the interaction and behavior of the OS cache and sharing of memory between processes? Is LMDB doing something to help the OS?
Nope, LMDB does nothing special with its mmap.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
I'm not looking for fairies :-) I was just curious exactly how mmap interacts with the OS in order to understand performance metrics better.
On Sat, Feb 6, 2016 at 5:46 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Thanks Howard. What's the best documentation on mmap you know, aside from read kernel source code?
Not sure what you're asking. Read the mmap(2) manpage and its related pages. If you're thinking there's something fancy hidden in there for you to take advantage of, I don't think you should be using mmap in the first place.
On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Hi
Our application do lots of caching using vmtouch, up to a point where there isn't a lot of memory left on the machine. We would like to use LMDB on the same machine to store around 40GiB data of a few hundred million entries.
How can we best understand the interaction and behavior of the OS cache and sharing of memory between processes? Is LMDB doing something to help the OS?
Nope, LMDB does nothing special with its mmap.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Mmapped memory goes through the exact same page cache as all other filesystem data -- you probably want to do exactly nothing for optimal performance, the kernel will be smart about what it keeps paged in
On Sat, Feb 6, 2016 at 12:44 PM, Kristoffer Sjögren stoffe@gmail.com wrote:
I'm not looking for fairies :-) I was just curious exactly how mmap interacts with the OS in order to understand performance metrics better.
On Sat, Feb 6, 2016 at 5:46 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Thanks Howard. What's the best documentation on mmap you know, aside from read kernel source code?
Not sure what you're asking. Read the mmap(2) manpage and its related
pages.
If you're thinking there's something fancy hidden in there for you to
take
advantage of, I don't think you should be using mmap in the first place.
On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Hi
Our application do lots of caching using vmtouch, up to a point where there isn't a lot of memory left on the machine. We would like to use LMDB on the same machine to store around 40GiB data of a few hundred million entries.
How can we best understand the interaction and behavior of the OS cache and sharing of memory between processes? Is LMDB doing something to help the OS?
Nope, LMDB does nothing special with its mmap.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Ok, so the page cache, that's exactly what I want to read more about. Not for tuning, but just to understand how the page cache works with the different subsystems in order to dimension the hardware properly. For example what IO devices affect the page cache? Why are memory pages reshuffled between different NUMA nodes? etc..
On Sat, Feb 6, 2016 at 7:15 PM, Jay Booth jaybooth@gmail.com wrote:
Mmapped memory goes through the exact same page cache as all other filesystem data -- you probably want to do exactly nothing for optimal performance, the kernel will be smart about what it keeps paged in
On Sat, Feb 6, 2016 at 12:44 PM, Kristoffer Sjögren stoffe@gmail.com wrote:
I'm not looking for fairies :-) I was just curious exactly how mmap interacts with the OS in order to understand performance metrics better.
On Sat, Feb 6, 2016 at 5:46 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Thanks Howard. What's the best documentation on mmap you know, aside from read kernel source code?
Not sure what you're asking. Read the mmap(2) manpage and its related pages. If you're thinking there's something fancy hidden in there for you to take advantage of, I don't think you should be using mmap in the first place.
On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Hi
Our application do lots of caching using vmtouch, up to a point where there isn't a lot of memory left on the machine. We would like to use LMDB on the same machine to store around 40GiB data of a few hundred million entries.
How can we best understand the interaction and behavior of the OS cache and sharing of memory between processes? Is LMDB doing something to help the OS?
Nope, LMDB does nothing special with its mmap.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
Maybe what I was looking for was resources like this https://0xax.gitbooks.io/linux-insides/content.
On Sat, Feb 6, 2016 at 7:54 PM, Kristoffer Sjögren stoffe@gmail.com wrote:
Ok, so the page cache, that's exactly what I want to read more about. Not for tuning, but just to understand how the page cache works with the different subsystems in order to dimension the hardware properly. For example what IO devices affect the page cache? Why are memory pages reshuffled between different NUMA nodes? etc..
On Sat, Feb 6, 2016 at 7:15 PM, Jay Booth jaybooth@gmail.com wrote:
Mmapped memory goes through the exact same page cache as all other filesystem data -- you probably want to do exactly nothing for optimal performance, the kernel will be smart about what it keeps paged in
On Sat, Feb 6, 2016 at 12:44 PM, Kristoffer Sjögren stoffe@gmail.com wrote:
I'm not looking for fairies :-) I was just curious exactly how mmap interacts with the OS in order to understand performance metrics better.
On Sat, Feb 6, 2016 at 5:46 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote:
Thanks Howard. What's the best documentation on mmap you know, aside from read kernel source code?
Not sure what you're asking. Read the mmap(2) manpage and its related pages. If you're thinking there's something fancy hidden in there for you to take advantage of, I don't think you should be using mmap in the first place.
On Fri, Feb 5, 2016 at 10:33 PM, Howard Chu hyc@symas.com wrote:
Kristoffer Sjögren wrote: > > > Hi > > Our application do lots of caching using vmtouch, up to a point where > there isn't a lot of memory left on the machine. We would like to use > LMDB on the same machine to store around 40GiB data of a few hundred > million entries. > > How can we best understand the interaction and behavior of the OS > cache and sharing of memory between processes? Is LMDB doing > something > to help the OS?
Nope, LMDB does nothing special with its mmap.
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
-- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
openldap-technical@openldap.org
-
Howard Chu -
Jay Booth -
Kristoffer Sjögren