Luca Foppiano wrote:
Dear everybody, I'm using one of your component (LMDB) via a java JNDI bindings implementation (https://github.com/deephacks/lmdbjni) and I'm having an issue when I deploy my LMDB file on a tempfs filesystem in RAM. The issue do not occur when the LMDB files are stored on a "normal" filesystem. When the data is in the tempfs ramdisk all the allocated memory ends up being in the Dirty area (it has not been written back to the Filesytem). Here an example using the ramdisk:
7ce320000000-7cfc20000000 r--s 00000000 00:26 2459 /ramfs/nerd/data/db/db-en/entityEmbeddings/data.mdb Size: 104857600 kB Rss: 1255680 kB Pss: 1255680 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 1255680 kB <--- Referenced: 1255680 kB Anonymous: 0 kB AnonHugePages: 0 kB Shared_Hugetlb: 0 kB Private_Hugetlb: 0 kB Swap: 0 kB SwapPss: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB VmFlags: rd sh mr mw me ms sd
and here an example without:
7ca4fc000000-7cbdfc000000 r--s 00000000 fd:00 11154951 /data/workspace/shared/nerd-data/db/db-en/entityEmbeddings/data.mdb Size: 104857600 kB Rss: 838124 kB Pss: 838124 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 838124 kB <---- Private_Dirty: 0 kB Referenced: 764872 kB Anonymous: 0 kB AnonHugePages: 0 kB ShmemPmdMapped: 0 kB Shared_Hugetlb: 0 kB Private_Hugetlb: 0 kB Swap: 0 kB SwapPss: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB VmFlags: rd sh mr mw me ms sd
According to my understanding the memory is dirty when 1)there are open transactions, 2) the data has not been written back to the filesystem
Your understanding is incorrect. Dirty pages remain dirty until they are written to stable storage (e.g., disk). A tmpfs/RAMdisk has no stable storage, all of its pages reside only in RAM. That's the point of a RAMdisk.
What I don't understand is why there is a difference between filesystem and ramdisk? Is there any reason? The application (listed above) is not writing on the lmdb, but just reading (using reading transaction). Thank you Luca
Using tmpfs is a waste of RAM. Just use LMDB on a regular filesystem and let the system's pagecache manager take care of memory.