Hi,
I'm looking for the right method to monitor the usage of an MDB database according to the limit of its size imposed via the parameter olcDbMaxSize. Currently, I am using the following command:
# mdb_stat -e /var/lib/ldap/accesslog/ Environment Info Map address: (nil) Map size: 4294967296 Page size: 4096 Max pages: 1048576 Number of pages used: 1048572 Last transaction ID: 24153425 Max readers: 126 Number of readers used: 18 Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 6
Here, I determine the percentage of database usage with the information "Number of pages used" and "Max pages".
This seemed to be a good method, but I noticed that after a significant purge of objects in the directory, the number of used pages does not seem to decrease.
Is there another and/or better method to monitor this? Failing that, is there a solution to release "falsely" used pages, ideally without too much downtime of the service (so not a dump/restore)? I read that it was possible to use the "mdb_copy -c" command, but I am a little concerned about the duration on a database with approximately 1,000,000 entries.
Thank you in advance.
Le 07/05/2024 à 17:07, Benjamin Renard a écrit :
Hi,
I'm looking for the right method to monitor the usage of an MDB database according to the limit of its size imposed via the parameter olcDbMaxSize. Currently, I am using the following command:
# mdb_stat -e /var/lib/ldap/accesslog/ Environment Info Map address: (nil) Map size: 4294967296 Page size: 4096 Max pages: 1048576 Number of pages used: 1048572 Last transaction ID: 24153425 Max readers: 126 Number of readers used: 18 Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 6
Here, I determine the percentage of database usage with the information "Number of pages used" and "Max pages".
This seemed to be a good method, but I noticed that after a significant purge of objects in the directory, the number of used pages does not seem to decrease.
Is there another and/or better method to monitor this? Failing that, is there a solution to release "falsely" used pages, ideally without too much downtime of the service (so not a dump/restore)? I read that it was possible to use the "mdb_copy -c" command, but I am a little concerned about the duration on a database with approximately 1,000,000 entries.
Thank you in advance.
Hello Benjamin,
if it helps, we have created a monitoring script for this: https://ltb-project.org/documentation/check_lmdb_usage.html
Clément OUDOT wrote:
Le 07/05/2024 à 17:07, Benjamin Renard a écrit :
Hi,
I'm looking for the right method to monitor the usage of an MDB database according to the limit of its size imposed via the parameter olcDbMaxSize. Currently, I am using the following command:
# mdb_stat -e /var/lib/ldap/accesslog/ Environment Info Map address: (nil) Map size: 4294967296 Page size: 4096 Max pages: 1048576 Number of pages used: 1048572 Last transaction ID: 24153425 Max readers: 126 Number of readers used: 18 Status of Main DB Tree depth: 1 Branch pages: 0 Leaf pages: 1 Overflow pages: 0 Entries: 6
Here, I determine the percentage of database usage with the information "Number of pages used" and "Max pages".
This seemed to be a good method, but I noticed that after a significant purge of objects in the directory, the number of used pages does not seem to decrease.
You also need to account for the freelist. Use `mdb_stat -ef`.
Is there another and/or better method to monitor this? Failing that, is there a solution to release "falsely" used pages, ideally without too much downtime of the service (so not a dump/restore)? I read that it was possible to use the "mdb_copy -c" command, but I am a little concerned about the duration on a database with approximately 1,000,000 entries.
There is no reason to release free pages. They will get reused again anyway.
Hello Benjamin,
if it helps, we have created a monitoring script for this: https://ltb-project.org/documentation/check_lmdb_usage.html
I think the question relates to the fact that "Used" pages are not always *actually* in use.
But it seems that this is not a very important question. The important thing to monitor is that the percentage of used does not get close to the max. But also, note that the maxsize can be larger than the underlying disk storage available space. This can be dangerous. It's always important to monitor both the used/max percentage as well as the used% of the underlying disk storage.
Now that we're on this topic. And this is only an academic question, probably: I'm wondering what are olmMDBPagesFree? It seems intuitive that (max - used) would equate to free, but it does not. Are "free" pages pages that were once used?
$ ldapsearch -ZH ldap://ldap2.rexpro.local:3892 -w $W -s base -b "cn=Database 1,cn=Databases,cn=Monitor" olmMDBPagesMax olmMDBPagesUsed olmMDBPagesFree dn: cn=Database 1,cn=Databases,cn=Monitor olmMDBPagesMax: 262144 olmMDBPagesUsed: 34465 olmMDBPagesFree: 395
--On Tuesday, May 7, 2024 6:07 PM +0200 Benjamin Renard brenard@easter-eggs.com wrote:
Hi,
I'm looking for the right method to monitor the usage of an MDB database according to the limit of its size imposed via the parameter olcDbMaxSize. Currently, I am using the following command:
If you are using a modern version of OpenLDAP, you can simply query the monitor backend to get the necessary information to calculate usage, no need to use mdb_stat at all.
--Quanah
Thank you all for your help. I had indeed not understood the notion of 'free pages' which affected the way to retrieve the actual number of used pages.
And indeed, with recent versions of OpenLDAP, I noted that it was possible to retrieve this information by querying the monitor database (attributes olmMDBPagesMax, olmMDBPagesUsed & olmMDBPagesFree for those wondering).
@Clément: I also implemented a similar monitoring plugin :) (https://exchange.icinga.com/brenard/check_mdb). Don't forget to share your works on this great website (and possibly on https://exchange.nagios.org) ;)
Le mar. 7 mai 2024 à 19:32, Benjamin Renard brenard@easter-eggs.com a écrit :
@Clément: I also implemented a similar monitoring plugin :) (https://exchange.icinga.com/brenard/check_mdb). Don't forget to share your works on this great website (and possibly on https://exchange.nagios.org) ;)
Great, thanks for the pointers!
If you want a monitor, Nagios XI is a pretty affordable and effective way. MDB monitoring is part of it.
From: Quanah Gibson-Mount quanah@fast-mail.org Sent: Tuesday, May 7, 2024 11:56 AM To: Benjamin Renard brenard@easter-eggs.com; openldap-technical@openldap.org Subject: [EXTERNAL] Re: How to properly monitor MDB usage
--On Tuesday, May 7, 2024 6: 07 PM +0200 Benjamin Renard <brenard@ easter-eggs. com> wrote: > Hi, > > I'm looking for the right method to monitor the usage of an MDB database > according to the limit of its size imposed via the
--On Tuesday, May 7, 2024 6:07 PM +0200 Benjamin Renard
<brenard@easter-eggs.commailto:brenard@easter-eggs.com> wrote:
Hi,
I'm looking for the right method to monitor the usage of an MDB database
according to the limit of its size imposed via the parameter
olcDbMaxSize. Currently, I am using the following command:
If you are using a modern version of OpenLDAP, you can simply query the
monitor backend to get the necessary information to calculate usage, no
need to use mdb_stat at all.
--Quanah
I accidentally dropped this email, replying so it shows up ont he list.
--Quanah
--On Wednesday, May 8, 2024 2:13 PM +0000 Bradley T Gill bgill@aep.com wrote:
If you want a monitor, Nagios XI is a pretty affordable and effective way. MDB monitoring is part of it.
From: Quanah Gibson-Mount quanah@fast-mail.org Sent: Tuesday, May 7, 2024 11:56 AM To: Benjamin Renard brenard@easter-eggs.com; openldap-technical@openldap.org Subject: [EXTERNAL] Re: How to properly monitor MDB usage
--On Tuesday, May 7, 2024 6: 07 PM +0200 Benjamin Renard <brenard@ easter-eggs. com> wrote: > Hi, > > I'm looking for the right method to monitor the usage of an MDB database > according to the limit of its size imposed via the
--On Tuesday, May 7, 2024 6:07 PM +0200 Benjamin Renard
brenard@easter-eggs.com wrote:
Hi,
I'm looking for the right method to monitor the usage of an MDB database
according to the limit of its size imposed via the parameter
olcDbMaxSize. Currently, I am using the following command:
If you are using a modern version of OpenLDAP, you can simply query the
monitor backend to get the necessary information to calculate usage, no
need to use mdb_stat at all.
--Quanah
On 5/8/24 6:13 AM, Bradley T Gill bgill@aep.com wrote:
If you want a monitor, Nagios XI is a pretty affordable and effective way. MDB monitoring is part of it.
I'm biased since Rex is a Nagios reseller. But after using Nagios for 20 years (and XI since it came out), I 100% agree. Nagios is the way to go for monitoring LDAP. In addition to monitoring MDB utilization, I would recommend in addition to monitoring MDB utilization, to monitor replication and response time.
openldap-technical@openldap.org
-
Benjamin Renard -
Bradley T Gill -
chris.paul＠rexconsulting.net -
Christopher Paul
-
Clément OUDOT -
Clément OUDOT -
Howard Chu -
Quanah Gibson-Mount