Hello everyone.
I could find (very) few results while searching the internet for MDB_MAP_RESIZED error in OpenLDAP, and they were of no help.
I'll describe my setup and the circumstances of the error.
I have OpenLDAP (version 2.4.47) and LMDB (version 0.9.22) installed from Debian 10 default packages in a amd64 box.
Along with some usual settings (my slapd.conf file is attached), I set the main database up using the mdb backend with maxsize = 7516192768 (7GB). The on-disk base size (the data.mdb file) is 134MB. I also have loglevel = stats.
Everything seems to work flawless and fast, so all of a sudden syslog prints this message:
Apr 14 09:47:56 my-ldap-box slapd[20736]: mdb_opinfo_get: err MDB_MAP_RESIZED: Database contents grew beyond environment mapsize(-30785)
and seconds later, the slapd daemon stop to answer requests, although the service appears to be still running. This has happened 7 times since April 7, when I moved the data from an old server to this one.
I changed loglevel to 'stats trace args shell' and I realized that when the error occurs, there were always some ADD operations logged right before.
I inspected the logs today, April 14, right after the occurrence of the error and attached a file containing: 1) the error message, which appears at 09:47:56, 2) the messages logged before, since 09:47:50, 3) the messages logged after, until 09:48:01. This time on, slapd logged no more messages and appeared to be stuck for the clients.
At 09:53:41 I restarted the daemon. The log generated from this time until the first request answered is also attached.
For the record, I initially suspected of the replica server which was running alongside the main server and which showed the same behavior. This was because at then I associated the error with syncrepl operations. But even after I stop the replica, the error keeps ocurring in the main server.
Any clues? Thanks for your consideration.
--On Wednesday, April 14, 2021 9:42 PM +0000 Leonardo Bruno Lopes leonardo@cefetmg.br wrote:
I have OpenLDAP (version 2.4.47) and LMDB (version 0.9.22) installed from Debian 10 default packages in a amd64 box.
First, upgrade to a current release. 2.4.47 is not safe to use with replication.
Apr 14 09:47:56 my-ldap-box slapd[20736]: mdb_opinfo_get: err MDB_MAP_RESIZED: Database contents grew beyond environment mapsize(-30785)
You failed to set a maxsize on the accesslog database, so it's using the default of 10MB. You need to increase its size, as it's clearly running out of space.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Quanah Gibson-Mount wrote:
--On Wednesday, April 14, 2021 9:42 PM +0000 Leonardo Bruno Lopes leonardo@cefetmg.br wrote:
I have OpenLDAP (version 2.4.47) and LMDB (version 0.9.22) installed from Debian 10 default packages in a amd64 box.
First, upgrade to a current release. 2.4.47 is not safe to use with replication.
Apr 14 09:47:56 my-ldap-box slapd[20736]: mdb_opinfo_get: err MDB_MAP_RESIZED: Database contents grew beyond environment mapsize(-30785)
You failed to set a maxsize on the accesslog database, so it's using the default of 10MB. You need to increase its size, as it's clearly running out of space.
No, that's MDB_MAP_FULL. This particular error means some other process opened the environment and increased the mapsize. Currently back-mdb doesn't handle this error code because nobody should ever be doing that to an OpenLDAP installation.
--On Wednesday, April 14, 2021 11:49 PM +0100 Howard Chu hyc@symas.com wrote:
Apr 14 09:47:56 my-ldap-box slapd[20736]: mdb_opinfo_get: err MDB_MAP_RESIZED: Database contents grew beyond environment mapsize(-30785)
You failed to set a maxsize on the accesslog database, so it's using the default of 10MB. You need to increase its size, as it's clearly running out of space.
No, that's MDB_MAP_FULL. This particular error means some other process opened the environment and increased the mapsize. Currently back-mdb doesn't handle this error code because nobody should ever be doing that to an OpenLDAP installation.
Ok, well, that's bizarre. The issue with the accesslog DB still exists too though. :) I would guess that that's the database that got resized. Guess the next question is, where did they get a program that's illegally accessing the slapd database and attempting to resize it.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Thank you again, Quanah
Ok, well, that's bizarre. The issue with the accesslog DB still exists too though. :) I would guess that that's the database that got resized. Guess the next question is, where did they get a program that's illegally accessing the slapd database and attempting to resize it.
Maybe I got this correctly, but besides the use of mdb as backend and, maybe the accesslog overlay, my setup is quite ordinary. Speaking in accesslog, I have logpurge 00+00:10 00+00:10, which I think is a little too often. However, the problem occurs in such a way that is well time spaced. It's trully bizarre.
Regards, Leonardo Lopes
--On Thursday, April 15, 2021 12:19 AM +0000 Leonardo Lopes leonardo@cefetmg.br wrote:
Thank you again, Quanah
Ok, well, that's bizarre. The issue with the accesslog DB still exists too though. :) I would guess that that's the database that got resized. Guess the next question is, where did they get a program that's illegally accessing the slapd database and attempting to resize it.
Maybe I got this correctly, but besides the use of mdb as backend and, maybe the accesslog overlay, my setup is quite ordinary. Speaking in accesslog, I have logpurge 00+00:10 00+00:10, which I think is a little too often. However, the problem occurs in such a way that is well time spaced. It's trully bizarre.
You seem to fail to comprehend that you have *TWO* entirely separate MDB databses defined in your configuration. You set the maxsize on one of them, you did not set the max size on the other. That's why <something> is attempting to resize your database. You'll have to track down what's attempting to do that resize, because it's not OpenLDAP. But the thing you really need to do is fix your configuration so both databases have a maxsize configured.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
You seem to fail to comprehend that you have *TWO* entirely separate MDB databses defined in your configuration. You set the maxsize on one of them, you did not set the max size on the other. That's why <something> is attempting to resize your database. You'll have to track down what's attempting to do that resize, because it's not OpenLDAP. But the thing you really need to do is fix your configuration so both databases have a maxsize configured.
Oh. Now I see. It's true, I have selected mdb backend for log database too, and as you said, maxsize for this database is not explicitly set and therefore slpad is using the default value which is not enough. I will correct this and see if the error stops. That totally makes sense, but only now I got it.
Thank you so much.
--On Thursday, April 15, 2021 12:50 AM +0000 Leonardo Lopes leonardo@cefetmg.br wrote:
You seem to fail to comprehend that you have *TWO* entirely separate MDB databses defined in your configuration. You set the maxsize on one of them, you did not set the max size on the other. That's why <something> is attempting to resize your database. You'll have to track down what's attempting to do that resize, because it's not OpenLDAP. But the thing you really need to do is fix your configuration so both databases have a maxsize configured.
Oh. Now I see. It's true, I have selected mdb backend for log database too, and as you said, maxsize for this database is not explicitly set and therefore slpad is using the default value which is not enough. I will correct this and see if the error stops. That totally makes sense, but only now I got it.
So you still need to track down what process was trying to modify slapd's database outside of slapd, as that's not legal to do. I'd hope it wasn't something Debian's done in some bizarre fashion.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Thak you, Howard
No, that's MDB_MAP_FULL. This particular error means some other process opened the environment and increased the mapsize. Currently back-mdb doesn't handle this error code because nobody should ever be doing that to an OpenLDAP installation.
I found a bug report filled years ago that seems to be related to this: https://bugs.openldap.org/show_bug.cgi?id=8179
Also, I think I didn't understand what you mean with "nobody should ever be doing that to an OpenLDAP installation". Shouldn't I use mdb as OpenLDAP backend?
Regards, Leonardo Lopes
--On Thursday, April 15, 2021 12:10 AM +0000 Leonardo Lopes leonardo@cefetmg.br wrote:
Thak you, Howard
No, that's MDB_MAP_FULL. This particular error means some other process opened the environment and increased the mapsize. Currently back-mdb doesn't handle this error code because nobody should ever be doing that to an OpenLDAP installation.
I found a bug report filled years ago that seems to be related to this: https://bugs.openldap.org/show_bug.cgi?id=8179
No, this has nothing to do with this issue.
Also, I think I didn't understand what you mean with "nobody should ever be doing that to an OpenLDAP installation". Shouldn't I use mdb as OpenLDAP backend?
You have an external process that's illegally modifying the mdb database outside of slapd.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
Tnaks, Quanah
First, upgrade to a current release. 2.4.47 is not safe to use with replication.
I think I'll make do without replication by now, as for me it is almost mandatory to stick to the official Debian packages.
You failed to set a maxsize on the accesslog database, so it's using the default of 10MB. You need to increase its size, as it's clearly running out of space.
That seems weird as I get no configuration error. slaptest -d config does not complain nor slapd in logs, but I'll double check.
Regards, Leonardo Lopes
--On Thursday, April 15, 2021 12:05 AM +0000 Leonardo Lopes leonardo@cefetmg.br wrote:
Tnaks, Quanah
First, upgrade to a current release. 2.4.47 is not safe to use with replication.
I think I'll make do without replication by now, as for me it is almost mandatory to stick to the official Debian packages.
I'd strongly recommend revising your "mandatory" policy. Running such an old release is an exceptionally bad idea, replication aside.
You failed to set a maxsize on the accesslog database, so it's using the default of 10MB. You need to increase its size, as it's clearly running out of space.
That seems weird as I get no configuration error. slaptest -d config does not complain nor slapd in logs, but I'll double check.
Why would slaptest complain? I clearly stated it's using the default size, so clearly it has a configured value. This is explicitly documented in the slapd-mdb(5) man page.
The default value is also clearly too small for your usage.
Regards, Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
I'd strongly recommend revising your "mandatory" policy. Running such an old release is an exceptionally bad idea, replication aside.
For sure. It's kinda lazy rule, as we suppose the packages are released with Debian in a reasonable version.
Why would slaptest complain? I clearly stated it's using the default size, so clearly it has a configured value. This is explicitly documented in the slapd-mdb(5) man page.
Yes, you did, but for me maxsize 7516192768 would be enough. I'll dig deeper in the docs.
I'd strongly recommend revising your "mandatory" policy. Running such an old release is an exceptionally bad idea, replication aside.
I've just upgraded OpenLDAP to the 2.4.57 version. For now it's the closer I can get to the most recent release. I don't know if the 2.4.57 version can safely do replication. Anyway I can live without replication for a while, so I'll let it disabled.
--On Thursday, April 15, 2021 2:13 AM +0000 Leonardo Lopes leonardo@cefetmg.br wrote:
I'd strongly recommend revising your "mandatory" policy. Running such an old release is an exceptionally bad idea, replication aside.
I've just upgraded OpenLDAP to the 2.4.57 version. For now it's the closer I can get to the most recent release. I don't know if the 2.4.57 version can safely do replication. Anyway I can live without replication for a while, so I'll let it disabled.
2.4.57 should generally be fine for replication outside of a few edge cases that you're unlikely to hit.
--Quanah
--
Quanah Gibson-Mount Product Architect Symas Corporation Packaged, certified, and supported LDAP solutions powered by OpenLDAP: http://www.symas.com
openldap-technical@openldap.org
-
Howard Chu -
Leonardo Bruno Lopes -
Leonardo Lopes
-
Quanah Gibson-Mount