Because this is so unexpected and it impacts OpenLDAP installations, I though I share this information here.
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data. I don't understand why the or a upgrade would want to purge data but that is what happens. I can't find anything on the internet explaining why this is unavoidable and necessary.
From what I have learned so far the best way to recover from this is to
completly remove OpenLDAP and start a fresh installation and import data from backup. I'm sure you will have it, for exactly this types of situations that are I'm afraid typical for OpenLDAP.
The reason is that it is unclear what state the upgrade leaves OpenLDAP in. I think it will take more time to figure out what was purged, what not, what was changed than it takes to do a complete reinstall.
On Sun, 2014-12-21 at 13:21 +0100, Onno van der Straaten wrote:
Because this is so unexpected and it impacts OpenLDAP installations, I though I share this information here.
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data. I don't understand why the or a upgrade would want to purge data but that is what happens. I can't find anything on the internet explaining why this is unavoidable and necessary.
From what I have learned so far the best way to recover from this is to completly remove OpenLDAP and start a fresh installation and import data from backup. I'm sure you will have it, for exactly this types of situations that are I'm afraid typical for OpenLDAP.
The reason is that it is unclear what state the upgrade leaves OpenLDAP in. I think it will take more time to figure out what was purged, what not, what was changed than it takes to do a complete reinstall.
stab in the dark. is there an update to the filesystems used (ext3/4 to xfs or something)?
On Sun, Dec 21, 2014 at 01:21:21PM +0100, Onno van der Straaten wrote:
Because this is so unexpected and it impacts OpenLDAP installations, I though I share this information here.
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data.
That would be a severe bug in the Ubuntu package. If you can reproduce this (for example, by restoring a backup and repeating the upgrade), please file a Launchpad bug ('ubuntu-bug slapd'), and we can discuss it there, since Ubuntu packaging bugs are off-topic for this list.
Florian Weimer wrote:
- Onno van der Straaten:
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data.
Is there an Ubuntu bug for this behavior? It's a bit odd.
One should generally be careful with system upgrades.
AFAIK on-disk-format might change because of Berkeley-DB version upgrade.
But without knowing about the backend configuration used it's impossible to track this down.
Ciao, Michael. (not an Ubuntu user)
At Sun, 21 Dec 2014 23:14:42 +0100 [** ISO-8859-1 charset **] Michael Strödermichael@stroeder.com wrote:
Florian Weimer wrote:
- Onno van der Straaten:
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data.
Is there an Ubuntu bug for this behavior? It's a bit odd.
One should generally be careful with system upgrades.
Esp. major version jumps.
AFAIK on-disk-format might change because of Berkeley-DB version upgrade.
Which suggests that one needs to dump the 12.04 database (to ldif text format) and then reload it into the fresh 14.04 database.
(This is what I did when I upgraded a old server running CentOS 5 to a new server running CentOS 6.)
But without knowing about the backend configuration used it's impossible to track this down.
This is the reason *RedHat* Enterprise Linux is never recomended to do upgrades across major versions. The recomended practice is allways a fresh install. (Of course, RHEL has a much longer lifetime and package APIs generally make a *big* jump.) Ubuntu has a shorter life span, so things *generally* don't make such a big jump, but it looks like that is not always the case.
Ciao, Michael. (not an Ubuntu user)
* Robert Heller:
This is the reason *RedHat* Enterprise Linux is never recomended to do upgrades across major versions. The recomended practice is allways a fresh install.
That's mostly because Red Hat does not want to support such installations of unknown genealogy. Technically, in-place upgrades for most systems work, as long as you take care of the usual offenders (e.g., Berkeley DB, as Michael pointed out, and PostgreSQL).
* Michael Ströder:
Florian Weimer wrote:
- Onno van der Straaten:
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data.
Is there an Ubuntu bug for this behavior? It's a bit odd.
One should generally be careful with system upgrades.
AFAIK on-disk-format might change because of Berkeley-DB version upgrade.
Unless OpenLDAP uses the HASH database type, it's sufficient to checkpoint the Berkeley DB environment (and perhaps some other undocumented magic) *before* updating OpenLDAP. The on-disk format of B-Tree-type databases is quite stable.
(Automated dump and reload might not work for large databases during automated package upgrades because of the time and disk space overhead.)
Florian Weimer wrote:
- Michael Ströder:
Florian Weimer wrote:
- Onno van der Straaten:
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data.
Is there an Ubuntu bug for this behavior? It's a bit odd.
One should generally be careful with system upgrades.
AFAIK on-disk-format might change because of Berkeley-DB version upgrade.
Unless OpenLDAP uses the HASH database type, it's sufficient to checkpoint the Berkeley DB environment (and perhaps some other undocumented magic) *before* updating OpenLDAP. The on-disk format of B-Tree-type databases is quite stable.
(Automated dump and reload might not work for large databases during automated package upgrades because of the time and disk space overhead.)
The original poster did a significant distribution upgrade and not just a package upgrade.
While there might be distributions which have some sort of upgrade magic I strongly recommend that everybody should follow the standard slapcat/slapadd procedure when doing significant upgrade. And of course do a backup right before upgrade after turning down writing to the directory. Everything else I would not trust enough for a serious deployment.
Ciao, Michael.
Florian Weimer wrote:
- Michael Ströder:
Florian Weimer wrote:
- Onno van der Straaten:
Ubuntu server upgrade from 12.04 to 14.04 LTS will purge the OpenLDAP directory data.
Is there an Ubuntu bug for this behavior? It's a bit odd.
One should generally be careful with system upgrades.
AFAIK on-disk-format might change because of Berkeley-DB version upgrade.
Unless OpenLDAP uses the HASH database type,
OpenLDAP abandoned the HASH format in OpenLDAP 2.2.
it's sufficient to checkpoint the Berkeley DB environment (and perhaps some other undocumented magic) *before* updating OpenLDAP. The on-disk format of B-Tree-type databases is quite stable.
But the environment and logfile formats have changed quite a bit, and while newer BDB versions will open older B+tree DB files, there's no telling what sub-optimal effects that has on operation.
A clean shutdown will do a checkpoint. You will still need to cleanup the environment and logfiles so that the new BDB version doesn't try to use them. (E.g., using db_recover and db_archive - all covered in the BDB documentation so I won't provide details here.)
None of this is ever guaranteed to work. The only upgrade procedure that is documented and will always work is slapcat/slapadd.
(Automated dump and reload might not work for large databases during automated package upgrades because of the time and disk space overhead.)
openldap-technical@openldap.org
-
Brendan Kearney -
Florian Weimer -
Howard Chu -
Michael Ströder -
Onno van der Straaten -
Robert Heller -
Ryan Tandy