Hi
I want to migrate our openldap db backend which currently using idbm to bdb, Matt have suggested following steps, would you please kind enough to put some more light on few steps
<snip> Migrate to BDB or HDB. Here's the rough idea: shutdown slapd slapcat -f /path/to/slapd.conf -l mydb.ldif vi /path/to/slapd.conf Change your database to a new directory. Read about tuning, cache sizing, and other stuff. Read openldap man pages. ( what to read ;) Read oracle's tuning docs for bdb. (we are not using any oracle things) Re-Read openldap man pages. (re-read for what :-S :wq slapadd -f /path/to/slapd.conf -l mydb.ldif
Fix permissions on /var/run so the slapd user can write there -- this one is pretty easy </snip>
Secondly, to be safe side, if something goes wrong, taking backup of ...
directory /var/lib/ldap/mydomain.com
cp -r /var/lib/ldap/mydomain.com /TosomeSafePlace
is enough to fallback to ibdm?
I want minimum downtime, heh frankly I can't aford my ass on fire :P
Thanks. Askar
<quote who="Asrai khn">
Hi
I want to migrate our openldap db backend which currently using idbm to bdb, Matt have suggested following steps, would you please kind enough to put some more light on few steps
<snip> Migrate to BDB or HDB. Here's the rough idea: shutdown slapd slapcat -f /path/to/slapd.conf -l mydb.ldif vi /path/to/slapd.conf Change your database to a new directory. Read about tuning, cache sizing, and other stuff. Read openldap man pages. ( what to read ;) Read oracle's tuning docs for bdb. (we are not using any oracle things) Re-Read openldap man pages. (re-read for what :-S :wq slapadd -f /path/to/slapd.conf -l mydb.ldif
Fix permissions on /var/run so the slapd user can write there -- this one is pretty easy
</snip>
Secondly, to be safe side, if something goes wrong, taking backup of ...
directory /var/lib/ldap/mydomain.com
cp -r /var/lib/ldap/mydomain.com /TosomeSafePlace
is enough to fallback to ibdm?
Above looks about right. Just keep your slapcat export safe, then you can fall back to any backend, if need be.
I want minimum downtime, heh frankly I can't aford my ass on fire :P
Hire someone with experience then! ;-)
Thanks. Askar
Asrai khn schrieb:
Hi
I want to migrate our openldap db backend which currently using idbm to bdb, Matt have suggested following steps, would you please kind enough to put some more light on few steps
<snip> Migrate to BDB or HDB. Here's the rough idea: shutdown slapd slapcat -f /path/to/slapd.conf -l mydb.ldif vi /path/to/slapd.conf Change your database to a new directory. Read about tuning, cache sizing, and other stuff. Read openldap man pages. ( what to read ;) Read oracle's tuning docs for bdb. (we are not using any oracle things) Re-Read openldap man pages. (re-read for what :-S :wq slapadd -f /path/to/slapd.conf -l mydb.ldif
Fix permissions on /var/run so the slapd user can write there -- this one is pretty easy
</snip>
Secondly, to be safe side, if something goes wrong, taking backup of ...
directory /var/lib/ldap/mydomain.com
cp -r /var/lib/ldap/mydomain.com /TosomeSafePlace
is enough to fallback to ibdm?
No, when you're changing from ldbm to bdb you have to change slapd.conf accordingly. So at least one additional step would be copying back your old config...
I want minimum downtime, heh frankly I can't aford my ass on fire :P
1. Well, it's your job ;) 2. Make a test setup and test the procedure! 3. Repeat step 2 until you have understood what's happening and what could go wrong including a working recovery plan. 4. Migrate the production setup with a defined desaster recovery plan.
Hmm, thats pretty basic advice and applies to a lot of scenarios, but in my experience there is no way around getting your hands dirty. Noone can *tell* you, so I suggest you try the steps Matt has posted and come back when it fails (in you test setup of course).
cheers Paul
Hi
Migrate to BDB or HDB. Here's the rough idea: shutdown slapd slapcat -f /path/to/slapd.conf -l mydb.ldif vi /path/to/slapd.conf Change your database to a new directory. Read about tuning, cache sizing, and other stuff. Read openldap man pages. ( what to read ;) Read oracle's tuning docs for bdb. (we are not using any oracle things) Re-Read openldap man pages. (re-read for what :-S :wq slapadd -f /path/to/slapd.conf -l mydb.ldif
Fix permissions on /var/run so the slapd user can write there -- this one is pretty easy
</snip>
I have successfully migrated ldap to use back-bdb, thanks Matt i followed the steps which you have suggested and it worked :)
However, at first run slapd starting without giving any error or warning but when i stop/start it using 'service ldap restart' it gives me weird warnings/errors....
Checking configuration files for slapd: bdb_db_open: unclean shutdown detected; attempting recovery. bdb_db_open: Recovery skipped in read-only mode. Run manual recovery if errors are encountered. bdb(dc=awpdc,dc=com): PANIC: fatal region error detected; run recovery bdb_db_open: Database cannot be opened, err -30974. Restore from backup! bdb(dc=awpdc,dc=com): DB_ENV->lock_id_free interface requires an environment configured for the locking subsystem backend_startup_one: bi_db_open failed! (-30974) slap_startup failed (test would succeed using the -u switch)
Running 'db_recover' solved this problem, now slapd stopping/starting working fine without any warning or errors.
Thanks guys I greatly appreciate all your help and support.
Regards, Askar
Asrai khn wrote, on 25. feb 2007 17:03:
[...]
Running 'db_recover' solved this problem, now slapd stopping/starting working fine without any warning or errors.
Thanks guys I greatly appreciate all your help and support.
While you're at it. why not upgrade your OpenLDAP version to 2.3 (2.3.34 is the latest version, though 2.3.32 is classed as "stable"). This will do your db_recover automatically, though you shouldn't be getting this problem with a properly built db and up-to-date bdb libraries. 2.3.32/4 has a number (like tons) of significant improvements over previous minors.
--Tonni
On 2/26/07, Tony Earnshaw tonni@hetnet.nl wrote:
While you're at it. why not upgrade your OpenLDAP version to 2.3 (2.3.34 is the latest version, though 2.3.32 is classed as "stable"). This will do your db_recover automatically, though you shouldn't be getting this problem with a properly built db and up-to-date bdb libraries. 2.3.32/4 has a number (like tons) of significant improvements over previous minors.
Hi Tony,
Thanks, soon we are upgrading this machine to FC6 which come with verion 2.3.27-4 of openldap. The current version of openldap on FC5 is 2.3.19-4
Thanks. Askar
Asrai khn wrote, on 26. feb 2007 10:51:
[...]
Thanks, soon we are upgrading this machine to FC6 which come with verion 2.3.27-4 of openldap. The current version of openldap on FC5 is 2.3.19-4
[...]
"Running 'db_recover' solved this problem, now slapd stopping/starting working fine without any warning or errors."
[...]
You should not need to run: "Running 'db_recover' solved this problem, now slapd stopping/starting working fine without any warning or errors" with *any* 2.3 version. With OpenLDAP 2.3 db_recover is automatic.
Either you or not running any OpenLDAP 2.3 version, or your vendor is conning you.
Best,
--Tonni
--On Monday, February 26, 2007 4:55 PM +0100 Tony Earnshaw tonni@hetnet.nl wrote:
Asrai khn wrote, on 26. feb 2007 10:51:
[...]
Thanks, soon we are upgrading this machine to FC6 which come with verion 2.3.27-4 of openldap. The current version of openldap on FC5 is 2.3.19-4
[...]
"Running 'db_recover' solved this problem, now slapd stopping/starting working fine without any warning or errors."
[...]
You should not need to run: "Running 'db_recover' solved this problem, now slapd stopping/starting working fine without any warning or errors" with *any* 2.3 version. With OpenLDAP 2.3 db_recover is automatic.
Either you or not running any OpenLDAP 2.3 version, or your vendor is conning you.
Using vendor packages to run an OpenLDAP server is nearly always a bad idea (Mandriva's a definite exception). There is almost nothing that motivates the vendor to use current releases, or backport stability fixes. There are known problems with the connection code in OL 2.3.27, for example, that were fixed in OpenLDAP 2.3.32. In general, use vendor packages at extreme risk.
--Quanah
-- Quanah Gibson-Mount Principal Software Developer ITS/Shared Application Services Stanford University GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
Quanah Gibson-Mount writes:
Using vendor packages to run an OpenLDAP server is nearly always a bad idea (Mandriva's a definite exception). There is almost nothing that motivates the vendor to use current releases, or backport stability fixes. There are known problems with the connection code in OL 2.3.27, for example, that were fixed in OpenLDAP 2.3.32. In general, use vendor packages at extreme risk.
Is there something particular which makes this more so for OpenLDAP than other packages, or are OpenLDAP releases more buggy than other packages, or are existing bugs more likely to be fatal, or...?
I had the impression that this was mostly a RedHat issue. But if it's more general, it sounds like the only likely fix would be in OpenLDAP or the release methods or something.
--On Monday, February 26, 2007 8:19 PM +0100 Hallvard B Furuseth h.b.furuseth@usit.uio.no wrote:
Quanah Gibson-Mount writes:
Using vendor packages to run an OpenLDAP server is nearly always a bad idea (Mandriva's a definite exception). There is almost nothing that motivates the vendor to use current releases, or backport stability fixes. There are known problems with the connection code in OL 2.3.27, for example, that were fixed in OpenLDAP 2.3.32. In general, use vendor packages at extreme risk.
Is there something particular which makes this more so for OpenLDAP than other packages, or are OpenLDAP releases more buggy than other packages, or are existing bugs more likely to be fatal, or...?
I had the impression that this was mostly a RedHat issue. But if it's more general, it sounds like the only likely fix would be in OpenLDAP or the release methods or something.
The distro operators are generally interested in one thing -- The LDAP C api, and the libraries provided by OpenLDAP. Using the software to actually run a slapd server is not something they particularly worry about. The only bugs they tend to backport are those ones involving a security vulnerability. I've tried for a few months now to get the Debian folks to pick up the connection code patch from 2.3.32 and apply it to their distribution. I'm still hopeful that they'll do it, but I removed myself from their packaging list after it became clear that creating a quality package in support of upstream was not anything they were interested in.
--Quanah
-- Quanah Gibson-Mount Principal Software Developer ITS/Shared Application Services Stanford University GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html
Quanah Gibson-Mount writes:
The distro operators are generally interested in one thing -- The LDAP C api, and the libraries provided by OpenLDAP. Using the software to actually run a slapd server is not something they particularly worry about. The only bugs they tend to backport are those ones involving a security vulnerability. (...)
Might help for that problem to make --disable-slapd the default for configure (and ask "slow" vendors to keep it that way). Then we instead get problems with users who have both preinstalled LDAP in /usr/ and self-compiled LDAP elsewhere... Or support linking slapd with the installed libraries as well as with new-built ones, if new server + old library tends to be better than old server + equally old library.
On 2/26/07, Hallvard B Furuseth h.b.furuseth@usit.uio.no wrote:
Quanah Gibson-Mount writes:
Using vendor packages to run an OpenLDAP server is nearly always a bad idea (Mandriva's a definite exception). There is almost nothing that motivates the vendor to use current releases, or backport stability fixes. There are known problems with the connection code in OL 2.3.27, for example, that were fixed in OpenLDAP 2.3.32. In general, use vendor packages at extreme risk.
Is there something particular which makes this more so for OpenLDAP than other packages, or are OpenLDAP releases more buggy than other packages, or are existing bugs more likely to be fatal, or...?
I had the impression that this was mostly a RedHat issue. But if it's more general, it sounds like the only likely fix would be in OpenLDAP or the release methods or something.
It's more a general problem with the decoupled nature of linux distros. OpenLDAP is simply mentioned a lot on this list. :)
matthew sporleder writes:
On 2/26/07, Hallvard B Furuseth h.b.furuseth@usit.uio.no wrote:
Quanah Gibson-Mount writes:
Using vendor packages to run an OpenLDAP server is nearly always a bad idea (Mandriva's a definite exception). There is almost nothing that motivates the vendor to use current releases, or backport stability fixes. There are known problems with the connection code in OL 2.3.27, for example, that were fixed in OpenLDAP 2.3.32. In general, use vendor packages at extreme risk.
Is there something particular which makes this more so for OpenLDAP than other packages, or (...) I had the impression that this was mostly a RedHat issue. (...)
It's more a general problem with the decoupled nature of linux distros. OpenLDAP is simply mentioned a lot on this list. :)
Ouch. Does that mean the way to have a solid system is to check the age of most of its major packages and compile newer ones myself?
Hallvard B Furuseth wrote:
matthew sporleder writes:
On 2/26/07, Hallvard B Furuseth h.b.furuseth@usit.uio.no wrote:
Quanah Gibson-Mount writes:
Using vendor packages to run an OpenLDAP server is nearly always a bad idea (Mandriva's a definite exception). There is almost nothing that motivates the vendor to use current releases, or backport stability fixes. There are known problems with the connection code in OL 2.3.27, for example, that were fixed in OpenLDAP 2.3.32. In general, use vendor packages at extreme risk.
Is there something particular which makes this more so for OpenLDAP than other packages, or (...) I had the impression that this was mostly a RedHat issue. (...)
It's more a general problem with the decoupled nature of linux distros. OpenLDAP is simply mentioned a lot on this list. :)
Ouch. Does that mean the way to have a solid system is to check the age of most of its major packages and compile newer ones myself?
Unfortunately, I think this reflects some immaturity of the software. We're still fighting with even the basic definition of the C API. There are other packages in the same situation: I just installed OpenSUSE 10.2 on my laptop yesterday, which is the latest and greatest version there, but it still shipped with old wireless lan drivers, that did not work on my laptop. I had to go download current drivers and firmware and build them manually before I had something usable.
The problems are exacerbated by the clumsy design of the pam_ldap/nss_ldap mechanisms, which cause system-level functionality to pollute user-program namespaces. If distros would take the proper steps to hide the symbols of their dynamically loaded pam/nss modules from user code, then most of the compatibility issues would disappear. (I note that we do this correctly in Symas' CNS packages. Hint, hint.)
I have a feeling that not enough people complain to their distro vendors about these problems, and so the distros continue to pretend the problems don't exist.
Howard Chu hyc@symas.com wrote: Hallvard B Furuseth wrote:
matthew sporleder writes:
On 2/26/07, Hallvard B Furuseth wrote:
Quanah Gibson-Mount writes:
Using vendor packages to run an OpenLDAP server is nearly always a bad idea (Mandriva's a definite exception). There is almost nothing that motivates the vendor to use current releases, or backport stability fixes. There are known problems with the connection code in OL 2.3.27, for example, that were fixed in OpenLDAP 2.3.32. In general, use vendor packages at extreme risk.
Is there something particular which makes this more so for OpenLDAP than other packages, or (...) I had the impression that this was mostly a RedHat issue. (...)
It's more a general problem with the decoupled nature of linux distros. OpenLDAP is simply mentioned a lot on this list. :)
Ouch. Does that mean the way to have a solid system is to check the age of most of its major packages and compile newer ones myself?
Unfortunately, I think this reflects some immaturity of the software. We're still fighting with even the basic definition of the C API. There are other packages in the same situation: I just installed OpenSUSE 10.2 on my laptop yesterday, which is the latest and greatest version there, but it still shipped with old wireless lan drivers, that did not work on my laptop. I had to go download current drivers and firmware and build them manually before I had something usable.
The problems are exacerbated by the clumsy design of the pam_ldap/nss_ldap mechanisms, which cause system-level functionality to pollute user-program namespaces. If distros would take the proper steps to hide the symbols of their dynamically loaded pam/nss modules from user code, then most of the compatibility issues would disappear. (I note that we do this correctly in Symas' CNS packages. Hint, hint.)
I have a feeling that not enough people complain to their distro vendors about these problems, and so the distros continue to pretend the problems don't exist.
Howard,
You mentioned that Symas is correctly hiding the symbols when loading in pam/nss libraries. How are you doing this? Are you using symbol versioning, dlopen flags other than RLTD_LOCAL, something else (or combination)? Also, is this being accomplished on Solaris as well? I'm fighting this same issue and haven't found a good way to accomplish this yet (with the Solaris linker).
Darin Broady
For Solaris, Sun has developer's documentation and a sample module at http://www.sun.com/software/solaris/pam/ which are linked correctly. Note the "mapfile".
Hallvard B Furuseth wrote:
matthew sporleder writes:
It's more a general problem with the decoupled nature of linux distros. OpenLDAP is simply mentioned a lot on this list. :)
Ouch. Does that mean the way to have a solid system is to check the age of most of its major packages and compile newer ones myself?
Yes.
Ciao, Michael.
Hallvard B Furuseth wrote, on 26. feb 2007 20:19:
[...]
Is there something particular which makes this more so for OpenLDAP than other packages, or are OpenLDAP releases more buggy than other packages, or are existing bugs more likely to be fatal, or...?
I had the impression that this was mostly a RedHat issue. But if it's more general, it sounds like the only likely fix would be in OpenLDAP or the release methods or something.
I guess I was born cynical; Red Hat always (through many years) had a lousy record of supporting OpenLDAP, whilst its support of stuff like Apache and MySQL has always been impeccable. Not only these packages, but the OS as a whole has been supported and maintained (with or without back porting) in a manner that I as a SysV and Red Hat aficionado would find it difficult (but not impossible) to leave for any competitor. One asks oneself what, for example, Centos will do in the circumstances.
Lately Red Hat has adopted (bought) it's own directory services from Netscape/Sun. Looking at Red Hat's service support conditions (available for anyone on the net), I see that Red Hat (nothing like repeating the name for cognizance) wishes to charge around $17,000 per site for support of its own directory services. I don't see any motive for continued Red Hat OpenLDAP support there. Red Hat's pecuniary philosophy is fast becoming a superset of Microsoft's.
My own experience over the last three years has been, that (latterly with Buchan Milne's continually updated Red Hat specs/srpms), I can probably do a far better job of supporting directory services using OpenLDAP on Red Hat and Fedora bases than Red Hat can using its own stuff; certainly for far less money. OpenLDAP is a crucial component of all Red Hat sites I have anything to do with.
--Tonni
Tony Earnshaw wrote:
Hallvard B Furuseth wrote, on 26. feb 2007 20:19:
[...]
Is there something particular which makes this more so for OpenLDAP than other packages, or are OpenLDAP releases more buggy than other packages, or are existing bugs more likely to be fatal, or...?
I had the impression that this was mostly a RedHat issue. But if it's more general, it sounds like the only likely fix would be in OpenLDAP or the release methods or something.
I guess I was born cynical; Red Hat always (through many years) had a lousy record of supporting OpenLDAP, whilst its support of stuff like Apache and MySQL has always been impeccable. Not only these packages, but the OS as a whole has been supported and maintained (with or without back porting) in a manner that I as a SysV and Red Hat aficionado would find it difficult (but not impossible) to leave for any competitor. One asks oneself what, for example, Centos will do in the circumstances.
Lately Red Hat has adopted (bought) it's own directory services from Netscape/Sun. Looking at Red Hat's service support conditions (available for anyone on the net), I see that Red Hat (nothing like repeating the name for cognizance) wishes to charge around $17,000 per site for support of its own directory services.
Or you can use Fedora Directory Server for free. Also note that the price is _per server_, not per entry or per user like many of the other large commercial LDAP server vendors. So if you have a large number of entries, it can be a good deal.
I don't see any motive for continued Red Hat OpenLDAP support there.
Red Hat has a large number of customers running the OpenLDAP server provided as part of the base OS, and these customers are supported, and will be for the lifetime of their support contracts with Red Hat. I won't get into the entire history, but there have been many customers who adamantly refused to upgrade OpenLDAP because it was "just working" (for some value of "working") and did not want to "fix something that wasn't broken". Of course this is a gross oversimplification but should convey the general idea. And yes, I've heard a thousand times that "yes, it is broken, they just don't know it - yet" (and so have the aforementioned Red Hat customers, to no avail).
Red Hat's pecuniary philosophy is fast becoming a superset of Microsoft's.
Hardly. But you're welcome to your own opinion. Red Hat spent a considerable amount of money to purchase the Netscape server products, open source them, and make them available _for free_ as Fedora Directory Server. A lot of the work to open source the code, replace proprietary components with open source alternatives, make the build process more friendly to the open source community, package-ify, etc. was done solely for the purpose of creating a viable open source project, and at the expense of adding other features to the product to attract more paying customers.
Does Microsoft's "pecuniary philosophy" include purchasing large, well established proprietary products, open sourcing them, and making them available for free?
Is Red Hat entitled to a fair return on its investment?
My own experience over the last three years has been, that (latterly with Buchan Milne's continually updated Red Hat specs/srpms), I can probably do a far better job of supporting directory services using OpenLDAP on Red Hat and Fedora bases than Red Hat can using its own stuff; certainly for far less money. OpenLDAP is a crucial component of all Red Hat sites I have anything to do with.
That's great. If it works for you, more power to you.
--Tonni
You should not need to run: "Running 'db_recover' solved this problem, now slapd stopping/starting working fine without any warning or errors" with *any* 2.3 version. With OpenLDAP 2.3 db_recover is automatic.
Sorry my bad I thinks db_recover is now automatically running, I didn't noticed it coz i got my hand dirty with master :)
Thanks. Askar
openldap-software@openldap.org