Dear list,
I am newbie in OpenLDAP. Prersently I am running an OpenLDAP as the authentication of Postfix+IMAP email server. How can I take backup and maintain the database of LDAP ? what are the commands ? I like to place the associated commands in a cron jobs. Please enlighten me. thanks
On 7/11/07, JOYDEEP j.bakshi@unlimitedmail.org wrote:
Dear list,
I am newbie in OpenLDAP. Prersently I am running an OpenLDAP as the authentication of Postfix+IMAP email server. How can I take backup and maintain the database of LDAP ? what are the commands ? I like to place the associated commands in a cron jobs. Please enlighten me. thanks
slapcat is the command for you. Also be sure you don't let bdb transaction logs take over your hdd. (see db_archive or options in DB_CONFIG)
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP. You can make a crontab task on midnight, or something like.
A very simple example of shell script.
#!/bin/bash
/etc/init.d/ldap stop
slapcat > /path/to/backup/backup_base.ldif tar -zcf /path/to/backup/backup_ldap.tar.gz /etc/ldap /path/to/base
/etc/init.d/ldap start
exit
Cheers.
On 7/11/07, JOYDEEP j.bakshi@unlimitedmail.org wrote:
Dear list,
I am newbie in OpenLDAP. Prersently I am running an OpenLDAP as the authentication of Postfix+IMAP email server. How can I take backup and maintain the database of LDAP ? what are the commands ? I like to place the associated commands in a cron jobs. Please enlighten me. thanks
On Wed, 11 Jul 2007, Gabriel Stein wrote:
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP. You can make a crontab task on midnight, or something like.
This shouldn't be true in most production configurations (at least, as of 2.3.16ish or so, with bdb/hdb). This is the case with legacy configurations (e.g. ldbm), hence the documentation warning...
Aaron Richton wrote:
On Wed, 11 Jul 2007, Gabriel Stein wrote:
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP. You can make a crontab task on midnight, or something like.
This shouldn't be true in most production configurations (at least, as of 2.3.16ish or so, with bdb/hdb). This is the case with legacy configurations (e.g. ldbm), hence the documentation warning...
Hi Aaron, Gabriel,matthew and others,
thanks a lot for ur kind response. Thanks Gabriel for your script .
Yes Aaron I'm using "bdb" database. I'm little confused here as Gabriel and matthew have suggested for slapcat but u r not in favour for that in case of "bdb" database. So what would be the solution ?
Again slapcat is for backup. Is there any command for maintenance and repair the bdb database ?
thanks to all of you for your kind support.
On Thursday, 12 July 2007, JOYDEEP wrote:
Aaron Richton wrote:
On Wed, 11 Jul 2007, Gabriel Stein wrote:
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP.
I will state here that this is false. In my environment it is never good to stop slapd. Many environments require formal approval to do something like stop slapd (and cron jobs that stop slapd can be career-limiting). All my production environments do automated backups to ldif with slapcat while slapd is running. This includes 1 environment with ~ 1.5 million entries.
While it could be accurate to say "if you don't need your LDAP server ~ 100% available, it can be easier to get a consistent point-in-time backup by running slapcat while slapd is not running", what is better for one environment may not be for another one.
You can make a crontab task on midnight, or something like.
This shouldn't be true in most production configurations (at least, as of 2.3.16ish or so, with bdb/hdb).
While it is good to recommend new versions, IIRC slapcat on bdb has been safe since 2.2.x was marked RELEASE (2.2.7?).
This is the case with legacy configurations (e.g. ldbm), hence the documentation warning...
Hi Aaron, Gabriel,matthew and others,
thanks a lot for ur kind response. Thanks Gabriel for your script .
I would suggest you avoid using a trivial script which has no error checking etc. which may not even work in your environment.
Yes Aaron I'm using "bdb" database. I'm little confused here as Gabriel and matthew have suggested for slapcat but u r not in favour for that in case of "bdb" database. So what would be the solution ?
The best solution for an ldif backup is to use slapcat.
Again slapcat is for backup. Is there any command for maintenance and repair the bdb database ?
With OpenLDAP 2.3, slapd does all maintenance, if you have configured the checkpoint setting, and if you have set the database environment to auto-remove transaction log files. If you do not want transaction log files to be automatically removed, you need to have a cron job to clean them up.
I wrote some scripts for this, which ship in the Mandriva packages (and run daily by default). You need at least ldap-common and ldap-hot-db-backup from:
http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/openldap/current/...
Regards, Buchan
Buchan Milne wrote:
Hi Buchan,
Thanks for your constructive discussion. I'm using OpenLDAP in a production system where Email authentication is based on it. Hence it is not possible to stop the OpenLDAP for backup and repairing. That's why I am very much interested to know the techniques to do a backup and repairing with running OpenLDAP. could you please enlighten me ? I'm using suse 9.3 here thanks
On Thursday, 12 July 2007, JOYDEEP wrote:
Aaron Richton wrote:
On Wed, 11 Jul 2007, Gabriel Stein wrote:
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP.
I will state here that this is false. In my environment it is never good to stop slapd. Many environments require formal approval to do something like stop slapd (and cron jobs that stop slapd can be career-limiting). All my production environments do automated backups to ldif with slapcat while slapd is running. This includes 1 environment with ~ 1.5 million entries.
While it could be accurate to say "if you don't need your LDAP server ~ 100% available, it can be easier to get a consistent point-in-time backup by running slapcat while slapd is not running", what is better for one environment may not be for another one.
You can make a crontab task on midnight, or something like.
This shouldn't be true in most production configurations (at least, as of 2.3.16ish or so, with bdb/hdb).
While it is good to recommend new versions, IIRC slapcat on bdb has been safe since 2.2.x was marked RELEASE (2.2.7?).
This is the case with legacy configurations (e.g. ldbm), hence the documentation warning...
Hi Aaron, Gabriel,matthew and others,
thanks a lot for ur kind response. Thanks Gabriel for your script .
I would suggest you avoid using a trivial script which has no error checking etc. which may not even work in your environment.
Yes Aaron I'm using "bdb" database. I'm little confused here as Gabriel and matthew have suggested for slapcat but u r not in favour for that in case of "bdb" database. So what would be the solution ?
The best solution for an ldif backup is to use slapcat.
Again slapcat is for backup. Is there any command for maintenance and repair the bdb database ?
With OpenLDAP 2.3, slapd does all maintenance, if you have configured the checkpoint setting, and if you have set the database environment to auto-remove transaction log files. If you do not want transaction log files to be automatically removed, you need to have a cron job to clean them up.
I wrote some scripts for this, which ship in the Mandriva packages (and run daily by default). You need at least ldap-common and ldap-hot-db-backup from:
http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/openldap/current/...
Regards, Buchan
On Thu, 12 Jul 2007, JOYDEEP wrote:
Aaron Richton wrote:
On Wed, 11 Jul 2007, Gabriel Stein wrote:
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP. You can make a crontab task on midnight, or something like.
This shouldn't be true in most production configurations (at least, as of 2.3.16ish or so, with bdb/hdb). This is the case with legacy configurations (e.g. ldbm), hence the documentation warning...
Yes Aaron I'm using "bdb" database. I'm little confused here as Gabriel and matthew have suggested for slapcat but u r not in favour for that in case of "bdb" database. So what would be the solution ?
I apologize; my context wasn't entirely clear, and I think Buchan already clarified to my intent. But to be explicit...slapcat is the recommended backup method. Stopping slapd first would not be my recommendation.
slapcat, with a suitable backend (which one should read to be bdb/hdb), has always been designed to be safe to run hot. Historically, there have been occasional {b,h}db deadlock bugs (IMO most notably #4088/fixed 2.3.12) that got in the way of this design. Therefore, to run slapcat hot safely, upgrade to the latest 2.3 series first.
I'd really wish this wouldn't be a statement of anything other than historical curiosity (keep in mind this was fixed in 2005), but experience with vendors and on this list proves otherwise. All the coded bug fixes in the world don't matter if they're not what you're actually executing.
To touch on your other question, there is no regular care or feeding of slapd necessary in 2.3. If you use bdb/hdb, Sleepycat logs should be maintained. This has already been discussed (in this thread, even) so I will not revisit the subject.
Aaron Richton wrote:
Thanks Aaron wish you a nice day !
On Thu, 12 Jul 2007, JOYDEEP wrote:
Aaron Richton wrote:
On Wed, 11 Jul 2007, Gabriel Stein wrote:
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP. You can make a crontab task on midnight, or something like.
This shouldn't be true in most production configurations (at least, as of 2.3.16ish or so, with bdb/hdb). This is the case with legacy configurations (e.g. ldbm), hence the documentation warning...
Yes Aaron I'm using "bdb" database. I'm little confused here as Gabriel and matthew have suggested for slapcat but u r not in favour for that in case of "bdb" database. So what would be the solution ?
I apologize; my context wasn't entirely clear, and I think Buchan already clarified to my intent. But to be explicit...slapcat is the recommended backup method. Stopping slapd first would not be my recommendation.
slapcat, with a suitable backend (which one should read to be bdb/hdb), has always been designed to be safe to run hot. Historically, there have been occasional {b,h}db deadlock bugs (IMO most notably #4088/fixed 2.3.12) that got in the way of this design. Therefore, to run slapcat hot safely, upgrade to the latest 2.3 series first.
I'd really wish this wouldn't be a statement of anything other than historical curiosity (keep in mind this was fixed in 2005), but experience with vendors and on this list proves otherwise. All the coded bug fixes in the world don't matter if they're not what you're actually executing.
To touch on your other question, there is no regular care or feeding of slapd necessary in 2.3. If you use bdb/hdb, Sleepycat logs should be maintained. This has already been discussed (in this thread, even) so I will not revisit the subject.
Quoting Gabriel Stein gabrielstein@gmail.com:
Hi Joy!
I suggest you to use slapcat, but its better stop the OpenLDAP. You can make a crontab task on midnight, or something like.
What if I have a webmail service, that say, offers something like Yahoo!. I would like to have my webmail to be online 24x7 . Is/are there alternatives to shutting down openLDAP to that backup?
-- roger
--------------------------------------------------- Sign Up for free Email at http://ureg.home.net.my/ ---------------------------------------------------
As noted by others, you can use slapcat or ldapsearch to perform nightly backups. If using ldapsearch, be sure your ACLs allow a full dump, including userPassword.
Also, be sure to put into place log monitoring for slapd (and possibly slurpd if you are replicating) to watch for errors. We also like to do routine and automated LDAP service checks via tools such as Big Brother.
How long have you been using OpenLDAP anyway?
-- Puryear IT, LLC Identity Management, Directory Services, Systems Integration Baton Rouge, LA * 225-706-8414 * http://www.puryear-it.com
"Best Practices for Managing Linux and UNIX Servers" http://www.puryear-it.com/pubs/linux-unix-best-practices
JOYDEEP wrote:
Dear list,
I am newbie in OpenLDAP. Prersently I am running an OpenLDAP as the authentication of Postfix+IMAP email server. How can I take backup and maintain the database of LDAP ? what are the commands ? I like to place the associated commands in a cron jobs. Please enlighten me. thanks
That's a FINE question, and there are a million ways to do this. Two quick ideas are:
* Use something like swatch or logwatch against your replication log files and syslog (local4 if I remember correctly) logs.
* Use Big Brother (insert preferred monitoring software <here>), write a simple log parser agent, and then trigger an alert if there is ever unexpected or error output in the log.
I prefer exception-style reporting, so the second option works better for me.
One of the biggest mistakes I see with clients when they are doing replication is not monitoring for a failure in the replication itself. Failures happen, always, and you need to be prepared to catch it. There is nothing worse than failing over to your standby to find out that it has data that is three weeks old. :)
-- Puryear IT, LLC Identity Management, Directory Services, Systems Integration Baton Rouge, LA * 225-706-8414 * http://www.puryear-it.com
"Best Practices for Managing Linux and UNIX Servers" http://www.puryear-it.com/pubs/linux-unix-best-practices
Thierry Lacoste wrote:
Also, be sure to put into place log monitoring for slapd (and possibly slurpd if you are replicating) to watch for errors.
What tool do you suggest for log monitoring?
Regards, Thierry.
On Tuesday 17 July 2007 17:45, Dustin Puryear wrote:
That's a FINE question, and there are a million ways to do this. Two quick ideas are:
- Use something like swatch or logwatch against your replication log
files and syslog (local4 if I remember correctly) logs.
- Use Big Brother (insert preferred monitoring software <here>), write a
simple log parser agent, and then trigger an alert if there is ever unexpected or error output in the log.
I prefer exception-style reporting, so the second option works better for me.
Thanks. Following your advice I found a perl script called slurpd_status.pl (*) that I integrated to my nagios server. It monitors entries in transition, rejected and waiting. I did a few satisfactory tests. What else do you think I should do concerning replication? I know slurpd is deprecated and I plan to move to syncrepl ASAP.
I'm using logcheck (from the FreeBSD ports) to monitor my system logs. AFAICS it is similar to logwatch and can certainly be used to monitor slapd's logs. What unexpected or error output do you think should raise alarms? What loglevel do you recommand for slapd?
Regards, Thierry.
Dustin Puryear wrote:
As noted by others, you can use slapcat or ldapsearch to perform nightly backups. If using ldapsearch, be sure your ACLs allow a full dump, including userPassword.
Also, be sure to put into place log monitoring for slapd (and possibly slurpd if you are replicating) to watch for errors. We also like to do routine and automated LDAP service checks via tools such as Big Brother.
How long have you been using OpenLDAP anyway?
Hi Dustin,
thanks for your response. I am just a newbie in openLDAP technology. I need this for egroupware server which is based on OpenLDAP for user authentication, storage of user and group information and for LDAP based addressbook. Hence I am looking for a solution which can allow me to do maintenance and reparing of OpenLDAP bdb database.
-- Puryear IT, LLC Identity Management, Directory Services, Systems Integration Baton Rouge, LA * 225-706-8414 * http://www.puryear-it.com
"Best Practices for Managing Linux and UNIX Servers" http://www.puryear-it.com/pubs/linux-unix-best-practices
JOYDEEP wrote:
Dear list,
I am newbie in OpenLDAP. Prersently I am running an OpenLDAP as the authentication of Postfix+IMAP email server. How can I take backup and maintain the database of LDAP ? what are the commands ? I like to place the associated commands in a cron jobs. Please enlighten me. thanks
openldap-software@openldap.org
-
Aaron Richton -
Buchan Milne -
Dustin Puryear -
Gabriel Stein -
JOYDEEP -
matthew sporleder -
Sn!per -
Thierry Lacoste