What to Back Up?

Your backup strategy needs to focus on what you want to back up, or in some cases, what not to back up.

As you can see in Table 5-3, some parts of the OS hard drive are important to back up, others are less important to backup on a regular basis, while still other directories don't need backup at all.

If you have any valuable system configuration data, be sure to back up /etc/. This is the most common area for system-wide configuration data. Don't forget it.

Other directories you don't really need to back up unless you're doing a full backup. These include /opt/ (for third party and FHS based applications), /usr/local, /bin/, /boot/, /dev/, and the other directories in the last section of Table 5-3.

For some directories you just do not want to back up, ever; of these, the two big ones that come to mind are /proc/ and /mnt/./proc/ is a special directory that does nothing more than give the kernel a place to mount itself so that user space apps can "talk" to it, look at, and in some cases change running kernel parameters on the fly. If you boot from a Linux boot floppy and take a look in your system's /proc/ directory on the hard drive, there's nothing in there. And if you try to back it up, you may run into problems.

The /mnt/ directory is kind of special. While the subdirectory names in this directory can be important, most of the important (system-wide ones) should be referenced in the /etc/fstab file and should be able to be restored fairly easily.

Backing up the Entire Operating System

Why wouldn't you want to backup the whole OS on a server? The UNIX-/Linux-based file systems are clearly divided by function. So when doing a recovery, many people assume that they'll be doing an OS reinstall anyway-so they don't bother backing up the parts of the OS that never change after an install is complete. This stance varies depending on what you define as critical for your environment. If you lose the main and only drive in a server, do you think you'll just be getting a new drive, reinstalling, reconfiguring fresh, and then restoring the data? Or do you want to restore everything (including the OS) from tape? Most administrators don't bother with OS backups of their servers. They get the users, application configurations, and just apply those to a fresh OS install.

When might you want to backup the entire OS? If you have a highly specialized OS nonserver or workstation install with many customized user application and OS settings (such as a developer workstation or remote desktop terminal server) you probably want to get the whole OS and all of its installed and configured apps.

When might you want not to backup everything? A likely example would be on a single-purpose server such as a user home-dir fileserver, vanilla web server, BIND/DNS server, or the like. These types of application configurations can usually be quickly brought back up by an OS reinstall, then using up2date (for RPM systems) or apt-get (for Debian systems) to bring the system up to a fully patched state, and finally putting the backed up data back in place from tape.

For example, after a reinstall a BIND DNS server would simply need its DNS zone files put back into place in either /var/named/ or /var/named/chroot/var/named/ (for chroot'd systems) along with the main named config file /etc/named.conf. Restart the named service and you're back up and running. It's kind of a waste to back up the rest of the OS (several GB) every week when it is not really warranted.

Most home/desktop systems only require backup of user data in /home/ and /etc/ (configuration data), and possible /var/ if you store e-mail, web, and/or ftp data there. In such cases only /home/ should probably get backed up, since home Linux users tend to do a lot of OS switching anyway. In fact, one very convenient thing I have found is to just keep your /home/ files ystem on its own dedicated drive. This way when you go to reinstall a new Linux distribution on your system, you just unplug the /home/ drive, install the OS on your main drive, and then plug in and mount the old /home/ drive back into place. Be sure you get the new /home/username/ directories' content copied over into the old home drive. Many desktop apps and environments need their newer config directories in place and will balk if you try to forcefeed them the older counterpart's config directories and settings.

Other Important System Config Files

Even if you do a total system backup, there are some important configurational aspects of your system that you should backup, such as your hard drive's partition layout, your important /etc/ configuration files, and so on.

To make your Linux system back up its partition table(s) to a flat text file, you can use the following commands:

# mkdir /etc/config-data ; chmod 700 /etc/config-data ;
  chown root.root/etc/ config-data
# fdisk -1/dev/hda > /etc/config-data/hda-fdisk.txt
# dd if=/dev/hda of=/etc/config-data/hda-part-boot.img bs=512 count=1
1+0 records in
1+0 records out
# fdisk -1 /dev/hdc > /etc/config-data/hdc-fdisk.txt
# dd if=/dev/hdc of=/etc/config-data/hdc-part-boot.img bs=512 count=1 1+0 records in
1+0 records out

The first command sets up a /etc/config-data/ directory and root-restricted permissions. The next fdisk line grabs the human-readable partition table config from drive /dev/hda (IDE primary-master) and dumps it out to the flat text file /etc/config-data/hda-fdisk.txt. The third line then backs up the actual partition table and MBR out to the file /etc/config-data/ hda-part-boot.img. Likewise, the last two command lines do the same for the IDE secondary-master or hdc.

Doing this will ensure that you have backups of your drive partition layouts as well as your actual master boot record (MBR), which can be used in case someone ever corrupts and renders your system unbootable. And since you always backup /etc/ to tape, you should have these archived safely.

Backing up Databases

Backing up a database (or any other file that is in a state of change) is like photocopying a sheet of paper. If you wiggle it while copying it, then you get a corrupted copy. If you want a good backup of a database (or other such "wiggling" file), you need to choose one of the following options:

• Backup directly with a backup agent-This is a special piece of backup software (usually added on in the form of a commercial backup module) that "talks" to both the database and to the backup software. The backup software starts copying the database and when the agent senses a change in the database, it finds out what that change was and communicates it back to the backup suite and so the uncorrupted change can be accounted for in the final backup. The database backup agents are not always available for all database systems or backup suites or may cost several hundred dollars, plus their use varies from vendor to vendor.

• Stop, copy, and back up-You stop the database or service, quickly copy the related file(s) somewhere else, and then restart the service and get your backup. This technique is appropriate for backing up files like mySQL/database,sendmail, and syslog.

• To use this method, see the Shutting off services section later in this chapter. Shutting off services is great for getting everything related to that service on tape. However, if you're on a production server, you may not be able to shut off mission-critical services such as web and databases.

• Freeze, copy, and back up-This method is generally more difficult, but you can usually lock your databases using simple script-driven programs like mysqldump or mysqlhotcopy (more on these later). This method uses the same concept as the previous technique, but you don't stop the services; you just use these tools to "freeze" them (read-only) while you quickly copy them out, and then unfreeze them when you're done with your copy. See the rest of this section for more information on how this is done with MySQL.

The Backup Agent method may not always be available to you, depending on what type of database and backup suite you're using. Some Linux databases are starting to get third-party commercial backup agents that will talk to the database and the backup system to effectively back up a live, or hot, database. (A MySQL backup agent for ARCserve is available at www3.ca.com/Solutions/ProductOption.asp?ID=4691.) However, MySQL comes with a couple of utilities to help you use the Freeze and Copy method.

For the Freeze, Copy, and Backup database backup method, there are various options depending on what database you are using. Because MySQL is the most popular open source database, let's take a look at that. MySQL is good in that there are a few tools with which to get the job done. The first, older program is called mysqldump. It does not get binary log files (required in some cases), and it dumps to flat .sq1 files (which can take up more room than the actual data base files). If neither of these issues bothers you, then this mini-app might be just what the doctor ordered. mysqldump's general shell usage looks like this:

# mysqldump --opt --all-databases --user=root >
  /root/all-databases_$( date +%Y-%m-%d).sql

This will freeze and copy all of the databases out to a time-stamped flat file under /root/.

In newer MySQL database Freeze and Copy option is another included tool called mysqlhotcopy. This does a table level or all-database read-only lock effectively, clones everything in the mysql data directory (/var/lib/mysql/for most RPM-based systems) out to a directory of your liking, and then unlocks the databases when done. This is what the shell usage looks like:

# mysqlhotcopy --user=root --allowold --regexp=".*" /root/mysql-backup

Your regular system backups will now get the static database files in /root/mysql-backup/ and you'll be good to go.

mysqlhotcopy is great if you're running MyISAM storage engine table types. But if you're running innodb table tapes, you really do need to shut down the whole service and do a cp-a /var/lib/mysql/*/root/mysql-backup/, otherwise you will not get your innodb database content. If you're not sure which engine/table type you're running, then you're probably running the default MyISAM type and you have nothing to worry about.

For more info, do a mysqlhotcopy--h or perldoc/usr/bin/mysqlhotcopy.

Setting up Tape Drive Devices

Commands from dump, restore, mt, and tar will all want to see the device file called /dev/ tape in place to backup and restore from. For example, mt rewind and mt-f /dev/st0 rewind should do the same thing if you have a symlink set up from /dev/tape to /dev/st0. But with some of these commands, if no device file is given /dev/tape is the default, so you should have this link set up. If you don't and you are using a SCSI tape drive (/dev/st0) set that up now, using the following commands:

# ln -s /dev/st0 /dev/tape
# ls -la /dev/tape lrwxrwxrwx lroot root 8 Mar 14 15:17 /dev/tape->/dev/st0

Shutting Off Services

As previously mentioned, you don't want to back up a system with a bunch of actively changing files on it. You get around this by shutting down these services or using the Stop and Copy method.

One of the most orderly ways of doing this is via runlevel 4. In most Linux distributions, runlevel 3 is the command line or headless mode in which you might run a web or e-mail server; you don't need a GUI/X11 display (as you would on a desktop). Desktops and workstations, on the other hand, need to run in a default runlevel of 5 (at least for Red Hat-based systems).

# chkconfig --list | grep -e 4:on -e on$

# chkconfig --level 4 syslog off
# chkconfig --level 4 sendmail off
# chkconfig --level 4 xinetd off
# chkconfig --level 4 mysqld off
...

#!/bin/bash

## Do a MYSQL-HOT-COPY to /root/mysql-bak
mysqlhotcopy --user=root --allowold --regexp=".*" /root/mysql-backup

## Rewind tape
mt rewind

## Change Run levels to 4, backup, back to 3
init 4
sleep 10
tar -czf /dev/tape /bin /boot /dev /etc /home /lib /lost+found
  /opt /root /sbin /tmp /usr /var > /dev/null 2>&1
echo -Backups complete...
## Eject tape...
mt eject
sleep 5
init 3
echo -Done