Team LiB
Previous Section Next Section

Verifying Your Files with RPM

In order for any file alteration tracking method to work, you must have a baseline snapshot of your system so that you can compare the potentially compromised files to the intact versions. In this section, we show you how to do host-based file integrity checks with RPM. Later in this chapter, we'll show you how to use it to create a system-wide security baseline, and how to automate file alteration scanning, using RPM so that it becomes a regular part of your security routine.

Note 

If you want to use RPM's file alteration tracking capabilities as part of a truly secure installation, you should consider moving your system's RPM database and some basic system tools off the hard drive and onto write-protected removable media. These tools include many of the programs in /bin, /sbin, /lib, and /var/lib/rpm, as well as RPM's binaries and some statically compiled tools like /sbin/sash or /bin/ash.static. The truly savvy cracker might be able to compromise RPM itself, making it meaningless as a security tool.

Using RPM for Maintenance

RPM is much like a scientific calculator. While it's got a huge array of functions and features, most people just use it to manage packages and installations, just like most people use scientific calculators to figure percentages and do long division. However, if you poke around RPM a bit, you'll find a number of ways to simplify package maintenance, track files and package versioning, and even utilize methods for bolstering system security.

You probably already use RPM for basic package management. For example, you can use the query mode to track package versions, with the following command

   # rprn -q sendmail
   sendmail-8.12.10-1.1 .1

RPM will show you all the packages on your system that have a given string in their name.

   # rprn -qa | grep mail
   mailx-8.1.1-31.1
   fetchmail-6.2.0-8
   sendmail-cf-8.12.10-1.1.1
   mailman-2.1.2-2
   mailcap-2.1.14-1.1
   sendmaiI-8.12.10-1.1.1
   sendmail-doc-8.12.10-1.1.1
   redhat-switch-mail-0.5.21-1
   mozilla-mail-1.4.1-18
   squirrelmail-1.4.0-1
   procmail-3.22-11

You can even use RPM to figure out what package a deleted or missing file belonged to, so that you can reinstall or repair the damage.

   # rm /bin/ping
   rm: remove regular file '/bin/ping'? Y
   # ping localhost
   -bash: ping: command not found
   # rpm -qf /bin/ping
   iputils-20020927-9.1

By querying the RPM system for the missing file (with -qf), you can see what package the file belonged to, and which package would need to be reinstalled to fix the system. Assuming that you have already downloaded the package to be installed, here's how to use RPM to upgrade (reinstall), verbosely (with hashes), the package iputils.

   # rpm -Uvh iputils-20020927-9.1.i386.rpm
   Preparing...                ################### [100%]
         package iputils-20020927-9.1 is already installed

Because the system thinks that the package is already installed, you may need to force it to reinstall (we'll also do the install from a remote copy of the package located on an FTP server).

   # rpm -Uvh --force ftp://example.com/pub/RPMS/iputils-20020927-
    9.1.i386.rpm
   Retrieving ftp://example.com/pub/RPMS/iputils-20020927-
   9.1.i386.rpm
   Preparing...   ####################################### [100%]
      1:iputils   ####################################### [100%]

The problem should be fixed. Use the following command to check it.

   #  ping -c 1 localhost
   PING localhost.localdomain (127.0.0.1) 56(84) bytes of data.
   64 bytes from localhost.localdomain (127.0.0.1): icmp_seq=0
    ttl=64 time-0.113 ms
   --- localhost.localdomain ping statistics ---
   1 packets transmitted, 1 received, 0% packet loss. time 0ms
   rtt min/avg/max/mdev = 0.113/0.113/0.113/0.000 ms, pipe 2

The example shows the usefulness of this functionality; however, most newer administrators never even use RPM to this level. If this basic functionality is surprising to you, just think about how much else you're missing out on.

How can you use RPM to track changes to critical system files? Let's revisit the example just shown.

   # rm /bin/ping
   rm: remove regular file '/bin/ping'? Y
   # rpm -V iputils
   missing /bin/ping

Here, we've used the --V switch, for verify. This option tells RPM to go through a listing of all of the files that were originally installed by the package being referenced, and compare all the file attributes (size, date, MDSsum, and so on) to those that now exist. Any differences are listed. If nothing is output from the command, then nothing has changed since the package install time.

This verify option can also be used against all packages installed on the system (and all their files), to see what files are missing or changed.

# rpm -Va | grep missing
missing     /usr/src/linux-2.4.22-1.2129.nptl/arch/i386/math-emu/.depend
missing     /bin/ping

RPM compares the existing file listings to the RPM database of files that were installed and should be present. Anything that's different is listed. In the example, we're missing a file called .depend, as well as our deleted ping.

Note 

Always send the output of such RPM verifies into grep to filter it down. Otherwise, the output will flood your console, since many files on the system get modified in one way or another after they're installed (for example, config files, user files, logs, and the like). You usually want to use grep like this to return only the things that you're interested in.

The RPM database is rebuilt each time a new package is installed. Every database entry contains a list of the files that should be included in each package and what these files should look like. Very useful.

Caution 

Using the -V or verify option is disk-and processor-intensive. Only run this command on a machine under light load, or run it during nonprime times. For example, the rpm -Va|grepmissing command shown above took nearly 15 minutes to run on a 1.6 GHz system, and drove the system load over 1.2 at times. This is not a tool for the heavily used production system.

Using RPM to Check Security

Yes, knowing whether a file is missing is helpful. What if a file still exists, but has changed? If a cracker has gotten into your system and replaced a critical system file with a version that includes a trojan or back door in place, you need to know about it. Use the verify all option again, but this time, limit the output to files that have the string bin/ in their path, like this.

   # rpm -Va | grep 'bin\/'
   .M......  /usr/bin/smbmnt
   S.5....T  /bin/netstat
   S.5....T  /bin/Is
   S.5....T  /usr/bin/passwd
   missing   /bin/ping
   S.?....T  /bin/login
   S.5....T  /bin/ps

This output shows that someone has modified several of the system files with bin/ in their path. The 5 mnemonic indicates that the MD5sum, or fingerprint, of the file has changed. A question mark shows that RPM could not calculate the MD5sum of the file for some reason. If you have one or more critical system files with a modified fingerprint, it's likely that you've been cracked.

Tip 

Commonly attacked files on Linux systems to watch out for include /bin/ls, /usr/bin/passwd, /bin/login, /bin/netstat, /bin/ps., /sbin/ifconflg, and /usr/bin/chattr, just to name a few.

Caution 

Once a system has been compromised, you can no longer trust anything, not even unmodified files. The only solution is to get the system off-line, get all nonexecutable content off the system, and reformat and reinstall fresh.

RPM stores a number of attributes about every file on the system for which it's responsible. These attributes are represented by the following mnemonics:

  • S: File size

  • M: Modes (includes permissions and file type)

  • 5: MD5sum

  • D: Device major/minor numbers

  • L: readLink(2) paths

  • U: User ownership

  • G: Group ownership

  • T: Timestamp

You can use RPM to check for differences between the install time and the existing file system using any of these attributes.

Caution 

If you're going to use RPM as a security tool, commit to it. Avoid mixing RPM package management with other forms, such as pkg, tarball, source code files, or converted Debian packages. Everything needs to be installed through RPM so that you can count on the RPM database when you need accurate information.

Once you've checked the existing files against the RPM's MD5sum information in it's database, you may need to look at the files a bit more closely. A compromised system sometimes includes other tracks or evidence. Collecting such information is referred to as intrusion forensics.

Other evidence you may detect on a compromised system may include special file attributes that shouldn't be on a normal Linux system, hidden within the ext2 and ext3 file system itself. One of these of these cracker-utilized file system attributes is the immutable control bit. On a compromised system, you might see settings in the /bin directory, from the output of the list attribute or lsattr command, that look like this.

   # lsattr/bin/* /sbin/* /usr/bin/* /usr/sbin/*| grep "i--"
   ----i-------- /bin/login
   ----i-------- /bin/netstat
   ----i-------- /bin/ps
   ----i-------- /sbin/insmod
   ----i-------- /sbin/kallsyms
   ----i-------- /sbin/ksyms
   ----i-------- /sbin/lsmod
   ----i-------- /sbin/modprobe
   ----i-------- /sbin/rmmod
   ----i-------- /usr/sbin/sshd

Unless you've set these files to be immutable yourself, this is another good indicator that your system has been cracked. These immutable bits keep even the root user from deleting, moving, changing, or even reinstalling the files. The ultimate fix, as previously mentioned, is a complete reinstall. However, the way in which you "un-set" these bits is with the chattr command, like this.

   # chattr -i /bin/login
   # lsattr /bin/login
   ------------- /bin/login

Now this file, at least, appears to be back to normal. Just remember, we really can't trust our own eyes on this system as even the kernel could be faking us out and we would never know it. This type of fix is really more of a band-aid to give you a stopgap measure on a production system (to get it as patched up as you can) until you can migrate all the content off and reformat the system.


Team LiB
Previous Section Next Section