9.8. Optimizing Network I/O Usage

When you know that a network problem is happening, Linux provides a set of tools to determine which applications are involved. However, when you are connected to external machines, the fix to a network problem is not always within your control.

Figure 9-6 shows the steps that we take to investigate a network performance problem.

To start the investigation, continue to Section 9.8.1.

9.8.1. Is Any Network Device Sending/Receiving Near the Theoretical Limit?

The first thing to do is to use ethtool to determine what hardware speed each Ethernet device is set to. If you record this information, you then investigate whether any of the network devices are saturated. Ethernet devices and/or switches can be easily mis-configured, and ethtool shows what speed each device believes that it is operating at. After you determine the theoretical limit of each of the Ethernet devices, use iptraf (of even ifconfig) to determine the amount of traffic that is flowing over each interface. If any of the network devices appear to be saturated, go to Section 9.8.3; otherwise, go to Section 9.8.2.

9.8.2. Is Any Network Device Generating a Large Number of Errors?

Network traffic can also appear to be slow because of a high number of network errors. Use ifconfig to determine whether any of the interfaces are generating a large number of errors. A large number of errors can be the result of a mismatched Ethernet card / Ethernet switch setting. Contact your network administrator, search the Web for people with similar problems, or e-mail questions to one of the Linux networking newsgroups.

Go to Section 9.9.

9.8.3. What Type of Traffic Is Running on That Device?

If a particular device is servicing a large amount of data, use iptraf to track down what types of traffic that device is sending and receiving. When you know the type of traffic that the device is handling, advance to Section 9.8.4.

9.8.4. Is a Particular Process Responsible for That Traffic?

Next, we want to determine whether a particular process is responsible for that traffic. Use netstat with the -p switch to see whether any process is handling the type of traffic that is flowing over the network port.

If an application is responsible, go to Section 9.8.6. If none are responsible, go to Section 9.8.5.

9.8.5. What Remote System Is Sending the Traffic?

If no application is responsible for this traffic, some system on the network may be bombarding your system with unwanted traffic. To determine which system is sending all this traffic, use iptraf or etherape.

If it is possible, contact the owner of this system and try to figure out why this is happening. If the owner is unreachable, it might be possible to set up ipfilters within the Linux kernel to always drop this particular traffic, or to set up a firewall between the remote machine and the local machine to intercept the traffic.

Go to Section 9.9.

9.8.6. Which Application Socket Is Responsible for the Traffic?

Determining which socket is being used is a two-step process. First, we can use strace to trace all the I/O system calls that an application is making by using strace -e trace=file. This shows which file descriptors the process is reading and writing from. Second, we map these file descriptors back to a socket by looking in the proc file system. The files in /proc/<pid>/fd/ are symbolic links from the file descriptor number to the actual files or sockets. An ls -la of this directory shows all the file descriptors of this particular process. Those with socket in the name are network sockets. You can then use this information to determine inside the program which socket is causing all the communication.

Go to Section 9.9.