|
|
< Day Day Up > |
|
7.2. Network Performance ToolsThis section describes the Linux network performance tools available to diagnose performance problems. We start with the tools to determine the lowest level of network performance (physical statistics) and add tools that can investigate the layers above that. 7.2.1. mii-tool (Media-Independent Interface Tool)mii-tool is an Ethernet-specific hardware tool primarily used to configure an Ethernet device, but it can also provide information about the current configuration. This information, such as the link speed and duplex setting, can be useful when tracking down the cause of an under-performing network device. 7.2.1.1 Network I/O Performance-Related Optionsmii-tool requires root access to be used. It is invoked with the following command line: mii-tool [-v] [device] mii-tool prints the Ethernet settings for the given device. If no devices are specified, mii-tool displays information about all the available Ethernet devices. If the -v option is used, mii-tool displays verbose statistics about the offered and negotiated network capabilities. 7.2.1.2 Example UsageListing 7.1 shows the configuration of eth0 on the system. The first line tells us that the Ethernet device is currently using a 100BASE-T full-duplex connection. The next few lines describe the capabilities of the network card in the machine and the capabilities that the card has detected of the network device on the other end of the wire. Listing 7.1.
[root@nohs linux-2.6.8-1.521]# /sbin/mii-tool -v eth0
eth0: negotiated 100baseTx-FD, link ok
product info: vendor 00:00:00, model 0 rev 0
basic mode: autonegotiation enabled
basic status: autonegotiation complete, link ok
capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
mii-tool provides low-level information about how the physical level of the etheRnet device is configured. 7.2.2. ethtoolethtool provides similar capabilities to mii-tool for configuration and display of statistics for Ethernet devices. However, ethtool is the more powerful tool and contains more configuration options and device statistics. 7.2.2.1 Network I/O Performance-Related Optionsethtool requires root access to be used. It is invoked with the following command line: ethtool [device] ethtool prints out configuration information about the given Ethernet device. If no devices are provided, ethtool prints statistics for all the Ethernet devices in the system. The options to change the current Ethernet settings are described in detail in the ethtool main page. 7.2.2.2 Example UsageListing 7.2 shows the configuration of eth0 on the system. Although the device supports many different speed and link settings, it is currently connected to a full-duplex 1,000Mbps link. Listing 7.2.
[root@scrffy tmp]# /sbin/ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes
ethtool is simple to run, and it can quickly provide information about an improperly configured network device. 7.2.3. ifconfig (Interface Configure)The primary job of ifconfig is to set up and configure the network interfaces in a Linux box. It also provides rudimentary performance statistics about all the network devices in the system. ifconfig is available on almost every Linux machine that uses networking. 7.2.3.1 Network I/O Performance-Related Optionsifconfig is invoked with the following command line: ifconfig [device] If no device is specified, ifconfig shows statistics about all the active network devices. Table 7-1 describes the performance statistics that ifconfig provides. Although primarily for network configuration, ifconfig provides a reasonable number of statistics that you can use to determine the health and performance of each of the network devices in the system. 7.2.3.2 Example UsageListing 7.3 shows the network performance statistics from all the devices in the system. In this case, we have an Ethernet card (eth0) and the loopback (lo) device. In this example, the Ethernet card has received ~790Mb of data and has transmitted ~319Mb. Listing 7.3.
[ezolt@wintermute tmp]$ /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:02:E3:15:A5:03
inet addr:192.168.0.4 Bcast:192.168.0.255 Mask:255.255.255.0
UP BROADCAST NOTRAILERS RUNNING MTU:1500 Metric:1
RX packets:1047040 errors:0 dropped:0 overruns:0 frame:0
TX packets:796733 errors:12 dropped:0 overruns:12 carrier:12
collisions:0 txqueuelen:1000
RX bytes:829403956 (790.9 Mb) TX bytes:334962327 (319.4 Mb)
Interrupt:19 Base address:0x3000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:102 errors:0 dropped:0 overruns:0 frame:0
TX packets:102 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6492 (6.3 Kb) TX bytes:6492 (6.3 Kb)
The statistics provided by ifconfig represent the cumulative amount since system boot. If you bring down a network device and then bring it back up, the statistics do not reset. If you run ifconfig at regular intervals, you can eyeball the rate of change in the various statistics. You can automate this by using the watch command or a shell script, both of which are described in the next chapter. 7.2.4. ipSome of the network tools, such as ifconfig, are being phased out in favor of the new command: ip. ip enables you to configure many different aspect of Linux networking, but it can also display performance statistics about each network device. 7.2.4.1 Network I/O Performance-Related OptionsWhen extracting performance statistics, you invoke ip with the following command line: ip -s [-s] link If you call ip with these options, it prints statistics about all the network devices in the system, including the loopback (lo) and simple Internet transition (sit0) device. The sit0 device allows IPv6 packets to be encapsulated in IPv4 packets and exists to ease the transition between IPv4 and IPv6. If the extra -s is provided to ip, it provides a more detailed list of low-level Ethernet statistics. Table 7-2 describes some of the performance statistics provided by ip. ip is a very versatile tool for Linux network configuration, and although its main function is the configuration of the network, you can use it to extract low-level device statistics as well. 7.2.4.2 Example UsageListing 7.4 shows the network performance statistics from all the devices in the system. In this case, we have an Ethernet card, the loopback device, and the sit0 tunnel device. In this example, the Ethernet card has received ~820Mb of data and has transmitted ~799Mb. Listing 7.4.
[ezolt@nohs ezolt]$ /sbin/ip -s link
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
RX: bytes packets errors dropped overrun mcast
4460 67 0 0 0 0
TX: bytes packets errors dropped carrier collsns
4460 67 0 0 0 0
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:10:b5:59:2c:82 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped overrun mcast
799273378 920999 0 0 0 0
TX: bytes packets errors dropped carrier collsns
820603574 930929 0 0 0 0
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
RX: bytes packets errors dropped overrun mcast
0 0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
0 0 0 0 0 0
Much like ifconfig, ip provides system totals for statistics since the system has booted. If you use watch (described in the next chapter), you can monitor how these values change over time. 7.2.5. sarAs discussed in previous chapters, sar is one of the most versatile Linux performance tools. It can monitor many different things, archive statistics, and even display information in a format that is usable by other tools. sar does not always provide as much detail as the area-specific performance tools, but it provides a good overview. Network performance statistics are no different. sar provides information about the link-level performance of the network, as do ip and ifconfig; however, it also provides some rudimentary statistics about the number of sockets opened by the transport layer. 7.2.5.1 Network I/O Performance-Related Optionssar collects network statistics using the following command: sar [-n DEV | EDEV | SOCK | FULL ] [DEVICE] [interval] [count] sar collects many different types of performance statistics. Table 7-3 describes the command-line options used by sar to display network performance statistics. The network performance options that sar provides are described in Table 7-4. Considering all the statistics that sar can gather, it really does provide the most system-level performance statistics in a single location. 7.2.5.2 Example UsageIn Listing 7.5, we examine the transmit and receive statistics of all the network devices in the system. As you can see, the eth0 device is the most active. In the first sample, eth0 is receiving ~63,000 bytes per second (rxbyt/s) and transmitting ~45,000 bytes per second (txbyt/s). No compressed packets are sent (txcmp) or received (rxcmp). (Compressed packets are usually present during SLIP or PPP connections.) Listing 7.5.[ezolt@wintermute sysstat-5.0.2]$ sar -n DEV 1 2 Linux 2.4.22-1.2174.nptlsmp (wintermute.phil.org) 06/07/04 21:22:29 IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s 21:22:30 lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 21:22:30 eth0 68.00 65.00 63144.00 45731.00 0.00 0.00 0.00 21:22:30 IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s 21:22:31 lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 21:22:31 eth0 80.39 47.06 45430.39 30546.08 0.00 0.00 0.00 Average: IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 74.26 55.94 54199.50 38063.37 0.00 0.00 0.00 In Listing 7.6, we examine the number of open sockets in the system. We can see the total number of open sockets and the TCP, RAW, and UDP sockets. sar also displays the number of fragmented IP packets. Listing 7.6.[ezolt@wintermute sysstat-5.0.2]$ sar -n SOCK 1 2 Linux 2.4.22-1.2174.nptlsmp (wintermute.phil.org) 06/07/04 21:32:26 totsck tcpsck udpsck rawsck ip-frag 21:32:27 373 118 8 0 0 21:32:28 373 118 8 0 0 Average: 373 118 8 0 0 sar provides a good overview of the system's performance. However, when we are investigating a performance problem, we really want to understand what processes or services are consuming a particular resource. sar does not provide this level of detail, but it does enable us to observe the overall system network I/O statistics. 7.2.6. gkrellmgkrellm is a graphical monitor that enables you to keep an eye on many different system performance statistics. It draws charts of different performance statistics, including CPU usage, disk I/O, and network usage. It can be "themed" to change its appearance, and even accepts plug-ins to monitor events not included in the default release. gkrellm provides similar information to sar, ip, and ipconfig, but unlike the other tools, it provides a graphical view of the data. In addition, it can provide information about the traffic flowing through particular UDP and TCP ports. This is the first tool that we have seen that can show which services are consuming different amounts of network bandwidth. 7.2.6.1 Network I/O Performance-Related Optionsgkrellm is invoked using the following command line: gkrellm None of gkrellm's command-line options configure the statistics that it monitors. You do all configurations graphically after gkrellm is started. To bring up the configuration screen, you can either right-click the gkrellm's title bar and select Configuration, or just press F1 when your cursor is in any area of the window. This brings up a configuration window (see Figure 7-1). Figure 7-1.Figure 7-2 shows the network configuration window. It is used to configure which statistics and which devices are shown in the final gkrellm output window. Figure 7-2.You can configure gkrellm to monitor the activity on a particular range of TCP ports. Doing so enables you to monitor the exact ports used by services such as HTTP or FTP and to measure the amount of bandwidth that they are using. In Figure 7-2, we have configured gkrellm to monitor the ports used by the bittorrent (BT) P2P application and the Web server (HTTP). gkrellm is a flexible and powerful graphical performance-monitoring tool. It enables you to see how the system is currently performing and how its performance changes over time. The most difficult aspect of using gkrellm is reading the small default text. However, the appearance of gkrellm can be easily themed, so presumably, this could be easily fixed. 7.2.6.2 Example UsageAs stated previously, gkrellm can monitor many different types of events. In Figure 7-3, we pruned the output so that only statistics relevant to network traffic and use is displayed. Figure 7-3.
As you can see in Figure 7-3, the top two graphs are the bandwidth used for the ports (BT and HTTP) that we set up in the configuration section, and the bottom two graphics are the statistics for each of the network devices (eth0 and lo). There is a small amount of bittorrent (BT) traffic, but no Web server traffic (HTTP). The Ethernet device eth0 had some large activity in the past, but is settling down now. The lighter shade in the eth0 indicates the number of bytes received, and the darker shade indicates the number of bytes transmitted. gkrellm is a powerful graphical tool that makes it easy to diagnose the status of the system at a glance. 7.2.7. iptrafiptraf is a real-time network monitoring tool. It provides a large number of modes to monitor network interfaces and traffic. iptraf is a console application, but its user interface is a cursor-based series of menus and windows. Like the other tools mentioned previously in this chapter, it can provide information about the rate at which each network device is sending frames. However, it can also display information about the type and size of the TCP/IP packet and about which ports are being used for network traffic. 7.2.7.1 Network I/O Performance-Related Optionsiptraf is invoked with the following command line: iptraf [-d interface] [-s interface] [-t <minutes>] If iptraf is called with no parameters, it brings up a menu that enables you to select the interface to monitor and type of information that you want to monitor. Table 7-5 describes the command-line parameters that enable you to see the amount of network traffic on a particular interface or network service. iptraf has many more modes and configuration options. Read its included documentation for more information. 7.2.7.2 Example Usageiptraf creates a display similar to Figure 7-4 when it is invoked with the following command: [root@wintermute tmp]# iptraf -d eth0 -t 1 Figure 7-4.
This command specifies that iptraf should display detailed statistics about Ethernet device eth0 and exit after it has run for 1 minute. In this case, we can see that 186.8kbps are received and 175.5kbps are transmitted by the eth0 network device. The next command, whose results are shown in Figure 7-5, asks iptraf to show information about the amount of network traffic from each UDP or TCP port. iptraf was invoked with the following command: Figure 7-5.
[root@wintermute etherape-0.9.0]# iptraf -s eth0 -t 10 Because the TCP or UDP ports of well-known services are fixed, you can use this to determine how much traffic each service is handling. Figure 7-5 shows that 29kb of HTTP data has been sent from eth0 and 25kb has been received. Because iptraf is a console-based application, it doesn't require an X server or X server libraries. Even though iptraf cannot be controlled with a mouse, it is easy to use and configurable. 7.2.8. netstatnetstat is a basic network-performance tool that is present on nearly every Linux machine with networking. You can use it to extract information about the number and types of network sockets currently being used and interface-specific statistics regarding the number of UDP or TCP packets flowing to and from the current system. It also enables you to trace the owner of a socket back to a particular process or PID, which can prove useful when trying to determine the application responsible for network traffic. 7.2.8.1 Network I/O Performance-Related Optionsnetstat is invoked with the following command line: netstat [-p] [-c] [–interfaces=<name>] [-s] [-t] [-u] [-w] If netstat is called without any parameters, it shows information about system-wide socket usage and displays information about both Internet and UNIX domain sockets. (UNIX domain sockets are used for interprocess communication on the local machine, but do not indicate network traffic.) To retrieve all the statistics that netstat is capable of displaying, you must run it as root. Table 7-6 describes the command-line options of netstat that modify the types of network statistics that netstat displays.
netstanetstat also accepts some command-line options not described here. See the netstat man page for more details. 7.2.8.2 Example UsageListing 7.7 asks netstat to show the active TCP connections and to continually update this information. Every second, netstat displays new TCP network statistics. netstat does not enable you to set the length of time that it will monitor, so it will only stop if it is killed or interrupted (Ctrl-C). Listing 7.7.[root@wintermute ezolt]# netstat -t -c Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 192.168.0.4:1023 fas.harvard.edu:ssh ESTABLISHED tcp 0 0 192.168.0.4:32844 216.239.39.147:http TIME_WAIT tcp 0 0 192.168.0.4:32843 216.239.39.147:http TIME_WAIT tcp 0 0 192.168.0.4:32853 skaiste.elekta.lt:http ESTABLISHED Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 192.168.0.4:1023 fas.harvard.edu:ssh ESTABLISHED tcp 0 0 192.168.0.4:32844 216.239.39.147:http TIME_WAIT tcp 0 0 192.168.0.4:32843 216.239.39.147:http TIME_WAIT tcp 0 0 192.168.0.4:32853 skaiste.elekta.lt:http ESTABLISHED Listing 7.8 asks netstat to once again print the TCP socket information, but this time, we also ask it to display the program that is responsible for this socket. In this case, we can see that SSH and mozilla-bin are the applications that are initiating the TCP connections. Listing 7.8.[root@wintermute ezolt]# netstat -t -p Active Internet connections (w/o servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 192.168.0.4:1023 fas.harvard.edu:ssh ESTABLISHED 1463/ssh tcp 0 0 192.168.0.4:32844 216.239.39.147:http TIME_WAIT - tcp 0 0 192.168.0.4:32843 216.239.39.147:http TIME_WAIT - tcp 0 0 192.168.0.4:32853 skaiste.elekta.lt:http ESTABLISHED 1291/mozilla-bin Listing 7.9 asks netstat to provide statistics about the UDP traffic that the system has received since boot. Listing 7.9.
[root@wintermute ezolt]# netstat -s -u
Udp:
125 packets received
0 packets to unknown port received.
0 packet receive errors
152 packets sent
Listing 7.10 asks netstat to provide information about the amount of network traffic flowing through the eth0 interface. Listing 7.10.[root@wintermute ezolt]# netstat –interfaces=eth0 Kernel Interface table Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg eth0 1500 0 52713 0 0 0 13711 1 0 1 BNRU netstat provides a great number of network performance statistics about sockets and interfaces in a running Linux system. It is the only network-performance tool that maps the sockets used back to the PID of the process that is using it, and is therefore very useful. 7.2.9. etherapeetherape (a pun on the Windows-based network tool etherman) provides a visualization of the current network traffic. By default, it observes all the network traffic flowing on the network, not just those packets that the current machine is sending or receiving. However, it can be configured to only display network information for the current machine. etherape is a little rough around the edges (in interface and documentation), but it provides a unique visual insight into how the network is connected, what types of services are being requested, and which nodes are requesting them. It creates a graph whose nodes represent the systems on the network. The nodes that are communicating have lines connecting them that increase in size as more network traffic flows between them. As a particular system's network usage increases, the size of the circle representing that system also increases. The lines connecting the different systems are colored differently depending on the protocols they are using to communicate with each other. 7.2.9.1 Network I/O Performance-Related Optionsetherape uses the libpcap library to capture the network packets and, as a result, it must be run as root. etherape is invoked using the following command line: etherape [-n] [-i <interface name>] Table 7-7 describes some of the command-line options that change the interface that etherape monitors and whether resolved host names are printed on each node.
All in all, etherape's documentation is rather sparse. The etherape man page describes a few more command lines that change its appearance and behavior, but the best way to learn it is to try it. In general, etherape is a great way to visualize the network. 7.2.9.2 Example UsageFigure 7-6 shows etherape monitoring a relatively simple network. If we match up the color of the protocol to the color of the biggest circle, we see that this node is generating a high amount of SSH traffic. From the figure, it can be difficult to determine which node is causing this SSH traffic. Although not pictured, if we double-click the big circle, etherape creates a window with statistics pertaining to the node responsible for the traffic. We can use this to investigate each of the generators of network traffic and investigate their node names. Figure 7-6. |
|
|
< Day Day Up > |
|
