30.5 TCP Concurrent Server, One Child per Client

Traditionally, a concurrent TCP server calls fork to spawn a child to handle each client. This allows the server to handle numerous clients at the same time, one client per process. The only limit on the number of clients is the OS limit on the number of child processes for the user ID under which the server is running. Figure 5.12 is an example of a concurrent server and most TCP servers are written in this fashion.

The problem with these concurrent servers is the amount of CPU time it takes to fork a child for each client. Years ago (the late 1980s), when a busy server handled hundreds or perhaps even a few thousand clients per day, this was acceptable. But the explosion of the Web has changed this attitude. Busy Web servers measure the number of TCP connections per day in the millions. This is for an individual host, and the busiest sites run multiple hosts, distributing the load among the hosts. (Section 14.2 of TCPv3 talks about a common way to distribute this load using what is called "DNS round robin.") Later sections will describe various techniques that avoid the per-client fork incurred by a concurrent server, but concurrent servers are still common.

Figure 30.4 shows the main function for our concurrent TCP server.

Figure 30.4 main function for TCP concurrent server.

server/serv01.c

 1 #include     "unp.h"

 2 int
 3 main(int argc, char **argv)
 4 {
 5     int     listenfd, connfd;
 6     pid_t   childpid;
 7     void    sig_chld(int), sig_int(int), web_child(int);
 8     socklen_t clilen, addrlen;
 9     struct sockaddr *cliaddr;

10     if (argc == 2)
11         listenfd = Tcp_listen(NULL, argv[1], &addrlen);
12     else if (argc == 3)
13         listenfd = Tcp_listen(argv[1], argv[2], &addrlen);
14     else
15         err_quit("usage: serv01 [ <host> ] <port#>");
16     cliaddr = Malloc(addrlen);

17     Signal(SIGCHLD, sig_chld);
18     Signal(SIGINT, sig_int);

19     for ( ; ; ) {
20         clilen = addrlen;
21         if  ( (connfd = accept(listenfd, cliaddr, &clilen)) < 0) {
22              if (errno == EINTR)
23                  continue;      /* back to for() */
24              else
25                  err_sys("accept error");
26         }

27         if ( (childpid = Fork()) == 0) { /* child process */
28             Close(listenfd);     /* close listening socket */
29             web_child(connfd);   /* process request */
30             exit(0);
31         }
32         Close(connfd);            /* parent closes connected socket */
33     }
34 }

This function is similar to Figure 5.12: It calls fork for each client connection and handles the SIGCHLD signals from the terminating children. This function, however, we have made protocol-independent by calling our tcp_listen function. We do not show the sig_chld signal handler: It is the same as Figure 5.11, with the printf removed.

We also catch the SIGINT signal, generated when we type our terminal interrupt key. We type this key after the client completes, to print the CPU time required for the program. Figure 30.5 shows the signal handler. This is an example of a signal handler that does not return.

Figure 30.5 Signal handler for SIGINT.

server/serv01.c

35 void
36 sig_int(int signo)
37 {
38     void    pr_cpu_time(void);

39     pr_cpu_time();
40     exit(0);
41 }

Figure 30.6 shows the pr_cpu_time function that is called by the signal handler.

Figure 30.6 pr_cpu_time function: prints total CPU time.

server/pr_cpu_time.c

 1 #include    "unp.h"
 2 #include    <sys/resource.h>

 3 #ifndef HAVE_GETRUSAGE_PROTO
 4 int     getrusage(int, struct rusage *);
 5 #endif

 6 void
 7 pr_cpu_time(void)
 8 {
 9     double user, sys;
10     struct rusage myusage, childusage;

11     if (getrusage(RUSAGE_SELF, &myusage) < 0)
12         err_sys("getrusage error");
13     if (getrusage(RUSAGE_CHILDREN, &childusage) < 0)
14         err_sys("getrusage error");

15     user = (double) myusage.ru_utime.tv_sec +
16         myusage.ru_utime.tv_usec / 1000000.0;
17     user += (double) childusage.ru_utime.tv_sec +
18         childusage.ru_utime.tv_usec / 1000000.0;
19     sys = (double) myusage.ru_stime.tv_sec +
20         myusage.ru_stime.tv_usec / 1000000.0;
21     sys += (double) childusage.ru_stime.tv_sec +
22         childusage.ru_stime.tv_usec / 1000000.0;

23     printf("\nuser time = %g, sys time = %g\n", user, sys);
24 }

The getrusage function is called twice to return the resource utilization of both the calling process (RUSAGE_SELF) and all the terminated children of the calling process (RUSAGE_CHILDREN). The values printed are the total user time (CPU time spent in the user process) and total system time (CPU time spent within the kernel, executing on behalf of the calling process).

The main function in Figure 30.4 calls the function web_child to handle each client request. Figure 30.7 shows this function.

Figure 30.7 web_child function to handle each client's request.

server/web_child.c

 1 #include    "unp.h"

 2 #define MAXN    16384     /* max # bytes client can request */

 3 void
 4 web_child(int sockfd)
 5 {
 6     int     ntowrite;
 7     ssize_t nread;
 8     char    line[MAXLINE], result[MAXN];

 9     for ( ; ; ) {
10         if ( (nread = Readline(sockfd, line, MAXLINE)) == 0)
11             return;             /* connection closed by other end */

12             /* line from client specifies #bytes to write back */
13         ntowrite = atol(line);
14         if ((ntowrite <= 0) || (ntowrite > MAXN))
15             err_quit("client request for %d bytes", ntowrite);

16         Writen(sockfd, result, ntowrite);
17     }
18 }

After the client establishes the connection with the server, the client writes a single line specifying the number of bytes the server must return to the client. This is some-what similar to HTTP: The client sends a small request and the server responds with the desired information (often an HTML file or a GIF image, for example). In the case of HTTP, the server normally closes the connection after sending back the requested data, although newer versions are using persistent connections, holding the TCP connection open for additional client requests. In our web_child function, the server allows additional requests from the client, but we saw in Figure 30.3 that our client sends only one request per connection and the client then closes the connection.

Row 1 of Figure 30.1 shows the timing result for this concurrent server. When compared to the subsequent lines in this figure, we see that the concurrent server requires the most CPU time, which is what we expect with one fork per client.

One server design that we do not measure in this chapter is one invoked by inetd, which we covered in Section 13.5. From a process control perspective, a server invoked by inetd involves a fork and an exec, so the CPU time will be even greater than the times shown in row 1 of Figure 30.1.