26.5 Thread-Specific Data

When converting existing functions to run in a threads environment, a common problem encountered is due to static variables. A function that keeps state in a private buffer, or one that returns a result in the form of a pointer to a static buffer, is not thread-safe because multiple threads cannot use the buffer to hold different things at the same time. When faced with this problem, there are various solutions:

• Use thread-specific data. This is nontrivial and then converts the function into one that works only on systems with threads support. The advantage to this approach is that the calling sequence does not change and all the changes go into the library function and not the applications that call the function. We show a version of readline that is thread-safe by using thread-specific data later in this section.

• Change the calling sequence so that the caller packages all the arguments into a structure, and also store in that structure the static variables from Figure 3.18. This was also done, and Figure 26.6 shows the new structure and new function prototypes.

Figure 26.6 Data structure and function prototype for re-entrant version of readline.

          typedef struct {
            int       read_fd;        /* caller's descriptor to read from */
            char     *read_ptr;       /* caller's buffer to read into */
            size_t    read_maxlen;    /* caller's max # bytes to read */
                          /* next three are used internally by the function */
            int       rl_cnt;         /* initialize to 0 */
            char     *rl_bufptr;      /* initialize to rl_buf */
            char      rl_buf[MAXLINE];
          } Rline;

          void    readline_rinit(int, void *, size_t, Rline *);
          ssize_t readline_r(Rline *);
          ssize_t Readline_r(Rline *);

These new functions can be used on threaded and nonthreaded systems, but all applications that call readline must change.

• Restructure the interface to avoid any static variables so that the function is thread-safe. For the readline example, this would be the equivalent of ignoring the speedups introduced in Figure 3.18 and going back to the older version in Figure 3.17. Since we said the older version was "painfully slow," taking this option is not always viable.

Thread-specific data is a common technique for making an existing function thread-safe. Before describing the Pthread functions that work with thread-specific data, we describe the concept and a possible implementation, because the functions appear more complicated than they really are.

Part of the complication in many texts on using threads is that their descriptions of thread-specific data read like the Pthreads standard, talking about key-value pairs and keys being opaque objects. We describe thread-specific data in terms of indexes and pointers because common implementations use a small integer index for the key, and the value associated with the index is just a pointer to a region that the thread mallocs.

Each system supports a limited number of thread-specific data items. POSIX requires this limit be no less than 128 (per process), and we assume this limit in the following example. The system (probably the threads library) maintains one array of structures per process, which we call key structures, as we show in Figure 26.7.

The flag in the Key structure indicates whether this array element is currently in use, and all the flags are initialized to be "not in use." When a thread calls pthread_key_create to create a new thread-specific data item, the system searches through its array of Key structures and finds the first one not in use. Its index, 0 through 127, is called the key, and this index is returned to the calling thread. We will talk about the "destructor pointer," the other member of the Key structure, shortly.

In addition to the process-wide array of Key structures, the system maintains numerous pieces of information about each thread within a process. We call this a Pthread structure and part of this information is a 128-element array of pointers, which we call the pkey array. We show this in Figure 26.8.

All entries in the pkey array are initialized to null pointers. These 128 pointers are the "values" associated with each of the possible 128 "keys" in the process.

When we create a key with pthread_key_create, the system tells us its key (index). Each thread can then store a value (pointer) for the key, and each thread normally obtains the pointer by calling malloc. Part of the confusion with thread-specific data is that the pointer is the value in the key-value pair, but the real thread-specific data is whatever this pointer points to.

We now go through an example of how thread-specific data is used, assuming that our readline function uses thread-specific data to maintain the per-thread state across successive calls to the function. Shortly we will show the code for this, modifying our readline function to follow these steps:

One item we have not addressed is: What happens when a thread terminates? If the thread has called our readline function, that function has allocated a region of memory that needs to be freed. This is where the "destructor pointer" from Figure 26.7 is used. When the thread that creates the thread-specific data item calls pthread_key_create, one argument to this function is a pointer to a destructor function. When a thread terminates, the system goes through that thread's pkey array, calling the corresponding destructor function for each non-null pkey pointer. What we mean by "corresponding destructor" is the function pointer stored in the Key array in Figure 26.7. This is how the thread-specific data is freed when a thread terminates.

The first two functions that are normally called when dealing with thread-specific data are pthread_once and pthread_key_create.

#include <pthread.h>
int pthread_once(pthread_once_t *onceptr, void (*init) (void));
int pthread_key_create(pthread_key_t *keyptr, void (*destructor) (void *value));
Both return: 0 if OK, positive Exxx value on error

pthread_once is normally called every time a function that uses thread-specific data is called, but pthread_once uses the value in the variable pointed to by onceptr to guarantee that the init function is called only one time per process.

pthread_key_create must be called only one time for a given key within a process. The key is returned through the keyptr pointer, and the destructor function, if the argument is a non-null pointer, will be called by each thread on termination if that thread has stored a value for this key.

Typical usage of these two functions (ignoring error returns) is as follows:

     pthread_key_t   rl_key;
     pthread_once_t  rl_once = PTHREAD_ONCE_INIT;

     void
     readline_destructor(void *ptr)
     {
         free(ptr);
     }

     void
     readline_once(void)
     {
         pthread_key_create(&rl_key, readline_destructor);
     }

     ssize_t
     readline( ... )
     {
         ...

         pthread_once(&rl_once, readline_once);

         if ( (ptr = pthread_getspecific(rl_key)) == NULL) {
             ptr = Malloc( ... );
             pthread_setspecific(rl_key, ptr);
             /* initialize memory pointed to by ptr */
         }
         ...
         /* use values pointed to by ptr */
     }

Every time readline is called, it calls pthread_once. This function uses the value pointed to by its onceptr argument (the contents of the variable rl_once) to make certain that its init function is called only one time. This initialization function, readline_once, creates the thread-specific data key that is stored in rl_key, and which readline then uses in calls to pthread_getspecific and pthread_setspecific.

The pthread_getspecific and pthread_setspecific functions are used to fetch and store the value associated with a key. This value is what we called the "pointer" in Figure 26.8. What this pointer points to is up to the application, but normally, it points to dynamically allocated memory.

#include <pthread.h>
void *pthread_getspecific(pthread_key_t key);
Returns: pointer to thread-specific data (possibly a null pointer)
int pthread_setspecific(pthread_key_t key, const void *value);
Returns: 0 if OK, positive Exxx value on error

Notice that the argument to pthread_key_create is a pointer to the key (because this function stores the value assigned to the key), while the arguments to the get and set functions are the key itself (probably a small integer index as discussed earlier).

Example: readline Function Using Thread-Specific Data

We now show a complete example of thread-specific data by converting the optimized version of our readline function from Figure 3.18 to be thread-safe, without changing the calling sequence.

Figure 26.11 shows the first part of the function: the pthread_key_t variable, the pthread_once_t variable, the readline_destructor function, the readline_once function, and our Rline structure that contains all the information we must maintain on a per-thread basis.

Figure 26.11 First part of thread-safe readline function.

threads/readline.c

 1 #include    "unpthread.h"

 2 static pthread_key_t rl_key;
 3 static pthread_once_t rl_once = PTHREAD_ONCE_INIT;

 4 static void
 5 readline_destructor(void *ptr)
 6 {
 7     free(ptr);
 8 }

 9 static void
10 readline_once(void)
11 {
12     Pthread_key_creat(&rl_key, readline_destructor);
13 }

14 typedef struct {
15     int     rl_cnt;                /* initialize to 0 */
16     char   *rl_bufptr;             /* initialize to rl_buf */
17     char    rl_buf[MAXLINE];
18 } Rline;

Destructor

4–8 Our destructor function just frees the memory that was allocated for this thread.

One-time function

9–13 We will see that our one-time function is called one time by pthread_once, and it just creates the key that is used by readline.

Rline structure

14–18 Our Rline structure contains the three variables that caused the problem by being declared static in Figure 3.18. One of these structures will be dynamically allocated per thread and then released by our destructor function.

Figure 26.12 shows the actual readline function, plus the function my_read it calls. This figure is a modification of Figure 3.18.

my_read function

19–35 The first argument to the function is now a pointer to the Rline structure that was allocated for this thread (the actual thread-specific data).

Allocate thread-specific data

42 We first call pthread_once so that the first thread that calls readline in this process calls readline_once to create the thread-specific data key.

Fetch thread-specific data pointer

43–46 pthread_getspecific returns the pointer to the Rline structure for this thread. But if this is the first time this thread has called readline, the return value is a null pointer. In this case, we allocate space for an Rline structure and the rl_cnt member is initialized to 0 by calloc. We then store the pointer for this thread by calling pthread_setspecific. The next time this thread calls readline, pthread_getspecific will return this pointer that was just stored.

Figure 26.12 Second part of thread-safe readline function.

threads/readline.c

19 static ssize_t
20 my_read(Rline *tsd, int fd, char *ptr)
21 {
22     if (tsd->rl_cnt < = 0 {
23       again:
24         if ( (tsd->rl_cnt = read(fd, tsd->rl_buf, MAXLINE)) < 0) {
25             if (error == EINTR)
26                 goto again;
27             return (-1);
28         } else if (tsd->rl_cnt == 0)
29             return (0);
30         tsd->rl_bufptr = tsd->rl_buf;
31     }

32     tsd->rl_cnt--;
33     *ptr = *tsd->rl_bufptr++;
34     return (1);
35 }

36 ssize_t
37 readline(int fd, void *vptr, size_t maxlen)
38 {
39     size_t n, rc;
40     char     c, *ptr;
41     Rline *tsd;

42     Pthread_once(&rl_once, readline_once);
43     if ( (tsd = pthread_getspecific(rl_key)) == NULL) {
44         tsd = Calloc(1, sizeof(Rline)); /* init to 0 */
45         Pthread_setspecific(rl_key, tsd);
46     }
47     ptr = vptr;
48     for (n = 1; n < maxlen; n++) {
49         if ( (rc = my_read(tsd, fd, &c)) == 1) {
50             *ptr++ = c;
51             if (c == '\n')
52                 break;
53         } else if (rc == 0) {
54             *ptr = 0;
55             return (n - 1);     /* EOF, n - 1 bytes read */
56         } else
57             return (-1);       /* error, errno set by read() */
58     }
59     *ptr = 0;
60     return (n);
61 }