8.3. Pthread management

This section explains some of the basic Pthread management routines used for creating, joining, exiting, and detaching Pthreads.

8.3.1. Creating and terminating Pthreads

In order to create new Pthreads (besides the initial thread), the program must call the pthread_create() sub-routine. Example 8-5 is our first multi-threaded sample.

Example 8-5. create-5threads.c

						#include <pthread.h>                                                 /* #A */
#include <stdio.h>
#include <stdlib.h>
#define NUM_OF_THREADS      6
#define EXIT_CODE -1

void *thr_func(void *id)
{
    int j;

    for (j = 0; j < 500000; j++) {
        ;                                                            /* #B */
    }
    printf("Hello world from Pthread %d!
", (int *)id);
    pthread_exit(NULL);                                              /* #C */
}

int main (int argc, char *argv[])
{
    int rc, i;
    pthread_t tid[NUM_OF_THREADS];

    for (i = 1; i < NUM_OF_THREADS; i++) {
        printf("Creating Pthread: %d
", i);
        rc = pthread_create(&tid[i], NULL, thr_func, (void *)i);    /* #D */
        if (rc != 0) {
            fprintf(stderr
                , "pthread_create() failed with rc = %d at %d in %s.
"
                , rc, __LINE__, __FILE__);
            exit(EXIT_CODE);
         }
     }
     pthread_exit(NULL);
}

This program creates five Pthreads, as illustrated in Figure 8-2.

The highlighted lines with comments #A, #B, #C, and #D in Example 8-5 on page 284 explain the following important programming manners when developing multi-threaded applications:

• Comment #A

  To use the Pthread subroutines, the pthread.h header file must be included as the first header file in the each source file using the Pthreads library. This is because it defines some important macros that affect other system header files. Having the pthread.h header file as the first included file ensures the usage of thread-safe subroutines.

• Comment #B

  A Pthread’s life-cycle can be very short; it may terminate just after it is created. In this example, a delay loop commented as #B is necessary to demonstrate that created Pthreads run in parallel. If this loop is removed or the program is compiled with optimizing options, all the created Pthreads could terminate before other Pthreads could be created.

  The programmer must be aware of this volatile nature of Pthreads.

• Comment #C

  If a Pthread calls pthread_exit(), it terminates; this marks the end of Pthread’s life-cycle. Thread resources for the Pthread will be freed after the termination. However, other resources created by the terminating Pthread, such as file descriptors and sockets, will not be freed.

• Comment #D

  Most Pthreads library sub-routines return an integer value, which is interpreted as an error code. A value of zero indicates that the call was successful. Other values are passed or retrieved through appropriately typed arguments.

Also, the initial thread executing the main() function could have executed other functions after creating the other Pthreads. Therefore, there were six independent control flows in the process, as depicted in Figure 8-2 on page 285.

Compiling multi-threaded programs

To compile multi-threaded programs, use one of the compiler drivers with the _r suffix (see Table 1-4 on page 30). To compile the program shown in Example 8-5 on page 284, we used cc_r, as shown in the following example:

$ cc_r create_5threads.c

The created executable file, a.out, is a 32-bit multi-threaded program, as shown in the following:

$ file a.out
a.out:          executable (RISC System/6000) or object module not stripped
$ ldd a.out
a.out needs:
         /usr/lib/threads/libc.a(shr.o)
         /usr/lib/libpthreads.a(shr_comm.o)
         /usr/lib/libpthreads.a(shr_xpg5.o)
         /unix
         /usr/lib/libcrypt.a(shr.o)

Running multi-threaded programs

We ran this application on an AIX 5L Version 5.2 partition on the pSeries 690, which is assigned two processors:

# lsdev -Cc processor
proc0 Available 00-00 Processor
proc1 Available 00-01 Processor

At the first invocation of this program, it produced the following output:

$ a.out
Creating Pthread: 1
Creating Pthread: 2
Creating Pthread: 3
Hello world from Pthread 1!
Hello world from Pthread 2!
Hello world from Pthread 3!
Creating Pthread: 4
Creating Pthread: 5
Hello world from Pthread 4!
Hello world from Pthread 5!

At the second invocation of this program, it produced the following output:

$ a.out
Creating Pthread: 1
Creating Pthread: 2
Creating Pthread: 3
Creating Pthread: 4
Creating Pthread: 5
Hello world from Pthread 2!
Hello world from Pthread 4!
Hello world from Pthread 5!
Hello world from Pthread 1!
Hello world from Pthread 3!

Apparently, the execution order of Pthreads is different between these two outputs. In fact, it could be any order and you cannot predict which Pthread is scheduled first in a multi-threaded process.

Passing arguments to Pthreads

We discuss several aspects of the pthread_create() sub-routine in the following sections:

Syntax

#include <pthread.h>
int pthread_create (thread, attr, start_routine (void), arg)
pthread_t     *thread;
const         pthread_attr_t *attr;
void          **start_routine (void);
void          *arg;

Parameters

Return values

If successful, the pthread_create function returns zero. Otherwise, an error number is returned to indicate the error.

Error codes

The pthread_create function will fail if:

^[9] Work Load Manager

To pass multiple arguments the start_routine, create a structure that contains all of the arguments, and then pass a pointer to the structure as the last parameter of the pthread_create() routine.

For example, assuming that the following structure is defined:

struct thread_data {
   int  thread_id;
   char *msg;
} thread_data_array[NUM_OF_THREADS];

then pass the structure to the pthread_create() routine as follows:

pthread_create(&tid[t], NULL, thr_func, (void *)&thread_data_array[i]);

8.3.2. Joining Pthreads

The pthread_join() subroutine will block the calling Pthread until the Pthread you specify has terminated, and then, optionally, store the terminated Pthread’s return value. Calling pthread_join() detaches the specified Pthread automatically.

When a Pthread is created, one of its attributes defines whether it is joinable or detached. Detachable means that it cannot be joined. To explicitly create a Pthread as joinable or detached, the second argument in the pthread_create() routine is used. The pthread_detach() routine can be used to explicitly detach a Pthread even though it was created as joinable. The following example demonstrates how to wait for Pthread completions by using the pthread_join() routine. The Pthreads in this example are explicitly created in a joinable state so that they can be joined later:

/* File: pthread-join.c*/
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#define NUM_OF_THREADS 3
#define EXIT_CODE -1

void *Func_Join(void *t)
{
   printf("The argument is %d
", t);
   pthread_exit((void *) 0);
}

int main(int argc, char *argv[])
{
    pthread_t thread[NUM_OF_THREADS];
    pthread_attr_t attr;
    int rc, t, status;

    /* Initialize and set thread detached attribute */
    pthread_attr_init(&attr);
    /*Creates a thread as Joinable*/
    pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
    for (t = 0; t < NUM_OF_THREADS; t++) {
        printf("Creating thread %d
", (t+1));
        rc = pthread_create(&thread[t], &attr, Func_Join, (void*)t);
        if (rc) {
            fprintf(stderr, "pthread_create() failed with rc = %d.
", rc);
            exit(EXIT_CODE);
        }
    }

    /* Free attribute and wait for the other threads */
    pthread_attr_destroy(&attr);
    for (t = 0; t < NUM_OF_THREADS; t++) {
        rc = pthread_join(thread[t], (void **)&status);
        if (rc) {
            fprintf(stderr, "pthread_join() failed with rc = %d.
", rc);
            exit(EXIT_CODE);
        }
        printf("Joining the thread %d 
", (t+1));
    }
    pthread_exit(NULL);
}

When executed, the above program printed the following output on our system:

Creating thread 1
Creating thread 2
The argument is 0
Creating thread 3
The argument is 1
Joining the thread 1
The argument is 2
Joining the thread 2
Joining the thread 3

As shown in the example above, to explicitly create a Pthread as joinable, the following steps are followed:

The following factors need to be considered when deciding whether a Pthread has to be joined or not:

• If a Pthread requires joining, consider (as shown above) creating it as joinable. This provides portability, as not all implementations may create Pthreads as joinable by default.

• If you know in advance that a Pthread will never need to join with another Pthread, consider creating it in a detached state. Some system resources may be able to be freed.

8.3.3. Detaching a Pthread

We have seen how Pthreads can be joined using the pthread_join() function. In fact, Pthreads that are in a joinable state must be joined by other Pthreads, or else their memory resources will not be fully cleaned out. This is similar to what happens with processes whose parents did not clean up after them (also called orphan or zombie processes).

If we have a Pthread that we wish would exit whenever it wants without the need to join it, we should put it in the detached state. This can be done either with an thread attribute object with a PTHREAD_CREATE_DETACHED attached to the pthread_create() function, or by using the pthread_detach() function.

The pthread_detach() function gets one parameter, of type pthread_t, that denotes the Pthread we wish to put in the detached state. For example, we can create a Pthread and immediately detach it with code similar to the following example:

pthread_t a_thread;              /* store the thread's structure here.  */
int rc;                          /* return value for pthread functions. */
extern void* thread_loop(void*); /* declare the thread's main function. */

/* create a new thread. if succeeded, detach the newly created thread. */
if ((rc = pthread_create(&a_thread, NULL, thread_loop, NULL)) == 0) {
    rc = pthread_detach(a_thread);
}

By default, the Pthread that is created using the pthread_create() routine is joinable (the detach state is set to PTHREAD_CREATE_JOINABLE) and the scope is set to the process level (PTHREAD_SCOPE_PROCESS), which means that the user Pthread is not bound to a particular kernel thread. This is often referred to as the M:N thread model, where M is the number of user threads and N is the number of kernel threads.

8.3.4. Thread stack

When a Pthread is created, its own thread stack is also created. All auto storage class variables for the Pthread are allocated in its thread stack. When the Pthread terminates, all data allocated in its thread stack will be unallocated.

On AIX, thread stacks (except for the initial thread’s stack) are created in the process heap. The thread stack of the initial thread is created when the process is created.^[10]

^[10] For 32-bit process processes see 3.2.6, “Resource limits in 32-bit model” on page 125; for 64-bit processes, see 3.3.8, “Resource limits in 64-bit mode” on page 136.

By default, a thread stack size is 96 KB in the 32-bit user process model while it is 192 KB in 64-bit. The thread stack size can be tuned through either environment variables or through function calls, as explained in “Changing a thread stack size” on page 295. At the end of a thread stack, there is a 4 KB read/write protected page referred to as a guard page or red zone (see “AIXTHREAD_GUARDPAGES” on page 331 for further information about the guard page).

Figure 8-3 on page 293 illustrates how multiple thread stacks are allocated in the process heap in the 32-bit default memory model. The default thread stack size of 96 KB plus additional memory for the several necessary data structures will be consumed from the process heap every time a new Pthread is created. If a Pthread terminates, then the memory area for the terminated Pthread can be used again if another Pthread is created.

Because the creation of Pthreads consumes the process heap, depending on number of Pthreads in a process, you may need to consider the large or very large memory model,^[11] when developing 32-bit multi-threaded applications. When developing 64-bit multi-threaded applications, the data resource limit value only affects the maximum number of Pthreads within a process.^[12]

^[11] See 3.2.2, “Large memory model” on page 116 and 3.2.3, “Very large memory model” on page 117.

^[12] See 3.3.8, “Resource limits in 64-bit mode” on page 136.

To demonstrate how many Pthreads can be created in the 32-bit user process model, we have prepared a short example program, which is shown in Example 8-6 on page 294.

To compile the program, do the following:

$ cc_r -D_LARGE_THREADS max_threads.c

If the _LARGE_THREADS macro is defined, the PTHREAD_THREADS_MAX compilation time symbolic constant macro, which defines the maximum number of Pthreads per process, is set to 32767. If not defined, PTHREAD_THREADS_MAX is set to 512, which is a sufficient number for most multi-threaded applications, regardless of 32- or 64-bit applications.

When executed, the program would print the number of Pthreads in the angle brackets (highlighted in the example) within the process after failing to create another Pthread, as shown in the following example:^[13]

^[13] The return code 11 means EAGAIN (resource temporary unavailable).

$ a.out
[ 1131]: pthread_create() failed with rc=11 at 21 in max_threads.c
$ for i in 1 2 3 4 5 6 7 8
do
echo "LDR_CNTRL=MAXDATA=0x${i}0000000"
LDR_CNTRL=MAXDATA=0x${i}0000000 a.out
done
LDR_CNTRL=MAXDATA=0x10000000
[ 2281]: pthread_create() failed with rc=11 at 21 in max_threads.c
LDR_CNTRL=MAXDATA=0x20000000
[ 4583]: pthread_create() failed with rc=11 at 21 in max_threads.c
LDR_CNTRL=MAXDATA=0x30000000
[ 6884]: pthread_create() failed with rc=11 at 21 in max_threads.c
LDR_CNTRL=MAXDATA=0x40000000
[ 9186]: pthread_create() failed with rc=11 at 21 in max_threads.c
LDR_CNTRL=MAXDATA=0x50000000
[11268]: pthread_create() failed with rc=11 at 21 in max_threads.c
LDR_CNTRL=MAXDATA=0x60000000
[13788]: pthread_create() failed with rc=11 at 21 in max_threads.c
LDR_CNTRL=MAXDATA=0x70000000
[15523]: pthread_create() failed with rc=11 at 21 in max_threads.c
LDR_CNTRL=MAXDATA=0x80000000
[18391]: pthread_create() failed with rc=11 at 21 in max_threads.c

Example 8-6. max_threads.c

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

void *thread_func(void *j)
{
    sleep(60);
    pthread_exit(0);
}

int main()
{
    int i, rc;
    pthread_t tid[PTHREAD_THREADS_MAX];

    for (i = 0; i < PTHREAD_THREADS_MAX; i++) {
        if ((rc = pthread_create(&tid[i], NULL, thread_func, (void *)i)) != 0){
            fprintf(stderr
                , "[%5d]: pthread_create() failed with rc=%d at %d in %s
"
                , i, rc, __LINE__, __FILE__);
            exit(1);
        }
    }

    for (i = 0; i < PTHREAD_THREADS_MAX; i++) {
        pthread_join(tid[i], NULL);
    }
}

Changing a thread stack size

On AIX, the minimum thread stack size for a Pthread is 8 KB and the maximum size is 256 MB. These values are defined by the compilation time symbolic constants PTHREAD_STACK_MIN and PTHREAD_STACK_MAX, respectively.

Although the default stack sizes of 96 KB for 32-bit and 192 KB for 64-bit applications suffice for most multi-threaded applications’ demand, the stack size can be changed using the two methods described in the next two sections.

Setting the AIXTHREAD_STKSIZE environment value

If this environment value is set before the execution of a multi-threaded application, then the default thread stack size of the application process will be set to the value specified by the environment value (see “AIXTHREAD_STK” on page 332). This is a process-basis setting.

Using the pthread_attr_setstacksize() sub-routine

By creating a thread attribute with the specified thread size before the creation of a Pthread, the new Pthread will have the specified thread size. The following pseudo-code explains how to use this sub-routine:

This is a Pthread-basis setting and overrides the effect of the AIXTHREAD_STKSIZE environment value.