8.8. Pthread specific environment variables in AIX
Tuning multi-threaded applications for optimal performance can be challenging on any system. AIX provides a set of environmental variables that modify the behavior of various threading functions. Let us look at some of these variables one at a time.
AIXTHREAD_SCOPE
This environment variable controls the mapping of the application-level Pthreads to entries in the OS scheduling queue. The default thread scope for AIX is process, and this applies to the creation of Pthreads only when the attribute argument to pthread_create() is NULL; otherwise, the contents of the attribute structure determines the scope of the Pthread. This implies that a single process may contain Pthreads of both scopes.
AIXTHREAD_SCOPE permits exploring application behavior by being able to change the thread scope without having to modify the application. The default thread scope can be set by setting the variable in the environment:
AIXTHREAD_SCOPE=[P|S]
Where:
| P | For process scope |
| S | For system scope |
The preferred setting for any particular application is dependent upon its behavior. The performance difference between process and system scope, in current levels of AIX 5L, has become almost negligible for certain applications.
AIXTHREAD_MNRATIO
This environment variable controls the scaling factor used within the Pthread library when creating and terminating Pthreads. The default M:N ratio for current levels of the operating system is 8:1, that is, eight Pthreads for every kernel thread. This default ratio may not be suitable for all workloads, however, so the user can modify the M:N ratio by setting and exporting the environmental variable as follows:
AIXTHREAD_MNRATIO=M:N
Where:
| M | Number of user threads |
| N | Number of kernel threads |
AIXTHREAD_MNRATIO applies only to process-scope threads. It may be useful for applications with a very large number of Pthreads.
SPINLOOPTIME
SPINLOOPTIME controls the number of times a Pthread will attempt to obtain a mutex before blocking on the mutex availability. For example, sometimes it makes more sense for a Pthread to keep trying just a bit longer to acquire a lock in hopes of the other Pthread releasing the lock while the current Pthread is awake and waiting. This can avoid the overhead of putting the Pthread to sleep, managing the event that occurs when the lock is released, and waking the Pthread up and setting it to run so that it can now acquire the lock.
On multiprocessor systems the default value for SPINLOOPTIME is 40. The spin loop time can be modified by setting and exporting the environmental variable:
SPINLOOPTIME=N
Where:
| N | Number of times to retry a busy lock before yielding to another Pthread. |
YIELDLOOPTIME
Similar to SPINLOOPTIME, YIELDLOOPTIME controls the number of times a Pthread will yield itself before blocking on a mutex. In some cases, yielding the processor will permit other Pthreads time to run, thus reducing the amount of time the waiting Pthread will have to spin and/or wait for the mutex to be unlocked. This allows work to be accomplished while one Pthread is waiting for a resource to be available. It also keeps the Pthread from going to sleep; instead, the Pthread is put back on the run queue to try again to acquire the lock. Keeping the Pthread from going to sleep can result in improved overall performance, depending upon how much more time the Pthread needs to wait for the lock acquisition.
The default for YIELDLOOPTIME is 0. If the Pthread spins on the lock and cannot get it, it will be put to sleep. Increasing the value of this variable will prevent the Pthread from going to sleep quite so quickly. We can modify the value of yield loop time by setting and exporting the environmental variable:
AIXTHREAD_YIELDLOOPTIME=N
Where:
| N | Number of times to yield to acquire a busy mutex or spin lock |
AIXTHREAD_MINKTHREADS
Sets the minimum number of kernel threads that should be used for a process. The default is eight, which means that an average threaded process will have at least eight kernel threads available as a resource for scheduling Pthreads. This environment variable can be set as follows:
AIXTHREAD_MINKTHREADS=N
Where:
| N | Number of kernel threads |
AIXTHREAD_MUTEX_DEBUG
AIX is capable of maintaining, in running threaded processes, information regarding mutexes; this information is then available for application debugging. On current levels of AIX 5L, the default setting for this variable is OFF, but that was not always the case. Since maintaining this information is not free, we may observe unexplainable and undesirable performance characteristics. If we suspect that our application may be impacted, set this variable in our environment to ensure mutex debugging information is not being collected:
AIXTHREAD_MUTEX_DEBUG=[ON|OFF]
If you plan on debugging the Pthreads in your application, you will want to turn this variable ON.
AIXTHREAD_COND_DEBUG
Similar to AIXTHREAD_MUTEX_DEBUG, this variable controls the collection of debug information for condition variables. Its default setting is now OFF. For the same conditions described above, we can set the variable in our environment:
AIXTHREAD_COND_DEBUG=[ON|OFF]
AIXTHREAD_RWLOCK_DEBUG
Causes the Pthreads library to maintain a list of all read-write locks that can be viewed by debugging programs. Like the other two debugging variables above, the default for this is now OFF. Set the variable in your environment to ON for debugging the application.
AIXTHREAD_RWLOCK_DEBUG=[ON|OFF]
AIXTHREAD_GUARDPAGES
Each Pthread is created with its own stack; the stack size is controlled via the thread attributes or via (for a default value) an environment variable. Since the stack is allocated in the process heap, there is some potential for inadequately sized stacks to allow a Pthread to cause a stack to grow beyond its maximum size. To assist in detecting this behavior, and also to guard against errant memory writes, the stack can be protected by setting pages at the top of the stack as read-only. Any attempt to write onto these pages will result in the application receiving a segmentation violation signal, which will occur immediately. Debugging the application will allow us to investigate the conditions at the time of the stack overflow and decide on an appropriate course of action (increase the stack size, redesign part of the application, change where the data is stored, and so on).
To enable and set the number of guard pages, we need to export the environmental variable:
AIXTHREAD_GUARDPAGES=N
where:
| N | Number of 4 KB size pages |
The default value is 0, which means no guard pages are created. If the application specifies its own stack, or uses large pages[17] for its process heap, no guard pages are created.
AIXTHREAD_SLPRATIO
The “sleep ratio” value tells the system how many kernel threads should be held in reserve for sleeping Pthreads. This tuning parameter allows greater management of kernel resources, as sleeping Pthreads do not require a matching kernel thread. And since Pthreads are (usually) woken one at a time, the scheduling and running of the Pthreads can be matched to existing resources without needing to acquire additional kernel threads.
One of the control variables that impact the scheduling of Pthreads is created with a process based contention scope. This environment variable will be set as follows:
AIXTHREAD_SLPRATIO=k:p
Where:
| k | The number of kernel threads that should be held in reserve for p sleeping pthreads |
Any positive integer value may be specified for p and k. If k > p, then the ratio is treated as 1:1 (that is, you cannot specify more kernel threads than Pthreads). The default sleep ratio is 1:12.
AIXTHREAD_STK
Each Pthread has its own stack; the default size (on current levels of AIX) is 96 KB for 32-bit applications and 192 KS for 64-bit applications. While we will find this default adequate for many applications, it is also true that there are exceptions. Instead of modifying code to adjust a parameter in the Pthread attribute structure and then recompiling and rebuilding our application, we can use this environment variable to specify stacks of up to 256MB in size. Keep in mind, however, that Pthreads stacks are created in the process heap, and thus impact the amount of space available for dynamically allocated memory.
We can modify the stack size for Pthreads created without specifying the stack size programmatically by exporting the environment variable:
AIXTHREAD_STK=N
Where:
| N | number of bytes |