4.3. Use of MALLOCDEBUG options

This section explains the use of the MALLOCDEBUG environment variable by providing the following sections:

• Section 4.3.1, “MALLOCDEBUG with the debug memory allocator” on page 182

• Section 4.3.2, “MALLOCDEBUG with memory allocators other than debug” on page 190

4.3.1. MALLOCDEBUG with the debug memory allocator

The debug memory allocator supports the following options specified by the MALLOCDEBUG environment variable:

• “align:N” on page 183

• “validate_ptrs” on page 185

• “postfree_checking” on page 186

• “allow_overreading” on page 188

• “override_signal_handling” on page 188

• “record_allocations” on page 189

• “report_allocations” on page 189

align:N

By default, malloc() returns a pointer aligned on a 2-word^[3] boundary in the 32-bit and 4-word boundary in the 64-bit user process environment. The align:N option can be used to change the default alignment, where N is the number of bytes to be aligned and can be any power of 2 between 0 and 4096 inclusive (for example, 0, 1, 2, 4, ...). The values 0 and 1 are treated as the same, that is, there is no alignment, so any memory accesses outside the allocated area will cause an abort().

^[3] The term word means an implementation dependent unit of memory. On AIX, a word is 32 bits (4 bytes).

The following formula can be used to calculate how many bytes of over-reads or over-writes the debug memory allocator will allow for a given allocation request when MALLOCDEBUG=align:N and size is the number of bytes to be allocated:

((((size / N) + 1) * N) - size) % N

The example program shown in Example 4-2 demonstrates the effect of the align:N option. This program allocates a character string array size of 10 bytes, then prompts user to input some data, which will be stored in the previously allocated array.

Example 4-2. debug_malloc_align.c

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#define MAX_SIZE 10
#define EXIT_CODE -1

int main(int argc, char *argv[])
{
    char *ptr = (char *)NULL;
    char str[BUFSIZ];

    if ((ptr = (char *)malloc(MAX_SIZE)) == (char *)NULL) {
        perror("malloc() failed.
");
        exit(EXIT_CODE);
    }
    printf("Enter the value for ptr: ");
    gets(str);
    strcpy(ptr, str);
    printf("ptr points at : %p
", ptr);
    printf("The value stored in ptr is : %s
", ptr);
    free(ptr);
}

Before executing the program, set the following environment variables to enable the align:N option from the command prompt:

$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=align:2

Applying the above mentioned formula for align:2, the number of bytes of over-read or over-write allowed is:

Therefore, the debug memory allocator will not allow any over-reads or over-writes. If executed, the program would print the following output (the character string 12345678901 is user input):

$ a.out
Enter the value for ptr: 12345678901
Segmentation fault(coredump)

The program is terminated by a segmentation fault, because the length of ptr is 12 (11 printable characters plus the NULL-termination character ’0x0’).

If align:4 or align:8 is specified, the allowed over-read or over-write memory byte region size would be 2 or 6 bytes, as shown in the following calculation:

For example, if align:4 is specified, the same program prints the following output (no segmentation fault occurs):

$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=align:4
$ a.out
Enter the value for ptr: 12345678901
ptr points at : 20001ff4
The value stored in ptr is : 12345678901

The following points should be considered while setting the align:N option:

• For allocated space to be word aligned, specify align:N with a value of 4.

• If the align:N option is not explicitly set, it defaults to 8.

validate_ptrs

By default, free() does not validate its input pointer to ensure that it actually references memory previously allocated by malloc(). If the parameter passed to free() is a NULL value, free() will return to the caller without taking any action. If the parameter is invalid, the results will be undefined. A core dump may or may not occur in this case, depending upon the value of the invalid parameter. Specifying the validate_ptrs option will cause free() to perform extensive validation on its input parameter. If the parameter is found to be invalid (that is, it does not reference memory previously allocated by a call to malloc() or realloc()), debug malloc will print an error message stating why it is invalid. The abort() function is then called to terminate the process and produce a core file.

The example program shown in Example 4-3 demonstrates the effect of the validate_ptrs option. This is slightly modified from Example 4-2 on page 183 and calls the free() subroutine twice at the end of the program. Though the second call of free() is an error, it does not abort the execution of the program in normal situations.

Example 4-3. debug_malloc_vptr.c

/*This program is a slightly modified version of debug_mallo_align.c. We are
just trying to free the ptr memory even after it is freed by the first free()
call*/

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#define MAX_SIZE 10
#define EXIT_CODE -1

int main(int argc, char*argv[])
{
    char *ptr = NULL;
    char str[BUFSIZ];
    if ((ptr = (char *)malloc(MAX_SIZE)) == (char *)NULL) {
        perror("malloc() failed.
");
        exit(EXIT_CODE);
    }
    printf("Enter the value for ptr: ");
    gets(str);
    strcpy(ptr, str);
    printf("ptr points at : %p
", ptr);
    printf("The value stored in ptr is : %s
", ptr);
    free(ptr);
    free(ptr); /* This is invalid call. ptr is already freed. */
}

Before executing the program, set the following environment variables to enable the validate_ptr option from the command prompt:

$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=validate_ptrs

If executed, the program would print the following output (the character string 1234567890 is user input):

$ a.out
Enter the value for ptr: 1234567890
ptr points at : 20001ff0
The value stored in ptr is : 1234567890
Debug Malloc: Buffer (0x20001ff0) has already been free'd.
IOT/Abort trap(coredump)

As highlighted in the output, the debug memory allocator has detected the invalid second free() call.

postfree_checking

By default, the malloc subsystem allows the calling program to access memory that has previously been freed. This should result in an error in the calling program. If the postfree_checking option is specified, any attempt to access memory after it is freed will cause the debug memory allocator to report the error and abort the program; then a core file will be produced.

If the same program shown in Example 4-3 on page 185 is executed with the postfree_checking by setting the following environment variables:

$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=postfree_checking

then it will result in a segmentation fault, though the reason for that is not clearly reported in the following output:

$ a.out
Enter the value for ptr: 1234567890
ptr points at : 20001ff0
The value stored in ptr is : 1234567890
Segmentation fault(coredump)

The postfree_checking option identifies the access (if any) to the memory after it is freed, but the validate_ptrs option does not. The example program shown Example 4-4 on page 187 illustrates the difference between the postfree_checking and validate_ptrs options. This program is trying to access the ptr memory after it is freed by the free() call.

Example 4-4. debug_malloc_pfc.c

#include <stdio.h>
#include <stdlib.h>
#include <errono.h>
#define MAX_SIZE 10
#define EXIT_CODE -1

int main(int argc, char *argv[])
{
    char *ptr = (char *)NULL;
    char str[BUFSIZ];
    if ((ptr = (char *)malloc(MAX_SIZE)) == (char *)NULL) {
        perror("malloc() failed.
");
        exit(EXIT_CODE);
    }
    printf("Enter the value for ptr: ");
    gets(str);
    strcpy(ptr, str);
    printf("ptr points at : %p
", ptr);
    printf("The value stored in ptr is : %s
", ptr);
    free(ptr);
    /* Wrong. trying to access memory after it is freed. */
    printf("The value stored in ptr is : %s
", ptr);
}

Before executing the program, set the following environment variables to enable the validate_ptr option from the command prompt:

$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=validate_ptrs

If executed, the program would print the following output (the highlighted character string 12345 is a user input):

$ a.out
Enter the value for ptr: 12345
ptr points at : 20001ff0
The value stored in ptr is : 12345

Apparently the debug memory allocator with the validate_ptr option did not detect the error. The reason is that the validate_ptrs option checks for the validity of ptr only when it is passed to a free() function call.

If the same program is executed with the validate_ptrs option, the program would print the following output (the highlighted character string 12345 is a user input):

$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=postfree_checking
$ a.out
Enter the value for ptr: 12345
ptr points at : 20001ff0
The value stored in ptr is : 12345
Segmentation fault(coredump)
						

With the postfree_checking option set, the debug memory allocator identifies the erroneous access to the memory after it is freed.

allow_overreading

By default, the debug memory allocator will respond with a segmentation violation and if the program attempts to read past the end of allocated memory. The allow_overreading option instructs the debug memory allocator to ignore over-reads of this nature so that other types of errors, which may be considered more serious, can be detected first.

override_signal_handling

The debug memory allocator reports errors in one of two ways:

• Memory access errors (such as trying to read or write past the end of allocated memory) will cause a segmentation violation (SIGSEGV), resulting in a core dump.

• For other types of errors (such as trying to free space that was already freed), the debug memory allocator will print an error message, then call abort(), which will send a SIGIOT signal to terminate the current process.

If the calling program is blocking or catching the SIGSEGV and/or the SIGIOT signals, the debug memory allocator will be prevented from reporting errors. The override_signal_handling option provides a means of addressing this situation without recording and rebuilding the application.

If the override_signal_handling option is specified, the debug memory allocator will perform the following actions upon each call to one of the memory allocation routines (malloc(), free(), realloc(), or calloc()):

When using the override_signal_handling option, keep in mind the following:

• If an application signal handler modifies the action for SIGSEGV between memory allocation routine calls and then attempts an invalid memory access, the debug memory allocator will be unable to report the error (the application will not exit and no core file will be produced).

• The override_signal_handling option may be ineffective in a multi-threaded application environment because the debug memory allocator uses sigprocmask() and many multi-threaded processes use pthread_sigmask().

• If a user thread calls sigwait() without including SIGSEGV and SIGIOT in the signal set and the debug memory allocator subsequently detects an error, the user thread will hang because the allocator can only generate SIGSEGV or SIGIOT.

record_allocations

The record_allocations option instructs the debug memory allocator to create an allocation record for each malloc() request. Each record contains the following information:

• The original address returned to the caller from malloc().

• Up to six function trace backs starting from the call to malloc().

Each allocation record will be retained until the memory associated with it is freed.

report_allocations

The report_allocations option instructs the debug memory allocator to report all active allocation records at application exit. An active allocation record will be listed for any memory allocation that was not freed prior to application exit.

To demonstrate how the report_allocations works, we have slightly modified the example program shown in Example 4-3 on page 185 by removing two free() lines (highlighted in the example).

Before executing the program, set the following environment variables to enable the validate_ptr option from the command prompt:

$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=report_allocations

If executed, the program would print the following output (the highlighted character string 12345678901 is a user input):

$ a.out
Enter the value for ptr: 12345678901
ptr points at : 20003ff0
The value stored in ptr is : 12345678901
Current allocation report:
    Allocation #1: 0x20003FF0
        Allocation size: 0xA
        Allocation traceback:
        0x100001B4 __start
        0x1000035C main
        0xD01D7104 malloc

    Allocation #2: 0x20001FF0
        Allocation size: 0x10
        Allocation traceback:
        0x1000035C  main
        0xD01D7104  malloc
        0xD01D6C28  init_malloc
        0xD01D6120  check_environment
        0xD022BA10  malloc_debug_start
        0xD01E42D4  atexit

Total allocations: 2.

The output contains the allocation report (Allocation #1) for the un-freed memory in the program (the memory address printed for Allocation #1 is same as the address location of ptr: 0x20003FF0). Because this memory has not been freed, the debug memory allocator detects it and then prints the allocation report.

4.3.2. MALLOCDEBUG with memory allocators other than debug

The options shown in Table 4-4 on page 191, which can be specified by the MALLOCDEBUG environment value, are supported by 3.1, default, and default with malloc buckets extension memory allocators:

^[a] The arena option is not supported by the 3.1 memory allocator.

Multiple options can be specified by separating them using a comma as follows:

MALLOCDEBUG=option1,option2,...

verbose

The verbose option instructs memory allocators that information on errors that occurred in the malloc subsystem will be reported and actions can be performed if specified.

The verbose option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:

MALLOCDEBUG=verbose

All errors caught in the malloc subsystem are output to standard error, along with detailed information.

The verbose option allows the user to provide a function that the malloc subsystem will call when it encounters an error. Before returning, the malloc subsystem calls the user-provided function, if it is specified.

A global function pointer is available for use by the user. In the code, the following function pointer should be set to the user’s function:

extern void (*malloc_err_function)(int, ...)

The following user-defined function must be implemented:

void malloc_err_function(int, ...)

For example, to use the user-defined function abort_sub, the following code must be inserted into the user’s application:

malloc_err_function = &abort_sub;

A sample program shown in Example 4-5 illustrates the use of the verbose option. When executed, the program prints the following error message:^[4]

^[4] Yorktown is an internal name used in the AIX development to refer to the default memory allocator.

$ MALLOCDEBUG=verbose a.out
Malloc Report: Corruption in the Yorktown Malloc arena has been detected.
IOT/Abort trap(coredump)

The default memory allocator with the verbose option has detected the corruption in this output.

Example 4-5. debug_malloc_verbose.c

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <malloc.h>

int main(int argc, char *argv[])
{
    char *ptr;
    char buf[BUFSIZ];

    if ((ptr = (char *)malloc(1200)) == (char *)NULL) {
        sprintf(buf, "malloc() failed at %d in %s with errno = %d"
            , __LINE__, __FILE__, errno);
        perror(buf);
    }
    free(ptr);
    memset(ptr - 8, (char)-1, 40);
    free(ptr - 4096);

    exit(0);
}

arena

The arena option instructs the malloc subsystem to check the structures that contain the free blocks before every allocation request is processed. This option will ensure that the arena is not corrupted. Also, the arena will also be checked when an error occurs.

The checkarena option checks for NULL pointers in the free tree or pointers that do not fall within a certain range. If an invalid pointer is encountered during the descent of the tree, the program might perform a core dump depending on the value of the invalid address.

The arena option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:

MALLOCDEBUG=checkarena

A sample program shown in Example 4-6 illustrates the use of the arena option. When executed, the program prints the following error message:

$ MALLOCDEBUG=verbose,arena a.out
Malloc Report: The address passed to free, 0x1ffff5f8, is outside the valid
range of addresses allocated by malloc (errno = 0).
IOT/Abort trap(coredump)

Example 4-6. debug_malloc_arena.c

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <malloc.h>

int main(int argc, char *argv[])
{
    char *ptr;
    char buf[BUFSIZ];

    if ((ptr = (char *)malloc(16)) == (char *)NULL) {
        sprintf(buf, "malloc() failed at %d in %s with errno = %d"
            , __LINE__, __FILE__, errno);
        perror(buf);
        exit(1);
    }
    free(ptr-4096);

    exit(0);
}

trace

The trace option instructs the malloc subsystem to use the trace facility. Traces of the malloc(), realloc(), and free() subroutines are recorded for use in problem determination and performance analysis.

The trace option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:

MALLOCDEBUG=trace

The trace option supports the following trace hook IDs:

• HKWD_LIB_MALL_COMMON (hook ID: 60a)

  When tracing is enabled for HKWD_LIB_MALL_COMMON, the input parameters, as well as return values for each call to malloc(), realloc(), and free() subroutines, are recorded in the trace subsystem. In addition to providing trace information about the malloc subsystem, the trace option also performs checks of its internal data structures. If these structures have been corrupted, these checks will likely detect the corruption and provide temporal data, which is useful in problem determination.

• HKWD_LIB_MALL_INTERNAL (hook ID: 60b)

  When tracing is enabled for HKWD_LIB_MALL_INTERNAL and corruption is detected, information about the internal data structures are logged through the trace subsystem.

To use the trace option, do the following:

# trace -j'60a,60b' -a

                               START Trace

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
                                                 [Entry Fields]
  EVENT GROUPS to trace                        []                       +
  ADDITIONAL event IDs to trace                []                       +
  Event Groups to EXCLUDE from trace           []                       +
  Event IDs to EXCLUDE from trace              []                       +
  Trace MODE                                   [alternate]              +
  STOP when log file full?                     [no]                     +
  LOG FILE                                     [/var/adm/ras/trcfile]
  SAVE PREVIOUS log file?                      [no]                     +
  Omit PS/NM/LOCK HEADER to log file?          [yes]                    +
  Omit DATE-SYSTEM HEADER to log file?         [no]                     +
  Run in INTERACTIVE mode?                     [no]                     +
  Trace BUFFER SIZE in bytes                   [131072]                  #
  LOG FILE SIZE in bytes                       [1310720]                 #
  Buffer Allocation                            [automatic]              +

F1=Help             F2=Refresh       F3=Cancel        F4=List
F5=Reset            F6=Command       F7=Edit          F8=Image
F9=Shell            F10=Exit         Enter=Do

# trcstop

# trcrpt -0 exec=y -0 pid=y -0 tid=y -0 svc=y -0 timestamp=1

Example 4-8. Trace output of the trace option

Fri Feb 28 14:29:20 2003
System: AIX 5.2 Node: murumuru
Machine: 000C91AD4C00
Internet Address: 0903046A 9.3.4.106
The system contains 2 cpus, of which 2 were traced.
Buffering: Kernel Heap
This is from a 32-bit kernel.
Tracing only these hooks, 60a,60b


/usr/bin/trace -j60a 60b -a


ID  PROCESS NAME   I SYSTEM CALL    ELAPSED   APPL     SYSCALL KERNEL  INTERRUPT

001 --1-                           0.000000                    TRACE ON channel 0
                                                               Fri Feb 28 14:29:20 2003
60A --1-                           5.561069   HKWD_LIBC_MALL_COMMON
                                              function=malloc() [Default Allocator]
                                              size=000A
                                              returnptr=20000728
60A --1-                           5.561261   HKWD_LIBC_MALL_COMMON
                                              function=free() [Default Allocator]
                                              inptr=20000728
60A --1-                          10.131345   HKWD_LIBC_MALL_COMMON
                                              function=malloc() [Default Allocator]
                                              size=0290
                                              returnptr=20005878

log

The log option instructs the malloc subsystem to record information on the number of active allocations of a given size and stack trace back of a user program. This data can be used in problem determination and performance analysis, if the user program is modified accordingly.

Data recorded with the log option

If the log option is enabled, the following data is recorded for each malloc or realloc subroutine invocation:

• The size of the allocation.

• The stack trace back of the invocation. The depth of the trace back that is recorded is a configurable option.

• The number of currently active allocations that match the size and stack trace back.

The data is stored into the following global structure:^[5]

^[5] This is included in the /usr/include/malloc.h header file.

struct malloc_log *malloc_log_table;
#ifndef MALLOC_LOG_STACKDEPTH
#define MALLOC_LOG_STACKDEPTH 4
#endif
    struct malloc_log {
    size_t size;
    size_t cnt;
    uintptr_t callers [MALLOC_LOG_STACKDEPTH];
}
size_t malloc_log_size;

The size of the malloc_log structure can change. If the default call-stack depth is greater than 4, the structure will have a larger size. The current size of the malloc_log structure is stored in the globally exported malloc_log_size variable. A user can define the MALLOC_LOG_STACKDEPTH macro to the stack depth that was configured at process start time.

The malloc_log_table can be accessed in the following ways:

• Using the get_malloc_log() sub-routine as follows:

  #include <malloc.h>
  size_t get_malloc_log (void *addr,void *buf,size_t bufsize);
  

  This function copies the data from malloc_log_table into the provided buffer. The data can then be accessed without modifying the malloc_log_table. The data represents a snapshot of the malloc log data for that moment of time.

• Using the get_malloc_log_live() sub-routine as follows:

  #include <malloc.h>
  struct malloc_log *get_malloc_log_live (void *addr);
  

  The advantage of this method is that no data needs to be copied, therefore performance suffers less. Disadvantages of this method are that the data referenced is volatile and the data may not encompass the entire malloc subsystem, depending on which malloc algorithm is being used.

To clear all existing data from the malloc log tables, use the reset_malloc_log() subroutine as follows:

#include malloc.h
void reset_malloc_log(void *addr);

The sample program shown in Example 4-9 on page 198 demonstrates the use of the get_malloc_log_live() sub-routine. After it is compiled, the program would print the following output, if the log option is enabled:

$ cc get_malloc_live.c
$ export MALLOCTYPE=
$ export MALLOCDEBUG=log
$ a.out
i is 1217.
The size of the allocations is 8.
The number of matching allocations is 1.
i is 1241.
The size of the allocations is 16.
The number of matching allocations is 1.
i is 3293.
The size of the allocations is 8.
The number of matching allocations is 1.

cc: 1501-230 Internal compiler error; please contact your Service
Representative.

The program has a for loop to dump the content of log_ptr, if there are referenced slots in the data structure referenced by log_ptr.

Example 4-9. get_malloc_log_live.c

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <malloc.h>

int main(int argc, char *argv[])
{
    char *ptr1 = (char *)NULL, *ptr2 = (char *)NULL, *ptr3 = (char *)NULL;
    int i;

    struct malloc_log *log_ptr;

    ptr1 = (char *)malloc(8);
    ptr2 = (char *)malloc(16);
    ptr3 = (char *)malloc(8);

    log_ptr = get_malloc_log_live(ptr1);

    for (i = 0; i < DEFAULT_RECS_PER_HEAP; i++) {
        if (log_ptr->cnt > 0) {
            printf("i is %d.
", i);
            printf("The size of the allocations is %d.
", log_ptr->size);
            printf("The number of matching allocations is %d.
"
                , log_ptr->cnt);
        }
        log_ptr++;
    }
    exit(0);
}

Enabling malloc log

The log option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:

MALLOCDEBUG=log

To enable the trace option with user-specified configuration options, set the MALLOCDEBUG environment variable as follows:

MALLOCDEBUG=log:records_per_heap:stack_depth
							

The predefined MALLOCDEBUG configuration options include the following:

Limitations

The performance of all programs can degrade when the trace option is enabled, due to the cost of storing data to memory. Memory usage will also increase.