This section explains the use of the MALLOCDEBUG environment variable by providing the following sections:
Section 4.3.1, “MALLOCDEBUG with the debug memory allocator” on page 182
Section 4.3.2, “MALLOCDEBUG with memory allocators other than debug” on page 190
The debug memory allocator supports the following options specified by the MALLOCDEBUG environment variable:
“align:N” on page 183
“validate_ptrs” on page 185
“postfree_checking” on page 186
“allow_overreading” on page 188
“override_signal_handling” on page 188
“record_allocations” on page 189
“report_allocations” on page 189
Note
The debug memory allocator does not support the MALLOCDEBUG options explained in 4.3.2, “MALLOCDEBUG with memory allocators other than debug” on page 190.
By default, malloc() returns a pointer aligned on a 2-word[3] boundary in the 32-bit and 4-word boundary in the 64-bit user process environment. The align:N option can be used to change the default alignment, where N is the number of bytes to be aligned and can be any power of 2 between 0 and 4096 inclusive (for example, 0, 1, 2, 4, ...). The values 0 and 1 are treated as the same, that is, there is no alignment, so any memory accesses outside the allocated area will cause an abort().
[3] The term word means an implementation dependent unit of memory. On AIX, a word is 32 bits (4 bytes).
The following formula can be used to calculate how many bytes of over-reads or over-writes the debug memory allocator will allow for a given allocation request when MALLOCDEBUG=align:N and size is the number of bytes to be allocated:
((((size / N) + 1) * N) - size) % N
The example program shown in Example 4-2 demonstrates the effect of the align:N option. This program allocates a character string array size of 10 bytes, then prompts user to input some data, which will be stored in the previously allocated array.
#include <stdio.h> #include <stdlib.h> #include <errno.h> #define MAX_SIZE 10 #define EXIT_CODE -1 int main(int argc, char *argv[]) { char *ptr = (char *)NULL; char str[BUFSIZ]; if ((ptr = (char *)malloc(MAX_SIZE)) == (char *)NULL) { perror("malloc() failed. "); exit(EXIT_CODE); } printf("Enter the value for ptr: "); gets(str); strcpy(ptr, str); printf("ptr points at : %p ", ptr); printf("The value stored in ptr is : %s ", ptr); free(ptr); } |
Before executing the program, set the following environment variables to enable the align:N option from the command prompt:
$ export MALLOCTYPE=debug $ export MALLOCDEBUG=align:2
Applying the above mentioned formula for align:2, the number of bytes of over-read or over-write allowed is:
align:2 | ((((10/2) + 1) * 2) - 10) % 2 = 0 |
Therefore, the debug memory allocator will not allow any over-reads or over-writes. If executed, the program would print the following output (the character string 12345678901 is user input):
$ a.out
Enter the value for ptr: 12345678901
Segmentation fault(coredump)
The program is terminated by a segmentation fault, because the length of ptr is 12 (11 printable characters plus the NULL-termination character ’0x0’).
If align:4 or align:8 is specified, the allowed over-read or over-write memory byte region size would be 2 or 6 bytes, as shown in the following calculation:
align:4 | ((((10/4) + 1) * 4) - 10) % 4 = 2 |
align:8 | ((((10/8) + 1) * 8) - 10) % 8 = 6 |
For example, if align:4 is specified, the same program prints the following output (no segmentation fault occurs):
$ export MALLOCTYPE=debug
$ export MALLOCDEBUG=align:4
$ a.out
Enter the value for ptr: 12345678901
ptr points at : 20001ff4
The value stored in ptr is : 12345678901
The following points should be considered while setting the align:N option:
For allocated space to be word aligned, specify align:N with a value of 4.
If the align:N option is not explicitly set, it defaults to 8.
By default, free() does not validate its input pointer to ensure that it actually references memory previously allocated by malloc(). If the parameter passed to free() is a NULL value, free() will return to the caller without taking any action. If the parameter is invalid, the results will be undefined. A core dump may or may not occur in this case, depending upon the value of the invalid parameter. Specifying the validate_ptrs option will cause free() to perform extensive validation on its input parameter. If the parameter is found to be invalid (that is, it does not reference memory previously allocated by a call to malloc() or realloc()), debug malloc will print an error message stating why it is invalid. The abort() function is then called to terminate the process and produce a core file.
The example program shown in Example 4-3 demonstrates the effect of the validate_ptrs option. This is slightly modified from Example 4-2 on page 183 and calls the free() subroutine twice at the end of the program. Though the second call of free() is an error, it does not abort the execution of the program in normal situations.
/*This program is a slightly modified version of debug_mallo_align.c. We are just trying to free the ptr memory even after it is freed by the first free() call*/ #include <stdio.h> #include <stdlib.h> #include <errno.h> #define MAX_SIZE 10 #define EXIT_CODE -1 int main(int argc, char*argv[]) { char *ptr = NULL; char str[BUFSIZ]; if ((ptr = (char *)malloc(MAX_SIZE)) == (char *)NULL) { perror("malloc() failed. "); exit(EXIT_CODE); } printf("Enter the value for ptr: "); gets(str); strcpy(ptr, str); printf("ptr points at : %p ", ptr); printf("The value stored in ptr is : %s ", ptr); free(ptr); free(ptr); /* This is invalid call. ptr is already freed. */ } |
Before executing the program, set the following environment variables to enable the validate_ptr option from the command prompt:
$ export MALLOCTYPE=debug $ export MALLOCDEBUG=validate_ptrs
If executed, the program would print the following output (the character string 1234567890 is user input):
$ a.out Enter the value for ptr: 1234567890 ptr points at : 20001ff0 The value stored in ptr is : 1234567890 Debug Malloc: Buffer (0x20001ff0) has already been free'd. IOT/Abort trap(coredump)
As highlighted in the output, the debug memory allocator has detected the invalid second free() call.
By default, the malloc subsystem allows the calling program to access memory that has previously been freed. This should result in an error in the calling program. If the postfree_checking option is specified, any attempt to access memory after it is freed will cause the debug memory allocator to report the error and abort the program; then a core file will be produced.
Note
Specifying the postfree_checking option automatically enables the validate_ptrs option.
If the same program shown in Example 4-3 on page 185 is executed with the postfree_checking by setting the following environment variables:
$ export MALLOCTYPE=debug $ export MALLOCDEBUG=postfree_checking
then it will result in a segmentation fault, though the reason for that is not clearly reported in the following output:
$ a.out
Enter the value for ptr: 1234567890
ptr points at : 20001ff0
The value stored in ptr is : 1234567890
Segmentation fault(coredump)
The postfree_checking option identifies the access (if any) to the memory after it is freed, but the validate_ptrs option does not. The example program shown Example 4-4 on page 187 illustrates the difference between the postfree_checking and validate_ptrs options. This program is trying to access the ptr memory after it is freed by the free() call.
#include <stdio.h> #include <stdlib.h> #include <errono.h> #define MAX_SIZE 10 #define EXIT_CODE -1 int main(int argc, char *argv[]) { char *ptr = (char *)NULL; char str[BUFSIZ]; if ((ptr = (char *)malloc(MAX_SIZE)) == (char *)NULL) { perror("malloc() failed. "); exit(EXIT_CODE); } printf("Enter the value for ptr: "); gets(str); strcpy(ptr, str); printf("ptr points at : %p ", ptr); printf("The value stored in ptr is : %s ", ptr); free(ptr); /* Wrong. trying to access memory after it is freed. */ printf("The value stored in ptr is : %s ", ptr); } |
Before executing the program, set the following environment variables to enable the validate_ptr option from the command prompt:
$ export MALLOCTYPE=debug $ export MALLOCDEBUG=validate_ptrs
If executed, the program would print the following output (the highlighted character string 12345 is a user input):
$ a.out
Enter the value for ptr: 12345
ptr points at : 20001ff0
The value stored in ptr is : 12345
Apparently the debug memory allocator with the validate_ptr option did not detect the error. The reason is that the validate_ptrs option checks for the validity of ptr only when it is passed to a free() function call.
If the same program is executed with the validate_ptrs option, the program would print the following output (the highlighted character string 12345 is a user input):
$ export MALLOCTYPE=debug $ export MALLOCDEBUG=postfree_checking $ a.out Enter the value for ptr: 12345 ptr points at : 20001ff0 The value stored in ptr is : 12345 Segmentation fault(coredump)
With the postfree_checking option set, the debug memory allocator identifies the erroneous access to the memory after it is freed.
By default, the debug memory allocator will respond with a segmentation violation and if the program attempts to read past the end of allocated memory. The allow_overreading option instructs the debug memory allocator to ignore over-reads of this nature so that other types of errors, which may be considered more serious, can be detected first.
The debug memory allocator reports errors in one of two ways:
Memory access errors (such as trying to read or write past the end of allocated memory) will cause a segmentation violation (SIGSEGV), resulting in a core dump.
For other types of errors (such as trying to free space that was already freed), the debug memory allocator will print an error message, then call abort(), which will send a SIGIOT signal to terminate the current process.
If the calling program is blocking or catching the SIGSEGV and/or the SIGIOT signals, the debug memory allocator will be prevented from reporting errors. The override_signal_handling option provides a means of addressing this situation without recording and rebuilding the application.
If the override_signal_handling option is specified, the debug memory allocator will perform the following actions upon each call to one of the memory allocation routines (malloc(), free(), realloc(), or calloc()):
Disables any existing signal handlers set up by the application.
Sets the action for both SIGIOT and SIGSEGV to the default (SIG_DFL).
Unblocks both SIGIOT and SIGSEGV.
When using the override_signal_handling option, keep in mind the following:
If an application signal handler modifies the action for SIGSEGV between memory allocation routine calls and then attempts an invalid memory access, the debug memory allocator will be unable to report the error (the application will not exit and no core file will be produced).
The override_signal_handling option may be ineffective in a multi-threaded application environment because the debug memory allocator uses sigprocmask() and many multi-threaded processes use pthread_sigmask().
If a user thread calls sigwait() without including SIGSEGV and SIGIOT in the signal set and the debug memory allocator subsequently detects an error, the user thread will hang because the allocator can only generate SIGSEGV or SIGIOT.
The record_allocations option instructs the debug memory allocator to create an allocation record for each malloc() request. Each record contains the following information:
The original address returned to the caller from malloc().
Up to six function trace backs starting from the call to malloc().
Each allocation record will be retained until the memory associated with it is freed.
The report_allocations option instructs the debug memory allocator to report all active allocation records at application exit. An active allocation record will be listed for any memory allocation that was not freed prior to application exit.
Note
Specifying the report_allocations option automatically enables the record_allocations option.
To demonstrate how the report_allocations works, we have slightly modified the example program shown in Example 4-3 on page 185 by removing two free() lines (highlighted in the example).
Before executing the program, set the following environment variables to enable the validate_ptr option from the command prompt:
$ export MALLOCTYPE=debug $ export MALLOCDEBUG=report_allocations
If executed, the program would print the following output (the highlighted character string 12345678901 is a user input):
$ a.out Enter the value for ptr: 12345678901 ptr points at : 20003ff0 The value stored in ptr is : 12345678901 Current allocation report: Allocation #1: 0x20003FF0 Allocation size: 0xA Allocation traceback: 0x100001B4 __start 0x1000035C main 0xD01D7104 malloc Allocation #2: 0x20001FF0 Allocation size: 0x10 Allocation traceback: 0x1000035C main 0xD01D7104 malloc 0xD01D6C28 init_malloc 0xD01D6120 check_environment 0xD022BA10 malloc_debug_start 0xD01E42D4 atexit Total allocations: 2.
The output contains the allocation report (Allocation #1) for the un-freed memory in the program (the memory address printed for Allocation #1 is same as the address location of ptr: 0x20003FF0). Because this memory has not been freed, the debug memory allocator detects it and then prints the allocation report.
Note
One allocation record will always be listed for the atexit() handler that prints the allocation records, as shown in the previous output (Allocation #2).
The options shown in Table 4-4 on page 191, which can be specified by the MALLOCDEBUG environment value, are supported by 3.1, default, and default with malloc buckets extension memory allocators:
Option | Feature name | Related section |
---|---|---|
verbose | Malloc report | “verbose” on page 191 |
arena[a] | “arena” on page 192 | |
trace | Malloc trace | “trace” on page 193 |
log | Malloc log | “log” on page 196 |
[a] The arena option is not supported by the 3.1 memory allocator.
Multiple options can be specified by separating them using a comma as follows:
MALLOCDEBUG=option1,option2,...
The verbose option instructs memory allocators that information on errors that occurred in the malloc subsystem will be reported and actions can be performed if specified.
The verbose option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:
MALLOCDEBUG=verbose
All errors caught in the malloc subsystem are output to standard error, along with detailed information.
The verbose option allows the user to provide a function that the malloc subsystem will call when it encounters an error. Before returning, the malloc subsystem calls the user-provided function, if it is specified.
A global function pointer is available for use by the user. In the code, the following function pointer should be set to the user’s function:
extern void (*malloc_err_function)(int, ...)
The following user-defined function must be implemented:
void malloc_err_function(int, ...)
For example, to use the user-defined function abort_sub, the following code must be inserted into the user’s application:
malloc_err_function = &abort_sub;
A sample program shown in Example 4-5 illustrates the use of the verbose option. When executed, the program prints the following error message:[4]
[4] Yorktown is an internal name used in the AIX development to refer to the default memory allocator.
$ MALLOCDEBUG=verbose a.out
Malloc Report: Corruption in the Yorktown Malloc arena has been detected.
IOT/Abort trap(coredump)
The default memory allocator with the verbose option has detected the corruption in this output.
#include <stdio.h> #include <stdlib.h> #include <errno.h> #include <malloc.h> int main(int argc, char *argv[]) { char *ptr; char buf[BUFSIZ]; if ((ptr = (char *)malloc(1200)) == (char *)NULL) { sprintf(buf, "malloc() failed at %d in %s with errno = %d" , __LINE__, __FILE__, errno); perror(buf); } free(ptr); memset(ptr - 8, (char)-1, 40); free(ptr - 4096); exit(0); } |
The arena option instructs the malloc subsystem to check the structures that contain the free blocks before every allocation request is processed. This option will ensure that the arena is not corrupted. Also, the arena will also be checked when an error occurs.
The checkarena option checks for NULL pointers in the free tree or pointers that do not fall within a certain range. If an invalid pointer is encountered during the descent of the tree, the program might perform a core dump depending on the value of the invalid address.
The arena option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:
MALLOCDEBUG=checkarena
A sample program shown in Example 4-6 illustrates the use of the arena option. When executed, the program prints the following error message:
$ MALLOCDEBUG=verbose,arena a.out Malloc Report: The address passed to free, 0x1ffff5f8, is outside the valid range of addresses allocated by malloc (errno = 0). IOT/Abort trap(coredump)
#include <stdio.h> #include <stdlib.h> #include <errno.h> #include <malloc.h> int main(int argc, char *argv[]) { char *ptr; char buf[BUFSIZ]; if ((ptr = (char *)malloc(16)) == (char *)NULL) { sprintf(buf, "malloc() failed at %d in %s with errno = %d" , __LINE__, __FILE__, errno); perror(buf); exit(1); } free(ptr-4096); exit(0); } |
The trace option instructs the malloc subsystem to use the trace facility. Traces of the malloc(), realloc(), and free() subroutines are recorded for use in problem determination and performance analysis.
The trace option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:
MALLOCDEBUG=trace
The trace option supports the following trace hook IDs:
HKWD_LIB_MALL_COMMON (hook ID: 60a)
When tracing is enabled for HKWD_LIB_MALL_COMMON, the input parameters, as well as return values for each call to malloc(), realloc(), and free() subroutines, are recorded in the trace subsystem. In addition to providing trace information about the malloc subsystem, the trace option also performs checks of its internal data structures. If these structures have been corrupted, these checks will likely detect the corruption and provide temporal data, which is useful in problem determination.
HKWD_LIB_MALL_INTERNAL (hook ID: 60b)
When tracing is enabled for HKWD_LIB_MALL_INTERNAL and corruption is detected, information about the internal data structures are logged through the trace subsystem.
To use the trace option, do the following:
1. | Start the trace subsystem before executing the target application. If you use the command line interface, type the following command as the root user: # trace -j'60a,60b' -a If you use SMIT, run smit -C trace and select START Trace, and then select the hook keywords 60a and 60b in the high-lighted field in Example 4-7. Example 4-7. Start trace
| |
2. | Run the target application. You should remember the process ID of the application. | |
3. | ||
4. | Generate a trace report. If you use the command line, type the following command as the root user: # trcrpt -0 exec=y -0 pid=y -0 tid=y -0 svc=y -0 timestamp=1 If you use SMIT, run smit -C trace and select Generate a Trace Report. |
Fri Feb 28 14:29:20 2003 System: AIX 5.2 Node: murumuru Machine: 000C91AD4C00 Internet Address: 0903046A 9.3.4.106 The system contains 2 cpus, of which 2 were traced. Buffering: Kernel Heap This is from a 32-bit kernel. Tracing only these hooks, 60a,60b /usr/bin/trace -j60a 60b -a ID PROCESS NAME I SYSTEM CALL ELAPSED APPL SYSCALL KERNEL INTERRUPT 001 --1- 0.000000 TRACE ON channel 0 Fri Feb 28 14:29:20 2003 60A --1- 5.561069 HKWD_LIBC_MALL_COMMON function=malloc() [Default Allocator] size=000A returnptr=20000728 60A --1- 5.561261 HKWD_LIBC_MALL_COMMON function=free() [Default Allocator] inptr=20000728 60A --1- 10.131345 HKWD_LIBC_MALL_COMMON function=malloc() [Default Allocator] size=0290 returnptr=20005878 |
The log option instructs the malloc subsystem to record information on the number of active allocations of a given size and stack trace back of a user program. This data can be used in problem determination and performance analysis, if the user program is modified accordingly.
If the log option is enabled, the following data is recorded for each malloc or realloc subroutine invocation:
The size of the allocation.
The stack trace back of the invocation. The depth of the trace back that is recorded is a configurable option.
The number of currently active allocations that match the size and stack trace back.
The data is stored into the following global structure:[5]
[5] This is included in the /usr/include/malloc.h header file.
struct malloc_log *malloc_log_table; #ifndef MALLOC_LOG_STACKDEPTH #define MALLOC_LOG_STACKDEPTH 4 #endif struct malloc_log { size_t size; size_t cnt; uintptr_t callers [MALLOC_LOG_STACKDEPTH]; } size_t malloc_log_size;
The size of the malloc_log structure can change. If the default call-stack depth is greater than 4, the structure will have a larger size. The current size of the malloc_log structure is stored in the globally exported malloc_log_size variable. A user can define the MALLOC_LOG_STACKDEPTH macro to the stack depth that was configured at process start time.
The malloc_log_table can be accessed in the following ways:
Using the get_malloc_log() sub-routine as follows:
#include <malloc.h> size_t get_malloc_log (void *addr,void *buf,size_t bufsize);
This function copies the data from malloc_log_table into the provided buffer. The data can then be accessed without modifying the malloc_log_table. The data represents a snapshot of the malloc log data for that moment of time.
Using the get_malloc_log_live() sub-routine as follows:
#include <malloc.h> struct malloc_log *get_malloc_log_live (void *addr);
The advantage of this method is that no data needs to be copied, therefore performance suffers less. Disadvantages of this method are that the data referenced is volatile and the data may not encompass the entire malloc subsystem, depending on which malloc algorithm is being used.
To clear all existing data from the malloc log tables, use the reset_malloc_log() subroutine as follows:
#include malloc.h void reset_malloc_log(void *addr);
The sample program shown in Example 4-9 on page 198 demonstrates the use of the get_malloc_log_live() sub-routine. After it is compiled, the program would print the following output, if the log option is enabled:
$ cc get_malloc_live.c $ export MALLOCTYPE= $ export MALLOCDEBUG=log $ a.out i is 1217. The size of the allocations is 8. The number of matching allocations is 1. i is 1241. The size of the allocations is 16. The number of matching allocations is 1. i is 3293. The size of the allocations is 8. The number of matching allocations is 1.
Note
Do not set the MALLOCTYPE=log environment variable before invoking the compiler, because it does not support this environment variable. If set, it prints the following message:
cc: 1501-230 Internal compiler error; please contact your Service Representative.
The program has a for loop to dump the content of log_ptr, if there are referenced slots in the data structure referenced by log_ptr.
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <malloc.h>
int main(int argc, char *argv[])
{
char *ptr1 = (char *)NULL, *ptr2 = (char *)NULL, *ptr3 = (char *)NULL;
int i;
struct malloc_log *log_ptr;
ptr1 = (char *)malloc(8);
ptr2 = (char *)malloc(16);
ptr3 = (char *)malloc(8);
log_ptr = get_malloc_log_live(ptr1);
for (i = 0; i < DEFAULT_RECS_PER_HEAP; i++) {
if (log_ptr->cnt > 0) {
printf("i is %d.
", i);
printf("The size of the allocations is %d.
", log_ptr->size);
printf("The number of matching allocations is %d.
"
, log_ptr->cnt);
}
log_ptr++;
}
exit(0);
}
|
The log option is not enabled by default, but can be enabled and configured prior to process startup by setting the MALLOCDEBUG environment variable as follows:
MALLOCDEBUG=log
To enable the trace option with user-specified configuration options, set the MALLOCDEBUG environment variable as follows:
MALLOCDEBUG=log:records_per_heap:stack_depth
Note
The records_per_heap and stack_depth parameters must be specified in order. Leaving a value blank will result in setting that parameter to the default value.
The predefined MALLOCDEBUG configuration options include the following:
records_per_heap | Used to specify the number of malloc log records that are stored for each heap. This parameter affects the amount of memory that is used by malloc log. The default value is 4096, the maximum value is 65535. |
stack_depth | Used to specify the depth of the function-call stack that is recorded for each allocation. This parameter affects the amount of memory used by malloc log, as well as the size of the malloc_log structure. The default value is 4, the maximum is 32. |
The performance of all programs can degrade when the trace option is enabled, due to the cost of storing data to memory. Memory usage will also increase.