3.5. Shared memory segments allocation order
This section explains how shared memory segments are allocated in the process address space. To demonstrate the allocation order, we have prepared the program shown in Example 3-19 on page 151. To compile this program, type the following command:
$ cc -DCALL_SHMDT grabshm.c
If -DCALL_SHMDT is not specified, then the shmdt() and shmctl() routines will not be called in the program; it then leaves shared memory segments after the execution. If the program is executed by the ausres01 user, it then leaves the shared memory segments highlighted in Example 3-18.
Example 3-18. ipcs -m
$ ipcs -m IPC status from /dev/mem as of Sun Feb 9 18:21:48 CST 2003 T ID KEY MODE OWNER GROUP Shared Memory: m 0 0xe4663d62 --rw-rw-rw- imnadm imnadm m 1 0x9308e451 --rw-rw-rw- imnadm imnadm m 2 0x52e74b4f --rw-rw-rw- imnadm imnadm m 3 0xc76283cc --rw-rw-rw- imnadm imnadm m 4 0x298ee665 --rw-rw-rw- imnadm imnadm m 5 0xffffffff --rw-rw---- root system m 6 0xffffffff --rw-rw---- root system m 48159 0xffffffff --rw------- ausres01 itsores m 48160 0xffffffff --rw------- ausres01 itsores m 48161 0xffffffff --rw------- ausres01 itsores m 48162 0xffffffff --rw------- ausres01 itsores |
To remove stale shared memory segments, type ipcrm -m shared_memory_ID. The shared_memory_IDs are shown in the second column in the example. If you need to remove many shared memory segments, do the following:
$ ipcs -m | grep user_name | awk '{print $2}' | while read id
> do
> ipcrm -m $id
> done
Where user_name is the user name of the application process that acquired the shared memory segments.
Example 3-19 on page 151 is a sample program to allocate shared memory segments. When executed, it requires two arguments:
Usage: a.out [-m|-s] digit-number
If you specify the -m option, then it tries to allocate multiplies of a 1 MB shared memory segment. If -s specified, it tries to allocate multiplies of a 256 MB shared memory segment. The argument digit-number specifies how many shared memory segments should be allocated.
In this program, the created shared memory segments cannot be shared with other processes, because the IPC_PRIVATE flag is specified with the shmget() routine. Also, please note that the attaching target address is specified as (void *)0 with the shmat() routine, so that we instruct the system to select the attaching target address of shared memory segments.
Example 3-19. grabshm.c
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <strings.h>
#include <errno.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#define MAX_SHM_SEGMENTS 131072
#define ONE_MB (1024 * 1024)
#define ONE_SEG (256 * 1024 * 1024)
char *size_of(int i)
{
if (i == ONE_MB) {
return("1 MB");
} else if (i == ONE_SEG) {
return("256 MB");
} else {
exit(1);
}
}
int main(int argc, char *argv[])
{
char *shmptr[MAX_SHM_SEGMENTS];
char buf[BUFSIZ];
int shmid[MAX_SHM_SEGMENTS];
int size_flag;
int cnt, max, rc;
if (argc != 3) {
fprintf(stderr, "Usage: %s [-m|-s] digit-number
", argv[0]);
exit(1);
} else {
if (!strcmp(argv[1], "-m")) {
size_flag = ONE_MB;
} else if (!strcmp(argv[1], "-s")) {
size_flag = ONE_SEG;
} else {
fprintf(stderr, "Usage: %s [-m|-s] digit-number
", argv[0]);
exit(1);
}
max = atoi(argv[2]);
}
for (cnt = 0; cnt < max; cnt++) {
if ((shmid[cnt] = shmget(IPC_PRIVATE, size_flag
, IPC_CREAT | S_IRUSR | S_IWUSR)) < 0) {
sprintf(buf
, "[%2d] shmget("%s") at %d in %s failed with errno = %d"
, cnt, size_of(size_flag), __LINE__, __FILE__, errno);
perror(buf);
}
if ((shmptr[cnt] = shmat(shmid[cnt], (void *)0, 0)) == (void *) -1) {
sprintf(buf, "[%2d] shmat() at %d in %s failed with errno = %d"
, cnt, __LINE__, __FILE__, errno);
perror(buf);
} else {
printf(
"[%2d] beginning address of %s shared memory segment is 0x%016p
"
, cnt, size_of(size_flag), shmptr[cnt]);
printf(
"[%2d] ending address of %s shared memory segment is 0x%016p
"
, cnt, size_of(size_flag), shmptr[cnt] + (size_flag - 1));
}
}
/* if you comment out the following code block,
* you need to explicitly remove the allocated shared memory segment
* by calling ipcrm -m ID.
*/
#if defined(CALL_SHMDT)
for (cnt = 0; cnt < max; cnt++) {
if ((rc = shmdt(shmptr[cnt])) == -1) {
sprintf(buf, "[%2d] shmdt() at %d in %s failed with errno = %d"
, cnt, __LINE__, __FILE__, errno);
perror(buf);
}
if (shmctl(shmid[cnt], IPC_RMID, 0) < 0) {
sprintf(buf, "[%2d] shmctl() at %d in %s failed with errno = %d"
, cnt, __LINE__, __FILE__, errno);
perror(buf);
}
}
#endif
exit(0);
}
|
Please note that this program is written in order to demonstrate the order of shared memory segments allocation. If you need to allocate a larger size of shared memory segment, you can get it with the one shmget() call.
3.5.1. Order in the 32-bit default memory model
If the program shown in Example 3-19 on page 151 is executed in the 32-bit default memory model, then segments 0x3 - 0xE will be sequentially used, excluding 0xD.
Example 3-20 shows the output from this program, when we have attempted to allocate twelve 256 MB shared memory segments. Because the 32-bit user process model supports up to 11 shared memory segments by default, if the EXTSHM=ON environment variable is not specified, the last attempt to allocate the 12th shared memory segment would fail.
Example 3-20. Acquiring twelve 256 MB shared memory segments[18]
$ a.out -s 12 [ 0] beginning address of 256 MB shared memory segment is 0x0000000030000000 [ 0] ending address of 256 MB shared memory segment is 0x000000003fffffff [ 1] beginning address of 256 MB shared memory segment is 0x0000000040000000 [ 1] ending address of 256 MB shared memory segment is 0x000000004fffffff ... [ 9] beginning address of 256 MB shared memory segment is 0x00000000c0000000 [ 9] ending address of 256 MB shared memory segment is 0x00000000cfffffff [10] beginning address of 256 MB shared memory segment is 0x00000000e0000000 [10] ending address of 256 MB shared memory segment is 0x00000000efffffff [11] shmat() at 58 in grabshm.c failed with errno = 24: Too many open files [11] shmdt() at 76 in grabshm.c failed with errno = 22: Invalid argument |
[18] The value 24 of errno means EMFILE (too many open files) and the value 22 of errno means EINVAL (invalid argument).
Example 3-21 shows the output from this program, when we have attempted to allocate twelve 1 MB shared memory segments. Because the 32-bit user process model supports up to 11 shared memory segments, by default, if the EXTSHM=ON environment variable is not specified, the last attempt to allocate the 12th shared memory segment would fail, even if the total shared memory size is just 11 MB.
Example 3-21. Acquiring twelve 1 MB shared memory segments
$ a.out -m 12 [ 0] beginning address of 1 MB shared memory segment is 0x0000000030000000 [ 0] ending address of 1 MB shared memory segment is 0x00000000300fffff [ 1] beginning address of 1 MB shared memory segment is 0x0000000040000000 [ 1] ending address of 1 MB shared memory segment is 0x00000000400fffff ... [ 9] beginning address of 1 MB shared memory segment is 0x00000000c0000000 [ 9] ending address of 1 MB shared memory segment is 0x00000000c00fffff [10] beginning address of 1 MB shared memory segment is 0x00000000e0000000 [10] ending address of 1 MB shared memory segment is 0x00000000e00fffff [11] shmat() at 58 in grabshm.c failed with errno = 24: Too many open files |
You may notice the system selected different segments for allocating just 1MB size shared memory segments. For example, the 0x300FFFFF - 0x3FFFFFFF address cannot be used to allocate another shared memory segment.
3.5.2. Order in the 32-bit very large memory model with DSA
If the program shown in Example 3-19 on page 151 is executed in the 32-bit very large memory model with DSA, then segments 0xF - 0x3 would be sequentially used; however, higher address segments cannot be used, depending on the maxdata variable.
Example 3-22 shows the output from this program, when we have attempted to allocate eleven 1 MB shared memory segments with LDR_CNTRL=MAXDATA=0x80000000@DSA. Because the 0x3 segment is always reserved for the process heap in the very large memory model, the 11th attempt to allocate another shared memory segment failed.
Note
If segments 0x3 - 0xE were attached to the process address heap for the process heap before allocating shared memory segments, this program could have failed earlier.
Example 3-22. LDR_CNTRL=MAXDATA=0x80000000@DSA
$ LDR_CNTRL=MAXDATA=0x80000000@DSA a.out -m 11 [ 0] beginning address of 1 MB shared memory segment is 0x00000000e0000000 [ 0] ending address of 1 MB shared memory segment is 0x00000000e00fffff [ 1] beginning address of 1 MB shared memory segment is 0x00000000c0000000 [ 1] ending address of 1 MB shared memory segment is 0x00000000c00fffff ... [ 8] beginning address of 1 MB shared memory segment is 0x0000000050000000 [ 8] ending address of 1 MB shared memory segment is 0x00000000500fffff [ 9] beginning address of 1 MB shared memory segment is 0x0000000040000000 [ 9] ending address of 1 MB shared memory segment is 0x00000000400fffff [10] shmat() at 58 in grabshm.c failed with errno = 24: Too many open files [10] shmdt() at 76 in grabshm.c failed with errno = 22: Invalid argument |
Example 3-23 on page 155 shows the output from this program, when we have attempted to allocate fourteen 1 MB shared memory segments with LDR_CNTRL=MAXDATA=0@DSA. Because the program ran out of all the available 13 segments, the 14th attempt to allocate another shared memory segment failed.
Example 3-23. LDR_CNTRL=MAXDATA=0@DSA
$ LDR_CNTRL=MAXDATA=0@DSA a.out -m 14 [ 0] beginning address of 1 MB shared memory segment is 0x00000000f0000000 [ 0] ending address of 1 MB shared memory segment is 0x00000000f00fffff [ 1] beginning address of 1 MB shared memory segment is 0x00000000e0000000 [ 1] ending address of 1 MB shared memory segment is 0x00000000e00fffff ... [11] beginning address of 1 MB shared memory segment is 0x0000000040000000 [11] ending address of 1 MB shared memory segment is 0x00000000400fffff [12] beginning address of 1 MB shared memory segment is 0x0000000030000000 [12] ending address of 1 MB shared memory segment is 0x00000000300fffff [13] shmat() at 58 in grabshm.c failed with errno = 24: Too many open files |
3.5.3. Extended mode shared memory segments
As explained in the previous sections, up to 11 shared memory segments are available in the 32-bit default memory model and up to 13 shared memory segments are available in the 32-bit very large memory model.
Although most 32-bit applications are satisfied with this limitation, some applications require more shared memory segments. To address this requirement, AIX supports another type of shared memory segments in the 32-bit user process called extended mode shared memory segments (EXTSHM).[19]
[19] The extended mode shared memory segment capability has been supported since AIX Version 4.2.1.
The extended mode shared memory segment capability is a process basis dynamic feature. To use the capability, simply define the following environment variable before executing your applications:
EXTSHM=ON
Note
The keyword must be upper case characters. EXTSHM=on is invalid.
Do not insert this line into the /etc/environment file on your system. Some 32-bit applications do not support this capability. Please consult with the publications shipped with the software products installed on your system.
Once defined, a 32-bit process can have shared memory segments up to 131,072 on AIX Version 4.3.2 and later.
Example 3-24 on page 156 shows the output from the program, when we have attempted to allocate 100 1 MB shared memory segments with EXTSHM=ON.
Example 3-24. Acquiring 100 shared memory segments with EXTSHM=ON
$ EXTSHM=ON a.out -m 100 [ 0] beginning address of 1 MB shared memory segment is 0x0000000030000000 [ 0] ending address of 1 MB shared memory segment is 0x00000000300fffff [ 1] beginning address of 1 MB shared memory segment is 0x0000000030100000 [ 1] ending address of 1 MB shared memory segment is 0x00000000301fffff ... [98] beginning address of 1 MB shared memory segment is 0x0000000036200000 [98] ending address of 1 MB shared memory segment is 0x00000000362fffff [99] beginning address of 1 MB shared memory segment is 0x0000000036300000 [99] ending address of 1 MB shared memory segment is 0x00000000363fffff |
Restrictions of extended shared memory segments
Although they seem very convenient, the extended shared memory segments have some restrictions, which are described in the following:
The EXTSHM environment variable will be ignored and has no effect in the 64-bit user process model.
When attaching a segment larger than 268,431,360 bytes (256 MB - 4 KB), the EXTSHM environment variable will be ignored, and the process attaches the segment with a granularity of 256 MB.
When calling the mmap() routine, the EXTSHM environment variable will be ignored, and the process attaches the segment with a granularity of 256 MB.
The SHM_SIZE parameter of shmctl() is not supported for segments created with EXTSHM=ON.
No raw I/O is allowed for segments created with EXTSHM=ON.
3.5.4. Order in the 64-bit memory model
Example 3-25 shows the output from this program, when we have attempted to allocate twelve 1 MB shared memory segments in the 64-bit user process model. Each 1 MB shared memory segment is allocated into separate 256 MB segments, starting from 0x0700_0000_0000_0000.
Example 3-25. Acquiring 12 shared memory segments in 64-bit mode
$ cc -q64 -DCALL_SHMDT grabshm.c $ file a.out a.out: 64-bit XCOFF executable or object module not stripped $ a.out -m 12 [ 0] beginning address of 1 MB shared memory segment is 0x0700000000000000 [ 0] ending address of 1 MB shared memory segment is 0x07000000000fffff [ 1] beginning address of 1 MB shared memory segment is 0x0700000010000000 [ 1] ending address of 1 MB shared memory segment is 0x07000000100fffff ... [10] beginning address of 1 MB shared memory segment is 0x07000000a0000000 [10] ending address of 1 MB shared memory segment is 0x07000000a00fffff [11] beginning address of 1 MB shared memory segment is 0x07000000b0000000 [11] ending address of 1 MB shared memory segment is 0x07000000b00fffff |
Although it is technically possible that a 64-bit process can request shared memory segments up to 131,072 on AIX Version 4.3.2 or later, it is your responsibility to assure that your application will not consume all the available system memory.
Note
The EXTSHM environment variable will be ignored and has no effect in the 64-bit user process model.