The Inter-Process Communication (IPC) facilities are used by processes to communicate with each other and to synchronize their activities.[15] IPC facilities, semaphores, message queues, and shared memory are quite common services in the modern UNIX operating systems, including AIX. In this section, we focus on the shared memory on AIX.
[15] Signals are also commonly used for IPC in the UNIX operating systems. However, we do not discuss signals in this redbook, because the signal management on AIX conforms very well to many UNIX standards, such as POSIX.
The shared memory is usually used for the following purposes:
Sharing memory segments between processes
Mapped shared memory segments can serve as a large pool for exchanging data among processes. The mechanism does not provide locks or access control among the processes by default. Therefore, processes using shared memory areas must set up a signal or semaphore control method to prevent access conflicts and to keep one process from changing data that another is using. Shared memory segments can be most beneficial when the amount of data to be exchanged between processes is too large to transfer with message queues or when many processes have to share common data.
Mapping files into memory segments
Memory mapped files provide a mechanism for a process to access files by directly incorporating file data into the process address space. The use of mapped files can significantly reduce I/O data movement because the file data does not have to be copied into process data buffers, as is done by the read and write subroutines. When more than one process maps the same file, its contents are shared among them, therefore providing a low-overhead mechanism by which processes can synchronize and communicate.
However, once shared memory segments are attached to the address space of your application process, those segments do not have to be shared with other processes. Therefore, some applications exploit shared memory segments as if they are an extended memory heap (see 4.4, “Heap management using MEMDBG” on page 199 for detail).
AIX supports the following two well-known services for memory mapping:
Section 3.4.1, “The shmat services” on page 141
Section 3.4.2, “The mmap services” on page 145
Both services address the same type of memory usage, but shmat is normally used to create and use shared memory segments within a program, and the mmap is mostly used for mapping files into the process address space, although it can be used for creating shared memory segments as well as mapping files.
The term shared memory segments is widely used to refer to the memory chunks allocated by these two services on UNIX operating systems, including AIX. Although a shared memory segment smaller than 256 MB usually consumes a 256 MB segment on AIX, if the EXTSHM=ON environment variable is not defined, the term is not the same concept referenced by segments (explained in 3.2, “The 32-bit user process model” on page 109).
For further information about the topics explained in this section, please refer to the “Shared Libraries and Shared Memory” section in AIX 5L Version 5.2 General Programming Concepts: Writing and Debugging Programs.
The shmat services supported by AIX include the following routines:
shmat() | Attaches a shared memory segment to a process. |
shmctl() | Controls shared memory operations. |
shmget() | Gets or creates a shared memory segment. |
shmdt() | Detaches a shared memory segment from a process. |
ftok() | Provides the key that the shmget() subroutine uses to create the shared segment. |
These subroutine references are provided in Appendix C, “Subroutine references for shmat and mmap services” on page 447.
In the shmat services, the following terms are frequently used:
There are some well-established programming disciplines about the use of these routines. We introduce a few of them in this section.
The following pseudo-code shows how to use shared memory segments with shmat services:
1. | Create a key to uniquely identify the shared segment. Use the ftok() subroutine to create the key. For example, to create the mykey key using a variable, proj (integer type), and a file name of /path/some_file, insert the following statement: mykey = ftok("/path/some_file", proj); |
2. | Either:
|
3. | Attach the shared segment to the process with the shmat() subroutine. For example, to attach a previously created segment, insert the following statement: ptr = shmat(mem_id, 0, SHM_MAP); If you specify address 0 as the second parameter of shmat(), then the operating system will select the attaching target address. This is the recommended programming manner. The variable ptr is a pointer to a structure that defines the fields in the shared segment. Use this structure to store and retrieve data in the shared segment. The definition of the structure should be the same for all processes using the segment. |
4. | |
5. | Detach from the segment using the shmdt() subroutine: shmdt(ptr); |
6. | If the shared segment is no longer needed, remove it from the system with the shmctl() subroutine: shmctl (mem_id, IPC_RMID, ptr); |
If a shared memory segment is created accordingly, multiple processes can attach it into their address space, as illustrated in Figure 3-11. In this figure, two processes, A and B, are sharing the sheared memory segment, which is attached to 0x3 in the process A’s address space and attached to 0x4 in the process B’s address space. If the process A writes data in this memory area, the process B can instantly read the data value from it.
Although it is a very efficient way to transfer data between processes, a data synchronization algorithm must be used between processes. Semaphores are commonly used to implement this data synchronization algorithm.
The shared memory segments created by the following methods can be shared between processes:
The MAP_SHARED flag is specified with mmap().
Appropriate permission modes are specified with shmget().
After creating a shared memory segment with shmget(), its permission modes are altered accordingly with the IPC_SET command flag with shmctl().
The following pseudo-code shows how to map files with shmat services. To do so, you need to map the file first, then set the end of the mapped file using lseek(). The shmat services do not provide any mechanisms to detect the end of mapped file.
1. | To create the mapped data file:
|
2. | To detect the end of the mapped file:
|
The mmap services supported by AIX include the following routines:
mmap() | Maps an object file into virtual memory. |
madvise() | Advises the system of a process’ expected paging behavior. |
mincore() | Determines residency of memory pages. |
mprotect() | Modifies the access protections of memory mapping. |
msync() | Synchronizes a mapped file with its underlying storage device. |
munmap() | Unmaps a mapped memory segment. |
These subroutine references are provided in Appendix C, “Subroutine references for shmat and mmap services” on page 447.
There are some well-established programming disciplines for the use of these routines. We introduce a few of them in this section.
The following pseudo-code shows how to map files with the mmap services:
Note
A file system object should not be simultaneously mapped using both the shmat and mmap services.
The following pseudo-code shows how to map memory segments with mmap services:
If you specify MAP_ANONYMOUS instead of MAP_FILE for the mmap()’s fourth parameter, then the file descriptor parameter must be specified as -1. This is called an anonymous memory segment, because there is no file system object associated. All pages in the anonymous memory segment are 0-filled:
ptr = mmap(addr, len, prot, MAP_ANONYMOUS, -1, 0)
This memory segment pointed to by ptr can be shared only with the descendants of the current process.
Work with the data in the segment.
Detach the segment from the address space using the munmap() subroutine:
munmap(addr, len);
Although both the shmat and mmap services can be used for the purposes explained in 3.4, “Introduction to shared memory” on page 140, there are significant differences between these two services:
The address for a shared memory segment mapped by shmat() must be SHMLBA (256 MB) aligned if the EXTSHM=ON environment variable is not set, while mmap() works on a page-size alignment (4 KB).
The mmap() memory mappings are a process resource and thus are cleaned up at process exit-time. The shmget() shared memory segments are a system-wide resource and are not cleaned up at process exit-time.
Therefore, use the shmat services under the following circumstances:
When mapping shared memory regions that need to be shared among unrelated processes (no parent-child relationship).
When mapping entire files.
Use the mmap services under the following circumstances:
Portability of the application is a concern.
Many files are mapped simultaneously.
Only a portion of a file needs to be mapped.
Page-level protection needs to be set on the mapping.
Private mapping (MAP_PRIVATE) is required.
Note
Section 3.4.4, “Shared memory limits” on page 147 and 3.5, “Shared memory segments allocation order” on page 149 are only applicable to the shared memory segments created by shmat().
Shared memory limits vary, depending on AIX versions, as shown in Table 3-1 on page 148.
To confirm system constants regarding shared memory, it is recommended to call the vmgetinfo()[16] routine instead of hardcoding numerical values (see the program example shown in Example 3-17 on page 149).
[16] The vmgetinfo() routine is supported on AIX 5L Version 5.2 and later.
Example 3-16 shows the output from this program. Please compare the values in this example with values for AIX 5L Version 5.2 in the Table 3-1.
$ a.out /* Shared memory limits */ max # of shared memory id's = 131072 64bit proc max shm segment size = 1099511627776 32bit proc max shm segment size = 2147483648 min shared memory segment size = 1 max # of shm segs per 64bit proc = 268435456 max # of shm segs per 32bit proc (without EXTSHM=ON) = 11 |
Example 3-17 on page 149 shows an example program to call the vmgetinfo() routine.
#include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <errno.h> #include <sys/vminfo.h> int main(int argc, char *argv[]) { int rc; char buf[BUFSIZ]; struct ipc_limits ipc_lim; if ((rc = vmgetinfo(&ipc_lim, IPC_LIMITS, sizeof(struct ipc_limits))) != 0) { sprintf(buf, "vmgetinfo() at %d in %s failed with errno = %d" , _LINE_, _FILE_, errno); perror(buf); exit(1); } printf("/* Shared memory limits */ "); printf("max # of shared memory id's = %lld ", ipc_lim.shmmni); printf("64bit proc max shm segment size = %lld ", ipc_lim.shmmax64); printf("32bit proc max shm segment size = %u ", ipc_lim.shmmax32); printf("min shared memory segment size = %u ", ipc_lim.shmmin); printf("max # of shm segs per 64bit proc = %lld ", ipc_lim.shmseg64); printf("max # of shm segs per 32bit proc (without EXTSHM=ON) = %u " , ipc_lim.shmseg32); exit(0); } |
[17] The structure type ipc_limits is defined in the /usr/include/sys/vminfo.h header file.