3.4. Introduction to shared memory

The Inter-Process Communication (IPC) facilities are used by processes to communicate with each other and to synchronize their activities.^[15] IPC facilities, semaphores, message queues, and shared memory are quite common services in the modern UNIX operating systems, including AIX. In this section, we focus on the shared memory on AIX.

^[15] Signals are also commonly used for IPC in the UNIX operating systems. However, we do not discuss signals in this redbook, because the signal management on AIX conforms very well to many UNIX standards, such as POSIX.

The shared memory is usually used for the following purposes:

• Sharing memory segments between processes

  Mapped shared memory segments can serve as a large pool for exchanging data among processes. The mechanism does not provide locks or access control among the processes by default. Therefore, processes using shared memory areas must set up a signal or semaphore control method to prevent access conflicts and to keep one process from changing data that another is using. Shared memory segments can be most beneficial when the amount of data to be exchanged between processes is too large to transfer with message queues or when many processes have to share common data.

• Mapping files into memory segments

  Memory mapped files provide a mechanism for a process to access files by directly incorporating file data into the process address space. The use of mapped files can significantly reduce I/O data movement because the file data does not have to be copied into process data buffers, as is done by the read and write subroutines. When more than one process maps the same file, its contents are shared among them, therefore providing a low-overhead mechanism by which processes can synchronize and communicate.

However, once shared memory segments are attached to the address space of your application process, those segments do not have to be shared with other processes. Therefore, some applications exploit shared memory segments as if they are an extended memory heap (see 4.4, “Heap management using MEMDBG” on page 199 for detail).

AIX supports the following two well-known services for memory mapping:

• Section 3.4.1, “The shmat services” on page 141

• Section 3.4.2, “The mmap services” on page 145

Both services address the same type of memory usage, but shmat is normally used to create and use shared memory segments within a program, and the mmap is mostly used for mapping files into the process address space, although it can be used for creating shared memory segments as well as mapping files.

The term shared memory segments is widely used to refer to the memory chunks allocated by these two services on UNIX operating systems, including AIX. Although a shared memory segment smaller than 256 MB usually consumes a 256 MB segment on AIX, if the EXTSHM=ON environment variable is not defined, the term is not the same concept referenced by segments (explained in 3.2, “The 32-bit user process model” on page 109).

For further information about the topics explained in this section, please refer to the “Shared Libraries and Shared Memory” section in AIX 5L Version 5.2 General Programming Concepts: Writing and Debugging Programs.

3.4.1. The shmat services

The shmat services supported by AIX include the following routines:

These subroutine references are provided in Appendix C, “Subroutine references for shmat and mmap services” on page 447.

Terms used in the shmat services

In the shmat services, the following terms are frequently used:

There are some well-established programming disciplines about the use of these routines. We introduce a few of them in this section.

Mapping memory with shmat()

The following pseudo-code shows how to use shared memory segments with shmat services:

mykey = ftok("/path/some_file", proj);

ptr = shmat(mem_id, 0, SHM_MAP);

shmdt(ptr);

shmctl (mem_id, IPC_RMID, ptr);

If a shared memory segment is created accordingly, multiple processes can attach it into their address space, as illustrated in Figure 3-11. In this figure, two processes, A and B, are sharing the sheared memory segment, which is attached to 0x3 in the process A’s address space and attached to 0x4 in the process B’s address space. If the process A writes data in this memory area, the process B can instantly read the data value from it.

Although it is a very efficient way to transfer data between processes, a data synchronization algorithm must be used between processes. Semaphores are commonly used to implement this data synchronization algorithm.

The shared memory segments created by the following methods can be shared between processes:

• The MAP_SHARED flag is specified with mmap().

• Appropriate permission modes are specified with shmget().

• After creating a shared memory segment with shmget(), its permission modes are altered accordingly with the IPC_SET command flag with shmctl().

Mapping files with shmat()

The following pseudo-code shows how to map files with shmat services. To do so, you need to map the file first, then set the end of the mapped file using lseek(). The shmat services do not provide any mechanisms to detect the end of mapped file.

3.4.2. The mmap services

The mmap services supported by AIX include the following routines:

These subroutine references are provided in Appendix C, “Subroutine references for shmat and mmap services” on page 447.

There are some well-established programming disciplines for the use of these routines. We introduce a few of them in this section.

Mapping files with mmap()

The following pseudo-code shows how to map files with the mmap services:

fd = open(pathname, permissions);

len = lseek(fd, 0, SEEK_END)

ptr = mmap(addr, len, prot, MAP_FILE, fd, 0)

close(fd);

munmap(addr, len);

Mapping memory with mmap()

The following pseudo-code shows how to map memory segments with mmap services:

3.4.3. Difference between shmat and mmap services

Although both the shmat and mmap services can be used for the purposes explained in 3.4, “Introduction to shared memory” on page 140, there are significant differences between these two services:

• The address for a shared memory segment mapped by shmat() must be SHMLBA (256 MB) aligned if the EXTSHM=ON environment variable is not set, while mmap() works on a page-size alignment (4 KB).

• The mmap() memory mappings are a process resource and thus are cleaned up at process exit-time. The shmget() shared memory segments are a system-wide resource and are not cleaned up at process exit-time.

Therefore, use the shmat services under the following circumstances:

• When mapping shared memory regions that need to be shared among unrelated processes (no parent-child relationship).

• When mapping entire files.

Use the mmap services under the following circumstances:

• Portability of the application is a concern.

• Many files are mapped simultaneously.

• Only a portion of a file needs to be mapped.

• Page-level protection needs to be set on the mapping.

• Private mapping (MAP_PRIVATE) is required.

3.4.4. Shared memory limits

Shared memory limits vary, depending on AIX versions, as shown in Table 3-1 on page 148.

To confirm system constants regarding shared memory, it is recommended to call the vmgetinfo()^[16] routine instead of hardcoding numerical values (see the program example shown in Example 3-17 on page 149).

^[16] The vmgetinfo() routine is supported on AIX 5L Version 5.2 and later.

Example 3-16 shows the output from this program. Please compare the values in this example with values for AIX 5L Version 5.2 in the Table 3-1.

Example 3-16. Output from vmgetinfo() routine

$ a.out
/* Shared memory limits */
max # of shared memory id's =                           131072
64bit proc max shm segment size =                        1099511627776
32bit proc max shm segment size =                        2147483648
min shared memory segment size =                         1
max # of shm segs per 64bit proc =                      268435456
max # of shm segs per 32bit proc (without EXTSHM=ON) =  11

Example 3-17 on page 149 shows an example program to call the vmgetinfo() routine.

Example 3-17. vmgetinfo.c^[17]

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <sys/vminfo.h>

int main(int argc, char *argv[])
{
    int                 rc;
    char                buf[BUFSIZ];
    struct ipc_limits   ipc_lim;

    if ((rc = vmgetinfo(&ipc_lim, IPC_LIMITS, sizeof(struct ipc_limits))) != 0)
    {
        sprintf(buf, "vmgetinfo() at %d in %s failed with errno = %d"
            , _LINE_, _FILE_, errno);
        perror(buf);
        exit(1);
    }

    printf("/* Shared memory limits */
");
    printf("max # of shared memory id's =				%lld
", ipc_lim.shmmni);
    printf("64bit proc max shm segment size =			%lld
", ipc_lim.shmmax64);
    printf("32bit proc max shm segment size =			%u
", ipc_lim.shmmax32);
    printf("min shared memory segment size =				%u
", ipc_lim.shmmin);
    printf("max # of shm segs per 64bit proc =			%lld
", ipc_lim.shmseg64);
    printf("max # of shm segs per 32bit proc (without EXTSHM=ON) =	%u
"
        , ipc_lim.shmseg32);

    exit(0);
}

^[17] The structure type ipc_limits is defined in the /usr/include/sys/vminfo.h header file.