CHAPTER 3

Installing Oracle Grid Infrastructure for a Stand-Alone Server

Exam Objectives

In this chapter, you will learn to

• 62.2.4.1    Configure Storage for Oracle Automatic Storage Management (ASM)

• 62.2.2.2    Install Oracle Grid Infrastructure for a Stand-Alone Server

This chapter describes the installation of Grid Infrastructure (GI). GI is a separately installed product that provides clustering, networking, storage, and high-availability services for Oracle databases. It is required for Real Application Clusters (RAC) databases and can also be used with single-instance databases. In a single-instance environment, GI is usually used to provide Automatic Storage Management (ASM) devices for storing database files (as described in this chapter) and to provide an automatic restart capability in the event of failures (as described in Chapter 4). The treatment of GI in the Oracle Certified Associate (OCA) syllabus is limited to using the restart capability in a single-instance environment, although concepts of ASM may also be examined.

Configure Storage for Oracle Automatic Storage Management

Grid Infrastructure includes the ASM capability to manage storage to be used for Oracle database files. In earlier releases (up to release 11.1.x), ASM was shipped as part of the database software, but from 11.2.x onward, it is part of GI. GI is a set of processes that run as part of the operating system: on Windows, services that run with Administrator privileges; on Linux, daemons that run with root privileges. The GI processes start an ASM instance. In some ways, an ASM instance resembles the relational database management system (RDBMS) instance with which all database administrators (DBAs) are familiar, but its purpose is very different: it manages devices that are made available to RDBMS instances for file storage. These devices must be configured by the system administrator before installing GI.

GI Architecture

GI consists of a set of processes, all of which are protected against failure. The core process is the High Availability Services daemon, the OHASD. This is protected by the operating system. If it fails, the operating system (OS) will restart it. The OHASD then starts and monitors a set of other processes, which in turn will start and monitor resources used by database instances. These resources are ASM and database listeners.

GI maintains a registry of resources that run under its control. These resources may include any or all of the following:

•  Database listeners

•  Virtual Internet Protocol (IP) addresses

•  An ASM instance (one only)

•  ASM disk groups

•  Database instances

•  Third-party products

The registry exists in a file called the Oracle Local Registry (OLR). The location of the OLR is specified by a platform-specific pointer. On Linux, the pointer is the file /etc/oracle/olr.loc. On Windows, it is the registry key HKEY_LOCAL_MACHINE/SOFTWARE/Oracle/olr/olrconfig_loc.

Resources are registered in the OLR using the crsctl utility. These registrations include details of how to start and stop the resources, the operating accounts under which they should run, and what to do if they fail. The GI daemons make use of this information to manage the resources.

The crsctl utility is also used to stop and start the GI processes, including the OHASD, and to administer registered resources. An alternative administration tool is the srvctl utility. srvctl can manage Oracle resources, not any third-party products that may have been registered and placed under GI control. It is generally much easier to use syntactically (and is less prone to error) than the crsctl utility.

ASM Architecture

ASM is a logical volume manager (LVM) that can be used to configure striped and mirrored volumes for storing Oracle database files. These volumes are known as disk groups. Disk groups are not formatted with a file system that is visible to the operating system; they can be used only for Oracle database files, and these files can be managed only with Oracle products and utilities.

To understand the relationship between an ASM file and an Oracle database file, study the entity-relationship diagram presented in Figure 3-1.

Figure 3-1    The ASM storage model

ASM files reside on a disk group, consisting of one or more ASM disks. The ASM disks are formatted into allocation units (AUs). AUs are grouped into file extents, and a file consists of one or more file extents. There is, in effect, a many-to-many relationship between ASM files and ASM disks. One file can be spread across many disks, and one disk can hold parts of many files. As good relational engineers, we do not permit many-to-many relationships; this one is resolved in two ways: by the disk group entity and by the file extent and AU route. Any one file extent consists of consecutive AUs on one ASM disk. An ASM file is defined by its extent map—that is, the list of pointers to the physical locations of its file extents.

ASM is managed by the ASM instance. The ASM instance consists of memory components and background processes; it is a lightweight structure that manages the ASM environment. It tracks the extent maps that define file locations, adding or removing file extents to or from files according to demands from the RDBMS instance using the file.

The ASM instance has a System Global Area (SGA), and it accepts sessions in the same way that an RDBMS instance does, but it is never in any mode other than nomount. An ASM instance never mounts a controlfile; it has no data dictionary. Therefore, the only way to log on is by using operating system authentication or password file authentication.

The ASM bootstrap does the following:

1.  The operating system starts the GI processes.

2.  GI starts the ASM instance.

3.  The ASM instance locates the ASM disk devices.

4.  The ASM instance mounts the disk groups and registers them with GI.

5.  The RDBMS instance starts.

6.  The RDBMS instance needs a file that is on an ASM disk group.

7.  The RDBMS requests the ASM instance address from GI.

8.  The RDBMS instance logs on to the ASM instance and requests access to the ASM file.

9.  The ASM instance returns the file’s extent map.

10.  The RDBMS instance opens the file.

In summary, when opening an ASM file, GI facilitates the connection between the RDBMS and ASM instances, and then the ASM instance facilitates the connection between the RDBMS instance and the file.

An ASM disk group can store only database files, but it is a broad definition of database files and incudes these file types:

•  Controlfiles

•  Datafiles

•  Tempfiles

•  Online logfiles

•  Archive logfiles

•  Server parameter file, the spfile

•  Password file

•  RMAN backups

•  Data Pump dump files

These file types cannot be stored on an ASM disk group:

•  The Oracle Home

•  Trace files or the alert log

•  A pfile initialization file

•  User-managed backups

ASM Disks

The term ASM disk is a little misleading because the devices are not actually disks. The following are the possibilities:

•  Partitions of directly attached storage (DAS) devices

•  Storage area network (SAN) devices

•  Network-attached storage (NAS) devices

DAS devices are disks physically attached to the server. Typically, this means SCSI disks or some variation thereof. In earlier releases, it was possible to give the entire, raw disk to ASM to use as an ASM disk. This was rarely advisable and is no longer possible. It is now required that the disks have a partition table, even if this defines only one partition covering the entire disk. The partition is then presented to ASM.

SAN devices will usually be striped (and possibly mirrored) volumes managed by a storage array and attached to the server over a fiber-optic channel. They will be presented to ASM as logical unit numbers (LUNs). There is no practical limit to the size that these volumes may be. Striping the volumes with whatever RAID algorithm is considered appropriate will improve performance. Mirroring can be enabled at the SAN level or managed by ASM.

NAS devices will typically be iSCSI volumes or NFS files. An iSCSI device is exported to the network by an iSCSI target machine and mounted on the database server by an iSCSI initiator. NFS files will exist as large zero-filled files on an NFS volume exported from an NFS server and mounted by the database server. The underlying storage for NAS devices may, of course, be a RAID device; the NAS layer will conceal this from ASM. The local area network connection between the database server and storage server should be high speed and dedicated to this function. Multipathing is advisable to add bandwidth and resilience against network failures.

DAS, SAN, or iSCSI devices (which should not be formatted with any file system) are identified by their device drivers. NFS files are identified by their fully qualified filename. Either way, the ASM instance needs to determine what devices it should use. An ASM instance is controlled by a parameter file, in the same way as an RDBMS instance. One critical parameter is ASM_DISKSTRING. This is a comma-separated list of values (which may include wildcard characters) that identify the ASM disks. The manner of naming the devices, and therefore the values to include in the ASM_DISKSTRING, is platform specific. These are the default values for some popular operating systems:

It is important to note that the default value may find disks that are not intended for ASM’s use. If this is the case, it is imperative to set the parameter such that it will find only appropriate devices.

Creating ASM Disks

Disk creation will be done by your system administrator. Subsequently, each disk device will be accessed through a device driver. Device drivers are created by the operating system as it boots. During boot-up, the operating system scans the various I/O buses and creates a device driver for each device it finds. Because this is a dynamic process, it is possible that any one device may be assigned to a different device driver on each boot. Furthermore, following creation, the device drivers will be root owned.

To make devices usable by ASM, it is necessary that the names be persistent across system restarts and that the drivers be readable and writeable by the Oracle processes. On Linux, there are three techniques for this.

•  The ASMLib kernel library    This library is available for some Linux distributions, depending on your Linux license.

•  The ASM Filter Driver    This facility is provided from release 12.1.0.2 and can replace the ASMLib. It adds functionality that will prevent any non-Oracle process from writing to the ASM disks.

•  The udev facility    This facility runs scripts written by your system administrator that identify devices and set appropriate ownership and permissions.

To configure devices on Windows, Oracle supplies the asmtool.exe utility. Other operating systems will have their own platform-specific techniques for managing device ownership and name persistence.

If you are using NFS files as ASM disks, create them with commands such as these, which create a 1TB file and give access to the Oracle owner:

The details of creating ASM disks are beyond the scope of the OCA examination, and there are many platform variations.

Install Oracle Grid Infrastructure for a Stand-Alone Server

GI is installed into a dedicated Oracle Home. The release of GI must be greater than or equal to the release of any database that intends to use the GI services. It is, for example, possible for a 12.1.x GI installation to service an 11.2.x database—but not the other way around. GI must always be running on the same nodes as the database instances. It is, however, possible in a clustered environment to configure Flex ASM, where only a small number of nodes (by default, three) run ASM instances.

The GI installation media include a copy of the Oracle Universal Installer (OUI). Run this (the executable is the runInstaller.sh shell script on Unix and is the setup.exe file on Windows) and follow the prompts. These are the major choices:

•  Download Software Updates    Choose whether to register the installation with My Oracle Support to facilitate downloading patches and updates.

•  Select Installation Option    What type of dialog box should the OUI present? The options are as follows:

•  Install and Configure Oracle Grid Infrastructure for a Cluster

•  Install and Configure Oracle Grid Infrastructure for a Standalone Server

•  Upgrade Oracle Grid Infrastructure or Oracle Automatic Storage Management

•  Install Oracle Grid Infrastructure Software Only

•  Create ASM Disk Group    Set the ASM_DISKSTRING discovery path and choose disks to be used for a disk group.

Figure 3-2 shows the window that prompts for creating a disk group. This window is part of the dialog box presented by the Install and Configure installation options.

Figure 3-2    Creating an ASM disk group during GI install

In the figure, note that by default Redundancy is set to Normal, meaning that unless specified otherwise, every extent of every file created on this disk group will be mirrored. This means that the group must consist of at least two disks because there would be little point in creating a mirror copy on the same device as the primary copy, and ASM will not permit this. The other radio buttons are High, meaning that three copies will be made of each extent (and at least three disks are required), and External, meaning that ASM will not mirror at all but rather rely on fault tolerance provided by the storage medium.

The allocation unit size defaults to 1MB. This has a knock-on effect on the file extent size. The first 20,000 extents of any file are one AU, the next 20,000 extents are four AUs, and beyond that they are 16 AUs. AU size can be set to 1, 2, 4, 8, 16, 32, or 64MB. It applies to all files on the disk group and can never be changed after disk group creation.

In Figure 3-2, no disks are shown. This is because the default search path has not detected any and no disk string has yet been set. Clicking the Change Discovery Path button will prompt for the location of the prepared ASM disks, which will then (if configured correctly) be listed as candidates for members of the disk group.

If you are using OUI to perform an upgrade, the OUI will detect the existing installation and transfer its configuration to the newly installed GI home. It is not possible to have two instances of GI running on one machine concurrently, and the installation wizards will take care of disabling the previous version.

It is considered best practice to install GI under a different operating system user than the database software. This is to allow separation of duties: one OS user can manage GI’s storage and high-availability features, making them available to several DBAs who each have their own OS account. At a small site, where the GI administrator and the DBA are the same person, there is no necessity to follow this rule. Nonetheless, the operating system will always separate these roles through the use of operating system groups. The GI administrator must be a member of the OSASM group. The name of this group is hard-coded on Windows as ORA_ASMDBA, and the group is created implicitly. On Unix, the group can be named anything and must be created before running OUI.

Exercise 3-1: Install Grid Infrastructure    In this exercise, you will install the GI software and configure it for use. It is assumed that the software has been downloaded and unzipped. The routine is slightly different on Windows and Linux.

1.  Follow these steps on Windows:

A.  Launch the OUI dialog box by running the setup.exe file. You will need to do this from a command prompt started with the Run As Administrator privilege.

B.  In the Download Software Updates window, select the Skip Software Updates radio button. Click Next.

C.  In the Select Installation Option window, select the Install Grid Infrastructure Software Only radio button. Click Next.

D.  In the Select Install Type window, select the Install Oracle Grid Infrastructure For A Standalone Server radio button. Click Next.

E.  In the Select Product Languages window, choose any languages you want. Click Next.

F.  In the Specify Installation Location, adjust the directory if you want (the default is usually fine). Click Next.

G.  In the Perform Prerequisite Checks window, any “failed” checks should be addressed and a decision made about whether they are likely to matter in your case. Warnings (for example, regarding Windows security issues) can usually be ignored. Check the Ignore All box, and click Next.

H.  In the Summary window, click Install.

I.  After the install completes, the Finish window, shown next, will prompt you to run the operating system command necessary to configure GI for a stand-alone server:

In the illustration, the command, which must be run as Administrator, is one long command with no line breaks (copy/paste will be useful).

After running the command, click Close to exit from the installer.

2.  Follow these steps on Linux:

A.  Launch the OUI dialog box in a graphical session by running the runInstaller shell script. This cannot be done as the root user. Using the same account that was used for the Oracle software install will be fine (although for a production site, you would usually create a separate OS user to install GI).

B.  In the Download Software Updates window, select the Skip Software Updates radio button. Click Next.

C.  In the Select Installation Option window, select the Install Grid Infrastructure Software Only radio button. Click Next.

D.  In the Select Product Languages window, choose any languages you want. Click Next.

E.  In the Privileged Operating System Groups window, if you are running the installer as the same user who installed the Oracle software, select the group dba for all three groups (OSASM, OSDBA for ASM, and OSOPER for ASM).

F.  In the Specify Installation Location window, adjust the directory if you want (the default is usually fine). Click Next.

G.  In the Perform Prerequisite Checks window, you can usually ignore warnings. Any “failed” checks should be addressed. For instance, a failure of the test for an NTP daemon will not matter; a failure on file permissions would. Check the Ignore All box, and click Next.

H.  In the Summary window, click Install.

I.  Execute Configuration Scripts is a pop-up window that will prompt you to run a script named something like /u01/app/12.1.0.grid/root.sh as root. Execute this script from a root session, accepting the defaults for any prompts. The following illustration shows the result of a typical run:

J.  Still in your root session, run the command generated by the root.sh script that will complete the configuration for a stand-alone server. This is one long command, with no line breaks. In the preceding illustration, the command is as follows:

Following completion, return to the OUI window and click OK and Close.

3.  To confirm the successful installation, use the crsctl utility. This will be in the bin directory beneath the directory chosen for the installation. Here is an example on Windows:

Here is an example on Linux:

These commands will show that the High Availability Service (HAS) is enabled and that the Event Management Daemon is running, as shown in the following illustration:

4.  Investigate the automatic startup of the GI processes.

A.  On Windows, GI is launched by a Windows service. Check its configuration and status with the sc utility, as shown in this illustration:

B.  On Linux, GI is launched through the rc system. Look for the file /etc/rc.d/init.d/init.ohasd and links to it in the various rc directories. There will also be an entry in /etc/inittab that will respawn the init.ohasd process if it fails.

Two-Minute Drill

Configure Storage for Oracle Automatic Storage Management

•  An ASM disk group stores only database files; it has no normal file system.

•  ASM disks are unformatted DAS, SAN, or NAS devices.

•  The ASM instance is a control structure only; no data passes through it.

Install Oracle Grid Infrastructure for a Stand-Alone Server

•  GI must be installed into its own Oracle Home.

•  Configuring GI requires root (Linux) or Administrator (Windows) privileges.

•  The ASM disk search string can be set at install time or changed later.

•  The GI daemons are launched by the operating system on boot-up.

Self Test

1.  What file types and directories can be stored with ASM? (Choose all that apply.)

A.  Alert log

B.  Controlfiles

C.  Datafiles

D.  Online redo log files

E.  Oracle Home directory

F.  Tempfiles

2.  Which of the following recovery files can be stored with ASM? (Choose all that apply.)

A.  Archive redo log files

B.  RMAN backup sets

C.  RMAN image copies

D.  User-managed backups

E.  The flash recovery area

3.  What are the default characteristics of ASM files? (Choose the best answer.)

A.  The files will be striped for performance but not mirrored for safety.

B.  The files will be mirrored for safety but not striped for performance.

C.  The files will be both striped and mirrored.

D.  The files will be neither striped nor mirrored.

4.  What statement is correct about ASM and logical volume managers (LVMs)? (Choose the best answer.)

A.  ASM is itself an LVM and cannot work with a third-party LVM.

B.  ASM can use LVM volumes if they are formatted with a file system.

C.  You can use ASM for striping and the LVM for mirroring.

D.  You can use ASM for mirroring and the LVM for striping.

5.  How can you connect to an ASM instance? (Choose the best answer.)

A.  By using operating system authentication only

B.  By using password file authentication only

C.  By using data dictionary authentication only

D.  By using either operating system or password file authentication

6.  What does ASM stripe? (Choose the best answer.)

A.  Files across all disk groups

B.  Disks across all disk groups

C.  Disk groups across all disks

D.  Files across all disks in a group

7.  You want to make Grid Infrastructure services available to your database. Where can the software be installed? (Choose the best answer.)

A.  Into the database Oracle Home if it is the same release

B.  Into an Oracle Home on a machine accessible to all databases over a network

C.  Into a dedicated Oracle Home on each machine running a database

D.  All of the above

8.  What utilities can you use to start and stop the GI High Availability Services? (Choose two correct answers.)

A.  crsctl

B.  SQL*Plus

C.  srvctl

D.  Windows net start utility

Self Test Answers

1.      B, C, D, and F. You can use ASM for database files, such as the controlfile, the datafiles, the tempfiles, and the online log files.
    A and E are incorrect. A is incorrect because the alert and trace files must be on conventional storage, and E is incorrect because your Oracle Home must be on conventional storage as well.

2.      A, B, C, and E. Archive logs, RMAN backups and image copies, and indeed the whole flash recovery area can be on ASM.
    D is incorrect. You cannot direct user-managed backups to ASM because operating system utilities cannot write to ASM devices.

3.      C. By default, files are both striped and mirrored because this is the default when creating a disk group and will be applied to all files unless specified otherwise.
    A, B, and D are incorrect. A is incorrect because normal redundancy mirroring is the default. B and D are incorrect because while you can disable mirroring by using the EXTERNAL REDUNDANCY option, you cannot disable striping (and you would not want to).

4.      C. This is probably the best way to use ASM: to rely on an LVM to provide fault tolerance and ASM to provide Oracle-aware striping.
    A, B, and D are incorrect. A is incorrect because an LVM can be used to create the disks presented to ASM. B is incorrect because LVMs must be presented as raw devices, not as “cooked” file systems. D is incorrect because you can (and should) use two layers if striping: first at the hardware level over physical volumes, then ASM will stripe on top of these.

5.      D. Both password file and operating system authentication will work.
    A, B, and C are incorrect. A and B are incorrect because both techniques can be used. C is incorrect because an ASM instance has no data dictionary.

6.      D. ASM stripes files across all disks in the group.
    A, B, and C are incorrect. A is incorrect because one file can exist only in one group. B and C are incorrect because it is file extents that are striped, not disks or diskgroups.

7.      C. GI must be installed into its own home directory.
    A, B, and D are incorrect. A is incorrect because it is not possible for the GI and database binaries to coexist in one Oracle Home. B and D are incorrect because it is not possible for a database to contact GI over a network connection.

8.      A and D. crsctl is the Oracle-provided tool that can manage the GI processes, and on Windows the service manager can also be used.
    B and C are incorrect. B is incorrect because SQL*Plus can manage an ASM instance, but not GI. C is incorrect because srvctl can manage GI registered resources, but not GI itself.