Settings for Oracle

When you use Oracle Cluster file System 2 in an Oracle RAC environment, set the O2CB_HEARTBEAT_THRESHOLD parameter in the /etc/sysconfig/o2cb file to the following value:

If the VSP One B20, VSP E series, VSP 5000 series, VSP Fx00 models, VSP Gx00 models, or VSP N series is being used:

(number-of-paths-connected-to-Oracle-Cluster-file-System-2 x 60 seconds / 2) + 1

When a host and an Oracle RAC or Oracle SEHA voting disk are connected by multiple paths, HDLM performs failover processing for those paths (in the same way as for normal paths) when an I/O timeout occurs for one of the paths.

Note that, depending on the settings of Oracle, Oracle might determine that a node error has occurred before the failover processing performed by HDLM is completed, and then reconfigure the cluster.

Therefore, if an Oracle RAC or Oracle SEHA voting disk is connected to an HDLM device, depending on the type or the operational requirements of the storage system, change the values of MISSCOUNT and DISKTIMEOUT for Oracle RAC 11g, Oracle RAC 12c, Oracle RAC 18c, Oracle RAC 19c, and Oracle SEHA 19c.

When prioritizing the availability of the storage system: For MISSCOUNT, specify the maximum allowed time before it is determined that the heartbeat cannot be recognized. For DISKTIMEOUT, specify the maximum allowed time before it is determined that an I/O operation for a voting disk cannot be performed. Calculate values to be specified by following the descriptions in Table 1 and Table 2, and then change the current value to a value greater than or equal to the value you have obtained. If availability is to be considered, the greater the number of paths, the greater the values to be specified for MISSCOUNT and DISKTIMEOUT will be, and the time before the reconfiguration of Oracle RAC starts becomes longer.

When prioritizing the time before the reconfiguration of Oracle RAC or Oracle SEHA starts: To reduce the time before the reconfiguration of Oracle RAC or Oracle SEHA starts, determine the values of MISSCOUNT and DISKTIMEOUT from the maximum time allowed before the reconfiguration of Oracle RAC or Oracle SEHA starts. In this case, the values of MISSCOUNT and DISKTIMEOUT might become smaller than the values obtained by the calculation formula. This will not affect HDLM operation. However, if an error that requires detection time (such as a timeout error) occurs, the values of MISSCOUNT and DISKTIMEOUT become smaller than the time taken for a failover to an online path to be completed. This increases the likelihood that the reconfiguration of Oracle RAC or Oracle SEHA will start, even if a failover is possible.

The number of paths contained in the formula for calculating MISSCOUNT and DISKTIMEOUT indicates the number of paths that connect a voting disk to the host for which MISSCOUNT and DISKTIMEOUT are to be set. If connections are established from a target host through two paths to the voting disk, the number of path will be 2.

Table. Formula for calculating MISSCOUNT
Storage system type	Formula for obtaining the value of MISSCOUNT
VSP One B20 VSP E series VSP 5000 series VSP Fx00 models VSP Gx00 models VSP N series	`number-of-paths-connected-to-the-voting-disk` x 60 seconds

Table. Formula for calculating DISKTIMEOUT
Storage system type	Number of paths connected to the voting disk	Formula for obtaining the value of DISKTIMEOUT
VSP One B20 VSP E series VSP 5000 series VSP Fx00 models VSP Gx00 models VSP N series	3 or less	You do not need to change the value of `DISKTIMEOUT`.
	4 or more	`number-of-paths-connected-to-the-voting-disk` x 60 seconds

Note

If the value of MISSCOUNT is greater than or equal to that of DISKTIMEOUT, an error might occur as a result of the specifications of Oracle. In such cases, in addition to changing the value of MISSCOUNT, specify settings so that the value of MISSCOUNT is less than that of DISKTIMEOUT, for example, by adding 1 to the value of DISKTIMEOUT.

For details, contact the company with which you have a contract for Oracle Support Services.

Note that when you remove HDLM from the above configuration, you must reset the values of MISSCOUNT and DISKTIMEOUT to their original values. Therefore, make a note of the original values of MISSCOUNT and DISKTIMEOUT before changing them.

If the Oracle command oracleasm is executed for HDLM devices, the KAPL05023-E message might be output to syslog.

If the following message indicating the successful completion of the oracleasm command is output after this message, there is no problem with HDLM operation.

oracleasm:  succeeded

If you use HDLM with Oracle RAC 12c, Oracle RAC 18c, or Oracle RAC 19c, and use ASMLib for Oracle RAC, you must change the settings in the ASMLib configuration file.

After installing ASMLib, do the following for each node:

Execute the following command to create the ASMLib configuration file:
- For Oracle RAC 12c, or ASMLib 2.1.0 or later:
  
  # /usr/sbin/oracleasm configure -i
- For ASMLib versions earlier than 2.1.0:
  
  # /etc/init.d/oracleasm configure
Open the ASMLib configuration file (/etc/sysconfig/oracleasm).
Change the ORACLEASM_SCANORDER line as follows:

ORACLEASM_SCANORDER ="sddlm"

If an ASM disk defined in an HDLM device already exists, perform this procedure, and then restart all the nodes.

This step must be done to enable the settings.

If you use ASMLib in an environment where HDLM is used with Oracle RAC 12c, Oracle RAC 18c, or Oracle RAC 19c, the CPU load might remain high for several minutes after the oracleasm scandisk command is executed.

You can avoid this by excluding devices other than those with the sdd prefix from scandisk targets, so that devices with the sddlm prefix are not excluded. Add the following settings to the ASMLib configuration file (/etc/sysconfig/oracleasm).

ORACLEASM_SCANEXCLUDE="sda sdb sdc sde sdf sdg sdh sdi sdj sdk sdl sdm sdn sdo sdp sdq sdr sds sdt sdu sdv sdw sdx sdy sdz"

If there is already an ASM disk defined for the HDLM device, perform this step, and then restart all the nodes.

This step must be done to enable the settings.

After configuring the above settings, specify and execute the oracleasm scandisk command as follows.

# /usr/sbin/oracleasm scandisks /dev/sddlm*

If you use ASMFD in an environment where HDLM is used with Oracle RAC 19c, and you want to use ASMFD to specify an HDLM device file (block device) for the disk area managed by ASM, use the following commands to enable HDLM devices. If you use the setting /dev/sd* as is, HDLM devices will not be recognized, so you must specify the setting. By specifying the setting, only HDLM devices will be recognized.

# $ORACLE_HOME/bin/asmcmd afd_dsget AFD discovery
string: /dev/sd*
# $ORACLE_HOME/bin/asmcmd afd_dsset '/dev/sddlm*'
# $ORACLE_HOME/bin/asmcmd afd_dsget AFD discovery
string: /dev/sddlm*

For details about ASMFD and the commands, see the Oracle documentation.

If you specify an HDLM device file (block device) for an Oracle ASM disk in an environment where HDLM is used with Oracle RAC 19c, note the following two points:

For an Oracle ASM disk, specify an HDLM device file that is created by default (example: /dev/sddlmaa) rather than specifying a renamed HDLM device file or a link file that is linked to an HDLM device file. In addition, you can specify a partition.

For details on how to specify an HDLM device file, see the Oracle documentation.
If you are using a udev rule file (example: /etc/udev/rules.d/99-oracle-asmdevices.rules) to change the attributes of an HDLM device file, specify the HDLM device name for KERNEL.

Note that if you additionally specify keys such as BUS="scsi" at the same time, the attribute cannot be changed correctly. For details on udev rule files, see the OS documentation.

The following shows an example of the /etc/udev/rules.d/99-oracle-asmdevices.rules file:
```
KERNEL=="sddlmaa", OWNER=="oracle", GROUP=="oinstall", MODE="0660"
KERNEL=="sddlmab", OWNER=="oracle", GROUP=="oinstall", MODE="0660"
KERNEL=="sddlmac", OWNER=="oracle", GROUP=="oinstall", MODE="0660"
KERNEL=="sddlmad", OWNER=="oracle", GROUP=="oinstall", MODE="0660"
```