Registering drive replacement information

[On the controller node] Verify whether drives in the Blockage status exist.

CLI: drive_list

If you receive no response after waiting approximately 1 minute, stop the CLI command forcibly with Ctrl+C, and then retry the operation. If the problem persists, contact the Hitachi design team.

The required parameters are as follows:

--ignore_certificate_errors
--user <user-with-Security-or-Storage-or-Monitor-or-Service-role>
--host <IP-address-obtained-in-step-1-in-Restoring>
--status Blockage

You will be prompted to enter the password after running the command. Ask the system administrator to enter the password.

If drives with the Blockage status exist, go to the next step. If there is no drive with the Blockage status, you do not need to take action in this section. Proceed to the next section.

[On the controller node] Make an SSH connection to the storage node selected in step 1 of Restoring.

Perform the subsequent operations of [On the storage node] for the target one node.

$ ssh -i <passphrase-of-private-key> -o UserKnownHostsFile=/dev/null \
-o StrictHostKeyChecking=no \
hitachi_rescue_person@<IP-address-of-storage-node>

Note:

The target private key files are the ones that you obtained in step 4 of Collecting information required for restore and that do not include ".pub" in their file names. When you created the key files as shown in the execution example of step 4, key files sample_ed25519 and sample_ed25519.pub exist, and sample_ed25519 serves as a private key file.

[On the storage node] Verify the IDs for each pair of faulty drive and new drive recorded in step 1 of Preparing reinstallation of a storage cluster in Reinstalling the storage cluster. The status of the displayed faulty drive and new drive is Blockage and Offline, respectively.

$ sudo /usr/local/bin/hsds-configuration-restore-replace-drive-list
StorageNode: sample-SN01 (00000000-0000-0000-0000-000000000001)
    Drives:
        id: <ID-of-faulty-drive>
        vendor name: Sample Vendor
        model name: Sample Disk
        wwid: xxx
        capacity: 1024
        status: Blockage   * This status indicates a faulty drive. 

        id: <ID-of-new-drive>
        vendor name: Sample Vendor
        model name: Sample Disk
        wwid: yyy
        capacity: 1024
        status: Offline   * This status indicates a new drive.

CAUTION:

If the WWID (WWN or EUI) of a drive that is not one of a replacement pair recorded in step 1 of Preparing reinstallation of a storage cluster in Reinstalling the Storage Cluster is displayed in the Blockage state, a normal drive might have been inadvertently removed. Also, if no WWID (WWN or EUI) for a faulty drive and new drive pair exists, the new drive might not be inserted, or a drive failure might have occurred.

In either case, drive replacement has not been properly performed. Verify whether you removed a normal drive inadvertently, the SKU (for more information, see VSP One SDS Block Bare Metal Hardware Compatibility Reference) of the new drive, its installation location, and whether there is any failure. If there is still a problem with drive replacement, replace the drive again. After that, reperform the operation from Installing the storage software in Reinstalling the storage cluster.

[On the controller node] Verify the storage pool ID.

CLI: pool_list

If you receive no response after waiting approximately 1 minute, stop the CLI command forcibly with Ctrl+C, and then retry the operation. If the problem persists, contact the Hitachi design team.

The required parameters are as follows:

--ignore_certificate_errors
--user <user-with-Security-or-Storage-or-Monitor-or-Service-role>
--host <IP-address-obtained-in-step-1-in-Restoring>

You will be prompted to enter the password after running the command. Ask the system administrator to enter the password.

[On the controller node] Expand the storage pool for the new drive that you verified in step 3.

Run the following command with the storage pool ID and the drive IDs of all the new drives specified.

CLI: pool_expand

If you receive no response after waiting approximately 1 minute, stop the CLI command forcibly with Ctrl+C, and then retry the operation. If the problem persists, contact the Hitachi design team.

The required parameters are as follows:

--ignore_certificate_errors
--user <user-with-Storage-role>
--host <IP-address-obtained-in-step-1-in-Restoring>
--id <Storage-pool-ID>
--drive_ids <ID-of-the-new-drive-that-you-verified-in-step-3>

You will be prompted to enter the password after running the command. Ask the system administrator to enter the password.

Note:

Performing this procedure might cause a warning event log to be output but this is a temporary message accompanying restore operation. Disregard this message and continue the subsequent operations.

[On the controller node] Verify whether the storage pool has been successfully expanded.

Run the following command with the job ID specified.

CLI: job_show

If you receive no response after waiting approximately 1 minute, stop the CLI command forcibly with Ctrl+C, and then retry the operation. If the problem persists, contact the Hitachi design team.

The required parameters are as follows:

--ignore_certificate_errors
--user <user-with-Security-or-Storage-or-Monitor-or-Service-role>
--host <IP-address-obtained-in-step-1-in-Restoring>
--job_id <Job-ID-of-expanding-the-storage-pool>

You will be prompted to enter the password after running the command. Ask the system administrator to enter the password.

If the job state is "Succeeded", the job is completed.

[On the storage node] Register drive replacement information.

Note:

Perform steps 7 and 8 for each pair of faulty drive and new drive recorded in step 1 of Preparing reinstallation of a storage cluster in Reinstalling the storage cluster.

$ sudo /usr/local/bin/hsds-configuration-restore-replace-drive \
--targetStorageDevice <ID-of-new-drive> \
--sourceStorageDevice <ID-of-faulty-drive>

CAUTION:

Verify again that the paired faulty drive and new drive are of the same SKU (for more information, see VSP One SDS Block Bare Metal Hardware Compatibility Reference) and that the installation location is the same before and after the hardware replacement. If a faulty drive and a new drive are improperly paired when registering drive replacement information, you might need to perform the restore procedure again.

[On the controller node] Verify that the information is properly registered.

CLI: event_log_list

If you receive no response after waiting approximately 1 minute, stop the CLI command forcibly with Ctrl+C, and then retry the operation. If the problem persists, contact the Hitachi design team.

The required parameters are as follows:

--ignore_certificate_errors
--user <user-with-Security-or-Storage-or-Monitor-or-Service-role>
--host <IP-address-obtained-in-step-1-in-Restoring>

You will be prompted to enter the password after running the command. Ask the system administrator to enter the password.

If the event log with the ID and message as shown in the following is output, the operation is completed.

Message Id: KARS13010-I

Message: The job has completed successfully. (Operation = DRIVE_REPLACE, Job ID = xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxxxxxx)

CAUTION:

If message KARS13011-I is output, drive replacement information could not be registered. Run the following command with the job ID indicated in the message specified.

CLI: job_show

If you receive no response after waiting approximately 1 minute, stop the CLI command forcibly with Ctrl+C, and then retry the operation. If the problem persists, contact the Hitachi design team.

The required parameters are as follows:
- --ignore_certificate_errors
- --user <user-with-Security-or-Storage-or-Monitor-or-Service-role>
- --host <IP-address-obtained-in-step-1-in-Restoring>
- --job_id <ID-of-the-job-indicated-in-the-event-log>
You will be prompted to enter the password after running the command. Ask the system administrator to enter the password.
If message KARS07262-E or KARS07263-E is output, a drive ID is not specified in the command in step 7. Specify the drive ID, and then perform the task in step 7.
If message KARS07264-E, KARS07265-E, KARS07266-E, or KARS07267-E is output, an incorrect drive ID is specified in the command in step 7. Review the drive ID, and then perform the task in step 7.
If message KARS07268-E is output, capacity of the new drive is less than that of the faulty drive to be replaced. The drive might not have been replaced properly.

Verify that a replacement pair of failed drive and new drive has the same model name (see the VSP One SDS Block Bare Metal Hardware Compatibility Reference), and that the installation location is the same before and after the hardware replacement. To do so, see the drive information recorded in step 1 of Preparing reinstallation of a storage cluster in Reinstalling the storage cluster.

After confirming that a replacement pair is correct for all the drives to be replaced, reperform the operation from Installing the storage software in Reinstalling the storage cluster.

[On the controller node] Verify that there are no faulty drives.

CLI: drive_list

If you receive no response after waiting approximately 1 minute, stop the CLI command forcibly with Ctrl+C, and then retry the operation. If the problem persists, contact the Hitachi design team.

The required parameters are as follows:

--ignore_certificate_errors
--user <user-with-security-storage-monitor-or-service-role>
--host <IP-address-obtained-in-step-1-of-Restoring>
--status Blockage

You will be prompted to enter a password after running the command. Request the system administrator to enter the password.

If drive information is not displayed, information registration for drive replacement is completed. If drives with the Blockage status exist, reperform the procedure from step 3.