Resolving failures in multiple locations

Identify the failure locations from the SIMs issued by the primary and secondary storage systems and using the SAN management software, and then recover from the failures.

If data has been lost from both volumes, recover from the backup data using ShadowImage, Thin Image, or Thin Image Advanced volumes, or backup software.

If I/O is stopped, resume I/O from the server.

If GAD pairs are suspended, resynchronize the pairs.

If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation.

If the pairs cannot be resynchronized, perform the following recovery procedures depending on the pair status and I/O mode.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
PSUE	COPY	Local	Block

Delete the pair forcibly from the S-VOL.
When you perform this step, delete the virtual LDEV ID so that the volume cannot be accessed from the server.
```
pairsplit -g oraHA -d dev1 -RF -IH2
```
Check if the virtual LDEV ID of the P-VOL is not deleted.
```
raidcom get ldev -ldev_id 0x1111 -fx -IH1
(Omitted)
LDEV : 1111
(Omitted)
```
The VIR_LDEV information is not displayed if it is the same as the LDEV information. If the virtual LDEV ID is deleted, set a correct virtual LDEV ID.
Delete the pair forcibly from the P-VOL.
When you perform this step, make sure not to delete the virtual LDEV ID, which allows the volume to be accessed from the server.
```
pairsplit -g oraHA -d dev1 -SFV -IH1
```
Check if the virtual LDEV ID of the S-VOL indicates GAD reserve.
```
raidcom get ldev -ldev_id 0x2222 -fx -IH2
(Omitted)
LDEV : 2222
VIR_LDEV : ffff	
(Omitted)
```
VIR_LDEV: ffff indicates GAD reserve. If it shows another value, set a correct virtual LDEV ID.
Re-create the pair by specifying the P-VOL.
```
paircreate -g oraHA -d dev1 -f never -vl -jq 1 -IH1
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation. If you accidentally delete the virtual LDEV ID of the P-VOL, you cannot create a GAD pair again. See Setting correct virtual LDEV ID values and set the correct value for the virtual LDEV ID.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
SMPL	COPY	Not applicable	Block

Delete the pair forcibly from the S-VOL.
When you perform this step, delete the virtual LDEV ID so that the volume cannot be accessed from the server.
```
pairsplit -g oraHA -d dev1 -RF -IH2
```
Check if the virtual LDEV ID of the S-VOL indicates GAD reserve.
```
raidcom get ldev -ldev_id 0x2222 -fx -IH2
(Omitted)
LDEV : 2222
VIR_LDEV : ffff	
(Omitted)
```
VIR_LDEV: ffff indicates GAD reserve. If it shows another value, set a correct virtual LDEV ID.
Check if the virtual LDEV ID of the P-VOL is not deleted.
```
raidcom get ldev -ldev_id 0x1111 -fx -IH1
(Omitted)
LDEV : 1111
(Omitted)
```
The VIR_LDEV information is not displayed if it is the same as the LDEV information. If the virtual LDEV ID is deleted, set a correct virtual LDEV ID.

Note: If shared memory in the secondary storage system becomes volatilized, the S-VOL pair status changes to SMPL, and GAD reserve is assigned to the virtual attribute.
Re-create the pair by specifying the P-VOL.
```
paircreate -g oraHA -d dev1 -f never -vl -jq 1 -IH1
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation. If you accidentally delete the virtual LDEV ID of the P-VOL, you cannot create a GAD pair again. See Setting correct virtual LDEV ID values and set the correct value for the virtual LDEV ID.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
PSUS/PSUE	SSWS	Block	Local

Resynchronize the pair by specifying the S-VOL.
```
pairresync -g oraHA -d dev1 -swaps -IH2
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
SMPL	SSWS	Not applicable	Local

Check if the virtual LDEV ID of the P-VOL indicates GAD reserve.
```
raidcom get ldev -ldev_id 0x1111 -fx -IH1
(Omitted)
LDEV : 1111
VIR_LDEV : ffff	
(Omitted)
```
VIR_LDEV: ffff indicates GAD reserve. If it shows another value, set a correct virtual LDEV ID.
Delete the pair forcibly from the S-VOL.
When you perform this step, make sure not to delete the virtual LDEV ID, which allows the volume to be accessed from the server.
```
pairsplit -g oraHA -d dev1 -RFV -IH2
```

Check if the virtual LDEV ID of the S-VOL is not deleted.

raidcom get ldev -ldev_id 0x2222 -fx -IH2
(Omitted)
LDEV : 2222
VIR_LDEV : 1111	
(Omitted)

Re-create the pair by specifying the S-VOL.
```
paircreate -g oraHA -d dev1 -f never -vl -jq 1 -IH2
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation. If you accidentally delete the virtual LDEV ID of the S-VOL, you cannot create a GAD pair again. See Setting correct virtual LDEV ID values and set the correct value for the virtual LDEV ID.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
PSUS/PSUE	SSUS/PSUE	Local	Block

Resynchronize the pair by specifying the P-VOL.
```
pairresync -g oraHA -d dev1 -IH1
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
SMPL	SSUS/PSUE	Not applicable	Block

Delete the pair forcibly from the S-VOL.
When you perform this step, make sure that you delete the virtual LDEV ID so that the volume cannot be accessed from the server.
```
pairsplit -g oraHA -d dev1 -RF -IH2
```
Check if the virtual LDEV ID of the S-VOL indicates GAD reserve.
```
raidcom get ldev -ldev_id 0x2222 -fx -IH2
(Omitted)
LDEV : 2222
VIR_LDEV : ffff	
(Omitted)
```
VIR_LDEV: ffff indicates GAD reserve. If it shows another value, set a correct virtual LDEV ID.
Check if the virtual LDEV ID of the P-VOL is not deleted.
```
raidcom get ldev -ldev_id 0x1111 -fx -IH1
(Omitted)
LDEV : 1111
(Omitted)
```
The VIR_LDEV information is not displayed if it is the same as the LDEV information. If the virtual LDEV ID is deleted, set a correct virtual LDEV ID.

Note: If shared memory in the secondary storage system becomes volatilized, the S-VOL pair status changes to SMPL, and GAD reserve is assigned to the virtual attribute.
Re-create the pair by specifying the P-VOL.
```
paircreate -g oraHA -d dev1 -f never -vl -jq 1 -IH1
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation. If you accidentally delete the virtual LDEV ID of the P-VOL, you cannot create a GAD pair again. See Setting correct virtual LDEV ID values and set the correct value for the virtual LDEV ID.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
PSUE	PSUE	Block	Block

Delete the pair forcibly from the S-VOL.
When you perform this step, delete the virtual LDEV ID so that the volume cannot be accessed from the server.
```
pairsplit -g oraHA -d dev1 -RF -IH2
```
Check if the virtual LDEV ID of the S-VOL indicates GAD reserve.
```
raidcom get ldev -ldev_id 0x2222 -fx -IH2
(Omitted)
LDEV : 2222
VIR_LDEV : ffff	
(Omitted)
```
VIR_LDEV: ffff indicates GAD reserve. If it shows another value, set a correct virtual LDEV ID.
Delete the pair forcibly from the P-VOL.
When you perform this step, make sure not to delete the virtual LDEV ID, which allows the volume to be accessed from the server.
```
pairsplit -g oraHA -d dev1 -SFV -IH1
```
Check if the virtual LDEV ID of the P-VOL is not deleted.
```
raidcom get ldev -ldev_id 0x1111 -fx -IH1
(Omitted)
LDEV : 1111
(Omitted)
```
The VIR_LDEV information is not displayed if it is the same as the LDEV information. If the virtual LDEV ID is deleted, set a correct virtual LDEV ID.
Re-create the pair by specifying the P-VOL.
```
paircreate -g oraHA -d dev1 -f never -vl -jq 1 -IH1
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation. If you accidentally delete the virtual LDEV ID of the P-VOL, you cannot create a GAD pair again. See Setting correct virtual LDEV ID values and set the correct value for the virtual LDEV ID.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
PSUS/PSUE	PSUS/PSUE	Local	Block

Resynchronize the pair by specifying the P-VOL.
```
pairresync -g oraHA -d dev1 -IH1
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
PSUS/PSUE	SMPL	Local	Not applicable

Check if the virtual LDEV ID of the S-VOL indicates GAD reserve.
```
raidcom get ldev -ldev_id 0x2222 -fx -IH2
(Omitted)
LDEV : 2222
VIR_LDEV : ffff	
(Omitted)
```
VIR_LDEV: ffff indicates GAD reserve. If it shows another value, set a correct virtual LDEV ID.
Delete the pair forcibly from the P-VOL.
When you perform this step, make sure not to delete the virtual LDEV ID, which allows the volume to be accessed from the server.
```
pairsplit -g oraHA -d dev1 -SFV -IH1
```
Check if the virtual LDEV ID of the P-VOL is not deleted.
```
raidcom get ldev -ldev_id 0x1111 -fx -IH1
(Omitted)
LDEV : 1111
(Omitted)
```
The VIR_LDEV information is not displayed if it is the same as the LDEV information. If the virtual LDEV ID is deleted, set a correct virtual LDEV ID.
Re-create the pair by specifying the P-VOL.
```
paircreate -g oraHA -d dev1 -f never -vl -jq 1 -IH1
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation. If you accidentally delete the virtual LDEV ID of the P-VOL, you cannot create a GAD pair again.

Pair status		I/O mode
P-VOL	S-VOL	P-VOL	S-VOL
PSUS/PSUE	SMPL	Block	Not applicable

Delete the pair forcibly from the P-VOL.
When you perform this step, make sure that you delete the virtual LDEV ID so that the volume cannot be accessed from the server.
```
pairsplit -g oraHA -d dev1 -SF -IH1
```
Check if the virtual LDEV ID of the P-VOL indicates GAD reserve.
```
raidcom get ldev -ldev_id 0x1111 -fx -IH1
(Omitted)
LDEV : 1111
VIR_LDEV : ffff	
(Omitted)
```
VIR_LDEV: ffff indicates GAD reserve. If it shows another value, set a correct virtual LDEV ID.
Check if the virtual LDEV ID of the S-VOL is not deleted.
```
raidcom get ldev -ldev_id 0x2222 -fx -IH2
(Omitted)
LDEV : 2222
VIR_LDEV : 1111
(Omitted)
```
If the virtual LDEV ID is deleted, set a correct virtual LDEV ID.

Note: If shared memory in the secondary storage system becomes volatilized, the S-VOL pair status changes to SMPL, and GAD reserve is assigned to the virtual attribute.
Re-create the pair by specifying the S-VOL.
```
paircreate -g oraHA -d dev1 -f never -vl -jq 1 -IH2
```
If a failure occurs after the one volume capacity of a GAD pair can be expanded, the creation, resync, swap resync, and horctakeover operations of the GAD pair cannot be performed because the capacity of both the volumes is not the same. Make sure to expand the other volume capacity so that the capacity of both the volumes is the same, and then retry the operation. If you accidentally delete the virtual LDEV ID of the S-VOL, you cannot create a GAD pair again. See Setting correct virtual LDEV ID values and set the correct value for the virtual LDEV ID.