Troubleshooting V-VOLs, pools, and capacity saving

• Usage of the pool has reached 100%.
• Something in the storage system is blocked.
• The maximum reservation value set for the pool limits the capacity of V-VOLs that can be created.
• The total capacity for the V-VOLs to be created exceeds the capacity for the estimated configurable V-VOL capacity.

  The estimated configurable V-VOL capacity can be confirmed by the following VSP block storage REST API and its attributes:

  GET <base URL>
  /simple/v1/objects/storage?
  withEstimatedConfigurableCapacities=true

  Attributes:

  • estimatedConfigurableMainframePoolCapacity
  • estimatedConfigurableMainframeVolumeCapacity

  For more information about how to use VSP block storage REST API, see the Hitachi Virtual Storage Platform Block Storage REST API Reference.

• The capacity requirements for deduplication system data volume are not satisfied.

Solutions:

• Add pool-VOLs to the pool. (See the raidcom add dp_pool command).
• Perform the task to reclaim zero pages to release pages in which zero data are stored. (See the raidcom modify ldev command).
• Revise the setting of the subscription limit of the pool. (See the raidcom modify pool command.)
• Add shared memory, and then retry the operation. Alternatively delete V-VOLs that are not used. (See the raidcom delete ldev command.)
• Contact customer support to solve the problem.

• 1,024 pool-VOLs are already defined in the pool.
• The pool-VOL does not fulfill the requirements for a pool-VOL.
• Something in the storage system is blocked.

Solution:

Change the setting of the LDEV to satisfy the requirement of the pool-VOL. See Pool-VOL requirements.

Among the asynchronous commands in the group to which the request (multiple LDEV creations) is assigned, one or more asynchronous commands cannot be run.

With CCI, VSP One Block Administrator, and Simple API, requests are processed using asynchronous commands internally. When multiple asynchronous command requests are received, they are grouped and run the asynchronous commands for each group. For details about multiple asynchronous command executions, see the Command Control Interface Command Reference.

Solution:

Create LDEVs one at a time.

• The number of LDEV creation requests assigned to a group exceeds the number of available unused LDEV numbers in that group.
• When the storage system receives multiple LDEV creation requests, it divides them into several groups and processes LDEV creation for each group. If the number of LDEV creation requests assigned to a group is greater than the number of available unused LDEV numbers, this error occurs and all LDEV creation requests assigned to the affected group fails.
• Conversely, if the number of available unused LDEV numbers in a group is sufficient for the number of assigned LDEV creation requests, all requests succeed for that group. Therefore, when multiple LDEV creation requests are issued to the storage system, some groups might successfully create LDEVs while others might fail.
• When a range of LDEV numbers has been specified, the specified range includes LDEV numbers that can only be used for mainframe system LDEVs. (For example, if a certain LDEV#X is used as a mainframe system LDEV, open system LDEVs cannot be created in the range from ↓(X/32)↓* 32 to ↓(X/32)↓* 32 + 31.)
• When a range of LDEV numbers has been specified, the specified range includes LDEV numbers used for PAV alias devices.

Solutions:

• If you have specified a range of LDEV numbers, increase the range and try creating the LDEV again.
• If there are unnecessary LDEVs, delete the unnecessary LDEVs and try creating the LDEV again.
• If you have specified a range of LDEV numbers, make sure that the range does not include LDEV numbers that can only be used for mainframe system LDEVs, and then try creating the LDEV again.
• If you have specified a range of LDEV numbers, make sure that the range does not include LDEV numbers used for PAV alias devices (aliases), and then try creating the LDEV again.

After performing the following operations, the write pending rate of the MP unit to which the target volume to be deleted is assigned has dropped below 70% by using System Monitor in VSP One Block Administrator:

• Change the volume assignment on the MP unit
• Delete a Thin Image Advanced pair
• Delete a ShadowImage pair
• Migrate a volume by using Volume Migration
• Delete a journal volume by using Universal Replicator

Solutions:

• Verify that the write pending rate of the MP unit to which the target volume to be deleted is assigned has dropped below 70% by using System Monitor in VSP One Block Administrator, and then retry the deletion operation after waiting about 10 minutes.
• Contact customer support to resolve the problem.

A failure occurred in data drives greater than the parity group redundancy.

Solution:

Ask customer support to solve the problem.

Ask customer support to solve the problem.

• Processing takes time because something in the storage system is blocked.
• Usage of the pool has reached 100%.

Solutions:

• Add some pool-VOLs to the pool to increase the capacity of the pool. (See the raidcom add dp_pool command.)
• Perform the operation to reclaim zero pages to release pages in which zero data are stored. (See the raidcom modify ldev command.)
• Delete V-VOLs that are not used. (See the raidcom delete ldev command.)
• Ask customer support to solve the problem.

• The pool usage is not 0.
• External volumes are removed from the pool before you delete the pool.
• V-VOLs have not been deleted.

Solutions:

• Confirm that the pool usage is 0 after the V-VOLs are deleted, and that you can delete the pool.
• Ask customer support to solve the problem.

• Free space of the pool is insufficient.
• Some areas in the storage system are blocked.

Solutions:

• Check the free space of the pool and increase the capacity of the pool.
• Perform the operation to reclaim zero pages to release pages in which zero data are stored. (See the raidcom modify ldev command.)
• Delete V-VOLs that are not used. (See the raidcom delete ldev command.)
• Ask customer support to solve the problem.

• Free space of the pool is insufficient.
• Some areas in the storage system are blocked.

Solutions:

• Check the free space of the pool and increase the capacity of the pool.
• Reclaim zero pages in order to release pages in which zero data are stored. (See the raidcom modify ldev command.)
• Delete V-VOLs that are not used. (See the raidcom delete ldev command.)
• Ask customer support to solve the problem.

Causes:

• The total capacity for the V-VOLs to be expanded exceeds the capacity for the estimated configurable V-VOL capacity.
  The estimated configurable V-VOL capacity can be confirmed by the following VSP Block Storage REST API and its attributes:

  GET <base URL>/simple/v1/objects/storage?
  withEstimatedConfigurableCapacities=true

  Attributes:

  • estimatedConfigurableMainframePoolCapacity
  • estimatedConfigurableMainframeVolumeCapacity

  For more information about how to use VSP Block Storage REST API, see the VSP Block Storage REST API Reference Guide.

Solutions:

• Verify whether the processing for increasing V-VOL capacity meets the conditions described in Working with DP-VOLs.
• Retry the operation after 10 minutes or so.
• Ask customer support to solve the problem.

Zero pages in the V-VOL cannot be reclaimed because the V-VOL does not meet conditions for releasing pages in a V-VOL. See raidcom modify ldev in Command Control Interface Command Reference.

Solution:

Make sure that the V-VOL meets the conditions described in .

Pages of the V-VOL are not released because the process of reclaiming zero pages was interrupted. See raidcom modify ldev in Command Control Interface Command Reference.

Solution:

Make sure that the V-VOL meets the conditions described in .

• The pool is full.
• The pool-VOL is blocked.
• The pool-VOL that is an external volume is blocked.

Solutions:

• Add pool-VOLs to the pool to increase the free space in the pool. (See the raidcom add dp_pool command.)
• Perform the reclaiming zero pages task to release pages in which zero data are stored. (See the raidcom modify ldev command.)
• Delete V-VOLs that are not used. (See the raidcom delete ldev command.)
• Contact customer support to restore the pool-VOL.
• If the blocked pool-VOL is an external volume, verify the status of the path blockade and the external storage system.
• After performing above solutions, release the protect attribute (Data Retention Utility) of the V-VOL by using the raidvchkset command.

Usage of the pool has reached 100%.

Solutions:

• Add pool-VOLs to the pool to increase the free space in the pool.
• Perform the task to reclaim zero pages in order to release pages in which zero data are stored.
• Delete V-VOLs that are not used.

The protect attribute of Data Retention Utility might have been set on V-VOLs. After performing the above solutions, release the protect attribute of the V-VOLs.

• Another pool is being formatted.
• The pool usage level reaches up to the threshold.
• The pool is full.
• The pool is blocked.
• I/O loads to the storage system are high.
• The cache memory is blocked.
• Pool-VOLs are blocked.
• Pool-VOLs that are external volumes are blocked.

Solutions:

• Confirm the display again after waiting for a while.
• Add pool-VOLs to the pool to increase the free space in the pool. (See the raidcom add dp_pool command.)
• Perform the task to reclaim zero pages to release pages in which zero data are stored. (See the raidcom modify ldev command.)
• Confirm the display again after decreasing I/O loads of the storage system.
• Delete V-VOLs that are not used. (See the raidcom delete ldev command.)
• Contact customer support to restore the cache memory.
• Contact customer support to restore the pool-VOL.
• If the blocked pool-VOL is an external volume, confirm these situations:
  • Path blockage
  • Status of the storage system

• Cache memory maintenance was performed while the shrink pool operation was in progress.
• A cache memory error occurred while the shrink pool operation was in progress.
• The I/O load to V-VOLs associated with the pool is too high.
• V-VOLs associated with the pool are blocked.

Solutions:

• Restore the cache memory, and then perform the shrink pool operation again.
• Wait until the I/O load to V-VOLs associated with the pool is lower, and then perform the shrink pool operation again.
• Delete or format V-VOLs associated with the pool, and then perform the shrink pool operation again.

An error occurs while the capacity saving status is changing, and then the processing aborts. After recovering from the error, the resumed status change process fails.

Solution:

Delete the target volume, and then retry the operation.

• The shared memory is volatilized and the storage system is restarted.
• The pool is initialized.
• The pool volumes are formatted.
• The processing for deleting the V-VOLs failed.

Solution:

Format the operation target V-VOLs.

• The shared memory is volatilized and the storage system is restarted.
• The pool is initialized.
• The pool volumes are formatted.
• The processing for deleting the V-VOL failed.

Solution:

1. Back up both of the following volumes:
   • All V-VOLs with deduplication data enabled that are associated with the pool
   • A V-VOL with the vClone attribute that belongs to a family of a V-VOL with the vClone Parent attribute enabled and with deduplication data enabled

   Note: During the backup, SIM code 680001 might be reported and the data might not be read. In this case, back up only the data that can be read. Even if the SIM code 680001 is reported, perform this operation.

2. Block V-VOLs in the following order:
   1. All V-VOLs with the vClone attribute that belong to the family of the V-VOL with the vClone Parent attribute enabled and with deduplication data enabled
   2. All V-VOLs with deduplication data enabled
   3. All DSD-VOLs
3. Run the raidcom initialize pool command by specifying a pool containing the deduplication system data volumes.
4. Perform LDEV formatting for V-VOLs in the following order:
   1. All DSD-VOLs (data store)
   2. All V-VOLs with the vClone attribute that belong to the family of the V-VOL with the vClone Parent attribute enabled and with deduplication data enabled

      Caution: Performing this operation changes the vClone Parent attribute of the V-VOL with the vClone Parent attribute in the family to disabled.

   3. All V-VOLs with deduplication data enabled

   Caution: Do not try to restore the LDEVs because these V-VOLs cannot be used again.

5. Verify the compression accelerator settings for the V-VOLs with deduplication data enabled (with deduplication and compression enabled) in the pool initialized in step 3, and for the deduplication system data volumes (data store) in the pool. (See the raidcom modify ldev command.)
6. Re-create the V-VOL with the vClone attribute.
7. Restore the backup data, or try to delete the LDEVs again.

Note: If you format V-VOLs for which deduplication data is disabled, the capacity saving for the V-VOLs might become compression. If you initialize duplicated data for a pool that has V-VOLs for which deduplication data is enabled, the capacity saving for the V-VOLs might become deduplication and compression. Therefore, verify the capacity saving of V-VOLs after formatting is complete.

When a DRS-VOL is removed, one of these problems occurs:

1. The capacity saving status of the V-VOL is Deleting Volume, and the progress indicates a hyphen (-).
2. The capacity saving status of the V-VOL is Disabled.

Retry the deletion of the V-VOL whose capacity saving status becomes disabled.

• The pool used capacity exceeded the depletion threshold.
• The pool with which the target V-VOL is associated is blocked, or the pool volumes are blocked.
• The deduplication system data volume of the pool with which the target V-VOL is associated is blocked.

Solutions:

If the pool used capacity exceeded the depletion threshold, take these actions:

• Verify the free capacity of the pool, and then expand the pool capacity for the capacity shortage.
• Release pages in V-VOLs to reclaim zero pages. (See the raidcom modify ldev command.)
• Delete V-VOLs that are not used. (See the raidcom delete ldev command.)

If the pool with which the target V-VOL is associated is blocked, or if the pool volumes are blocked, restore the pool or pool volumes.

If the saved capacity is not increased, the initial capacity saving processing might have been stopped. The initial capacity saving processing is performed on the existed data in V-VOLs. These are causes for stopping the initial capacity saving processing:

1. The cache write pending rate of the CLPR containing the V-VOLs is 50% or more.
2. The controller of the storage system is blocked.
3. The ratio of a free pool space with which the V-VOLs are associated is 1% or less.
4. The size of a free pool space with which the V-VOLs are associated is 120GB or less.
5. The pool with which the V-VOLs are associated is shrunk.
6. The pool volumes are blocked.
7. The operation targeted DRS-VOLs with capacity saving enabled.
8. The used capacity rate of the data store area for the operation targeted DRS-VOLs with capacity saving enabled at 80% or more.
9. The average operating ratio for MPs on the basis of a controller is 70% or more.

Solutions:

• For item number 1, reduce the written data size from hosts.
• For item number 2, restore the blocked controller.
• For item number 3 and 4, expand the pool capacity.
• For item number 5, wait until the pool shrink task is completed.
• For item number 6, restore the blocked pool volumes.
• For item number 7, restore the blocked V-VOLs of the operation target.
• For item number 8, split the operation targeted V-VOLs into DRS-VOLs with capacity saving enabled and a size of 2.4TB or less.
• For item number 9, the storage system reduces the number of processes of the initial capacity saving. If you want to increase the amount of the processing, balance the loading on the basis of a controller.

If the garbage data continues to increase, these are the potential causes. However, the garbage data is included in the system data:

• The amount of written data by hosts (the average performance of host writing) exceeds the throughput of the garbage collection (the garbage collection performance).
• The processing of the garbage collection is stopped.

Solutions:

Perform these actions if the amount of written data by hosts (the average performance of host writing) exceeds the throughput of the garbage collection (the garbage collection performance):

• If the amount of written data by hosts exceeds the throughput of the garbage collection on a basis of a storage system, perform one of these actions:
  • Reduce the amount of written data by hosts.
  • Verify the design of pools and V-VOLs again.
• If the amount of written data of hosts exceeds the throughput of the garbage collection on a basis of a MP unit, perform this action:

  To rebalance the load (the writing I/Os from hosts) to MP units, change the assignment of MP units.

• If the amount of written data by hosts exceeds the throughput of the garbage collection on a basis of a DRS-VOL, perform this action:

  To rebalance the load (the writing I/Os from hosts) to DRS-VOLs, divide V-VOLs.

If the processing of the garbage collection is stopped, here are the possible causes:

1. The cache write pending rate of the CLPR containing the V-VOL is 83% or more.
2. There is no free space in the pool with which V-VOLs are associated.
3. Pool volumes are blocked.
4. DRS-VOLs are blocked.

Solutions:

• For item number 1, reduce the amount of written data by hosts.
• For item number 2, expand the pool capacity.
• For item number 3, restore the blocked pool volumes.
• For item number 4, restore the blocked V-VOLs.

• The setting is not correct.
• The pool used capacity exceeds the threshold.
• The DRS-VOLs are used in a task which is not suited with using of the capacity saving function.

Solutions:

1. Verify the setting. If the setting is correct, go to step 2. If the capacity saving mode is accidentally set to Inline, change the mode to Post process. (raidcom modify ldev -capacity_saving_mode command)
2. Verify whether the pool used capacity exceeds the warning threshold by looking at Pool Usage. If the pool used capacity has not exceeded the warning threshold, go to step 3.

   If the pool usage has exceeded the warning threshold, verify the garbage data size by looking at System Data.

   If the ratio of system data size in the data capacity before capacity saving is more than 13%, the pool usage will go below the warning threshold by the garbage collection. The ratio of the used capacity before saving goes below the warning threshold ratio so that the I/O performance will be improved. If you know that the performance decrement is temporary and you can wait until the recovery time, waiting is the workable alternative. However, if you do not have time to wait until the recovery, expand the pool capacity.

   If the ratio of the system data for the data capacity before capacity saving is less than 13%, expand the pool capacity.

3. This problem might be caused by overload due to the copy processing in a replication program product. In this case, the copy threshold option might improve the performance. This option has a side effect, and therefore determine whether to apply this option by referring to Copy threshold.

After deduplication system data volumes are created, an error occurs while V-VOLs are created, and then the processing stops.

Solution:

Resolve the cause of the error. Then, re-create V-VOLs with deduplication and compression enabled.

After deduplication system data volumes are created, an error occurs while the settings for V-VOLs are changed, and then the processing stops.

Solution:

Resolve the causes of the error. Then, change the capacity saving setting for V-VOLs again.

An error occurs while deduplication system data volumes are deleted, and then the processing stops.

Solution:

Resolve the cause of the error. Then, delete deduplication system data volumes using the CCI command, raidcom modify pool -pool -delete dsd_volumes.

• Cause: The number of pools to which V-VOLs with deduplication and compression enabled belong has reached the maximum number of pools containing V-VOLs with deduplication and compression enabled.

  Solution: For more information about the maximum number of pools, see Pool specifications and requirements.

• Cause: The total capacity of created V-VOLs exceeds the estimated configurable capacity of V-VOL capacity.

  Solution: Add shared memory, and then retry the operation. Alternatively, delete the V-VOLs that are not used.

• Cause: This problem is caused due to the same causes as those for the "Cannot create a V-VOL" problem listed in this table.

  Solution: Take actions by referring to the causes and the solutions for the "Cannot create a V-VOL" problem in this table.

• Cause: The number of pools to which V-VOLs with deduplication and compression enabled belong has reached the maximum number of pools containing V-VOLs with deduplication and compression enabled.

  Solution: For more information about the maximum number of pools, see Pool specifications and requirements.

• Cause: The total capacity of created V-VOLs exceeds the estimated configurable capacity of V-VOL capacity.

  Solution: Add shared memory, and then retry the operation. Alternatively, delete the V-VOLs that are not used.

• Cause: This problem is caused due to the same causes as those for the "Cannot create a V-VOL" problem listed in this table.

  Solution: Take actions by referring to the causes and the solutions for the "Cannot create a V-VOL" problem in this table.

• Cause: The used capacity of the pool containing the target V-VOL exceeds the depletion threshold. The free capacity percentage of the pool is 1% or less, or the free capacity of the pool is 120 GB or less.

  Solutions:

  • Verify the free capacity of the pool, and then expand the pool capacity for the capacity shortage.
  • Release pages to reclaim zero pages. (See the raidcom modify ldev command.)
  • Delete a V-VOL that is not used. (See the raidcom delete ldev command.)
• Cause: The target V-VOL is blocked.

  Solution: Restore the blocked V-VOL.

• Cause: The pool containing the target V-VOL is blocked, or a pool volume is blocked.

  Solution: Restore the blocked pool or pool volume.

• Cause: The deduplication system data volume in the pool containing the target V-VOL is blocked.

  Solution: Restore the blocked pool or deduplication system data volume.

• Cause: The pool containing the target V-VOL is being shrunk.

  Solution: Wait for the pool shrinking operation to complete.

• Cause: The cache write-pending rate for the CLPR containing the V-VOL is 30% or more.

  Solution: Reduce the amount of data written from the host.

• Cause: The controller is blocked.

  Solution: Restore the blocked controller.