Overview of the data reduction function for volumes

During volume creation, volumes with data reduction enabled are created if any of the following are true. Volumes with data reduction enabled will be subject to data compression and fixed pattern elimination.

When volumes are created with "Compression" specified in the savingSetting (CLI: --saving_setting) parameter
When the VPS where you want to create volumes has the data reduction function enabled (savingSettingOfVolume is "Compression")

Data compression uses the OSS "LZ4" compression algorithm.

When fixed patterns are excluded, if the amount of 0x00 and 0xFF data exceeds 8 KiB, the data itself is not written and only the control information is written. Thus, consumption of the storage pool is reduced.

For a volume with data reduction enabled, volumeType is "Normal" (as with regular volumes) and savingSetting is "Compression."

The processing of the data reduction function uses the inline method in which data compression for new or updated data is performed simultaneously with write operation.

Volumes with data reduction enabled should be used with understanding of the following:

The effect of data compression differs depending on how easily the data compression can be performed.
If data is compressed by a user's application, there might be no reduction effect by the data reduction function. Therefore, verify reduction effect by using actual data in advance.
I/O performance of a volume with data reduction enabled might be lower than that of a volume with data reduction disabled because data compression and decompression are executed.

CAUTION:

To use the data reduction function in the cloud model, you need to newly perform installation with r7i.8xlarge or r6i.8xlarge set for the instance type. For details about how to set the instance type, see the VSP One SDS Block Cloud Setup and Configuration Guide.

Requirements for enabling the data reduction function for volumes

The requirements are met if setup was performed based on the assumption that the data reduction function would be used for volumes.

If you want to enable the data reduction function for volumes after setup, see Enabling the data reduction function for volumes to meet the requirements.

Write back mode with cache protection is enabled.
The physical memory capacity on each storage node is 256 GiB or more.
(Cloud) Instance type "r7i.8xlarge" or "r6i.8xlarge" is set.
The storage pool has sufficient capacity.

Tip:

To verify the memory capacity allocated to each storage node, see Obtaining a list of information about storage nodes or Obtaining information about individual storage nodes.

Overview of data write to a volume with data reduction enabled

When data is written, the data is compressed in memory, and then written to the storage pool (drive).

For a volume with data reduction enabled, metadata specific to the data reduction function will be created. The amount of the metadata is approximately 3% of the amount of compressed data.

When the data reduction function is used, data will be appended to the storage pool when data is updated (this does not occur when the data reduction function is not used). The old data will be invalidated and handled as garbage. When a certain amount of garbage accumulates, the garbage collection process is performed asynchronously to free the area occupied by garbage.

Information on the data reduction effect by the data reduction function

You can obtain information on data reduction effect by the data reduction function by:

Obtaining a list of information about volumes
Obtaining information about individual volumes
Obtaining a list of information about storage pools
Obtaining information about individual storage pools
Obtaining information about storage clusters

You can obtain the information about data reduction on volumes in dataReductionEffects of volume information.

The following figure shows the relationship between various information on volume data reduction.

Supplementary notes on data reduction information

Keep the following in mind when viewing the capacity of volumes with data reduction enabled.

Even immediately after a volume is created, from 42 MiB up to several hundreds of MiB are required as the space for metadata depending on the volume capacity.
preCapacityDataReductionWithoutSystemData [MiB] is displayed as a multiple of 42 because the actual consumed amount of the storage pool is in units of pages (42 MiB per page).
Data reduction-related information (compressedCapacity, reclaimedCapacity, and systemDataCapacity) is managed on a block-by-block basis (512 bytes). Data reduction-related information is displayed on a block basis in units of MiB. Therefore, information data less than 1 MiB is truncated. Therefore, the actual value and displayed value might differ.
```
Example: dataReductionCapacity [MiB] = compressedCapacity [MiB] + reclaimedCapacity [MiB] - systemDataCapacity [MiB]
```
(Example) The actual values for compressedCapacity, reclaimedCapacity, and systemDataCapacity are 2489.5 [MiB], 6551.5 [MiB], and 179.0 [MiB], respectively.
- Calculated results by using the actual values:
  
  2489.5 [MiB] + 6551.5 [MiB] - 179.0 [MiB] = 8862.0 [MiB]
- The result calculated from the values that are rounded down to 1 MiB:
  
  2489 [MiB] + 6551 [MiB] - 179 [MiB] = 8861.0 [MiB]
In the above case, the resulting values have a difference of 1 MiB.

Note:

Immediately after the use of a volume with data reduction enabled is started, the amount of metadata might become larger than the amount of written data. Also, while data is being written, the amount of garbage might be larger than the amount of written data.

If the amount of system data being used (the amount of metadata and garbage) becomes larger than the amount of written data, the following might be displayed:
- systemDataCapacity [MiB] becomes larger than preCapacityDataReductionWithoutSystemData [MiB].
- dataReductionCapacity [MiB] becomes 0.
- postCapacityDataReduction [MiB] becomes larger than preCapacityDataReductionWithoutSystemData [MiB].
- dataReductionRate [%] becomes less than 100.
When the Write Same/Unmap command is run from the host for a volume with data reduction enabled, fixed patterns are excluded in the relevant area. This results in a reduction in consumption of the storage pool used by the volume with data reduction enabled but not in consumption of the volume.

Amount of data reduced state

A volume with data reduction enabled is in one of the following statuses.

Meaning of status	Combination of statuses			Necessary action
Meaning of status	status Summary	status	dataReduction Status	Necessary action
Normal status after volume creation	Normal	Normal	Enabled	No action required
Volume settings being updated or volume capacity being expanded	Normal	Updating or Expanding	Enabled	No action required
Volume being deleted	Normal	Deleting	Deleting	No action required
Volume creation unsuccessful	Error	CreationFailed	Enabled	After deleting the volume whose creation was unsuccessful, create a volume again.
Status after recovery operation is performed in the "Failed" status after unsuccessful volume operation	Error	DeletionFailed, UpdateFailed, or ExpansionFailed	Enabled	Retry the volume operation.
Deletion unsuccessful due to a cause other than metadata mismatch	Error	DeletionFailed	Deleting	Take action according to the VSP One SDS Block Troubleshooting Guide.
Status in which metadata integrity is not guaranteed	Error	MetadataConsistencyError	Failed
Status in which the storage cluster was restarted after failures exceeding redundancy, and others, occurred during volume operation	Error	DeletionFailed, or ExpansionFailed	Failed
Status in which the storage cluster was restarted after failures exceeding redundancy, and others, occurred during volume operation	Error	IOSuppressed	Failed

Effect of enabling the data reduction function

The data reduction function, when it is used, limits the write speed if the storage controller capacity is about to be exhausted. The following table shows the conditions and impact of the data reduction function on write operation.

Cases in which write performance is affected	Impact on write performance	Remarks
One of the following states: The usage rate of the storage pool managed by the storage controller that manages the volume that received the write request is 99% or more, or the free space is 40 GiB or less. The internal resource usage of the storage controller that manages the volume that received the write request has temporarily increased.	I/O performance is reduced because a response is returned to the host after making sure that the storage pool can be written to. In addition, writes are accepted according to the amount of garbage collected by the garbage collection process. For this reason, write performance might be degraded.
One of the following states: The write speed is limited from when free space in the storage pool managed by the storage controller that manages the volume that received the write request is 40 GiB or less until it is restored to 80 GiB. The write speed is limited from when free space in the storage pool managed by the storage controller is 80 GiB or less and the first volume with data reduction enabled is created in the storage pool, until the free space is restored to 80 GiB.	Writes are accepted according to the amount of garbage collected by the garbage collection process. For this reason, write performance might be degraded.	If the situation described in the left column occurs, the event log KARS06213-W will be notified. When the write restriction is canceled, the event log KARS06212-I will be notified.
The usage rate of the storage pool managed by the storage controller that manages the volume that received the write request is 80% or more.	This situation might degrade write performance because processing resources are preferentially allocated to the garbage collection process.
The internal resource usage managed by the storage controller (that manages the volume that received the write request) has temporarily increased.	Write performance of the volume with data reduction enabled might be degraded. In addition, writes are accepted according to the amount of garbage collected by the garbage collection process. For this reason, write performance might be degraded.	If the situation described in the left column occurs, event log KARS06217-W will be notified. When the write restriction is canceled, event log KARS06216-I will be notified.
The internal resource usage of the storage controller that manages the volume that received the write request has temporarily increased.	The entire storage controller is affected. I/O performance will be reduced until there is sufficient free space in the storage pool.	If the situation described in the left column occurs, event log KARS06218-W will be notified.

The following are actions to be taken when the garbage collection process occurs or to reduce the frequency of its occurrence.

If the ratio of the system data usage (systemDataCapacity [MiB]), which is the sum of metadata and garbage data, is 10% or more of the volume used for storage controller information (usedVolumeCapacity [MiB]), reduce host I/Os so that the ratio becomes less than 10%. Or, expand the storage pool by adding more drives or more storage nodes.
If the ratio of the system data usage (systemDataCapacity [MiB]), which is the sum of metadata and garbage data, is less than 10% of the volume used for storage controller information (usedVolumeCapacity [MiB]), expand the storage pool by performing drive addition or storage node addition.

Application of the data reduction function to snapshot volumes

You can take snapshots of a volume with data reduction enabled (P-VOL) to create snapshot volumes (S-VOLs). However, the data reduction function is not enabled for the S-VOLs. Manage the storage pool capacity based on this condition.