Sample coding for re-creating a parity group (data encryption)

REST API Reference Guide for Virtual Storage Platform 5000, Virtual Storage Platform E Series, and Virtual Storage Platform G/F Series

Version
93-07-0x
90-09-0x
88-08-10
Audience
anonymous
Part Number
MK-98RD9014-17
This section provides the sample coding for re-creating a parity group (encrypting data).
Note:

This operation is performed as part of the procedure for encrypting existing data without changing the drive configuration. For details on the entire procedure, see the description of the flow of data encryption operations.

Before deleting a parity group, be sure to back up the data in the parity group to a volume in another parity group.

Sample coding operation flow for re-creating a parity group

The following table shows the sample coding operation flow for re-creating a parity group and the corresponding code constructs.
Step

Sample coding operation flow

Code constructs

1

Import necessary libraries and set parameters.

-

2

Define headers.

Specifying request headers (for the default HTTP headers)

3

Define functions for issuing an HTTP request and for verifying the status of asynchronous processing.

Getting information about the job status by performing a GET operation

Setting user authentication information (for authentication by using session-based authentication)

Generating a request body in JSON format

Getting the job execution results

Getting the URLs of the resources to which the operation results have been applied

Getting error codes

4

Check the version of the REST API.

Getting information about the version of the REST API by performing a GET operation

5

Generate a session.

Getting the URLs of the resources (when object IDs are not specified)

Setting user authentication information (for authentication by using a user ID and a password)

Creating objects by performing a POST operation

6

Get information about the parity group.

Getting objects by performing a GET operation

Setting user authentication information (for authentication by using session-based authentication)

Outputting the acquired information

7

Get information about drives.

Getting objects by performing a GET operation

Setting user authentication information (for authentication by using session-based authentication)

Outputting the acquired information

8

Delete the parity group.

Getting the URLs of the resources (when no object ID is specified)

Setting user authentication information (for authentication by using session-based authentication)

Generating a request body in JSON format

Deleting objects by performing a DELETE operation

9

Create a parity group.

Getting the URLs of the resources (when no object ID is specified)

Setting user authentication information (for authentication by using session-based authentication)

Generating a request body in JSON format

Creating objects by performing a POST operation

10

Get information about the parity group.

Getting objects by performing a GET operation

Setting user authentication information (for authentication by using session-based authentication)

Outputting the acquired information

11

Output error messages.

Outputting error messages

12

Discard the session.

Getting the URLs of the resources (when object IDs that are obtained from the operation results are specified)

Deleting objects by performing a DELETE operation

Expected system configuration

This sample coding assumes the system configuration is as shown in the following figure.

The following table shows the values specified for the parameters in the sample coding. If necessary, change the settings to match the system environment and requirements.

Parameter

Value

Description

USER_CREDENTIAL

("user1", "pass1")

This is the authentication information to be used for authentication in the storage system. The coding sample shows a setting example when the user ID is user1, and the password is pass1. The user needs the Storage Administrator (Provisioning) role.

PARITY_GROUP_ID

1-1

The ID of the parity group to be recreated

FIRST_WAIT_TIME

1

The first interval (seconds) for collecting the execution result of asynchronous processing. Normally, you do not need to change this value.

MAX_RETRY_COUNT

6

The maximum number of retries for collecting the execution result of asynchronous processing. Normally, you do not need to change this value.

Contents of the sample coding

This subsection explains the sample coding.

  1. Import necessary libraries and set parameters.

    Before starting the volume allocation processing, the sample coding imports the required libraries or classes. In addition to the common libraries, the sample coding also imports the BlockStorageAPI class that defines the function that generates URLs.

    # coding:utf-8
    
    """
    recreate_parity_group
    
    This program requires API version 1.12.0 or newer.
    """
    
    import requests
    import json
    import sys
    import http.client
    import time
    import traceback
    import rest_server_param
    import storage_param
    
    from block_storage_api import BlockStorageAPI
    

    Set parameters to be used in the sample coding.

    # #################Initialize parameters################# #
    # Change the following parameters to fit your environment
    
    # This parameter defines the first interval to access
    # an asynchronous job. (Unit: Second)
    FIRST_WAIT_TIME = 1
    
    # This parameter defines the maximum retry time
    # to confirm job status.
    MAX_RETRY_COUNT = 6
    
    # An user id and password of the target storage
    USER_CREDENTIAL = ("user1", "pass1")
    
    # A parity group id
    PARITY_GROUP_ID = "1-1"
    
    ###########################################################
    
  2. Define headers.

    Define the HTTP request header. Because the REST API only supports JSON format data, the sample coding defines header information so that data is handled in JSON format.

    # ###You don't have to change the following parameters### #
    block_storage_api = BlockStorageAPI(
        rest_server_param.REST_SERVER_IP_ADDR,
        rest_server_param.REST_SERVER_PORT,
        storage_param.STORAGE_MODEL,
        storage_param.SERIAL_NUMBER)
    
    headers = {"content-type": "application/json",
               "accept": "application/json",
               "Response-Job-Status": "Completed"}
    
    REQUIRED_MAJOR_VERSION = 1
    REQUIRED_MINOR_VERSION = 12
    
    ###########################################################
    
  3. Define the function for issuing an HTTP request and for verifying the status of asynchronous processing (the invoke_async_command function).

    Define the function that issues an HTTP request and verifies the status of asynchronous processing. Call and use this function from the main volume allocation operation. For details on this function, see the section explaining the functions used in the sample coding.

    Tip: To prevent errors that occur if the server certificate used for SSL communication between the REST API client and the storage system is a self-signed certificate, the sample coding skips the verification of the server certificate by specifying verify=False when a request is issued.
    """
    Check whether the asynchronous command was finished.
    @param job_id the job ID to identify
           the asynchronous command
    @return r the response data
    """
    
    
    def check_update(job_id):
        url = block_storage_api.job(str(job_id))
        r = requests.get(url, headers=headers, verify=False)
        return r
    
    
    """
    Execute the HTTP request (POST, PUT or DELETE)
    @param method_type HTTP request method (POST, PUT or DELETE)
    @param url URL to execute HTTP method
    @param body The information of a resource
    @return job_result.json()["affectedResources"][0]
             URL of an affected resource
    """
    
    
    def invoke_async_command(method_type, url, body):
        if method_type == "put":
            if body is None:
                r = requests.put(url, headers=headers, verify=False)
            else:
                r = requests.put(url, headers=headers,
                                 data=json.dumps(body), verify=False)
        elif method_type == "post":
            r = requests.post(
                url,
                headers=headers,
                data=json.dumps(body),
                verify=False)
        elif method_type == "delete":
            r = requests.delete(
                url,
                headers=headers,
                verify=False)
        if r.status_code != http.client.ACCEPTED:
            raise requests.HTTPError(r)
        print("Request was accepted. JOB URL : " +
              r.json()["self"])
        status = "Initializing"
        job_result = None
        retry_count = 1
        wait_time = FIRST_WAIT_TIME
        while status != "Completed":
            if retry_count > MAX_RETRY_COUNT:
                raise Exception("Timeout Error! "
                                "Operation was not completed.")
            time.sleep(wait_time)
            job_result = check_update(r.json()["jobId"])
            status = job_result.json()["status"]
            double_time = wait_time * 2
            if double_time < 120:
                wait_time = double_time
            else:
                wait_time = 120
            retry_count += 1
        if job_result.json()["state"] == "Failed":
            error_obj = job_result.json()["error"]
            if "errorCode" in error_obj:
                if "SSB1" in error_obj["errorCode"]:
                    print("Error! SSB code : ",
                          error_obj["errorCode"]["SSB1"],
                          ", ", error_obj["errorCode"]["SSB2"])
                elif "errorCode" in error_obj["errorCode"]:
                    print("Error! error code : ",
                          error_obj["errorCode"]["errorCode"])
            raise Exception("Job Error!", job_result.text)
        if "affectedResources" in job_result.json():
            print("Async job was succeeded. affected resource : " +
                  job_result.json()["affectedResources"][0])
            return job_result.json()["affectedResources"][0]
        else:
            print("Async job was succeeded.")
            return None
    
    
  4. Check the version of the REST API.

    Get information about the version of the REST API to make sure that the version is supported.

    """
    Check whether this API version allows the REST
     Server to execute this program
    
    @param api_version api version of this REST Server
    @param required_major_version the lowest number of
           the major version that this program requires
    @param required_minor_version the lowest number of
           the minor version that this program requires
    
    """
    
    
    def check_api_version(api_version, required_major_version,
                          required_minor_version):
        version = api_version.split(".")
        major_version = int(version[0])
        minor_version = int(version[1])
        if not ((major_version == required_major_version and
                 minor_version >= required_minor_version) or
                major_version >= required_major_version + 1):
            sys.exit("This program requires API Version " +
                     str(required_major_version) + "." +
                     str(required_minor_version) +
                     "." + "x or newer.\n")
    
    try:
        # step1 Check the API version #
        print("Check the API version")
        url = block_storage_api.api_version()
        r = requests.get(url, headers=headers, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        check_api_version(
            r.json()["apiVersion"],
            REQUIRED_MAJOR_VERSION,
            REQUIRED_MINOR_VERSION)
    
  5. Generate a session.

    Generate sessions by using the REST API server.

        # step2 Generate a session #
        print("Generate a session")
        url = block_storage_api.generate_session()
        r = requests.post(url, headers=headers,
                          auth=USER_CREDENTIAL, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        token = r.json()["token"]
        auth = "Session " + token
        session_id = r.json()["sessionId"]
        headers["Authorization"] = auth
    

    When you generate a session, a session ID and a token are returned. When running the API, specify the token for the Authentication header as the required authentication information for the subsequent operations. Use the session ID to discard the session after a set of operations is completed.

  6. Get information about the parity group.

    Get information about the parity group by specifying its parity group number. From the acquired information, the sample coding outputs the following information: number of LDEVs in the parity group, usage rate of the parity group, available capacity, RAID level, RAID type, CLPR number, code indicating the drive type of the drives belonging to the parity group, drive type of the drives that belong to the parity group, rotation speed of the drives belonging to the parity group, value of the encryption setting of the parity group, total logical capacity of the parity group, total physical capacity of the parity group, and value of the accelerated compression setting of the parity group.

        # step3 Get the parity group #
        print("Get the parity group")
        url = block_storage_api.parity_group(PARITY_GROUP_ID)
        r = requests.get(url, headers=headers,
                         auth=USER_CREDENTIAL, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        
        raidType = r.json()["raidType"]
        isCopyBackModeEnabled = r.json()["isEncryptionEnabled"]
        isAcceleratedCompressionEnabled = r.json()["isAcceleratedCompressionEnabled"]
        clprId = r.json()["clprId"]
        print("NUM OF LDEVS : " + str(r.json()["numOfLdevs"]))
        print("USED CAPACITY RATE : " + str(r.json()["usedCapacityRate"]))
        print("AVAILABLE VOLUME CAPACITY : " + str(r.json()["availableVolumeCapacity"]))
        print("RAID LEVEL : " + str(r.json()["raidLevel"]))
        print("RAID TYPE : " + str(r.json()["raidType"]))
        print("CLPR ID : " + str(r.json()["clprId"]))
        print("DRIVE TYPE : " + r.json()["driveType"])
        print("DRIVE TYPE NAME : " + str(r.json()["driveTypeName"]))
        print("DRIVE SPEED : " + str(r.json()["driveSpeed"]))
        print("IS ENCRYPTION ENABLED : " + str(r.json()["isEncryptionEnabled"]))
        print("TOTAL CAPACITY : " + str(r.json()["totalCapacity"]))
        print("PHYSICAL TOTAL CAPACITY : " + str(r.json()["physicalCapacity"]))
        print("IS ACCELERATED COMPRESSION ENABLED : " + str(r.json()["isAcceleratedCompressionEnabled"]))
        print()
    
    
  7. Get information about drives.

    Get information about drives that belong to a parity group by specifying its parity group number.

        # step4 Get drives #
        print("Get drives")
        url = block_storage_api.drives_parity_group(PARITY_GROUP_ID)
        r = requests.get(url, headers=headers,
                         auth=USER_CREDENTIAL, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        
        drive_ids = []
        for d in r.json()["data"]:
            drive_ids.append(d["driveLocationId"])
            print("DRIVE LOCATION ID : " + str(r.json()["driveLocationId"]))
        print()
    
    
  8. Delete the parity group.

    Delete the parity group by specifying its parity group number.

        # step5 Delete the parity group #
        print("Delete the parity group")
        url = block_storage_api.parity_group(PARITY_GROUP_ID)
        invoke_async_command("delete", url, None)
    

    The invoke_async_command function issues a request to delete the parity group, checks the execution statuses of the jobs that were executed asynchronously, and then returns the URL of the deleted parity group as the execution result.

  9. Create a parity group.

    Create a parity group by specifying a parity group number and drive IDs.

        # step6 Create the parity group #
        print("Create the parity group")
        url = block_storage_api.parity_groups()
        body = {
            "parityGroupId": PARITY_GROUP_ID,
            "driveLocationIds": drive_ids,
            "raidType": raidType,
            "isEncryptionEnabled": True,
            "isCopyBackModeEnabled": isCopyBackModeEnabled,
            "isAcceleratedCompressionEnabled": isAcceleratedCompressionEnabled,
            "clprId": clprId
        }
        invoke_async_command("post", url, body)
    
    

    The invoke_async_command function issues a request to create a parity group, checks the execution statuses of jobs that were executed asynchronously, and then returns the URL of the created parity group as the execution result.

  10. Get information about the parity group.

    To check whether the operations up to this step have been correctly applied to the resource, get information about the parity group by specifying the parity group number you acquired when you created the parity group. From the acquired information, the sample coding outputs the following information: number of LDEVs in the parity group, usage rate of the parity group, available capacity, RAID level, RAID type, CLPR number, code indicating the drive type of the drives belonging to the parity group, drive type of the drives that belong to the parity group, rotation speed of the drives belonging to the parity group, value of the encryption setting of the parity group, total logical capacity of the parity group, total physical capacity of the parity group, and value of the accelerated compression setting of the parity group.

        # step7 Get the parity group #
        print("Get the parity group")
        url = block_storage_api.parity_group(PARITY_GROUP_ID)
        r = requests.get(url, headers=headers,
                         auth=USER_CREDENTIAL, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        
        print("NUM OF LDEVS : " + str(r.json()["numOfLdevs"]))
        print("USED CAPACITY RATE : " + str(r.json()["usedCapacityRate"]))
        print("AVAILABLE VOLUME CAPACITY : " + str(r.json()["availableVolumeCapacity"]))
        print("RAID LEVEL : " + str(r.json()["raidLevel"]))
        print("RAID TYPE : " + str(r.json()["raidType"]))
        print("CLPR ID : " + str(r.json()["clprId"]))
        print("DRIVE TYPE : " + r.json()["driveType"])
        print("DRIVE TYPE NAME : " + str(r.json()["driveTypeName"]))
        print("DRIVE SPEED : " + str(r.json()["driveSpeed"]))
        print("IS ENCRYPTION ENABLED : " + str(r.json()["isEncryptionEnabled"]))
        print("TOTAL CAPACITY : " + str(r.json()["totalCapacity"]))
        print("PHYSICAL TOTAL CAPACITY : " + str(r.json()["physicalCapacity"]))
        print("IS ACCELERATED COMPRESSION ENABLED : " + str(r.json()["isAcceleratedCompressionEnabled"]))
        print()
    
    
  11. Output error messages.

    In the sample coding, processing for communication errors, HTTP request errors, and job execution errors is described. If a communication error occurs, an error message is output. If an HTTP request error occurs, the error code, the error message, and the response body are output. If a job execution error occurs, all of the contents included in the job execution result are output.

    except requests.ConnectionError:
        sys.stderr.write("Connection Error!\n")
        sys.stderr.write(traceback.format_exc())
    except requests.HTTPError as he:
        sys.stderr.write("HTTP Error! status code : ")
        sys.stderr.write(str(he.args[0].status_code) + "\n")
        sys.stderr.write(he.args[0].text + "\n")
    except Exception as e:
        sys.stderr.write(traceback.format_exc())
        for msg in e.args:
            sys.stderr.write(str(msg) + "\n")
    
    
  12. Discard the session.

    After a set of operations is completed, discard the session. Specify the session ID that was obtained when the session was generated. The "finally" statement in the sample coding makes sure that the session will be discarded even if an error occurs while the API is running. After the session is discarded, the processing ends.

    finally:
        # step8 Discard the session #
        print("Discard the session")
        url = block_storage_api.discard_session(session_id)
        r = requests.delete(url, headers=headers, verify=False)
        try:
            if r.status_code != http.client.OK:
                raise requests.HTTPError(r)
        except requests.HTTPError as he:
            sys.stderr.write("HTTP Error! status code : ")
            sys.stderr.write(str(he.args[0].status_code) + "\n")
            sys.stderr.write(he.args[0].text + "\n")
    
        print("Operation was completed.")
        sys.exit()