Sample coding for TrueCopy pair operations

REST API Reference Guide for Virtual Storage Platform 5000, Virtual Storage Platform E Series, and Virtual Storage Platform G/F Series

Version
93-07-0x
90-09-0x
88-08-10
Audience
anonymous
Part Number
MK-98RD9014-17
This section explains the sample coding for the TrueCopy pair operations.

Sample coding operation flow for TrueCopy pair operations

The following table shows the sample coding operation flow for TrueCopy pair operations and the corresponding code constructs.

Step

Sample coding operation flow

Code constructs

1

Import necessary libraries and set parameters.

-

2

Define headers.

Specifying request headers (for the default HTTP headers)

3

Define the function for getting status changes for asynchronous processing.

Getting information about the job status by performing a GET operation

Setting user authentication information (for authentication by using session-based authentication)

Getting the job execution results

Getting error codes

4

Check the version of the REST API.

Getting information about the version of the REST API by performing a GET operation

5

Generate a session.

Getting the URLs of the resources (when object IDs are not specified)

Setting user authentication information (for authentication by using a user ID and a password)

Creating objects by performing a POST operation

6

Lock resources.

Getting the URLs of the resources (when object IDs are not specified)

Setting user authentication information (for authentication by using session-based authentication)

Running actions for services by performing a POST operation

7

Create a TrueCopy pair.

Getting the URLs of the resources (when object IDs are not specified)

Getting the URLs of the resources (when object IDs that are obtained from the operation results are specified)

Setting user authentication information (for authentication by using session-based authentication)

Generating a request body in JSON format

Operations that require sessions to be generated on multiple devices (remote copy operation)

Creating objects by performing a POST operation

8

Unlock resources.

Getting the URLs of the resources (when object IDs are not specified)

Setting user authentication information (for authentication by using session-based authentication)

Running actions for services by performing a POST operation

9

Confirm that the pair has been created.

-

10

Get information about a TrueCopy pair.

Getting an object by performing a GET operation

Setting user authentication information (for authentication by using session-based authentication)

Getting the URLs of the resources to which the operation results have been applied

Getting error codes

11

Output error messages.

Outputting error messages

12

Discard the session.

Getting the URLs of the resources (when object IDs that are obtained from the operation results are specified)

Setting user authentication information (for authentication by using session-based authentication)

Deleting objects by performing a DELETE operation

Expected system configuration

This sample coding assumes the system configuration is as shown in the following figure.

The following table shows the values specified for the parameters in the sample coding. If necessary, change the settings to match the system environment and requirements.

Parameter

Value

Description

LOCAL_USER _CREDENTIAL

("local_copy_user", "local_copy_pass")

This is the authentication information to be used for authentication in the local storage system. The coding sample shows a setting example when the user ID is local_copy_user, and the password is local_copy_pass. The user needs the Storage Administrator (Provisioning) role and the Storage Administrator (Remote Copy) role.

REMOTE_USER _CREDENTIAL

("remote_copy_user", "remote_copy_pass")

This is the authentication information to be used for authentication in the remote storage system. The coding sample shows a setting example when the user ID is remote_copy_user, and the password is remote_copy_pass. The user needs the Storage Administrator (Provisioning) role and the Storage Administrator (Remote Copy) role.

COPY_GROUP_NAME

"TC_GROUP"

The copy group name for a TrueCopy pair to be created

COPY_PAIR_NAME

"p_347-348"

The copy pair name for a TrueCopy pair to be created

PVOL_LDEV_ID

347

The number of the already created LDEV to be used for the primary volume

SVOL_LDEV_ID

348

The number of the already created LDEV to be used for the secondary volume

FIRST_WAIT_TIME

1

The first interval for collecting the execution result of asynchronous processing. Normally, you do not need to change this value.

MAX_RETRY_COUNT

10

The maximum number of retries for collecting the execution result of asynchronous processing. Normally, you do not need to change this value.

The following table shows the parameters and values defined in the remote_copy_param.py file, which can be used in coding samples as common variables for the information about the local and the remote storage systems. If necessary, change the settings to match the system environment and requirements.

Parameter

Value

Description

LOCAL_REST_SERVER_IP_ADDR

192.0.2.100

The IP address of the REST API server of the local storage system

LOCAL_PORT

443

The SSL communication port for the REST API server of the local storage system

LOCAL_STORAGE_MODEL

VSP G900

The model name of the local storage system

LOCAL_SERIAL_NUMBER

410000

The serial number of the local storage system

REMOTE_REST_SERVER_IP_ADDR

192.0.2.200

The IP address of the REST API server of the remote storage system

REMOTE_PORT

443

The SSL communication port for the REST API server of the remote storage system

REMOTE_STORAGE_MODEL

VSP G900

The model name of the remote storage system

REMOTE_SERIAL_NUMBER

420000

The serial number of the remote storage system

Contents of the sample coding

This subsection explains the sample coding.

  1. Import necessary libraries and set parameters.

    Before starting operations for the TrueCopy pair, the sample coding imports the required libraries or classes. In addition to the common libraries, the sample coding also imports the BlockStorageAPI class that defines the function that generates URLs.

    # coding:utf-8
    
    """
    synchronous_remote_copy
    
    This program requires API version 1.9.0 or newer.
    """
    
    import traceback
    import requests
    import json
    import sys
    import http.client
    import time
    import remote_copy_param
    
    from block_storage_api import BlockStorageAPI
    

    Set parameters to be used in the sample coding.

    # #################Initialize parameters################# #
    # Change the following parameters to fit your environment
    
    # A copy group name
    COPY_GROUP_NAME = "TC_GROUP"
    
    # A copy pair name
    COPY_PAIR_NAME = "p_347-348"
    
    # A primary volume ID
    # Specify already created and allocated volume ID by decimal
    PVOL_LDEV_ID = 347
    
    # A secondary volume ID which has the exactly same size
    # as the primary volume
    # Specify already created and allocated volume ID by decimal
    SVOL_LDEV_ID = 348
    
    # This parameter defines the first interval to access
    # an asynchronous job. (Unit: Second)
    FIRST_WAIT_TIME = 1
    
    # This parameter defines the maximum retry time
    # to confirm job status.
    MAX_RETRY_COUNT = 10
    
    # An user id and password of the local storage
    LOCAL_USER_CREDENTIAL = ("local_copy_user",
                             "local_copy_pass")
    
    # An user id and password of the remote storage
    REMOTE_USER_CREDENTIAL = ("remote_copy_user",
                              "remote_copy_pass")
    
    ###########################################################
    
  2. Define headers.

    Define the HTTP request header. Because the REST API only supports JSON format data, the sample coding defines header information so that data is handled in JSON format.

    # ###You don't have to change the following parameters### #
    local_storage_api = BlockStorageAPI(
        remote_copy_param.LOCAL_REST_SERVER_IP_ADDR,
        remote_copy_param.LOCAL_PORT,
        remote_copy_param.LOCAL_STORAGE_MODEL,
        remote_copy_param.LOCAL_SERIAL_NUMBER)
    
    remote_storage_api = BlockStorageAPI(
        remote_copy_param.REMOTE_REST_SERVER_IP_ADDR,
        remote_copy_param.REMOTE_PORT,
        remote_copy_param.REMOTE_STORAGE_MODEL,
        remote_copy_param.REMOTE_SERIAL_NUMBER)
    
    local_headers = {"content-type": "application/json",
                     "accept": "application/json",
                     "Response-Job-Status": "Completed"}
    
    remote_headers = {"content-type": "application/json",
                      "accept": "application/json",
                      "Response-Job-Status": "Completed"}
    
    REQUIRED_MAJOR_VERSION = 1
    REQUIRED_MINOR_VERSION = 9
    
    local_session_id = 0
    remote_session_id = 0
    
    ###########################################################
    
  3. Define the function for getting status changes for asynchronous processing. (wait_until_jobstatus_is_changed function)

    Define the function for getting status changes for asynchronous processing. Call and use this function from the main TrueCopy pair operation. For details on this function, see the description of functions used in the sample coding.

    Tip: To prevent errors that occur when the server certificate used for SSL communication between the REST API client and the storage system is a self-signed certificate, the sample coding specifies verify=False in the request message to skip verification of the server certificate.
    """
    Check whether the asynchronous command was finished.
    
    @param storage_api storage_api
    @param job_id the job ID to identify
          the asynchronous command
    @param headers the array of the http headers
    @return r the response data
    """
    
    
    def check_update(storage_api, job_id, headers):
        url = storage_api.job(str(job_id))
        r = requests.get(url, headers=headers, verify=False)
        return r
    
    """
    Wait until the job status is changed
    
    @param storage_api storage_api
    @param headers the array of the http headers
    @param job_id the job ID to identify
           the asynchronous command
    @param changed_status job status after waiting
    @param is_retry_count_enabled if true, wait
           until MAX_RETRY_COUNT. if false, wait forever
           until job status is changed.
    @return job_result.json()["affectedResources"][0]
             URL of an affected resource
    """
    
    
    def wait_until_jobstatus_is_changed(
            storage_api,
            headers,
            job_id,
            changed_status,
            is_retry_count_enabled):
        status = "Initializing"
        retry_count = 1
        wait_time = FIRST_WAIT_TIME
        while status != changed_status:
            if status == "Completed":
                print("Status was already changed" +
                      "to Completed.")
                break
            if is_retry_count_enabled and \
                    retry_count > MAX_RETRY_COUNT:
                raise Exception("Timeout Error! "
                                "Operation was not completed.")
            time.sleep(wait_time)
            job_result = check_update(storage_api,
                                      job_id, headers)
            status = job_result.json()["status"]
            double_time = wait_time * 2
            if double_time < 120:
                wait_time = double_time
            else:
                wait_time = 120
            retry_count += 1
        if job_result.json()["state"] == "Failed":
            error_obj = job_result.json()["error"]
            if "errorCode" in error_obj:
                if "SSB1" in error_obj["errorCode"]:
                    print("Error! SSB code : ",
                          error_obj["errorCode"]["SSB1"],
                          ", ", error_obj["errorCode"]["SSB2"])
                elif "errorCode" in error_obj["errorCode"]:
                    print("Error! error code : ",
                          error_obj["errorCode"]["errorCode"])
            raise Exception("Job Error!", job_result.text)
        print("Async job was succeeded. affected resource : " +
              job_result.json()["affectedResources"][0])
        return job_result.json()["affectedResources"][0]
    
    
  4. Check the version of the REST API.

    Get information about the version of the REST API for both the local and the remote storage systems by using the REST API server of each system to make sure that the version is supported.

    """
    Check whether this API version allows the REST
     Server to execute this program
    
    @param api_version api version of this REST Server
    @param required_major_version the lowest number of
           the major version that this program requires
    @param required_minor_version the lowest number of
           the minor version that this program requires
    
    """
    
    
    def check_api_version(api_version, required_major_version,
                          required_minor_version):
        version = api_version.split(".")
        major_version = int(version[0])
        minor_version = int(version[1])
        if not ((major_version == required_major_version and
                 minor_version >= required_minor_version) or
                major_version >= required_major_version + 1):
            sys.exit(
                "This program requires API Version " +
                str(required_major_version) +
                "." +
                str(required_minor_version) +
                "." +
                "x or newer.\n")
    
    try:
        # step1 Check the API version of the local REST API #
        print("Check the API version of the local REST API")
        url = local_storage_api.api_version()
        r = requests.get(url, headers=local_headers,
                         verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        check_api_version(r.json()["apiVersion"],
                          REQUIRED_MAJOR_VERSION,
                          REQUIRED_MINOR_VERSION)
    
        # step1 Check the API version of the remote REST API #
        print("Check the API version of the remote REST API")
        url = remote_storage_api.api_version()
        r = requests.get(url, headers=remote_headers,
                         verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        check_api_version(r.json()["apiVersion"],
                          REQUIRED_MAJOR_VERSION,
                          REQUIRED_MINOR_VERSION)
    
  5. Generate a session.

    Generate a session in both the local and the remote storage systems by using the REST API server of each system.

        # step2 Generate a local session #
        print("Generate a local session")
        url = local_storage_api.generate_session()
        r = requests.post(
            url,
            headers=local_headers,
            auth=LOCAL_USER_CREDENTIAL,
            verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        local_token = r.json()["token"]
        local_auth = "Session " + local_token
        local_session_id = r.json()["sessionId"]
    
        # step2 Generate a remote session #
        print("Generate a remote session")
        url = remote_storage_api.generate_session()
        r = requests.post(
            url,
            headers=remote_headers,
            auth=REMOTE_USER_CREDENTIAL,
            verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        remote_token = r.json()["token"]
        remote_auth = "Session " + remote_token
        remote_session_id = r.json()["sessionId"]
    

    When a session is generated, a session ID and a token are returned. When running the API, specify the token for the Authentication header as the required authentication information for the subsequent operations. Use the session ID to discard the session after a set of operations is completed.

  6. Lock resources.

    Obtain a lock to prevent other users from performing operations on the target volume. For the local storage system, lock the resource group to which the LDEV for the primary volume belongs. For the remote storage system, lock the resource group to which the LDEV for the secondary volume belongs.

        try:
            # step3 Lock the local resource group #
            print("Lock the local resource group")
            url = local_storage_api.lock()
            local_headers["Authorization"] = local_auth
            r = requests.post(url, headers=local_headers,
                             verify=False)
            if r.status_code != http.client.ACCEPTED:
                raise requests.HTTPError(r)
            print("Request was accepted. JOB URL : " +
                  r.json()["self"])
            wait_until_jobstatus_is_changed(
                local_storage_api,
                local_headers,
                r.json()["jobId"],
                "Completed",
                True)
    
            # step3 Lock the remote resource group #
            print("Lock the remote resource group")
            remote_headers["Authorization"] = remote_auth
            url = remote_storage_api.lock()
            r = requests.post(url, headers=remote_headers,
                             verify=False)
            if r.status_code != http.client.ACCEPTED:
                raise requests.HTTPError(r)
            print("Request was accepted. JOB URL : " +
                  r.json()["self"])
            wait_until_jobstatus_is_changed(
                remote_storage_api,
                remote_headers,
                r.json()["jobId"],
                "Completed",
                True)
    

    The wait_until_jobstatus_is_changed function checks the execution status of the jobs that were run asynchronously, and waits until the job status changes to the specified status. In the sample coding, the job execution status is confirmed to have changed to "Completed" and to have been locked.

  7. Create a TrueCopy pair.

    Use the already created LDEV to create a TrueCopy pair. Also create a new copy group. Specify the copy group name, copy pair name, and the LDEV number of the volume to be used, which are defined in advance in the parameters. In addition, specify items such as the copy pair type, whether to create a copy group, and the fence level, and then issue a request for creating a TrueCopy pair. The block_storage_api function is used to generate the URL.

            # step4 Create a remote copy pair #
            print("Create a remote copy pair")
            url = local_storage_api.remote_copy_pairs()
            body = {
                "copyGroupName": COPY_GROUP_NAME,
                "copyPairName": COPY_PAIR_NAME,
                "replicationType": "TC",
                "remoteStorageDeviceId": remote_storage_api.
                get_storage_id(),
                "pvolLdevId": PVOL_LDEV_ID,
                "svolLdevId": SVOL_LDEV_ID,
                "isNewGroupCreation": "true",
                "fenceLevel": "data",
            }
            local_headers["Remote-Authorization"] = remote_auth
            r = requests.post(
                url,
                headers=local_headers,
                data=json.dumps(body),
                verify=False)
            if r.status_code != http.client.ACCEPTED:
                raise requests.HTTPError(r)
            print("Create remote copy pair request " +
                  "was accepted. JOB URL : " + r.json()["self"])
            wait_until_jobstatus_is_changed(
                local_storage_api,
                local_headers,
                r.json()["jobId"],
                "StorageAccepted",
                False)
            jobid = r.json()["jobId"]
    
            print("Status changed to StorageAccepted")
    

    The wait_until_jobstatus_is_changed function checks the execution status of the jobs that were run asynchronously, and waits until the job status changes to the specified status. In the sample coding, it is confirmed that the job execution status has changed to "StorageAccepted" and the request for creating a TrueCopy pair has been received by the storage system.

  8. Unlock resources.

    After having confirmed that the storage system received processing for creating the pair, cancel the obtained lock. The "finally" statement in the sample coding makes sure that the lock will be canceled even if an error occurs while the API is running.

        finally:
            # step5 Unlock the local resource group #
            print("Unlock the local resource group")
            url = local_storage_api.unlock()
            r = requests.post(url, headers=local_headers,
                             verify=False)
            if r.status_code != http.client.ACCEPTED:
                raise requests.HTTPError(r)
            print("Request was accepted. JOB URL : " +
                  r.json()["self"])
            wait_until_jobstatus_is_changed(
                local_storage_api, local_headers,
                r.json()["jobId"], "Completed", True)
    
            # step5 Unlock the remote resource group #
            print("Unlock the remote resource group")
            url = remote_storage_api.unlock()
            r = requests.post(url, headers=remote_headers,
                             verify=False)
            if r.status_code != http.client.ACCEPTED:
                raise requests.HTTPError(r)
            print("Request was accepted. JOB URL : " +
                  r.json()["self"])
            wait_until_jobstatus_is_changed(
                remote_storage_api,
                remote_headers,
                r.json()["jobId"],
                "Completed",
                True)
    
  9. Confirm that the pair has been created.

    Confirm that processing for creating the pair is completed in the storage system. Use the wait_until_jobstatus_is_changed function to confirm that the job execution status has changed to "Completed".

        # step6 Wait until the operation is complete #
        affected_resource_path = wait_until_jobstatus_is_changed(
            local_storage_api, local_headers,
            jobid, "Completed", False)
    
  10. Get information about a TrueCopy pair.

    To confirm that the pair has been correctly created, get information about the pair by using the URL of the pair that was obtained when the TrueCopy pair was created. In the sample coding, the following items are output: the copy group name, copy pair name, pair type, LDEV numbers for the P-VOL and S-VOL, pair volume status, and the storage device ID.

        # step7 Print the remote copy pair #
        print("Print the remote copy pair")
        url = local_storage_api.affected_resource(
            affected_resource_path)
        r = requests.get(url, headers=local_headers,
                         verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        print("COPY GROUP NAME : " +
              str(r.json()["copyGroupName"]))
        print("COPY PAIR NAME : " +
              str(r.json()["copyPairName"]))
        print("REPLICATION TYPE : " +
              str(r.json()["replicationType"]))
        print("PVOL LDEV ID : " + str(r.json()["pvolLdevId"]))
        print("SVOL LDEV ID : " + str(r.json()["svolLdevId"]))
        print("PVOL STATUS : " + str(r.json()["pvolStatus"]))
        print("SVOL STATUS : " + str(r.json()["svolStatus"]))
        print("PVOL STORAGE DEVICE ID : "
              + str(r.json()["pvolStorageDeviceId"]))
        print("SVOL STORAGE DEVICE ID : "
              + str(r.json()["svolStorageDeviceId"]))
        print("REMOTE MIRROR COPY PAIR ID : "
              + str(r.json()["remoteMirrorCopyPairId"]))
        print()
    
  11. Output error messages.

    In the sample coding, processing for communication errors, HTTP request errors, and job execution errors is described. If a communication error occurs, an error message is output. If an HTTP request error occurs, the error code, the error message, and the response body are output. If a job execution error occurs, all of the contents included in the job execution result are output.

    except requests.ConnectionError:
        sys.stderr.write("Connection Error!\n")
        sys.stderr.write(traceback.format_exc())
    except requests.HTTPError as he:
        sys.stderr.write("HTTP Error! status code : ")
        sys.stderr.write(str(he.args[0].status_code) + "\n")
        sys.stderr.write(he.args[0].text + "\n")
    except Exception as e:
        sys.stderr.write(traceback.format_exc())
        for msg in e.args:
            sys.stderr.write(str(msg) + "\n")
    
  12. Discard the session.

    After a set of operations is completed, discard the session by using the REST API server of both the local and the remote storage systems. Specify the session ID that was obtained when the session was created. The "finally" statement in the sample coding makes sure that the session will be discarded even if an error occurs while the API is running. After the session is discarded, the processing ends.

    finally:
        # step8 Discard the local session #
        print("Discard the local session")
        url = local_storage_api. \
            discard_session(local_session_id)
        r = requests.delete(url, headers=local_headers,
                            verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
    
        # step8 Discard the remote session #
        print("Discard the remote session")
        url = remote_storage_api.discard_session(
            remote_session_id)
        r = requests.delete(url,
                            headers=remote_headers, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
    
        print("Operation was completed.")
        sys.exit()