Sample coding for uploading files (specifying the transfer destinations of audit log files)

REST API Reference Guide for Virtual Storage Platform 5000, Virtual Storage Platform E Series, and Virtual Storage Platform G/F Series

Version
93-07-0x
90-09-0x
88-08-10
Audience
anonymous
Part Number
MK-98RD9014-17
This section provides sample code for specifying the transfer destinations of the audit log files of storage systems. This sample coding includes the procedure for uploading the client certificates to a storage system.

Operation flow of the sample code for specifying the transfer destinations of the audit log files of storage systems

The following is the operation flow of the sample code for specifying the transfer destinations of the audit log files of storage systems and the corresponding code constructs.

Step

Sample coding operation flow

Code constructs

1

Import necessary libraries and set parameters.

-

2

Define headers.

Specifying request headers (for the default HTTP headers)

3

Define functions for issuing an HTTP request and for verifying the status of asynchronous processing.

Getting information about the job status by performing a GET operation

Setting user authentication information (for authentication by using session-based authentication)

Generating a request body in JSON format

Getting the job execution results

Getting the URLs of the resources to which the operation results have been applied

Getting error codes

4

Check the version of the REST API.

Getting information about the version of the REST API by performing a GET operation

5

Generate a session.

Getting the URLs of the resources (when object IDs are not specified)

Setting user authentication information (for authentication by using a user ID and a password)

Creating objects by performing a POST operation

6

Upload the root certificate of the syslog server to which audit log files are to be transferred.

Getting the URLs of the resources (when object IDs are not specified)

Creating form data

Running actions for objects by using the POST operation

7

Upload the client certificate of the syslog server to which audit log files are to be transferred.

Getting the URLs of the resources (when object IDs are not specified)

Creating form data

Running actions for objects by using the POST operation

8

Specify the syslog server to which audit log files are to be transferred.

Getting the URLs of the resources (when fixed object IDs such as those for single instances are specified)

Generating a request body in JSON format

Changing the attribute of an object by using the PATCH operation

9

Send a test message to the syslog server to which audit log files are to be transferred.

Getting the URLs of the resources (when fixed object IDs such as those for single instances are specified)

Running actions for objects by using the POST operation

10

Obtain the configuration information of the transfer destinations of audit log files

Getting the URLs of the resources (when fixed object IDs such as those for single instances are specified)

Obtaining an object by using the GET operation (when obtaining a specific object)

Outputting obtained information

11

Output error messages.

Outputting error messages

12

Discard the session.

Getting the URLs of the resources (when object IDs that are obtained from the operation results are specified)

Deleting objects by performing a DELETE operation

Expected system configuration

The sample code assumes the system configuration in the following figure. Note that the transfer destination is a single syslog server.

The following table shows the values specified for the parameters in the sample coding. If necessary, change the settings to match the system environment and requirements.

Parameter

Value

Description

USER_CREDENTIAL

("user1", "pass1")

Authentication information used for authentication by the storage system. In the sample code, the user ID is user1 and the password is pass1. The user must have the Audit Log Administrator (View & Modify) role.

ROOT_CERT_FILE_PATH

"D:\\cert\\"

The path where the root certificate file of the syslog server (the file to be uploaded to the storage system) is stored.

The sample code specifies for SSL communication to be used between the storage system and the syslog server. Prepare the root certificate of the syslog server in advance.

ROOT_CERT_FILE_NAME

"root.crt"

The name of the root certificate file of the syslog server (the file to be uploaded to the storage system).

CLIENT_CERT_FILE_PATH

"D:\\cert\\"

The path where the client certificate file of the syslog server (the file to be uploaded to the storage system) is stored.

The sample code specifies for SSL communication to be used between the storage system and the syslog server. Prepare the client certificate of the syslog server in advance.

CLIENT_CERT_FILE_NAME

"client.pfx"

The name of the client certificate file of the syslog server (the file to be uploaded to the storage system).

TRANSFER_PROTOCOL

"TLS"

The protocol to be used when transferring the audit logs to the syslog server.

LOCATION_NAME

"STORAGE_SYSTEM_1"

The name that identifies the storage system from which the audit logs are to be transferred.

RETRY_INTERVAL

5

The interval (in seconds) between retries when communication with the syslog server fails.

PRIMARY_SYSLOG_SERVER_IP_ADDR

"192.0.1.101"

The IP address of the syslog server.

PRIMARY_SYSLOG_SERVER_PORT

"12345"

The port number used by the syslog server.

CLIENT_CERT_FILE_PASSWORD

"certFilePass"

The password for the client certificate file.

FIRST_WAIT_TIME

1

The first interval (in seconds) for collecting the execution result of asynchronous processing. Normally, you do not need to change this value.

MAX_RETRY_COUNT

6

The maximum number of retries for collecting the execution result of asynchronous processing. Normally, you do not need to change this value.

Contents of the sample coding

This subsection explains the sample coding.

  1. Import necessary libraries and set parameters.

    Before starting the volume allocation processing, the sample coding imports the required libraries or classes. In addition to the common libraries, the sample coding also imports the BlockStorageAPI class that defines the function that generates URLs.
    # coding:utf-8
    
    """
    auditlog_syslog_server setting
    
    This program requires API version 1.9.0 or newer.
    """
    
    import requests
    import json
    import sys
    import http.client
    import time
    import traceback
    import rest_server_param
    import storage_param
    
    from block_storage_api import BlockStorageAPI
    
    Set parameters to be used in the sample coding.
    # #################Initialize parameters################# #
    # Change the following parameters to fit your environment
    
    # A path of root certificate
    ROOT_CERT_FILE_PATH = "D:\\cert\\"
    
    # A root certificate name
    ROOT_CERT_FILE_NAME = "root.crt"
    
    # A path of client certificate
    CLIENT_CERT_FILE_PATH = "D:\\cert\\"
    
    # A client certificate name
    CLIENT_CERT_FILE_NAME = "client.pfx"
    
    # A transfer protocol
    TRANSFER_PROTOCOL = "TLS"
    
    # A location name
    LOCATION_NAME = "STORAGE_SYSTEM_1"
    
    # A retry interval
    RETRY_INTERVAL = 5
    
    # A primary syslog server IP address
    PRIMARY_SYSLOG_SERVER_IP_ADDR = "192.0.1.101"
    
    # A primary syslog server port number
    PRIMARY_SYSLOG_SERVER_PORT = "12345"
    
    # A password of the client certificate
    CLIENT_CERT_FILE_PASSWORD = "certFilePass"
    
    # This parameter defines the first interval to access
    # an asynchronous job. (Unit: Second)
    FIRST_WAIT_TIME = 1
    
    # This parameter defines the maximum retry time
    # to confirm job status.
    MAX_RETRY_COUNT = 6
    
    # An user id and password of the target storage
    USER_CREDENTIAL = ("user1", "pass1")
    
    ###########################################################
  2. Define headers.

    Define the HTTP request header. In addition to the JSON format, which is the standard format for the REST API, define the header information so that the form data format, which used in by the API function for uploading the file, can also be handled.
    # ###You don't have to change the following parameters### #
    block_storage_api = BlockStorageAPI(
        rest_server_param.REST_SERVER_IP_ADDR,
        rest_server_param.REST_SERVER_PORT,
        storage_param.STORAGE_MODEL,
        storage_param.SERIAL_NUMBER)
    
    headers = {"content-type": "application/json",
               "accept": "application/json"}
    file_upload_headers = {"accept": "application/json",
                           "Expect": ""}
    
    REQUIRED_MAJOR_VERSION = 1
    REQUIRED_MINOR_VERSION = 9
    
    session_id = 0
    
    ###########################################################
  3. Define the function for issuing an HTTP request and for verifying the status of asynchronous processing (the invoke_async_command function).

    Define the function that issues an HTTP request and verifies the status of asynchronous processing. Call and use this function from the main volume allocation operation. For details on this function, see the section explaining the functions used in the sample coding.

    Tip: To prevent errors that occur when the server certificate used for SSL communication between the REST API client and the storage system is a self-signed certificate, the sample coding specifies verify=False in the request message to skip verification of the server certificate.
    """
    Check whether the asynchronous command was finished.
    @param job_id the job ID to identify
           the asynchronous command
    @return r the response data
    """
    
    
    def check_update(job_id):
        url = block_storage_api.job(str(job_id))
        r = requests.get(url, headers=headers, verify=False)
        return r
    
    """
    Execute the HTTP request (POST or PATCH)
    @param method_type HTTP request method (POST or PATCH)
    @param url URL to execute HTTP method
    @param body The information of a resource
    @return job_result.json()["affectedResources"][0]
             URL of an affected resource
    """
    
    
    def invoke_async_command(method_type, url, body):
        if method_type == "patch":
            r = requests.patch(url, headers=headers,
                             data=json.dumps(body), verify=False)
        elif method_type == "post":
            if body is None:
                r = requests.post(
                    url,
                    headers=headers,
                    verify=False)
            else:
                r = requests.post(
                    url,
                    headers=headers,
                    data=json.dumps(body),
                    verify=False)
        if r.status_code != http.client.ACCEPTED:
            raise requests.HTTPError(r)
        print("Request was accepted. JOB URL : " +
              r.json()["self"])
    
        if "affectedResources" in job_result.json():
            print("Async job was succeeded." +
                  " affected resource : " +
                  job_result.json()["affectedResources"][0])
            return job_result.json()["affectedResources"][0]
        else:
            print("Async job was succeeded.")
            return None
    
  4. Check the version of the REST API.

    Get information about the version of the REST API to make sure that the version is supported.
    """
    Check whether this API version allows the REST
     Server to execute this program
    
    @param api_version api version of this REST Server
    @param required_major_version the lowest number of
           the major version that this program requires
    @param required_minor_version the lowest number of
           the minor version that this program requires
    
    """
    
    
    def check_api_version(api_version, required_major_version,
                          required_minor_version):
        version = api_version.split(".")
        major_version = int(version[0])
        minor_version = int(version[1])
        if not ((major_version == required_major_version and
                 minor_version >= required_minor_version) or
                major_version >= required_major_version + 1):
            sys.exit("This program requires API Version " +
                     str(required_major_version) + "." +
                     str(required_minor_version) +
                     "." + "x or newer.\n")
    
    try:
        # step1 Check the API version #
        print("Check the API version")
        url = block_storage_api.api_version()
        r = requests.get(url, headers=headers, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        check_api_version(
            r.json()["apiVersion"],
            REQUIRED_MAJOR_VERSION,
            REQUIRED_MINOR_VERSION)
    
  5. Generate a session.

    Generate sessions by using the REST API server.
        # step2 Generate a session #
        print("Generate a session")
        url = block_storage_api.generate_session()
        r = requests.post(url, headers=headers, auth=USER_CREDENTIAL,
                          verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        token = r.json()["token"]
        auth = "Session " + token
        session_id = r.json()["sessionId"]
        headers["Authorization"] = auth
        file_upload_headers["Authorization"] = auth
    

    When a session is generated, a session ID and a token are returned. When running the API, specify the token for the Authentication header as the required authentication information for the subsequent operations. Use the session ID to discard the session after a set of operations is completed.

  6. Upload the root certificate of the syslog server to which the audit log files are to be transferred.

    As preparation for transferring the audit log files to the syslog server by using SSL communication, upload the root certificate of the syslog server to the storage system.
        # step3 Upload a root certificate #
        print("Upload a root certificate")
        url = block_storage_api.file_upload()
        files = {"file": (ROOT_CERT_FILE_NAME,
            open(ROOT_CERT_FILE_PATH + ROOT_CERT_FILE_NAME, "rb"),
            "application/octet-stream")}
        r = requests.post(url, headers=file_upload_headers,
            data={"fileType": "AuditSyslogPrimaryRootCertFile"},
            files=files, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
    
  7. Upload the client certificate of the syslog server to which audit log files are to be transferred.

    As preparation for transferring the audit log files to the syslog server by using SSL communication, upload the client certificate of the syslog server to the storage system.
        # step4 Upload a client certificate #
        print("Upload a client certificate")
        files = {"file": (CLIENT_CERT_FILE_NAME,
            open(CLIENT_CERT_FILE_PATH + CLIENT_CERT_FILE_NAME,
            "rb"), "application/octet-stream")}
        r = requests.post(url, headers=file_upload_headers,
            data={"fileType":
            "AuditSyslogPrimaryClientCertFile"},
            files=files, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
    
  8. Specify the syslog server to which audit log files are to be transferred.

    Specify the settings for transferring audit log files to the syslog server. Because the sample code specifies for SSL communication to be used for communication with the syslog server, in addition to information about the syslog server to which audit log files are to be transferred, the sample code also specifies information about the certificate for SSL communication.
        # step5 Modify the syslog server #
        print("Modify the syslog server")
        url = block_storage_api.auditlog_syslog()
        body = {
            "transferProtocol": TRANSFER_PROTOCOL,
            "locationName": LOCATION_NAME,
            "retries": True,
            "retryInterval": RETRY_INTERVAL,
            "primarySyslogServer": {
                "isEnabled": True,
                "ipAddress": PRIMARY_SYSLOG_SERVER_IP_ADDR,
                "port": PRIMARY_SYSLOG_SERVER_PORT,
                "clientCertFileName": CLIENT_CERT_FILE_NAME,
                "clientCertFilePassword": CLIENT_CERT_FILE_PASSWORD,
                "rootCertFileName": ROOT_CERT_FILE_NAME
            },
            "secondarySyslogServer": {
                "isEnabled": False
            }
        }
        invoke_async_command("patch", url, body)
    

    The invoke_async_command function issues a request to specify the transfer destination of audit log files and checks the execution status of jobs that are executed asynchronously. The function then returns, as the execution result, the URL of information about the transfer destination of audit log files that is specified for the storage system.

  9. Send a test message to the syslog server to which audit log files are to be transferred.

    Send a test message to check whether the syslog server that is set as the transfer destination of audit log files can correctly receive audit log data.
        # step6 Send a test message to the syslog server #
        print("Send a test message to the syslog server")
        url = block_storage_api.auditlog_syslog_send_test()
        invoke_async_command("post", url, None)
    
  10. Obtain the configuration information of the transfer destinations of audit log files.

    To make sure that the operations in the previous steps (operations to set information on the storage system) have been successful, obtain and output the configuration information of the transfer destinations of audit logs. The sample code obtains and outputs the following information: the protocol to be used, the identifier of the storage system from which audit log files are to be transferred, retry settings and the interval for retries when communication fails, and the IP address and port number of the syslog server.
        # step7 Get the syslog server #
        print("get the syslog server")
        url = block_storage_api.auditlog_syslog()
        r = requests.get(url, headers=headers, verify=False)
        if r.status_code != http.client.OK:
            raise requests.HTTPError(r)
        print("TRANSFER PROTOCOL : " +
              str(r.json()["transferProtocol"]))
        print("LOCATION NAME : " +
              str(r.json()["locationName"]))
        print("RETRIES : " +
              str(r.json()["retries"]))
        print("RETRY INTERVAL : " +
              str(r.json()["retryInterval"]))
        print("PRIMARY SYSLOG SERVER")
        print("IP ADDRESS : " +
              str(r.json()["primarySyslogServer"]["ipAddress"]))
        print("PORT : " +
              str(r.json()["primarySyslogServer"]["port"]))
    
  11. Output error messages.

    In the sample coding, processing for communication errors, HTTP request errors, and job execution errors is described. If a communication error occurs, an error message is output. If an HTTP request error occurs, the error code, the error message, and the response body are output. If a job execution error occurs, all of the contents included in the job execution result are output.
    except requests.ConnectionError:
        sys.stderr.write("Connection Error!\n")
        sys.stderr.write(traceback.format_exc())
    except requests.HTTPError as he:
        sys.stderr.write("HTTP Error! status code : ")
        sys.stderr.write(str(he.args[0].status_code) + "\n")
        sys.stderr.write(he.args[0].text + "\n")
    except Exception as e:
        sys.stderr.write(traceback.format_exc())
        for msg in e.args:
            sys.stderr.write(str(msg) + "\n")
    
  12. Discard the session.

    After a set of operations is completed, discard the session. Specify the session ID that was obtained when the session was generated. The "finally" statement in the sample coding makes sure that the session will be discarded even if an error occurs while the API is running. After the session is discarded, the processing ends.
    finally:
        # step8 Discard the session #
        print("Discard the session")
        url = block_storage_api.discard_session(session_id)
        r = requests.delete(url, headers=headers, verify=False)
        try:
            if r.status_code != http.client.OK:
                raise requests.HTTPError(r)
        except requests.HTTPError as he:
            sys.stderr.write("HTTP Error! status code : ")
            sys.stderr.write(str(he.args[0].status_code) + "\n")
            sys.stderr.write(he.args[0].text + "\n")
    
        print("Operation was completed.")
        sys.exit()