The following table describes the services that HCP for cloud scale runs. Each service runs within its own Docker container. For each service, the table lists:
- Configuration settings: The settings you can configure for the service.
- RAM needed per instance: The amount of RAM that, by default, the service needs on each instance on which it's deployed. For all services except for System services, this value is also the default Docker value of Container Memory for the service.
- Number of instances: Shows both:
- The minimum number of instances on which a service must run to function properly.
- The best number of instances on which a service should run. If the system includes more than the minimum number of instances, you should take advantage of the instances by running services on them.
Note: Unused services do not need to be scaled. - Service unit cost: For HCP for cloud scale, you can safely ignore these values.
- Whether the service is stateful (that is, it saves data permanently to disk) or stateless (that is, it does not save data to disk).
- Whether the service is persistent (that is, it must run on a specific instance) or supports floating (that is, it can run on any instance).
- Whether the service is scalable or not.
Best practices when using system services
- For HCP for cloud scale services, you cannot set the size of Max Heap Size larger than the value of the Container Memory setting.
- For the services which allow you to set the size of Max Heap Size, it should not be made to be larger than the value of the Container Memory setting. It's best to set this value to half the size of the container memory.
- The decision to scale a service beyond the minimum recommended nodes should be made on the basis of a performance evaluation and applied on a case-by-case judgment for each individual service. When the scale of a service is changed to exceed the minimum recommended nodes, it then becomes a performance sensitive use case. When this occurs, you must follow the rules contained within the "Sizing and scaling models" section found under Best practices for system sizing and scaling.
- Similarly, the decision to customize RAM settings beyond their recommended values should be made on the basis of a performance evaluation of the entire cluster and applied on a case-by-case judgment for each individual service.
- When scaling services, it is important to ensure that all nodes of a cluster have enough RAM allocated to them in order to run the service under the new parameters. Attempting to scale services to spare nodes in order to compensate for the loss of a node may fail due to an insufficient amount of available RAM.
- On existing systems which are currently running services below their recommended RAM sizing, it is highly advised to not apply the recommended values without a prior performance evaluation as stated above.
Service name and description | Configuration settings (changes cause the service to redeploy) | Properties |
---|---|---|
Product services: These services perform HCP for cloud scale functions. You can move and reconfigure these services. | ||
Cassandra Decentralized database, used to stores some configuration and system update packages |
Container Options: Default
Service Options
Advanced Options Compaction Frequency: How often the database is compacted. The options are Weekly (default) and Daily. Caution: Changing this setting can negatively affect the service. Use with caution. |
RAM needed per instance: 2.4 GB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateful Persistent or floating? Persistent Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
Chronos Job scheduler |
Container Options: Default
Service Options
|
RAM needed per instance: 712 MB Number of instances: Unscalable, permanently runs on 1 Service unit cost: 1 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? Yes Single or multiple types? Single Scalable? Yes |
Data Lifecycle Processes lifecycle policies |
Container Options: Default
Service Options
|
RAM needed per instance: 8 GB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? Yes Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
Elasticsearch Indexes metrics and event logs |
Container Options: Default
Service Options
|
RAM needed per instance: 8 GB Number of instances:
Service unit cost: 25 Stateful or stateless? Stateful Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? Yes |
Grafana Collects data and displays dashboard metrics |
Container Options: Default
Service Options
|
RAM needed per instance: 768 MB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateful Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |
Kafka Handles metrics and event logs |
Container Options: Default
Service Options
|
RAM needed per instance: 2 GB Number of instances:
Service unit cost: 5 Stateful or stateless? Stateful Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? Yes |
Key Management Server Manages storage component encryption keys |
Container Options: Default
Service Options None. |
RAM needed per instance: 2 GB Number of instances:
Important: In order to save on system resources, 1 instance is only recommended if you do not plan on using the encryption service. The minimum number of instances required when encryption is enabled is 3, which must be scaled prior to enabling encryption.
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? Yes Single or multiple types? Single Scalable? Yes |
Logstash Handles metrics and event logs |
Container Options: Default
Service Options
|
RAM needed per instance: 700 MB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
MAPI Gateway Serves MAPI endpoints |
Container Options: Default
Service Options
|
RAM needed per instance: 768 MB Number of instances: Unscalable, permanently runs on 1 Service unit cost: 5 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
Message Queue Coordinates and distributes messages to other services |
Container Options: Default
Service Options
|
RAM needed per instance: 8 GB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateful Persistent or floating? Persistent Supports volume configuration? No Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
Metadata Cache Cache for HCP for cloud scale metadata Note: This service is deprecated but cannot be removed. |
Container Options: Default
Service Options
|
RAM needed per instance: 768 GB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Persistent Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
Metadata Coordination Coordinates Metadata Gateway service instances and coordinates scaling and balancing of metadata partitions |
Container Options: Default
Service Options
|
RAM needed per instance: 2 GB Number of instances: Unscalable, permanently runs on 1 Service unit cost: 5 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes Important: Your partition count must be kept under 1500 partitions per node. Once you reach this limit, system performance may be affected. If you are encountering issues, contact your Hitachi Vantara representative for further assistance.
|
Metadata Gateway Stores and protects metadata and serves it to other services |
Container Options: Default
Service Options
|
RAM needed per instance:
Number of instances:
Service unit cost: 50 Stateful or stateless? Stateful Persistent or floating? Persistent Supports volume configuration? No Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
Metrics Gathers metrics from all services and instances and supplies them to GUI and API |
Container Options: Default
Service Options
|
RAM needed per instance: 4 GB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateful Persistent or floating? Persistent Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
Mirror In Executes synch-from policies |
Container Options: Default
Service Options
|
RAM needed per instance: 4 GB Number of instances:
Important: The default value of instances for this service is 0. In order to utilize this service, you must scale it up to a minimum of 1 instance. This change is irreversible and can not be scaled back down.
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
Mirror Out Executes system synch-to policies |
Container Options: Default
Service Options
|
RAM needed per instance: 4 GB Number of instances:
Important: The default value of instances for this service is 0. In order to utilize this service, you must scale it up to a minimum of 1 instance. This change is irreversible and can not be scaled back down.
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
Policy Engine Executes system policies |
Container Options: Default
Service Options
|
RAM needed per instance: 4 GB Number of instances:
Service unit cost: 25 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
S3 Gateway Serves S3 API methods and communicates with storage components |
Container Options: Default
Service Options
HTTP Options
HTTPS Options
|
RAM needed per instance:
Number of instances:
Service unit cost: 25 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
S3 Notifications Executes S3 notifications |
Container Options: Default
Service Options
|
RAM needed per instance: 4 GB Number of instances:
Important: The default value of instances for this service is 0. In order to utilize this service, you must scale it up to a minimum of 1 instance. This change is irreversible and can not be scaled back down.
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes (but not recommended on master instances) |
Tracing Agent Listens for incoming tracing of S3 API and MAPI calls, batches them, and sends them to Tracing Collector service |
Container Options: Default
Service Options
|
RAM needed per instance: 2 GB Number of instances:
Service unit cost: 1 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
Tracing Collector Collects traces from Tracing Agent service instances and stores them in tracing database |
Container Options: Default
Service Options
|
RAM needed per instance: 2 GB Number of instances:
Service unit cost: 10 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
Tracing Query UI and API endpoint access for distributed tracing for S3 API and MAPI calls |
Container Options: Default
Service Options
|
RAM needed per instance: 768 MB Number of instances:
Service unit cost: 5 Stateful or stateless? Stateless Persistent or floating? Floating Supports volume configuration? No Single or multiple types? Single Scalable? Yes |
System services: These services manage system resources and ensure that the HCP for cloud scale system remains available and accessible. These services are persistent and cannot be moved, scaled, or reconfigured. | ||
Admin App The System Management application |
Service Options
|
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on all 3 master nodes Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |
Cluster Coordination Manages hardware resource allocation |
None. |
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on all 3 master nodes Persistent or floating? Persistent Supports volume configuration? No Single or multiple types? Single Scalable? No |
Cluster Worker Agent for Cluster Coordination on each instance; reports on resource utilization and availability, deploys services |
None. |
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on all nodes Service unit cost: 5 Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |
Network Proxy Network request load balancer |
Security Protocol: Select which Transport Layer Security (TLS) versions to use:
SSL Ciphers: To use another cipher suite, type it here. Custom Global Configuration: Select Enable Advanced Global Configuration to enable adding custom parameters to the HAProxy "global" section. Custom Defaults Configuration: Select Enable Defaults Configuration to enable adding custom parameters to the HAProxy "global" section. |
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on all nodes Service unit cost: 1 Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |
Sentinel Runs internal system processes and monitors the health of other services |
Service Options
|
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on 1 Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |
Service Deployment Handles deployment of high-level services (that is, the services that you can configure) |
None. |
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on all 3 master nodes Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |
Synchronization Coordinates service configuration settings and other information across service instances |
Service Options
|
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on all 3 master nodes Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |
Watchdog Responsible for initial system startup; monitors other System services and restarts them if necessary |
Service Options
|
RAM needed per instance: N/A Number of instances: Unscalable, permanently runs on all nodes Service unit cost: 5 Persistent or floating? Persistent Supports volume configuration? Yes Single or multiple types? Single Scalable? No |