Content Intelligence Workflow Designer Help

Version: 2.1.x
Audience: anonymous
Part Number: MK-HCI007-01
Published: 2023-10-23

Quick Start: Indexing and searching
Create a data connection to one of your data sources
Create an index collection
Create, edit, and run a workflow
Search for files
Administering Hitachi Content Search
About Hitachi Content Search
Search App
Configuring your user's search experience
Enabling full-content search
Search result highlighting
Making images, audio, and video appear in search results
Query suggestions
Configuring what query suggestions are based on
Adding facets to the Search App
Search result sorting
Configuring per document security
Adding file type icons to search results
Special cases and scenarios
Using aggregations and triggers to monitor a workflow
Create the trigger pipeline
Create the trigger
Run your workflow
Writing HCI custom metadata to HCP objects
Writing file contents as custom metadata to HCP objects
Adding document caching to your workflows
Copying data from Amazon S3 to HCP
Parsing and indexing CSV and log files
Conditionally, separate log files into individual entries
Create a workflow to parse and index individual log entries
Documents
Workflows
Creating a workflow
Viewing workflows
Searching workflows
Workflow inputs
Adding existing inputs to a workflow
Adding new inputs to a workflow
Removing inputs from a workflow
Workflow pipelines
Adding pipelines to a workflow
Reordering pipelines in a workflow
Removing pipelines from a workflow
Workflow outputs
Adding existing outputs to a workflow
Adding new index collections to a workflow
Removing outputs from a workflow
Aggregations
Viewing aggregation values
Aggregation types
Default aggregations
Aggregations, triggers, and workflow notifications
Adding aggregations to workflows
Viewing aggregations for a workflow
Triggers
Trigger expressions
List of trigger comparators
Example trigger expression
Trigger pipelines
Trigger pipeline documents
Example trigger pipeline document
Adding triggers to a workflow
Example trigger pipeline
Workflow metrics information for triggers
Workflow tasks
Running workflow tasks
Run a workflow task
Running multiple workflows at the same time
Configuring where workflows run
Pausing and resuming tasks
Task settings
Document Discovery settings
Performance settings
Metrics settings
Error Handling settings
Memory settings
Reconfiguring tasks
Scheduling tasks
Determining when a running task will stop
Task details, status, and results
Task Status
Status messages
Status icons
Failures
Metrics
Performance
Aggregations task details
Triggers
Viewing workflow task details
Task status icons
Working with fields discovered by a workflow task
Clearing workflow tasks
Starting a task over
Downloading task reports
Testing individual document failures
Retrying task failures
Testing workflows
Deleting a workflow
Importing and exporting workflow components
Exporting workflow components
Importing workflow components
Resuming halted workflows
Actions
Supported actions
Permissions for data connection actions
Performing actions
Execute Action stage advantages
Workflow output action advantages
Data connections
Data connection types and settings
Box data connection (Preview Mode)
Authentication
List-based data connection
Configuration settings
Supported actions
Document fields
Microsoft Sharepoint data connection (Preview Mode)
Authentication
List-based data connection
Configuration settings
Supported actions
Document fields
Microsoft OneDrive for Business data connection
Authentication
List-based data connection
Configuration settings
Supported actions
Document fields
HCP (Hitachi Content Platform) data connection
HCP system requirements
Configuration settings
Supported actions
Authentication and action permissions
HCP object versioning
Checking for updates with an HCP connector
How this data connection determines which file to perform an action on
How this data connection populates HCI_relativePath field
Setting up the HCP connector to perform a Copy File action across HCP tenants
HCP MQE (Hitachi Content Platform Metadata Query Engine) data connection
HCP system requirements
HCP MQE object versioning
Authentication and action permissions
Checking for updates with an MQE connector
Configuration settings
Supported actions
How the data connection determines which file to perform an action on
How this data connection populates the HCI_relativePath field
HCP Anywhere (Single user) data connection
HCP Anywhere system requirements
Checking for updates with an HCP Anywhere single user connector
Configuration settings
Supported actions
How an HCP Anywhere data connection populates HCI_relativePath
HCP Anywhere (System-wide) data connection
HCP Anywhere system and user requirements
Checking for updates with an HCP Anywhere system connector
Configuration settings
Supported actions
How this data connection determines which file to perform an action on
HCP for Cloud Scale Monitoring data connection (Preview Mode)
Configuration settings
Amazon® S3 data connection
Authentication
Checking for updates with an Amazon S3 connector
Configuration settings
Supported actions
How the data connection determines which file to perform an action on
How this data connection populates the HCI_relativePath field
Considerations - Amazon S3 connections
S3 Compatible data connection
Authentication
Connecting to HCP over S3
Checking for updates with an S3 compatible connector
Configuration settings
Supported actions
How the data connection determines which file to perform an action on
How this data connection populates the HCI_relativePath field
Considerations - S3 compatible connections
HCP Monitoring data connection
How this data connection collects information
Supported HCP versions
Fields added to all documents
Fields added by the Node Status API
Fields added by MAPI
Fields added by SNMP
Configuration settings
HCP Syslog Kafka Queue data connection
HCP requirements
Checking for updates with this connector
Configuration settings
Testing this connection
Kafka Queue data connection
Checking for updates with this connector
Configuration settings
Supported actions
PostgreSQL JDBC data connection
Authentication
Avoid restarting workflow tasks that use this data connection
Checking for updates for an SQL data connector
Configuration settings
Supported actions
MySQL and MariaDB JDBC data connection
Authentication
Avoid restarting workflow tasks that use this data connection
Checking for updates for an SQL data connector
Configuration settings for a MySQL and MariaDB JDBC connection
Supported actions
Solr JDBC data connection
Avoid restarting workflow tasks that use this data connection
Checking for updates for an SQL data connector
Configuration settings for a MySQL and MariaDB JDBC connection
Configuration settings for a Solr JDBC data connection
Supported actions
Hadoop File System data connection
Authentication
Checking for updates with a Hadoop data connection
Configuration settings
Supported actions
How this data connection determines which file to perform an action on
How this data connection populates the HCI_relativePath field
Local File System data connection (FOR TEST USE ONLY)
Permissions for use
Connecting to NFS file systems
POSIX metadata
Configuration settings
Supported actions
How this data connection determines where to perform an action
How this data connection populates the HCI_relativePath field
GUID-ADCDEB60-D25B-45B6-B5C0-A4E4D5BD4AE1.html
Permissions for use
Authentication
Configuration settings
Supported actions
How this data connection determines where to perform an action
How this data connection populates the HCI_relativePath field
Solr Query data connections
Solr Query data connection
Internal Index Query data connection
Configuration settings for a Solr Query connector
Configuration settings for an Internal Index Query connector
Crawling behavior for Solr Query connectors
Creating data connections
Viewing data connections
Editing data connections
Testing data connections
Adding data connections to workflows
Deleting data connections
Maintaining timestamp metadata on CIFS, HDFS, and LFS connectors
Processing pipelines and stages
Creating and modifying processing pipelines
Creating a new pipeline
Adding stages to a pipeline
Configuring stages
Adding conditional statements to a pipeline
Adding actions to a pipeline
Removing stages from a pipeline
Viewing pipelines
Built-in stages
Add Metadata stage
Appender stage
Attach Stream stage
Cache Stream stage
Content Class Extraction stage
Date Conversion stage
Decompression stage
DICOM Metadata Extraction stage
Drop Documents stage
Document Security stage
Email Expansion stage
Email Notification stage
Execute Action stage
Filter stage
Field Parser stage
Geocoding stage
HCP Access Log Parser stage
HCP System Events Parser stage
HCP Chargeback Report Parser stage
HCP Resource Metrics Parser stage
JavaScript stage
Single Field Transformation Mode
Document Transformation Mode
Mapping stage
Mbox Expansion stage
MIME Type Detection stage
PST Expansion stage
Read Lines stage
Replace stage
Reject Documents stage
Snippet Extraction stage
SRT Text Extraction stage
Syslog Notification stage
Tagging stage
TAR Expansion stage
Text and Metadata Extraction stage
Tokenizer stage
URL Encoder/Decoder stage
XML Formatter stage
ZIP Expansion stage
Built-in pipelines
Default pipeline
Basic Preprocessing pipeline
Basic Extraction pipeline
Conditional statements
Creating your own stages
Testing pipelines
Testing pipelines
Viewing stream contents
Working with fields discovered by a test
Evaluating pipeline performance
Obtain and analyze pipeline performance data
Reducing time spent indexing
Reducing time spent expanding archive files
Reducing time spent by non-expansion stages
Pipeline execution modes
When to use Workflow-Agent mode
When to use Preprocessing mode
Setting pipeline execution modes
How workflow recursion settings interact with pipeline execution modes
Deleting pipelines
Index collections
Index collection types and settings
Initial schema options
Adding index collections
Overview tab
Viewing index collections and statistics
HCI Index statistics
Apache Solr statistics
Elasticsearch statistics
HDDS statistics
Index shards
Specifying the shard count for an index
Multiple shards on a single-instance system
Balancing shards across instances
Index protection level for HCI Indexes
Editing Index Protection Level for an index
Configuring external indexes
Index collection schemas
Defined dynamic and copy fields
Field types
Adding and editing fields in an index collection schema
Use cases
Handling multi value fields
Field attributes
Importing fields from a workflow into an index collection
Using Language Select
Directly editing configuration files for HCI Indexes
Querying an index
Query syntax
Deleting an index collection
Unregistering an index collection
Considerations - External Apache Solr index collections
Query settings
Creating and editing query settings
Configuring advanced search options
Query setting configuration options
Configuring index display name
Enabling and disabling query suggestions
Enabling search result download
Configuring fields in query settings
Configuring facets in query settings
Enabling bulk actions on an index
Visualizations
Configuring filtering in query settings
Configuring sortable fields in query settings
Boosting field relevancy
Configuring search result layouts
Configuring access control settings in query settings
Configuring custom parameters in query settings
Filtering queries by user
Enabling and disabling query settings
Deleting query settings
Index Collection Delete-by-query
Controlling user access to search indexes
Granting limited access to a search index
Granting public access to a search index
Enabling multiple query settings for an index collection
Hitachi Content Intelligence fields
SQL queries
Solr index backup and restore
Index backup API
Index restore API
Backup and restore status API
List backups API
Delete backup API
Solr aliases
Creating a Solr alias
Solr aliases Overview tab
Querying an alias
Query syntax for aliases
Query settings for Solr aliases
Controlling user access to Solr alias searches
Adding an index to a Solr alias
Removing an index from a Solr alias
Deleting a Solr alias
Content classes
Creating and editing content classes
Viewing content classes
Testing content classes
Using content classes
Deleting content classes
Best practices
For configuring stages and pipelines
Test your pipelines to learn how each stage affects your documents
Order your stages and pipelines correctly
Identify pipeline performance issues
Use conditional statements to limit unnecessary document processing
Process only the documents that your users want
Use the Reject Documents stage to debug your pipelines
Use Workflow Recursion to simplify your pipelines
Use temporary fields
Follow the best practices for each stage type
For running workflow tasks
Schedule tasks to run when user demand on the system is low
Favor processing large numbers of small documents
Optimize your pipeline before changing task performance settings
Experiment with task performance setting changes
Run the Workflow-Agent jobs on as many instances as possible
Optionally, avoid processing old versions of HCP objects
Avoid or minimize metrics collection in production
Avoid unnecessary file rereads
For designing an index
Scale your index appropriately to ensure availability and fast query responses
For HCI Indexes, increase the Index Protection Level setting to maintain index availability and performance
Use the appropriate initial schema value
Favor query settings changes over index collection schema changes
Minimize field attributes in your index collection schema
Minimize the number of copy fields
Do not use dynamic fields in a production index
Use the smallest field types possible for numerical values
Follow this example procedure to create an index for use in production
Use the Index action in your pipeline to test the effects of pipeline stages on your index
For connecting to HCP
HCP data connection
HCP MQE data connection
S3 Compatible data connection
For end-user search experience
Expose as many searchable fields as is useful
Keep searches fast
Enable extra search features
For writing content classes
Have Hitachi Content Intelligence create content properties for you
Keep content property types in separate content classes
Normalize field names within a content class
Troubleshooting
Workflow task issues
Data connection issues
Pipeline and task performance issues
System event issues
HCP for Cloud Scale Bucket Indexing (Preview Mode)
Creating a bucket index
Editing a bucket index
Deleting a bucket index
Reference
Writing and testing your own plugins
Cloning components
Supported browsers
Accessing other apps
REST API reference
Input and output format
Access and authentication
Viewing and using REST API methods
Error responses
CLI reference
Accessing the CLI tools on a system instance
Accessing the CLI tools from your computer
Syntax
Viewing available commands
Viewing request models
Editing configuration preferences
System error responses
Glossary

Back to home page