Using the Text File Input step on the Spark engine

Pentaho Data Integration

Version
9.3.x
Audience
anonymous
Part Number
MK-95PDIA003-15

You can set up the Text file input step to run on the Spark engine. Spark processes null values differently than the Pentaho engine, so you may need to adjust your transformation to process null values following Spark's processing rules.

Note: If you are using this step to extract data from Amazon Simple Storage Service (S3), browse to the URI of the S3 system or specify the Uri field option in the Additional output fields tab. S3 and S3n are supported.

If you are running your transformation on the Spark engine, use the following instructions to set up the Text File Input step.