| Option | Description |
|---|---|
| Hadoop Cluster |
Specify which Hadoop cluster configuration to use. The environment must match the Spark cluster. You can specify information like host names and ports for HDFS, Job Tracker, and other big data cluster components through the Hadoop Cluster configuration dialog box. Click Edit to edit an existing cluster configuration in the dialog box, or click New to create a new configuration with the dialog box. Once created, Hadoop cluster configurations settings can be reused by other transformation steps and job entries. See Connecting to a Hadoop cluster with the PDI client for more details on the configuration settings. |
| Folder/File | Specify the location and/or name of the output text file written to the Hadoop Cluster. Click Browse to navigate to the source file or folder in the VFS browser. |
| Create Parent Folder | Indicate if a parent folder should be created for the output text file. |
| Do not create file at start | This field is either not used by the Spark engine or not implemented for Spark on AEL. |
| Accept file name from field? | This field is either not used by the Spark engine or not implemented for Spark on AEL. |
| File name field | This field is either not used by the Spark engine or not implemented for Spark on AEL. |
| Extension | Adds the .csv extension to the end of the file name. |
| Include stepnr in filename | This field is either not used by the Spark engine or not implemented for Spark on AEL. |
| Include partition nr in file name? | This field is either not used by the Spark engine or not implemented for Spark on AEL. |
| Include date in file name | Include the system date in the filename (_20181231 for example). |
| Include time in file name | Include the system time in the filename (_235959 for example). |
| Specify Date time format | Indicate if you want to specify the date time format from the list in the Date time format drop-down list. |
| Date time format | Specify date time formats. |
| Show file name(s) | Display a list of the files generated. The list is a simulation and depends on the number of rows that go into each file. |
| Add filenames to result | This field is either not used by the Spark engine or not implemented for Spark on AEL. |