Set up transformation logging

Pentaho Data Integration

Version
9.3.x
Audience
anonymous
Part Number
MK-95PDIA003-15
Follow the instructions below to create a log table that keeps a history of information associated with your field information.
  1. Ask your system administrator to create a database or table space called PdiLog.
  2. Right-click in the workspace (canvas) where you have an open transformation and select Properties, or press CTRL T .
    The Transformation properties dialog box appears.


    Transformation properties dialog box

  3. In the Transformation properties dialog box, click the Logging tab. In the left-side navigation pane, select which type of logging you want to use.

    Logging tab
  4. In the Logging tab, enter the following information.
    Option Description
    Log Connection

    Specify the database connection you are using for logging. You can configure a new connection by clicking New.

    Log table schema Specify the schema name, if supported by your database.
    Log table name

    Specify the name of the log table.

    Note: If you are also using job logging, use a different table name for transformation logging.
    Logging interval (seconds)

    Specify the interval in which logs are written to the table.

    This property only applies to Transformation and Performance logging types.

    Log record timeout (in days)

    Specify the number of days to keep log entries in the table before they are deleted.

    This property only applies to Transformation and Performance logging types.

    If you find that data in the log table is not deleted as expected, see Log table data is not deleted in Troubleshooting for best practice information.

    Log size limit in lines

    Enter the limit for the number of lines that are stored in the LOG_FIELD. PDI stores logging for the transformation in a long text field (CLOB).

    This property only applies to the Transformation logging type.

  5. Select the fields you want to log in the Fields to log pane, or keep the default selections.
    Note: For effective deletion of expired logs, the LOGDATE and TRANSNAME fields in the Fields to log pane are enabled by default.
  6. Click the SQL button.
    PDI checks the log table.
  7. If the Simple SQL editor opens, go to step 8. Otherwise, proceed to step 10.
  8. Verify your table name, field selections and indices in the Simple SQL editor, and then click Execute to create your log table.
    PDI creates or revises the table and displays the results in the Results dialog box.
  9. Click OK to exit the Results dialog box. Click Close to exit the Simple SQL editor.
  10. Click OK to exit the Transformation properties dialog box.
The next time you run your transformation, logging information will appear under the Execution History tab.
Note: Monitoring the LOG_FIELD field can negatively impact Pentaho Server performance. However, if you do not select all fields, including LOG_FIELD, when configuring transformation logging, you will not see information about this transformation in the Operations Mart logging.