Data360 Analyze - How to check if a field exists and add it if it doesn't

Scenario: You want to check if a field exists in a data set, and if it doesn't, the field should be added to the data set with a specified default value.

Solution: This can be achieved using some Python scripting in a Transform node, as follows:

The ConfigureFields script uses the fields function to access the input metadata for the Transform node. This is converted to a Python dictionary using todict() and the keys for the dictionary are extracted using keys(). This results in a list object containing the names of the input fields.

The script then defines the name of the field that is required to be present in the input data, together with the data type for the field and the default value to be used if the field does not exist.

The contents of the FieldList are then checked to determine whether the field exists. If it is present, the 'foundField' variable is set to True. If it isn't present the variable is set to False and the required field metadata is added to the Transform node's output metadata.

The ProcessRecords script outputs the field values from the input record and then, if the required field was not present in the input data, it outputs the default value for field.

Please see the attached example data flow.