When processing data it is often necessary to obtain and use today's date as a reference point for selecting or filtering data. This article describes some ways in which you can generate a value for the current date when the data flow is run.
Using the Calculate Fields Node
In this example the Calculate Fields node is used to create two fields containing the value of today's date.
The 'Today_String' field is defined as a unicode string type. The expression uses Run Property Substitution to evaluate the RunDate run property and this string representation of the current run date is assigned as the value of the Today_String field. The format of the resulting string is YYYY-MM-DD.
The 'Today_Date' field is a date type. In this case the expression uses the Python strptime() function to parse the date string from the substitution of the RunDate run property. The strptime() function returns a datetime value rather than a date value so the date() method is used to exract the date component of the datetime value. The date value is assigned as the value of the Today_Date field.
Using the Generate Data Node
The Generate Data node is used to generate records at the start of a sub-data flow (note, the Transform node can be configured in a similar way to the following example and can be used when the date field needs to be added to an existing data set).
The metadata for the output fields is defined in the ConfigureFields script. As with the previous example that used the Create Data node, the node defines a unicode type field ('Today_String') and a date type field ('Today_Date').
The ProcessRecords script uses Run Property Substitution in the expressions to obtain the current run date value. The unicode and date type values are output per the previous example.
You can also use the Modify Fields node to convert an existing field that contains a string representation of a date value into a date type. In the example below, the 'Today_String' value is parsed using the defined format. The date value is output in the (renamed) 'Today_Date_2' field.
As the RunDate run property was introduced inthe v.3.5.1 release, v.3.5.1 or above is required to run the attached example. Users of earlier releases may want to substitute the (now deprecated) CurrentDate run property for the RunDate run property.
Comments
0 comments
Please sign in to leave a comment.