Data3Sixty™ Analyze 3.2.7 is Now Available

I'm excited to announce the Data3Sixty™ Analyze 3.2.7 release is now available.

This Generally Available maintenance release provides the following enhancements to the Data3Sixty Analyze application:

• The Modify Fields node - which allows you to include fields, exclude fields, rename and update field metadata without scripting. 
• Enables the use of execution-related variables in user-defined expressions. in the Calculate Fields node.
• Delivers support for Single Sign On (SSO) for centralised user authentication.
• Introduces a range of usability improvements.

The release also introduces a number of operational, performance and stability improvements.

Node Enhancements

Modify Fields Node

The Modify Fields node allows you to modify field metadata. Using a ‘point-and-click’ approach you can:

• Select which input fields are to be included in/ excluded from the output data
• Rename an input field and rename fields using a pattern
• Change an input field’s data type.

 

 

The OutputFields grid displays a list of field metadata when the metadata is available at the node’s input (i.e. it is connected to a node that has run and shows a green output pin).

The default pass through strategy for the output fields is to pass through all input fields to the output. However, you can change the default strategy by clicking on the ‘Input Fields’ button and selecting the required strategy. The icon for the ‘Input Fields’ button indicates the configured default operation:

 - all fields passed through unless explicitly excluded

- all fields are excluded unless they have been explicitly included.

You can change the pass through strategy by clicking on the ‘Input Fields’ button and selecting the required strategy. When a field is excluded, its name is displayed with a strikethrough font and the attributes are greyed out.

 

You can use the checkboxes to select/deselect individual fields to override the current default strategy:

 

To help you identify fields of interest, the input fields list can be filtered by entering the required search text:

 

When blending data from multiple input sources you may want to rename fields to indicate their lineage. The ‘Rename by Pattern’ property can be configured to rename all input fields by adding a prefix or suffix. The new names are displayed in the grid:

 

If required, you can override the name created by the pattern:

 

When the metadata for a field has been explicitly changed you can revert to the original metadata by clicking on the revert button displayed in the grid:

 

A field’s data type can be changed using the Type drop-down menu. The contents of the list displays the available data types, which depend on the field’s current data type:

 

When a field’s data type has been changed, the displayed Type changes to have a blue background. You can click on the revert icon for the field to restore the original data type:

 

When converting from a textual data type (unicode, string) to date/time/datetime data types, the node parses the text string using a default format. Similarly, when changing fields with data types such as date, time and datetime to a textual data type the node uses a default format. When the default format is used, the ‘Format’ attribute displays ‘Auto’. You can use format specifications to control how the node parses the input text string or how the output value will be formatted. You can click on the Format attribute to see the available format specifications, which depend on the input data type:

 

When the node is run it will parse/format the input data using the selected format specification. If required, you can customize the format - for example, to extract just the year or month component of a date by editing the format specification. See the node documentation for further information on the use of the format specifications:

 

When a Double type (floating point) number is converted to an integer or long data type, the value may be rounded. Similarly, when an input field with a  unicode or string data type is output as an integer or long data type and the values are textual representations of a floating point number then rounding will occur. You can specify the method used to round non-integer values using the ‘RoundingMode’ property. The default is Half Even.

 

When converting a string or unicode field to a date data type the node will attempt to automatically determine the format of the date from a sample of the data. However because of regional differences, date values can be ambiguous because the format can be Month First or Day First, e.g.

7/4/2019 -> July 4th 2019 using M/d/yyyy format
7/4/2019 -> April 7th 2019 using d/M/yyyy format

By default, the node will use the locale of the server to determine the format to use. If required, the ‘AmbiguousDateResolutionMode’ property can be set to explicitly configure the date format:

 

You can explicitly set the locale to be used when converting unicode or string fields to other data types. The local must be specified using the following format:

<language code>[_<country code>[_<variant code>]]

• US English           en_US
• British English      en_GB
• Austrian German  de_AT

The node supports the conversion of string representations of numeric values that contain the decimal point and thousands separator symbols, for example:

• Floating point:  1,234.56  (en_US) or  9.876,54 (de_DE)
• Integer:            4,444 (en_US) or 20.000 (de_DE)

The interpretation of the symbols uses the default or defined locale.

 

The node uses a sample of the data when detecting the data type and format of a field. The ‘SampleSize’ property determines the maximum number of input records that are used to analyze the data when converting to date/time and numeric data types. By default the maximum is 1000 records.

You can specify a default value to be used for different output data types when the input value is Null. By default input Null values will be passed through to the output as Null values.

 

A number of properties are provided to allow you to control how processing exceptions will be handled. For instance this enables you to determine how the node will handle exceptions when attempting to convert the field to an a different data type, e.g.:

• double data type (floating point) values of NaN, Infinity, -Infinity to integer/long data type
• Value overflow for long to integer data type
  

You can also configure whether records with conversion errors are written to the main output pin or to the errors pin. The threshold point at which the node will error can be configured in terms of the number of records with errors or as a percentage of the number of input records.

 

Calculate Fields Node

The Calculate Fields node now supports additional built-in variables for expressions where the result depends on the record being processed.

• node.firstExec
  • Boolean object that is True only for the first record
  • Allows variable values to be set when the first record is being processed
• node.lastExec
  • Boolean object that is True only for the last record
  • Enables expressions to derive values when all records have been processed
• node.execCount
  • Provides a count of the current record number
  • Can be used to add a record ID to the output data

 

Application Enhancements

Single Sign On Support

Customers using the Data3Sixty Analyze Enterprise Server edition can now integrate with CA Single Sign On (SiteMinder) to provide centralized user authentication using SSO. See the online Help for further details.

 

Usability Improvements

View Data at Input Pins

You can now view the data at a node’s input pin, a Composite Input or a Composite Output (i.e. when the pin/output is green).

 

Data Viewer Filter Configuration

When creating a Filter in the Data Viewer, the Filter value is now automatically populated with the value of in the selected cell in the Data Viewer.

 

Improved Node Search

When searching for nodes in the Favorites section of the node pallet, the results now include a link to matches found in the All Nodes section.

 

Additional Context Menu Options

Additional right-click menu options are provided for selecting and running nodes.

The context menu displayed when you right-click on a node now includes options enabling you to run/ re-run a node and to clear the status of a node that has been run:

 

When you right click on the canvas with no nodes selected, the context menu now displays the Select All option:

 

API Enhancements

The REST API has been modified to improve user sign-in and authentication. The API documentation has also been improved.

Additionally, enhancements have been made to the API documentation describing how to use the API to run a data flow.

See the API developer documentation topic in the Help for further details.

 

Resolved Issues

The following issues have been resolved in this release:

LAE-9704, LAE-9860, LAE-9868, LAE-10171, LAE-10442

LAE-10530, LAE-10532, LAE-10539, LAE-10667, LAE-10724

LAE-10794, LAE-10827, LAE-10845, LAE-10849, LAE-10853

LAE-10858, LAE-10861, LAE-10862, LAE-10863, LAE-21231

 

Superseded Nodes

The Modify Field Prefix node has been superseded. Equivalent functionality is provided by the new Modify Fields node or the Transform node.

The Modify Field Prefix (Superseded) node remains available in the node library but the new Modify Fields node is recommended for use in current projects where appropriate.