Data360® Analyze enhancements in this Long Term Support Maintenance release include the following changes.
Note: the items listed below comprises the changes implemented in the Generally Available 3.8.0 release and the internal-only 3.7.x releases.
Azure Data Lake Storage List Node
The new Azure Data Lake List node enables you to list files stored in Azure Data Lake Storage Gen2.
Azure Data Lake Storage Get Node
The new Azure Data Lake Get node enables you to download files/objects stored in Azure Data Lake Storage Gen2.
Azure Data Lake Storage Put Node
The new Azure Data Lake Put node enables you to upload files/objects to Azure Data Lake Storage Gen2.
Azure Data Lake Storage Delete Node
The new Azure Data Lake Delete node enables you to delete files/objects stored in Azure Data Lake Storage Gen2.
Multiparty Request Generator Node
The new Multiparty Request Generator node enables you to prepare a valid multi-part body, which can then be passed to the HTTP node for processing.
Azure Blob List Node
The Azure Blob List node has been updated to return additional metadata.
CSV/Delimited Input Node
You can now specify how the node handles duplicate input field names by specifying the ‘DuplicateFieldNameBehavior’ property.
The ‘MissingFieldBehavior’ and ‘AdditionalFieldBehavior’ properties allow you to specify how the node handles missing fields or additional fields in the input file data.
The ‘MissingFieldNameBehavior’ property allows you to specify how the node handles a missing field name.
You can now specify multiple characters for the field delimiter. Similarly, Record delimiters can now have up to two characters.
You can now specify whether the node should substitute invalid characters from the input file data that would be used in the record metadata.
The node now provides the ‘HeaderMode’ property. The property enables you specify whether the input file has: No header; Untyped header; Typed header.
The node now provides the ‘DateFormat’, ‘TimeFormat’ and ‘DateTimeFormat’ properties. These properties enable you to specify the format of the data that is to be assumed when the node attempts to parse input fields with the specified type.
The node now provides the ‘NullMode’ property. The property enables you to specify what data values are to be interpreted as Null values.
The node now provides the ‘MaxCharsPerColumn’ and ‘MaxColumns’ properties. These properties enable you to limit the amount of data that is imported from the source files and are useful when handling files that contain a very large amount of data in a single field value or contain many columns that are not relevant to your analysis.
The new ‘CommentCharacter’ property allows you to specify the character that identifies comment lines in the input data.
S3 List Node
The S3 List node has been updated to return any user-defined metadata as a JSON blob.
Database Metadata Node
The Database Metadata node has been changed to support a comma-separated list of patterns.
Database Profiler Node
The Database Profiler node now allows you to choose whether to enable the default set of Logical Types, which are based on the locale.
JDBC Node Libraries
The JDBC Query, JDBC Execute and JDBC Store nodes now provide the DbDriverNativeLibraries and DbDriverNativeLibraryDirectories properties to enable you to specify the DB driver native libraries to load and where to load them from.
Java Node Libraries
The Java node now provides the NativeLibraryPreloads and NativeLibraryDirectories properties to enable you to specify the native libraries (.so, .dlls) to load and where to load them from.
Publish to DQ+ Node (Experimental)
The Publish to DQ+ node now provides the ‘InvalidCharactersInFieldBehavior’.This property defines the action to take when Data360 Analyze encounters a field format that is not supported by Data360 DQ+.
Pivot Data to Names Node
A new Pivot Data to Names node is available in the node library. The new node has been updated to not depend on Superseded nodes and allows you to pivot on non-string values.
Full Merge Node
The new Full Merge node is a replacement for the functionality provided by the Deep X-Ref node. The Full Merge node is not dependent on Superseded nodes.
X-Ref Metadata Node
The X-Ref Metadata node has been updated to not depend on Superseded nodes.
K-Means Advisor Node
A new K-Means Advisor node is available in the node library. The new node has been updated to not use the Bar Chart Engine library node.
JSON Data Node and XML Data Node
The JSON Data node and XML Data node now provide the ‘ExcludeFieldPaths’ property. You can specify field paths within the input data which should not be written to the outputs.
Previously, if the value of a field was always null or empty, it would never appear in the output of the XML or JSON nodes. An 'OutputAllNullFields' property has been introduced on the nodes, to determine whether or not such fields should be produced. The default is to output the fields. See the release notes for details of how to maintain system-wide compatibility with the previous operation of the nodes.
The DeepSQL node now supports lineage generation for Google BigQuery data.
Design-time Performance Improvements
A range of changes have been made to improve the user experience when designing and building data flows.
Users with the administrative role can now view and unlock documents.
You can now insert a text substitution property reference to any run property defined on the parent run property set.
Running Node Duration
When a node is running you can now see how long it has been running in the node’s properties panel.
The version of Tomcat that is used by the application has been upgraded to 9.0.44.
Third-party Library Updates
A number of third-party libraries have been updated to benefit from the latest security updates.
The version of Python that is used by the application has been upgraded to 2.7.18.
The OpenSSL library has been updated to 1.1.1k.
Status of Backup Progress in the Logs
Additional logging has been introduced to the audit log, which shows the progress of a system back up. The new audit codes are: “backupWaiting”, “backupStarted”, and “backupFinished”.
Third-Party License Information
Third-party license information is now available in the About Data360 Analyze dialog.
The version of Postgres used by Analyze has been updated to version 13.3.
Note: Due to this change, upgrades to this release will require more disk space than normal. The additional space required is 1.5 times that of the current Postgres database size.
The Data360 Analyze command line interface (AnalyzeCli) tool is now available.
Changes to Supported Platforms
Windows Server Core 2019 is now a supported platform
Windows Server 2012 R2 64-bit is no longer a supported platform
Red Hat Enterprise Linux 8 is now a supported platform
Red Hat Enterprise Linux 6 is no longer a supported platform
SUSE Linux Enterprise Server 12 is no longer a supported platform.
For Windows Server installations you must now also install Visual C++ Redistributable Packages for Visual Studio 2019.
Installer Check for Required .dll Files
The Windows version of the Data360 Analyze installer now checks for the required .dll files.
QVD File Node
The QVD File node has been deprecated in this release.
The following Spark nodes are no longer supported and have been removed from the product:
Spark Agg, Spark Data Viewer, Spark Download, Spark End Session, Spark Filter, Spark Java Submit, Spark Join, Spark Sample, Spark Sort, Spark SQL Query, Spark Start Session, Spark Upload.
The XMLpy Node is no longer supported and has been removed from the product.
SharePoint 2010 Nodes
The following deprecated nodes are no longer supported and have been removed from the product:
Download for Sharepoint 2010; Folder List for Sharepoint 2010; Upload for Sharepoint 2010.
The following issues have been resolved in this release
LAE-8880, LAE-9613, LAE-9851, LAE-9988, LAE-10423
LAE-10846, LAE-22936, LAE-24352, LAE-23575, LAE-23865
LAE-23882, LAE-24521, LAE-24646, LAE-24692, LAE-24911
LAE-24956, LAE-24964, LAE-24989, LAE-25185, LAE-25277
LAE-25280, LAE-25468, LAE-25538, LAE-25550, LAE-25635
LAE-25678, LAE-25815, LAE-25816, LAE-25863, LAE-25902
LAE-25909, LAE-25911, LAE-25932, LAE-26026, LAE-26055
LAE-26240, LAE-26246, LAE-26287, LAE-26311, LAE-26382
See the release notes for details of the resolved issues: