I am currently testing the Data360-app with the aim to migrate my graphs from LAE to Data360.
In general, I am facing serious performance issues in Data360 compared to LAE. Please see below further information on my HW/SW-specifications, and the issues I am having:
- Machine Specifications: Intel Core i7,4core CPU, 16GB internal memory, 1TB SSD Hard-drive (90% Free disk space), Windows10, Google Chrome V. 87.0.4280.141
- LAE version 6.1.40
- Data360 version: Windows desktop version 126.96.36.19958
- Complexity Dataflows: Data flows are relatively simple (no clocks, cartesian products, fuzzy matching etc.) containing mainly data transformation and -aggregation steps.
- I run max. 4 nodes at the same time within a dataflow
- Amount of data to be processed is relatively large: For most of my data flows/graphs the raw input data contains appr. 300mln records.
- Overall a significant more processing time required by D360 compared to LAE (Appr. 20% increase in processing time for dataflows that run from 2 up to 7 hours)
- Where in LAE i was able to perform tasks in other applications, running the flows in D360 consumes ~100% of CPU- and internal storage capacity
- Regardless the size of the dataflow, related datasources the D360-app is very slow (a lot of "loading" waiting time) for (this increases even more when a flow is actually running):
- Starting-up D360
- Browsing and toggling thru the Directory User Interface
- Opening a dataflow
- Browsing and toggling thru a dataflow (e.g. entering and exesting composite nodes)
- Connecting nodes and/or bending connectors
- Toggling thru the Proporties panel
- Opening data in DataViewer
I have gone thru all the materials that can be found in the documentation and community regarding performance issues, but unfortunately i did not find anything that could help me in tackling the issues. Therefore:
- What can I do improve overall performance of D360?
- Is there an option similar to the LAE "Run in agressive mode" within D360 to help improve processing speed?
Thank you in advance for your reply and support.
Please sign in to leave a comment.