Within DQ+ there is a log limit that may be set within the "analysis settings". If the number of logs generated exceeds the limit, the Analysis will be terminated :
If the limit is reached, the following message will appear within the logs / UI :
com.infogix.component.executionengine.export.spark.SparkLogLimitExceededException: Log limit exceeded.
This generally occurs when there is a record-level issue and each record is processed, hitting an ERROR. For example, inserting a duplicate record into the data store. Each record that has an issue will produce an ERROR up to the "Log Limit" setting. After which, the analysis will stop processing and mark itself as failed.
Comments
0 comments
Please sign in to leave a comment.