Data analyzer execution error

Follow

RTountcheva

• January 29, 2018 18:51

Hello - I am completely stumped by the below error. Help would be much appreciated!

 

0

• 

  RTountcheva

  • January 29, 2018 19:03

  I pulled in a brand new analyzer node, connected it, and ran without changing any configurations and got the same error:

   

  

  0

  Comment actions Permalink
• 

  Adrian Williams

  • January 31, 2018 09:52

  Hi Rodi,

  The issue is the data contains a 'date' type field and one or more values are Null (i.e. None in Python). This is a known issue with the Data Analyzer node that only occurs when the input field type is 'date' - i.e. it does not happen when the data type is a string.

  The simplest way around this, assuming you are just using the Data Analyzer node to change the data types rather than using the node's histogram functionality, is to use the Data Converter node rather than the Data Analyzer node. The Data Converter node can be found in the Aggregation and Transformation category of the node library.

  If you really need to use the Data Analyzer for some reason then you could modify the input data by inserting a Transform node and configuring it to change the affected field to an empty string so that the Data Analyzer can handle the missing values. The following example assumes the affected field name is "dt":

   

  ##ConfigureFields property

  #Configure all fields from input 'in1' to be mapped
  #to the corresponding fields on the output 'out1'
  outputs[0] += inputs[0]

  ## Remove the input date type field
  outputs[0] -= inputs[0]['dt']

  ## Recreate the field as a string type field
  outputs[0]['dt'] = str

   

  ##ProcessRecords

  #Copy all fields from input 'in1' to the corresponding output fields
  #in output 'out1'. Copies the values of the fields that have been setup
  #in the mapping defined in the ConfigureFields property
  outputs[0] += inputs[0]

  ## Output the date type field as a string and replace Null with an empty string
  outputs[0]['dt'] = str(inputs[0]['dt']).replace('Null','')

   

  0

  Comment actions Permalink
• 

  RTountcheva

  • January 31, 2018 14:29

  Ok, that is helpful to know. Is this known issue in the pipeline to be resolved? Thanks.

  0

  Comment actions Permalink
• 

  Adrian Williams

  • February 01, 2018 11:46

  A fix to this has not been committed to a release at this time.  Where practicable, the Data Converter should be used and this node also provides improved performance when processing larger data sets.

  0

  Comment actions Permalink

Please sign in to leave a comment.