We recommend switching to the latest versions of Edge, Firefox, Chrome or Safari. Using Internet Explorer will result in a loss of website functionality.

Concatenate / Merge multiple Datasets based on dynamic number of inputs

Comments

3 comments

  • Avatar
    Adrian Williams

    Hi John,

    I'm not entirely sure I understand what you mean by "a defined input is empty". If the input has zero records but the metadata is defined then the Cat node should concatenate the remaining inputs with non-zero records.

    Perhaps you could try writing each of the non-zero data sets from your API responses to a set of temporary files in a specified directory and then, once all response files have been written you could use a run dependency to enable a Directory List node to run against the directory. The filename list can then be passed to the relevant input connector node (e.g. CSV/Delimited or JSON Data) which will then concatenate the data from the individual files. When the node has run you could use another run dependency to enable a Transform node that is configured to delete all the listed temporary files.

    0
    Comment actions Permalink
  • Avatar
    John Taylor

    Thanks Adrian,

    I'll consider the best way approach this. Currently if you try and connect a number of JSON nodes to a CAT and one of those JSON nodes are empty (or a meta check doesn't output) then the CAT node doesn't like to run.

    I was hoping there was a direct to do this without having to write out to a file.

    Thanks, John

    0
    Comment actions Permalink
  • Avatar
    John Taylor

    Morning Adrian,

    Just a follow up note on this one. 

    Turns out it was more to do with the JSON node not outputting a value for the cat node to consume rather than the cat node doing anything odd.

    Tweaking the JSON node that is connected to the HTTP response to ignore errors then allowed for an input to the cat node and then setting the the cat type to "Union" allowed me to do exactly what I needed to do.

    Thanks for your help - it got my mind on the right track to solve this one :)

    0
    Comment actions Permalink

Please sign in to leave a comment.