Union of list of tables

Follow

Holger Schimanski

• August 29, 2018 12:57

I have a config file with a list of tables. These tables should be merged and written into one table.

The Cat node does the job of merging resp. the different tables.

But I have up to 100 tables for the Cat union and the list of tables needs to be a config file or parameter rather that individual Input nodes for each table and adding the new Input node to the workflow when adding a table.

I guess I need some Do While node to iterate over the list of tables from the config file, but I don't know, how to connect it with the Cat node.

Any ideas how to solve this problem?

Best regards
Holger

0

• 

  Gerry Mullin

  • September 06, 2018 15:27

  It sounds like a Do While would work and you could keep on appending your logic to a brd file that you finally read when the loop has completed.

  I'm attaching an example of how I would configure a Do While that would cycle through a list of 5 tables.

   

  Attached files

  Loop - 6 Sept 2018.lna

   

  0

  Comment actions Permalink
• 

  Holger Schimanski

  • September 07, 2018 08:54

  Thank you very much for providing a demo workflow! :-)

  I notice, that you are using the Output BRD File node and not the CAT node.

  But my input tables have slightly different column structures.

  CAT has ConcatenationMode = Union ("Outputs all of the fields across the inputs, filling in nulls for fields not present in a given input"), but Output BRD File with Append = True ("If true and the output file exists, the field names and types of the existing file and the input stream are compared to make sure they are identical.") does not accept different columns.

  Any ideas?

  Best regards
  Holger

  0

  Comment actions Permalink
• 

  Gerry Mullin

  • September 07, 2018 12:53

  Rather than using Append, I added reading back in the BRD file and then using Cat (union) to join back up to the current iteration. Then we will overwrite a new BRD file each time. That will get you past that error.

   

  Attached files

  Loop with Cat (Union) - 7 Sept 2018.lna

   

  0

  Comment actions Permalink

Please sign in to leave a comment.