If you notice that your temp files for a particular dataflow have increased dramatically in size, it's likely that you're using Merge node.
Data volumes can explode on a merge node depending on the use case.
For example, If you have 1000 records on the left input and 1000 records on the right input, and you specify the merge keys on that node as some field K on both, where K is a constant value, then you have 1000*1000=1M records on the match output. Therefore, the size can blow up pretty quickly depending on what you are doing.
Comments
0 comments
Please sign in to leave a comment.