Select Node is taking Huge amount of time to get executed
We have a Dataflow that will insert data from one table to another. In doing so we have been doing Lookup and then based on the output of the lookup we are passing a parameter to 'JDBC execute' node. And based on that Parameter, we are forming insert into select query in the JDBC execute Node.
When we try and run all the Data-Flow, it's taking a huge amount of time at that last insert step. and we are wondering how can we get around that?
We also wanted to understand, why it's happening? Is it because for each parameter value that JDBC execute is running and running for ** no of times is taking a huge amount of time? Or is it a limitation or issue w.r.t to Data3Sixty?
Because if we run the same insert into select without any parameter value it's getting executed in less than a minute on SQL Server Management Studio.
-
Have you tried using JDBC Store node in place of the JDBC Execute? On the Option tab you can set the LoadMethod to Optimized with a CommitFrequency of 1000. I've seen dramatic increases speed rather than using Insert doing this.
Please sign in to leave a comment.
Comments
2 comments