Select Node is taking Huge amount of time to get executed

Follow

Yashvanth Cheedella

• November 12, 2019 12:48

We have a Dataflow that will insert data from one table to another. In doing so we have been doing Lookup and then based on the output of the lookup we are passing a parameter to 'JDBC execute' node. And based on that Parameter, we are forming insert into select query in the JDBC execute Node. 

When we try and run all the Data-Flow, it's taking a huge amount of time at that last insert step. and we are wondering how can we get around that?

We also wanted to understand, why it's happening? Is it because for each parameter value that JDBC execute is running and running for ** no of times is taking a huge amount of time? Or is it a limitation or issue w.r.t to Data3Sixty? 

Because if we run the same insert into select without any parameter value it's getting executed in less than a minute on SQL Server Management Studio.

 

 

0

• 

  Gerry Mullin

  • November 12, 2019 22:41

  Have you tried using JDBC Store node in place of the JDBC Execute? On the Option tab you can set the LoadMethod to Optimized with a CommitFrequency of 1000. I've seen dramatic increases speed rather than using Insert doing this.

  https://d3sa-preview.infogixsaas.com/docs/dist/help/Default.htm#e-node-help/Publishing/jdbc-store.html

  0

  Comment actions Permalink
• 

  Yashvanth Cheedella

  • November 13, 2019 03:43

  Hey Gerry, Thanks for the response. I am not able to use JDBC Store node because i have an identity column in my table and JDBC Store node expects value for each column in that table. 

  0

  Comment actions Permalink

Please sign in to leave a comment.