We recommend switching to the latest versions of Edge, Firefox, Chrome or Safari. Using Internet Explorer will result in a loss of website functionality.

Setting resource manager queue

Comments

7 comments

  • Avatar
    Jeffery Brown

    Hi Blair,

      There is an approach that we think can get you there, but let me confirm it first. 

     

    Thanks,

    Jeff

    0
    Comment actions Permalink
  • Avatar
    Jeffery Brown

    Hi Blair,

     After speaking with the team, they confirmed that it is possible change to a different YARN queue, but starting in DQ+ 3.3. To do it, you can add an Execution Property at the Analysis Level:  spark.yarn.queue and set it to the name of the custom queue. You can get to Execution Properties through Edit Stage / Settings (gear in top right) / Execution Property.

    Thanks,

    Jeff

    0
    Comment actions Permalink
  • Avatar
    Rob McCurley

    Jeff, the method you suggest requires creating a setting for each individual analysis created. Is there a way to set spark.yarn.queue to a global default so it doesn't have to be set for each analysis? Perhaps something added in 4.3?

    0
    Comment actions Permalink
  • Avatar
    Jeffery Brown

    Hi Rob, there is not a way to alter the global default for each analysis.  One alternative would be to create an Execution Profile with the default execution property settings that you wan to set, then apply the Execution Profile on a per analysis basis.  This would mean setting this once at each analysis, but after that you can change the default anytime. 

    0
    Comment actions Permalink
  • Avatar
    Jeffery Brown

    Hi Rob... I did create an enhancement for this inquiry.   One idea that I can think of would be to create a Default Execution Setting execution profile that could be applied to each analysis chain (by defualt), then allow the Admin to update that profile as necessary.

    0
    Comment actions Permalink
  • Avatar
    William Williford

    Hi Jeff:  We also are experiencing this issue. Can you give us a time frame (month/year) when this enhancement will be implemented or was this resolved in Analyze 3.6.8?

    0
    Comment actions Permalink
  • Avatar
    Jeffery Brown

    Hi Hal, This enhancement is related to DQ+ and not Analyze.  If you are having issues with this topic in Analyze, please feel free to post here:

    https://support.infogix.com/hc/en-us/community/topics/360001073633-Data360-Analyze

     

    0
    Comment actions Permalink

Please sign in to leave a comment.