Error on look up node - Maximum Buffer Size

Comments

3 comments

  • Avatar
    Stony Smith

    My suggestion is to do what I refer to as a "reverse lookup".

    Add another lookup in front of the one you have.. switch the inputs on it. put the 1.7m records on top and the 5 records on the bottom. 

    The code will be simply:

    emit 1:*
    where matchIsFound

    This way, only the records that could possibly match from pin2 will be fed to your existing lookup.

    There is a place where you can change the buffer setting, but you may fare better with trying the trick above.

    0
    Comment actions Permalink
  • Avatar
    Tim Segall

    Martin,

    As per the documentation https://preview-desktop.lavastorm.com/#e-node-help/Correlation/lookup.html?Highlight=lookup this node 'Enriches a large data set (left table) with a small data set (right table) by joining the two data sets based on common fields.'

    This node is meant for one input that is large and one that is small (e.g. a lookup table).  Given you have two large data sets you will need to use the Join or X-Ref node.

    Cheers, Tim.

    0
    Comment actions Permalink
  • Avatar
    Martin Hamilton

    Thanks Both

    i have run both options provided and each works for me.

    cheers

    Martin 

    0
    Comment actions Permalink

Please sign in to leave a comment.



Powered by Zendesk