We recommend switching to the latest versions of Edge, Firefox, Chrome or Safari. Using Internet Explorer will result in a loss of website functionality.

HDFS Input Data Stores: Regular Expression for File Path?

Comments

3 comments

  • Avatar
    Brenda Stoetzer

    Hi Rob,

    Can you try this expression for File Path .bucket_.*

    0
    Comment actions Permalink
  • Avatar
    Rob McCurley

    Brenda, the you've worded it you seem to suggest I try this:

    But that doesn't work ("Incomplete HDFS URI"). But I think you're suggesting to try this:

    That doesn't throw an error like the first attempt,but doesn't retrieve any data from the directory, even though the editor regex parser confirms the expression should match "/bucket_00000", which is a non-empty file in the directory. Are the "Path:" and "Regular Expression for File Path:" concatenated to determine access? Or should "Path:" be empty with the other parameter is used?  

     

    0
    Comment actions Permalink
  • Avatar
    Brenda Stoetzer

    Hi Rob,

    Your second approach seems to be correct with the regular expression as .*bucket_.* If the file is a text file then filter extension should be mentioned as .txt.

    Regarding your question if the patch should be empty...No it should not be empty. It can have full path or even the root directory (in this case /tmp/some_directory) if it is unique.

     

     

    0
    Comment actions Permalink

Please sign in to leave a comment.