We recommend switching to the latest versions of Edge, Firefox, Chrome or Safari. Using Internet Explorer will result in a loss of website functionality.
Our Support systems migrated on Saturday, May 21. We'll automatically forward you to the new location for this content.

Import data from a Tableau TDE file

Comments

9 comments

  • Avatar
    Infogix Support
    Originally posted by: awilliams1024

    Hi Dave,

    the Tableau Data Extract (TDE) file format is Tableau proprietary. Looking online, there have been a number of requests for Tableau to open source the TDE formats so that other applications can read extract files. However, there is a statement on the Tableau community website that says: "The existing TDE API will continue to create TDE extracts but we have no plan to open source it."

    The post mentions that the new Tableau Hyper extract will replace the existing Tableau Data Engine and that "there will be an API for creating and updating Hyper extracts". No details of the API are available at this time.

    Regards,
    Adrian
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: gmullin

    If you can't easily read the TDE file an easy solution would be to write/append the data to a BRD file at the same time you write the TDE file. Then you can easily read back in whatever you had in TDE by using that BRD file.
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: dglavastorm

    Thanks all, curious how Lavastorm can create a TDE but not read one ?

    Also the second issue underlying this is that the data sets I'm working with are massive. Using BRD or CSV the file is over 5GB, but LS compresses this down to 100mb as a TDE. I'm trying desperately to balance the required work with a workable storage solution, and I can't shrink the underlying data any further.

    So I guess a secondary question is, if we can't read from a TDE, is there any other format I can use that heavily compresses the data ? XLSX won't work, the node just doesn't run even if I adjust the JvmMaxHeapSize, and I would prefer it in a single file.

    Data Warehousing sadly is not an option.
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: stonysmith

    Originally posted by: dglavastorm
    					

    Thanks all, curious how Lavastorm can create a TDE but not read one ?
    We use a library of code provided by Tableau - that library can write a TDE, but can't read one.
    In simple terms, Tableau themselves haven't provided a mechanism for reading, and their format is proprietary - they have not yet provided information sufficient for other companies to read the TDE files.
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: dglavastorm

    Makes sense, thanks.

    So are there any formats the LS can write to that offer good compression ? And I don't mean compress to an archive like ZIP, it needs to be readable.

    BRD didn't really compress, it just made reading the data back in faster.
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: gmullin

    I think the answer to your question is no. I tried to write out a BRD file of around 50mb that as TDE was 50kb but any other format I tried the file was of similar size if not larger. The only other option is to compress the file, but you said that's not an option for you. Maybe somebody has a custom node out there? I'd be happy to be proven wrong.
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: dglavastorm

    Thanks, that unfortunately matches everything I've tested as well, including using a Teradata datalab. Whatever Tableau is doing, their compression is sensational.
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: dstacey

    Are there plans to upgrade the Output Tableau node to support the new hyper format? Thanks.
    0
    Comment actions Permalink
  • Avatar
    Infogix Support
    Originally posted by: awilliams1024

    Hi,
    we have no plans to provide support for the Tableau .hyper extract format on the Lavastorm product.
    However, this may be something that could be considered for the Dataverse product.
    0
    Comment actions Permalink

Please sign in to leave a comment.