Data Streaming Input sources
Hi Everyone,
Please let me know if D3S Analyze supports real-time data streaming (Apache Kafka) as an input source or do we have to store the real-time data into HDFS or hadoop and then source data using one of the in-built nodes for HDFS within Analyze to source data?
Best,
Rakshit
-
Data3Sixty Analyze does not currently provide a pre-built input connector for real-time data streaming sources such as Apache Kafka. However you may be able to leverage the Kafka Java client as part of a custom node using Analyze's Java node, or use the Kafka Python client in conjunction with Analyze's Python node.
If you have a Kafka HDFS Sink Connector that is able to export the contents of a Kafka topic to an HDFS file then an Analyze HDFS Download node could be used to read the contents of the file.
-
The Data360 DQ+ product supports streaming data using Kafka. See the DQ+ documentation for further details:
https://doc.infogixsaas.com/dqplus/Default.htm#h-nodes/nodes-streaming.htm
Please sign in to leave a comment.
Comments
2 comments