Key Features of the HDFS Reader/Writer Snaps
Enterprise IT organizations and developers are now able to process and store massive volumes and varieties of data, which is then ending up in big data lakes, hubs, and repositories. Feeding, reading and analyzing large amounts of unstructured, complex, or social data can prove challenging for these organizations. The HDFS Reader/Writer Snaps are part of the Hadoop Snap Pack.
With the HDFS Snaps, organizations can now tap into their Big Data to gain contextual business insights and make appropriate decisions to grow their business.
The HDFS Snaps supports read and write capabilities on files delimited by single characters or arbitrary strings on Hadoop DFS file system. The kerberized HDFS Snap provides extended support for Groundplex by allowing users to push data into HDFS and run reports and analytics in the HDFS environment. Additional capabilities include:
- Delimiters that are arbitrary strings
- If a field in the source being read has no data, the HDFS Snap produces an empty string for a corresponding output field of STRING type, or a null value for an output field of NUMBER type
- If a field in the source being read is missing, the HDFS Snap produces a null value
- If an input record field has no data, the HDFS Snap outputs nothing for the field
- The HDFS Snap terminates each record it writes with an ‘\n’. It only writes “\n” on all platforms. It does not make OS specific interpretation of ‘\n’ (such as writing ‘\r\n’ on Windows)
- User impersonation for non-kerberized clusters
Learn more about how Snaps work with the SnapLogic Intelligent Integration Platform here.
