Quantcast
Channel: SCN : All Content - All Communities
Viewing all articles
Browse latest Browse all 8608

HADOOP HDFS Explorer built with HANA XS and SAPUI5

$
0
0

For those of you that have started to explore HADOOP you might be familiarly with HUE (Hadoop User Interface)  or Linux command line for moving files to and from the HADOOP Distributed File system (HDFS).

 

As a bit of a novelty I thought it'd be interesting to build a prototype of an HDFS Explorer, using HANA XS & SAPUI5.

 

 

HUE File Browser                                                            HANA: HADOOP HDFS Explorer                           

 

  

 

 

HUE  is primarily is built using PYTHON with an number of  PYTHON plugins, such as DJANGO and MAKO.

 

My HANA  HDFS Explorer is built using HANA XS,  SAPUI5 (Horizontal Splitter, Tree & Table controls), a sprinkling of custom Javascript & the HADOOP WebHDFS REST API (accessible via xshttpdest).  It's a very rough prototype, and is not indeed reproduce all the same functionality of HUE file browser.

 

NOTE: as standard the webHDFS REST service may run off ports 50070 & 50075 on your HADOOP cluster. I needed to redirect these to 50001 & 50014 respectively to enable xshttpdest to work. For Hana cloud platform the valid ports to use are documented here SAP HANA Cloud Platform

 

 

Here a few screen shots comparing the original and the new improved HANA version

Original HUE:


HANA XS Version (with Download option):

 

Select the file and then click the Download button:

 

 

The dataflow of HANA XS version is:

 

 

The HADOOP webHDFS Rest service makes uses different url parameters  [operations (op)]  for interacting with HDFS.

I only used the following operations in this example:

FILESTATUS          Which acts like 'dir' or 'ls' commands on a specified HDFS directory

OPEN                    Which acts like an ftp 'get' command. 

            

NOTE: OPEN is called twice. 1st on the HADOOP Name node to get the DATA node the file is stored, then secondly on the specified DATA node download the file.

 

The full code is available to download here: https://github.com/AronMacDonald/HanaHdfsExplorer


Viewing all articles
Browse latest Browse all 8608

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>