Hi Big Elephant Lovers,
The Hue Team is glad to release Hue 3.5!
This new version comes with many improvements (254!), for example:
Hue 3.5 and more will be available early next year in CDH5beta2. If you feel adventurous download the tarball!
In Hue 3.5, a new assistant was added to the Pig Editor: Navigator.
Sorted by category
Auto-completable (as well as HDFS paths and Metastore tables)
So now, get started with Apache Pig!
With HUE-1476, users can submit Oozie jobs directly from HDFS. Just upload your configuration or browse an existing workspace and select a workflow, coordinator or bundle. A submit button will appear and let you execute the job in one click!
File Browser supports:
Parameters from workflow.xml, coordinator.xml, bundle.xml
Parameters from job.properties
Oozie Dashboard supports:
Dynamic progress and log report
One click MapReduce log access
Stop, Pause, Rerun buttons
Here is the workflow tutorial used in the video demo.
Of course, the Oozie Editor is still recommended if you want to avoid any XML :)
Hue makes it easy to create Hive tables.
With HUE-1746, Hue guesses the columns names and types (int, string, float…) directly by looking at your data. If your data starts with a header, this one will automatically be used and skipped while creating the table.
Quoted CSV fields are also compatible thanks to HUE-1747.
Here is the data file used:
This is the SerDe for reading quoted CSV:
And the command to switch the SerDe used by the table:
ALTER TABLE banks SET SERDE 'com.bizo.hive.serde.csv.CSVSerde'
The app is not totally new: it consists of a rebasing from Hue 1 to Hue 3 of the ZooKeeper UI made by Andrei during his Google Summer of Code 3 years ago.
The main two features are:
Listing of the ZooKeeper cluster stats and clients
Browsing and editing of the ZNode hierarchy
ZooKeeper Browser requires the ZooKeeper REST service to be running. Here is how to set it up:
First get and build ZooKeeper:
git clone https://github.com/apache/zookeeper cd zookeeper ant Buildfile: /home/hue/Development/zookeeper/build.xml init: [mkdir] Created dir: /home/hue/Development/zookeeper/build/classes [mkdir] Created dir: /home/hue/Development/zookeeper/build/lib [mkdir] Created dir: /home/hue/Development/zookeeper/build/package/lib [mkdir] Created dir: /home/hue/Development/zookeeper/build/test/lib …
Then start the REST service:
cd src/contrib/rest nohup ant run&
If ZooKeeper and the REST service are not on the same machine as Hue, please update the Hue settings and specify the correct hostnames and ports:
[zookeeper] [[clusters]] [[[default]]] # Zookeeper ensemble. Comma separated list of Host/Port. # e.g. localhost:2181,localhost:2182,localhost:2183 ## host_ports=localhost:2181 # The URL of the REST contrib service ## rest_url=http://localhost:9998
And that’s it, jump up to ZooKeeper Browser!
Hue usage and community is growing tremendously and development has been particularly active, culminating with a big Hue 3. This led to a good timing for having the Hue team go celebrate and scout for some exotic inspiration!
After some tough debates, Caribbean and South America were eliminated (for next time!) and the team flew around the globe and landed at Bangkok, shortly followed by the Elephant Island: Ko Chang!
There, the team immersed into the Thai culture and its welcoming inhabitants. A hard and refreshing week followed:
Thai food, sun, sunsets…
Hadoop elephants were also spotted.
In Thailand, a brand new application that enables viewing data in MySQL or PostgreSQL has been committed.
Inspired from the Beeswax application, it allows you to query a relational database and view it in a table.
For example, let’s integrate Tableau:
To create a new app:
build/env/bin/hue create_proxy_app my_hue http://gethue.com tools/app_reg/app_reg.py --install my_hue --relative-paths
If you want to update the url later, change it in the ini:
In this final episode (previous one was about Search) of the season 2 of the Hadoop Tutorial series let’s see how simple it becomes to export our Yelp results into a MySql table!
Sqoop2 currently only Comma Separated Values files. Moreover, Sqoop2 currently require on export for String constants to be enclosed in single quotes.
Then, as detailed in the video we specify an export job, set the input path as the output of our previous Pig job. The data is in on HDFS and the path can either be a single file or a directory.
We previously created a MySql table ‘stats’ with this SQL script. This table is going to store the exported data.
Here are the properties of our job. They are explained in more depth in the previous Sqoop2 App blog post.
Table name: yelp_cool_test Input directory: /user/hdfs/test_sqoop Connector: mysql JDBC Driver Class : com.mysql.jdbc.Driver JDBC Connection String: jdbc:mysql://hue.com/test
Then click ‘Save & Execute’, and here we go, the data is now available in MySql!
mysql> select * from yelp_cool_test limit 2; +------+------+------+------+ | a | b | c | d | +------+------+------+------+ | 1 | 2 | 3 | 4 | | 2 | 3 | 4 | 5 | +------+------+------+------+ 2 rows in set (0.00 sec)
Data stored in Hive or HBase can not be sqooped natively yet by Sqoop2. A current (less efficient) workaround would be to dump it to a HDFS directory with Hive or Pig and then do a similar Sqoop export.
Thank you for watching this season 2, stay stuned, season 3 is approaching!
comments powered by Disqus