Hi Spark Makers!
We previously released the app with an Oozie submission backend but switched to the Spark Job Server (SPARK-818) contributed by Ooyala and Evan’s team at the last Spark Summit. This new server will enable a real interactivity with Spark and is closer to the community.
We hope to work with the community and have support for Python, Java, direct script submission without compiling/uploading and other improvements in the future!
We assume you have Scala installed on your system.
Currently on github on this branch:
git clone https://github.com/ooyala/incubator-spark.git spark-server cd spark-server git checkout -b jobserver-preview-2013-12 origin/jobserver-preview-2013-12
sbt/sbt project jobserver re-start
Currently only on github (will be in CDH5b2):
If Hue and Spark Job Server are not on the same machine update the hue.ini property in desktop/conf/pseudo-distributed.ini:
[spark] # URL of the Spark Job Server. server_url=http://localhost:8090/
Then follow this walk-through and create the example jar that is used in the video demo.
comments powered by Disqus