Spark and hadoop anyone ?

I should have published this a while back - slipped my mind :-)


Spark now supports batch mode on top of Hadoop v2 (Yarn) thanks to the work we did in Yahoo! Labs Bangalore !
The pull requests are here, here, here, here, here and here (to be applied in order). Have been committed to spark master and will be in 0.8 release !
Take it for a spin and try it out on PB of data !!

Sourangshu was involved in this along with me in getting this polished and submitted to spark.