Tuesday, August 06, 2013

Spark and hadoop anyone ?

I should have published this a while back - slipped my mind :-)

Spark now supports batch mode on top of Hadoop v2 (Yarn) thanks to the work we did in Yahoo! Labs Bangalore !
The pull requests are here, here, here, here, here and here (to be applied in order). Have been committed to spark master and will be in 0.8 release !
Take it for a spin and try it out on PB of data !!

Sourangshu was involved in this along with me in getting this polished and submitted to spark.


Anonymous Anonymous said...

Hey , heard you are in Bay area, any chance we can catch up on that Chess engine you have. As mentioned earlier I am ready to give top dollar for it.

- Daniel Shroff

3/21/2015 09:47:00 AM  

Post a Comment

Links to this post:

Create a Link

<< Home