Thursday, January 5, 2012

Is Hadoop reinventing the wheel?

In Hadoop 0.23 the resource management (RM) and the application management (AM) have been seperated to make Hadoop more scalable. Though, Hadoop has got enough media attention, looks like there are a couple of alternatives for RM/AM. Here is an interesting mail from the Hadoop groups.

As mentioned earlier Hadoop doesn't solve everything. It's very important to have a good idea of the ecosystem before jumping into an architecture or selecting a framework to meet the requirements.

Although some of the frameworks (like Storm) are independent of Hadoop, most of the new frameworks are dependent on Hadoop for either MapReduce runtime or HDFS file system or implement some of the MR/HDFS API (like MapR) in a different way. Hadoop acts like a kernel (similar to Linux) on which other frameworks run.

So, if you are getting started with Big Data, look for the different alternative solutions and approaches.

No comments:

Post a Comment