'Next Generation MR' or 'NextGen MR' or 'MRv2' or 'MR2' is a major revamp of the MapReduce engine and will part of the 0.23 release. MRv1 or the old MapReduce engine will be not be supported in 0.23 release. The underlying engine has been revamped in 0.23, but the API to interface with the engine remains the same. So, the existing MapReduce code for MRv1 engine should run without modifications on MRv2.
The architecture, information for building and running MRv2 is spread across and this blog entry will try to consolidate and present all the information available on MRv2. I will keep-on updating this blog entry as I get more information about MRv2, instead of creating a new one. So, bookmark this and check it often :).
MRv1 vs MRv2
http://www.ibm.com/developerworks/library/bd-hadoopyarn/index.html
Current Status
http://hortonworks.com/blog/apache-hadoop-yarn-background-and-an-overview/ - 9th August, 2012
http://www.hortonworks.com/update-on-apache-hadoop-0-23/ - 27th September, 2011
http://www.cloudera.com/blog/2011/11/apache-hadoop-0-23-0-has-been-released/ - 15th November, 2011
http://hortonworks.com/apache-hadoop-is-here/ - 16th November, 2011
Home Page
http://hadoop.apache.org/common/docs/r0.23.0/
Architecture
MRv2 Introduction
The Hadoop Map-Reduce Capacity Scheduler
The Next Generation of Apache Hadoop MapReduce
Next Generation of Apache Hadoop MapReduce – The Scheduler
Detailed document on MRv2
HortonWorks Series
http://hortonworks.com/blog/apache-hadoop-yarn-background-and-an-overview/
http://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/
http://hortonworks.com/blog/apache-hadoop-yarn-resourcemanager/
http://hortonworks.com/blog/apache-hadoop-yarn-nodemanager/
Developing applications on YARN
http://continuuity.github.io/weave/
Applications ported to Yarn
A lot of applications are being migrated to YARN.
http://wiki.apache.org/hadoop/PoweredByYarn
Presentation
Quick view of MRv2
JIRAs
https://issues.apache.org/jira/browse/MAPREDUCE-279
Books
3rd edition of the Hadoop : The Definitive Guide due on April 30th, 2012 has a chapter on MRv2.
Videos
Next Generation Hadoop MapReduce by Arun C. Murthy
Code
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23/
Building from code and running a sample
On a single node - http://www.thecloudavenue.com/2012/01/getting-started-with-nextgen-mapreduce.html
On a cluster - http://www.thecloudavenue.com/2012/01/getting-started-with-nextgen-mapreduce_11.html
http://www.cloudera.com/blog/2011/11/building-and-deploying-mr2/
http://www.crobak.org/2011/12/getting-started-with-apache-hadoop-0-23-0/
The architecture, information for building and running MRv2 is spread across and this blog entry will try to consolidate and present all the information available on MRv2. I will keep-on updating this blog entry as I get more information about MRv2, instead of creating a new one. So, bookmark this and check it often :).
MRv1 vs MRv2
http://www.ibm.com/developerworks/library/bd-hadoopyarn/index.html
Current Status
http://hortonworks.com/blog/apache-hadoop-yarn-background-and-an-overview/ - 9th August, 2012
http://www.hortonworks.com/update-on-apache-hadoop-0-23/ - 27th September, 2011
http://www.cloudera.com/blog/2011/11/apache-hadoop-0-23-0-has-been-released/ - 15th November, 2011
http://hortonworks.com/apache-hadoop-is-here/ - 16th November, 2011
Home Page
http://hadoop.apache.org/common/docs/r0.23.0/
Architecture
MRv2 Introduction
The Hadoop Map-Reduce Capacity Scheduler
The Next Generation of Apache Hadoop MapReduce
Next Generation of Apache Hadoop MapReduce – The Scheduler
Detailed document on MRv2
HortonWorks Series
http://hortonworks.com/blog/apache-hadoop-yarn-background-and-an-overview/
http://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/
http://hortonworks.com/blog/apache-hadoop-yarn-resourcemanager/
http://hortonworks.com/blog/apache-hadoop-yarn-nodemanager/
Developing applications on YARN
http://continuuity.github.io/weave/
Applications ported to Yarn
A lot of applications are being migrated to YARN.
http://wiki.apache.org/hadoop/PoweredByYarn
Presentation
Quick view of MRv2
JIRAs
https://issues.apache.org/jira/browse/MAPREDUCE-279
Books
3rd edition of the Hadoop : The Definitive Guide due on April 30th, 2012 has a chapter on MRv2.
Videos
Next Generation Hadoop MapReduce by Arun C. Murthy
Code
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23/
Building from code and running a sample
On a single node - http://www.thecloudavenue.com/2012/01/getting-started-with-nextgen-mapreduce.html
On a cluster - http://www.thecloudavenue.com/2012/01/getting-started-with-nextgen-mapreduce_11.html
http://www.cloudera.com/blog/2011/11/building-and-deploying-mr2/
http://www.crobak.org/2011/12/getting-started-with-apache-hadoop-0-23-0/