Yarn 框架代码详细分析分享

目录
一、yarn 简单介绍 .................................................................................................................................................... 2
1.1、概述 ............................................................................................................................................................... 3
1.2、YARN 的优势.................................................................................................................................................. 4
1.3、新框架下的软件设计模式............................................................................................................................. 4
1.4、HADOOP 2.0.0- alpha 工程结构 ..................................................................................................................... 4
二、yarn 模块详细分析............................................................................................................................................. 5
2.1、接口 ............................................................................................................................................................... 5
2.2、各大模块分析 ................................................................................................................................................ 6
2.2.1、RM .......................................................................................................................................................... 6
2.2.2、NM .......................................................................................................................................................... 8
2.2.3、MRAppMaster ......................................................................................................................................... 9
2.2.4、MRYarnChild .......................................................................................................................................... 10
三、功能点详细分析 ............................................................................................................................................... 11
3.1、Jobhistory 机制 ............................................................................................................................................ 11
3.1.1． NM 收集日志 ...................................................................................................................................... 11
3.1.2． MRAppMaster 收集 JobHistory ........................................................................................................... 12
3.1.3． JobHistoryServer .................................................................................................................................. 12
3.2、RM 调度器 ................................................................................................................................................... 13
3.2.1、简述 ...................................................................................................................................................... 13
3.2.2、FairScheduler 代码分析 ........................................................................................................................ 14
3.2.3、FairScheduler 资源预分配 ..................................................................................................................... 14
3.2.4、FairScheduler 抢占资源 ........................................................................................................................ 15
3.2.5、FairScheduler container 分配 ................................................................................................................ 15
3.3、MRAppMaster 分配器 ................................................................................................................................. 17
3.3.1、代码分析............................................................................................................................................... 17
3.3.2、任务周期管理及资源分配 .................................................................................................................... 17
3.3、shuffle .......................................................................................................................................................... 19
3.5、NM 的资源下载 ........................................................................................................................................... 20
Yarn框架代码详细分析V0.3.zip (1.36 MB, 下载次数: 220)

2013-11-17 21:23 上传

点击文件名下载附件

谢谢楼主分享

thanks a lot