oozie 提交任务到 yarn阶段没有问题,执行阶段显示错误。请各位给予指点。
hread-67] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 02016-02-05 16:53:40,720 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://master:8020/user/root/.staging/job_1454661506698_0003/job_1454661506698_0003_1.jhist to hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003-1454662388942-root-oozie%3Alauncher%3AT%3Dhive%3AW%3Dhive%2Dwf%3AA%3Dhive%2Dnode%3AID%3D000-1454662420643-1-0-SUCCEEDED-default-1454662397885.jhist_tmp2016-02-05 16:53:40,822 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003-1454662388942-root-oozie%3Alauncher%3AT%3Dhive%3AW%3Dhive%2Dwf%3AA%3Dhive%2Dnode%3AID%3D000-1454662420643-1-0-SUCCEEDED-default-1454662397885.jhist_tmp2016-02-05 16:53:40,825 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://master:8020/user/root/.staging/job_1454661506698_0003/job_1454661506698_0003_1_conf.xml to hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003_conf.xml_tmp2016-02-05 16:53:40,870 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003_conf.xml_tmp2016-02-05 16:53:40,873 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003.summary_tmp to hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003.summary2016-02-05 16:53:40,876 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003_conf.xml_tmp to hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003_conf.xml2016-02-05 16:53:40,878 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003-1454662388942-root-oozie%3Alauncher%3AT%3Dhive%3AW%3Dhive%2Dwf%3AA%3Dhive%2Dnode%3AID%3D000-1454662420643-1-0-SUCCEEDED-default-1454662397885.jhist_tmp to hdfs://master:8020/mr-history/tmp/root/job_1454661506698_0003-1454662388942-root-oozie%3Alauncher%3AT%3Dhive%3AW%3Dhive%2Dwf%3AA%3Dhive%2Dnode%3AID%3D000-1454662420643-1-0-SUCCEEDED-default-1454662397885.jhist2016-02-05 16:53:40,880 INFO [Thread-67] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()2016-02-05 16:53:40,885 INFO [Thread-67] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to 2016-02-05 16:53:40,885 INFO [Thread-67] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://master:19888/jobhistory/job/job_1454661506698_00032016-02-05 16:53:40,893 INFO [Thread-67] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.2016-02-05 16:53:41,896 INFO [Thread-67] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:1 ContRel:0 HostLocal:0 RackLocal:02016-02-05 16:53:41,898 INFO [Thread-67] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://master:8020 /user/root/.staging/job_1454661506698_00032016-02-05 16:53:41,901 INFO [Thread-67] org.apache.hadoop.ipc.Server: Stopping server on 520962016-02-05 16:53:41,903 INFO [IPC Server listener on 52096] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 520962016-02-05 16:53:41,911 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder2016-02-05 16:53:41,914 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
网上有说是map-reduce配置问题。但没有贴出解决办法。各位如有知道,请帮忙指点。谢谢。
map-reduce配置文件见附件。
截取部分内容:
MR Map Java Heap Size -Xmx1024m MR Reduce Java Heap Size -Xmx1638mmapreduce.admin.map.child.java.opts -server -XX:NewRatio=8 -Djava.net.preferIPv4Stack=true -Dhdp.version=${hdp.version}mapreduce.admin.reduce.child.java.opts -server -XX:NewRatio=8 -Djava.net.preferIPv4Stack=true -Dhdp.version=${hdp.version}
|
|