分享

Oozie使用问题

pangge2007 发表于 2014-5-4 16:42:05 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 7 23140
我使用Oozie向Hadoop2.2发送测试用例Map-Reduce报如下异常,哪位大神帮我解答一下,谢谢。。。

4 15:17:52,690 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:17:53,695 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:17:54,699 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:17:55,703 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:17:56,707 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:17:57,712 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:17:58,716 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:17:59,720 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:18:30,732 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:18:31,736 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:18:32,744 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:18:33,749 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:18:34,753 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:18:35,757 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)2014-05-04 15:18:36,207 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a signal. Signaling RMCommunicator and JobHistoryEventHandler.2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that iSignalled is: true2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: false2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that shouldUnregistered is: false2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: false2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is false

已有(8)人评论

跳转到指定楼层
pangge2007 发表于 2014-5-4 16:44:11
在Hadoop能够成功看到Oozie已成功提交的Job,上面的日志是在Hadoop的子节点中提取的。。。。
回复

使用道具 举报

hyj 发表于 2014-5-4 17:04:57


Retrying connect to server: 0.0.0.0/0.0.0.0:8030. Already tried 2 time(s);
红字部分,ip不正确。你看看是什么原因造成的


回复

使用道具 举报

pangge2007 发表于 2014-5-4 17:44:52
hyj 发表于 2014-5-4 17:04
红字部分,ip不正确。你看看是什么原因造成的

[Thread-1] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that iSignalled is: true2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: false 2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified that shouldUnregistered is: false  2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: false  2014-05-04 15:18:36,208 INFO [Thread-1] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is false


IP那个应该没啥问题,主要是下面的几个false的问题。。。
回复

使用道具 举报

sstutu 发表于 2014-5-4 17:58:42
你可以排除定位一下错误:看看是你的程序问题,还是Oozie的问题。
你可以先不通过Oozie来提交,你直接提交看看是否会产生这个问题。

还有像发生这种错误,就是没有远程服务器没有响应。
你看看ssh,防火墙,hdfs空间等等排查一下。

回复

使用道具 举报

hyj 发表于 2014-5-4 18:06:54
本帖最后由 hyj 于 2014-5-4 18:13 编辑
pangge2007 发表于 2014-5-4 17:44
[Thread-1] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: RMCommunicator notified tha ...

很多异常都会有你标识的红字部分,建议看看为什么通信失败。
就是楼上说的,ssh,防火墙之类的,确保都没有问题。
回复

使用道具 举报

pangge2007 发表于 2014-6-11 11:00:53
hyj 发表于 2014-5-4 18:06
很多异常都会有你标识的红字部分,建议看看为什么通信失败。
就是楼上说的,ssh,防火墙之类的,确保都 ...

通信都没问题,我在本机发送hadoop任务都能成功执行。。。

点评

应该是Oozie的配置问题,你可以检查下配置,该注册的是否注册。  发表于 2014-6-11 11:38
回复

使用道具 举报

pangge2007 发表于 2014-6-12 13:54:20
pangge2007 发表于 2014-6-11 11:00
通信都没问题,我在本机发送hadoop任务都能成功执行。。。

ooize提交的任务和普通的map——reduce任务比较,在tmp--->staging->job***文件夹中的内容有不同
oozie提交的结果中没有job.jar,普通map_reduce提交的结果中有job.jar

回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条