图文精华

About云-梭伦科技»专题 › 交流区 › 技术交流 › Pig › pig执行完之后，正确输出，但是最后还有一大截ipc连接问 ...
分享

pig执行完之后，正确输出，但是最后还有一大截ipc连接问题

Joker 发表于 2014-12-11 15:02:44 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 4 13345 
  1. 

  2. 2014-12-11 14:21:20,943 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: liubei/10.10.80.144:60269. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  3. 2014-12-11 14:21:21,944 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: liubei/10.10.80.144:60269. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  4. 2014-12-11 14:21:22,945 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: liubei/10.10.80.144:60269. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  5. 2014-12-11 14:21:23,053 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server

  6. 2014-12-11 14:21:25,606 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: guangyu/10.10.80.145:45352. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  7. 2014-12-11 14:21:26,607 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: guangyu/10.10.80.145:45352. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  8. 2014-12-11 14:21:27,608 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: guangyu/10.10.80.145:45352. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  9. 2014-12-11 14:21:27,712 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server

  10. 2014-12-11 14:21:28,956 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:34849. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  11. 2014-12-11 14:21:29,956 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:34849. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  12. 2014-12-11 14:21:30,957 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:34849. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  13. 2014-12-11 14:21:31,061 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server

  14. 2014-12-11 14:21:32,237 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:54561. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  15. 2014-12-11 14:21:33,238 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:54561. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  16. 2014-12-11 14:21:34,238 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:54561. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  17. 2014-12-11 14:21:34,342 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server

  18. 2014-12-11 14:21:35,499 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhangfei/10.10.80.142:44738. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  19. 2014-12-11 14:21:36,500 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhangfei/10.10.80.142:44738. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  20. 2014-12-11 14:21:37,501 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhangfei/10.10.80.142:44738. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  21. 2014-12-11 14:21:37,605 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server

  22. 2014-12-11 14:21:38,750 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:40037. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  23. 2014-12-11 14:21:39,751 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:40037. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  24. 2014-12-11 14:21:40,752 [main] INFO  org.apache.hadoop.ipc.Client - Retrying connect to server: zhaoyun/10.10.80.143:40037. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)

  25. 2014-12-11 14:21:40,856 [main] INFO  org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server

  26. 2014-12-11 14:21:40,987 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!

复制代码
Hadoop的配置是更具这个链接进行hadoop集群搭建


我看了resourcemanager和nodemanager的log之前是虚拟内存1gb of 2gb后来改成2.1gb的虚拟内存

还是一直这样，大家有遇到过吗？

你就把pig看成MR就行
回复

使用道具 举报

自定义广告语

已有(4)人评论

跳转到指定楼层
正序浏览
muyannian 发表于 2014-12-11 19:48:30
一直尝试连接这台机器，但是始终连接不上，这个应该是网络问题，看看host，防火墙，网络之类的。
liubei/10.10.80.144
回复

使用道具 举报

Joker 发表于 2014-12-11 20:32:08
muyannian 发表于 2014-12-11 19:48
一直尝试连接这台机器，但是始终连接不上，这个应该是网络问题，看看host，防火墙，网络之类的。
liubei/ ...

host配置好的，并且注释掉localhost写上机器名字了
防火墙机器默认不启动。 都在同一网段，相互之间可以ping通
回复

使用道具 举报

yuwenge 发表于 2014-12-11 21:41:03
Joker 发表于 2014-12-11 20:32
host配置好的，并且注释掉localhost写上机器名字了
防火墙机器默认不启动。 都在同一网段，相互之间可以 ...
尽量都贴出截图来，每一个程序员都认为自己写的程序是正确的，可是就是找不到原因。所以只有贴出真实的内容，别人才能帮忙解决问题。

尝试这样操作，重新启动hadoop，然后格式化namenode
回复

使用道具 举报

Joker 发表于 2014-12-12 11:01:38
我想这个异常是没有关系的。
我在nodemanager log中查看到
state: C_RUNNING, diagnostics: ", exit_status: -1000
当然，前面还有一大段字符串，这里上不去服务器手敲的~

还有一小段是。165.5MB of 1GB physical memory used 1.2GB if 2.1GB virtual memory used

可以看到，虽然container分配的内存为1GB，但是在使用到165.5MB（165.5MB of 1GB physical memory used）时task被kill掉了）


但是container并没有关闭，而是在执行任务，但是任务已经关闭掉了，所有出现ipc连接问题。



回复

使用道具 举报

高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

发表新帖
Joker

高级会员

关注

  • 44

    主题

  • 271

    帖子

  • 3

    粉丝

TA的主题
关闭

推荐上一条 /2 下一条

关于我们· 联系我们· 加入我们· 小黑屋· 合作伙伴   

Copyright © 2001-2024 About云-梭伦科技 Powered by Discuz! X3.4 Licensed Discuz Team.

简书 / Smrz 京ICP备2020039040号 Wxb 简书网举报电话：021-34700000