分享

运行wordcount不成功

[root@lcx mapreduce]# hadoop jar hadoop-mapreduce-examples-2.2.0.jar wordcount hdfs://lcx:9000/words hdfs://lcx:9000/wcount
15/03/20 08:02:41 INFO client.RMProxy: Connecting to ResourceManager at lcx/192.168.1.100:8032
15/03/20 08:02:42 INFO input.FileInputFormat: Total input paths to process : 1
15/03/20 08:02:42 INFO mapreduce.JobSubmitter: number of splits:1
15/03/20 08:02:42 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
15/03/20 08:02:42 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
15/03/20 08:02:42 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
15/03/20 08:02:42 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
15/03/20 08:02:42 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
15/03/20 08:02:42 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1426840918345_0008
15/03/20 08:02:43 INFO impl.YarnClientImpl: Submitted application application_1426840918345_0008 to ResourceManager at lcx/192.168.1.100:8032
15/03/20 08:02:43 INFO mapreduce.Job: The url to track the job: http://lcx:8088/proxy/application_1426840918345_0008/
15/03/20 08:02:43 INFO mapreduce.Job: Running job: job_1426840918345_0008
15/03/20 08:02:47 INFO mapreduce.Job: Job job_1426840918345_0008 running in uber mode : false
15/03/20 08:02:47 INFO mapreduce.Job:  map 0% reduce 0%
15/03/20 08:02:47 INFO mapreduce.Job: Job job_1426840918345_0008 failed with state FAILED due to: Application application_1426840918345_0008 failed 2 times due to AM Container for appattempt_1426840918345_0008_000002 exited with  exitCode: -1 due to: .Failing this attempt.. Failing the application.
15/03/20 08:02:47 INFO mapreduce.Job: Counters: 0

运行wordcount不成功jps显示4965 NodeManager
4287 NameNode
34406 Jps
4626 SecondaryNameNode
4756 ResourceManager
4397 DataNode
都启动了
用的系统是centos6.4 64位 jdk1.7  hadoop2.2.0稳定版
求帮忙解决,万分感谢。

已有(14)人评论

跳转到指定楼层
arsenduan 发表于 2015-3-22 21:42:40
单节点,伪分布?
这个是自己写的,还是系统的,列出具体信息

参考这个:
这有个例子参考下:
hadoop2.2使用手册2:如何运行自带wordcount
回复

使用道具 举报

xiaoqian3599 发表于 2015-3-22 21:44:27
arsenduan 发表于 2015-3-22 21:42
单节点,伪分布?
这个是自己写的,还是系统的,列出具体信息

伪分布式系统,运行系统自带的worcount程序
回复

使用道具 举报

xiaoqian3599 发表于 2015-3-22 21:51:01
arsenduan 发表于 2015-3-22 21:42
单节点,伪分布?
这个是自己写的,还是系统的,列出具体信息

按那参考做过了,之前都运行正常到15/03/22 06:30:29 INFO mapreduce.Job:  map 0% reduce 0%

这 都卡在百分之0就报错提醒
15/03/22 06:30:29 INFO mapreduce.Job: Job job_1426840918345_0025 failed with state FAILED due to: Application application_1426840918345_0025 failed 2 times due to AM Container for appattempt_1426840918345_0025_000002 exited with  exitCode: -1 due to: .Failing this attempt.. Failing the application.
15/03/22 06:30:30 INFO mapreduce.Job: Counters: 0

文件可以正常上传hdfs
端口8088显示
点击history显示


YMIMFQX$O5L[{28`%36ZI76.jpg
回复

使用道具 举报

arsenduan 发表于 2015-3-22 22:02:39
xiaoqian3599 发表于 2015-3-22 21:51
按那参考做过了,之前都运行正常到15/03/22 06:30:29 INFO mapreduce.Job:  map 0% reduce 0%

这 都卡 ...

点击看看日志。
然后进入输入文件和输出目录,列出他们的权限看看,权限是否足够
回复

使用道具 举报

xiaoqian3599 发表于 2015-3-22 22:14:23
arsenduan 发表于 2015-3-22 22:02
点击看看日志。
然后进入输入文件和输出目录,列出他们的权限看看,权限是否足够

hadoop-root-datanode-lcx.log           hadoop-root-secondarynamenode-lcx.out.1
hadoop-root-datanode-lcx.out           SecurityAuth-root.audit
日志文件有这些,请问下看哪一个?
hadoop-root-namenode-lcx.log           yarn-root-nodemanager-lcx.log
hadoop-root-namenode-lcx.out           yarn-root-nodemanager-lcx.out
hadoop-root-namenode-lcx.out.1         yarn-root-resourcemanager-lcx.log
hadoop-root-secondarynamenode-lcx.log  yarn-root-resourcemanager-lcx.out
hadoop-root-secondarynamenode-lcx.out  yarn-root-resourcemanager-lcx.out.1
如何查看权限?

回复

使用道具 举报

arsenduan 发表于 2015-3-22 22:25:28
xiaoqian3599 发表于 2015-3-22 22:14
hadoop-root-datanode-lcx.log           hadoop-root-secondarynamenode-lcx.out.1
hadoop-root-datano ...

如下图:



看关于yarn的日志
回复

使用道具 举报

xiaoqian3599 发表于 2015-3-22 23:01:02

[root@lcx mapreduce]# hadoop fs -ls /
Found 5 items
drwx------   - root supergroup          0 2015-03-22 06:20 /hadoop
drwx------   - root supergroup          0 2015-03-22 06:20 /lcx
-rw-r--r--   1 root supergroup       1366 2015-03-20 03:03 /readme.txt
drwx------   - root supergroup          0 2015-03-20 03:07 /tmp
-rw-r--r--   1 root supergroup         56 2015-03-20 03:19 /words
可以看到文件

回复

使用道具 举报

arsenduan 发表于 2015-3-22 23:09:31
本帖最后由 arsenduan 于 2015-3-22 23:11 编辑
xiaoqian3599 发表于 2015-3-22 23:01
[root@lcx mapreduce]# hadoop fs -ls /
Found 5 items
drwx------   - root supergroup          0 20 ...


输入文件words,最好授予更多权限
输出文件夹没有看到

回复

使用道具 举报

12下一页
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条