分享

word_count运行卡住不会动~~~!

运行代码如下:
[mw_shl_code=shell,true]chen@chen-K84HR ~/hadoop-2.7.1 $ hadoop jar ~/hadoop-2.7.1/share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar -jobconf mapred.reduce.tasks=4 -mapper /home/chen/mapreduce/word_count/mapper.py -reducer /home/chen/mapreduce/word_count/reduce.py -input book/* -output book-output[/mw_shl_code]

然后到到 INFO mapreduce.Job: Running job: job_1437734035333_00就不会动了!
整个运行如下:
[mw_shl_code=shell,true]15/07/24 19:20:42 WARN streaming.StreamJob: -jobconf option is deprecated, please use -D instead.
OpenJDK Server VM warning: You have loaded library /home/chen/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
15/07/24 19:20:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
packageJobJar: [/tmp/hadoop-unjar4200297893364560010/] [] /tmp/streamjob3830355201014675690.jar tmpDir=null
15/07/24 19:20:43 INFO client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8032
15/07/24 19:20:43 INFO client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8032
15/07/24 19:20:44 INFO mapred.FileInputFormat: Total input paths to process : 1
15/07/24 19:20:44 INFO mapreduce.JobSubmitter: number of splits:2
15/07/24 19:20:44 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
15/07/24 19:20:44 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1437734035333_0003
15/07/24 19:20:45 INFO impl.YarnClientImpl: Submitted application application_1437734035333_0003
15/07/24 19:20:45 INFO mapreduce.Job: The url to track the job: http://chen-K84HR:8088/proxy/application_1437734035333_0003/
15/07/24 19:20:45 INFO mapreduce.Job: Running job: job_1437734035333_00[/mw_shl_code]

Error.png

求助~求助~

已有(4)人评论

跳转到指定楼层
yuwenge 发表于 2015-7-25 14:35:51
楼主的输入输出文件是否有问题。
是如何操作的。
估计是参数或则文件的问题
回复

使用道具 举报

smartleon 发表于 2015-7-25 14:42:54
那么多参数,都知道是啥意思不?把最基本的输入输出写上,先试试最简单的看看
回复

使用道具 举报

mzcyx2011 发表于 2015-7-25 18:41:23
smartleon 发表于 2015-7-25 14:42
那么多参数,都知道是啥意思不?把最基本的输入输出写上,先试试最简单的看看

看起来多而已,,实际上就输入了mapper reducer 还有输入输出而已。。
回复

使用道具 举报

smartleon 发表于 2015-7-26 17:12:21
mzcyx2011 发表于 2015-7-25 18:41
看起来多而已,,实际上就输入了mapper reducer 还有输入输出而已。。

问题解决了吗?
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条