新手指导，该如何在开发环境中，创建mapreduce程序

langke93 · 发表于 2015-4-19 15:55:20

yunmin 发表于 2015-4-19 15:05
楼上那个什么问题造成的？每台机的InputPath和OutputPath都应该不一样吗？

下面改成自己的路径

static final String INPUT_PATH = "hdfs://aboutyun:9000/yy1";
static final String OUT_PATH = "hdfs://aboutyun:9000/Output";

yunmin · 发表于 2015-4-22 09:24:20

langke93 发表于 2015-4-19 15:55
下面改成自己的路径

static final String INPUT_PATH = "hdfs://aboutyun[/backcolor ...

已经解决了！谢谢Qq

fzu3144 · 发表于 2015-7-15 16:24:41

[mw_shl_code=java,true]SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/I:/%e5%ae%89%e8%a3%85%e9%83%a8%e7%bd%b2/%e5%bc%95%e5%85%a5jar%e5%8c%85/hadoop-eclipse-plugin-2.2.0%e4%be%9d%e8%b5%96%e5%8c%85/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/I:/%e5%ae%89%e8%a3%85%e9%83%a8%e7%bd%b2/%e5%bc%95%e5%85%a5jar%e5%8c%85/hadoop-2.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/F:/master/hbase-0.98.12.1-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2015-07-15 16:21:20,201 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-07-15 16:21:21,279 INFO  [main] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at master/192.168.37.136:8032
2015-07-15 16:21:23,270 WARN  [main] mapreduce.JobResourceUploader (JobResourceUploader.java:uploadFiles(64)) - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2015-07-15 16:21:23,442 WARN  [main] mapreduce.JobResourceUploader (JobResourceUploader.java:uploadFiles(171)) - No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
2015-07-15 16:21:23,609 INFO  [main] input.FileInputFormat (FileInputFormat.java:listStatus(283)) - Total input paths to process : 1
2015-07-15 16:21:25,497 INFO  [main] mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(201)) - number of splits:1
2015-07-15 16:21:26,388 INFO  [main] mapreduce.JobSubmitter (JobSubmitter.java:printTokens(290)) - Submitting tokens for job: job_1436614419410_0031
2015-07-15 16:21:26,838 INFO  [main] mapred.YARNRunner (YARNRunner.java:createApplicationSubmissionContext(371)) - Job jar is not present. Not adding any jar to the list of resources.
2015-07-15 16:21:27,433 INFO  [main] impl.YarnClientImpl (YarnClientImpl.java:submitApplication(273)) - Submitted application application_1436614419410_0031
2015-07-15 16:21:27,537 INFO  [main] mapreduce.Job (Job.java:submit(1294)) - The url to track the job: http://master:8088/proxy/application_1436614419410_0031/
2015-07-15 16:21:27,539 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1339)) - Running job: job_1436614419410_0031
2015-07-15 16:21:32,640 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1360)) - Job job_1436614419410_0031 running in uber mode : false
2015-07-15 16:21:32,646 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1367)) -  map 0% reduce 0%
2015-07-15 16:21:32,710 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1380)) - Job job_1436614419410_0031 failed with state FAILED due to: Application application_1436614419410_0031 failed 2 times due to AM Container for appattempt_1436614419410_0031_000002 exited with  exitCode: 1
For more detailed output, check application tracking page:http://master:8088/cluster/app/application_1436614419410_0031Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1436614419410_0031_02_000001
Exit code: 1
Exception message: /bin/bash: line 0: fg: no job control

Stack trace: ExitCodeException exitCode=1: /bin/bash: line 0: fg: no job control

at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
2015-07-15 16:21:32,779 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1385)) - Counters: 0
[/mw_shl_code]

Alkaloid0515 · 发表于 2015-7-15 21:44:08

fzu3144 发表于 2015-7-15 16:24
[mw_shl_code=java,true]SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in ...

应该是引用包重复了

blue5whale · 发表于 2016-4-28 21:58:50

quenlang 发表于 2014-9-28 23:01
楼主，我在eclipse中运行WordCount的实例程序时，没有任何报错信息，设置好输入路径和输出路径后，run，然 ...

我最近是刚接触hadoop，也碰到了跟你一模一样的问题，想问问你以前解决了这个问题了吗？

when30 · 发表于 2016-4-28 22:01:58

blue5whale 发表于 2016-4-28 21:58
我最近是刚接触hadoop，也碰到了跟你一模一样的问题，想问问你以前解决了这个问题了吗？

最有可能环境不同造成的，上图看看

blue5whale · 发表于 2016-4-28 23:51:15

when30 发表于 2016-4-28 22:01
最有可能环境不同造成的，上图看看

这个图片是运行后的界面

blue5whale · 发表于 2016-4-28 23:57:06

when30 发表于 2016-4-28 22:01
最有可能环境不同造成的，上图看看

抱歉，对发图不怎么熟悉，不过我描述给你就知道了主要问题就是我在处理/user/Echizen/mytest下面的文件note.txt后，在mytest下面得到了out输出文件夹，但是里面没有内容，我每次刷新后out文件夹括号后面的数字就突然变成1然后又变回0了，总之就是没有输出结果，我感觉是log4j的配置问题，但是不知道如何配置