hadoop2.2的环境问题

强烈支持，顺便轻松拿金币

继续 问，如何通过eclipse而不是hadoop jar的方式运行mapreduce程序，hadoop版本为haoop2.2

首先你搭建好环境，可以参考下面配置Hadoop 2.x开发环境（Eclipse）

搭建完环境，你写一个例子就ok了

zhangcd123 发表于 2014-4-28 19:01
core-site.xml:

你的配置文件错了：你看看红字部分不可能是这样的

hdfs-site.xml:
<configuration>
<property>
  <name>dfs.namenode.name.dir</name>
<value>file:/usr/hadoop/hdfs/name</value>
  <final>true</final>
</property>
<property>
  <name>dfs.datanode.data.dir</name>
<value>file:/usr/hadoop/hdfs/data</value>
  <final>true</final>
</property>
<property>
   <name>dfs.replication</name>
   <value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>

可以参考这个：
Hadoop Mapreduce 2.0 Yarn

<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>/hadoop/dfs/name</value>
<description> </description>
</property>

<property>
<name>dfs.datanode.data.dir</name>
<value>/hadoop/dfs/data</value>
<description> </description>
</property>

<property>
<name>dfs.replication</name>
<value>2</value>
</property>

</configuration>

楼上的你好，那个配置文件改成下面你写的那样还是出同样的问题

zhangcd123 发表于 2014-4-30 15:15
楼上的你好，那个配置文件改成下面你写的那样还是出同样的问题

参考一下这个，出现的错误跟你的类似
Eclipse调用hadoop2运作MR程序

下面解决办法你参考下，出自：Hadoop2.3.0+Hbase0.96.1.1+Hive0.14.0+Zookeeper3.4.6+Sqoop1.99.3安装配置指导

运行job最后状态为failed，检查日志报错

Application application_1397717698978_0004failed 2 times due to AM Container for appattempt_1397717698978_0004_000002exited with exitCode: 1 due to: Exception from container-launch:org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
atorg.apache.hadoop.util.Shell.runCommand(Shell.java:505)
atorg.apache.hadoop.util.Shell.run(Shell.java:418)
atorg.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
atorg.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
atorg.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
atjava.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
atjava.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Container exited with a non-zero exit code1
.Failing this attempt.. Failing theapplication.
复制代码

执行如下命令测试job：

$HADOOP_HOME/bin/hadoop jar$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.3.0.jar pi 10 5
复制代码

如果报错则说明hadoop配置有问题，检查$HADOOP_HOME/etc/hadoop/mapred-site.xml
发现mapreduce.jobhistory.webapp.address这个参数配错了，应为qzj05:19888