分享

eclipse 运行mapreduce出现ava.lang.NoClassDefFound

wlzxnet 发表于 2015-7-30 09:38:02 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 3 12396
我本地(非hadoop所在的虚拟机)的yarn-site.xml  配置信息如下:
<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>
    <property>
        <name>yarn.application.classpath</name>
        <value>
           /project/hadoop/etc/hadoop,
/project/hadoop/share/hadoop/common/*,
/project/hadoop/share/hadoop/common/lib/*,
/project/hadoop/share/hadoop/hdfs/*,
/project/hadoop/share/hadoop/hdfs/lib/*,
/project/hadoop/share/hadoop/mapreduce/*,
/project/hadoop/share/hadoop/mapreduce/lib/*,
/project/hadoop/share/hadoop/yarn/*,
/project/hadoop/share/hadoop/yarn/lib/*
        </value>
    </property>
eclipse 输出的错误信息如下:
15/07/30 09:07:00 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/07/30 09:07:01 INFO client.RMProxy: Connecting to ResourceManager at aaa/192.168.1.107:8032
15/07/30 09:07:01 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
15/07/30 09:07:01 WARN mapreduce.JobSubmitter: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
15/07/30 09:07:01 INFO input.FileInputFormat: Total input paths to process : 1
15/07/30 09:07:02 INFO mapreduce.JobSubmitter: number of splits:1
15/07/30 09:07:03 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
15/07/30 09:07:03 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1438218283134_0001
15/07/30 09:07:03 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
15/07/30 09:07:03 INFO impl.YarnClientImpl: Submitted application application_1438218283134_0001
15/07/30 09:07:03 INFO mapreduce.Job: The url to track the job: http://aaa:8088/proxy/application_1438218283134_0001/
15/07/30 09:07:03 INFO mapreduce.Job: Running job: job_1438218283134_0001
15/07/30 09:07:09 INFO mapreduce.Job: Job job_1438218283134_0001 running in uber mode : false
15/07/30 09:07:09 INFO mapreduce.Job:  map 0% reduce 0%
15/07/30 09:07:09 INFO mapreduce.Job: Job job_1438218283134_0001 failed with state FAILED due to: Application application_1438218283134_0001 failed 2 times due to AM Container for appattempt_1438218283134_0001_000002 exited with  exitCode: 1
For more detailed output, check application tracking page:http://aaa:8088/proxy/application_1438218283134_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1438218283134_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
15/07/30 09:07:09 INFO mapreduce.Job: Counters: 0
我查看日志后:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService [见楼下]

已有(3)人评论

跳转到指定楼层
wlzxnet 发表于 2015-7-30 09:39:55
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        ... 13 more
回复

使用道具 举报

Alkaloid0515 发表于 2015-7-30 10:23:39
wlzxnet 发表于 2015-7-30 09:39
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeServic ...



下面引用是不正确的:

<value>
           /project/hadoop/etc/hadoop,
/project/hadoop/share/hadoop/common/*,
/project/hadoop/share/hadoop/common/lib/*,
/project/hadoop/share/hadoop/hdfs/*,
/project/hadoop/share/hadoop/hdfs/lib/*,
/project/hadoop/share/hadoop/mapreduce/*,
/project/hadoop/share/hadoop/mapreduce/lib/*,
/project/hadoop/share/hadoop/yarn/*,
/project/hadoop/share/hadoop/yarn/lib/*
        </value>
改成下面:
<value>
           /project/hadoop/etc/hadoop,/project/hadoop/share/hadoop/common/*,/project/hadoop/share/hadoop/common/lib/*,/project/hadoop/share/hadoop/hdfs/*,/project/hadoop/share/hadoop/hdfs/lib/*,/project/hadoop/share/hadoop/mapreduce/*,/project/hadoop/share/hadoop/mapreduce/lib/*,/project/hadoop/share/hadoop/yarn/*,/project/hadoop/share/hadoop/yarn/lib/*
        </value>

不要有空格或则换行等方式

还有注意版本一致,包引用全面

回复

使用道具 举报

wlzxnet 发表于 2015-7-30 11:14:27
Alkaloid0515 发表于 2015-7-30 10:23
下面引用是不正确的:

版本是一致的,包也全部导入了。按照你的方法试了。仍然报同样的错误。咋办?

回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条