我也遇到这个错,通过在spark-env.sh中添加export JAVA_HOME=/usr/local/java/jdk1.7.0_80 问题得到解决。 |
原因没找到,我hadoop是cdh5.5版本 spark也是cdh5.5版本,报以上错误,把spark换成官方下的版本1.5运行再没报那个错了~~ 可能cdh的缺东西。。。 |
arsenduan 发表于 2015-12-22 18:10 多谢,只是不知道具体语句怎么写,我没学过java 搞的是运维,公司让搭个环境给他们开发的用,还有我把那些包都复制到/opt/spark/lib里了,这个方式启动还是报一样的错,还是因为缺包吗? |
HawCHing 发表于 2015-12-22 13:31 试试下面方式: drive -class-............ 例子: ./bin/spark-submit \ --class org.apache.spark.examples.mllib.JavaALS \ --master yarn-cluster \ --driver-class-path $(echo /opt/cloudera/parcels/CDH/lib/hadoop-yarn/*.jar |sed 's/ /:/g'):/opt/cloudera/parcels/CDH-5.1.2-1.cdh5.1.2.p0.3/lib/hadoop-hdfs/hadoop-hdfs-2.3.0-cdh5.1.2.jar \ /opt/cloudera/parcels/CDH-5.1.2-1.cdh5.1.2.p0.3/lib/hadoop-yarn/lib/spark-examples_2.10-1.0.0-cdh5.1.2.jar \ /user/data/netflix_rating 10 10 /user/data/result 来自: spark on yarn运行产生缺jar包错误及解决办法【分享】 http://www.aboutyun.com/thread-16701-1-1.html |
arsenduan 发表于 2015-12-22 13:03 我下了个包解压了,然后路径加到classpath里了,但是运行还是会报这个错,是还需要链接还是怎么样吗? /opt/spark/logs [root@nameNode logs]# cat spark-root-org.apache.spark.deploy.master.Master-1-nameNode.out Spark Command: /usr/java/jdk1.7.0_79/bin/java -cp /opt/spark/sbin/../conf/:/opt/spark/lib/spark-assembly-1.5.0-cdh5.5.0-hadoop2.6.0-cdh5.5.0.jar:/opt/hadoop/etc/hadoop/:/opt/hadoop/etc/hadoop/ -Xms1g -Xmx1g -XX:MaxPermSize=256m org.apache.spark.deploy.master.Master --ip 192.168.2.100 --port 7077 --webui-port 8080 ======================================== Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/Logger at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2615) at java.lang.Class.getMethod0(Class.java:2856) at java.lang.Class.getMethod(Class.java:1668) at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486) Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 6 more [root@nameNode logs]# echo $CLASSPATH .:/usr/java/jdk1.7.0_79/lib:/usr/java/jdk1.7.0_79/jre/lib:/usr/java/slf4j-1.7.6 [root@nameNode logs]# cd /usr/java/slf4j-1.7.6 [root@nameNode slf4j-1.7.6]# ls integration log4j-over-slf4j-1.7.6.jar slf4j-android-1.7.6-sources.jar slf4j-jcl-1.7.6-sources.jar slf4j-migrator-1.7.6.jar jcl-over-slf4j log4j-over-slf4j-1.7.6-sources.jar slf4j-api slf4j-jdk14 slf4j-nop jcl-over-slf4j-1.7.6.jar osgi-over-slf4j slf4j-api-1.7.6.jar slf4j-jdk14-1.7.6.jar slf4j-nop-1.7.6.jar jcl-over-slf4j-1.7.6-sources.jar osgi-over-slf4j-1.7.6.jar slf4j-api-1.7.6-sources.jar slf4j-jdk14-1.7.6-sources.jar slf4j-nop-1.7.6-sources.jar jul-to-slf4j osgi-over-slf4j-1.7.6-sources.jar slf4j-ext slf4j-log4j12 slf4j-simple jul-to-slf4j-1.7.6.jar pom.xml slf4j-ext-1.7.6.jar slf4j-log4j12-1.5.11.jar slf4j-simple-1.7.6.jar jul-to-slf4j-1.7.6-sources.jar site slf4j-ext-1.7.6-sources.jar slf4j-log4j12-1.7.6.jar slf4j-simple-1.7.6-sources.jar LICENSE.txt slf4j-android slf4j-jcl slf4j-log4j12-1.7.6-sources.jar slf4j-site log4j-over-slf4j slf4j-android-1.7.6.jar slf4j-jcl-1.7.6.jar slf4j-migrator src [root@nameNode slf4j-1.7.6]# |
HawCHing 发表于 2015-12-22 12:45 https://yunpan.cn/cus2LtbQTheKX 访问密码 44bc |
langke93 发表于 2015-12-22 12:00 无奈做伸手党,因为java不熟,网上搜了试下没搜到这个jar,方便给个类似的下载链接吗,谢谢了! |
org.slf4j.logger jar 没有找到这个jar包,下载个添加进去,或则看看路径是不是有问题了 |