【解决】hadoop 运行自己写的wordcount下例子报错：error opening job jar

本帖最后由 howtodown 于 2014-4-11 02:22 编辑

报错如下：

[root@hadoop hadoop]# hadoop jar mapreduce.jar com.wordCount.WordCount /usr/hello/hello.txt /usr/hello/count.txt14/02/21 23:35:01 INFO mapred.JobClient: Cleaning up the staging area hdfs://hadoop:9000/usr/local/hadoop/tmp/mapred/staging/root/.staging/job_201402211715_0002
14/02/21 23:35:01 ERROR security.UserGroupInformation: PriviledgedActionException as:root cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /usr/hello/count.txt already exists
Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /usr/hello/count.txt already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:949)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at com.wordCount.WordCount.main(WordCount.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
[root@hadoop hadoop]# hadoop jar mapreduce.jar com.wordCount.WordCount /usr/hello/hello.txt /usr/hello/count.txt
14/02/21 23:36:33 INFO mapred.JobClient: Cleaning up the staging area hdfs://hadoop:9000/usr/local/hadoop/tmp/mapred/staging/root/.staging/job_201402211715_0003
14/02/21 23:36:33 ERROR security.UserGroupInformation: PriviledgedActionException as:root cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /usr/hello/count.txt already exists
Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory /usr/hello/count.txt already exists
at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:137)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:949)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at com.wordCount.WordCount.main(WordCount.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

[root@hadoop hadoop]# hadoop fs -ls /usr/hello
Found 3 items
drwxr-xr-x - root supergroup 0 2014-02-21 23:25 /usr/hello/count.txt
-rw-r--r-- 1 root supergroup 18 2014-02-21 22:59 /usr/hello/hello.txt
-rw-r--r-- 1 root supergroup 3468 2014-02-21 23:16 /usr/hello/mapreduce.jar

[root@hadoop hadoop]# hadoop fs -rmr /usr/hello/count.txt
Deleted hdfs://hadoop:9000/usr/hello/count.txt
[root@hadoop hadoop]# hadoop fs -ls /usr/hello
Found 2 items
-rw-r--r-- 1 root supergroup 18 2014-02-21 22:59 /usr/hello/hello.txt
-rw-r--r-- 1 root supergroup 3468 2014-02-21 23:16 /usr/hello/mapreduce.jar

运行失败：

[root@hadoop hadoop]# hadoop jar mapreduce.jar com.wordCount.WordCount /usr/hello/hello.txt /usr/hello/count.txt
14/02/21 23:38:10 INFO input.FileInputFormat: Total input paths to process : 1
14/02/21 23:38:10 INFO util.NativeCodeLoader: Loaded the native-hadoop library
14/02/21 23:38:10 WARN snappy.LoadSnappy: Snappy native library not loaded
14/02/21 23:38:11 INFO mapred.JobClient: Running job: job_201402211715_0004
14/02/21 23:38:12 INFO mapred.JobClient: map 0% reduce 0%
14/02/21 23:38:21 INFO mapred.JobClient: map 50% reduce 0%
14/02/21 23:42:26 INFO mapred.JobClient: Task Id : attempt_201402211715_0004_m_000000_0, Status : FAILED