WARN util.NativeCodeLoader: Unable to load native-hadoop library for your pl...

whx@whx-desktop:/usr/local/hadoop$ hadoop jar hadoop-examples-*.jar wordcount input output
15/11/22 17:30:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/11/22 17:30:21 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
15/11/22 17:30:21 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
15/11/22 17:30:21 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/usr/local/hadoop_tmp/mapred/staging/whx274145175/.staging/job_local274145175_0001
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost:9000/user/whx/input
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:321)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:264)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:385)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:597)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:614)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:492)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
    at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
whx@whx-desktop:/usr/local/hadoop$
whx@whx-desktop:/usr/local/hadoop$ hadoop dfs -copyFromLocal conf/* input
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

15/11/22 17:33:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
copyFromLocal: `input': No such file or directory
whx@whx-desktop:/usr/local/hadoop$

本地没有创建input 文件夹，或则没有找到input文件夹

jixianqiuxue 发表于 2015-11-22 18:10
本地没有创建input 文件夹，或则没有找到input文件夹

whx@whx-desktop:/usr/local/hadoop$ ls -l
total 200
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 bin
drwxrwxrwx 3 hduser hadoop   4096 11月 14  2014 etc
-rwxrwxrwx 1 whx    whx    142726 11月 22 17:29 hadoop-examples-1.2.0.jar
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 include
drwxrwxrwx 2 hduser hadoop   4096 10月 17 21:13 input
drwxrwxrwx 3 hduser hadoop   4096 11月 14  2014 lib
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 libexec
-rw-rw-rw- 1 hduser hadoop  15429 11月 14  2014 LICENSE.txt
drwxrwxrwx 3 whx    whx      4096 11月 22 17:25 logs
-rw-rw-rw- 1 hduser hadoop    101 11月 14  2014 NOTICE.txt
-rw-rw-rw- 1 hduser hadoop   1366 11月 14  2014 README.txt
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 sbin
drwxrwxrwx 4 hduser hadoop   4096 11月 14  2014 share
whx@whx-desktop:/usr/local/hadoop$
这里面不是有吗？

往事也加 发表于 2015-11-22 18:29
whx@whx-desktop:/usr/local/hadoop$ ls -l
total 200
drwxrwxrwx 2 hduser hadoop   4096 11月 14  20 ...

-rwxrwxrwx 1 whx    whx    142726 11月 22 17:29 hadoop-examples-1.2.0.jar
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 include
drwxrwxrwx 2 hduser hadoop   4096 10月 17 21:13 input

犯了同样的错误，用户权限又不对。

jixianqiuxue 发表于 2015-11-22 18:33
-rwxrwxrwx 1 whx    whx    142726 11月 22 17:29 hadoop-examples-1.2.0.jar
drwxrwxrwx 2 hduser had ...

whx@whx-desktop:/usr/local/hadoop#sudo  cat /etc/sudoers

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

# Allow members of group sudo to execute any command
%sudo    ALL=(ALL:ALL) ALL
whx     ALL=(ALL) NOPASSWD:ALL
hduser  ALL=(ALL) NOPASSWD:ALL
postgres ALL=(ALL) NOPASSWD:ALL
# See sudoers(5) for more information on "#include" directives:
这样做也不行吗？

jixianqiuxue 发表于 2015-11-22 18:33
-rwxrwxrwx 1 whx    whx    142726 11月 22 17:29 hadoop-examples-1.2.0.jar
drwxrwxrwx 2 hduser had ...

那使用root,whx,hduser,这其中的那个用户呢？是否需要修改一下下面的文件(夹)的拥有者呢？
root@whx-desktop:/usr/local/hadoop# ls -l
total 200
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 bin
drwxrwxrwx 3 hduser hadoop   4096 11月 14  2014 etc
-rwxrwxrwx 1 whx    whx    142726 11月 22 17:29 hadoop-examples-1.2.0.jar
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 include
drwxrwxrwx 2 hduser hadoop   4096 10月 17 21:13 input
drwxrwxrwx 3 hduser hadoop   4096 11月 14  2014 lib
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 libexec
-rw-rw-rw- 1 hduser hadoop  15429 11月 14  2014 LICENSE.txt
drwxrwxrwx 3 whx    whx      4096 11月 22 17:25 logs
-rw-rw-rw- 1 hduser hadoop    101 11月 14  2014 NOTICE.txt
-rw-rw-rw- 1 hduser hadoop   1366 11月 14  2014 README.txt
drwxrwxrwx 2 hduser hadoop   4096 11月 14  2014 sbin
drwxrwxrwx 4 hduser hadoop   4096 11月 14  2014 share
root@whx-desktop:/usr/local/hadoop#

往事也加 发表于 2015-11-22 18:40
那使用root,whx,hduser,这其中的那个用户呢？是否需要修改一下下面的文件(夹)的拥有者呢？
root@whx-des ...

修改下权限拥有者。让他们保持一致

jixianqiuxue 发表于 2015-11-22 18:37
修改下权限拥有者。让他们保持一致

那我全部修改成root，变成类似如下应该可以吧
root@whx-desktop:/usr# ls -l
total 172
drwxr-xr-x   2 root root 69632 11月 22 12:43 bin
drwxr-xr-x   4 root root  4096 10月  8 16:18 Flash_player11
drwxr-xr-x   2 root root  4096  7月 23  2014 games
drwxr-xr-x   9 root root  4096 10月 31 08:43 Gem5-GPGPU
drwxr-xr-x   3 root root  4096 10月 14 18:12 GPGPU-SIMS
drwxr-xr-x  45 root root 16384 11月 22 09:38 include
drwxr-xr-x 166 root root 20480 11月 22 12:42 lib
drwxr-xr-x   3 root root  4096 10月 13 08:46 lib32
drwxr-xr-x  20 root root  4096 11月 22 09:10 local
drwxr-xr-x   3 root root  4096 10月 13 14:27 nvidia
drwxr-xr-x   3 root root  4096 10月 17 15:18 PostgreSQL
drwxr-xr-x   2 root root 12288 11月 21 11:20 sbin
drwxr-xr-x 328 root root 12288 11月 22 12:43 share
drwxr-xr-x  12 root root  4096 11月 21 12:13 src

往事也加 发表于 2015-11-22 18:48
那我全部修改成root，变成类似如下应该可以吧
root@whx-desktop:/usr# ls -l
total 172

可以的

jixianqiuxue 发表于 2015-11-22 18:49
可以的

whx@whx-desktop:/usr/local/hadoop$ start-dfs.sh
15/11/22 19:03:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
whx@localhost's password:
localhost: namenode running as process 14972. Stop it first.
whx@localhost's password:
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-whx-datanode-whx-desktop.out
Starting secondary namenodes [0.0.0.0]
whx@0.0.0.0's password:
0.0.0.0: secondarynamenode running as process 15297. Stop it first.
15/11/22 19:03:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
whx@whx-desktop:/usr/local/hadoop$ start-yarn.sh
starting yarn daemons
resourcemanager running as process 15467. Stop it first.
whx@localhost's password:
localhost: nodemanager running as process 15791. Stop it first.
whx@whx-desktop:/usr/local/hadoop$ hadoop dfs -copyFromLocal conf/* input  
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

15/11/22 19:04:12 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
copyFromLocal: `input': No such file or directory
whx@whx-desktop:/usr/local/hadoop$ ls -l
total 200
drwxrwxrwx 2 root root   4096 11月 14  2014 bin
drwxrwxrwx 3 root root   4096 11月 14  2014 etc
-rwxrwxrwx 1 root root 142726 11月 22 17:29 hadoop-examples-1.2.0.jar
drwxrwxrwx 2 root root   4096 11月 14  2014 include
drwxrwxrwx 2 root root   4096 10月 17 21:13 input
drwxrwxrwx 3 root root   4096 11月 14  2014 lib
drwxrwxrwx 2 root root   4096 11月 14  2014 libexec
-rw-rw-rw- 1 root root  15429 11月 14  2014 LICENSE.txt
drwxrwxrwx 3 root root   4096 11月 22 19:04 logs
-rw-rw-rw- 1 root root    101 11月 14  2014 NOTICE.txt
-rw-rw-rw- 1 root root   1366 11月 14  2014 README.txt
drwxrwxrwx 2 root root   4096 11月 14  2014 sbin
drwxrwxrwx 4 root root   4096 11月 14  2014 share
whx@whx-desktop:/usr/local/hadoop$