WARN util.NativeCodeLoader: Unable to load native-hadoop library for your pl...

往事也加 发表于 2015-11-22 19:10
whx@whx-desktop:/usr/local/hadoop$ start-dfs.sh
15/11/22 19:03:38 WARN util.NativeCodeLoader: Un ...

安装有问题，建议参考这个文档
hadoop（2.x）以hadoop2.2为例完全分布式最新高可靠安装文档
http://www.aboutyun.com/thread-7684-1-1.html

jixianqiuxue 发表于 2015-11-22 18:49
可以的

我是用的是whx用户，但是还是出现错误
我按照的是：http://blog.csdn.net/zhaoyl03/article/details/8657104
里面的内容操作的
这时注意程序是在文件系统dfs运行的，创建的文件也都基于文件系统：

首先在dfs中创建input目录

• hadoop@derekUbun:/usr/local/hadoop$ hadoop dfs -mkdir input   （出现错误）！！！！！！！！！！1
  

错误如下：
whx@whx-desktop:/usr/local/hadoop$ hadoop dfs -mkdir input
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

15/11/22 19:12:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: `input/': No such file or directory
whx@whx-desktop:/usr/local/hadoop$

whx@whx-desktop:/usr/local/hadoop$

将conf中的文件拷贝到dfs中的input

• hadoop@derekUbun:/usr/local/hadoop$ hadoop dfs -copyFromLocal conf/* input  
  
  

错误如下：

whx@whx-desktop:/usr/local/hadoop$ hadoop dfs -copyFromLocal conf/* input  
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

15/11/22 19:13:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
copyFromLocal: `input': No such file or directory
whx@whx-desktop:/usr/local/hadoop$

jixianqiuxue 发表于 2015-11-22 19:13
安装有问题，建议参考这个文档
hadoop（2.x）以hadoop2.2为例完全分布式最新高可靠安装文档
http:/ ...

我安装的是hadoop2.6.0，在ubuntu14.04 64位的操作系统
下面是hadoop版本
whx@whx-desktop:/usr/local/hadoop$ file $HADOOP_HOME/lib/native/libhadoop.so.1.0.0
/usr/local/hadoop/lib/native/libhadoop.so.1.0.0: ELF 64-bit LSB  shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=3a80422c78d708c9a1666c1a8edd23676ed77dbb, not stripped
whx@whx-desktop:/usr/local/hadoop$ hadoop version
Hadoop 2.6.0
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1
Compiled by jenkins on 2014-11-13T21:10Z
Compiled with protoc 2.5.0
From source with checksum 18e43357c8f927c0695f1e9522859d6a
This command was run using /usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.0.jar
whx@whx-desktop:/usr/local/hadoop$
whx@whx-desktop:/usr/local/hadoop$ lsb_release -a
No LSB modules are available.
Distributor ID:    Ubuntu
Description:    Ubuntu 14.04.3 LTS
Release:    14.04
Codename:    trusty
whx@whx-desktop:/usr/local/hadoop$
whx@whx-desktop:/usr/local/hadoop$ jps             （不是这个出来了，就表示正确安装了吗？）
30998 Jps
15467 ResourceManager
15297 SecondaryNameNode
14972 NameNode
24522 org.eclipse.equinox.launcher_1.3.0.dist.jar
15791 NodeManager
whx@whx-desktop:/usr/local/hadoop$

往事也加 发表于 2015-11-22 19:28
我安装的是hadoop2.6.0，在ubuntu14.04 64位的操作系统
下面是hadoop版本
whx@whx-desktop:/usr/local/ ...

whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat core-site.xml    
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>hadoop.native.lib</name>
  <value>true</value>
  <description>Should native hadoop libraries, if present, be used.</description>
</property>
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://localhost:9000</value>
</property>   

<property>
  <name>io.file.buffer.size</name>
  <value>131072</value>
</property>
<property>
  <name>hadoop.tmp.dir</name>
  <value>file:/usr/local/hadoop_tmp</value>
  <description>Abasefor other temporary directories.</description>
</property>
<property>
  <name>hadoop.proxyuser.spark.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.spark.groups</name>
  <value>*</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>dfs.replication</name>
  <value>2</value>
</property>
<property>
   <name>dfs.namenode.name.dir</name>
   <value>file:/usr/local/hadoop_tmp/hdfs/namenode</value>
</property>
<property>
   <name>dfs.datanode.data.dir</name>
   <value>file:/usr/local/hadoop_tmp/hdfs/datanode</value>
</property>
<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>hadoop.native.lib</name>
  <value>true</value>
  <description>Should native hadoop libraries, if present, be used.</description>
</property>
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://localhost:9000</value>
</property>

<property>
  <name>io.file.buffer.size</name>
  <value>131072</value>
</property>
<property>
  <name>hadoop.tmp.dir</name>
  <value>file:/usr/local/hadoop_tmp</value>
  <description>Abasefor other temporary directories.</description>
</property>
<property>
  <name>hadoop.proxyuser.spark.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.spark.groups</name>
  <value>*</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat mapred-
mapred-env.cmd              mapred-queues.xml.template  mapred-site.xml~            
mapred-env.sh               mapred-site.xml             mapred-site.xml.template   
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$

往事也加 发表于 2015-11-22 19:28
我安装的是hadoop2.6.0，在ubuntu14.04 64位的操作系统
下面是hadoop版本
whx@whx-desktop:/usr/local/ ...

下面的配置有问题吗？(能加一个QQ好不呢？)
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>hadoop.native.lib</name>
  <value>true</value>
  <description>Should native hadoop libraries, if present, be used.</description>
</property>
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://localhost:9000</value>
</property>

<property>
  <name>io.file.buffer.size</name>
  <value>131072</value>
</property>
<property>
  <name>hadoop.tmp.dir</name>
  <value>file:/usr/local/hadoop_tmp</value>
  <description>Abasefor other temporary directories.</description>
</property>
<property>
  <name>hadoop.proxyuser.spark.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.spark.groups</name>
  <value>*</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>dfs.replication</name>
  <value>2</value>
</property>
<property>
   <name>dfs.namenode.name.dir</name>
   <value>file:/usr/local/hadoop_tmp/hdfs/namenode</value>
</property>
<property>
   <name>dfs.datanode.data.dir</name>
   <value>file:/usr/local/hadoop_tmp/hdfs/datanode</value>
</property>
<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
  <name>hadoop.native.lib</name>
  <value>true</value>
  <description>Should native hadoop libraries, if present, be used.</description>
</property>
<property>
  <name>fs.defaultFS</name>
  <value>hdfs://localhost:9000</value>
</property>

<property>
  <name>io.file.buffer.size</name>
  <value>131072</value>
</property>
<property>
  <name>hadoop.tmp.dir</name>
  <value>file:/usr/local/hadoop_tmp</value>
  <description>Abasefor other temporary directories.</description>
</property>
<property>
  <name>hadoop.proxyuser.spark.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.spark.groups</name>
  <value>*</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat mapred-
mapred-env.cmd              mapred-queues.xml.template  mapred-site.xml~            
mapred-env.sh               mapred-site.xml             mapred-site.xml.template   
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$ cat mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
whx@whx-desktop:/usr/local/hadoop/etc/hadoop$

往事也加 发表于 2015-11-22 19:28
我安装的是hadoop2.6.0，在ubuntu14.04 64位的操作系统
下面是hadoop版本
whx@whx-desktop:/usr/local/ ...

我能加一下你的QQ不呢？

往事也加 发表于 2015-11-22 22:14
我能加一下你的QQ不呢？

我是安装这个网站：http://www.tuicool.com/articles/QvYVju 安装hadoop2.6.0的
但是在整合的时候报错
whx@whx-desktop:~$ jps
22702 Jps
15467 ResourceManager
22455 org.eclipse.equinox.launcher_1.3.0.dist.jar
15297 SecondaryNameNode
15791 NodeManager
32671 NameNode
whx@whx-desktop:~$ hadoop fs -mkdir input
15/11/22 21:31:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
mkdir: `input': No such file or directory
whx@whx-desktop:~$

这个网站：http://blog.sina.com.cn/s/blog_6d932f2a0101pgk3.html 使如何解决15/11/22 21:31:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable