lzw 发表于 2013-12-22 18:35:15

hadoop 伪分布式搭建

下载hadoop1.0.4版本,和jdk1.6版本或更高版本:
1. 安装JDK,安装目录大家可以自定义,下面是我的安装目录:/usr/jdk1.6.0_22配置环境变量:# vi /etc/profile将环境变量添加到profile文件底部:export JAVA_HOME=/usr/jdk1.6.0_22
export PATH=$PATH:$JAVA_HOME/bin激活环境变量:# source /etc/profile2. 安装hadoop
创建hadoop用户及其hadoop组。
配置IP对应域名在$ vi /etc/hosts在hosts文件底部中添加如下:192.168.0.101   hadoop.master接下来下载hadoop安装包,大家可以到apache官网下载1.0.4版本
我这里将hadoop安装到hadoop用户目录下,解压hadoop安装文件,hadoop有三个核心配置文件core-site.xml,hdfs-site.xml,mapred-site.xml,
配置core-site.xml<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://hadoop.master:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
</property>
</configuration>配置hdfs-site.xml<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
   <name>dfs.replication</name>
   <value>1</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/hdfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/hadoop/hdfs/data</value>
</property>
</configuration>配置 mapred-site.xml<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
<property>
   <name>mapred.job.tracker</name>
   <value>hadoop.master:9001</value>
</property>
</configuration>最后配置haoop-env.sh文件# Set Hadoop-specific environment variables here.

# The only required environment variable is JAVA_HOME.All others are
# optional.When running a distributed configuration it is best to
# set JAVA_HOME in this file, so that it is correctly defined on
# remote nodes.

# The java implementation to use.Required.
export JAVA_HOME=/usr/jdk1.6.0_22

# Extra Java CLASSPATH elements.Optional.
export HADOOP_HOME=/home/hadoop/hadoop-1.0.4
# export HADOOP_CLASSPATH=

# The maximum amount of heap to use, in MB. Default is 1000.
# export HADOOP_HEAPSIZE=2000

# Extra Java runtime options.Empty by default.
# export HADOOP_OPTS=-server

# Command specific options appended to HADOOP_OPTS when specified
export HADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote $HADOOP_NAMENODE_OPTS"
export HADOOP_SECONDARYNAMENODE_OPTS="-Dcom.sun.management.jmxremote $HADOOP_SECONDARYNAMENODE_OPTS"
export HADOOP_DATANODE_OPTS="-Dcom.sun.management.jmxremote $HADOOP_DATANODE_OPTS"
export HADOOP_BALANCER_OPTS="-Dcom.sun.management.jmxremote $HADOOP_BALANCER_OPTS"
export HADOOP_JOBTRACKER_OPTS="-Dcom.sun.management.jmxremote $HADOOP_JOBTRACKER_OPTS"
# export HADOOP_TASKTRACKER_OPTS=
# The following applies to multiple commands (fs, dfs, fsck, distcp etc)
# export HADOOP_CLIENT_OPTS

# Extra ssh options.Empty by default.
# export HADOOP_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HADOOP_CONF_DIR"

# Where log files are stored.$HADOOP_HOME/logs by default.
# export HADOOP_LOG_DIR=${HADOOP_HOME}/logs

# File naming remote slave hosts.$HADOOP_HOME/conf/slaves by default.
# export HADOOP_SLAVES=${HADOOP_HOME}/conf/slaves

# host:path where hadoop code should be rsync'd from.Unset by default.
# export HADOOP_MASTER=master:/home/$USER/src/hadoop

# Seconds to sleep between slave commands.Unset by default.This
# can be useful in large clusters, where, e.g., slave rsyncs can
# otherwise arrive faster than the master can service them.
# export HADOOP_SLAVE_SLEEP=0.1

# The directory where pid files are stored. /tmp by default.
# export HADOOP_PID_DIR=/var/hadoop/pids

# A string representing this instance of hadoop. $USER by default.
# export HADOOP_IDENT_STRING=$USER

# The scheduling priority for daemon processes.See 'man nice'.
# export HADOOP_NICENESS=10

export PATH=$PATH:$HADOOP_HOME/bin最后在配置hadoop环境变量在跟jdk在同一个文件,在jdk环境变量下面增加hadoop环境变量:export HADOOP_HOME=/home/hadoop/hadoop-1.0.4
export PATH=$PATH:$HADOOP_HOME/bin到这里所有环境配置已经完成:
再一次启动时需要namenode format,命令如下:$ hadoop namenode -format格式完成后,启动hadoop,使用命令$ ./start-all.sh使用jps查看hadoop启动的守护进程:$ jps
2734 NameNode
2860 DataNode
2996 SecondaryNameNode
3090 JobTracker
3261 TaskTrackerhadoop 伪分布式安装已经完成。





兰君云 发表于 2014-3-4 23:12:26

windows下如何搭建?

weijia365 发表于 2014-5-6 17:32:06

顶顶更健康!

weijia365 发表于 2014-5-6 17:34:07

初学者,顶起!

weijia365 发表于 2014-5-6 23:03:30

顶顶更健康!

旋风来袭 发表于 2014-6-3 11:33:23

还是不太懂,我是个新手,可以详细点吗??

ascentzhen 发表于 2014-7-18 14:52:06

官方文档建议在非root下部署,不过作为练习环境可以了

跃阳紫 发表于 2014-7-29 09:07:05

初学者飘过

hery 发表于 2014-9-5 14:36:07

回顾一下,重点点出来了

monkey_d 发表于 2014-12-24 13:46:27

需要搭建双节点吗
页: [1] 2 3
查看完整版本: hadoop 伪分布式搭建