Ubuntu Server 14部署CDH5.7实战教程

本帖最后由 levycui 于 2016-9-27 13:58 编辑
问题导读：
1、如何规划CDH集群节点？
2、如何安装配置CDH？
3、如何安装CDH所用数据库？
4、如何安装Cloudera Manager？

【环境】
① 物理主机
@Server       Wave P8000
@CPU          Intel Xeon CPU E5-2620 v3 2.40GHz * 2
@Memory       16GB * 8
@Storage       SAS 300GB * 3
@Network       Gigabit Ethernet Card * 6
@OS          Windows Server 2012 R2 DataCenter

② 集群环境
@OS          Ubuntu Server 14.04.5 LTS (64bit)
@JDK          JDK 1.7.0_67
@CDH          CDH 5.7.2
@Hadoop       Hadoop 2.6.0
@Spark       Spark 1.6.0

1. 节点系统安装配置
1.1 集群节点规划

硬件配置
共5个节点：1个master节点，4个slave节点，硬件配置如下：

网络配置

主机用户名和密码

服务用户名和密码

1.2 节点系统安装

在VMware中创建虚拟机，安装Ubuntu Server 14.04.5 LTS

创建虚拟机，设置CPU核心数、内存大小、存储大小。
添加虚拟网卡，采用桥接方式，桥接到物理网络。
加载Ubuntu Server 14.04.5 LTS系统镜像
启动系统，选择时区、语言，设置主机名和用户名、密码。
Partition disks配置步骤：
① 单独设置分区/dev/sda1，大小200MB，用于挂载/boot
选择手动配置-->选择硬盘-->创建新的分区表-->选择空闲空间-->新建分区-->设置大小200MB-->选择主分区-->选择开始位置-->设置文件系统格式Ext4-->设置挂载点/boot-->设置分区。
② 设置LVM，方便扩展存储，挂载/和swap
配置逻辑卷LVM-->将修改写入磁盘并配置LVM-->新建卷组-->卷组名称：vg_system-->按空格键选中剩余空闲空间/dev/sda-->将修改写入磁盘并配置LVM-->新建逻辑卷-->选择卷组vg_system-->逻辑卷名：lv_root-->设置逻辑卷大小：91GB-->新建逻辑卷-->选择卷组vg_system-->逻辑卷名：lv_swap-->设置逻辑卷大小为剩余空间-->完成。
③ 分别对每一个逻辑卷进行配置
磁盘格式：lv_root选择ext4，lv_swap选择swap area。
挂载点：lv_root选择"/"。
最后选择完成分区并将更改写入磁盘，点击确定即可。
选择服务器默认安装软件包
空格键选择OpenSSH Server，方便使用SSH传输文件。
安装Grub。
按照上述步骤，创建其他节点服务器并安装Ubuntu Server 14.04.5 LTS系统

1.3 节点系统基本环境配置

启用root用户，设置root密码
sudo passwd root

设置静态IP/DNS
注：以master节点为例
① 修改文件sudo vi /etc/network/interfaces，将默认生成的（下面内容可能没有，可能不一样，配置时视自己情况而定）：
[mw_shl_code=applescript,true]auto eth0
iface eth0 inet dhcp[/mw_shl_code]

改为：
[mw_shl_code=applescript,true]auto eth0
iface eth0 inet static
address 192.168.199.100 #IP地址
netmask 255.255.255.0 #子网掩码
gateway 192.168.199.1 #网关
dns-nameserver 192.168.199.1 #DNS[/mw_shl_code]

② 重启网卡，使配置生效：
[mw_shl_code=applescript,true]sudo service networking restart[/mw_shl_code]

注：若找不到上述命令或重启无效可尝试下面命令：

[mw_shl_code=applescript,true]sudo ifdown eth0
sudo ifdown eth1
sudo ifup eth0
sudo ifup eth1[/mw_shl_code]

注：若提示ifdown: interface eth0 not configured，可换下面命令尝试：
[mw_shl_code=applescript,true]sudo ifconfig eth0 down
sudo ifconfig eth1 down
sudo ifconfig eth0 up
sudo ifconfig eth1 up[/mw_shl_code]

③ 运行ifconfig -a查看网络配置信息。
④ 通过ping -c2 www.baidu.com测试能否正常访问外网。若不能，可sudo reboot重启节点试试。
⑤ 配置完成后，可在其他主机，通过Xshell连接master节点。
注：若XShell连接失败，提示Xshell no matching outgoing encryption，可升级Xshell版本,在链接的属性(SSH-->安全性)的加密算法列表中选择aes256-ctr，mac加密列表中选择hmac-sha2-256，保存即可。
注：slave节点配置方法类似

关闭防火墙
[mw_shl_code=applescript,true]sudo ufw disable[/mw_shl_code]

配置hosts文件
以master节点为例
要求所有的节点主机之间都是通过host互信的，编辑所有节点主机的/etc/hosts文件，配置完成后在每台主机上ping通其他主机名。
① 执行sudo vi /etc/hosts，添加如下内容
[mw_shl_code=applescript,true]192.168.199.100 master
192.168.199.101 slave1
192.168.199.102 slave2
192.168.199.103 slave3
192.168.199.104 slave4[/mw_shl_code]

② 注释掉除127.0.0.1之外的环回地址项，如下：
[mw_shl_code=applescript,true]#127.0.1.1    master[/mw_shl_code]
注：slave节点配置方法类似

服务器时区设置
① 执行sudo tzselect-->选择亚洲（Asia）-->选择中国（China)-->选择北京(Beijing)-->
② 复制文件到/etc目录下：
sudo cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
注：若在系统安装时已设置正确时区，可跳过该步骤。

设置区域语言
安装en_US.UTF-8：
[mw_shl_code=applescript,true]sudo locale-gen en_US.UTF-8[/mw_shl_code]
修改配置文件sudo vi /etc/default/locale如下：
[mw_shl_code=applescript,true]LANG="en_US.UTF-8"
LANGUAGE="en_US:en"
LC_ALL="en_US.UTF-8"[/mw_shl_code]

重新配置区域：
[mw_shl_code=applescript,true]sudo dpkg-reconfigure locales[/mw_shl_code]
注销或重启电脑，执行locale查看是否修改成功。

配置ubuntu的更新源
① 备份源列表
[mw_shl_code=applescript,true]sudo cp /etc/apt/sources.list /etc/apt/sources.list_backup[/mw_shl_code]

② 选择一个更新源，修改配置文件sudo vi /etc/apt/sources.list，添加新的源，国内推荐使用阿里源：
[mw_shl_code=applescript,true]       deb http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
      deb http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
      deb http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
      deb http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
      deb http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse
      deb-src http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
      deb-src http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
      deb-src http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
      deb-src http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
      deb-src http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse[/mw_shl_code]

更新系统
① 更新系统
[mw_shl_code=applescript,true]       sudo apt update
      sudo apt upgrade
      sudo apt dist-upgrade
      sudo apt autoremove[/mw_shl_code]

可能遇到的问题：
Ubuntu系统/boot容量不足引起的无法更新系统
解决方法：
查看/boot分区容量df -h /boot；查看当前使用内核版本号uname -a；删除旧内核，输入sudo apt-get remove linux-image-，接着按两下tab键，将显示所有的内核版本，把目前使用的版本之前的版本全部删除。最后执行
[mw_shl_code=applescript,true]       sudo update-grub
      sudo grub-mkconfig -o /boot/grub/grub.cfg[/mw_shl_code]

② 查看系统发行版本和内核版本
[mw_shl_code=applescript,true]       ubuntu@master:~$ sudo lsb_release -a
      No LSB modules are available.
      Distributor ID: Ubuntu
      Description: Ubuntu 14.04.5 LTS
      Release: 14.04
      Codename: trusty
      ubuntu@master:~$ uname -srpi
      Linux 4.4.0-36-generic x86_64 x86_64[/mw_shl_code]

免密码SSH登录
① 在每个节点上生成密钥对：
[mw_shl_code=applescript,true]       ssh-keygen -t rsa[/mw_shl_code]

② 将每个节点（包括master和slave）上的公钥复制到master节点，执行如下命令，输入master登录密码，完成复制：
[mw_shl_code=applescript,true]       ssh-copy-id -i ~/.ssh/id_rsa.pub master[/mw_shl_code]

③ 查看所有节点公钥是否已经添加到了master节点，在master节点执行：
[mw_shl_code=applescript,true]       cat ~/.ssh/authorized_keys[/mw_shl_code]

④ 把master节点的authorized_keys发送到所有slave节点，在master节点执行：
[mw_shl_code=applescript,true]       scp ~/.ssh/authorized_keys ubuntu@slave1:/home/ubuntu/.ssh/
      scp ~/.ssh/authorized_keys ubuntu@slave2:/home/ubuntu/.ssh/
      scp ~/.ssh/authorized_keys ubuntu@slave3:/home/ubuntu/.ssh/
      scp ~/.ssh/authorized_keys ubuntu@slave4:/home/ubuntu/.ssh/[/mw_shl_code]

⑤ 在所有节点上修改ssh配置文件使root用户也能免密码登录：
修改配置文件sudo vi /etc/ssh/sshd_config
[mw_shl_code=applescript,true]       PermitRootLogin yes[/mw_shl_code]

统一时间源，安装NTP服务
① 在master节点上安装NTP服务：sudo apt-get install chrony。
重启NTP使配置生效：sudo service chrony restart
注：确保网络时间服务器IP地址可以被DNS解析，如果不能，则在/etc/hosts文件中手工加入。
② 在所有slave节点上安装NTP服务：sudo apt-get install chrony。
修改配置文件sudo vi /etc/chrony/chrony.conf，删除全部默认设置，只添加如下内容：
[mw_shl_code=applescript,true]       server master iburst[/mw_shl_code]

重启NTP服务：sudo service chrony restart
③ 验证是否配置正确：
[mw_shl_code=applescript,true]       chronyc sources -v [/mw_shl_code]

2. 安装配置CDH

Cloudera官方给出了三种安装方法：

Path A - Automated Installation by Cloudera Manager (Non-Production Mode)
Path B - Installation Using Cloudera Manager Parcels or Packages
Path C - Manual Installation Using Cloudera Manager Tarballs

注：本文采用第2种方法，手动安装JDK、MariaDB Database、Cloudera Manager Software Package。

Cloudera服务组件：
① Cloudera Express Version
      • Cloudera Manager - Contains all the information about services you have configured and their role assignments, all configuration history, commands, users, and running processes.
      • Oozie Server - Contains Oozie workflow, coordinator, and bundle data. Can grow very large.
      • Sqoop Server - Contains entities such as the connector, driver, links and jobs. Relatively small.
      • Activity Monitor - Contains information about past activities. In large clusters, this database can grow large. Configuring an Activity Monitor database is only necessary if a MapReduce service is deployed.
      • Hive Metastore Server - Contains Hive metadata. Relatively small.
      • Sentry Server - Contains authorization metadata. Relatively small.

② Cloudera Enterprise Version
      • Reports Manager - Tracks disk utilization and processing activities over time. Medium-sized.
      • Cloudera Navigator Audit Server - Contains auditing information. In large clusters, this database can grow large.
      • Cloudera Navigator Metadata Server - Contains authorization, policies, and audit report metadata. Relatively small

2.1 Phase 1：安装Oracle JDK 7

在所有节点安装JDK 1.7

配置Cloudera软件仓库
Cloudera官方文档提供了两种管理仓库策略：
• Standard Cloudera repositories
• Internally hosted repositories

本文采用Standard Cloudera repositories，步骤如下：
① 在master节点下载cloudera.list
[mw_shl_code=applescript,true]       wget https://archive.cloudera.com/cm5 ... 13753785.1471598714[/mw_shl_code]

② 在master节点新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list，复制cloudera.list的内容
[mw_shl_code=applescript,true]       sudo cp ~/cloudera.list /etc/apt/sources.list.d/cloudera-manager.list[/mw_shl_code]

③ 将cloudera-manager.list分发到所有slave节点：
[mw_shl_code=applescript,true]       scp /etc/apt/sources.list.d/cloudera-manager.list root@slave1:/etc/apt/sources.list.d/cloudera-manager.list
      scp /etc/apt/sources.list.d/cloudera-manager.list root@slave2:/etc/apt/sources.list.d/cloudera-manager.list
      scp /etc/apt/sources.list.d/cloudera-manager.list root@slave3:/etc/apt/sources.list.d/cloudera-manager.list
      scp /etc/apt/sources.list.d/cloudera-manager.list root@slave4:/etc/apt/sources.list.d/cloudera-manager.list[/mw_shl_code]

④ 向apt添加cloudera源公钥，在所有节点执行如下命令：
[mw_shl_code=applescript,true]       curl -s http://archive.cloudera.com/cm5/ ... md64/cm/archive.key | sudo apt-key add -[/mw_shl_code]

⑤ 在所有节点，更新源
      sudo apt-get update

若更新出错，提示如下：
W: GPG error: http://archive.cloudera.com trusty-cm5 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 327574EE02A818DD

可执行如下命令，添加cloudera源的公钥，重新更新源：
[mw_shl_code=applescript,true]curl -s http://archive.cloudera.com/cm5/ ... md64/cm/archive.key | sudo apt-key add -[/mw_shl_code]

安装JDK1.7
CDH要求至少是Oracle JDK7，Ubuntu自带的是OpenJDK，Ubuntu没有自带的Oracle JDK,必须添加外部源才能安装JDK。
① 采用Cloudera软件源
在上一步配置Cloudera软件源后，可直接执行下面命令安装：
[mw_shl_code=applescript,true]       sudo apt-get install oracle-j2sdk1.7[/mw_shl_code]

② 采用PPP源
添加PPA repository到系统,配置PPP源安装：
[mw_shl_code=applescript,true]       sudo add-apt-repository ppa:webupd8team/java
      sudo apt-get update
      sudo apt-get upgrade
      sudo apt-get install oracle-java7-installer
      sudo apt-get install oracle-java7-set-default
      java -version[/mw_shl_code]

注：若提示如下信息：
sha256sum mismatch jdk-7u80-linux-x64.tar.gz
Oracle JDK 7 is NOT installed.

可能是jdk软件包没有下载完全，可执行如下命令删除后重新安装：
[mw_shl_code=applescript,true]       sudo apt-get purge oracle-java7-installer[/mw_shl_code]

③ 手动安装
从Oracle官网下载JDK7u80：jdk-7u80-linux-x64.tar.gz，解压到/usr/java/jdk.1.7.0_80
[mw_shl_code=applescript,true]       mkdir /usr/java
      sudo tar -zxvf jdk-7u80-linux-x64.tar.gz -C /usr/java[/mw_shl_code]

配置JDK环境变量：
在~/.bashrc最后添加：
[mw_shl_code=applescript,true]       export JAVA_HOME=/usr/java/jdk.1.7.0_80[/mw_shl_code]

再执行：
[mw_shl_code=applescript,true]       source ~/.bashrc[/mw_shl_code]

2.2 Phase 2：安装数据库

需要安装：MariaDB和PostgreSQL Database

安装MariaDB
注：只有master节点安装MariaDB
[mw_shl_code=applescript,true]       sudo apt-get install mariadb-server[/mw_shl_code]

注：提示输入root密码时，可直接选择OK，跳过，在后面配置。

配置和启动MariaDB Server
① 停止MariaDB Server
[mw_shl_code=applescript,true]       sudo service mysql stop[/mw_shl_code]

② 备份旧的InnoDB日志文件/var/lib/mysql/ib_logfile0和/var/lib/mysql/ib_logfile1
[mw_shl_code=applescript,true]       sudo mv /var/lib/mysql/ib_logfile0 backup_location #备份位置
      sudo mv /var/lib/mysql/ib_logfile1 backup_location[/mw_shl_code]

③ 新建文件sudo vi /etc/mysql/conf.d/cloudera.cnf，添加如下配置信息：
[mw_shl_code=applescript,true]       [mysqld]
      transaction-isolation = READ-COMMITTED
      # Disabling symbolic-links is recommended to prevent assorted security risks;
      # to do so, uncomment this line:
      # symbolic-links = 0
      bind-address = master # master hostname
      default-storage-engine = innodb
      collation-server = utf8_general_ci
      character-set-server = utf8
      key_buffer = 16M
      key_buffer_size = 32M
      max_allowed_packet = 32M
      thread_stack = 256K
      thread_cache_size = 64
      query_cache_limit = 8M
      query_cache_size = 64M
      query_cache_type = 1
      max_connections = 550
      #expire_logs_days = 10
      #max_binlog_size = 100M
      #log_bin should be on a disk with enough free space. Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your system
      #and chown the specified folder to the mysql user.
      log_bin=/var/lib/mysql/mysql_binary_log
      binlog_format = mixed
      read_buffer_size = 2M
      read_rnd_buffer_size = 16M
      sort_buffer_size = 8M
      join_buffer_size = 8M
      # InnoDB settings
      innodb_file_per_table = 1
      innodb_flush_log_at_trx_commit  = 2
      innodb_log_buffer_size = 64M
      innodb_buffer_pool_size = 4G
      innodb_thread_concurrency = 8
      innodb_flush_method = O_DIRECT
      innodb_log_file_size = 512M
      [mysqld_safe]
      log-error=/var/log/mysqld.log
      pid-file=/var/run/mysqld/mysqld.pid[/mw_shl_code]

④ 重新启动MariaDB数据库，确保MariaDB开机启动：
[mw_shl_code=applescript,true]       sudo service mysql restart
      sudo service mysql status
      netstat -ntlp #查看端口监听情况[/mw_shl_code]

⑤ 设置MariaDB数据库root用户密码：
执行sudo /usr/bin/mysql_secure_installation，开启MySQL安全保护措施，根据提示输入MariaDB数据库root用户密码，根据实际需要选择后续操作。
[mw_shl_code=applescript,true]       $
      [...]
      Enter current password for root (enter for none):
      OK, successfully used password, moving on...
      [...]
      Set root password? [Y/n] Y
      New password:
      Re-enter new password:
      Remove anonymous users? [Y/n] Y
      [...]
      Disallow root login remotely? [Y/n] N
      [...]
      Remove test database and access to it [Y/n] Y
      [...]
      Reload privilege tables now? [Y/n] Y
      All done![/mw_shl_code]

⑥ 验证MariaDB是否安装配置正确
[mw_shl_code=applescript,true]       mysql -uroot -p
      mysql -h localhost -uroot -p[/mw_shl_code]

⑦ 授权root可以访问所有数据库：
[mw_shl_code=applescript,true]       mysql -uroot -proot_password
      grant all privileges on *.* to 'root'@'%' identified by 'root_password' with grant option;
      flush privileges;[/mw_shl_code]

安装MariaDB JDBC Driver
Cloudera推荐MariaDB数据库使用MySQL JDBC驱动。
[mw_shl_code=applescript,true]       sudo apt-get install libmysql-java[/mw_shl_code]

安装PostgreSQL Database
① 设置区域
[mw_shl_code=applescript,true]       export LANGUAGE=en_US.UTF-8
      export LANG=en_US.UTF-8
      export LC_ALL=en_US.UTF-8
      sudo locale-gen en_US.UTF-8
      sudo dpkg-reconfigure locales[/mw_shl_code]

注：上述方法好像无效，安装数据库时，还是原来系统区域语言。
② 安装PostgreSQL软件包
[mw_shl_code=applescript,true]       sudo apt-get install postgresql[/mw_shl_code]

③ 配置并启动PostgreSQL Server
注：默认第一次启动PostgreSQL会自动初始化，有的版本若第一次启动不进行初始化，可显示初始化：
[mw_shl_code=applescript,true]       sudo service postgresql initdb[/mw_shl_code]

修改配置文件sudo vi /etc/postgresql/9.3/main/pg_hba.conf，启用MD5认证，将行
[mw_shl_code=applescript,true]       host all all 127.0.0.1/32 md5[/mw_shl_code]

修改为：
[mw_shl_code=applescript,true]       host all all 192.168.199.100/32 md5 //指定主机IP[/mw_shl_code]

或：
[mw_shl_code=applescript,true]       host all all master md5 //指定主机名[/mw_shl_code]

注：若默认配置文件中存在行host all all 127.0.0.1/32 ident，则需确保行host all all 192.168.199.100/32 md5在其之前。
修改配置文件sudo vi /etc/postgresql/9.3/main/postgresql.conf，添加：

[mw_shl_code=applescript,true]       listen_addresses = '*'[/mw_shl_code]

同时根据需要修改其他默认配置，如对于中小型集群可设置：
[mw_shl_code=applescript,true]       shared_buffers = 128MB
      wal_buffers = 8MB
      checkpoint_segments = 16
      checkpoint_completion_target = 0.9[/mw_shl_code]

④ 重启PostgreSQL Database
[mw_shl_code=applescript,true]       sudo service postgresql restart[/mw_shl_code]

⑤ 设置PostgresSQL默认用户postgres登录密码
[mw_shl_code=applescript,true]       sudo su - postgres
      psql
      \password postgres
      \q[/mw_shl_code]

2.3 Phase 3：安装Cloudera Manager Server

安装Cloudera Manager Server软件包
在master节点上执行：
[mw_shl_code=applescript,true]       sudo apt-get install cloudera-manager-daemons cloudera-manager-server [/mw_shl_code]

创建Cloudera Manager Server数据库scm

[mw_shl_code=applescript,true]       mysql -u root -p
      create database scm DEFAULT CHARACTER SET utf8;
      grant all on scm.* TO 'scm'@'%' IDENTIFIED BY 'scm_password';
      flush privileges;[/mw_shl_code]

配置启动Cloudera Manager Server
[mw_shl_code=applescript,true]       sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm scm scm_password
      sudo service cloudera-scm-server start[/mw_shl_code]

若启动报错：

/etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM PermGen is set less than 256m, CM server may run out of PermGen space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")

/etc/init.d/cloudera-scm-server: line 292: [[: [error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.: syntax error: operand expected (error token is "[error] JVM MaxHeap is set less than 2G, CM server may run out of Heap space. Update CMF_JAVA_OPTS in /etc/default/cloudera-scm-server to fix this.")

修改配置文件/etc/default/cloudera-scm-server，找到export CM_JAVA_OPTS部分，将-Xmx2G修改为-Xmx4G

查看日志，验证是否成功启动
[mw_shl_code=applescript,true]       tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log[/mw_shl_code]

2.4 Phase 4：安装Cloudera Manager Agents

安装Cloudera Manager Agent软件包
在所有节点（包括master）上执行：
[mw_shl_code=applescript,true]       sudo apt-get install cloudera-manager-agent cloudera-manager-daemons[/mw_shl_code]

配置Cloudera Manager Agent
在所有节点上，修改配置文件sudo vi /etc/cloudera-scm-agent/config.ini，配置Cloudera Manager Server的主机和端口：
[mw_shl_code=applescript,true]       server_host=master
      server_port=7182 #默认端口即可[/mw_shl_code]

启动Cloudera Manager Agent
[mw_shl_code=applescript,true]       sudo service cloudera-scm-agent start[/mw_shl_code]

2.5 Phase 5：安装CDH和管理服务软件
登录Cloudera Manager Admin Console
URL：http://192.168.199.100:7180 (http://master:7180)
用户名：admin
密码：admin
选择Cloudera Manager Edition：Cloudera Express

为CDH集群安装指定主机
前面已经启动Cloudera Manager Server & Agent，若所有配置正确，在已经管理的主机中应该可以看到所有主机。勾选所有主机，点击“继续”，进入Cluster Installation Select Repository页。
注：若没有看到主机，可查看Cloudera Manager Agent服务是否正常启动，或者指定IP或主机名匹配模式进行搜索。

选择软件安装类型并安装软件
① 推荐采用Parcels安装类型。
② 点击“更多选项”，可以配置Parcel目录（修改后需重启所有主机Cloudera Manager Agent服务）、本地Parcels库路径和远程Parcels库路径。
③ 删除系统默认的远程Parcels库路径，只添加一条远程Parcels库路径：https://archive.cloudera.com/cdh5/parcels/5.7.2/，安装CDH5.7.2
④ CDH版本选择CDH-5.7.2-1.cdh5.7.2.p0.18，点击“继续”，进入Installing Selected Parcels页，开始安装所选软件包。
注：由于前面手动安装了JDK，所以不会弹出Install Oracle Java SE Development Kit (JDK)页，直接进入Installing Selected Parcels页。
⑤ 完成后，CDH5.7.2软件包将下载分发到各节点。

检查主机正确性
点击“主机”-->“所有主机”-->“检查所有主机”（右上角）-->“查看检查结果”，确认所有项都通过检查，没有警告和错误。若出现如下警告：
① Cloudera 建议将/proc/sys/vm/swappiness 设置为 0。当前设置为 30。
在所有节点主机上，修改配置文件sudo vi /etc/sysctl.conf，设置vm.swappiness = 0，最后执行sudo sysctl -p。
② 已启用“透明大页面”，它可能会导致重大的性能问题。
[mw_shl_code=applescript,true]
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
sudo vi /etc/rc.local
echo never >/sys/kernel/mm/transparent_hugepage/enabled
echo never >/sys/kernel/mm/transparent_hugepage/defrag[/mw_shl_code]

完成以上步骤之后，就可以进行集群配置了。

2.6 Phase 6：创建配置启动CDH和相关服务

添加服务Cloudera Management Service
点击“集群”-->选择一个“Cluster 1”-->“添加Cloudera管理服务”（右上角）-->选择默认配置安装。
注：若首页显示Unable to issue query: could not connect to the Host Monitor，则是因为没有安装Cloudera Management Service。

添加Activity Monitor服务
① 创建Activity Monitor服务数据库amon
[mw_shl_code=applescript,true]mysql -u root -p
create database amon DEFAULT CHARACTER SET utf8;
grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
flush privileges;[/mw_shl_code]

② 配置过程需指定数据库amon及用户amon。

添加Core Hadoop服务
注：选择合适的主机分配策略
① HDFS
若Namenode提示一直处于元数据升级中：
Concerning : NameNode summary: master (Availability: Active, Health: Concerning). This health test reflects the health of the active NameNode.

可在Actions中手动执行Finalize Metadata Upgrade。
② YARN (MR2 Included)
③ ZooKeeper
④ Oozie
创建Oozie服务数据库：

[mw_shl_code=applescript,true]mysql -uroot -p
create database oozie DEFAULT CHARACTER SET utf8;
grant all on oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie_password';
grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie_password';
flush privileges;[/mw_shl_code]

复制MySQL JDBC JAVA驱动
[mw_shl_code=applescript,true]sudo cp /usr/share/java/mysql-connector-java-5.1.28.jar /var/lib/oozie[/mw_shl_code]

配置过程需指定数据库oozie及用户oozie。
⑤ Hive
创建Hive Metadata服务数据库：
[mw_shl_code=applescript,true]mysql -uroot -p
create database metastore DEFAULT CHARACTER SET utf8;
grant all on metastore.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';
flush privileges;[/mw_shl_code]

配置过程需指定数据库metastore及用户hive。
⑥ Hue
添加服务Hue，然后配置Hue Server使用MariaDB数据库。
详见：Configuring the Hue Server to Store Data in MariaDB
创建Hue Server服务数据库：
[mw_shl_code=applescript,true]create database hue DEFAULT CHARACTER SET utf8;
grant all on hue.* TO 'hue'@'%' IDENTIFIED BY 'hue_password';[/mw_shl_code]

注：在执行完Actions > Synchronize Database后，官方指南略有差别，登录时需要指定主机名，然后选择hue数据库，如下：

[mw_shl_code=applescript,true]mysql -h hostname -uhue -p
use hue;
SHOW CREATE TABLE auth_permission;
ALTER TABLE auth_permission DROP FOREIGN KEY content_type_id_refs_id_XXXXXX; //仅针对InnoDB
DELETE FROM hue.django_content_type;[/mw_shl_code]

在Load Database后，执行：
ALTER TABLE auth_permission ADD FOREIGN KEY (content_type_id) REFERENCES django_content_type (id);

特别提示：本文Hue该部分只列出了配置的关键步骤，详细步骤请查看官方指南。

HBase
Impala
Cloudera Search：Solr
Spark
Key-Value Store Indexer
Sqoop 2

① 创建Sqoop服务数据库和用户：
[mw_shl_code=applescript,true]sudo su - postgres
psql -U postgres
CREATE ROLE sqoop LOGIN ENCRYPTED PASSWORD 'sqoop_password' NOSUPERUSER INHERIT CREATEDB NOCREATEROLE;
CREATE DATABASE "sqoop" WITH OWNER = sqoop  ENCODING = 'UTF8' TABLESPACE = pg_default LC_COLLATE = 'en_US.UTF8' LC_CTYPE = 'en_US.UTF8' CONNECTION LIMIT = -1;
\q[/mw_shl_code]

注：创建数据库若提示错误：

ERROR:  new collation (en_US.UTF8) is incompatible with the collation of the template database (en_HK.UTF-8)
HINT:  Use the same collation as in the template database, or use template0 as template.

可执行如下命令：

[mw_shl_code=applescript,true]CREATE DATABASE "sqoop" WITH OWNER = sqoop  ENCODING = 'UTF8' TABLESPACE = pg_default LC_COLLATE = 'en_US.UTF8' LC_CTYPE = 'en_US.UTF8' CONNECTION LIMIT = -1 TEMPLATE template0;[/mw_shl_code]

② 为Sqoop配置PostgresSQL Database
详见：Configure Sqoop 2 to use PostgreSQL
注： Sqoop需要部署在PostgresSQL所在节点，测试发现部署在其他节点会报错。

Sentry
① 创建数据库

[mw_shl_code=applescript,true]mysql -uroot -p
create database sentry DEFAULT CHARACTER SET utf8;
grant all on sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry_password';
flush privileges;[/mw_shl_code]

② 在Cloudera Manager中添加Sentry服务，选择服务安装节点，若选择除master之外的其他slave节点，则需要在该slave节点上安装MySQL JDBC驱动，否则提示没有MySQL JDBC驱动。如下：
[mw_shl_code=applescript,true]sudo apt-get install libmysql-java[/mw_shl_code]

Kafka
① 在Parcels Configuration中配置Kafka远程库路径：https://archive.cloudera.com/kafka/parcels/latest/，然后下载、分发、激活Kafka软件包。
② 在集群配置中，添加Kafka服务。
若提示，或者错误日志中提示Java Heap空间太小：
Missing required value: Destination Broker List
Missing required value: Source Broker List

可按如下方法配置后重试即可：
a. 填写Source Brokers List
填写Kafka Broker所在节点构成的列表（用逗号分隔），如下（本文在所有节点部署了Kafka Broker）：
master:9092,slave1:9092,slave2:9092,slave3:9092,slave4:9092

b. 填写Destination Brokers List
若添加了Kafka MirrorMaker，则可填写其所在节点构成的列表；若未添加Kafka MirrorMaker，可填写任意服务器即可，如下：
master:9092,slave1:9092,slave2:9092,slave3:9092,slave4:9092
或：
example.com:9092

c. 修改Java Heap Size
填写上面列表后，点击继续，出错后，Kafka服务未启动。返回集群配置，打卡Kafka服务配置页，查找“Java Heap Size of Broker”项，将对大小从50MB修改为256MB。
d. 配置Topic Whitelist
配置Topic Whitelist项为正则表达式：(?!x)x，保存更改。然后添加角色实例，重新配置即可。
参考链接：adding a Kafka service failed

1) Fill in "Source Brokers List" --> "nameOfClouderaKafkaBrokerServer.yourdomain.com:9092". This is the Server (or Servers) where you configured the Kafka Broker (NOT the MirrorMaker).

2) Fill in "Destination Brokers List" --> "nameOfRemoteBrokerServer.otherdomain.com:9092". This is supposed to be a remote Cluster that will receive Topics sent over by your MirrorMaker. If you have one, put in that one. Otherwise just put in another Server in your network, whatever Server.

Please note that both this Server Names must be FQDN and resolvable by your DNS (or hosts file), otherwise you'll get other errors. Also the format with the trailing Port Number is mandatory!

3) Click "Continue". Service will NOT start (error). Do not navigate away from that screen

4) Open another Cloudera Manager in another browser pane. You should now see "Kafka" in the list of Services (red, but it should be there). Click on the Kafka Service and then "Configure".

5) Search for the "java heap space" Configuration Property. The standard Java Heap Space you'll find already set up should be 50 MBytes. Put in at least 256 MBytes. The original value is simply not enough.

6) Now search for the "whitelist" Configuration Property. In the field, put in "(?!x)x" (without the quotation marks). That's a regular expression that does not match anything. Given that apparently a Whitelist is mandatory for the Mirrormaker Service to start, and I'm assuming you don't want to replicate any topics remotely right now, just put in something that won't replicate anything e.g. that regular expression.

7) Save the changes and go back to the original Configuration Screen on the othe browser pane. Click "Retry", or wathever, or even exit that screen and manually restart the Kafka Service in Cloudera Manager.

3. 测试

MapReduce
[mw_shl_code=applescript,true]sudo -uhdfs hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 20 10[/mw_shl_code]

Spark
[mw_shl_code=applescript,true]sudo -uhdfs spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client /opt/cloudera/parcels/CDH/lib/spark/lib/spark-examples-1.6.0-cdh5.7.2-hadoop2.6.0-cdh5.7.2.jar[/mw_shl_code]

打开http://master:8088可以查看Job执行情况。

4. 遇到的问题
4.1 apt update无法更新Ubuntu Server 14.04

问题
执行sudo apt update，提示如下错误Packages Hash Sum mismatch：
......
W: Failed to fetch http://mirrors.aliyun.com/ubuntu ... inary-i386/Packages  Hash Sum mismatch
W: Failed to fetch http://mirrors.aliyun.com/ubuntu ... inary-i386/Packages  Hash Sum mismatch
W: Failed to fetch http://mirrors.aliyun.com/ubuntu ... i18n/Translation-en  Hash Sum mismatch
W: Failed to fetch http://mirrors.aliyun.com/ubuntu ... i18n/Translation-en  Hash Sum mismatch
E: Some index files failed to download. They have been ignored, or old ones used instead.

原因
① 网上有的帖子写到可能和以前本地缓存冲突，可删除本地缓存的软件源列表信息，sudo rm -rf /var/lib/apt/lists/*，但该方法经测试无效。
② 最后发现是由于网络运营商的HTTP缓存，没有及时更新，导致用户更新 Ubuntu 系统时从ISP缓存下载软件包，报错「校验和不符」。

解决方法
① 使用HTTP代理：
如果是设置只对当前会话有效的临时代理，执行：
[mw_shl_code=applescript,true]export http_proxy=http://yourproxyaddress:proxyport
sudo apt-get update
sudo apt-get upgrade[/mw_shl_code]

如果要设置持久代理，编辑sudo vi /etc/apt/apt.conf，添加一行：
Acquire::http::Proxy "http://yourproxyaddress:proxyport";

注：经测试，我的环境下，只有这种设置持久代理的方法才有效。
参考链接：XX-NET代理配置
如果没有HTTP代理，只有SOCKS代理，首先安装proxychains程序，编辑/etc/proxychains.conf，指定SOCKS服务器的IP地址和端口，执行：
[mw_shl_code=applescript,true]sudo proxychains apt-get update
sudo apt-get upgrade[/mw_shl_code]

② 换协议：
网络服务商只使用了HTTP缓存，如果软件源还支持HTTPS或者FTP协议，修改/etc/apt/sources.list，把其中所有的http://换成ftp://，再执行系统更新。
注：软件源不支持FTP协议访问，Ubuntu主服务器支持，但是网络速度会比较慢。
③ 修改apt包的源代码：更新 Ubuntu 系统，避免报错「校验和不符」

来源：https://www.zybuluo.com/ncepuwanghui/note/474966
作者：Sam Wong

lingyufeng · 发表于 2016-9-28 10:02:12

谢谢分享，很详细

若曦难得 · 发表于 2016-9-28 10:39:55

很好啊

huj427 · 发表于 2016-10-3 11:24:38

本帖最后由 huj427 于 2016-10-3 11:36 编辑

have a look
thanks for sharing
good done~~~~~~~~~~~
硬件配置
共5个节点：1个master节点，4个slave节点，硬件配置如下：

master:24gb
slave:16gb
这个是必须的吗？
最少需要多少？