主要参考官方文档进行安装。

安装与配置MySQL 数据库

$ sudo apt-get install mysql-server
$ sudo apt-get install libmysql-java

根据文档进行配置，使用InnoDB 引擎，添加并更新以下设置。

$ sudo vim /etc/mysql/my.cnf

[mysqld]
max_connections = 100

# InnoDB settings
innodb_additional_mem_pool_size = 16M
innodb_data_file_path = ibdata1:512M:autoextend
innodb_log_files_in_group = 2
transaction-isolation = READ-COMMITTED
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 1
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 512M
innodb_thread_concurrency = 5
innodb_flush_method = O_DIRECT
innodb_log_file_size = 256M

删除/var/lib/mysql 目录下的ib_logfile0、ib_logfile1 和ibdata1 文件后，重启mysql 服务。

登录MySQL，创建以下数据库并授权给用户hadoop。

create database amon DEFAULT CHARACTER SET utf8;
create database rman DEFAULT CHARACTER SET utf8;
create database metastore DEFAULT CHARACTER SET utf8;
create database sentry DEFAULT CHARACTER SET utf8;
create database nav DEFAULT CHARACTER SET utf8;
create database navms DEFAULT CHARACTER SET utf8;

grant all on amon.* TO 'hadoop'@'%' IDENTIFIED BY 'password';
grant all on rman.* TO 'hadoop'@'%' IDENTIFIED BY 'password';
grant all on metastore.* TO 'hadoop'@'%' IDENTIFIED BY 'password';
grant all on sentry.* TO 'hadoop'@'%' IDENTIFIED BY 'password';
grant all on nav.* TO 'hadoop'@'%' IDENTIFIED BY 'password';
grant all on navms.* TO 'hadoop'@'%' IDENTIFIED BY 'password';

create database oozie DEFAULT CHARACTER SET utf8;
grant all on oozie.* TO 'hadoop'@'%' IDENTIFIED BY 'password';

create database scm DEFAULT CHARACTER SET utf8;
grant all on scm.* TO 'hadoop'@'%' IDENTIFIED BY 'password';

安装Cloudera Manager

# master执行
$ mkdir ~/cdh_install
$ cd ~/cdh_install

$ wget https://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/cloudera.list
$ sudo cp cloudera.list /etc/apt/sources.list.d/
$ wget https://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/archive.key -O archive.key
$ sudo apt-key add archive.key

$ sudo apt-get update
$ sudo apt-get install oracle-j2sdk1.7
$ sudo apt-get install cloudera-manager-daemons cloudera-manager-server

如果要安装指定版本，需要修改cloudera.list 文件，默认安装的最新的版本号。

deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5.6.0 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5.6.0 contrib

运行脚本初始化数据库schema。

$ sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql scm hadoop

更改配置文件/etc/init.d/cloudera-scm-server，添加以下内容

USER=cloudera-scm
GROUP=cloudera-scm
MAX_PERMGEN=256
MAX_HEAP=2048

启动cloudera-scm-sever，等待几分钟后，可以通过浏览器访问http://localhost:7180 进入管理界面，用户名和密码默认为admin。

$ sudo service cloudera-scm-server start

使用Cloudera Manager 管理Agent 节点

进入管理界面后

• 选择安装Cloudera Express 版本。
• 添加集群内的所有节点（包括master），多个IP 地址使用逗号分隔。
• 选择Parcel 方式进行安装，同时添加Kafka parcel。
• 安装JDK。
• 启用Single User Mode。
• ssh 登录中选择devops 用户。

安装成功后，所有节点通过cloudera-scm-agent 进行管理。单用户模式下默认账号为cloudera-scm。根据官方文档进行单用户模式下的配置，以便之后的服务安装有足够的权限。

使用Cloudera Manger 安装服务

首先安装并启动Cloudera Management Service，在master 上配置Host Monitor、Service Monitor、Event Server、Alert Publisher。配置完成后可以监控到每台host 的健康状况。

由于各个服务之间有依赖关系，请按照以下顺序进行安装启动，并以相反的顺序停止服务。
ZooKeeper -> HDFS -> YARN ->HBase

因为启用了Single User Mode，在安装服务的过程中，要确保cloudera-scm 用户有权限访问相关的目录。

$ sudo mkdir -p /var/lib/cloudera-service-monitor
$ sudo mkdir -p /var/lib/cloudera-host-monitor
$ sudo mkdir -p /var/lib/cloudera-scm-eventserver
$ sudo mkdir -p /var/lib/cloudera-scm-navigator
$ sudo mkdir -p /var/lib/cloudera-scm-headlamp
$ sudo mkdir -p /var/lib/cloudera-scm-firehose
$ sudo mkdir -p /var/lib/cloudera-scm-alertpublisher
$ sudo mkdir -p /var/log/cloudera-service-monitor
$ sudo mkdir -p /var/log/cloudera-host-monitor
$ sudo mkdir -p /var/log/cloudera-scm-eventserver
$ sudo mkdir -p /var/log/cloudera-scm-navigator
$ sudo mkdir -p /var/log/cloudera-scm-headlamp
$ sudo mkdir -p /var/log/cloudera-scm-firehose
$ sudo mkdir -p /var/log/cloudera-scm-alertpublisher

$ sudo chown cloudera-scm:cloudera-scm /var/lib/cloudera-service-monitor
$ sudo chown cloudera-scm:cloudera-scm /var/lib/cloudera-host-monitor
$ sudo chown cloudera-scm:cloudera-scm /var/lib/cloudera-scm-eventserver
$ sudo chown cloudera-scm:cloudera-scm /var/lib/cloudera-scm-navigator
$ sudo chown cloudera-scm:cloudera-scm /var/lib/cloudera-scm-headlamp
$ sudo chown cloudera-scm:cloudera-scm /var/lib/cloudera-scm-firehose
$ sudo chown cloudera-scm:cloudera-scm /var/lib/cloudera-scm-alertpublisher
$ sudo chown cloudera-scm:cloudera-scm /var/log/cloudera-service-monitor
$ sudo chown cloudera-scm:cloudera-scm /var/log/cloudera-host-monitor
$ sudo chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-eventserver
$ sudo chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-navigator
$ sudo chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-headlamp
$ sudo chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-firehose
$ sudo chown cloudera-scm:cloudera-scm /var/log/cloudera-scm-alertpublisher

配置HDFS

$ sudo vim /etc/security/limits.conf

soft memlock unlimited
hard memlock unlimited

HDFS 默认的备份数为3，建议至少要有3个DataNode，否则会出现under replicated block 警告。

配置HIVE 数据库

create database hive DEFAULT CHARACTER SET utf8;
grant all on hive.* TO 'hadoop'@'%' IDENTIFIED BY 'password';

配置Kafka

更改csd.sh 文件提升调用命令的权限，将以下内容中的exec 更改为bash

sudo vim /usr/lib/cmf/service/csd/csd.sh

bash $CONF_DIR/$CM_CSD_SCRIPT "$@"

参考

Cloudera 官方安装步骤
Cloudera 官方MySQL 配置
Cloudera 官方Single User Mode 配置
在Ubuntu 14.04下通过Cloudera CDH 5.4.8搭建Hadoop 集群