Cloudera Hadoop 离线安装

1,前期环境检查

• hosts文件
• jdk版本
• 主机操作系统是否与CDH版本匹配
• 检查主机hostname是否合法(主机域名不能带'_',血泪史。)
• ntp时间同步
• 防火墙(iptables)

官方支持版本检查文档
https://www.cloudera.com/documentation/enterprise/release-notes/topics/rn_consolidated_pcm.html#cm_cdh_compatibility

2,下载cloudera manager相关安装包

官方地址:http://archive.cloudera.com/cm5/

下载包列表

(以5.9.0为例,完整地址:http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.9.0/RPMS/x86_64/)

• cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el7.x86_64.rpm
• cloudera-manager-server-5.9.0-1.cm590.p0.249.el7.x86_64.rpm
• cloudera-manager-server-db-2-5.9.0-1.cm590.p0.249.el7.x86_64.rpm
• oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
• cloudera-manager-agent-5.9.0-1.cm590.p0.249.el7.x86_64.rpm

3,安装cloudera manager server相关服务

$ sudo yum --nogpgcheck localinstall oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
$ sudo yum --nogpgcheck localinstall cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el7.x86_64.rpm
$ sudo yum --nogpgcheck localinstall cloudera-manager-server-*.rpm

4,启动内置数据库

$ sudo service cloudera-scm-server-db start
# scm 密码：/etc/cloudera-scm-server/db.properties
# postgre 密码: /var/lib/cloudera-scm-server-db/data/generated_password.txt [psql -h localhost -p 7432 -U cloudera-scm -d postgres]

5,启动cloudera manager server服务

$ sudo service cloudera-scm-server start
# 日志目录:/var/log/cloudera-scm-server/cloudera-scm-server.log

6,检查前台页面是否能够访问

地址:hostip:7180/

用户名/密码:admin/admin

6,启动cloudera manager agent服务

# 修改/etc/cloudera-scm-agent/config.ini 中 server_host项
$ vi /etc/cloudera-scm-agent/config.ini
server_host={你搭建的cloudera manager server ip地址}
$ sudo service cloudera-manager-agent start
# 日志目录:/var/log/cloudera-scm-agent/cloudera-scm-agent.log

7,登录页面查看hosts中是否存在这台主机

8,在其他节点安装并配置agent

9,下载CDH安装包

地址:http://archive.cloudera.com/cdh5/parcels

下载包列表

(以cdh5.8.4为例,完整地址:http://archive.cloudera.com/cdh5/parcels/5.8.4/)

• CDH-5.8.4-1.cdh5.8.4.p0.5-el7.parcel
• CDH-5.8.4-1.cdh5.8.4.p0.5-el7.parcel.sha1

10,将文件上传至cloudera manager server主机 /opt/cloudera/parcel-repo目录

11,登录CM页面,开始安装服务。

---

问题排查

agent未启动且无日志信息

问题描述：

$ sudo service cloudera-scm-agent start
Starting Cloudera Manager Agent...

BEGIN /sbin/service cloudera-scm-agent status 
● cloudera-scm-agent.service - LSB: Cloudera SCM Agent 
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-agent; bad; vendor preset: disabled) 
Active: inactive (dead) 
Docs: man:systemd-sysv-generator(8) 
END (3) 
BEGIN /sbin/service cloudera-scm-agent start 
Starting cloudera-scm-agent (via systemctl): [ OK ] 
END (0) 

# 显示agent服务启动失败,检查/var/log/cloudera-scm-agent目录下只有.out文件,无.log日志文件,执行'/etc/rc.d/init.d/cloudera-scm-agent start'返回值为0。
# 使用CM页面安装时报错如下:
Installation failed. Failed to receive heartbeat from agent.
Ensure that the host's hostname is configured properly.
Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).
Ensure that ports 9000 and 9001 are not in use on the host being added.
Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details).
If Use TLS Encryption for Agents is enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.ini has use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here.

参考链接:https://community.cloudera.com/t5/Cloudera-Manager-Installation/Installation-failed-Failed-to-receive-heartbeat-from-agent/td-p/53921

解决方案:修改主机hostname(主机名必须符合规范,不能包含下划线'_',如果社区没这先例,我估计只能以死谢罪了)

问题描述：

安装hive服务时使用外置mysql数据库，测试连接报错:
User can't run DDL statements on the specified database. Attempt to create and drop a table failed.

报错信息为执行ddl失败,我们后台使用命令行连接正常,找dba调了半天的权限,还是不行。
后面在页面测试时DBA发现数据库并未建立连接，开始查看官方文档，发现5.8.4版本只支持5.6版本的mysql,而我们DBA使用的是5.7版本。重建5.5版本数据库后解决问题。这报错不是一般的坑,居然说ddl执行失败。

解决方案:检查数据库版本是否在cdh版本支持列表中.