cdh部署

0、注意安装时需要给server免密ssh登陆所有机器的权限

需要用到的命令：ssh -p 端口号 用户@ip     scp -P  端口号  文件名 用户@ip:路径

1、jdk 注意指定的版本(链接失效可以从官网找，或从其他机器拷贝)（每台）

wget http://download.oracle.com/otn/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jre-8u131-linux-x64.tar.gz

注意建软链接

ln -s jdk1.8.0_151/ java

sh> vi /etc/profile

# java

export JAVA_HOME=/usr/local/java

export PATH=$JAVA_HOME/bin:$PATH 

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar 

注：此时配的JAVA_HOME，cdh还不能识别，后面还需要再次配置

2、安装psycopg2（每台）

yes | yum install postgresql-devel

yes | yum install python-devel

yes | yum install libffi-devel

yes | yum install libpqxx-devel

pip install psycopg2

3、配置数据库连接的Java库（每台）

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.44.tar.gz

解压后取出mysql-connector-java-5.1.44-bin.jar 放到 /usr/share/java/ 

并 ln -s mysql-connector-java-5.1.44-bin.jar mysql-connector-java.jar

4、安装NTP（每台）

yum install ntp -y

cp /etc/ntp.conf /etc/ntp.conf.bak

vi /etc/ntp.conf

加入

server ntpupdate.tencentyun.com

再重新启动

sudo systemctl restart ntpd.service

sudo systemctl status ntpd.service

等5-10分钟看

ntpstat

未连接上

[root@GZ-BigData-data-193-204 ~]# ntpstat

unsynchronised

  time server re-starting

   polling server every 8 s

连接上

[root@GZ-BigData-data-193-204 ~]# ntpstat

synchronised to NTP server (169.254.0.2) at stratum 5 

   time correct to within 362 ms

   polling server every 64 s

chkconfig ntpd on

5、关闭透明大页面压缩（每台）

echo never > /sys/kernel/mm/transparent_hugepage/defrag

echo never > /sys/kernel/mm/transparent_hugepage/enabled

server端部署，在第6步开始改为（server端机器）

6、

从这里

http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64/

下载

cloudera-manager-agent-5.13.0-1.cm5130.p0.55.el7.x86_64.rpm

cloudera-manager-daemons-5.13.0-1.cm5130.p0.55.el7.x86_64.rpm

cloudera-manager-server-5.13.0-1.cm5130.p0.55.el7.x86_64.rpm  (server端用)

yum localinstall cloudera-manager-daemons-5.13.0-1.cm5130.p0.55.el7.x86_64.rpm

yum localinstall cloudera-manager-server-5.13.0-1.cm5130.p0.55.el7.x86_64.rpm

7、安装mysql

wget https://dev.mysql.com/get/Downloads/MySQL-5.5/mysql-5.5.58-linux-glibc2.12-x86_64.tar.gz

（https://cdn.mysql.com//Downloads/MySQL-5.5/mysql-5.5.58-linux-glibc2.12-x86_64.tar.gz）

安装说明：https://dev.mysql.com/doc/refman/5.5/en/binary-installation.html

参考：https://www.cnblogs.com/lunatic-cto/p/6123490.html 

create database amon DEFAULT CHARACTER SET utf8;

create database rman DEFAULT CHARACTER SET utf8;

create database metastore DEFAULT CHARACTER SET utf8;

create database sentry DEFAULT CHARACTER SET utf8;

create database nav DEFAULT CHARACTER SET utf8;

create database navms DEFAULT CHARACTER SET utf8;

create database oozie default character set utf8;

create database hive default character set utf8;

create database hue default character set utf8 default collate utf8_general_ci;

grant all on amon.* TO 'amon'@'10.%' IDENTIFIED BY 'xxx';

grant all on rman.* TO 'rman'@'10.%' IDENTIFIED BY 'xxx;

grant all on metastore.* TO 'metastore'@'10.%' IDENTIFIED BY 'xxx';

grant all on sentry.* TO 'sentry'@'10.%' IDENTIFIED BY 'xxx;

grant all on nav.* TO 'nav'@'10.%' IDENTIFIED BY 'xxx';

grant all on navms.* TO 'navms'@'10.%' IDENTIFIED BY 'xxx';

grant all on oozie.* TO 'oozie'@'10.%' IDENTIFIED BY 'xxx';

grant all on hive.* TO 'hive'@'10.%' IDENTIFIED BY 'xxx';

grant all on hue.* TO 'hue'@'10.%' IDENTIFIED BY 'xxx';

create database cmf DEFAULT CHARACTER SET utf8;

grant all on cmf.* TO 'cmf'@'10.%' IDENTIFIED BY 'xxx';

flush privileges;

（数据库账号说明：https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5 ）

8、配置数据库

vi/etc/cloudera-scm-server/db.properties

填入数据库地址和账号

并修改

（针对错误2017-03-06 00:02:44,675 ERROR MainThread:com.cloudera.server.cmf.Main: Server failed.

java.lang.IllegalArgumentException: Invalid value set for db.setupType, the valid values are EMBEDDED or EXTERNAL）

com.cloudera.cmf.db.setupType=EXTERNAL

设置java路径

参考：https://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_jdk_installation.html#topic_29

9、启动 cloudera-scm-server

sudo service cloudera-scm-server start

查看日志

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

10、安装cdh客户端（需要安装客户端的机器）

yum localinstall cloudera-manager-daemons-5.13.0-1.cm5130.p0.55.el7.x86_64.rpm

yum localinstall cloudera-manager-agent-5.13.0-1.cm5130.p0.55.el7.x86_64.rpm

11、配置连接的server，并启动客户端

/etc/cloudera-scm-agent/config.ini 

server_host=xxx

sudo service cloudera-scm-agent start

登录 xxxxxxxxxxxxxxx:7180 

在web上操作

注意安装时需要给server免密ssh登陆所有机器的权限

===========================================