美文网首页
CDH5.14在线安装

CDH5.14在线安装

作者: 飞起的书包 | 来源:发表于2019-02-26 16:17 被阅读12次

    硬件配置:
    阿里云(强调!!!!!!!!!!! 虚拟机请自行配置静态ip)

    物理CPU数 逻辑CPU总核数 内存 存储 CPU
    1--4核 8核 64G 500G Intel(R) Xeon(R) CPU E5-2682 v4 @ 2.50GHz

    操作系统：
    Centos 7.4

    查看centos版本:

    cat /etc/redhat-release

    查看内核版本信息

    uname -a

    查看内网ip

    ip addr

    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno16780032: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:0c:29:7f:5c:48 brd ff:ff:ff:ff:ff:ff
    inet ************** brd ************ scope global eno16780032
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe7f:5c48/64 scope link 
       valid_lft forever preferred_lft forever

    集群规划：

    集群 角色
    hadoop01 master
    hadoop02 slave1
    hadoop03 slave2
    hadoop04 slave3

    基础环境配置：

    1.集群免密登录:
    ①修改主机名
    查看主机名:

    hostname

    临时修改主机名:

    hostname hadoop01
hostname hadoop02
hostname hadoop03
hostname hadoop04

    永久修改主机名:

    [root@hadoop01 ~]# vi /etc/sysconfig/network

hostname=hadoop01

[root@hadoop02 ~]# vi /etc/sysconfig/network

hostname=hadoop02

[root@hadoop03 ~]# vi /etc/sysconfig/network

hostname=hadoop03

[root@hadoop04 ~]# vi /etc/sysconfig/network

hostname=hadoop04

    ② hosts文件配置:

    vi /etc/hosts

    hadoop01 hadoop02 hadoop03 都修改同一的配置(内网ip 主机名 域名)

    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.184.11 hadoop01 www.xy01.com

192.168.184.12 hadoop02 www.xy02.com

192.168.184.13 hadoop03 www.xy03.com

192.168.184.14 hadoop04 www.xy04.com

    ③ssh 免密配置:

    两两配置免密:

    1.生成密钥(每台机器上都执行):

    ssh-keygen -t rsa
ssh-copy-id   hadoop01
ssh-copy-id   hadoop02
ssh-copy-id   hadoop03
ssh-copy-id   hadoop03

    2.JDK.1.8.0_191 安装:
    ①解压

    tar -zxvf jdk-8u191-linux-x64.tar.gz -C /usr/local/

    ②配置环境变量

    export JAVA_HOME=/usr/local/jdk1.8.0_191

export CLASSPATH=$JAVA_HOME/lib/

export PATH=$PATH:$JAVA_HOME/bin

    配置完之后 执行

     source /etc/profile

    使配置文件生效

    3.时间同步设置:

    (若有则无需设置)

    查看时间

    [root@hadoop01 home]# date

Fri Feb 22 22:58:41 CST 2019

[root@hadoop02 home]# date

Fri Feb 22 22:58:45 CST 2019

[root@hadoop03 home]# date

Fri Feb 22 22:58:50 CST 2019

[root@hadoop04 home]# date

Fri Feb 22 22:58:57 CST 2019

    安装ntp(hadoop01 hadoop02 hadoop03 hadoop04)

    yum install ntp
chkconfig ntpd on
ntpdate -u ntp.sjtu.edu.cn
*/2 * * * * /usr/sbin/ntpdate -u ntp.sjtu.edu.cn >> /dev/null 2>&1

    date 查看是否同步

    4.关闭防火墙和selinux:
    查看防火墙状态:s

    systemctl status firewalld.service

    关闭防火墙:

    systemctl stop firewalld.service

    永久防火墙开机自启动:

    systemctl disable firewalld

    开机关闭SELinux:

    SELINUX=disabled

    vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

    CDH 5.14.4 在线安装:
    设置(所有节点)yum源:
    1、打开centos的yum文件夹
    输入命令

    cd /etc/yum.repos.d/

    2、用wget下载repo文件
    输入命令:

    wget http://mirrors.aliyun.com/repo/Centos-7.repo

    如果wget命令不生效，说明还没有安装wget工具，输入

    yum -y install wget

    当前目录是/etc/yum.repos.d/，刚刚下载的Centos-7.repo也在这个目录上
    3、备份系统原来的repo文件

    mv CentOs-Base.repo CentOs-Base.repo.bak

    即是重命名 CentOs-Base.repo -> CentOs-Base.repo.bak
    4、替换系统原理的repo文件

    mv Centos-7.repo CentOs-Base.repo

    即是重命名 Centos-7.repo -> CentOs-Base.repo

    5、执行yum源更新命令

    yum clean all
yum makecache
yum update

    主节点hadoop01 yum安装mysql:
    下载mysql源:

    wget http://dev.mysql.com/get/mysql57-community-release-el7-7.noarch.rpm

    安装到yum 源:

    yum localinstall -y mysql57-community-release-el7-7.noarch.rpm

    安装MySQL:

    yum install -y mysql-community-server

    启动MySQL服务:

    systemctl start mysqld.service

    查看初始密码

    grep 'temporary password' /var/log/mysqld.log

    修改密码:

    mysql -u root -p 初始密码
SET PASSWORD = PASSWORD('your new password');
ALTER USER 'root'@'localhost' PASSWORD EXPIRE NEVER;
flush privileges;

    设置远程访问权限:

    GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'your password' WITH GRANT OPTION;

    java 设置(所有节点)
    cdh启用时, 默认会从/usr/java/default路径读取java环境, 所以此处需要做个软链

    #生成软链
mkdir /usr/java 
ln -s /usr/local/jdk1.8.0_191 /usr/java/default

    登录mysql添加数据库, 用户并授权:

    create database cm DEFAULT CHARACTER SET utf8
GRANT ALL PRIVILEGES ON cm.* TO 'cm'@'%' IDENTIFIED BY 'admin123@XY'

    hadoop上执行
    下载对应系统cdh官方yum源:

    cd /etc/yum.repos.d/
wget http://archive.cloudera.com/cm5/redhat/7/x86_64/cm/cloudera-manager.repo
wget https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/cloudera-cdh5.repo

    修改yum源, 配置cdh的版本(默认是最新的)

    
vim /etc/yum.repos.d/cloudera-cdh5.repo
vim /etc/yum.repos.d/cloudera-manager.repo

    [cloudera-manager]
# Packages for Cloudera Manager, Version 5, on RedHat or CentOS 7 x86_64              
name=Cloudera Manager
baseurl=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.14/
gpgkey =https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera    
gpgcheck = 1

    [cloudera-cdh5]
# Packages for Cloudera's Distribution for Hadoop, Version 5, on RedHat or CentOS 7 x86_64
name=Cloudera's Distribution for Hadoop, Version 5
baseurl=https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5.14/
gpgkey =https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/RPM-GPG-KEY-cloudera    
gpgcheck = 1

    安装 cm server

    yum install cloudera-manager-daemons cloudera-manager-server -y

    修改配置文件

    vim /etc/cloudera-scm-server/db.properties

    com.cloudera.cmf.db.type=mysql

# The database host
# If a non standard port is needed, use 'hostname:port'
com.cloudera.cmf.db.host=192.168.184.11:3306

# The database name
com.cloudera.cmf.db.name=cm

# The database user
com.cloudera.cmf.db.user=cm

# The database user's password
com.cloudera.cmf.db.password=admin23@XY

    尝试启动

    /etc/init.d/cloudera-scm-server restart

    查看日志是否报错

    tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

    如果启动失败, 可能报错jdbc驱动找不到

    wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.47.tar.gz

    解压后,将其jar包复制到cdh公共lib目录下

    mv mysql-connector-java-5.1.47-bin.jar mysql-connector-java.jar

    cp mysql-connector-java.jar /usr/share/cm/lib/
cp mysql-connector-java.jar /usr/share/cm/common_jars/
cp mysql-connector-java.jar $JAVA_HOME/lib/

    若启动成功则:
    在页面访问

    192.168.184.11:7180

    可能出现的问题:

    待续>>>>>>>>>>>>>>

    相关文章

      网友评论

          本文标题：CDH5.14在线安装

          本文链接：https://www.haomeiwen.com/subject/egtayqtx.html

          延伸阅读

          深度阅读

          • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

          栏目导航

          热点阅读

          关于我们|服务条款|联系我们|CDH5.14在线安装|投稿指南|网站地图|RSS订阅|排版工具|手机版

          提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

          Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

          备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

          本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

          所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！