美文网首页
clouderamanager-cdh

clouderamanager-cdh

作者: 阿文很淘气 | 来源:发表于2020-05-08 14:47 被阅读0次
    基础环境准备
    
    操作基本依赖
    yum install -y net-tools.x86_64  vim wget httpd
    
    1、设置静态ip
    vim /etc/sysconfig/network-scripts/ifcfg-ens33
    TYPE="Ethernet"
    PROXY_METHOD="none"
    BROWSER_ONLY="no"
    BOOTPROTO="static"
    IPADDR=192.168.13.180
    NETMASK=255.255.255.0
    GATEWAY=192.168.13.2
    DNS1=192.168.13.2
    DEFROUTE="yes"
    IPV4_FAILURE_FATAL="no"
    IPV6INIT="yes"
    IPV6_AUTOCONF="yes"
    IPV6_DEFROUTE="yes"
    IPV6_FAILURE_FATAL="no"
    IPV6_ADDR_GEN_MODE="stable-privacy"
    NAME="ens33"
    UUID="9677b95a-fd83-4e19-915b-1aee52751109"
    DEVICE="ens33"
    ONBOOT="yes"
    
    service network restart
    2、配置主机名映射
    
    
    vi /etc/hosts
    
    
    192.168.1.11 node-1
    
    
    192.168.1.12 node-2
    
    
    192.168.1.13 node-3
    
    
    3、配置ssh免密登陆
    生成ssh免登录密钥
    
    
    ssh-keygen -t rsa  
    (四个回车)
    
    执行完这个命令后,会生成id_rsa(私钥)、id_rsa.pub(公钥)
    
    
    将公钥拷贝到要免密登陆的目标机器 上
    
    ssh-copy-id node-2
    
    
    
    
    ssh node-3
    
    无需密码直接登录上
    
    exit 
    
    退出当前登录的node-3
    
    
    4、配置防火墙和SELINUX
    
    查看防火墙状态
    
    service iptables status
    systemctl status firewalld
    
    
    关闭防火wall
    
    service iptables stop
    
    
    systemctl stop firewalld
    查看防火墙开机启动状态
    
    chkconfig iptables --list
    systemctl is-enabled firewalld
    
    
    关闭防火墙开机启动
    
    chkconfig  iptables off
    
    systemctl disable firewalld
    
    关闭SELINUX(重启才有效)
    vim /etc/selinux/config
    SELINUX=disabled
    查看SELINUX状态:
    /usr/sbin/sestatus –v
    
    5、JDK环境安装
    scp -r jdk1.8  root@192.168.13.180:/usr/local/
    vim /etc/profile
    export JAVA_HOME=/usr/local/jdk1.8/jdk1.8.0_162
    export PATH=$PATH:$JAVA_HOME/bin
    source /etc/profile
    java -version
    
    6、ntp 时间同步
    date 每节点时间 要一致。
    所有节点安装NTP:
    yum install ntp
    
    配置开机启动:
    chkconfig ntpd on
    
    检查是否设置成功:
    chkconfig --list ntpd (2-5为on状态则成功)
    
    设置同步:
    ntpdate -u ntp.sjtu.edu.cn(时钟服务器根据实际环境设置、本文采用210.72.145.44-国家授时中心服务器IP地址)
    
    7、节点依赖
    yum install -y chkconfig python  bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse portmap fuse-libs redhat-lsb
    8、mysql 数据库安装
    
    mysql
    rpm -qa|grep mysql
    for i in $(rpm -qa|grep mysql);do rpm -e $i --nodeps ;done
    
    
    wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
    yum -y install mysql57-community-release-el7-10.noarch.rpm
    yum -y install mysql-community-server
    systemctl enable mysqld
    systemctl is-enabled mysqld
    systemctl start mysqld
    systemctl status mysqld
    
    grep "password" /var/log/mysqld.log
    mysql -uroot -p
    alter user 'root'@'localhost' IDENTIFIED BY 'Zhujie0814.';
    vi /etc/my.cnf
    [client]  此行
    default-character-set=utf8    此行
    # For advice on how to change settings please see
    # http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
    
    [mysqld]
    。
    。
    。
    log-error=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid
    
    character-set-server=utf8  此行
    collation-server=utf8_general_ci   此行
    
    systemctl restart mysqld
    mysql -uroot -p
    grant all PRIVILEGES on *.* to root@'%'  identified by 'Zhujie0814.' with grant option;
    flush privileges;
    
    use mysql;
    update user set  `Select_priv` = 'Y',  `Insert_priv` = 'Y',  `Update_priv` = 'Y',  `Delete_priv` = 'Y',  `Create_priv` = 'Y',  `Drop_priv` = 'Y',  `Reload_priv` = 'Y',  `Shutdown_priv` = 'Y',  `Process_priv` = 'Y',  `File_priv` = 'Y',  `Grant_priv` = 'Y',  `References_priv` = 'Y',  `Index_priv` = 'Y',  `Alter_priv` = 'Y',  `Show_db_priv` = 'Y',  `Super_priv` = 'Y',  `Create_tmp_table_priv` = 'Y',  `Lock_tables_priv` = 'Y',  `Execute_priv` = 'Y',  `Repl_slave_priv` = 'Y',  `Repl_client_priv` = 'Y',  `Create_view_priv` = 'Y',  `Show_view_priv` = 'Y',  `Create_routine_priv` = 'Y',  `Alter_routine_priv` = 'Y',  `Create_user_priv` = 'Y',  `Event_priv` = 'Y',  `Trigger_priv` = 'Y',  `Create_tablespace_priv` = 'Y'  where user='root' and host='%';
    flush privileges;
    
    
    mkdir /opt/cloudera-manager
    tar -zxvf cloudera-manager-centos7-cm5.15.2_x86_64.tar.gz -C /opt/cloudera-manager
    在每个节点都要创建该用户
    useradd --system --no-create-home --shell=/bin/false  --comment "Cloudera SCM User" cloudera-scm
    
    vim /opt/cloudera-manager/cm-5.15.2/etc/cloudera-scm-agent/config.ini
    修改server 和port
    
    
    mkdir -p /usr/share/java
    mv /opt/mysql-connector-java-5.1.26-bin.jar  /usr/share/java/mysql-connector-java.jar
    
    create database scm DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
    
    grant all on *.* to 'temp'@'%' identified by 'Zhujie0814.' with grant option;
    grant all on *.* to 'scm'@'%' identified by 'Zhujie0814.' with grant option;
    cd /opt/cloudera-manager/cm-5.15.2/share/cmf/schema/
    ./scm_prepare_database.sh mysql -h node-1 -utemp -pZhujie0814. --scm-host node-1 scm scm scm
     (对应于:数据库类型、数据库服务器、数据库用户名、数据库密码、CMServer所在节点…….) 第一个scm 库名,第二个用户名,第三个,是scm库密码
    
    /usr/share/cmf/schema/scm_prepare_database.sh mysql -h<mysql-host> -u<mysql-user> -p<mysql-pwd> --scm-host <scm-host> <scm-user> <scm-dbname> <scm-pwd>
    其中
    
    <mysql-host>,<mysql-user> <mysql-pwd> 是需要创建的 scm 库所在的 mysql 位置,对应的 mysql 的主机名,用户名和登录密码;
    <scm-host>,<scm_user>,<scm-dbname>,<scm-pwd>分别对应 cloudera-scm-server 部署在的主机名,scm 库的登录用户,scm 库的库名,scm 库的登录密码。
    
    
    ./scm_prepare_database.sh mysql -h node-1 -uroot -pZhujie0814. --scm-host node-1 scm scm Zhujie0814.
    
    mkdir -p /opt/cloudera/parcel-repo
    
    
    Manager节点创建目录/opt/cloudera/parcel-repo,执行:
        mkdir -p /opt/cloudera/parcel-repo
        chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
    将下载好的文件(CDH-5.4.0-1.cdh5.4.0.p0.27-el6.parcel、CDH-5.4.0-1.cdh5.4.0.p0.27-el6.parcel.sha、manifest.json)拷贝到该目录下。
        
        Agent节点创建目录/opt/cloudera/parcels,执行:
    mkdir -p /opt/cloudera/parcels
        chown cloudera-scm:cloudera-scm /opt/cloudera/parcels
    
    在node-1上  把cloudera-manager 复制到其它节点
    scp -r cloudera-manager root@node-2:/opt
    定位所有节点位置到
    cd /opt/cloudera-manager/cm-5.15.2/etc/init.d 
    Server:/opt/cloudera-manager/cm-5.15.2/etc/init.d/cloudera-scm-server start
    Agents:/opt/cloudera-manager/cm-5.15.2/etc/init.d/cloudera-scm-agent start
    
    
    chown cloudera-scm:cloudera-scm -R  /opt/cloudera
    chown cloudera-scm:cloudera-scm -R  /opt/cloudera-manager
    
    
    当检查主机时出现该问题,一般是由于之前装过cloudera-scm-agent .
    
    解决办法:cd  /opt/cloudera-manager/cm-5.15.2/lib/cloudera-scm-agent
    
    rm -f  cm_guid  
    
    /opt/cloudera-manager/cm-5.15.2/etc/init.d/cloudera-scm-agent restart
    
    
    
    
    

    相关文章

      网友评论

          本文标题:clouderamanager-cdh

          本文链接:https://www.haomeiwen.com/subject/ixtvwhtx.html