美文网首页
Ambari2.6.2 和HDP2.6.5搭建Hadoop集群-

Ambari2.6.2 和HDP2.6.5搭建Hadoop集群-

作者: wk86 | 来源:发表于2019-06-20 09:55 被阅读0次

    1.安装JDK

    1.1、卸载系统自带OPEN-JDK(所有节点)

    执行以下命令查看系统中有哪些OpenJk相关包:

    rpm -qa | grep java
    

    其中有如下包必须卸载

    rpm -e --nodeps java-1.5.0-gcj-1.5.0.0-29.1.el6.x86_64
    rpm -e --nodeps java-1.6.0-openjdk-1.6.0.0-1.66.1.13.0.el6.x86_64
    rpm -e --nodeps java-1.6.0-openjdk-devel-1.6.0.0-1.66.1.13.0.el6.x86_64
    
    1.2.下载安装 jdk-8u121-linux-x64.rpm

    安装jdk

    chmod +x jdk-8u121-linux-x86.rpm
    rpm -ivh jkd-8u121-linux-86.rpm
    
    1.3.配置环境变量
    [root@iZbp1barmvtqfdn6lceoyxZ tmp]# vi /etc/profile
    

    追加配置:

    export JAVA_HOME=/usr/java/jdk1.8.0_121
    export JRE_HOME=$JAVA_HOME/jre
    export PATH=$JAVA_HOME/bin:$PATH
    

    保存配置文件source /etc/profile激活配置,不然重开机后可能没有成功修改配置文件

    2.修改swappiness配置文件

    执行命令cat /proc/sys/vm/swappiness 一般情况下都是60
    vi /etc/sysctl.conf 添加 vm.swappiness=0 保存退出;重启后再查看就回是0
    操作这个是为了最后一步安装检查集群节点时出现警告
    具体原理请看http://blog.csdn.net/wulantian/article/details/36184943

    3.关闭防火墙(所有节点)


    centos7之前
    临时关闭:service iptables stop
    永久关闭:chkconfig iptables off
    查看关闭状况:service iptables status


    centos从7开始默认用的是firewalld,这个是基于iptables的,虽然有iptables的核心,但是iptables的服务是没安装的。所以你只要停止firewalld服务即可:

    sudo systemctl stop firewalld.service && sudo systemctl disable firewalld.service
    

    4.关闭SELINUX关、配置umask值

    vi /etc/selinux/config
    

    setenforce 0 (临时生效)修改 SELINUX=disabled (重启后永久生效)
    查看SELINUX状态
    若SELinux status 为disabled 即为关闭

    echo umask 0022 >> /etc/profile
    

    source /etc/profile

    5.修改主机名称

    vi /etc/hostname
    

    6.修改域名映射

    vi /etc/hosts
    

    添加以下信息:
    192.168.12.191 Master1.Hadoop node1
    192.168.12.192 Master2.Hadoop node2
    192.168.12.193 Slave1.Hadoop node3
    192.168.12.194 Slave2.Hadoop node4
    192.168.12.195 Slave3.Hadoop node5

    重启网络服务生效:

    service network restart
    

    7.SSH无密码登陆

    7.1 生成公钥和私钥 (每台机器)

    输入命令:(一路回车)

    ssh-keygen -t rsa
    
    7.2 执行ssh-copy-id(每台机器)
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@node1
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@node2
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@node3
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@node4
    ssh-copy-id -i ~/.ssh/id_rsa.pub root@node5
    
    • ssh-copy-id将pub值写入远程机器的~/.ssh/authorized_key中

    8.NTP时间服务设置

    按照此文章配置NTP即可

    Server端补充添加国内可用NTP服务

    server ntp1.aliyun.com iburst minpoll 4 maxpoll 10
    restrict ntp1.aliyun.com nomodify notrap nopeer noquery
    server ntp1.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp1.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp10.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp10.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp11.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp11.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp12.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp12.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp2.aliyun.com iburst minpoll 4 maxpoll 10
    restrict ntp2.aliyun.com nomodify notrap nopeer noquery
    server ntp2.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp2.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp3.aliyun.com iburst minpoll 4 maxpoll 10
    restrict ntp3.aliyun.com nomodify notrap nopeer noquery
    server ntp3.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp3.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp4.aliyun.com iburst minpoll 4 maxpoll 10
    restrict ntp4.aliyun.com nomodify notrap nopeer noquery
    server ntp4.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp4.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp5.aliyun.com iburst minpoll 4 maxpoll 10
    restrict ntp5.aliyun.com nomodify notrap nopeer noquery
    server ntp5.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp5.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp6.aliyun.com iburst minpoll 4 maxpoll 10
    restrict ntp6.aliyun.com nomodify notrap nopeer noquery
    server ntp6.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp6.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp7.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp7.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp8.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp8.cloud.aliyuncs.com nomodify notrap nopeer noquery
    server ntp9.cloud.aliyuncs.com iburst minpoll 4 maxpoll 10
    restrict ntp9.cloud.aliyuncs.com nomodify notrap nopeer noquery
    
    

    相关文章

      网友评论

          本文标题:Ambari2.6.2 和HDP2.6.5搭建Hadoop集群-

          本文链接:https://www.haomeiwen.com/subject/xwzrqctx.html