【Hadoop】Ambari 安装

本篇结构：

• 准备条件
• 使用本地仓库离线安装
• 配置 Ambari
• 启动 Ambari
• Log In to Apache Ambari
• 创建集群

一、准备条件

1.1、软件需求

On each of your hosts:

• yum and rpm (RHEL/CentOS/Oracle/Amazon Linux)

• zypper and php_curl (SLES)

• apt (Debian/Ubuntu)

• scp, curl, unzip, tar, and wget

• OpenSSL (v1.01, build 16 or later)

• Python

  • For SLES 12:

    Python 2.7.x

  • For Amazon Linux 2, CentOS 7, Ubuntu 14, Ubuntu 16, Ubuntu 18, and Debian 9:

    Python 2.7.x

1.2、内存需求

The Ambari host should have at least 1 GB RAM, with 500 MB free.

To check available memory on any host, run:

free -m

1.3、最大打开文件需求

The recommended maximum number of open file descriptors is 10000, or more. To check the current value set for the maximum number of open file descriptors, execute the following shell commands on each host:

ulimit -Sn
ulimit -Hn

If the output is not greater than 10000, run the following command to set it to a suitable default:

ulimit -n 10000

1.4、Collect Information

The base directories you want to use as mount points for storing:

• NameNode data
• DataNodes data
• Secondary NameNode data
• Oozie data
• YARN data
• ZooKeeper data, if you install ZooKeeper
• Various log, pid, and db files, depending on your install type

Important
You must use base directories that provide persistent storage locations for your components and your Hadoop data. Installing components in locations that may be removed from a host may result in cluster failure or data loss. For example: Do Not use /tmp in a base directory path.

1.5、Prepare the Environment

• SSH 免密 https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_password-less_ssh.html

  # 生成密钥对
  ssh-keygen -t rsa 
  # 将密码插入到authorized_keys
  cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
  # 配置远程主机SSH免密登录
  ssh-copy-id -i user@node1
  

  ssh-copy-id -i user@node1就是将本机的公钥插入到远程主机node1的用户user的authorized_keys文件中
  所以也可以手动通过scp将自己的公钥复制到远程主机，然后再将其手机复制到远程主机authorized_keys文件的结尾

• 设置服务用户帐户https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_service_user_accounts.html

• 在集群和浏览器主机上启用NTPhttps://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/enable_ntp_on_the_cluster_and_on_the_browser_host.html

# buntu
apt-get install ntp
update-rc.d ntp defaults

• 每台主机修改 hosts，添加对应的域名配置解析,将主机名与IP地址相对应(注意IP地址与DNS要做到一一对应,会存在反解析操作,否则有坑)
• 防火墙设置：https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/configuring_iptables.html

# Ubuntu
sudo ufw disable
sudo iptables -X
sudo iptables -t nat -F
sudo iptables -t nat -X
sudo iptables -t mangle -F
sudo iptables -t mangle -X
sudo iptables -P INPUT ACCEPT
sudo iptables -P FORWARD ACCEPT
sudo iptables -P OUTPUT ACCEPT

二、使用本地仓库离线安装

2.1、Create an HTTP server

可以使用 Tomcat，也可以使用 Ngnix，这里用 Ngnix：

sudo apt-get install nginx

2.2、On your mirror server, create a directory for your web server.

mkdir -p /var/www/ambari/HDP-UTILS
cd /var/www/ambari

2.3、Obtain the compressed tape archive file (tarball) for the repository you want to create.

从ambari仓库和HDP仓库下载符合你的操作系统的ambari和HDP安装程序，这里下载的是Ubuntu 16的安装包

image

wget http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.1.0/ambari-2.7.1.0-ubuntu16.tar.gz

image

wget http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.0.1.0/HDP-3.0.1.0-ubuntu16-deb.tar.gz

wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16/HDP-UTILS-1.1.0.22-ubuntu16.tar.gz

wget http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.0.1.0/HDP-GPL-3.0.1.0-ubuntu16-gpl.tar.gz

2.4、解压上面下载的文件到指定目录

tar -zxvf ambari-2.7.1.0-ubuntu16.tar.gz
tar -zxvf HDP-GPL-3.0.1.0-ubuntu16-gpl.tar.gz
tar -zxvf HDP-3.0.1.0-ubuntu16-deb.tar.gz 
tar -zxvf HDP-UTILS-1.1.0.22-ubuntu16.tar.gz -C HDP-UTILS

可以看到当前目录(/var/www/ambari)下有如下文件夹：

ambari/  HDP/  HDP-GPL/  HDP-UTILS/

2.5、创建 nginx 配置文件 ambari

cd /etc/nginx/sites-available
vi ambari

ambari 文件内容如下：

server {
    listen 80;
    listen [::]:80;

    server_name <host-name>; # <host-name>为服务器主机名

    root /var/www/ambari;

    location ^~ / {
        autoindex on;
        autoindex_exact_size off;
        autoindex_localtime on;
    }


    location ~ /\.ht {
        deny all;
    }

}

重启nginx服务器:

cd ../sites-enabled
ln -s ../sites-available/ambari ambari
service nginx restart

2.6、指定仓库地址

进入到/etc/apt/sources.list.d文件夹下

创建以下文件，记得把<host-name>替换为当前机器的ip：

vi ambari.list

# 因ambari每个大版本后都有小版本号，因此请自行检查路径是否正确
deb http://<host-name>/ambari/ubuntu16/2.7.1.0-169/ Ambari main

vi hdp.list

# 因ambari每个大版本后都有小版本号，因此请自行检查路径是否正确
deb http://<host-name>/HDP/ubuntu16/3.0.1.0-187/ HDP main
deb http://<host-name>/HDP-GPL/ubuntu16/3.0.1.0-187/ HDP-GPL main
deb http://<host-name>/HDP-UTILS/ubuntu16/1.1.0.22/ HDP-UTILS main

将这两个文件拷贝到集群的所有机器上，现在本地仓库已经安装完成。

三、安装 Ambari

保证准备工作进行正确。首先执行下面命令导入key：

sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com B9733A7A07513CAD

sudo apt-get update

使用下面的命令检查update是否成功，即包信息是否已经被缓存。

apt-cache showpkg ambari-server
apt-cache showpkg ambari-agent
apt-cache showpkg ambari-metrics-assembly

果一切正常，就可以开始安装Ambari了。

sudo apt-get install ambari-server

三、配置 Ambari

执行sudo ambari-server setup配置Ambari。
遇到Customize user account for ambari-server daemon选择n，使Ambari 在 root 下执行，当然也可以选择其它用户。
然后输入数字，选择要安装的JDK版本并等待下载安装完成。
Enter advanced database configuration选择n来使用内置数据库，选择 y 自定义数据库，根据提示输入相应信息即可。

https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.1.0/bk_ambari-installation/content/set_up_the_ambari_server.html

四、启动 Ambari

sudo ambari-server start
sudo ambari-server status
sudo ambari-server stop

五、Log In to Apache Ambari

1.Point your web browser to

http://<your.ambari.server>:8080

2.Log in to the Ambari Server using the default user name/password: admin/admin. You can change these credentials later.

六、创建集群

image

点击Launch Install Wizard按钮，进入Ambari集群安装向导。根据页面提示操作一步步完成即可。

1.给集群命名，名称不能包含特殊字符

2.选择和本地仓库下载的对应的版本

3.Install Options 这步主要配置所有的节点FQDNS，并上传主节点私钥

4.Confirm Hosts 这里观察安装进度

我这里安装出了错误，参考下面解决

https://community.hortonworks.com/content/supportkb/188002/ioerror-request-to-httpslocalhost8441agentv1regist.html

force_https_protocol=PROTOCOL_TLSv1_2

image

5.Choose Services

Choose which services you want to install on your cluster.

6.Assign Masters

7.Assign Slaves and Clients

8.Customize Services

9.Configure Identities

10.Review

11.Install, Start and Test