16.1 hue

一、介绍

image.png

二、下载

到cdh网站下载,下载版本为hue-3.7-cdh5.3.6版本过高可能与hive版本不兼容,高版本的hue需要和高版本的hive配合使用

三、解压

放到第三台机器上。

四、安装

参考官网http://archive.cloudera.com/cdh5/cdh/5/hue-3.9.0-cdh5.10.0/manual.html
需要安装依赖软件，第三台机器需要联网。

yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel openldap-devel python-devel sqlite-devel openssl-devel mysql-devel gmp-devel

五、编译

若安装位置发生移动，也需要重新编译

## 在hue的安装根目录下执行
$ rm app.reg
$ rm -r build
$ make apps

六、基本配置

修改/opt/modules/hue-3.9.0-cdh5.10.0/desktop/conf/hue.ini

image.png

secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
http_host=bigdata-pro03.kfk.com
http_port=8888
time_zone=Asia/Shanghai

修改权限,让other有写的权限

/opt/modules/hue-3.9.0-cdh5.10.0/desktop
[kfk@bigdata-pro03 desktop]$ sudo chmod o+w desktop.db

启动服务

/opt/modules/hue-3.7.0-cdh5.3.6/build/env/bin/supervisor

查看地址
http://bigdata-pro03.kfk.com:8888/desktop/dump_config

七、与hdfs集成

7.1、修改hue.ini

image.png

# Enter the filesystem uri
      fs_defaultfs=hdfs://ns

       webhdfs_url=http://bigdata-pro01.kfk.com:50070/webhdfs/v1
        
       hadoop_hdfs_home=/opt/modules/hadoop-2.5.0
       
       hadoop_bin=/opt/modules/hadoop-2.5.0/bin

      # Directory of the Hadoop configuration
       hadoop_conf_dir=/opt/modules/hadoop-2.5.0/etc/hadoop

修改hdfs上tmp的权限

image.png

7.2、修改所有节点上的hadoop的core-site.xml

##dfs.webhdfs.enabled可以不写，默认为true
<property>
    <name>dfs.webhdfs.enabled</name>
    <value>true</value>
  </property>
  <property>
    <name>hadoop.proxyuser.hue.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.hue.groups</name>
    <value>*</value>
  </property>

八 与yarn的集成

8.1

注意:resourceManager_host写集群id,为rs
如果hue页面上看不到job执行情况，则说明resourcemanager_api_url所填写的节点地址不是active。解决办法是让手动设置节点为active，或者手动修改配置文件上resourcemanager_api_url的节点地址。

image.png

# Enter the host on which you are running the ResourceManager
      resourcemanager_host=rs

      # The port where the ResourceManager IPC listens on
      resourcemanager_port=8032

      # Whether to submit jobs to this cluster
      submit_to=True

      # URL of the ResourceManager API
      resourcemanager_api_url=http://bigdata-pro01.kfk.com:8088

      # URL of the ProxyServer API
      proxy_api_url=http://bigdata-pro01.kfk.com:8088

      # URL of the HistoryServer API
      history_server_api_url=http://bigdata-pro01.kfk.com:19888

九 与hive集成

image.png
image.png

hive_server_host：hive的地址为节点3的地址。

[beeswax]

  # Host where HiveServer2 is running.
  # If Kerberos security is enabled, use fully-qualified domain name (FQDN).
  hive_server_host=bigdata-pro03.kfk.com

  # Port where HiveServer2 Thrift server runs on.
  hive_server_port=10000

  # Hive configuration directory, where hive-site.xml is located
  hive_conf_dir=/opt/modules/hive-0.13.1-cdh5.3.0/conf

在hive-site.xml中添加以下配置

<property>
        <name>hbase.zookeeper.quorum</name>
        <value>bigdata-pro01.kfk.com,bigdata-pro02.kfk.com,bigdata-pro03.kfk.com</value>
    </property>

hive的启动方式

bin/hiveserver2

十、与mysql的集成

image.png
image.png

mysql的服务器地址也为节点3

[librdbms]
  # The RDBMS app can have any number of databases configured in the databases
  # section. A database is known by its section name
  # (IE sqlite, mysql, psql, and oracle in the list below).

  [[databases]]
    # sqlite configuration.
    [[[sqlite]]]
      # Name to show in the UI.
      nice_name=SQLite

      # For SQLite, name defines the path to the database.
      name=/opt/modules/hue-3.7.0-cdh5.3.6/desktop/desktop.db

      # Database backend to use.
      engine=sqlite

      # Database options to send to the server when connecting.
      # https://docs.djangoproject.com/en/1.4/ref/databases/
      ## options={}

    # mysql, oracle, or postgresql configuration.
    [[[mysql]]]
      # Name to show in the UI.
      nice_name="My SQL DB"

      # For MySQL and PostgreSQL, name is the name of the database.
      # For Oracle, Name is instance of the Oracle server. For express edition
      # this is 'xe' by default.
      name=metastore

      # Database backend to use. This can be:
      # 1. mysql
      # 2. postgresql
      # 3. oracle
      engine=mysql

      # IP or hostname of the database to connect to.
      host=bigdata-pro03.kfk.com

      # Port the database server is listening to. Defaults are:
      # 1. MySQL: 3306
      # 2. PostgreSQL: 5432
      # 3. Oracle Express Edition: 1521
      port=3306

      # Username to authenticate with when connecting to the database.
      user=root

      # Password matching the username to authenticate with when
      # connecting to the database.
      password=123456

      # Database options to send to the server when connecting.
      # https://docs.djangoproject.com/en/1.4/ref/databases/
      ## options={}

十一、与hbase的集成

image.png

[hbase]
  # Comma-separated list of HBase Thrift servers for clusters in the format of '(name|host:port)'.
  # Use full hostname with security.
  hbase_clusters=(Cluster|bigdata-pro01.kfk.com:9090)

  # HBase configuration directory, where hbase-site.xml is located.
  hbase_conf_dir=/opt/modules/hbase-0.98.6-cdh5.3.0/conf

启动测试

解决无法获取日志的问题

无法获取日志的原因可能是我们启动hive方式有问题.若以客户端的方式启动则不会出现这个问题

image.png