Hue是Cloudera开源的一个Hadoop UI,由Cloudera Desktop演化而来。面向用户提供方便的UI用于平时的Hadoop操作中。Apache Ambari面向的是管理员,用于安装、维护集群,而不是使用集群。两者针对的是不同需求。
依赖安装:
sudo yum install krb5-devel cyrus-sasl-gssapi cyrus-sasl-deve libxml2-devel libxslt-devel mysql mysql-devel openldap-devel python-devel python-simplejson sqlite-devel
如何已经安装了mysql最好把它提前卸载了。
JDK安装
Maven和Ant安装
Hue的安装过程需要依赖于Maven和Ant,按常规方法安装Maven和Ant,配置MAVEN_HOME,ANT_HOME,并将bin目录加入PATH
- MAV(apache-maven 或者 maven3 tarball )这里安装的是apache-maven
sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo
sudo sed -i s/\$releasever/6/g /etc/yum.repos.d/epel-apache-maven.repo
sudo yum install -y apache-maven
mvn --version
下载安装hue
sudo wget http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6.tar.gz
tar zxvf hue-3.7.0-cdh5.3.6.tar.gz
cd hue-4.0.1.tgz
安装
make apps
如果编译出现问题,需要重新clean,再编译
make clean
make apps
配置Hue
Hue的配置集中在$HUE_HOME/desktop/conf/hue.ini文件,针对不同的Hadoop组件分段配置。详细配置文件参考这里。
- Hue hue.ini
# Set this to a random string, the longer the better.
# This is used for secure hashing in the session store.
secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
# Webserver listens on this address and port
http_host=hadoop-senior.beifeng.com
http_port=8888
# Time zone name
time_zone=Asia/Shanghai
# Enable or disable Django debug mode.
django_debug_mode=false
# Enable or disable backtrace for server error
http_500_debug_mode=false
# Enable or disable memory profiling.
## memory_profiler=false
# Server email for internal error messages
## django_server_email='hue@localhost.localdomain'
# Email backend
## django_email_backend=django.core.mail.backends.smtp.EmailBackend
# Webserver runs as this user
server_user=beifeng
server_group=beifeng
- Hdfs
[[hdfs_clusters]]
# HA support by using HttpFs
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://hadoop-senior.beifeng.com:8020
# NameNode logical name.
## logical_name=
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
webhdfs_url=http://hadoop-senior.beifeng.com:50070/webhdfs/v1
# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false
# Default umask for file and directory creation, specified in an octal value.
## umask=022
# Directory of the Hadoop configuration
hadoop_conf_dir=/opt/cdh5.3.6/hadoop-2.5.0-cdh5.3.6/etc/hadoop
- Yarn 配置
[[yarn_clusters]]
[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=hadoop-senior.beifeng.com
# The port where the ResourceManager IPC listens on
resourcemanager_port=8032
# Whether to submit jobs to this cluster
submit_to=True
# Resource Manager logical name (required for HA)
## logical_name=
# Change this if your YARN cluster is Kerberos-secured
## security_enabled=false
# URL of the ResourceManager API
resourcemanager_api_url=http://hadoop-senior.beifeng.com:8088
# URL of the ProxyServer API
proxy_api_url=http://hadoop-senior.beifeng.com:8088
# URL of the HistoryServer API
history_server_api_url=http://hadoop-senior.beifeng.com:19888
# In secure mode (HTTPS), if SSL certificates from Resource Manager's
# Rest Server have to be verified against certificate authority
## ssl_cert_ca_verify=False
# HA support by specifying multiple clusters
# e.g.
# [[[ha]]]
# Resource Manager logical name (required for HA)
## logical_name=my-rm-name
hadoop配置
- 在hdfs-site.xml添加
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
- 在core-site.xml添加
<!-- enable WebHDFS in the NameNode and DataNodes -->
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<!-- Hue WebHDFS proxy user setting -->
<property>
<name>hadoop.proxyuser.beifeng.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.beifeng.groups</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.httpfs.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.httpfs.groups</name>
<value>*</value>
</property>
注意:proxyuser 后面是Hue.ini配置文件中的server_user 值
server_user=beifeng
server_group=beifeng
启动服务
build/env/bin/supervisor
停止服务(跳过)
netstat -apn |grep 8888
kill -9 33284
访问地址
http://hadoop-senior.beifeng.com:8888/about/
第一次登陆Hue时,会有提示,随便输入用户和密码,必须要记住,这就是以后登陆Hue管理员的用户名和密码。
网友评论