CDH安装phoenix

作者: Rex_2013 | 来源:发表于2020-08-05 11:32 被阅读0次

CDH安装phoenix
解决：cdh版本的Phoenix安装的问题
CDH5.8 HBase安装Phoenix
CDH5.8 HBase安装Phoenix
Phoenix编译支持cdh5.10.2
Phoenix从4.7升级到4.14
phoenix for cloudera
大数据之CM+CDH5.16.1（二）
Centos7.6部署CDH6.1.0大数据平台教程
Hbase-2.4.2（四）phoenix 安装及使用

一、准备文件

https://archive.cloudera.com/phoenix/6.2.0/csd/PHOENIX-1.0.jar
https://archive.cloudera.com/phoenix/6.2.0/parcels/PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel
https://archive.cloudera.com/phoenix/6.2.0/parcels/PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel.sha
https://archive.cloudera.com/phoenix/6.2.0/parcels/manifest.json

[root@node01 phoenix]# pwd
/var/www/html/phoenix
[root@node01 phoenix]# ll
total 392808
-rw-r--r--. 1 root root      2478 Jul 31 16:09 manifest.json
-rw-r--r--. 1 root root      5306 Aug  1  2019 PHOENIX-1.0.jar
-rw-r--r--. 1 root root 402216960 Jul 31 16:12 PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel
-rw-r--r--. 1 root root        41 Jul 31 16:07 PHOENIX-5.0.0-cdh6.2.0.p0.1308267-el7.parcel.sha

二、配置安装

将文件放入 /opt/cloudera/csd 目录下

[root@node01 csd]# mv /var/www/html/phoenix/PHOENIX-1.0.jar .
[root@node01 csd]# ll
total 8
-rw-r--r--. 1 root root 5306 Aug  1  2019 PHOENIX-1.0.jar

重启 cloudera-scm-server 服务

[root@node01 ~]# systemctl restart cloudera-scm-server

打开CM 添加的服务

image.png

[root@node01 html]# python -m SimpleHTTPServer 8900
Serving HTTP on 0.0.0.0 port 8900 ...

image.png

在管理页面中部署hbase客户端配置

image.png

[root@node02 conf]# pwd
/etc/hbase/conf
[root@node02 conf]# vim hbase-site.xml

<!-- phoenix -->
<property>
        <name>phoenix.schema.isNamespaceMappingEnabled</name>
        <value>true</value>
</property>
<property>
        <name>phoenix.schema.mapSystemTablesToNamespace</name>
        <value>true</value>
</property>

Phoenix 表映射

Phoenix 表和 HBase 表的关系

默认情况下, 直接在 HBase 中创建的表通过 Phoenix 是查不到
hbase-shell

hbase(main):009:0> create 'test','name','company'
Created table test
Took 0.7872 seconds
=> Hbase::Table - test
hbase(main):010:0> list
TABLE
SYSTEM:CATALOG
SYSTEM:FUNCTION
SYSTEM:LOG
SYSTEM:MUTEX
SYSTEM:SEQUENCE
SYSTEM:STATS
test
7 row(s)
Took 0.0066 seconds
=> ["SYSTEM:CATALOG", "SYSTEM:FUNCTION", "SYSTEM:LOG", "SYSTEM:MUTEX", "SYSTEM:SEQUENCE", "SYSTEM:STATS", "test"]

phoenix

0: jdbc:phoenix:node03,node04,node05:2181> !tables
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+--------------+-----------------+---------------+---------------+----------+
| TABLE_CAT  | TABLE_SCHEM  | TABLE_NAME  |  TABLE_TYPE   | REMARKS  | TYPE_NAME  | SELF_REFERENCING_COL_NAME  | REF_GENERATION  | INDEX_STATE  | IMMUTABLE_ROWS  | SALT_BUCKETS  | MULTI_TENANT  | VIEW_STA |
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+--------------+-----------------+---------------+---------------+----------+
|            | SYSTEM       | CATALOG     | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
|            | SYSTEM       | FUNCTION    | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
|            | SYSTEM       | LOG         | SYSTEM TABLE  |          |            |                            |                 |              | true            | 32            | false         |          |
|            | SYSTEM       | SEQUENCE    | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
|            | SYSTEM       | STATS       | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+--------------+-----------------+---------------+---------------+----------+

如果要在 Phoenix 中操作直接在 HBase 中创建的表，则需要在 Phoenix 中进行表的映射。
映射方式有两种： 1. 视图映射 2. 表映射

视图映射
Phoenix 创建的视图是只读的, 所以只能用来查询, 无法通过视图对数据进行修改等操作

在Phoenix上创建test表视图

0: jdbc:phoenix:node03,node04,node05:2181> create view "test" (empid varchar primary key,"name"."firstname" varchar,"name"."lastname" varchar,"company"."name" varchar,"company"."address" varchar);
No rows affected (6.491 seconds)
0: jdbc:phoenix:node03,node04,node05:2181> !tables
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+--------------+-----------------+---------------+---------------+----------+
| TABLE_CAT  | TABLE_SCHEM  | TABLE_NAME  |  TABLE_TYPE   | REMARKS  | TYPE_NAME  | SELF_REFERENCING_COL_NAME  | REF_GENERATION  | INDEX_STATE  | IMMUTABLE_ROWS  | SALT_BUCKETS  | MULTI_TENANT  | VIEW_STA |
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+--------------+-----------------+---------------+---------------+----------+
|            | SYSTEM       | CATALOG     | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
|            | SYSTEM       | FUNCTION    | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
|            | SYSTEM       | LOG         | SYSTEM TABLE  |          |            |                            |                 |              | true            | 32            | false         |          |
|            | SYSTEM       | SEQUENCE    | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
|            | SYSTEM       | STATS       | SYSTEM TABLE  |          |            |                            |                 |              | false           | null          | false         |          |
|            |              | test        | VIEW          |          |            |                            |                 |              | false           | null          | false         |          |
+------------+--------------+-------------+---------------+----------+------------+----------------------------+-----------------+--------------+-----------------+---------------+---------------+----------+

hbase shell 上插入数据

hbase(main):006:0> put 'test','001','name:firtname','hello'
Took 0.1830 seconds
hbase(main):007:0> put 'test','001','name:firstname','hello'
Took 0.0244 seconds
hbase(main):008:0> put 'test','001','name:name','hello phonix'
Took 0.0071 seconds
hbase(main):009:0> put 'test','001','company:name','saleforce'
Took 0.0133 seconds

在Phoenix上查看视图test

0: jdbc:phoenix:node03,node04,node05:2181> select * from "test";
+--------+------------+-----------+------------+----------+
| EMPID  | firstname  | lastname  |    name    | address  |
+--------+------------+-----------+------------+----------+
| 001    | hello      |           | saleforce  |          |
+--------+------------+-----------+------------+----------+
1 row selected (0.05 seconds)

表映射
使用 Phoenix创建对 HBase 的表映射，有两种方法：

2.1. 当 HBase 中已经存在表时，可以以类似创建视图的方式创建关联表，只需要将create view改为create table即可。在 HBase 中创建表:

hbase(main):010:0> create 'test1', 'name', 'company'
Created table test1

0: jdbc:phoenix:node03,node04,node05:2181> create table "test1" 
(empid varchar primary key,
"name"."firstname" varchar,
"name"."lastname" varchar,
"company"."name" varchar,
"company"."address" varchar) 
column_encoded_bytes=0;

说明: 添加这个参数之后, 在 HBase 中添加的数据在 Phoenix 中也可以查询到. 否则 HBase 中添加的数据在 Phoenix 中查询不到

hbase(main):015:0> put 'test1','001','name:firstname','test1'
Took 0.0062 seconds

0: jdbc:phoenix:node03,node04,node05:2181> select * from "test1";
+--------+------------+-----------+-------+----------+
| EMPID  | firstname  | lastname  | name  | address  |
+--------+------------+-----------+-------+----------+
| 001    | test1      |           |       |          |
+--------+------------+-----------+-------+----------+

2.2 当 HBase 中不存在表时，可以直接使用create table指令创建需要的表,系统将会自动在 Phoenix 和 HBase 中创建 person_infomation 的表，并会根据指令内的参数对表结构进行初始化。
2.3 视图映射和表映射的对比与总结
相比于直接创建映射表，视图的查询效率会低，原因是：创建映射表的时候，Phoenix 会在表中创建一些空的键值对，这些空键值对的存在可以用来提高查询效率。
使用create table创建的关联表，如果对表进行了修改，源数据也会改变，同时如果关联表被删除，源表也会被删除。但是视图就不会，如果删除视图，源数据不会发生改变。

Phoenix 图形化工具: Squirrel

下载 Squirrel
http://squirrel-sql.sourceforge.net/
选择 java 打开方式
一路 next 安装
配置 Squirrel
4.1 复制 jar 包
复制 Phoenix 的 phoenix-4.14.2-HBase-1.3-client.jar 到 Squirrel 的安装目录下: 安装目录\lib
4.2 启动 Squirrel
4.3 配置 Driver

Snipaste_2020-08-04_17-48-22.png

Name=Phoenix
Example URL = jdbc:phoenix:node03,node04,node05:2181
JavaClass Path phoenix-4.14.2-HBase-1.3-client.jar
Class Name = org.apache.phoenix.jdbc.PhoenixDriver

4.4 配置Alias

Snipaste_2020-08-04_17-50-57.png

squirrel 连接Phoenix遇到的问题

error

dev_phoenix: ERROR 2007 (INT09): Outdated jars. Newer Phoenix clients can't communicate with older Phoenix servers. The following servers require an updated phoenix-[version]-server.jar to be put in the classpath of HBase: region=SYSTEM:CATALOG,,1596513241693.d18d6a556415cdf92d4e5220871f43f2., hostname=node02,16020,1596512982029, seqNum=2
class java.sql.SQLException: ERROR 2007 (INT09): Outdated jars. Newer Phoenix clients can't communicate with older Phoenix servers. The following servers require an updated phoenix-[version]-server.jar to be put in the classpath of HBase: region=SYSTEM:CATALOG,,1596513241693.d18d6a556415cdf92d4e5220871f43f2., hostname=node02,16020,1596512982029, seqNum=2

分析原因
Phoenix 版本5.0-Hbase-2.0 跟客户端driver phoenix-4.14.2-HBase-1.3 没对应上
解决方法
下载跟Phoenix 对应的版本，解压获取phoenix-5.0.0-HBase-2.0-client.jar 替代Squirrel 目录lib文件夹中的 phoenix-4.14.2-HBase-1.3-client.jar
http://phoenix.apache.org/download.html

error

test: ERROR 726 (43M10):  Inconsistent namespace mapping properties. Cannot initiate connection as SYSTEM:CATALOG is found but client does not have phoenix.schema.isNamespaceMappingEnabled enabled
class java.sql.SQLException: ERROR 726 (43M10):  Inconsistent namespace mapping properties. Cannot initiate connection as SYSTEM:CATALOG is found but client does not have phoenix.schema.isNamespaceMappingEnabled enabled

分析原因
缺少isNamespaceMappingEnabled 配置
解决方法
使用WinRAR 打开 phoenix-5.0.0-HBase-2.0-client.jar

image.png

获取和hbase-default.xml ，添加配置

image.png

   <property>
        <name>phoenix.schema.isNamespaceMappingEnabled</name>
        <value>true</value>
</property>
<property>
        <name>phoenix.schema.mapSystemTablesToNamespace</name>
        <value>true</value>
</property>

把hbase-default.xml 移动到 phoenix-5.0.0-HBase-2.0-client.jar 覆盖配置
重新squirrel连接测试成功

image.png

CDH安装phoenix
一、准备文件二、配置安装将文件放入 /opt/cloudera/csd 目录下重启 cloudera-sc...
解决：cdh版本的Phoenix安装的问题
问题背景：因为我安装的都是cdh5.7.0系列版本，Phoenix官网上没有cdh5.7.0版本，所以就看大神们...
CDH5.8 HBase安装Phoenix
1、phoenix的简介hbase的java api或者其语法很难用，可以认为phoenix是一个中间件，提供了访...
CDH5.8 HBase安装Phoenix
1、phoenix的简介hbase的java api或者其语法很难用，可以认为phoenix是一个中间件，提供了访...
Phoenix编译支持cdh5.10.2
前言由于Phoenix还没有发布支持cdh5.10以上的版本，这里在在4.9-HBase-1.2-cdh5.9的...
Phoenix从4.7升级到4.14
之前有过一篇文章讲述CDH5.16.1如何整合phoenix，但是，但是，但是，这个版本整合的是phoenix4....
phoenix for cloudera
软件版本: 下载CDH版本的phoenix，最新版本目前只有phoenix 4.9，不过在4.10开始才可以使用s...
大数据之CM+CDH5.16.1（二）
接着上一篇继续安装CDH环境三、安装CM和CDH 3.1 CDH下载以CentOS7.5和CDH5.16.1举...
Centos7.6部署CDH6.1.0大数据平台教程
目录一、CDH介绍二、CDH6.1.0安装前环境的部署三、CDH6.1.0安装四、CDH6.1.0的初始化...
Hbase-2.4.2（四）phoenix 安装及使用
安装 http://phoenix.apache.org/download.html[http://phoenix...