hive启用sentry

基于CDH6.2.0环境

安装sentry服务

在cloudera manager中添加sentry服务：

1595380703.jpg 1595380751(1).jpg 1595380789(1).jpg

并在hive中开启sentry服务：

1595381082(1).jpg
1595381142(1).jpg

如果集群中没有开启kerbos，需要在sentry-site.xml中添加如下配置：

<property>
  <name>sentry.hive.testing.mode</name>
  <value>true</value>
</property>

企业微信截图_1595389436803.png

在beeline中登录hive：

beeline -u jdbc:hive2://YOUR_HIVE_HOST:10000/default -n hive

执行：

create role admin_role;
grant all on server server1 to role admin_role;
grant role admin_role to group hive;

这样就可以通过hive用户管理sentry权限

问题

在cdh文档中，sentry创建函数需要CREATE ON SERVER或CREATE ON DATABASE权限，但是实测不需要（使用Direct JAR Reference Configuration，未使用推荐的Reloadable Aux JAR Configuration），只需要给对应的hdfs uri权限以及/tmp目录权限即可，暂时没深究

0: jdbc:hive2://cdh1:10000/default> show current roles;
...
+-------------------+
|       role        |
+-------------------+
| david_2020_test1  |
+-------------------+
...
0: jdbc:hive2://cdh1:10000/default> show grant role david_2020_test1;
...
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+----------------+----------+
|                    database                     | table  | partition  | column  |  principal_name   | principal_type  | privilege  | grant_option  |   grant_time   | grantor  |
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+----------------+----------+
| default                                         |        |            |         | david_2020_test1  | ROLE            | SELECT     | false         | 1595414036000  | --       |
| file:///tmp/                                    |        |            |         | david_2020_test1  | ROLE            | *          | false         | 1595413722000  | --       |
| hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar  |        |            |         | david_2020_test1  | ROLE            | *          | false         | 1595390462000  | --       |
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+----------------+----------+
...
0: jdbc:hive2://cdh1:10000/default> create function LowerUDF as 'com.test.LowerUDF' using jar 'hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar';
INFO  : Compiling command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6): create function LowerUDF as 'com.test.LowerUDF' using jar 'hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar'
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO  : Completed compiling command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6); Time taken: 0.125 seconds
INFO  : Executing command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6): create function LowerUDF as 'com.test.LowerUDF' using jar 'hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar'
INFO  : Starting task [Stage-0:FUNC] in serial mode
INFO  : Added [/tmp/a114e074-ee0e-4342-bb12-04e6e83f42ae_resources/test-udf-1.0-SNAPSHOT.jar] to class path
INFO  : Added resources: [hdfs://cdh1:8020/tmp/test-udf-1.0-SNAPSHOT.jar]
INFO  : Completed executing command(queryId=hive_20200723154145_61a10681-8dca-44ae-b62a-8f19e1824ba6); Time taken: 0.017 seconds
INFO  : OK
No rows affected (0.158 seconds)

需要 file:///tmp/权限的原因在HiveAuthzBindingHook中

     case HiveParser.TOK_CREATEFUNCTION:
        String udfClassName = BaseSemanticAnalyzer.unescapeSQLString(ast.getChild(1).getText());
        try {
          CodeSource udfSrc =
              Class.forName(udfClassName, true, Utilities.getSessionSpecifiedClassLoader())
                  .getProtectionDomain().getCodeSource();
          if (udfSrc == null) {
            throw new SemanticException("Could not resolve the jar for UDF class " + udfClassName);
          }
          String udfJar = udfSrc.getLocation().getPath();
          if (udfJar == null || udfJar.isEmpty()) {
            throw new SemanticException("Could not find the jar for UDF class " + udfClassName +
                "to validate privileges");
          }
          udfURIs.add(parseURI(udfSrc.getLocation().toString(), true));
        } catch (ClassNotFoundException e) {
          List<String> functionJars = getFunctionJars(ast);
          if (functionJars.isEmpty()) {
            throw new SemanticException("Error retrieving udf class:" + e.getMessage(), e);
          } else {
            // Add the jars from the command "Create function using jar" to the access list
            // Defer to hive to check if the class is in the jars
            for(String jar : functionJars) {
              udfURIs.add(parseURI(jar, false));
            }
          }
        }

        // create/drop function is allowed with any database
        currDB = Database.ALL;
        break;

• udfSrc会从classpath中找udf对应的jar包，如果找不到，抛出ClassNotFoundException，使用create function语句中的uri。
• 函数一旦创建，当前session会创建类似/tmp/a114e074-ee0e-4342-bb12-04e6e83f42ae_resources/xxx.jar之类的缓存，并加入classpath。
• 后续语句的校验都会走classpath流程，且获取的uri路径是file:///tmp/a114e074-ee0e-4342-bb12-04e6e83f42ae_resources/xxx.jar，如果没有file:///tmp/权限，则权限校验会报错。
• 这可能是sentry的一个bug，获取jar资源的uri应该永远返回using jar语句中的uri才对。

参考：
Direct JAR Reference Configuration
Hive Aux JARs Directory Configuration
Reloadable Aux JAR Configuration
Managing the Sentry Service
Managing Apache Hive User-Defined Functions