hive 开启 python udf

背景

hive需要开启python udf功能，线上代码分支丢失，只能其他分支的一个类进行替换

在不开启时执行transfrom语句报错

   org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAccessControlException: Query with transform clause is disallowed in current configuration.
            at org.apache.hadoop.hive.ql.security.authorization.plugin.DisallowTransformHook.run(DisallowTransformHook.java:34)
            at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1681)
            at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1166)
            at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:242)
            at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
            at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:334)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
            at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:347)
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)

操作

1、修改代码

源代码

public class DisallowTransformHook implements ExecuteWithHookContext {

      @Override
      public void run(HookContext hookContext) throws Exception {
        QueryProperties qProps = hookContext.getQueryPlan().getQueryProperties();
        if (null == qProps) {
          return; // its a ddl query.
        } 
        if (qProps.usesScript()) {
           throw new HiveAccessControlException("Query with transform clause is disallowed in"
               + " current configuration.");
        }
      }
    }

修改

public class DisallowTransformHook implements ExecuteWithHookContext {
      @Override
      public void run(HookContext hookContext) throws Exception {
        QueryProperties qProps = hookContext.getQueryPlan().getQueryProperties();
        if (null == qProps) {
          return; // its a ddl query.
        } 
    //    if (qProps.usesScript()) {
    //       throw new HiveAccessControlException("Query with transform clause is disallowed in"
    //           + " current configuration.");
    //    }
      }
    }

2、编译打包

 mvn clean install -Dmaven.test.skip=true

3、替换包class文件

3.1、将线上包复制

cp $HIVE_HOME/lib/hive-exec-2.1.1.jar   /work/hive-exec-2.1.1.zip

3.2、解压包

unzip /work/hive-exec-2.1.1.zip -d /work

3.3、替换文件

将编译后的包在其他目录执行上述步骤，并将文件替换到线上包中

3.4、重新打包

 cd /work
 zip -r hive-exec.zip *
 mv hive-exec.zip hive-exec-2.1.1.jar

3.5、替换原有hive-exec-2.1.1.jar并重启hive

成功demo

image.png