hive自定义udf的pom文件,可自动加载依赖包

作者: 堂哥000 | 来源:发表于2022-01-05 09:57 被阅读0次

hive自定义udf的pom文件,可自动加载依赖包
Scala实现hive自定义函数udf
Hive 3 - UDF函数
2019-10-12_springboot配置双数据源并调用
spark读写mysql、hive、kafka数据demo
Hive从入门到精通8：Hive自定义函数（UDF）
python udf 函数使用
【Hive】Hive UDF
java读取hive的orc文件
自定义UDF函数，从hive保存到mongodb

执行注意jdk和jre版本是否一致

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.example</groupId>
  <artifactId>demoUDF</artifactId>
  <version>1.0-SNAPSHOT</version>
  <properties>
    <java.version>1.8</java.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
    <scala.version>2.11.7</scala.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.apache.hive</groupId>
      <artifactId>hive-exec</artifactId>
      <version>2.3.1</version>
      <exclusions>
        <exclusion>
          <artifactId>ST4</artifactId>
          <groupId>org.antlr</groupId>
        </exclusion>
        <exclusion>
          <artifactId>antlr-runtime</artifactId>
          <groupId>org.antlr</groupId>
        </exclusion>
        <exclusion>
          <artifactId>avro</artifactId>
          <groupId>org.apache.avro</groupId>
        </exclusion>
        <exclusion>
          <artifactId>commons-collections</artifactId>
          <groupId>commons-collections</groupId>
        </exclusion>
        <exclusion>
          <artifactId>commons-compress</artifactId>
          <groupId>org.apache.commons</groupId>
        </exclusion>
        <exclusion>
          <artifactId>slf4j-log4j12</artifactId>
          <groupId>org.slf4j</groupId>
        </exclusion>
        <exclusion>
          <artifactId>slf4j-api</artifactId>
          <groupId>org.slf4j</groupId>
        </exclusion>
        <exclusion>
          <artifactId>servlet-api</artifactId>
          <groupId>javax.servlet</groupId>
        </exclusion>
        <exclusion>
          <artifactId>protobuf-java</artifactId>
          <groupId>com.google.protobuf</groupId>
        </exclusion>
        <exclusion>
          <artifactId>metrics-core</artifactId>
          <groupId>io.dropwizard.metrics</groupId>
        </exclusion>
        <exclusion>
          <artifactId>log4j</artifactId>
          <groupId>log4j</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jline</artifactId>
          <groupId>jline</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jackson-xc</artifactId>
          <groupId>org.codehaus.jackson</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jackson-mapper-asl</artifactId>
          <groupId>org.codehaus.jackson</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jsr305</artifactId>
          <groupId>com.google.code.findbugs</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jackson-jaxrs</artifactId>
          <groupId>org.codehaus.jackson</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jackson-databind</artifactId>
          <groupId>com.fasterxml.jackson.core</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jackson-core-asl</artifactId>
          <groupId>org.codehaus.jackson</groupId>
        </exclusion>
        <exclusion>
          <artifactId>httpcore</artifactId>
          <groupId>org.apache.httpcomponents</groupId>
        </exclusion>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>org.apache.httpcomponents</groupId>
        </exclusion>
        <exclusion>
          <artifactId>hadoop-annotations</artifactId>
          <groupId>org.apache.hadoop</groupId>
        </exclusion>
        <exclusion>
          <artifactId>guava</artifactId>
          <groupId>com.google.guava</groupId>
        </exclusion>
        <exclusion>
          <artifactId>curator-framework</artifactId>
          <groupId>org.apache.curator</groupId>
        </exclusion>
        <exclusion>
          <artifactId>curator-client</artifactId>
          <groupId>org.apache.curator</groupId>
        </exclusion>
        <exclusion>
          <artifactId>commons-logging</artifactId>
          <groupId>commons-logging</groupId>
        </exclusion>
        <exclusion>
          <artifactId>commons-lang3</artifactId>
          <groupId>org.apache.commons</groupId>
        </exclusion>
        <exclusion>
          <artifactId>commons-lang</artifactId>
          <groupId>commons-lang</groupId>
        </exclusion>
        <exclusion>
          <artifactId>commons-httpclient</artifactId>
          <groupId>commons-httpclient</groupId>
        </exclusion>
        <exclusion>
          <artifactId>zookeeper</artifactId>
          <groupId>org.apache.zookeeper</groupId>
        </exclusion>
        <exclusion>
          <artifactId>mail</artifactId>
          <groupId>javax.mail</groupId>
        </exclusion>
        <exclusion>
          <artifactId>log4j-web</artifactId>
          <groupId>org.apache.logging.log4j</groupId>
        </exclusion>
        <exclusion>
          <artifactId>log4j-slf4j-impl</artifactId>
          <groupId>org.apache.logging.log4j</groupId>
        </exclusion>
        <exclusion>
          <artifactId>log4j-core</artifactId>
          <groupId>org.apache.logging.log4j</groupId>
        </exclusion>
        <exclusion>
          <artifactId>xmlenc</artifactId>
          <groupId>xmlenc</groupId>
        </exclusion>
        <exclusion>
          <artifactId>velocity</artifactId>
          <groupId>org.apache.velocity</groupId>
        </exclusion>
        <exclusion>
          <artifactId>stax-api</artifactId>
          <groupId>stax</groupId>
        </exclusion>
        <exclusion>
          <artifactId>stax-api</artifactId>
          <groupId>javax.xml.stream</groupId>
        </exclusion>
        <exclusion>
          <artifactId>pentaho-aggdesigner-algorithm</artifactId>
          <groupId>org.pentaho</groupId>
        </exclusion>
        <exclusion>
          <artifactId>parquet-hadoop-bundle</artifactId>
          <groupId>org.apache.parquet</groupId>
        </exclusion>
        <exclusion>
          <artifactId>metrics-json</artifactId>
          <groupId>io.dropwizard.metrics</groupId>
        </exclusion>
        <exclusion>
          <artifactId>orc-core</artifactId>
          <groupId>org.apache.orc</groupId>
        </exclusion>
        <exclusion>
          <artifactId>activation</artifactId>
          <groupId>javax.activation</groupId>
        </exclusion>
        <exclusion>
          <artifactId>ant</artifactId>
          <groupId>org.apache.ant</groupId>
        </exclusion>
        <exclusion>
          <artifactId>ant-launcher</artifactId>
          <groupId>org.apache.ant</groupId>
        </exclusion>
        <exclusion>
          <artifactId>aopalliance</artifactId>
          <groupId>aopalliance</groupId>
        </exclusion>
        <exclusion>
          <artifactId>apache-curator</artifactId>
          <groupId>org.apache.curator</groupId>
        </exclusion>
        <exclusion>
          <artifactId>apacheds-i18n</artifactId>
          <groupId>org.apache.directory.server</groupId>
        </exclusion>
        <exclusion>
          <artifactId>apacheds-kerberos-codec</artifactId>
          <groupId>org.apache.directory.server</groupId>
        </exclusion>
        <exclusion>
          <artifactId>api-asn1-api</artifactId>
          <groupId>org.apache.directory.api</groupId>
        </exclusion>
        <exclusion>
          <artifactId>api-util</artifactId>
          <groupId>org.apache.directory.api</groupId>
        </exclusion>
        <exclusion>
          <artifactId>asm</artifactId>
          <groupId>asm</groupId>
        </exclusion>
        <exclusion>
          <artifactId>asm-commons</artifactId>
          <groupId>asm</groupId>
        </exclusion>
        <exclusion>
          <artifactId>avatica</artifactId>
          <groupId>org.apache.calcite.avatica</groupId>
        </exclusion>
        <exclusion>
          <artifactId>avatica-metrics</artifactId>
          <groupId>org.apache.calcite.avatica</groupId>
        </exclusion>
        <exclusion>
          <artifactId>calcite-core</artifactId>
          <groupId>org.apache.calcite</groupId>
        </exclusion>
        <exclusion>
          <artifactId>calcite-druid</artifactId>
          <groupId>org.apache.calcite</groupId>
        </exclusion>
        <exclusion>
          <artifactId>calcite-linq4j</artifactId>
          <groupId>org.apache.calcite</groupId>
        </exclusion>
        <exclusion>
          <artifactId>commons-beanutils</artifactId>
          <groupId>commons-beanutils</groupId>
        </exclusion>
        <exclusion>
          <artifactId>gson</artifactId>
          <groupId>com.google.code.gson</groupId>
        </exclusion>
        <exclusion>
          <artifactId>groovy-all</artifactId>
          <groupId>org.codehaus.groovy</groupId>
        </exclusion>
        <exclusion>
          <artifactId>datanucleus-core</artifactId>
          <groupId>org.datanucleus</groupId>
        </exclusion>
        <exclusion>
          <artifactId>ivy</artifactId>
          <groupId>org.apache.ivy</groupId>
        </exclusion>
        <exclusion>
          <artifactId>hive-vector-code-gen</artifactId>
          <groupId>org.apache.hive</groupId>
        </exclusion>
        <exclusion>
          <artifactId>hive-shims</artifactId>
          <groupId>org.apache.hive</groupId>
        </exclusion>
        <exclusion>
          <artifactId>libfb303</artifactId>
          <groupId>org.apache.thrift</groupId>
        </exclusion>
        <exclusion>
          <artifactId>hive-llap-tez</artifactId>
          <groupId>org.apache.hive</groupId>
        </exclusion>
        <exclusion>
          <artifactId>log4j-1.2-api</artifactId>
          <groupId>org.apache.logging.log4j</groupId>
        </exclusion>
        <exclusion>
          <artifactId>log4j-api</artifactId>
          <groupId>org.apache.logging.log4j</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.bouncycastle</groupId>
      <artifactId>bcprov-jdk15on</artifactId>
      <version>1.52</version>
    </dependency>

  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.6.1</version>
        <configuration>
          <compilerVersion>${java.version}</compilerVersion>
          <source>${java.version}</source>
          <target>${java.version}</target>
          <encoding>UTF-8</encoding>
          <useIncrementalCompilation>false</useIncrementalCompilation>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <configuration>
          <archive>
            <manifest>
              <mainClass>com.demo.udf</mainClass>
            </manifest>
          </archive>
          <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
          <appendAssemblyId>false</appendAssemblyId>
        </configuration>
        <executions>
          <execution>
            <id>make-assembly</id>
            <phase>package</phase>
            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

</project>

hive自定义udf的pom文件,可自动加载依赖包
执行注意jdk和jre版本是否一致
Scala实现hive自定义函数udf
1、hive自定义函数引入pom依赖注意版本号依据自己实际情况修改。继承UDF类重写evaluate方法 ...
Hive 3 - UDF函数
Eclipse编写自定义UDF函数步骤 1、配置 pom 添加两个dependency 2、下载 hive-sit...
2019-10-12_springboot配置双数据源并调用
项目结构 pom文件jar包依赖 application.properties配置自定义配置
spark读写mysql、hive、kafka数据demo
读取hive库数据 pom.xml依赖配置读取hive数据demo 读取mysql数据 pom.xml配置文件 ...
Hive从入门到精通8：Hive自定义函数（UDF）
1.什么是Hive UDF？ Hive中的用户自定义函数（即User Defined Function，简称UDF...
python udf 函数使用
参考资料用python实现Hive中的UDF函数 1.创建表 ——hive窗口或者命令行 2.加载数据文件路径...
【Hive】Hive UDF
[TOC] 一、UDF 介绍 UDF（User-Defined Functions）即是用户自定义的hive函数。...
java读取hive的orc文件
hive API2hive udf 读取HDFS的orc文件
自定义UDF函数，从hive保存到mongodb
（可以通过idea工具调试UDF函数，第二步中会提供参考）一、自定义UDF函数： 1、首先是pom.xml文件 ...