Hive(3.1) debug env with Intelli

Prepare

please install JDK-1.8, Hadoop3.1 and get Hive-3.1 source code first

• JDK
  JDK-version=1.8
  export JAVA_HOME=/path/to/jdk/home

• Hadoop
  hadoop-version=3.1
  export HADOOP_HOME=/path/to/hadoop/home

• Hive Source Code
  hive-version=3.1

• IntelliJ IDEA

Start Hadoop

cd $HADOOP_HOME
./sbin/start-all.sh

Hive package install

cd /path/to/hive/source
mvn clean package -DskipTests -Phadoop-2 -Pdist

Hive config

only when the last step finished(hive package install)

cd $HIVE_SRC/packaging/target/apache-hive-{version}-bin/apache-hive-{version}-bin/conf
# find HADOOP_HOME in hive-env.sh, replace it to our $HADOOP_HOME
cd $HIVE_SRC/packaging/target/apache-hive-{version}-bin/apache-hive-{version}-bin

# init metadata database(using derby, if you need mysql, you should install mysql first)
bin/schematool -dbType derby -initSchema

Hive debug

start hive debug console(listening port 8000 default)

cd $HIVE_SRC/packaging/target/apache-hive-{version}-bin/apache-hive-{version}-bin
bin/hive --debug -hiveconf hive.root.logger=DEBUG,console

Config IDEA

• find class CliDriver(entry for hive cli)
• Edit configurations -> add -> choose "remote"
  debug.jpg

Finally!

We can add break points in idea for debugging, try to input "show databases" in console(which we execute the command "bin/hive --debug -hiveconf hive.root.logger=DEBUG,console")