1、修改字段注释字符集
alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
2、修改表注释字符集
alter table TABLE_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;
3、修改分区表参数，以支持分区键能够用中文表示。
alter table PARTITION_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;
alter table PARTITION_KEYS modify column PKEY_COMMENT varchar(4000) character set utf8;
select count(*) 得0的问题，hive表有记录，涉及count计算有关设置
hive默认设置 hive.compute.query.using.stats=true //改为false 不从元数据取值，可计数，但性能不高，不推荐
set hive.compute.query.using.stats=false
beeline 后台执行
/home/ocdp/.bash_profile 添加如下行
export HADOOP_CLIENT_OPTS="-Djline.terminal=jline.UnsupportedTerminal"
递归删除partion
#!/bin/bash
for i in 201704 201705 201706 201707 201708 201709 201710 201711 201712 201801 201802 201803 201804
do
for j in 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
do
echo "alter table bass.kpi_scp_cd005 drop if exists partition(stat_month="\'${i}\'",stat_date="\'${i}${j}\'");"
beeline -u jdbc:hive2://10.31.10.29:10000/bass -n bass -p bass -e "alter table bass.kpi_scp_cd005 drop if exists partition(stat_month="\'${i}\'",stat_date="\'${i}${j}\'");"
done
done
exit 0
更改hive GC CMS 方式
Ambari > Hive > Configs > Advanced hive-env > hive-env template > Add the follow properties.
if [ "$SERVICE" = "hiveserver2" ]; then
if [ -z "$DEBUG" ]; then
export HADOOP_OPTS="-server -XX:ParallelGCThreads=30 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hive/$USER/hs_err_pid%p.log -XX:NewSize=2048m -XX:MaxNewSize=2048m -Xloggc:/var/log/hadoop/$USER/gc.log-`date +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Xms4096m -Xmx193275m"
else
export HADOOP_OPTS="-server -XX:ParallelGCThreads=30 -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/hive/$USER/hs_err_pid%p.log -XX:NewSize=2048m -XX:MaxNewSize=2048m -Xloggc:/var/log/hadoop/$USER/gc.log-`date +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Xms4096m -Xmx193275m"
fi
fi
Make sure that the new settings has been applied successfully, as well as the heap sizes.
/usr/jdk64/jdk1.8.0_112/bin/jcmd <hiveserver_pid> VM.flags
/usr/jdk64/jdk1.8.0_112/bin/jmap -heap <hiveserver_pid>
网友评论