spark-sql使用hive无法创建表

作者: zhouhaolong1 | 来源:发表于2017-09-23 11:56 被阅读1269次

spark-sql使用hive无法创建表
Hive使用与Sqoop安装与使用
hive进阶学习
Hive SQL （HQL）小结
Spark SQL中Kryo反序列化问题分析
Hive日常使用
Hive练习（三）
2018-11-28
Hive wordcount
Hive框架基础（二）

使用spark-sql 执行hive的建表语句 create table person (id bigint,name string ,age int) row format delimited fields terminated by ','; 报出如下错误:

ERROR SparkSQLDriver: Failed in [create table person4 (id bigint,name string ,age int) row format delimited fields terminated by ',']
org.apache.spark.sql.execution.QueryExecutionException: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:file:/user/hive/warehouse/person4 is not a directory or unable to create one)

解决办法:

把$HIVE_HOME/conf/hive-site.xml复制到$SPARK_HOME/conf/下

在$SPARK_HOME/conf/hive-site.xml修改hive.metastore.warehouse.dir的属性值.其默认属性值是/user/hive/warehouse

 <property>

     <name>hive.metastore.warehouse.dir</name>

     <value>hdfs://mini1:9000/user/hive/warehouse</value>

     <description>hive.metastore.warehouse.dir</description>

 </property>

重启spark-sql

 spark-sql> exit;

 [hadoop@mini1 ~]$ spark-sql --master spark://mini1:7077

附上我的$SPARK_HOME/conf/hive-site.xml内容

    <configuration>
    <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://mini1:3306/hive?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
    </property>
    
    <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
    </property>
    
    <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>username to use against metastore database</description>
    </property>
    
    <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>root</value>
    <description>password to use against metastore database</description>
    </property>
    
    <!-- add by zhou ############################# -->
    
    <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>hdfs://mini1:9000/user/hive/warehouse</value>
    <description>hive.metastore.warehouse.dir</description>
    </property>
    
    <!-- add by zhou ############################# -->
    
    </configuration>