Ark*_*ski 8 hadoop hive apache-spark apache-spark-sql
通过spark-shell
2.0 查询Hive时:
spark.sql("SELECT * FROM schemaname.tablename")
Run Code Online (Sandbox Code Playgroud)
它抛出一个错误:
16/08/13 09:24:17 INFO execution.SparkSqlParser: Parsing command: SELECT * FROM schemaname.tablename
org.apache.spark.sql.AnalysisException: Table or view not found: `schemaname`.`tablename`; line 1 pos 14
...
Run Code Online (Sandbox Code Playgroud)
Hive访问似乎通过正确配置hive-site.xml
.在shell中Spark正在打印:
scala> spark.conf.get("spark.sql.warehouse.dir")
res5: String = /user/hive/warehouse
Run Code Online (Sandbox Code Playgroud)
在内部conf/hive-site.xml
,配置了Hive,可以在Spark上访问其配置.列出数据库时,它显示现有default
数据库.但它没有显示内部的表格default
.
scala> spark.catalog.listDatabases.show(false)
+-------+----------------+---------------------------------------------+
|name |description |locationUri |
+-------+----------------+---------------------------------------------+
|default|default database|hdfs://hdfs-server-uri:8020/user/hive/warehouse|
+-------+----------------+---------------------------------------------+
scala> spark.catalog.listTables("default").show()
+----+--------+-----------+---------+-----------+
|name|database|description|tableType|isTemporary|
+----+--------+-----------+---------+-----------+
+----+--------+-----------+---------+-----------+
Run Code Online (Sandbox Code Playgroud)
访问Hive时可能会遗漏什么?
归档时间: |
|
查看次数: |
9971 次 |
最近记录: |