相关疑难解决方法(0)

我在HDP 2.3上有一个Spark(版本1.4.1)应用程序.在YARN-Client模式下运行它时工作正常.但是,在YARN-Cluster模式下运行时,应用程序无法找到任何Hive表.

我这样提交申请:

./bin/spark-submit 
    --class com.myCompany.Main 
    --master yarn-cluster 
    --num-executors 3 
    --driver-memory 4g 
    --executor-memory 10g 
    --executor-cores 1 
    --jars lib/datanucleus-api-jdo-3.2.6.jar,lib/datanucleus-rdbms-3.2.9.jar,lib/datanucleus-core-3.2.10.jar /home/spark/apps/YarnClusterTest.jar  
    --files /etc/hive/conf/hive-site.xml

这是日志的摘录:

5/12/02 11:05:13 INFO hive.HiveContext: Initializing execution hive, version 0.13.1
15/12/02 11:05:14 INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
15/12/02 11:05:14 INFO metastore.ObjectStore: ObjectStore, initialize called
15/12/02 11:05:14 INFO DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
15/12/02 11:05:14 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
15/12/02 11:05:14 INFO storage.BlockManagerMasterEndpoint: Registering block manager worker2.xxx.com:34697 with …