auxService:在hive 1.2.0上不存在mapreduce_shuffle

Question

我使用的是hive 1.2.0和hadoop 2.6.0.每当我在我的机器上运行配置单元...选择查询工作正常,但如果count(*)它显示以下错误:

此任务的诊断消息:container_1434646588807_0001_01_000005的容器启动失败: org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException:sun.reflect.NativeConstructorAccessorImpl的sun.reflect.NativeConstructorAccessorImpl.newInstance0 (本地方法)中不存在auxService:mapreduce_shuffle. newInstance(NativeConstructorAccessorImpl.java:62)位于org.apache.hadoop.yarn.api的java.lang.reflect.Constructor.newInstance(Constructor.java:422)的sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)位于org.apache.hadoop的org.apache.hadoop.yarn.api.records.impl.pb.SerializedExceptionPBImpl.deSerialize(SerializedExceptionPBImpl.java:106)的.records.impl.pb.SerializedExceptionPBImpl.instantiateException(SerializedExceptionPBImpl.java:168) .mapreduce.v2.app.launcher.ContainerLauncherImpl $ Container.launch(ContainerLauncherImpl.java:155)at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl $ EventProce ssor.run(ContainerLauncherImpl.java:369)java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)java.lang上的java.util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617) .Thread.run(Thread.java:745)

FAILED:执行错误,从org.apache.hadoop.hive.ql.exec.mr.MapRedTask返回代码2启动MapReduce作业:Stage-Stage-1:Map:1 Reduce:1 HDFS读取:0 HDFS写入:0 FAIL Total MapReduce CPU耗时:0毫秒

Answer 1

请在yarn-site.xml中使用它; 当您将框架设置为用作纱线时,它会开始查找这些值.

<configuration>
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
</configuration>