为仅地图作业运行Hadoop Map Reduce时出错

Question

我想运行一个只有地图的工作Hadoop MapReduce,这是我的代码:

    Configuration conf = new Configuration();
    Job job = new Job(conf);
    job.setJobName("import");

    job.setMapperClass(Map.class);//Custom Mapper
    job.setInputFormatClass(TextInputFormat.class);
    job.setNumReduceTasks(0);

    TextInputFormat.setInputPaths(job, new Path("/home/jonathan/input"));

但我得到错误:

13/07/17 18:22:48 ERROR security.UserGroupInformation: PriviledgedActionException
as: jonathan cause:org.apache.hadoop.mapred.InvalidJobConfException: 
Output directory not set. 
Exception in thread "main" org.apache.hadoop.mapred.InvalidJobConfException: 
 Output directory not set.

然后我试着用这个:

job.setOutputFormatClass(org.apache.hadoop.mapred.lib.NullOutputFormat.class);

但它给了我一个编译错误:

java: method setOutputFormatClass in class org.apache.hadoop.mapreduce.Job 
  cannot be applied to given types;
  required: java.lang.Class<? extends org.apache.hadoop.mapreduce.OutputFormat>
  found: java.lang.Class<org.apache.hadoop.mapred.lib.NullOutputFormat>
  reason: actual argument java.lang.Class
  <org.apache.hadoop.mapred.lib.NullOutputFormat> cannot be converted to 
  java.lang.Class<? extends org.apache.hadoop.mapreduce.OutputFormat> 
  by method invocation conversion

我究竟做错了什么？

Answer 1

仅映射作业仍需要指定输出位置.正如错误所说,你没有指明这一点.

我认为你的意思是你的工作完全没有输出.Hadoop仍然希望您指定输出位置,但不需要编写任何内容.

你org.apache.hadoop.mapreduce.lib.output.NullOutputFormat不想要org.apache.hadoop.mapred.lib.NullOutputFormat,这是第二个错误表明虽然它是微妙的.