如何在触发java代码时设置hadoop作业的优先级？

Question

这就是我的职业运动员的样子.如何在触发此作业时设置低优先级？

public int run(String[] args) throws Exception {
    this.initJob();
    Path outputPath = new Path(args[2]);
    FileInputFormat.setInputPaths(job, args[0]);
    FileOutputFormat.setOutputPath(job, outputPath);

    job.getConfiguration().set("FREQUENCY", args[3]);
    job.getConfiguration().set("TIMEZONE", args[4]);

    boolean rc = job.waitForCompletion(true);
    if (rc) {
        return 0;
    }
    return 1;
}
public void initJob() throws IOException {
    job = new Job(getConf(), "Stats Data Cruncher");
    job.setMapOutputKeyClass(Text.class);       
    job.setMapOutputValueClass(Text.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(NullWritable.class);
    Job.setInputFormatClass(LzoTextInputFormat.class);  

    job.setJarByClass(JobRunner.class);
    job.setMapperClass(StatsMapper.class);
    job.setCombinerClass(StatsCombiner.class);        
    job.setReducerClass(StatsReducer.class);
}

该行是否有效设置LOW优先级？

job.getConfiguration().set("PRIORITY", "LOW");

Answer 1

您可以使用三种方法来设置作业优先级:

1. 在xml配置文件中设置用户的作业优先级.

2. 使用Hadoop命令设置您的工作如下:

   hadoop job -set-priority

3. 在java代码中,您可以在配置中设置作业优先级:

   conf.set("mapred.job.priority",JOBPRIORITY.toString());

JOBPRIORITY的值可能是:

JobPriority.VERY_HIGH

JobPriority.HIGH

JobPriority.NORMAL

JobPriority.LOW

JobPriority.VERY_LOW