相关疑难解决方法(0)

hadoop中的MultipleOutputFormat

我是Hadoop的新手.我正在尝试Wordcount程序.

现在尝试多个输出文件,我使用MultipleOutputFormat.这个链接帮助我做到了这一点.http://hadoop.apache.org/common/docs/r0.19.0/api/org/apache/hadoop/mapred/lib/MultipleOutputs.html

在我的司机课上我有

    MultipleOutputs.addNamedOutput(conf, "even",
            org.apache.hadoop.mapred.TextOutputFormat.class, Text.class,
            IntWritable.class);

    MultipleOutputs.addNamedOutput(conf, "odd",
            org.apache.hadoop.mapred.TextOutputFormat.class, Text.class,
            IntWritable.class);`
Run Code Online (Sandbox Code Playgroud)

而我的减少课就变成了这个

public static class Reduce extends MapReduceBase implements
        Reducer<Text, IntWritable, Text, IntWritable> {
    MultipleOutputs mos = null;

    public void configure(JobConf job) {
        mos = new MultipleOutputs(job);
    }

    public void reduce(Text key, Iterator<IntWritable> values,
            OutputCollector<Text, IntWritable> output, Reporter reporter)
            throws IOException {
        int sum = 0;
        while (values.hasNext()) {
            sum += values.next().get();
        }
        if (sum % 2 == 0) {
            mos.getCollector("even", reporter).collect(key, new IntWritable(sum));
        }else …
Run Code Online (Sandbox Code Playgroud)

java hadoop mapreduce

16
推荐指数
1
解决办法
8190
查看次数

标签 统计

hadoop ×1

java ×1

mapreduce ×1