Joh*_*nGa 10 java hadoop mapreduce
public static class Map extends MapReduceBase implements Mapper
Run Code Online (Sandbox Code Playgroud)
MapReduceBase
,Mapper
并JobConf
在Hadoop 0.20.203中弃用.
我们现在应该怎么用?
编辑1 - 对于Mapper
和MapReduceBase
,我发现我们只需要扩展Mapper
public static class Map extends Mapper
<LongWritable, Text, Text, IntWritable> {
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(LongWritable key, Text value,
OutputCollector<Text, IntWritable> output,
Reporter reporter) throws IOException {
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
word.set(tokenizer.nextToken());
output.collect(word, one);
}
}
}
Run Code Online (Sandbox Code Playgroud)
编辑2 - 因为JobConf
我们应该使用这样的配置:
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = new Job(conf);
job.setMapperClass(WordCount.Map.class);
}
Run Code Online (Sandbox Code Playgroud)
编辑3 - 我根据新的API找到了一个很好的教程:http://sonerbalkir.blogspot.com/2010/01/new-hadoop-api-020x.html
Javadoc包含有关使用此depellyated类的instaed的信息:
例如http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/JobConf.html
Deprecated. Use Configuration instead
Run Code Online (Sandbox Code Playgroud)
编辑:当您使用Maven和开放类的声明(F3)Maven可以自动下载源代码,你会看到与解释javadoc注释的内容.
归档时间: |
|
查看次数: |
9538 次 |
最近记录: |