我最近开始使用Hadoop,我在使用Mapfile作为MapReduce作业的输入时遇到了问题.
下面的工作代码在hdfs中编写了一个名为"TestMap"的简单MapFile,其中有三个Text类型的键和三个类型为BytesWritable的值.
这里是TestMap的内容:
$ hadoop fs -text /user/hadoop/TestMap/data
11/01/20 11:17:58 INFO util.NativeCodeLoader: Loaded the native-hadoop library
11/01/20 11:17:58 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
11/01/20 11:17:58 INFO compress.CodecPool: Got brand-new decompressor
A 01
B 02
C 03
以下是创建TestMap Mapfile的程序:
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.io.MapFile;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.BytesWritable;
import org.apache.hadoop.io.SequenceFile;
import org.apache.hadoop.io.IOUtils;
public class CreateMap {
public static void main(String[] args) throws IOException{
Configuration conf = new Configuration();
FileSystem hdfs = FileSystem.get(conf);
Text key = …