使用 HBase 和 Parquet,我编写了代码来从 HBase 获取值并将值映射到 Object 类,但是我无法使用 Dataset 用 Parquet 复制它。
HBase:
JavaPairRDD<ImmutableBytesWritable, Result> data = sc.newAPIHadoopRDD(getHbaseConf(),
TableInputFormat.class, ImmutableBytesWritable.class, Result.class);
JavaRDD<List<Tuple3<Long, Integer, Double>>> tempData = data
.values()
//Uses HBaseResultToSimple... class to parse the data.
.map(value -> {
SimpleObject object = oParser.call(value);
// Get the sample property, remove leading and ending spaces and split it by comma
// to get each sample individually
List<Tuple2<String, Integer>> samples = zipWithIndex((object.getSamples().trim().split(",")));
// Gets the unique identifier for that sp.
Long sp = object.getPos(); …Run Code Online (Sandbox Code Playgroud)