Cho*_*een 1 build bazel apache-flink
如何在进行 bazel 构建时传递参数。实际上我的程序将接受两个参数。一个是目录,另一个是 target.csv 我需要获取目录下的文件并将其写入 csv。
import java.io.File;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collector;
import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.DataSet;
import org.apache.flink.api.java.ExecutionEnvironment;
import org.apache.flink.api.java.tuple.Tuple1;
import org.apache.flink.api.java.utils.ParameterTool;
public class ReadFiles {
/**
* @param args
*/
public static void main(String[] args) throws Exception {
// set up the execution environment
try {
final ParameterTool params = ParameterTool.fromArgs(args);
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1); // without this multiple files creating
env.getConfig().setGlobalJobParameters(params);
List<String> paths = new ArrayList<String>();
File dir = new File(params.getRequired("input"));
for (File f : dir.listFiles()) {
if (f.isFile()) {
paths.add(f.getName());
}
}
DataSet<String> data = env.fromCollection(paths).rebalance();
DataSet<Tuple1<String>> output = data.flatMap(new CSVSplitter());
env.execute();
} catch (Exception e) {
e.printStackTrace();
}
}
}
// The operations are defined by specialized classes, here the Splitter class.
class CSVSplitter implements FlatMapFunction<String, Tuple1<String>> {
public void flatMap(String value, Collector<Tuple1<String>> out) throws Exception {
out.collect(new Tuple1<String>(value));
}
}
Run Code Online (Sandbox Code Playgroud)
以及我正在尝试的构建 bazel 的语法 -
bazel build FlinkEx/com/practice:read_files
Run Code Online (Sandbox Code Playgroud)
执行此操作后面临问题。
Worker process sent response with exit code: 1.
error: wrong number of type arguments; required 3
Run Code Online (Sandbox Code Playgroud)
我需要传递类似 bazel build FlinkEx/com/practice:read_files c:/ c:/target.csv 之类的参数
你的意思是bazel run?您可以通过将参数与 Bazel 调用分开来传递参数--:
bazel run FlinkEx/com/practice:read_files -- c:/ c:/target.csv
Run Code Online (Sandbox Code Playgroud)
或者您可以直接运行二进制文件并“正常”传递参数:
bazel build FlinkEx/com/practice:read_files
bazel-bin/FlinkEx/com/practice:read_files c:/ c:/target.csv
Run Code Online (Sandbox Code Playgroud)
(这是假设bazel build是把read_files在bazel-bin。如果它下bazel-genfiles,刚刚从那里运行它,而不是。)
| 归档时间: |
|
| 查看次数: |
6253 次 |
| 最近记录: |