如何使用Google的Cloud Command Tool Tool/SDK构建Docker容器？
网址https://sdk.cloud.google.com上的脚本似乎需要用户输入,因此无法在docker文件中使用.
当我运行我的Dataflow管道时,我得到以下异常抱怨我的DoFn无法序列化.我该如何解决？
这是堆栈跟踪:
Caused by: java.lang.IllegalArgumentException: unable to serialize contrail.dataflow.AvroMRTransforms$AvroReducerDoFn@bba0fc2
at com.google.cloud.dataflow.sdk.util.SerializableUtils.serializeToByteArray(SerializableUtils.java:51)
at com.google.cloud.dataflow.sdk.util.SerializableUtils.ensureSerializable(SerializableUtils.java:81)
at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner$Evaluator.ensureSerializable(DirectPipelineRunner.java:784)
at com.google.cloud.dataflow.sdk.transforms.ParDo.evaluateHelper(ParDo.java:1025)
at com.google.cloud.dataflow.sdk.transforms.ParDo.evaluateSingleHelper(ParDo.java:963)
at com.google.cloud.dataflow.sdk.transforms.ParDo.access$000(ParDo.java:441)
at com.google.cloud.dataflow.sdk.transforms.ParDo$1.evaluate(ParDo.java:951)
at com.google.cloud.dataflow.sdk.transforms.ParDo$1.evaluate(ParDo.java:946)
at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner$Evaluator.visitTransform(DirectPipelineRunner.java:611)
at com.google.cloud.dataflow.sdk.runners.TransformTreeNode.visit(TransformTreeNode.java:200)
at com.google.cloud.dataflow.sdk.runners.TransformTreeNode.visit(TransformTreeNode.java:196)
at com.google.cloud.dataflow.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:109)
at com.google.cloud.dataflow.sdk.Pipeline.traverseTopologically(Pipeline.java:204)
at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner$Evaluator.run(DirectPipelineRunner.java:584)
at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.run(DirectPipelineRunner.java:328)
at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.run(DirectPipelineRunner.java:70)
at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:145)
at contrail.stages.DataflowStage.stageMain(DataflowStage.java:51)
at contrail.stages.NonMRStage.execute(NonMRStage.java:130)
at contrail.stages.NonMRStage.run(NonMRStage.java:157)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at contrail.stages.ValidateGraphDataflow.main(ValidateGraphDataflow.java:139)
... 6 more
Caused by: java.io.NotSerializableException: org.apache.hadoop.mapred.JobConf
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
at com.google.cloud.dataflow.sdk.util.SerializableUtils.serializeToByteArray(SerializableUtils.java:47)
... 27 more
我使用Dataflow的项目编译得很好
mvn compile
但是当我将项目导入eclipse时,eclipse无法构建项目并出现以下错误
The project was not built since its build path is incomplete. Cannot find the class file for com.google.common.reflect.TypeToken. Fix the build path then try building this project