Flink：流拓扑中没有定义算子。无法执行

Question

我正在尝试设置一个非常基本的 flink 作业。当我尝试运行时，出现以下错误：

Caused by: java.lang.IllegalStateException: No operators defined in streaming topology. Cannot execute.
    at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getStreamGraph(StreamExecutionEnvironment.java:1535)
    at org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:53)
    at org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.scala:654)
    at com.test.flink.jobs.TestJobRunnable$.run(TestJob.scala:223)

该错误是由以下代码引起的：

val streamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
val messageStream = streamExecutionEnvironment.addSource(kafkaConsumer)
messageStream.keyBy(_ => "S")

streamExecutionEnvironment.execute("Test Job")

当我print()向流的末尾添加调用时，错误消失了：

val streamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
val messageStream = streamExecutionEnvironment.addSource(kafkaConsumer)
messageStream.keyBy(_ => "S")
messageStream.print()

streamExecutionEnvironment.execute("Test Job")

我很困惑为什么要print()解决这个问题。在引入接收器之前，流式拓扑不会处理其任何运算符的想法吗？被print()用作这里水槽？任何帮助，将不胜感激。谢谢。

Answer 1

In programming language theory, lazy evaluation, or call-by-need is an evaluation strategy which delays the evaluation of an expression until its value is needed and which also avoids repeated evaluations. The opposite of lazy evaluation is eager evaluation, sometimes known as strict evaluation. The benefits of lazy evaluation include:

• The ability to define control flow (structures) as abstractions instead of primitives.
• The ability to define potentially infinite data structures. This allows for more straightforward implementation of some algorithms.
• Performance increases by avoiding needless calculations, and avoiding error conditions when evaluating compound expressions.

Lazy evaluation can lead to reduction in memory footprint, since values are created when needed. However, lazy evaluation is difficult to combine with imperative features such as exception handling and input/output, because the order of operations becomes indeterminate.

Generally, Flink divided operations into two class: transformations operations and sink operations. As you guess, Flink transformations are lazy, meaning that they are not executed until a sink operation is invoked.

Flink 程序是实现分布式集合转换（例如，过滤、映射、更新状态、连接、分组、定义窗口、聚合）的常规程序。集合最初是从源创建的（例如，通过从文件、Kafka 主题或本地内存集合中读取）。结果通过接收器返回，例如，接收器可以将数据写入（分布式）文件或标准输出（例如命令行终端）。