无法将替换解析为值:AWS Lambda中的$ {akka.stream.materializer}

Question

我有一个java应用程序,我正在使用它Flink Api.所以基本上我正在尝试用代码做的是创建两个记录很少的数据集,然后将它们作为两个表和必要的字段注册.

 DataSet<Company> comp = env.fromElements(
                new Company("Aux", 1),
                new Company("Comp2", 2),
                new Company("Comp3", 3));

        DataSet<Employee> emp = env.fromElements(
                new Employee("Kula", 1),
                new Employee("Ish", 1),
                new Employee("Kula", 3));


        tEnv.registerDataSet("Employee", emp, "name, empId");
        tEnv.registerDataSet("Company", comp, "cName, empId");

然后我尝试使用以下方法连接这两个表Table API:

Table anotherJoin = tEnv.sql("SELECT Employee.name, Employee.empId, Company.cName FROM " +
                "Employee RIGHT JOIN Company on Employee.empId = Company.empId");

而我只是在控制台上打印结果.这在我的机器上完美地工作.我创建了一个fat-jar使用maven-shade-pluginwith依赖项,我正在尝试在AWS中执行它Lambda.

因此,当我尝试在那里执行它时,我会被抛出以下异常(我只发布前几行):

reference.conf @ file:/var/task/reference.conf:804:无法将替换解析为值:$ {akka.stream.materializer}:com.typesafe.config.ConfigException $ UnresolvedSubstitution com.typesafe.config.ConfigException $ UnresolvedSubstitution:reference.conf @ file:/var/task/reference.conf:804:无法解析替换为值:$ {akka.stream.materializer} at com.typesafe.config.impl.ConfigReference.resolveSubstitutions(ConfigReference) .java:111)com.typesafe.config.impl.ResolveContext.realResolve(ResolveContext.java:179)at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:142)

我在Lambda中执行它之前解压缩了jar,并且碰巧看到所有依赖项都在那里.我无法弄清楚哪里出错了？

任何帮助都可以得到赞赏.

Answer 1

您需要在pom - > maven-shaded-plugin - >配置部分添加此代码:

<transformers>
    <!-- append default configs -->
    <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
        <resource>reference.conf</resource>
    </transformer>
</transformers>

Answer 2

最终能够解决这个问题，这是我的 pom.xml 中的一些主要版本问题。然后我将所有依赖项降级Flink 1.3.2并添加到插件<relocations>中shade。现在可以了。我附上整个 pom，以便有一天它会对某人有所帮助：

<build>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.7.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>


       <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.1.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <transformers>
                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                  <mainClass>com.ink.FlinkLambdaTest.FlinkToLambda</mainClass>
                </transformer>
                <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                  <resource>reference.conf</resource>
                </transformer>
              </transformers>
              <relocations>
                <relocation>
                  <pattern>org.codehaus.plexus.util</pattern>
                  <shadedPattern>org.shaded.plexus.util</shadedPattern>
                  <excludes>
                    <exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude>
                    <exclude>org.codehaus.plexus.util.xml.pull.*</exclude>
                  </excludes>
                </relocation>
              </relocations>
            </configuration>
          </execution>
        </executions>
      </plugin>  
    </plugins>
  </build>

<dependencies>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-table_2.10</artifactId>
      <version>1.3.2</version>
    </dependency>

    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-java</artifactId>
      <version>1.3.2</version>
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-streaming-java_2.10</artifactId>
      <version>1.3.2</version>
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-clients_2.10</artifactId>
      <version>1.3.2</version>
    </dependency>

    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-scala_2.10</artifactId>
      <version>1.3.2</version>
    </dependency>

    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-streaming-scala_2.10</artifactId>
      <version>1.3.2</version>
    </dependency>

    </dependencies>

确保将主类更改为您的主类。