使用多阶段 Docker 构建在容器中构建本机可执行文件时内存不足

Question

我正在尝试将quarkus-quickstarts/kafka-quickstart/项目（来自 Quarkus 入门资源）构建为具有多阶段 Docker 构建的容器中的本机可执行文件。在“quarkus-maven-plugin:0.21.2:native-image”阶段出现内存不足之前，构建会卡住很长一段时间。

从解决方案中，我使用 docker compose 启动 Kafka 代理run docker-compose up，然后添加下一个多阶段 Dockerfile：

## Stage 1 : build with maven builder image with native capabilities
FROM quay.io/quarkus/centos-quarkus-maven:19.2.0 AS build
COPY src /usr/src/app/src
COPY pom.xml /usr/src/app
USER root
RUN chown -R quarkus /usr/src/app
USER quarkus
RUN mvn -f /usr/src/app/pom.xml -Pnative clean package

## Stage 2 : create the docker final image
FROM registry.access.redhat.com/ubi8/ubi-minimal
WORKDIR /work/
COPY --from=build /usr/src/app/target/*-runner /work/application
RUN chmod 775 /work
EXPOSE 8080
CMD ["./application", "-Dquarkus.http.host=0.0.0.0"]

最后我试图建立 docker build -f src/main/docker/Dockerfile.multistage -t quarkus-quickstart/kafka-quickstart .

但由于 OOM，进程被卡住了：

...
[INFO] [io.quarkus.creator.phase.runnerjar.RunnerJarPhase] Building jar: /usr/src/app/target/kafka-quickstart-1.0-SNAPSHOT-runner.jar
[INFO]
[INFO] --- quarkus-maven-plugin:0.21.2:native-image (default) @ kafka-quickstart ---
[INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] Running Quarkus native-image plugin on OpenJDK 64-Bit GraalVM CE 19.2.0
[INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] /opt/graalvm/bin/native-image -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dio.netty.leakDetection.level=DISABLED -J-Dvertx.disableDnsResolver=true -J-Dio.netty.noUnsafe=true --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar kafka-quickstart-1.0-SNAPSHOT-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:+PrintAnalysisCallTree -H:-AddAllCharsets -H:EnableURLProtocols=http -H:-SpawnIsolates -H:+JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace
[kafka-quickstart-1.0-SNAPSHOT-runner:126]    classlist:  11,600.15 ms
[kafka-quickstart-1.0-SNAPSHOT-runner:126]        (cap):   1,349.80 ms
[kafka-quickstart-1.0-SNAPSHOT-runner:126]        setup:   3,497.19 ms
15:47:12,961 INFO  [org.jbo.threads] JBoss Threads version 3.0.0.Beta5
15:47:15,275 INFO  [org.xnio] XNIO version 3.7.2.Final
15:47:15,989 INFO  [org.xni.nio] XNIO NIO Implementation Version 3.7.2.Final
[kafka-quickstart-1.0-SNAPSHOT-runner:126]     analysis: 402,209.12 ms

因为 OOM 不清楚，所以我尝试指示 Maven 构建从容器内部生成可执行文件 mvnw package -Pnative -Dnative-image.docker-build=true ，并重现了相同的行为：

...
[INFO] [io.quarkus.creator.phase.runnerjar.RunnerJarPhase] Building jar: C:\Users\tim\workspace\tutoquarkus\second-chance\kafka-quickstart\target\kafka-quickstart-1.0-SNAPSHOT-runner.jar
[INFO]
[INFO] --- quarkus-maven-plugin:0.21.2:native-image (default) @ kafka-quickstart ---
[INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] Running Quarkus native-image plugin on OpenJDK 64-Bit GraalVM CE 19.2.0
[INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] docker run -v C:\Users\tim\workspace\tutoquarkus\second-chance\kafka-quickstart\target:/project:z --rm quay.io/quarkus/ubi-quarkus-native-image:19.1.1 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dio.netty.leakDetection.level=DISABLED -J-Dvertx.disableDnsResolver=true -J-Dio.netty.noUnsafe=true --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar kafka-quickstart-1.0-SNAPSHOT-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:+PrintAnalysisCallTree -H:-AddAllCharsets -H:EnableURLProtocols=http -H:-SpawnIsolates -H:+JNI -H:-UseServiceLoaderFeature -H:+StackTrace
Build on Server(pid: 22, port: 41567)*
[kafka-quickstart-1.0-SNAPSHOT-runner:22]    classlist:  15,449.39 ms
[kafka-quickstart-1.0-SNAPSHOT-runner:22]        (cap):   1,568.93 ms
[kafka-quickstart-1.0-SNAPSHOT-runner:22]        setup:   3,580.83 ms
16:02:45,267 INFO  [org.jbo.threads] JBoss Threads version 3.0.0.Beta5
16:02:46,840 INFO  [org.xnio] XNIO version 3.7.2.Final
16:02:47,019 INFO  [org.xni.nio] XNIO NIO Implementation Version 3.7.2.Final
[kafka-quickstart-1.0-SNAPSHOT-runner:22]     analysis: 403,409.32 ms
Fatal error: java.lang.InternalError: java.lang.InternalError: linkToTargetMethod=Lambda(a0:L,a1:L)=>{
    t2:L=MethodHandle.invokeBasic(a1:L,a0:L);t2:L}
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593)
        at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
        at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:457)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:308)
        at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:446)
        at com.oracle.svm.hosted.server.NativeImageBuildServer.executeCompilation(NativeImageBuildServer.java:394)
        at com.oracle.svm.hosted.server.NativeImageBuildServer.lambda$processCommand$8(NativeImageBuildServer.java:331)
        at com.oracle.svm.hosted.server.NativeImageBuildServer.withJVMContext(NativeImageBuildServer.java:412)
        at com.oracle.svm.hosted.server.NativeImageBuildServer.processCommand(NativeImageBuildServer.java:328)
        at com.oracle.svm.hosted.server.NativeImageBuildServer.processRequest(NativeImageBuildServer.java:272)
        at com.oracle.svm.hosted.server.NativeImageBuildServer.lambda$serve$7(NativeImageBuildServer.java:232)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InternalError: linkToTargetMethod=Lambda(a0:L,a1:L)=>{
    t2:L=MethodHandle.invokeBasic(a1:L,a0:L);t2:L}
        at java.lang.invoke.MethodHandleStatics.newInternalError(MethodHandleStatics.java:127)
        at java.lang.invoke.LambdaForm.compileToBytecode(LambdaForm.java:660)
        at java.lang.invoke.Invokers.callSiteForm(Invokers.java:381)
        at java.lang.invoke.Invokers.linkToTargetMethod(Invokers.java:347)
        at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:314)
        at java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
        at com.oracle.graal.pointsto.reports.CallTreePrinter.buildCallTree(CallTreePrinter.java:156)
        at com.oracle.graal.pointsto.reports.CallTreePrinter.print(CallTreePrinter.java:62)
        at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:753)
        at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:522)
        at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:440)
        at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
Error: Image build request failed with exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  12:55 min
[INFO] Finished at: 2019-09-12T18:13:28+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:0.21.2:native-image (default) on project kafka-quickstart: Failed to generate a native image: Failed to build native image: Image generation failed -> [Help 1]

我期望得到与本机可执行文件的构建相同的结果mvnw package -Pnative

最后我尝试mvnw package -Pnative从 quay.io/quarkus/centos-quarkus-maven:19.2.0 容器运行，但我遇到了完全相同的问题。我很难理解这个问题是否涉及更多 GraalVM 还是 Quarkus 构建。

Answer 1

GraalVM 原生镜像工具需要大量内存！至少需要6GB，但如果你能给他8GB就更好了。

为了让您了解一下，当我构建本机映像时，我几乎关闭了 16Gb 笔记本电脑上运行的所有内容。

您可以再次尝试构建具有更多可用内存的本机映像吗？如果您使用的是 Windows 版 Docker，请注意为其使用的 VM 提供足够的内存（至少 8Gb）。

如果你通过 docker 运行，你可以Xmx在命令行上设置：

mvn clean package -Dnative -Dquarkus.native.container-build=true \
    -Dquarkus.native.native-image-xmx=6g