如何在 Gitlab CI 管道上运行 Flutter 集成测试？

Question

我正在开发一个 Flutter 项目，需要在 Gitlab CI 管道上运行集成测试。

测试在本地运行良好，启动chromedriver -port=4444然后flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart.

但是，我无法让它们在 CI 中运行。我的 yaml 文件如下所示：

...

test_ui_integration:
  needs: 
    - build_deploy_user_service
    - populate_db
  tags:
    - docker
  stage: test_ui_integration
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: always
  before_script:
    - chromedriver --port=4444 & # launching the chromedriver
    - sleep 10
  script:
    - cd $CI_PROJECT_DIR/ui
    - npm ${NODE_INSTALL_OPTIONS:-i}
    - npm run build
    - flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart -d chrome --web-run-headless --headless
    # - flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart  -d web-server --headless
    # - flutter drive --driver=integration_test/test_driver/integration_test.dart --target=integration_test/integration/app_test.dart   -d web-server --release --web-run-headless --headless
  timeout: 40 minutes

（请注意，管道使用安装了 chromedriver 和 Flutter 的 docker img）

我已经尝试了各种不同的标志组合，也如此处所示。然而，这些似乎都不起作用：即使在本地，我也无法让测试无头运行（没有浏览器）。

CI 步骤通常会陷入这样的困境，直到超时：

=== running with flags -d web-server --web-run-headless --headless ===

Waiting for connection from debug service on Web Server...         37.1s
integration_test/integration/app_test.dart is being served at http://localhost:44415
The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
...

=== running with flags -d chrome --web-run-headless --headless ===

Waiting for connection from debug service on Chrome...             37.5s
[CHROME]:[1391:1391:0609/090100.591700:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
Failed to launch browser after 3 tries. Command used to launch it: google-chrome --user-data-dir=/tmp/flutter_tools.RDVJEL/flutter_tools_chrome_device.KRPHNF --remote-debugging-port=41647 --disable-background-timer-throttling --disable-extensions --disable-popup-blocking --bwsi --no-first-run --no-default-browser-check --disable-default-apps --disable-translate http://localhost:35691

我还注意到，您可以使用 本地运行集成测试flutter run path/to/my/test/dart，这是官方文档从未提及的。两者有区别吗？

我应该尝试使用移动模拟器运行 CI 测试吗？

非常感谢任何帮助！

编辑：根据@MarkosTh09的要求，这里是dockerfile和相关的CI工作流程：

FROM fedora:35

# Add basic tools
RUN dnf -y install git make gcc gcc-c++ wget unzip zip jq diffutils \
 && dnf clean all

# Requirement of openapi generator
RUN dnf install -y java \
 && dnf clean all

# Install python
RUN dnf install -y python3 \
 && dnf clean all

# Install chromedriver
RUN wget https://chromedriver.storage.googleapis.com/99.0.4844.35/chromedriver_linux64.zip \
    && unzip chromedriver_linux64.zip \
    && cp chromedriver /usr/bin/chromedriver \
    && chown root /usr/bin/chromedriver \
    && chmod +x /usr/bin/chromedriver \
    && chmod 755 /usr/bin/chromedriver

# Install chrome
RUN wget https://dl.google.com/linux/chrome/rpm/stable/x86_64/google-chrome-stable-99.0.4844.84-1.x86_64.rpm \
    && yum install ./google-chrome-*.rpm -y

# Install AWS CLI V2
RUN curl -s https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip > awscliv2.zip \
 && unzip awscliv2.zip \
 && ./aws/install \
 && rm -rf aws awscliv2.zip

# Install and configure nvm for npm
# nvm environment variables
ENV NVM_DIR /opt/nvm
RUN mkdir -p $NVM_DIR
ENV NODE_VERSION 16.14.0

# install nvm
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash

# install node and npm
RUN source $NVM_DIR/nvm.sh \
 && nvm install $NODE_VERSION \
 && nvm alias default $NODE_VERSION \
 && nvm use default

# add node and npm to path so the commands are available
ENV NODE_PATH $NVM_DIR/versions/node/v$NODE_VERSION/lib/node_modules
ENV NVM_BIN $NVM_DIR/versions/node/v$NODE_VERSION/bin
ENV NVM_INC $NVM_DIR/versions/node/v$NODE_VERSION/include/node
ENV PATH $NVM_BIN:$PATH

# Install serverless
RUN npm install -g serverless@3.2.0

# Install Flutter
RUN curl -s https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_3.0.0-stable.tar.xz > flutter.tar.xz \
 && tar xf flutter.tar.xz -C /opt/ \
 && rm flutter.tar.xz
ENV PATH $PATH:/opt/flutter/bin

RUN git config --global --add safe.directory /opt/flutter

CMD [ "/bin/bash" ]
ENTRYPOINT [ "/bin/bash", "-c" ]

=== running with flags -d web-server --web-run-headless --headless ===

Waiting for connection from debug service on Web Server...         37.1s
integration_test/integration/app_test.dart is being served at http://localhost:44415
The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
...

Answer 1

我不再参与那个特定的 Flutter 项目，但我被告知 e2e 测试的设置类似于以下内容：

\n

Dockerfile

\n

FROM centos:8\n\n# Failed to download metadata for repo \xe2\x80\x98AppStream\xe2\x80\x99 [CentOS] : https://techglimpse.com/failed-metadata-repo-appstream-centos-8/\nRUN cd /etc/yum.repos.d/ && sed -i \'s/mirrorlist/#mirrorlist/g\' /etc/yum.repos.d/CentOS-* && sed -i \'s|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g\' /etc/yum.repos.d/CentOS-*\n\n# Add basic tools\nRUN dnf install -y git make gcc gcc-c++ wget unzip zip jq diffutils zlib.i686 ncurses-libs.i686 bzip2-libs.i686 curl which mesa-libGLU patch \\\n    && dnf clean all\n\n# ............\n\n# Install Flutter\nRUN wget https://storage.googleapis.com/flutter_infra_release/releases/stable/linux/flutter_linux_3.13.9-stable.tar.xz \\\n    && tar xf flutter_linux_3.13.9-stable.tar.xz && rm flutter_linux_3.13.9-stable.tar.xz\n\nENV PATH "$PATH:/flutter/bin"\n\n# Install firebase CLI\nRUN npm install -g firebase-tools\n\n\n# Install Android SDK\nENV VERSION_TOOLS "9477386"\nENV ANDROID_SDK_ROOT "/sdk"\n# Keep alias for compatibility\nENV ANDROID_HOME "${ANDROID_SDK_ROOT}"\nENV PATH "$PATH:${ANDROID_SDK_ROOT}/cmdline-tools/latest/bin:${ANDROID_SDK_ROOT}/platform-tools"\n\nRUN curl -s https://dl.google.com/android/repository/commandlinetools-linux-${VERSION_TOOLS}_latest.zip > /cmdline-tools.zip \\\n    && mkdir -p ${ANDROID_SDK_ROOT}/cmdline-tools \\\n    && unzip /cmdline-tools.zip -d ${ANDROID_SDK_ROOT}/cmdline-tools \\\n    && mv ${ANDROID_SDK_ROOT}/cmdline-tools/cmdline-tools ${ANDROID_SDK_ROOT}/cmdline-tools/latest \\\n    && chmod -R 777 $ANDROID_SDK_ROOT \\\n    && rm -v /cmdline-tools.zip\n\nRUN mkdir -p $ANDROID_SDK_ROOT/licenses/ \\\n    && echo "8933bad161af4178b1185d1a37fbf41ea5269c55\\nd56f5187479451eabf01fb78af6dfcb131a6481e\\n24333f8a63b6825ea9c5514f83c2829b004d1fee" > $ANDROID_SDK_ROOT/licenses/android-sdk-license \\\n    && echo "84831b9409646a918e30573bab4c9c91346d8abd\\n504667f4c0de7af1a06de9f4b1727b84351f2910" > $ANDROID_SDK_ROOT/licenses/android-sdk-preview-license \\\n    && yes | sdkmanager --licenses >/dev/null \\\n    && chmod -R 777 $ANDROID_SDK_ROOT/licenses/\n\nRUN mkdir -p /root/.android \\\n    && touch /root/.android/repositories.cfg \\\n    && sdkmanager --update\n\nADD docker-env/build/packages.txt /sdk\nRUN sdkmanager --package_file=/sdk/packages.txt && chmod -R 777 $ANDROID_SDK_ROOT\nRUN sdkmanager "system-images;android-33;google_apis;x86_64"\nRUN avdmanager create avd --force --name Pixel5 --abi google_apis/x86_64 --package "system-images;android-33;google_apis;x86_64" --device "pixel_5"\n\n# Configure android emulator\nRUN sed -i \'s/disk.dataPartition.size=800M/disk.dataPartition.size=2000M/g\' /root/.android/avd/Pixel5.avd/config.ini\nRUN sed -i \'s/hw.keyboard=no/hw.keyboard=yes/g\' /root/.android/avd/Pixel5.avd/config.ini\n\nCMD [ "/bin/bash" ]\nENTRYPOINT [ "/bin/bash", "-c" ]\n

\n

.yml CI 文件

\n

显然我们不需要flutter drive在 CI 步骤中再使用：

\n

test_ui:\n  tags:\n    - docker\n  stage: test_ui\n  needs:\n    - ...\n    - ...\n  rules:\n    - if: $CI_PIPELINE_SOURCE == "merge_request_event"\n      when: always\n  before_script:\n    - !reference [.aws-prep, before_script]\n  script:\n    - git fetch origin\n    - export PUB_CACHE=$CI_PROJECT_DIR/.pub-cache\n    - export PATH="$PATH":"$PUB_CACHE/bin"\n    - echo "to see android emulator test in live, go to http://RUNNER_IP:6080"\n    - cd $CI_PROJECT_DIR/mobile/android\n    - bundle install\n    - bundle exec fastlane android bump_patch\n    - cd $CI_PROJECT_DIR/mobile\n    - flutter pub get\n    - flutter test\n    - adb connect <docker_host_ip>:5555\n    - flutter test integration_test/*.dart --dart-define=AWS_STAGE=${AWS_STAGE} --dart-define=AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID:-${GITLAB_TEST_AWS_ACCESS_KEY_ID}} --dart-define=AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY:-${GITLAB_TEST_AWS_SECRET_ACCESS_KEY}}\n  timeout: 60 minute\n  retry: 1\n\n

\n