CircleCi、TestContainers 使用 Docker Executor 和远程 Docker 环境

Question

我在 CircleCi 的远程 docker 环境中运行 Testcontainers，容器上打开的端口不可用。这可以在不恢复到 Machine 执行器的情况下工作吗？

Answer 1

注意：这已经过时了，Testcontainers 现在只能在 Circleci 上运行

截至 2023 年 4 月，这似乎不再必要，端口已绑定并公开在本地主机上，因为所有远程 docker 和 docker 执行器都在同一虚拟机上运行。请参阅：https ://discuss.circleci.com/t/changes-to-remote-docker-reporting-pricing/47759/1

您可以将 Testcontainers 与docker执行程序一起使用，但存在一些限制，因为这将是一个受防火墙保护且只能通过 SSH 访问的远程 Docker 环境。

从概念上讲，您需要遵循以下步骤：

• 添加setup-remote-docker​.circleci/config.yml
• 如果您在测试期间需要私有容器镜像，请添加登录步骤。
• 设置环境变量TESTCONTAINERS_HOST_OVERRIDE=localhost。端口通过 SSH 映射到 localhost。
• 为每个公开端口创建进入远程 Docker 的隧道。原因是远程 docker 有防火墙，只能通过 ssh remote-docker. 在下面的示例中.circleci/autoforward.py，在后台运行，监视 docker 端口映射并动态创建 SSH 端口转发到本地主机。

示例配置.circleci/config.yml

version: 2.1
jobs:
    test:
        docker:
            # choose an image that has: 
            #    ssh, java, git, docker-cli, tar, gzip, python3
            - image: cimg/openjdk:16.0.0
        steps:
            - checkout
            - setup_remote_docker:
                  version: 20.10.2
                  docker_layer_caching: true
            - run:
                  name: Docker login
                  command: |
                      # access private container images during tests
                      echo ${DOCKER_PASS} | \
                        docker login ${DOCKER_REGISTRY_URL} \
                          -u ${DOCKER_USER} \ 
                          --password-stdin
            - run:
                  name: Setup Environment Variables
                  command: |
                      echo "export TESTCONTAINERS_HOST_OVERRIDE=localhost" \
                        >> $BASH_ENV
            - run:
                  name: Testcontainers tunnel
                  background: true
                  command: .circleci/autoforward.py
            - run: ./gradlew clean test --stacktrace
workflows:
    test:
        jobs:
            - test

处理端口转发的脚本：.circleci/autoforward.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import dataclasses
import threading
import sys
import signal
import subprocess
import json
import re
import time


@dataclasses.dataclass(frozen=True)
class Forward:
    port: int

    def __ne__(self, other):
        return not self.__eq__(other)

    @staticmethod
    def parse_list(ports):
        r = []
        for port in ports.split(","):
            port_splits = port.split("->")
            if len(port_splits) < 2:
                continue
            host, ports = Forward.parse_host(port_splits[0], "localhost")
            for port in ports:
                r.append(Forward(port))
        return r

    @staticmethod
    def parse_host(s, default_host):
        s = re.sub("/.*$", "", s)
        hp = s.split(":")
        if len(hp) == 1:
            return default_host, Forward.parse_ports(hp[0])
        if len(hp) == 2:
            return hp[0], Forward.parse_ports(hp[1])
        return None, []

    @staticmethod
    def parse_ports(ports):
       port_range = ports.split("-")
       start = int(port_range[0])
       end = int(port_range[0]) + 1
       if len(port_range) > 2 or len(port_range) < 1:
           raise RuntimeError(f"don't know what to do with ports {ports}")
       if len(port_range) == 2:
           end = int(port_range[1]) + 1
       return list(range(start, end))

class PortForwarder:
    def __init__(self, forward, local_bind_address="127.0.0.1"):
        self.process = subprocess.Popen(
            [
                "ssh",
                "-N",
                f"-L{local_bind_address}:{forward.port}:localhost:{forward.port}",
                "remote-docker",
            ]
        )

    def stop(self):
        self.process.kill()


class DockerForwarder:
    def __init__(self):
        self.running = threading.Event()
        self.running.set()

    def start(self):
        forwards = {}
        try:
            while self.running.is_set():
                new_forwards = self.container_config()
                existing_forwards = list(forwards.keys())
                for forward in new_forwards:
                    if forward in existing_forwards:
                        existing_forwards.remove(forward)
                    else:
                        print(f"adding forward {forward}")
                        forwards[forward] = PortForwarder(forward)

                for to_clean in existing_forwards:
                    print(f"stopping forward {to_clean}")
                    forwards[to_clean].stop()
                    del forwards[to_clean]
                time.sleep(0.8)
        finally:
            for forward in forwards.values():
                forward.stop()

    @staticmethod
    def container_config():
        def cmd(cmd_array):
            out = subprocess.Popen(
                cmd_array,
                universal_newlines=True,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
            )
            out.wait()
            return out.communicate()[0]

        try:
            stdout = cmd(["docker", "ps", "--format", "'{{json .}}'"])
            stdout = stdout.replace("'", "")
            configs = map(lambda l: json.loads(l), stdout.splitlines())
            forwards = []
            for c in configs:
                if c is None or c["Ports"] is None:
                    continue
                ports = c["Ports"].strip()
                if ports == "":
                    continue
                forwards += Forward.parse_list(ports)
            return forwards
        except RuntimeError:
            print("Unexpected error:", sys.exc_info()[0])
            return []

    def stop(self):
        print("stopping")
        self.running.clear()


def main():
    forwarder = DockerForwarder()

    def handler(*_):
        forwarder.stop()

    signal.signal(signal.SIGINT, handler)

    forwarder.start()


if __name__ == "__main__":
    main()