gitlab-ci 运行程序需要很长时间才能从 S3 缓存下载/上传(没有“ServerAddress”)

rma*_*how 5 gitlab gitlab-ci-runner gitlab-runner

我们有一个 gitlab 运行器在 AWS 上的 kubernetes 集群中运行。测试工作:

test-job:
  image: centos
  stage: build
  cache:
    paths:
     - output
  script:
    - mkdir -p output
    - date +"%Y-%m-%d" > output/date.txt
  tags:
    - docker
  when: always
takes about 4 minutes to run - but only 4s if i remove the "cache" section.
Run Code Online (Sandbox Code Playgroud)

我的跑步者的 config.toml 看起来像这样

[[runners]]
  name = "gitlab-runner..."
  url = "https://gitlab...."
  token = "....."
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    Type = "s3"
    Shared = true
    [runners.cache.s3]
      AccessKey = "...."
      SecretKey = "...."
      BucketName = "runner-cache-bucket-name"
      BucketLocation = "eu-central-1"
  [runners.docker]
    tls_verify = false
    image = "ruby:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
Run Code Online (Sandbox Code Playgroud)

我还尝试使用实例配置文件(相同的配置但没有访问/密钥)。

这个配置有效......它只是在“脚本”部分之前和之后挂了很长时间。配置似乎非常简单。

这是跑步者日志:

[[runners]]
  name = "gitlab-runner..."
  url = "https://gitlab...."
  token = "....."
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    Type = "s3"
    Shared = true
    [runners.cache.s3]
      AccessKey = "...."
      SecretKey = "...."
      BucketName = "runner-cache-bucket-name"
      BucketLocation = "eu-central-1"
  [runners.docker]
    tls_verify = false
    image = "ruby:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
Run Code Online (Sandbox Code Playgroud)

请注意,右侧的时间加起来大约为 4 秒——禁用缓存时所需的时间。

编辑=====================================

我将其添加到配置中:

ServerAddress = "s3.amazonaws.com"
Run Code Online (Sandbox Code Playgroud)

现在它按预期工作了。所以,我的问题是:为什么它可以在没有这种配置的情况下工作——尽管很

mar*_*one 1

使用这些变量可以加快压缩速度:

variables:
  FF_USE_FASTZIP: "true" # enable fastzip - a faster zip implementation that also supports level configuration.
  ARTIFACT_COMPRESSION_LEVEL: default # can also be set to fastest, fast, slow and slowest. If just enabling fastzip is not enough try setting this to fastest or fast.
  CACHE_COMPRESSION_LEVEL: default # same as above, but for caches
  TRANSFER_METER_FREQUENCY: 5s # will display transfer progress every 5 seconds for artifacts and remote caches.

Run Code Online (Sandbox Code Playgroud)

参考: https: //gitlab.com/gitlab-org/gitlab-runner/-/issues/1797