GitLab CI - 缓存无法正常工作

Question

我目前正在使用GitLab与CI运行程序一起运行我的项目的单元测试,以加快引导我使用内置缓存功能的测试的过程,但这似乎不起作用.

每当有人提交掌握时,我的跑步者会做一个git fetch并继续删除所有缓存文件,这意味着我必须盯着我的屏幕大约10分钟等待测试完成,同时跑步者重新下载所有依赖项(NPM和PIP是最大的杀手锏).

CI跑者的输出:

Fetching changes...

Removing bower_modules/jquery/  --+-- Shouldn't happen!
Removing bower_modules/tether/    |
Removing node_modules/            |
Removing vendor/                --'
HEAD is now at 7c513dd Update .gitlab-ci.yml

目前我的.gitlab-ci.yml

image: python:latest

services:
  - redis:latest
  - node:latest

cache:
  key: "$CI_BUILD_REF_NAME"
  untracked: true
  paths:
  - ~/.cache/pip/
  - vendor/
  - node_modules/
  - bower_components/


before_script:
  - python -V

  # Still gets executed even though node is listed as a service??
  - '(which nodejs && which npm) || (apt-get update -q && apt-get -o dir::cache::archives="vendor/apt/" install nodejs npm -yqq)'
  - npm install -g bower gulp

  # Following statements ignore cache!
  - pip install -r requirements.txt
  - npm install --only=dev
  - bower install --allow-root
  - gulp build

test:
  variables:
    DEBUG: "1"
  script:
  - python -m unittest myproject

我已经尝试阅读以下文章寻求帮助,但它们似乎都没有解决我的问题:

• http://docs.gitlab.com/ce/ci/yaml/README.html#cache
• https://fleschenberg.net/gitlab-pip-cache/
• https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/336

Answer 1

事实证明我做错了一些事情:

• 第一个是我没有使用工件的事实.
• 您的脚本无法缓存项目范围之外的文件,而是创建虚拟环境,缓存允许您缓存您的pip模块.
• 最重要的是:您的测试必须成功,才能缓存文件.

使用以下配置后,我得到了-3分钟的时差:

目前我的配置如下所示,适合我.

# Official framework image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python
image: python:latest

# Pick zero or more services to be used on all builds.
# Only needed when using a docker container to run your tests in.
# Check out: http://docs.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-service
services:
- mysql:latest
- redis:latest

cache:
    untracked: true
    key: "$CI_BUILD_REF_NAME"
    paths:
    - venv/
    - node_modules/
    - bower_components/


# This is a basic example for a gem or script which doesn't use
# services such as redis or postgres
before_script:

# Check python installation
- python -V

# Install NodeJS (Gulp & Bower)
# Default repository is outdated, this is the latest version
- 'curl -sL https://deb.nodesource.com/setup_8.x | bash -'
- apt-get install -y nodejs
- npm install -g bower gulp

# Install dependencie
- pip install -U pip setuptools
- pip install virtualenv


test:

    # Indicate to the framework that it's being unit tested
    variables:
        DEBUG: "1"

    # Test script
    script:

    # Set up virtual environment
    - virtualenv venv -ppython3
    - source venv/bin/activate
    - pip install coverage
    - pip install -r requirements.txt

    # Install NodeJS & Bower + Compile JS
    - npm install --only=dev
    - bower install --allow-root
    - gulp build

    # Run all unit tests
    - coverage run -m unittest project.tests
    - coverage report -m project/**/*.py

这导致以下输出:

Fetching changes...
Removing .coverage                              --+-- Don't worry about this
Removing bower_components/                        |
Removing node_modules/                            |
Removing venv/                                  --`
HEAD is now at 24e7618 Fix for issue #16
From https://git.example.com/repo
85f2f9b..42ba753  master     -> origin/master
Checking out 42ba7537 as master...
Skipping Git submodules setup
Checking cache for master...                    --+-- The files are back now :)
Successfully extracted cache                    --`

...

project/module/script.py                  157      9    94%   182, 231-244
---------------------------------------------------------------------------
TOTAL                                          1084    328    70%
Creating cache master...
Created cache
Uploading artifacts...
venv/: found 9859 matching files                   
node_modules/: found 7070 matching files           
bower_components/: found 982 matching files 
Trying to load /builds/repo.tmp/CI_SERVER_TLS_CA_FILE ... 
Dialing: tcp git.example.com:443 ...         
Uploading artifacts to coordinator... ok            id=127 responseStatus=201 Created token=XXXXXX
Job succeeded

对于覆盖率报告,我使用了以下正则表达式:

^TOTAL\s+(?:\d+\s+){2}(\d{1,3}%)$