GitLab CI runner无法连接到kubernetes中的unix:///var/run/docker.sock

Question

GitLab在kubernetes集群中运行.Runner无法使用构建工件构建docker镜像.我已经尝试了几种方法来解决这个问题,但没有运气.以下是一些配置片段:

.gitlab-ci.yml

image: docker:latest
services:
  - docker:dind

variables:
  DOCKER_DRIVER: overlay

stages:
  - build
  - package
  - deploy

maven-build:
  image: maven:3-jdk-8
  stage: build
  script: "mvn package -B --settings settings.xml"
  artifacts:
    paths:
      - target/*.jar

docker-build:
  stage: package
  script:
  - docker build -t gitlab.my.com/group/app .
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN gitlab.my.com/group/app
  - docker push gitlab.my.com/group/app

config.toml

concurrent = 1
check_interval = 0

[[runners]]
  name = "app"
  url = "https://gitlab.my.com/ci"
  token = "xxxxxxxx"
  executor = "kubernetes"
  [runners.kubernetes]
    privileged = true
    disable_cache = true

包阶段日志:

running with gitlab-ci-multi-runner 1.11.1 (a67a225)
  on app runner (6265c5)
Using Kubernetes namespace: default
Using Kubernetes executor with image docker:latest ...
Waiting for pod default/runner-6265c5-project-4-concurrent-0h9lg9 to be running, status is Pending
Waiting for pod default/runner-6265c5-project-4-concurrent-0h9lg9 to be running, status is Pending
Running on runner-6265c5-project-4-concurrent-0h9lg9 via gitlab-runner-3748496643-k31tf...
Cloning repository...
Cloning into '/group/app'...
Checking out 10d5a680 as master...
Skipping Git submodules setup
Downloading artifacts for maven-build (61)...
Downloading artifacts from coordinator... ok        id=61 responseStatus=200 OK token=ciihgfd3W
$ docker build -t gitlab.my.com/group/app .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
ERROR: Job failed: error executing remote command: command terminated with non-zero exit code: Error executing in Docker Container: 1

我究竟做错了什么？

Answer 1

不需要使用这个:

DOCKER_DRIVER: overlay

因为似乎不支持OVERLAY,所以svc-0容器无法启动它:

$ kubectl logs -f `kubectl get pod |awk '/^runner/{print $1}'` -c svc-0
time="2017-03-20T11:19:01.954769661Z" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]"
time="2017-03-20T11:19:01.955720778Z" level=info msg="libcontainerd: new containerd process, pid: 20"
time="2017-03-20T11:19:02.958659668Z" level=error msg="'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded."

另外,添加export DOCKER_HOST="tcp://localhost:2375"到docker-build:

 docker-build:
  stage: package
  script:
  - export DOCKER_HOST="tcp://localhost:2375"
  - docker build -t gitlab.my.com/group/app .
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN gitlab.my.com/group/app
  - docker push gitlab.my.com/group/app

Answer 2

使用Kubernetes时,您必须调整Build图像以连接Docker引擎.

添加到您的构建图像:

DOCKER_HOST=tcp://localhost:2375

从文档引用:

运行docker:dind也称为docker-in-docker映像也是可能的,但遗憾的是需要容器以特权模式运行.如果你愿意承担这种风险,那么其他问题就会出现,乍一看似乎并不那么直接.因为docker守护程序通常在.gitlab-ci.yaml中作为服务启动,所以它将作为pod中的单独容器运行.基本上,pod中的容器只共享分配给它们的卷和一个IP地址,它们可以使用localhost相互连接./var/run/docker.sock不是由docker共享的:dind容器和docker二进制文件默认尝试使用它.要覆盖它并使客户端使用tcp联系另一个容器中的docker守护程序,请确保在构建容器的环境变量中包含DOCKER_HOST = tcp:// localhost:2375.

关于Kubernetes的Gitlab-CI