如何作为Kubernetes工作顺序运行容器？

Question

我正在尝试用Kubernetes工作替换我的旧作业调度程序,并想知道如何将顺序作业写为Kubernetes作业.

首先,我编写了以下脚本来执行job1并按job2书面顺序执行,但它没有按预期工作.

apiVersion: batch/v1
kind: Job
metadata:
  name: sequential
spec:
  activeDeadlineSeconds: 100
  template:
    metadata:
      name: sequential_jobs
    spec:
      containers:
      - name: job1
        image: image1
      - name: job2
        image: image2
      restartPolicy: Never

上述工作似乎运行job1和job2并行.有没有什么好的方式来运行job1和job2书面订单？

追加.

我最近发现https://github.com/argoproj/argo非常适合我的用例.

Answer 1

经过几次尝试,我做到了这一点,解决了基本问题(类似于OP发布的内容).此配置确保job-1在job-2开始之前完成.如果job-1失败,job-2则不启动容器.我仍然需要处理重试和故障处理,但基础工作正常.希望,这将有助于其他人:

apiVersion: v1
kind: Pod
metadata:
  name: sequential-job
spec:
  initContainers:
  - name: job-1
    image: busybox
    # runs for 15 seconds; echoes job name and timestamp
    command: ['sh', '-c', 'for i in 1 2 3; do echo "job-1 `date`" && sleep 5s; done;']
  - name: job-2
    image: busybox
    # runs for 15 seconds; echoes job name and timestamp
    command: ['sh', '-c', 'for i in 1 2 3; do echo "job-2 `date`" && sleep 5s; done;']
  # I don't really need the 'containers', but syntax requires 
  # it so, I'm using it as a place where I can report the 
  # completion status
  containers:
  - name: job-done
    image: busybox
    command: ['sh', '-c', 'echo "job-1 and job-2 completed"']
  restartPolicy: Never

更新

与上述相同的配置也适用于作业规范:

apiVersion: batch/v1
kind: Job
metadata:
  name: sequential-jobs
spec:
  template:
    metadata:
      name: sequential-job
    spec:
      initContainers:
      - name: job-1
        image: busybox
        command: ['sh', '-c', 'for i in 1 2 3; do echo "job-1 `date`" && sleep 5s; done;']
      - name: job-2
        image: busybox
        command: ['sh', '-c', 'for i in 1 2 3; do echo "job-2 `date`" && sleep 5s; done;']
      containers:
      - name: job-done
        image: busybox
        command: ['sh', '-c', 'echo "job-1 and job-2 completed"']
      restartPolicy: Never

Answer 2

Argo 工作流程将适合您的用例。Argo将支持顺序、并行、DAG 作业处理。

# This template demonstrates a steps template and how to control sequential vs. parallel steps.
# In this example, the hello1 completes before the hello2a, and hello2b steps, which run in parallel.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: steps-
spec:
  entrypoint: hello-hello-hello

  templates:
  - name: hello-hello-hello
    steps:
    - - name: hello1
        template: whalesay
        arguments:
          parameters: [{name: message, value: "hello1"}]
    - - name: hello2a
        template: whalesay
        arguments:
          parameters: [{name: message, value: "hello2a"}]
      - name: hello2b
        template: whalesay
        arguments:
          parameters: [{name: message, value: "hello2b"}]

  - name: whalesay
    inputs:
      parameters:
      - name: message
    container:
      image: docker/whalesay
      command: [cowsay]
      args: ["{{inputs.parameters.message}}"]

Answer 3

一般来说，Kubernetes 设置中不存在顺序概念，也没有捕获跨容器/Pod 的依赖关系。

就您而言，如果作业规范（甚至是 pod 规范）中有 2 个容器，则这 2 个容器没有排序。同样，如果您相继解雇了 2 项工作，那么这些工作也不存在排序的概念。

理想情况下，如果有任何东西需要测序，您应该将其捕获在单个单元（容器）中。

与您的问题稍微有些切线，当作业依赖于另一个现有服务（例如以 k8s 服务为前端的部署）时，我看到了另一种常见模式：

作业中的容器向 k8s 服务发出请求，如果服务未按预期响应，则失败。这样，作业就会不断重新启动，最终当服务启动时，作业就会成功执行并完成。