Gitlab CI SAST 在后续阶段访问 gl-sast-report.json 工件

Question

我想在 CI 的后续阶段使用 SAST 过程中创建的 gl-sast-report.json 文件，但找不到该文件。

ci.yml

include:
  - template: Security/SAST.gitlab-ci.yml

stages:
  - test
  - .post
sast:
  rules:
    - if: $CI_COMMIT_TAG

send-reports:
  stage: .post
  dependencies: 
    - sast
  script: 
    - ls
    - echo "in post stage"
    - cat gl-sast-report.json

输出：

Running with gitlab-runner 13.2.1 (efa30e33)
on blah blah blah
Preparing the "docker" executor
00:01
.
.
.

Preparing environment
00:01
Running on runner-zqk9bcef-project-4296-concurrent-0 via ff93ba7b6ee2...
Getting source from Git repository
00:01
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in blah blah
Checking out 9c2edf67 as 39-test-dso...
Removing gl-sast-report.json
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:03
$ ls
<stuff in the repo>
$ echo "in .post stage"
in post stage
$ cat gl-sast-report.json
cat: can't open 'gl-sast-report.json': No such file or directory
ERROR: Job failed: exit code 1

Removing gl-sast-report.json你可以看到我认为是问题所在的那一行。

我在https://gitlab.com/gitlab-org/gitlab/-/blob/v11.11.0-rc2-ee/lib/gitlab/ci/templates的 SAST.gitlab-ci.yml 中没有看到这一点/安全/SAST.gitlab-ci.yml#L33-45

关于如何在 CI 管道的下一阶段使用此工件有什么想法吗？

更新：

所以我尝试了下面 k33g_org 的建议，但没有成功。似乎这是由于 sast 模板的限制所致。做了如下测试。

include:
  - template: Security/SAST.gitlab-ci.yml

stages:
  - test
  - upload

something:
  stage: test
  script:
      - echo "in something"
      - echo "this is something" > something.txt
  artifacts:
      paths: [something.txt]

sast:
  before_script:
      - echo "hello from before sast"
      - echo "this is in the file" > test.txt
  artifacts:
    reports:
      sast: gl-sast-report.json
    paths: [gl-sast-report.json, test.txt]

send-reports:
  stage: upload
  dependencies:
    - sast
    - something
  before_script:
      - echo "This is the send-reports before_script"
  script:
    - echo "in send-reports job"
    - ls
  artifacts:
      reports:
          sast: gl-sast-report.json

三个变化：

1. 根据 k33g_org 的建议更新了代码
2. 在 sast 作业中创建了另一个工件（以查看它是否会传递到发送报告作业）
3. 创建了一个新的作业（某物），我在其中创建了一个新的Something.txt工件（以查看它是否会传递到发送报告作业）

输出：

Preparing environment
00:01
Running on runner-zqx7qoq-project-4296-concurrent-0 via e3fe672984b4...
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /<repo>
Checking out 26501c44 as <branch_name>...
Removing something.txt
Skipping Git submodules setup
Downloading artifacts
00:00
Downloading artifacts for something (64950)...
Downloading artifacts from coordinator... ok        id=64950 
responseStatus=200 OK token=zoJwysdq
Executing "step_script" stage of the job script
00:01
$ echo "This is the send-reports before_script"
This is the send-reports before_script
$ echo "in send-reports job"
in send-reports job
$ ls
...<other stuff in repo>
something.txt
Uploading artifacts for successful job
00:01
Uploading artifacts...
WARNING: gl-sast-report.json: no matching files    
ERROR: No files to upload                          
Cleaning up file based variables
00:01
Job succeeded

笔记：

• Something.txt 完成了这项工作
• sast 作业中的所有工件都不会进入后续作业

我只能得出结论，sast 模板内部有一些东西不允许工件传播到后续作业。

Answer 1

我花了一整天的时间来解决这个问题，试图访问内置 IaC 扫描仪生成的 gl-sast-report.json 文件。这最终对我有用：

首先也是最重要的，不要使用 GitLab 文档建议的代码：

include:
  - template: Security/SAST-IaC.latest.gitlab-ci.yml

如果您只想扫描 IaC 漏洞并稍后从 GitLab UI 下载报告，则上述代码可以正常工作。但谁愿意这么做呢？我想在下一份工作中访问该报告，如果报告中存在中等以上漏洞，则管道会失败！

如果这就是您想要做的，您需要将官方 GitLab IaC 扫描仪模板中的所有代码添加到您的管道中，然后进行一些修改。您可以在这里找到最新的模板代码，或者使用我下面的示例。

修改后的模板：

# Read more about this feature here: https://docs.gitlab.com/ee/user/application_security/iac_scanning/
#
# Configure SAST with CI/CD variables (https://docs.gitlab.com/ee/ci/variables/index.html).
# List of available variables: https://docs.gitlab.com/ee/user/application_security/iac_scanning/index.html

variables:
  # Setting this variable will affect all Security templates
  # (SAST, Dependency Scanning, ...)
  TEMPLATE_REGISTRY_HOST: 'registry.gitlab.com'
  SECURE_ANALYZERS_PREFIX: "$TEMPLATE_REGISTRY_HOST/security-products"
  SAST_IMAGE_SUFFIX: ""

  SAST_EXCLUDED_PATHS: "spec, test, tests, tmp"

iac-sast:
  stage: test
  artifacts:
    name: sast
    paths:
      - gl-sast-report.json
    #reports:
    #  sast: gl-sast-report.json
    when: always
  rules:
    - when: never
  # `rules` must be overridden explicitly by each child job
  # see https://gitlab.com/gitlab-org/gitlab/-/issues/218444
  variables:
    SEARCH_MAX_DEPTH: 4
  allow_failure: true
  script:
    - /analyzer run

kics-iac-sast:
  extends: iac-sast
  image:
    name: "$SAST_ANALYZER_IMAGE"
  variables:
    SAST_ANALYZER_IMAGE_TAG: 3
    SAST_ANALYZER_IMAGE: "$SECURE_ANALYZERS_PREFIX/kics:$SAST_ANALYZER_IMAGE_TAG$SAST_IMAGE_SUFFIX"
  rules:
    - if: $SAST_DISABLED
      when: never
    - if: $SAST_EXCLUDED_ANALYZERS =~ /kics/
      when: never
    - if: $CI_COMMIT_BRANCH
  

Enforce Compliance:
  stage: Compliance
  before_script:
    - apk add jq
  script:
    - jq -r '.vulnerabilities[] | select(.severity == "Critical") | (.severity, .message, .location, .identifiers[].url)' gl-sast-report.json > results.txt
    - jq -r '.vulnerabilities[] | select(.severity == "High") | (.severity, .message, .location, .identifiers[].url)' gl-sast-report.json >> results.txt
    - jq -r '.vulnerabilities[] | select(.severity == "Medium") | (.severity, .message, .location, .identifiers[].url)' gl-sast-report.json >> results.txt
    - chmod u+x check-sast-results.sh
    - ./check-sast-results.sh

您还需要确保在管道中添加两个阶段（如果您还没有）：

stages:
  # add these to whatever other stages you already have
  - test
  - Compliance

注意：尝试访问 gl-sast-report.json（本例中为“合规性”）的作业与 sast 扫描自身（本例中为“测试”）不在同一阶段，这一点非常重要。如果是，那么您的作业将尝试在报告存在之前访问该报告并失败。

我将在管道中引用我的 shell 脚本，以防您也想使用它：

#!/bin/sh

if [ -s results.txt ]; then
        echo ""
        echo ""
        cat results.txt
        echo ""
        echo "ERROR: SAST SCAN FOUND VULNERABILITIES - FIX ALL VULNERABILITIES TO CONTINUE"
        echo ""
        exit 1
fi

这是一个基本脚本，用于检查“results.txt”文件是否有任何内容。如果存在，则会以代码 1 退出，以破坏管道并打印漏洞。如果文件中没有内容，脚本将以代码 0 退出并且管道继续（允许您部署基础设施）。将上面的文件保存为 GitLab 存储库根目录中的“check-sast-results.sh”（与“.gitlab-ci.yml”所在的级别相同）。

希望这对那里的人有帮助。