Checkout part of a branch in Azure DevOps Pipelines (GetSources)

Question

My repository in my organisation's devops project contains a lot of .net solutions and some unity projects as well. When I run my build pipeline, it fails due to several of these:

Error MSB3491: Could not write lines to file "obj\Release\path\to\file". There is not enough space on the disk.

I would like the pipeline to only checkout and fetch parts of the repository that are required for a successful build. This might also help with execution time of the pipeline since it currently also fetches the whole of my unity projects with gigabytes of resources which takes forever.

I would like to spread my projects across multiple repositories but the admin won't give me more than the one I already have. It got a lot better when I configured git fetch as shallow (--depth=1) but I still get the error every now and then.

This is how I configured the checkout:

steps:
- checkout: self
  clean: true
  # shallow fetch
  fetchDepth: 1
  lfs: false
  submodules: false

The build is done using VSBuild@1 task.

I can't find a valid solution to my problem except for using multiple repositories, which is not an option right now.

Edit: Shayki Abramczyk's solution #1 works perfectly. Here is my full implementation.

GitSparseCheckout.yml:

parameters:
  access: ''
  repository: ''
  sourcePath: ''

steps:
- checkout: none

- task: CmdLine@2
  inputs:
    script: |
      ECHO ##[command] git init
      git init
      ECHO ##[command] git sparse-checkout: ${{ parameters.sourcePath }}
      git config core.sparsecheckout true
      echo ${{ parameters.sourcePath }} >> .git/info/sparse-checkout
      ECHO ##[command] git remote add origin https://${{ parameters.repository }}
      git remote add origin https://${{ parameters.access }}@${{ parameters.repository }}
      ECHO ##[command] git fetch --progress --verbose --depth=1 origin master
      git fetch --progress --verbose --depth=1 origin master
      ECHO ##[command] git pull --progress --verbose origin master
      git pull --progress --verbose origin master

Checkout is called like this (where template path has to be adjusted):

- template: ../steps/GitSparseCheckout.yml
  parameters:
    access: anything:<YOUR_PERSONAL_ACCESS_TOKEN>
    repository: dev.azure.com/organisation/project/_git/repository
    sourcePath: path/to/files/

Answer 1

在 Azure DevOps 中，您无法选择仅获取存储库的一部分，但有一种解决方法：禁用“获取源”步骤并通过在脚本中手动执行相应的 git 命令来仅获取所需的源。

要禁用默认的“获取源”，只需none在结帐语句中指定：

- checkout: none

在管道中添加 CMD/PowerShell 任务以使用以下 2 个选项之一手动获取源：

1.使用git sparse-checkout仅获取部分 repo 。例如，仅获得目录src_1和src_2在中test的文件夹（开始用线REM ###只是一般的批评论）：

- script: |
    REM ### this will create a 'root' directory for your repo and cd into it
    mkdir myRepo
    cd myRepo
    REM ### initialize Git in the current directory
    git init
    REM ### set Git sparsecheckout to TRUE
    git config core.sparsecheckout true
    REM ### write the directories that you want to pull to the .git/info/sparse-checkout file (without the root directory)
    REM ### you can add multiple directories with multiple lines
    echo test/src_1/ >> .git/info/sparse-checkout
    echo test/src_2/ >> .git/info/sparse-checkout
    REM ### fetch the remote repo using your access token
    git remote add -f origin https://your.access.token@path.to.your/repo
    REM ### pull the files from the source branch of this build, using the build-in Azure DevOps variable for the branch name
    git pull origin $(Build.SourceBranch)
    displayName: 'Get only test/src_1 & test/src_2 directories'

现在在构建任务中创建myRepo工作目录。使用访问令牌获取远程存储库是必要的，因为使用checkout: none将阻止使用您的登录凭据。在管道的末尾，您可能需要添加步骤来清理myRepo目录。

2.使用Azure DevOps Rest API获取部分 repo （Git - Items - Get Items Batch）。

Answer 2

其他答案效果很好，但我发现了一种使用 git 潜在更新功能的不同方法。

这将获取深度为 1 并显示根文件夹中的所有文件以及folder1,folder2和folder3

        - task: CmdLine@2
          inputs:
            script: |
              git init
              git sparse-checkout init --cone
              git sparse-checkout set folder1 folder2 folder3
              git remote add origin https://<github-username>:%GITHUB_TOKEN%@<your-git-repo>
              git fetch --progress --verbose --depth=1 origin
              git switch develop
          env:
            GITHUB_TOKEN: $(GITHUB_TOKEN)

Answer 3

在 Ubuntu 和 Windows 代理上支持 LFS

parameters:
  folders: '*'

steps:
- bash: |
      set -ex
      export ORIGIN=$(Build.Repository.Uri)
      export REF=$(Build.SourceVersion)
      export FOLDERS='${{ parameters.folders }}'
      git version
      git lfs version
      git init
      git sparse-checkout init --cone
      git sparse-checkout add $FOLDERS
      git remote add origin $ORIGIN
      git config core.sparsecheckout true
      git config gc.auto 0
      git config advice.detachedHead false
      git config http.version HTTP/1.1
      git lfs install --local
      git config uploadpack.allowReachableSHA1InWant true
      git config http.extraheader "AUTHORIZATION: bearer $(System.AccessToken)"
      git fetch --force --no-tags --progress --depth 1 origin $REF
      git checkout $REF --progress --force
  displayName: Fast sparse Checkout

然后作为步骤使用

  steps:
  - checkout: none

  - template: fastCheckout.yaml
    parameters:
      folders: 'Folder1 src/Folder2'

您可以将文件夹作为参数传递

导出是为了更容易在本地测试脚本。

结账时间从 10 分钟缩短至 2 分钟