使用 Docker Registry API 获取托管在 Docker Hub 上的公共 Docker 映像的清单

Question

我正在尝试找出用于此目的的正确 URL。例如，假设我想获取 alpine:3.9 标签的清单。我试过https://hub.docker.com/v2/repositories/library/alpine/manifests/3.9但这会产生 404 错误。

我发现 Docker Hub 的注册表实现与他们的文档并不真正匹配。例如https://docs.docker.com/registry/spec/api/#tags表示获取标签列表的 URL 是 v2/<name>/tags/list，但是在查询 Docker Hub 时，实际上需要去掉 URL 的“列表”部分：https : //hub.docker.com/v2/repositories/library/alpine/tags/。所以这让我现在在查询 Docker Hub 注册表时质疑他们文档的所有内容。

Answer 1

以下是一些 cURL 命令，用于练习某些 V2 端点。我对hub.docker.com端点的用途感到非常困惑( https://hub.docker.com/v2/users/login, https://hub.docker.com/v2/repositories/library/, 等等) 但我认为/v2/这完全是在误导并且与注册表V2 API无关？这篇文章使用hub.docker.com可以让你得到标签，但不能得到清单。

DOCKERHUB_USERNAME=$(jq -r '.username' < ~/.secrets/docker.json)
DOCKERHUB_PASSWORD=$(jq -r '.password' < ~/.secrets/docker.json)

TARGET_NS_REPO=library/debian

# yes, you need a new token for each repository, maybe you can have multiple scopes though?
PARAMS="service=registry.docker.io&scope=repository:$TARGET_NS_REPO:pull"
TOKEN=$(curl --user "$DOCKERHUB_USERNAME:$DOCKERHUB_PASSWORD" \
    "https://auth.docker.io/token?$PARAMS" \
    | jq -r '.token'
)

curl "https://registry-1.docker.io/v2/$TARGET_NS_REPO/tags/list" \
    -H "Authorization:Bearer $TOKEN" \
    | jq '.tags[:10]'

TAG="10-slim"
curl "https://registry-1.docker.io/v2/$TARGET_NS_REPO/manifests/$TAG" \
    -H "Authorization:Bearer $TOKEN" \
    | jq '.fsLayers'

输出：

[
  "10-slim",
  "10.0-slim",
  "10.0",
  "10",
  "6.0.10",
  "6.0.8",
  "6.0.9",
  "6.0",
  "6",
  "7-slim"
]
[
  {
    "blobSum": "sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4"
  },
  {
    "blobSum": "sha256:1ab2bdfe97783562315f98f94c0769b1897a05f7b0395ca1520ebee08666703b"
  }
]

逆向工程

我基本上不得不使用mitmproxy 对它进行逆向工程。如果你想知道其他任何东西是如何工作的：

1. 安装/运行mitmproxy。通过以下方式检查它是否正常工作：

curl -x localhost:8080 http://mitm.it/cert/pem  # should print out a cert

2. 获取/安装它的证书（给 MITM 自己）：

# Ubuntu 18.04, other distros may vary
MITM_CERT_PATH=/usr/local/share/ca-certificates/mitmproxy.crt
sudo cp ~/.mitmproxy/mitmproxy-ca-cert.cer "$MITM_CERT_PATH"
sudo chown root:root "$MITM_CERT_PATH"
sudo chmod 644 "$MITM_CERT_PATH"
sudo update-ca-certificates

# Verify MITM root cert accepted
curl -x localhost:8080 https://sha256.badssl.com/

# Troubleshooting
# - see if installed (https://unix.stackexchange.com/a/97252/42385)
awk -v cmd='openssl x509 -noout -subject' \
    '/BEGIN/{close(cmd)};{print | cmd}' \
    < /etc/ssl/certs/ca-certificates.crt \
    | grep -i mitmproxy

# - print the cert used (OpenSSL 1.1.0+)
openssl s_client -proxy localhost:8080 -showcerts -connect sha256.badssl.com:443 </dev/null

如果需要，稍后卸载证书

sudo rm /usr/local/share/ca-certificates/mitmproxy.crt
sudo update-ca-certificates

Check not in the list
awk -v cmd='openssl x509 -noout -subject' \
    '/BEGIN/{close(cmd)};{print | cmd}' \
    < /etc/ssl/certs/ca-certificates.crt \
    | grep -i mitmproxy

# Double-check MITM root cert rejected
curl -x localhost:8080 https://sha256.badssl.com/

3. dockerd使用HTTPS_PROXYset运行（如果服务已经在运行，则停止服务）

sudo HTTPS_PROXY=http://localhost:8080/ dockerd  # bash
# sudo env HTTPS_PROXY=http://localhost:8080/ dockerd  # fish

4. 告诉 Docker 守护进程做一些事情，例如docker pull alpine。在 mitmproxy 中你会看到类似的东西

Flows
   GET https://registry-1.docker.io/v2/
       ? 401 application/json 87b 213ms
   GET https://auth.docker.io/token?account=youraccount&scope=repository%3Alibrary%2Fal
       pine%3Apull&service=registry.docker.io
       ? 200 application/json 4.18k 245ms
>> GET https://registry-1.docker.io/v2/library/alpine/manifests/latest
       ? 200 application/vnd.docker.distribution.manifest.list.v2+json 1.6k 294ms
   GET https://registry-1.docker.io/v2/library/alpine/manifests/sha256:57334c50959f26ce
       1ee025d08f136c2292c128f84e7b229d1b0da5dac89e9866
       ? 200 application/vnd.docker.distribution.manifest.v2+json 528b 326ms
   GET https://registry-1.docker.io/v2/library/alpine/blobs/sha256:b7b28af77ffec6054d13
       378df4fdf02725830086c7444d9c278af25312aa39b9
       ? 307 text/html 242b 288ms
   GET https://registry-1.docker.io/v2/library/alpine/blobs/sha256:0503825856099e6adb39
       c8297af09547f69684b7016b7f3680ed801aa310baaa
       ? 307 text/html 242b 322ms
   GET https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sh
       a256/b7/b7b28af77ffec6054d13378df4fdf02725830086c7444d9c278af25312aa39b9/data?…
       ? 200 application/octet-stream 1.48k 191ms
   GET https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sh
       a256/05/0503825856099e6adb39c8297af09547f69684b7016b7f3680ed801aa310baaa/data?…
       ? 200 application/octet-stream 2.66m 207ms
?  [27/32]                                                                     [*:8080]

5. 检查请求。选择...manifests/latest要查看的请求：

Flow Details
2019-08-20 13:43:44 GET https://registry-1.docker.io/v2/library/alpine/manifests/latest
         ? 200 OK application/vnd.docker.distribution.manifest.list.v2+json 1.6k 294ms
       [[ Request ]]             Response                  Detail
Host:             registry-1.docker.io
User-Agent:       docker/19.03.1 go/go1.12.5 git-commit/74b1e89 kernel/4.15.0-55-generic
                  os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.1\\(linux\\))
Accept:           application/vnd.docker.distribution.manifest.v2+json
Accept:           application/vnd.docker.distribution.manifest.list.v2+json
Accept:           application/vnd.oci.image.index.v1+json
Accept:           application/vnd.docker.distribution.manifest.v1+prettyjws
Accept:           application/json
Accept:           application/vnd.oci.image.manifest.v1+json
Authorization:    Bearer eyJhbGci...(a big JWT returned by the auth.docker.io req.)
Accept-Encoding:  gzip
Connection:       close