Kubernetes中微服务之间如何做TLS？

Question

抱歉我的英语不好，但我不知道如何解决我的问题。\n所以...\n简介：\n我有 2 个微服务（我称它们为 gRPCClient 和 gRPCServer，尽管它\xe2\x80\x99 并不重要）确切地）。他们需要通过 TLS 进行通信。没有 Kubernetes，一切都非常简单。我通过 cfssl 在 docker 容器中创建我的 CA，然后从 CA 获取根证书，并将其置于我的 grpc 应用程序的信任中（我在 Dockerfile 中执行此操作），以便我的 CA 签名的任何证书都能通过测试。

\n

现在 Kubernetes 已包含在游戏中。我正在本地使用 minikube 进行游戏。我在 mac 上创建本地集群“minikube start”（也许这很重要，我不\xe2\x80\x99t知道...）

\n

问题：\n此流程如何与 Kubernetes 配合使用？据我了解，Kubernetes 内部已经有一个 CA（如果不是这样，请纠正我）。我读了很多文章，但我真的什么都看不懂\xe2\x80\x99。我尝试了本文中的示例https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/一步一步：

\n

\n
1. 创建签名请求
\n

\n

    cat <<EOF | cfssl genkey - | cfssljson -bare server\n    {\n      "hosts": [\n        "my-svc.my-namespace.svc.cluster.local",\n        "my-pod.my-namespace.pod.cluster.local",\n        "192.0.2.24",\n        "10.0.34.2"\n      ],\n      "CN": "my-pod.my-namespace.pod.cluster.local",\n      "key": {\n        "algo": "ecdsa",\n        "size": 256\n      }\n    }\n    EOF\n

\n

The first thing I did not understand was the hosts. For example, my-svc.my-namespace.svc.cluster.local is the full name of my service? (I mean the service in Kubernetes as kind: Service). I have it in the namespace "dev" and its name is user-app-sesrvice. Should I specify user-app-sesrvice.dev.svc.cluster.local then? or it just user-app-sesrvice. Or is there some kind of command to get the full name of the service? 192.0.2.24 - as I understand it, is the IP of service, it is also unclear whether it is mandatory to specify it or is it possible only the name of the service? What if I have clusterIP: None installed, then I don\'t have IP for it. my-pod.my-namespace.pod.cluster.local - Should I specify this? If I have several pods, should I list them all? Then the problem is in the dynamics, because the pods are recreated, deleted and added, and I need to send a new request for signature each time. The same questions that I asked about service including some parts "my-pod" and "namespace"? Is it possible to see the full name of the pod with all this data. 10.0.34.2 - pods\' IP. The same question about pods\' IP.

\n

I tried to specify the host and CN as name of my service name "user-app-service" (as if I was working without a Kubernetes). I created a signature and a key. Then all the steps, created a request object for signature in the Kubernetes

\n

    cat <<EOF | kubectl apply -f -\n    apiVersion: certificates.k8s.io/v1beta1\n    kind: CertificateSigningRequest\n    metadata:\n      name: my-svc.my-namespace\n    spec:\n      request: $(cat server.csr | base64 | tr -d \'\\n\')\n      usages:\n      - digital signature\n      - key encipherment\n      - server auth\n    EOF\n

\n

Then I made it and I received a certificate\nFurther, based on security, I need to store the key and a certificate in secrets and then get it in the container (for the purposes of the test, I just put them in the container in the docker file, hard-coded), this is in the gRPC server. I run the deployment and created a client on golang, specifying config: = &tls.Config{} in the code so that it would pull the trusted certificates from the system itself, I thought that the Kubernetes has a CA, but did not find how to get its certificate in the docs. I thought the Kubernetes adds them to all the containers himself. But I got the error Unavailable desc = connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority". How should all this work? Where can I get a CA certificate from a Kubernetes? And then, do I need to add it to each container with my hands in dockerfile? or is this not the right tactic and is there some kind of automation from the Kubernetes?

\n

I found another way, this is to try to deploy cfssl https://hub.docker.com/r/cfssl/cfssl/ on the Kubernetes and already work with it, like there was no Kubernetes (I have not tried this method yet)

\n

How to put all this into a working system, what options to use and why? Maybe there are some full articles. I wrote a lot, but I hope it\xe2\x80\x99s clear. I really need the help.

\n

Answer 1

我将把我的答案分为几个部分：

Kubernetes 服务和 DNS 发现

一般来说，建议将 Service 放在 Kubernetes 中管理 pod 的 Deployment 前面。该服务为 Pod 创建稳定的 DNS 和 IP 端点，这些端点可能会被删除，并在重新创建时被分配不同的 IP 地址。DNS 服务发现是通过 ClusterIP 类型服务自动启用的，其格式为：<service name>.<kubernetes namespace>.svc.<cluster domain>其中群集域通常为 cluster.local。这意味着我们可以在证书的备用名称中使用自动创建的 DNS 和分配的 ClusterIP。

Kubernetes 内部 CA

Kubernetes 确实有一个内部 CA 以及用于发布 CSR 并由 CA 签署这些 CSR 的 API 方法，但我不会使用内部 CA 来保护微服务。内部 CA 主要由 kubelet 和其他内部集群进程用于向 Kubernetes API 服务器进行身份验证。没有自动续订功能，我认为证书将始终签名 30 天。

Kubernetes 原生证书管理

您可以安装并使用cert-manager，让集群使用自定义资源自动为您创建和管理证书。他们的网站上有很好的例子，所以我鼓励您检查一下是否感兴趣。您应该能够使用CA 颁发者类型并创建 证书资源，该资源将创建证书作为 Kubernetes Secret。对于替代名称，请参阅我的回复的手动部分中的以下证书生成步骤。

手动创建和部署证书

您应该能够使用 cfssl 使用“无 Kubernetes”方法来实现相同的结果：

1. 使用 cfssl 生成 CA

2. 将 CA 添加为映像中受信任的 CA（使用 Dockerfile 方法）

3. 创建 Kubernetes 服务（出于我将使用的示例目的kubectl create）

   $ kubectl create service clusterip grpcserver --tcp=8000
   

4. 描述创建的 Kubernetes 服务，注意 IP 在您的情况下很可能会有所不同

   $ kubectl describe service/grpcserver
   Name:              grpcserver
   Namespace:         default
   Labels:            app=grpcserver
   Annotations:       <none>
   Selector:          app=grpcserver
   Type:              ClusterIP
   IP:                10.108.125.158
   Port:              8000  8000/TCP
   TargetPort:        8000/TCP
   Endpoints:         <none>
   Session Affinity:  None
   Events:            <none>
   

5. 为 CN 为 grpcserver.default.svc.cluster.local 的 gRPCServer 生成证书，并使用以下替代名称：

   • 服务端
   • grpcserver.default.svc
   • grpcserver.default.svc.cluster.local
   • 10.108.125.158

6. 使用 cfssl 生成客户端证书

7. 将两个证书放入 Secret 对象中

   kubectl create secret tls server --cert=server.pem --key=server.key
   kubectl create secret tls client --cert=client.pem --key=client.key
   

8. 将秘密安装到 podspec 中

Answer 2

使用这种定制方法需要做很多样板工作。如果您有选择，我建议探索服务网格（例如 istio 或 linkerd），以在 kubernetes 中使用 TLS 保护微服务之间的通信。