如何在没有 kubeconfig 文件的情况下为 C# Kubernetes 客户端启用 TLS 验证？

Question

我希望 C# 代码连接到 K8s 集群，现在只需列出命名空间。我尝试的以下 C# 代码可以工作，并连接到 Kubernetes 集群，但仅在“SkipTlsVerify = true”时才有效。当我将 SkipTlsVerify 设置为 false 时，出现以下错误：

Unhandled exception: k8s.Exceptions.KubeConfigException: A CA must be set when SkipTlsVerify === false

我不想跳过 TLS 验证。如何向 C# Kubernetes 客户端提供必要的 CA 信息以启用 TLS 验证？

我不想使用 BuildConfigFromConfigFile()。这些都无法进入文件系统来读取或写入文件。现在我只能使用局部变量

我一开始尝试过，它有效，但它跳过了 TLS 验证：

        var contextName = "mycontext, i copied this from my kubeconfig file"
        var server = "https://...*** copied from 'server' in kubeconfig file *** "

        var config = new KubernetesClientConfiguration()
        {
            Host = server,
            AccessToken = accessToken,
            SkipTlsVerify = true,
        };
        var client = new Kubernetes(config);
        var namespaces = client.CoreV1.ListNamespace();
        foreach (var ns in namespaces)
        {
            Console.WriteLine(ns.Name());
        }

我也尝试了以下代码，并得到了相同的错误消息结果。除非 config.SkipTlsVerify 设置为 false，否则以下代码有效。内部的 SkipTlsVerify 没有任何影响，但外部的 SkipTlsVerify 确实会影响结果。我假设“ClientCertificateKeyData”C# 字段与我的 kubeconfig 文件中的“client-key-data”匹配。

        var clientCertificateData = "*** copied and pasted from client-certificate-data in kube-config ****";
        var clientKeyData = "*** copied from client-key-data in kube-config ****";
        var certificateAuthorityData = "...copied from certificate-authority-data in kube-config"
        var config = KubernetesClientConfiguration.BuildConfigFromConfigObject(new K8SConfiguration
        {
            ApiVersion = "v1",
            Clusters = new List<Cluster>
            {
                new()
                {
                    ClusterEndpoint = new ClusterEndpoint
                    {
                        CertificateAuthorityData = certificateAuthorityData,
                        Server = server,
                        //SkipTlsVerify = true // This one has no effect. I still get the same 
                                               //error even when setting this to true
                    },
                    Name = contextName
                }
            }
        }, masterUrl: server); // I think it's a little strange that I need to put in server here  
                               // If I omit masterUrl, i get the error 
                               //"k8s.Exceptions.KubeConfigException: 
                               // Cannot infer server host url either from context or masterUrl"
        //config.SkipTlsVerify = true;  // uncommenting this makes it work
        config.Host = server;
        config.AccessToken = accessToken;
        config.ClientCertificateData = clientCertificateData;
        config.ClientCertificateKeyData = clientKeyData; // I assume this line is supposed to be client-key-data from the kubeconfig?
        var client2 = new Kubernetes(config);
        var namespaces2 = client2.CoreV1.ListNamespace();
        foreach (var ns in namespaces2)
        {
            Console.WriteLine(ns.Name());
        }

       

Answer 1

对不起，我生锈的 dotnet，我主要是一个 python 程序员。这个答案主要基于https://github.com/kubernetes-client/csharp/issues/621#issuecomment-843584577的代码

所以算法归结为：

1. 创建名为 config 的 KubernetesClientConfiguration，就像您已经做的那样
2. certificate-authority-data如果您想使用该字符串，请从 base64-encoded 中提取 CA 证书
3. 从该字符串创建 X509Certificate2
4. 创建 X509Certificate2Collection，包含以前的 X509Certificate2
5. 将此 X509Certificate2Collection 添加为 config.SslCaCerts

这是我的测试，我的集群使用基于证书的身份验证而不是令牌，因此配置对象有点不同，您应该像以前一样使用 accessToken 。使用最新mcr.microsoft.com/dotnet/sdk图像构建。

调查X509.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>


  <ItemGroup>
    <PackageReference Include="KubernetesClient" Version="5.0.5" />
  </ItemGroup>

</Project>

调查X509.cs

namespace InvestigateX509
{
    using System;
    using System.Security.Cryptography.X509Certificates;
    using k8s;

    class Program
    {

        static void Main(string[] args)
        {
            var server = "https://YOUR_HOST:6443";
            var certificateAuthorityData = "YOUR_BASE64_ENCODED_certificate-authority-data_FROM_KUBECONFIG";
            var clientCertificateData = "YOUR_BASE64_ENCODED_client-certificate-data_FROM_KUBECONFIG";
            var clientCertificateKeyData = "YOUR_BASE64_ENCODED_client-key-data_FROM_KUBECONFIG";
            var myNamespace = "ingress-nginx";

            var config = new KubernetesClientConfiguration()
            {
                Host = server,
                ClientCertificateKeyData = clientCertificateKeyData,
                ClientCertificateData = clientCertificateData,
            };
            byte[] decodedCertificateAuthorityData = Convert.FromBase64String(certificateAuthorityData);
            X509Certificate2 caCert = new X509Certificate2(decodedCertificateAuthorityData);
            X509Certificate2Collection caCollection = new X509Certificate2Collection();
            caCollection.Add(caCert);
            config.SslCaCerts = caCollection;
            var client = new Kubernetes(config);
            var pods = client.ListNamespacedPod(myNamespace);
            foreach (var pod in pods.Items)
            {
                Console.WriteLine($"Pod: {pod.Metadata.Name}");
            }
        }
    }
}

我的 dotnet 容器的输出示例

root@84fc24bd18c2:/src/InvestigateX509# ./bin/Debug/net7.0/InvestigateX509 
Pod: ingress-nginx-1656961013-controller-97k82
Pod: ingress-nginx-1656961013-controller-h7dhm
Pod: ingress-nginx-1656961013-controller-lndjw

它打印我的 3 节点集群中 ingress-nginx ingress 的所有 3 个 pod