基于 Calico 的 pod 没有出现

Question

我正在使用 calico 设置一个 pod，但它一直失败并出现一些授权错误。默认情况下，以下是我系统的节点 cidr：

[root@k8master-1 ~]# kubeadm config view | grep Subnet
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12

我已经使用以下步骤设置了 ippools：

https://docs.projectcalico.org/getting-started/kubernetes/flannel/flannel

IP池创建

- apiVersion: projectcalico.org/v3
  kind: IPPool
  metadata:
    name: rack-ip-pool 
  spec:
    blockSize: 26
    cidr: 10.244.1.0/24
    ipipMode: Never
    natOutgoing: true
    nodeSelector: all()
    vxlanMode: Never

ip池列表

[root@k8master-1 ~]# calicoctl get ippool -o wide
NAME          CIDR            NAT    IPIPMODE   VXLANMODE   DISABLED   SELECTOR   
rack-ip-pool   10.244.1.0/24   true   Never      Never       false      all()   

Pod Yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: testcalico
  labels:
    cracklerack: "1"
spec:
  serviceName: testcalico-svc
  selector:
    matchLabels:
      cracklerack: "1"
  template:
    metadata:
      labels:
        cracklerack: "1"
      annotations:
       cni.projectcalico.org/ipv4pools: "[\"rack-ip-pool\"]"
    spec:
      runtimeClassName: kata-containers
      containers:
      - name: testcalico
        image: cracklelinux:7
        ports:
        - containerPort: 80
        command: [/usr/sbin/init]
        securityContext:
          privileged: true
---
apiVersion: v1
kind: Service
metadata:
 name: testcalico-svc
spec:
 clusterIP: None
 selector:
   cracklerack: "1"

当我创建一个 pod 时，它会抛出以下错误：

错误

 Warning  FailedCreatePodSandBox  112s  kubelet, k8worker-1  Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_xxxxx-0_default_45357eab-bf40-4fe7-a470-da42c9668116_0(579e2c258154fcdc2e85df4a1e35264ea9550b0dd1c4384331abc471f552456d): connection is unauthorized: ipamconfigs.crd.projectcalico.org "default" is forbidden: User "system:serviceaccount:kube-system:canal" cannot get resource "ipamconfigs" in API group "crd.projectcalico.org" at the cluster scope

Answer 1

看起来你有一个 RBAC 问题，你的 pod 无法读取 Kubernetes 的IPAMConfigCRD。

我查看了https://docs.projectcalico.org/manifests/canal.yaml中的清单，我发现它ipamconfigs在几个 RBAC ClusterRoles 中丢失了。所以你可以继续尝试添加它们。

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: calico-kube-controllers
rules:
  # Nodes are watched to monitor for deletions.
  - apiGroups: [""]
    resources:
      - nodes
    verbs:
      - watch
      - list
      - get
  # Pods are queried to check for existence.
  - apiGroups: [""]
    resources:
      - pods
    verbs:
      - get
  # IPAM resources are manipulated when nodes are deleted.
  - apiGroups: ["crd.projectcalico.org"]
    resources:
      - ippools
    verbs:
      - list
  - apiGroups: ["crd.projectcalico.org"]
    resources:
      - blockaffinities
      - ipamblocks
      - ipamhandles
      - ipamconfigs  add here
 ...

然后是另一个 ClusterRole：

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: calico-node
rules:
...
  # Calico monitors various CRDs for config.
  - apiGroups: ["crd.projectcalico.org"]
    resources:
      - globalfelixconfigs
      - felixconfigurations
      - bgppeers
      - globalbgpconfigs
      - bgpconfigurations
      - ippools
      - ipamblocks
      - ipamconfigs  add here
      - globalnetworkpolicies
      - globalnetworksets
      - networkpolicies
      - networksets
      - clusterinformations
      - hostendpoints
      - blockaffinities
    verbs:
      - get
      - list
      - watch
  # Calico must create and update some CRDs on startup.
  - apiGroups: ["crd.projectcalico.org"]
    resources:
      - ippools
      - ipamconfigs  just in case
      - felixconfigurations
      - clusterinformations
    verbs:
      - create
      - update
...

然后运行：

kubectl apply -f canal.yaml

应用此功能后，您可能需要重新启动集群（至少在我的 minikube 上需要）。