调试kubernetes中的NoDiskConflict

Question

我们有一个吊舱工作了大约一个月,突然间它不能再安排了.描述pod似乎表明磁盘已满或无法使用,但它不是非常具体(请参阅下面描述pod的完整输出).

我已经确认此节点上的磁盘有足够的空间(95G),它引用的GCEPersistentDisk也有足够的空间(450G).还有什么可以让我再次使用它？

到目前为止,我已经尝试重新启动节点,甚至删除节点从头开始.这是GKE上的单节点集群.

谢谢你的任何提示!

> kubectl --namespace=bakery-production describe pods bakery-deployment-3841321805-l84nc
Name:       bakery-deployment-3841321805-l84nc
Namespace:  bakery-production
Node:       /
Labels:     pod-template-hash=3841321805,service=bakery
Status:     Pending
IP:     
Controllers:    ReplicaSet/bakery-deployment-3841321805
Containers:
  bakery:
    Image:  gcr.io/pear-deck-production/bakery:38fda09f727493e4e88def14d49fe36883414e08
    Port:   80/TCP
    QoS Tier:
      cpu:  BestEffort
      memory:   BestEffort
    Environment Variables:
      PEARDECK_CONTAINER_REGISTRY:  gcr.io/pear-deck-production
Volumes:
  docker-images:
    Type:   GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine)
    PDName: bakery-docker-images
    FSType: ext4
    Partition:  0
    ReadOnly:   false
  bakery-secret-volume:
    Type:   Secret (a volume populated by a Secret)
    SecretName: bakery-secret
  default-token-z3ew1:
    Type:   Secret (a volume populated by a Secret)
    SecretName: default-token-z3ew1
Events:
  FirstSeen LastSeen    Count   From            SubobjectPath   Type        Reason          Message
  --------- --------    -----   ----            -------------   --------    ------          -------
  20s       13s     4   {default-scheduler }            Warning     FailedScheduling    pod (bakery-deployment-3841321805-l84nc) failed to fit in any node
fit failure on node (gke-peardeck-infrastructure-0f42f748-node-qa5a): NoDiskConflict

Answer 1

如果您尝试计划引用已由另一个(已调度的)窗格引用的卷并且该卷不支持多个装入的窗格,则调度程序将返回NoDiskConflict.只有当它们都是只读时,GCE PD才允许多次安装.

因此,请确保在读写模式下只有一个pod引用GCE PD.

请参阅https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/algorithm/predicates/predicates.go#L105