我有一个使用 Rancher 的 K3s 部署的轻量级 Kubernetes 集群。
大多数时候 Pod 在其中运行良好,但我注意到它有时会遇到NodeDiskPressure,这会导致现有 Pod 被驱逐。
通过查看主机中的可用磁盘,我发现在此问题发生之前较高的集群负载与Containerd运行时存储中的高使用量一致。在正常情况下,这些卷的已用空间量为 70%,但它们会上升到 +90%,这可能会导致 pod 被驱逐。
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/3cd5b4cad915d0914436df95359e7685aa89fcd3f95f0b51e9a3d7db6f11d01b/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/fd2a513ce2736f10e98a203939aaa60bd28fbbb4f9ddbbd64a0aedbf75cae216/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/73865fcfa8b448d71b9b7c8297192b16612bd01732e3aa56d6e6a3936305b4a2/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/fc68e6653cec69361823068b3afa2ac51ecd6caf791bf4ae9a65305ec8126f37/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/7fcd3e8789f0ca7c8cabdc7522722697f76456607cbd0e179dd4826393c177ec/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/9334ed12649bcdb1d70f4b2e64c80168bdc86c897ddf699853daf9229516f5cf/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/de1c6f47cf82ff3362f0fc3ed4d4b7f5326a490d177513c76641e8f1a7e5eb1a/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/079c26817021c301cb516dab2ddcf31f4e224431d6555847eb76256369510482/rootfs
overlay 6281216 4375116 1906100 70% /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/d0da2f62430306d25565072edf478ad92752255a40830544101aeb576b862a5f/rootfs
overlay 6281216 4375116 1906100 70% …Run Code Online (Sandbox Code Playgroud) 是否可以在(特权)docker 容器内安装覆盖文件系统?至少我的直觉方法在容器外运行良好,但失败了:
> mkdir /tmp/{up,low,work,merged}
> mount -t overlay overlay -o lowerdir=/tmp/low/,upperdir=/tmp/up/,workdir=/tmp/work/ /tmp/merged/
mount: /tmp/merged: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error.
Run Code Online (Sandbox Code Playgroud)
附加信息:
OpenWrt 使用overlayfs
# mount | grep overlay
/dev/ubi0_2 on /overlay type ubifs (rw,noatime)
overlayfs:/overlay on / type overlay (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
Run Code Online (Sandbox Code Playgroud)
不知何故真正的lowerdir是 /rom
# mount | grep rom
mtd:ubi_rootfs on /rom type squashfs (ro,relatime)
Run Code Online (Sandbox Code Playgroud)
如何恢复文件以使其成为那样/rom?从覆盖中删除记录:whiteouts 和不透明的目录。
解决方法
cp /rom/path/to/file /overlay/upper/path/to/file
Run Code Online (Sandbox Code Playgroud) overlayfs ×3
docker ×2
containerd ×1
filesystems ×1
k3s ×1
kubernetes ×1
linux ×1
openwrt ×1
overlay ×1
unionfs ×1