Docker上的Kubernetes使用空的serviceaccount创建容器,没有令牌导致容器崩溃和重新启动

Question

仍然遇到类似问题1.3.0和高达1.4.0-alpha.0

在我的情况下(基于docker的设置),无论是trusty还是kubedns都会从api服务器中获得未授权.

奇怪的是,我发现实例内部没有秘密,在路径/var/run/secrets/kubernetes.io/serviceaccount下

[root@ ... ]#  kubectl exec -it kube-dns-v13-htfjo ls /bin/sh
/ #
/ # ls /var/run/secrets/kubernetes.io/serviceaccount
/ #

虽然它们似乎在节点和代理实例中

tmpfs on /var/lib/kubelet/pods/3de53b0c-45bb-11e6-9f03-08002776167a/volumes/kubernetes.io~secret/default-token-8axd8 type
tmpfs on /var/lib/kubelet/pods/3de5591e-45bb-11e6-9f03-08002776167a/volumes/kubernetes.io~secret/default-token-8axd8 type
tmpfs on /var/lib/kubelet/pods/f29f35c7-45cc-11e6-9f03-08002776167a/volumes/kubernetes.io~secret/default-token-ql88q type

• 删除密钥并删除pod然后重新创建它们没有任何效果
• 卸载和删除文件夹后重新启动群集也没有任何效果

当然,这导致kubedns无法启动.记录下面

I0709 09:04:11.578816       1 dns.go:394] Received DNS Request:kubernetes.default.svc.cluster.local., exact:false
I0709 09:04:11.578873       1 dns.go:427] records:[], retval:[], path:[local cluster svc default kubernetes]
I0709 09:04:11.579657       1 dns.go:394] Received DNS Request:kubernetes.default.svc.cluster.local., exact:false
I0709 09:04:11.579677       1 dns.go:427] records:[], retval:[], path:[local cluster svc default kubernetes]
E0709 09:04:11.786646       1 reflector.go:216] pkg/dns/dns.go:128: Failed to list *api.Service: serializer for text/html; charset=utf-8 doesn't exist
E0709 09:04:11.786995       1 reflector.go:216] pkg/dns/dns.go:127: Failed to list *api.Endpoints: serializer for text/html; charset=utf-8 doesn't exist
I0709 09:04:12.488674       1 dns.go:145] Ignoring error while waiting for service default/kubernetes: serializer for text/html; charset=utf-8 doesn't exist. Sleeping 1s before retrying.
E0709 09:04:12.879701       1 reflector.go:216] pkg/dns/dns.go:128: Failed to list *api.Service: serializer for text/html; charset=utf-8 doesn't exist
E0709 09:04:12.880000       1 reflector.go:216] pkg/dns/dns.go:127: Failed to list *api.Endpoints: serializer for text/html; charset=utf-8 doesn't exist
I0709 09:04:13.582561       1 dns.go:145] Ignoring error while waiting for service default/kubernetes: serializer for text/html; charset=utf-8 doesn't exist. Sleeping 1s before retrying.

Answer 1

这个似乎还是一个仍然存在的bug

https://github.com/kubernetes/kubernetes/issues/26943

有效的解决方法是在kubelet安装中添加rslave选项,如 --volume =/var/lib/kubelet:/ var/lib/kubelet:rw,rslave,如下所示.

该解决方案还取决于平台.阅读错误报告中的注释.

## Start kubernetes master
sudo docker run \
--volume=/:/rootfs:ro \
--volume=/sys:/sys:ro \
--volume=/var/lib/docker/:/var/lib/docker:rw \
**--volume=/var/lib/kubelet:/var/lib/kubelet:rw,rslave** \
--volume=/var/run:/var/run:rw \
--net=host \
--privileged=true \
--pid=host \
-d \
gcr.io/google_containers/hyperkube-amd64:${K8S_VERSION} \
/hyperkube kubelet \
    --allow-privileged=true \
    --api-servers=http://localhost:8080 \
    --v=2 \
    --address=0.0.0.0 \
    --enable-server \
    --hostname-override=127.0.0.1 \
    --config=/etc/kubernetes/manifests-multi \
    --containerized \
    --cluster-dns=10.0.0.10 \
    --cluster-domain=cluster.local