无法在私有 GKE 集群中的 pod 上访问互联网

Question

我目前无法从我的私有 Kubernetes 集群访问/ping/连接到 Google 之外的任何服务。Pod 正在运行 Alpine linux。

路由表

/sleepez/api # ip route show table all
default via 10.52.1.1 dev eth0
10.52.1.0/24 dev eth0 scope link  src 10.52.1.4
broadcast 10.52.1.0 dev eth0 table local scope link  src 10.52.1.4
local 10.52.1.4 dev eth0 table local scope host  src 10.52.1.4
broadcast 10.52.1.255 dev eth0 table local scope link  src 10.52.1.4
broadcast 127.0.0.0 dev lo table local scope link  src 127.0.0.1
local 127.0.0.0/8 dev lo table local scope host  src 127.0.0.1
local 127.0.0.1 dev lo table local scope host  src 127.0.0.1
broadcast 127.255.255.255 dev lo table local scope link  src 127.0.0.1
local ::1 dev lo  metric 0
local fe80::ac29:afff:fea1:9357 dev lo  metric 0
fe80::/64 dev eth0  metric 256
ff00::/8 dev eth0  metric 256
unreachable default dev lo  metric -1  error -101

pod 肯定有一个分配的 IP，并且连接到它的网关没有问题：

PS C:\...\> kubectl get pods -o wide -n si-dev
NAME                              READY     STATUS    RESTARTS   AGE       IP          NODE
sleep-intel-api-79bf57bd9-c4l8d   1/1       Running   0          52m       10.52.1.4   gke-sez-production-default-pool-74b75ebc-6787

ip addr 输出

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1460 qdisc noqueue state UP
    link/ether 0a:58:0a:34:01:04 brd ff:ff:ff:ff:ff:ff
    inet 10.52.1.4/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::ac29:afff:fea1:9357/64 scope link
       valid_lft forever preferred_lft forever

Ping 网关工程

/sleepez/api # ping 10.52.1.1
PING 10.52.1.1 (10.52.1.1): 56 data bytes
64 bytes from 10.52.1.1: seq=0 ttl=64 time=0.111 ms
64 bytes from 10.52.1.1: seq=1 ttl=64 time=0.148 ms
64 bytes from 10.52.1.1: seq=2 ttl=64 time=0.137 ms
^C
--- 10.52.1.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.111/0.132/0.148 ms

Ping 1.1.1.1 失败

/sleepez/api # ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
^C
--- 1.1.1.1 ping statistics ---
6 packets transmitted, 0 packets received, 100% packet loss

系统服务状态

PS C:\...\> kubectl get deploy -n kube-system
NAME                    DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
event-exporter-v0.1.7   1         1         1            1           18m
heapster-v1.4.3         1         1         1            1           18m
kube-dns                2         2         2            2           18m
kube-dns-autoscaler     1         1         1            1           18m
l7-default-backend      1         1         1            1           18m
tiller-deploy           1         1         1            1           14m

跟踪路由（谷歌内部）

/sleepez/api # traceroute -In 74.125.69.105
 1  10.52.1.1  0.007 ms  0.006 ms  0.006 ms
 2  *  *  *
 3  *  *  *
 4  *  *

跟踪路由（外部）

traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 46 byte packets
 1  10.52.1.1  0.009 ms  0.003 ms  0.004 ms
 2  *  *  *
 3  *  *  *
 [continues...]

Answer 1

私有 GKE 集群中的节点没有外部 IP 地址，因此它们无法与 Google 之外的站点通信。https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#pulling_a_container_image_from_a_registry