1.集群 DNS 没有正常运行

步骤 1：找集群 dns 的 service

#查找dns的service
$kubectl get svc -nkube-system -owide
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE   SELECTOR
kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   32d   k8s-app=kube-dns

步骤 2：查看 service 详情

$kubectl describe svc -nkube-system kube-dns
Name:              kube-dns
Namespace:         kube-system
Labels:            k8s-app=kube-dns
                   kubernetes.io/cluster-service=true
                   kubernetes.io/name=KubeDNS
Annotations:       prometheus.io/port: 9153
                   prometheus.io/scrape: true
Selector:          k8s-app=kube-dns
Type:              ClusterIP
IP Families:       <none>
IP:                10.96.0.10
IPs:               10.96.0.10
Port:              dns  53/UDP
TargetPort:        53/UDP
Endpoints:         10.244.0.34:53,10.244.0.66:53,10.244.0.8:53
Port:              dns-tcp  53/TCP
TargetPort:        53/TCP
Endpoints:         10.244.0.34:53,10.244.0.66:53,10.244.0.8:53
Port:              metrics  9153/TCP
TargetPort:        9153/TCP
Endpoints:         10.244.0.34:9153,10.244.0.66:9153,10.244.0.8:9153
Session Affinity:  None
Events:            <none>

步骤 3：查看 endpoint 对应的 pod 状态

$kubectl get pod -nkube-system -owide |grep 10.244.0.34
coredns-659f5bbffd-w5vzw                   1/1     Running   0          2d   10.244.0.34     master-0002    <none>           <none>
$kubectl get pod -nkube-system -owide |grep 10.244.0.66
coredns-659f5bbffd-qrzl8                   1/1     Running   0         2d   10.244.0.66     master-0003    <none>           <none>
$kubectl get pod -nkube-system -owide |grep 10.244.0.8
coredns-659f5bbffd-rfr79                   1/1     Running   0          2d   10.244.0.8      master-0001    <none>           <none>

2.Pod 与 DNS 的网通不通

步骤 1：pod 里 curl 一下 dns 的 53 端口，参考 curl 命令详解

$curl -vi 10.244.0.31:53
*   Trying 10.244.0.31:53...
* connect to 10.244.0.31 port 53 failed: Connection timed out
* Failed to connect to 10.244.0.31 port 53: Connection timed out
* Closing connection 0
curl: (28) Failed to connect to 10.244.0.31 port 53: Connection timed out

步骤 2：如果网络不通，需要排查一下网络配置
① 检查 Pod 所在节点的安全组，需要对容器网络开放 53 端口
② 检查防火墙规则 iptables，参考【K8s 精选】深入剖析 Kubernetes 网络模型