Kubernetes 集群拉取 Docker 镜像报错 “pul

作者: awker | 来源:发表于2020-12-30 18:59 被阅读0次

Kubernetes 集群拉取 Docker 镜像报错 “pul
Kubernetes-在Kubernetes集群上搭建Hadoo
Docker-入门
使用docker运行一个lamp环境
在Docker中安装Verdaccio（Sinopia的新版本）
docker环境搭建zookeeper
docker 安装consul集群
Elasticsearch 单机集群及多机集群Docker实践
docker安装kibana+elasticsearch
kubeadm init从私有仓库拉取镜像

问题现象：

使用 kubectl create -f release.yaml 创建 deployment 时，启动的 pod 无法拉取镜像，报错 Unknown desc = Error response from daemon: pull access denied for cr.registry.platform.foobar.cn/prod/foobar, repository does not exist or may require 'docker login' 如下：

# kubectl describe pod foobar-699df4f77b-8zkvj
省略其他……
Events:
  Type     Reason     Age                 From                                                  Message
  ----     ------     ----                ----                                                  -------
  Normal   Scheduled  80s                 default-scheduler                                     Successfully assigned default/foobar-699df4f77b-8zkvj to cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc8
  Normal   Pulling    72s (x3 over 108s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc8  Pulling image "cr.registry.platform.foobar.cn/prod/foobar:1.5.11"
  Warning  Failed     72s (x3 over 108s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc8  Failed to pull image "cr.registry.platform.foobar.cn/prod/foobar:1.5.11": rpc error: code = Unknown desc = Error response from daemon: pull access denied for cr.registry.platform.foobar.cn/prod/foobar, repository does not exist or may require 'docker login'
  Warning  Failed     72s (x3 over 108s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc8  Error: ErrImagePull
  Normal   BackOff    37s (x6 over 107s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc8  Back-off pulling image "cr.registry.platform.foobar.cn/prod/foobar:1.5.11"
  Warning  Failed     37s (x6 over 107s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc8  Error: ImagePullBackOff

直接在 node 节点上 docker pull 可以拉取镜像。

# docker pull cr.registry.platform.foobar.cn/prod/foobar:1.5.11
1.5.11: Pulling from prod/foobar
840caab23da4: Pull complete 
c73e835deefd: Pull complete 
2eec68d27246: Pull complete 
c5cfd8d4e231: Pull complete 
db1ee2b864ce: Pull complete 
8d01cb45f656: Pull complete 
b47d363c0228: Pull complete 
0dd590ba0e0b: Pull complete 
b15216aaf651: Pull complete 
da20bc470edc: Pull complete 
ae966cba2cfc: Pull complete 
642b4b98622b: Pull complete 
Digest: sha256:f523eb8b8f1c1da7c659a6a2a60bfc611ef7287e42e724eb4e99e8a37076f53e
Status: Downloaded newer image for cr.registry.platform.foobar.cn/prod/foobar:1.5.11

原因分析：

cr.registry.platform.foobar.cn/prod/foobar:1.5.11 存放在自建的私有镜像仓库中。kubernetes 并不能直接使用 docker 的 /root/.docker/config.json 凭证拉取这个镜像，需要单独配置自己的凭证来拉取镜像。

解决办法：

方式一：使用密钥编排容器，在 Pod 上指定 ImagePullSecrets。

// 查看 kubernetes 的 secret
# kubectl get secret -o wide
NAME                      TYPE                                  DATA   AGE
aliyun-acr-credential-a   kubernetes.io/dockerconfigjson        1      24d
aliyun-acr-credential-b   kubernetes.io/dockerconfigjson        1      24d
default-token-x9m7q       kubernetes.io/service-account-token   3      24d

// 创建 kubernetes 拉取私有镜像仓库的凭证 foobar-registry。
# kubectl create secret docker-registry foobar-registry --docker-server=私有镜像仓库地址 --docker-username=用户名 --docker-password=密码 --docker-email=邮箱
secret/foobar-registry created

# kubectl get secret -o wide
NAME                      TYPE                                  DATA   AGE
aliyun-acr-credential-a   kubernetes.io/dockerconfigjson        1      24d
aliyun-acr-credential-b   kubernetes.io/dockerconfigjson        1      24d
default-token-x9m7q       kubernetes.io/service-account-token   3      24d
foobar-registry             kubernetes.io/dockerconfigjson        1      2s

// 在 Pod 上指定 ImagePullSecrets 为 foobar-registry 来拉取镜像。
# vim release.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: foobar
  labels:
    app: foobar
spec:
  replicas: 2
  selector:
    matchLabels:
      app: foobar
  template:
    metadata:
      labels:
        app: foobar
    spec:
      containers:
      - name: foobar
        image: cr.registry.platform.foobar.cn/prod/foobar:1.5.11
        imagePullPolicy: Always
      imagePullSecrets:
      - name: foobar-registry


# kubectl create -f release.yaml
// 查看 kubernetes pod 事件，可以知道成功拉取了私有镜像仓库的镜像
# kubectl describe pod foobar-699df4f77b-8zkvj
省略其他……
Events:
  Type     Reason     Age                From                                                  Message
  ----     ------     ----               ----                                                  -------
  Normal   Pulling    31s (x3 over 45s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Pulling image "cr.registry.platform.foobar.cn/prod/foobar:1.5.11"
  Normal   Pulled     31s (x3 over 45s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Successfully pulled image "cr.registry.platform.foobar.cn/prod/foobar:1.5.11"
  Normal   Created    31s (x3 over 45s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Created container foobar
  Normal   Started    31s (x3 over 45s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Started container foobar

方式二：实现无密钥编排，为服务账户添加 ImagePullSecrets。

// 查看 kubernetes 的 serviceaccounts 
# kubectl get serviceaccounts default -o yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: "2020-12-04T10:04:26Z"
  name: default
  namespace: default
  resourceVersion: "344"
  selfLink: /api/v1/namespaces/default/serviceaccounts/default
  uid: 1740b59c-3618-11eb-848c-00163e01064f
secrets:
- name: default-token-x9m7q

// 导出 服务账户 的配置文件
# kubectl get serviceaccounts default -o yaml > /tmp/sa.yaml
# cat /tmp/sa.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: "2020-12-04T10:04:26Z"
  name: default
  namespace: default
  resourceVersion: "344"
  selfLink: /api/v1/namespaces/default/serviceaccounts/default
  uid: 1740b59c-3618-11eb-848c-00163e01064f
secrets:
- name: default-token-x9m7q

// 删除了 resourceVersion 字段，添加了 imagePullSecrets 字段，name 为上述添加的 secret foobar-registry
# vim /tmp/sa.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  creationTimestamp: "2020-12-04T10:04:26Z"
  name: default
  namespace: default
  selfLink: /api/v1/namespaces/default/serviceaccounts/default
  uid: 1740b59c-3618-11eb-848c-00163e01064f
secrets:
- name: default-token-x9m7q
imagePullSecrets:
- name: foobar-registry

// 替换现有的 服务账户 的配置文件
# kubectl replace serviceaccount default -f /tmp/sa.yaml 
serviceaccount/default replaced
# kubectl get serviceaccounts default -o yaml 
apiVersion: v1
imagePullSecrets:
- name: foobar-registry
kind: ServiceAccount
metadata:
  creationTimestamp: "2020-12-04T10:04:26Z"
  name: default
  namespace: default
  resourceVersion: "5918451"
  selfLink: /api/v1/namespaces/default/serviceaccounts/default
  uid: 1740b59c-3618-11eb-848c-00163e01064f
secrets:
- name: default-token-x9m7q

// 在 Pod 上不用单独指定 ImagePullSecrets 来拉取镜像。因为配置了 serviceaccounts 的 ImagePullSecrets，可以直接拉取镜像。
# vim release.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: foobar
  labels:
    app: foobar
spec:
  replicas: 2
  selector:
    matchLabels:
      app: foobar
  template:
    metadata:
      labels:
        app: foobar
    spec:
      containers:
      - name: foobar
        image: cr.registry.platform.foobar.cn/prod/foobar:1.5.11
        imagePullPolicy: Always

# kubectl create -f release.yaml
// 查看 kubernetes pod 事件，可以知道成功拉取了私有镜像仓库的镜像
# kubectl describe pod foobar-699df4f77b-ngg7f
Events:
  Type     Reason     Age                From                                                  Message
  ----     ------     ----               ----                                                  -------
  Normal   Pulling    26s (x3 over 41s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Pulling image "cr.registry.platform.foobar.cn/prod/foobar:1.5.11"
  Normal   Pulled     26s (x3 over 41s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Successfully pulled image "cr.registry.platform.foobar.cn/prod/foobar:1.5.11"
  Normal   Created    26s (x3 over 41s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Created container foobar
  Normal   Started    26s (x3 over 40s)  kubelet, cn-shanghai-foobar-d01.i-a0z01nzs2pkled5iikc9  Started container foobar

参考：

使用私有仓库：https://kubernetes.io/zh/docs/concepts/containers/images/#using-a-private-registry
在 Pod 上指定 ImagePullSecrets：https://kubernetes.io/zh/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
为服务账户添加 ImagePullSecrets：https://kubernetes.io/zh/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account
在Kubernetes集群如何支持私有镜像：https://help.aliyun.com/knowledge_detail/86562.html

Kubernetes 集群拉取 Docker 镜像报错 “pul
问题现象：使用 kubectl create -f release.yaml 创建 deployment 时，启...
Kubernetes-在Kubernetes集群上搭建Hadoo
准备工作 Hadoop镜像，到docker hub上拉取 Kubernetes集群参考：Kubernetes-离线...
Docker-入门
一、pull镜像 Docker镜像官方好像只提供Linux，拉取官方最新版的ubuntu镜像：docker pul...
使用docker运行一个lamp环境
执行步骤拉取lamp镜像docker search lamp本文挑选 fauria/lampdocker pul...
在Docker中安装Verdaccio（Sinopia的新版本）
Verdaccio官网地址：https://verdaccio.org/en/ 一、拉取镜像 docker pul...
docker环境搭建zookeeper
1、用 docker images 查看本地已拉取镜像如果本地还没有zk镜像，那么使用命令：docker pul...
docker 安装consul集群
docker 安装consul集群 consul架构拉取镜像 https://hub.docker.com/_/...
Elasticsearch 单机集群及多机集群Docker实践
（一）Docker 单机集群部署拉取镜像 docker pull docker.elastic.co/elast...
docker安装kibana+elasticsearch
1、首先还是拉取镜像，我们这里与前面安装的elasticsearch保持一直，都用6.8.1 docker pul...
kubeadm init从私有仓库拉取镜像
查看kubeadm config所需的镜像第一种：中转首先从Kubernetes国内Docker镜像拉取镜像，...