关于版本的选择

openshift 4的安装需要用到很多文件，如下：

ocp_bios: "https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.3/4.3.8/rhcos-4.3.8-x86_64-metal.x86_64.raw.gz"
ocp_initramfs: "https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.3/4.3.8/rhcos-4.3.8-x86_64-installer-initramfs.x86_64.img"
ocp_install_kernel: "https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.3/4.3.8/rhcos-4.3.8-x86_64-installer-kernel-x86_64"
ocp_client: "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.3.8/openshift-client-linux-4.3.8.tar.gz"
ocp_installer: "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.3.8/openshift-install-linux-4.3.8.tar.gz"

• 首先我们要确认版本，主要还是确认rhcos相关文件的版本，比如我打开链接： https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.3/，里面只包含了rhcos 4.3.8相关的文件，其中latest也是4.3.8：
  1
• 其次再确认oc 和openshift-install文件的版本，打开链接：https://mirror.openshift.com/pub/openshift-v4/clients/ocp/，里面包含了很多版本：
  2

• 具体4.3.8的版本内容：

  
  3
• 在 https://cloud.redhat.com/openshift/install/metal/user-provisioned上下载的client和installer也是最新版本：
  4
• 总结
  为了确保所有软件版本的一致性，所以建议oc和installer以rhcos相关文件版本为主去下载，而不是使用最新版。

开始制作离线包

我使用在美国的VPS下载离线镜像，体验了一下飞一般的速度，不到一分钟就同步好镜像了，不过从VPS拉取打包的镜像文件时还是遇到网络慢的问题，我搭建了一个http服务，用迅雷去下载镜像文件，速度好很多也稳定。

• 下载oc命令客户端

wget https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.3.8/openshift-client-linux-4.3.8.tar.gz

• 解压，并且将oc命令复制到

tar -zxvf openshift-client-linux-4.3.8.tar.gz
cp oc /usr/local/bin

• 安装工具

yum -y install podman httpd-tools

• 创建准备给本地私有镜像仓库的相关目录

mkdir -p /opt/registry/{auth,certs,data}

• 创建证书，然后会提示你输入相关信息，

注意：Common Name (eg, your name or your server's hostname) 字段要填写镜像仓库的域名，不能使用IP。其他的可以空着，这一步我错了几次，如下：

cd /opt/registry/certs
openssl req -newkey rsa:4096 -nodes -sha256 -keyout domain.key -x509 -days 365 -out domain.crt
Generating a 4096 bit RSA private key
....................................++
................................................................................................................................................................................................................................................................................................................++
writing new private key to 'domain.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:cn
State or Province Name (full name) []:shanghai
Locality Name (eg, city) [Default City]: 
Organization Name (eg, company) [Default Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server's hostname) []:registry.vps.apo.ocp4.com
Email Address []:

• 创建密码文件

htpasswd -bBc /opt/registry/auth/htpasswd admin admin

• 使用一个镜像来部署私有镜像仓库：

podman run --name mirror-registry -p 5000:5000 \ 
     -v /opt/registry/data:/var/lib/registry:z \
     -v /opt/registry/auth:/auth:z \
     -e "REGISTRY_AUTH=htpasswd" \
     -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
     -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
     -v /opt/registry/certs:/certs:z \
     -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
     -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
     -d docker.io/library/registry:2

注意需要放行特定的防火墙端口，我这里直接关闭防火墙，所以不开放了，也可以按照官方文档开通防火墙：

firewall-cmd --add-port=5000/tcp --zone=internal --permanent 
firewall-cmd --add-port=5000/tcp --zone=public   --permanent 
firewall-cmd --reload

• 更新Linux系统证书

cp /opt/registry/certs/domain.crt /etc/pki/ca-trust/source/anchors/
update-ca-trust

• 测试镜像仓库是否能正常访问：

需要在/etc/hosts文件中添加域名解析，否则无法解析 registry.vps.apo.ocp4.com地址。

curl -u admin:admin -k https://registry.vps.apo.ocp4.com:5000/v2/_catalog
{"repositories":[]}

• 将镜像仓库的用户名密码镜像base64加密

echo -n ’admin:admin' | base64 -w0
YWRtaW46YWRtaW4=

• 去下载pull-secret.text文件，然后安装jq，为了能够将密钥文件进行json格式化，好看些。

yum install jq

• json格式化生成新的文件

cat ./pull-secret.text | jq .  >  pull-secret2.text

• 文件内容如下：

{
  "auths": {
    "cloud.openshift.com": {
      "auth": "b3BlbnNo...",
      "email": "you@example.com"
    },
    "quay.io": {
      "auth": "b3BlbnNo...",
      "email": "you@example.com"
    },
    "registry.connect.redhat.com": {
      "auth": "NTE3Njg5Nj...",
      "email": "you@example.com"
    },
    "registry.redhat.io": {
      "auth": "NTE3Njg5Nj...",
      "email": "you@example.com"
    }
  }
}

• 然后添加一下私有镜像仓库的内容，用于将quay的镜像同步到私有镜像仓库过程中需要做认证，这就是个认证密码文件，类似如下。

  "auths": {
...
    "<local_registry_host_name>:<local_registry_host_port>": { 
      "auth": "<credentials>", 
      "email": "you@example.com"
  },
...

• 导入一些拉取镜像需要的环境变量参数，就是一些镜像仓库的地址，版本之类的信息

export OCP_RELEASE=4.3.8-x86_64
export LOCAL_REGISTRY='registry.vps.apo.ocp4.com:5000'
export LOCAL_REPOSITORY='ocp4/openshift4'
export PRODUCT_REPO='openshift-release-dev'
export LOCAL_SECRET_JSON='/opt/registry/certs/pull-secret2.text'
export RELEASE_NAME="ocp-release" 

• 使用oc命令去同步

oc adm -a ${LOCAL_SECRET_JSON} release mirror \
     --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE} \
     --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY} \
     --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}

• 同步的输出日志如下

info: Mirroring 103 images to registry.vps.apo.ocp4.com:5000/ocp4/openshift4 ...
registry.vps.apo.ocp4.com:5000/
  ocp4/openshift4
    blobs:
Success
Update image:  registry.vps.apo.ocp4.com:5000/ocp4/openshift4:4.3.8-x86_64
Mirror prefix: registry.vps.apo.ocp4.com:5000/ocp4/openshift4

To use the new mirrored repository to install, add the following section to the install-config.yaml:

imageContentSources:
- mirrors:
  - registry.vps.apo.ocp4.com:5000/ocp4/openshift4
  source: quay.io/openshift-release-dev/ocp-release
- mirrors:
  - registry.vps.apo.ocp4.com:5000/ocp4/openshift4
  source: quay.io/openshift-release-dev/ocp-v4.0-art-dev


To use the new mirrored repository for upgrades, use the following to create an ImageContentSourcePolicy:

apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
  name: example
spec:
  repositoryDigestMirrors:
  - mirrors:
    - registry.vps.apo.ocp4.com:5000/ocp4/openshift4
    source: quay.io/openshift-release-dev/ocp-release
  - mirrors:
    - registry.vps.apo.ocp4.com:5000/ocp4/openshift4
    source: quay.io/openshift-release-dev/ocp-v4.0-art-dev

• 打包镜像仓库文件

在openshift3的时候是直接拉取镜像，然后打包镜像，在4没有提供这样命令，所以我就将镜像仓库文件直接全部打包，然后拉回国内，再放到一个虚拟机上，启动镜像仓库。

cd /opt/registry/data/
tar -czvf ocp4.3.8-images.tar.gz docker/
ll
drwxr-xr-x 3 root root       4096 Apr 21 07:10 docker
-rw-r--r-- 1 root root 5585707233 Apr 21 07:24 ocp4.3.8-images.tar.gz

有意思的地方

在同步镜像仓库的时候输出的日志，可以看见镜像的名称都是为openshift4，而tag为openshift版本加上具体的组件名称，并且还有对应的hash值

sha256:d57b9ab77f64cb3cc667d957d53248f004dfa3ba5c8e3270ad06465815fca9e1 registry.vps.apo.ocp4.com:5000/ocp4/openshift4:4.3.8-openshift-state-metrics