Kopia是一个快速、安全的开源备份/恢复工具，用于管理远程存储中的文件系统快照。如何使用kopia来备份Jenkins呢？
一、搭建Jenkins
关于Jenkins的搭建，本文不再赘述，参考https://www.jenkins.io/doc/book/installing/进行部署。
二、部署minio服务端
1、使用docker部署minio
docker run -d -p 9001:9000 -p 9002:9002 --name minio \
-e "MINIO_ACCESS_KEY=admin" \
-e "MINIO_SECRET_KEY=admin@minio" \
-v /opt/minio/data:/data \
-v /opt/minio/config:/root/.minio \
minio/minio:latest server /data --console-address ":9002"
访问minio控制台（http://192.168.116.128:9002/login）
2、创建bucket
创建名为jenkins-backup的存储桶。
三、kopia
1、部署kopia
//install the GPG signing key to verify authenticity of the releases
rpm --import https://kopia.io/signing-key
//Install Yum repository
cat <<EOF | sudo tee /etc/yum.repos.d/kopia.repo
[Kopia]
name=Kopia
baseurl=http://packages.kopia.io/rpm/stable/\$basearch/
gpgcheck=1
enabled=1
gpgkey=https://kopia.io/signing-key
EOF
//install Kopia
sudo yum install kopia
2、创建repository
kopia repository create s3 \
--endpoint=192.168.116.128:9001 \
--bucket=jenkins-backup \
--access-key=admin \
--secret-access-key=admin@minio \
--disable-tls
执行以上命令，并根据提示设置仓库密码。
[root@localhost ~]# kopia repository create s3 \
> --endpoint=192.168.116.128:9001 \
> --bucket=jenkins-backup \
> --access-key=admin \
> --secret-access-key=admin@minio \
> --disable-tls
Enter password to create new repository:
Re-enter password for verification:
Initializing repository with:
block hash: BLAKE2B-256-128
encryption: AES256-GCM-HMAC-SHA256
splitter: DYNAMIC-4M-BUZHASH
Connected to repository.
NOTICE: Kopia will check for updates on GitHub every 7 days, starting 24 hours after first use.
To disable this behavior, set environment variable KOPIA_CHECK_FOR_UPDATES=false
Alternatively you can remove the file "/root/.config/kopia/repository.config.update-info.json".
Retention:
Annual snapshots: 3 (defined for this target)
Monthly snapshots: 24 (defined for this target)
Weekly snapshots: 4 (defined for this target)
Daily snapshots: 7 (defined for this target)
Hourly snapshots: 48 (defined for this target)
Latest snapshots: 10 (defined for this target)
Compression disabled.
To find more information about default policy run 'kopia policy get'.
To change the policy use 'kopia policy set' command.
NOTE: Kopia will perform quick maintenance of the repository automatically every 1h0m0s
and full maintenance every 24h0m0s when running as root@localhost.
See https://kopia.io/docs/advanced/maintenance/ for more information.
NOTE: To validate that your provider is compatible with Kopia, please run:
$ kopia repository validate-provider
按照提示执行kopia repository validate-provider进行验证。
[root@localhost ~]# kopia repository validate-provider
Validating storage capacity and usage
Validating blob list responses
Validating non-existent blob responses
Writing blob (5000000 bytes)
Validating conditional creates...
Validating list responses...
Validating partial reads...
Validating full reads...
Validating metadata...
Running concurrency test for 30s...
All good.
Cleaning up temporary data...
3、使用命令kopia repository status查看存储库的状态
[root@localhost ~]# kopia repository status
Config file: /root/.config/kopia/repository.config
Description: Repository in S3: 192.168.116.128:9001 jenkins-backup
Hostname: localhost
Username: root
Read-only: false
Format blob cache: 15m0s
Storage type: s3
Storage capacity: unbounded
Storage config: {
"bucket": "jenkins-backup",
"endpoint": "192.168.116.128:9001",
"doNotUseTLS": true,
"accessKeyID": "admin",
"secretAccessKey": "***********",
"sessionToken": ""
}
Unique ID: 203d2b34e4b92e5def58be71652bc7070bfe54cf2664c59e843302976d01e8ee
Hash: BLAKE2B-256-128
Encryption: AES256-GCM-HMAC-SHA256
Splitter: DYNAMIC-4M-BUZHASH
Format version: 2
Content compression: true
Password changes: true
Max pack length: 20 MiB
Index Format: v2
Epoch Manager: enabled
Current Epoch: 0
Epoch refresh frequency: 20m0s
Epoch advance on: 20 blobs or 10 MiB, minimum 24h0m0s
Epoch cleanup margin: 4h0m0s
Epoch checkpoint every: 7 epochs
4、创建快照
接下来创建jenkins_home目录的快照：kopia snapshot create /var/jenkins_home
[root@localhost ~]# kopia snapshot create /var/jenkins_home
Snapshotting root@localhost:/var/jenkins_home ...
* 0 hashing, 3936 hashed (442.8 MB), 0 cached (0 B), uploaded 407.4 MB, estimated 442.8 MB (100.0%) 0s left
Created snapshot with root k83ef262e4cb8b8a9b0fb9b38afde032c and ID d2c3ca1c3adb2b58fe5d72dc8ffe7b66 in 7s
Running full maintenance...
Looking for active contents...
Looking for unreferenced contents...
Rewriting contents from short packs...
Not enough time has passed since previous successful Snapshot GC. Will try again next time.
Skipping blob deletion because not enough time has passed yet (59m59s left).
Cleaned up 0 logs.
Cleaning up old index blobs which have already been compacted...
Finished full maintenance.
列出文件和目录的快照
[root@localhost ~]# kopia snapshot list
root@localhost:/var/jenkins_home
2022-04-27 18:54:35 CST k83ef262e4cb8b8a9b0fb9b38afde032c 442.8 MB drwxr-xr-x files:3936 dirs:1240 (latest-1,hourly-1,daily-1,weekly-1,monthly-1,annual-1)
5、回到Jenkins，运行几次任务，然后再次创建快照
[root@localhost ~]# kopia snapshot create --all
Snapshotting root@localhost:/var/jenkins_home ...
* 0 hashing, 25 hashed (24.6 KB), 3930 cached (442.8 MB), uploaded 203 B, estimating...
Created snapshot with root k69a63ee3adaf12012c9d55aff4a38fa3 and ID 04defac5438cf141af2b744f35a9ee8f in 0s
此时，再次查看快照列表
[root@localhost ~]# kopia snapshot list
root@localhost:/var/jenkins_home
2022-04-27 18:54:35 CST k83ef262e4cb8b8a9b0fb9b38afde032c 442.8 MB drwxr-xr-x files:3936 dirs:1240 (latest-2,hourly-2)
2022-04-27 19:05:24 CST k69a63ee3adaf12012c9d55aff4a38fa3 442.8 MB drwxr-xr-x files:3955 dirs:1245 (latest-1,hourly-1,daily-1,weekly-1,monthly-1,annual-1)
6、可以通过kopia diff来获取两次快照的差异
kopia diff k83ef262e4cb8b8a9b0fb9b38afde032c k69a63ee3adaf12012c9d55aff4a38fa3
7、前面是针对整个jenkins_home做的快照，但是有些目录不需要做快照，比如.cache，.groovy，.java等目录。
首先，查看当前的快照策略
[root@localhost ~]# kopia policy show /var/jenkins_home/
Policy for root@localhost:/var/jenkins_home:
Retention:
Annual snapshots: 3 inherited from (global)
Monthly snapshots: 24 inherited from (global)
Weekly snapshots: 4 inherited from (global)
Daily snapshots: 7 inherited from (global)
Hourly snapshots: 48 inherited from (global)
Latest snapshots: 10 inherited from (global)
Files policy:
Ignore cache directories: true inherited from (global)
No ignore rules:
Read ignore rules from files: inherited from (global)
.kopiaignore
Scan one filesystem only: false inherited from (global)
Error handling policy:
Ignore file read errors: false inherited from (global)
Ignore directory read errors: false inherited from (global)
Ignore unknown types: true inherited from (global)
Scheduling policy:
Scheduled snapshots:
None.
Manual snapshot: false inherited from (global)
Uploads:
Max parallel snapshots (server/UI): 1 inherited from (global)
Max parallel file reads: - inherited from (global)
Compression disabled.
No actions defined.
Logging details (0-none, 10-maximum):
Directory snapshotted: 5 inherited from (global)
Directory ignored: 5 inherited from (global)
Entry snapshotted: 0 inherited from (global)
Entry ignored: 5 inherited from (global)
Entry cache hit: 0 inherited from (global)
Entry cache miss: 0 inherited from (global)
接下来，设置忽略提及的几个目录
[root@localhost ~]# kopia policy set --add-ignore .cache/ --add-ignore .groovy/ --add-ignore .java/ /var/jenkins_home/
Setting policy for root@localhost:/var/jenkins_home
- adding ".cache/" to "ignore rules"
- adding ".groovy/" to "ignore rules"
- adding ".java/" to "ignore rules"
再次查看策略
[root@localhost ~]# kopia policy show /var/jenkins_home/
Policy for root@localhost:/var/jenkins_home:
Retention:
Annual snapshots: 3 inherited from (global)
Monthly snapshots: 24 inherited from (global)
Weekly snapshots: 4 inherited from (global)
Daily snapshots: 7 inherited from (global)
Hourly snapshots: 48 inherited from (global)
Latest snapshots: 10 inherited from (global)
Files policy:
Ignore cache directories: true inherited from (global)
Ignore rules: (defined for this target)
.cache/
.groovy/
.java/
......
通过kopia policy list可以查看策略列表：
[root@localhost ~]# kopia policy list
4f3609b9bc6670c3e9e61b6f09cd1d0d (global)
aff64b90cc1993f4c968eec2a940b123 root@localhost:/var/jenkins_home
四、定时创建快照
前面的内容，我们都是手动执行来进行备份。但在实际应用中，可以结合crontab设置定时任务来执行。
创建定时备份脚本jenkins-kopia.sh
#!/bin/bash
export KOPIA_PASSWORD=123
kopia repository connect s3 --endpoint=192.168.116.128:9001 --bucket=jenkins-backup --access-key=admin --secret-access-key=admin@minio --disable-tls --no-check-for-updates
kopia snapshot create --all
结合crontab配置定时任务，
*/1 * * * * /opt/minio/jenkins-kopia.sh 2>&1
五、从快照恢复jenkins
进入jenkins_home，删除目录内的所有文件，然后通过kopia snapshot restore进行恢复，恢复后运行Jenkins进行检查。
[root@localhost ~]# cd /var/jenkins_home/
[root@localhost jenkins_home]# rm -rf *
[root@localhost jenkins_home]# ll
总用量 0
[root@localhost jenkins_home]#
[root@localhost jenkins_home]# kopia snapshot restore k69a63ee3adaf12012c9d55aff4a38fa3 /var/jenkins_home
Restoring to local filesystem (/var/jenkins_home) with parallelism=8...
Processed 1727 (124.8 MB) of 5199 (442.8 MB) 124.8 MB/s (28.2%) remaining 1s.
Processed 2125 (236.5 MB) of 5199 (442.8 MB) 118.2 MB/s (53.4%) remaining 0s.
Processed 2672 (272.5 MB) of 5199 (442.8 MB) 89.4 MB/s (61.5%) remaining 0s.
Processed 2995 (319.3 MB) of 5199 (442.8 MB) 78.8 MB/s (72.1%) remaining 0s.
Processed 3505 (352.7 MB) of 5199 (442.8 MB) 69.8 MB/s (79.7%) remaining 0s.
Processed 4950 (409 MB) of 5199 (442.8 MB) 67.5 MB/s (92.4%) remaining 0s.
Processed 5200 (442.8 MB) of 5199 (442.8 MB) 68.6 MB/s (100.0%) remaining 0s.
Restored 3955 files, 1245 directories and 0 symbolic links (442.8 MB).
参考：
https://kopia.io/docs/installation/#linux-installation-using-rpm-redhat-centos-fedora
https://kopia.io/docs/repositories/#amazon-s3
网友评论