缓存用途
在gitlab-ci中，缓存分为两种
-
artifacts：称作"制品"，一般是构建阶段生成的产物，比如C程序编译后的可执行文件，很可能是之后需要拿去测试发布。制品可以在不同的stage间传递。 -
cache：缓存一般用于存储项目的依赖，比如pip、npm、vendor，项目依赖变动不大的情况下使用缓存可以极大地加速构建过程。
缓存存放位置：
缓存可以指定存储到
不同executor存放cache的位置如下：
- shell：
Locally, stored under the gitlab-runner user’s home directory: /home/gitlab-runner/cache/<user>/<project>/<cache-key>/cache.zip - docker/docker machine：
Locally, stored under Docker volumes: /var/lib/docker/volumes/<volume-id>/_data/<user>/<project>/<cache-key>/cache.zip
这里以docker为例
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9cea7d5079ca 0cc85081e891 "gitlab-runner-helpe…" 4 minutes ago Exited (0) 4 minutes ago runner-BKHmmDWP-project-612-concurrent-0-cache-c33bcaa1fd2c77edfc3893b41966cea8
728e17ed6ad2 0cc85081e891 "gitlab-runner-helpe…" 4 minutes ago Exited (0) 4 minutes ago runner-BKHmmDWP-project-612-concurrent-0-cache-3c3f060a0374fc8bc39395164f415a70
0c855de03b08 0cc85081e891 "bash" 26 minutes ago Exited (127) 25 minutes ago pensive_banzai
639d15c4db90 gitlab/gitlab-runner:latest "/usr/bin/dumb-init …" 5 weeks ago Up 10 days gitlab-runner
# 9cea这个container是用来保存artifacts的
$ docker inspect 9cea | grep volume
"Type": "volume",
"Source": "/var/lib/docker/volumes/f375877b31044f7ebd0542408b4a118d481f6a7439f6c80200fe693a2baad3ab/_data"
$ docker inspect 9cea | grep com.gitlab.gitlab-runner.cache.dir
"com.gitlab.gitlab-runner.cache.dir": "/builds",
# 728e这个container是用来保存cache的
$docker inspect 728e17ed6ad2 | grep volume
"Type": "volume",
"Source": "/var/lib/docker/volumes/e8bd577e7737a8001487f6e509814f315cf85a41de47abfa0460258656893d73/_data",
docker inspect 728e | grep com.gitlab.gitlab-runner.cache.dir
"com.gitlab.gitlab-runner.cache.dir": "/cache",
# 搜索cache.zip的位置
$ find /var/lib/docker -name cache.zip
/var/lib/docker/volumes/e8bd577e7737a8001487f6e509814f315cf85a41de47abfa0460258656893d73/_data/yl2121/npm_module_test/default/cache.zip
- 查看两个容器的详细信息
docker inspect 9cea7d5079ca
[
{
"Id": "9cea7d5079cab30e5cc65cb600a5659924265e1c432206222eac935c35059296",
"Created": "2019-09-20T08:45:41.244822667Z",
"Path": "gitlab-runner-helper",
"Args": [
"cache-init",
"/builds"
],
"State": {
"Status": "exited",
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 0,
"Error": "",
"StartedAt": "2019-09-20T08:45:41.8873507Z",
"FinishedAt": "2019-09-20T08:45:41.882976815Z"
},
"Image": "sha256:0cc85081e89120d42ec6943bca0033664d387d7dee1b8d323ca32a2547506d3d",
"Mounts": [
{
"Type": "volume",
"Name": "f375877b31044f7ebd0542408b4a118d481f6a7439f6c80200fe693a2baad3ab",
"Source": "/var/lib/docker/volumes/f375877b31044f7ebd0542408b4a118d481f6a7439f6c80200fe693a2baad3ab/_data",
"Destination": "/builds",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "9cea7d5079ca",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": [
"gitlab-runner-helper",
"cache-init",
"/builds"
],
"Image": "sha256:0cc85081e89120d42ec6943bca0033664d387d7dee1b8d323ca32a2547506d3d",
"Volumes": {
"/builds": {}
},
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": {
"com.gitlab.gitlab-runner.cache.dir": "/builds",
"com.gitlab.gitlab-runner.job.before_sha": "0000000000000000000000000000000000000000",
"com.gitlab.gitlab-runner.job.id": "59503",
"com.gitlab.gitlab-runner.job.ref": "develop-2222",
"com.gitlab.gitlab-runner.job.sha": "3b5d33683230bfb34bc2d22bad4304adcf63b912",
"com.gitlab.gitlab-runner.project.id": "612",
"com.gitlab.gitlab-runner.runner.id": "BKHmmDWP",
"com.gitlab.gitlab-runner.runner.local_id": "0",
"com.gitlab.gitlab-runner.type": "cache"
}
},
]
docker inspect 728e17ed6ad2
[
{
"Id": "728e17ed6ad2098b6642f1ff81daf4bd5a67beaa70e15ced3775c3e4d8c4da87",
"Created": "2019-09-20T08:45:39.395724588Z",
"Path": "gitlab-runner-helper",
"Args": [
"cache-init",
"/cache"
],
"State": {
"Status": "exited",
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 0,
"Error": "",
"StartedAt": "2019-09-20T08:45:40.20530802Z",
"FinishedAt": "2019-09-20T08:45:40.201265468Z"
},
"Image": "sha256:0cc85081e89120d42ec6943bca0033664d387d7dee1b8d323ca32a2547506d3d",
"Mounts": [
{
"Type": "volume",
"Name": "e8bd577e7737a8001487f6e509814f315cf85a41de47abfa0460258656893d73",
"Source": "/var/lib/docker/volumes/e8bd577e7737a8001487f6e509814f315cf85a41de47abfa0460258656893d73/_data",
"Destination": "/cache",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "728e17ed6ad2",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": [
"gitlab-runner-helper",
"cache-init",
"/cache"
],
"Image": "sha256:0cc85081e89120d42ec6943bca0033664d387d7dee1b8d323ca32a2547506d3d",
"Volumes": {
"/cache": {}
},
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": {
"com.gitlab.gitlab-runner.cache.dir": "/cache",
"com.gitlab.gitlab-runner.job.before_sha": "0000000000000000000000000000000000000000",
"com.gitlab.gitlab-runner.job.id": "59503",
"com.gitlab.gitlab-runner.job.ref": "develop-2222",
"com.gitlab.gitlab-runner.job.sha": "3b5d33683230bfb34bc2d22bad4304adcf63b912",
"com.gitlab.gitlab-runner.project.id": "612",
"com.gitlab.gitlab-runner.runner.id": "BKHmmDWP",
"com.gitlab.gitlab-runner.runner.local_id": "0",
"com.gitlab.gitlab-runner.type": "cache"
}
},
...
}
}
}
]
缓存覆盖问题
如果不使用key，不同stage的缓存都会存在default下，生成cache.zip覆盖原来的缓存。
推荐使用以下方式：
stages:
- build
- test
job A:
stage: build
script: make build
cache:
key: same-key
paths:
- public/
job B:
stage: test
script: make test
cache:
key: same-key
paths:
- vendor/
清除所有项目的cache
set -e
docker version >/dev/null 2>/dev/null
echo Clearing docker cache...
CONTAINERS=$(docker ps -a -q \
--filter=status=exited \
--filter=status=dead \
--filter=label=com.gitlab.gitlab-runner.type=cache)
if [ -n "${CONTAINERS}" ]; then
docker rm -v ${CONTAINERS}
fi
网友评论