Server-side Apply
k8s 在1.16 支持了 Server-side Apply
什么是Server-side Apply, 简单的说就是多个Controller控制一个资源
多个Controller控制一个资源, 通过managedFields来记录哪个Field被哪个资源控制
Multi Controller example
拿depolyment举例, deployment包含了镜像,quota额度(CPU,memory),副本数等一系列资源,
我现在想让副本数和镜像由不同的Controller控制, 每个Controller 控制哪些字段就记录在
managedFields 列里面
它详细记录的该资源的哪些Field(镜像地址,CPU资源,Mem资源等等)由哪些Controller控制
看下面的示例
假设
使用WorkloadController负责创建deployment
使用ScalerController负责deployment副本数和资源使用量
即两个Controller同时控制一个deployment
WorkloadController
负责创建deployment
ao := []client.PatchOption{client.ForceOwnership, client.FieldOwner(c1.GetUID())}
client.Patch(ctx,desired, client.Apply, ao...)
ScalerController
负责修改deployment的副本数
// 我们使用这种Patch方法, 会更新任何Field同时更新 managedFields
client.Patch(ctx, desired, client.MergeFrom(current), client.FieldOwner(c2.GetUID()))
mechanism
- 使用Server-side Apply 必须使用client.Apply
- 每次只能更新自己管理的部分, 否则更新失败, 得到类似
Apply failed with 5 conflicts的错误
Server-side Apply 只能Patch 需要增量部分,即delta部分, 如果包含了不是自己managedFields管理的部分, 需要相等
staging/src/k8s.io/apiserver/pkg/endpoints/handlers/fieldmanager/fieldmanager.go
问题
如果我们由于实现机制的原理, 我们Apply和MergeFrom都是全量
在WorkloadController使用Apply时候
如果目标资源已经存在通过managedFields Pre Patch 当前对象, 保证
WorkloadController的Apply不会修改ScalerController控制的Field
附
// ObjectMeta is metadata that all persisted resources must have, which includes all objects
// users must create.
type ObjectMeta struct {
// ManagedFields maps workflow-id and version to the set of fields
// that are managed by that workflow. This is mostly for internal
// housekeeping, and users typically shouldn't need to set or
// understand this field. A workflow can be the user's name, a
// controller's name, or the name of a specific apply path like
// "ci-cd". The set of fields is always in the version that the
// workflow used when modifying the object.
//
// +optional
ManagedFields []ManagedFieldsEntry `json:"managedFields,omitempty" protobuf:"bytes,17,rep,name=managedFields"`
}
// ManagedFieldsEntry is a workflow-id, a FieldSet and the group version of the resource
// that the fieldset applies to.
type ManagedFieldsEntry struct {
// Manager is an identifier of the workflow managing these fields.
Manager string `json:"manager,omitempty" protobuf:"bytes,1,opt,name=manager"`
// Operation is the type of operation which lead to this ManagedFieldsEntry being created.
// The only valid values for this field are 'Apply' and 'Update'.
Operation ManagedFieldsOperationType `json:"operation,omitempty" protobuf:"bytes,2,opt,name=operation,casttype=ManagedFieldsOperationType"`
// APIVersion defines the version of this resource that this field set
// applies to. The format is "group/version" just like the top-level
// APIVersion field. It is necessary to track the version of a field
// set because it cannot be automatically converted.
APIVersion string `json:"apiVersion,omitempty" protobuf:"bytes,3,opt,name=apiVersion"`
// Time is timestamp of when these fields were set. It should always be empty if Operation is 'Apply'
// +optional
Time *Time `json:"time,omitempty" protobuf:"bytes,4,opt,name=time"`
// Fields is tombstoned to show why 5 is a reserved protobuf tag.
//Fields *Fields `json:"fields,omitempty" protobuf:"bytes,5,opt,name=fields,casttype=Fields"`
// FieldsType is the discriminator for the different fields format and version.
// There is currently only one possible value: "FieldsV1"
FieldsType string `json:"fieldsType,omitempty" protobuf:"bytes,6,opt,name=fieldsType"`
// FieldsV1 holds the first JSON version format as described in the "FieldsV1" type.
// +optional
FieldsV1 *FieldsV1 `json:"fieldsV1,omitempty" protobuf:"bytes,7,opt,name=fieldsV1"`
}
type FieldsV1 struct {
// Raw is the underlying serialization of this object.
Raw []byte `json:"-" protobuf:"bytes,1,opt,name=Raw"`
}
apiVersion: v1
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
aios.4pd.io/ManualScalerTrait: "true"
aios.4pd.io/component-abs: '{"containerList":[{"name":"simple-web-0","Ports":[{"name":"tcpa","containerPort":8000,"protocol":"TCP"}]}]}'
aios.4pd.io/sage-namespace: prophet
aios.4pd.io/user.accesskey: c60362a6-5c9f-49f8-becd-eba7828ef67c
aios.4pd.io/workspace.id: "3"
deployment.kubernetes.io/revision: "4"
creationTimestamp: "2020-12-29T15:39:41Z"
generation: 4
labels:
aios.4pd.io/sage-namespace: prophet
aios.4pd.io/service-name: simple-web-34
aios.4pd.io/workload-type: ServerWorkload
app.oam.dev/component: simple-web-n.v-1.0.1
app.oam.dev/name: simple-web-34
app.oam.dev/resourceType: WORKLOAD
app.oam.dev/revision: simple-web-n.v-1.0.1-v1
managedFields:
- apiVersion: apps/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:aios.4pd.io/ManualScalerTrait: {}
f:aios.4pd.io/component-abs: {}
f:aios.4pd.io/sage-namespace: {}
f:aios.4pd.io/user.accesskey: {}
f:aios.4pd.io/workspace.id: {}
f:labels:
f:aios.4pd.io/sage-namespace: {}
f:aios.4pd.io/service-name: {}
f:aios.4pd.io/workload-type: {}
f:app.oam.dev/component: {}
f:app.oam.dev/name: {}
f:app.oam.dev/resourceType: {}
f:app.oam.dev/revision: {}
f:ownerReferences:
k:{"uid":"158ebbd8-f9ec-4b5e-9587-e43c681b9ec3"}:
.: {}
f:apiVersion: {}
f:blockOwnerDeletion: {}
f:controller: {}
f:kind: {}
f:name: {}
f:uid: {}
f:spec:
f:selector:
f:matchLabels:
f:aios.4pd.io/sage-namespace: {}
f:aios.4pd.io/serverworkload: {}
f:aios.4pd.io/service-name: {}
f:aios.4pd.io/workload-type: {}
f:app.kubernetes.io/name: {}
f:app.oam.dev/component: {}
f:app.oam.dev/name: {}
f:app.oam.dev/resourceType: {}
f:app.oam.dev/revision: {}
f:template:
f:metadata:
f:creationTimestamp: {}
f:labels:
f:aios.4pd.io/sage-namespace: {}
f:aios.4pd.io/serverworkload: {}
f:aios.4pd.io/service-name: {}
f:aios.4pd.io/workload-type: {}
f:app.kubernetes.io/name: {}
f:app.oam.dev/component: {}
f:app.oam.dev/name: {}
f:app.oam.dev/resourceType: {}
f:app.oam.dev/revision: {}
f:spec:
f:containers:
k:{"name":"simple-web-0"}:
.: {}
f:env:
k:{"name":"ENV_TEST_KEY"}:
.: {}
f:name: {}
f:value: {}
k:{"name":"MY_NODE_NAME"}:
.: {}
f:name: {}
f:valueFrom:
f:fieldRef:
f:fieldPath: {}
k:{"name":"MY_POD_IP"}:
.: {}
f:name: {}
f:valueFrom:
f:fieldRef:
f:fieldPath: {}
k:{"name":"MY_POD_NAME"}:
.: {}
f:name: {}
f:valueFrom:
f:fieldRef:
f:fieldPath: {}
k:{"name":"MY_POD_NAMESPACE"}:
.: {}
f:name: {}
f:valueFrom:
f:fieldRef:
f:fieldPath: {}
k:{"name":"MY_POD_SERVICE_ACCOUNT"}:
.: {}
f:name: {}
f:valueFrom:
f:fieldRef:
f:fieldPath: {}
k:{"name":"SAGE_NAMESPACE"}:
.: {}
f:name: {}
f:value: {}
f:image: {}
f:livenessProbe:
f:httpGet:
f:path: {}
f:port: {}
f:initialDelaySeconds: {}
f:periodSeconds: {}
f:timeoutSeconds: {}
f:name: {}
f:ports:
k:{"containerPort":8000,"protocol":"TCP"}:
.: {}
f:containerPort: {}
f:name: {}
f:protocol: {}
f:readinessProbe:
f:httpGet:
f:path: {}
f:port: {}
f:initialDelaySeconds: {}
f:periodSeconds: {}
f:timeoutSeconds: {}
f:volumeMounts:
k:{"mountPath":"/home/work/local/conf"}:
.: {}
f:mountPath: {}
f:name: {}
f:imagePullSecrets:
k:{"name":"docker-secret"}:
.: {}
f:name: {}
f:nodeSelector:
f:prophet.4paradigm.com/online: {}
f:volumes:
k:{"name":"config-file-simple-web-34-0-0"}:
.: {}
f:configMap:
f:defaultMode: {}
f:name: {}
f:name: {}
manager: ServerWorkload-simple-web-34
operation: Apply
time: "2020-12-29T15:41:20Z"
- apiVersion: apps/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:ownerReferences:
k:{"uid":"5bf6b2b3-ba69-4828-82c9-0da01bdc6a1a"}:
.: {}
f:apiVersion: {}
f:blockOwnerDeletion: {}
f:controller: {}
f:kind: {}
f:name: {}
f:uid: {}
f:spec:
f:replicas: {}
f:template:
f:spec:
f:containers:
k:{"name":"simple-web-0"}:
f:resources:
f:limits:
f:cpu: {}
f:memory: {}
f:requests:
f:cpu: {}
f:memory: {}
manager: ManualScalerTrait-simple-web-34
operation: Update
time: "2020-12-29T15:41:20Z"
- apiVersion: apps/v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:deployment.kubernetes.io/revision: {}
f:status:
f:availableReplicas: {}
f:conditions:
.: {}
k:{"type":"Available"}:
.: {}
f:lastTransitionTime: {}
f:lastUpdateTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Progressing"}:
.: {}
f:lastTransitionTime: {}
f:lastUpdateTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
f:observedGeneration: {}
f:readyReplicas: {}
f:replicas: {}
f:updatedReplicas: {}
manager: kube-controller-manager
operation: Update
time: "2020-12-29T15:41:37Z"
name: simple-web-34
namespace: prophet-resource-cdh
ownerReferences:
- apiVersion: core.4paradigm.com/v1alpha2
blockOwnerDeletion: true
controller: true
kind: ServerWorkload
name: simple-web-34
uid: 158ebbd8-f9ec-4b5e-9587-e43c681b9ec3
- apiVersion: core.4paradigm.com/v1alpha2
blockOwnerDeletion: true
controller: false
kind: ManualScalerTrait
name: simple-web-34
uid: 5bf6b2b3-ba69-4828-82c9-0da01bdc6a1a
resourceVersion: "38658396"
selfLink: /apis/apps/v1/namespaces/prophet-resource-cdh/deployments/simple-web-34
uid: 28070692-40dd-4a9a-8dde-071d0698ffbe
spec:
progressDeadlineSeconds: 600
replicas: 3
revisionHistoryLimit: 10
selector:
matchLabels:
aios.4pd.io/sage-namespace: prophet
aios.4pd.io/serverworkload: simple-web-34
aios.4pd.io/service-name: simple-web-34
aios.4pd.io/workload-type: ServerWorkload
app.kubernetes.io/name: simple-web-34
app.oam.dev/component: simple-web-n.v-1.0.1
app.oam.dev/name: simple-web-34
app.oam.dev/resourceType: WORKLOAD
app.oam.dev/revision: simple-web-n.v-1.0.1-v1
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
aios.4pd.io/sage-namespace: prophet
aios.4pd.io/serverworkload: simple-web-34
aios.4pd.io/service-name: simple-web-34
aios.4pd.io/workload-type: ServerWorkload
app.kubernetes.io/name: simple-web-34
app.oam.dev/component: simple-web-n.v-1.0.1
app.oam.dev/name: simple-web-34
app.oam.dev/resourceType: WORKLOAD
app.oam.dev/revision: simple-web-n.v-1.0.1-v1
spec:
containers:
- env:
- name: MY_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: MY_POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: MY_POD_IP
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: status.podIP
- name: MY_POD_SERVICE_ACCOUNT
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.serviceAccountName
- name: ENV_TEST_KEY
value: this value from service
- name: SAGE_NAMESPACE
value: prophet
image: allenhaozi/simple-go-web:v4
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8000
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 3
successThreshold: 1
timeoutSeconds: 3
name: simple-web-0
ports:
- containerPort: 8000
name: tcpa
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8000
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 3
successThreshold: 1
timeoutSeconds: 3
resources:
limits:
cpu: 10m
memory: 20Mi
requests:
cpu: 10m
memory: 20Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /home/work/local/conf
name: config-file-simple-web-34-0-0
dnsPolicy: ClusterFirst
imagePullSecrets:
- name: docker-secret
nodeSelector:
prophet.4paradigm.com/online: "true"
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 511
name: config-file-simple-web-34-0-0
name: config-file-simple-web-34-0-0
status:
availableReplicas: 3
conditions:
- lastTransitionTime: "2020-12-29T15:39:41Z"
lastUpdateTime: "2020-12-29T15:40:29Z"
message: ReplicaSet "simple-web-34-7d79ddcf8" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
- lastTransitionTime: "2020-12-29T15:41:37Z"
lastUpdateTime: "2020-12-29T15:41:37Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
observedGeneration: 4
readyReplicas: 3
replicas: 3
updatedReplicas: 3
kind: List
metadata:
resourceVersion: ""
selfLink: ""
网友评论