如果您希望根據Pod實際的資源使用情況,動態調整和優化資源配置,以確保Pod能夠調度到資源充足的節點上,建議您為業務容器開啟垂直伸縮VPA(Vertical Pod Autoscaler)功能。VPA能夠監控Pod的資源消耗模式,靈活推薦CPU和內存資源分配的配置,并在適當的情況下自動進行調整,而不調整Pod的副本數量。這種能力更適用于需要穩定資源配置的有狀態應用的擴容等場景。
閱讀前提示
為了幫助您更好地使用VPA功能,建議您在閱讀本文前參見Kubernetes社區VPA介紹了解VPA的基本信息、使用流程、已知限制等。
此外,ACK集群提供了多種工作負載伸縮(調度層彈性)和節點伸縮(資源層彈性)方案,建議您在使用本文檔前閱讀彈性伸縮概述,了解不同方案的適用場景、使用限制等。
ack-vertical-pod-autoscaler組件介紹
VPA基于ack-vertical-pod-autoscaler組件實現,以根據實際資源需求推薦并自動調整Pod中容器的CPU及內存的Request和Limit參數。同時,VPA也會遵循容器初始配置中定義的資源Request和Limit占比。ack-vertical-pod-autoscaler組件有以下組成部分:
Admission Controller組件:為新建Pod設置正確的資源請求值Requests。安裝admission-controller組件前,您需要用此腳本為Webhook生成證書。
Recommender組件:監控容器目前和過去的資源使用情況,并基于檢測數據推薦資源配置,以更準確地匹配資源使用需求。
Updater組件:檢查VPA管理的Pod資源量配置是否正確。如果不正確,Updater會終止這些Pod,以便后續重建。
注意事項
VPA功能目前處于Beta階段,其性能未在大型集群中測試,請謹慎使用。如遇相關問題或有相關產品建議,請提交工單聯系容器服務團隊。
更新正在運行的Pod資源配置會導致Pod的重建和重啟,且有可能會被調度到其他節點上。VPA動態更新現有Pod的資源Request時,支持無需重啟的更新機制,但此機制仍處于測試階段。
VPA不會驅逐沒有在副本控制器(Replication Controller)管理下的Pod。在VPA中,此類Pod的Auto模式等同于Initial模式,即VPA會為新的Pod設置資源請求和限制,但不會自動刪除和重新創建那些資源分配不當的運行中的Pod。
不推薦您為同一個工作負載部署多個VPA。多個VPA同時匹配同一個Pod時,可能會造成不可預測的行為。
VPA對Pod資源Request的修改值可能超過實際的資源上限,例如節點資源上限、空閑資源或資源配額,從而造成Pod處于Pending狀態無法被調度。此時,使用節點自動伸縮可能可以解決這個問題:當Pod的資源使用率超出擴容閾值后,Pod將擴容,但此時節點資源不足,導致Pod調度失敗并處于Pending狀態,繼而觸發節點擴容。更多信息,請參見啟用節點自動伸縮。
目前,VPA和HPA的Controller無法完全兼容。如果您同時使用VPA和HPA來監控CPU和內存使用情況,這可能會產生沖突。如果您的HPA僅監控其他定制化的或者外部的資源使用情況,可以避免此沖突。
VPA使用Admission Webhook作為其準入控制器。請確保集群中存在的其他Admission Webhook不會與VPA發生沖突。
說明準入控制器的執行順序定義可在API Server的配置參數中查找。
VPA會處理出現的絕大多數OOM(Out Of Memory)的事件,但無法保證所有的場景下都有效。
關于VPA更多的已知限制,請參見VPA已知限制。
步驟一:安裝ack-vertical-pod-autoscaler組件
您可以通過控制臺(v1.26及以上集群)或kubectl(v1.26以下集群)的方式安裝ack-vertical-pod-autoscaler組件。
通過控制臺安裝(推薦)
推薦您通過容器服務管理控制臺安裝ack-vertical-pod-autoscaler組件,屏蔽組件底層復雜性,使用門檻和維護成本更低。
前提條件
已創建v1.26及以上的ACK托管集群。具體操作,請參見創建Kubernetes托管版集群。如需升級集群,請參見手動升級集群。
已使用命令行工具連接集群,請參見獲取集群KubeConfig并通過kubectl工具連接集群。
如果您之前通過kubectl的方式安裝了VPA,建議您卸載該VPA,通過控制臺重新安裝。具體操作,請參見下文如何通過控制臺管理使用kubectl安裝的VPA?。
安裝步驟
登錄容器服務管理控制臺,在左側導航欄選擇集群。
在集群列表頁面,單擊目標集群名稱,然后在左側導航欄,選擇 。
在組件管理頁面,定位ack-vertical-pod-autoscaler組件,然后按照頁面提示完成安裝。
通過kubectl安裝
前提條件
已創建v1.26以下的ACK托管集群。具體操作,請參見創建Kubernetes托管版集群。
已使用命令行工具連接集群,請參見獲取集群KubeConfig并通過kubectl工具連接集群。
如果您已在集群中部署VPA,請卸載該VPA,以避免新安裝的VPA與舊版VPA沖突。
操作步驟
保存RBAC權限的YAML文件,并執行命令,創建RBAC權限文件。
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-reader rules: - apiGroups: - "metrics.k8s.io" resources: - pods verbs: - get - list --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-actor rules: - apiGroups: - "" resources: - pods - nodes - limitranges verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - get - list - watch - create - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-status-actor rules: - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers/status verbs: - get - patch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-checkpoint-actor rules: - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalercheckpoints verbs: - get - list - watch - create - patch - delete - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalercheckpoints verbs: - get - list - watch - create - patch - delete - apiGroups: - "" resources: - namespaces verbs: - get - list --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:evictioner rules: - apiGroups: - "apps" - "extensions" resources: - replicasets verbs: - get - apiGroups: - "" resources: - pods/eviction verbs: - create --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-reader roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-reader subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-status-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-status-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-checkpoint-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-checkpoint-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-target-reader rules: - apiGroups: - '*' resources: - '*/scale' verbs: - get - watch - apiGroups: - "" resources: - replicationcontrollers verbs: - get - list - watch - apiGroups: - apps resources: - daemonsets - deployments - replicasets - statefulsets verbs: - get - list - watch - apiGroups: - batch resources: - jobs - cronjobs verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-target-reader-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-target-reader subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system - kind: ServiceAccount name: vpa-admission-controller namespace: kube-system - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-evictioner-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:evictioner subjects: - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-admission-controller namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-recommender namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-admission-controller rules: - apiGroups: - "" resources: - pods - configmaps - nodes - limitranges verbs: - get - list - watch - apiGroups: - "admissionregistration.k8s.io" resources: - mutatingwebhookconfigurations verbs: - create - delete - get - list - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "coordination.k8s.io" resources: - leases verbs: - create - update - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-admission-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-admission-controller subjects: - kind: ServiceAccount name: vpa-admission-controller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-status-reader rules: - apiGroups: - "coordination.k8s.io" resources: - leases verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-status-reader-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-status-reader subjects: - kind: ServiceAccount name: vpa-updater namespace: kube-system
kubectl apply -f rbac.yaml
保存CRD的YAML文件,并執行命令,創建ack-vertical-pod-autoscaler組件的CRD。
說明CRD可以提高Kubernetes的擴展能力,詳情請參見Extend the Kubernetes API with CustomResourceDefinitions。
CRD YAML定義如下:
1.22≤ 集群版本<1.26
apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 controller-gen.kubebuilder.io/version: v0.9.2 creationTimestamp: null name: verticalpodautoscalercheckpoints.autoscaling.k8s.io spec: group: autoscaling.k8s.io names: kind: VerticalPodAutoscalerCheckpoint listKind: VerticalPodAutoscalerCheckpointList plural: verticalpodautoscalercheckpoints shortNames: - vpacheckpoint singular: verticalpodautoscalercheckpoint scope: Namespaced versions: - name: v1 schema: openAPIV3Schema: description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal state of VPA that is used for recovery after recommender's restart. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: containerName: description: Name of the checkpointed container. type: string vpaObjectName: description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint object. type: string type: object status: description: Data of the checkpoint. properties: cpuHistogram: description: Checkpoint of histogram for consumption of CPU. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object firstSampleStart: description: Timestamp of the fist sample from the histograms. format: date-time nullable: true type: string lastSampleStart: description: Timestamp of the last sample from the histograms. format: date-time nullable: true type: string lastUpdateTime: description: The time when the status was last refreshed. format: date-time nullable: true type: string memoryHistogram: description: Checkpoint of histogram for consumption of memory. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object totalSamplesCount: description: Total number of samples in the histograms. type: integer version: description: Version of the format of the stored data. type: string type: object type: object served: true storage: true - name: v1beta2 schema: openAPIV3Schema: description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal state of VPA that is used for recovery after recommender's restart. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: containerName: description: Name of the checkpointed container. type: string vpaObjectName: description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint object. type: string type: object status: description: Data of the checkpoint. properties: cpuHistogram: description: Checkpoint of histogram for consumption of CPU. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object firstSampleStart: description: Timestamp of the fist sample from the histograms. format: date-time nullable: true type: string lastSampleStart: description: Timestamp of the last sample from the histograms. format: date-time nullable: true type: string lastUpdateTime: description: The time when the status was last refreshed. format: date-time nullable: true type: string memoryHistogram: description: Checkpoint of histogram for consumption of memory. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object totalSamplesCount: description: Total number of samples in the histograms. type: integer version: description: Version of the format of the stored data. type: string type: object type: object served: true storage: false --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 controller-gen.kubebuilder.io/version: v0.9.2 creationTimestamp: null name: verticalpodautoscalers.autoscaling.k8s.io spec: group: autoscaling.k8s.io names: kind: VerticalPodAutoscaler listKind: VerticalPodAutoscalerList plural: verticalpodautoscalers shortNames: - vpa singular: verticalpodautoscaler scope: Namespaced versions: - additionalPrinterColumns: - jsonPath: .spec.updatePolicy.updateMode name: Mode type: string - jsonPath: .status.recommendation.containerRecommendations[0].target.cpu name: CPU type: string - jsonPath: .status.recommendation.containerRecommendations[0].target.memory name: Mem type: string - jsonPath: .status.conditions[?(@.type=='RecommendationProvided')].status name: Provided type: string - jsonPath: .metadata.creationTimestamp name: Age type: date name: v1 schema: openAPIV3Schema: description: VerticalPodAutoscaler is the configuration for a vertical pod autoscaler, which automatically manages pod resources based on historical and real time resource utilization. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the behavior of the autoscaler. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: recommenders: description: Recommender responsible for generating recommendation for this object. List should be empty (then the default recommender will generate the recommendation) or contain exactly one recommender. items: description: VerticalPodAutoscalerRecommenderSelector points to a specific Vertical Pod Autoscaler recommender. In the future it might pass parameters to the recommender. properties: name: description: Name of the recommender responsible for generating recommendation for this object. type: string required: - name type: object type: array resourcePolicy: description: Controls how the autoscaler computes recommended resources. The resource policy may be used to set constraints on the recommendations for individual containers. If not specified, the autoscaler computes recommended resources for all containers in the pod, without additional constraints. properties: containerPolicies: description: Per-container resource policies. items: description: ContainerResourcePolicy controls how autoscaler computes the recommended resources for a specific container. properties: containerName: description: Name of the container or DefaultContainerResourcePolicy, in which case the policy is used by the containers that don't have their own policy specified. type: string controlledResources: description: Specifies the type of recommendations that will be computed (and possibly applied) by VPA. If not specified, the default of [ResourceCPU, ResourceMemory] will be used. items: description: ResourceName is the name identifying various resources in a ResourceList. type: string type: array controlledValues: description: Specifies which resource values should be controlled. The default is "RequestsAndLimits". enum: - RequestsAndLimits - RequestsOnly type: string maxAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the maximum amount of resources that will be recommended for the container. The default is no maximum. type: object minAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the minimal amount of resources that will be recommended for the container. The default is no minimum. type: object mode: description: Whether autoscaler is enabled for the container. The default is "Auto". enum: - Auto - "Off" type: string type: object type: array type: object targetRef: description: TargetRef points to the controller managing the set of pods for the autoscaler to control - e.g. Deployment, StatefulSet. VerticalPodAutoscaler can be targeted at controller implementing scale subresource (the pod set is retrieved from the controller's ScaleStatus) or some well known controllers (e.g. for DaemonSet the pod set is read from the controller's spec). If VerticalPodAutoscaler cannot use specified target it will report ConfigUnsupported condition. Note that VerticalPodAutoscaler does not require full implementation of scale subresource - it will not use it to modify the replica count. The only thing retrieved is a label selector matching pods grouped by the target resource. properties: apiVersion: description: API version of the referent type: string kind: description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' type: string name: description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' type: string required: - kind - name type: object x-kubernetes-map-type: atomic updatePolicy: description: Describes the rules on how changes are applied to the pods. If not specified, all fields in the `PodUpdatePolicy` are set to their default values. properties: minReplicas: description: Minimal number of replicas which need to be alive for Updater to attempt pod eviction (pending other checks like PDB). Only positive values are allowed. Overrides global '--min-replicas' flag. format: int32 type: integer updateMode: description: Controls when autoscaler applies changes to the pod resources. The default is 'Auto'. enum: - "Off" - Initial - Recreate - Auto type: string type: object required: - targetRef type: object status: description: Current information about the autoscaler. properties: conditions: description: Conditions is the set of conditions required for this autoscaler to scale its target, and indicates whether or not those conditions are met. items: description: VerticalPodAutoscalerCondition describes the state of a VerticalPodAutoscaler at a certain point. properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another format: date-time type: string message: description: message is a human-readable explanation containing details about the transition type: string reason: description: reason is the reason for the condition's last transition. type: string status: description: status is the status of the condition (True, False, Unknown) type: string type: description: type describes the current condition type: string required: - status - type type: object type: array recommendation: description: The most recently computed amount of resources recommended by the autoscaler for the controlled pods. properties: containerRecommendations: description: Resources recommended by the autoscaler for each container. items: description: RecommendedContainerResources is the recommendation of resources computed by autoscaler for a specific container. Respects the container resource policy if present in the spec. In particular the recommendation is not produced for containers with `ContainerScalingMode` set to 'Off'. properties: containerName: description: Name of the container. type: string lowerBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Minimum recommended amount of resources. Observes ContainerResourcePolicy. This amount is not guaranteed to be sufficient for the application to operate in a stable way, however running with less resources is likely to have significant impact on performance/availability. type: object target: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Recommended amount of resources. Observes ContainerResourcePolicy. type: object uncappedTarget: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: The most recent recommended resources target computed by the autoscaler for the controlled pods, based only on actual resource usage, not taking into account the ContainerResourcePolicy. May differ from the Recommendation if the actual resource usage causes the target to violate the ContainerResourcePolicy (lower than MinAllowed or higher that MaxAllowed). Used only as status indication, will not affect actual resource assignment. type: object upperBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Maximum recommended amount of resources. Observes ContainerResourcePolicy. Any resources allocated beyond this value are likely wasted. This value may be larger than the maximum amount of application is actually capable of consuming. type: object required: - target type: object type: array type: object type: object required: - spec type: object served: true storage: true subresources: {} - deprecated: true deprecationWarning: autoscaling.k8s.io/v1beta2 API is deprecated name: v1beta2 schema: openAPIV3Schema: description: VerticalPodAutoscaler is the configuration for a vertical pod autoscaler, which automatically manages pod resources based on historical and real time resource utilization. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the behavior of the autoscaler. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: resourcePolicy: description: Controls how the autoscaler computes recommended resources. The resource policy may be used to set constraints on the recommendations for individual containers. If not specified, the autoscaler computes recommended resources for all containers in the pod, without additional constraints. properties: containerPolicies: description: Per-container resource policies. items: description: ContainerResourcePolicy controls how autoscaler computes the recommended resources for a specific container. properties: containerName: description: Name of the container or DefaultContainerResourcePolicy, in which case the policy is used by the containers that don't have their own policy specified. type: string maxAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the maximum amount of resources that will be recommended for the container. The default is no maximum. type: object minAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the minimal amount of resources that will be recommended for the container. The default is no minimum. type: object mode: description: Whether autoscaler is enabled for the container. The default is "Auto". enum: - Auto - "Off" type: string type: object type: array type: object targetRef: description: TargetRef points to the controller managing the set of pods for the autoscaler to control - e.g. Deployment, StatefulSet. VerticalPodAutoscaler can be targeted at controller implementing scale subresource (the pod set is retrieved from the controller's ScaleStatus) or some well known controllers (e.g. for DaemonSet the pod set is read from the controller's spec). If VerticalPodAutoscaler cannot use specified target it will report ConfigUnsupported condition. Note that VerticalPodAutoscaler does not require full implementation of scale subresource - it will not use it to modify the replica count. The only thing retrieved is a label selector matching pods grouped by the target resource. properties: apiVersion: description: API version of the referent type: string kind: description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' type: string name: description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' type: string required: - kind - name type: object x-kubernetes-map-type: atomic updatePolicy: description: Describes the rules on how changes are applied to the pods. If not specified, all fields in the `PodUpdatePolicy` are set to their default values. properties: updateMode: description: Controls when autoscaler applies changes to the pod resources. The default is 'Auto'. enum: - "Off" - Initial - Recreate - Auto type: string type: object required: - targetRef type: object status: description: Current information about the autoscaler. properties: conditions: description: Conditions is the set of conditions required for this autoscaler to scale its target, and indicates whether or not those conditions are met. items: description: VerticalPodAutoscalerCondition describes the state of a VerticalPodAutoscaler at a certain point. properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another format: date-time type: string message: description: message is a human-readable explanation containing details about the transition type: string reason: description: reason is the reason for the condition's last transition. type: string status: description: status is the status of the condition (True, False, Unknown) type: string type: description: type describes the current condition type: string required: - status - type type: object type: array recommendation: description: The most recently computed amount of resources recommended by the autoscaler for the controlled pods. properties: containerRecommendations: description: Resources recommended by the autoscaler for each container. items: description: RecommendedContainerResources is the recommendation of resources computed by autoscaler for a specific container. Respects the container resource policy if present in the spec. In particular the recommendation is not produced for containers with `ContainerScalingMode` set to 'Off'. properties: containerName: description: Name of the container. type: string lowerBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Minimum recommended amount of resources. Observes ContainerResourcePolicy. This amount is not guaranteed to be sufficient for the application to operate in a stable way, however running with less resources is likely to have significant impact on performance/availability. type: object target: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Recommended amount of resources. Observes ContainerResourcePolicy. type: object uncappedTarget: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: The most recent recommended resources target computed by the autoscaler for the controlled pods, based only on actual resource usage, not taking into account the ContainerResourcePolicy. May differ from the Recommendation if the actual resource usage causes the target to violate the ContainerResourcePolicy (lower than MinAllowed or higher that MaxAllowed). Used only as status indication, will not affect actual resource assignment. type: object upperBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Maximum recommended amount of resources. Observes ContainerResourcePolicy. Any resources allocated beyond this value are likely wasted. This value may be larger than the maximum amount of application is actually capable of consuming. type: object required: - target type: object type: array type: object type: object required: - spec type: object served: true storage: false
集群版本<1.22
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: verticalpodautoscalers.autoscaling.k8s.io annotations: "api-approved.kubernetes.io": "https://github.com/kubernetes/kubernetes/pull/63797" spec: group: autoscaling.k8s.io scope: Namespaced names: plural: verticalpodautoscalers singular: verticalpodautoscaler kind: VerticalPodAutoscaler shortNames: - vpa version: v1beta1 versions: - name: v1beta1 served: false storage: false - name: v1beta2 served: true storage: true - name: v1 served: true storage: false validation: # openAPIV3Schema is the schema for validating custom objects. openAPIV3Schema: type: object properties: spec: type: object required: [] properties: targetRef: type: object updatePolicy: type: object properties: updateMode: type: string resourcePolicy: type: object properties: containerPolicies: type: array items: type: object properties: containerName: type: string controlledValues: type: string enum: ["RequestsAndLimits", "RequestsOnly"] mode: type: string enum: ["Auto", "Off"] minAllowed: type: object maxAllowed: type: object controlledResources: type: array items: type: string enum: ["cpu", "memory"] --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: verticalpodautoscalercheckpoints.autoscaling.k8s.io annotations: "api-approved.kubernetes.io": "https://github.com/kubernetes/kubernetes/pull/63797" spec: group: autoscaling.k8s.io scope: Namespaced names: plural: verticalpodautoscalercheckpoints singular: verticalpodautoscalercheckpoint kind: VerticalPodAutoscalerCheckpoint shortNames: - vpacheckpoint version: v1beta1 versions: - name: v1beta1 served: false storage: false - name: v1beta2 served: true storage: true - name: v1 served: true storage: false
kubectl apply -f crd.yaml
安裝ack-vertical-pod-autoscaler組件的Admission Controller、Recommender和Updater組件。
1.22≤ 集群版本<1.26
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-admission-controller namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-admission-controller template: metadata: labels: app: vpa-admission-controller spec: serviceAccountName: vpa-admission-controller securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: admission-controller image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-admission-controller:0.13.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: tls-certs mountPath: "/etc/tls-certs" readOnly: true resources: limits: cpu: 200m memory: 500Mi requests: cpu: 50m memory: 200Mi ports: - containerPort: 8000 - name: prometheus containerPort: 8944 volumes: - name: tls-certs secret: secretName: vpa-tls-certs --- apiVersion: v1 kind: Service metadata: name: vpa-webhook namespace: kube-system spec: ports: - port: 443 targetPort: 8000 selector: app: vpa-admission-controller
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-recommender namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-recommender template: metadata: labels: app: vpa-recommender spec: serviceAccountName: vpa-recommender securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: recommender image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-recommender:0.13.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - name: prometheus containerPort: 8942
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-updater namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-updater template: metadata: labels: app: vpa-updater spec: serviceAccountName: vpa-updater securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: updater image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-updater:0.13.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - name: prometheus containerPort: 8943
集群版本<1.22
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-admission-controller namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-admission-controller template: metadata: labels: app: vpa-admission-controller spec: serviceAccountName: admin containers: - name: admission-controller image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-admission-controller:0.7.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: tls-certs mountPath: "/etc/tls-certs" readOnly: true resources: limits: cpu: 200m memory: 500Mi requests: cpu: 50m memory: 200Mi ports: - containerPort: 8000 volumes: - name: tls-certs secret: secretName: vpa-tls-certs --- apiVersion: v1 kind: Service metadata: name: vpa-webhook namespace: kube-system spec: ports: - port: 443 targetPort: 8000 selector: app: vpa-admission-controller
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-recommender namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-recommender template: metadata: labels: app: vpa-recommender spec: serviceAccountName: admin containers: - name: recommender image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-recommender:0.7.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - containerPort: 8080
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-updater namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-updater template: metadata: labels: app: vpa-updater spec: serviceAccountName: admin containers: - name: updater image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-updater:0.7.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - containerPort: 8080
步驟二:驗證安裝VPA
保存以下YAML文件,執行命令創建名為nginx-deployment-basic的Deployment。
說明將Deployment中的資源
requests
和limits
留空。apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment-basic labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
kubectl apply -f nginx-deployment-basic.yaml
保存以下YAML文件,執行命令創建名為nginx-deployment-basic-vpa的VPA資源。
說明您可以按需設置
updateMode
為Off
或者Auto
。Off
(推薦):根據集群資源消耗推薦Pod的資源Request和Limit配置,但不會自動更新Pod資源配置。Auto
:根據集群資源消耗推薦Pod的資源Request和Limit配置,并自動更新Pod資源配置。
apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: nginx-deployment-basic-vpa spec: targetRef: apiVersion: "apps/v1" kind: Deployment name: nginx-deployment-basic updatePolicy: updateMode: "Off" # 設置為off。
kubectl apply -f nginx-deployment-basic-vpa.yaml
執行以下命令,查詢VPA為Deployment推薦的CPU和內存資源的
requests
值。kubectl describe vpa nginx-deployment-basic-vpa
預期輸出需要等待兩分鐘左右。正常情況下,預期輸出如下,展示了VPA為Deployment推薦的值。
Recommendation: Container Recommendations: Container Name: nginx Lower Bound: Cpu: 25m Memory: 262144k Target: Cpu: 25m Memory: 262144k Uncapped Target: Cpu: 25m Memory: 262144k Upper Bound: Cpu: 11601m Memory: 12128573170
您可以根據VPA的推薦值來配置Deployment中實際資源的
requests
。VPA會持續監控應用資源的使用情況,并提供優化建議。
如何通過控制臺管理使用kubectl安裝的VPA?
對于v1.26及以上的集群,推薦您使用控制臺進行ack-vertical-pod-autoscaler組件的安裝和VPA的管理。為避免新舊VPA沖突,請卸載舊的VPA(通過kubectl安裝的VPA),并通過控制臺重新安裝。
步驟1:卸載集群中已經部署的VPA
卸載集群中已經部署的VPA,包括清理Deployment、RBAC、Secret、CRD、Service等資源,以避免新安裝的VPA與舊版VPA沖突而導致VPA不可用。
在舊版VPA的YAML文件中(本小節以下方示例nginx-deployment-basic-vpa.yaml為例),刪除
metadata
中的信息,僅保留name
和namespace
字段即可,同時刪除status
字段信息。然后,保存此YAML文件供后續步驟使用。apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"autoscaling.k8s.io/v1","kind":"VerticalPodAutoscaler","metadata":{"annotations":{},"name":"nginx-deployment-basic-vpa","namespace":"default"},"spec":{"targetRef":{"apiVersion":"apps/v1","kind":"Deployment","name":"nginx-deployment-basic"},"updatePolicy":{"updateMode":"Off"}}} creationTimestamp: "2024-02-29T06:03:35Z" generation: 1 name: nginx-deployment-basic-vpa namespace: default resourceVersion: "56264" uid: 9f128737-d12e-46f6-b254-c1a7505c19c6 spec: targetRef: apiVersion: apps/v1 kind: Deployment name: nginx-deployment-basic updatePolicy: updateMode: "Off" status: conditions: - lastTransitionTime: "2024-02-29T06:03:55Z" status: "True" type: RecommendationProvided recommendation: containerRecommendations: - containerName: nginx lowerBound: cpu: 25m memory: 262144k target: cpu: 25m memory: 262144k uncappedTarget: cpu: 25m memory: 262144k upperBound: cpu: 25m memory: 262144k
kubectl get vpa nginx-deployment-basic-vpa -oyaml
執行以下命令,刪除VPA組件安裝的資源。
// 刪除Deployment和Service kubectl delete deployment vpa-admission-controller vpa-recommender vpa-updater -n kube-system kubectl delete svc vpa-webhook -n kube-system // 刪除ClusterRole kubectl delete clusterrole system:metrics-reader system:vpa-actor system:vpa-status-actor system:vpa-checkpoint-actor system:evictioner system:vpa-target-reader system:vpa-admission-controller system:vpa-status-reader // 刪除ClusterRoleBinding kubectl delete clusterrolebinding system:metrics-reader system:vpa-actor system:vpa-status-actor system:vpa-checkpoint-actor system:vpa-target-reader-binding system:vpa-evictioner-binding system:vpa-admission-controller system:vpa-status-reader-binding // 刪除ServiceAccount kubectl delete sa vpa-admission-controller vpa-recommender vpa-updater -n kube-system // 刪除Secret kubectl delete secret vpa-tls-certs -n kube-system //刪除CRD kubectl delete crd verticalpodautoscalercheckpoints.autoscaling.k8s.io verticalpodautoscalers.autoscaling.k8s.io
步驟2:安裝ack-vertical-pod-autoscaler組件
登錄容器服務管理控制臺,在左側導航欄選擇集群。
在集群列表頁面,單擊目標集群名稱,然后在左側導航欄,選擇 。
在組件管理頁面,定位ack-vertical-pod-autoscaler組件,然后按照頁面提示完成安裝。
步驟3:重新部署VPA YAML文件
執行以下命令,重新部署步驟1:卸載集群中已經部署的VPA保存的VPA YAML文件,本步驟以nginx-deployment-basic-vpa.yaml為例。
kubectl apply -f nginx-deployment-basic-vpa.yaml