服務網格 ASM(Service Mesh)支持與Karmada(Kubernetes Armada)配合,使您能夠在多個Kubernetes集群中部署和運行云原生應用程序,而無需更改應用程序。Karmada提供了為云原生場景下的多集群應用程序管理提供即插即用的自動化,具有集中式多云管理、高可用性、故障恢復和流量調度等關鍵功能。本文介紹如何使用ASM與Karmada實現多集群應用管理。
背景信息
Karmada使用Kubernetes原生API定義聯邦資源模板,以便輕松與現有Kubernetes采用的工具進行集成。同時,Karmada也提供了一個獨立的Propagation (placement) Policy API 來定義多集群的調度要求。
支持1:N的策略映射機制。您無需每次創建聯邦應用時都標明調度約束。
在使用默認策略的情況下,您可以直接與Kubernetes API交互。
集群模式
Karmada支持通過Push
和Pull
兩種模式來管理成員集群。Push
和Pull
模式的主要區別在于部署資源清單時,訪問成員集群的方式。
Push模式
Karmada控制平面將直接訪問成員集群的kube-apiserver
,以獲取集群狀態并部署資源清單。
Pull模式?
Karmada 控制平面不會直接訪問成員集群,而是將其請求下放給名為karmada-agent
的額外組件處理。
每個karmada-agent
服務于一個集群,并承擔以下職責:
將集群注冊到 Karmada 中(創建
Cluster
對象)維護集群狀態并向 Karmada 報告(更新
Cluster
對象的狀態)監聽來自 Karmada 執行空間(命名空間,格式為
karmada-es-<cluster name>
)的資源清單,并將監聽到的資源部署到其所服務的集群中。
關鍵組件
Karmada的控制面包括以下組件:
Karmada API Server。
Karmada Controller Manager。
Karmada Scheduler。
ETCD存儲了Karmada API對象,API Server是所有其他組件通訊的REST端點,Karmada Controller Manager根據您通過API服務器創建的API對象執行操作。
前提條件
在同一VPC下,已創建兩個ACK集群(本示例為member1和member2)。具體操作,請參見創建ACK托管集群。
說明本示例以同一個VPC的兩個集群作為示例場景。如果使用不同的VPC,則需要額外配置以使跨VPC能夠在物理網絡上連通。
在創建集群時,建議使用企業安全組。
已創建ASM實例(本示例為mesh1),且已在default命名空間上配置Sidecar注入策略。
已部署一個Karmada主集群實例(本示例為karmada-master),并將上述兩個ACK集群作為成員集群(本示例為member1和member2)添加到Karmada主集群中。具體操作,請參見Karmada Installation。
已添加兩個集群到ASM實例(mesh1),并創建Serverless入口網關。具體操作,請參見添加集群到ASM實例并創建Serverless入口網關。
步驟一:使用Karmada在多集群中部署應用
與使用ASM Serverless網關實現多集群入口中相同,本文同樣使用Bookinfo示例。不同于手動將review-3和review-1、review-2分別部署到兩個集群中的方式,本節演示基于karmada的Propagation Policy,只需在karmada-master集群中創建傳播策略,即可實現同樣的效果。
使用以下內容創建bookinfo-karmada.yaml。
# Details service apiVersion: v1 kind: Service metadata: name: details labels: app: details service: details spec: ports: - port: 9080 name: http selector: app: details --- apiVersion: v1 kind: ServiceAccount metadata: name: bookinfo-details labels: account: details --- apiVersion: apps/v1 kind: Deployment metadata: name: details-v1 labels: app: details version: v1 spec: replicas: 1 selector: matchLabels: app: details version: v1 template: metadata: labels: app: details version: v1 spec: serviceAccountName: bookinfo-details containers: - name: details image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-details-v1:1.19.1 imagePullPolicy: IfNotPresent ports: - containerPort: 9080 --- # Ratings service apiVersion: v1 kind: Service metadata: name: ratings labels: app: ratings service: ratings spec: ports: - port: 9080 name: http selector: app: ratings --- apiVersion: v1 kind: ServiceAccount metadata: name: bookinfo-ratings labels: account: ratings --- apiVersion: apps/v1 kind: Deployment metadata: name: ratings-v1 labels: app: ratings version: v1 spec: replicas: 1 selector: matchLabels: app: ratings version: v1 template: metadata: labels: app: ratings version: v1 spec: serviceAccountName: bookinfo-ratings containers: - name: ratings image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-ratings-v1:1.19.1 imagePullPolicy: IfNotPresent ports: - containerPort: 9080 --- # Reviews service apiVersion: v1 kind: Service metadata: name: reviews labels: app: reviews service: reviews spec: ports: - port: 9080 name: http selector: app: reviews --- apiVersion: v1 kind: ServiceAccount metadata: name: bookinfo-reviews labels: account: reviews --- apiVersion: apps/v1 kind: Deployment metadata: name: reviews-v1 labels: app: reviews version: v1 spec: replicas: 1 selector: matchLabels: app: reviews version: v1 template: metadata: labels: app: reviews version: v1 spec: serviceAccountName: bookinfo-reviews containers: - name: reviews image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-reviews-v1:1.19.1 imagePullPolicy: IfNotPresent ports: - containerPort: 9080 --- apiVersion: apps/v1 kind: Deployment metadata: name: reviews-v2 labels: app: reviews version: v2 spec: replicas: 1 selector: matchLabels: app: reviews version: v2 template: metadata: labels: app: reviews version: v2 spec: serviceAccountName: bookinfo-reviews containers: - name: reviews image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-reviews-v2:1.19.1 imagePullPolicy: IfNotPresent ports: - containerPort: 9080 --- apiVersion: apps/v1 kind: Deployment metadata: name: reviews-v3 labels: app: reviews version: v3 spec: replicas: 1 selector: matchLabels: app: reviews version: v3 template: metadata: labels: app: reviews version: v3 spec: serviceAccountName: bookinfo-reviews containers: - name: reviews image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-reviews-v3:1.19.1 imagePullPolicy: IfNotPresent ports: - containerPort: 9080 --- # Productpage services apiVersion: v1 kind: Service metadata: name: productpage labels: app: productpage service: productpage spec: ports: - port: 9080 name: http selector: app: productpage --- apiVersion: v1 kind: ServiceAccount metadata: name: bookinfo-productpage labels: account: productpage --- apiVersion: apps/v1 kind: Deployment metadata: name: productpage-v1 labels: app: productpage version: v1 spec: replicas: 1 selector: matchLabels: app: productpage version: v1 template: metadata: labels: app: productpage version: v1 spec: serviceAccountName: bookinfo-productpage containers: - name: productpage image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-productpage-v1:1.19.1 imagePullPolicy: IfNotPresent ports: - containerPort: 9080 volumeMounts: - name: tmp mountPath: /tmp securityContext: runAsUser: 1000 volumes: - name: tmp emptyDir: {} ---
執行以下命令,在karmada主集群中部署bookinfo應用。
kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f bookinfo-karmada.yaml
說明karmada主集群的kubeconfig文件默認保存位置為
/etc/karmada/karmada-apiserver.config
,如果您在部署主集群時選擇了其他安裝位置,請將--kubeconfig
后的路徑替換為實際的路徑。使用以下內容創建propagation.yaml。
apiVersion: policy.karmada.io/v1alpha1 kind: PropagationPolicy metadata: name: service-propagation spec: resourceSelectors: - apiVersion: v1 kind: Service name: productpage - apiVersion: v1 kind: Service name: details - apiVersion: v1 kind: Service name: reviews - apiVersion: v1 kind: Service name: ratings placement: clusterAffinity: clusterNames: - member1 - member2 --- apiVersion: policy.karmada.io/v1alpha1 kind: PropagationPolicy metadata: name: produtpage-propagation spec: resourceSelectors: - apiVersion: apps/v1 kind: Deployment name: productpage-v1 - apiVersion: v1 kind: ServiceAccount name: bookinfo-productpage placement: clusterAffinity: clusterNames: - member1 --- apiVersion: policy.karmada.io/v1alpha1 kind: PropagationPolicy metadata: name: details-propagation spec: resourceSelectors: - apiVersion: apps/v1 kind: Deployment name: details-v1 - apiVersion: v1 kind: ServiceAccount name: bookinfo-details placement: clusterAffinity: clusterNames: - member2 --- apiVersion: policy.karmada.io/v1alpha1 kind: PropagationPolicy metadata: name: reviews-propagation spec: resourceSelectors: - apiVersion: apps/v1 kind: Deployment name: reviews-v1 - apiVersion: apps/v1 kind: Deployment name: reviews-v2 - apiVersion: apps/v1 kind: Deployment name: reviews-v3 - apiVersion: v1 kind: ServiceAccount name: bookinfo-reviews placement: clusterAffinity: clusterNames: - member1 - member2 --- apiVersion: policy.karmada.io/v1alpha1 kind: PropagationPolicy metadata: name: ratings-propagation spec: resourceSelectors: - apiVersion: apps/v1 kind: Deployment name: ratings-v1 - apiVersion: v1 kind: ServiceAccount name: bookinfo-ratings placement: clusterAffinity: exclude: - member1
PropagationPolicy的
.spec.placement.clusterAffinity
表示對某一特定集群的調度限制,如果沒有這個限制,任何集群都可以作為調度候選者。它有四個字段可以設置。
配置項
說明
LabelSelector
用于通過標簽選擇成員集群的過濾器。如果它非空且不為nil,則只會選擇與此過濾器匹配的集群。
FieldSelector
用于通過字段選擇成員集群。如果它非空且不為nil,僅匹配此篩選器的集群將被選擇。
ClusterNames
您可以設置字段以指定所選集群的
ClusterNames
。ExcludeClusters
您可以設置字段以排除所選集群的
ClusterNames
。本示例只使用了
ClusterNames
和ExcludeClusters
兩個字段,更多關于.spec.placement.clusterAffinity
字段的信息,請參見Resource Propagating。執行以下命令,部署PropagationPolicy。
kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f propagation.yaml
分別使用member1集群和member2集群的kubeconfig查看Deployment。
member1
kubectl --kubeconfig member1 get deployment
預期輸出:
NAME READY UP-TO-DATE AVAILABLE AGE productpage-v1 1/1 1 1 12m reviews-v1 1/1 1 1 12m reviews-v2 1/1 1 1 12m reviews-v3 1/1 1 1 12m
member2
kubectl --kubeconfig member2 get deployment
預期輸出:
NAME READY UP-TO-DATE AVAILABLE AGE details-v1 1/1 1 1 16m ratings-v1 1/1 1 1 16m reviews-v1 1/1 1 1 16m reviews-v2 1/1 1 1 16m reviews-v3 1/1 1 1 16m
步驟二:添加虛擬服務和網關規則
在default命名空間下,使用以下內容創建一個名為bookinfo的虛擬服務。具體操作,請參見管理虛擬服務。
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: bookinfo spec: hosts: - "*" gateways: - bookinfo-gateway http: - match: - uri: exact: /productpage - uri: prefix: /static - uri: exact: /login - uri: exact: /logout - uri: prefix: /api/v1/products route: - destination: host: productpage port: number: 9080
在default命名空間下,使用以下內容創建一個名為bookinfo-gateway的網關規則。具體操作,請參見管理網關規則。
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: bookinfo-gateway spec: selector: istio: ingressgateway # use istio default controller servers: - port: number: 80 name: http protocol: HTTP hosts: - "*"
步驟三:訪問驗證
獲取入口網關的IP地址。具體操作,請參見獲取入口網關地址。
在瀏覽器地址欄,輸入
http://{Serverless入口網關的IP地址}/productpage
,并多次刷新頁面。可以看到頁面出現reviews的3個版本,且比例接近1:1:1。雖然review-v3和其他服務不在同一個集群中,也可以正常顯示。