在CI Pipeline中Clone私有Git倉庫
ACK One工作流集群構建CI Pipeline,其使用BuildKit Cache和NAS存儲Go mode cache,可大幅加速CI Pipeline的流程。通過工作流集群構建基于Golang項目CI Pipeline時,若您使用的Git倉庫為私有倉庫,您需要在CI流程中先成功Clone該私有倉庫,再進行CI Pipeline的構建操作。本文為您介紹如何在CI Pipeline中Clone私有Git倉庫。
背景信息
使用公共Git倉庫構建CI Pipeline的最佳實踐,請參見基于工作流集群構建Golang項目的CI Pipeline。
若您使用私有Git倉庫,則需要在上述最佳實踐操作前,先Clone私有Git倉庫。
本文為您提供以下三種方法Clone私有Git倉庫:
在工作流集群中保存私有倉庫憑據(jù)
Clone私有倉庫前,您需要先在工作流集群中執(zhí)行如下命令保存私有倉庫所需的用戶名、密碼和ssh private key
。
username
、password
和ssh-private-key
需要替換為您實際使用的參數(shù)值。
kubectl create secret generic git-creds --from-literal="username=${username}" --from-literal="password=${password or token}" --from-file=ssh-private-key=${ssh private key path}
# example
# kubectl create secret generic git-creds --from-literal="username=demo" --from-literal="password=ghp_GePB****************d407" --from-file=ssh-private-key=$HOME/.ssh/id_rsa
方法一:基于Argo Workflows Git Artifact與用戶名密碼
該方法主要是在執(zhí)行構建CI Pipeline的操作前,先執(zhí)行Git Clone私有倉庫操作,再進行Git Checkout操作。
以下YAML為了方便展示,和上文預置工作流模板內(nèi)容相比,僅保留了上述CI Pipeline中的git-checkout-pr
任務(其他方法均相同),基于此增加git-clone
任務,并設置git-checkout-pr
依賴git-clone
。
git-checkout-pr
的command
中,shell script無需修改。git-clone
的artifacts
中引用保存的私有倉庫憑據(jù)的git-creds
secret中的用戶名、密碼。
示例模板
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
name: ci-git-artifact
spec:
entrypoint: main
volumes:
- name: run-test
emptyDir: {}
- name: workdir
persistentVolumeClaim:
claimName: pvc-nas
- name: docker-config
secret:
secretName: docker-config
arguments:
parameters:
- name: repo_url
value: ""
- name: repo_name
value: ""
- name: target_branch
value: "main"
templates:
- name: main
dag:
tasks:
- name: git-clone
arguments:
artifacts:
- name: git-repo
path: /workdir
git:
repo: "{{arguments.parameters.repo_url}}"
revision: main
usernameSecret:
name: git-creds
key: username
passwordSecret:
name: git-creds
key: password
sshPrivateKeySecret:
name: git-creds
key: ssh-private-key
inline:
container:
image: golang:1.10
command:
- sh
- -c
- |
cd {{workflow.parameters.repo_name}}
git status && ls
workingDir: /workdir
volumeMounts:
- name: "workdir"
mountPath: /workdir
- name: git-checkout-pr
inline:
container:
image: alpine:latest
command:
- sh
- -c
- |
set -eu
apk --update add git
cd /workdir
echo "Start to Clone "{{workflow.parameters.repo_url}}
git -C "{{workflow.parameters.repo_name}}" pull || git clone {{workflow.parameters.repo_url}}
cd {{workflow.parameters.repo_name}}
echo "Start to Checkout target branch" {{workflow.parameters.target_branch}}
git checkout {{workflow.parameters.target_branch}}
echo "Get commit id"
git rev-parse --short origin/{{workflow.parameters.target_branch}} > /workdir/{{workflow.parameters.repo_name}}-commitid.txt
commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
echo "Commit id is got: "$commitId
echo "Git Clone and Checkout Complete."
volumeMounts:
- name: "workdir"
mountPath: /workdir
resources:
requests:
memory: 1Gi
cpu: 1
activeDeadlineSeconds: 1200
depends: git-clone
提交Workflow參數(shù)說明
涉及參數(shù)和CI Pipeline保持一致,如下圖所示:
方法二:基于Argo Workflows Git Artifact與SSH Private Key
和方法一基本相同,主要差異如下:
git-clone
的artifacts
中引用保存的私有倉庫憑據(jù)的git-creds
secret中的ssh private key
。
在提交Workflow時,
repo_url
需要為ssh格式,例如:git@github.com:ivan-cai/gitops-demo-private.git
。
示例模板
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
name: ci-git-artifact-sshkey
spec:
entrypoint: main
volumes:
- name: run-test
emptyDir: {}
- name: workdir
persistentVolumeClaim:
claimName: pvc-nas
- name: docker-config
secret:
secretName: docker-config
arguments:
parameters:
- name: repo_url
value: ""
- name: repo_name
value: ""
- name: target_branch
value: "main"
templates:
- name: main
dag:
tasks:
- name: git-clone
arguments:
artifacts:
- name: git-repo
path: /workdir
git:
repo: "{{arguments.parameters.repo_url}}"
revision: main
sshPrivateKeySecret:
name: git-creds
key: ssh-private-key
inline:
container:
image: golang:1.10
command:
- sh
- -c
- |
cd {{workflow.parameters.repo_name}}
git status && ls
workingDir: /workdir
volumeMounts:
- name: "workdir"
mountPath: /workdir
- name: git-checkout-pr
inline:
container:
image: alpine:latest
command:
- sh
- -c
- |
set -eu
apk --update add git
cd /workdir
echo "Start to Clone "{{workflow.parameters.repo_url}}
git -C "{{workflow.parameters.repo_name}}" pull || git clone {{workflow.parameters.repo_url}}
cd {{workflow.parameters.repo_name}}
echo "Start to Checkout target branch" {{workflow.parameters.target_branch}}
git checkout {{workflow.parameters.target_branch}}
echo "Get commit id"
git rev-parse --short origin/{{workflow.parameters.target_branch}} > /workdir/{{workflow.parameters.repo_name}}-commitid.txt
commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
echo "Commit id is got: "$commitId
echo "Git Clone and Checkout Complete."
volumeMounts:
- name: "workdir"
mountPath: /workdir
resources:
requests:
memory: 1Gi
cpu: 1
activeDeadlineSeconds: 1200
depends: git-clone
提交Workflow參數(shù)說明
涉及參數(shù)如下:
repo_url
需要為ssh格式。
方法三:基于Git Clone命令與用戶名密碼
和前兩種方法不同,該方法不需要增加DAG(Directed Acyclic Graph)任務,但需要修改git-checkout-pr
中git clone的命令,并通過env引用git-creds
secret中的用戶名、密碼。命令如下:
git clone https://${GIT_USER}:${GIT_TOKEN}@github.com/${GITHUB_REPOSITORY}
示例模板
apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
name: ci-git
spec:
entrypoint: main
volumes:
- name: run-test
emptyDir: {}
- name: workdir
persistentVolumeClaim:
claimName: pvc-nas
- name: docker-config
secret:
secretName: docker-config
arguments:
parameters:
- name: repo_url
value: ""
- name: repo_name
value: ""
- name: target_branch
value: "main"
templates:
- name: main
dag:
tasks:
- name: git-checkout-pr
inline:
container:
image: alpine:latest
env:
- name: GIT_USER
valueFrom:
secretKeyRef:
name: git-creds
key: username
- name: GIT_TOKEN
valueFrom:
secretKeyRef:
name: git-creds
key: password
command:
- sh
- -c
- |
set -eu
apk --update add git
cd /workdir
echo "Start to Clone "{{workflow.parameters.repo_url}}
git -C "{{workflow.parameters.repo_name}}" pull || git clone https://$GIT_USER:$GIT_TOKEN@{{workflow.parameters.repo_url}}
cd {{workflow.parameters.repo_name}}
echo "Start to Checkout target branch" {{workflow.parameters.target_branch}}
git checkout {{workflow.parameters.target_branch}}
echo "Get commit id"
git rev-parse --short origin/{{workflow.parameters.target_branch}} > /workdir/{{workflow.parameters.repo_name}}-commitid.txt
commitId=$(cat /workdir/{{workflow.parameters.repo_name}}-commitid.txt)
echo "Commit id is got: "$commitId
echo "Git Clone and Checkout Complete."
volumeMounts:
- name: "workdir"
mountPath: /workdir
resources:
requests:
memory: 1Gi
cpu: 1
activeDeadlineSeconds: 1200
提交Workflow參數(shù)說明
涉及參數(shù)如下:
repo_url
參數(shù)不能包含https://
前綴。
相關文檔
使用公共Git倉庫構建CI Pipeline的最佳實踐,請參見基于工作流集群構建Golang項目的CI Pipeline。