我們經常遇到從某臺線上的機器把日志文件拷貝出來或者是進程卡了,通過jstack輸出到某個文件再進行分析。我們使用osscmd把數據導到OSS里,但是手動配置過程實在是過于繁瑣。通過如下模板您可很容易地處理此需求。
解決方案
從實例中拷貝文件可以利用云助手的RunCommand去到機器上去調用ossutil做數據拷貝,關鍵問題是AK怎么傳,當然我們可以把AK當參數以命令的方式傳入,但是這種太不安全了,相當于AK明文傳輸。那么有沒有別的辦法解決AK明文傳輸的問題呢?此時就想到了InstanceRole的功能,可以通過給instance attach一個ram role,那么在實例內部curl某個固定地址就能拿到以這個role身份產生的sts-token。詳細參見概述。
curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}
以上AK的問題解決了,另外的問題ossutil在實例中訪問不了公網的情況怎么下載呢?咨詢了OSS的支持同事,他們并不提供內網下載的地址。只能自己搭建了,我們自己創建了個bucket為oos-public的bucket提供全局只讀的權限可以方便內網下載。解析JSON我們需要個方便的工具jq,我們同時把這個文件傳到公共bucket上經過調試最終的腳本如下。
test -e ossutil64 || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/ossutil64 && chmod 755 ossutil64 test -e jq || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/jq && chmod 755 jq stsToken=`curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}` accessKeyId=`echo $stsToken |./jq .AccessKeyId | awk -F'\"' '{print $2}'` accessKeySecret=`echo $stsToken | ./jq .AccessKeySecret | awk -F'\"' '{print $2}' securityToken=`echo $stsToken | ./jq .SecurityToken | awk -F'\"' '{print $2}' endpoint=https://oss-{{ACS::RegionId}}.aliyuncs.com; ./ossutil64 -i $accessKeyId -k $accessKeySecret -t $securityToken -e $endpoint cp {{SrcUrl}} {{DestUrl}}
再轉換成OOS模板。
FormatVersion: OOS-2019-06-01 Description: Copy ECS instance file to OSS Parameters: InstanceId: Type: String Description: The Ecs instance SrcUrl: Type: String Description: Src url DestUrl: Type: String Description: Dest url InstanceAssumeRole: Type: String Description: '' OOSAssumeRole: Type: String Description: oos assume this role to execution task. Default: OOSServiceRole RamRole: '{{OOSAssumeRole}}' Tasks: - Name: checkInstanceReady Action: ACS::CheckFor Description: describe instances with specified parameters. Properties: API: DescribeInstances Service: ECS PropertySelector: Instances.Instance[].Status DesiredValues: - Running Parameters: InstanceIds: - '{{ InstanceId }}' - Name: runCommand Action: ACS::ECS::RunCommand Description: '' Properties: commandContent: Fn::Join: - |+ - - test -e oos || mkdir oos; - cd oos; - test -e ossutil64 || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/ossutil64 && chmod 755 ossutil64 - test -e jq || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/jq && chmod 755 jq - stsToken=`curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}` - accessKeyId=`echo $stsToken |./jq .AccessKeyId | awk -F'"' '{print $2}'` ; - accessKeySecret=`echo $stsToken | ./jq .AccessKeySecret | awk -F'"' '{print $2}'` ; - securityToken=`echo $stsToken | ./jq .SecurityToken | awk -F'"' '{print $2}'` ; - endpoint=https://oss-{{ACS::RegionId}}-internal.aliyuncs.com; - ./ossutil64 -i $accessKeyId -k $accessKeySecret -t $securityToken -e $endpoint cp {{SrcUrl}} {{DestUrl}} commandType: RunShellScript instanceId: '{{InstanceId}}' Outputs: CommandOutput: Type: String ValueSelector: InvocationResult[].Output
{ "FormatVersion": "OOS-2019-06-01", "Description": "Copy ECS instance file to OSS", "Parameters": { "InstanceId": { "Type": "String", "Description": "The Ecs instance" }, "SrcUrl": { "Type": "String", "Description": "Src url" }, "DestUrl": { "Type": "String", "Description": "Dest url" }, "InstanceAssumeRole": { "Type": "String", "Description": "" }, "OOSAssumeRole": { "Type": "String", "Description": "oos assume this role to execution task.", "Default": "OOSServiceRole" } }, "RamRole": "{{OOSAssumeRole}}", "Tasks": [ { "Name": "checkInstanceReady", "Action": "ACS::CheckFor", "Description": "describe instances with specified parameters.", "Properties": { "API": "DescribeInstances", "Service": "ECS", "PropertySelector": "Instances.Instance[].Status", "DesiredValues": [ "Running" ], "Parameters": { "InstanceIds": [ "{{ InstanceId }}" ] } } }, { "Name": "runCommand", "Action": "ACS::ECS::RunCommand", "Description": "", "Properties": { "commandContent": { "Fn::Join": [ "\n", [ "test -e oos || mkdir oos;", "cd oos;", "test -e ossutil64 || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/ossutil64 && chmod 755 ossutil64", "test -e jq || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/jq && chmod 755 jq", "stsToken=`curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}`", "accessKeyId=`echo $stsToken |./jq .AccessKeyId | awk -F'\"' '{print $2}'` ;", "accessKeySecret=`echo $stsToken | ./jq .AccessKeySecret | awk -F'\"' '{print $2}'` ;", "securityToken=`echo $stsToken | ./jq .SecurityToken | awk -F'\"' '{print $2}'` ;", "endpoint=https://oss-{{ACS::RegionId}}-internal.aliyuncs.com;", "./ossutil64 -i $accessKeyId -k $accessKeySecret -t $securityToken -e $endpoint cp {{SrcUrl}} {{DestUrl}}" ] ] }, "commandType": "RunShellScript", "instanceId": "{{InstanceId}}" }, "Outputs": { "CommandOutput": { "Type": "String", "ValueSelector": "InvocationResult[].Output" } } } ] }
傳入參數實例ID,拷貝的文件,拷貝路徑,傳好之后執行。
執行成功后的結果。
根據Execution的執行日志可以看到執行過程base64 decode出來后可以看到Shell執行的具體信息,從結果上看是成功了,再到OSS上看,文件在不在。
總結
以上我們介紹了如果使用系統運維管理方便的從實例中拷貝文件到OSS上,結合拷貝文件的場景與執行命令的組合,我們可以方便的做出到某臺機器jstack輸出文件并拷貝出來等相應場景。