通義千問Audio
說明
支持的領域 / 任務:aigc
通義千問Audio是阿里云研發的大規模音頻語言模型。通義千問Audio可以以多種音頻 (包括說話人語音、自然音、音樂、歌聲)和文本作為輸入,并以文本作為輸出。通義千問Audio模型的特點包括:
1、全類型音頻感知:通義千問Audio是一個性能卓越的通用音頻理解模型,支持30秒內的自然音、人聲、音樂等類型音頻理解,如多語種語音識別,時間抽定位,說話人情緒、性別識別,環境識別,音樂的樂器、風格、情感識別等。
2、基于音頻推理:通義千問Audio支持基于音頻內容進行相關推理和創作,如語義理解,場景推理,相關推薦,內容創作等。
3、支持多輪音頻和文本對話:通義千問Audio支持多音頻分析、多輪音頻-文本交錯對話。
快速開始
前提條件
已開通服務并獲得API-KEY:API-KEY的獲取與配置。
已安裝最新版SDK:安裝DashScope SDK。
示例代碼
以下示例展示了調用通義千問Audio API對一個用戶指令進行響應的代碼。
說明
需要使用您的API-KEY替換示例中的 YOUR_DASHSCOPE_API_KEY,代碼才能正常運行。
設置API-KEY
export DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEY
from http import HTTPStatus
import dashscope
def simple_multimodal_conversation_call():
"""Simple single round multimodal conversation call.
"""
messages = [
{
"role": "user",
"content": [
{"audio": "https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3"},
{"text": "這段音頻在說什么?"}
]
}
]
response = dashscope.MultiModalConversation.call(model='qwen-audio-turbo',
messages=messages)
# The response status_code is HTTPStatus.OK indicate success,
# otherwise indicate request is failed, you can get error code
# and message from code and message.
if response.status_code == HTTPStatus.OK:
print(response)
else:
print(response.code) # The error code.
print(response.message) # The error message.
if __name__ == '__main__':
simple_multimodal_conversation_call()
// Copyright (c) Alibaba, Inc. and its affiliates.
import java.util.Arrays;
import java.util.Collections;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversation;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationParam;
import com.alibaba.dashscope.aigc.multimodalconversation.MultiModalConversationResult;
import com.alibaba.dashscope.common.MultiModalMessage;
import com.alibaba.dashscope.common.Role;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.exception.UploadFileException;
public class Main {
public static void simpleMultiModalConversationCall()
throws ApiException, NoApiKeyException, UploadFileException {
MultiModalConversation conv = new MultiModalConversation();
MultiModalMessage userMessage = MultiModalMessage.builder().role(Role.USER.getValue())
.content(Arrays.asList(Collections.singletonMap("audio", "https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3"),
Collections.singletonMap("text", "這段音頻在說什么?"))).build();
MultiModalConversationParam param = MultiModalConversationParam.builder()
.model("qwen-audio-turbo")
.message(userMessage)
.build();
MultiModalConversationResult result = conv.call(param);
System.out.println(result);
}
public static void main(String[] args) {
try {
simpleMultiModalConversationCall();
} catch (ApiException | NoApiKeyException | UploadFileException e) {
System.out.println(e.getMessage());
}
System.exit(0);
}
}
python 調用成功后,將會返回如下示例結果。
{
"status_code": 200,
"request_id": "cd828016-bcf5-94c7-82ed-5b45bf06886c",
"code": "",
"message": "",
"output": {
"choices": [
{
"finish_reason": null,
"message": {
"role": "assistant",
"content": [
{
"text": "這段音頻在說中文:\"歡迎使用阿里云\"。"
}
]
}
}
]
},
"usage": {
"input_tokens": 1276,
"audio_tokens": 768,
"output_tokens": 19
}
}
了解更多
有關通義千問Audio API的詳細調用文檔可前往API詳情頁面進行了解。
文檔內容是否對您有幫助?