具有自定义语音的 Azure 呼叫自动化

Question

我使用 Azure 呼叫自动化创建了一个语音机器人。

def get_chat_completions_async(system_prompt,user_prompt): 
    openai.api_key = AZURE_OPENAI_SERVICE_KEY
    openai.api_base = AZURE_OPENAI_SERVICE_ENDPOINT # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
    openai.api_type = 'azure'
    openai.api_version = '2023-05-15' # this may change in the future
    
    # Define your chat completions request
    chat_request = [
        {"role": "system", "content": f"{system_prompt}"},
        {"role": "user", "content": f"In less than 200 characters: respond to this question: {user_prompt}?"}
    ]
  
    global response_content
    try:
        response = ChatCompletion.create(model=AZURE_OPENAI_DEPLOYMENT_MODEL,deployment_id=AZURE_OPENAI_DEPLOYMENT_MODEL_NAME, messages=chat_request,max_tokens = 1000)
    except ex:
        app.logger.info("error in openai api call : %s",ex)
       
    # Extract the response content
    if response is not None :
         response_content  =  response['choices'][0]['message']['content']
    else :
         response_content=""    
    return response_content  

我寻求整合我已经训练和部署的自定义语音。根据 Microsoft docs，您可以简单地在呼叫自动化应用程序中使用 tts 的自定义语音模型的端点。然而，这似乎不起作用。

有人以前有使用 Azure 将自定义语音集成到呼叫自动化应用程序中的经验吗？

我可以使用端点进行语音合成，但使用端点进行呼叫自动化不起作用 - 因为没有生成发生 - 它不会给出错误。

Answer 1

要将自定义语音集成到 Azure 呼叫自动化应用程序中，您需要确保在呼叫自动化流程中正确使用自定义语音端点。 对于呼叫自动化，我们需要 Azure 通信服务中的

callConnectionId

我按照这个MSDOC获取了callConnectionId Azure 通信 服务。

POST https://contoso.communications.azure.com/calling/callConnections?api-version=2023-10-15
Authorization: Bearer {access_token}

{
  "callbackUri": "https://app.contoso.com/callback",
  "targets": [
    {
      "kind": "phoneNumber",
      "phoneNumber": {
        "value": "+919"
      }
    }
  ],
  "sourceCallerIdNumber": {
    "value": "+183"
  },
  "sourceDisplayName": "Contoso Support",
  "callIntelligenceOptions": {
    "cognitiveServicesEndpoint": "https://<your-cognitive-service-endpoint>.cognitiveservices.azure.com/"
  }
}

以下 Python 代码用于使用 Azure 通信服务呼叫自动化 SDK 通过播放操作自定义语音提示。

import os
import sys
from azure.communication.callautomation import (
    CallAutomationClient,
    CommunicationUserIdentifier
)

sys.path.append("..")

class CallAutomationCreateCallSample(object):

    connection_string = os.getenv("COMMUNICATION_CONNECTION_STRING")

    def create_call_to_single(self):
        callautomation_client = CallAutomationClient.from_connection_string(self.connection_string)

        # Creating a call
        user = CommunicationUserIdentifier("8:acs:123")
        callback_uri = "https://contoso.com/event"

        call_connection_properties = callautomation_client.create_call(
            target_participant=user,
            callback_url=callback_uri
        )

        # callconnection id of the call
        print(call_connection_properties.call_connection_id)

if __name__ == '__main__':
    sample = CallAutomationCreateCallSample()
    sample.create_call_to_single()

我参考了适用于 Python 的 Azure 通信呼叫自动化客户端库的 MSDOC

  # callconnection id of the call
        call_connection_id = call_connection_properties.call_connection_id
        print(f"Call connection ID: {call_connection_id}")

  # Play the audio to the target participant
        call_connection.play_media(play_source=play_source, play_to=[target_participant])

if __name__ == '__main__':
    sample = CallAutomationCreateCallSample()
    
   
    call_connection_id, user = sample.create_call_to_single()

   
    audio_uri = "https://some.file.azure.com/sample.wav"
    play_source_cache_id = "string"  # Optional cache ID

    
    sample.play_audio_with_cache(call_connection_id, user, audio_uri, play_source_cache_id)