如何修复发送不完整数据的 Azure AI 流响应？

Question

我正在从 Azure AI Hub 的 Phi3 mini 模型的无服务器部署流式传输聊天完成响应。尽管我遇到了响应中遗漏随机字符或单词的问题。在禁用流式传输的情况下进行测试时，该 API 工作得很好。我已经尝试过来自（https://github.com/thivy/azure-openai-js-stream）的SSEParser，尽管它遇到了同样的问题。这是一个简单脚本的结果，该脚本记录已解析的块并记录解析失败的块。

data: {"id":"cmpl-6efec8012ebb48768a85a7489211369f","object":"
AP PHYSICS 2 (3 Sources for Physics Lesson
Key Topics to Study:
Introduction todata: {"id":"cmpl-6efec8012ebb48768a85a7489211369f","
ources: Descriptions of point, line, and surface sources, field patterns, and direction of fields
Radio Waves: Generation, propagation, and applications of radio waves
data: {"id":"cmpl-6efec8012ebb48768a85a7489211369f","object":"chat.co
 and Wave Propagation: Transmission modes, reflection, refraction, and absorption in various media
Study Tips:
Understanding Source Types:data: {"id":"cmpl-6efec8012ebb48768a85a7489
iarize yourself with diagrams of different source types and their field patterns.
Practice Problems: Work through problems related to field strength, intensity, and coverage areas ofdata: {"id":"cmpl-6efec8012e
 sources.
Visual Aids: Use animations and simulations to visualize wave propagation and interactions with various media.
Resources:
Textbook: Review the physics ofdata: {"id":"cmpl-6efec8012ebb48768a85a7489
 and sources chapter in your AP Physics textbook.
OpenStax: Access "AP Physics 2: Physics for College Students" by OpenStax for free, whichdata: {"id":"cmpl-6efec8012ebb48768a85a7489211369f","obje       
 a range of problems and explanations.
MIT OpenCourseWare: Explore the "AP Physics 2: Physics for College Students" course materials on Mdata: {"id":"cmpl-6efec8012ebb487
's open courseware website, which includes video lectures and problem sets.
Remember to align your study efforts with the particular questions and concepts that will likely appear on AP Physics 2 exam, with an emphasis on understanding the principles of wave sources, as these are fundamental to many AP Physics topics.

正如您所看到的，流式响应中有一些不完整的部分，揭示了为什么随机字符或单词被遗漏的根本问题。有什么办法可以解决这个问题吗？这是由 Azure 引起的还是在流式传输响应时需要处理的问题，如果是这样，我是否可以查看任何资源来执行此类操作？

相关信息：

请求组建

      const headers = {
        "Content-Type": "application/json",
        "Authorization": "Bearer " + env.AI_KEY
      }
  
      const body = {
        "messages": [
          {
            "role": "user",
            "content": "..."
          },
          {
            "role": "user",
            "content": course + " " + name
          }
        ],
        "max_tokens": 1024,
        "temperature": 0.8,
        "top_p": 1,
        "stream": true
      }
  
      const response = await fetch(endpoint, {
        method: "POST",
        headers: headers,
        body: JSON.stringify(body)
      })

读者

      const SSEEvents = {
        onError: (error: any) => {
          console.error(error);
          controller.error(error)
        },
        onData: (data: string) => {
          const queue = new TextEncoder().encode(data);
          process.stdout.write(queue)
          controller.enqueue(queue);
        },
        onComplete: () => {
          //console.log("Done")
          controller.close()
        },
      };
      const decoder = new TextDecoder();
      const sseParser = new SSEParser(SSEEvents);
  
      const reader = response.body?.getReader()
      
      while (true) {

        const book = await reader?.read();

        if (book?.done) break;

        const chunk = decoder.decode(book?.value);
        try {
          sseParser.parseSSE(chunk);
        } catch (error) {
          console.log(chunk)
          //console.log(error)
        }
        
      }

      controller.close()
    }

Answer 1

当我尝试使用 for 循环读取 Python 中的块时，我遇到了同样的问题。我改用 SSE 客户端库，现在事件以完整的 JSON 可解析块的形式出现。也许，您可以尝试使用不同的 SSE 解析器库。参考

这个