LLM 端点托管在 Azure databricks 响应 200 中，但主体已锁定

Question

我在使用 ReactJS 和 Flask 创建并托管在 azure 中的 POC 中遇到了这个问题，我获得了一个托管在 azure databricks 中的 LLM 端点。当我在邮递员处尝试时，端点工作正常，但当我尝试在我的应用程序中调用它时，响应为 200 但正文被锁定。我在之前的项目中从未遇到过这个问题，并且我无法查看是否有任何权限或限制选项可以在天蓝色中查看主体。下面是我的代码片段和端点响应。

我省略了端点路径，但我确信它是正确的

#Flask app.py
import requests
from flask import Flask, request, jsonify
from flask_cors import CORS

app = Flask(__name__)

CORS(app, resources={r"/api/*": {"origins": "http://localhost:5173"}})

@app.route("my-endpoint", methods=["POST"])
def handle_invocations(endpoint_name):
    try:
        payload = request.json
        base_url = "https:{my-url}.azuredatabricks.net"
    
        target_url = f"{my-target URL}"
        

        headers = {
        "Content-Type": "application/json",
        "Authorization": "{my token}"
        }
    
        response = requests.post(target_url, json=payload, headers=headers)
    
        return jsonify(response.json()), response.status_code
    except Exception as e:
        return jsonify({"error": str(e)}), 500

if __name__ == "__main__":
    app.run(debug=True)

# Reactjs Homepage.jsx
const onCardClick = async (question) => {
    const payload = {
        messages: [
            {
                role: "user",
                content: String(question),
            },
        ],
    };

    try {
        const response = await fetch({my-endpoint}, {
            method: "POST",
            headers: {
                "Content-Type": "application/json",
                "Authorization": {my-token}
            },
            
            body: JSON.stringify(payload),
        });

        if (!response.ok) {
          let errorDetails = null;
  
          
          try {
              errorDetails = await response.json();
          } catch (jsonError) {
              console.error("Failed to parse error response as JSON.");
          }
  
          console.error("HTTP Error:", {
              status: response.status,
              statusText: response.statusText,
              details: errorDetails
          });
  
          throw new Error(`HTTP error! status: ${response.status} - ${response.statusText}`);
      }

        const result = await response.json();

        
        setTextFieldContent(result.content || "No response content available.");
    } catch (error) {
        console.error("Error:", error);
        setTextFieldContent("An error occurred while fetching the response.");
    }
};

Answer 1

您收到的回复是正确且有效的，请访问如下内容。

const  content = result.choices[0].message.content || "No response content available.";

根据您传递的参数和正文，来自端点的响应 json 数据将如下所示，请检查您使用的端点或 LLM 模型文档。

{
  "id": "chatcmpl_cec8b3cd-856d-417b-a629-907e15a55762",
  "object": "chat.completion",
  "created": 1734327739,
  "model": "dbrx-instruct-071224",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Databricks is a data and analytics company that provides a unified analytics platform for data science teams to collaborate on large-scale data engineering, data science, and machine learning workloads. The company was founded by the original creators of Apache Spark, Delta Lake, and MLflow. Databricks provides a managed platform for these open-source technologies, as well as additional proprietary features, to help organizations simplify and scale their data-driven initiatives. The platform enables data teams to process and analyze large volumes of data, build and deploy machine learning models, and share insights across the organization."
      },
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 251,
    "completion_tokens": 115,
    "total_tokens": 366
  }
}

接下来，

body

被锁定，因为只读一次。

我尝试时遇到以下错误。

代码片段

console.log(response)
const bd = await response.body.getReader().read();
console.log(bd)
const result = await response.json();
const content = result.choices[0].message.content || "No response content available.";

错误

TypeError: Failed to execute 'json' on 'Response': body stream already read

您可以在下面的屏幕截图中看到，使用阅读器对象读取正文获取的值，但

.json()

函数失败。

只有一次可以读取锁定的主体，因此将其读取为 json 并存储在变量中。