如何流式传输 OpenAI 的完成 API？

Question

我想通过 OpenAI 的 API传输完成结果。

该文档提到使用服务器发送的事件- 似乎这对于 Flask 来说并不是开箱即用的，所以我试图在客户端进行处理（我知道这会暴露 API 密钥）。但是，由于 OpenAI API 要求将其发布，因此它似乎与 eventSource API 不兼容。我尝试通过 fetch （使用可读流）来完成此操作，但是当我尝试通过示例转换为 JSON 时，出现以下错误：（Uncaught (in promise) SyntaxError: Unexpected token 'd', "data: {"id"... is not valid JSON我知道这不是有效的 JSON）。看起来它正在解析整个结果而不是每个单独的流。

data: {"id": "cmpl-5l11I1kS2n99uzNiNVpTjHi3kyied", "object": "text_completion", "created": 1661887020, "choices": [{"text": " to", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-002"}

data: {"id": "cmpl-5l11I1kS2n99uzNiNVpTjHi3kyied", "object": "text_completion", "created": 1661887020, "choices": [{"text": " AL", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-002"}

data: {"id": "cmpl-5l11I1kS2n99uzNiNVpTjHi3kyied", "object": "text_completion", "created": 1661887020, "choices": [{"text": "I", "index": 0, "logprobs": null, "finish_reason": null}], "model": "text-davinci-002"}

我会喜欢一些关于如何执行此操作的指针或简单的代码示例，因为我已经为此努力了一段时间了。谢谢！

Answer 1

终于得到了这个工作代码：

import { Configuration, OpenAIApi } from "openai";
import dotenv from "dotenv";
dotenv.config({ override: true });

const openai = new OpenAIApi(new Configuration({ apiKey: process.env.OPENAI_KEY }));

const getText = async (prompt, callback) => {
    const completion = await openai.createCompletion(
        {
            model: "text-davinci-003",
            prompt: prompt,
            max_tokens: 1000,
            stream: true,
        },
        { responseType: "stream" }
    );
    return new Promise((resolve) => {
        let result = "";
        completion.data.on("data", (data) => {
            const lines = data
                ?.toString()
                ?.split("\n")
                .filter((line) => line.trim() !== "");
            for (const line of lines) {
                const message = line.replace(/^data: /, "");
                if (message == "[DONE]") {
                    resolve(result);
                } else {
                    let token;
                    try {
                        token = JSON.parse(message)?.choices?.[0]?.text;
                    } catch {
                        console.log("ERROR", json);
                    }
                    result += token;
                    if (token) {
                        callback(token);
                    }
                }
            }
        });
    });
};
    
console.log(await getText("Who was the latest president of USA?", (c) => process.stdout.write(c)));

Answer 2

在浏览器中，您可以使用 fetch API，例如：

const response = await fetch('https://api.openai.com/v1/completions', {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${config.apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model: 'text-davinci-003',
        prompt: input,
        stream: true,
      }),
    });
    const reader = response.body?.pipeThrough(new TextDecoderStream()).getReader();
    if (!reader) return;
    // eslint-disable-next-line no-constant-condition
    while (true) {
      // eslint-disable-next-line no-await-in-loop
      const { value, done } = await reader.read();
      if (done) break;
      let dataDone = false;
      const arr = value.split('\n');
      arr.forEach((data) => {
        if (data.length === 0) return; // ignore empty message
        if (data.startsWith(':')) return; // ignore sse comment message
        if (data === 'data: [DONE]') {
          dataDone = true;
          return;
        }
        const json = JSON.parse(data.substring(6));
        console.log(json);
      });
      if (dataDone) break;
    }

Answer 3

使用此代码：

const { Configuration, OpenAIApi } = require("openai");

const configuration = new Configuration({
    apiKey: process.env.REACT_APP_APIKEY,// your api key
  });
const openai = new OpenAIApi(configuration);
let fetchData = async () => {
        await openai
          .createCompletion({
            model: "text-davinci-002",
            prompt: `hello i am searched text`,
            max_tokens: 500,
            temperature: 0,
          })
          .then(response => {
         
            console.log(response.data.choices[0].text);
          })
          .catch(err => console.log(err));
      };
      fetchData();

data您将在 object -> -> choices[0]->中接收数据text。