-
Notifications
You must be signed in to change notification settings - Fork 429
Description
I have a complete API interface that returns content in a streaming manner, designed for JavaScript reception and display in a browser.
However, I've encountered a strange issue: I must output a
tag to the browser before accessing the streaming content, otherwise the subsequent stream, though already output to the browser, will be buffered and not displayed until the entire content is received. If I comment out the
tag line, the browser seems to hold the streamed content in a buffer and waits until all content is available before displaying it. Here is the key part of the code.
The operating system is Windows Server 2019, and the browser version is Chrome 120. Please Help me.
var pStart = Encoding.UTF8.GetBytes("
");
await foreach (var res in chat.StreamResponseEnumerableFromChatbotAsync())
{
output = res;
var data = Encoding.UTF8.GetBytes(output);
await Response.Body.WriteAsync(data, 0, data.Length);
_logger.LogInformation($"Res:{output} Length:{data.Length}");
}
Full Code
[HttpPost("AIChatStream")]
public async Task AIChatStream([FromBody] ChatGPT_Request request)
{
try
{
OpenAIAPI api = new OpenAIAPI(_openAIOption.APIKey);
var chat = api.Chat.CreateConversation();
chat.Model = OpenAI_API.Models.Model.GPT4_Vision;
chat.RequestParameters.Temperature = 0.2;
chat.RequestParameters.MaxTokens = 2000;
// 将聊天内容封装到ChatMessage对象中
foreach (var message in request.messages)
{
if (message.role == "user")
{
chat.AppendUserInput(message.content, ImageArrayToImageListArray(message.images));
}
if (message.role == "assistant")
{
chat.AppendMessage(new ChatMessage(ChatMessageRole.Assistant, message.content));
}
if (message.role == "system")
{
chat.AppendMessage(new ChatMessage(ChatMessageRole.System, message.content));
}
}
var pStart = Encoding.UTF8.GetBytes("<br />");
//var pStart = Encoding.UTF8.GetBytes(Environment.NewLine);
await Response.Body.WriteAsync(pStart, 0, pStart.Length);
var output = string.Empty;
await foreach (var res in chat.StreamResponseEnumerableFromChatbotAsync())
{
output = res;
var data = Encoding.UTF8.GetBytes(output);
await Response.Body.WriteAsync(data, 0, data.Length);
_logger.LogInformation($"Res:{output} Length:{data.Length}");
}
}
catch (Exception ex)
{
_logger.LogError($"Exception:{ex}");
await Response.Body.WriteAsync(Encoding.UTF8.GetBytes(ex.Message));
}
}