Vercel AI SDK OpenAIStream
Unanswered
Spectacled Caiman posted this in #help-forum
Spectacled CaimanOP
Not sure if this is the right place to ask, but I am using vercel's ai sdk and OpenAIStream: https://sdk.vercel.ai/docs/api-reference/openai-stream
I noticed that if the response is streaming to the user, but the user refreshes the page, the response gets cut off and the backend gets cut access to the response.
Example use case: I am streaming a chatgpt api response, and on complete I want to add it to a database, but if the user leaves the page or refreshes I still can't process the rest of the response on the backend and just add it when complete, since the ReadableStream got cut off.
Anyone know any fix for this?
I noticed that if the response is streaming to the user, but the user refreshes the page, the response gets cut off and the backend gets cut access to the response.
Example use case: I am streaming a chatgpt api response, and on complete I want to add it to a database, but if the user leaves the page or refreshes I still can't process the rest of the response on the backend and just add it when complete, since the ReadableStream got cut off.
Anyone know any fix for this?
4 Replies
i guess Vercel implemented it on purpose. this could help.
https://sdk.vercel.ai/docs/concepts/backpressure-and-cancellation
https://sdk.vercel.ai/docs/concepts/backpressure-and-cancellation
besides that, if it is hosted on Vercel, instances won't run so long. u need self-hosted. or ask at OpenAI Discord server.
@tafutada777 i guess Vercel implemented it on purpose. this could help.
https://sdk.vercel.ai/docs/concepts/backpressure-and-cancellation
Spectacled CaimanOP
read through this already, and it just explains why the cancellation is happening but not how to stop it
@tafutada777 besides that, if it is hosted on Vercel, instances won't run so long. u need self-hosted. or ask at OpenAI Discord server.
Spectacled CaimanOP
nothing is being hosted its just a function