I’m hitting a super weird issue with OpenAI’s streaming API.
I’m building a multi-message streaming thing in plain JS (no frameworks) where I need to pipe multiple OpenAI responses one after another. It looks fine on localhost, but in production (node 18) when two streams overlap even by a few ms, the whole thing starts mixing chunks between streams like they share the same reader or something. It feels like some async iterator is leaking state; I'm not sure.
I’m using the official OpenAI node SDK (the new one) and the .response.body.getReader() thing. Sometimes the chunks arrive duplicated, sometimes out of order, sometimes a chunk from stream A literally shows up inside stream B. There are no shared variables, no global state, nothing.
I’m guessing something about the reader or WebStream polyfill inside node is borked when you run multiple concurrent reads, or maybe I’m doing something wrong with the loop. It only happens on the prod server under real traffic, which makes this hell to debug.
Any idea what the “correct” way is to handle 2–3 parallel streaming completions without chunk bleed? Is there a known issue with the SDK or node streams?
Here’s a stripped version of the code that still breaks randomly:
import OpenAI from "openai";
const client = new OpenAI({ apiKey: process.env.OPENAI_KEY });
async function runStream(prompt) {
const res = await client.chat.completions.create({
model: "gpt-4.1",
messages: [{ role: "user", content: prompt }],
stream: true,
});
const reader = res.response.body.getReader();
let buffer = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = new TextDecoder().decode(value);
// this is where random corruption happens:
// sometimes chunk contains leftover bytes from another stream??
buffer += chunk;
}
return buffer;
}
// simulate parallel calls
Promise.all([
runStream("msg A"),
runStream("msg B"),
runStream("msg C"),
]).then(console.log);
If anyone knows why streams bleed into each other under concurrency, or what’s the correct approach to isolate them, I’d appreciate it because right now I’m just logging hex dumps and crying inside.