Reproduction
I think @react-router/node package introduced a memory leak in writeReadableStreamToWritable. In #15071 the implementation of writeReadableStreamToWritable was changed and this line was added inside the loop:
let { done, value } = await writableError.race(reader.read());
Which basically is equivalent of await Promise.race([promise, writableErrorPromise]); on every iteration. The problem is that the writableErrorPromise is a long-lived promise and repeatedly racing against the same long-lived promise can retain memory over time. Please note that this is a known issue in node and has been discussed in nodejs/node#17469 and this comment:
The leak described here isn’t just that an increasing number of promise reactions are created for a non-settling promise; this would be more gradual and difficult to detect. In actuality, when you call Promise.race with a long-running promise, the resolved value of the returned promise gets retained for as long as each of its promises do not settle. This means that the severity of this leak is determined by whatever else you’re passing to Promise.race, and a high throughput will crash the process quickly.
I've created a minimal reproduction here.
$ pnpm --dir packages/react-router-node run repro:promise-race-memory-leak
current writeReadableStreamToWritable (after 50511dc): 336.01 MB retained heap
baseline without Promise.race (before 50511dc): 0.07 MB retained heap
difference: 335.94 MB more retained heap
System Info
Used Package Manager
pnpm
Expected Behavior
writeReadableStreamToWritable should not retain memory as it waits for additional chunks. It should behave similarly to the previous implementation that awaits reader.read() directly.
Actual Behavior
writeReadableStreamToWritable retains heap while waiting for the next chunk. In the attached repro, the current implementation retains about 336 MB of heap, while the previous implementation (baseline) without the repeated Promise.race(...) retains about 0.07 MB
Reproduction
I think
@react-router/nodepackage introduced a memory leak inwriteReadableStreamToWritable. In #15071 the implementation ofwriteReadableStreamToWritablewas changed and this line was added inside the loop:Which basically is equivalent of
await Promise.race([promise, writableErrorPromise]);on every iteration. The problem is that thewritableErrorPromiseis a long-lived promise and repeatedly racing against the same long-lived promise can retain memory over time. Please note that this is a known issue in node and has been discussed in nodejs/node#17469 and this comment:I've created a minimal reproduction here.
System Info
Used Package Manager
pnpm
Expected Behavior
writeReadableStreamToWritableshould not retain memory as it waits for additional chunks. It should behave similarly to the previous implementation that awaitsreader.read()directly.Actual Behavior
writeReadableStreamToWritableretains heap while waiting for the next chunk. In the attached repro, the current implementation retains about 336 MB of heap, while the previous implementation (baseline) without the repeatedPromise.race(...)retains about 0.07 MB