Out of order streaming

[LazyClicks]

What is it about?

allow page rendering while still using routeLoaders by streaming the response

What's the motivation for this proposal?

Problems you are trying to solve:

• Page rendering is blocked by current routeLoaders

Goals you are trying to achieve:

• non blocking streaming routeLoaders

Any other context or information you want to share:

Im aware that this has been discussed before but just thought it would be cool to add it here as it would fit the usage of this proposals repo

---

Proposed Solution / Feature

What do you propose?

api for it already exists using with onPending and onResolved but the render blocking is still there

Code examples

find below the example by gioboa

---

Links / References

https://qwik.dev/docs/cookbook/streaming-deferred-loaders/

[wmertens]

This is out-of-order streaming and we'll take a look at how to do it after v2 is released.
We'll need to figure out how to stream state (making sure object references are retained) and how to make it work on no-js clients (shadow dom?).

[ianlet]

Just a note to mention that the same goes for useResource. Some people in the Discord and in the issues requested useResource to behave more like <Suspense> and prevent blocking rendering in SSR.

Should we update this RFC to be about Out of order streaming? Which would enable non-blocking routeLoaders and useResource.

[LazyClicks]

personally speaking useResource sounds like it would be out of order rendering , and that was actually my initial understanding of it

[thejackshelton]

Summarizing Misko here:

div with id (marker in Dom → stuff later)

when spit out → correct html wrapped in whatever

spit out tiny script tag that looks up the id’s and switches them

this needs to work with multiple qwik apps on a page

Unique id that won’t conflict with anything else

Out of order execution → is basically a promise that is not resolved

When we serialize output, we wait for all promises to resolve, then serialize

In this case we need to do a different kind of serialization. Serialize the promise as “not serialized”, then spit out the promise as a script tag later on.

It’s not just about streaming the html, but dealing with the fact that you need to lazily deal with an unresolved promise, later on we spit out the script tag that refers back to the promise, once it’s resolved.

The script tag also contains the resolved promise state.

It’s not a simple switch, needs to be conditional

if Qwik not yet awoken, then switch

if Qwik woke, resolve promise and let qwik do the rest to render

We need to remember:
- Suspense or resource like behavior with a loading state
- Come across resource, make decision, do I wait or do OOO? If I do OOO, then i leave behind temporary thing and know how to replace myself. Serialize promise as unresolved, keep going

When it wakes up, resolve the promise then update the resource.

change serialization format to deal with unresolved promises

constraint that state is serialized in one script tag, change that so that state is serialized across multiple script tags, such that the connections are “unresolved promises”, with a unique id

“I will deliver promises later → new script tag that shows up”

leave behind special comments for worker to understand if it’s safe to pause stream and stream something else. Only way it knows is if we tell it (emit a comment inside the html)

“Am I blocked? Ok, dom worker here is special comment, here is a safe place you can stop streaming me and stream someone else temporarily”

worker A: OOO (A portion of app)

worker B: OOO (B portion of app)

worker C: merges two streams together (host of all the MFE containers).

worker C is currently naive → sends everything from A then everything from B.

multiplexer can’t randomly decide to pause stream. 

tell the thing that it is ok to inject it in the stream and it will not break anything.

[shairez]

Following our RFC monthly
This is all the potential things to consider for the implementation of OOO streaming 👆

(leaving this here as pointers to when we'll start the implementation)

[wmertens]

Some more thoughts Right now we do two passes over data that we serialize, to identify shared references. When we do out of order streaming, we can't do this any more. This means that we should serialize in a single pass, and keep a map of all objects and their emitted location. If we simply emit everything in the root array, we'll get a very long array and the references will take more bytes. Instead, we could encode a reference as the path to the object in the nested serialized array, as an array of indexes. To prevent having to specify this every time, we would store this as a root object. On my phone so hard to draw a picture sorry. So the serialization would become: - single pass, but await promises - keep a map between each object and it's nested array output location - when you encounter an object again, output the backreference as a root object that you then reference. So a `[Ref, 123]` to a `[Ref, 4, 3, 1]` to the object. Maybe skip the double ref if the path is short. - streaming just continues adding to the root array - the vnode map needs similar handling, perhaps it should just be output as part of the serialization. - the object map must be kept in memory until SSR is complete Wout.

…

On Fri, 20 Dec 2024, 23:29 Shai Reznik, ***@***.***> wrote: Following our RFC monthly This is all the potential things to consider for the implementation of OOO streaming 👆 — Reply to this email directly, view it on GitHub <#16 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAANNFRVMQICTWZZHA2NPQD2GR42XAVCNFSM6AAAAABNY6ACYWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRTGM2DGNY> . You are receiving this because you commented.Message ID: ***@***.***>

[wmertens]

Actually, for Promises we could remember their position and emit the value as a root ref with empty offset. Then when deserializing, we scan root for these results and we patch the promised ref Wout.

…

On Sat, 21 Dec 2024, 12:28 Wout Mertens, ***@***.***> wrote: Some more thoughts Right now we do two passes over data that we serialize, to identify shared references. When we do out of order streaming, we can't do this any more. This means that we should serialize in a single pass, and keep a map of all objects and their emitted location. If we simply emit everything in the root array, we'll get a very long array and the references will take more bytes. Instead, we could encode a reference as the path to the object in the nested serialized array, as an array of indexes. To prevent having to specify this every time, we would store this as a root object. On my phone so hard to draw a picture sorry. So the serialization would become: - single pass, but await promises - keep a map between each object and it's nested array output location - when you encounter an object again, output the backreference as a root object that you then reference. So a `[Ref, 123]` to a `[Ref, 4, 3, 1]` to the object. Maybe skip the double ref if the path is short. - streaming just continues adding to the root array - the vnode map needs similar handling, perhaps it should just be output as part of the serialization. - the object map must be kept in memory until SSR is complete Wout. On Fri, 20 Dec 2024, 23:29 Shai Reznik, ***@***.***> wrote: > Following our RFC monthly > This is all the potential things to consider for the implementation of > OOO streaming 👆 > > — > Reply to this email directly, view it on GitHub > <#16 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAANNFRVMQICTWZZHA2NPQD2GR42XAVCNFSM6AAAAABNY6ACYWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNRTGM2DGNY> > . > You are receiving this because you commented.Message ID: > ***@***.***> >

[colexbruhn]

It may be a niche use-case, but out-of-order streaming would be super ergonomic for search forms.

Rather than calling a routeAction during a Form submission, you could just do an SPA navigation to add url parameters. This would re-trigger the routeLoader and run a search. Assuming the loader can be deferred, the navigation would happen instantly. Ideally, this would interop cleanly with a Resource element to immediately display a loading state, but it currently just blocks rendering instead (I know someone else already mentioned this behavior).

This would simplify a lot of logic for some projects. Currently, you need to mix loaders + actions, or resort to useVisibleTask$ to be able to handle url parameter searches in a way that still loads the page and provides feedback instantly.

Overall, I think deferred/streaming loaders and the ability to opt-out of the Resource element blocking rendering would be amazing to have since they can allow apps to load instantly and feel much faster.

[wmertens]

Good thinking, but I'd like it to still allow SPA, and the route loader data would just stream in. Seems possible.

[wmertens]

Some more thoughts:

Out-of-order streaming

when we allow out-of-order streaming but we want interactivity while the page is still streaming, we can't scan the state once for cycles/Promises/references, because later state might refer to earlier state.

Therefore we must make the serializer single-pass. To do this, we could store each object as a root, but that will make references take more bytes. Instead, we could implement sub-paths for references, and when we encounter a reference, we output that instead.

For later references it would be best to have a root reference, so we'd probably add a root for the sub-reference and later we can refer to that.

We'll still need to keep track of Promises. We can do this by waiting, but that halts streaming. Instead, we could write a forward reference id, and when the promise is resolved, we store the result as the next root item. At the end of the stream we can emit a mapping from forward references to root index.

Then later, when we send out-of-order state, it will append to the existing state on the client.

This single-pass approach might just be better in general, because several bugs were found due to differences between the first and second pass code, and it allows streaming the data while the promises are being resolved.