RFC: LWC server runtime proposal

[pmdartus]

Server-side Rendering (SSR) is a popular technique for rendering pages and components that would usually run in a browser on the server. SSR takes as input a root component and a set of properties and renders an HTML fragment and optionally one or multiple stylesheets. This technique can be used in many different ways: improve initial page render time, generate static websites, …

Due to the size and complexity of this feature, the rollout of SSR is broken up in 3 different phases and is spread over multiple releases:

• Phase 1: Decouple the LWC engine from the DOM and allow it to run in a JavaScript environment without access to the DOM APIs (like Node.js). As part of this phase, the LWC server engine will also generate a prototype version of the serialized HTML (Phase 2).
• Phase 2: Settle on an HTML serialization format.
• Phase 3: Rehydrate in a browser the DOM tree resulting from the serialization into an LWC component tree.

As part of phase 1, the goal is to provide a capability for LWC to render components transparently on the server. From the component standpoint, the LWC SSR and DOM versions provide identical APIs and will run the component in the same fashion on both platforms.

---

Rendered

[tedconn]

Suggested change

	This package injects mock DOM APIs in the `@lwc/engine-core` rendering engine. Those DOM APIs produces a lightweight DOM structure that can be serialized into a string by the `renderToString` method. As described in the Appendix, this package is also in charge of attaching on the global object a mock `CustomEvent`.
	This package injects mock DOM APIs in the `@lwc/engine-core` rendering engine. Those DOM APIs produce a lightweight DOM structure that can be serialized into a string by the `renderToString` method. As described in the Appendix, this package is also in charge of attaching on the global object a mock `CustomEvent`.

[tedconn]

Those DOM APIs produce a lightweight DOM structure

Which DOM implementation do you plan to use for that?

[caridy]

I think it will be our own bake implementation... We do not plan to allow user-code to talk to the dom, and if they do, we will throw or ignore. The mocks needed here (btw, I will not call them mocks, but implementation) are those APIs needed for the engine to function, so it goes hand to hand.

[tedconn]

I would also add scalability. The mocked APIs would have to be kept in sync with what's actually being used in the engine. Also the engine still wouldn't be reusable in other environments without a corresponding "mock" layer.

[pmdartus]

I don't think it is the case. There is only a well-defined set of DOM APIs the engine requires to operate. Each runtime (dom and server) is in charge of implementing this contract by passing an implementation of those APIs to the engine.

[tedconn]

Maybe it has stabilized by now but when I took that approach I had to add/change multiple mocked APIs during the couple months I touched that code.

[caridy]

I don't think that will be that challenging, we will own both packages, and we can keep them in sync.

[davidturissini]

Not a fan of the name. engine-dom describes the LWC execution context and doesn't imply a client or server environment. engine-server describes one possible environment for a dom-less execution context. We can bike shed on naming, but I'd prefer engine-string or something similar

[pmdartus]

Agreed, I wasn't happy with the naming.

What do you think about engine-dom-renderer and engine-string-renderer? Even if the package names are more verbose I don't think it's a problem because the only thing developers have to remember if lwc.

[ekashida]

How about @lwc/engine-core, @lwc/engine, and @lwc/engine-string? This would minimize the impact of renaming existing packages.

[apapko]

I also like the inverted version: @lwc/client-engine, @lwc/server-engine, @lwc/engine

[pmdartus]

I don't think there will be a renaming impact here. The @lwc/engine module should never be referenced directly. Userland code should only reference lwc, the only place where we might need to change is on the resolver.

[caridy]

I'm with @ekashida on this, this is NOT about client and server I'm afraid, this is about dom vs dom string. Imagine that in the future we decide to use the dom string output in a client for some reason.

[pmdartus]

[...] this is about dom vs dom string.

I fully agree with this statement. In this regard, I think it would less confusing if the module that is loaded in a DOM environment contains the term dom in its name. I would suggest reaming @lwc/engine to @lwc/engine-dom in @ekashida proposal.

[davidturissini]

One thing worth exploring: Can we pass @api data for the top level component here? If values are known at "tree creation time", we can probably hydrate a good portion of the LWC tree sync as the initial tree is being created

[caridy]

this is probably another area to explore in a different RFC since it touches on how much can you render on the server synchronously.

[pmdartus]

If the data is known before the component is rendered, the current proposal should be sufficient.

import { createElement, renderToString } from "lwc";
import App from "c/app";

const app = createElement("c-app", { is: App });
app.foo = 'Some value';
app.bar = ['Some', 'other', 'value.'];
const str = renderToString(app);

console.log(str);

Like a standard HTMLElement returned by createElement when LWC is running in the browser, the ServerHTMLElement return by created createElement` on the server would expose:

• the component public properties
• the global HTML properties
• aria reflection properties

[blittle]

This would probably be a major version change requiring new documentation.

[ekashida]

How about @lwc/engine-core, @lwc/engine, and @lwc/engine-string? This would minimize the impact of renaming existing packages.

[priandsf]

How are the components stylesheets passed as part of the string? Are they pre-pended to the string as a <style> tag?

[pmdartus]

This is one area that will require more investigation. The serialization format is outside the scope of this proposal and will be subject to another RFC: https://github.com/salesforce/lwc-rfcs/blob/ccbd9f38a7ff30d2bd604107e6eb13608366aed2/text/0112-server-runtime.md#scope

[priandsf]

@seckardt Maybe you want to share your investigation in this area?

[seckardt]

On our first baby steps towards SSR I didn't reinvent the wheel in that area. I simply reused existing functionality as provided by the StylesheetFactory.

[priandsf]

The POC directly serializes the VDOM, without the need of a DOM structure. Why is a lightweight DOM necessary? Isn't that an extra generation step that would impact the performance?

[pmdartus]

Yes, this DOM structure will have a performance impact however there needs to be a DOM structure (even really lightweight) to attach the event listeners and dispatch the DOM events.

[tedconn]

@pmdartus "LWC Server Runtime" sounds like an application framework for running LWC, like LWR (and runtime is super overloaded). Can we rename this to "LWC Server Engine" or "LWC DOM Decoupling"?

[gamedevsam]

I'm hoping someone will take a look at the complexity of implementing a one time snapshot of wired data during SSR. If SSR comes out with wire adapters disabled it will significantly decrease its appeal.

At least expose an option? Or a plugin mechanism to enable it to work?

[gamedevsam]

Disabling wire adapters during SSR eliminates the ability to use wire protocol to inject initial state including localized text in the SSR output. Can't we invoke the wire's constructor, connect and disconnect and capture a one time snapshot init data of the wire?

[pmdartus]

As of today, the invocation timing of the update callback is different if the wire adapter has dynamic parameters or not. This difference makes it impossible to invoke wire adapters in a reliable fashion.

Changing the current invocation timing is an observable change that might result in breaking existing customers on the platform. We decided to set wire adapters aside from the initial SSR implementation but it is something that we will reconsider in the future.

[gamedevsam]

Perhaps this can enabled as an optional configuration param when we call renderLwcToString or something like that. By default wire adapters can be skipped, but users can opt-in if their use case is deterministic like in my case, I don't connect wire adapters to network requests, they are simply connected to redux style stores to decouple state and UI.

[gamedevsam]

Perhaps a reasonable compromise would be to support Wire adapters without dynamic parameters during SSR, which would cover several use-cases (including my own).

[pmdartus]

IMO, we should make wire execution predictable (both on the client and on the server). It would be surprising that for a developer to discover that a wire stops getting executed in SSR after adding a dynamic param.

Over the last 6 months, the LWC team has been investigating how we could roll out potentially risky changes like this one (changing the invocation timing of wires) without breaking existing customers in production. I hope that we could leverage one of the approaches we are discussing to solve this issue and enable wire adapter execution for SSR.

/cc @nolanlawson

[AllanOricil]

@pmdartus
Did you thought about creating a new hook that fetches data on the server, before serving the component? Nuxt does that pretty well. In nuxt we have a method called fetch. I believe it would be something really great for lwc. With a fetch method, components could be served straight away with the data developers need from Salesforce. Pages would be rendered so much faster, not just because the initial state of the component was rendered on the server, but also because components would no longer need to wait for the wired method - the one necessary for the initial state of the component - to be added to the stack call on the client, which can be slow depending on the specs of the computer, and how many other wired methods were added to the stack.

I also believe that with a fetch method, lots of use cases for the wired methods would no longer exist.

How would ssr work on a lightning page? Would salesforce let us control which lwcs are rendered on the server, or would it be all or none?

export default class comp extends LightningComponents {

    fetch(context){
        /* context 

          user   => exposes current session permissions
          query => runs queries soql/sosl
          isBrowser => this could also be available on the middleware. This way we can interact with data stores in the middleware if available


        inside this scope, the server could let us use "some" of node functions/modules, such as the native fetch???? With the fetchrm function I would be able to get data from other authorized services (maybe controlled by remote site settings entries). 
 I could also use the `query` object from `context` to fetch data from Salesforce in a secure way. 
*/
     }
}

[AllanOricil]

Middlewares that can run on both client/server could also be useful to control the initial state of a component depending on the User permissions. If ssr is enabled on salesforce, the context object could expose objects to interact with Salesforce APIs

export default class comp extends LightningElement{

     middleware(context){
      /*context has objects such as 
      user   => exposes current session permissions
     query => runs queries soql/sosl
     isBrowser => this could also be available on the middleware. This way we can interact with data stores in the middleware if available
     */
     }

     Or

     this.middleware = commonMiddleware //must be imported as another module. It is useful for sharing middlewares to many components
     
}

[gamedevsam]

@AllanOricil the fetch method is an interesting approach, NextJS offers something called getServerSideProps that does something similar.

My issue with this proposal, or any other proposal that provides an alternative to wire adapters is that it basically requires developers who leverage wire adapters for their data fetching to re-engineer their components to be server side rendered. This may not be avoidable due to some foundational architecture decision, but it's a real shame.

The wire adapter API is one of the best things about LWC, forking the ecosystem by saying on clients wire adapters are supported, but on servers they are not, is going to be a serious barrier for adopting server side rendering.

Perhaps a reasonable compromise would be to support Wire adapters without dynamic parameters during SSR, which would cover several use-cases (including my own).

[AllanOricil]

@AllanOricil the fetch method is an interesting approach, NextJS offers something called getServerSideProps that does something similar.

My issue with this proposal, or any other proposal that provides an alternative to wire adapters is that it basically requires developers who leverage wire adapters for their data fetching to re-engineer their components to be server side rendered. This may not be avoidable due to some foundational architecture decision, but it's a real shame.

The wire adapter API is one of the best things about LWC, forking the ecosystem by saying on clients wire adapters are supported, but on servers they are not, is going to be a serious barrier for adopting server side rendering.

Perhaps a reasonable compromise would be to support Wire adapters without dynamic parameters during SSR, which would cover several use-cases (including my own).

@gamedevsam
Can't ssr be optional, and maybe configurable per component, and not just routes/pages. If so, maybe it would not be necessary to rewrite anything that is already working. Wire adapters eval could happen after the page is sent to the client. Why exactly do you think people would have to re-engineer their components?

More about nuxt fetch

The fetch method, if set, is called every time before loading the component (only for page components). It will be called server-side once (on the first request to the Nuxt app) and client-side when navigating to further routes.

If you think a fetch method is a good idea, I would like to add an addition to its proposal. fetch must not be a Salesforce org feature, like wire methods are. It must run everywhere. But when running on Salesforce, it must be injected with a Salesforce context object.
For me, wire methods always felt weird because they can only be used inside a Salesforce org. I think they shouldn't even be considered a lwc feature.

[gamedevsam]

We should maybe take this discussion elsewhere (maybe Slack? Hit me up @Samuel Batista) since it's getting a bit unrelated to the current topic at hand, but I'd like to answer a few misconceptions in your previous comment:

Wire adapters eval could happen after the page is sent to the client. Why exactly do you think people would have to re-engineer their components?

Wire adapters can result in an invocation of a wire method, as well as wire values directly into member variables. If a wire adapter invokes a method on a component, that method often assigns values into member variables of the component. These variables can (and often are) accessed in the template of the component. If we render the template without invoking wire adapters, the HTML will be rendered using default values provided for those member variables. This may be acceptable for some use cases, but in my case it will break the template rendering, since we assume wire adapters will be invoked before the initial render, and will provide the initial values for member variables. Here's an example of code that would break in several of our repos, where we leverage wire adapters to inject localized text, ex:

export default class I18nComponentExample {
    // this:
    @wire(WireI18n) i18n;

    // is equivalent to this:
    i18n;
    @wire(WireI18n)
    i18nConnected(i18n) {
        this.i18n = i18n;
    }
}

Rendering the template without the wire adapter will break this component:

<p>{i18n.keyToLocalizedText}</p>

For me, wire methods always felt weird because they can only be used inside a Salesforce org. I think they shouldn't even be considered a lwc feature.

I use wire methods / wire adapter API outside of Salesforce, we use it to wire data from Redux-like data stores in applications hosted on Heroku. Wire adapters are frequently used to pipe networked data into components, and that is the main use case for components rendered within Salesforce. But that isn't their only use case. AppExchange uses them to provide reactivity / connect them to external stores that manage our state. Wire adapters for me are like React hooks, they are in my opinion one of the best features of LWC OSS.

[AllanOricil]

We should maybe take this discussion elsewhere (maybe Slack? Hit me up @Samuel Batista) since it's getting a bit unrelated to the current topic at hand, but I'd like to answer a few misconceptions in your previous comment:

Wire adapters eval could happen after the page is sent to the client. Why exactly do you think people would have to re-engineer their components?

Wire adapters can result in an invocation of a wire method, as well as wire values directly into member variables. If a wire adapter invokes a method on a component, that method often assigns values into member variables of the component. These variables can (and often are) accessed in the template of the component. If we render the template without invoking wire adapters, the HTML will be rendered using default values provided for those member variables. This may be acceptable for some use cases, but in my case it will break the template rendering, since we assume wire adapters will be invoked before the initial render, and will provide the initial values for member variables. Here's an example of code that would break in several of our repos, where we leverage wire adapters to inject localized text, ex:

export default class I18nComponentExample {
    // this:
    @wire(WireI18n) i18n;

    // is equivalent to this:
    i18n;
    @wire(WireI18n)
    i18nConnected(i18n) {
        this.i18n = i18n;
    }
}

Rendering the template without the wire adapter will break this component:

<p>{i18n.keyToLocalizedText}</p>

For me, wire methods always felt weird because they can only be used inside a Salesforce org. I think they shouldn't even be considered a lwc feature.

I use wire methods / wire adapter API outside of Salesforce, we use it to wire data from Redux-like data stores in applications hosted on Heroku. Wire adapters are frequently used to pipe networked data into components, and that is the main use case for components rendered within Salesforce. But that isn't their only use case. AppExchange uses them to provide reactivity / connect them to external stores that manage our state. Wire adapters for me are like React hooks, they are in my opinion one of the best features of LWC OSS.

@gamedevsam my bad

https://lwc.dev/guide/wire_adapter#wire-adapters

After reading this doc, now I understand how it actually works.

[pmdartus]

There are a lot of interesting points that have been raised on this thread. Let me try to answer some of them.

How would ssr work on a lightning page? Would salesforce let us control which lwcs are rendered on the server, or would it be all or none?

For the record, there is currently no plan to use SSR in the context of Lightning Experience. Salesforce is considering using SSR for other experiences where initial page load time and SEO are important.

As a side note, the portability of existing LWC components on the Salesforce platform from client-side rendering only to client + server-side rendering is also something that we discussed at length. While we understand that customers have put a lot of effort into building LWC components on top of the platform, it's important to acknowledge that building client-side-only UI components is vastly different than build isomorphic components. And this, regardless of the UI framework.

We are in the middle of porting some of our existing internal LWC components to be SSR compatible, and as part of this journey, some components had to be redesigned from the ground up. Documentation and linting rules are in preparation to help with this migration.

Did you thought about creating a new hook that fetches data on the server, before serving the component?

Yes, it was top of mind as we designed SSR for LWC. We are currently investigating an approach similar to Next getServerSideProps and Nuxt asyncData for data retrieval at the page level.

We also discussed exposing the capability to retrieve async data in arbitrary components (like fetch in Nuxt), but it's something that we have set aside for now as it would require a large rewrite of the LWC engine to make all the internals async. We believe the page-level data retrieval would be sufficient for the initial version of SSR. And as always it's something we might reconsider in the future based on developers' feedback.

/cc @nolanlawson @divmain