Custom Server ESM Integration

[cbratschi]

Link to reproduction

private repo

To Reproduce

See below.

Describe the Bug

We tried to integrate Payload as a custom server based on the example from https://github.com/payloadcms/payload/blob/master/examples/custom-server. Our project uses type module i.e. uses the ESM loader instead of CommonJS.

npm run dev fails with "Unknown file extension ".ts" for .../apps/gg-nextjs/src/payload/server.ts".

This workaround in nodemon.config gets the server to be started:

node --experimental-specifier-resolution=node --loader ts-node/esm src/payload/server.ts

Whereas this fails:

ts-node --project tsconfig.server.json src/payload/server.ts

The following payload.config.ts example uses ESM imports but in parallel __dirname:

https://github.com/payloadcms/payload/blob/master/examples/custom-server/src/payload.config.ts

__dirname should not be available in ESM modules. Probably Webpack allows this but a correctly implemented load won't accept it. We use import.meta.url which works fine for the server part but we have to load the payload.config.ts ourselves and pass it as a parameter to payload.init(). However, Payload still loads payload.config.ts again which leads to a Webpack issue because the url Node.js package cannot be found. Right now we avoid using the url package and __dirname. At least Payload is running in dev mode.

Payload should used the config in payload.init() as singleton and not load it again. This would allow us to load the ES module once and avoid conflicts between two repositories.

Payload Version

1.11.8

[cbratschi]

Update: the type had to be set to CommonJS in package.json. Could not get it to work with ES modules.

[sannajammeh]

I've had many problems with TS Node to the point where I abandoned it completely. I recommend trying out @swc-node/register/esm and see if that works.

[cbratschi]

@sannajammeh I am using ts-node with "swc: true". Internally it is using SWC.

[sannajammeh]

@sannajammeh I am using ts-node with "swc: true". Internally it is using SWC.

Yes but their resolution logic was the problem for me. Regardless of SWC being used it still goes through tsnode

[tyteen4a03]

I've had many problems with TS Node to the point where I abandoned it completely. I recommend trying out @swc-node/register/esm and see if that works.

Do you have an example setup?

[AlessioGr]

The following payload.config.ts example uses ESM imports but in parallel __dirname:

Typescript allows us to use ESM-style imports, but the whole of payload is still on 100% CJS, which is why you see stuff like __dirname and require being used.

We're looking to switch to ESM in 3.0 - until then, I recommend to just use CommonJS. Maybe Bun will be able to solve this in the meantime, as it can handle both ESM and CJS. I've converted it to a Discussion, as it's more of a feature request / improvement, rather than a bug.

[AlessioGr]

We did! 3.0 is now 100% ESM

[fpolli]

That's awesome. This is the problem of coming into a discussion late. :)

[mryellow]

TypeScript isn't allowing ESM-style imports. It's making the entire package harder to use and requires extra dependencies.

[AlessioGr]

@mryellow can you give me an example? TypeScript does allow ESM-style imports. In fact, even if your project was on CJS, TypeScript still made it possible to use ESM-style imports through "module": "CommonJS", which wouldn't be possible in plain JS

[mryellow]

TypeScript does allow ESM-style imports.

It might when you're compiling TypeScript. When you're using Javascript instead it just breaks everything and makes everything harder.

Despite the lack of documentations and examples managed to get it all working as normal Javascript under CommonJS by not using any of the build scripts provided but instead simply copying files into the /dist directory and committing that.

"module": "CommonJS"

I am not writing a module.

...

src/server.js

const express = require('express');
const payload = require('payload');

require('dotenv').config();
const app = express()

app.get('/', (_, res) => {
	res.redirect('/admin')
});

const start = () => {
	return Promise.resolve()
		.then(() => payload.init({
			debug: true,
			telemetry: false,
			secret: process.env.PAYLOAD_SECRET,
			express: app,
			onInit: () => {
				payload.logger.info(`Payload Admin URL: ${payload.getAdminURL()}`);
			},
		}))
		.then(() => {
			console.log('Starting server!', process.env.PAYLOAD_PORT || 3000);
			app.listen(process.env.PAYLOAD_PORT || 3000);
		});
};

start();

src/payload.config.js

const fspath = require('path');

//const payloadCloud = require('@payloadcms/plugin-cloud').payloadCloud;
const mongooseAdapter = require('@payloadcms/db-mongodb').mongooseAdapter;
const webpackBundler = require('@payloadcms/bundler-webpack').webpackBundler;
const slateEditor = require('@payloadcms/richtext-slate').slateEditor;
const buildConfig = require('payload/config').buildConfig;

const Users = require('./collections/Users');
const Posts = require('./collections/Posts');

module.exports = buildConfig({
	admin: {
		user: Users.slug,
		bundler: webpackBundler(),
		components: {},
	},
	editor: slateEditor({}),
	collections: [Users, Posts],
	//typescript: {
	//	outputFile: fspath.resolve('payload-types.ts'),
	//},
	graphQL: {
		disable: true, // NOTE: Running with "NODE_ENV=production" causes issues when not using TypeScript
		disablePlaygroundInProduction: true,
		schemaOutputFile: fspath.resolve('generated-schema.graphql'),
	},
	//plugins: [payloadCloud()],
	plugins: [],
	db: mongooseAdapter({
		url: process.env.DATABASE_URI,
	}),
});

[carloslato]

I was trying to run Astro build with SSR mode on the same express server as Payload CMS to use the Local API. But nodeJS gives error when importing the 'entry.mjs' file from astro because Astro compiles for ES6. Any suggestions ?