refactor(ast): Introduce flexible serialization encoding for ast

[quininer]

Description:

Following our previous discussion, I am trying to introduce a self-describing serialization encoding for ast to replace #11069 .

Since serde is currently used for json serialization, it is hard to modify serde to support both json and binary encodings (due to the untagged and flatten attr). So I made our own Encode and Decode proc macro, which also facilitate unknown variant support.

I've done some simple benchmarks with a modified swc ast, and I believe we don't have an unacceptable performance loss relative to rkyv.
I'm using cbor4ii here, which I developed, so I'd prefer it if there's no performance difference. but it's easy to migrate to bincode or other scheme.

This also introduces an unknown feature for ecma_ast crate, which should only be enabled by the wasm plugin to ensure that old plugins can decode new AST.

BREAKING CHANGE:

This PR itself only introduces new encoding and will not cause any breakage. However, it may cause breakage when switch the wasm plugin communication scheme in future.

Related issue (if exists):

[changeset-bot]

⚠️ No Changeset found

Latest commit: 041a36d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[codspeed-hq]

CodSpeed Performance Report

Merging #11100 will not alter performance

_{Comparing quininer:r/self-desc-encode (041a36d) with main (101c3b7)¹}

Summary

✅ 140 untouched

Footnotes

1. No successful run was found on main (3ef7ce0) during the generation of this report, so 101c3b7 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[socket-security]

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action	Severity	Alert  (click "▶" to expand/collapse)
Warn		@babel/[email protected] has a Critical CVE. CVE: GHSA-67hx-6x53-jw92 Babel vulnerable to arbitrary code execution when compiling specifically crafted malicious code (CRITICAL) Affected versions: < 7.23.2; >= 8.0.0-alpha.0 < 8.0.0-alpha.4 Patched version: 7.23.2 From: crates/swc_ecma_minifier/yarn.lock → npm/@babel/[email protected] ℹ Read more on: This package | This alert | What is a critical CVE? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at [email protected]. Suggestion: Remove or replace dependencies that include known critical CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/@babel/[email protected]. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

[quininer]

This also requires a serialization test, include unknown variants.

Since I'm busy with other work, it will be a little slower.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@​babel/​traverse@​7.22.1
	@​types/​terser@​3.12.0
	has@​1.0.3
	@​swc/​counter@​0.1.3
	node-releases@​2.0.13
	hasown@​2.0.2
	path-parse@​1.0.7
	is-core-module@​2.12.1
	function-bind@​1.1.2
	@​swc/​plugin-jest@​1.5.117
	@​types/​parse-json@​4.0.0
	is-arrayish@​0.2.1
	axios@​0.21.4
	@​babel/​plugin-transform-react-jsx-development@​7.18.6
	jest-regex-util@​25.2.6 ⏵ 29.6.3
	jest-get-type@​25.2.6 ⏵ 29.6.3			+1
	escape-string-regexp@​2.0.0 ⏵ 1.0.5			-5
	@​jest/​globals@​25.5.2 ⏵ 29.7.0			+1
	@​babel/​plugin-transform-dotall-regex@​7.18.6
	@​babel/​plugin-transform-unicode-regex@​7.18.6
	@​jest/​source-map@​25.5.0 ⏵ 29.6.3
	@​babel/​plugin-transform-unicode-sets-regex@​7.22.3
	@​babel/​plugin-transform-exponentiation-operator@​7.18.6
	@​babel/​plugin-transform-reserved-words@​7.18.6
	@​babel/​plugin-transform-class-properties@​7.22.3
	@​babel/​plugin-transform-private-methods@​7.22.3
	@​babel/​plugin-transform-sticky-regex@​7.18.6
	core-util-is@​1.0.2 ⏵ 1.0.3			-1
	collect-v8-coverage@​1.0.1
	@​babel/​plugin-transform-named-capturing-groups-regex@​7.22.3
	jest-resolve-dependencies@​25.5.4 ⏵ 29.7.0	+1		+1
	@​babel/​plugin-transform-property-literals@​7.18.6
	@​babel/​plugin-transform-optional-catch-binding@​7.22.3
See 403 more rows in the dashboard

View full report