Skip to content

Commit 14ead0c

Browse files
polyhbpolyhb
committed
🚀(init) bench main ts framework
Added to git the work I have done until now. I tried to test the main frameworks that we could use in Polyconseil and compared them. The test cases maybe a bit too simple but still reveal some pain points in the frameworks.
0 parents  commit 14ead0c

File tree

145 files changed

+23794
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

145 files changed

+23794
-0
lines changed

‎.gitignore

+57
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
### Node template
2+
.DS_Store
3+
# Logs
4+
logs
5+
*.log
6+
npm-debug.log*
7+
8+
# Runtime data
9+
pids
10+
*.pid
11+
*.seed
12+
13+
# Directory for instrumented libs generated by jscoverage/JSCover
14+
lib-cov
15+
16+
# Coverage directory used by tools like istanbul
17+
coverage
18+
19+
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
20+
.grunt
21+
22+
# node-waf configuration
23+
.lock-wscript
24+
25+
# Compiled binary addons (http://nodejs.org/api/addons.html)
26+
build/Release
27+
28+
# Dependency directory
29+
# https://docs.npmjs.com/misc/faq#should-i-check-my-node-modules-folder-into-git
30+
node_modules
31+
.npmrc
32+
*.log
33+
34+
# Typings
35+
typings/
36+
37+
# Typescript
38+
/**/*.js
39+
/**/*.js.map
40+
test/**/*.js
41+
test/**/*.js.map
42+
43+
# Test
44+
/.tmp
45+
/.nyc_output
46+
47+
# IDE
48+
.vscode
49+
.idea
50+
51+
# Project
52+
/public
53+
**/dist
54+
55+
#env
56+
.env.local
57+
.env.development

‎README.md

+199
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
# Poly Typescript Backend Testing
2+
3+
The goal of this project is to test the limits of the typescript backend frameworks. The project will be a simple CRUD API with a single entity. Each framework will be tested for the following:
4+
- Type safety
5+
- Community support
6+
- Developer support
7+
- Developer experience
8+
- Documentation
9+
- Performance
10+
11+
The type safe is a major factor in the evaluation of the frameworks. Most of our projects in Poly were written in typescript, and while we do like the language, we have had some issues with type safety on several backend frameworks. We hope to find a framework that is both completely type safe and has a good developer experience.
12+
13+
## Tested libraries
14+
15+
- [x] [hono](https://www.npmjs.com/package/hono)
16+
- [x] [elysia](https://www.npmjs.com/package/elysia)
17+
- [x] [express-zod-api](https://www.npmjs.com/package/express-zod-api)
18+
- [x] [encore.ts](https://www.npmjs.com/package/encore.ts)
19+
- [x] [fastify](https://www.fastify.io/)
20+
- [x] [tsed](https://tsed.dev/)
21+
- [x] [adonisJs](https://adonisjs.com/)
22+
- [x] [nestjs](https://nestjs.com/)
23+
- [ ] [orpc](https://github.com/unnoq/orpc)
24+
- [ ] [koa](https://www.npmjs.com/package/koa)
25+
- [ ] [loopback](https://loopback.io/doc/en/lb4)
26+
- [ ] [sails](https://sailsjs.com/)
27+
28+
29+
### Test Plan
30+
31+
Each project will try to have theses features:
32+
33+
- An auth middleware that will add a user to the request / context so that routes can access it (should be typed)
34+
- Route `authors/:id` with a `PUT` method that takes a body and a param and will return an author with with combination of the body and the param and the user from the auth middleware (everything should be typed)
35+
- A properly constructed openapi file
36+
- The open api file should be easily configurable
37+
- The type between the openapi file and the route should be checked
38+
- A `/reference` route that will serve a scalar ui website thanks to the openapi file
39+
40+
41+
## Development feedback
42+
43+
### Hono
44+
45+
I need to use a third party libraries for validation, a complete type safety:
46+
47+
- :no_entry: The [hono-openapi](https://hono.dev/examples/hono-openapi) does not type check the result types and the schema given in the openapi file.
48+
49+
- :white_check_mark: The [Zod OpenAPI](https://hono.dev/examples/zod-openapi) generates the openapi file from the zod schema and the response type is checked between the route and the schema which is a good thing, but the library is a bit verbose.
50+
51+
52+
### Elysia
53+
54+
Easy to setup / use and type safe everywhere.
55+
56+
It looks like scalar may have a problems with 'examples' in params.
57+
58+
example of openapi file given by Elysia:
59+
```JSON
60+
"parameters": [
61+
{
62+
"description": "The ID of the author",
63+
"examples": [
64+
"12"
65+
],
66+
"schema": {
67+
"type": "string",
68+
"minLength": 1
69+
},
70+
"in": "path",
71+
"name": "id",
72+
"required": true
73+
}
74+
],
75+
```
76+
77+
example of what is given by Hono
78+
```JSON
79+
"parameters": [
80+
{
81+
"schema": {
82+
"type": "string",
83+
"minLength": 3,
84+
"description": "The ID of the user",
85+
"example": "1212121"
86+
},
87+
"required": true,
88+
"name": "id",
89+
"in": "path"
90+
}
91+
],
92+
```
93+
94+
Everywhere else the examples field works fine but not in the params, problem with scalar ?
95+
96+
Elysia may have an older version of Typebox as dependence, so if our project wants to use the latest release of Typebox, we will have to create an alias in package.json to use this version. (Also, keep it mind that it wont be compatible with the Elysia controller types)
97+
98+
### express-zod-api
99+
100+
Distinction between the body, the params and body are not clear.
101+
102+
The hot reloading is not working properly ?
103+
104+
Based on express so very slow.
105+
106+
### Encore.ts
107+
108+
As of 24/12/2024, the framework looks a bit incomplete:
109+
- Cannot add objects to context
110+
- OpenAPI is still experimental
111+
112+
Look promising.
113+
114+
### Koa
115+
116+
Terrible doc
117+
Stopped testing after trying to find a library that would properly type the routes, found one with 60 github stars but way less complete than the express zod api one.
118+
[koa-zod-router](https://github.com/jakefenley/koa-zod-router#readme)
119+
120+
121+
### Fastify
122+
123+
Hard time to find how to properly type a decorator. Cannot ?
124+
125+
### Tsed
126+
127+
Easy to use, but the it suffers from the same problem as nestjs, the decorators and the typescript are not typed together.
128+
129+
5 min to shut down the server ?
130+
131+
based on express so very slow.
132+
133+
### AdonisJs
134+
135+
Adding context is not easy and truly typed.
136+
137+
OpenAPI is not easily configurable (BY COMMENT !!) AND not typed safely.
138+
139+
Some requests are failing when benching it.
140+
141+
142+
### NestJs
143+
144+
Was really easy to setup and make it work.
145+
146+
Personal preferences but chaining decorators is not easily readable.
147+
148+
@ApiResponse is not typed with the return type of the route.
149+
150+
Doc looks great
151+
152+
You can give custom "context" thanks to the custom param decorator but the type is not well checked.
153+
154+
based on express so very slow.
155+
156+
## Results
157+
158+
| | body typed | param typed | context typed | open api | open api typed end to end | front end client | perf (avg request) |
159+
|-----------------|--------------------|--------------------|--------------------|--------------------|---------------------------|--------------------|--------------------|
160+
| hono | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | 28.4k |
161+
| elysia | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | 45.3k |
162+
| express-zod-api | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | 5.2k |
163+
| encore.ts | :white_check_mark: | :white_check_mark: | :no_entry: | :no_entry: | :no_entry: | :white_check_mark: | 8.5k |
164+
| fastify | :white_check_mark: | :white_check_mark: | :no_entry: | :white_check_mark: | :white_check_mark: | :no_entry: | 10.3k |
165+
| tsed | :white_check_mark: | :white_check_mark: | :no_entry: | :white_check_mark: | :no_entry: | :no_entry: | 3.4k |
166+
| adonisJs | :white_check_mark: | :white_check_mark: | :no_entry: | :white_check_mark: | :no_entry: | :no_entry: | 6k |
167+
| nestjs | :white_check_mark: | :white_check_mark: | :no_entry: | :white_check_mark: | :no_entry: | :no_entry: | 4.3k |
168+
169+
170+
### Remarks
171+
172+
We can see that express based frameworks are slower than the others.
173+
174+
### The express like frameworks
175+
176+
We do have a three frameworks that achieve the type safety we were looking for and are express like (chaining functions, middleware, etc):
177+
178+
- hono
179+
- elysia
180+
- express-zod-api
181+
182+
[Hono](https://github.com/honojs/hono) has 20k stars, and it looks like it has a good community / plugins. Also the performance is good for a javascript framework. In order to have the complete type safety, we need to use a third party library [Zod OpenAPI](https://hono.dev/examples/zod-openapi) which is a bit verbose.
183+
184+
[Elysia](https://github.com/elysiajs/elysia) has 11k stars, has a bit less community than hono. It has a very good performance, the type safety is complete, and completely integrated in the framework. It was initially made for Bun but it is now an agnostic framework.
185+
186+
[Express-zod-api](https://github.com/RobinTail/express-zod-api) has 600 stars, has mediocre performance because it is based on express. The type safety is complete. The advantage of this framework is that it is based on express, we can use the insane amount of plugins available for express.
187+
188+
### The more "architecture opinionated" frameworks
189+
190+
We can see that when a framework is more architecture opinionated, the type safety is not complete. The frameworks using decorators like nest.js cannot achieve a complete type safety.
191+
192+
I find [encore.ts](https://github.com/encoredev/encore) a promising framework. It has 8.3k stars, the type safety is not complete yet and some features are missing but it looks like it can provides a good developer experience.
193+
194+
195+
### Result are a bit biased
196+
197+
Some of these frameworks are huge, and I did not graded them on all the features they offer. Obviously complete type safety and performance is not the only thing we have to look for when we are choosing a typescript backend server.
198+
199+
Also the bench is done of my computer and is based on a simple route. The performance can be different with a more complex route, or with a database, with more features, etc.

‎_bench/.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# deps
2+
node_modules/

‎_bench/.prettierrc

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"useTabs": false,
3+
"trailingComma": "none",
4+
"semi": true,
5+
"singleQuote": true,
6+
"tabWidth": 2,
7+
"printWidth": 100,
8+
"overrides": [
9+
{
10+
"files": "*.ts"
11+
}
12+
]
13+
}

‎_bench/README.md

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
To install dependencies:
2+
3+
```sh
4+
bun install
5+
```
6+
7+
To run:
8+
9+
```sh
10+
bun run dev
11+
```
12+
13+
open http://localhost:3000

‎_bench/bun.lockb

64 KB
Binary file not shown.

‎_bench/eslint.config.mjs

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
import eslint from '@eslint/js';
2+
import tseslint from 'typescript-eslint';
3+
import unusedImports from 'eslint-plugin-unused-imports';
4+
import eslintPluginPrettierRecommended from 'eslint-plugin-prettier/recommended';
5+
6+
export default tseslint.config({
7+
files: ['**/*.ts', '**/*.tsx'],
8+
extends: [
9+
eslint.configs.recommended,
10+
...tseslint.configs.recommended,
11+
...tseslint.configs.strict,
12+
...tseslint.configs.stylistic,
13+
eslintPluginPrettierRecommended
14+
],
15+
languageOptions: {
16+
ecmaVersion: 'latest',
17+
parserOptions: {
18+
project: ['./tsconfig.json'],
19+
tsconfigRootDir: import.meta.dirname
20+
}
21+
// globals: {
22+
// ...globals.recommended,
23+
// Stripe: true,
24+
// cy: true,
25+
// Cypress: true,
26+
// },
27+
},
28+
plugins: {
29+
'unused-imports': unusedImports
30+
},
31+
rules: {}
32+
});

‎_bench/package.json

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"name": "autocannon",
3+
"type": "commonjs",
4+
"scripts": {
5+
"dev": "bun src/index.ts",
6+
"lint": "eslint ./src --max-warnings 0 --fix"
7+
},
8+
"dependencies": {},
9+
"devDependencies": {
10+
"@types/autocannon": "^7.12.5",
11+
"@types/bun": "^1.1.14",
12+
"autocannon": "^8.0.0",
13+
"eslint": "^9.16.0",
14+
"eslint-config-prettier": "^9.1.0",
15+
"eslint-plugin-prettier": "^5.2.1",
16+
"eslint-plugin-unused-imports": "^4.1.4",
17+
"typescript-eslint": "^8.17.0"
18+
}
19+
}

‎_bench/src/index.ts

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
import autocannon from 'autocannon';
2+
3+
async function run() {
4+
const url = 'http://localhost:3000/authors/12';
5+
6+
console.log('Running benchmark...');
7+
8+
const instance = await autocannon({
9+
url,
10+
connections: 100, // default
11+
duration: 10, // default
12+
headers: {
13+
'Content-Type': 'application/json'
14+
},
15+
body: JSON.stringify({ age: 30 }),
16+
method: 'PUT'
17+
});
18+
19+
console.log('Average requests', instance.requests.average);
20+
}
21+
22+
run();

‎_bench/tsconfig.json

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"compilerOptions": {
3+
"strict": true,
4+
"jsx": "react-jsx",
5+
"jsxImportSource": "hono/jsx",
6+
"moduleResolution": "bundler",
7+
"module": "ES2022"
8+
}
9+
}

‎adonisjs/.editorconfig

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# http://editorconfig.org
2+
3+
[*]
4+
indent_style = space
5+
indent_size = 2
6+
end_of_line = lf
7+
charset = utf-8
8+
trim_trailing_whitespace = true
9+
insert_final_newline = true
10+
11+
[*.json]
12+
insert_final_newline = unset
13+
14+
[**.min.js]
15+
indent_style = unset
16+
insert_final_newline = unset
17+
18+
[MakeFile]
19+
indent_style = space
20+
21+
[*.md]
22+
trim_trailing_whitespace = false

0 commit comments

Comments
 (0)