Multithreading support

[RastislavMirek]

This might be a feature request or a question depending on the feature's state.

Can Leto handle multiple GraphQL requests to the endpoint in parallel? Or does the second request need to wait for the first to complete?

Say we first requests coming and taking 5 seconds because it is computationally expensive. After 1 second, another request comes. Does the second request wait for 4 seconds before it is handled?

If parallel processing is not possible, I would like to request this feature.

[juancastillo0]

Hi, thanks for your question!

Sorry for the delay I was not available for a few days.

Concurrent processing works by default since the resolvers can be async, they would use Dart's threads to schedule work. Within a single request, mutations execute resolvers in series following the GraphQL specification and queries execute them concurrently. But multiple different requests will be handled concurrently in a single core (using Dart's scheduling).

Parallel processing (multiple cores) is also possible, but you need to make sure the necessary state is share if it needs to be shared or just create a bunch of executors and schedule each request in parallel with Dart isolates. If the server is horizontally scalable you could probably use an executor for each isolate and there should not be any problems.

Also you probably would want to investigate how shelf (or the web server you are using) would interact with multiple cores. I have not benchmark these changes to make sure they would improve performance significantly.

Please let me know if I missed something or if there is something more I could help you with.