Add support for Rails built-in Parallel Testing

[zoras]

Use Rails 6 built-in parallelizer API for running parallel RSpec executors

https://edgeguides.rubyonrails.org/testing.html#parallel-testing

rails/rails#31900 (comment)

[JonRowe]

We'd love to support parallel tests with RSpec itself, its a rather large task however. If we get parallelisation support into rspec-core you can be sure we'll integrate with the rails helpers.

[jrochkind]

Note that linked rspec/rspec#7 says:

There are bunch of gems that allow for parallel execution, but all of these are process based and not thread based.

I believe the parallel testing now built into Rails can be process-based or thread-based, and by default is process-based.

Is supporting Rails process-based parallel testing less of a challenge? It looks like one of the main things Rails does for you when supporting this is setting up a separate database for each test worker, as well as orchestrate getting each worker to run different tests and aggregating the results.

[JonRowe]

In the short term all we could offer is to help integrate 3rd party parallel testing with Rails tools.

Our own parallel journey requires changes to rspec-core which are mostly identical for process or thread based.

Please note there are parallel testing extensions for RSpec already!

[p8]

Discourse now has a solution based on parallel_tests discourse/discourse#7778

[iainbeeston]

Is there any update on supporting the activesupport parallelize helper in rspec-rails, now that rails 6 has been released?

[iainbeeston]

Sorry, I meant to include a link with more details:

rails/rails#31900

[JonRowe]

No, RSpec itself needs to support parallelisation in order for it to supported

[iainbeeston]

Sorry @JonRowe - can I please clarify something?

rspec-core needs updating even for multi-process parallelisation, right? (not just multi-threaded parallelisation as described in rspec/rspec#7, which I would assume is more challenging)

[JonRowe]

The main rspec-core runner doesn't support parallelisation, so it has no means of collating results from workers, be they threads or processes. Process based parallelisation is simpler than thread based due to well, the GIL on MRI rubies and lack of 100% thread safety within the other gems. (I'm fairly certain for example that there are threaded bugs lurking within the mocks code).

My ideal plan for taking us to parallelisation is to use the "fork" model that the bisect runner uses, maybe with a fall back to a "shell" model (essentially processes) but I haven't had the time to work much on it.

[krtschmr]

to summarize: regardless if thread or processes: rspec-core isn't supporting it yet, so if we run a rails 6.0 application and use rspec, we won't benefit from it. correct?

[JonRowe]

The helpers are there to support parallelisation, rspec on its own doesn't support parallelisation so you'll need an extension gem to benefit from it.

[p8]

My ideal plan for taking us to parallelisation is to use the "fork" model that the bisect runner uses, maybe with a fall back to a "shell" model (essentially processes) but I haven't had the time to work much on it.

@JonRowe Would you copy all bisect code and renaming the namespace to parallel and make the required changes?

[JonRowe]

No I'm writing it from first principles using the bisect runner as a guide and some other external gems.