Fix "Cannot initialize Sdl from more than one thread" for tests / CI

[Cobrand]

Related to #1278

Hello all,

Many of you are waiting for a new release on crates.io, but realistically I cannot do that when the whole pipeline is failing for all platforms. The main issue that we are having is this Cannot initialize Sdl from more than one thread for tests (run them locally and you'll see, or see here).

cargo test -- --test-threads=1 is not enough for this (even though it used to be), so there has to be something else that makes the tests run on different threads. Perhaps on the rust side, perhaps on this lib's side.

Just to be clear I looked it up a little but this is a bit more complicated than expected, and I don't have the time to tackle this right now.

Once all the CI goes green, I will merge everything in the PRs that is mergeable and push another version.

Side note: I have been the sole maintainer of this lib for almost 7 years, and I would really appreciate if someone could at least help me maintain this, especially now that I have less and less time. If you are interested, please send me a message.

[Cobrand]

#1241 needs to be fixed for next release

[HollayHorvath]

Preface: On Linux it seems like that cargo-nextest is running the tests fine. Although nextest doesn't handle doc tests but it can be combined with cargo test --doc. Also it looks like that nextest uses a different threading model and it does not require to be forced to single threaded execution, but it must be installed on the CI making it slower and more complicated (I can give it a try if you'd like).

After mentioning this, here is what I found about the original issue.
I did a quick research trying to bisect which Rust release caused the tests to fail. The first version that SDL v0.35.2 compiles with is 1.65 and the tests are working fine there.

The last stable it works with is 1.66.1 (2023-01-10) and the first one it fails with is 1.67.0 (2023-02-09).

For the nightly versions it fails both with nightly-2023-01-10 (v1.66.1) and with nightly-2022-12-15 (v1.66.0) but works with nightly-2022-11-03 (v1.65) (which makes sense with the Rust release model). I kept going back on nightly and bisecting after I found a version that it works with. I found that the last working version is nightly-2022-11-04.

Now checking the commits between 2022-11-03 and 2022-11-04, the only one that seems to be relevant to me as it is affecting how concurrency on tests works is this one: libtest: run all tests in their own thread, if supported by the host

I will not pretend that I have any idea what's going on in this commit but this was the only one I found that has anything to do with tests and concurrency and is contained in this given timeframe. It's not much but it might help others solving this issue.

I will try to set up a test to see if cargo test -j1 actually runs tests concurrently.

Here is the whole log of the bisect if somebody is interested

1.60, 1.63, 1.64 - too old 1.65 - good (2022-11-03) 1.67.1 - fails (2023-02-09) 1.66.1 - good (2023-01-10)

Just for the sake of completion I tried:
1.66.0 - good (2022-12-15)
1.67.0 - fails (2023-01-26)

Now, nightly versions:
nightly-2023-01-10 - fails
nightly-2022-12-15 - fails
nightly-2022-11-03 - good (finally!)
nightly-2022-11-24 - fails
nightly-2022-11-13 - fails
nightly-2022-11-08 - fails
nightly-2022-11-05 - fails
nightly-2022-11-04 - good

[Cobrand]

Thanks for the comment, really helpful. I think we either:

• Have to find a way to run the tests non-concurrently via some kind of lock, which would wait every time to Init + Quit SDL2, without cargo flags
• Create an issue in rust-lang to have this option added