[MAINT] Reactivate GUI tests from the CI

[krzywon]

Description

This is currently a draft because I don't know if linux will handle the tests as well as Windows.

This is a comprehensive redo of the GUI tests. This includes a number of changes to fix a series of the tests, but I couldn't get them all working. Running the tests locally, I got pretty far, but need to stop where I am and move on to other things. Currently, 38 tests are labelled as skip or xfail, and four test files are not run (more below).

== 386 passed, 7 skipped, 31 xfailed, 226 warnings in 212.79s (0:03:32) ===

Things changed

• Enabled all GUI tests during the CI
• Many changes with the PyQt to PySide6 transition (I couldn't fix them all...)
• Enabled many tests previously marked as xfail and skip
• Disabled a few tests because of either PySide or underlying changes to the GUI that I couldn't fix myself.
• Moved the AddMultiEditorTest.py to a sub folder of Utilites\UnitTesting and added that directory to the pytest norecursedirs list (Loading the editor is segfaulting right now...)
• Added the Inversion perspective to the pytest norecursedirs list because of the major changes to it - most all of the tests currently fail
• Added the ParticleEditor perspective to the pytest norecursedirs list because it is still experimental and many of its tests fail
• Fixed some tests (that I could)

Fixes #2322

How Has This Been Tested?

Ran python -m pytest --disable-warnings -p no:logging .\src\sas\qtgui\ from the base sasview directory.

Review Checklist:

[Documentation (check at least one)

• There is nothing that needs documenting
• Documentation changes are in this PR
• There is an issue open for the documentation (link?)

Installers

• There is a chance this will affect the installers, if so
  • Windows installer (GH artifact) has been tested (installed and worked)
  • MacOSX installer (GH artifact) has been tested (installed and worked)
  • Wheels installer (GH artifact) has been tested (installed and worked)

Licensing (untick if necessary)

• The introduced changes comply with SasView license (BSD 3-Clause)

[backmari]

@krzywon I wasn't able to reproduce the segfault locally on Ubuntu. I started fixing some other things, but I can add those in a separate PR if you are successful in making the pipeline pass 🤞

[krzywon]

@backmari - I figured out the segfault issue (thanks to a not-be-named AI agent). I had to set to the QT_QPA_PLATFORM environment variable to offscreen to allow Ubuntu to run the gui elements while in headless mode.

Currently, the CI 'passes', but there are still a few segfaults that are being swallowed during the GUI tests. Once those are fixed, feel free to see if your changes conflict with what I've done and feel free to push here, if you can.

[krzywon]

Okay, @backmari. I think the GUI tests should be ready. Assuming this last CI passes, feel free to add your changes here.

[backmari]

@krzywon These were just some minor changes, probably nothing that will help with the segfaults (and hopefully won't break anything.)
I had problems with the tests creating an empty config file that would cause an error the second time I would run the test suite, so I mocked away saving a config file in two places.

[krzywon]

I had problems with the tests creating an empty config file that would cause an error the second time I would run the test suite, so I mocked away saving a config file in two places.

I had the same issue and couldn't figure out where it was coming from. Thank you!

[DrPaulSharp]

This all looks good, it'll be great to get these tests back up and running. Alongside the specific comments:

• The command suggested in the PR does not work properly - specifically the -p no:logging disables the caplog fixture which causes errors in the guiUtils tests (alongside other tests skipped due to the use of caplog). Using --show-capture=no instead should have the required effect without disabling caplog
• Using the command python -m pytest --disable-warnings --show-capture=no .\src\sas\qtgui\ on my Windows machine, I find: 367 passed, 3 skipped, 29 xfailed, 129 warnings in 146.50s (0:02:26)
• We should look into a way of closing dialogs generated by the test suite at a test or module level. They are correctly closed at the end of the session, but hang around for a while and may persist if an error occurs in the test suite.
• We need to work out the sequencing of this PR and PRs #3881 and #3888.

[krzywon]

@DrPaulSharp, thanks for the review. I'll respond to your comments soon, but when I was working on this, I prioritized getting things working and passing, not understanding whey I had to change the tests. I'll work on the proposed changes soon.

As far as merge order, I think this one -> #3888 -> #3881 would be best. Each PR should be rebased to main after others have been merged to be sure the unit tests continue to pass.

[DrPaulSharp]

@DrPaulSharp, thanks for the review. I'll respond to your comments soon, but when I was working on this, I prioritized getting things working and passing, not understanding whey I had to change the tests. I'll work on the proposed changes soon.

As far as merge order, I think this one -> #3888 -> #3881 would be best. Each PR should be rebased to main after others have been merged to be sure the unit tests continue to pass.

This all makes sense. @jellybean2004 are you happy with the proposed merge order and process here?

[krzywon]

My latest test results, using periodictable v2.1.0 which fixes a large number of the deprecation warnings:

=== 366 passed, 3 skipped, 30 xfailed, 20 warnings in 95.80s (0:01:35) ====

[DrPaulSharp]

Ok, I'm still getting a lot of warnings after upgrading periodictable: 366 passed, 3 skipped, 30 xfailed, 129 warnings in 164.43s (0:02:44) but that aside, I think we can merge.

We need to raise an issue on the graphs and dialogs that are kept open throughout the test suite.

[DrPaulSharp]

IMPORTANT - lines 203-207 of pyproject.toml needs to be changed to:

testpaths = [
    "src",
    "tests",
]

otherwise running pytest without specifying a directory tries to run tests in sascalc that don't work.

Do we want to run the GUI tests in the CI as well? At present we only run the tests in the test directory.

[krzywon]

Do we want to run the GUI tests in the CI as well? At present we only run the tests in the test directory.

That's the point of this PR. Our CI now runs against src/sas/qtgui/ in line 336 with these changes. See for reference: https://github.com/SasView/sasview/actions/runs/22691197363/job/65787668530. If the tests failed, the CI will gate (see previous failed CIs from this PR, https://github.com/SasView/sasview/actions/runs/22101402544/job/63872286307).

I'm planning to merge this soon and will add two issues once I've done that:

1. GUI tests need to properly close all test windows (widget.close() or equivalent).
2. Running pytest against the base directory throws multiple errors.

[DrPaulSharp]

Ahh yes, I was thinking about the existing test section of the CI and forgot that you'd added the new test GUI section.

For reference, the issues addressing outstanding points are #3891 and #3892.

[llimeht]

@krzywon just noticed this now (sorry) - setting DISPLAY like this here but allowing xvfb-run to pick its own display number with -a is fragile. -a is for automatically determining which display to use, starting at :99. It also should be unnecessary as xvfb-run sets DISPLAY on its own and sets it to the value of the display that it has created. It should be safe(r) to remove this when next touching the code.

[krzywon]

I'll open an issue to remove it. This was something I tried after reading the commented link, but wasn't sure it was doing anything, and based on what you're suggesting, it likely isn't.