Enable EVLOSER GPU backends in hybrid mode

[tamar-dewilde]

Description

This PR allows the external EVLOSER interface to use CUDA or HIP refactorization in hybrid mode when HiOp data is stored in host memory.

Previously, use_device_ was used both to track where the HiOp data was stored and to decide whether ReSolve should use a GPU backend. Because of this, hybrid mode continued to use KLU when the HiOp data was stored on the host.

This PR separates those decisions so HiOp can keep its data in host memory while ReSolve uses the selected GPU backend.

Closes #6

@pelesh

Proposed changes

• Replace use_device_ with two separate flags:
  • input_on_device_ tracks where the HiOp input data is stored.
  • compute_on_device_ tracks whether ReSolve uses a CUDA or HIP GPU backend.
• Copy the RHS and solution between the memory used by HiOp and ReSolve.
• Synchronize updated matrix values to device memory before GPU refactorization.
• Add explicit driver options for EVLOSER with CPU KLU, CUDA GLU, CUDA RF, and HIP RF.
• Add the -evloser option to NlpSparseEx1 and set verbosity_level to 3 so the selected backend is printed for the regression test.
• Add regression tests that use FAIL_REGULAR_EXPRESSION on the Refactorization: and Use IR: output lines to reject the wrong backend configuration.
• Replace the legacy cuSOLVER LU option in NlpSparseEx4 with the external EVLOSER interface.
• Correct the duplicated use_ginkgo_cuda initialization in NlpSparseEx1 and NlpSparseEx4.

Checklist

• All tests pass (make test and make test_install). Code tested on:
  • CPU backend
  • CUDA backend
  • HIP backend
• I have manually run the relevant examples. The registered NlpSparseEx1 and NlpSparseEx2 self checks pass to six digits, and NlpSparseEx4 completes successfully. Code tested on:
  • CPU backend
  • CUDA backend
  • HIP backend
• Code compiles cleanly with flags -Wall -Wpedantic -Wconversion -Wextra.
  • The warning builds show that this PR does not introduce new warnings.
• Regression tests were updated and run.
• The new code is documented.
• The feature branch is rebased with respect to the target branch.

Further comments

Using compute_mode=auto with data in host memory continues to use KLU. Selecting compute_mode=hybrid or compute_mode=gpu enables the available GPU backend.

The previous general hybrid test was replaced with explicit tests for each supported EVLOSER backend.

NlpSparseEx4 is not currently registered as a CTest, and its existing self-check saved values do not match the current problem, so adding a registration would require separate work. The updated EVLOSER path was therefore validated manually by checking successful completion and solver status. Updating the Ex4 regression coverage is outside the scope of this PR.

[tamar-dewilde]

Validated against HiOp 0c24f62 and ReSolve develop at 6b14bd4.

CPU

• Ubuntu 22.04, GCC 11.4.0
• KLU
• HiOp full test suite, install validation, and relevant EVLOSER examples: passed

CUDA

• Ubuntu 22.04, GCC 11.4.0, CUDA 12.8
• NVIDIA RTX 2060, compute capability 7.5
• KLU, GLU, RF
• HiOp full test suite, install validation, and relevant EVLOSER examples: passed

HIP

• Frontier, PrgEnv-cray 8.6.0, CCE 18.0.1, ROCm 6.4.2
• AMD MI250X, gfx90a
• KLU, RF
• HiOp full test suite, install validation, and relevant EVLOSER examples: passed

The targeted tests confirmed the expected Refactorization: and Use IR: selections, and the self-checks passed to six digits.

Full validation logs:

• cpu-pr6-validation.log
• cuda-pr6-validation.log
• hip-pr6-validation.log

[nkoukpaizan]

Looks good to me. Tested on the HIP and CUDA backends.

[nkoukpaizan]

Is there a particular reason for use_device_ --> use_accelerator_? You might also consider something like compute_on_device_ to mirror the input_on_device_ variable.

[tamar-dewilde]

No particular reason beyond distinguishing the ReSolve backend from the input memory location. compute_on_device_ makes that clearer and mirrors input_on_device_ better. It has been renamed.