[Perf Test] DirectML "performance_preference" option non-functional due to catch22 bug

[virajwad]

Describe the issue

When running onnxruntime_perf_test.exe -h option, we see for DirectML there is a "performance_preference" option. The selectable values can be 'minimum_power' or 'high_performance'

I try out the following command:

onnxruntime_perf_test.exe -e dml -i "performance_preference|minimum_power" -m times -r 1 <path to onnx model>

But trying to use the option value 'minimum_power' leads to an error. The error says to instead use 'minimal_power'

When using 'minimal_power' actually there is a different error specifying that this is not an allowable option. When I check the code in the dml_provider_factory.cc file, it looks like it wants "minimum_power" instead.

This seems to be a catch22 situation where I can't pick either "minimum_power" or "minimal_power" since there are 2 different code checks, each check wanting the other option :)

To reproduce

Any onnx model, tried on a built onnxruntime 1.21.0 and used onnxruntime_perf_test.exe

Urgency

Low Urgency - Just want to report this bug since it is a non-functional option

Platform

Windows

OS Version

Windows 11

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

Release 1.21.0

ONNX Runtime API

C++

Architecture

X64

Execution Provider

DirectML

Execution Provider Library Version

DirectML 1.15.4

[fdwr]

This seems to be a catch22 situation where I can't pick either "minimum_power" or "minimal_power" since there are 2 different code checks, each check wanting the other option :)

🤔 Which should change to which?

[virajwad]

@fdwr
Might be easiest to use "minimum_power" (since that is also the option in -h)

The dml_provider_factory.cc file looks for the "minimum_power" string:

Looks like 2 changes would need to be changed in ort_test_session.cc ->

https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/test/perftest/ort_test_session.cc#L421-L428

Not sure if it needs to be changed anywhere else, at least that is where I see the error!

[fdwr]

Might be easiest to use "minimum_power" (since that is also the option in -h)
Looks like 2 changes would need to be changed in ort_test_session.cc

Makes sense, as it's safer to update the tool according to the product code than the other way (in case ParsePerformancePreference is called by someone else outside the perf tool). Thanks for the catch. Willing to update it in a fork and tag me?