feat: Add average parameter to Accuracy for per-label multilabel output - #3810
feat: Add average parameter to Accuracy for per-label multilabel output#3810zongyang078 wants to merge 7 commits into
Conversation
Add average parameter to Accuracy, aligning its API with Precision/Recall for multilabel classification. When average=False and is_multilabel=True, compute() returns a per-label accuracy tensor instead of a scalar. Fixes pytorch#513
…ment - cast() the num_correct accumulator before .item() in compute(), matching the pattern Precision already uses for the same int|Tensor duality - drop a comment that just restated the line below it - test_running_average.py: compare _num_correct in a type-agnostic way, since it may now be a plain int (right after reset(), before update()) instead of always a tensor Fixes pytorch#513
|
My bad I missed the updates |
| :class:`~ignite.engine.engine.Engine`'s ``process_function``'s output into the | ||
| form expected by the metric. This can be useful if, for example, you have a multi-output model and | ||
| you want to compute the metric with respect to one of the outputs. | ||
| subset_accuracy: if True (default), computes subset accuracy for multilabel data |
There was a problem hiding this comment.
I dont actually like this name its confusing it does not tell what it does by just its name, maybe we can try label_average or something similar I am not sure.
There was a problem hiding this comment.
Renamed to per_label: bool = False — polarity now matches what it returns (per_label=True → per-label tensor).
|
Also I feel many of the tests are redundant and can be removed or be paramatized. Sorry for late response, i missed the updates on the pr |
… tests Renames the parameter so its True/False polarity matches what it returns (per-label tensor vs the existing scalar), instead of naming it after only one side of the toggle. Also collapses the all_correct/all_wrong/ label_with_no_positives tests into one parametrized test, and drops the module-level engine-integration test that duplicated TestDistributed.test_integration_multilabel_per_label's gloo_cpu coverage.
Merged the all_correct/all_wrong/no_positives tests into one parametrized test, and dropped the duplicate module-level engine test (verified TestDistributed's gloo_cpu case already covers the same reset-per-epoch scenario by default). |
|
@vfdev-5 LGTM |
2 participants
Fixes #513
Description:
Add
averageparameter toAccuracyfor multilabel classification, aligning its API withPrecision/Recallwhich already support per-label output via_BasePrecisionRecall.When
average=Falseandis_multilabel=True,compute()returns a per-label accuracy tensor of shape(num_labels,)using elementwise comparison, rather than the default subset accuracy scalar. Default behavior (average=None) is unchanged.Prior attempts (PR #516, PR #542) stalled in 2019 over API design — both proposed a
labelwise=Trueboolean flag, which was rejected. This PR uses theaverageparameter convention already established byPrecision/Recall.Scoped to not touch
Accuracy's internal structure (no_prepare_output), staying orthogonal to #3568 / #3610.Also updates two pre-existing tests (
test_accuracy.py::test_accumulator_device,test_running_average.py::test_integration_batchwise) that assumed_num_correctis always a tensor right afterreset()— it's now a plainintuntil the firstupdate(), matching theint | torch.TensorpatternPrecision/Recallalready use for_numerator/_denominator.Check list: