How to compose two metrics?

[celsofranssa]

Currently, I am computing two metrics at the validation_epoch_end, as shown in the code snippet below.

class LTModel(pl.LightningModule):

    def __init__(self, hparams):
        super(LTModel, self).__init__()
        self.entity_cls_head = # entity classifier 
        self.relation_cls_head = # relation classifier

        # metrics
        self.entity_metrics = self._get_metrics(prefix="entity_")
        self.relation_metrics = self._get_metrics(prefix="relation_")

        # loss
        self.loss = ...

    def _get_metrics(self, prefix):
        pass
        return MetricCollection(
             metrics={
                 "Wei-F1": F1(num_classes=self.hparams.num_entities, average="weighted"),
                 ...
             },
             prefix=prefix)

However, I would like to aggregate these two metrics to form a single one.
Then, is there an example I can use to achieve this goal?

[Borda]

To aggregate multiple metrics into one output, TorchMetrics provides the powerful MetricCollection class. This class allows you to wrap several metric objects into a single callable metric, enabling you to compute and log many metrics together and, if needed, manually aggregate their computed values.
If you want a single aggregated value (e.g., average or weighted sum of outputs), you can compute each metric, then combine their results as needed.
Here is a minimal code sample for combining two metrics and aggregating their values:

import torch
from torchmetrics import MetricCollection, F1

# Example: Define two metrics
entity_metric = F1(num_classes=NUM_ENTITIES, average="weighted")
relation_metric = F1(num_classes=NUM_RELATIONS, average="weighted")

# Compose with MetricCollection
metrics = MetricCollection({
    "entity_f1": entity_metric,
    "relation_f1": relation_metric,
})

# Forward pass (in validation_step or validation_epoch_end)
results = metrics(preds, targets)  # dict: {'entity_f1': ..., 'relation_f1': ...}

# Aggregate (e.g., mean)
aggregate_f1 = torch.mean(torch.stack(list(results.values())))
print("Aggregated F1:", aggregate_f1.item())

This approach gives you flexibility to aggregate in any way you want, after the metric results are computed as a dictionary. You can log both individual and aggregated values as needed.
For more complex metric compositions, TorchMetrics supports arithmetic operations directly on metrics (since v1.8), allowing you to add or average metrics with simple Python syntax.