Remove duplicate addition of callbacks/make optional

[romeokienzler]

in cli_tools.py:

        parser.add_lightning_class_args(StateDictAwareModelCheckpoint, "ModelCheckpoint")
        parser.set_defaults({"ModelCheckpoint.filename": "{epoch}", "ModelCheckpoint.monitor": "val/loss"})

        parser.add_lightning_class_args(StateDictAwareModelCheckpoint, "StateDictModelCheckpoint")
        parser.set_defaults(
            {
                "StateDictModelCheckpoint.filename": "{epoch}_state_dict",
                "StateDictModelCheckpoint.save_weights_only": True,
                "StateDictModelCheckpoint.monitor": "val/loss",
            }
        )

We need to check if it makes sense to have both callbacks added at the same time, also need to remove those if the config already provides those

@paolofraccaro

[paolofraccaro]

@romeokienzler We also have the issue in here that we add this with no choice of disabling (as far as I can tell), but more importantly the 'mode' (e.g. min/max) is not something we can change from here. Min (the default) is OK if we refer to loss, but not if we referred to something different. We also assume that there will always be a "val/loss" item to monitor. For object detection for example, I am monitoring MAP that has to go up.

    
        monitor = "val_map"
        mode = 'max'
        
        parser.add_lightning_class_args(StateDictAwareModelCheckpoint, "ModelCheckpoint")
        parser.set_defaults({"ModelCheckpoint.filename": "{epoch}", "ModelCheckpoint.monitor": monitor, "ModelCheckpoint.mode": mode})

        parser.add_lightning_class_args(StateDictAwareModelCheckpoint, "StateDictModelCheckpoint")
        parser.set_defaults(
            {
                "StateDictModelCheckpoint.filename": "{epoch}_state_dict",
                "StateDictModelCheckpoint.save_weights_only": True,
                "StateDictModelCheckpoint.monitor": monitor,
                "StateDictModelCheckpoint.mode": mode,
             }
        )

[blumenstiel]

@paolofraccaro @romeokienzler Currently you can overwrite them in the config on the highest level, not in the Trainer. Not sure if you can deactive them (This is also quite annoying for Iterate).

StateDictModelCheckpoint:
     filename: "..."
...

But I agree it is not intuitve and not explained by terratorch afaikt. It might be better to define default callbacks in general (checkpoints, LR montior, rich progress bar) and add them if the use does not overwrite them.

[Joao-L-S-Almeida]

@romeokienzler @blumenstiel @paolofraccaro Maybe the solution proposed in #454 is the way to solve it.