feat(pydantic-validation): pydantic config validation

[AWarno]

pydantic config validation with extra field not permitted

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[AWarno]

When we have e.g. typo:

Traceback (most recent call last):
  File "/opt/venv/bin/eval-factory", line 7, in <module>
    sys.exit(run_eval())
             ^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/entrypoint.py", line 188, in run_eval
    run(args)
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/entrypoint.py", line 150, in run
    evaluate(eval_cfg=eval_cfg, target_cfg=target_cfg, metadata=metadata_cfg)
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/evaluate.py", line 73, in evaluate
    evaluation = validate_configuration(run_config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/input.py", line 407, in validate_configuration
    check_task_invocation(run_config)
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/input.py", line 245, in check_task_invocation
    framework_evals_mapping, _, all_evals_mapping = get_available_evaluations()
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/input.py", line 206, in get_available_evaluations
    ) = _get_framework_evaluations(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/input.py", line 153, in _get_framework_evaluations
    ) = get_framework_evaluations(def_file)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/nemo_evaluator/core/input.py", line 127, in get_framework_evaluations
    evaluation = Evaluation(
                 ^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/pydantic/main.py", line 250, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 1 validation error for Evaluation
config.params.n_repeats
  Extra inputs are not permitted [type=extra_forbidden, input_value=1, input_type=int]
    For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden

[AWarno]

/ok to test d85da06

[AWarno]

/ok to test 2eed38c

[AWarno]

/ok to test ea15b2f

[agronskiy]

This sounds legit to me!

[agronskiy]

nit: feels like this belongs to the top level import up there (you already have from pytdantic). And same for logging.

[AWarno]

/ok to test 4de6d2d