Model name formatting during evaluation loading

[sjoerdgunneweg]

Model loading fails due to name formatting mismatch during evaluation

Error

huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name'

The given config files (eg. src/modernvbert/contrastive_training/modernvbert/doc/config/colmodel_text_300k.yaml)
have no model_name_or_path parameter.

Resulting in the if-else statement in lines 109-112 of src/modernvbert/contrastive_training/evaluate.py to add "/final/" to the output dir:

huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/lustre/fsn1/projects/rech/nwd/unj98sd/models/flagship_postrainings/colmodernvbert_text_300k/final'. Use `repo_type` argument if needed.

Workaround

add model_name_or_path: ModernVBERT/colmodernvbert to config file.

New issue after workaround!

Lines 114-117 in src/modernvbert/contrastive_training/evaluate.py, manipulate the model name by splitting on / and only keeping the final component (colmodernvbert)

This breaks valid HuggingFace identifiers of the form:

organization/model_name

Resulting in:

Value error, Model name must be in the format 'organization/model_name'

Workaround

replacing lines 114-117 by:

name = model_name_or_path