fix(afrixnli): use Jinja braces in prompt_1 doc_to_text so premise and hypothesis are substituted

[AmosBunde]

What this fixes

afrixnli prompt_1 declared an inline doc_to_text using Python .format() brace syntax ({premise}, {hypothesis}). doc_to_text is rendered through Jinja2, where single braces are literal. The premise and hypothesis were never substituted, so every document in the task produced the same constant prompt and the model scored a three-way multiple choice against an empty stem.

Evidence

Run with --log_samples on afrixnli_eng_prompt_1, Qwen/Qwen2.5-0.5B, --limit 200, CPU. English is used as a control so that "the model does not know the language" cannot explain the result.

	documents	distinct doc_hash	distinct prompt_hash
before	200	200	1
after	200	200	200

Two hundred distinct documents rendered one identical prompt. After the fix they render two hundred distinct prompts.

The rendered arg_0 before the fix:

What does not change, stated up front

Accuracy and F1 are identical before and after: acc 0.3300 (± 0.0333), f1 0.1638, on both eng and swa. Qwen2.5-0.5B collapses to a single label in both conditions, so the score cannot demonstrate the fix. A weighted F1 of 0.1638 is close to the 1/6 that always-predict-one-class yields on a balanced three-class set, which is what a constant prompt guarantees but is also what this model does anyway at this size.

Runtime is the independent confirmation that the prompts really did change: the same 200-document run took 3 seconds before and 38 minutes after, since the prompts now carry the actual premise and hypothesis.

I do not have GPU access to run a model large enough to show a score delta. If a maintainer can run this at around 7B, I would expect the fixed task to separate from chance and the broken one not to.

Changes

• lm_eval/tasks/afrixnli/gen_utils.py: prompt_1 now uses {{premise}} and {{hypothesis}}, matching how prompt_3, prompt_4 and prompt_5 already write them. prompt_2 is unaffected as it has no inline doc_to_text.
• Regenerated the affected task YAMLs with the repo's own generator rather than editing them by hand:

python gen_utils.py --output-dir ./direct    --mode prompt_1 --overwrite
python gen_utils.py --output-dir ./translate --mode prompt_1 --overwrite

• 18 files under direct/prompt_1/, 17 under translate/prompt_1/, 35 tasks total.
• Also cleared a pre-existing ruff PLC0206 in gen_utils.py (for lang in languages.keys() to .items()). This is unrelated to the bug, but the pre-commit hook fails on it and therefore blocks any change to that file. Regenerating after the lint fix produces byte-identical YAMLs.

Notes for reviewers

1. Alternative approach. lm_eval/tasks/afrixnli/direct/prompt_1/utils.py already contains a correct doc_to_text function, but nothing references it, because the templates carry no doc_to_text key and the generated inline string is the only one in the chain. An alternative fix would be to add doc_to_text: !function utils.doc_to_text to the templates and drop the inline strings entirely, which would also revive that dead function. That is a larger change and would want the same treatment across all five prompts, so I kept this PR minimal. Happy to do it that way instead if you prefer.

2. Separate generator issue, not fixed here. Running gen_utils.py in translate mode emits translate/prompt_1/afrixnli_translate_eng.yaml, an English-to-English translate task that is not in the repo and was presumably removed deliberately. I deleted it after regenerating in order to preserve the committed state. The generator is not idempotent against the repo as committed. Worth a separate fix.

3. I make no claim about any published results. This PR describes only what the harness renders today.

Testing

• pytest --showlocals -s -vv -n=auto --ignore=tests/models/test_openvino.py: 607 passed, 16 skipped, matching the baseline on main.
• pre-commit run --files <the changed files>: passing.

[CLAassistant]

All committers have signed the CLA.

[AmosBunde]

Note: #3931 also fixes #3930. Mine changes gen_utils.py and regenerates rather than
editing the generated YAMLs, and clears the ruff PLC0206 in that file that otherwise
blocks the pre-commit hook. Happy to close in favour of the other PR if a maintainer
prefers it.