Hi. I tried to follow the README example, and when I came to the Evaluation step, an IndexError occurred when I ran python evaluate.py --model openlm-research/open_llama_3b --domain ID --result ParaRel_openllama_3b:
I think the error is due to that in evaluate.py, the SURE and UNSURE are lists of encodings but not their ids:
SURE.append(tokenizer("sure")[0]) UNSURE.append(tokenizer("unsure")[0])
Hi. I tried to follow the README example, and when I came to the Evaluation step, an IndexError occurred when I ran
python evaluate.py --model openlm-research/open_llama_3b --domain ID --result ParaRel_openllama_3b:I think the error is due to that in evaluate.py, the SURE and UNSURE are lists of encodings but not their ids:
SURE.append(tokenizer("sure")[0]) UNSURE.append(tokenizer("unsure")[0])