Support for https://huggingface.co/CohereForAI/c4ai-command-r-plus

[bks5881]

Can you please add support or point me to a branch for https://huggingface.co/CohereForAI/c4ai-command-r-plus ? currently vllm fails to load and throws error. I updated transformers to the latest and still fails. i am able to load it directly with transformers though.

The error vllm throws is
KeyError: 'model.layers.14.self_attn.k_norm.weight' [repeated 2x across cluster]

[martin416]

how did you load it in huggingface's transformers ,thanks

[bks5881]

@martin416 i used the latest transformers and installed from git, and used their code on hf.