how to config rerank model of bge-reranker-v2-m3 deploy locally

[reallax]

I deployed bge-reranker-v2-m3 model locally and need to access through litellm, how to config that?
I tested huggingface rerank configuration, but it didn't work. I'm not sure if I configured it correctly.
Could anyone please offer some advise or help? Thanks a lot.

[shengkui]

I meet same issue, and tried different provider, and found infinity provider works:

model_list:
  - model_name: bge-reranker-v2-m3
    litellm_params:
      model: infinity/bge-reranker-v2-m3
      api_base: http://192.168.1.108:9997
      api_key: sk-1234

NOTES: DO NOT add "/v1" to api_base.