[Bug]: vertex_ai rerank drops document text from response even when return_documents is true

[AbhAy120204]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When calling litellm.rerank() with a Vertex AI semantic-ranker model and return_documents=True (the default), none of the results carry a document field. The results only contain index and relevance_score.

What makes this different from a simple omission: the request side already does the right thing. transform_rerank_request sets ignoreRecordDetailsInResponse = not return_documents (litellm/llms/vertex_ai/rerank/transformation.py:146), so Vertex returns content and title on every record when return_documents=True. The response transformer then reads only id and score from each record and silently discards content. Vertex is paying to return the text and LiteLLM throws it away.

The result is that results[i].document.text is always absent regardless of what the caller passes for return_documents, making the parameter a no-op on the response side.

User Flow

Before a fix: a RAG pipeline using Vertex rerank to retrieve ranked source passages gets no document text back

1. They call litellm.rerank(model="vertex_ai/semantic-ranker-default@latest", query="...", documents=["doc 0", "doc 1", ...], return_documents=True)
2. The response comes back with results: [{"index": 1, "relevance_score": 0.95}], no document field, despite Vertex having returned the record content
3. They access results[0]["document"]["text"] and get KeyError: 'document'
4. Setting return_documents=False produces identical output, so the parameter is a no-op

After a fix: the same call returns document text in each result

1. Same call with return_documents=True
2. The response comes back with results: [{"index": 1, "relevance_score": 0.95, "document": {"text": "doc 1"}}]
3. results[0]["document"]["text"] returns the ranked source passage
4. Setting return_documents=False omits the document field as expected

Proof the bug occurs

Verified on commit 8fb4545 (litellm v1.97.0).

The bug is in the response transformer and reproduces without a live Vertex call. Running the regression test before the fix:

FAILED test_vertex_rerank_return_documents_true_populates_document_text
  AssertionError: return_documents=True must populate document
  got {'index': 1, 'relevance_score': 0.95}  -- no document field

FAILED test_vertex_rerank_return_documents_true_async  
  KeyError: 'document'

Root cause in transform_rerank_response (transformation.py:182-186):

results.append(
    {
        "index": int(record["id"]),
        "relevance_score": record.get("score", 0.0),
        # record["content"] is present when ignoreRecordDetailsInResponse=false
        # but is never read here
    }
)

The fix is to read record.get("content") and populate document.text when return_documents is truthy, which is simpler than the Bedrock equivalent because Vertex returns the text directly in the response rather than requiring back-fill from the original request.

This is the same class of bug as #38006 (Bedrock rerank), which was fixed in #38007.

What part of LiteLLM is this about?

SDK (litellm Python package)

What LiteLLM version are you on ?

v1.97.0

Twitter / LinkedIn details

https://www.linkedin.com/in/abhay-tiwari-/