Update perplexity cost tracking #15556

Sameerlite · 2025-10-15T08:10:42Z

Title

Add Perplexity cost extraction from API response

Relevant issues

Fixes #15547

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
I have added a screenshot of my new test passing locally
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

Bug FIx

Changes

This PR adds cost extraction functionality to Perplexity's chat transformation, allowing Perplexity to override LiteLLM's cost calculation by providing its own cost information in the API response.

I have updated perplexity provider to base_llm_http_handler and its own transformation methods. And I have used concept of over riding the cost as now perplexity provides cost in its response itself

This change ensures Perplexity users get accurate cost tracking that reflects the provider's actual pricing, improving cost transparency and accuracy for Perplexity API usage.

Change in perplexity API repsonse:

"usage": {
    "prompt_tokens": 5,
    "completion_tokens": 308,
    "total_tokens": 313,
    "search_context_size": "low",
    "cost": {
      "input_tokens_cost": 0,
      "output_tokens_cost": 0,
      "request_cost": 0.005,
      "total_cost": 0.005
    }
  }

This fixes Claude's models via the Converse API, which should also fix Claude Code.

* fix(router): update model_name_to_deployment_indices on deployment removal When a deployment is deleted, the model_name_to_deployment_indices map was not being updated, causing stale index references. This could lead to incorrect routing behavior when deployments with the same model_name were dynamically removed. Changes: - Update _update_deployment_indices_after_removal to maintain model_name_to_deployment_indices mapping - Remove deleted indices and decrement indices greater than removed index - Clean up empty entries when no deployments remain for a model name - Update test to verify proper index shifting and cleanup behavior * fix(router): remove redundant index building during initialization Remove duplicate index building operations that were causing unnecessary work during router initialization: 1. Removed redundant `_build_model_id_to_deployment_index_map` call in __init__ - `set_model_list` already builds all indices from scratch 2. Removed redundant `_build_model_name_index` call at end of `set_model_list` - the index is already built incrementally via `_create_deployment` -> `_add_model_to_list_and_index_map` Both indices (model_id_to_deployment_index_map and model_name_to_deployment_indices) are properly maintained as lookup indexes through existing helper methods. This change eliminates O(N) duplicate work during initialization without any behavioral changes. The indices continue to be correctly synchronized with model_list on all operations (add/remove/upsert).

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

merge main

* fix(router): update model_name_to_deployment_indices on deployment removal When a deployment is deleted, the model_name_to_deployment_indices map was not being updated, causing stale index references. This could lead to incorrect routing behavior when deployments with the same model_name were dynamically removed. Changes: - Update _update_deployment_indices_after_removal to maintain model_name_to_deployment_indices mapping - Remove deleted indices and decrement indices greater than removed index - Clean up empty entries when no deployments remain for a model name - Update test to verify proper index shifting and cleanup behavior * fix(router): remove redundant index building during initialization Remove duplicate index building operations that were causing unnecessary work during router initialization: 1. Removed redundant `_build_model_id_to_deployment_index_map` call in __init__ - `set_model_list` already builds all indices from scratch 2. Removed redundant `_build_model_name_index` call at end of `set_model_list` - the index is already built incrementally via `_create_deployment` -> `_add_model_to_list_and_index_map` Both indices (model_id_to_deployment_index_map and model_name_to_deployment_indices) are properly maintained as lookup indexes through existing helper methods. This change eliminates O(N) duplicate work during initialization without any behavioral changes. The indices continue to be correctly synchronized with model_list on all operations (add/remove/upsert).

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

Add tiered pricing and cost calculation for xai

…15503)

…_block_repair Add support for thinking blocks and redacted thinking blocks in Anthropic v1/messages API

(feat) Add voyage model integration in sagemaker

Add support for extended thinking in Anthropic's models via Bedrock's Converse API

* docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * add application level encryption in SQS * add application level encryption in SQS --------- Co-authored-by: Krrish Dholakia <[email protected]> Co-authored-by: Ishaan Jaff <[email protected]> Co-authored-by: deepanshu <[email protected]>

…t/completions API with LiteLLM (#15509) * docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * add AnthropicCitation * fix async_post_call_success_deployment_hook * fix add vector_store_custom_logger to global callbacks * test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call * async_post_call_success_deployment_hook * add async_post_call_streaming_deployment_hook * async def test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call_streaming(setup_vector_store_registry): * fix _call_post_streaming_deployment_hook * fix async_post_call_streaming_deployment_hook * test update * docs: Accessing Search Results * docs KB * fix chatUI * fix searchResults * fix onSearchResults * fix kb --------- Co-authored-by: Krrish Dholakia <[email protected]>

* docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * fix: KeyRequestBase * fix rpm_limit_type * fix dynamic rate limits * fix use dynamic limits here * fix _should_enforce_rate_limit * fix _should_enforce_rate_limit * fix counter * test_dynamic_rate_limiting_v3 * use _create_rate_limit_descriptors --------- Co-authored-by: Krrish Dholakia <[email protected]>

Litellm fix mypy ruff errors1

vercel · 2025-10-15T08:10:50Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Oct 17, 2025 3:26am

ishaan-jaff

ishaan-jaff · 2025-10-15T16:05:58Z

litellm/llms/perplexity/chat/transformation.py

    def _get_openai_compatible_provider_info(
-        self, api_base: Optional[str], api_key: Optional[str]
-    ) -> Tuple[Optional[str], Optional[str]]:
-        api_base = api_base or get_secret_str("PERPLEXITY_API_BASE") or "https://api.perplexity.ai"  # type: ignore


this won't work on python 3.8, please don't change this

ishaan-jaff · 2025-10-15T16:06:07Z

litellm/llms/perplexity/chat/transformation.py

-        encoding: Any,
-        api_key: Optional[str] = None,
-        json_mode: Optional[bool] = None,
+        encoding: Any,  # noqa: ANN401


same point ^

ishaan-jaff · 2025-10-15T16:07:14Z

litellm/llms/perplexity/chat/transformation.py

+                    # Store cost in hidden params for the cost calculator to use
+                    if not hasattr(model_response, "_hidden_params"):
+                        model_response._hidden_params = {}  # noqa: SLF001
+                    if "additional_headers" not in model_response._hidden_params:  # noqa: SLF001


can we not avoid all these linting rules

krrishdholakia · 2025-10-16T16:26:08Z

litellm/llms/perplexity/chat/transformation.py

-        api_key: Optional[str] = None,
-        json_mode: Optional[bool] = None,
+        encoding: Any,  
+        api_key: str | None = None,


revert this - it will fail on python 3.8

CLAassistant · 2025-10-18T19:09:59Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
5 out of 6 committers have signed the CLA.

✅ lcfyi
✅ ishaan-jaff
✅ Sameerlite
✅ AlexsanderHamir
✅ deepanshululla
❌ LoadingZhang
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

ishaan-jaff

please rebase with main

Sameerlite · 2025-10-20T18:11:27Z

Closing this due to many conflicts. New one - #15743

lcfyi and others added 30 commits October 5, 2025 03:36

Implement fix for thinking_blocks and converse API calls

dd7d12e

This fixes Claude's models via the Converse API, which should also fix Claude Code.

Add thinking literal

1c3ec18

Fix mypy issues

310e3b3

Type fix for redacted thinking

d07455b

Add voyage model integration in sagemaker

25769b5

Add tiered pricing and cost calculation for xai

a33c348

Add config file logic

6002880

Use already exiting voyage transformation

8c7d7a3

Use generic cost calculator

d0e26e2

UI new build

cb9b65d

fix(prometheus): Fix Prometheus metric collection in a multi-workers …

6060537

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

Merge pull request #15477 from BerriAI/main

75ef5df

merge main

Merge pull request #15478 from BerriAI/main

c496097

merge main

fix conversion of thinking block

a39d263

Resolve conflicts in generated HTML files

ef0d6f0

fix(prometheus): Fix Prometheus metric collection in a multi-workers …

b49ab39

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

Merge pull request #15368 from BerriAI/litellm_add_above_128k_pricing

7a68c57

Add tiered pricing and cost calculation for xai

Remove penalty params as supported params for gemini preview model (#…

2afae44

…15503)

Merge pull request #15501 from BerriAI/litellm_fix_anthropic_thinking…

939d499

…_block_repair Add support for thinking blocks and redacted thinking blocks in Anthropic v1/messages API

refactor code as per comments

6fadfd3

fix merge error

c748321

refactor code as per comments

0906b99

refactor code as per comments

a6ed454

Merge branch 'litellm_staging_oct' into litellm_voyage_sagemaker

2653d6a

Merge pull request #15367 from BerriAI/litellm_voyage_sagemaker

2bd7c88

(feat) Add voyage model integration in sagemaker

Merge branch 'litellm_staging_oct' into lcfyi/add-support-for-thinking

cee4b58

Merge pull request #15220 from lcfyi/lcfyi/add-support-for-thinking

0175c13

Add support for extended thinking in Anthropic's models via Bedrock's Converse API

ishaan-jaff and others added 5 commits October 13, 2025 20:05

fix mypy and lint errors

4339f52

Merge pull request #15533 from BerriAI/litellm_fix_mypy_ruff_errors1

298fa0c

Litellm fix mypy ruff errors1

Update perplexity cost tracking

25508c2

ishaan-jaff requested changes Oct 15, 2025

View reviewed changes

fix lint errors

9601918

vercel bot deployed to Preview October 15, 2025 19:01 View deployment

Sameerlite requested a review from ishaan-jaff October 16, 2025 03:39

krrishdholakia reviewed Oct 16, 2025

View reviewed changes

fix code

162f40d

Sameerlite requested a review from krrishdholakia October 17, 2025 03:26

vercel bot deployed to Preview October 17, 2025 03:26 View deployment

Sameerlite force-pushed the litellm_staging_oct branch from 789eec0 to 5a637cc Compare October 17, 2025 18:44

ishaan-jaff requested changes Oct 18, 2025

View reviewed changes

Sameerlite changed the base branch from litellm_staging_oct to litellm_sameer_oct_staging October 20, 2025 18:04

Sameerlite closed this Oct 20, 2025

Uh oh!

Update perplexity cost tracking #15556

Update perplexity cost tracking #15556

Uh oh!

Conversation

Sameerlite commented Oct 15, 2025

Title

Relevant issues

Pre-Submission checklist

Type

Changes

Change in perplexity API repsonse:

Uh oh!

vercel bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ishaan-jaff left a comment

Choose a reason for hiding this comment

Uh oh!

ishaan-jaff Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

ishaan-jaff Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Sameerlite Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

ishaan-jaff Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Sameerlite Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

krrishdholakia Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

Sameerlite Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

CLAassistant commented Oct 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ishaan-jaff left a comment

Choose a reason for hiding this comment

Uh oh!

Sameerlite commented Oct 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

vercel bot commented Oct 15, 2025 •

edited

Loading

CLAassistant commented Oct 18, 2025 •

edited

Loading