Add OpenAI video generation and content retrieval support #15541

Sameerlite · 2025-10-14T17:27:51Z

Title

Add OpenAI video generation and content retrieval support

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
I have added a screenshot of my new test passing locally
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature

Changes

Video Generation (litellm.video_generation):
- Text-to-video generation using OpenAI Sora models
- Support for video editing with reference images
- Configurable video duration and dimensions
- Async support with litellm.avideo_generation
Video Content Retrieval (litellm.video_content):
- Download generated videos as MP4 bytes
- Async support with litellm.avideo_content
- Proper error handling and status checking

API Examples:

# Video Generation
response = litellm.video_generation(
    prompt="A cat playing with a ball of yarn",
    model="sora-2",
    seconds="8",
    size="720x1280"
)

# Video Content Retrieval
video_bytes = litellm.video_content(
    video_id=response.data[0].id,
    model="sora-2"
)

Working end to end

vercel · 2025-10-14T17:27:56Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Error			Oct 18, 2025 2:59pm

💡 Enable Vercel Agent with $100 free credit for automated AI reviews

krrishdholakia · 2025-10-14T23:24:59Z

docs/my-website/docs/providers/openai.md

-```
+```
+
+## Video Generation


can you have this be a separate doc - it'll be easier to find

have it be in the same openai folder though

krrishdholakia · 2025-10-14T23:25:47Z

litellm/llms/openai/video_retrieval/transformation.py

it's odd to call this folder video_retrieval and the top-level api - video_content

Can you pick one?

Done, rewrote everything to video_content

krrishdholakia · 2025-10-14T23:26:02Z

litellm/model_prices_and_context_window_backup.json

        "supports_vision": true,
        "supports_web_search": true
+    },
+    "sora-2": {


there's no cost for this model?

added cost tracking

krrishdholakia · 2025-10-14T23:26:36Z

litellm/videos/main.py

+        # Check for mock response first
+        mock_response = kwargs.get("mock_response", None)
+        if mock_response is not None:
+            if isinstance(mock_response, str):


keep in separate function

krrishdholakia · 2025-10-14T23:27:22Z

litellm/llms/openai/video_generation/transformation.py

add unit testing for this new endpoint @Sameerlite

krrishdholakia · 2025-10-16T19:23:13Z

litellm/cost_calculator.py

-                        _usage = ResponseAPILoggingUtils._transform_response_api_usage_to_chat_usage(
-                            _usage
-                        ).model_dump()
+                        # Skip token validation for video generation as it doesn't use token-based pricing


does this matter? if usage obj doesn't exist we will have an empty usage here

ideally we should avoid having endpoint specific if/else statements in here.

since there are other endpoints that don't need usage objects and can use this function, i wonder why we need an exception here.

Removed the conditioning.

krrishdholakia · 2025-10-16T19:24:24Z

litellm/cost_calculator.py

+        return output_cost_per_second * duration_seconds
+
+    # If no cost information found, return 0
+    verbose_logger.warning(


this will spam logs in traffic, probably emit as either debug or info log.

warnings will trigger alerting

Yes, changed it to info

* fix(router): update model_name_to_deployment_indices on deployment removal When a deployment is deleted, the model_name_to_deployment_indices map was not being updated, causing stale index references. This could lead to incorrect routing behavior when deployments with the same model_name were dynamically removed. Changes: - Update _update_deployment_indices_after_removal to maintain model_name_to_deployment_indices mapping - Remove deleted indices and decrement indices greater than removed index - Clean up empty entries when no deployments remain for a model name - Update test to verify proper index shifting and cleanup behavior * fix(router): remove redundant index building during initialization Remove duplicate index building operations that were causing unnecessary work during router initialization: 1. Removed redundant `_build_model_id_to_deployment_index_map` call in __init__ - `set_model_list` already builds all indices from scratch 2. Removed redundant `_build_model_name_index` call at end of `set_model_list` - the index is already built incrementally via `_create_deployment` -> `_add_model_to_list_and_index_map` Both indices (model_id_to_deployment_index_map and model_name_to_deployment_indices) are properly maintained as lookup indexes through existing helper methods. This change eliminates O(N) duplicate work during initialization without any behavioral changes. The indices continue to be correctly synchronized with model_list on all operations (add/remove/upsert).

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

…15503)

* docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * add application level encryption in SQS * add application level encryption in SQS --------- Co-authored-by: Krrish Dholakia <[email protected]> Co-authored-by: Ishaan Jaff <[email protected]> Co-authored-by: deepanshu <[email protected]>

…t/completions API with LiteLLM (#15509) * docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * add AnthropicCitation * fix async_post_call_success_deployment_hook * fix add vector_store_custom_logger to global callbacks * test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call * async_post_call_success_deployment_hook * add async_post_call_streaming_deployment_hook * async def test_e2e_bedrock_knowledgebase_retrieval_with_llm_api_call_streaming(setup_vector_store_registry): * fix _call_post_streaming_deployment_hook * fix async_post_call_streaming_deployment_hook * test update * docs: Accessing Search Results * docs KB * fix chatUI * fix searchResults * fix onSearchResults * fix kb --------- Co-authored-by: Krrish Dholakia <[email protected]>

* docs: fix doc * docs(index.md): bump rc * [Fix] GEMINI - CLI - add google_routes to llm_api_routes (#15500) * fix: add google_routes to llm_api_routes * test: test_virtual_key_llm_api_routes_allows_google_routes * build: bump version * bump: version 1.78.0 → 1.78.1 * fix: KeyRequestBase * fix rpm_limit_type * fix dynamic rate limits * fix use dynamic limits here * fix _should_enforce_rate_limit * fix _should_enforce_rate_limit * fix counter * test_dynamic_rate_limiting_v3 * use _create_rate_limit_descriptors --------- Co-authored-by: Krrish Dholakia <[email protected]>

krrishdholakia · 2025-10-17T20:21:03Z

@Sameerlite can you make sure your PR passes testing + linting

Sameerlite · 2025-10-20T18:36:05Z

Closing and creating a new one as this has gotten too many conflicts - #15745

Sameerlite added 4 commits October 14, 2025 12:32

Add openai videos generation and retrieval support

f6f6480

add retrieval endpoint

8322c62

Add docs

6e9b69e

Add imports

16c888c

vercel bot deployed to Preview October 14, 2025 17:27 View deployment

remove orjson

269e40a

vercel bot deployed to Preview October 14, 2025 17:31 View deployment

remove double import

5c0e9c8

vercel bot deployed to Preview October 14, 2025 17:37 View deployment

krrishdholakia reviewed Oct 14, 2025

View reviewed changes

fix openai videos format

9018a4c

vercel bot deployed to Preview October 15, 2025 18:07 View deployment

remove mock code

4e3dded

vercel bot deployed to Preview October 15, 2025 18:11 View deployment

remove not required comments

4ef28a4

vercel bot deployed to Preview October 15, 2025 18:14 View deployment

Add tests

06b8bb1

vercel bot deployed to Preview October 15, 2025 18:19 View deployment

Add tests

87666a1

vercel bot deployed to Preview October 15, 2025 18:23 View deployment

Sameerlite requested a review from krrishdholakia October 15, 2025 18:50

Add other video endpoints

c44298b

krrishdholakia reviewed Oct 16, 2025

View reviewed changes

Fix cost calculation and transformation

8894bee

ishaan-jaff and others added 20 commits October 17, 2025 23:59

UI new build

a5ae176

fix(prometheus): Fix Prometheus metric collection in a multi-workers …

02389ff

…environment (#14929) Co-authored-by: sotazhang <[email protected]>

Add tiered pricing and cost calculation for xai

3d4b956

Use generic cost calculator

71fc43d

Resolve conflicts in generated HTML files

d526b41

Remove penalty params as supported params for gemini preview model (#…

1b3d998

…15503)

fix conversion of thinking block

b64b540

Add google rerank endpoint

6543cb2

Add docs

5da242e

fix mypy error

acb2d2d

fix mypy and lint errors

39e8e40

Add haiku 4.5 integration

edd05d9

Add haiku 4.5 integration for other regions as well

a7a4bee

Handle citation field correctly

deed425

Fix filtering headers for signature calcs

1b173d0

Add haiku 4.5 integration (#15650)

5a637cc

Sameerlite force-pushed the litellm_staging_oct branch from 789eec0 to 5a637cc Compare October 17, 2025 18:44

Merge branch 'litellm_staging_oct' into litellm_videos_openai_retrieval

53c94f3

vercel bot had a problem deploying to Preview October 18, 2025 14:40 Failure

Fixed mypy tests

cb6683b

vercel bot had a problem deploying to Preview October 18, 2025 14:59 Failure

Sameerlite requested a review from krrishdholakia October 20, 2025 02:35

Sameerlite changed the base branch from litellm_staging_oct to litellm_sameer_oct_staging October 20, 2025 18:03

Sameerlite closed this Oct 20, 2025

Uh oh!

Add OpenAI video generation and content retrieval support #15541

Add OpenAI video generation and content retrieval support #15541

Uh oh!

Conversation

Sameerlite commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Title

Relevant issues

Pre-Submission checklist

Type

Changes

API Examples:

Uh oh!

vercel bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

krrishdholakia commented Oct 17, 2025

Uh oh!

Sameerlite commented Oct 20, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Sameerlite commented Oct 14, 2025 •

edited

Loading

vercel bot commented Oct 14, 2025 •

edited

Loading