Fixed async pre call hook not getting called #16531
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
||
| def log_pre_api_call(self, model, messages, kwargs): | ||
| pass | ||
| async_impl = type(self).async_log_pre_api_call |
Contributor
There was a problem hiding this comment.
why run the async method for pre api call? isn't it always a sync call?
Collaborator
Author
There was a problem hiding this comment.
Yes that is what the codebase showed. But since I saw those methods and client was using the async method, I thought it was not implemented
Contributor
|
It's dangerous to try and guess if we should run the sync vs. async pre api call hook. I believe we can just document the behaviour, to call out the sync pre api call is what is called in the codebase. |
Collaborator
Author
|
Understood. Will update the doc accordingly |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Fixed async pre-call hook not being invoked for custom loggers
Relevant issues
Fixes #16254
Pre-Submission checklist
I have Added testing in the
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsI have added a screenshot of my new test passing locally
My PR passes all unit tests on
make test-unitMy PR's scope is as isolated as possible, it only solves 1 specific problem
Type
🐛 Bug Fix
Changes
CustomLogger.log_pre_api_callto invokeasync_log_pre_api_callwhen an async implementation is provided, while falling back to the existing sync stub for legacy loggers.litellm._async_input_callback(and dynamic async input callbacks) are awaited before the provider call.pre_call_test.pyto confirm that the async pre-call hook now runs.