Why is cache_creation_input_tokens not included in prompt_tokens? #14890
Unanswered
anthony-liner
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
When using
"cache_control": {"type": "ephemeral"}
for Claude prompt caching as mentioned in the litellm docs, it seems that for cache write the cache_creation_input_tokens value is not being included in prompt_tokens in the responseUsage
object.Is this intended behavior? Currently this is leading to undercounting of input(prompt) tokens in downstream LLM tracing tools like langfuse.
Beta Was this translation helpful? Give feedback.
All reactions