how to measure time to first token (TTFT) and time between tokens (TBT) #13251
Unanswered
afsara-ben
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I cannot seem to find any metric for
time to first token (TTFT)ortime between tokens (TBT)inllama.cpp. I can ofcourse manually calculate them but was wondering if its already thereBeta Was this translation helpful? Give feedback.
All reactions