Display benchmark progress in non-interactive consoles

[sjmonson]

Problem Statement

When running GuideLLM in a non-interactive environment such as a K8s Pod there is no indication of benchmark progress until the entire run completes.

Proposed Solution

Create a system for emitting status updates in non-interactive mode.

Alternatives Considered

No response

Usage Examples

Additional Context

No response

[dbutenhof]

Is this a duplicate of #518?

[sjmonson]

Oh yes it is. I misinterpreted the original issue back when I responded to it. I assumed it was about getting a full live metrics readout. Rereading now I guess that is not the case.

[thameem-abbas]

I wouldn't classify it as a duplicate. This is not about recording in a file. We should have some progress be displayed on the terminal at regular intervals. In cases where the benchmark is long -> multiple minutes, there is no visibility into the progression of the benchmark. Writing it to a file isn't very useful for observability without creating another layer to read and emit that.

I think it would be ideal if we can emit some log at every 0.1 x benchmark load (duration or requests). I'm not sure how this could apply directly in multi-turn and a couple of complex scenarios. For a normal user though, this would help a lot.

Another note is that most systems expect some progress for automation to understand that the benchmark is still alive. So this could also help with those systems

[dbutenhof]

Perhaps. I think that, at least, there's a substantive overlap, which is what made me search for the existing "non-interactive feedback" issue when I saw this. It may be that we can look at this as a well-defined subset of the fairly loose scope of #518 ... in which case it would probably make sense to operate on this first and then expand/generalize.