Skip to content

Commit 3d6e811

Browse files
authored
docs: add realtime session stats guidance
Document realtime WebSocket session-stats diagnostics for long sessions with speaker diarization.
1 parent 5d79631 commit 3d6e811

2 files changed

Lines changed: 26 additions & 0 deletions

File tree

docs/vllm_guide.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -607,6 +607,19 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/
607607
Set `--ws-ping-interval 0` only when an external gateway already owns
608608
keepalive/reconnect policy.
609609

610+
For long-session debugging, especially with `--enable-spk`, enable periodic
611+
session-state logs:
612+
613+
```bash
614+
CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py \
615+
--port 10095 --language 中文 --enable-spk \
616+
--log-session-stats-interval 30
617+
```
618+
619+
This prints a `Session stats:` line every 30 seconds. Include the last few
620+
lines in issue reports together with tail RTF, process RSS, GPU memory, and
621+
the final disconnect log.
622+
610623
### 6.3 WebSocket Protocol
611624

612625
**Connection**: `ws://host:10095`
@@ -728,6 +741,7 @@ Because each refresh re-encodes from the sentence start, the longer a sentence,
728741
- **SPK is of limited effectiveness on Fun-ASR-Nano** (see #2944); most real-time ASR scenarios do not need speaker separation.
729742
- **Streaming SPK is expensive and grows with the session**: each sentence re-clusters **all historical embeddings** (**O(N²)**, more expensive per sentence as the session grows) and **synchronously blocks the event loop**; the session also **re-clusters everything again** at the end, so the per-sentence clustering during streaming is overwritten by the final result — redundant as far as the final output is concerned. This is especially pronounced under long sessions + high concurrency.
730743
- **Recommendation**: keep the default off for multi-client live ASR; if diarization is required, add `--enable-spk` and treat the final STOP-time labels as authoritative.
744+
- **Long-session diagnostics**: when a session still slows down or disconnects, rerun with `--log-session-stats-interval 30` and check whether `audio_buffer_samples`, `locked_sentences`, `speaker_history_chunks`, `speaker_history_embeddings`, and `speaker_centers` stay bounded. If those counters stay near their limits while RTF keeps rising, the remaining bottleneck is more likely model inference, response payload size, or environment scheduling rather than retained session state.
731745

732746
### 6.7 Production concurrency and multi-process deployment
733747

docs/vllm_guide_zh.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -610,6 +610,17 @@ CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/
610610
只有在外部网关已经统一负责 keepalive / reconnect 策略时,才考虑设置
611611
`--ws-ping-interval 0` 关闭服务端 ping。
612612

613+
长会话排障,尤其是启用 `--enable-spk` 时,可以打开周期性 session 状态日志:
614+
615+
```bash
616+
CUDA_VISIBLE_DEVICES=0 python examples/industrial_data_pretraining/fun_asr_nano/serve_realtime_ws.py \
617+
--port 10095 --language 中文 --enable-spk \
618+
--log-session-stats-interval 30
619+
```
620+
621+
服务端会每 30 秒输出一行 `Session stats:`。提交 issue 时请连同最后几行
622+
`Session stats`、末段 RTF、进程 RSS、GPU memory 和断线前后的服务端日志一起提供。
623+
613624
### 6.3 WebSocket 协议
614625

615626
**连接**`ws://host:10095`
@@ -731,6 +742,7 @@ Fun-ASR-Nano 的声学编码器(SenseVoice)是**全上下文、非流式**
731742
- **Fun-ASR-Nano 上 SPK 效果有限**(见 #2944),多数实时 ASR 场景并不需要说话人分离。
732743
- **流式 SPK 代价高且随会话变长**:每句对**全部历史 embedding** 做一次全量重聚类(**O(N²)**,会话越长每句越贵),且**同步阻塞事件循环**;而会话结束时还会**全量重聚一遍**,流式期间每句的聚类结果会被最终结果覆盖——对最终输出而言属于重复计算。长会话 + 高并发下尤其明显。
733744
- **建议**:多客户端实时转写优先保持默认关闭;确需 diarization 时再加 `--enable-spk`,并以 STOP 后的最终 `spk` 标签为准。
745+
- **长会话诊断**:如果 session 仍然逐渐变慢或断开,请用 `--log-session-stats-interval 30` 复测,并观察 `audio_buffer_samples``locked_sentences``speaker_history_chunks``speaker_history_embeddings``speaker_centers` 是否保持有界。如果这些计数都接近上限但 RTF 仍持续升高,剩余瓶颈更可能在模型推理、返回 payload 大小或环境调度,而不是 session 状态继续泄漏。
734746

735747
### 6.7 生产并发与多进程部署
736748

0 commit comments

Comments
 (0)