[WIP][docs] add aclgraph developer guide

[zzzzwwjj]

What this PR does / why we need it?

Add aclgraph developer guide.

Does this PR introduce any user-facing change?

How was this patch tested?

• vLLM version: v0.11.0rc3
• vLLM main: vllm-project/vllm@c9461e0

[gemini-code-assist]

Code Review

This pull request adds a developer guide for ACLGraph. The documentation is a good start, but it contains several grammatical errors and unclear phrasing that could be improved for better readability. I've provided a detailed suggestion to revise the text for clarity and correctness. Addressing these will make the guide much more helpful for developers.

[gemini-code-assist]

This section contains several grammatical errors, typos, and awkward phrases that make it difficult to read and understand. For a developer guide, clarity is crucial. I've suggested a rewrite of this section to improve readability and correctness. Notably, 'Grading' in the subheading is likely a typo for 'Bucketing' or a similar term.

During LLM inference, each token requires nearly a thousand operator executions. When the CPU is slower at launching operators than the GPU is at executing them, the process can become host-bound. In severe cases, the GPU may be idle for more than half of the time. To solve this problem, we use graph-based execution in LLM inference.

eager mode:

cpu: | launch op1 | launch op2 | launch op3 | launch op4 | launch op5 |

gpu: | run op1 |free| run op2 |free| run op3 |free| run op4 |free| run op5 |

 | <-----                           total time                                 -----> |

graph mode:

cpu: | launch graph |

gpu: | run op1 | run op2 | run op3 | run op4 | run op5 |

 | <-----                    total time                      -----> |

## How to use ACLGraph?

ACLGraph is enabled by default in the V1 Engine. To use it, simply ensure you are using the V1 Engine.

## How it works?

In short, graph mode works in two steps: **capture and replay**. When the engine starts, we capture all operations in the model's forward pass and save them as a graph. When a request arrives, we simply replay the graph on the GPUs and wait for the result.

But in reality, graph mode is not that simple.

### Padding and Bucketing

Since a graph can only replay operations that were captured beforehand, it cannot perform dynamic actions like tiling or input validation. Therefore, we must ensure the consistency of the graph's input. However, the model's input shape depends on the requests scheduled by the Scheduler, so we cannot guarantee this consistency.

One solution is to capture the graph for the largest possible shape and pad all model inputs to match it. However, this introduces significant redundant computation and degrades performance. A better approach is to capture multiple graphs with different shapes and pad the model input to the nearest captured graph shape. This greatly reduces redundant computation. The trade-off is that when `max_num_batched_tokens` is very large, the number of graphs to capture also becomes very large. We also know that for large input tensor shapes, the computation time is long, making graph mode less beneficial. Therefore, the strategy is as follows:
1. Set a threshold.
2. When `num_scheduled_tokens` is larger than the threshold, use `eager_mode`.
3. Capture multiple graphs for a range of sizes below the threshold.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.