SCC25 bonus implementation and documentation of LLM multi node inference

[hsuanchi1506]

✅ PR Checklist – Previous Submission Round repo (v5.0)

This repository adds additional implementation guidance on enabling multi-node inference for Llama-70B using Triton Server.

🔒 Safeguard Against Accidental Disclosure

• I confirm that I am not committing results or system updates intended for a future round.
• This PR does not contain any result, metadata, or logs for an unreleased submission round.

🛠️ Valid Post-Publication Changes (if any)

• This PR does not alter the published accuracy, latency, or system identity of any prior result.

📄 PR Communication

• I have clearly explained the reason for this change in the PR description.

[github-actions]

MLCommons CLA bot:
Thank you very much for your submission, we really appreciate it. Before we can accept your contribution, we ask that you sign the MLCommons CLA (Apache 2). Please use this [Google form] (https://forms.gle/Ew1KkBVpyeJDuRw67) to initiate authorization. If you are from an MLCommons member organization, we will request that you be added to the CLA. If you are not from a member organization, we will email you a CLA to sign. For any questions, please contact [email protected].
0 out of 1 committers have signed the MLCommons CLA.
❌ @hsuanchi1506
_{You can retrigger this bot by commenting recheck in this Pull Request}

[hanyunfan]

Hi @hsuanchi1506 Are you are member of MLPerf Inference week group?

[anandhu-eng]

Hi @hanyunfan , this and the following PRs are raised as part of Student Cluster Competition 2025 where MLPerf Inference is one of the benchmarks.

Other PRs:
#20
mlcommons/inference#2387

Edit: I'm converting these PR's to drafts since these would be used to award bonus points for teams, there is no urgent need to merge.