Can I deploy DeepSeek V3 on 4 nodes with 14 GPUs?

[groklab]

Hi!

Can I deploy DeepSeek V3 on 4 nodes with 14 H100s?

The setup:

node 1

node 2

node 3

node 4

---

My commands are:

# node 1
python3 -m sglang.launch_server --model-path /models/dsv3 --tp 14 --dist-init-addr 10.21.99.116:5000 --nnodes 4 --node-rank 0 --trust-remote-code --enable-torch-compile --host 0.0.0.0 --port 30000 --grammar-backend xgrammar

# node 2
python3 -m sglang.launch_server --model-path /models/dsv3 --tp 14 --dist-init-addr 10.21.99.116:5000 --nnodes 4 --node-rank 1 --trust-remote-code --enable-torch-compile --grammar-backend xgrammar

# node 3
python3 -m sglang.launch_server --model-path /models/dsv3 --tp 14 --dist-init-addr 10.21.99.116:5000 --nnodes 4 --node-rank 2 --trust-remote-code --enable-torch-compile --grammar-backend xgrammar

# node 4
python3 -m sglang.launch_server --model-path /models/dsv3 --tp 14 --dist-init-addr 10.21.99.116:5000 --nnodes 4 --node-rank 3 --trust-remote-code --enable-torch-compile --grammar-backend xgrammar

Then got error: AssertionError: tp_size must be divisible by number of nodes

[ispobock]

The TP size needs to ensure that some dimensions of model weights are divisible. It is recommended to set TP size as the power of 2.