File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -119,7 +119,7 @@ docker pull vllm/vllm-openai:v0.19.0-ubuntu2404
119119Or using with pip :
120120
121121` ` ` bash
122- pip install -U "vllm>=0.17 .0"
122+ pip install -U "vllm>=0.19 .0"
123123` ` `
124124
125125Launch the service :
@@ -144,15 +144,13 @@ docker pull lmsysorg/sglang:v0.5.10
144144Or using with pip :
145145
146146` ` ` bash
147- pip install "sglang>=0.5.9 "
147+ pip install "sglang>=0.5.10 "
148148` ` `
149149
150150Launch the service :
151151
152152` ` ` bash
153- pip install "transformers>=5.3.0"
154-
155- sglang serve --model zai-org/GLM-OCR --port 8080 --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --served-model-name glm-ocr
153+ SGLANG_ENABLE_SPEC_V2=1 sglang serve --model-path zai-org/GLM-OCR --port 8080 --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --served-model-name glm-ocr
156154` ` `
157155
158156>Note
Original file line number Diff line number Diff line change @@ -118,7 +118,7 @@ docker pull vllm/vllm-openai:v0.19.0-ubuntu2404
118118或者使用 pip :
119119
120120` ` ` bash
121- pip install -U "vllm>=0.17 .0"
121+ pip install -U "vllm>=0.19 .0"
122122` ` `
123123
124124启动服务:
@@ -143,15 +143,13 @@ docker pull lmsysorg/sglang:v0.5.10
143143或者使用 pip :
144144
145145` ` ` bash
146- pip install "sglang>=0.5.9 "
146+ pip install "sglang>=0.5.10 "
147147` ` `
148148
149149启动服务:
150150
151151` ` ` bash
152- pip install "transformers>=5.3.0"
153-
154- sglang serve --model zai-org/GLM-OCR --port 8080 --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --served-model-name glm-ocr
152+ SGLANG_ENABLE_SPEC_V2=1 sglang serve --model-path zai-org/GLM-OCR --port 8080 --speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --served-model-name glm-ocr
155153` ` `
156154
157155> Note
You can’t perform that action at this time.
0 commit comments