-
Notifications
You must be signed in to change notification settings - Fork 658
[usage] add self-host detailed README #189
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -166,6 +166,22 @@ For specialized deployment scenarios, see the detailed guides: | |
| - **[Apple Silicon with mlx-vlm](examples/mlx-deploy/README.md)** - Optimized for Apple Silicon Macs | ||
| - **[Ollama Deployment](examples/ollama-deploy/README.md)** - Simple local deployment with Ollama | ||
|
|
||
| #### Option 4: SDK Server + Client (GPU-less Client) | ||
|
|
||
| Deploy the SDK Server on a GPU machine, then use any machine as a client — no GPU needed on the client side. The client connects via the MaaS-compatible protocol, pointing `api_url` at your self-hosted server. | ||
|
|
||
| ```yaml | ||
| # Client config.yaml | ||
| pipeline: | ||
| maas: | ||
| enabled: true | ||
| api_url: http://<SERVER_IP>:5002/glmocr/parse | ||
| api_key: any-string # self-hosted server does not validate keys | ||
| verify_ssl: false | ||
|
Comment on lines
+171
to
+180
|
||
| ``` | ||
|
|
||
| See the full guide: **[Self-hosted SDK Server + Client](examples/self-host/README.md)** | ||
|
|
||
| #### Update Configuration | ||
|
|
||
| After launching the service, configure `config.yaml`: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -164,6 +164,22 @@ sglang serve --model zai-org/GLM-OCR --port 8080 --speculative-algorithm NEXTN - | |
| - **[Apple Silicon 使用 mlx-vlm](examples/mlx-deploy/README.md)** - 针对 Apple Silicon Mac 优化 | ||
| - **[Ollama 部署](examples/ollama-deploy/README.md)** - 使用 Ollama 进行简单的本地部署 | ||
|
|
||
| #### 方式 4: SDK Server + Client(Client 无需 GPU) | ||
|
|
||
| 在 GPU 机器上部署 SDK Server,其他机器作为 Client 通过网络调用——Client 端无需 GPU。Client 通过 MaaS 兼容协议连接,将 `api_url` 指向自建 Server 即可。 | ||
|
|
||
| ```yaml | ||
| # Client 端 config.yaml | ||
| pipeline: | ||
| maas: | ||
| enabled: true | ||
| api_url: http://<SERVER_IP>:5002/glmocr/parse | ||
| api_key: any-string # 自建 Server 不校验 API key | ||
| verify_ssl: false | ||
|
Comment on lines
+169
to
+178
|
||
| ``` | ||
|
|
||
| 完整指南见:**[自部署 SDK Server + Client](examples/self-host/README.md)** | ||
|
|
||
| ##### 更新配置 | ||
|
|
||
| 启动服务后,配置 `config.yaml`: | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,103 @@ | ||||||||
| # Self-hosted SDK Server + Client Mode | ||||||||
|
|
||||||||
| The GLM-OCR SDK supports a split **Server / Client** deployment: run the SDK Server on a GPU machine, and connect from any other machine over HTTP — no GPU required on the client side. | ||||||||
|
|
||||||||
| ## Architecture | ||||||||
|
|
||||||||
| ``` | ||||||||
| ┌──────────────────────┐ HTTP ┌──────────────────────────────┐ | ||||||||
| │ Client (no GPU) │ ────────────────→ │ Server (GPU machine) │ | ||||||||
| │ │ POST /glmocr/parse│ │ | ||||||||
| │ glmocr CLI / Python │ ←──────────────── │ python -m glmocr.server │ | ||||||||
| │ │ JSON response │ (layout + OCR pipeline) │ | ||||||||
| └──────────────────────┘ └──────────────────────────────┘ | ||||||||
| ``` | ||||||||
|
|
||||||||
| The Server runs the full OCR pipeline (layout detection + parallel OCR). The Client calls it over HTTP with zero local computation. | ||||||||
|
|
||||||||
| ## Server Side | ||||||||
|
|
||||||||
| On the GPU machine, start the Server: | ||||||||
|
|
||||||||
| ```bash | ||||||||
| # 1. Install (includes selfhosted pipeline + server) | ||||||||
| pip install "glmocr[selfhosted,server]" | ||||||||
|
|
||||||||
| # 2. Configure self-hosted mode and point to your local vLLM / SGLang | ||||||||
| # In config.yaml: set pipeline.maas.enabled: false and configure ocr_api | ||||||||
|
|
||||||||
| # 3. Start the server | ||||||||
| python -m glmocr.server --config config.yaml | ||||||||
| ``` | ||||||||
|
|
||||||||
| The server listens on `0.0.0.0:5002` by default, with the API endpoint at `/glmocr/parse`. | ||||||||
|
|
||||||||
| ## Client Side | ||||||||
|
|
||||||||
| On any machine (no GPU needed), point the SDK's MaaS client at your self-hosted server: | ||||||||
|
|
||||||||
| ```bash | ||||||||
| pip install glmocr | ||||||||
| ``` | ||||||||
|
|
||||||||
| Edit `config.yaml`: | ||||||||
|
|
||||||||
| ```yaml | ||||||||
| pipeline: | ||||||||
| maas: | ||||||||
| enabled: true | ||||||||
| api_url: http://<SERVER_IP>:<SERVER_PORT>/glmocr/parse | ||||||||
| api_key: any-string # Self-hosted server does not validate API keys | ||||||||
| verify_ssl: false # Internal networks typically lack HTTPS | ||||||||
|
||||||||
| verify_ssl: false # Internal networks typically lack HTTPS | |
| verify_ssl: false # Only relevant for https:// URLs; no effect with http://. Use false only for self-signed/internal HTTPS, otherwise keep true. |
Copilot
AI
Apr 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This guide notes the server doesn’t validate API keys, but it should also include an explicit warning about the security implications (no auth by default) and recommend running it only on trusted networks or behind a reverse proxy with TLS/auth when used beyond localhost.
Copilot
AI
Apr 8, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The “Or set via environment variable” example only sets ZHIPU_API_KEY and then runs glmocr parse without --config. Without also overriding the MaaS api_url (e.g., via GLMOCR_API_URL or a config file), the CLI will use the packaged default cloud endpoint instead of the self-hosted server. Update this section to either keep using --config config.yaml or show the necessary env vars to point pipeline.maas.api_url at the self-hosted server.
| # Or set via environment variable | |
| # Or set via environment variables | |
| export GLMOCR_API_URL=http://<SERVER_IP>:5002/glmocr/parse |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR description currently contains a generic contribution guide template and doesn’t describe the self-hosted README additions. Please update the PR description to match the actual change (adding a self-hosted SDK server/client guide) so reviewers and release notes consumers have accurate context.