Skip to content

Commit eb83849

Browse files
docs: add APISIX reverse proxy example
Document the route settings needed for forwarded headers, streaming responses, and long-running inference behind Apache APISIX. Closes #11215 Assisted-by: Codex:gpt-5
1 parent 8a80830 commit eb83849

1 file changed

Lines changed: 68 additions & 1 deletion

File tree

docs/content/advanced/reverse-proxy-tls.md

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: TLS Reverse Proxy Configuration
3-
description: Configure LocalAI behind a TLS termination reverse proxy (HAProxy, Apache, Nginx)
3+
description: Configure LocalAI behind a TLS termination reverse proxy (HAProxy, Apache, Nginx, APISIX)
44
weight: 100
55
---
66

@@ -147,6 +147,73 @@ the proxy stops waiting before LocalAI finishes, clients receive a proxy-generat
147147
Nginx Proxy Manager, Caddy, Traefik, HAProxy, and ingress controllers have
148148
equivalent upstream response timeout settings.
149149

150+
## Apache APISIX Configuration
151+
152+
[Apache APISIX](https://apisix.apache.org/) is an open source API and AI
153+
gateway. Put it between clients and LocalAI when you want to manage the
154+
OpenAI-compatible API with the same gateway used for other services. In
155+
addition to TLS termination, APISIX can add authentication, rate limiting,
156+
load balancing, observability, and other policies through plugins without
157+
changing LocalAI.
158+
159+
The request path is:
160+
161+
```text
162+
OpenAI-compatible client -> APISIX -> LocalAI (:8080)
163+
```
164+
165+
Install APISIX using its
166+
[getting started guide](https://apisix.apache.org/docs/apisix/getting-started/README/),
167+
then create a route that forwards the external scheme and host, allows
168+
long-running inference, and disables response buffering for streaming
169+
completions. This example assumes APISIX can resolve `localai` and reach it on
170+
port `8080`:
171+
172+
```bash
173+
curl http://127.0.0.1:9180/apisix/admin/routes/localai \
174+
--request PUT \
175+
--header "X-API-KEY: ${admin_key}" \
176+
--data '{
177+
"uri": "/*",
178+
"plugins": {
179+
"proxy-rewrite": {
180+
"headers": {
181+
"set": {
182+
"X-Forwarded-Proto": "$scheme",
183+
"X-Forwarded-Host": "$host"
184+
}
185+
}
186+
},
187+
"proxy-buffering": {
188+
"disable_proxy_buffering": true
189+
}
190+
},
191+
"timeout": {
192+
"connect": 60,
193+
"send": 3600,
194+
"read": 3600
195+
},
196+
"upstream": {
197+
"type": "roundrobin",
198+
"pass_host": "pass",
199+
"nodes": {
200+
"localai:8080": 1
201+
}
202+
}
203+
}'
204+
```
205+
206+
Set `X-Forwarded-Prefix` in the `proxy-rewrite` header map as well if LocalAI
207+
is exposed under a sub-path. Adjust the timeout values, in seconds, for the
208+
slowest request you expect to serve. Keep the APISIX Admin API private and
209+
replace `${admin_key}` with the key configured for your deployment.
210+
211+
Clients continue to use LocalAI's OpenAI-compatible paths through the APISIX
212+
address, for example `https://localai.example.com/v1/chat/completions`; only
213+
the base URL changes. For policies beyond a transparent reverse proxy, see the
214+
[APISIX AI Gateway overview](https://apisix.apache.org/ai-gateway/) and its
215+
authentication, traffic management, and observability plugins.
216+
150217
For bulk jobs on a trusted private network, you can also bypass the public
151218
reverse proxy and connect directly to LocalAI, for example
152219
`http://localai-host:8080/v1`.

0 commit comments

Comments
 (0)