|
1 | 1 | --- |
2 | 2 | title: TLS Reverse Proxy Configuration |
3 | | -description: Configure LocalAI behind a TLS termination reverse proxy (HAProxy, Apache, Nginx) |
| 3 | +description: Configure LocalAI behind a TLS termination reverse proxy (HAProxy, Apache, Nginx, APISIX) |
4 | 4 | weight: 100 |
5 | 5 | --- |
6 | 6 |
|
@@ -147,6 +147,73 @@ the proxy stops waiting before LocalAI finishes, clients receive a proxy-generat |
147 | 147 | Nginx Proxy Manager, Caddy, Traefik, HAProxy, and ingress controllers have |
148 | 148 | equivalent upstream response timeout settings. |
149 | 149 |
|
| 150 | +## Apache APISIX Configuration |
| 151 | + |
| 152 | +[Apache APISIX](https://apisix.apache.org/) is an open source API and AI |
| 153 | +gateway. Put it between clients and LocalAI when you want to manage the |
| 154 | +OpenAI-compatible API with the same gateway used for other services. In |
| 155 | +addition to TLS termination, APISIX can add authentication, rate limiting, |
| 156 | +load balancing, observability, and other policies through plugins without |
| 157 | +changing LocalAI. |
| 158 | + |
| 159 | +The request path is: |
| 160 | + |
| 161 | +```text |
| 162 | +OpenAI-compatible client -> APISIX -> LocalAI (:8080) |
| 163 | +``` |
| 164 | + |
| 165 | +Install APISIX using its |
| 166 | +[getting started guide](https://apisix.apache.org/docs/apisix/getting-started/README/), |
| 167 | +then create a route that forwards the external scheme and host, allows |
| 168 | +long-running inference, and disables response buffering for streaming |
| 169 | +completions. This example assumes APISIX can resolve `localai` and reach it on |
| 170 | +port `8080`: |
| 171 | + |
| 172 | +```bash |
| 173 | +curl http://127.0.0.1:9180/apisix/admin/routes/localai \ |
| 174 | + --request PUT \ |
| 175 | + --header "X-API-KEY: ${admin_key}" \ |
| 176 | + --data '{ |
| 177 | + "uri": "/*", |
| 178 | + "plugins": { |
| 179 | + "proxy-rewrite": { |
| 180 | + "headers": { |
| 181 | + "set": { |
| 182 | + "X-Forwarded-Proto": "$scheme", |
| 183 | + "X-Forwarded-Host": "$host" |
| 184 | + } |
| 185 | + } |
| 186 | + }, |
| 187 | + "proxy-buffering": { |
| 188 | + "disable_proxy_buffering": true |
| 189 | + } |
| 190 | + }, |
| 191 | + "timeout": { |
| 192 | + "connect": 60, |
| 193 | + "send": 3600, |
| 194 | + "read": 3600 |
| 195 | + }, |
| 196 | + "upstream": { |
| 197 | + "type": "roundrobin", |
| 198 | + "pass_host": "pass", |
| 199 | + "nodes": { |
| 200 | + "localai:8080": 1 |
| 201 | + } |
| 202 | + } |
| 203 | + }' |
| 204 | +``` |
| 205 | + |
| 206 | +Set `X-Forwarded-Prefix` in the `proxy-rewrite` header map as well if LocalAI |
| 207 | +is exposed under a sub-path. Adjust the timeout values, in seconds, for the |
| 208 | +slowest request you expect to serve. Keep the APISIX Admin API private and |
| 209 | +replace `${admin_key}` with the key configured for your deployment. |
| 210 | + |
| 211 | +Clients continue to use LocalAI's OpenAI-compatible paths through the APISIX |
| 212 | +address, for example `https://localai.example.com/v1/chat/completions`; only |
| 213 | +the base URL changes. For policies beyond a transparent reverse proxy, see the |
| 214 | +[APISIX AI Gateway overview](https://apisix.apache.org/ai-gateway/) and its |
| 215 | +authentication, traffic management, and observability plugins. |
| 216 | + |
150 | 217 | For bulk jobs on a trusted private network, you can also bypass the public |
151 | 218 | reverse proxy and connect directly to LocalAI, for example |
152 | 219 | `http://localai-host:8080/v1`. |
|
0 commit comments