Skip to content

docs: use a NIC values overlay for AI Gateway wiring, fix stale links - #150

Open
tylerpotts wants to merge 1 commit into
mainfrom
docs/envoy-gateway-values-overlay
Open

docs: use a NIC values overlay for AI Gateway wiring, fix stale links#150
tylerpotts wants to merge 1 commit into
mainfrom
docs/envoy-gateway-values-overlay

Conversation

@tylerpotts

Copy link
Copy Markdown
Contributor

Two independent changes

1. Section 6.1 told operators to make an edit that NIC silently reverts

The installation guide's step 6.1 said to merge the AI Gateway keys into spec.source.helm.values on NIC's envoy-gateway ArgoCD Application. NIC regenerates apps/*.yaml from its own templates, so those edits are discarded by the next nic deploy --regen-apps, taking the extension wiring with them.

That failure is quiet, which is what makes it worth fixing rather than documenting: per the guide's own section 6, without enableBackend/backendResources the HTTPRoute translates to a 500, and without the extensionManager block ext_proc never lands in the filter chain and per-model requests 404. So the symptom is "LLM routing broke" some arbitrary time later, with nothing in the ArgoCD UI pointing at a deploy that ran days earlier.

Now: 6.1 leads with a values overlay committed to values/envoy-gateway/overlays/20-ai-gateway.yaml, a directory NIC never writes to or deletes from. The previous Application edit is kept as a documented legacy path with an explicit warning, because clusters that predate NIC's values/ layout still need it.

The section opens with a check of which case applies rather than naming a nic version:

ls <git_repository.path>/values/envoy-gateway/

New examples/envoy-gateway-overlay.yaml contains only the keys that differ from NIC's base values. Helm deep-merges maps, so config.envoyGateway.gateway.controllerName from base.yaml survives even though the overlay also writes under config.envoyGateway, and deployment resources, replica count and Service type are no longer restated. That removes a real hazard in the old instructions, which asked operators to hand-preserve those values while editing and warned "preserve any existing deployment / service / podDisruptionBudget values" in a comment.

I verified this rather than asserting it, by simulating Helm's merge against NIC's actual templates/values/envoy-gateway/base.yaml:

controllerName survives: gateway.envoyproxy.io/gatewayclass-controller
enableBackend:           True
extMgr hostname:         ai-gateway-controller.envoy-ai-gateway-system.svc.cluster.local
backendResources:        [{group: inference.networking.k8s.io, kind: InferencePool, version: v1}]
deployment.replicas:     1
deployment resources:    {cpu: 500m, memory: 512Mi}
service.type:            ClusterIP

The 20- prefix is deliberate: overlays apply in lexical filename order with last-wins, so keeping this pack at 20- leaves 30- and later free for operator overrides without editing a pack-owned file.

examples/envoy-gateway.yaml keeps its full-Application form and gains a header marking it as the legacy path.

2. Every broken link in the README

All of them are fallout from the nebari-llm-serving-pack -> llm-serving-pack rename.

Link Was Now
Installation guide nebari-dev.github.io/nebari-llm-serving-pack/installation/#6-... (404) packs.nebari.dev/llm-serving-pack/installation/#6-...
Local Development guide nebari-dev.github.io/nebari-llm-serving-pack/local-development/ (404) packs.nebari.dev/llm-serving-pack/local-development/
Issue #44 (x2) github.com/nebari-dev/nebari-llm-serving-pack/issues/44 (404) github.com/nebari-dev/llm-serving-pack/issues/44
Issue #61 github.com/nebari-dev/nebari-llm-serving-pack/issues/61 (404) github.com/nebari-dev/llm-serving-pack/issues/61

Worth recording why the docs links could not simply be renamed in place: GitHub does not redirect <owner>.github.io/<old-repo>/ project-page paths after a rename, and the site has since moved to the packs.nebari.dev custom domain, at which point the github.io path stops serving at all. Both halves of the change independently break the old URL, so there is no redirect to follow. nebari-dev.github.io/llm-serving-pack/installation/ 404s too.

All 15 external README links verified 200 after the change. All 10 relative links were already fine.

Also included

  • The H1 still read # nebari-llm-serving-pack; changed to # llm-serving-pack. Cosmetic and outside the strict "broken links" ask, so easy to drop if you would rather the display name differ from the repo name. Note the docs site title is "LLM Serving Pack".
  • The prerequisites bullet now points at the overlay example first, with the full-Application example labelled as legacy.

Verification

  • npm run build in docs/ passes, 11 pages built, no errors.
  • The bookmarked anchor #6-reconfigure-envoy-gateway-with-ai-gateway-extension-wiring is preserved, confirmed in the built HTML.
  • One anchor did change: #61-edit-nics-envoy-gateway-application is now #61-apply-the-values, since the heading no longer describes what the step does. Nothing in the repo references the old anchor, but external bookmarks to it will break.
  • Both example YAMLs parse.
  • Matched the house style for callouts: the docs use markdown blockquotes rather than Starlight asides, so the warning is a blockquote.

Dependency

NIC's values overlay seam is nebari-dev/nebari-infrastructure-core#499, approved but not yet merged, and not in nic v0.11.0. The overlay path here only works once that ships.

Because the guide branches on whether values/ exists rather than on a version number, this is safe to merge before then: operators on current nic fall through to the legacy path, which is what they are already doing. If you would rather wait for the nic release, the README link fixes are independent and could be split out.

The installation guide told operators to hand-edit inline helm values
on NIC's envoy-gateway ArgoCD Application. NIC regenerates apps/*.yaml
from its own templates, so those edits are silently discarded by the
next `nic deploy --regen-apps`, taking the AI Gateway extension wiring
with them. The failure is quiet: routes return 404 or 500 again with
nothing in the ArgoCD UI to explain it.

Section 6.1 now leads with a values overlay committed to
values/envoy-gateway/overlays/20-ai-gateway.yaml, which NIC never
writes to or deletes from, and keeps the Application edit as a
documented legacy path for clusters that predate the values/ layout,
with an explicit warning. The guide starts with a check of which case
applies rather than asserting a nic version.

Adds examples/envoy-gateway-overlay.yaml, containing only the keys that
differ from NIC's base values. Helm deep-merges maps, so
config.envoyGateway.gateway.controllerName from base.yaml survives, and
deployment resources, replica count and Service type are no longer
restated (and so can no longer be dropped by mistranscription).
Verified by simulating the merge against NIC's actual
templates/values/envoy-gateway/base.yaml. The 20- prefix leaves 30- and
later free for operator overrides, since overlays apply in lexical
order with last-wins.

Marks examples/envoy-gateway.yaml as the legacy path in its header.

Also fixes every broken link in the README, all of them fallout from
the nebari-llm-serving-pack -> llm-serving-pack rename:

- Two docs links pointed at nebari-dev.github.io/nebari-llm-serving-pack,
  which 404s. GitHub Pages does not redirect project-page paths after a
  rename, and the site has since moved to the packs.nebari.dev custom
  domain, so the old github.io path stopped serving entirely.
- Three issue links used the old repo name and 404ed.
- The H1 still read "nebari-llm-serving-pack".

All 15 external README links verified 200. The prerequisites bullet now
points at the overlay example first.
@github-actions

Copy link
Copy Markdown

📄 Docs preview for docs/envoy-gateway-values-overlay:
https://docs-envoy-gateway-values-ov.llm-serving-pack.pages.dev

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants