Skip to content

Conversation

codefromthecrypt
Copy link
Contributor

@codefromthecrypt codefromthecrypt commented Oct 13, 2025

What this PR does / why we need it:

Before, all paths were hard coded to start with /tmp/envoy-gateway, making parallel runs impractical, especially when certs change.

This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility.

The implementation introduces four configurable directory paths:

  • configHome: certificates and configuration files (default: ~/.config/envoy-gateway)
  • dataHome: Envoy binaries, sharable across configs (default: ~/.local/share/envoy-gateway)
  • stateHome: logs and persistent state (default: ~/.local/state/envoy-gateway)
  • runtimeDir: ephemeral runtime files (default: /tmp/envoy-gateway-${UID})

Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts.

These paths are propagated to func-e which creates subdirectories as needed:

  • dataHome/envoy-versions/ for Envoy binaries
  • stateHome/envoy-runs/{runID}/ for per-run logs
  • runtimeDir/{runID}/ for per-run runtime files

@codefromthecrypt codefromthecrypt requested a review from a team as a code owner October 13, 2025 17:56
@codefromthecrypt
Copy link
Contributor Author

license check fails due to not indexed yet. I'll fix the other things later https://deps.dev/go/github.com%2Ftetratelabs%2Ffunc-e

@codefromthecrypt codefromthecrypt force-pushed the feat/configurable-xdg-paths branch from 6383591 to 5398829 Compare October 14, 2025 10:49
@codefromthecrypt codefromthecrypt deleted the feat/configurable-xdg-paths branch October 14, 2025 10:49
@codecov
Copy link

codecov bot commented Oct 14, 2025

Codecov Report

❌ Patch coverage is 61.71875% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.19%. Comparing base (5e41fac) to head (935470e).

Files with missing lines Patch % Lines
internal/infrastructure/host/infra.go 0.00% 18 Missing ⚠️
internal/cmd/certgen.go 28.57% 15 Missing ⚠️
internal/globalratelimit/runner/runner.go 66.66% 6 Missing and 1 partial ⚠️
internal/gatewayapi/runner/runner.go 83.33% 2 Missing and 1 partial ⚠️
internal/infrastructure/host/paths.go 90.62% 2 Missing and 1 partial ⚠️
internal/xds/runner/runner.go 78.57% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7225      +/-   ##
==========================================
+ Coverage   71.12%   71.19%   +0.07%     
==========================================
  Files         228      229       +1     
  Lines       40703    40806     +103     
==========================================
+ Hits        28949    29051     +102     
+ Misses      10061    10059       -2     
- Partials     1693     1696       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codefromthecrypt codefromthecrypt restored the feat/configurable-xdg-paths branch October 14, 2025 11:10
@codefromthecrypt
Copy link
Contributor Author

🤞 build is green now

@codefromthecrypt
Copy link
Contributor Author

@arkodg @nacx @missBerg fyi I am off next week but wanted to eliminate this nagging tech debt for good. This PR uses XDG conventions (like prometheus, goose etc), which map into the same change made in func-e. The result is no more hard-coded /tmp/envoy-gateway and flexibility to change things in docker etc.

Most importantly, envoy-ai-gateway is blocked on this because the certs directory is hard-coded here. This causes problems as when you run tests that update certs it breaks your host configuration, as there's no way to overwrite it before this change.

So, TL;DR; this isn't just a random TODO fix, this is a big deal for sustainability forward and we should merge soon, especially as I can finish up the whole thing soup-to-nuts including ai gateway if folks can prioritize it this week!

@codefromthecrypt
Copy link
Contributor Author

OIDC_with_BackendCluster is a very flaky test.

Please kick the build. Maybe we can prioritize deflaking this after merge, especially as you can use a clean certs directory after. The certs being hard-coded directory smashed by other tests, caused flakes in ai-gateway also.

@codefromthecrypt
Copy link
Contributor Author

incidentally this flake reminded me certs should be in the ConfigHome as they are configuration bound (for exactly the same reason as flakes when one test needs certs for its config). changing this now

@codefromthecrypt codefromthecrypt force-pushed the feat/configurable-xdg-paths branch from 5398829 to 66871b3 Compare October 14, 2025 12:40
@codefromthecrypt
Copy link
Contributor Author

updated wrt cert location

This adds configuration fields to EnvoyGatewayHostInfrastructureProvider to allow users to specify custom paths for configuration, data, state, and runtime directories, following XDG Base Directory Specification conventions while maintaining backward compatibility.

The implementation introduces four configurable directory paths:

- configHome: certificates and configuration files (default: ~/.config/envoy-gateway)
- dataHome: Envoy binaries, sharable across configs (default: ~/.local/share/envoy-gateway)
- stateHome: logs and persistent state (default: ~/.local/state/envoy-gateway)
- runtimeDir: ephemeral runtime files (default: /tmp/envoy-gateway-${UID})

Certificates are stored under configHome to ensure isolation between different configurations when running multiple EnvoyGateway instances in parallel, preventing certificate conflicts.

These paths are propagated to func-e which creates subdirectories as needed:
- dataHome/envoy-versions/ for Envoy binaries
- stateHome/envoy-runs/{runID}/ for per-run logs
- runtimeDir/{runID}/ for per-run runtime files

The changes include path resolution helpers, updated runners to use configurable paths, comprehensive test coverage, and updated documentation.

Signed-off-by: Adrian Cole <[email protected]>
@jukie
Copy link
Contributor

jukie commented Oct 16, 2025

/retest

@jukie jukie requested a review from a team October 16, 2025 04:00
@zirain
Copy link
Member

zirain commented Oct 16, 2025

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants