fix: remove wrapping double-quotes from GCP guided_regex #1402

sukumargaonkar · 2025-10-21T15:45:06Z

Description
This PR removes the wrapping double-quotes returned by GCP when using guided_regex

GCP doesn't natively support REGEX response modes, so we instead express them as json schema.
This causes the response to be wrapped in double-quotes.
E.g. "positive" (the double-quotes at the start and end are unwanted)

Signed-off-by: Sukumar Gaonkar <[email protected]>

codecov-commenter · 2025-10-21T15:48:19Z

Codecov Report

❌ Patch coverage is 90.62500% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.63%. Comparing base (f134ecd) to head (06983d4).

Files with missing lines	Patch %	Lines
internal/extproc/translator/gemini_helper.go	89.28%	3 Missing ⚠️

❌ Your project status has failed because the head coverage (78.63%) is below the target coverage (86.00%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1402      +/-   ##
==========================================
+ Coverage   78.61%   78.63%   +0.02%     
==========================================
  Files         139      139              
  Lines       13745    13763      +18     
==========================================
+ Hits        10805    10823      +18     
  Misses       2285     2285              
  Partials      655      655

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Sukumar Gaonkar <[email protected]>

internal/extproc/translator/gemini_helper.go

tiswanso · 2025-10-22T12:56:52Z

internal/extproc/translator/gemini_helper.go

+				// This causes the response to be wrapped in double-quotes.
+				// E.g. `"positive"` (the double-quotes at the start and end are unwanted)
+				// Here we remove the wrapping double-quotes.
+				if len(part.Text) > 2 && part.Text[0] == '"' && part.Text[len(part.Text)-1] == '"' {


I'm curious whether there's a reason to not use strings.Trim() here: https://pkg.go.dev/strings#Trim

nice catch,
mainly being defensive for scenarios where first and last words are quoted but not connected
"ERROR" Unable to connect to database "DatabaseModule"
trim would remove more than intended
added a test case for this
thoughts?

I see... makes sense. TrimPrefix(), TrimSuffix() can be used to trim exact strings so it'd work to limit to the single char. https://pkg.go.dev/strings#TrimPrefix

+1 to strings.Trim*

@sukumargaonkar i think this is the only pending unresolved comment

Addressed in c73f6aa

i don't think this if conditions is necessary as strings.Trim* is noop if not applicable

TrimPrefix returns s without the provided leading prefix string. If s doesn't start with prefix, s is returned unchanged.

addressed in f2d8620

Signed-off-by: Sukumar Gaonkar <[email protected]>

…equests Signed-off-by: Sukumar Gaonkar <[email protected]>

sukumargaonkar · 2025-10-22T15:04:32Z

/retest

sukumargaonkar · 2025-10-22T15:29:21Z

The ubuntu based CI failed but macOS based passed

error in ubuntu based CI
https://github.com/envoyproxy/ai-gateway/actions/runs/18718793788/job/53391903589?pr=1402

--- FAIL: TestRun_enabled (0.00s)
    pprof_test.go:33: 
        	Error Trace:	/home/runner/work/ai-gateway/ai-gateway/internal/pprof/pprof_test.go:33
        	Error:      	Received unexpected error:
        	            	Get "http://localhost:6060/debug/pprof/cmdline": dial tcp [::1]:6060: connect: connection refused
        	Test:       	TestRun_enabled

seems unrelated to this PR. maybe @mathetake knows more

mathetake · 2025-10-22T15:30:58Z

yeah sorry you can ignore the flake

mathetake · 2025-10-22T15:49:45Z

internal/extproc/translator/gemini_helper.go

+	ResponseModeNone  ResponseMode = "NONE"
+	ResponseModeText  ResponseMode = "TEXT"
+	ResponseModeJSON  ResponseMode = "JSON"
+	ResponseModeEnum  ResponseMode = "ENUM"
+	ResponseModeRegex ResponseMode = "REGEX"


also prefix gemini*

addressed in f871ff7

mathetake · 2025-10-22T15:50:12Z

internal/extproc/translator/gemini_helper.go

 )

+// ResponseMode represents the type of response mode for Gemini requests
+type ResponseMode string


Suggested change

type ResponseMode string

type geminiResponseMode string

Addressed in f871ff7

Signed-off-by: Sukumar Gaonkar <[email protected]>

aabchoo · 2025-10-23T19:46:12Z

internal/extproc/translator/gemini_helper.go


+	// ResponseFormat and guidedJSON/guidedChoice/guidedRegex are mutually exclusive.
+	// Verify only one is specified.
+	if formatSpecifiedCount > 1 {


What happens when formatSpecifiedCount == 0

nothing really
this is only to validate more than one format specifiers are not specified

Signed-off-by: Sukumar Gaonkar <[email protected]>

fix: remove wrapping double-quotes from GCP guided_regex

8652530

Signed-off-by: Sukumar Gaonkar <[email protected]>

sukumargaonkar added 2 commits October 21, 2025 14:56

tests

2d665a3

Signed-off-by: Sukumar Gaonkar <[email protected]>

Merge remote-tracking branch 'upstream/main' into fix-guided-regex

b0bc3d2

sukumargaonkar marked this pull request as ready for review October 21, 2025 19:20

sukumargaonkar requested a review from a team as a code owner October 21, 2025 19:20

aabchoo reviewed Oct 21, 2025

View reviewed changes

internal/extproc/translator/gemini_helper.go Show resolved Hide resolved

tiswanso reviewed Oct 22, 2025

View reviewed changes

sukumargaonkar added 3 commits October 22, 2025 09:56

add test case for edge case

eeefb3d

Signed-off-by: Sukumar Gaonkar <[email protected]>

validate mutual exclusivity of response format specifiers in guided r…

9bc5b8e

…equests Signed-off-by: Sukumar Gaonkar <[email protected]>

Merge remote-tracking branch 'upstream/main' into fix-guided-regex

cb6f0af

mathetake reviewed Oct 22, 2025

View reviewed changes

sukumargaonkar added 3 commits October 22, 2025 12:41

address pr comments (refactoring)

f871ff7

Signed-off-by: Sukumar Gaonkar <[email protected]>

Merge remote-tracking branch 'upstream/main' into fix-guided-regex

7c49ff9

Merge remote-tracking branch 'upstream/main' into fix-guided-regex

704b778

mathetake assigned aabchoo Oct 23, 2025

sukumargaonkar added 2 commits October 23, 2025 15:00

Merge remote-tracking branch 'upstream/main' into fix-guided-regex

8c39a5e

pr comment - use strings trimPrefix trimSuffix

c73f6aa

Signed-off-by: Sukumar Gaonkar <[email protected]>

aabchoo reviewed Oct 23, 2025

View reviewed changes

sukumargaonkar added 2 commits October 23, 2025 16:00

address pr comment

f2d8620

Signed-off-by: Sukumar Gaonkar <[email protected]>

Merge remote-tracking branch 'upstream/main' into fix-guided-regex

06983d4

fix: remove wrapping double-quotes from GCP guided_regex #1402

Are you sure you want to change the base?

fix: remove wrapping double-quotes from GCP guided_regex #1402

Conversation

sukumargaonkar commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sukumargaonkar Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sukumargaonkar commented Oct 22, 2025

Uh oh!

sukumargaonkar commented Oct 22, 2025

Uh oh!

mathetake commented Oct 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mathetake Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sukumargaonkar commented Oct 21, 2025 •

edited

Loading

codecov-commenter commented Oct 21, 2025 •

edited

Loading

sukumargaonkar Oct 22, 2025 •

edited

Loading

mathetake Oct 22, 2025 •

edited

Loading