Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to custom handle redirects #5678

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions api/internal/loader/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,12 @@ var (
ErrHTTP = errors.Errorf("HTTP Error")
ErrRtNotDir = errors.Errorf("must build at directory")
)

type RedirectionError struct {
NewPath string
Msg string
}

func (e *RedirectionError) Error() string {
return e.Msg
}
19 changes: 15 additions & 4 deletions api/internal/loader/fileloader.go
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,9 @@ type FileLoader struct {
cleaner func() error
}

// This redirect code does not process automaticaly by http client and we can process it manually
const MultipleChoicesRedirectCode = 300

// Repo returns the absolute path to the repo that contains Root if this fileLoader was created from a url
// or the empty string otherwise.
func (fl *FileLoader) Repo() string {
Expand Down Expand Up @@ -318,11 +321,19 @@ func (fl *FileLoader) httpClientGetContent(path string) ([]byte, error) {
defer resp.Body.Close()
// response unsuccessful
if resp.StatusCode < 200 || resp.StatusCode > 299 {
_, err = git.NewRepoSpecFromURL(path)
if err == nil {
return nil, errors.Errorf("URL is a git repository")
if resp.StatusCode == MultipleChoicesRedirectCode {
var newPath string = resp.Header.Get("Location")
return nil, &RedirectionError{
Msg: "Response is redirect",
NewPath: newPath,
}
Comment on lines +325 to +329
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe That is better to solve redirects here than return RedirectionError and a new path.

Copy link
Author

@abogdanov37 abogdanov37 Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! It does not work in all cases. For example if I return redirect to git. In that case httpClient has no credentials for download content

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example if I return redirect to git. In that case httpClient has no credentials for download content

Sorry, I couldn't understand about this sentence.
In my understanding, that repo uses the HTTPS protocol to download when HTTP redirects to git.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My explanation was not clear. Sorry for that. I'll try again.
Example
We get next url in resource section https://example.com/apps/app1?version=1.1.0
When kustomize (in build proccess) go to that url, it get Redirect 300 with newPath https://customgit.org/somegrop/project-app1.git?refs=1.1.0. The repository is PRIVATE.
In this case if we go to that git repo using httpClient we get page with ask for credential (((
But if we create a new loader kustomize will use console git which already have credentials and can clone repository

} else {
_, err = git.NewRepoSpecFromURL(path)
if err == nil {
return nil, errors.Errorf("URL is a git repository")
}
return nil, fmt.Errorf("%w: status code %d (%s)", ErrHTTP, resp.StatusCode, http.StatusText(resp.StatusCode))
}
return nil, fmt.Errorf("%w: status code %d (%s)", ErrHTTP, resp.StatusCode, http.StatusText(resp.StatusCode))
}
content, err := io.ReadAll(resp.Body)
return content, errors.Wrap(err)
Expand Down
30 changes: 30 additions & 0 deletions api/internal/loader/fileloader_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import (
"sigs.k8s.io/kustomize/api/ifc"
"sigs.k8s.io/kustomize/api/internal/git"
"sigs.k8s.io/kustomize/api/konfig"
"sigs.k8s.io/kustomize/kyaml/errors"
"sigs.k8s.io/kustomize/kyaml/filesys"
)

Expand Down Expand Up @@ -659,6 +660,35 @@ func TestLoaderHTTP(t *testing.T) {
_, err := l2.Load(x.path)
require.Error(err)
}

var testCaseRedirect = []testData{
{
path: "https://example.com/resource.yaml",
expectedContent: "https content",
},
koba1t marked this conversation as resolved.
Show resolved Hide resolved
}
for _, x := range testCaseRedirect {
expectedLocation := "https://test.com/resource.yaml"
hc := makeFakeHTTPClient(func(req *http.Request) *http.Response {
response := &http.Response{
StatusCode: MultipleChoicesRedirectCode,
Body: io.NopCloser(bytes.NewBufferString("")),
Header: make(http.Header),
}
response.Header.Add("Location", expectedLocation)
return response
})
l2 := l1
koba1t marked this conversation as resolved.
Show resolved Hide resolved
l2.http = hc
_, err := l2.Load(x.path)
require.Error(err)
var redErr *RedirectionError
var path string = ""
if errors.As(err, &redErr) {
path = redErr.NewPath
}
require.Equal(expectedLocation, path)
}
}

// setupOnDisk sets up a file system on disk and directory that is cleaned after
Expand Down
5 changes: 5 additions & 0 deletions api/internal/localizer/localizer.go
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,11 @@ func (lc *localizer) localizeResource(path string) (string, error) {
}
}
if fileErr != nil {
var redErr *loader.RedirectionError
if errors.As(fileErr, &redErr) {
path = redErr.NewPath
}

var rootErr error
locPath, rootErr = lc.localizeRoot(path)
if rootErr != nil {
Expand Down
5 changes: 5 additions & 0 deletions api/internal/target/kusttarget.go
Original file line number Diff line number Diff line change
Expand Up @@ -423,7 +423,12 @@ func (kt *KustTarget) accumulateResources(
if errors.Is(errF, load.ErrHTTP) {
return nil, errF
}
var redErr *load.RedirectionError
if errors.As(errF, &redErr) {
path = redErr.NewPath
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this code is using redirect Location path directly.
I am worried that newPath will be used by no check.
like the redirected path requires redirecting more, or the redirected path is missing contents.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I investigated in these scenarious

  1. new loader works only with directory and git repo because of that all checks will be done in common way in
    func (fl *FileLoader) New(path string) (ifc.Loader, error) {
  2. If newPath will be a simple url in new loader it will be treated as directory and kustomize finish work with error in common way
  3. If some redirects (301 to 309 http codes) will happen in loading process
    resp, err := hc.Get(path)
    They will be processed by http client out of the box
    Following this points I think that all possible checks have been done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, Maybe I explained it wrong.

The scenario I was concerned about that the redErr.NewPath address returning a 300 status code.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have understood! Sorry ((( I does not implement this case to avoid recursion. If you think that this case should be implemented I'll do it.

}
ldr, err := kt.ldr.New(path)

if err != nil {
// If accumulateFile found malformed YAML and there was a failure
// loading the resource as a base, then the resource is likely a
Expand Down
Loading