panic: runtime error

[frjcomp]

Hi I've tried to integrate this lib and I receive panics.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2487472]

goroutine 1 [running]:
github.com/gofri/go-github-pagination/githubpagination/searchresult.(*Merger).Digest(0xc00106b630, {0x77ee315c3848, 0xc008488460})
        /go/pkg/mod/github.com/gofri/go-github-pagination@v1.0.0/githubpagination/searchresult/gh_merger.go:31 +0xf2
github.com/gofri/go-github-pagination/githubpagination/jsonmerger.(*UnprocessedMap).ReadNext(0xc0018ad4e8, {0x327a168?, 0xc008488460?})
        /go/pkg/mod/github.com/gofri/go-github-pagination@v1.0.0/githubpagination/jsonmerger/map_merger.go:34 +0x9e
github.com/gofri/go-github-pagination/githubpagination/jsonmerger.(*merger).ReadNext(0xc001efb518, {0x3278180?, 0xc002364f90?})
        /go/pkg/mod/github.com/gofri/go-github-pagination@v1.0.0/githubpagination/jsonmerger/json_merger.go:33 +0x82
github.com/gofri/go-github-pagination/githubpagination/drivers.(*SyncPaginationDriver).OnNextResponse(0xc00640ab40?, 0xc006408870?, 0x4f4db80?, 0x77ee7af395b8?)
        /go/pkg/mod/github.com/gofri/go-github-pagination@v1.0.0/githubpagination/drivers/sync_driver.go:29 +0x26
github.com/gofri/go-github-pagination/githubpagination.(*GitHubPagination).RoundTrip(0xc002248a08, 0xc00640ab40)
        /go/pkg/mod/github.com/gofri/go-github-pagination@v1.0.0/githubpagination/pagination.go:71 +0x1c8
github.com/google/go-github/v69/github.(*Client).WithAuthToken.func1(0x12?)
        /go/pkg/mod/github.com/google/go-github/v69@v69.2.0/github/github.go:352 +0x19c
github.com/google/go-github/v69/github.roundTripperFunc.RoundTrip(0x1?, 0x32653c0?)
        /go/pkg/mod/github.com/google/go-github/v69@v69.2.0/github/github.go:1772 +0x19
net/http.send(0xc00640aa00, {0x32653c0, 0xc0012f7b00}, {0x1?, 0x3?, 0x0?})
        /usr/local/go/src/net/http/client.go:259 +0x5e4
net/http.(*Client).send(0xc0012f7ad0, 0xc00640aa00, {0xc001df16f0?, 0x41d945?, 0x0?})
        /usr/local/go/src/net/http/client.go:180 +0x98
net/http.(*Client).do(0xc0012f7ad0, 0xc00640aa00)
        /usr/local/go/src/net/http/client.go:725 +0x8bc
net/http.(*Client).Do(...)
        /usr/local/go/src/net/http/client.go:590
github.com/google/go-github/v69/github.(*Client).bareDo(0xc0009b2488, {0x3290248, 0x4f4db80}, 0xc0012f7ad0, 0xc00640a8c0)
        /go/pkg/mod/github.com/google/go-github/v69@v69.2.0/github/github.go:871 +0x287
github.com/google/go-github/v69/github.(*Client).BareDo(...)
        /go/pkg/mod/github.com/google/go-github/v69@v69.2.0/github/github.go:957
github.com/google/go-github/v69/github.(*Client).Do(0x0?, {0x3290248?, 0x4f4db80?}, 0xc0009b2488?, {0x273a520, 0xc0016eebb8})
        /go/pkg/mod/github.com/google/go-github/v69@v69.2.0/github/github.go:1025 +0x6b
github.com/google/go-github/v69/github.(*ActionsService).ListRepositoryWorkflowRuns(0xc0009b27b0, {0x3290248, 0x4f4db80}, {0xc0009675ba?, 0x2d13d04?}, {0xc0000574d0, 0x11}, 0xc00847db80)
        /go/pkg/mod/github.com/google/go-github/v69@v69.2.0/github/actions_workflow_runs.go:198 +0x18f
github.com/CompassSecurity/pipeleak/cmd/github.iterateWorkflowRuns(0xc0009b2488, 0xc001070708)
        /workspaces/pipeleak/src/pipeleak/cmd/github/scan.go:266 +0xa5
github.com/CompassSecurity/pipeleak/cmd/github.searchRepositories(0xc0009b2488, {0x7fff88dae6f4, 0xd})
        /workspaces/pipeleak/src/pipeleak/cmd/github/scan.go:169 +0x117
github.com/CompassSecurity/pipeleak/cmd/github.scan(0xc0009b2488)
        /workspaces/pipeleak/src/pipeleak/cmd/github/scan.go:107 +0x19c
github.com/CompassSecurity/pipeleak/cmd/github.Scan(0xc0015fa600?, {0x2d12aa6?, 0x4?, 0x2d12aaa?})
        /workspaces/pipeleak/src/pipeleak/cmd/github/scan.go:87 +0x11d
github.com/spf13/cobra.(*Command).execute(0xc0022c2c08, {0xc000962f30, 0x9, 0x9})
        /go/pkg/mod/github.com/spf13/cobra@v1.9.1/command.go:1019 +0xa7b
github.com/spf13/cobra.(*Command).ExecuteC(0x4b2d640)
        /go/pkg/mod/github.com/spf13/cobra@v1.9.1/command.go:1148 +0x40c
github.com/spf13/cobra.(*Command).Execute(...)
        /go/pkg/mod/github.com/spf13/cobra@v1.9.1/command.go:1071
github.com/CompassSecurity/pipeleak/cmd.Execute(...)
        /workspaces/pipeleak/src/pipeleak/cmd/root.go:27
main.main()
        /workspaces/pipeleak/src/pipeleak/main.go:11 +0x1b
exit status 2

Using the following setup:

rateLimiter := github_ratelimit.New(nil,
		github_primary_ratelimit.WithLimitDetectedCallback(func(ctx *github_primary_ratelimit.CallbackContext) {
			log.Info().Str("category", string(ctx.Category)).Time("reset", *ctx.ResetTime).Msg("Primary rate limit detected")
		}),
		github_secondary_ratelimit.WithLimitDetectedCallback(func(ctx *github_secondary_ratelimit.CallbackContext) {
			log.Info().Time("reset", *ctx.ResetTime).Dur("totalSleep", *ctx.TotalSleepTime).Msg("Primary rate limit detected")
		}),
	)
	paginator := githubpagination.NewClient(rateLimiter,
		githubpagination.WithPerPage(100),
	)
	client := github.NewClient(paginator).WithAuthToken(options.AccessToken)

Then in the following function the client is used.

func iterateWorkflowRuns(client *github.Client, repo *github.Repository) {
	opt := github.ListWorkflowRunsOptions{	}
	wfCount := 0
	for {
               # thats where the error occurs
		workflowRuns, resp, err := client.Actions.ListRepositoryWorkflowRuns(context.Background(), *repo.Owner.Login, *repo.Name, &opt)
		if err != nil {
			log.Error().Stack().Err(err).Msg("Failed Fetching Workflow Runs")
			return
		}

		for _, workflowRun := range workflowRuns.WorkflowRuns {
			log.Debug().Str("name", *workflowRun.DisplayTitle).Str("repo", *repo.HTMLURL).Msg("Workflow Run")
			downloadWorkflowRunLog(client, repo, workflowRun)

			if options.Artifacts {
				listArtifacts(client, workflowRun)
			}

			wfCount = wfCount + 1
			if wfCount >= options.MaxWorkflows && options.MaxWorkflows > 0 {
				log.Debug().Str("name", *workflowRun.DisplayTitle).Str("repo", *repo.HTMLURL).Msg("Reached MaxWorkflow runs, skip remaining")
				return
			}
		}

		if resp.NextPage == 0 {
			break
		}
		opt.Page = resp.NextPage
	}
}

The following branch can be used to reproduce the behavior:
https://github.com/CompassSecurity/pipeleak/blob/repro-go-github-ratelimit/src/pipeleak/cmd/github/scan.go#L266

cd src/pipeleak
go run main.go gh scan -t github_pat_[redacted]  --confidence high,high-verified --maxWorkflows 10 --search infra-as-code -v

Not sure if I'm doing something wrong on my side or if its a bug. Thx for having a look!

[gofri]

Hi @frjcomp,
Thanks for opening the issue. It appears that the issue is actually with go-github-pagination.
Do you mind reopening the issue there?
https://github.com/gofri/go-github-pagination/issues

p.s.1. I'm a bit confused - why do you iterate the pages if you use the pagination middleware?
p.s.2. I'm looking into it anyway, thanks for adding a reproduction.

[gofri]

I think I fixed the panic, but my best quick guess is that it happened because you iterate over the pages although you use the middleware.
please use go-github-pagination @ v1.0.1 and let me know if it works for you :)
for reference: gofri/go-github-pagination#11

[frjcomp]

Thx for the fast feedback! Yes, seems to have fixed it and have mixed some things up copy pasting from the readme :)