Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maximize overall coverage by showing overlapping benchmarks #356

Open
DonggeLiu opened this issue Jun 19, 2024 · 3 comments
Open

Maximize overall coverage by showing overlapping benchmarks #356

DonggeLiu opened this issue Jun 19, 2024 · 3 comments

Comments

@DonggeLiu
Copy link
Collaborator

Extended from #354.
Context:
Currently, OSS-Fuzz-Gen selects functions with low coverage. However, some selected functions may execute the same code blocks of the project, leading to minimal project-level coverage improvement.

Goal:
Displaying the "per-project coverage increment" alongside the "per-benchmark" coverage increment will help us identify instances where overlapping code blocks are being tested. This insight will highlight cases where we should re-select benchmarks to enhance overall efficiency.

@DavidKorczynski
Copy link
Collaborator

Related: Fuzz Introspector has an analysis feature for "Optimal targets" which does a recursive analysis on which functions to target. Basically, pick the first best function to target, and then FI analyses the impact of this before proceeding to analyse which is the next best target to analyse. It's been around for some time and is in the UI e.g.: https://storage.googleapis.com/oss-fuzz-introspector/htslib/inspector-report/20240617/fuzz_report.html#Optimal-target-analysis

I'll expose this as an oracle API so we can use it for benchmark generation.

@DonggeLiu
Copy link
Collaborator Author

ACK, I will close this then.

@DavidKorczynski
Copy link
Collaborator

I still think this is a good thing to pursue though! Introspector's "Optimal targets" is just one heuristic trying to solve a similar problem, and the feature suggested in this issue is also relevant now that we can extract target functions from a diverse set of oracles.

@DonggeLiu DonggeLiu reopened this Jun 20, 2024
DavidKorczynski added a commit to ossf/fuzz-introspector that referenced this issue Jul 2, 2024
AdamKorcz pushed a commit to ossf/fuzz-introspector that referenced this issue Jul 2, 2024
* webapp: apis: enable sorting out non-declared functins

Signed-off-by: David Korczynski <[email protected]>

* webapp: add optimal target analysis API

Ref:
google/oss-fuzz-gen#356 (comment)

Signed-off-by: David Korczynski <[email protected]>

---------

Signed-off-by: David Korczynski <[email protected]>
DavidKorczynski added a commit that referenced this issue Jul 2, 2024
Create benchmarks using FI's optimal target analysis.

Ref:
#356 (comment)

Signed-off-by: David Korczynski <[email protected]>
DavidKorczynski added a commit that referenced this issue Jul 3, 2024
Create benchmarks using FI's optimal target analysis.

Ref:

#356 (comment)

---------

Signed-off-by: David Korczynski <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants