Incorrect Benchmark Status on report's Index Page

[DonggeLiu]

The status of each benchmark on the index page is currently determined solely by the trials included in the report. This approach leads to misleading statuses when not all trials are accounted for in the report.

Example:

• A benchmark has 10 trials in total.
• The report has the status of only 1 trial, and it is marked as Done.
• The index page shows the benchmark status as Done.

Expected Behavior:
The benchmark's status should be Running because results for the remaining 9 trials are still pending.

[ay-bh]

@DonggeLiu Hi, I'd like to work on this issue

[DonggeLiu]

Sure, thanks @ay-bh !

[Jett-Code]

Can this issue be assigned to multiple person? @DonggeLiu
I was working on the same but didn't ask for it.

[ay-bh]

Hi @Jett-Code,
Since this issue is relatively small, it might not be efficient for both of us to work on it simultaneously. I've already implemented a potential fix. It might be more helpful if you could tackle another issue instead. Hope that's okay!
Thanks for understanding!

[ay-bh]

Hey @DonggeLiu,

Would like to get your thoughts on this, I feel this should address it:

In report/common.py

Creating a helper method _get_expected_trials_count:
- This method scans the relevant benchmark directories (fuzz_targets and raw_targets).
- It identifies all filenames representing trials based on specific file extensions.
- Counts unique trial IDs extracted from these filenames to accurately determine the total expected number of trials.

Updating the logic in match_benchmark:
- Retrieve the expected number of trials using _get_expected_trials_count.
- Mark benchmarks as "Done" only if:
- There are actual results available.
- The total number of completed results exactly matches the expected trial count.
- All individual results have the completed status.
- Otherwise, benchmarks are marked as "Running" to reflect that trials are still in progress or incomplete.

if this looks good to you, I'll raise a PR.
Thanks

[DonggeLiu]

Thanks! Replied in your PR.