Experiment request

[zukatsinadze]

Hi @DonggeLiu,

would it be possible to run these two experiments for us? It's for an ongoing project at EPFL, HexHive Lab

Experiment 1: no seeds, no dictionaries, all targets

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-03-31-libafl-scheduling1 --fuzzers libafl_naive libafl_random libafl_weighted libafl_covaccount libafl_fast libafl_explore --no-seeds --no-dictionaries

Experiment 2: oss-fuzz corpus, all targets

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-03-31-libafl-scheduling2 --fuzzers libafl_naive libafl_random libafl_weighted libafl_covaccount libafl_fast libafl_explore --oss-fuzz-corpus

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-04-01-libafl-scheduling1 --fuzzers libafl_naive libafl_random libafl_weighted libafl_covaccount libafl_fast libafl_explore --no-seeds --no-dictionaries

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-04-01-libafl-scheduling2 --fuzzers libafl_naive libafl_random libafl_weighted libafl_covaccount libafl_fast libafl_explore --oss-fuzz-corpus

[DonggeLiu]

Experiment 2025-04-01-libafl-scheduling1 data and results will be available later at:
The experiment data.
The experiment report.
The experiment report(experimental).

Experiment 2025-04-01-libafl-scheduling2 data and results will be available later at:
The experiment data.
The experiment report.
The experiment report(experimental).

BTW, I am not very confident that these experiments can work, because of some recent flacky issue in FuzzBench.
We are working on a solution, but it will take some time.

[zukatsinadze]

Hi,

Looks like there is something wrong with both experiments (especially the second one).
Is it related to the fuzzbench issue that you mentioned?

[DonggeLiu]

Hi @zukatsinadze,
I reckon these two reports are available at:
The experiment report(experimental).
The experiment report(experimental).

[zukatsinadze]

Hi @DonggeLiu,

Yes, I meant those two reports.
The first one is missing trials and seems to be stuck for a few days already.
The second one is random garbage.

[DonggeLiu]

The first one is missing trials and seems to be stuck for a few days already.

Yes this is strange. I just checked the cloud log, it appears to be still running, which is unexpected.

The second one is random garbage.

The second finished in time at 2025-04-03 20:09:55.848.
But it indeed had the problem I mentioned earlier, which likely breaks coverage measurement.

Unfortunately the upgraded version of this will take some time to finish and we don't have the capacity to address this problem in the short-term.