Support passing arbitrary flags to run_all_experiments.py via request_pr_exp.py.

[oliverchang]

Currently there is very little control over flags passed to PR experiments:

oss-fuzz-gen/ci/k8s/pr-exp.yaml

Line 34 in 3dd239a

command: ["/bin/bash", "report/docker_run.sh", "${GKE_EXP_BENCHMARK}", "${GKE_EXP_NAME}", "${GKE_EXP_FUZZING_TIMEOUT}", "ofg-pr", "${GKE_EXP_LLM}", "${GKE_EXP_DELAY}"]

We should ideally support piping arbitrary flags via request_pr_exp.py.

e.g.

request_pr_exp.py -p 123 -n NAME -- --model=foo --context 

[DonggeLiu]

Another issue about docker_run.sh is that it assumes the order of command line args.
This is a good opportunity to upgrade it to a Python script to allow more flexible args.

[KKSurendran06]

Hi @DonggeLiu

I’d like to start working on converting docker_run.sh into a Python script (docker_run.py) to improve flexibility in argument handling. My plan is to:

• Replacedocker_run.sh with docker_run.py, using named arguments (--flag value) instead of fixed positional arguments.
• Update pr-exp.yaml to call docker_run.py instead of docker_run.sh.
• Modify docker_run.py to dynamically parse command line arguments and construct the command to execute run_all_experiments.py.
• Support passing additional flags so they are correctly forwarded to run_all_experiments.py.

Would this approach align with what you had in mind? Also, do you have any specific requirements or constraints I should be aware of before starting?

Looking forward to your guidance!

Thanks!

[DonggeLiu]

Thanks @KKSurendran06! That's very kind of you.

Replacedocker_run.sh with docker_run.py, using named arguments (--flag value) instead of fixed positional arguments.

Yep, this is a good starting point.

Update pr-exp.yaml to call docker_run.py instead of docker_run.sh.
Modify docker_run.py to dynamically parse command line arguments and construct the command to execute run_all_experiments.py.
Support passing additional flags so they are correctly forwarded to run_all_experiments.py.

Yes, and since the main goal is to allow request_pr_exp.py to pass arbitrary flags to run_all_experiments.py, we will need to modify request_pr_exp.py too.

[Snehil-Shah]

Hey @KKSurendran06, I am already working on this, as assigned here.

[KKSurendran06]

@Snehil-Shah

I see what happened now. I didn’t realize this was a sub-issue of #830, and since this one remained unclaimed, I went ahead and worked on it. Now that I see you were assigned the parent issue but not this one specifically, I understand the mix-up.

That said, I’ve already completed the changes, and they’re ready to be merged. But since neither of us expected this mix-up, I’m happy to step back if you’d prefer to continue solo. Alternatively, if you’d like to collaborate, I’d be happy to work together on this.

Thanks!

[Snehil-Shah]

Hey @KKSurendran06, as you have already completed the changes, feel free to make a PR. there's no point in doing double work again :)