Prioritize Bash if LLM responses both Bash and Conclusion

[DonggeLiu]

Previously, when the LLM response included both bash commands and a conclusion, we ignored the bash command and compiled the fuzz target in the conclusion alone. This approach is suboptimal, as conclusions may be premature, and the bash command often provides useful context to refine them.

To improve accuracy, we should prioritize the bash command when both are present in a response.

A more robust solution would involve structuring tasks to prevent mixed responses (e.g., using task-focused agents). But until then, prioritizing bash commands in mixed responses provides a fallback/default solution.

[DonggeLiu]

/gcbrun exp -n dg -ag

[DonggeLiu]

https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2024-11-10-716-dg-comparison/index.html

[DonggeLiu]

Thanks @mihaimaruseac : )
I plan to break these steps into individual agents in the next step, do you have any advices/suggestions?

[mihaimaruseac]

Oh, that's great, the prompt is quite long right now and I think that splitting it over multiple agents might bring more benefits.

I think one best rule of thumb is to have each agent use only one tool. So we should not have an agent use both bash and have access to the compiler, for example