Description
Recently, support was added to the perfbot to build the artifacts of all the PRs in a rollup. For example, here we can see each sha1 artifact that was built. This is to help perf triage, by manually enqueuing a perf run for suspicious PRs.
These artifacts could also be used by cargo-bisect-rustc for some bisection cases, as a limited workaround to today's stopping bisection at a rollup altogether. Note however that these artifacts are of the PRs themselves, not the accumulated commits of all previous PRs in the rollup: the artifacts for the 3rd PR in the rollup are only of that PR, and do not contain the 1st and 2nd PR's commits.
This is a limitation compared to fully supporting rollups here, but is a sensible step towards that goal: in my experience, it's often the case, but not always, that single PRs within rollups are the source of ICEs or bugs, rather than a unfortunate combination of some of the PRs in the rollup (rare in practice, as the rolled-up PRs are also quite random in general).
Today, bisecting within a rollup requires manually building rustc at each of the rolled-up PR's merge commit, whereas using these rolled-up PRs artifacts would be faster and simpler to find, what I believe is the more common source of bugs: a single PR.
It would be more informative than authoritative: not finding the regression source using these individual artifacts would mean that it's caused by a >1 PRs combination, and proceeding would still require building rustc (and that is fine and expected).
cc @rylev who asked me to open an issue about this