Akin to https://github.com/tidyverse/ellmer/pull/760 and https://github.com/tidyverse/ellmer/pull/333