runtime benchmarks

[nikomatsakis]

We're using this issue to track work on making benchmark suite for tracking efficiency of generated code (as opposed to efficiency of compilation time itself).

TODO: It might be good to copy over the list of items from description on rust-lang/rust#31265 . Or figure out some other way to encode them/curate them as a todo list for investigation points.

(original description follows)

---

It'd be great if there were an option to add a make bench target to makefiles that would run runtime benchmarks. We could scrape the output from cargo bench, for example, and include those as data points.

I could then merge the entries from https://github.com/nikomatsakis/rust-runtime-benchmarks and use those.

[Mark-Simulacrum]

Functionality wise, most of the code here is semi-compatible with another "time" field, though updates would be needed. However, I think the primary work needs to be done in the benchmarks repository more so than here. Once that is done it shouldn't be difficult to work with the data, though displaying it will be somewhat difficult, especially until we deprecate and remove the JS backend from consideration (I'd prefer to avoid maintaining two separate, incompatible versions of frontend code).

[nikomatsakis]

@Mark-Simulacrum ok; I can investigate the data gathering if you would be willing (at some later point) to tackle the data processing :)

[Mark-Simulacrum]

Sounds good to me. I think the best option here would be to add a runtimes field to the top-level JSON object in the processed/ directory. I'm uncertain specifically what data we will be gathering, so I can't speak much as to the layout within the field.

[lqd]

almost 7 years to the day, great work @Kobzol 🎉