Skip to content

[CI] Full Benchmark Suite Tests #3241

@AlexandreSinger

Description

@AlexandreSinger

We should really have full test suite runs in the CI somewhere. For example, the full VTR, Koios, Titan, and Titanium benchmark suites. The idea is that we can check that master runs all of the tests in these suites and quickly get QoR with the click of a button.

This can turn into a "monthly" or "release" CI test that runs every so often. From personal experience, I found that running all of these test suites on 16 cores will take around 18 hours; but this was when I was running baseline and my changes in the same flow, so it may take much less time.

For AP, I have something similar to what I want:
https://github.com/AlexandreSinger/fpga-analytical-placement/tree/main/testing/test_suites

A couple of circuits take a very very long time to run, so we would either need to disable them or move them to a larger device.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions