-
Notifications
You must be signed in to change notification settings - Fork 424
Description
We should really have full test suite runs in the CI somewhere. For example, the full VTR, Koios, Titan, and Titanium benchmark suites. The idea is that we can check that master runs all of the tests in these suites and quickly get QoR with the click of a button.
This can turn into a "monthly" or "release" CI test that runs every so often. From personal experience, I found that running all of these test suites on 16 cores will take around 18 hours; but this was when I was running baseline and my changes in the same flow, so it may take much less time.
For AP, I have something similar to what I want:
https://github.com/AlexandreSinger/fpga-analytical-placement/tree/main/testing/test_suites
A couple of circuits take a very very long time to run, so we would either need to disable them or move them to a larger device.