Skip to content

Conversation

@yangw-dev
Copy link
Contributor

@yangw-dev yangw-dev commented Nov 14, 2025

#Overview
Add regression report for helion

frequency

the summary report is generated daily

what is considered as regression for helion

  1. we find baseline point used to judge new data: median of (4th - 8th day speedup data)
  2. if more than 2 new dp of helion speedup are continuously 15% lower than the basline value, considered as regression

Demo Regression Report

https://torchci-git-addhelionregressionreport-fbopensource.vercel.app/benchmark/regression/report/ada0e5ba-874b-47ff-b76c-b281ac08d179

image

Notification

Currently we do not trigger workplace chat notification, this is experimental.
But we do:

  1. have a github issue that will be used to accept regression report as link, can be used for notification when it's ready
  2. have helion dashboard to access to the regression report list, and signal regression if find any

#7472

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 14, 2025
@vercel
Copy link

vercel bot commented Nov 14, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Preview Updated (UTC)
torchci Ignored Ignored Preview Nov 14, 2025 5:45pm

@yangw-dev yangw-dev changed the title Add Helion Regression Add Helion Regression Config Nov 14, 2025
@yangw-dev yangw-dev requested a review from huydhn November 14, 2025 03:47
@yangw-dev yangw-dev marked this pull request as ready for review November 14, 2025 03:47
@yangw-dev yangw-dev requested a review from oulgen November 14, 2025 03:56
@yangw-dev yangw-dev merged commit 39523dd into main Nov 15, 2025
6 of 7 checks passed
@yangw-dev yangw-dev deleted the addHelionRegressionReport branch November 15, 2025 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants