Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add frontend performance CI integration #4761

Open
wants to merge 54 commits into
base: develop
Choose a base branch
from

Conversation

schlessera
Copy link
Collaborator

@schlessera schlessera commented May 23, 2020

Summary

This PR adds a GitHub Actions workflow called "Build, Test & Measure" with currently two separate jobs:

release-zip

This job checks out the source, fetches Composer and npm dependencies and does a complete build.

It then uploads the produced ZIP file as an artifact with the filename having the form amp-{branch}-{short-sha}.zip.
Image 2020-05-23 at 3 50 06 PM
Todo:

  • The branch name extraction probably needs fine-tuning to work across all events.

Note: This job is meant to go into a different PR and is only located in here as it is a requirement for the next job.

lighthouse

(depends on release-zip)

This job sets up a complete WordPress installation, installs both the AMP and the PWA plugins and then runs the Lighthouse CI over a set of URLs to audit the performance.

The assertions are currently disabled, because we're not passing the default recommendation yet, and haven't discussed what our hard targets are.

Todo:

  • The build token needs to be added as a GitHub secret so that we can upload the gathered metrics to the Lighthouse CI server.
  • The Lighthouse CI server URL needs to be provided via a GitHub secret as well to point to the correct domain.
  • The site needs to be populated with test data.
  • An MU Plugin is needed that allows for disabling the optimizer and/or SSR via URL arguments.

Fixes #4569

Checklist

  • My pull request is addressing an open issue (please create one otherwise).
  • My code is tested and passes existing tests.
  • My code follows the Engineering Guidelines (updates are often made to the guidelines, check it out periodically).

@schlessera schlessera added the Infrastructure Changes impacting testing infrastructure or build tooling label May 23, 2020
@googlebot googlebot added the cla: yes Signed the Google CLA label May 23, 2020
@schlessera schlessera force-pushed the add/4569-frontend-performance-ci branch 2 times, most recently from 25fbaf0 to 1f6db44 Compare May 23, 2020 15:20
schlessera added a commit that referenced this pull request May 25, 2020
This file was initially built within #4761, but moved into a separate PR for faster merging.
@schlessera schlessera mentioned this pull request May 25, 2020
7 tasks
@schlessera schlessera marked this pull request as ready for review June 2, 2020 06:37
@schlessera
Copy link
Collaborator Author

schlessera commented Jun 2, 2020

The Lighthouse data collection works and can be seen here: http://lhci.amp-wp.org/app/projects/amp-for-wordpress/dashboard

Current state of a Twentytwenty homepage:
Image 2020-06-02 at 8 34 26 AM

@schlessera
Copy link
Collaborator Author

The Lighthouse CI server does not provide a way for making comparisons across URLs, so it's unnecessarily difficult to see the impact of the Optimizer or SSR right now.

I've opened an issue on the server repo: GoogleChrome/lighthouse-ci#338

@github-actions
Copy link
Contributor

github-actions bot commented Jun 10, 2020

Plugin builds for 883c28c are ready 🛎️!

@schlessera schlessera force-pushed the add/4569-frontend-performance-ci branch from 7b2a8c8 to 309c7a1 Compare July 20, 2020 07:07
@schlessera
Copy link
Collaborator Author

^^^ ⚠️ Force-pushed ⚠️ ^^^

@schlessera schlessera changed the base branch from develop to add/4571-build-reference-site July 20, 2020 07:09
@schlessera schlessera added WS:Perf Work stream for Metrics, Performance and Optimizer and removed WS:Perf Work stream for Metrics, Performance and Optimizer labels Aug 12, 2020
Base automatically changed from add/4571-build-reference-site to develop October 2, 2020 20:16
@CLAassistant
Copy link

CLAassistant commented Oct 8, 2020

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes Signed the Google CLA Infrastructure Changes impacting testing infrastructure or build tooling WS:Perf Work stream for Metrics, Performance and Optimizer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add performance metrics to CI
3 participants