Build pip wheel and validate AOT workflow on different backends & host environments in CI

[larryliu0820]

🚀 The feature, motivation and pitch

We need to guard our AOT workflow in CI, to make sure our pip wheel can give users out-of-the-box user experience.

In CI we should cover the flow of:

• Build pip wheel on PR or trunk, for MacOS or Linux
  • This exists right now but need to leverage the artifact in following steps.
• Install pip wheel
• Export a simple model (MV3) with common backend delegation (XNNPACK, Coreml, QNN) and retrieve .pte files
• Load the .pte files in python and run sample input (optional, depend on feasibility)

	MacOS	Linux
XNNPACK
CoreML		No need to run
QNN

Alternatives

No response

Additional context

No response

RFC (Optional)

No response

cc @lucylq

[jathu]

What are your thoughts on creating a new workflow just to check this flow?

For example:

# .github/workflows/just-works.yml

name: Just Works

on:
  pull_request:

jobs:
  configuration-matrix:
    strategy:
      matrix:
        os:
          - ubuntu-latest
          - macos-m1-stable
    runs-on: ${{ matrix.os }}
    steps:
      - uses: Build wheel
      - name: Install wheel
      - name: Test wheel on host
      - name: Test wheel on iOS
      - name: Test wheel on Android

This would have a few overlaps with build-wheels-m1.yml / build-wheels-linux. However, that workflow only runs when actually releasing the wheel — in fact it always attempts to upload the artifact. The main motivation for a separate workflow:

• Focus on the core work flows we described in "Just Works": export → run on host → run on target
• Don't interfere with the more sensitive artifact upload workflow
• Run this workflow on every PR and main pushes

[larryliu0820]

@jathu We actually build nightly wheels. I was hoping that we can extend the nightly wheel build job to include the test we want to add. Here's the nightly flow:

[jathu]

@larryliu0820 it would be great if we can do it in every PR, but nightly is good enough too. So we currently run a simple model test in the wheel workflow. We will be updating the workflow to build MV3, and also test the respective outputs on iOS and Android?

[larryliu0820]

We will be updating the workflow to build MV3, and also test the respective outputs on iOS and Android?

Yeah, though the test may not have to be in the same environment as the nightly flow, because you may need a different docker container for Android test. I think we can figure out a way to write a new flow that depends on the nightly flow artifacts.