[RFC] Predictable and (almost) fully automated WAMR releases

[loganek]

Feature

I'd like to discuss with the community standardizing release cadence of WAMR releases.

Benefit

The customers can know in advance any WAMR releases and plan accordingly. Customers no longer have to request ad-hoc requests.

Implementation

As a first step we need to make sure all of the tests are automated and publicly available, so Intel engineers no longer have to be overloaded with release requests; any maintainer can coordinate the release. Based on the discussion we had in the community meetings, there are two major actions to achieve this:

1. decouple WAMR IDE from WAMR runtime, as IDE right now requires a bunch of manual tests (and the effort to automate them would likely be significant)
2. enable tests that require QEMU

Whereas it's unknown yet whether 2. can be implemented completely using github actions, I think the problem overall is solvable one way or another (e.g. by using instances from 3rd party cloud providers etc.) so whoever picks this up can figure out the implementation detail.
The 1. can look a bit concerning, so I'd like to clarify that I'm not suggesting for it to completely decouple IDE from the WAMR project but instead keep it around with the same versioning schema, but make releases on best-effort basis.

The regular releases will include any changes that happened since the previous release, and if the particular feature will not be ready, it'll have to wait for another release (hence I'm proposing relatively frequent releases).

When it comes to the versioning, I propose incrementing MINOR version by default and change it when:

• there are mainly bugfixes and minor improvements, only the PATCH number should be increased
• for any breaking changes, MAJOR version should be incremented

I guess the preference would be not to change the MAJOR version too often, so for that we should likely maintain a dev branch where eventually all the breaking changes are included, and then added to the main branch through a single pull request to make sure they all are included in a single release.

This is just a proposal and I do understand it might not touch on all the potential problems and details. I'm open for feedback and suggestions; feel free to leave your comment below.

[no1wudi]

1. decouple WAMR IDE from WAMR runtime, as IDE right now requires a bunch of manual tests (and the effort to automate them would likely be significant)

Agreed. As a tool used during the development phase, the cost of rolling out IDE updates to end-users is significantly lower compared to the core runtime, allowing for a faster and more flexible release cadence.

enable tests that require QEMU

If possible, we should incorporate as many automated tests as we can. Ideally, if quality can be guaranteed, we should be able to perform a release operation at any commit point.
Of course, it is also necessary to take into account the features and bug fixes included in certain versions.

[lum1n0us]

decouple WAMR IDE from WAMR runtime, as IDE right now requires a bunch of manual tests (and the effort to automate them would likely be significant)

Even inclined to add "lldb wamr patched" to a list of items that are only released if changes have been made.

[lum1n0us]

When it comes to the versioning, I propose incrementing MINOR version by default and change it when: there are mainly bugfixes and minor improvements, only the PATCH number should be increased
for any breaking changes, MAJOR version should be incremented
I guess the preference would be not to change the MAJOR version too often, so for that we should likely maintain a dev branch where eventually all the breaking changes are included, and then added to the main branch through a single pull request to make sure they all are included in a single release.

Agreed. This approach makes it possible for a GitHub action to determine the next version string.

[lum1n0us]

#4171