Feature request: Generation of release artifacts

[MarkoSagadin]

Is your feature request related to a problem? Please describe.

In our firmware development workflow, release artifact generation is an essential process that ensures built binaries and related files are organized, named, and packaged in a way that makes them easily accessible and understandable to end users. This structure simplifies firmware distribution and deployment, particularly for users who are not familiar with Zephyr's build system.

To clarify: by end users I mean "people downloading and using the artifacts in some way". So, QA, DevOps, etc. non-firmware technical people, that don't know Zephyr that well.

By following a structured artifact generation process, we ensure:

• Clear versioning and traceability of firmware builds.
• Easy navigation for users downloading firmware from GitHub releases.
• Retention of necessary files while removing unnecessary ones.

Our current release folder structure looks like this:

release
├── apps
│   └── app
│       ├── debug
│       │   ├── custom_nrf52840dk
│       │   │   ├── app-custom_nrf52840dk-v1.0.0-debug-99d4fc7.bin
│       │   │   ├── app-custom_nrf52840dk-v1.0.0-debug-99d4fc7.elf
│       │   │   └── app-custom_nrf52840dk-v1.0.0-debug-99d4fc7.hex
│       │   └── nrf52840dk_nrf52840
│       │       ├── app-nrf52840dk_nrf52840-v1.0.0-debug-99d4fc7.bin
│       │       ├── app-nrf52840dk_nrf52840-v1.0.0-debug-99d4fc7.elf
│       │       └── app-nrf52840dk_nrf52840-v1.0.0-debug-99d4fc7.hex
│       ├── release
│       │   ├── custom_nrf52840dk
│       │   │   ├── app-custom_nrf52840dk-v1.0.0-99d4fc7.bin
│       │   │   ├── app-custom_nrf52840dk-v1.0.0-99d4fc7.elf
│       │   │   └── app-custom_nrf52840dk-v1.0.0-99d4fc7.hex
│       │   └── nrf52840dk_nrf52840
│       │       ├── app-nrf52840dk_nrf52840-v1.0.0-99d4fc7.bin
│       │       ├── app-nrf52840dk_nrf52840-v1.0.0-99d4fc7.elf
│       │       └── app-nrf52840dk_nrf52840-v1.0.0-99d4fc7.hex
│       └── uart
│           ├── custom_nrf52840dk
│           │   ├── app-custom_nrf52840dk-v1.0.0-uart-99d4fc7.bin
│           │   ├── app-custom_nrf52840dk-v1.0.0-uart-99d4fc7.elf
│           │   └── app-custom_nrf52840dk-v1.0.0-uart-99d4fc7.hex
│           └── nrf52840dk_nrf52840
│               ├── app-nrf52840dk_nrf52840-v1.0.0-uart-99d4fc7.bin
│               ├── app-nrf52840dk_nrf52840-v1.0.0-uart-99d4fc7.elf
│               └── app-nrf52840dk_nrf52840-v1.0.0-uart-99d4fc7.hex
├── app-v1.0.0-99d4fc7.zip
├── app-v1.0.0-debug-99d4fc7.zip
└── app-v1.0.0-uart-99d4fc7.zip

Some comments about it:

• In apps folder we can have many different apps, above there is only one, named app.
• app had three different build configurations, release, uart, and debug.
• Inside each build configuration folder there is folder for each board.
• Inside of that board folder are the release artifacts, with full name that makes easily identifiable, even if they aren't located in the above structure.
• In the top release folder there are zip folders that contain release artifacts for release, debug, uart build configurations. In our process those get uploaded to the GitHub release page as assets.
• Although the above example only lists app firmware, the tool also works with samples or tests, in other words they aren't treated any differently.

Until now, this process has been managed by our in-house tool, East, which automates firmware builds, final structuring and renaming. However, we are now migrating to Zephyr’s Twister framework as it provides greater flexibility and aligns better with the Zephyr ecosystem. The issue we face is that Twister can generate build artifacts (more specifically it creates build folders) but lacks functionality for renaming and packaging them into a structured format. Additionally, it would be a quite hard task for a non-Zephyr user to navigate the build directory and pick correct file which they would have to flash/DFU to the target.

Twister's --prep-artifacts-for-testing flag partially addresses this but does not fully meet our needs. The resulting folder structure includes unnecessary files and does not retain Nordic's build system-generated files (like dfu_application.zip).

Describe the solution you'd like

We propose enhancing Twister with an optional feature to rename and package release artifacts into a structured format. This could be implemented in one of the following ways:

• As an additional Twister flag to enable structured artifact packaging.
• As a separate west command that processes the twister-out folder and generates the final release directory.

In both cases the there will probably be a need for some file that specifies the naming convention of how the files are named and which files to rename. Although the tool could offer some sane defaults, it is expected that that can differ widely from project to project.

I have seen that --keep-artifacts flag was recently added to the Twister. Something similar would be needed for the described feature plus the renaming part.

Describe alternatives you've considered

• Use Twister as is, and add only renaming part to our internal tool.

Additional context

Our current approach using East follows a structured artifact naming and packaging convention. More details on our process can be found here:

• East release command documentation
• Zephyr Developer Summit talk on East
• Current naming convention guidelines

Keep in mind: our artifact naming convention works for us (or it at least did, until we started using XIP in our projects), but it probably won't work for the this possible enhancement.Do members of Zephyr community have the similar artifact packaging needs? If yes, how are they currently resolved?

Would the Zephyr community find value in such an enhancement? If so, which approach would be preferable for upstream contribution?

[henrikbrixandersen]

Twister is intended to be a test runner, not a release building tool.