CI: integrate macOS

[ChinYikMing]

The current CI pipeline utilizes Linux-based x86-64 and Aarch64 machines for verification and validation (V&V). To enhance the portability of rv32emu, it is both reasonable and feasible to integrate macOS into the CI pipeline.

[ChinYikMing]

A simple CI pipeline for system emulation:

...
+macOS:
+    runs-on: macos-latest
+   steps:
+    - uses: actions/checkout@v4
+    - name: install-dependencies
+      run: |
+            brew install make dtc expect
+    - name: boot Linux kernel test
+      run: |
+            make clean && make ENABLE_SYSTEM=1 && make ENABLE_SYSTEM=1 artifact -j$(sysctl -n hw.logicalcpu)
+            .ci/boot-linux.sh
+            make ENABLE_SYSTEM=1 clean
 ...

[ChinYikMing]

The fetching of the checksum occasionally fails in previous actions(1, 2. 3, ...).

To fully close this issue, the failure should not occur too frequently and the macOS pipeline should be capable of performing V&V for both user and system emulation.

Found at #508

[ChinYikMing]

The occasional failure could be fixed by #536.

[ChinYikMing]

CI breakage on macOS/M1 when running JIT test: CI link

It obviously shown that the function is not called ever. @vacantron Can you check it?

[vacantron]

It seems like it's been a long time since the last test on Arm64/MacOS. I'll fix it later.

[ChinYikMing]

I will try to integrate the minimal runnable CI for macOS first.

[ChinYikMing]

Shall we support macOS/Intel? The CI [1] breaks since it expects llvm18 library installed at path /usr/local/opt/ but Makefile detects at path /opt/homebrew/opt/ which is the default path of M series chips, so macOS/M1 runners do not suffer this when running JIT test suite.

Additionally, the behavior of gcc-14 on macOS/M1 does not align to clang, as different errors are displayed in [1], with most of them clearly related to the linker. Will try to fix it and embrace gcc-14 for macOS/M1.

[1] macOS CI breakage

[jserv]

Shall we support macOS/Intel?

Since Apple Silicon (Arm64) has become the predominant hardware platform for macOS, we should focus our CI pipeline on macOS/Arm64 rather than maintaining x64 support.

[yaoyhu]

uname -m on my M1 Pro Mac returns arm64, not aarch64. This required adding arm64 specific entries to common.mk. Is this the intended configuration, or am I missing a step in the setup process?

diff --git a/mk/common.mk b/mk/common.mk
index 2c721a6..c03dd7e 100644
--- a/mk/common.mk
+++ b/mk/common.mk
@@ -10,6 +10,8 @@ ifeq ($(UNAME_M),x86_64)
     HOST_PLATFORM := x86
 else ifeq ($(UNAME_M),aarch64)
     HOST_PLATFORM := aarch64
+else ifeq ($(UNAME_M),arm64)
+    HOST_PLATFORM := arm64
 else
     $(error Unsupported platform.)
 endif

[jserv]

uname -m on my M1 Pro Mac returns arm64, not aarch64. This required adding arm64 specific entries to common.mk.

Review the master branch, which now includes commit b23860d that adds macOS build support.

[ChinYikMing]

macOS runners still suffering occasionally from fetching the latest release tag of prebuilt binaries.

This is the code snippet where the macOS runners fail:
https://github.com/sysprog21/rv32emu/blob/b8d6fa757fb65992af1e123835c0797dc9a446c8/mk/artifact.mk#L57C1-L63C10

I planned to create three tag name which are always point to the latest prebuilt release. So that, no need to fetch the latest release tag name.
For ELF binaries, call ELF-latest.
For Linux images, call Linux-image-latest.
For Sail model, call Sail-latest.
If has any other prefer name, please let me know.

Before uploading built artifact, delete the previous tag name from one of three of them by git push -d origin xxx (where xxx is one of three of the tag name). Then create the tag name and link to the release prebuilt binaries.

What do you guys think?

[ChinYikMing]

macOS runners still suffering occasionally from fetching the latest release tag of prebuilt binaries.

This is the code snippet where the macOS runners fail: https://github.com/sysprog21/rv32emu/blob/b8d6fa757fb65992af1e123835c0797dc9a446c8/mk/artifact.mk#L57C1-L63C10

I planned to create three tag name which are always point to the latest prebuilt release. So that, no need to fetch the latest release tag name. For ELF binaries, call ELF-latest. For Linux images, call Linux-image-latest. For Sail model, call Sail-latest. If has any other prefer name, please let me know.

Before uploading built artifact, delete the previous tag name from one of three of them by git push -d origin xxx (where xxx is one of three of the tag name). Then create the tag name and link to the release prebuilt binaries.

What do you guys think?

Specifically, the changes to the build-artifact CI would be as following:

diff --git a/.github/workflows/build-artifact.yml b/.github/workflows/build-artifact.yml
index c2d4b15..b0d027b 100644
--- a/.github/workflows/build-artifact.yml
+++ b/.github/workflows/build-artifact.yml
@@ -65,6 +65,7 @@ jobs:
         uses: actions/checkout@v4
         with:
           submodules: 'true'
+          token: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
       - name: Install dependencies
         run: |
           sudo apt-get update -q=2
@@ -86,11 +87,17 @@ jobs:
         env:
           GH_TOKEN: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
         run: |
-          RELEASE_TAG=$(date +'%Y.%m.%d'-Linux-Image)
+          RELEASE_REPO=https://github.com/sysprog21/rv32emu-prebuilt.git
+          RELEASE_TAG=Linux-image-latest
+          RELEASE_NOTE="Build at $(date +'%Y.%m.%d')"
           cd /tmp
+          git push --delete $RELEASE_REPO $RELEASE_TAG
+          # delay some time to let github API sync status
+          sleep 1
           gh release create $RELEASE_TAG \
             --repo sysprog21/rv32emu-prebuilt \
-            --title "$RELEASE_TAG""-nightly"
+            --title "$RELEASE_TAG""-nightly" \
+            --note "$RELEASE_NOTE"
           gh release upload $RELEASE_TAG \
             rv32emu-linux-image-prebuilt.tar.gz \
             sha1sum-linux-image \
@@ -105,6 +112,7 @@ jobs:
         uses: actions/checkout@v4
         with:
           submodules: 'true'
+          token: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
       - name: Install dependencies
         run: |
           sudo apt-get update -q=2
@@ -127,11 +135,17 @@ jobs:
         env:
           GH_TOKEN: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
         run: |
-          RELEASE_TAG=$(date +'%Y.%m.%d'-ELF)
+          RELEASE_REPO=https://github.com/sysprog21/rv32emu-prebuilt.git
+          RELEASE_TAG=ELF-latest
+          RELEASE_NOTE="Build at $(date +'%Y.%m.%d')"
           cd /tmp
+          git push --delete $RELEASE_REPO $RELEASE_TAG
+          # delay some time to let github API sync status
+          sleep 1
           gh release create $RELEASE_TAG \
             --repo sysprog21/rv32emu-prebuilt \
-            --title "$RELEASE_TAG""-nightly"
+            --title "$RELEASE_TAG""-nightly" \
+            --note "$RELEASE_NOTE"
           gh release upload $RELEASE_TAG \
             rv32emu-prebuilt.tar.gz \
             sha1sum-linux-x86-softfp \

The build date is stored in the release note.

The Sail model is not the case since we built it ourselves and then uploaded it to rv32emu-prebuilt.

[ChinYikMing]

macOS runners still suffering occasionally from fetching the latest release tag of prebuilt binaries.
This is the code snippet where the macOS runners fail: https://github.com/sysprog21/rv32emu/blob/b8d6fa757fb65992af1e123835c0797dc9a446c8/mk/artifact.mk#L57C1-L63C10
I planned to create three tag name which are always point to the latest prebuilt release. So that, no need to fetch the latest release tag name. For ELF binaries, call ELF-latest. For Linux images, call Linux-image-latest. For Sail model, call Sail-latest. If has any other prefer name, please let me know.
Before uploading built artifact, delete the previous tag name from one of three of them by git push -d origin xxx (where xxx is one of three of the tag name). Then create the tag name and link to the release prebuilt binaries.
What do you guys think?

Specifically, the changes to the build-artifact CI would be as following:

diff --git a/.github/workflows/build-artifact.yml b/.github/workflows/build-artifact.yml
index c2d4b15..b0d027b 100644
--- a/.github/workflows/build-artifact.yml
+++ b/.github/workflows/build-artifact.yml
@@ -65,6 +65,7 @@ jobs:
uses: actions/checkout@v4
with:
submodules: 'true'

•      token: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
   - name: Install dependencies
     run: |
       sudo apt-get update -q=2
  

@@ -86,11 +87,17 @@ jobs:
env:
GH_TOKEN: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
run: |

•      RELEASE_TAG=$(date +'%Y.%m.%d'-Linux-Image)
  

•      RELEASE_REPO=https://github.com/sysprog21/rv32emu-prebuilt.git
  

•      RELEASE_TAG=Linux-image-latest
  

•      RELEASE_NOTE="Build at $(date +'%Y.%m.%d')"
       cd /tmp
  

•      git push --delete $RELEASE_REPO $RELEASE_TAG
  

•      # delay some time to let github API sync status
  

•      sleep 1
       gh release create $RELEASE_TAG \
         --repo sysprog21/rv32emu-prebuilt \
  

•        --title "$RELEASE_TAG""-nightly"
  

•        --title "$RELEASE_TAG""-nightly" \
  

•        --note "$RELEASE_NOTE"
       gh release upload $RELEASE_TAG \
         rv32emu-linux-image-prebuilt.tar.gz \
         sha1sum-linux-image \
  

@@ -105,6 +112,7 @@ jobs:
uses: actions/checkout@v4
with:
submodules: 'true'

•      token: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
   - name: Install dependencies
     run: |
       sudo apt-get update -q=2
  

@@ -127,11 +135,17 @@ jobs:
env:
GH_TOKEN: ${{ secrets.RV32EMU_PREBUILT_TOKEN }}
run: |

•      RELEASE_TAG=$(date +'%Y.%m.%d'-ELF)
  

•      RELEASE_REPO=https://github.com/sysprog21/rv32emu-prebuilt.git
  

•      RELEASE_TAG=ELF-latest
  

•      RELEASE_NOTE="Build at $(date +'%Y.%m.%d')"
       cd /tmp
  

•      git push --delete $RELEASE_REPO $RELEASE_TAG
  

•      # delay some time to let github API sync status
  

•      sleep 1
       gh release create $RELEASE_TAG \
         --repo sysprog21/rv32emu-prebuilt \
  

•        --title "$RELEASE_TAG""-nightly"
  

•        --title "$RELEASE_TAG""-nightly" \
  

•        --note "$RELEASE_NOTE"
       gh release upload $RELEASE_TAG \
         rv32emu-prebuilt.tar.gz \
         sha1sum-linux-x86-softfp \
  

The build date is stored in the release note.

The Sail model is not the case since we built it ourselves and then uploaded it to rv32emu-prebuilt.

We can skip this manner by the favor of ad16cd8.