Push docker image to DockerHub on merge into master

The rv32emu image build process will take around 6 minutes on GitHub 
CI. 

The base image build process will take around 20 and 10 minutes when 
building a native image on the native arch for gcc and sail, 
respectively, if internet connection is fast.

The multi-arch base image build will take a long time (4 hours on a 
8-core machine with fast internet) as we will use QEMU emulation for 
cross compilation. At the time of writing, we can only run multi-arch 
build successfully on x86 machines.

The base image will need to be built using the script provided in the 
docker directory, manually. The rv32emu image will be built 
automatically on PR and merge into the master branch.

Author: henrybear327

.github/workflows/docker.yml

@@ -0,0 +1,46 @@
+name: Publish Docker image to Docker Hub
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+    branches: [ master ]
+
+# https://docs.docker.com/build/ci/github-actions/multi-platform/
+jobs:
+  docker:
+    name: build-and-publish
+    runs-on: ubuntu-22.04
+    steps:
+      
+      - name: Check out the repo
+        uses: actions/checkout@v4
+      - name: Test changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v40
+        with:
+          files: |
+              src/**
+              build/**
+              mk/**
+              tests/**
+              tools/**
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+        if: ${{ steps.changed-files.outputs.any_changed == 'true' }}
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+        if: ${{ steps.changed-files.outputs.any_changed == 'true' }}
+      - name: Login to Docker Hub
+        uses: docker/login-action@v3
+        if: ${{ steps.changed-files.outputs.any_changed == 'true' && github.event_name == 'push'}}
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }}
+      - name: Build and push
+        uses: docker/build-push-action@v5
+        if: ${{ steps.changed-files.outputs.any_changed == 'true' }}
+        with:
+          context: .
+          platforms: linux/amd64,linux/arm64/v8
+          push: ${{ github.event_name == 'push' }}
+          tags: sysprog21/rv32emu:latest

Dockerfile

@@ -1,55 +1,29 @@
-FROM ubuntu:22.04
-LABEL maintainer="[email protected]"
+FROM sysprog21/rv32emu-gcc as base_gcc
+FROM sysprog21/rv32emu-sail as base_sail
 
-# Install packages required for the emulator to compile and execute correctly
+FROM ubuntu:22.04 as final
+
+# Install extra packages for the emulator to compile and execute with full capabilities correctly
 RUN apt-get update && \
     DEBIAN_FRONTEND=noninteractive apt-get install -y \
-    libsdl2-dev libsdl2-mixer-dev python3-pip git 
+    libsdl2-dev libsdl2-mixer-dev python3-pip git && \
+    rm -rf /var/lib/apt/lists/*
 
 RUN python3 -m pip install git+https://github.com/riscv/riscof 
 
 # set up the timezone
 ENV TZ=Asia/Taipei
 RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
 
-# when using apt install gcc-riscv64-unknown-elf, this will cause "unsupported ISA subset 'z'" during compilation
-# thus, we are building from scratch, following the version here -> https://github.com/sysprog21/rv32emu/blob/master/.ci/riscv-toolchain-install.sh
-# for x86, we can optimize this part and take the nightly build directly, but not for aarch64 
-ENV RISCV=/opt/riscv
-ENV PATH=$RISCV/bin:$PATH
-WORKDIR $RISCV
-RUN apt install -y autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev
-RUN git clone --recursive https://github.com/riscv/riscv-gnu-toolchain
-RUN cd riscv-gnu-toolchain && \
-    git checkout tags/2023.10.06 && \
-    ./configure --prefix=/opt/riscv --with-arch=rv32gc --with-abi=ilp32d && \
-    make -j$(nproc) && \
-    make clean
-
-# the default reference emulator is x86-based
-# we need to build it ourselves if we are using it on aarch64
-# https://riscof.readthedocs.io/en/stable/installation.html#install-plugin-models
-# the above commands are modified to match the current build flow as indicated in the Github CI -> https://github.com/riscv/sail-riscv/blob/master/.github/workflows/compile.yml
-WORKDIR /home/root/
-RUN apt install -y opam zlib1g-dev pkg-config libgmp-dev z3 device-tree-compiler
-RUN opam init --disable-sandboxing -y
-RUN opam install -y sail
-RUN git clone https://github.com/riscv/sail-riscv.git
-# based on this commit https://github.com/sysprog21/rv32emu/commit/01b00b6f175f57ef39ffd1f4fa6a611891e36df3#diff-3b436c5e32c40ecca4095bdacc1fb69c0759096f86e029238ce34bbe73c6e68f
-# we infer that the sail-riscv binary was taken from commit 9547a30bf84572c458476591b569a95f5232c1c7
-RUN cd sail-riscv && \
-    git checkout 9547a30bf84572c458476591b569a95f5232c1c7 && \
-    eval $(opam env) && \
-    make && \
-    ARCH=RV32 make 
-
 # copy in the source code
 WORKDIR /home/root/rv32emu
 COPY . .
 
+# Copy the GNU Toolchain files
+ENV RISCV=/opt/riscv
+ENV PATH=$RISCV/bin:$PATH
+COPY --from=base_gcc /opt/riscv/ /opt/riscv/
+
 # replace the emulator (riscv_sim_RV32) with the arch that the container can execute 
 RUN rm /home/root/rv32emu/tests/arch-test-target/sail_cSim/riscv_sim_RV32
-RUN cp /home/root/sail-riscv/c_emulator/riscv_sim_RV32 /home/root/rv32emu/tests/arch-test-target/sail_cSim/riscv_sim_RV32
-
-# clean up apt cache
-RUN rm -rf /var/lib/apt/lists/*
+COPY --from=base_sail /home/root/riscv_sim_RV32 /home/root/rv32emu/tests/arch-test-target/sail_cSim/riscv_sim_RV32

README.md

@@ -67,11 +67,7 @@ The usage and limitations of Doom and Quake demo are listed in [docs/demo.md](do
 
 ### Docker image
 
-The image containing all the necessary tools for development and testing can be built by executing `docker build -t rv32emu .`. 
-
-This image works for both x86 and aarch64 (Apple's M1 chip) machines. Note that the image building time will be long since it will perform builds of the toolchain and the reference emulator from source. Also, it will take up about 16GB of disk space.
-
-After the image is built, you can create a docker container by executing `docker run -it rv32emu` at the project root. This will give you can interactive bash shell where you can experiment with the current codebase.
+The image containing all the necessary tools for development and testing can be executed by `docker run -it sysprog21/rv32emu:latest`. It works for both x86-64 and aarch64 (Apple's M1 chip) machines.
 
 ### Customization
 

docker/.gitignore

@@ -0,0 +1 @@
+*.log

docker/Dockerfile-gcc

@@ -0,0 +1,20 @@
+FROM ubuntu:22.04 as base
+
+# when using apt install gcc-riscv64-unknown-elf, this will cause "unsupported ISA subset 'z'" during compilation
+# thus, we are building from scratch, following the version here -> https://github.com/sysprog21/rv32emu/blob/master/.ci/riscv-toolchain-install.sh
+# for x86-64, we can optimize this part and take the nightly build directly, but not for aarch64 
+RUN apt-get update && \
+    apt install -y \
+    git autoconf automake autotools-dev curl libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev && \
+    rm -rf /var/lib/apt/lists/*
+RUN git clone --recursive https://github.com/riscv/riscv-gnu-toolchain
+RUN cd riscv-gnu-toolchain && \
+    git checkout tags/2023.10.06 && \
+    ./configure --prefix=/opt/riscv --with-arch=rv32gc --with-abi=ilp32d && \
+    make -j$(nproc) && \
+    make clean
+
+FROM ubuntu:22.04 as final
+
+# Keep the GNU Toolchain files only
+COPY --from=base /opt/riscv/ /opt/riscv/

docker/Dockerfile-sail

@@ -0,0 +1,29 @@
+# builds the base image of rv32emu (the toolchains)
+# docker build --progress=plain -t sysprog21/rv32emu-base -f Dockerfile-base . 2>&1 | tee build.log
+FROM ubuntu:22.04 as base
+
+# the default reference emulator is x86-64-based
+# we need to build it ourselves if we are using it on aarch64
+# https://riscof.readthedocs.io/en/stable/installation.html#install-plugin-models
+# the above commands are modified to match the current build flow as indicated in the Github CI -> https://github.com/riscv/sail-riscv/blob/master/.github/workflows/compile.yml
+RUN apt-get update && \
+    apt install -y opam zlib1g-dev pkg-config libgmp-dev z3 device-tree-compiler && \
+    rm -rf /var/lib/apt/lists/*
+RUN opam init --disable-sandboxing -y
+RUN opam switch create ocaml-base-compiler.4.13.1 # opam switch list-available
+RUN opam search sail
+# https://opam.ocaml.org/packages/sail/sail.0.16/
+RUN opam install -y sail.0.16 # latest version 0.17.x will cause complication issues for sail-riscv on commit 9547a3
+RUN git clone https://github.com/riscv/sail-riscv.git
+# based on this commit https://github.com/sysprog21/rv32emu/commit/01b00b6f175f57ef39ffd1f4fa6a611891e36df3#diff-3b436c5e32c40ecca4095bdacc1fb69c0759096f86e029238ce34bbe73c6e68f
+# we infer that the sail-riscv binary was taken from commit 9547a30bf84572c458476591b569a95f5232c1c7
+RUN cd sail-riscv && \
+    git checkout 9547a30bf84572c458476591b569a95f5232c1c7 && \
+    eval $(opam env) && \
+    make && \
+    ARCH=RV32 make 
+
+FROM ubuntu:22.04 as final
+
+# keep the emulator only
+COPY --from=base /sail-riscv/c_emulator/riscv_sim_RV32 /home/root/riscv_sim_RV32

docker/build.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash
+
+# To enable multi-arch image build
+# https://www.docker.com/blog/how-to-rapidly-build-multi-architecture-images-with-buildx/
+docker buildx create --name cross-platform-builder --use --bootstrap
+
+docker buildx build --progress=plain --push --platform linux/amd64,linux/arm64/v8 --tag sysprog21/rv32emu-gcc -f Dockerfile-gcc . 2>&1 | tee build-gcc.log
+rm build-gcc.log
+
+docker buildx build --progress=plain --push --platform linux/amd64,linux/arm64/v8 --tag sysprog21/rv32emu-sail -f Dockerfile-sail . 2>&1 | tee build-sail.log
+rm build-sail.log

docs/base-image.md

@@ -0,0 +1,15 @@
+# Base images
+
+The goal is to support both `x86-64` and `aarch64` docker images for development and testing. 
+
+The base images contain `gcc` and `sail` images. We compile from the toolchain from source and use a specific version of the toolchains.
+
+The build process should be used when an update on the toolchain is required. When a new build is made by running `build.sh`, build and push to Docker Hub are automated. 
+
+## Details
+
+For `gcc`, we are using the compiler version `tags/2023.10.06`, as using the binaries from `apt install gcc-riscv64-unknown-elf` will cause `"unsupported ISA subset 'z'"` error during the compilation of the emulator on M1.
+
+For `sail`, we set up the toolchain to use `sail-0.16`, as the latest version `0.17.x` series will cause compilation errors.
+
+We are using the commit `9547a30bf84572c458476591b569a95f5232c1c7` from `sail-riscv` for the reference simulator, as this is the commit cloest to the time when the [x86-64 reference sail emulator](https://github.com/sysprog21/rv32emu/commit/01b00b6f175f57ef39ffd1f4fa6a611891e36df3#diff-3b436c5e32c40ecca4095bdacc1fb69c0759096f86e029238ce34bbe73c6e68f) was added to the `rv32emu` Github repo.