Preliminary support for Ubuntu 24 and Ubuntu 24 ARM (UBC-Thunderbots#3316)

* Update to python 3.10.0 and bazel 5.4.0

* No need to import `typing` for collections as of python 3.9

* Update black formatter to 24.4.2

* Run fix_formatting.sh

* Remove PyOpenGL-accelerate dependency

* Install latest pip in setup_software.sh

* Install latest pip in setup_software.sh

* Testing

* Testing

* Remove testing

* Bump pyqtgraph to 0.13.3

* Update setup_software.sh

* Try fixing precommit

* Try fixing precommit

* Try fixing precommit

* Replace black/autoflake with ruff

* Remove black binary

* Remove git attribute

* Remove unused check_formatting_ci.sh

* Run fix_formatting.sh

* Add simulated/field test fixtures to conftest.py

* Address UBC-Thunderbots#3251

* Python code cleanup and fix Thunderscope

* Enable pydocstyle rules in ruff

* Add compile pip requirements to pre-commit

* Formatting and fix pre-commit

* Fix pre-commit

* Fix pre-commit

* Fix pre-commit and fsm_diagram_generator.py

* Update docstrings

* Formatting

* Fix type errors

* Nuke jetson_nano/display :(

* Switch to pyqtdarktheme

* Bump pytqtgraph to 0.13.7

* If build fails, download autoref from Google Drive mirror

* Fix setup_software.sh

* If build fails, download autoref from releases in github fork

* Remove Qt deps from ER Force Simulator and improve code quality

* Remove Qt from bazel and setup_software.sh

* Misc code quality changes

* Rename formatting_scripts to scripts

* Fix docstring

* FIx docstring

* [pre-commit.ci lite] apply automatic fixes

* Add ASCII diagram to robotmesh.cpp

* Fix pyqtdarktheme dep missing

* Misc changes

* Nits

* Use python dataclasses, remove unused stuff, nits

* [pre-commit.ci lite] apply automatic fixes

* Use dataclass

* Update setup_software for Ubuntu 24 ARM

Start migrating to python3.12

* fixup installation issues

* Working on pybind11 compilation

* downgrade pybind11 slightly

* thunderscope is running, but slowly + autoref problems

* play around with newer protobuf version

* Fix python protobuf slowness

* update ssl protos

* resolve some build issues

* Seems to compile correctly on x86 Ubuntu 2020

* fix a whole bunch of issues and leave a bunch more

* Wip add support for new Tigers Autoref

* Remove auto_continue from old gamecontroller proto message

* Wip update autoref wrapper with new autoref

* [pre-commit.ci lite] apply automatic fixes

* Fixup last little niggly issues with game controller and autoref migration

* fixing up formatting

* migrate to gcc 10 and upgrade boost

* improve jetson toolchain

* work on nano build

* fix jetson nano

* fix ensurepip for ubuntu 24

* update linux gcc extlibs paths

* not yet compiling, adding arm64 support

* update cross compile toolchain names

* wip

* wip

* [pre-commit.ci lite] apply automatic fixes

* update setup software for pyqt6

* install nvm

* works on William (not William's) computer

* update g3log and some docs

* update tbots_protobuf_test

* [pre-commit.ci lite] apply automatic fixes

* update github 24 arm machine

* hopefully fix platform io in CI

* wip toolchain resolution issue

* fixup github builds mayhaps

* cleanup PR and fixup tests

* address PR comments

* cleanup documentation

* [pre-commit.ci lite] apply automatic fixes

* fixup clang format for ARM builds

---------

Co-authored-by: williamckha <[email protected]>
Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com>
Co-authored-by: arun <[email protected]>
Co-authored-by: wmostrenko <[email protected]>

Author: 5 people

.github/workflows/main.yml

@@ -11,9 +11,13 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  ubuntu22-tests:
-    name: Ubuntu 22.04 Build
-    runs-on: ubuntu-22.04
+  multiplatform-build:
+    strategy:
+      matrix:
+        platform: [ ubuntu-22.04, ubuntu-24.04 ]
+
+    name: Ubuntu Alternate Builds
+    runs-on: ${{ matrix.platform }}
     steps:
       # checks-out the repository under $GITHUB_WORKSPACE
       - uses: actions/checkout@v4
@@ -25,22 +29,24 @@ jobs:
       - name: Software Build Test
         run: |
           cd src
-          bazel build --show_timestamps       \
-            -- //... -//software:unix_full_system \
-            -//software/simulated_tests/...  \
-            -//software/ai/hl/...            \
-            -//software/field_tests/...      \
-            -//software/embedded/...      \
+          bazel build --show_timestamps --copt=-O3  \
+            -- //... -//software:unix_full_system   \
+            -//software/simulated_tests/...         \
+            -//software/ai/hl/...                   \
+            -//software/field_tests/...             \
+            -//software/embedded/...                \
+            -//cc_toolchain/...
 
       - name: Jetson Nano Build Test
         run: |
           cd src
-          bazel build --cpu=jetson_nano //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO
+          bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO --platforms=//cc_toolchain:robot
 
       - name: Raspberry Pi Build Test
         run: |
           cd src
-          bazel build --cpu=jetson_nano //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI
+          bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI --platforms=//cc_toolchain:robot
+
 
   software-tests:
     name: Software Tests
@@ -61,7 +67,8 @@ jobs:
             -//software/simulated_tests/...  \
             -//software/ai/hl/...            \
             -//software/field_tests/...      \
-            -//software/ai/navigator/...
+            -//software/ai/navigator/...     \
+            -//cc_toolchain/...
 
   robot-tests:
     name: Robot Software Tests
@@ -90,12 +97,14 @@ jobs:
       - name: Jetson Nano Build
         run: |
          cd src
-         bazel build --cpu=jetson_nano //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO
+         bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO --platforms=//cc_toolchain:robot
+
 
       - name: Raspberry Pi Build
         run: |
          cd src
-         bazel build --cpu=jetson_nano //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI
+         bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI --platforms=//cc_toolchain:robot
+
 
   simulated-gameplay-tests:
     name: Simulated Gameplay Tests
@@ -112,9 +121,9 @@ jobs:
         run: |
           cd src
           bazel test --copt=-O3 --flaky_test_attempts=3 --show_timestamps       \
-            //software:unix_full_system           \
-            //software/simulated_tests/...   \
-            //software/ai/hl/... \
+            //software:unix_full_system       \
+            //software/simulated_tests/...    \
+            //software/ai/hl/...              \
             //software/ai/navigator/...
 
       - name: Upload simulated test proto logs

docs/getting-started-wsl.md

@@ -39,7 +39,7 @@ If you are not using Windows 11 and would prefer not to upgrade, you can follow
 3. Now, let's install Ubuntu.
     - Download the WSL2 kernel from [here](https://docs.microsoft.com/en-us/windows/wsl/wsl2-kernel).
     - Open a PowerShell window and run command `wsl --set-default-version 2` to use WSL2 by default.
-    - Install Ubuntu 20.04 LTS or Ubuntu 22.04 LTS from the Microsoft Store.
+    - Install Ubuntu 20.04 LTS, Ubuntu 22.04 LTS or Ubuntu 24.04 LTS from the Microsoft Store.
     - Open the Ubuntu app in the Start menu. It will open a command prompt and ask you to create a new UNIX username and password for your WSL2 Ubuntu installation. 
 
 ### X Server Setup

docs/getting-started.md

@@ -66,9 +66,15 @@ These instructions assume you have a basic understanding of Linux and the comman
 
 ### Operating systems
 
-We currently only support Linux, specifically Ubuntu 20.04 LTS and Ubuntu 22.04 LTS. You are welcome to use a different version or distribution of Linux, but may need to make some tweaks in order for things to work.
+We currently only support Linux, specifically Ubuntu.
 
-You can use Ubuntu 20.04 LTS and Ubuntu 22.04 LTS inside Windows through Windows Subsystem for Linux, by following [this guide](./getting-started-wsl.md). **Running and developing Thunderbots on Windows is experimental and not officially supported.**
+If you have a X86_64 machine, we support Ubuntu 20.04 LTS, Ubuntu 22.04 LTS and Ubuntu 24.04 LTS.
+
+If you have a ARM64 (also known as AARCH64) machine, we support Ubuntu 24.04 LTS.
+
+You are welcome to use a different version or distribution of Linux, but may need to make some tweaks in order for things to work.
+
+You can use Ubuntu 20.04 LTS, Ubuntu 22.04 LTS or Ubuntu 24.04 LTS inside Windows through Windows Subsystem for Linux, by following [this guide](./getting-started-wsl.md). **Running and developing Thunderbots on Windows is experimental and not officially supported.**
 
 ### Getting the Code
 
@@ -344,15 +350,15 @@ Tracy also samples call stacks. If the profiled binary is run with root permissi
 
 ## Building for the robot
 
-To build for the robot computer, build the target with the `--cpu=jetson_nano` flag and the toolchain will automatically build using the ARM toolchain. For example, `bazel build --cpu=jetson_nano //software/geom/...`.
+To build for the robot computer, build the target with the `--platforms=//cc_toolchain:robot` flag and the toolchain will automatically build using the ARM toolchain. For example, `bazel build --platforms=//cc_toolchain:robot //software/geom/...`.
 
 ## Deploying Robot Software to the robot
 
 We use Ansible to automatically update software running on the robot. [More info here.](useful-robot-commands.md#flashing-the-robots-compute-module) 
 
 To update binaries on a working robot, you can run:
 
-`bazel run //software/embedded/ansible:run_ansible --cpu=jetson_nano --//software/embedded:host_platform=<platform> -- --playbook deploy_robot_software.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+`bazel run //software/embedded/ansible:run_ansible --platforms=//cc_toolchain:robot --//software/embedded:host_platform=<platform> -- --playbook deploy_robot_software.yml --hosts <robot_ip> --ssh_pass <robot_password>`
 
 Where `<platform>` is the robot platform you are deploying to (`PI` or `NANO`), and `<robot_ip>` is the IP address of the robot you are deploying to. The `robot_password` is the password used to login to the `robot` user on the robot.
 

docs/robot-software-architecture.md

@@ -20,7 +20,10 @@
 
 For a more detailed look at how Ansible works, [see the RFC](https://docs.google.com/document/d/1hN3Us2Vjr8z6ihqUVp_3L7rrjKc-EZ-l2hZJc31gNOc/edit)
 
-Example command: `bazel run //software/embedded/ansible:run_ansible --cpu=jetson_nano --//software/embedded:host_platform=<platform> -- --playbook deploy_robot_software.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+Example command: `bazel run //software/embedded/ansible:run_ansible --platforms=//cc_toolchain:robot --//software/embedded:host_platform=<platform> -- --playbook deploy_robot_software.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+* <platform>: `PI` or `NANO` depending on the computer on the robot
+* <robot_ip>: IP address of the robot
+* <robot_password>: Password of the robot
 
 More commands available [here](useful-robot-commands.md#off-robot-commands)
 

docs/useful-robot-commands.md

@@ -86,8 +86,8 @@ The IP address of the robots on the tbots network is `192.168.0.20<robot_id>` so
 
 Individual miscellaneous tasks (ex reboot, shutdown, rtt test) can be run through the `misc.yml` playbook by specifying the corresponding tag.
 
-To view a list of supported arguments, run  
-`bazel run //software/embedded/ansible:run_ansible --cpu=jetson_nano -- -h` 
+To view a list of supported arguments, run:
+`bazel run //software/embedded/ansible:run_ansible --platforms=//cc_toolchain:robot -h` 
 
 If desired, the `-ho`, `--hosts` argument can be replaced with `-p`, `--port`, defining a port to listen to for Announcements from hosts.
 
@@ -101,7 +101,7 @@ This will stop the current Systemd services, replace and restart them. Binaries
 
 <b>This will trigger motor calibration meaning the wheels may spin. Please elevate the robot so the wheels are not touching the ground for proper calibration.</b>
 
-`bazel run //software/embedded/ansible:run_ansible --cpu=jetson_nano --//software/embedded:host_platform=<platform> -- --playbook deploy_robot_software.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+`bazel run //software/embedded/ansible:run_ansible --platforms=//cc_toolchain:robot --//software/embedded:host_platform=<platform> -- --playbook deploy_robot_software.yml --hosts <robot_ip> --ssh_pass <robot_password>`
 * \<platform\> is the host platform on the robot (either `PI` or `NANO`)
 * <robot_ip> is the IP address of the robot
 * <robot_password> is the password of the `robot` user account
@@ -119,11 +119,11 @@ Example: Flashing robots 1, 4, and 7 that have a Raspberry Pi
 
 ## Flashing the powerboard
 
-This will flash powerloop, the current firmware in `software/power/`, onto the power board. It will prompt the user into setting the powerboard into bootloader mode by holding the boot button (left if looking from the back of the robot) and pressing the reset button (right if looking from the back of the robot), then releasing the reset button first, then the boot button. Once the board is flashed, pressing the reset button after to use the new firmware.  
+This will flash powerloop, the current firmware in `software/power/`, onto the power board. It will prompt the user into setting the powerboard into bootloader mode by holding the boot button (left if looking from the back of the robot) and pressing the reset button (right if looking from the back of the robot), then releasing the reset button first, then the boot button. Once the board is flashed, pressing the reset button after to use the new firmware.
 
 Looking from the back of the robot the reset and boot buttons are on right side of the battery holder on the lowest board with the reset being on the left and the boot on the right. <b>Warning it may kick/chip when pressed.</b>
 
-`bazel run //software/embedded/ansible:run_ansible --cpu=jetson_nano -- --playbook deploy_powerboard.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+`bazel run //software/embedded/ansible:run_ansible --platforms=//cc_toolchain:robot -- --playbook deploy_powerboard.yml --hosts <robot_ip> --ssh_pass <robot_password>`
 
 ## Setting up the embedded host
 
@@ -133,11 +133,11 @@ This section refers to setting up the computer on the robot for the first time.
 
 ### Jetson Nano
 
-`bazel run //software/embedded/ansible:run_ansible --cpu=jetson_nano --//software/embedded:host_platform=NANO -- --playbook setup_nano.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+`bazel run //software/embedded/ansible:run_ansible --platforms=//cc_toolchain:robot --//software/embedded:host_platform=NANO -- --playbook setup_nano.yml --hosts <robot_ip> --ssh_pass <robot_password>`
 
 ### Raspberry Pi
 
-`bazel run //software/embedded/ansible:run_ansible --cpu=jetson_nano --//software/embedded:host_platform=PI -- --playbook setup_raspberry_pi.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+`bazel run //software/embedded/ansible:run_ansible --platforms=//cc_toolchain:robot --//software/embedded:host_platform=PI -- --playbook setup_raspberry_pi.yml --hosts <robot_ip> --ssh_pass <robot_password>`
 
 ## Robot Diagnostics
 
@@ -164,7 +164,7 @@ Runs the robot auto test fixture on a robot through Ansible, which tests the mot
 
 From Software/src:
 
-`bazel run //software/embedded/ansible:run_ansible --//software/embedded:host_platform=<platform> --cpu=jetson_nano -- --playbook robot_auto_test_playbook.yml --hosts <robot_ip> --ssh_pass <robot_password>`
+`bazel run //software/embedded/ansible:run_ansible --//software/embedded:host_platform=<platform> --platforms=//cc_toolchain:robot -- --playbook robot_auto_test_playbook.yml --hosts <robot_ip> --ssh_pass <robot_password>`
 * replace the \<platform\> with the target platform for the robot (either `PI` or `NANO`)
 * replace the \<robot_ip\> with the actual ip address of the jetson nano for the ssh connection.
 * replace the <robot_password> with the actual password for the jetson nano for the ssh connection.