Skip to content

Commit ae1fa33

Browse files
Paciente8159Paciente8159
authored
Grbl compatibility Unity Tests (#988)
* unit tests virtual emulator API * new unity tests implementation via serial - new unity tests implementation supported mostly on serial stream * unit tests and parser fixes - unit tests tuning - some parser fixes * tests resolution and parser fixes * fixed M2 parser reset * unity tests optimizations and fixes * unit tests and fixes - fixed $20 and $22 dependent behavior - fixed starting blocks set command response * unit tests fixes * $Nx setting executes the command like Grbl - $Nx setting executes the command like Grbl - this will still erase the previous command if error (not Grbl compliant) * shrink AVR compilation * Final tests and fixes * linux backend update * Update pio-build.yaml * Update pio-test.yaml * updated github actions * fix PIO registry issues in CI testing * Update .gitignore --------- Co-authored-by: Paciente8159 <12116536+Paciente8159@users.noreply.github.com>
1 parent 9bd553f commit ae1fa33

51 files changed

Lines changed: 3958 additions & 178 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/pio-build.yaml‎

Lines changed: 44 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,45 @@ on:
99
- master
1010

1111
jobs:
12+
unit-tests:
13+
name: Linux Unit Tests
14+
runs-on: ubuntu-latest
15+
16+
permissions:
17+
contents: read
18+
19+
steps:
20+
- uses: actions/checkout@v7
21+
22+
- name: Cache pip
23+
uses: actions/cache@v5
24+
with:
25+
path: ~/.cache/pip
26+
key: pip-${{ runner.os }}-${{ hashFiles('**/requirements.txt') }}
27+
restore-keys: pip-${{ runner.os }}-
28+
29+
- name: Cache PlatformIO
30+
uses: actions/cache@v5
31+
with:
32+
path: ~/.platformio
33+
key: pio-v2-${{ runner.os }}-${{ hashFiles('platformio.ini', 'uCNC/src/hal/boards/**/*.ini') }}
34+
35+
- name: Set up Python
36+
uses: actions/setup-python@v7
37+
with:
38+
python-version: "3.12"
39+
40+
- name: Install PlatformIO
41+
run: |
42+
python -m pip install --upgrade pip
43+
pip install "platformio==6.1.18"
44+
45+
- name: Run Linux unit tests
46+
run: pio test -e EMULATOR_LINUX_TEST
47+
1248
build:
1349
name: Build Firmware
50+
needs: unit-tests
1451
runs-on: ubuntu-latest
1552

1653
permissions:
@@ -45,30 +82,30 @@ jobs:
4582
- ESP32-C3
4683

4784
steps:
48-
- uses: actions/checkout@v4
85+
- uses: actions/checkout@v7
4986

5087
- name: Cache pip
51-
uses: actions/cache@v4
88+
uses: actions/cache@v5
5289
with:
5390
path: ~/.cache/pip
5491
key: pip-${{ runner.os }}-${{ hashFiles('**/requirements.txt') }}
5592
restore-keys: pip-${{ runner.os }}-
5693

5794
- name: Cache PlatformIO
58-
uses: actions/cache@v4
95+
uses: actions/cache@v5
5996
with:
6097
path: ~/.platformio
61-
key: pio-${{ runner.os }}-${{ hashFiles('**/lockfiles') }}
98+
key: pio-v2-${{ runner.os }}-${{ hashFiles('platformio.ini', 'uCNC/src/hal/boards/**/*.ini') }}
6299

63100
- name: Set up Python
64-
uses: actions/setup-python@v5
101+
uses: actions/setup-python@v7
65102
with:
66103
python-version: "3.12"
67104

68105
- name: Install PlatformIO
69106
run: |
70107
python -m pip install --upgrade pip
71-
pip install --upgrade platformio
108+
pip install "platformio==6.1.18"
72109
73110
- name: Run PlatformIO
74111
run: pio run -e ${{ matrix.boards }}
@@ -77,7 +114,7 @@ jobs:
77114
run: zip ${{ matrix.boards }}.zip .pio/build/${{ matrix.boards }}/firmware.*
78115

79116
- name: Upload binaries
80-
uses: actions/upload-artifact@v4
117+
uses: actions/upload-artifact@v7
81118
with:
82119
name: ${{ matrix.boards }}
83120
path: ./${{ matrix.boards }}.zip

‎.github/workflows/pio-release.yaml‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
steps:
1919
- name: Create GitHub Release
2020
id: create_release
21-
uses: softprops/action-gh-release@v2
21+
uses: softprops/action-gh-release@v3
2222
with:
2323
tag_name: ${{ github.ref_name }}
2424
name: µCNC ${{ github.ref_name }}
@@ -71,30 +71,30 @@ jobs:
7171
- ESP32-C3
7272

7373
steps:
74-
- uses: actions/checkout@v4
74+
- uses: actions/checkout@v7
7575

7676
- name: Cache pip
77-
uses: actions/cache@v4
77+
uses: actions/cache@v5
7878
with:
7979
path: ~/.cache/pip
8080
key: pip-${{ runner.os }}-${{ hashFiles('**/requirements.txt') }}
8181
restore-keys: pip-${{ runner.os }}-
8282

8383
- name: Cache PlatformIO
84-
uses: actions/cache@v4
84+
uses: actions/cache@v5
8585
with:
8686
path: ~/.platformio
87-
key: pio-${{ runner.os }}-${{ hashFiles('**/lockfiles') }}
87+
key: pio-v2-${{ runner.os }}-${{ hashFiles('platformio.ini', 'uCNC/src/hal/boards/**/*.ini') }}
8888

8989
- name: Set up Python
90-
uses: actions/setup-python@v5
90+
uses: actions/setup-python@v7
9191
with:
9292
python-version: "3.12"
9393

9494
- name: Install PlatformIO
9595
run: |
9696
python -m pip install --upgrade pip
97-
pip install --upgrade platformio
97+
pip install "platformio==6.1.18"
9898
9999
- name: Build firmware
100100
run: pio run -e ${{ matrix.boards }}
@@ -103,13 +103,13 @@ jobs:
103103
run: zip ${{ matrix.boards }}.zip .pio/build/${{ matrix.boards }}/firmware.*
104104

105105
- name: Upload artifact
106-
uses: actions/upload-artifact@v4
106+
uses: actions/upload-artifact@v7
107107
with:
108108
name: ${{ matrix.boards }}
109109
path: ${{ matrix.boards }}.zip
110110

111111
- name: Upload Release Asset
112-
uses: softprops/action-gh-release@v2
112+
uses: softprops/action-gh-release@v3
113113
with:
114114
files: ${{ matrix.boards }}.zip
115115
tag_name: ${{ github.ref_name }}

‎.github/workflows/pio-test.yaml‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,45 @@ on:
44
push:
55

66
jobs:
7+
unit-tests:
8+
name: Linux Unit Tests
9+
runs-on: ubuntu-latest
10+
11+
permissions:
12+
contents: read
13+
14+
steps:
15+
- uses: actions/checkout@v7
16+
17+
- name: Cache pip
18+
uses: actions/cache@v5
19+
with:
20+
path: ~/.cache/pip
21+
key: pip-${{ runner.os }}-${{ hashFiles('**/requirements.txt') }}
22+
restore-keys: pip-${{ runner.os }}-
23+
24+
- name: Cache PlatformIO
25+
uses: actions/cache@v5
26+
with:
27+
path: ~/.platformio
28+
key: pio-v2-${{ runner.os }}-${{ hashFiles('platformio.ini', 'uCNC/src/hal/boards/**/*.ini') }}
29+
30+
- name: Set up Python
31+
uses: actions/setup-python@v7
32+
with:
33+
python-version: "3.12"
34+
35+
- name: Install PlatformIO
36+
run: |
37+
python -m pip install --upgrade pip
38+
pip install "platformio==6.1.18"
39+
40+
- name: Run Linux unit tests
41+
run: pio test -e EMULATOR_LINUX_TEST
42+
743
build:
844
name: Build & Test Firmware
45+
needs: unit-tests
946
runs-on: ubuntu-latest
1047

1148
strategy:
@@ -32,30 +69,30 @@ jobs:
3269
contents: read
3370

3471
steps:
35-
- uses: actions/checkout@v4
72+
- uses: actions/checkout@v7
3673

3774
- name: Cache pip
38-
uses: actions/cache@v4
75+
uses: actions/cache@v5
3976
with:
4077
path: ~/.cache/pip
4178
key: pip-${{ runner.os }}-${{ hashFiles('**/requirements.txt') }}
4279
restore-keys: pip-${{ runner.os }}-
4380

4481
- name: Cache PlatformIO
45-
uses: actions/cache@v4
82+
uses: actions/cache@v5
4683
with:
4784
path: ~/.platformio
48-
key: pio-${{ runner.os }}-${{ hashFiles('**/lockfiles') }}
85+
key: pio-v2-${{ runner.os }}-${{ hashFiles('platformio.ini', 'uCNC/src/hal/boards/**/*.ini') }}
4986

5087
- name: Set up Python
51-
uses: actions/setup-python@v5
88+
uses: actions/setup-python@v7
5289
with:
5390
python-version: "3.12"
5491

5592
- name: Install PlatformIO
5693
run: |
5794
python -m pip install --upgrade pip
58-
pip install --upgrade platformio
95+
pip install "platformio==6.1.18"
5996
6097
- name: Run PlatformIO Build
6198
run: pio run -e ${{ matrix.boards }}

‎test/README.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Grbl conformity tests (Windows emulator)
2+
3+
These Unity fixtures exercise the default µCNC three-axis Cartesian build as a
4+
Grbl 1.1 black box. Commands enter through the test stream and most assertions
5+
are made against the resulting Grbl serial transcript. Motion, spindle, probe,
6+
and limit fixtures additionally inspect the virtual MCU state where the serial
7+
protocol cannot prove a physical output transition.
8+
9+
Each `test_grbl_*` directory is compiled and launched by PlatformIO as a
10+
separate native Windows process. This isolates controller globals, planner
11+
state, timers, streams, and RAM-only settings between fixture groups.
12+
13+
Run the complete baseline suite with:
14+
15+
```powershell
16+
C:\Users\JCEM\.platformio\penv\Scripts\pio.exe test -e EMULATOR_WINDOWS_TEST
17+
```
18+
19+
Run one fixture with, for example:
20+
21+
```powershell
22+
C:\Users\JCEM\.platformio\penv\Scripts\pio.exe test `
23+
-e EMULATOR_WINDOWS_TEST `
24+
-f test_grbl_motion
25+
```
26+
27+
## Fixture coverage
28+
29+
- `test_domain1_a_lexical`: all 24 D1-A preprocessing and numeric lexical cases.
30+
- `test_domain1_b_supported`: all 53 D1-B supported-command cases, capability-gated where required.
31+
- `test_domain1_c_rejection`: all 26 D1-C unsupported-command and modal-conflict cases.
32+
- `test_domain1_d_parameters`: all 23 D1-D missing, invalid, and unused-parameter cases.
33+
- `test_domain1_e_modal`: all 16 D1-E modal-report cases.
34+
- `test_grbl_protocol`: startup, status, help, modal, coordinate, and settings reports.
35+
- `test_grbl_motion`: absolute, incremental, linear, and arc endpoint validation.
36+
- `test_grbl_spindle`: M3/M4/M5 modal behavior and virtual PWM/direction outputs.
37+
38+
## Domain 2 motion conformance
39+
40+
Domain 2 fixtures execute motion in a fresh default three-axis Cartesian emulator and validate
41+
reported `MPos` endpoints, sampled line/arc geometry, coordinate transforms, probing, stored
42+
positions, tool outputs, program flow, and feed constraints:
43+
44+
- `test_domain2_a_linear`: status reporting, G0/G1, absolute/incremental modes, units, and G80.
45+
- `test_domain2_b_arcs`: G2/G3 IJK and radius arcs in G17/G18/G19 plus helical motion.
46+
- `test_domain2_c_coordinates`: G54-G59, G10 L2/L20, G53, G92, and G92.1.
47+
- `test_domain2_d_stored_tool`: G28/G30 stored positions, intermediate legs, G43.1, and G49.
48+
- `test_domain2_e_probing`: G38.2-G38.5 contact/release and failure semantics.
49+
- `test_domain2_f_tools`: spindle PWM/direction, coolant capability behavior, and program end.
50+
- `test_domain2_g_program_feed`: dwell, pause/resume, status feed, and maximum-rate constraints.
51+
52+
Trajectory assertions use status samples rather than fixed completion delays. Receipt of `ok`
53+
only acknowledges that Grbl accepted a line; every motion assertion synchronizes on `<Idle>`.
54+
55+
## Domain 3 controller-state conformance
56+
57+
Domain 3 uses a persistent controller service loop within each fresh PlatformIO fixture and tests
58+
serial framing, status content, settings, Check/Sleep/startup modes, jogging, realtime state
59+
transitions, overrides, physical control inputs, limits, probe/homing alarms, recovery, and the
60+
state-dependent command acceptance matrix:
61+
62+
- `test_domain3_a_interface`: reset/query responses, status framing, partial lines, and realtime interleaving.
63+
- `test_domain3_b_settings`: standard setting read/write coverage, validation, and dependencies.
64+
- `test_domain3_c_modes`: Check, unlock, Alarm recovery, Sleep, and startup blocks.
65+
- `test_domain3_d_jog`: validation, locking, cancellation, queueing, and modal isolation.
66+
- `test_domain3_e_realtime`: hold/resume, reset, safety door, and physical hold/start inputs.
67+
- `test_domain3_f_overrides`: feed, rapid, spindle, and coolant realtime overrides.
68+
- `test_domain3_g_limits_alarms`: hard/soft limits and alarms 1-5 with command locking.
69+
- `test_domain3_h_homing`: disabled homing and homing alarms 6, 7, and 9.
70+
- `test_domain3_i_state_matrix`: Idle/Run/Hold/Jog/Alarm/Check command acceptance and pin reporting.
71+
- `test_grbl_inputs`: probing and hard-limit input behavior.
72+
- `test_grbl_jog_realtime`: jog state and realtime jog cancellation.
73+
- `test_grbl_feed_hold`: realtime feed hold and cycle-start/resume.
74+
75+
The five Domain 1 fixtures execute all 142 cases from D1-A through D1-E. Every
76+
case begins with a soft reset and a fresh Check-mode session. Responses use
77+
exact Grbl status codes, rejected commands are checked for modal-state
78+
preservation, and every mismatch is reported with its specification case ID.
79+
Table-driven mismatches are aggregated so one non-conforming command does not
80+
prevent the remaining cases in that section from executing.
81+
82+
## Current scope
83+
84+
This is the default-configuration baseline, not a claim of mathematical
85+
exhaustiveness. Configuration-dependent features such as homing (disabled by
86+
default), safety-door parking, mist coolant, extra axes, and alternative tools
87+
need separate build profiles before their expected behavior can be asserted.
88+
Physical timing/electrical characteristics also remain hardware-in-the-loop
89+
tests rather than emulator tests.

0 commit comments

Comments
 (0)