Merge pull request #267 from ianfixes/2021-01-07_beta

Beta test branch for next release

Author: ianfixes

.github/workflows/linux.yaml

@@ -33,6 +33,7 @@ jobs:
         run: |
           g++ -v
           cd SampleProjects/NetworkLib
-          sh ./scripts/install.sh
           bundle install
+          bundle exec ensure_arduino_installation.rb
+          sh ./scripts/install.sh
           bundle exec arduino_ci.rb

.github/workflows/macos.yaml

@@ -33,6 +33,7 @@ jobs:
         run: |
           g++ -v
           cd SampleProjects/NetworkLib
-          sh ./scripts/install.sh
           bundle install
+          bundle exec ensure_arduino_installation.rb
+          sh ./scripts/install.sh
           bundle exec arduino_ci.rb

.github/workflows/windows.yaml

@@ -33,6 +33,7 @@ jobs:
         run: |
           g++ -v
           cd SampleProjects/NetworkLib
-          bash -x ./scripts/install.sh
           bundle install
+          bundle exec ensure_arduino_installation.rb
+          bash -x ./scripts/install.sh
           bundle exec arduino_ci.rb

CHANGELOG.md

@@ -7,14 +7,22 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [Unreleased]
 ### Added
+- Better indications of the build phases in the test runner `arduino_ci.rb`
+- Better indications of which example sketch is being compiled as part of testing
 
 ### Changed
+- Topmost installtion instructions now suggest `gem install arduino_ci` instead of using a `Gemfile`.  Reasons for using a `Gemfile` are listed and discussed separately further down the README.
 
 ### Deprecated
 
 ### Removed
+- scanning of `library.properties`; this can and should now be performed by the standalone [`arduino-lint` tool](https://arduino.github.io/arduino-lint).
 
 ### Fixed
+- Example sketches with no configured platforms were printing the wrong configuration values to the debug message
+- Libraries directory was not being automatically created prior to attempting to change directory into it
+- A style error whose "fix" caused an _actual_ error.
+- Proper installation order for NetworkLib in CI workflow configuration
 
 ### Security
 

README.md

@@ -8,7 +8,7 @@
 Arduino CI was created to enable better collaboration among Arduino library maintainers and contributors, by enabling automated code checks to be performed as part of a pull request process.
 
 * enables running unit tests against the library **without hardware present**
-* provides a system of mocks that allow fine-grained control over the hardare inputs, including the system's clock
+* provides a system of mocks that allow fine-grained control over the hardware inputs, including the system's clock
 * verifies compilation of any example sketches included in the library
 * can test a wide range of arduino boards with different hardware options available
 * compares entries in `library.properties` to the contents of the library and reports mismatches
@@ -29,69 +29,91 @@ Windows  | [![Windows Build status](https://github.com/Arduino-CI/arduino_ci/wor
 
 ## Quick Start
 
-### You Need Your Arduino Library
+This project has the following dependencies:
 
-For a fairly minimal practical example of a unit-testable library repo that you can copy from, see [the `Arduino-CI/Blink` repository](https://github.com/Arduino-CI/Blink).
+* `ruby` 2.5 or higher
+* A compiler like `g++` (on OSX, `clang` works; on Cygwin, use the `mingw-gcc-c++` package)
+* `python` (if using a board architecutre that requires it, e.g. ESP32, ESP8266; see [this issue](https://github.com/Arduino-CI/arduino_ci/issues/235#issuecomment-739629243)). Consider `pyserial` as well.
+
+In that environment, you can install by running `gem install arduino_ci`.  To update to a latest version, use `gem update arduino_ci`.
+
+You can now test your library by simply running the command `arduino_ci.rb` from your library directory.  This will perform the following:
 
-> Note: The `SampleProjects` directory you see within _this_ repo contains tests for validing the `arduino_ci` framework itself, and due to that coupling will not be helpful to duplicate.  That said, the [SampleProjects/TestSomething](SampleProjects/TestSomething) project contains [test files](SampleProjects/TestSomething/test/) (each named after the type of feature being tested) that may be illustrative of testing strategy and capabilities _on an individual basis_.
+* validation of some fields in `library.properties`, if it exists
+* running unit tests from files found in `test/`, if they exist
+* testing compilation of example sketches found in `examples/`, if they exist
+
+### Assumptions About Your Repository
 
 Arduino expects all libraries to be in a specific `Arduino/libraries` directory on your system.  If your library is elsewhere, `arduino_ci` will _automatically_ create a symbolic link in the `libraries` directory that points to the directory of the project being tested.  This simplifieds working with project dependencies, but **it can have unintended consequences on Windows systems**.
 
 > If you use a Windows system **it is recommended that you only run `arduino_ci` from project directories that are already inside the `libraries` directory** because [in some cases deleting a folder that contains a symbolic link to another folder can cause the _entire linked folder_ to be removed instead of just the link itself](https://superuser.com/a/306618).
 
+### Changes to Your Repository
 
-### You Need Ruby and Bundler
-
-You'll need Ruby version 2.5 or higher, and to `gem install bundler` if it's not already there.
+Unit testing binaries created by `arduino_ci` should not be commited to the codebase.  To avoid that, add the following to your `.gitignore`:
 
+```ignore-list
+# arduino_ci unit test binaries and artifacts
+*.bin
+*.bin.dSYM
+```
 
-### You Need A Compiler (`g++`)
+### A Quick Example
 
-For unit testing, you will need a compiler; [g++](https://gcc.gnu.org/) is preferred.
+For a fairly minimal practical example of a unit-testable library repo that you can copy from, see [the `Arduino-CI/Blink` repository](https://github.com/Arduino-CI/Blink).
 
-* **Linux**: `gcc`/`g++` is likely pre-installed.
-* **OSX**: `g++` is an alias for `clang`, which is provided by Xcode and the developer tools.  You are free to `brew install gcc` as well; this is also tested and working.
-* **Windows**: you will need Cygwin, and the `mingw-gcc-g++` package.
 
+## Advanced Start
 
-### You _May_ Need `python`
+New features and bugfixes reach GitHub before they reach a released ruby gem.  Alternately, it may be that (for your own reasons) you do not wish to install `arduino_ci` globally on your system.  A few additional setup steps are required if you wish to do this.
 
-ESP32 and ESP8266 boards have [a dependency on `python` that they don't install themselves](https://github.com/Arduino-CI/arduino_ci/issues/235#issuecomment-739629243).  If you intend to test on these platforms (which are included in the default list of platforms to test against), you will need to make `python` (and possibly `pyserial`) available in the test environment.
+### You Need Ruby _and_ Bundler
 
-Alternately, you might configure `arduino_ci` to simply not test against these.  Consult the reference for those details.
+In addition to version 2.5 or higher, you'll also need to `gem install bundler` to a minimum of version 2.0 if it's not already there.  You may find it easiest to do this by using [`rbenv`](https://github.com/rbenv/rbenv).
 
+You will need to add a file called `Gemfile` (no extension) to your Arduino project.
 
-### Changes to Your Repo
+#### Non-root installation
 
-Add a file called `Gemfile` (no extension) to your Arduino project:
+If you are simply trying to avoid the need to install `arduino_ci` system-wide (which may require administrator permissions), your `Gemfile` would look like this:
 
 ```ruby
 source 'https://rubygems.org'
-gem 'arduino_ci' '~> 1.1'
+
+# Replace 1.2 with the desired version of arduino_ci.  See https://guides.rubygems.org/patterns/#pessimistic-version-constraint
+gem 'arduino_ci', '~> 1.2'
 ```
 
-At the time of this writing, `1.1` is the latest version available, and the `~>` syntax will allow your system to update it to the latest `1.x.x` version.  The list of all available versions can be found on [rubygems.org](https://rubygems.org/gems/arduino_ci) if you prefer to explicitly pin a higher version.
+It would also make sense to add the following to your `.gitignore`:
+```ignore-list
+/.bundle/
+vendor
+```
+
+> Note: this used to be the recommended installation method, but with the library's maturation it's better to avoid the use of `Gemfile` and `bundle install` by just installing as per the "Quick Start" instructions above.
+
 
-It would also make sense to add the following to your `.gitignore`, or copy [the `.gitignore` used by this project](.gitignore):
+#### Using the latest-available code
+
+If you want to use the latest code on GitHub, your `Gemfile` would look like this:
+
+```ruby
+source 'https://rubygems.org'
 
+# to use the latest github code in a given repo and branch, replace the below values for git: and ref: as needed
+gem 'arduino_ci', git: 'https://github.com/ArduinoCI/arduino_ci.git', ref: '<your desired ref, branch, or tag>'
 ```
-/.bundle/
-/.yardoc
-Gemfile.lock
-/_yardoc/
-/coverage/
-/doc/
-/pkg/
-/spec/reports/
-vendor
-*.gem
 
-# rspec failure tracking
-.rspec_status
 
-# C++ stuff
-*.bin
-*.bin.dSYM
+#### Using a version of `arduino_ci` source code on your local machine
+
+First, Thanks!  See [CONTRIBUTING.md](CONTRIBUTING.md).  Your `Gemfile` would look like this:
+
+```ruby
+source 'https://rubygems.org'
+
+gem 'arduino_ci', path: '/path/to/development/dir/for/arduino_ci'
 ```
 
 
@@ -103,17 +125,18 @@ $ bundle install   # adds packages to global library (may require admin rights)
 $ bundle install --path vendor/bundle   # adds packages to local library
 ```
 
+This will create a `Gemfile.lock` in your project directory, which you may optionally check into source control.  A broader introduction to ruby dependencies is outside the scope of this document.
 
-### Running tests
+
+
+### Running `arduino_ci.rb` To Test Your Library
 
 With that installed, just the following shell command each time you want the tests to execute:
 
-```
+```console
 $ bundle exec arduino_ci.rb
 ```
 
-`arduino_ci.rb` is the main entry point for this library.  This command will iterate over all the library's `examples/` and attempt to compile them.  If you set up unit tests, it will run those as well.
-
 
 ### Reference
 
@@ -128,18 +151,14 @@ For more information on the usage of `arduino_ci.rb`, see [REFERENCE.md](REFEREN
 
 ## Setting up Pull Request Testing and/or External CI
 
-The following prerequisites must be fulfilled:
-
-* A GitHub (or other repository-hosting) project for your library
-* A CI system like [Travis CI](https://travis-ci.org/) or [Appveyor](https://www.appveyor.com/) that is linked to your project
-
+> **Note:** `arduino_ci.rb` expects to be run from the root directory of your Arduino project library.
 
-### Testing with remote CI
+### Arduino CI's Own GitHub action
 
-> **Note:** `arduino_ci.rb` expects to be run from the root directory of your Arduino project library.
+[![GitHub Marketplace](https://img.shields.io/badge/Get_it-on_Marketplace-informational.svg)](https://github.com/marketplace/actions/arduino_ci)
 
 
-#### GitHub Actions
+### Your Own Scripted GitHub Action
 
 GitHub Actions allows you to automate your workflows directly in GitHub.
 No additional steps are needed.
@@ -156,12 +175,12 @@ jobs:
         with:
           ruby-version: 2.6
       - run: |
-          bundle install
-          bundle exec arduino_ci_remote.rb
+          gem install arduino_ci
+          arduino_ci.rb
 ```
 
 
-#### Travis CI
+### Travis CI
 
 You'll need to go to https://travis-ci.org/profile/ and enable testing for your Arduino project.  Once that happens, you should be all set.  The script will test all example projects of the library and all unit tests.
 
@@ -171,12 +190,12 @@ Next, you need this in `.travis.yml` in your repo
 sudo: false
 language: ruby
 script:
-  - bundle install
-  - bundle exec arduino_ci.rb
+  - gem install arduino_ci
+  - arduino_ci.rb
 ```
 
 
-#### Appveyor CI
+### Appveyor CI
 
 You'll need to go to https://ci.appveyor.com/projects and add your project.
 
@@ -185,8 +204,8 @@ Next, you'll need this in `appveyor.yml` in your repo.
 ```yaml
 build: off
 test_script:
-  - bundle install
-  - bundle exec arduino_ci.rb
+  - gem install arduino_ci
+  - arduino_ci.rb
 ```
 
 

REFERENCE.md

@@ -46,7 +46,7 @@ This allows a file (or glob) pattern to be executed in your tests directory, cre
 
 ### `CUSTOM_INIT_SCRIPT` environment variable
 
-If set, testing will execute (using `/bin/sh`) the script referred to by this variable -- relative to the current working directory.  This enables use cases like the GitHub action to install custom library versions (i.e. a version of a library that is different than what the library manager would automatically install by name) prior to CI test runs.
+If set, testing will execute (using `/bin/sh`) the script referred to by this variable -- relative to the current working directory (i.e. the root directory of the library).  The script will _run_ in the Arduino Libraries directory (changing to the Libraries directory, running the script, and returning to the individual library root afterward).  This enables use cases like the GitHub action to install custom library versions (i.e. a version of a library that is different than what the library manager would automatically install by name) prior to CI test runs.
 
 
 ### `USE_SUBDIR` environment variable
@@ -64,11 +64,6 @@ If set, testing will fail if no unit test files are detected (or if the director
 If set, testing will fail if no example sketches are detected.  This is to avoid communicating a passing status in cases where a commit may have accidentally moved or deleted the examples.
 
 
-### `SKIP_LIBRARY_PROPERTIES` environment variable
-
-If set, testing will skip validating `library.properties` entries.  This is to work around any possible bugs in `arduino_ci`'s interpretation of what is "correct".
-
-
 ## Indirectly Overriding Build Behavior (medium term use), and Advanced Options
 
 For build behavior that you'd like to persist across commits (e.g. defining the set of platforms to test against, disabling a test that you expect to re-enable at some future point), a special configuration file called `.arduino-ci.yml` can be used.  There are 3 places you can put them:

SampleProjects/README.md

@@ -1,10 +1,9 @@
 Arduino Sample Projects
 =======================
 
-This directory contains projects that are intended solely for testing the various features of this gem -- to test the testing framework itself.  The RSpec tests refer specifically to these projects.
-
-Because of this, these projects include some intentional quirks that differ from what a well-formed an Arduino project for testing with `arduino_ci` might contain.  See other projects in the "Arduino-CI" GitHub organization for practical examples.
+This directory contains projects that are intended solely for testing the various features of this gem -- to test the testing framework itself.  The RSpec tests refer specifically to these projects, and as a result _some are explicity designed to fail_.
 
+> **If you are a first-time `arduino_ci` user an are looking for an example to copy from, see [the `Arduino-CI/Blink` repository](https://github.com/Arduino-CI/Blink) instead.**
 
 * "TestSomething" contains a minimial library, but tests for all the C++ compilation feature-mocks of arduino_ci.
 * "DoSomething" is a simple test of the testing framework (arduino_ci) itself to verfy that passes and failures are properly identified and reported.  Because of this, it includes test files that are expected to fail -- they are prefixed with "bad-".
@@ -13,3 +12,4 @@ Because of this, these projects include some intentional quirks that differ from
 * "OnePointFiveDummy" is a non-functional library meant to test file inclusion logic on libraries conforming to the ["1.5" specfication](https://arduino.github.io/arduino-cli/latest/library-specification/)
 * "DependOnSomething" is a non-functional library meant to test file inclusion logic with dependencies
 * "ExcludeSomething" is a non-functional library meant to test directory exclusion logic
+* "NetworkLib" tests the Ethernet library

SampleProjects/TestSomething/README.md

@@ -1,5 +1,6 @@
 # TestSomething
 
-This is a "beater" example that is referenced by tests of the Arduino CI module itself.
 
 All the tests of our mocked `Arduino.h` implementation live here.
+
+This example project is tightly coupled to the tests of the Arduino CI module itself.  In that sense, each of the individual test files will be illustrative of the testing strategy and capabilities of the core library itself.