Skip to content

Install standardized CI system #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Oct 4, 2021
Merged

Install standardized CI system #9

merged 18 commits into from
Oct 4, 2021

Conversation

per1234
Copy link
Contributor

@per1234 per1234 commented Oct 2, 2021

It is standard practice to install a standardized set of GitHub Actions workflows and tasks in Arduino tooling repositories.

These are maintained in a centralized location, from/to which improvements and fixes are synced:
https://github.com/arduino/tooling-project-assets

It is not possible to add the Go-related assets at this time because they were designed for use with Go module based projects, which this project is not.

Resolves (partially) #8


Check license file

Whenever one of the recognized license file names are modified in the repository, the workflow runs licensee to check whether the license can be recognized and whether it is of the expected type.

GitHub has a useful automated license detection system that determines the license type used by a repository, and surfaces that information in the repository home page, the search web interface, and the GitHub API. This license detection system requires that the license be defined by a dedicated file with one of several standardized filenames and paths.

GitHub's license detection system uses the popular licensee tool, so this file also serves to define the license type for any other usages of licensee, as well as to human readers of the file.

For this reason, and to ensure it remains a valid legal instrument, it's important that there be no non-standard modifications to the license file or collisions with other supported license files. This workflow ensures that any changes which would change the license type or which license file is used by the detection are caught automatically.


General formatting

On every push, pull request, and periodically, use editorconfig-checker check whether the repository's files are formatted according to .editorconfig.


Prettier

On every push and pull request that affects relevant files, check whether the formatting of supported files is compliant with the Prettier style.


Shell scripts

On every push or pull request that modifies one of the shell scripts in the repository, and periodically, the workflow:

  • Runs ShellCheck to detect common problems.
  • Runs shfmt to check formatting.
  • Checks for forgotten executable script file permissions.

Taskfiles

On every push or pull request that affects the repository's Taskfiles, and periodically, validate them against the JSON schema.


GitHub Actions workflows

On every push or pull request that affects the repository's GitHub Actions workflows, and periodically, validate them against the JSON schema.


YAML

On every push and pull request that affects relevant files, run yamllint to check the YAML files of the repository for issues.

The .yamllint.yml file is used to configure yamllint:
https://yamllint.readthedocs.io/en/stable/configuration.html


Spell check

On every push, pull request, and periodically, use codespell to check for commonly misspelled words.

In the event of a false positive, the problematic word should be added, in all lowercase, to the ignore-words-list field of .codespellrc. Regardless of the case of the word in the false positive, it must be in all lowercase in the ignore list. The ignore list is comma-separated with no spaces.


Markdown

On every push and pull request that affects relevant files, and periodically, check the repository's Markdown files for
problems:

The Arduino tooling Markdown style is defined by the .markdownlint.yml file.

In the event the repository contains externally maintained Markdown files, markdownlint can be configured to ignore them via a .markdownlintignore file:
https://github.com/igorshubovych/markdownlint-cli#ignoring-files

markdown-link-check is configured via the .markdown-link-check.json file:
https://github.com/tcort/markdown-link-check#config-file-format

per1234 added 18 commits October 1, 2021 13:14
This file is not written in Markdown, so the previous `.md` file extension was incorrect. The options for an appropriate standardized file name are:

- `LICENSE.txt`
- `LICENSE`

I chose the former because the file is in text format and the `.txt` file extension clearly communicates this to humans
and machines both, making the information in the file more accessible than would be the case with the latter file name.
This is the standard license file name in Arduino tooling projects for this reason.
Whenever one of the recognized license file names are modified in the repository, the workflow runs to check whether the
license can be recognized and whether it is of the expected type.

GitHub has a useful automated license detection system that determines the license type used by a repository, and
surfaces that information in the repository home page, the search web interface, and the GitHub API. This license
detection system requires that the license be defined by a dedicated file with one of several standardized filenames and
paths.

GitHub's license detection system uses the popular licensee tool, so this file also serves to define the license type
for any other usages of licensee, as well as to human readers of the file.

For this reason, and to ensure it remains a valid legal instrument, it's important that there be no non-standard
modifications to the license file or collisions with other supported license files. This workflow ensures that any
changes which would change the license type or which license file is used by the detection are caught automatically.
On every push, pull request, and periodically, check whether the repository's files are formatted according to
.editorconfig.
On every push and pull request that affects relevant files, check whether the formatting of supported
files is compliant with the Prettier style.
On every push or pull request that modifies one of the shell scripts in the repository, and periodically, the workflow:

- Runs ShellCheck to detect common problems.
- Runs shfmt to check formatting.
- Checks for forgotten executable script file permissions.
On every push or pull request that affects the repository's Taskfiles, and periodically, validate them
against the JSON schema.
On every push or pull request that affects the repository's GitHub Actions workflows, and periodically, validate them
against the JSON schema.
On every push and pull request that affects relevant files, run yamllint to check the YAML files of
the repository for issues.

The .yamllint.yml file is used to configure yamllint:
https://yamllint.readthedocs.io/en/stable/configuration.html
On every push, pull request, and periodically, use codespell to check for commonly
misspelled words.

In the event of a false positive, the problematic word should be added, in all lowercase, to the `ignore-words-list`
field of `.codespellrc`. Regardless of the case of the word in the false positive, it must be in all lowercase in the
ignore list. The ignore list is comma-separated with no spaces.
On every push and pull request that affects relevant files, and periodically, check the repository's Markdown files for
problems:

- Use markdownlint to check for common problems and formatting.
- Use markdown-link-check to check for broken links.

The Arduino tooling Markdown style is defined by the `.markdownlint.yml` file.

In the event the repository contains externally maintained Markdown files, markdownlint can be configured to ignore them
via a `.markdownlintignore` file:
https://github.com/igorshubovych/markdownlint-cli#ignoring-files

markdown-link-check is configured via the `.markdown-link-check.json` file:
https://github.com/tcort/markdown-link-check#config-file-format
Arduino tooling projects use two spaces for indentation in shell scripts. Previously, the script used a random mixture of
 three space and tab indents which didn't even match the program structure.
Superfluous blank lines are not permitted by the shfmt formatting tool now used to maintain consistent shell code
formatting in this repository.
The previous backticks command substitution syntax is discouraged in favor of the modern `$()` syntax for the reasons described here:
http://mywiki.wooledge.org/BashFAQ/082
Stinginess with quoting in shell scripts is the most common cause of confusing bugs that usually only occur under
specific uncommon conditions. For this reason, all variables must be quoted.

More information:
https://github.com/koalaman/shellcheck/wiki/SC2086
The deploy script contains a fairly complex command used to parse the code to get the current version. This command used
`cat` to pipe the contents of the file to `grep`. But `grep` has a filename argument, so this was unnecessary.

More information:
https://github.com/koalaman/shellcheck/wiki/SC2002
The valued advice provided by the ShellCheck shell script static analysis tool is:
https://github.com/koalaman/shellcheck/wiki/SC2162

> You should always use `-r` unless you have a good reason not to.

In order to achieve ShellCheck compliance, it is necessary to do one of the following:

- Add the `-r` flag to the `read` command.
- Add a ShellCheck directive to the script to disable rule SC2162 for this line.

In this particular application, `-r` is not necessary, but it also does no harm. So I think the first option is best.
This is the shell script code formatting style required for compliance with the shfmt formatter tool.
This is the shell script code formatting style required for compliance with the shfmt formatter tool.
@per1234 per1234 added type: enhancement Proposed improvement topic: infrastructure Related to project infrastructure labels Oct 2, 2021
@per1234 per1234 merged commit 797c5a0 into arduino:master Oct 4, 2021
@per1234 per1234 deleted the ci branch October 4, 2021 12:00
@per1234 per1234 self-assigned this Nov 23, 2021
@per1234 per1234 linked an issue Sep 15, 2022 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: infrastructure Related to project infrastructure type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bring CI up to tooling standard for the arduinoOTA repo
2 participants