9pm is a flexible and efficient framework for running test cases or suites defined in YAML files or specified via command-line arguments. It supports nested suites, detailed logging, and robust error handling, making it ideal for simply managing complex test setups.

---

Features

• Arbitrary Test Execution: Run individual tests or entire suites from the command line.
• YAML-Defined Suites: Organize tests in structured, nested YAML files for reusability.
• Color-Coded Terminal Output: Easily identify test statuses with intuitive colors.
• "On-Fail" Logic: Define custom actions for failed tests to improve debugging.
• Masked Failures: Optionally ignore specific test failures or skips without halting the suite.
• JSON Export: Export comprehensive test results in JSON format for further processing.
• Isolated Environment: Use temporary directories and files for scratch area. Ensuring nothing is left after test execution, even if the test itself crashes.

---

Installation

1. Clone the repository:

   git clone <repository-url>
   cd 9pm

2. Install core dependencies:

   pip install pyyaml

Note

This is all you need to run tests and export JSON results. For report generation, see the Report Generation section below.

---

Usage

Run the framework with:

./9pm.py [OPTIONS] SUITE | TEST ...

Harness Command-Line Options

Option	Description
-a, --abort	Stop execution after the first failure.
-v, --verbose	Enable verbose output.
-p, --proj	Specify an explicit project configuration.

Test Case Command-Line Options (Passed to Test Cases)

Option	Description
-c, --config	Test Case config.
-d, --debug	Enable test case debug.
-o, --option	Test case options (repeatable).

Example:

./9pm.py -o "ssh" suites/main.yaml cases/cleanup.sh

---

Writing Test Suites

Test suites are YAML files that organize individual test cases or reference nested suites. Test cases must be executable files.

Simple Test Suite Example

- case: "tests/smoke.sh"
- case: "tests/hammer.pl"
- case: "tests/cleanup.py"

Nested Test Suite Example

- case: "tests/smoke.sh"
- case: "tests/unit_test.tcl"
- suite: "suites/integration.yaml"
  name: "Integration-Tests"
- suite: "suites/regression.yaml"
  name: "Regression-Tests"

Option Naming

Name can be used effectivly when passing different options to the same test case.

- case: "tests/scp.sh"
  opts:
    - "192.168.1.1"
    - "bootloader.bin"
  name: "upload-bootloader"
- case: "tests/scp.sh"
  opts:
    - "192.168.1.1"
    - "linux.sqfs"
  name: "upload-os"

Resulting in the following:

o Execution
|-- o 0001-upload-bootloader
`-- o 0002-upload-os.sh

Report Generation

9pm exports test results as JSON by default. Additional report formats can be generated using the report.py tool.

JSON Export (Always Available)

Every test run generates a comprehensive result.json file containing:

• Test metadata and timestamps
• Complete hierarchical test structure
• Embedded test output logs
• Summary statistics

Optional Report Formats

Install additional dependencies for report generation:

# For PDF reports
gem install --user-install asciidoctor-pdf rouge

# On Debian/Ubuntu
sudo apt install ruby-asciidoctor-pdf ruby-rouge

Generate reports from JSON:

# Generate specific format
python3 report.py result.json github     # -> result-gh.md
python3 report.py result.json markdown   # -> result.md
python3 report.py result.json asciidoc   # -> report.adoc

# Generate all formats
python3 report.py result.json all

# Generate PDF from AsciiDoc
make report  # Generates report.pdf

Report Formats

1. GitHub Markdown (result-gh.md): GitHub-compatible with emoji status indicators
2. Plain Markdown (result.md): Simple markdown format
3. AsciiDoc (report.adoc): Comprehensive report with embedded test logs
4. PDF (report.pdf): Professional report generated from AsciiDoc

GitHub Emoji Legend

Status	Emoji
Passed	:white_check_mark:
Failed	:red_circle:
Skipped	:large_orange_diamond:
Masked Failure	:o:
Masked Skip	:small_orange_diamond:

---

Logging and Environment

• Logs: Stored in timestamped directories under the configured LOG_PATH. A symlink last points to the latest log directory.
• Scratch Directory: Temporary directories are created for each test run and cleaned up automatically.
• Temporary Database File: A temporary database file is used during the test lifecycle.

---

Writing Test Cases

Tests must output results in TAP (Test Anything Protocol) format.

Example Test Case

#!/bin/bash
echo "1..1"
echo "ok 1 - Sample test case"

---

License

This project is licensed under GPL-2.0 license. For contributions or issues, open a pull request or issue on GitHub.