9pm is a flexible and efficient framework for running test cases or suites defined in YAML files or specified via command-line arguments. It supports nested suites, detailed logging, and robust error handling, making it ideal for simply managing complex test setups.
- Arbitrary Test Execution: Run individual tests or entire suites from the command line.
- YAML-Defined Suites: Organize tests in structured, nested YAML files for reusability.
- Color-Coded Terminal Output: Easily identify test statuses with intuitive colors.
- "On-Fail" Logic: Define custom actions for failed tests to improve debugging.
- Masked Failures: Optionally ignore specific test failures or skips without halting the suite.
- Rich Reporting: Generate markdown summaries for easy sharing and tracking.
- Isolated Environment: Use temporary directories and files for scratch area. Ensuring nothing is left after test execution, even if the test itself crashes.
- Clone the repository:
git clone <repository-url> cd 9pm
- Install dependencies:
pip install pyyaml
Run the framework with:
./9pm.py [OPTIONS] SUITE | TEST ...
Option | Description |
---|---|
-a, --abort |
Stop execution after the first failure. |
-c, --config |
Specify a custom configuration file. |
-d, --debug |
Enable debug mode for verbose output. |
-o, --option |
Pass options to tests/suites (repeatable). |
Example:
./9pm.py -d -o "testopt1" test/suites/main.yaml test/cases/cleanup.sh
Test suites are YAML files that organize individual test cases or reference nested suites. Test cases must be executable files.
- case: "tests/smoke.sh"
- case: "tests/hammer.tcl"
- case: "tests/cleanup.py"
- case: "tests/smoke.sh"
- case: "tests/unit_test.tcl"
- suite: "suites/integration.yaml"
name: "Integration-Tests"
- suite: "suites/regression.yaml"
name: "Regression-Tests"
Name can be used effectivly when passing different options to the same test case.
- case: "tests/scp.sh"
opts:
- "192.168.1.1"
- "bootloader.bin"
name: "upload-bootloader"
- case: "tests/scp.sh"
opts:
- "192.168.1.1"
- "linux.sqfs"
name: "upload-os"
Resulting in the following:
o Execution
|-- o 0001-upload-bootloader
`-- o 0002-upload-os.sh
9pm generates detailed reports:
- Human-Readable Markdown:
result.md
- GitHub-Compatible Markdown:
result-gh.md
Status | Emoji |
---|---|
Passed | :white_check_mark: |
Failed | :red_circle: |
Skipped | :large_orange_diamond: |
Masked Failure | :o: |
Masked Skip | :small_orange_diamond: |
- Logs: Stored in timestamped directories under the configured
LOG_PATH
. A symlinklast
points to the latest log directory. - Scratch Directory: Temporary directories are created for each test run and cleaned up automatically.
- Temporary Database File: A temporary database file is used during the test lifecycle.
Tests must output results in TAP (Test Anything Protocol) format.
#!/bin/bash
echo "1..1"
echo "ok 1 - Sample test case"
This project is licensed under GPL-2.0 license. For contributions or issues, open a pull request or issue on GitHub.