99 lines
3.2 KiB
Markdown
99 lines
3.2 KiB
Markdown
---
|
|
name: regression-validation-gate-runner
|
|
description: Use when Codex needs to dry-run or execute project-declared regression, validation, test, lint, build, schema, export, or review gates from a JSON, YAML, or Markdown manifest and capture logs before review, release, handoff, CCRA/local review, or lifecycle-sensitive work.
|
|
---
|
|
|
|
# Regression Validation Gate Runner
|
|
|
|
## Purpose
|
|
|
|
Run or verify explicitly declared validation gates and write a durable evidence package. Use the bundled script instead of ad hoc terminal copying when a reviewer needs command declarations, exit codes, logs, durations, skipped gates, and changed-file notes.
|
|
|
|
Passing gates are engineering evidence only. They do not imply product acceptance, lifecycle approval, Owner approval, or CCRA approval.
|
|
|
|
## Command
|
|
|
|
```powershell
|
|
conda run -n skills-vault python .\scripts\regression_validation_gate_runner.py `
|
|
--project-root C:\path\project `
|
|
--gate-manifest C:\path\gate-manifest.yaml `
|
|
--output-dir C:\path\helper-outputs `
|
|
--mode dry_run
|
|
```
|
|
|
|
Use `--mode run` only when the user has asked to execute the manifest commands. Add `--include-optional` only when optional gates should run.
|
|
|
|
When installed under `C:\Users\wangq\.agents\skills\regression-validation-gate-runner`, run from that Skill directory or use the installed script path.
|
|
|
|
## Manifest
|
|
|
|
The manifest may be JSON, YAML, YML, or Markdown containing one fenced `yaml`, `yml`, or `json` block.
|
|
|
|
```yaml
|
|
gates:
|
|
- gate_id: unit-tests
|
|
command:
|
|
- conda
|
|
- run
|
|
- -n
|
|
- skills-vault
|
|
- python
|
|
- -B
|
|
- -m
|
|
- unittest
|
|
- discover
|
|
- -s
|
|
- skills/example/tests
|
|
- -v
|
|
working_directory: .
|
|
expected_exit_code: 0
|
|
log_file: logs/unit-tests.log
|
|
timeout_seconds: 120
|
|
required_before_review: true
|
|
```
|
|
|
|
`command` may be a string or an argv list. Lists avoid shell quoting surprises on Windows.
|
|
|
|
## Modes
|
|
|
|
- `dry_run`: parse and validate the manifest, then report declared and skipped gates without executing commands.
|
|
- `run`: execute required gates. Optional gates are skipped unless `--include-optional` is passed.
|
|
|
|
## Outputs
|
|
|
|
The script writes under `output_dir`:
|
|
|
|
```text
|
|
gate-run-report.md
|
|
gate-run-report.json
|
|
logs/<gate-id>.log
|
|
```
|
|
|
|
Reports include:
|
|
|
|
- commands declared, run, and skipped
|
|
- exit codes and pass/fail/timeout status
|
|
- duration and log paths
|
|
- environment notes
|
|
- project files changed during each gate
|
|
- machine-readable summary
|
|
|
|
Exit code is `0` for `PASS` or `DRY_RUN`, `1` for failed or timed-out required gates, and `2` for manifest/input errors.
|
|
|
|
## Safety
|
|
|
|
- Run only commands explicitly listed in the manifest.
|
|
- Keep `working_directory` under `project_root`.
|
|
- Write logs and reports only under `output_dir`.
|
|
- Do not install dependencies, invent commands, edit project files directly, or repair failures.
|
|
- Record changed project files if a command creates, modifies, or deletes files; do not hide side effects.
|
|
|
|
## Validation
|
|
|
|
After changing runner behavior, run:
|
|
|
|
```powershell
|
|
conda run -n skills-vault python -B -m unittest discover -s skills/regression-validation-gate-runner/tests -v
|
|
conda run -n skills-vault python -B C:\Users\wangq\.codex\skills\.system\skill-creator\scripts\quick_validate.py skills/regression-validation-gate-runner
|
|
```
|