feat: add lifecycle status guard scan skill
This commit is contained in:
parent
419d70ab94
commit
a2e8f1bd7a
|
|
@ -22,6 +22,7 @@ PowerShell:
|
|||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File .\scripts\install-skill.ps1 -Skill bundle-zip
|
||||
powershell -ExecutionPolicy Bypass -File .\scripts\install-skill.ps1 -Skill lifecycle-status-guard-scan
|
||||
powershell -ExecutionPolicy Bypass -File .\scripts\install-skill.ps1 -Skill repair-markdown-citations
|
||||
```
|
||||
|
||||
|
|
@ -29,6 +30,7 @@ Use `-Force` to back up and replace an existing installed copy:
|
|||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File .\scripts\install-skill.ps1 -Skill bundle-zip -Force
|
||||
powershell -ExecutionPolicy Bypass -File .\scripts\install-skill.ps1 -Skill lifecycle-status-guard-scan -Force
|
||||
powershell -ExecutionPolicy Bypass -File .\scripts\install-skill.ps1 -Skill repair-markdown-citations -Force
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -15,6 +15,7 @@ Existing CCPE content is not migrated here.
|
|||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| `bundle-zip` | Create zip archives from explicit file lists while preserving source-relative paths and validating entries by readback. | `C:\Users\wangq\Documents\Codex\skills-vault\skills\bundle-zip` | `installed` | `default` | `C:\Users\wangq\.agents\skills\bundle-zip` | none |
|
||||
| `fix-title` | Shift Markdown ATX heading depth for pasted LLM replies before insertion under parent sections. | `C:\Users\wangq\Documents\Codex\knowledge-vault\skills\fix-title` | `installed` | `default` | `C:\Users\wangq\.agents\skills\fix-title` | none |
|
||||
| `lifecycle-status-guard-scan` | Scan configured Markdown, JSON, YAML, and text files for lifecycle/status overclaim candidates and missing local evidence markers. | `C:\Users\wangq\Documents\Codex\skills-vault\skills\lifecycle-status-guard-scan` | `installed` | `default` | `C:\Users\wangq\.agents\skills\lifecycle-status-guard-scan` | none |
|
||||
| `repair-markdown-citations` | Repair ChatGPT/Deep Research Markdown citation tokens into standard footnotes and a deduplicated reference section using exact turn-ID metadata mapping. | `C:\Users\wangq\Documents\Codex\skills-vault\skills\repair-markdown-citations` | `installed` | `default` | `C:\Users\wangq\.agents\skills\repair-markdown-citations` | none |
|
||||
| `voice-generation` | Generate spoken audio from Markdown scripts using the local `voice-gen` CLI and MiniMax `mmx`, including custom voice clone registration and voice registry management. | `D:\AI\ClaudeCode\voice-ge` | `installed` | `conditional` | `C:\Users\wangq\.agents\skills\voice-generation` | none |
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,41 @@
|
|||
# Lifecycle Status Guard Scan
|
||||
|
||||
Source Skill for scanning lifecycle/status overclaim candidates in Markdown, JSON, YAML, and text files.
|
||||
|
||||
The canonical entry point is `SKILL.md`; the deterministic scanner is in `scripts/lifecycle_status_guard_scan.py`.
|
||||
|
||||
## Original Source
|
||||
|
||||
```text
|
||||
Original source: C:\Users\wangq\Documents\Codex\ccpe-system\requirements\skills-vault\2026-06-19-lifecycle-status-guard-scan.md
|
||||
Migration date: 2026-06-19
|
||||
Migration status: first public Skill implementation
|
||||
Behavior preserved: deterministic scan only; no lifecycle judgment or file edits
|
||||
Known gaps: no project-specific policy preset beyond the example fixture
|
||||
```
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
SKILL.md
|
||||
README.md
|
||||
agents/openai.yaml
|
||||
fixtures/lifecycle-guard-config.example.yaml
|
||||
scripts/lifecycle_status_guard_scan.py
|
||||
tests/test_lifecycle_status_guard_scan.py
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```powershell
|
||||
conda run -n skills-vault python .\scripts\lifecycle_status_guard_scan.py `
|
||||
--scan-root C:\path\project `
|
||||
--config C:\path\lifecycle-guard-config.yaml `
|
||||
--output-dir C:\path\helper-outputs
|
||||
```
|
||||
|
||||
## Tests
|
||||
|
||||
```powershell
|
||||
conda run -n skills-vault python -B -m unittest discover -s skills/lifecycle-status-guard-scan/tests -v
|
||||
```
|
||||
|
|
@ -0,0 +1,102 @@
|
|||
---
|
||||
name: lifecycle-status-guard-scan
|
||||
description: Use when Codex needs to scan Markdown, JSON, YAML, or text files for lifecycle/status overclaims such as stable, accepted, owner-approved, CCRA-approved, released, production-ready, final, or equivalent Chinese phrasing before review, release, handoff, or governance work.
|
||||
---
|
||||
|
||||
# Lifecycle Status Guard Scan
|
||||
|
||||
## Purpose
|
||||
|
||||
Run a deterministic, configurable scan for lifecycle/status claim candidates. Use the bundled script to produce Markdown and JSON reports that a human, Owner, CCRA reviewer, or upper-layer agent can judge.
|
||||
|
||||
The scanner is evidence-producing only. It must not edit status fields, promote or downgrade lifecycle state, or decide whether an approval claim is true.
|
||||
|
||||
## Command
|
||||
|
||||
```powershell
|
||||
conda run -n skills-vault python .\scripts\lifecycle_status_guard_scan.py `
|
||||
--scan-root C:\path\project `
|
||||
--config C:\path\lifecycle-guard-config.yaml `
|
||||
--output-dir C:\path\helper-outputs
|
||||
```
|
||||
|
||||
When installed under `C:\Users\wangq\.agents\skills\lifecycle-status-guard-scan`, run from that Skill directory or use the installed script path.
|
||||
|
||||
## Config
|
||||
|
||||
```yaml
|
||||
watched_paths:
|
||||
- "**/*.md"
|
||||
- "**/*.json"
|
||||
- "**/*.yaml"
|
||||
status_fields:
|
||||
- status
|
||||
- lifecycle
|
||||
forbidden_status_values:
|
||||
- stable
|
||||
- accepted
|
||||
warning_status_values:
|
||||
- candidate
|
||||
required_evidence_markers:
|
||||
- owner_decision
|
||||
- ccra_review
|
||||
approved_phrases:
|
||||
- owner-approved
|
||||
- ccra-approved
|
||||
- 所有者批准
|
||||
forbidden_phrases:
|
||||
- production-ready
|
||||
allowed_contexts:
|
||||
- do not say
|
||||
- example:
|
||||
```
|
||||
|
||||
Config files may be JSON, YAML, or YML.
|
||||
|
||||
## Rules
|
||||
|
||||
- Treat `approved_phrases` as approval-claim phrases requiring evidence. They are not a whitelist and do not mean the claim is accepted.
|
||||
- If an approval claim phrase lacks a nearby `required_evidence_markers` match, report a blocking phrase finding.
|
||||
- If an approval claim phrase has nearby evidence, report only an evidence-present observation. Do not judge whether the approval is real.
|
||||
- For structured fields, associate evidence only within the same front matter block or JSON/YAML object that contains the matched field.
|
||||
- For phrase claims, associate evidence only within the configured nearby text window, default 240 characters.
|
||||
- Apply `allowed_contexts` only to phrase-level claims, such as negations, examples, or policy text. Never use allowed contexts to exempt structured fields like `status: stable`.
|
||||
|
||||
## Outputs
|
||||
|
||||
The script writes:
|
||||
|
||||
```text
|
||||
lifecycle-status-guard-scan.md
|
||||
lifecycle-status-guard-scan.json
|
||||
```
|
||||
|
||||
Reports include:
|
||||
|
||||
- files scanned
|
||||
- possible overclaims
|
||||
- field-level findings
|
||||
- phrase-level findings
|
||||
- missing evidence markers
|
||||
- warnings
|
||||
- blocking errors
|
||||
- evidence-present observations
|
||||
- machine-readable summary
|
||||
|
||||
Exit code is `1` when blocking findings exist and `0` when the scan has only warnings or observations.
|
||||
|
||||
## Safety
|
||||
|
||||
- Read only configured files under `scan_root`.
|
||||
- Write only the two report files under `output_dir`.
|
||||
- Do not modify scanned files.
|
||||
- Do not treat evidence markers as proof of approval; they only prevent a missing-evidence blocking result.
|
||||
|
||||
## Validation
|
||||
|
||||
After changing scanner behavior, run:
|
||||
|
||||
```powershell
|
||||
conda run -n skills-vault python -B -m unittest discover -s skills/lifecycle-status-guard-scan/tests -v
|
||||
conda run -n skills-vault python -B C:\Users\wangq\.codex\skills\.system\skill-creator\scripts\quick_validate.py skills/lifecycle-status-guard-scan
|
||||
```
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
interface:
|
||||
display_name: "Lifecycle Status Guard Scan"
|
||||
short_description: "Scan lifecycle claims for evidence gaps"
|
||||
default_prompt: "Use $lifecycle-status-guard-scan to scan configured project files for lifecycle or approval claims that need evidence."
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
watched_paths:
|
||||
- "**/*.md"
|
||||
- "**/*.json"
|
||||
- "**/*.yaml"
|
||||
- "**/*.yml"
|
||||
status_fields:
|
||||
- status
|
||||
- lifecycle
|
||||
- model.status
|
||||
- model.lifecycle
|
||||
forbidden_status_values:
|
||||
- stable
|
||||
- accepted
|
||||
- owner-approved
|
||||
- ccra-approved
|
||||
warning_status_values:
|
||||
- candidate
|
||||
- ready-for-review
|
||||
required_evidence_markers:
|
||||
- owner_decision
|
||||
- owner decision
|
||||
- ccra_review
|
||||
- ccra review
|
||||
- web ccra
|
||||
- 所有者决定记录
|
||||
approved_phrases:
|
||||
- owner-approved
|
||||
- ccra-approved
|
||||
- Owner approved
|
||||
- CCRA approved
|
||||
- 所有者批准
|
||||
- CCRA批准
|
||||
forbidden_phrases:
|
||||
- production-ready
|
||||
- released to production
|
||||
- final approved
|
||||
allowed_contexts:
|
||||
- do not say
|
||||
- must not claim
|
||||
- example:
|
||||
- 示例:
|
||||
evidence_window_chars: 240
|
||||
|
|
@ -0,0 +1,448 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import fnmatch
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
from collections.abc import Iterable, Mapping
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
REPORT_BASENAME = "lifecycle-status-guard-scan"
|
||||
DEFAULT_EVIDENCE_WINDOW_CHARS = 240
|
||||
|
||||
|
||||
def as_list(value: Any) -> list[str]:
|
||||
if value is None:
|
||||
return []
|
||||
if isinstance(value, str):
|
||||
return [value]
|
||||
if isinstance(value, Iterable):
|
||||
return [str(item) for item in value if item is not None]
|
||||
return [str(value)]
|
||||
|
||||
|
||||
def lower_set(values: Iterable[str]) -> set[str]:
|
||||
return {value.casefold() for value in values}
|
||||
|
||||
|
||||
def normalize_config(raw_config: Mapping[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"watched_paths": as_list(raw_config.get("watched_paths")) or ["**/*"],
|
||||
"status_fields": as_list(raw_config.get("status_fields")),
|
||||
"forbidden_status_values": lower_set(as_list(raw_config.get("forbidden_status_values"))),
|
||||
"warning_status_values": lower_set(as_list(raw_config.get("warning_status_values"))),
|
||||
"required_evidence_markers": as_list(raw_config.get("required_evidence_markers")),
|
||||
"approved_phrases": as_list(raw_config.get("approved_phrases")),
|
||||
"forbidden_phrases": as_list(raw_config.get("forbidden_phrases")),
|
||||
"allowed_contexts": as_list(raw_config.get("allowed_contexts")),
|
||||
"evidence_window_chars": int(
|
||||
raw_config.get("evidence_window_chars", DEFAULT_EVIDENCE_WINDOW_CHARS)
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def load_config(path: pathlib.Path) -> dict[str, Any]:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
if path.suffix.lower() == ".json":
|
||||
parsed = json.loads(text)
|
||||
else:
|
||||
parsed = yaml.safe_load(text)
|
||||
if not isinstance(parsed, Mapping):
|
||||
raise ValueError("Config must be a JSON or YAML object.")
|
||||
return normalize_config(parsed)
|
||||
|
||||
|
||||
def rel_path(path: pathlib.Path, root: pathlib.Path) -> str:
|
||||
return path.relative_to(root).as_posix()
|
||||
|
||||
|
||||
def discover_files(scan_root: pathlib.Path, watched_paths: list[str]) -> list[pathlib.Path]:
|
||||
files: dict[pathlib.Path, None] = {}
|
||||
for pattern in watched_paths:
|
||||
for candidate in scan_root.glob(pattern):
|
||||
if candidate.is_file():
|
||||
files[candidate.resolve()] = None
|
||||
return sorted(files)
|
||||
|
||||
|
||||
def parse_markdown_front_matter(text: str) -> Any | None:
|
||||
if not text.startswith("---"):
|
||||
return None
|
||||
lines = text.splitlines()
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return None
|
||||
for index in range(1, len(lines)):
|
||||
if lines[index].strip() == "---":
|
||||
front_matter = "\n".join(lines[1:index])
|
||||
parsed = yaml.safe_load(front_matter) if front_matter.strip() else {}
|
||||
return parsed if isinstance(parsed, Mapping) else None
|
||||
return None
|
||||
|
||||
|
||||
def parse_structured_content(path: pathlib.Path, text: str) -> Any | None:
|
||||
suffix = path.suffix.lower()
|
||||
try:
|
||||
if suffix == ".json":
|
||||
return json.loads(text)
|
||||
if suffix in {".yaml", ".yml"}:
|
||||
return yaml.safe_load(text)
|
||||
if suffix in {".md", ".markdown"}:
|
||||
return parse_markdown_front_matter(text)
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def scalar_values(value: Any) -> list[str]:
|
||||
if isinstance(value, (str, int, float, bool)):
|
||||
return [str(value)]
|
||||
if isinstance(value, list):
|
||||
values: list[str] = []
|
||||
for item in value:
|
||||
values.extend(scalar_values(item))
|
||||
return values
|
||||
return []
|
||||
|
||||
|
||||
def object_contains_marker(value: Any, markers: list[str]) -> bool:
|
||||
if not markers:
|
||||
return False
|
||||
marker_needles = [marker.casefold() for marker in markers]
|
||||
|
||||
def walk(current: Any) -> bool:
|
||||
if isinstance(current, Mapping):
|
||||
for key, nested in current.items():
|
||||
key_text = str(key).casefold()
|
||||
if any(marker in key_text for marker in marker_needles):
|
||||
return True
|
||||
if walk(nested):
|
||||
return True
|
||||
return False
|
||||
if isinstance(current, list):
|
||||
return any(walk(item) for item in current)
|
||||
text = str(current).casefold()
|
||||
return any(marker in text for marker in marker_needles)
|
||||
|
||||
return walk(value)
|
||||
|
||||
|
||||
def line_col(text: str, index: int) -> tuple[int, int]:
|
||||
line = text.count("\n", 0, index) + 1
|
||||
last_newline = text.rfind("\n", 0, index)
|
||||
column = index + 1 if last_newline == -1 else index - last_newline
|
||||
return line, column
|
||||
|
||||
|
||||
def make_context(text: str, index: int, length: int, window: int) -> str:
|
||||
start = max(0, index - window)
|
||||
end = min(len(text), index + length + window)
|
||||
return re.sub(r"\s+", " ", text[start:end]).strip()
|
||||
|
||||
|
||||
def context_has_any(context: str, values: list[str]) -> bool:
|
||||
folded = context.casefold()
|
||||
return any(value.casefold() in folded for value in values)
|
||||
|
||||
|
||||
def find_phrase_matches(text: str, phrase: str) -> Iterable[int]:
|
||||
if not phrase:
|
||||
return []
|
||||
folded_text = text.casefold()
|
||||
folded_phrase = phrase.casefold()
|
||||
matches: list[int] = []
|
||||
start = 0
|
||||
while True:
|
||||
index = folded_text.find(folded_phrase, start)
|
||||
if index == -1:
|
||||
break
|
||||
matches.append(index)
|
||||
start = index + max(len(folded_phrase), 1)
|
||||
return matches
|
||||
|
||||
|
||||
def path_matches(field_path: str, key: str, status_fields: list[str]) -> bool:
|
||||
if not status_fields:
|
||||
return False
|
||||
candidates = {field_path, key}
|
||||
return any(
|
||||
field in candidates or fnmatch.fnmatch(field_path, field) or fnmatch.fnmatch(key, field)
|
||||
for field in status_fields
|
||||
)
|
||||
|
||||
|
||||
def child_path(parent_path: str, key: str) -> str:
|
||||
return f"{parent_path}.{key}" if parent_path else key
|
||||
|
||||
|
||||
def list_path(parent_path: str, index: int) -> str:
|
||||
return f"{parent_path}[{index}]" if parent_path else f"[{index}]"
|
||||
|
||||
|
||||
def scan_structured(
|
||||
data: Any,
|
||||
file_rel: str,
|
||||
config: dict[str, Any],
|
||||
report: dict[str, Any],
|
||||
) -> None:
|
||||
status_fields = config["status_fields"]
|
||||
forbidden_values = config["forbidden_status_values"]
|
||||
warning_values = config["warning_status_values"]
|
||||
evidence_markers = config["required_evidence_markers"]
|
||||
|
||||
def walk(current: Any, path: str, parent: Any) -> None:
|
||||
if isinstance(current, Mapping):
|
||||
for key_obj, value in current.items():
|
||||
key = str(key_obj)
|
||||
current_path = child_path(path, key)
|
||||
if path_matches(current_path, key, status_fields):
|
||||
for scalar in scalar_values(value):
|
||||
value_key = scalar.casefold()
|
||||
if value_key in forbidden_values:
|
||||
evidence_present = object_contains_marker(parent, evidence_markers)
|
||||
record = {
|
||||
"file": file_rel,
|
||||
"source": "structured",
|
||||
"path": current_path,
|
||||
"field": key,
|
||||
"value": scalar,
|
||||
"severity": "blocking" if not evidence_present else "observation",
|
||||
"evidence_present": evidence_present,
|
||||
"message": (
|
||||
"Forbidden lifecycle/status value lacks local evidence marker."
|
||||
if not evidence_present
|
||||
else "Lifecycle/status value has a local evidence marker; approval validity is not decided by this scan."
|
||||
),
|
||||
}
|
||||
if evidence_present:
|
||||
observation = dict(record)
|
||||
observation["kind"] = "field_evidence_present"
|
||||
report["observations"].append(observation)
|
||||
else:
|
||||
report["field_level_findings"].append(record)
|
||||
report["blocking_errors"].append(record["message"])
|
||||
report["missing_evidence_markers"].append(
|
||||
{
|
||||
"file": file_rel,
|
||||
"path": current_path,
|
||||
"required_evidence_markers": evidence_markers,
|
||||
}
|
||||
)
|
||||
elif value_key in warning_values:
|
||||
report["warnings"].append(
|
||||
{
|
||||
"file": file_rel,
|
||||
"source": "structured",
|
||||
"path": current_path,
|
||||
"field": key,
|
||||
"value": scalar,
|
||||
"severity": "warning",
|
||||
"message": "Warning lifecycle/status value matched configured policy.",
|
||||
}
|
||||
)
|
||||
walk(value, current_path, value if isinstance(value, Mapping) else current)
|
||||
elif isinstance(current, list):
|
||||
for index, item in enumerate(current):
|
||||
walk(item, list_path(path, index), item if isinstance(item, Mapping) else parent)
|
||||
|
||||
walk(data, "", data)
|
||||
|
||||
|
||||
def scan_phrases(
|
||||
text: str,
|
||||
file_rel: str,
|
||||
config: dict[str, Any],
|
||||
report: dict[str, Any],
|
||||
) -> None:
|
||||
evidence_markers = config["required_evidence_markers"]
|
||||
allowed_contexts = config["allowed_contexts"]
|
||||
window = config["evidence_window_chars"]
|
||||
|
||||
for phrase in config["approved_phrases"]:
|
||||
for index in find_phrase_matches(text, phrase):
|
||||
context = make_context(text, index, len(phrase), window)
|
||||
if context_has_any(context, allowed_contexts):
|
||||
continue
|
||||
evidence_present = context_has_any(context, evidence_markers)
|
||||
line, column = line_col(text, index)
|
||||
record = {
|
||||
"file": file_rel,
|
||||
"source": "text",
|
||||
"phrase": phrase,
|
||||
"line": line,
|
||||
"column": column,
|
||||
"severity": "blocking" if not evidence_present else "observation",
|
||||
"evidence_present": evidence_present,
|
||||
"context": context,
|
||||
"message": (
|
||||
"Approval claim phrase lacks nearby evidence marker."
|
||||
if not evidence_present
|
||||
else "Approval claim phrase has nearby evidence marker; approval validity is not decided by this scan."
|
||||
),
|
||||
}
|
||||
if evidence_present:
|
||||
observation = dict(record)
|
||||
observation["kind"] = "evidence_present_claim"
|
||||
report["observations"].append(observation)
|
||||
else:
|
||||
report["phrase_level_findings"].append(record)
|
||||
report["blocking_errors"].append(record["message"])
|
||||
report["missing_evidence_markers"].append(
|
||||
{
|
||||
"file": file_rel,
|
||||
"line": line,
|
||||
"phrase": phrase,
|
||||
"required_evidence_markers": evidence_markers,
|
||||
}
|
||||
)
|
||||
|
||||
for phrase in config["forbidden_phrases"]:
|
||||
for index in find_phrase_matches(text, phrase):
|
||||
context = make_context(text, index, len(phrase), window)
|
||||
if context_has_any(context, allowed_contexts):
|
||||
continue
|
||||
line, column = line_col(text, index)
|
||||
record = {
|
||||
"file": file_rel,
|
||||
"source": "text",
|
||||
"phrase": phrase,
|
||||
"line": line,
|
||||
"column": column,
|
||||
"severity": "blocking",
|
||||
"evidence_present": context_has_any(context, evidence_markers),
|
||||
"context": context,
|
||||
"message": "Forbidden phrase matched configured policy.",
|
||||
}
|
||||
report["phrase_level_findings"].append(record)
|
||||
report["blocking_errors"].append(record["message"])
|
||||
|
||||
|
||||
def empty_report(scan_root: pathlib.Path, config_path: pathlib.Path) -> dict[str, Any]:
|
||||
return {
|
||||
"scan_root": str(scan_root),
|
||||
"config_path": str(config_path),
|
||||
"files_scanned": [],
|
||||
"possible_overclaims": [],
|
||||
"field_level_findings": [],
|
||||
"phrase_level_findings": [],
|
||||
"missing_evidence_markers": [],
|
||||
"warnings": [],
|
||||
"blocking_errors": [],
|
||||
"observations": [],
|
||||
"machine_readable_summary": {},
|
||||
}
|
||||
|
||||
|
||||
def finalize_report(report: dict[str, Any]) -> None:
|
||||
report["possible_overclaims"] = [
|
||||
*report["field_level_findings"],
|
||||
*report["phrase_level_findings"],
|
||||
]
|
||||
report["machine_readable_summary"] = {
|
||||
"status": "FAIL" if report["blocking_errors"] else "PASS",
|
||||
"files_scanned_count": len(report["files_scanned"]),
|
||||
"field_finding_count": len(report["field_level_findings"]),
|
||||
"phrase_finding_count": len(report["phrase_level_findings"]),
|
||||
"missing_evidence_count": len(report["missing_evidence_markers"]),
|
||||
"warning_count": len(report["warnings"]),
|
||||
"blocking_count": len(report["blocking_errors"]),
|
||||
"observation_count": len(report["observations"]),
|
||||
}
|
||||
|
||||
|
||||
def write_markdown_report(report: dict[str, Any], output_path: pathlib.Path) -> None:
|
||||
summary = report["machine_readable_summary"]
|
||||
lines = [
|
||||
"# Lifecycle Status Guard Scan",
|
||||
"",
|
||||
"## Summary",
|
||||
"",
|
||||
f"- Status: `{summary['status']}`",
|
||||
f"- Files scanned: {summary['files_scanned_count']}",
|
||||
f"- Blocking findings: {summary['blocking_count']}",
|
||||
f"- Warnings: {summary['warning_count']}",
|
||||
f"- Evidence-present observations: {summary['observation_count']}",
|
||||
"",
|
||||
"## Blocking Findings",
|
||||
"",
|
||||
]
|
||||
if report["possible_overclaims"]:
|
||||
for finding in report["possible_overclaims"]:
|
||||
location = finding.get("path") or f"line {finding.get('line')}"
|
||||
claim = finding.get("value") or finding.get("phrase")
|
||||
lines.append(f"- `{finding['file']}` {location}: `{claim}` - {finding['message']}")
|
||||
else:
|
||||
lines.append("- None")
|
||||
lines.extend(["", "## Warnings", ""])
|
||||
if report["warnings"]:
|
||||
for warning in report["warnings"]:
|
||||
lines.append(
|
||||
f"- `{warning['file']}` {warning.get('path', '')}: `{warning.get('value', '')}` - {warning['message']}"
|
||||
)
|
||||
else:
|
||||
lines.append("- None")
|
||||
lines.extend(["", "## Observations", ""])
|
||||
if report["observations"]:
|
||||
for observation in report["observations"]:
|
||||
location = observation.get("path") or f"line {observation.get('line')}"
|
||||
claim = observation.get("value") or observation.get("phrase")
|
||||
lines.append(
|
||||
f"- `{observation['file']}` {location}: `{claim}` - evidence marker present; approval validity not decided."
|
||||
)
|
||||
else:
|
||||
lines.append("- None")
|
||||
lines.extend(["", "## Files Scanned", ""])
|
||||
for file_name in report["files_scanned"]:
|
||||
lines.append(f"- `{file_name}`")
|
||||
output_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
|
||||
def run(scan_root: pathlib.Path, config_path: pathlib.Path, output_dir: pathlib.Path) -> int:
|
||||
scan_root = scan_root.resolve()
|
||||
config_path = config_path.resolve()
|
||||
if not scan_root.is_dir():
|
||||
raise ValueError(f"Scan root is not a directory: {scan_root}")
|
||||
if not config_path.is_file():
|
||||
raise ValueError(f"Config file does not exist: {config_path}")
|
||||
|
||||
config = load_config(config_path)
|
||||
report = empty_report(scan_root, config_path)
|
||||
|
||||
for path in discover_files(scan_root, config["watched_paths"]):
|
||||
file_rel = rel_path(path, scan_root)
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
report["files_scanned"].append(file_rel)
|
||||
structured = parse_structured_content(path, text)
|
||||
if isinstance(structured, (Mapping, list)):
|
||||
scan_structured(structured, file_rel, config, report)
|
||||
scan_phrases(text, file_rel, config, report)
|
||||
|
||||
finalize_report(report)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
json_path = output_dir / f"{REPORT_BASENAME}.json"
|
||||
md_path = output_dir / f"{REPORT_BASENAME}.md"
|
||||
json_path.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
write_markdown_report(report, md_path)
|
||||
return 1 if report["blocking_errors"] else 0
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description="Scan configured lifecycle/status claims.")
|
||||
parser.add_argument("--scan-root", required=True, type=pathlib.Path)
|
||||
parser.add_argument("--config", required=True, type=pathlib.Path)
|
||||
parser.add_argument("--output-dir", required=True, type=pathlib.Path)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
try:
|
||||
return run(args.scan_root, args.config, args.output_dir)
|
||||
except Exception as exc:
|
||||
print(f"ERROR: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -0,0 +1,265 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import pathlib
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
|
||||
SKILL_ROOT = pathlib.Path(__file__).resolve().parents[1]
|
||||
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
|
||||
SCRIPT = SKILL_ROOT / "scripts" / "lifecycle_status_guard_scan.py"
|
||||
TMP_ROOT = REPO_ROOT / "tmp" / "lifecycle-status-guard-scan-tests"
|
||||
|
||||
|
||||
def run_cli(*args: str) -> subprocess.CompletedProcess[str]:
|
||||
return subprocess.run(
|
||||
[sys.executable, str(SCRIPT), *args],
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def write_text(path: pathlib.Path, text: str) -> pathlib.Path:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(text, encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def write_config(path: pathlib.Path, config: dict[str, object]) -> pathlib.Path:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(config, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def load_report(output_dir: pathlib.Path) -> dict[str, object]:
|
||||
return json.loads((output_dir / "lifecycle-status-guard-scan.json").read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
class LifecycleStatusGuardScanTest(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
TMP_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
self.tmp_dir = tempfile.TemporaryDirectory(dir=TMP_ROOT)
|
||||
self.root = pathlib.Path(self.tmp_dir.name) / "scan-root"
|
||||
self.output_dir = pathlib.Path(self.tmp_dir.name) / "output"
|
||||
self.config_path = pathlib.Path(self.tmp_dir.name) / "config.json"
|
||||
self.root.mkdir()
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.tmp_dir.cleanup()
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls) -> None:
|
||||
if TMP_ROOT.exists():
|
||||
shutil.rmtree(TMP_ROOT)
|
||||
|
||||
def run_scan(self, config: dict[str, object]) -> tuple[subprocess.CompletedProcess[str], dict[str, object]]:
|
||||
write_config(self.config_path, config)
|
||||
result = run_cli(
|
||||
"--scan-root",
|
||||
str(self.root),
|
||||
"--config",
|
||||
str(self.config_path),
|
||||
"--output-dir",
|
||||
str(self.output_dir),
|
||||
)
|
||||
return result, load_report(self.output_dir)
|
||||
|
||||
def test_markdown_front_matter_stable_without_evidence_is_blocking(self) -> None:
|
||||
write_text(
|
||||
self.root / "model-card.md",
|
||||
"---\nstatus: stable\nname: QPI\n---\n\nEngineering checks passed.\n",
|
||||
)
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.md"],
|
||||
"status_fields": ["status"],
|
||||
"forbidden_status_values": ["stable", "accepted"],
|
||||
"required_evidence_markers": ["owner_decision", "ccra_review"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertEqual(report["machine_readable_summary"]["blocking_count"], 1)
|
||||
finding = report["field_level_findings"][0]
|
||||
self.assertEqual(finding["field"], "status")
|
||||
self.assertEqual(finding["value"], "stable")
|
||||
self.assertFalse(finding["evidence_present"])
|
||||
|
||||
def test_draft_status_is_allowed(self) -> None:
|
||||
write_text(self.root / "model-card.md", "---\nstatus: draft\n---\n\nStill in draft.\n")
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.md"],
|
||||
"status_fields": ["status"],
|
||||
"forbidden_status_values": ["stable", "accepted"],
|
||||
"warning_status_values": ["candidate"],
|
||||
"required_evidence_markers": ["owner_decision"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(report["field_level_findings"], [])
|
||||
self.assertEqual(report["blocking_errors"], [])
|
||||
|
||||
def test_owner_approved_phrase_without_nearby_evidence_is_blocking(self) -> None:
|
||||
write_text(self.root / "handoff.md", "The model is owner-approved for stable use.\n")
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.md"],
|
||||
"approved_phrases": ["owner-approved"],
|
||||
"required_evidence_markers": ["owner decision", "owner_decision"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
finding = report["phrase_level_findings"][0]
|
||||
self.assertEqual(finding["phrase"], "owner-approved")
|
||||
self.assertEqual(finding["severity"], "blocking")
|
||||
self.assertFalse(finding["evidence_present"])
|
||||
|
||||
def test_owner_approved_phrase_with_nearby_evidence_is_observation_only(self) -> None:
|
||||
write_text(
|
||||
self.root / "handoff.md",
|
||||
"The model is owner-approved. Owner decision: accepted in review note 2026-06-19.\n",
|
||||
)
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.md"],
|
||||
"approved_phrases": ["owner-approved"],
|
||||
"required_evidence_markers": ["owner decision"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(report["blocking_errors"], [])
|
||||
self.assertEqual(report["observations"][0]["kind"], "evidence_present_claim")
|
||||
|
||||
def test_ccra_approved_phrase_without_review_reference_is_blocking(self) -> None:
|
||||
write_text(self.root / "review.md", "Local output says ccra-approved and final.\n")
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.md"],
|
||||
"approved_phrases": ["ccra-approved"],
|
||||
"required_evidence_markers": ["ccra review", "web ccra"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertEqual(report["phrase_level_findings"][0]["phrase"], "ccra-approved")
|
||||
|
||||
def test_json_status_evidence_must_be_in_same_object(self) -> None:
|
||||
write_text(
|
||||
self.root / "models.json",
|
||||
json.dumps(
|
||||
{
|
||||
"models": [
|
||||
{"name": "qpi", "status": "stable"},
|
||||
{"name": "other", "owner_decision": "approved elsewhere"},
|
||||
]
|
||||
},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
)
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.json"],
|
||||
"status_fields": ["status"],
|
||||
"forbidden_status_values": ["stable"],
|
||||
"required_evidence_markers": ["owner_decision"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
finding = report["field_level_findings"][0]
|
||||
self.assertEqual(finding["source"], "structured")
|
||||
self.assertEqual(finding["path"], "models[0].status")
|
||||
self.assertFalse(finding["evidence_present"])
|
||||
|
||||
def test_json_status_with_evidence_in_same_object_is_observation_only(self) -> None:
|
||||
write_text(
|
||||
self.root / "models.json",
|
||||
json.dumps(
|
||||
{"model": {"status": "stable", "owner_decision": "owner accepted"}},
|
||||
ensure_ascii=False,
|
||||
),
|
||||
)
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.json"],
|
||||
"status_fields": ["status"],
|
||||
"forbidden_status_values": ["stable"],
|
||||
"required_evidence_markers": ["owner_decision"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(report["blocking_errors"], [])
|
||||
self.assertEqual(report["observations"][0]["kind"], "field_evidence_present")
|
||||
|
||||
def test_allowed_context_exempts_phrase_but_not_structured_status_field(self) -> None:
|
||||
write_text(
|
||||
self.root / "policy.md",
|
||||
"---\nstatus: stable\n---\n\nPolicy: do not say owner-approved without a decision.\n",
|
||||
)
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.md"],
|
||||
"status_fields": ["status"],
|
||||
"forbidden_status_values": ["stable"],
|
||||
"approved_phrases": ["owner-approved"],
|
||||
"allowed_contexts": ["do not say"],
|
||||
"required_evidence_markers": ["owner decision"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
self.assertEqual(len(report["field_level_findings"]), 1)
|
||||
self.assertEqual(report["phrase_level_findings"], [])
|
||||
|
||||
def test_chinese_approval_phrase_and_utf8_filename(self) -> None:
|
||||
write_text(self.root / "模型状态.md", "本轮已经获得所有者批准,可以进入稳定状态。\n")
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.md"],
|
||||
"approved_phrases": ["所有者批准"],
|
||||
"required_evidence_markers": ["所有者决定记录"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 1, result.stderr)
|
||||
finding = report["phrase_level_findings"][0]
|
||||
self.assertEqual(finding["phrase"], "所有者批准")
|
||||
self.assertIn("模型状态.md", finding["file"])
|
||||
|
||||
def test_warning_status_value_does_not_fail_scan(self) -> None:
|
||||
write_text(self.root / "model.yml", "lifecycle: candidate\n")
|
||||
|
||||
result, report = self.run_scan(
|
||||
{
|
||||
"watched_paths": ["**/*.yml"],
|
||||
"status_fields": ["lifecycle"],
|
||||
"warning_status_values": ["candidate"],
|
||||
}
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(report["warnings"][0]["severity"], "warning")
|
||||
self.assertEqual(report["blocking_errors"], [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Loading…
Reference in New Issue