skills-vault/skills/routing-behavior-diff-audit/SKILL.md

name	description
routing-behavior-diff-audit	Use when Codex needs to compare before/after selector, routing, QPI, classifier, rules-engine, triage, assignment, or model-governance outputs to identify targeted route changes, collateral non-target changes, no-call changes, missing cases, and new or resolved expected-route mismatches before review, release, handoff, CCRA/local review, or regression closeout.

Routing Behavior Diff Audit

Purpose

Run a deterministic before/after behavior diff for rules-based routing outputs. Use the bundled script to compare case IDs, routes, optional expected routes, missing cases, no-call transitions, and targeted versus non-target changes.

This Skill reports observed differences only. It must not edit routing rules, change regression cases, update expected labels, infer product meaning, or decide whether a patch should close.

Command

conda run -n skills-vault python .\scripts\routing_behavior_diff_audit.py `
  --before-results C:\path\before-results `
  --after-results C:\path\after-results `
  --output-dir C:\path\review-run `
  --case-id-field case_id `
  --route-field route `
  --expected-route-field expected_route `
  --targeted-case CASE-001

When installed under C:\Users\wangq\.agents\skills\routing-behavior-diff-audit, run from that Skill directory or use the installed script path.

Inputs

• --before-results: JSON, JSONL, or Markdown file, or a directory containing those files.
• --after-results: JSON, JSONL, or Markdown file, or a directory containing those files.
• --output-dir: directory where reports are written.
• --case-id-field: field containing the stable case ID. Default: case_id.
• --route-field: field containing the routing decision. Default: route.
• --expected-route-field: optional field containing the expected route for failure comparison.
• --targeted-case: repeatable case ID that the patch intentionally targets.
• --targeted-cases-file: optional JSON/YAML list, or object with targeted_cases, case_ids, or cases.

JSON is the stable input target. Directory inputs are scanned recursively for .json, .jsonl, .md, and .markdown; Markdown support expects a fenced JSON or YAML block.

Outputs

The script writes under output_dir:

routing-behavior-diff.md
routing-behavior-diff.json

The JSON report includes:

• total_cases_compared
• unchanged_cases
• changed_cases
• targeted_changes
• non_target_changes
• new_failures
• resolved_failures
• missing_before_cases
• missing_after_cases
• diff_table
• machine_readable_summary

machine_readable_summary.status is PASS, ATTENTION, or FAIL for automation routing only. It is not a patch approval, owner approval, CCRA decision, or release decision.

Comparison Rules

• Compare only case IDs present in both before and after for total_cases_compared.
• Classify after-only IDs as missing_before_cases.
• Classify before-only IDs as missing_after_cases.
• Treat a common case as changed when the route, expected route, no-call status, or expected-route failure status changes.
• Classify changed cases listed by --targeted-case or --targeted-cases-file as targeted_changes.
• Classify all other changed cases as non_target_changes.
• Compute new_failures and resolved_failures only when --expected-route-field is supplied and present in the records.
• Treat empty, null-like, and common no-call spellings such as no_call, no-call, and no call as no-call values.

Safety

• Read only configured before/after result paths and optional targeted-cases file.
• Write only routing-behavior-diff.md and routing-behavior-diff.json under output_dir.
• Do not edit selectors, routing rules, regression cases, expected labels, review bundles, model cards, or governance artifacts.
• Do not decide acceptance, lifecycle status, patch closure, release readiness, or owner approval.

Validation

After changing diff behavior, run:

conda run -n skills-vault python -B -m unittest discover -s skills/routing-behavior-diff-audit/tests -v
conda run -n skills-vault python -B C:\Users\wangq\.codex\skills\.system\skill-creator\scripts\quick_validate.py skills/routing-behavior-diff-audit