109 lines
5.0 KiB
Markdown
109 lines
5.0 KiB
Markdown
# Round 04.1 Selector Rule Patch Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Fix Round 04 selector boundary failures without changing model lifecycle status or introducing an LLM selector.
|
|
|
|
**Architecture:** Add targeted regression cases from Round 04 review, patch rule-based selector configuration and minimal selector logic, rerun the frozen Round 04 input pool as post-patch verification, and package 04.1 review materials. The original Round 04 blind output remains frozen; 04.1 produces comparison reports.
|
|
|
|
**Tech Stack:** JSON, Markdown, Python standard library, existing selector scripts, installed `bundle-zip` Skill.
|
|
|
|
## Global Constraints
|
|
|
|
- Do not change the original Round 04 blind routing report.
|
|
- Do not introduce an LLM selector.
|
|
- Do not add a third model.
|
|
- Do not upgrade QPI or Intellectual Archaeology.
|
|
- Do not implement RAG, vector database, frontend, backend, user account, or QA product.
|
|
- Treat Round 04.1 as post-patch routing verification, not a second blind test.
|
|
- Use `bundle-zip` for source-relative review bundle zips.
|
|
|
|
---
|
|
|
|
### Task 1: Add Targeted Regression Cases
|
|
|
|
**Files:**
|
|
- Modify: `tests/qpi.regression.json`
|
|
- Modify: `tests/intellectual_archaeology.regression.json`
|
|
- Modify: `tests/regression_cases.json`
|
|
|
|
**Interfaces:**
|
|
- Consumes: Round 04 review cases R04-BI-002, R04-BI-024, R04-BI-035, R04-BI-036.
|
|
- Produces: selector regression failures before patch.
|
|
|
|
- [ ] Add R04-BI-002 as a QPI no-call regression.
|
|
- [ ] Add R04-BI-024 as an IA hard-exclusion regression with QPI allowed.
|
|
- [ ] Add R04-BI-035 and R04-BI-036 as depth-limiting QPI override regressions.
|
|
- [ ] Run `python scripts\run_selector_regression.py` and confirm expected failures.
|
|
|
|
### Task 2: Patch Selector Rules and Minimal Logic
|
|
|
|
**Files:**
|
|
- Modify: `selector/selector_rules.json`
|
|
- Modify if needed: `scripts/run_selector_demo.py`
|
|
- Modify: `selector/README.md`
|
|
- Modify: `docs/DECISIONS.md`
|
|
|
|
**Interfaces:**
|
|
- Consumes: failing regression cases from Task 1.
|
|
- Produces: selector behavior that passes targeted regressions.
|
|
|
|
- [ ] Add translation no-call intent phrases beyond one exact wording.
|
|
- [ ] Add instruction / payload separation for direct transformation tasks before scoring payload complexity.
|
|
- [ ] Add IA hard-exclusion variants such as `不要进入思想考古`.
|
|
- [ ] Add depth-limiting QPI override phrases for `主导稀缺` and `缺信息 / 缺路径 / 缺共识` forms.
|
|
- [ ] Document the rule patch.
|
|
|
|
### Task 3: Post-Patch Routing Comparison
|
|
|
|
**Files:**
|
|
- Create: `scripts/run_round04_post_patch_verification.py`
|
|
- Create: `reports/Round04_1_post_patch_routing_verification_report_2026-06-18.md`
|
|
- Test: `tests/test_round04_blind_routing.py` or new focused test if needed.
|
|
|
|
**Interfaces:**
|
|
- Consumes: frozen `selector/round04_blind_inputs.json` and frozen Round 04 report.
|
|
- Produces: before / after comparison for targeted expected routing labels.
|
|
|
|
- [ ] Generate current post-patch results for all 38 frozen inputs.
|
|
- [ ] Compare targeted cases R04-BI-002, R04-BI-024, R04-BI-035, R04-BI-036 against expected labels.
|
|
- [ ] Include original selected models / no-call from Round 04 and new selected models / no-call.
|
|
|
|
### Task 4: Build Round 04.1 Review Bundle
|
|
|
|
**Files:**
|
|
- Create: `ccra_review_bundle/round-04.1_2026-06-18_selector-rule-patch/00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_04.1.md`
|
|
- Create: `ccra_review_bundle/round-04.1_2026-06-18_selector-rule-patch/01_RULE_PATCH_SUMMARY_AND_DIFF_04.1.md`
|
|
- Create: `ccra_review_bundle/round-04.1_2026-06-18_selector-rule-patch/02_EXPECTED_ROUTING_LABELS_04.1.md`
|
|
- Create: `ccra_review_bundle/round-04.1_2026-06-18_selector-rule-patch/03_POST_PATCH_ROUTING_VERIFICATION_REPORT_04.1.md`
|
|
- Create: `ccra_review_bundle/round-04.1_2026-06-18_selector-rule-patch/04_REGRESSION_AND_COMMAND_LOG_04.1.md`
|
|
- Create: `ccra_review_bundle/round-04.1_2026-06-18_selector-rule-patch/BUNDLE_FILE_MANIFEST_04.1.md`
|
|
- Create: `ccra_review_bundle/round-04.1_2026-06-18_selector-rule-patch/optional_raw_changed_files_04.1.zip`
|
|
|
|
**Interfaces:**
|
|
- Consumes: patch diffs, post-patch report, validation output, source file list.
|
|
- Produces: GPT / CCRA review package.
|
|
|
|
- [ ] Write required review files.
|
|
- [ ] Use `bundle-zip` to create source-relative raw zip.
|
|
- [ ] Verify zip output using `bundle-zip` JSON summary.
|
|
|
|
### Task 5: Verify
|
|
|
|
**Files:**
|
|
- Read: full changed set.
|
|
|
|
**Interfaces:**
|
|
- Consumes: completed 04.1 patch and review bundle.
|
|
- Produces: verification evidence.
|
|
|
|
- [ ] Run `python scripts\rebuild_indexes.py --check`.
|
|
- [ ] Run `python -m unittest discover -s tests -p "test*.py" -v`.
|
|
- [ ] Run `python scripts\validate_model_library.py`.
|
|
- [ ] Run `python scripts\check_card_contract.py`.
|
|
- [ ] Run `python scripts\run_selector_demo.py`.
|
|
- [ ] Run `python scripts\run_selector_regression.py`.
|
|
- [ ] Run `python scripts\run_selector_calibration_smoke.py`.
|
|
- [ ] Run `python scripts\check_model_card_sync.py`.
|
|
- [ ] Confirm model JSON status remains `draft / B / pending`.
|