# Round 04 Blind Routing Evaluation Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Prepare and run Round 04 blind input routing evaluation without changing selector rules or model lifecycle status.

**Architecture:** Round 04 uses a reviewed blind input pool as source material, freezes it into a machine-readable JSON input set, runs the existing rule-based selector through a thin batch runner, and packages the outputs for GPT / CCRA review. The runner reuses `scripts/run_selector_demo.py::recommend` and does not add answer generation, LLM selector logic, or expected routing labels.

**Tech Stack:** Markdown, JSON, Python standard library, existing selector scripts.

## Global Constraints

- Do not modify `selector/selector_rules.json` before the blind evaluation.
- Do not merge blind inputs into `selector/selector_calibration_inputs.json`.
- Do not add expected behavior before the first blind run.
- Do not add a third model.
- Do not upgrade QPI or Intellectual Archaeology.
- Do not introduce an LLM selector.
- Do not implement RAG, vector database, frontend, backend, user account, or QA product.
- Run first, analyze failures later.

---

### Task 1: Revise Human-Readable Blind Input Source

**Files:**
- Modify: `reports/Round04_blind_input_candidates_2026-06-18.md`

**Interfaces:**
- Consumes: owner review notes for R04-BI-018, R04-BI-021, R04-BI-025, R04-BI-026, and new R04-BI-031 through R04-BI-038.
- Produces: 38 candidate inputs with review-only metadata and no expected routing labels.

- [ ] **Step 1: Mark control-case notes**

Add metadata notes:
- R04-BI-021 and R04-BI-025: `model-name-exposed control case`
- R04-BI-018: `positive control`
- R04-BI-026: natural IA / QPI-to-IA boundary sample

- [ ] **Step 2: Add R04-BI-031 through R04-BI-038**

Add the eight owner-proposed cases exactly, preserving their categories, input text, and inclusion rationale.

- [ ] **Step 3: Update coverage**

Set total to 38 candidate inputs and include a short note that 2 inputs expose model names by design.

### Task 2: Freeze Machine-Readable Blind Input JSON

**Files:**
- Create: `selector/round04_blind_inputs.json`

**Interfaces:**
- Consumes: reviewed Markdown source from Task 1.
- Produces: JSON with `blind_input_set_id`, `status`, `source_document`, `constraints`, and `inputs`.

- [ ] **Step 1: Create JSON without expected routing**

Each input object contains:
- `input_id`
- `category_for_owner_review`
- `input_text`
- `why_included`
- optional `control_case_type`

- [ ] **Step 2: Preserve blindness constraints**

Include top-level notes that no expected behavior is present and that selector rules must not be modified before first run.

### Task 3: Add Failing Tests for the Round 04 Runner

**Files:**
- Create: `tests/test_round04_blind_routing.py`
- Create later in Task 4: `scripts/run_round04_blind_routing.py`

**Interfaces:**
- Consumes: `selector/round04_blind_inputs.json`
- Produces: test expectations for load, evaluation shape, and report content.

- [ ] **Step 1: Write tests before implementation**

Test that a blind input item can be loaded, evaluated through `recommend`, and rendered without expected routing labels.

- [ ] **Step 2: Run tests and confirm failure**

Run: `python -m unittest tests.test_round04_blind_routing -v`

Expected: failure because `scripts.run_round04_blind_routing` does not exist.

### Task 4: Implement and Run the Round 04 Runner

**Files:**
- Create: `scripts/run_round04_blind_routing.py`
- Create: `reports/Round04_blind_routing_evaluation_report_2026-06-18.md`

**Interfaces:**
- Consumes: `selector/round04_blind_inputs.json`, `scripts/run_selector_demo.py::recommend`
- Produces: JSON-shaped evaluation results and Markdown report.

- [ ] **Step 1: Implement standard-library runner**

Provide functions:
- `load_blind_inputs(root)`
- `evaluate_blind_input(root, item)`
- `write_report(root, input_set, results)`
- `main()`

- [ ] **Step 2: Run focused tests**

Run: `python -m unittest tests.test_round04_blind_routing -v`

- [ ] **Step 3: Run blind evaluation**

Run: `python scripts\run_round04_blind_routing.py`

Expected: report written; command exits 0.

### Task 5: Build Round 04 Review Bundle

**Files:**
- Create: `ccra_review_bundle/round-04_2026-06-18_blind-input-routing-evaluation/00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_04.md`
- Create: `ccra_review_bundle/round-04_2026-06-18_blind-input-routing-evaluation/01_ROUND03_CLOSURE_AND_ROUND04_SCOPE_04.md`
- Create: `ccra_review_bundle/round-04_2026-06-18_blind-input-routing-evaluation/02_BLIND_INPUT_SET_04.md`
- Create: `ccra_review_bundle/round-04_2026-06-18_blind-input-routing-evaluation/03_ROUTING_EVALUATION_REPORT_04.md`
- Create: `ccra_review_bundle/round-04_2026-06-18_blind-input-routing-evaluation/04_REVIEW_QUESTIONS_FOR_GPT_04.md`
- Create: `ccra_review_bundle/round-04_2026-06-18_blind-input-routing-evaluation/BUNDLE_FILE_MANIFEST_04.md`
- Create: `ccra_review_bundle/round-04_2026-06-18_blind-input-routing-evaluation/optional_raw_changed_files_04.zip`

**Interfaces:**
- Consumes: Round 03 closure summary, frozen blind input JSON, runner output report, current runtime assets.
- Produces: GPT / CCRA review package.

- [ ] **Step 1: Write brief and scope files**

Include Round 03 closure summary, Round 04 target, and non-goals.

- [ ] **Step 2: Write input and evaluation mirrors**

Mirror the frozen blind input set and routing evaluation report for upload.

- [ ] **Step 3: Create raw zip**

Include current runtime assets, blind input set, runner, tests, and reports. Preserve source-relative paths.

### Task 6: Verify

**Files:**
- Read: all changed files

**Interfaces:**
- Consumes: full working tree after Tasks 1-5.
- Produces: verification evidence.

- [ ] **Step 1: Run focused Round 04 test**

Run: `python -m unittest tests.test_round04_blind_routing -v`

- [ ] **Step 2: Run existing validation gates**

Run:
- `python scripts\rebuild_indexes.py --check`
- `python -m unittest discover -s tests -p "test*.py" -v`
- `python scripts\validate_model_library.py`
- `python scripts\check_card_contract.py`
- `python scripts\run_selector_demo.py`
- `python scripts\run_selector_regression.py`
- `python scripts\run_selector_calibration_smoke.py`
- `python scripts\check_model_card_sync.py`

- [ ] **Step 3: Inspect git diff**

Confirm no selector rule changes and no model lifecycle status changes.