the-mindscape-of-bro-tsong/local_ccra_reviews/round-04/pilot-02/next-review-requirements.md

301 lines
16 KiB
Markdown

# Local CCRA Next Review Requirements
## 1. Return Metadata
- project_id: `the-mindscape-of-bro-tsong`
- review_round: `round-04`
- local_pass: `pilot-02`
- turn_type: `planning_turn`
- runtime_id: `ccra-local-review`
- runtime_version: `0.1.1`
- agent_id: `ccra-local-reviewer`
- agent_version: `0.1.1`
- invocation_mode: `prompt_path_reference`
- carrier_type: `codex_thread`
- child_session_id: `019edcde-24be-7593-99b2-5cf014467340`
- same_child_session_as_review_turn: `yes`
- first_turn_report_path: `local_ccra_reviews/round-04/pilot-02/04_LOCAL_CCRA_REVIEW_REPORT.md`
- owner_decision_path: `local_ccra_reviews/round-04/pilot-02/owner-decision.md`
- planning_output_path: `local_ccra_reviews/round-04/pilot-02/next-review-requirements.md`
- generated_at: `2026-06-19`
## 2. Planning Scope
- next_review_goal: define the submitted materials and verification evidence required for a Round 04.1 post-patch routing verification / regression pass.
- target_round_or_pass: `round-04.1_selector-rule-patch_verification`
- planning_basis:
- `local_ccra_reviews/round-04/pilot-02/04_LOCAL_CCRA_REVIEW_REPORT.md`
- `local_ccra_reviews/round-04/pilot-02/owner-decision.md`
- `local_ccra_reviews/round-04/pilot-02/agent-invocation-packet.md`
- explicit_non_goals:
- Do not re-review the full initial Round 04 bundle in this planning turn.
- Do not use this planning output as Owner approval.
- Do not treat Round 04.1 verification as Round 05 acceptance.
- Do not promote QPI or Intellectual Archaeology lifecycle status.
- Do not modify selector rules, tests, knowledge assets, model files, or review bundles in this turn.
Owner decision status is still `pending`. This plan therefore defines required review materials and recommended defaults; it does not convert the first-turn findings into accepted work.
## 3. Required Submitted Materials
```text
- material_id: M-01-review-brief
required_path_or_bundle_slot: ccra_review_bundle/<round-04.1>/00_OPEN_THIS_FIRST_*.md
purpose: state the review goal, target failures, non-goals, and authority boundary.
required: yes
notes: Must say this is a post-patch verification pass, not a second blind test and not lifecycle promotion.
- material_id: M-02-rule-patch-summary-and-diff
required_path_or_bundle_slot: ccra_review_bundle/<round-04.1>/01_RULE_PATCH_SUMMARY_AND_DIFF_*.md
purpose: show exactly what selector rule surfaces changed and why.
required: yes
notes: Must map changes to R04-BI-002, R04-BI-024, R04-BI-035, R04-BI-036, and any accepted same-class case such as R04-BI-022.
- material_id: M-03-expected-routing-labels-or-owner-decisions
required_path_or_bundle_slot: ccra_review_bundle/<round-04.1>/02_EXPECTED_ROUTING_LABELS_*.md or owner-decision-backed equivalent
purpose: prevent Web CCRA from inferring target labels from prose.
required: yes
notes: Must separate must-fix, same-class boundary, calibration-only, and explicitly-not-patched cases.
- material_id: M-04-post-patch-routing-verification-report
required_path_or_bundle_slot: reports/Round04_1_post_patch_routing_verification_report_*.md and bundle copy
purpose: provide full frozen-pool rerun after the patch.
required: yes
notes: Must include all 38 Round 04 inputs, not only target cases.
- material_id: M-05-before-after-routing-diff
required_path_or_bundle_slot: helper-outputs/routing-behavior-diff.md or a report section with equivalent table
purpose: show target changes, non-target changes, accepted collateral behavior, and regressions.
required: yes
notes: If the helper cannot run, include a manual before/after table and explain the missing helper risk.
- material_id: M-06-regression-and-command-log
required_path_or_bundle_slot: ccra_review_bundle/<round-04.1>/04_REGRESSION_AND_COMMAND_LOG_*.md
purpose: prove executed tests and validation gates, not dry-run declarations.
required: yes
notes: Include exact commands, exit codes, timestamps or run context, and changed-file impact.
- material_id: M-07-existing-selector-regression
required_path_or_bundle_slot: reports/selector_regression_report*.md or command log section
purpose: prove existing no-call, model-exclusion, QPI-before-IA, and depth-limiting protections still pass.
required: yes
notes: Must not rely only on targeted target-case checks.
- material_id: M-08-calibration-smoke
required_path_or_bundle_slot: reports/selector_calibration_smoke_report*.md or command log section
purpose: prove direct-execution, QPI, and IA boundaries remain sane after patch.
required: yes
notes: Especially watch translation/direct transformation and IA trigger over-expansion.
- material_id: M-09-bundle-manifest
required_path_or_bundle_slot: ccra_review_bundle/<round-04.1>/BUNDLE_FILE_MANIFEST_*.md
purpose: let reviewers inspect package completeness without relying on upload order.
required: yes
notes: Use versioned filenames to avoid Web ChatGPT file confusion.
- material_id: M-10-optional-raw-changed-files
required_path_or_bundle_slot: ccra_review_bundle/<round-04.1>/optional_raw_changed_files_<round>.zip
purpose: allow exact inspection of selector, tests, scripts, reports, and relevant docs.
required: yes
notes: Preserve source-relative paths. Include zip validation/readback summary.
- material_id: M-11-local-ccra-first-turn-report
required_path_or_bundle_slot: local_ccra_reviews/round-04/pilot-02/04_LOCAL_CCRA_REVIEW_REPORT.md
purpose: optional context for Web CCRA on why these verification requirements exist.
required: no
notes: Include in Web upload only if Owner explicitly asks Web CCRA to review Local CCRA findings.
- material_id: M-12-owner-decision-record
required_path_or_bundle_slot: local_ccra_reviews/round-04/pilot-02/owner-decision.md or round-specific decision note
purpose: show which findings and same-class boundaries Owner accepted, rejected, or deferred.
required: yes before final acceptance claims
notes: Current status is pending; absence blocks acceptance language, not evidence collection.
```
## 4. Required Verification Evidence
```text
targeted_regression:
required: yes
cases: R04-BI-002, R04-BI-024, R04-BI-035, R04-BI-036
conditional_cases: R04-BI-022 if Owner accepts it as same-class QPI-limited intent
expected: target failures change in the intended direction only.
full_frozen_pool_rerun_or_before_after_comparison:
required: yes
scope: all 38 Round 04 inputs
expected: selected_models, rejected_models, no_call, score, reasons, penalties, and routing_notes before/after.
existing_regression_or_unit_tests:
required: yes
minimum: round04 runner tests, selector regression, any post-patch verification tests, and touched selector helper tests.
expected: executed, not dry-run.
calibration_smoke:
required: yes
expected: no-call, direct execution, QPI problem-definition, IA refusal, QPI-before-IA, and depth-limiting smoke coverage.
validation_report:
required: yes
minimum: index check and model library validation if package claims model-library consistency.
expected: no lifecycle upgrade from validation pass.
command_log:
required: yes
minimum: command, working directory, exit code, output file path, and whether files changed.
expected: dry-run must be labeled dry-run and cannot support pass claims.
dry_run_acceptable: no for final verification; yes only for planning/preflight manifest readability.
```
## 5. Expected Before / After Comparisons
```text
- comparison_id: C-01-target-fixes
before_source: reports/Round04_blind_routing_evaluation_report_2026-06-18.md
after_source: reports/Round04_1_post_patch_routing_verification_report_<date>.md
expected_fields: input_id, input_text, selected_models, rejected_models, no_call, score, reasons, penalties, routing_notes
target_cases: R04-BI-002, R04-BI-024, R04-BI-035, R04-BI-036
non_target_cases_to_watch: R04-BI-001, R04-BI-003, R04-BI-004, R04-BI-005, R04-BI-021, R04-BI-025, R04-BI-031, R04-BI-032, R04-BI-033, R04-BI-034
- comparison_id: C-02-same-class-boundary
before_source: reports/Round04_blind_routing_evaluation_report_2026-06-18.md
after_source: post-patch frozen-pool rerun
expected_fields: no_call, QPI selected/rejected, IA selected/rejected, exclusion reason
target_cases: R04-BI-022 if accepted into patch scope
non_target_cases_to_watch: all depth-limiting and IA-refusal cases, especially R04-BI-021 and R04-BI-023
- comparison_id: C-03-direct-execution-collateral
before_source: initial Round 04 report plus existing selector regression
after_source: post-patch report plus regression output
expected_fields: no_call status and reasons for fact lookup, translation, rewrite, formatting, direct execution, how-to, dictionary judgment
target_cases: R04-BI-002
non_target_cases_to_watch: R04-BI-001, R04-BI-003, R04-BI-004, R04-BI-005, R04-BI-031, R04-BI-032, R04-BI-033, R04-BI-034
- comparison_id: C-04-calibration-not-patched
before_source: first-turn report section 12
after_source: post-patch report calibration section
expected_fields: unchanged/deferred status, reason, Owner/Web question
target_cases: none
non_target_cases_to_watch: R04-BI-010 through R04-BI-017, R04-BI-018, R04-BI-026, R04-BI-027, R04-BI-030, R04-BI-037, R04-BI-038
```
## 6. Open Owner Decisions
```text
- decision_id: OD-R04-001
decision_needed: accept the four must-fix initial Round 04 target failures as the patch target.
options: accept_all | accept_subset | request_web_baseline_comparison_first
recommended_default: accept_all
blocks_next_review: no for evidence collection; yes for final acceptance language.
- decision_id: OD-R04-002
decision_needed: classify R04-BI-022 as must-fix same-class, calibration-only, or acceptable conservative no-call.
options: include_in_patch | calibration_only | leave_no_call
recommended_default: include_in_patch
blocks_next_review: yes if the next package claims R04-BI-022 as accepted collateral or target behavior.
- decision_id: OD-R04-003
decision_needed: define policy for descriptive governance/capacity statements without explicit requests.
options: provisional_qpi | ask_clarifying_question | no_call_until_request
recommended_default: ask_clarifying_question or provisional_qpi with low confidence.
blocks_next_review: no for narrow patch verification; yes for broader selector maturity claims.
- decision_id: OD-R04-004
decision_needed: define direct IA versus QPI-first route for natural model-extraction/deep-mechanism inputs.
options: direct_ia_when_depth_intent_clear | qpi_first_always | qpi_plus_ia_when_both_signals_present
recommended_default: qpi_plus_ia_when both clear depth intent and model-extraction intent appear.
blocks_next_review: no for narrow patch verification; yes for IA route maturity claims.
- decision_id: OD-R04-005
decision_needed: decide whether helper profile tuning is required before formal Round 05 Local CCRA use.
options: tune_now | document_limitations | defer
recommended_default: document_limitations for this pilot, tune before repeated formal use.
blocks_next_review: no, if limitations are explicit.
```
## 7. Explicit Non-Goals
```text
- non_goal: no lifecycle status upgrade
reason: QPI and Intellectual Archaeology remain draft / B / pending until Owner / CCRA lifecycle authority changes that.
- non_goal: no Round 05 acceptance claim
reason: Round 04.1 verification only proves a scoped selector patch if evidence passes.
- non_goal: no second blind test claim
reason: after expected labels and target cases exist, this is a post-patch verification pass, not blind evaluation.
- non_goal: no broad governance/capacity keyword patch
reason: those cases are calibration-only until Owner/Web policy is explicit.
- non_goal: no IA trigger broadening without calibration
reason: IA is a deep model and should not become default because of words like structure, mechanism, long-term, or model.
- non_goal: no LLM selector, RAG, database, frontend, backend, user system, or third model
reason: outside model_library_mvp selector patch scope.
- non_goal: no helper exit-code absolutism
reason: helper output is evidence only; profile mismatch, dry-run, and broad scans must be judged by the reviewer.
```
## 8. Recommended Web CCRA Review Questions
1. Do the Round 04.1 submitted materials prove the four target cases were fixed without creating non-target regressions across the full frozen 38-case pool?
2. Should `R04-BI-022` be treated as an accepted same-class QPI-limited override, calibration-only, or conservative no-call?
3. Are governance/capacity descriptive cases (`R04-BI-010` through `R04-BI-017`) better handled as provisional QPI, clarification, or no-call until the user asks for analysis?
4. Should natural model-extraction/deep-mechanism cases (`R04-BI-018`, `026`, `027`, `038`) directly select IA, select QPI first, or select both when depth intent is explicit?
5. Is the verification package sufficient to avoid a further small patch round, or should remaining boundary issues be deferred into a broader calibration round?
6. Does any wording in the package imply lifecycle promotion, Round 05 acceptance, or Owner/Web approval beyond the evidence?
## 9. Material Package Checklist
```text
review_brief:
required: yes
pass_condition: states target scope, non-goals, and post-patch verification identity.
rule_or_model_diff:
required: yes
pass_condition: selector rule changes mapped to target cases; no model content/lifecycle changes unless separately justified.
expected_labels_or_decisions:
required: yes
pass_condition: target, same-class, calibration-only, and explicitly-not-patched cases are separated.
post_patch_verification_report:
required: yes
pass_condition: all 38 frozen inputs rerun with score/reason details.
regression_and_command_log:
required: yes
pass_condition: commands executed with exit codes; no dry-run used as pass evidence.
bundle_manifest:
required: yes
pass_condition: versioned manifest names recognized by human reviewer even if generic helper profile is noisy.
optional_raw_changed_files:
required: yes
pass_condition: source-relative zip entries and readback/validation summary included.
local_ccra_report:
required: optional
pass_condition: included only with Owner approval; otherwise referenced locally, not uploaded as formal Web evidence.
owner_decision_record:
required: yes before acceptance claims
pass_condition: records accepted, rejected, deferred, or escalated findings; current `pending` is not acceptance.
```
## 10. Limits And Carry-Forward Risks
- Owner decision remains pending. The next package may gather evidence, but it should not claim accepted fixes without a decision record.
- The first-turn report identified clear `fix_now` targets but did not inspect any post-patch evidence. The next review must not cite this plan as proof that the patch worked.
- `routing-behavior-diff-audit` was skipped in review_turn for good reason, but before/after routing comparison is required for the post-patch pass.
- `regression-validation-gate-runner` was dry-run only. The next package needs executed gates if it claims verification.
- `review-bundle-audit` used a generic profile that did not recognize versioned bundle manifests. Human-readable bundle manifest evidence is still required, and helper profile limitations should be documented.
- `lifecycle-status-guard-scan` was broad and noisy. The next review should include a lifecycle/scope guard section focused on the submitted package, not a whole-repository false-positive dump.
- Calibration-only cases must stay explicitly out of the narrow patch acceptance claim. They can be listed as next-round policy questions, not evidence that the patch is incomplete.
- Current project docs may contain later Round 04.1 language. The next review should clearly distinguish source evidence, patch evidence, closeout language, and lifecycle claims.