Close Round 03 selector review
This commit is contained in:
parent
cd9f63e9eb
commit
77114cee5b
|
|
@ -174,8 +174,8 @@ Files intended for GPT / CCRA upload inside a review bundle should include the r
|
|||
|
||||
## 13. Next Steps
|
||||
|
||||
1. Submit the Round 03.2 selector negative-gate patch for GPT / CCRA review.
|
||||
1. Round 03 is closed after the Round 03.2a CCRA / GPT pass decision.
|
||||
2. Keep QPI and Intellectual Archaeology at `draft / B / pending` until Owner / CCRA review accepts stronger status.
|
||||
3. Run a QPI blind-test round with new unlabelled inputs only after Round 03.2 passes.
|
||||
3. Start Round 04 blind input routing evaluation only after an explicit owner instruction; do not modify selector rules before that evaluation.
|
||||
4. Defer any third-model expansion until the current QPI selector and case layer are accepted.
|
||||
5. Route missing reusable extraction, inspection, or stability-scoring tools to `requirements/skills-vault/` or `requirements/ccpe/` instead of improvising local platform features.
|
||||
|
|
|
|||
12
cards/qpi.md
12
cards/qpi.md
|
|
@ -145,6 +145,8 @@ QPI 必须区分：
|
|||
- 不要展开
|
||||
- 只改错别字
|
||||
|
||||
说明：`不要展开` / `不要深入分析` 默认抑制 QPI；但如果同时出现 `只做 QPI`、`只做问题定性`、`只判断主导稀缺` 等限定分析意图，则允许 QPI 做窄范围判断，并拒绝进入思想考古。
|
||||
|
||||
## 相关模型
|
||||
|
||||
- `intellectual_archaeology`: 当 QPI 判断问题偏中重型难题或课题时，可进入思想考古做深度建模。
|
||||
|
|
@ -202,7 +204,7 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
|||
- 机制稳定性：中高，扫描匮乏物和匹配处理范式的机制明确。
|
||||
- 边界清晰度：中等，混合型问题、多视角分歧和上下文不足场景仍需更多测试。
|
||||
- 来源证据质量：较高，主体性和动态性来自 2025 原文，核心匮乏物和误框定规则来自 2026 原文，应用规则来自综合文档。
|
||||
- 回归测试表现：pending，当前已有 55 条 QPI 回归用例；其中 Round 03 / 03.1 已纳入 owner-reviewed case layer、selector no-call、multi-perspective 和低上下文 provisional 边界用例，Round 03.2 已纳入普通 how-to / 词义判断 no-call 陷阱。生命周期仍保持 draft。
|
||||
- 回归测试表现：pending，当前已有 58 条 QPI 回归用例；其中 Round 03 / 03.1 已纳入 owner-reviewed case layer、selector no-call、multi-perspective 和低上下文 provisional 边界用例，Round 03.2 已纳入普通 how-to / 词义判断 no-call 陷阱，Round 03.2a 已纳入 depth-limiting QPI override 陷阱。生命周期仍保持 draft。
|
||||
|
||||
评级理由：三分结构清晰，适合作为入口路由模型，但需要补充大量边界案例，防止过度升维或降维。
|
||||
|
||||
|
|
@ -212,7 +214,7 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
|||
|
||||
`pending`
|
||||
|
||||
当前已有 55 条 QPI 回归用例，覆盖 positive、boundary、misuse、no_call、selector_gate 和 pipeline。Round 03 / 03.1 重点新增：
|
||||
当前已有 58 条 QPI 回归用例，覆盖 positive、boundary、misuse、no_call、selector_gate 和 pipeline。Round 03 / 03.1 重点新增：
|
||||
|
||||
- owner-reviewed flow / disappointment / organizational year-end-review cases；
|
||||
- `case_qpi_international_logistics_no_call_001`
|
||||
|
|
@ -228,6 +230,12 @@ Round 03.2 重点新增：
|
|||
- `case_qpi_how_to_markdown_pdf_no_call_001`
|
||||
- `case_qpi_dictionary_judgment_no_call_001`
|
||||
|
||||
Round 03.2a 重点新增：
|
||||
|
||||
- `case_qpi_depth_limited_qpi_only_001`
|
||||
- `case_qpi_depth_limited_problem_definition_001`
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`
|
||||
|
||||
## 示例输入
|
||||
|
||||
- “我们团队每次都说要长期主义，但一到季度 KPI 就回到短期动作，这到底是什么问题？”
|
||||
|
|
|
|||
|
|
@ -0,0 +1,52 @@
|
|||
# Round 03.2a CCRA Review Brief
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||
|
||||
Patch title: Depth-Limiting Override For QPI
|
||||
|
||||
## Scope
|
||||
|
||||
Round 03.2a is a micro-patch after GPT's Round 03.2 `REVISE-light` review.
|
||||
|
||||
It fixes one selector precedence issue:
|
||||
|
||||
> `不要展开` / `不要深入分析` should remain global no-call by default, but when paired with explicit QPI-limited intent such as `只做 QPI`, `只做问题定性`, or `只判断主导稀缺`, they should allow QPI and reject IA.
|
||||
|
||||
This patch does not:
|
||||
|
||||
- add a third model;
|
||||
- upgrade QPI or Intellectual Archaeology lifecycle status;
|
||||
- introduce an LLM selector;
|
||||
- introduce RAG, database, frontend, backend, user accounts, or platform features;
|
||||
- start Round 04 blind testing.
|
||||
|
||||
Current model status remains:
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
||||
## What Changed
|
||||
|
||||
1. Added `depth_limiting_no_call_signals`.
|
||||
2. Added `qpi_limited_analysis_override_phrases`.
|
||||
3. Modified hard no-call precedence so depth-limiting phrases do not globally no-call when QPI-limited intent is explicit.
|
||||
4. Suppressed QPI's own depth-limiting negative penalty only in that explicit QPI-limited combination.
|
||||
5. Expanded IA hard exclusion triggers to cover QPI-limited judgment phrases.
|
||||
6. Added 3 selector regression cases and 5 calibration smoke inputs.
|
||||
|
||||
## Suggested Review Order
|
||||
|
||||
1. `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_03.2a.md`
|
||||
2. `01_PATCH_MATRIX_03.2a.md`
|
||||
3. `02_CURRENT_ASSET_PACK_03.2a.md`
|
||||
4. `03_VALIDATION_AND_COMMAND_LOG_03.2a.md`
|
||||
5. `04_REVIEW_QUESTIONS_FOR_GPT_03.2a.md`
|
||||
6. `optional_raw_changed_files_03.2a.zip` only if exact changed-file inspection is needed
|
||||
|
||||
## Primary Review Question
|
||||
|
||||
Does Round 03.2a close the depth-limiting override issue enough to close Round 03 and proceed to Round 04 blind input evaluation?
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# Round 03.2a Patch Matrix
|
||||
|
||||
## GPT Requirement Mapping
|
||||
|
||||
| Requirement | Implementation | Files |
|
||||
| --- | --- | --- |
|
||||
| Add QPI-limited override phrases | Added `qpi_limited_analysis_override_phrases` | `selector/selector_rules.json` |
|
||||
| Keep `不要展开` / `不要深入分析` as no-call by default | Added separate `depth_limiting_no_call_signals`; did not remove original hard no-call phrases | `selector/selector_rules.json` |
|
||||
| Let QPI-limited depth requests continue scoring | Hard no-call branch now skips global no-call only when depth-limiting and QPI-limited phrases both hit | `scripts/run_selector_demo.py` |
|
||||
| Do not let QPI's own negative trigger penalty suppress the override | Suppressed QPI depth-limiting penalty only for explicit QPI-limited combinations | `scripts/run_selector_demo.py` |
|
||||
| IA must remain rejected | Added QPI-limited judgment phrases to IA hard exclusion triggers | `selector/selector_rules.json` |
|
||||
| Add regression / calibration cases | Added 3 regression cases and 5 calibration inputs | `tests/qpi.regression.json`, `selector/selector_calibration_inputs.json` |
|
||||
|
||||
## Added Regression Cases
|
||||
|
||||
- `case_qpi_depth_limited_qpi_only_001`
|
||||
- `case_qpi_depth_limited_problem_definition_001`
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`
|
||||
|
||||
## Added Calibration Inputs
|
||||
|
||||
- `selector_calibration_qpi_depth_limited_qpi_only_001`
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_001`
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_002`
|
||||
- `selector_calibration_qpi_depth_limited_problem_or_issue_001`
|
||||
- `selector_calibration_qpi_depth_limited_dominant_scarcity_001`
|
||||
|
||||
## Current Counts
|
||||
|
||||
| Asset | Round 03.2 | Round 03.2a |
|
||||
| --- | ---: | ---: |
|
||||
| QPI regression cases | 55 | 58 |
|
||||
| IA regression cases | 19 | 19 |
|
||||
| Aggregate regression cases | 74 | 77 |
|
||||
| Selector calibration inputs | 90 | 95 |
|
||||
|
||||
## Non-Goals Preserved
|
||||
|
||||
- No new model.
|
||||
- No lifecycle upgrade.
|
||||
- No LLM selector.
|
||||
- No RAG / database / frontend / backend.
|
||||
- No Round 04 blind-test execution.
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
# Round 03.2a Current Asset Pack
|
||||
|
||||
## Canonical Changed Assets
|
||||
|
||||
- `selector/selector_rules.json`
|
||||
- `selector/selector_calibration_inputs.json`
|
||||
- `tests/qpi.regression.json`
|
||||
- `tests/regression_cases.json`
|
||||
- `tests/regression_index.json`
|
||||
- `scripts/run_selector_demo.py`
|
||||
|
||||
## Generated / Derived Assets
|
||||
|
||||
- `models/model_index.json`
|
||||
- `cards/card_index.md`
|
||||
- `reports/index_rebuild_report.md`
|
||||
- `reports/validation_report.md`
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
- `reports/model_card_sync_report_v0.2.md`
|
||||
|
||||
## Documentation Updated
|
||||
|
||||
- `README.md`
|
||||
- `docs/DECISIONS.md`
|
||||
- `selector/README.md`
|
||||
- `cards/qpi.md`
|
||||
- `reports/content_review_report_v0.2.md`
|
||||
|
||||
## Required Behavior
|
||||
|
||||
The following inputs should now select QPI, reject IA, and set `no_call=false`:
|
||||
|
||||
```text
|
||||
不要展开到思想考古，只做 QPI 复核。
|
||||
不要展开到思想考古，只做问题定性。
|
||||
不要深入分析，只做问题定性。
|
||||
不要深入分析背后的理论，只判断这是问题还是课题。
|
||||
不要展开，只判断主导稀缺。
|
||||
```
|
||||
|
||||
Depth-limiting phrases without explicit QPI-limited intent remain no-call / low-depth signals.
|
||||
|
||||
## Current Lifecycle State
|
||||
|
||||
No lifecycle upgrade was made.
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
# Round 03.2a Validation and Command Log
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
## Red Evidence
|
||||
|
||||
After adding the 03.2a traps and before implementation, selector checks failed as expected.
|
||||
|
||||
`python scripts\run_selector_regression.py`
|
||||
|
||||
- `case_qpi_depth_limited_qpi_only_001`: expected primary QPI, got none.
|
||||
- `case_qpi_depth_limited_problem_definition_001`: expected primary QPI, got none.
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`: expected primary QPI, got none.
|
||||
|
||||
`python scripts\run_selector_calibration_smoke.py`
|
||||
|
||||
- five depth-limiting QPI override inputs failed with expected QPI selection missing.
|
||||
|
||||
## Green Verification
|
||||
|
||||
The following commands passed after the Round 03.2a patch:
|
||||
|
||||
```powershell
|
||||
python scripts\rebuild_indexes.py --check
|
||||
python -m unittest discover -s tests -p "test*.py" -v
|
||||
python scripts\validate_model_library.py
|
||||
python scripts\check_card_contract.py
|
||||
python scripts\run_selector_demo.py
|
||||
python scripts\run_selector_regression.py
|
||||
python scripts\run_selector_calibration_smoke.py
|
||||
python scripts\check_model_card_sync.py
|
||||
```
|
||||
|
||||
Results:
|
||||
|
||||
- Index check: PASS
|
||||
- Unit tests: PASS, 17 tests
|
||||
- Model library validation: PASS
|
||||
- Card contract: PASS
|
||||
- Selector demo: PASS
|
||||
- Selector regression: PASS
|
||||
- Selector calibration smoke: PASS, 95 inputs, 0 failures, 0 warnings
|
||||
- Model/card sync: PASS
|
||||
|
||||
## Important Interpretation
|
||||
|
||||
Validation pass does not mean model content is stable or owner-approved.
|
||||
|
||||
It only means the Round 03.2a selector precedence patch is structurally consistent and passes current local checks.
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# Round 03.2a Review Questions for GPT
|
||||
|
||||
Please review Round 03.2a only.
|
||||
|
||||
Do not expand into Round 04 blind testing yet.
|
||||
|
||||
## Questions
|
||||
|
||||
1. Does the selector now correctly treat `不要展开 / 不要深入分析 + 只做 QPI / 只做问题定性` as QPI-limited analysis rather than global no-call?
|
||||
2. Does IA remain properly rejected in all five depth-limiting override cases?
|
||||
3. Is it acceptable that depth-limiting phrases without QPI-limited intent still remain no-call / low-depth signals?
|
||||
4. Can Round 03 now be closed with the following conclusion?
|
||||
|
||||
```text
|
||||
QPI and Intellectual Archaeology remain draft / B / pending.
|
||||
QPI is accepted as a draft-callable routing asset at report level.
|
||||
Selector v0.2 passes no-call, calibration smoke, model-level exclusion, QPI-before-IA, and depth-limiting override gates.
|
||||
Proceed to Round 04 blind input evaluation.
|
||||
```
|
||||
|
||||
## Suggested Answer Format
|
||||
|
||||
Please answer with:
|
||||
|
||||
- `pass`, `revise`, or `block`;
|
||||
- specific failing cases if `revise` or `block`;
|
||||
- whether Round 04 blind input evaluation may start after this patch.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
# Round 03.2a Bundle File Manifest
|
||||
|
||||
## Review Files
|
||||
|
||||
- `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_03.2a.md`
|
||||
- `01_PATCH_MATRIX_03.2a.md`
|
||||
- `02_CURRENT_ASSET_PACK_03.2a.md`
|
||||
- `03_VALIDATION_AND_COMMAND_LOG_03.2a.md`
|
||||
- `04_REVIEW_QUESTIONS_FOR_GPT_03.2a.md`
|
||||
- `BUNDLE_FILE_MANIFEST_03.2a.md`
|
||||
- `optional_raw_changed_files_03.2a.zip`
|
||||
|
||||
## Optional Raw Changed Files
|
||||
|
||||
The zip is optional and should be used only if exact file inspection is needed.
|
||||
|
||||
It preserves source-relative paths.
|
||||
|
||||
It excludes:
|
||||
|
||||
- `knowledge_assets/`
|
||||
- `__pycache__/`
|
||||
- `*.pyc`
|
||||
- previous review bundle zip files
|
||||
|
||||
## Source-Relative Files Included in Raw Zip
|
||||
|
||||
- `README.md`
|
||||
- `cards/qpi.md`
|
||||
- `docs/DECISIONS.md`
|
||||
- `reports/content_review_report_v0.2.md`
|
||||
- `reports/Codex新会话交接文档_Round03.2a评审准备_2026-06-17.md`
|
||||
- `reports/index_rebuild_report.md`
|
||||
- `reports/model_card_sync_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/validation_report.md`
|
||||
- `scripts/run_selector_demo.py`
|
||||
- `selector/README.md`
|
||||
- `selector/selector_calibration_inputs.json`
|
||||
- `selector/selector_rules.json`
|
||||
- `tests/qpi.regression.json`
|
||||
- `tests/regression_cases.json`
|
||||
- `tests/regression_index.json`
|
||||
- `models/model_index.json`
|
||||
- `cards/card_index.md`
|
||||
Binary file not shown.
|
|
@ -181,3 +181,36 @@ The current Round 03.2 implementation keeps this scoped:
|
|||
- It does not add an LLM selector, RAG, database, service, frontend, backend, or third model.
|
||||
- It preserves QPI explicit-analysis override, but QPI selection does not re-enable IA when IA is explicitly refused.
|
||||
- Generic bare QPI gate words such as `如何` and `判断` are removed in favor of compound problem-definition phrases.
|
||||
|
||||
## Decision 015: Treat depth-limiting QPI requests as scoped analysis, not global no-call
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Reason:
|
||||
|
||||
Round 03.2 review found that global hard no-call phrases such as `不要展开` and `不要深入分析` could incorrectly block QPI when the user also asked for a limited QPI action such as `只做 QPI`, `只做问题定性`, or `只判断主导稀缺`.
|
||||
|
||||
The correct interpretation is scoped depth control:
|
||||
|
||||
- `不要展开` or `不要深入分析` alone remains a no-call / low-depth signal.
|
||||
- When paired with explicit QPI-limited intent, it means do not enter deeper models such as Intellectual Archaeology.
|
||||
- QPI may still be selected for problem definition, problem-type judgment, or dominant-scarcity judgment.
|
||||
- IA remains rejected through model-level hard exclusion.
|
||||
|
||||
This remains a selector precedence rule only. It does not add a new model, lifecycle upgrade, LLM selector, RAG, database, service, frontend, backend, user system, or Round 04 blind-test behavior.
|
||||
|
||||
## Decision 016: Close Round 03 after the Round 03.2a pass decision
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Reason:
|
||||
|
||||
The Round 03.2a review accepted the depth-limiting QPI override patch and confirmed that Round 03 can close.
|
||||
|
||||
Closure statement:
|
||||
|
||||
- Round 03 is closed.
|
||||
- QPI and Intellectual Archaeology remain `draft / B / pending`.
|
||||
- QPI is accepted as a draft-callable routing asset at report level only; this does not change JSON model lifecycle status.
|
||||
- Selector v0.2 passes no-call, calibration smoke, model-level exclusion, QPI-before-IA, and depth-limiting override gates.
|
||||
- Round 04 blind input routing evaluation is the next candidate phase, but it is not started by this closure.
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@
|
|||
"card_file": "cards/qpi.md",
|
||||
"source_article_count": 3,
|
||||
"source_evidence_count": 10,
|
||||
"regression_case_count": 55,
|
||||
"regression_case_count": 58,
|
||||
"stability_level": "B",
|
||||
"regression_status": "pending",
|
||||
"status": "draft",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,112 @@
|
|||
# Codex 新会话交接文档：Round 03.2a 评审准备与关闭记录
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||
|
||||
Current phase: `model_library_mvp`
|
||||
|
||||
## 1. 本轮目标
|
||||
|
||||
根据 GPT 对 Round 03.2 的 `REVISE-light` 评审意见，完成 03.2a 微修：
|
||||
|
||||
```text
|
||||
不要展开 / 不要深入分析
|
||||
+
|
||||
只做 QPI / 只做问题定性 / 只判断主导稀缺
|
||||
=
|
||||
允许 QPI，拒绝 IA，不做全局 no-call
|
||||
```
|
||||
|
||||
## 2. 已完成
|
||||
|
||||
- 新增 `depth_limiting_no_call_signals`。
|
||||
- 新增 `qpi_limited_analysis_override_phrases`。
|
||||
- 修改 selector hard no-call precedence：
|
||||
- depth-limiting phrase 单独出现时仍可 no-call；
|
||||
- depth-limiting phrase 与 QPI-limited intent 同时出现时继续评分；
|
||||
- QPI 可被选中；
|
||||
- IA 仍通过 hard exclusion 被拒绝。
|
||||
- 避免 QPI 自身 negative trigger penalty 在该组合下把 QPI 压到阈值以下。
|
||||
- 新增 3 条 QPI regression：
|
||||
- `case_qpi_depth_limited_qpi_only_001`
|
||||
- `case_qpi_depth_limited_problem_definition_001`
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`
|
||||
- 新增 5 条 selector calibration input。
|
||||
- 同步 aggregate regression、regression index、model/card index 和 reports。
|
||||
- 生成 03.2a 评审包：
|
||||
|
||||
```text
|
||||
ccra_review_bundle/round-03.2a_2026-06-17_depth-limiting-qpi-override-patch/
|
||||
```
|
||||
|
||||
## 3. 当前计数
|
||||
|
||||
- QPI regression cases: 58
|
||||
- Intellectual Archaeology regression cases: 19
|
||||
- Aggregate regression cases: 77
|
||||
- Selector calibration inputs: 95
|
||||
|
||||
模型状态没有升级：
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
||||
## 4. 验证结果
|
||||
|
||||
以下命令已通过：
|
||||
|
||||
```powershell
|
||||
python scripts\rebuild_indexes.py --check
|
||||
python -m unittest discover -s tests -p "test*.py" -v
|
||||
python scripts\validate_model_library.py
|
||||
python scripts\check_card_contract.py
|
||||
python scripts\run_selector_demo.py
|
||||
python scripts\run_selector_regression.py
|
||||
python scripts\run_selector_calibration_smoke.py
|
||||
python scripts\check_model_card_sync.py
|
||||
```
|
||||
|
||||
结果：
|
||||
|
||||
- Index check: PASS
|
||||
- Unit tests: PASS, 17 tests
|
||||
- Model library validation: PASS
|
||||
- Card contract: PASS
|
||||
- Selector demo: PASS
|
||||
- Selector regression: PASS
|
||||
- Selector calibration smoke: PASS, 95 inputs, 0 failures, 0 warnings
|
||||
- Model/card sync: PASS
|
||||
|
||||
## 5. 需要 GPT / Owner 判断
|
||||
|
||||
- Round 03.2a 已经由 GPT / CCRA 判断为 `pass`。
|
||||
- Round 03 系列可以正式关闭。
|
||||
- Round 04 blind input evaluation 可以作为下一候选阶段，但本次关闭动作不进入 Round 04。
|
||||
|
||||
## 6. 建议下一步
|
||||
|
||||
1. 记录 Round 03 closure。
|
||||
2. 提交并同步 Round 03.2a 工程资产、评审包、交接文档和关闭记录。
|
||||
3. 仅在后续 Owner 明确要求时启动 Round 04 blind input evaluation；启动前不修改 selector 规则。
|
||||
|
||||
## 7. Round 03 关闭结论
|
||||
|
||||
```text
|
||||
Round 03 closed.
|
||||
|
||||
QPI and Intellectual Archaeology remain draft / B / pending.
|
||||
QPI is accepted as a draft-callable routing asset at report level.
|
||||
Selector v0.2 passes no-call, calibration smoke, model-level exclusion, QPI-before-IA, and depth-limiting override gates.
|
||||
Round 04 blind input routing evaluation is not started in this closeout.
|
||||
```
|
||||
|
||||
注意：`draft-callable` 只能作为评审报告语言，不改变 JSON model lifecycle status。
|
||||
|
||||
## 8. 本次关闭修复
|
||||
|
||||
- 按评审意见在 `cards/qpi.md` 补充 depth-limiting override 的说明：`不要展开` / `不要深入分析` 默认抑制 QPI，但与 `只做 QPI`、`只做问题定性`、`只判断主导稀缺` 等限定分析意图组合时，允许 QPI 窄范围判断并拒绝思想考古。
|
||||
- 在 `docs/DECISIONS.md` 记录 Round 03 关闭决策。
|
||||
- 在 `README.md` 更新下一步状态：Round 03 已关闭，Round 04 只作为后续候选阶段，不在本轮启动。
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Date: 2026-06-16
|
||||
|
||||
Status note: this report is a pre-case-promotion review snapshot. It remains useful for the v0.2 evidence / contract / selector baseline, but its regression counts do not reflect the Round 03 / 03.1 QPI owner-reviewed case promotion or the Round 03.2 selector negative-gate patch. Current regression counts are tracked in `tests/*.regression.json`, `models/model_index.json`, and the latest review bundle.
|
||||
Status note: this report is a pre-case-promotion review snapshot. It remains useful for the v0.2 evidence / contract / selector baseline, but its regression counts do not reflect the Round 03 / 03.1 QPI owner-reviewed case promotion, the Round 03.2 selector negative-gate patch, or the Round 03.2a depth-limiting override patch. Current regression counts are tracked in `tests/*.regression.json`, `models/model_index.json`, and the latest review bundle.
|
||||
|
||||
## 1. 本轮修改摘要
|
||||
|
||||
|
|
@ -111,7 +111,7 @@ Coverage:
|
|||
- 已覆盖误用、防误召回、no-call 和 pipeline gate。
|
||||
- 仍需人工审查用例真实性和遗漏边界。
|
||||
|
||||
Round 03 / 03.1 / 03.2 update: QPI has since expanded to 55 regression cases after owner-reviewed case promotion, selector/no-call repair, and broad-gate no-call traps. Intellectual Archaeology has expanded to 19 regression cases after Round 03.2 IA hard-exclusion traps. This v0.2 table is retained as a historical baseline, not the current count.
|
||||
Round 03 / 03.1 / 03.2 / 03.2a update: QPI has since expanded to 58 regression cases after owner-reviewed case promotion, selector/no-call repair, broad-gate no-call traps, and depth-limiting QPI override traps. Intellectual Archaeology has expanded to 19 regression cases after Round 03.2 IA hard-exclusion traps. This v0.2 table is retained as a historical baseline, not the current count.
|
||||
|
||||
## 6. Selector Regression 结论
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ Status: `PASS`
|
|||
|
||||
Command: `python scripts/run_selector_calibration_smoke.py`
|
||||
|
||||
Calibration inputs checked: 90
|
||||
Calibration inputs checked: 95
|
||||
Failures: 0
|
||||
Warnings: 0
|
||||
|
||||
|
|
@ -99,4 +99,9 @@ Warnings: 0
|
|||
- `selector_calibration_qpi_how_to_excel_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||
- `selector_calibration_qpi_how_to_markdown_pdf_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||
- `selector_calibration_qpi_dictionary_judgment_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||
- `selector_calibration_qpi_depth_limited_qpi_only_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_002`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_problem_or_issue_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_dominant_scarcity_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_analysis_override_should_call_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Status: `PASS`
|
||||
|
||||
Cases checked: 18
|
||||
Cases checked: 21
|
||||
Failures: 0
|
||||
|
||||
## Cases
|
||||
|
|
@ -24,4 +24,7 @@ Failures: 0
|
|||
- `case_qpi_how_to_excel_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_how_to_markdown_pdf_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_dictionary_judgment_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_depth_limited_qpi_only_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_depth_limited_problem_definition_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_analysis_override_should_call_001`: PASS; selected=['qpi']; no_call=False
|
||||
|
|
|
|||
|
|
@ -33,6 +33,12 @@ def hard_no_call_hits(user_input, global_rules):
|
|||
return hit_any(user_input, signals)
|
||||
|
||||
|
||||
def depth_limited_qpi_override_hits(user_input, global_rules):
|
||||
depth_hits = hit_any(user_input, global_rules.get("depth_limiting_no_call_signals", []))
|
||||
override_hits = hit_any(user_input, global_rules.get("qpi_limited_analysis_override_phrases", []))
|
||||
return depth_hits, override_hits
|
||||
|
||||
|
||||
def model_hard_exclusion_hits(user_input, model_rule):
|
||||
return hit_any(user_input, model_rule.get("hard_exclusion_triggers", []))
|
||||
|
||||
|
|
@ -45,7 +51,9 @@ def score_model(model, model_rule, global_rules, user_input, task_type="", pipel
|
|||
penalties = []
|
||||
|
||||
negative_hits = hit_any(user_input, model_rule.get("negative_triggers", [])) + hit_any(user_input, model.get("negative_triggers", []))
|
||||
if negative_hits and not has_analysis_override(user_input, global_rules):
|
||||
depth_hits, qpi_limited_hits = depth_limited_qpi_override_hits(user_input, global_rules)
|
||||
suppress_qpi_depth_penalty = model_id == "qpi" and depth_hits and qpi_limited_hits
|
||||
if negative_hits and not has_analysis_override(user_input, global_rules) and not suppress_qpi_depth_penalty:
|
||||
penalty_key = "model_negative_penalty_qpi" if model_id == "qpi" else "model_negative_penalty_default"
|
||||
score -= weights.get(penalty_key, 0.5)
|
||||
penalties.append("negative trigger: " + "、".join(sorted(set(negative_hits))[:4]))
|
||||
|
|
@ -146,7 +154,14 @@ def recommend(root, user_input, task_type="", pipeline_position="", threshold=No
|
|||
threshold = global_rules.get("no_call_threshold", 0.35) if threshold is None else threshold
|
||||
|
||||
no_call_hits = hard_no_call_hits(user_input, global_rules)
|
||||
if global_rules.get("hard_no_call_first", True) and no_call_hits and not has_analysis_override(user_input, global_rules):
|
||||
depth_hits, qpi_limited_hits = depth_limited_qpi_override_hits(user_input, global_rules)
|
||||
has_depth_limited_qpi_override = bool(depth_hits and qpi_limited_hits)
|
||||
if (
|
||||
global_rules.get("hard_no_call_first", True)
|
||||
and no_call_hits
|
||||
and not has_analysis_override(user_input, global_rules)
|
||||
and not has_depth_limited_qpi_override
|
||||
):
|
||||
return hard_no_call_result(root, user_input, no_call_hits)
|
||||
|
||||
model_rules = rules_by_model(selector_rules)
|
||||
|
|
|
|||
|
|
@ -25,6 +25,11 @@ Round 03.2 selector rule:
|
|||
- A hard exclusion rejects only the named model; it does not globally no-call the request.
|
||||
- Generic bare QPI gate words such as `如何` and `判断` should not be used alone. Use compound problem-definition phrases instead.
|
||||
|
||||
Round 03.2a selector rule:
|
||||
|
||||
- Depth-limiting phrases such as `不要展开` and `不要深入分析` remain global no-call signals by default.
|
||||
- If a depth-limiting phrase is paired with explicit QPI-limited intent such as `只做 QPI`, `只做问题定性`, or `只判断主导稀缺`, the selector should continue scoring, allow QPI, and still reject IA.
|
||||
|
||||
Current files:
|
||||
|
||||
- `selector_rules.json`
|
||||
|
|
|
|||
|
|
@ -626,6 +626,41 @@
|
|||
"expected_selector_behavior": "no_call",
|
||||
"expected_notes": "Dictionary meaning request should not be routed to QPI by bare 判断."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_qpi_only_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要展开到思想考古，只做 QPI 复核。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Depth-limiting phrase should not become global no-call when QPI-only intent is explicit."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_problem_definition_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要展开到思想考古，只做问题定性。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Do not call IA, but still route to QPI for problem definition."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_problem_definition_002",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要深入分析，只做问题定性。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Depth-limiting no-call phrase should be overridden by explicit QPI-limited problem definition."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_problem_or_issue_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要深入分析背后的理论，只判断这是问题还是课题。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Problem-or-issue judgment is QPI-limited analysis, not global no-call."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_dominant_scarcity_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要展开，只判断主导稀缺。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Dominant-scarcity-only judgment should select QPI and reject IA."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_analysis_override_should_call_001",
|
||||
"category": "round_03_1_analysis_override",
|
||||
|
|
|
|||
|
|
@ -35,6 +35,18 @@
|
|||
"不用解释",
|
||||
"生成图片"
|
||||
],
|
||||
"depth_limiting_no_call_signals": [
|
||||
"不要展开",
|
||||
"不要深入分析"
|
||||
],
|
||||
"qpi_limited_analysis_override_phrases": [
|
||||
"只做 QPI",
|
||||
"只做问题定性",
|
||||
"只做问题定性复核",
|
||||
"只判断主导稀缺",
|
||||
"只判断这是问题还是课题",
|
||||
"只判断问题类型"
|
||||
],
|
||||
"direct_execution_no_call_signals": [
|
||||
"排一下",
|
||||
"排床位",
|
||||
|
|
@ -217,7 +229,11 @@
|
|||
"不要调用思想考古",
|
||||
"不调用思想考古",
|
||||
"只做 QPI",
|
||||
"只做问题定性"
|
||||
"只做问题定性",
|
||||
"只做问题定性复核",
|
||||
"只判断主导稀缺",
|
||||
"只判断这是问题还是课题",
|
||||
"只判断问题类型"
|
||||
],
|
||||
"pipeline_position": "modeling_depth_analysis",
|
||||
"selection_priority": 7,
|
||||
|
|
|
|||
|
|
@ -1634,6 +1634,93 @@
|
|||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2 no-call trap: dictionary judgment is not QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_qpi_only_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要展开到思想考古，只做 QPI 复核。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA；“不要展开”在此表示限制深度，不是全局 no-call。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"depth_limited_qpi_override"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: global no-call phrase plus QPI-only intent."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_problem_definition_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要深入分析，只做问题定性。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA；“不要深入分析”限制深度，但问题定性仍需要 QPI。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"depth_limited_problem_definition"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: limited problem definition should route to QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_dominant_scarcity_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要展开，只判断主导稀缺。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA；主导稀缺判断属于 QPI 限定分析。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"dominant_scarcity_gate"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: dominant-scarcity-only judgment should route to QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_analysis_override_should_call_001",
|
||||
"model_id": "qpi",
|
||||
|
|
|
|||
|
|
@ -1633,6 +1633,93 @@
|
|||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2 no-call trap: dictionary judgment is not QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_qpi_only_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要展开到思想考古，只做 QPI 复核。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA；“不要展开”在此表示限制深度，不是全局 no-call。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"depth_limited_qpi_override"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: global no-call phrase plus QPI-only intent."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_problem_definition_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要深入分析，只做问题定性。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA；“不要深入分析”限制深度，但问题定性仍需要 QPI。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"depth_limited_problem_definition"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: limited problem definition should route to QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_dominant_scarcity_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要展开，只判断主导稀缺。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA；主导稀缺判断属于 QPI 限定分析。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"dominant_scarcity_gate"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: dominant-scarcity-only judgment should route to QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_analysis_override_should_call_001",
|
||||
"model_id": "qpi",
|
||||
|
|
|
|||
|
|
@ -7,14 +7,14 @@
|
|||
{
|
||||
"model_id": "qpi",
|
||||
"regression_file": "tests/qpi.regression.json",
|
||||
"case_count": 55,
|
||||
"case_count": 58,
|
||||
"case_type_counts": {
|
||||
"boundary": 13,
|
||||
"misuse": 16,
|
||||
"no_call": 6,
|
||||
"pipeline": 1,
|
||||
"positive": 16,
|
||||
"selector_gate": 3
|
||||
"selector_gate": 6
|
||||
}
|
||||
},
|
||||
{
|
||||
|
|
|
|||
Loading…
Reference in New Issue