Compare commits
2 Commits
ddaa0cf35b
...
77114cee5b
| Author | SHA1 | Date |
|---|---|---|
|
|
77114cee5b | |
|
|
cd9f63e9eb |
|
|
@ -170,10 +170,12 @@ Indexes follow `docs/INDEX_MAINTENANCE_PROTOCOL.md`: every asset change must syn
|
|||
|
||||
CCRA review packages are archived by round under `ccra_review_bundle/round-NN_YYYY-MM-DD_topic/`. Do not place new review bundle files directly in `ccra_review_bundle/`; create or update the current round directory instead.
|
||||
|
||||
Files intended for GPT / CCRA upload inside a review bundle should include the round version in the filename, for example `_03.2`. Raw changed-file zips should also include the round version while preserving source-relative paths inside the archive.
|
||||
|
||||
## 13. Next Steps
|
||||
|
||||
1. Submit the Round 03.1 selector/no-call/regression patch for GPT / CCRA review.
|
||||
1. Round 03 is closed after the Round 03.2a CCRA / GPT pass decision.
|
||||
2. Keep QPI and Intellectual Archaeology at `draft / B / pending` until Owner / CCRA review accepts stronger status.
|
||||
3. Run a QPI blind-test round with new unlabelled inputs after Round 03.1 passes.
|
||||
3. Start Round 04 blind input routing evaluation only after an explicit owner instruction; do not modify selector rules before that evaluation.
|
||||
4. Defer any third-model expansion until the current QPI selector and case layer are accepted.
|
||||
5. Route missing reusable extraction, inspection, or stability-scoring tools to `requirements/skills-vault/` or `requirements/ccpe/` instead of improvising local platform features.
|
||||
|
|
|
|||
|
|
@ -175,7 +175,7 @@
|
|||
- 机制稳定性:中等,七层结构清晰,但层级之间的真实递进关系需要更多案例验证。
|
||||
- 边界清晰度:中等,最小充分下潜原则已明确,但执行时容易过度下钻。
|
||||
- 来源证据质量:较高,核心定义和七层结构来自原文,停止条件来自综合应用材料。
|
||||
- 回归测试表现:pending,已有五条样板用例,尚未扩展到更多领域。
|
||||
- 回归测试表现:pending,当前已有 19 条回归用例,已覆盖 positive、boundary、misuse、no_call、selector_gate 和 pipeline;Round 03.2 已纳入明确拒绝思想考古的模型级 hard exclusion 陷阱。
|
||||
|
||||
评级理由:纵向下潜机制清晰,适合深度建模,但七层结构和停止条件需要进一步稳定。
|
||||
|
||||
|
|
@ -185,13 +185,10 @@
|
|||
|
||||
`pending`
|
||||
|
||||
当前已有五条回归用例:
|
||||
当前已有 19 条回归用例,覆盖 positive、boundary、misuse、no_call、selector_gate 和 pipeline。Round 03.2 重点新增:
|
||||
|
||||
- `case_ia_positive_deep_modeling_001`
|
||||
- `case_ia_positive_resilience_001`
|
||||
- `case_ia_boundary_minimum_depth_001`
|
||||
- `case_ia_misuse_philosophizing_001`
|
||||
- `case_ia_misuse_no_validation_001`
|
||||
- `case_ia_negative_explicit_do_not_call_001`
|
||||
- `case_ia_negative_only_qpi_001`
|
||||
|
||||
## 示例输入
|
||||
|
||||
|
|
|
|||
20
cards/qpi.md
20
cards/qpi.md
|
|
@ -145,6 +145,8 @@ QPI 必须区分:
|
|||
- 不要展开
|
||||
- 只改错别字
|
||||
|
||||
说明:`不要展开` / `不要深入分析` 默认抑制 QPI;但如果同时出现 `只做 QPI`、`只做问题定性`、`只判断主导稀缺` 等限定分析意图,则允许 QPI 做窄范围判断,并拒绝进入思想考古。
|
||||
|
||||
## 相关模型
|
||||
|
||||
- `intellectual_archaeology`: 当 QPI 判断问题偏中重型难题或课题时,可进入思想考古做深度建模。
|
||||
|
|
@ -202,7 +204,7 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
|||
- 机制稳定性:中高,扫描匮乏物和匹配处理范式的机制明确。
|
||||
- 边界清晰度:中等,混合型问题、多视角分歧和上下文不足场景仍需更多测试。
|
||||
- 来源证据质量:较高,主体性和动态性来自 2025 原文,核心匮乏物和误框定规则来自 2026 原文,应用规则来自综合文档。
|
||||
- 回归测试表现:pending,当前已有 52 条 QPI 回归用例;其中 Round 03 / 03.1 已纳入 owner-reviewed case layer、selector no-call、multi-perspective 和低上下文 provisional 边界用例。生命周期仍保持 draft。
|
||||
- 回归测试表现:pending,当前已有 58 条 QPI 回归用例;其中 Round 03 / 03.1 已纳入 owner-reviewed case layer、selector no-call、multi-perspective 和低上下文 provisional 边界用例,Round 03.2 已纳入普通 how-to / 词义判断 no-call 陷阱,Round 03.2a 已纳入 depth-limiting QPI override 陷阱。生命周期仍保持 draft。
|
||||
|
||||
评级理由:三分结构清晰,适合作为入口路由模型,但需要补充大量边界案例,防止过度升维或降维。
|
||||
|
||||
|
|
@ -212,7 +214,7 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
|||
|
||||
`pending`
|
||||
|
||||
当前已有 52 条 QPI 回归用例,覆盖 positive、boundary、misuse、no_call、selector_gate 和 pipeline。Round 03 / 03.1 重点新增:
|
||||
当前已有 58 条 QPI 回归用例,覆盖 positive、boundary、misuse、no_call、selector_gate 和 pipeline。Round 03 / 03.1 重点新增:
|
||||
|
||||
- owner-reviewed flow / disappointment / organizational year-end-review cases;
|
||||
- `case_qpi_international_logistics_no_call_001`
|
||||
|
|
@ -222,6 +224,18 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
|||
- `case_qpi_direct_summary_no_call_001`
|
||||
- `case_qpi_analysis_override_should_call_001`
|
||||
|
||||
Round 03.2 重点新增:
|
||||
|
||||
- `case_qpi_how_to_excel_no_call_001`
|
||||
- `case_qpi_how_to_markdown_pdf_no_call_001`
|
||||
- `case_qpi_dictionary_judgment_no_call_001`
|
||||
|
||||
Round 03.2a 重点新增:
|
||||
|
||||
- `case_qpi_depth_limited_qpi_only_001`
|
||||
- `case_qpi_depth_limited_problem_definition_001`
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`
|
||||
|
||||
## 示例输入
|
||||
|
||||
- “我们团队每次都说要长期主义,但一到季度 KPI 就回到短期动作,这到底是什么问题?”
|
||||
|
|
@ -280,7 +294,7 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
|||
|
||||
QPI 应作为问题回答系统的前置路由模型,用于防止系统在问题类型错误的情况下直接给答案。
|
||||
|
||||
selector 应优先使用 `trigger_keywords`、`negative_triggers`、`pipeline_position` 和 `selection_priority` 做规则推荐。
|
||||
selector 应优先使用 `trigger_keywords`、`negative_triggers`、模型级 hard exclusion、`pipeline_position` 和 `selection_priority` 做规则推荐。
|
||||
|
||||
## 版本信息
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,55 @@
|
|||
# Round 03.2 CCRA Review Brief
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||
|
||||
Patch title: Selector Negative-Gate Patch
|
||||
|
||||
## Scope
|
||||
|
||||
Round 03.2 is a tiny follow-up to Round 03.1.
|
||||
|
||||
It addresses GPT review finding:
|
||||
|
||||
> IA / 思想考古 negative triggers were still score penalties, not model-level hard exclusions.
|
||||
|
||||
This patch does not:
|
||||
|
||||
- add a third model;
|
||||
- upgrade QPI or Intellectual Archaeology lifecycle status;
|
||||
- introduce an LLM selector;
|
||||
- introduce RAG, database, frontend, backend, user accounts, or platform features;
|
||||
- start Round 04 blind testing.
|
||||
|
||||
Current model status remains:
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
||||
## What Changed
|
||||
|
||||
1. Added model-level hard exclusion for explicit IA refusal.
|
||||
2. Removed bare QPI gate signals `如何` and `判断`.
|
||||
3. Replaced broad gates with compound problem-definition signals.
|
||||
4. Strengthened selector calibration smoke checks:
|
||||
- `no_call` requires `selected_models == []` and `no_call == true`;
|
||||
- `no_call_or_low_priority` rejects both QPI and IA;
|
||||
- `select_qpi_reject_ia` requires QPI selected and IA rejected;
|
||||
- IA selection expectations fail if IA hard exclusion is present.
|
||||
5. Added five GPT-requested traps to regression and calibration data.
|
||||
|
||||
## Suggested Review Order
|
||||
|
||||
1. `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_03.2.md`
|
||||
2. `01_PATCH_MATRIX_03.2.md`
|
||||
3. `02_CURRENT_ASSET_PACK_03.2.md`
|
||||
4. `03_VALIDATION_AND_COMMAND_LOG_03.2.md`
|
||||
5. `04_REVIEW_QUESTIONS_FOR_GPT_03.2.md`
|
||||
6. `optional_raw_changed_files_03.2.zip` only if exact changed-file inspection is needed
|
||||
|
||||
## Primary Review Question
|
||||
|
||||
Does Round 03.2 close the model-level negative gate issue enough to close Round 03 and proceed to Round 04 blind input evaluation?
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
# Round 03.2 Patch Matrix
|
||||
|
||||
## GPT Requirement Mapping
|
||||
|
||||
| Requirement | Implementation | Files |
|
||||
| --- | --- | --- |
|
||||
| IA explicit refusal must be hard exclusion, not score penalty | Added `hard_exclusion_triggers` for IA and forced rejected decision when matched | `selector/selector_rules.json`, `scripts/run_selector_demo.py` |
|
||||
| Include refusal phrases such as `不要思想考古`, `先不要思想考古`, `不调用思想考古`, `只做 QPI`, `只做问题定性` | Added IA hard exclusion trigger list | `selector/selector_rules.json` |
|
||||
| Preserve QPI explicit-analysis override but do not let it re-enable IA | Hard exclusion applies per model after scoring and is not bypassed by QPI override | `scripts/run_selector_demo.py` |
|
||||
| Remove or narrow bare `如何` and `判断` QPI gates | Removed bare signals; added compound problem-definition phrases | `selector/selector_rules.json` |
|
||||
| Strict `no_call` smoke behavior | `no_call` now requires no selected models and `no_call=true` | `scripts/run_selector_calibration_smoke.py` |
|
||||
| Strict `no_call_or_low_priority` behavior | Fails if QPI or IA is selected | `scripts/run_selector_calibration_smoke.py` |
|
||||
| Add `select_qpi_reject_ia` behavior | Requires QPI selected and IA rejected | `scripts/run_selector_calibration_smoke.py`, `selector/selector_calibration_inputs.json` |
|
||||
| Fail IA selection expectations if IA hard exclusion exists | Added hard-exclusion check for `select_intellectual_archaeology` expected cases | `scripts/run_selector_calibration_smoke.py` |
|
||||
| Add five traps | Added two IA hard-exclusion traps and three generic how-to / dictionary no-call traps | `tests/*.regression.json`, `selector/selector_calibration_inputs.json` |
|
||||
|
||||
## Added Regression Cases
|
||||
|
||||
### Intellectual Archaeology
|
||||
|
||||
- `case_ia_negative_explicit_do_not_call_001`
|
||||
- `case_ia_negative_only_qpi_001`
|
||||
|
||||
### QPI
|
||||
|
||||
- `case_qpi_how_to_excel_no_call_001`
|
||||
- `case_qpi_how_to_markdown_pdf_no_call_001`
|
||||
- `case_qpi_dictionary_judgment_no_call_001`
|
||||
|
||||
## Added Calibration Inputs
|
||||
|
||||
- `selector_calibration_ia_negative_explicit_do_not_call_001`
|
||||
- `selector_calibration_ia_negative_only_qpi_001`
|
||||
- `selector_calibration_qpi_how_to_excel_no_call_001`
|
||||
- `selector_calibration_qpi_how_to_markdown_pdf_no_call_001`
|
||||
- `selector_calibration_qpi_dictionary_judgment_no_call_001`
|
||||
|
||||
## Current Counts
|
||||
|
||||
| Asset | Round 03.1 | Round 03.2 |
|
||||
| --- | ---: | ---: |
|
||||
| QPI regression cases | 52 | 55 |
|
||||
| IA regression cases | 17 | 19 |
|
||||
| Aggregate regression cases | 69 | 74 |
|
||||
| Selector calibration inputs | 85 | 90 |
|
||||
|
||||
## Non-Goals Preserved
|
||||
|
||||
- No new model.
|
||||
- No lifecycle upgrade.
|
||||
- No LLM selector.
|
||||
- No RAG / database / frontend / backend.
|
||||
- No Round 04 blind-test execution.
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
# Round 03.2 Current Asset Pack
|
||||
|
||||
## Canonical Changed Assets
|
||||
|
||||
- `selector/selector_rules.json`
|
||||
- `selector/selector_calibration_inputs.json`
|
||||
- `tests/qpi.regression.json`
|
||||
- `tests/intellectual_archaeology.regression.json`
|
||||
- `tests/regression_cases.json`
|
||||
- `tests/regression_index.json`
|
||||
- `scripts/run_selector_demo.py`
|
||||
- `scripts/run_selector_calibration_smoke.py`
|
||||
|
||||
## Generated / Derived Assets
|
||||
|
||||
- `models/model_index.json`
|
||||
- `cards/card_index.md`
|
||||
- `reports/index_rebuild_report.md`
|
||||
- `reports/validation_report.md`
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
- `reports/model_card_sync_report_v0.2.md`
|
||||
|
||||
## Documentation Updated
|
||||
|
||||
- `README.md`
|
||||
- `docs/DECISIONS.md`
|
||||
- `selector/README.md`
|
||||
- `scripts/README.md`
|
||||
- `reports/README.md`
|
||||
- `cards/qpi.md`
|
||||
- `cards/intellectual_archaeology.md`
|
||||
- `reports/content_review_report_v0.2.md`
|
||||
|
||||
## Selector Behavior Summary
|
||||
|
||||
### IA explicit refusal
|
||||
|
||||
Inputs such as:
|
||||
|
||||
```text
|
||||
QPI 已判断:这是 Issue。先不要思想考古,只做问题定性复核。
|
||||
不要思想考古,只做 QPI 复核。
|
||||
```
|
||||
|
||||
now select QPI and reject `intellectual_archaeology`.
|
||||
|
||||
### Generic how-to / dictionary no-call
|
||||
|
||||
Inputs such as:
|
||||
|
||||
```text
|
||||
如何在 Excel 里冻结首行?
|
||||
如何把 Markdown 转 PDF?
|
||||
判断这个词是什么意思。
|
||||
```
|
||||
|
||||
now produce no selected models and `no_call=true`.
|
||||
|
||||
### Still preserved
|
||||
|
||||
QPI remains callable for explicit problem-definition or owner-reviewed calibration inputs.
|
||||
|
||||
IA remains callable when IA is explicitly requested without hard exclusion and the heavy-depth / QPI-completed gate is satisfied.
|
||||
|
||||
## Current Lifecycle State
|
||||
|
||||
No lifecycle upgrade was made.
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
# Round 03.2 Validation and Command Log
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
## Red Evidence
|
||||
|
||||
After adding the five GPT-requested traps and before implementation, selector checks failed as expected.
|
||||
|
||||
`python scripts\run_selector_regression.py`
|
||||
|
||||
- `case_ia_negative_explicit_do_not_call_001`: IA was selected despite explicit refusal.
|
||||
- `case_ia_negative_only_qpi_001`: QPI was not selected.
|
||||
- `case_qpi_how_to_excel_no_call_001`: QPI was selected.
|
||||
- `case_qpi_how_to_markdown_pdf_no_call_001`: QPI was selected.
|
||||
- `case_qpi_dictionary_judgment_no_call_001`: QPI was selected.
|
||||
|
||||
`python scripts\run_selector_calibration_smoke.py`
|
||||
|
||||
- `selector_calibration_ia_negative_explicit_do_not_call_001`: expected IA rejection.
|
||||
- `selector_calibration_ia_negative_only_qpi_001`: expected QPI selection.
|
||||
- three generic how-to / dictionary traps expected no QPI selection.
|
||||
|
||||
## Green Verification
|
||||
|
||||
The following commands passed after the Round 03.2 patch:
|
||||
|
||||
```powershell
|
||||
python scripts\rebuild_indexes.py --check
|
||||
python -m unittest discover -s tests -p "test*.py" -v
|
||||
python scripts\validate_model_library.py
|
||||
python scripts\check_card_contract.py
|
||||
python scripts\run_selector_demo.py
|
||||
python scripts\run_selector_regression.py
|
||||
python scripts\run_selector_calibration_smoke.py
|
||||
python scripts\check_model_card_sync.py
|
||||
```
|
||||
|
||||
Results:
|
||||
|
||||
- Index check: PASS
|
||||
- Unit tests: PASS, 17 tests
|
||||
- Model library validation: PASS
|
||||
- Card contract: PASS
|
||||
- Selector demo: PASS
|
||||
- Selector regression: PASS, 18 selector-relevant cases
|
||||
- Selector calibration smoke: PASS, 90 inputs, 0 failures, 0 warnings
|
||||
- Model/card sync: PASS
|
||||
|
||||
## Generated Reports
|
||||
|
||||
- `reports/index_rebuild_report.md`
|
||||
- `reports/validation_report.md`
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
- `reports/model_card_sync_report_v0.2.md`
|
||||
|
||||
## Important Interpretation
|
||||
|
||||
Validation pass does not mean model content is stable or owner-approved.
|
||||
|
||||
It only means the Round 03.2 selector negative-gate patch is structurally consistent and passes current local checks.
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# Round 03.2 Review Questions for GPT
|
||||
|
||||
Please review Round 03.2 only.
|
||||
|
||||
Do not expand into Round 04 blind testing yet.
|
||||
|
||||
## Questions
|
||||
|
||||
1. Does model-level hard exclusion now correctly handle explicit IA refusal?
|
||||
2. Are the current hard exclusion phrases sufficient for Round 03 closure?
|
||||
3. Is it acceptable that hard exclusion rejects only IA while still allowing QPI when the user says `只做 QPI` or `只做问题定性`?
|
||||
4. Is removing bare `如何` and `判断` while preserving compound problem-definition phrases enough to reduce QPI over-selection?
|
||||
5. Are the strengthened calibration smoke assertions strict enough to become a long-term selector gate?
|
||||
6. Can Round 03 now be closed with the following conclusion?
|
||||
|
||||
```text
|
||||
QPI and Intellectual Archaeology remain draft / B / pending.
|
||||
QPI is accepted as a draft-callable routing asset at report level.
|
||||
Selector v0.2 passes no-call, calibration smoke, model-level exclusion, and QPI-before-IA gates.
|
||||
Proceed to Round 04 blind input evaluation.
|
||||
```
|
||||
|
||||
## Suggested Answer Format
|
||||
|
||||
Please answer with:
|
||||
|
||||
- `pass`, `revise`, or `block`;
|
||||
- specific failing cases if `revise` or `block`;
|
||||
- whether Round 04 blind input evaluation may start after this patch.
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
# Round 03.2 Bundle File Manifest
|
||||
|
||||
## Review Files
|
||||
|
||||
- `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_03.2.md`
|
||||
- `01_PATCH_MATRIX_03.2.md`
|
||||
- `02_CURRENT_ASSET_PACK_03.2.md`
|
||||
- `03_VALIDATION_AND_COMMAND_LOG_03.2.md`
|
||||
- `04_REVIEW_QUESTIONS_FOR_GPT_03.2.md`
|
||||
- `BUNDLE_FILE_MANIFEST_03.2.md`
|
||||
- `optional_raw_changed_files_03.2.zip`
|
||||
|
||||
## Optional Raw Changed Files
|
||||
|
||||
The zip is optional and should be used only if exact file inspection is needed.
|
||||
|
||||
It preserves source-relative paths.
|
||||
|
||||
It excludes:
|
||||
|
||||
- `knowledge_assets/`
|
||||
- `__pycache__/`
|
||||
- `*.pyc`
|
||||
- previous review bundle zip files
|
||||
|
||||
## Source-Relative Files Included in Raw Zip
|
||||
|
||||
- `README.md`
|
||||
- `cards/intellectual_archaeology.md`
|
||||
- `cards/qpi.md`
|
||||
- `docs/DECISIONS.md`
|
||||
- `docs/FILE_TAXONOMY.md`
|
||||
- `reports/README.md`
|
||||
- `reports/content_review_report_v0.2.md`
|
||||
- `reports/Codex新会话交接文档_Round03.2评审准备_2026-06-17.md`
|
||||
- `reports/index_rebuild_report.md`
|
||||
- `reports/model_card_sync_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/validation_report.md`
|
||||
- `scripts/README.md`
|
||||
- `scripts/run_selector_calibration_smoke.py`
|
||||
- `scripts/run_selector_demo.py`
|
||||
- `selector/README.md`
|
||||
- `selector/selector_calibration_inputs.json`
|
||||
- `selector/selector_rules.json`
|
||||
- `tests/intellectual_archaeology.regression.json`
|
||||
- `tests/qpi.regression.json`
|
||||
- `tests/regression_cases.json`
|
||||
- `tests/regression_index.json`
|
||||
- `models/model_index.json`
|
||||
- `cards/card_index.md`
|
||||
Binary file not shown.
|
|
@ -0,0 +1,52 @@
|
|||
# Round 03.2a CCRA Review Brief
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||
|
||||
Patch title: Depth-Limiting Override For QPI
|
||||
|
||||
## Scope
|
||||
|
||||
Round 03.2a is a micro-patch after GPT's Round 03.2 `REVISE-light` review.
|
||||
|
||||
It fixes one selector precedence issue:
|
||||
|
||||
> `不要展开` / `不要深入分析` should remain global no-call by default, but when paired with explicit QPI-limited intent such as `只做 QPI`, `只做问题定性`, or `只判断主导稀缺`, they should allow QPI and reject IA.
|
||||
|
||||
This patch does not:
|
||||
|
||||
- add a third model;
|
||||
- upgrade QPI or Intellectual Archaeology lifecycle status;
|
||||
- introduce an LLM selector;
|
||||
- introduce RAG, database, frontend, backend, user accounts, or platform features;
|
||||
- start Round 04 blind testing.
|
||||
|
||||
Current model status remains:
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
||||
## What Changed
|
||||
|
||||
1. Added `depth_limiting_no_call_signals`.
|
||||
2. Added `qpi_limited_analysis_override_phrases`.
|
||||
3. Modified hard no-call precedence so depth-limiting phrases do not globally no-call when QPI-limited intent is explicit.
|
||||
4. Suppressed QPI's own depth-limiting negative penalty only in that explicit QPI-limited combination.
|
||||
5. Expanded IA hard exclusion triggers to cover QPI-limited judgment phrases.
|
||||
6. Added 3 selector regression cases and 5 calibration smoke inputs.
|
||||
|
||||
## Suggested Review Order
|
||||
|
||||
1. `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_03.2a.md`
|
||||
2. `01_PATCH_MATRIX_03.2a.md`
|
||||
3. `02_CURRENT_ASSET_PACK_03.2a.md`
|
||||
4. `03_VALIDATION_AND_COMMAND_LOG_03.2a.md`
|
||||
5. `04_REVIEW_QUESTIONS_FOR_GPT_03.2a.md`
|
||||
6. `optional_raw_changed_files_03.2a.zip` only if exact changed-file inspection is needed
|
||||
|
||||
## Primary Review Question
|
||||
|
||||
Does Round 03.2a close the depth-limiting override issue enough to close Round 03 and proceed to Round 04 blind input evaluation?
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# Round 03.2a Patch Matrix
|
||||
|
||||
## GPT Requirement Mapping
|
||||
|
||||
| Requirement | Implementation | Files |
|
||||
| --- | --- | --- |
|
||||
| Add QPI-limited override phrases | Added `qpi_limited_analysis_override_phrases` | `selector/selector_rules.json` |
|
||||
| Keep `不要展开` / `不要深入分析` as no-call by default | Added separate `depth_limiting_no_call_signals`; did not remove original hard no-call phrases | `selector/selector_rules.json` |
|
||||
| Let QPI-limited depth requests continue scoring | Hard no-call branch now skips global no-call only when depth-limiting and QPI-limited phrases both hit | `scripts/run_selector_demo.py` |
|
||||
| Do not let QPI's own negative trigger penalty suppress the override | Suppressed QPI depth-limiting penalty only for explicit QPI-limited combinations | `scripts/run_selector_demo.py` |
|
||||
| IA must remain rejected | Added QPI-limited judgment phrases to IA hard exclusion triggers | `selector/selector_rules.json` |
|
||||
| Add regression / calibration cases | Added 3 regression cases and 5 calibration inputs | `tests/qpi.regression.json`, `selector/selector_calibration_inputs.json` |
|
||||
|
||||
## Added Regression Cases
|
||||
|
||||
- `case_qpi_depth_limited_qpi_only_001`
|
||||
- `case_qpi_depth_limited_problem_definition_001`
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`
|
||||
|
||||
## Added Calibration Inputs
|
||||
|
||||
- `selector_calibration_qpi_depth_limited_qpi_only_001`
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_001`
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_002`
|
||||
- `selector_calibration_qpi_depth_limited_problem_or_issue_001`
|
||||
- `selector_calibration_qpi_depth_limited_dominant_scarcity_001`
|
||||
|
||||
## Current Counts
|
||||
|
||||
| Asset | Round 03.2 | Round 03.2a |
|
||||
| --- | ---: | ---: |
|
||||
| QPI regression cases | 55 | 58 |
|
||||
| IA regression cases | 19 | 19 |
|
||||
| Aggregate regression cases | 74 | 77 |
|
||||
| Selector calibration inputs | 90 | 95 |
|
||||
|
||||
## Non-Goals Preserved
|
||||
|
||||
- No new model.
|
||||
- No lifecycle upgrade.
|
||||
- No LLM selector.
|
||||
- No RAG / database / frontend / backend.
|
||||
- No Round 04 blind-test execution.
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
# Round 03.2a Current Asset Pack
|
||||
|
||||
## Canonical Changed Assets
|
||||
|
||||
- `selector/selector_rules.json`
|
||||
- `selector/selector_calibration_inputs.json`
|
||||
- `tests/qpi.regression.json`
|
||||
- `tests/regression_cases.json`
|
||||
- `tests/regression_index.json`
|
||||
- `scripts/run_selector_demo.py`
|
||||
|
||||
## Generated / Derived Assets
|
||||
|
||||
- `models/model_index.json`
|
||||
- `cards/card_index.md`
|
||||
- `reports/index_rebuild_report.md`
|
||||
- `reports/validation_report.md`
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
- `reports/model_card_sync_report_v0.2.md`
|
||||
|
||||
## Documentation Updated
|
||||
|
||||
- `README.md`
|
||||
- `docs/DECISIONS.md`
|
||||
- `selector/README.md`
|
||||
- `cards/qpi.md`
|
||||
- `reports/content_review_report_v0.2.md`
|
||||
|
||||
## Required Behavior
|
||||
|
||||
The following inputs should now select QPI, reject IA, and set `no_call=false`:
|
||||
|
||||
```text
|
||||
不要展开到思想考古,只做 QPI 复核。
|
||||
不要展开到思想考古,只做问题定性。
|
||||
不要深入分析,只做问题定性。
|
||||
不要深入分析背后的理论,只判断这是问题还是课题。
|
||||
不要展开,只判断主导稀缺。
|
||||
```
|
||||
|
||||
Depth-limiting phrases without explicit QPI-limited intent remain no-call / low-depth signals.
|
||||
|
||||
## Current Lifecycle State
|
||||
|
||||
No lifecycle upgrade was made.
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
# Round 03.2a Validation and Command Log
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
## Red Evidence
|
||||
|
||||
After adding the 03.2a traps and before implementation, selector checks failed as expected.
|
||||
|
||||
`python scripts\run_selector_regression.py`
|
||||
|
||||
- `case_qpi_depth_limited_qpi_only_001`: expected primary QPI, got none.
|
||||
- `case_qpi_depth_limited_problem_definition_001`: expected primary QPI, got none.
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`: expected primary QPI, got none.
|
||||
|
||||
`python scripts\run_selector_calibration_smoke.py`
|
||||
|
||||
- five depth-limiting QPI override inputs failed with expected QPI selection missing.
|
||||
|
||||
## Green Verification
|
||||
|
||||
The following commands passed after the Round 03.2a patch:
|
||||
|
||||
```powershell
|
||||
python scripts\rebuild_indexes.py --check
|
||||
python -m unittest discover -s tests -p "test*.py" -v
|
||||
python scripts\validate_model_library.py
|
||||
python scripts\check_card_contract.py
|
||||
python scripts\run_selector_demo.py
|
||||
python scripts\run_selector_regression.py
|
||||
python scripts\run_selector_calibration_smoke.py
|
||||
python scripts\check_model_card_sync.py
|
||||
```
|
||||
|
||||
Results:
|
||||
|
||||
- Index check: PASS
|
||||
- Unit tests: PASS, 17 tests
|
||||
- Model library validation: PASS
|
||||
- Card contract: PASS
|
||||
- Selector demo: PASS
|
||||
- Selector regression: PASS
|
||||
- Selector calibration smoke: PASS, 95 inputs, 0 failures, 0 warnings
|
||||
- Model/card sync: PASS
|
||||
|
||||
## Important Interpretation
|
||||
|
||||
Validation pass does not mean model content is stable or owner-approved.
|
||||
|
||||
It only means the Round 03.2a selector precedence patch is structurally consistent and passes current local checks.
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# Round 03.2a Review Questions for GPT
|
||||
|
||||
Please review Round 03.2a only.
|
||||
|
||||
Do not expand into Round 04 blind testing yet.
|
||||
|
||||
## Questions
|
||||
|
||||
1. Does the selector now correctly treat `不要展开 / 不要深入分析 + 只做 QPI / 只做问题定性` as QPI-limited analysis rather than global no-call?
|
||||
2. Does IA remain properly rejected in all five depth-limiting override cases?
|
||||
3. Is it acceptable that depth-limiting phrases without QPI-limited intent still remain no-call / low-depth signals?
|
||||
4. Can Round 03 now be closed with the following conclusion?
|
||||
|
||||
```text
|
||||
QPI and Intellectual Archaeology remain draft / B / pending.
|
||||
QPI is accepted as a draft-callable routing asset at report level.
|
||||
Selector v0.2 passes no-call, calibration smoke, model-level exclusion, QPI-before-IA, and depth-limiting override gates.
|
||||
Proceed to Round 04 blind input evaluation.
|
||||
```
|
||||
|
||||
## Suggested Answer Format
|
||||
|
||||
Please answer with:
|
||||
|
||||
- `pass`, `revise`, or `block`;
|
||||
- specific failing cases if `revise` or `block`;
|
||||
- whether Round 04 blind input evaluation may start after this patch.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
# Round 03.2a Bundle File Manifest
|
||||
|
||||
## Review Files
|
||||
|
||||
- `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF_03.2a.md`
|
||||
- `01_PATCH_MATRIX_03.2a.md`
|
||||
- `02_CURRENT_ASSET_PACK_03.2a.md`
|
||||
- `03_VALIDATION_AND_COMMAND_LOG_03.2a.md`
|
||||
- `04_REVIEW_QUESTIONS_FOR_GPT_03.2a.md`
|
||||
- `BUNDLE_FILE_MANIFEST_03.2a.md`
|
||||
- `optional_raw_changed_files_03.2a.zip`
|
||||
|
||||
## Optional Raw Changed Files
|
||||
|
||||
The zip is optional and should be used only if exact file inspection is needed.
|
||||
|
||||
It preserves source-relative paths.
|
||||
|
||||
It excludes:
|
||||
|
||||
- `knowledge_assets/`
|
||||
- `__pycache__/`
|
||||
- `*.pyc`
|
||||
- previous review bundle zip files
|
||||
|
||||
## Source-Relative Files Included in Raw Zip
|
||||
|
||||
- `README.md`
|
||||
- `cards/qpi.md`
|
||||
- `docs/DECISIONS.md`
|
||||
- `reports/content_review_report_v0.2.md`
|
||||
- `reports/Codex新会话交接文档_Round03.2a评审准备_2026-06-17.md`
|
||||
- `reports/index_rebuild_report.md`
|
||||
- `reports/model_card_sync_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/validation_report.md`
|
||||
- `scripts/run_selector_demo.py`
|
||||
- `selector/README.md`
|
||||
- `selector/selector_calibration_inputs.json`
|
||||
- `selector/selector_rules.json`
|
||||
- `tests/qpi.regression.json`
|
||||
- `tests/regression_cases.json`
|
||||
- `tests/regression_index.json`
|
||||
- `models/model_index.json`
|
||||
- `cards/card_index.md`
|
||||
Binary file not shown.
|
|
@ -156,5 +156,61 @@ This keeps second-round and third-round materials comparable without overwriting
|
|||
|
||||
Round 03.1 added two bundle hygiene rules:
|
||||
|
||||
- `optional_raw_changed_files.zip` must preserve source-relative paths. Do not create flat archives that can collide on names such as `README.md`.
|
||||
- `optional_raw_changed_files_<round-version>.zip` must preserve source-relative paths. Do not create flat archives that can collide on names such as `README.md`.
|
||||
- `knowledge_assets/` is excluded from review bundles by default because the project owner manually syncs stable long-term knowledge assets into GPT knowledge storage. Include it only when the current review explicitly targets the knowledge asset itself.
|
||||
|
||||
Round 03.2 added one ChatGPT upload hygiene rule:
|
||||
|
||||
- Every file intended for GPT / CCRA upload inside a review bundle must include the round version in the filename, such as `_03.2` or `03.2`, because uploading same-named files from multiple rounds in one Web ChatGPT conversation can cause file-reading ambiguity. This applies especially to raw changed-file zip archives, e.g. `optional_raw_changed_files_03.2.zip`.
|
||||
- The version suffix belongs to the bundle file names. Zip entries should still preserve source-relative repository paths so reviewers can inspect actual changed files without flattening path context.
|
||||
|
||||
## Decision 014: Treat explicit model refusal as model-level hard exclusion
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Reason:
|
||||
|
||||
Round 03.1 fixed QPI default over-selection, but GPT review found that explicit Intellectual Archaeology refusal phrases such as `先不要思想考古` could still lose to positive IA signals and QPI-completed bonuses.
|
||||
|
||||
A user instruction that names a model and refuses it should not be represented as a normal score penalty. It must hard-exclude that model unless a later owner-approved rule defines an explicit reversal phrase.
|
||||
|
||||
The current Round 03.2 implementation keeps this scoped:
|
||||
|
||||
- It rejects only the explicitly refused model.
|
||||
- It does not create a global no-call.
|
||||
- It does not add an LLM selector, RAG, database, service, frontend, backend, or third model.
|
||||
- It preserves QPI explicit-analysis override, but QPI selection does not re-enable IA when IA is explicitly refused.
|
||||
- Generic bare QPI gate words such as `如何` and `判断` are removed in favor of compound problem-definition phrases.
|
||||
|
||||
## Decision 015: Treat depth-limiting QPI requests as scoped analysis, not global no-call
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Reason:
|
||||
|
||||
Round 03.2 review found that global hard no-call phrases such as `不要展开` and `不要深入分析` could incorrectly block QPI when the user also asked for a limited QPI action such as `只做 QPI`, `只做问题定性`, or `只判断主导稀缺`.
|
||||
|
||||
The correct interpretation is scoped depth control:
|
||||
|
||||
- `不要展开` or `不要深入分析` alone remains a no-call / low-depth signal.
|
||||
- When paired with explicit QPI-limited intent, it means do not enter deeper models such as Intellectual Archaeology.
|
||||
- QPI may still be selected for problem definition, problem-type judgment, or dominant-scarcity judgment.
|
||||
- IA remains rejected through model-level hard exclusion.
|
||||
|
||||
This remains a selector precedence rule only. It does not add a new model, lifecycle upgrade, LLM selector, RAG, database, service, frontend, backend, user system, or Round 04 blind-test behavior.
|
||||
|
||||
## Decision 016: Close Round 03 after the Round 03.2a pass decision
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Reason:
|
||||
|
||||
The Round 03.2a review accepted the depth-limiting QPI override patch and confirmed that Round 03 can close.
|
||||
|
||||
Closure statement:
|
||||
|
||||
- Round 03 is closed.
|
||||
- QPI and Intellectual Archaeology remain `draft / B / pending`.
|
||||
- QPI is accepted as a draft-callable routing asset at report level only; this does not change JSON model lifecycle status.
|
||||
- Selector v0.2 passes no-call, calibration smoke, model-level exclusion, QPI-before-IA, and depth-limiting override gates.
|
||||
- Round 04 blind input routing evaluation is the next candidate phase, but it is not started by this closure.
|
||||
|
|
|
|||
|
|
@ -97,7 +97,8 @@ Rules:
|
|||
|
||||
- Keep review bundles under dated round directories.
|
||||
- Do not place new review files directly under `ccra_review_bundle/`.
|
||||
- `optional_raw_changed_files.zip` must preserve source-relative paths.
|
||||
- `optional_raw_changed_files_<round-version>.zip` must preserve source-relative paths.
|
||||
- GPT / CCRA upload files inside review bundles must include the round version in the filename, for example `_03.2`, to avoid same-name ambiguity when multiple rounds are uploaded in one Web ChatGPT conversation.
|
||||
- Do not flatten zip contents, because duplicate filenames such as `README.md` can collide.
|
||||
- `knowledge_assets/` is excluded from review zips by default because the Owner manually syncs stable knowledge assets into GPT knowledge storage.
|
||||
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@
|
|||
"card_file": "cards/intellectual_archaeology.md",
|
||||
"source_article_count": 2,
|
||||
"source_evidence_count": 5,
|
||||
"regression_case_count": 17,
|
||||
"regression_case_count": 19,
|
||||
"stability_level": "B",
|
||||
"regression_status": "pending",
|
||||
"status": "draft",
|
||||
|
|
@ -26,7 +26,7 @@
|
|||
"card_file": "cards/qpi.md",
|
||||
"source_article_count": 3,
|
||||
"source_evidence_count": 10,
|
||||
"regression_case_count": 52,
|
||||
"regression_case_count": 58,
|
||||
"stability_level": "B",
|
||||
"regression_status": "pending",
|
||||
"status": "draft",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,112 @@
|
|||
# Codex 新会话交接文档:Round 03.2a 评审准备与关闭记录
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||
|
||||
Current phase: `model_library_mvp`
|
||||
|
||||
## 1. 本轮目标
|
||||
|
||||
根据 GPT 对 Round 03.2 的 `REVISE-light` 评审意见,完成 03.2a 微修:
|
||||
|
||||
```text
|
||||
不要展开 / 不要深入分析
|
||||
+
|
||||
只做 QPI / 只做问题定性 / 只判断主导稀缺
|
||||
=
|
||||
允许 QPI,拒绝 IA,不做全局 no-call
|
||||
```
|
||||
|
||||
## 2. 已完成
|
||||
|
||||
- 新增 `depth_limiting_no_call_signals`。
|
||||
- 新增 `qpi_limited_analysis_override_phrases`。
|
||||
- 修改 selector hard no-call precedence:
|
||||
- depth-limiting phrase 单独出现时仍可 no-call;
|
||||
- depth-limiting phrase 与 QPI-limited intent 同时出现时继续评分;
|
||||
- QPI 可被选中;
|
||||
- IA 仍通过 hard exclusion 被拒绝。
|
||||
- 避免 QPI 自身 negative trigger penalty 在该组合下把 QPI 压到阈值以下。
|
||||
- 新增 3 条 QPI regression:
|
||||
- `case_qpi_depth_limited_qpi_only_001`
|
||||
- `case_qpi_depth_limited_problem_definition_001`
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`
|
||||
- 新增 5 条 selector calibration input。
|
||||
- 同步 aggregate regression、regression index、model/card index 和 reports。
|
||||
- 生成 03.2a 评审包:
|
||||
|
||||
```text
|
||||
ccra_review_bundle/round-03.2a_2026-06-17_depth-limiting-qpi-override-patch/
|
||||
```
|
||||
|
||||
## 3. 当前计数
|
||||
|
||||
- QPI regression cases: 58
|
||||
- Intellectual Archaeology regression cases: 19
|
||||
- Aggregate regression cases: 77
|
||||
- Selector calibration inputs: 95
|
||||
|
||||
模型状态没有升级:
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
||||
## 4. 验证结果
|
||||
|
||||
以下命令已通过:
|
||||
|
||||
```powershell
|
||||
python scripts\rebuild_indexes.py --check
|
||||
python -m unittest discover -s tests -p "test*.py" -v
|
||||
python scripts\validate_model_library.py
|
||||
python scripts\check_card_contract.py
|
||||
python scripts\run_selector_demo.py
|
||||
python scripts\run_selector_regression.py
|
||||
python scripts\run_selector_calibration_smoke.py
|
||||
python scripts\check_model_card_sync.py
|
||||
```
|
||||
|
||||
结果:
|
||||
|
||||
- Index check: PASS
|
||||
- Unit tests: PASS, 17 tests
|
||||
- Model library validation: PASS
|
||||
- Card contract: PASS
|
||||
- Selector demo: PASS
|
||||
- Selector regression: PASS
|
||||
- Selector calibration smoke: PASS, 95 inputs, 0 failures, 0 warnings
|
||||
- Model/card sync: PASS
|
||||
|
||||
## 5. 需要 GPT / Owner 判断
|
||||
|
||||
- Round 03.2a 已经由 GPT / CCRA 判断为 `pass`。
|
||||
- Round 03 系列可以正式关闭。
|
||||
- Round 04 blind input evaluation 可以作为下一候选阶段,但本次关闭动作不进入 Round 04。
|
||||
|
||||
## 6. 建议下一步
|
||||
|
||||
1. 记录 Round 03 closure。
|
||||
2. 提交并同步 Round 03.2a 工程资产、评审包、交接文档和关闭记录。
|
||||
3. 仅在后续 Owner 明确要求时启动 Round 04 blind input evaluation;启动前不修改 selector 规则。
|
||||
|
||||
## 7. Round 03 关闭结论
|
||||
|
||||
```text
|
||||
Round 03 closed.
|
||||
|
||||
QPI and Intellectual Archaeology remain draft / B / pending.
|
||||
QPI is accepted as a draft-callable routing asset at report level.
|
||||
Selector v0.2 passes no-call, calibration smoke, model-level exclusion, QPI-before-IA, and depth-limiting override gates.
|
||||
Round 04 blind input routing evaluation is not started in this closeout.
|
||||
```
|
||||
|
||||
注意:`draft-callable` 只能作为评审报告语言,不改变 JSON model lifecycle status。
|
||||
|
||||
## 8. 本次关闭修复
|
||||
|
||||
- 按评审意见在 `cards/qpi.md` 补充 depth-limiting override 的说明:`不要展开` / `不要深入分析` 默认抑制 QPI,但与 `只做 QPI`、`只做问题定性`、`只判断主导稀缺` 等限定分析意图组合时,允许 QPI 窄范围判断并拒绝思想考古。
|
||||
- 在 `docs/DECISIONS.md` 记录 Round 03 关闭决策。
|
||||
- 在 `README.md` 更新下一步状态:Round 03 已关闭,Round 04 只作为后续候选阶段,不在本轮启动。
|
||||
|
|
@ -0,0 +1,125 @@
|
|||
# Codex 新会话交接文档:Round 03.2 评审准备
|
||||
|
||||
Date: 2026-06-17
|
||||
|
||||
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||
|
||||
Current phase: `model_library_mvp`
|
||||
|
||||
## 1. 本轮目标
|
||||
|
||||
根据 GPT 对 Round 03.1 的 `REVISE` 评审意见,完成一个极小的 Round 03.2 selector negative-gate patch,并准备 03.2 评审资料。
|
||||
|
||||
## 2. 已完成
|
||||
|
||||
- 将 IA / 思想考古的明确拒绝语从普通 penalty 升级为 model-level hard exclusion。
|
||||
- 保留 QPI 显式分析 / 复核入口,但不允许 QPI 选择重新启用被明确拒绝的 IA。
|
||||
- 移除 QPI gate 中裸 `如何` 和裸 `判断`。
|
||||
- 增加复合 problem-definition 信号,避免普通 how-to / 词义判断误召回 QPI。
|
||||
- 加严 `scripts/run_selector_calibration_smoke.py`:
|
||||
- `no_call` 必须 `selected_models == []` 且 `no_call == true`;
|
||||
- `no_call_or_low_priority` 不允许 QPI 或 IA 被选中;
|
||||
- `select_qpi_reject_ia` 必须选中 QPI 且拒绝 IA;
|
||||
- `select_intellectual_archaeology` 如果命中 IA hard exclusion 则失败。
|
||||
- 增加 5 条 GPT 指定陷阱:
|
||||
- `case_ia_negative_explicit_do_not_call_001`
|
||||
- `case_ia_negative_only_qpi_001`
|
||||
- `case_qpi_how_to_excel_no_call_001`
|
||||
- `case_qpi_how_to_markdown_pdf_no_call_001`
|
||||
- `case_qpi_dictionary_judgment_no_call_001`
|
||||
- 同步 per-model regression、aggregate regression、regression index、model/card index 和相关报告。
|
||||
- 生成 Round 03.2 评审包:
|
||||
|
||||
```text
|
||||
ccra_review_bundle/round-03.2_2026-06-17_selector-negative-gate-patch/
|
||||
```
|
||||
- Round 03.2 评审包内上传给 GPT / CCRA 的文件名均带 `03.2` 版本号;raw zip 命名为 `optional_raw_changed_files_03.2.zip`,zip 内保持 source-relative paths。
|
||||
|
||||
## 3. 当前计数
|
||||
|
||||
- QPI regression cases: 55
|
||||
- Intellectual Archaeology regression cases: 19
|
||||
- Aggregate regression cases: 74
|
||||
- Selector calibration inputs: 90
|
||||
|
||||
模型状态没有升级:
|
||||
|
||||
```text
|
||||
qpi: draft / B / pending
|
||||
intellectual_archaeology: draft / B / pending
|
||||
```
|
||||
|
||||
## 4. 主要变更文件
|
||||
|
||||
- `selector/selector_rules.json`
|
||||
- `scripts/run_selector_demo.py`
|
||||
- `scripts/run_selector_calibration_smoke.py`
|
||||
- `selector/selector_calibration_inputs.json`
|
||||
- `tests/qpi.regression.json`
|
||||
- `tests/intellectual_archaeology.regression.json`
|
||||
- `tests/regression_cases.json`
|
||||
- `tests/regression_index.json`
|
||||
- `models/model_index.json`
|
||||
- `cards/card_index.md`
|
||||
- `cards/qpi.md`
|
||||
- `cards/intellectual_archaeology.md`
|
||||
- `docs/DECISIONS.md`
|
||||
- `README.md`
|
||||
- `selector/README.md`
|
||||
- `scripts/README.md`
|
||||
- `reports/README.md`
|
||||
- `reports/content_review_report_v0.2.md`
|
||||
- generated reports under `reports/`
|
||||
- Round 03.2 review bundle under `ccra_review_bundle/round-03.2_2026-06-17_selector-negative-gate-patch/`
|
||||
|
||||
## 5. 验证结果
|
||||
|
||||
以下命令已通过:
|
||||
|
||||
```powershell
|
||||
python scripts\rebuild_indexes.py --check
|
||||
python -m unittest discover -s tests -p "test*.py" -v
|
||||
python scripts\validate_model_library.py
|
||||
python scripts\check_card_contract.py
|
||||
python scripts\run_selector_demo.py
|
||||
python scripts\run_selector_regression.py
|
||||
python scripts\run_selector_calibration_smoke.py
|
||||
python scripts\check_model_card_sync.py
|
||||
```
|
||||
|
||||
结果:
|
||||
|
||||
- Index check: PASS
|
||||
- Unit tests: PASS, 17 tests
|
||||
- Model library validation: PASS
|
||||
- Card contract: PASS
|
||||
- Selector demo: PASS
|
||||
- Selector regression: PASS
|
||||
- Selector calibration smoke: PASS, 90 inputs, 0 failures, 0 warnings
|
||||
- Model/card sync: PASS
|
||||
|
||||
## 6. 假设
|
||||
|
||||
- GPT 的 `Round 03.2 Selector Negative-Gate Patch` 指令是本轮唯一修复范围。
|
||||
- `只做 QPI` 和 `只做问题定性` 在当前上下文中表示拒绝 IA,同时允许 QPI。
|
||||
- `draft-callable` 只能作为评审报告语言,不写入模型 JSON status。
|
||||
|
||||
## 7. 仍不代表已经完成的事
|
||||
|
||||
- 没有进入 Round 04 blind test。
|
||||
- 没有证明 QPI 或 IA 内容已稳定。
|
||||
- 没有升级 `status`、`stability_level` 或 `regression_status`。
|
||||
- 没有新增第三模型、LLM selector、RAG、数据库、前后端或完整问答系统。
|
||||
|
||||
## 8. 需要 GPT / Owner 判断
|
||||
|
||||
- Round 03.2 是否 `pass / revise / block`。
|
||||
- Round 03 是否可以正式关闭。
|
||||
- 是否允许进入 Round 04 blind input evaluation。
|
||||
- 当前 IA hard exclusion 词表是否足够,是否需要定义“用户反转前文拒绝”的显式规则。
|
||||
|
||||
## 9. 建议下一步
|
||||
|
||||
1. 将 Round 03.2 review bundle 提交给 GPT。
|
||||
2. 如果 GPT pass,关闭 Round 03 系列。
|
||||
3. 仅在 GPT / Owner 同意后启动 Round 04 blind input evaluation。
|
||||
|
|
@ -10,6 +10,7 @@ Current reports:
|
|||
- `baseline_before_content_stabilization_v0.2.md`
|
||||
- `evidence_coverage_report_v0.2.md`
|
||||
- `selector_regression_report_v0.2.md`
|
||||
- `selector_calibration_smoke_report.md`
|
||||
- `model_card_sync_report_v0.2.md`
|
||||
- `content_review_report_v0.2.md`
|
||||
- `model_review_status.json`
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Date: 2026-06-16
|
||||
|
||||
Status note: this report is a pre-case-promotion review snapshot. It remains useful for the v0.2 evidence / contract / selector baseline, but its regression counts do not reflect the Round 03 / 03.1 QPI owner-reviewed case promotion. Current QPI regression counts are tracked in `tests/qpi.regression.json`, `models/model_index.json`, and the Round 03.1 review bundle.
|
||||
Status note: this report is a pre-case-promotion review snapshot. It remains useful for the v0.2 evidence / contract / selector baseline, but its regression counts do not reflect the Round 03 / 03.1 QPI owner-reviewed case promotion, the Round 03.2 selector negative-gate patch, or the Round 03.2a depth-limiting override patch. Current regression counts are tracked in `tests/*.regression.json`, `models/model_index.json`, and the latest review bundle.
|
||||
|
||||
## 1. 本轮修改摘要
|
||||
|
||||
|
|
@ -111,7 +111,7 @@ Coverage:
|
|||
- 已覆盖误用、防误召回、no-call 和 pipeline gate。
|
||||
- 仍需人工审查用例真实性和遗漏边界。
|
||||
|
||||
Round 03 / 03.1 update: QPI has since expanded to 52 regression cases after owner-reviewed case promotion and selector/no-call repair. This v0.2 table is retained as a historical baseline, not the current count.
|
||||
Round 03 / 03.1 / 03.2 / 03.2a update: QPI has since expanded to 58 regression cases after owner-reviewed case promotion, selector/no-call repair, broad-gate no-call traps, and depth-limiting QPI override traps. Intellectual Archaeology has expanded to 19 regression cases after Round 03.2 IA hard-exclusion traps. This v0.2 table is retained as a historical baseline, not the current count.
|
||||
|
||||
## 6. Selector Regression 结论
|
||||
|
||||
|
|
@ -124,10 +124,13 @@ Selector v0.2 仍为规则 selector。
|
|||
- QPI-before-IA gate
|
||||
- score breakdown
|
||||
- rejected model reasons
|
||||
- model-level hard exclusion for explicit model refusal
|
||||
- stricter calibration smoke checks for no-call, no-call-or-low-priority, and select-qpi-reject-ia
|
||||
|
||||
报告位于:
|
||||
|
||||
- `reports/selector_regression_report_v0.2.md`
|
||||
- `reports/selector_calibration_smoke_report.md`
|
||||
|
||||
结论:
|
||||
|
||||
|
|
|
|||
|
|
@ -4,8 +4,9 @@ Status: `PASS`
|
|||
|
||||
Command: `python scripts/run_selector_calibration_smoke.py`
|
||||
|
||||
Calibration inputs checked: 85
|
||||
Calibration inputs checked: 95
|
||||
Failures: 0
|
||||
Warnings: 0
|
||||
|
||||
## Cases
|
||||
|
||||
|
|
@ -93,4 +94,14 @@ Failures: 0
|
|||
- `selector_calibration_qpi_research_safety_accountability_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_research_penalty_integrity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_direct_summary_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||
- `selector_calibration_ia_negative_explicit_do_not_call_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_ia_negative_only_qpi_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_how_to_excel_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||
- `selector_calibration_qpi_how_to_markdown_pdf_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||
- `selector_calibration_qpi_dictionary_judgment_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||
- `selector_calibration_qpi_depth_limited_qpi_only_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_problem_definition_002`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_problem_or_issue_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_qpi_depth_limited_dominant_scarcity_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||
- `selector_calibration_analysis_override_should_call_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
Status: `PASS`
|
||||
|
||||
Cases checked: 13
|
||||
Cases checked: 21
|
||||
Failures: 0
|
||||
|
||||
## Cases
|
||||
|
|
@ -13,10 +13,18 @@ Failures: 0
|
|||
- `case_ia_selector_gate_qpi_first_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_ia_selector_gate_no_heavy_signal_001`: PASS; selected=[]; no_call=True
|
||||
- `case_ia_pipeline_after_qpi_001`: PASS; selected=['intellectual_archaeology', 'qpi']; no_call=False
|
||||
- `case_ia_negative_explicit_do_not_call_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_ia_negative_only_qpi_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_no_call_direct_edit_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_selector_gate_fact_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_pipeline_before_ia_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_disappointment_mismatch_diagnostics_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_international_logistics_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_direct_summary_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_how_to_excel_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_how_to_markdown_pdf_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_dictionary_judgment_no_call_001`: PASS; selected=[]; no_call=True
|
||||
- `case_qpi_depth_limited_qpi_only_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_depth_limited_problem_definition_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_depth_limited_dominant_scarcity_001`: PASS; selected=['qpi']; no_call=False
|
||||
- `case_qpi_analysis_override_should_call_001`: PASS; selected=['qpi']; no_call=False
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ Current script:
|
|||
- `check_model_card_sync.py`
|
||||
- `run_selector_regression.py`
|
||||
- `run_selector_demo.py`
|
||||
- `run_selector_calibration_smoke.py`
|
||||
|
||||
Run:
|
||||
|
||||
|
|
@ -44,7 +45,10 @@ Run selector regression and model/card sync checks:
|
|||
|
||||
```powershell
|
||||
python scripts\run_selector_regression.py
|
||||
python scripts\run_selector_calibration_smoke.py
|
||||
python scripts\check_model_card_sync.py
|
||||
```
|
||||
|
||||
`run_selector_calibration_smoke.py` checks selector calibration inputs, including strict `no_call`, `no_call_or_low_priority`, `select_qpi_reject_ia`, and IA heavy-depth gate expectations.
|
||||
|
||||
Prefer Python standard library before adding dependencies.
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import json
|
|||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from run_selector_demo import load_selector_rules, recommend
|
||||
from run_selector_demo import load_selector_rules, model_hard_exclusion_hits, recommend
|
||||
|
||||
|
||||
def read_json(path):
|
||||
|
|
@ -24,9 +24,24 @@ def evaluate_input(root, selector_rules, item):
|
|||
selected_ids = selected_model_ids(result)
|
||||
expected = item.get("expected_selector_behavior")
|
||||
errors = []
|
||||
warnings = []
|
||||
|
||||
if expected in {"no_call", "no_call_or_low_priority"} and "qpi" in selected_ids:
|
||||
errors.append("expected no QPI selection")
|
||||
model_rules = {
|
||||
model_rule["model_id"]: model_rule
|
||||
for model_rule in selector_rules.get("models", [])
|
||||
}
|
||||
|
||||
if expected == "no_call":
|
||||
if selected_ids:
|
||||
errors.append(f"expected no selected models, got {selected_ids}")
|
||||
if result.get("no_call") is not True:
|
||||
errors.append("expected no_call=true")
|
||||
|
||||
if expected == "no_call_or_low_priority":
|
||||
if "qpi" in selected_ids:
|
||||
errors.append("expected no QPI selection")
|
||||
if "intellectual_archaeology" in selected_ids:
|
||||
errors.append("expected no IA selection")
|
||||
|
||||
if expected in {"select_qpi", "select_qpi_low_confidence", "select_qpi_reject_ia"}:
|
||||
if "qpi" not in selected_ids:
|
||||
|
|
@ -40,6 +55,15 @@ def evaluate_input(root, selector_rules, item):
|
|||
errors.append("expected IA selection")
|
||||
if not has_ia_gate(item["input"], selector_rules):
|
||||
errors.append("IA selected expectation lacks heavy-depth or QPI-completed gate")
|
||||
ia_hard_exclusion_hits = model_hard_exclusion_hits(
|
||||
item["input"],
|
||||
model_rules.get("intellectual_archaeology", {}),
|
||||
)
|
||||
if ia_hard_exclusion_hits:
|
||||
errors.append("IA selection expected despite IA hard exclusion signal")
|
||||
|
||||
if expected == "select_qpi" and set(selected_ids) - {"qpi"}:
|
||||
warnings.append("unexpected extra selected models: " + ", ".join(sorted(set(selected_ids) - {"qpi"})))
|
||||
|
||||
return {
|
||||
"case_id": item["case_id"],
|
||||
|
|
@ -47,12 +71,14 @@ def evaluate_input(root, selector_rules, item):
|
|||
"selected_models": selected_ids,
|
||||
"no_call": result.get("no_call"),
|
||||
"errors": errors,
|
||||
"warnings": warnings,
|
||||
}
|
||||
|
||||
|
||||
def write_report(root, results):
|
||||
report_path = root / "reports" / "selector_calibration_smoke_report.md"
|
||||
failures = [result for result in results if result["errors"]]
|
||||
warnings = [result for result in results if result["warnings"]]
|
||||
lines = [
|
||||
"# Selector Calibration Smoke Report",
|
||||
"",
|
||||
|
|
@ -62,6 +88,7 @@ def write_report(root, results):
|
|||
"",
|
||||
f"Calibration inputs checked: {len(results)}",
|
||||
f"Failures: {len(failures)}",
|
||||
f"Warnings: {len(warnings)}",
|
||||
"",
|
||||
"## Cases",
|
||||
"",
|
||||
|
|
@ -74,6 +101,8 @@ def write_report(root, results):
|
|||
)
|
||||
for error in result["errors"]:
|
||||
lines.append(f" - {error}")
|
||||
for warning in result["warnings"]:
|
||||
lines.append(f" - WARNING: {warning}")
|
||||
report_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
return report_path
|
||||
|
||||
|
|
|
|||
|
|
@ -33,6 +33,16 @@ def hard_no_call_hits(user_input, global_rules):
|
|||
return hit_any(user_input, signals)
|
||||
|
||||
|
||||
def depth_limited_qpi_override_hits(user_input, global_rules):
|
||||
depth_hits = hit_any(user_input, global_rules.get("depth_limiting_no_call_signals", []))
|
||||
override_hits = hit_any(user_input, global_rules.get("qpi_limited_analysis_override_phrases", []))
|
||||
return depth_hits, override_hits
|
||||
|
||||
|
||||
def model_hard_exclusion_hits(user_input, model_rule):
|
||||
return hit_any(user_input, model_rule.get("hard_exclusion_triggers", []))
|
||||
|
||||
|
||||
def score_model(model, model_rule, global_rules, user_input, task_type="", pipeline_position=""):
|
||||
model_id = model.get("model_id")
|
||||
weights = global_rules.get("weights", {})
|
||||
|
|
@ -41,7 +51,9 @@ def score_model(model, model_rule, global_rules, user_input, task_type="", pipel
|
|||
penalties = []
|
||||
|
||||
negative_hits = hit_any(user_input, model_rule.get("negative_triggers", [])) + hit_any(user_input, model.get("negative_triggers", []))
|
||||
if negative_hits and not has_analysis_override(user_input, global_rules):
|
||||
depth_hits, qpi_limited_hits = depth_limited_qpi_override_hits(user_input, global_rules)
|
||||
suppress_qpi_depth_penalty = model_id == "qpi" and depth_hits and qpi_limited_hits
|
||||
if negative_hits and not has_analysis_override(user_input, global_rules) and not suppress_qpi_depth_penalty:
|
||||
penalty_key = "model_negative_penalty_qpi" if model_id == "qpi" else "model_negative_penalty_default"
|
||||
score -= weights.get(penalty_key, 0.5)
|
||||
penalties.append("negative trigger: " + "、".join(sorted(set(negative_hits))[:4]))
|
||||
|
|
@ -142,14 +154,28 @@ def recommend(root, user_input, task_type="", pipeline_position="", threshold=No
|
|||
threshold = global_rules.get("no_call_threshold", 0.35) if threshold is None else threshold
|
||||
|
||||
no_call_hits = hard_no_call_hits(user_input, global_rules)
|
||||
if global_rules.get("hard_no_call_first", True) and no_call_hits and not has_analysis_override(user_input, global_rules):
|
||||
depth_hits, qpi_limited_hits = depth_limited_qpi_override_hits(user_input, global_rules)
|
||||
has_depth_limited_qpi_override = bool(depth_hits and qpi_limited_hits)
|
||||
if (
|
||||
global_rules.get("hard_no_call_first", True)
|
||||
and no_call_hits
|
||||
and not has_analysis_override(user_input, global_rules)
|
||||
and not has_depth_limited_qpi_override
|
||||
):
|
||||
return hard_no_call_result(root, user_input, no_call_hits)
|
||||
|
||||
model_rules = rules_by_model(selector_rules)
|
||||
scored = [
|
||||
score_model(model, model_rules.get(model.get("model_id"), {}), global_rules, user_input, task_type, pipeline_position)
|
||||
for model in load_models(root)
|
||||
]
|
||||
scored = []
|
||||
for model in load_models(root):
|
||||
model_id = model.get("model_id")
|
||||
model_rule = model_rules.get(model_id, {})
|
||||
item = score_model(model, model_rule, global_rules, user_input, task_type, pipeline_position)
|
||||
exclusion_hits = model_hard_exclusion_hits(user_input, model_rule)
|
||||
if exclusion_hits:
|
||||
item["score"] = 0.0
|
||||
item["hard_excluded"] = True
|
||||
item["penalties"].append("model hard exclusion signal: " + "、".join(sorted(set(exclusion_hits))[:4]))
|
||||
scored.append(item)
|
||||
no_call = all(item["score"] < threshold for item in scored)
|
||||
selected = []
|
||||
rejected = []
|
||||
|
|
@ -160,7 +186,7 @@ def recommend(root, user_input, task_type="", pipeline_position="", threshold=No
|
|||
"score": item["score"],
|
||||
"reasons": item["reasons"],
|
||||
"penalties": item["penalties"],
|
||||
"decision": "selected" if item["score"] >= threshold and not no_call else "rejected"
|
||||
"decision": "selected" if item["score"] >= threshold and not no_call and not item.get("hard_excluded") else "rejected"
|
||||
}
|
||||
if output_item["decision"] == "selected":
|
||||
selected.append(output_item)
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ It should use simple matching rules:
|
|||
- Trigger keywords
|
||||
- Input types
|
||||
- Negative triggers
|
||||
- Model-level hard exclusion triggers
|
||||
- Pipeline position
|
||||
- Selection priority
|
||||
- Negative trigger first
|
||||
|
|
@ -18,6 +19,17 @@ It should use simple matching rules:
|
|||
- Direct-execution no-call signals
|
||||
- Calibration smoke checks
|
||||
|
||||
Round 03.2 selector rule:
|
||||
|
||||
- Explicit refusal of a specific model, such as `不要思想考古` or `只做 QPI`, is a model-level hard exclusion for that model.
|
||||
- A hard exclusion rejects only the named model; it does not globally no-call the request.
|
||||
- Generic bare QPI gate words such as `如何` and `判断` should not be used alone. Use compound problem-definition phrases instead.
|
||||
|
||||
Round 03.2a selector rule:
|
||||
|
||||
- Depth-limiting phrases such as `不要展开` and `不要深入分析` remain global no-call signals by default.
|
||||
- If a depth-limiting phrase is paired with explicit QPI-limited intent such as `只做 QPI`, `只做问题定性`, or `只判断主导稀缺`, the selector should continue scoring, allow QPI, and still reject IA.
|
||||
|
||||
Current files:
|
||||
|
||||
- `selector_rules.json`
|
||||
|
|
|
|||
|
|
@ -591,6 +591,76 @@
|
|||
"expected_selector_behavior": "no_call",
|
||||
"expected_notes": "Round 03.1 direct summary request must not select QPI by default score."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_ia_negative_explicit_do_not_call_001",
|
||||
"category": "round_03_2_model_level_negative_gate",
|
||||
"input": "QPI 已判断:这是 Issue。先不要思想考古,只做问题定性复核。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Explicit IA refusal must be a model-level hard exclusion, not a score penalty."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_ia_negative_only_qpi_001",
|
||||
"category": "round_03_2_model_level_negative_gate",
|
||||
"input": "不要思想考古,只做 QPI 复核。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Only-QPI instruction should reject IA even when IA terms appear."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_how_to_excel_no_call_001",
|
||||
"category": "round_03_2_broad_qpi_gate_no_call",
|
||||
"input": "如何在 Excel 里冻结首行?",
|
||||
"expected_selector_behavior": "no_call",
|
||||
"expected_notes": "Generic tool how-to should not be routed to QPI by bare 如何."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_how_to_markdown_pdf_no_call_001",
|
||||
"category": "round_03_2_broad_qpi_gate_no_call",
|
||||
"input": "如何把 Markdown 转 PDF?",
|
||||
"expected_selector_behavior": "no_call",
|
||||
"expected_notes": "Generic conversion how-to should not be routed to QPI by bare 如何."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_dictionary_judgment_no_call_001",
|
||||
"category": "round_03_2_broad_qpi_gate_no_call",
|
||||
"input": "判断这个词是什么意思。",
|
||||
"expected_selector_behavior": "no_call",
|
||||
"expected_notes": "Dictionary meaning request should not be routed to QPI by bare 判断."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_qpi_only_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要展开到思想考古,只做 QPI 复核。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Depth-limiting phrase should not become global no-call when QPI-only intent is explicit."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_problem_definition_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要展开到思想考古,只做问题定性。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Do not call IA, but still route to QPI for problem definition."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_problem_definition_002",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要深入分析,只做问题定性。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Depth-limiting no-call phrase should be overridden by explicit QPI-limited problem definition."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_problem_or_issue_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要深入分析背后的理论,只判断这是问题还是课题。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Problem-or-issue judgment is QPI-limited analysis, not global no-call."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_qpi_depth_limited_dominant_scarcity_001",
|
||||
"category": "round_03_2a_depth_limited_qpi_override",
|
||||
"input": "不要展开,只判断主导稀缺。",
|
||||
"expected_selector_behavior": "select_qpi_reject_ia",
|
||||
"expected_notes": "Dominant-scarcity-only judgment should select QPI and reject IA."
|
||||
},
|
||||
{
|
||||
"case_id": "selector_calibration_analysis_override_should_call_001",
|
||||
"category": "round_03_1_analysis_override",
|
||||
|
|
|
|||
|
|
@ -35,6 +35,18 @@
|
|||
"不用解释",
|
||||
"生成图片"
|
||||
],
|
||||
"depth_limiting_no_call_signals": [
|
||||
"不要展开",
|
||||
"不要深入分析"
|
||||
],
|
||||
"qpi_limited_analysis_override_phrases": [
|
||||
"只做 QPI",
|
||||
"只做问题定性",
|
||||
"只做问题定性复核",
|
||||
"只判断主导稀缺",
|
||||
"只判断这是问题还是课题",
|
||||
"只判断问题类型"
|
||||
],
|
||||
"direct_execution_no_call_signals": [
|
||||
"排一下",
|
||||
"排床位",
|
||||
|
|
@ -79,8 +91,15 @@
|
|||
"这是问题",
|
||||
"执行路径问题",
|
||||
"组织共识问题",
|
||||
"如何",
|
||||
"判断",
|
||||
"如何判断这是什么问题",
|
||||
"如何定义这个问题",
|
||||
"如何提高流量",
|
||||
"如何区分 Q/P/I",
|
||||
"判断主导稀缺",
|
||||
"判断这是问题还是课题",
|
||||
"判断应该调用哪个模型",
|
||||
"只做 QPI",
|
||||
"只做问题定性",
|
||||
"真实调用",
|
||||
"上下文",
|
||||
"审计边界",
|
||||
|
|
@ -92,6 +111,8 @@
|
|||
"行动杠杆",
|
||||
"依赖风险",
|
||||
"误路由",
|
||||
"真实独立 agent 调用记录",
|
||||
"正式输出",
|
||||
"过度升维",
|
||||
"负激励",
|
||||
"成本转嫁",
|
||||
|
|
@ -202,6 +223,18 @@
|
|||
"先不要思想考古",
|
||||
"不要思想考古"
|
||||
],
|
||||
"hard_exclusion_triggers": [
|
||||
"不要思想考古",
|
||||
"先不要思想考古",
|
||||
"不要调用思想考古",
|
||||
"不调用思想考古",
|
||||
"只做 QPI",
|
||||
"只做问题定性",
|
||||
"只做问题定性复核",
|
||||
"只判断主导稀缺",
|
||||
"只判断这是问题还是课题",
|
||||
"只判断问题类型"
|
||||
],
|
||||
"pipeline_position": "modeling_depth_analysis",
|
||||
"selection_priority": 7,
|
||||
"routing_notes": "通常在 QPI 判断为中重型难题或课题后调用;遇到轻量事实检索应避免调用。"
|
||||
|
|
|
|||
|
|
@ -17,6 +17,8 @@
|
|||
{"case_id": "case_ia_misuse_layer_jump_001", "model_id": "intellectual_archaeology", "case_type": "misuse", "input": "我们还没弄清应用层事实,就想直接讨论哲学基岩。", "expected_behavior": "应阻止层级跳跃,从应用层事实和证据缺口开始。", "failure_signal": "跳过中间层级。", "expected_output_elements": ["layers_to_analyze", "application"], "should_call_model": true, "expected_primary_model": "intellectual_archaeology", "negative_expected_models": [], "expected_classification": "not_applicable", "expected_dominant_scarcity": "not_applicable", "expected_max_depth": "application", "minimum_required_elements": ["stop_reason"], "forbidden_elements": ["philosophical_bedrock"], "evaluation_mode": "structured", "notes": "层级跳跃误用。"},
|
||||
{"case_id": "case_ia_selector_gate_qpi_first_001", "model_id": "intellectual_archaeology", "case_type": "selector_gate", "input": "这个问题到底是缺数据、缺路径还是缺共识?", "expected_behavior": "selector 应先选 QPI,不应直接选思想考古。", "failure_signal": "IA 抢在 QPI 前。", "expected_output_elements": ["qpi first"], "should_call_model": false, "expected_primary_model": "qpi", "negative_expected_models": ["intellectual_archaeology"], "expected_classification": "not_applicable", "expected_dominant_scarcity": "not_applicable", "expected_max_depth": "no_call", "minimum_required_elements": ["rejected_models"], "forbidden_elements": ["selected IA"], "evaluation_mode": "structured", "notes": "QPI-before-IA gate。"},
|
||||
{"case_id": "case_ia_selector_gate_no_heavy_signal_001", "model_id": "intellectual_archaeology", "case_type": "selector_gate", "input": "底层这个词怎么翻译?只要一个英文。", "expected_behavior": "虽然有底层一词,也不应召回思想考古。", "failure_signal": "关键词误召回。", "expected_output_elements": ["rejected_models"], "should_call_model": false, "expected_primary_model": "none", "negative_expected_models": ["intellectual_archaeology"], "expected_classification": "not_applicable", "expected_dominant_scarcity": "not_applicable", "expected_max_depth": "no_call", "minimum_required_elements": ["negative trigger"], "forbidden_elements": ["七层"], "evaluation_mode": "structured", "notes": "关键词误召回防护。"},
|
||||
{"case_id": "case_ia_pipeline_after_qpi_001", "model_id": "intellectual_archaeology", "case_type": "pipeline", "input": "QPI 已判断为中重型课题,且需要抽取底层机制和验证路径。", "expected_behavior": "IA 可作为 QPI 后续模型被调用。", "failure_signal": "在 QPI 放行后仍拒绝。", "expected_output_elements": ["QPI 后续", "should_call=true"], "should_call_model": true, "expected_primary_model": "intellectual_archaeology", "negative_expected_models": [], "expected_classification": "not_applicable", "expected_dominant_scarcity": "not_applicable", "expected_max_depth": "core_mechanism", "minimum_required_elements": ["entry_reason"], "forbidden_elements": [], "evaluation_mode": "structured", "notes": "QPI 后续 pipeline。"}
|
||||
{"case_id": "case_ia_pipeline_after_qpi_001", "model_id": "intellectual_archaeology", "case_type": "pipeline", "input": "QPI 已判断为中重型课题,且需要抽取底层机制和验证路径。", "expected_behavior": "IA 可作为 QPI 后续模型被调用。", "failure_signal": "在 QPI 放行后仍拒绝。", "expected_output_elements": ["QPI 后续", "should_call=true"], "should_call_model": true, "expected_primary_model": "intellectual_archaeology", "negative_expected_models": [], "expected_classification": "not_applicable", "expected_dominant_scarcity": "not_applicable", "expected_max_depth": "core_mechanism", "minimum_required_elements": ["entry_reason"], "forbidden_elements": [], "evaluation_mode": "structured", "notes": "QPI 后续 pipeline。"},
|
||||
{"case_id": "case_ia_negative_explicit_do_not_call_001", "model_id": "intellectual_archaeology", "case_type": "selector_gate", "input": "QPI 已判断:这是 Issue。先不要思想考古,只做问题定性复核。", "expected_behavior": "selector 应选择 QPI,并硬拒绝思想考古。", "failure_signal": "因为“思想考古”和“QPI 已判断”加分而仍选中 IA。", "expected_output_elements": ["selected=qpi", "rejected=intellectual_archaeology"], "should_call_model": false, "expected_primary_model": "qpi", "negative_expected_models": ["intellectual_archaeology"], "expected_classification": "not_applicable", "expected_dominant_scarcity": "not_applicable", "expected_max_depth": "no_call", "minimum_required_elements": ["model_level_hard_exclusion"], "forbidden_elements": ["selected IA"], "evaluation_mode": "structured", "notes": "Round 03.2 hard exclusion trap: explicit IA refusal must override score."},
|
||||
{"case_id": "case_ia_negative_only_qpi_001", "model_id": "intellectual_archaeology", "case_type": "selector_gate", "input": "不要思想考古,只做 QPI 复核。", "expected_behavior": "selector 应选择 QPI,并硬拒绝思想考古。", "failure_signal": "IA 负触发只被当作 penalty,仍可能被选中。", "expected_output_elements": ["selected=qpi", "rejected=intellectual_archaeology"], "should_call_model": false, "expected_primary_model": "qpi", "negative_expected_models": ["intellectual_archaeology"], "expected_classification": "not_applicable", "expected_dominant_scarcity": "not_applicable", "expected_max_depth": "no_call", "minimum_required_elements": ["model_level_hard_exclusion"], "forbidden_elements": ["selected IA"], "evaluation_mode": "structured", "notes": "Round 03.2 hard exclusion trap: only-QPI instruction rejects IA."}
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1550,6 +1550,177 @@
|
|||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.1 direct execution no-call."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_how_to_excel_no_call_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "no_call",
|
||||
"input": "如何在 Excel 里冻结首行?",
|
||||
"expected_behavior": "应 no_call:普通工具 how-to 不应因裸“如何”触发 QPI。",
|
||||
"failure_signal": "selector 选中 QPI。",
|
||||
"expected_output_elements": [
|
||||
"classification=no_call"
|
||||
],
|
||||
"should_call_model": false,
|
||||
"expected_primary_model": "none",
|
||||
"negative_expected_models": [
|
||||
"qpi",
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "no_call",
|
||||
"expected_dominant_scarcity": "not_applicable",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"no_call"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"problem_definition"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2 no-call trap: narrow bare qpi_gate_signals."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_how_to_markdown_pdf_no_call_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "no_call",
|
||||
"input": "如何把 Markdown 转 PDF?",
|
||||
"expected_behavior": "应 no_call:普通格式转换 how-to 不应调用 QPI。",
|
||||
"failure_signal": "selector 选中 QPI。",
|
||||
"expected_output_elements": [
|
||||
"classification=no_call"
|
||||
],
|
||||
"should_call_model": false,
|
||||
"expected_primary_model": "none",
|
||||
"negative_expected_models": [
|
||||
"qpi",
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "no_call",
|
||||
"expected_dominant_scarcity": "not_applicable",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"no_call"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"problem_definition"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2 no-call trap: generic how-to is not QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_dictionary_judgment_no_call_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "no_call",
|
||||
"input": "判断这个词是什么意思。",
|
||||
"expected_behavior": "应 no_call:普通词义判断不应因裸“判断”触发 QPI。",
|
||||
"failure_signal": "selector 选中 QPI。",
|
||||
"expected_output_elements": [
|
||||
"classification=no_call"
|
||||
],
|
||||
"should_call_model": false,
|
||||
"expected_primary_model": "none",
|
||||
"negative_expected_models": [
|
||||
"qpi",
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "no_call",
|
||||
"expected_dominant_scarcity": "not_applicable",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"no_call"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"problem_definition"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2 no-call trap: dictionary judgment is not QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_qpi_only_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要展开到思想考古,只做 QPI 复核。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA;“不要展开”在此表示限制深度,不是全局 no-call。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"depth_limited_qpi_override"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: global no-call phrase plus QPI-only intent."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_problem_definition_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要深入分析,只做问题定性。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA;“不要深入分析”限制深度,但问题定性仍需要 QPI。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"depth_limited_problem_definition"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: limited problem definition should route to QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_depth_limited_dominant_scarcity_001",
|
||||
"model_id": "qpi",
|
||||
"case_type": "selector_gate",
|
||||
"input": "不要展开,只判断主导稀缺。",
|
||||
"expected_behavior": "应选择 QPI 并拒绝 IA;主导稀缺判断属于 QPI 限定分析。",
|
||||
"failure_signal": "selector 返回 no_call=true 或选中 IA。",
|
||||
"expected_output_elements": [
|
||||
"selected=qpi",
|
||||
"rejected=intellectual_archaeology",
|
||||
"no_call=false"
|
||||
],
|
||||
"should_call_model": true,
|
||||
"expected_primary_model": "qpi",
|
||||
"negative_expected_models": [
|
||||
"intellectual_archaeology"
|
||||
],
|
||||
"expected_classification": "mixed",
|
||||
"expected_dominant_scarcity": "unknown",
|
||||
"expected_max_depth": "not_applicable",
|
||||
"minimum_required_elements": [
|
||||
"dominant_scarcity_gate"
|
||||
],
|
||||
"forbidden_elements": [
|
||||
"global_no_call"
|
||||
],
|
||||
"evaluation_mode": "structured",
|
||||
"notes": "Round 03.2a depth-limiting override trap: dominant-scarcity-only judgment should route to QPI."
|
||||
},
|
||||
{
|
||||
"case_id": "case_qpi_analysis_override_should_call_001",
|
||||
"model_id": "qpi",
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -1,33 +1,33 @@
|
|||
{
|
||||
"index_version": "0.2",
|
||||
"last_updated": "2026-06-16",
|
||||
"last_updated": "2026-06-17",
|
||||
"source_of_truth": "per_model_files",
|
||||
"aggregate_file": "tests/regression_cases.json",
|
||||
"models": [
|
||||
{
|
||||
"model_id": "qpi",
|
||||
"regression_file": "tests/qpi.regression.json",
|
||||
"case_count": 17,
|
||||
"case_count": 58,
|
||||
"case_type_counts": {
|
||||
"positive": 9,
|
||||
"boundary": 3,
|
||||
"misuse": 2,
|
||||
"no_call": 1,
|
||||
"selector_gate": 1,
|
||||
"pipeline": 1
|
||||
"boundary": 13,
|
||||
"misuse": 16,
|
||||
"no_call": 6,
|
||||
"pipeline": 1,
|
||||
"positive": 16,
|
||||
"selector_gate": 6
|
||||
}
|
||||
},
|
||||
{
|
||||
"model_id": "intellectual_archaeology",
|
||||
"regression_file": "tests/intellectual_archaeology.regression.json",
|
||||
"case_count": 17,
|
||||
"case_count": 19,
|
||||
"case_type_counts": {
|
||||
"positive": 5,
|
||||
"boundary": 3,
|
||||
"misuse": 3,
|
||||
"no_call": 3,
|
||||
"selector_gate": 2,
|
||||
"pipeline": 1
|
||||
"pipeline": 1,
|
||||
"positive": 5,
|
||||
"selector_gate": 4
|
||||
}
|
||||
}
|
||||
]
|
||||
|
|
|
|||
Loading…
Reference in New Issue