fix: harden qpi selector round 03.1 review
This commit is contained in:
parent
f034d3fd9d
commit
ddaa0cf35b
12
README.md
12
README.md
|
|
@ -148,6 +148,7 @@ Foundation repair in progress for:
|
||||||
Current foundation assets include:
|
Current foundation assets include:
|
||||||
|
|
||||||
- GPT plan localization protocol.
|
- GPT plan localization protocol.
|
||||||
|
- File taxonomy for canonical, generated, review archive, and temporary files.
|
||||||
- Model extraction rules.
|
- Model extraction rules.
|
||||||
- Model card contract.
|
- Model card contract.
|
||||||
- Model extraction workflow.
|
- Model extraction workflow.
|
||||||
|
|
@ -161,7 +162,7 @@ Current foundation assets include:
|
||||||
- A standard-library validation script that writes `reports/validation_report.md`.
|
- A standard-library validation script that writes `reports/validation_report.md`.
|
||||||
- A standard-library index rebuild/check script that writes `reports/index_rebuild_report.md`.
|
- A standard-library index rebuild/check script that writes `reports/index_rebuild_report.md`.
|
||||||
- ChatGPT handoff rules.
|
- ChatGPT handoff rules.
|
||||||
- Long-term knowledge asset rules and initial `knowledge_assets/` documents.
|
- Long-term knowledge asset rules and `knowledge_assets/` documents, including `09_数据治理与模型调用机制说明.md`.
|
||||||
|
|
||||||
Current QPI and Intellectual Archaeology model contents pass the local contract and remain `draft` pending product review.
|
Current QPI and Intellectual Archaeology model contents pass the local contract and remain `draft` pending product review.
|
||||||
|
|
||||||
|
|
@ -171,7 +172,8 @@ CCRA review packages are archived by round under `ccra_review_bundle/round-NN_YY
|
||||||
|
|
||||||
## 13. Next Steps
|
## 13. Next Steps
|
||||||
|
|
||||||
1. Review QPI and Intellectual Archaeology content for product correctness.
|
1. Submit the Round 03.1 selector/no-call/regression patch for GPT / CCRA review.
|
||||||
2. Review whether the new `knowledge_assets/` package is sufficient for ChatGPT / CCRA reuse.
|
2. Keep QPI and Intellectual Archaeology at `draft / B / pending` until Owner / CCRA review accepts stronger status.
|
||||||
3. Decide whether to expand to a third core model.
|
3. Run a QPI blind-test round with new unlabelled inputs after Round 03.1 passes.
|
||||||
4. Decide whether extraction, inspection, and stability scoring should become CCPE or skills-vault requests.
|
4. Defer any third-model expansion until the current QPI selector and case layer are accepted.
|
||||||
|
5. Route missing reusable extraction, inspection, or stability-scoring tools to `requirements/skills-vault/` or `requirements/ccpe/` instead of improvising local platform features.
|
||||||
|
|
|
||||||
18
cards/qpi.md
18
cards/qpi.md
|
|
@ -202,7 +202,7 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
||||||
- 机制稳定性：中高，扫描匮乏物和匹配处理范式的机制明确。
|
- 机制稳定性：中高，扫描匮乏物和匹配处理范式的机制明确。
|
||||||
- 边界清晰度：中等，混合型问题、多视角分歧和上下文不足场景仍需更多测试。
|
- 边界清晰度：中等，混合型问题、多视角分歧和上下文不足场景仍需更多测试。
|
||||||
- 来源证据质量：较高，主体性和动态性来自 2025 原文，核心匮乏物和误框定规则来自 2026 原文，应用规则来自综合文档。
|
- 来源证据质量：较高，主体性和动态性来自 2025 原文，核心匮乏物和误框定规则来自 2026 原文，应用规则来自综合文档。
|
||||||
- 回归测试表现：pending，已有五条样板用例，尚未经过真实案例扩展。
|
- 回归测试表现：pending，当前已有 52 条 QPI 回归用例；其中 Round 03 / 03.1 已纳入 owner-reviewed case layer、selector no-call、multi-perspective 和低上下文 provisional 边界用例。生命周期仍保持 draft。
|
||||||
|
|
||||||
评级理由：三分结构清晰，适合作为入口路由模型，但需要补充大量边界案例，防止过度升维或降维。
|
评级理由：三分结构清晰，适合作为入口路由模型，但需要补充大量边界案例，防止过度升维或降维。
|
||||||
|
|
||||||
|
|
@ -212,13 +212,15 @@ QPI 的三分结构、核心匮乏物、主体性和动态性有清晰来源支
|
||||||
|
|
||||||
`pending`
|
`pending`
|
||||||
|
|
||||||
当前已有五条回归用例：
|
当前已有 52 条 QPI 回归用例，覆盖 positive、boundary、misuse、no_call、selector_gate 和 pipeline。Round 03 / 03.1 重点新增：
|
||||||
|
|
||||||
- `case_qpi_positive_question_001`
|
- owner-reviewed flow / disappointment / organizational year-end-review cases；
|
||||||
- `case_qpi_positive_problem_001`
|
- `case_qpi_international_logistics_no_call_001`
|
||||||
- `case_qpi_positive_issue_001`
|
- `case_qpi_research_capacity_problem_not_issue_001`
|
||||||
- `case_qpi_boundary_mixed_001`
|
- `case_qpi_multi_perspective_requires_viewpoint_output_001`
|
||||||
- `case_qpi_misuse_inflation_001`
|
- `case_qpi_low_context_provisional_no_high_confidence_001`
|
||||||
|
- `case_qpi_direct_summary_no_call_001`
|
||||||
|
- `case_qpi_analysis_override_should_call_001`
|
||||||
|
|
||||||
## 示例输入
|
## 示例输入
|
||||||
|
|
||||||
|
|
@ -283,5 +285,5 @@ selector 应优先使用 `trigger_keywords`、`negative_triggers`、`pipeline_po
|
||||||
## 版本信息
|
## 版本信息
|
||||||
|
|
||||||
- version: `0.1`
|
- version: `0.1`
|
||||||
- last_updated: `2026-06-16`
|
- last_updated: `2026-06-17`
|
||||||
- status: `draft`
|
- status: `draft`
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,61 @@
|
||||||
|
# CCRA / GPT Review Brief: Round 03.1 Selector No-Call Regression Patch
|
||||||
|
|
||||||
|
Date: 2026-06-17
|
||||||
|
|
||||||
|
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||||
|
|
||||||
|
Phase: `model_library_mvp`
|
||||||
|
|
||||||
|
## 1. Scope
|
||||||
|
|
||||||
|
Round 03.1 is a small patch over the formal Round 03 review package.
|
||||||
|
|
||||||
|
It fixes:
|
||||||
|
|
||||||
|
- selector over-selection of QPI by base score;
|
||||||
|
- direct-execution no-call coverage;
|
||||||
|
- selector calibration smoke coverage;
|
||||||
|
- QPI regression coverage for no-call, low-context, multi-perspective, and capacity boundary cases;
|
||||||
|
- QPI digest field drift;
|
||||||
|
- stale QPI card / report counts;
|
||||||
|
- review bundle hygiene.
|
||||||
|
|
||||||
|
It does not:
|
||||||
|
|
||||||
|
- add a third model;
|
||||||
|
- upgrade QPI or Intellectual Archaeology to stable;
|
||||||
|
- introduce an LLM selector;
|
||||||
|
- introduce RAG, database, frontend, backend, user system, or full QA system.
|
||||||
|
|
||||||
|
Lifecycle states remain:
|
||||||
|
|
||||||
|
| Model ID | Status | Stability | Regression Status |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `qpi` | draft | B | pending |
|
||||||
|
| `intellectual_archaeology` | draft | B | pending |
|
||||||
|
|
||||||
|
## 2. Main Fix
|
||||||
|
|
||||||
|
QPI can no longer be selected only by base score plus selection priority.
|
||||||
|
|
||||||
|
The selector now requires a positive signal for QPI unless the request is explicitly in the problem-definition path. Direct execution inputs such as summary, table formatting, bed assignment, and narrow translation are no-call unless an explicit analysis override exists.
|
||||||
|
|
||||||
|
## 3. Current Counts
|
||||||
|
|
||||||
|
| Artifact | Count |
|
||||||
|
| --- | ---: |
|
||||||
|
| QPI case digests | 62 |
|
||||||
|
| Selector calibration inputs | 85 |
|
||||||
|
| QPI regression cases | 52 |
|
||||||
|
| Aggregate regression cases | 69 |
|
||||||
|
| Unit tests | 17 |
|
||||||
|
|
||||||
|
## 4. Review Focus
|
||||||
|
|
||||||
|
Please review:
|
||||||
|
|
||||||
|
- whether the selector no-call gate now blocks direct execution without suppressing explicit analysis override;
|
||||||
|
- whether the calibration smoke test is an acceptable Round 03.1 guardrail;
|
||||||
|
- whether `misclassification_risk` and `qpi_complexity_pattern` resolve the digest contract drift;
|
||||||
|
- whether multi-perspective digests now have enough viewpoint traceability for draft-callable review;
|
||||||
|
- whether QPI remains a routing model and not a solution engine.
|
||||||
|
|
@ -0,0 +1,28 @@
|
||||||
|
# Round 03.1 Patch Matrix
|
||||||
|
|
||||||
|
| GPT / Owner instruction | Status | Evidence |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| QPI must not be selected only by base score + selection priority | Done | `scripts/run_selector_demo.py`, `selector/selector_rules.json` |
|
||||||
|
| Add direct execution no-call signals | Done | `selector/selector_rules.json` |
|
||||||
|
| Preserve explicit analysis override behavior | Done | `scripts/run_selector_demo.py`, `case_qpi_analysis_override_should_call_001` |
|
||||||
|
| Normalize rule-level selection priority scale | Done | QPI rule priority `9`, IA rule priority `7` |
|
||||||
|
| Add required Round 03.1 QPI regression cases | Done | `tests/qpi.regression.json` |
|
||||||
|
| Sync new QPI cases into aggregate regression file | Done | `tests/regression_cases.json` |
|
||||||
|
| Add selector calibration smoke test | Done | `scripts/run_selector_calibration_smoke.py` |
|
||||||
|
| Normalize `misframing_risks` to `misclassification_risk` | Done | `selector/qpi_case_digests.json`, `scripts/validate_model_library.py` |
|
||||||
|
| Rename/document `mixed_or_multi_perspective` | Done | `qpi_complexity_pattern`, documented in `selector/README.md` and `docs/QPI_CONTEXTUAL_ROUTING_RULES.md` |
|
||||||
|
| Require viewpoint detail for multi-perspective digests | Done | validator requires `classification_by_viewpoint` or `viewpoint_summary` |
|
||||||
|
| Update stale QPI card count and date | Done | `cards/qpi.md`, `models/qpi.model.json` |
|
||||||
|
| Mark v0.2 content report as pre-case-promotion | Done | `reports/content_review_report_v0.2.md` |
|
||||||
|
| Preserve source paths in review zip | Done | `optional_raw_changed_files.zip` uses relative paths |
|
||||||
|
| Exclude `knowledge_assets` from review resources | Done | Not included in Round 03.1 raw zip |
|
||||||
|
| Include IA model/card if IA contract completion is referenced | Done | `models/intellectual_archaeology.model.json`, `cards/intellectual_archaeology.md` included in zip |
|
||||||
|
|
||||||
|
## Added QPI Regression Cases
|
||||||
|
|
||||||
|
- `case_qpi_international_logistics_no_call_001`
|
||||||
|
- `case_qpi_research_capacity_problem_not_issue_001`
|
||||||
|
- `case_qpi_multi_perspective_requires_viewpoint_output_001`
|
||||||
|
- `case_qpi_low_context_provisional_no_high_confidence_001`
|
||||||
|
- `case_qpi_direct_summary_no_call_001`
|
||||||
|
- `case_qpi_analysis_override_should_call_001`
|
||||||
|
|
@ -0,0 +1,51 @@
|
||||||
|
# Current Asset Pack
|
||||||
|
|
||||||
|
## 1. Selector And Calibration
|
||||||
|
|
||||||
|
- `selector/selector_rules.json`
|
||||||
|
- `selector/selector_calibration_inputs.json`
|
||||||
|
- `selector/qpi_case_digests.json`
|
||||||
|
- `selector/README.md`
|
||||||
|
- `scripts/run_selector_demo.py`
|
||||||
|
- `scripts/run_selector_regression.py`
|
||||||
|
- `scripts/run_selector_calibration_smoke.py`
|
||||||
|
|
||||||
|
## 2. Regression And Validation
|
||||||
|
|
||||||
|
- `tests/qpi.regression.json`
|
||||||
|
- `tests/regression_cases.json`
|
||||||
|
- `tests/test_validate_model_library.py`
|
||||||
|
- `scripts/validate_model_library.py`
|
||||||
|
- `scripts/check_card_contract.py`
|
||||||
|
- `scripts/check_model_card_sync.py`
|
||||||
|
- `scripts/rebuild_indexes.py`
|
||||||
|
|
||||||
|
## 3. Model/Card Sync
|
||||||
|
|
||||||
|
- `models/qpi.model.json`
|
||||||
|
- `cards/qpi.md`
|
||||||
|
- `models/model_index.json`
|
||||||
|
- `cards/card_index.md`
|
||||||
|
- `models/intellectual_archaeology.model.json`
|
||||||
|
- `cards/intellectual_archaeology.md`
|
||||||
|
|
||||||
|
## 4. Reports
|
||||||
|
|
||||||
|
- `reports/validation_report.md`
|
||||||
|
- `reports/index_rebuild_report.md`
|
||||||
|
- `reports/selector_regression_report_v0.2.md`
|
||||||
|
- `reports/selector_calibration_smoke_report.md`
|
||||||
|
- `reports/model_card_sync_report_v0.2.md`
|
||||||
|
- `reports/content_review_report_v0.2.md`
|
||||||
|
|
||||||
|
## 5. Rule Docs
|
||||||
|
|
||||||
|
- `docs/QPI_CONTEXTUAL_ROUTING_RULES.md`
|
||||||
|
|
||||||
|
## 6. Excluded From Review Zip
|
||||||
|
|
||||||
|
`knowledge_assets/` is not included. The owner manually syncs long-term knowledge assets into GPT knowledge storage, so review bundles should focus on current executable / auditable project assets.
|
||||||
|
|
||||||
|
Current long-term related asset, manually synced by Owner rather than included in this patch zip:
|
||||||
|
|
||||||
|
- `knowledge_assets/09_数据治理与模型调用机制说明.md`
|
||||||
|
|
@ -0,0 +1,31 @@
|
||||||
|
# Validation And Command Log
|
||||||
|
|
||||||
|
## 1. Commands Run
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
python scripts\rebuild_indexes.py --write
|
||||||
|
python -m unittest discover -s tests -p "test*.py" -v
|
||||||
|
python scripts\check_card_contract.py
|
||||||
|
python scripts\validate_model_library.py
|
||||||
|
python scripts\run_selector_demo.py
|
||||||
|
python scripts\run_selector_regression.py
|
||||||
|
python scripts\run_selector_calibration_smoke.py
|
||||||
|
python scripts\check_model_card_sync.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## 2. Results
|
||||||
|
|
||||||
|
- Index rebuild `--write`: PASS.
|
||||||
|
- Unit tests: PASS, 17 tests.
|
||||||
|
- Card contract: PASS.
|
||||||
|
- Model library validation: PASS.
|
||||||
|
- Selector demo: PASS; selected `qpi`, rejected `intellectual_archaeology`.
|
||||||
|
- Selector regression: PASS.
|
||||||
|
- Selector calibration smoke: PASS, 85 calibration inputs checked.
|
||||||
|
- Model/card sync: PASS.
|
||||||
|
|
||||||
|
## 3. Notable Debugging Notes
|
||||||
|
|
||||||
|
- Lowering QPI base score exposed that existing owner-reviewed calibration inputs needed richer QPI semantic signals rather than default QPI selection.
|
||||||
|
- Broad `翻译` matching was narrowed so literal translation stays no-call while metaphorical organizational translation can still select QPI.
|
||||||
|
- `先不要思想考古` was added as an IA negative trigger to preserve QPI-before-IA routing when the user explicitly asks not to call IA.
|
||||||
|
|
@ -0,0 +1,18 @@
|
||||||
|
# Review Questions For GPT
|
||||||
|
|
||||||
|
Please return:
|
||||||
|
|
||||||
|
1. pass / revise / block judgment for Round 03.1;
|
||||||
|
2. whether selector no-call behavior is now safe enough for draft-callable review;
|
||||||
|
3. whether the calibration smoke test should become a permanent gate;
|
||||||
|
4. whether any new no-call or override traps are missing;
|
||||||
|
5. whether digest field normalization is acceptable;
|
||||||
|
6. whether QPI lifecycle should remain `draft / B / pending`.
|
||||||
|
|
||||||
|
Required non-goal check:
|
||||||
|
|
||||||
|
- no third model;
|
||||||
|
- no stable upgrade;
|
||||||
|
- no LLM selector;
|
||||||
|
- no full QA system;
|
||||||
|
- no RAG / vector database / frontend / backend platform.
|
||||||
|
|
@ -0,0 +1,50 @@
|
||||||
|
# Bundle File Manifest
|
||||||
|
|
||||||
|
## Recommended Upload Order
|
||||||
|
|
||||||
|
1. `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF.md`
|
||||||
|
2. `01_PATCH_MATRIX.md`
|
||||||
|
3. `02_CURRENT_ASSET_PACK.md`
|
||||||
|
4. `03_VALIDATION_AND_COMMAND_LOG.md`
|
||||||
|
5. `04_REVIEW_QUESTIONS_FOR_GPT.md`
|
||||||
|
6. `optional_raw_changed_files.zip` only if exact file inspection is needed
|
||||||
|
|
||||||
|
## Zip Policy
|
||||||
|
|
||||||
|
`optional_raw_changed_files.zip` preserves source-relative paths. It is not a flat archive.
|
||||||
|
|
||||||
|
`knowledge_assets/` is intentionally excluded from this review bundle because the owner manually syncs long-term knowledge assets into GPT knowledge storage.
|
||||||
|
|
||||||
|
Current related long-term asset outside this review zip:
|
||||||
|
|
||||||
|
- `knowledge_assets/09_数据治理与模型调用机制说明.md`
|
||||||
|
|
||||||
|
## Raw Files Included In Zip
|
||||||
|
|
||||||
|
- `selector/selector_rules.json`
|
||||||
|
- `selector/selector_calibration_inputs.json`
|
||||||
|
- `selector/qpi_case_digests.json`
|
||||||
|
- `selector/README.md`
|
||||||
|
- `scripts/run_selector_demo.py`
|
||||||
|
- `scripts/run_selector_regression.py`
|
||||||
|
- `scripts/run_selector_calibration_smoke.py`
|
||||||
|
- `scripts/validate_model_library.py`
|
||||||
|
- `scripts/check_card_contract.py`
|
||||||
|
- `scripts/check_model_card_sync.py`
|
||||||
|
- `scripts/rebuild_indexes.py`
|
||||||
|
- `tests/qpi.regression.json`
|
||||||
|
- `tests/regression_cases.json`
|
||||||
|
- `tests/test_validate_model_library.py`
|
||||||
|
- `models/qpi.model.json`
|
||||||
|
- `cards/qpi.md`
|
||||||
|
- `models/model_index.json`
|
||||||
|
- `cards/card_index.md`
|
||||||
|
- `models/intellectual_archaeology.model.json`
|
||||||
|
- `cards/intellectual_archaeology.md`
|
||||||
|
- `reports/validation_report.md`
|
||||||
|
- `reports/index_rebuild_report.md`
|
||||||
|
- `reports/selector_regression_report_v0.2.md`
|
||||||
|
- `reports/selector_calibration_smoke_report.md`
|
||||||
|
- `reports/model_card_sync_report_v0.2.md`
|
||||||
|
- `reports/content_review_report_v0.2.md`
|
||||||
|
- `docs/QPI_CONTEXTUAL_ROUTING_RULES.md`
|
||||||
Binary file not shown.
|
|
@ -153,3 +153,8 @@ ccra_review_bundle/round-NN_YYYY-MM-DD_topic/
|
||||||
```
|
```
|
||||||
|
|
||||||
This keeps second-round and third-round materials comparable without overwriting prior evidence. Temporary review bundles remain session evidence and should not be copied into `knowledge_assets/`.
|
This keeps second-round and third-round materials comparable without overwriting prior evidence. Temporary review bundles remain session evidence and should not be copied into `knowledge_assets/`.
|
||||||
|
|
||||||
|
Round 03.1 added two bundle hygiene rules:
|
||||||
|
|
||||||
|
- `optional_raw_changed_files.zip` must preserve source-relative paths. Do not create flat archives that can collide on names such as `README.md`.
|
||||||
|
- `knowledge_assets/` is excluded from review bundles by default because the project owner manually syncs stable long-term knowledge assets into GPT knowledge storage. Include it only when the current review explicitly targets the knowledge asset itself.
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,152 @@
|
||||||
|
# File Taxonomy
|
||||||
|
|
||||||
|
Date: 2026-06-17
|
||||||
|
|
||||||
|
Status: accepted
|
||||||
|
|
||||||
|
## 1. Purpose
|
||||||
|
|
||||||
|
This document defines file identities for the `model_library_mvp` phase.
|
||||||
|
|
||||||
|
The goal is to prevent every generated report, review bundle, cache, and model asset from being treated as the same kind of artifact.
|
||||||
|
|
||||||
|
`docs/KNOWLEDGE_ASSET_RULES.md` governs one specific category: long-term reusable knowledge assets.
|
||||||
|
|
||||||
|
This document is broader. It governs the whole repository.
|
||||||
|
|
||||||
|
## 2. File Identity Classes
|
||||||
|
|
||||||
|
Every meaningful file should fit one of four classes:
|
||||||
|
|
||||||
|
| Class | Meaning | Default retention | Examples |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Canonical source of truth | Files that define model/library behavior or reviewed content | Keep and version | `models/*.model.json`, `cards/*.md`, `sources/*.json`, `tests/*.regression.json`, `selector/*.json`, `schemas/*.json`, operative `docs/*.md` |
|
||||||
|
| Generated / derived | Files rebuilt or checked from canonical assets | Keep when useful, rebuild at handoff/release | `models/model_index.json`, `cards/card_index.md`, `reports/validation_report.md`, `reports/index_rebuild_report.md`, `reports/model_card_sync_report_v0.2.md`, `reports/selector_regression_report_v0.2.md` |
|
||||||
|
| Review archive | Per-round evidence for Owner / CCRA / GPT review | Keep by round, do not treat as runtime truth | `ccra_review_bundle/round-*`, `reports/Codex*.md`, `reports/GPT*.md`, `reports/model_case_preprocessing/*` |
|
||||||
|
| Temporary / local runtime | Caches or local command byproducts | Do not commit | `__pycache__/`, `*.pyc`, temporary extraction folders, ad hoc local scratch files |
|
||||||
|
|
||||||
|
## 3. Canonical Source Of Truth
|
||||||
|
|
||||||
|
Canonical files answer:
|
||||||
|
|
||||||
|
```text
|
||||||
|
What is the current model/library behavior?
|
||||||
|
What is the reviewed source or contract?
|
||||||
|
What should code and future agents trust?
|
||||||
|
```
|
||||||
|
|
||||||
|
Canonical files include:
|
||||||
|
|
||||||
|
- model JSON specs under `models/`;
|
||||||
|
- human-readable cards under `cards/`;
|
||||||
|
- source records and evidence excerpts under `sources/`;
|
||||||
|
- regression source files under `tests/*.regression.json`;
|
||||||
|
- selector rules and calibration files under `selector/`;
|
||||||
|
- schemas under `schemas/`;
|
||||||
|
- operative rules and protocols under `docs/`;
|
||||||
|
- stable explanatory assets under `knowledge_assets/`.
|
||||||
|
|
||||||
|
Canonical files should be edited deliberately and validated after change.
|
||||||
|
|
||||||
|
## 4. Generated / Derived Files
|
||||||
|
|
||||||
|
Generated files answer:
|
||||||
|
|
||||||
|
```text
|
||||||
|
What did the current canonical assets produce when checked or rebuilt?
|
||||||
|
```
|
||||||
|
|
||||||
|
They may be committed when they are part of the file-first workflow, but they are not independent truth.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
- `models/model_index.json`
|
||||||
|
- `cards/card_index.md`
|
||||||
|
- `tests/regression_cases.json`
|
||||||
|
- `reports/validation_report.md`
|
||||||
|
- `reports/index_rebuild_report.md`
|
||||||
|
- `reports/model_card_sync_report_v0.2.md`
|
||||||
|
- `reports/selector_regression_report_v0.2.md`
|
||||||
|
- `reports/selector_calibration_smoke_report.md`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- Rebuild indexes after model/card/source/test changes.
|
||||||
|
- Regenerate reports after validation or selector checks.
|
||||||
|
- Do not manually edit generated reports to hide validation failure.
|
||||||
|
- If generated output disagrees with canonical source, fix the source or generator, then regenerate.
|
||||||
|
|
||||||
|
## 5. Review Archive Files
|
||||||
|
|
||||||
|
Review archives answer:
|
||||||
|
|
||||||
|
```text
|
||||||
|
What was submitted, reviewed, or handed off in a specific round?
|
||||||
|
```
|
||||||
|
|
||||||
|
They preserve evidence and context, but they do not override canonical files.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
- `ccra_review_bundle/round-*/`
|
||||||
|
- `reports/Codex*.md`
|
||||||
|
- `reports/GPT*.md`
|
||||||
|
- `reports/model_case_preprocessing/*`
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- Keep review bundles under dated round directories.
|
||||||
|
- Do not place new review files directly under `ccra_review_bundle/`.
|
||||||
|
- `optional_raw_changed_files.zip` must preserve source-relative paths.
|
||||||
|
- Do not flatten zip contents, because duplicate filenames such as `README.md` can collide.
|
||||||
|
- `knowledge_assets/` is excluded from review zips by default because the Owner manually syncs stable knowledge assets into GPT knowledge storage.
|
||||||
|
|
||||||
|
## 6. Temporary / Local Runtime Files
|
||||||
|
|
||||||
|
Temporary files answer no durable project question.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
|
||||||
|
- `__pycache__/`
|
||||||
|
- `*.pyc`
|
||||||
|
- local unpacked zip folders;
|
||||||
|
- scratch command outputs not referenced by a report;
|
||||||
|
- editor backups.
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- Do not commit these files.
|
||||||
|
- Add broad safe ignore rules in `.gitignore`.
|
||||||
|
- If a temporary command output becomes useful evidence, convert it into a deliberate report under `reports/` or a review bundle.
|
||||||
|
|
||||||
|
## 7. Relationship To Knowledge Assets
|
||||||
|
|
||||||
|
`knowledge_assets/` is a canonical long-term explanatory layer, but not every canonical file belongs there.
|
||||||
|
|
||||||
|
Put a document in `knowledge_assets/` only when it answers:
|
||||||
|
|
||||||
|
```text
|
||||||
|
What durable context, mechanism, rule, or map should ChatGPT / CCRA / Owner remember across sessions?
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not put these in `knowledge_assets/`:
|
||||||
|
|
||||||
|
- concrete model cards that already live in `cards/`;
|
||||||
|
- machine-readable model JSON;
|
||||||
|
- validation reports;
|
||||||
|
- per-round review bundles;
|
||||||
|
- command logs;
|
||||||
|
- temporary handoffs.
|
||||||
|
|
||||||
|
See `docs/KNOWLEDGE_ASSET_RULES.md` for detailed knowledge-asset rules.
|
||||||
|
|
||||||
|
## 8. Commit Checklist
|
||||||
|
|
||||||
|
Before commit:
|
||||||
|
|
||||||
|
1. Confirm new files have the correct identity.
|
||||||
|
2. Confirm temporary files are not staged.
|
||||||
|
3. Rebuild or check indexes when canonical model/card/source/test files changed.
|
||||||
|
4. Regenerate validation reports after validation commands.
|
||||||
|
5. Keep review-bundle zips path-preserving.
|
||||||
|
6. Keep lifecycle status conservative; validation pass does not imply stable.
|
||||||
|
|
@ -6,6 +6,8 @@
|
||||||
|
|
||||||
These documents are not raw source archives, not temporary handoffs, and not implementation-only files. They are the durable explanation layer that helps the project owner, ChatGPT, Codex, and future CCRA work understand the model library without reconstructing context from scattered files.
|
These documents are not raw source archives, not temporary handoffs, and not implementation-only files. They are the durable explanation layer that helps the project owner, ChatGPT, Codex, and future CCRA work understand the model library without reconstructing context from scattered files.
|
||||||
|
|
||||||
|
For the broader repository-wide distinction between canonical, generated, review archive, and temporary files, see `docs/FILE_TAXONOMY.md`.
|
||||||
|
|
||||||
## What Belongs In `knowledge_assets/`
|
## What Belongs In `knowledge_assets/`
|
||||||
|
|
||||||
Use `knowledge_assets/` for:
|
Use `knowledge_assets/` for:
|
||||||
|
|
@ -17,6 +19,7 @@ Use `knowledge_assets/` for:
|
||||||
- stability rating rules
|
- stability rating rules
|
||||||
- durable process records
|
- durable process records
|
||||||
- reusable architecture summaries
|
- reusable architecture summaries
|
||||||
|
- durable data-governance and model-invocation mechanism explanations
|
||||||
|
|
||||||
Do not use `knowledge_assets/` for:
|
Do not use `knowledge_assets/` for:
|
||||||
|
|
||||||
|
|
@ -26,6 +29,7 @@ Do not use `knowledge_assets/` for:
|
||||||
- generated cache files
|
- generated cache files
|
||||||
- implementation scripts
|
- implementation scripts
|
||||||
- source-of-truth JSON model assets
|
- source-of-truth JSON model assets
|
||||||
|
- per-round review bundles or optional raw changed-file zips
|
||||||
|
|
||||||
## Relationship To Other Directories
|
## Relationship To Other Directories
|
||||||
|
|
||||||
|
|
@ -39,6 +43,8 @@ Do not use `knowledge_assets/` for:
|
||||||
|
|
||||||
`knowledge_assets/` contains stable explanatory knowledge distilled from those sources.
|
`knowledge_assets/` contains stable explanatory knowledge distilled from those sources.
|
||||||
|
|
||||||
|
The project owner may manually sync stable `knowledge_assets/` documents into GPT knowledge storage. Review bundles should therefore not include `knowledge_assets/` by default unless the current review explicitly targets the long-term knowledge asset itself.
|
||||||
|
|
||||||
## Naming Rule
|
## Naming Rule
|
||||||
|
|
||||||
Use numeric prefixes for reading order.
|
Use numeric prefixes for reading order.
|
||||||
|
|
@ -54,6 +60,7 @@ Examples:
|
||||||
03_核心模型抽取样板.md
|
03_核心模型抽取样板.md
|
||||||
06_模型稳固性评级规则.md
|
06_模型稳固性评级规则.md
|
||||||
07_产品规划过程记录.md
|
07_产品规划过程记录.md
|
||||||
|
09_数据治理与模型调用机制说明.md
|
||||||
```
|
```
|
||||||
|
|
||||||
## Model Card Sample Rule
|
## Model Card Sample Rule
|
||||||
|
|
|
||||||
|
|
@ -174,5 +174,16 @@ Each case digest should preserve:
|
||||||
- `codex_candidate_judgment`;
|
- `codex_candidate_judgment`;
|
||||||
- `owner_review_needed`.
|
- `owner_review_needed`.
|
||||||
|
|
||||||
Raw case processing is intentionally deferred until owner materials are available.
|
Round 03 completed the first owner-reviewed QPI case promotion into:
|
||||||
|
|
||||||
|
- `reports/model_case_preprocessing/qpi/round-01/*.cases.md`;
|
||||||
|
- `selector/qpi_case_digests.json`;
|
||||||
|
- `selector/selector_calibration_inputs.json`;
|
||||||
|
- `tests/qpi.regression.json`.
|
||||||
|
|
||||||
|
Round 03.1 digest field rules:
|
||||||
|
|
||||||
|
- Use `misclassification_risk`, not `misframing_risks`, to match the QPI structured output contract.
|
||||||
|
- Use `qpi_complexity_pattern`, not `mixed_or_multi_perspective`, for judgment-structure complexity.
|
||||||
|
- `qpi_complexity_pattern=intra_frame_mixed` does not require `classification=mixed`; final routing classification and judgment complexity are separate.
|
||||||
|
- If `classification_scope=multi_perspective` or `qpi_complexity_pattern=inter_viewpoint_divergence`, the digest must include `classification_by_viewpoint` or `viewpoint_summary`.
|
||||||
|
|
|
||||||
|
|
@ -117,3 +117,5 @@ For long-term reusable knowledge rules, see `docs/KNOWLEDGE_ASSET_RULES.md`.
|
||||||
When a rule, model map, schema explanation, workflow summary, or product context becomes stable enough to reuse across sessions, create or update a file under `knowledge_assets/`.
|
When a rule, model map, schema explanation, workflow summary, or product context becomes stable enough to reuse across sessions, create or update a file under `knowledge_assets/`.
|
||||||
|
|
||||||
Do not leave durable knowledge only in temporary reports.
|
Do not leave durable knowledge only in temporary reports.
|
||||||
|
|
||||||
|
For repository-wide file identity rules, including canonical files, generated reports, review archives, and temporary files, see `docs/FILE_TAXONOMY.md`.
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,629 @@
|
||||||
|
# 数据治理与模型调用机制说明
|
||||||
|
|
||||||
|
version: 0.1
|
||||||
|
|
||||||
|
last_updated: 2026-06-17
|
||||||
|
|
||||||
|
status: stable explanatory asset
|
||||||
|
|
||||||
|
source_basis:
|
||||||
|
|
||||||
|
- `C:\Users\wangq\Documents\Codex\knowledge-vault\work\internal\强哥的思想宇宙\GPT成果\CCRA_数据治理与模型调用机制说明_v0.1.md`
|
||||||
|
- current repository contracts, selector rules, regression files, validation scripts, and review-bundle workflow
|
||||||
|
|
||||||
|
## 1. 文档定位
|
||||||
|
|
||||||
|
本文档是 `model_library_mvp` 阶段的数据治理与模型调用机制说明。
|
||||||
|
|
||||||
|
它不是某一轮评审的 PASS / FAIL 记录，也不是对 Owner 质疑的逐条回复。它沉淀的是长期可复用的项目机制：
|
||||||
|
|
||||||
|
- 为什么模型库不等于普通知识库；
|
||||||
|
- 文章、模型卡、selector、regression、validation、review bundle 分别承担什么治理职责；
|
||||||
|
- QPI 和思想考古学未来如何被调用；
|
||||||
|
- 哪些文件是长期源头，哪些只是过程证据；
|
||||||
|
- 后续新增模型应按什么最低资产结构进入系统。
|
||||||
|
|
||||||
|
具体模型内容的 source of truth 仍在：
|
||||||
|
|
||||||
|
- `models/*.model.json`
|
||||||
|
- `cards/*.md`
|
||||||
|
- `sources/*.json`
|
||||||
|
- `tests/*.regression.json`
|
||||||
|
- `selector/*.json`
|
||||||
|
|
||||||
|
本文档只解释机制，不替代上述文件。
|
||||||
|
|
||||||
|
## 2. 项目当前性质
|
||||||
|
|
||||||
|
`the-mindscape-of-bro-tsong` 当前处于 `model_library_mvp` 阶段。
|
||||||
|
|
||||||
|
它不是：
|
||||||
|
|
||||||
|
- 完整产品；
|
||||||
|
- 聊天机器人；
|
||||||
|
- 前端平台；
|
||||||
|
- 后端服务；
|
||||||
|
- RAG 系统；
|
||||||
|
- 知识图谱；
|
||||||
|
- 数据库应用；
|
||||||
|
- 商业交付系统。
|
||||||
|
|
||||||
|
它当前验证的是：
|
||||||
|
|
||||||
|
```text
|
||||||
|
少量核心认知模型
|
||||||
|
能否被整理成 file-first 的模型资产，
|
||||||
|
并具备可读、可追溯、可调用、可拒绝调用、可测试、可路由的最低能力。
|
||||||
|
```
|
||||||
|
|
||||||
|
第一批样板模型是：
|
||||||
|
|
||||||
|
- QPI：前置问题定性和路由模型；
|
||||||
|
- 思想考古学：中重型问题的深度建模模型。
|
||||||
|
|
||||||
|
QPI 的价值不只在于它本身，而在于它能压力测试一整套模型治理机制：调用条件、拒绝条件、输出契约、误用边界、selector 校准、regression 防退化、Owner / CCRA 审核。
|
||||||
|
|
||||||
|
## 3. 为什么不是直接把文章喂给 AI
|
||||||
|
|
||||||
|
如果只是让 AI 读取文章并回答问题，确实不需要模型库结构。
|
||||||
|
|
||||||
|
但这种做法不满足模型资产化要求：
|
||||||
|
|
||||||
|
- 不可追溯：很难回查系统用了哪篇文章、哪段证据、哪条人工判断；
|
||||||
|
- 不可稳定调用：同一输入可能每次触发不同判断和输出结构；
|
||||||
|
- 不可拒绝调用：模型容易被滥用，例如所有复杂问题都套 QPI 或思想考古；
|
||||||
|
- 不可回归：改规则后无法知道旧边界是否被破坏；
|
||||||
|
- 不可交接 Codex：Codex 不能只凭一篇文章稳定构造 schema、selector、validator 和测试；
|
||||||
|
- 不可产品化：文章是内容资产，模型库需要可组合、可运行、可验证的认知工具资产。
|
||||||
|
|
||||||
|
因此本项目做的是：
|
||||||
|
|
||||||
|
```text
|
||||||
|
原始文章 / 人工素材
|
||||||
|
-> 来源记录
|
||||||
|
-> 证据片段
|
||||||
|
-> 人读模型卡
|
||||||
|
-> 机器可读模型卡
|
||||||
|
-> 输出契约
|
||||||
|
-> 调用规则
|
||||||
|
-> 负向触发条件
|
||||||
|
-> selector
|
||||||
|
-> calibration input
|
||||||
|
-> regression cases
|
||||||
|
-> validation scripts
|
||||||
|
-> review bundle
|
||||||
|
-> Owner / CCRA 审核意见
|
||||||
|
```
|
||||||
|
|
||||||
|
这些文件不是平行内容，而是不同治理层。
|
||||||
|
|
||||||
|
## 4. 数据治理的六个目标
|
||||||
|
|
||||||
|
### 4.1 来源治理
|
||||||
|
|
||||||
|
每个模型必须知道它从哪里来：
|
||||||
|
|
||||||
|
- 来源文章是什么；
|
||||||
|
- 代表性文本是什么；
|
||||||
|
- 哪些字段由原文直接支持；
|
||||||
|
- 哪些字段是从原文推导；
|
||||||
|
- 哪些字段是产品化决策；
|
||||||
|
- 哪些字段是 Owner / CCRA 人工判断；
|
||||||
|
- 哪些证据仍是 placeholder 或需要复核。
|
||||||
|
|
||||||
|
目标是防止模型后来变成“像作者思想，但已经无法回到原文”的漂移资产。
|
||||||
|
|
||||||
|
### 4.2 结构治理
|
||||||
|
|
||||||
|
模型不能只是一段定义。
|
||||||
|
|
||||||
|
每个模型至少要被拆成：
|
||||||
|
|
||||||
|
- `model_id`
|
||||||
|
- `model_type`
|
||||||
|
- `pipeline_position`
|
||||||
|
- 核心问题
|
||||||
|
- 核心机制
|
||||||
|
- 输入类型
|
||||||
|
- 输出类型
|
||||||
|
- 适用场景
|
||||||
|
- 不适用场景
|
||||||
|
- 负向触发条件
|
||||||
|
- 常见误用
|
||||||
|
- 失败信号
|
||||||
|
- 稳固性等级
|
||||||
|
- 输出契约
|
||||||
|
|
||||||
|
结构治理让模型既能被人审，也能被机器读取。
|
||||||
|
|
||||||
|
### 4.3 调用治理
|
||||||
|
|
||||||
|
模型进入系统后，不能默认“能用就用”。
|
||||||
|
|
||||||
|
每个模型都必须回答：
|
||||||
|
|
||||||
|
- 什么输入应该调用它；
|
||||||
|
- 什么输入不该调用它；
|
||||||
|
- 是否必须先经过其他模型；
|
||||||
|
- 是否只能在某个流程阶段使用；
|
||||||
|
- 是否需要重型分析门槛；
|
||||||
|
- 是否存在 hard no-call 条件；
|
||||||
|
- 是否存在 explicit analysis override。
|
||||||
|
|
||||||
|
这就是 selector 的职责。
|
||||||
|
|
||||||
|
### 4.4 输出治理
|
||||||
|
|
||||||
|
模型输出不能随意发挥。
|
||||||
|
|
||||||
|
以 QPI 为例，它不是简单输出“这是 Question / Problem / Issue”，而是必须围绕主体、场景、责任范围、期望现实落差、主导稀缺物、分类置信度、证据缺口、误分类风险、下一步候选模型等字段工作。
|
||||||
|
|
||||||
|
以思想考古学为例，它不能无限哲学化，而是必须说明是否应该调用、为什么调用、最多下潜到哪层、哪些层需要分析、什么时候停止。
|
||||||
|
|
||||||
|
### 4.5 边界治理
|
||||||
|
|
||||||
|
解释力强的模型更容易被滥用。
|
||||||
|
|
||||||
|
典型误用包括：
|
||||||
|
|
||||||
|
- 暴力降维：把复杂 Issue 当成简单 Problem；
|
||||||
|
- 恶意升维：把简单执行任务夸大成复杂课题；
|
||||||
|
- 手段错配：本该查资料，却启动深度模型；本该组织协商，却只做文档润色；
|
||||||
|
- 认知重工业化：一个轻量问题被多模型、多智能体、深层考古压爆。
|
||||||
|
|
||||||
|
边界治理不是削弱模型，而是让模型该用时有力，不该用时安静。
|
||||||
|
|
||||||
|
### 4.6 生命周期治理
|
||||||
|
|
||||||
|
模型不能因为 JSON 能解析、schema 通过、demo 能跑，就升级为 stable。
|
||||||
|
|
||||||
|
升级至少需要经过：
|
||||||
|
|
||||||
|
- evidence review；
|
||||||
|
- content review；
|
||||||
|
- regression review；
|
||||||
|
- selector review；
|
||||||
|
- Owner / CCRA review。
|
||||||
|
|
||||||
|
当前 QPI 和思想考古学仍保持：
|
||||||
|
|
||||||
|
```text
|
||||||
|
status: draft
|
||||||
|
stability_level: B
|
||||||
|
regression_status: pending
|
||||||
|
```
|
||||||
|
|
||||||
|
`draft-callable` 只能作为评审报告语言，不能替代模型生命周期字段。
|
||||||
|
|
||||||
|
## 5. 文件身份治理
|
||||||
|
|
||||||
|
项目中的文件应按身份区分，而不是都视为同等资产。
|
||||||
|
|
||||||
|
| 文件身份 | 是否长期保留 | 典型路径 | 作用 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Canonical source of truth | 是 | `models/*.model.json`, `cards/*.md`, `sources/*.json`, `tests/*.regression.json`, `selector/*.json` | 模型本体、来源、测试、调用规则 |
|
||||||
|
| Stable governance docs | 是 | `docs/*.md`, `knowledge_assets/*.md` | 长期规则、协议、解释层 |
|
||||||
|
| Generated / derived artifacts | 可重建，但可保留报告 | `models/model_index.json`, `cards/card_index.md`, `reports/*_report*.md` | 检查、导航、验证结果 |
|
||||||
|
| Round / review artifacts | 阶段归档 | `ccra_review_bundle/round-*`, `reports/Codex*.md` | 交接和审核证据 |
|
||||||
|
| Temporary / cache files | 不应提交 | `__pycache__/`, `*.pyc`, 临时 zip 展开目录 | 本地运行产物 |
|
||||||
|
|
||||||
|
判断标准：
|
||||||
|
|
||||||
|
```text
|
||||||
|
回答“以后一直怎么做”的文档，可以进入 docs/ 或 knowledge_assets/。
|
||||||
|
回答“这轮做了什么、哪些 PASS/FAIL”的文档，应留在 reports/ 或 ccra_review_bundle/。
|
||||||
|
```
|
||||||
|
|
||||||
|
## 6. 主要文件层
|
||||||
|
|
||||||
|
### 6.1 来源层
|
||||||
|
|
||||||
|
用途：回答“模型从哪里来”。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `sources/source_articles.json`
|
||||||
|
- `sources/source_excerpts.json`
|
||||||
|
- `sources/evidence_coverage.matrix.json`
|
||||||
|
|
||||||
|
### 6.2 人读模型层
|
||||||
|
|
||||||
|
用途：让 Owner、CCRA 和协作者读懂模型。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `cards/qpi.md`
|
||||||
|
- `cards/intellectual_archaeology.md`
|
||||||
|
- `cards/card_index.md`
|
||||||
|
|
||||||
|
### 6.3 机器模型层
|
||||||
|
|
||||||
|
用途：让 selector、validator、未来运行时读取模型。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `models/qpi.model.json`
|
||||||
|
- `models/intellectual_archaeology.model.json`
|
||||||
|
- `models/model_index.json`
|
||||||
|
|
||||||
|
### 6.4 契约与规则层
|
||||||
|
|
||||||
|
用途：约束模型卡、输出字段、数据结构和调用规则。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `schemas/model_card.schema.json`
|
||||||
|
- `docs/DATA_CONTRACT.md`
|
||||||
|
- `docs/QPI_CONTEXTUAL_ROUTING_RULES.md`
|
||||||
|
- `docs/INTELLECTUAL_ARCHAEOLOGY_DEPTH_GATE.md`
|
||||||
|
- `docs/DECISIONS.md`
|
||||||
|
|
||||||
|
### 6.5 Selector 层
|
||||||
|
|
||||||
|
用途：决定当前输入该调用哪些模型，以及不该调用哪些模型。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `selector/selector_rules.json`
|
||||||
|
- `selector/selector_examples.json`
|
||||||
|
- `selector/selector_calibration_inputs.json`
|
||||||
|
- `selector/qpi_case_digests.json`
|
||||||
|
- `scripts/run_selector_demo.py`
|
||||||
|
- `scripts/run_selector_regression.py`
|
||||||
|
- `scripts/run_selector_calibration_smoke.py`
|
||||||
|
|
||||||
|
### 6.6 Regression 层
|
||||||
|
|
||||||
|
用途：保护模型边界，避免以后修改规则时把模型改坏。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `tests/qpi.regression.json`
|
||||||
|
- `tests/intellectual_archaeology.regression.json`
|
||||||
|
- `tests/regression_cases.json`
|
||||||
|
|
||||||
|
### 6.7 Validation 层
|
||||||
|
|
||||||
|
用途：机械检查文件是否一致、字段是否完整、index 是否漂移、模型卡是否同步。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `scripts/validate_model_library.py`
|
||||||
|
- `scripts/check_card_contract.py`
|
||||||
|
- `scripts/check_model_card_sync.py`
|
||||||
|
- `scripts/rebuild_indexes.py`
|
||||||
|
- `reports/validation_report.md`
|
||||||
|
- `reports/index_rebuild_report.md`
|
||||||
|
- `reports/model_card_sync_report_v0.2.md`
|
||||||
|
|
||||||
|
### 6.8 Review Bundle 层
|
||||||
|
|
||||||
|
用途：每轮把 Codex 工作打包给 CCRA / GPT 审核，避免散文件上传。
|
||||||
|
|
||||||
|
典型文件：
|
||||||
|
|
||||||
|
- `ccra_review_bundle/round-XX_YYYY-MM-DD_topic/00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF.md`
|
||||||
|
- `ccra_review_bundle/round-XX_YYYY-MM-DD_topic/BUNDLE_FILE_MANIFEST.md`
|
||||||
|
- `ccra_review_bundle/round-XX_YYYY-MM-DD_topic/optional_raw_changed_files.zip`
|
||||||
|
|
||||||
|
Review bundle 是交接层，不是长期核心资产。
|
||||||
|
|
||||||
|
## 7. Selector 机制
|
||||||
|
|
||||||
|
Selector 是模型库的入口调度器和误召回防火墙。
|
||||||
|
|
||||||
|
它不负责回答问题。它负责判断：
|
||||||
|
|
||||||
|
- 当前输入是否需要模型加工；
|
||||||
|
- 如果需要，优先调用哪些模型；
|
||||||
|
- 哪些模型应该被拒绝；
|
||||||
|
- 拒绝理由是什么；
|
||||||
|
- 是否命中 no-call；
|
||||||
|
- 每个模型的分数、触发信号、惩罚项是什么。
|
||||||
|
|
||||||
|
当前 selector 仍然是 rule-based，不是 LLM selector。
|
||||||
|
|
||||||
|
基本流程：
|
||||||
|
|
||||||
|
```text
|
||||||
|
输入
|
||||||
|
-> 检查 hard no-call
|
||||||
|
-> 检查 explicit analysis override
|
||||||
|
-> 检查模型触发词
|
||||||
|
-> 检查复杂度信号
|
||||||
|
-> 检查模型特定 gate
|
||||||
|
-> 计算 score
|
||||||
|
-> 输出 selected / rejected models
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7.1 为什么当前不用 LLM selector
|
||||||
|
|
||||||
|
当前阶段最重要的是可审计。
|
||||||
|
|
||||||
|
LLM selector 可能更灵活，但会带来：
|
||||||
|
|
||||||
|
- 为什么选这个模型说不清；
|
||||||
|
- 为什么拒绝另一个模型说不清；
|
||||||
|
- 修改后是否破坏边界不好测；
|
||||||
|
- 容易把所有复杂问题交给重型模型；
|
||||||
|
- 不利于 Codex 本地测试和回归。
|
||||||
|
|
||||||
|
规则 selector 更保守，但更可控。
|
||||||
|
|
||||||
|
### 7.2 Selector 的核心价值
|
||||||
|
|
||||||
|
Selector 保护三件事：
|
||||||
|
|
||||||
|
1. 防止不该调用时调用：明确事实查询、轻量改写、直接执行任务不应启动 QPI 或思想考古。
|
||||||
|
2. 防止重型模型过早进入：思想考古学不应仅因出现“底层”“模型”“哲学”等词就被召回。
|
||||||
|
3. 让模型组合可解释：未来不是一个模型回答所有问题，而是多个模型按流程协作。
|
||||||
|
|
||||||
|
## 8. Regression 机制
|
||||||
|
|
||||||
|
Regression 在本项目中不是普通单元测试，而是模型边界保护机制。
|
||||||
|
|
||||||
|
它要回答：
|
||||||
|
|
||||||
|
- 该调用模型时是否调用；
|
||||||
|
- 不该调用模型时是否拒绝；
|
||||||
|
- Q / P / I / mixed / no-call 是否被误判；
|
||||||
|
- mixed 输入是否暴露证据缺口；
|
||||||
|
- 是否出现暴力降维；
|
||||||
|
- 是否出现恶意升维；
|
||||||
|
- 是否把轻量问题过度重型化；
|
||||||
|
- 是否把深度模型误召回；
|
||||||
|
- 修改 selector 后，过去关键边界是否被破坏。
|
||||||
|
|
||||||
|
Regression case 是“防止系统退化的钉子”，不是普通示例。
|
||||||
|
|
||||||
|
至少覆盖：
|
||||||
|
|
||||||
|
- `positive`
|
||||||
|
- `boundary`
|
||||||
|
- `misuse`
|
||||||
|
- `no_call`
|
||||||
|
- `selector_gate`
|
||||||
|
- `pipeline`
|
||||||
|
|
||||||
|
## 9. Digest、Calibration、Regression 的区别
|
||||||
|
|
||||||
|
以 QPI 为例，Owner 提供人工素材后，Codex 将材料拆为 `.cases.md`、digest、calibration、regression 四层。
|
||||||
|
|
||||||
|
### 9.1 `.cases.md`
|
||||||
|
|
||||||
|
人读的案例审阅稿。
|
||||||
|
|
||||||
|
作用：
|
||||||
|
|
||||||
|
- 保留原始案例；
|
||||||
|
- 保留 Owner / GPT 审查判断；
|
||||||
|
- 保留人能看懂的推理；
|
||||||
|
- 便于后续人工复核。
|
||||||
|
|
||||||
|
### 9.2 Case Digest
|
||||||
|
|
||||||
|
压缩后的结构化案例摘要。
|
||||||
|
|
||||||
|
作用：
|
||||||
|
|
||||||
|
- 让案例变得可检索、可审计；
|
||||||
|
- 保留核心分类、主导稀缺、误用风险、边界说明；
|
||||||
|
- 作为 selector / regression 的候选素材池。
|
||||||
|
|
||||||
|
Digest 是案例资产层，不是最终测试层。
|
||||||
|
|
||||||
|
### 9.3 Calibration Input
|
||||||
|
|
||||||
|
给 selector 调参和校准用的输入。
|
||||||
|
|
||||||
|
作用：
|
||||||
|
|
||||||
|
- 告诉 selector 哪些输入应该选 QPI；
|
||||||
|
- 哪些输入应该 no-call；
|
||||||
|
- 哪些输入应该低优先级；
|
||||||
|
- 哪些输入需要先 QPI 再进入思想考古；
|
||||||
|
- 哪些输入容易误召回。
|
||||||
|
|
||||||
|
Calibration 是“调方向”。
|
||||||
|
|
||||||
|
### 9.4 Regression Case
|
||||||
|
|
||||||
|
高价值边界测试。
|
||||||
|
|
||||||
|
作用：
|
||||||
|
|
||||||
|
- 以后每次改规则都要检查；
|
||||||
|
- 防止关键边界被破坏；
|
||||||
|
- 不要求覆盖所有案例；
|
||||||
|
- 只保留最容易出错、最值得保护的判断。
|
||||||
|
|
||||||
|
Regression 是“守底线”。
|
||||||
|
|
||||||
|
## 10. QPI 调用机制
|
||||||
|
|
||||||
|
QPI 不是最终答案模型，而是入口路由模型。
|
||||||
|
|
||||||
|
它处理的不是“怎么解决问题”，而是：
|
||||||
|
|
||||||
|
```text
|
||||||
|
当前输入到底是什么性质的问题？
|
||||||
|
```
|
||||||
|
|
||||||
|
运行方式：
|
||||||
|
|
||||||
|
```text
|
||||||
|
用户输入
|
||||||
|
-> selector 判断是否需要 QPI
|
||||||
|
-> QPI 分析主体、场景、责任范围、期望—现实落差
|
||||||
|
-> 判断主导稀缺物
|
||||||
|
-> 输出 Q / P / I / mixed / no-call
|
||||||
|
-> 给出证据缺口、误分类风险、下一步模型候选
|
||||||
|
-> 进入后续模型或直接行动
|
||||||
|
```
|
||||||
|
|
||||||
|
QPI 的五种结果：
|
||||||
|
|
||||||
|
| QPI 输出 | 含义 | 系统下一步 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Question | 数据不足 | 搜索、查证、补信息 |
|
||||||
|
| Problem | 路径、方法或资源不足 | 做方案、流程、SOP、资源约束分析 |
|
||||||
|
| Issue | 共识、秩序、确定性或治理结构不足 | 做多视角分析、动态权衡、思想考古或冲突处理 |
|
||||||
|
| mixed | 多类稀缺同时存在 | 拆分问题，分别路由 |
|
||||||
|
| no-call | 不需要问题定性 | 直接执行、改写、翻译、查事实、整理格式 |
|
||||||
|
|
||||||
|
QPI 不应直接输出组织、人事、法律、财务、运营解决方案。
|
||||||
|
|
||||||
|
它最多回答：
|
||||||
|
|
||||||
|
- 这是什么类型的问题；
|
||||||
|
- 为什么这样分类；
|
||||||
|
- 证据是否足够；
|
||||||
|
- 误判风险是什么；
|
||||||
|
- 下一步应该进入哪类处理。
|
||||||
|
|
||||||
|
## 11. 思想考古学调用机制
|
||||||
|
|
||||||
|
思想考古学不是默认分析流程，而是深度建模模型。
|
||||||
|
|
||||||
|
适合使用的条件：
|
||||||
|
|
||||||
|
- 问题表层现象很多，但底层假设不清；
|
||||||
|
- 需要识别概念、模型或判断背后的深层机制；
|
||||||
|
- QPI 已判断这是中重型 Problem / Issue；
|
||||||
|
- 继续下潜会改变判断、路径、验证方式或行动边界。
|
||||||
|
|
||||||
|
不适合：
|
||||||
|
|
||||||
|
- 明确事实查询；
|
||||||
|
- 低风险轻量改写；
|
||||||
|
- 用户只需要直接执行；
|
||||||
|
- 材料不足，无法区分真实假设和空泛哲学化表达。
|
||||||
|
|
||||||
|
关键原则：
|
||||||
|
|
||||||
|
```text
|
||||||
|
最小充分下潜。
|
||||||
|
如果继续下潜不再改变判断、路径、验证方式或行动边界，就应停止。
|
||||||
|
```
|
||||||
|
|
||||||
|
未来系统不是“QPI 一调用就自动思想考古”，而是：
|
||||||
|
|
||||||
|
```text
|
||||||
|
QPI 先判断问题性质
|
||||||
|
-> selector 判断是否满足思想考古 depth gate
|
||||||
|
-> 思想考古只分析必要层级
|
||||||
|
-> 达到充分深度就停止
|
||||||
|
```
|
||||||
|
|
||||||
|
## 12. 未来新增模型的最低资产结构
|
||||||
|
|
||||||
|
每个未来模型都不应只是一个概念。
|
||||||
|
|
||||||
|
最低需要七类资产：
|
||||||
|
|
||||||
|
1. 人读解释：`cards/*.md`
|
||||||
|
2. 机器可读定义：`models/*.model.json`
|
||||||
|
3. 来源证据：`sources/source_articles.json`、`sources/source_excerpts.json`
|
||||||
|
4. 调用规则：`selector/selector_rules.json`、`selector/selector_calibration_inputs.json`
|
||||||
|
5. 输出契约：模型 JSON 中的 `structured_output_contract`
|
||||||
|
6. 回归案例：`tests/*.regression.json`
|
||||||
|
7. 审核与版本状态：reports、review bundle、model/card index
|
||||||
|
|
||||||
|
新增模型不得用来绕过当前模型边界未稳定的问题。
|
||||||
|
|
||||||
|
## 13. 未来运行时调用流程
|
||||||
|
|
||||||
|
未来真正运行时，系统可按以下流程工作：
|
||||||
|
|
||||||
|
```text
|
||||||
|
1. 用户输入问题 / 话题 / 文本 / 任务
|
||||||
|
|
||||||
|
2. 输入预处理
|
||||||
|
- 识别语言
|
||||||
|
- 判断是否是直接执行任务
|
||||||
|
- 判断是否需要认知加工
|
||||||
|
- 抽取显性任务目标
|
||||||
|
|
||||||
|
3. Selector 路由
|
||||||
|
- 先检查 hard no-call
|
||||||
|
- 再检查 explicit analysis override
|
||||||
|
- 再根据模型触发条件打分
|
||||||
|
- 输出 selected / rejected models、分数和理由
|
||||||
|
|
||||||
|
4. 前置模型
|
||||||
|
- 常见情况下先调用 QPI
|
||||||
|
- QPI 判断 Q / P / I / mixed / no-call
|
||||||
|
- 输出下一步模型候选
|
||||||
|
|
||||||
|
5. 深度或专项模型
|
||||||
|
- 如果是中重型 Problem / Issue，可能进入思想考古
|
||||||
|
- 不满足 gate 的模型不得调用
|
||||||
|
|
||||||
|
6. 多模型结果汇总
|
||||||
|
- 比较不同模型输出
|
||||||
|
- 标记冲突
|
||||||
|
- 标记证据缺口
|
||||||
|
- 标记适用边界
|
||||||
|
|
||||||
|
7. 输出给用户
|
||||||
|
- 包含判断路径、模型调用理由、边界、下一步动作
|
||||||
|
|
||||||
|
8. 记录反馈
|
||||||
|
- 用户纠正分类或边界
|
||||||
|
- 重要反馈进入 calibration 或 regression
|
||||||
|
```
|
||||||
|
|
||||||
|
## 14. Codex 运作原则
|
||||||
|
|
||||||
|
后续 Codex 应遵守：
|
||||||
|
|
||||||
|
1. 不把 GPT 规划直接当本地规则，必须先本地化为 schema、workflow、validator、index。
|
||||||
|
2. 不把文章摘要当模型抽取。
|
||||||
|
3. 不把模型卡完整当成模型稳定。
|
||||||
|
4. 不把 selector demo pass 当成内容稳定。
|
||||||
|
5. 不把 validation pass 当成 Owner 审核通过。
|
||||||
|
6. 不因为素材增多就无限扩展 regression。
|
||||||
|
7. 不把 calibration 全部升级成 regression。
|
||||||
|
8. 不新增模型来解决当前模型边界没稳定的问题。
|
||||||
|
9. 每个新增文件必须说明身份：canonical / generated / report / temporary。
|
||||||
|
10. 每轮交接必须用 review bundle，不让 Owner / CCRA 面对散乱文件。
|
||||||
|
|
||||||
|
## 15. 与 GPT 知识库同步的关系
|
||||||
|
|
||||||
|
`knowledge_assets/` 是长期解释层。
|
||||||
|
|
||||||
|
Owner 可以手动将其中稳定文档同步到 GPT 知识库。评审包不应重复打包 `knowledge_assets/`，除非某轮评审明确要求审核长期资产本身。
|
||||||
|
|
||||||
|
当前规则：
|
||||||
|
|
||||||
|
- 长期机制说明放在 `knowledge_assets/`；
|
||||||
|
- 当前执行资产放在 `models/`、`cards/`、`selector/`、`tests/`、`scripts/`；
|
||||||
|
- 每轮评审资料放在 `ccra_review_bundle/round-*`；
|
||||||
|
- `optional_raw_changed_files.zip` 应保留源路径，避免扁平化覆盖；
|
||||||
|
- `knowledge_assets/` 默认不放入评审 zip，由 Owner 自行同步到 GPT 知识库。
|
||||||
|
|
||||||
|
## 16. 结论
|
||||||
|
|
||||||
|
本项目不是把少量文章堆进知识库。
|
||||||
|
|
||||||
|
它在做的是：
|
||||||
|
|
||||||
|
```text
|
||||||
|
把文章形式存在的个人认知模型
|
||||||
|
转化为可被 AI 软件稳定调用的模型资产库；
|
||||||
|
同时建立调用门、拒绝门、输出契约、边界测试和人机交接机制。
|
||||||
|
```
|
||||||
|
|
||||||
|
QPI 是第一个压力测试样板。
|
||||||
|
|
||||||
|
思想考古学是第二个深度模型样板。
|
||||||
|
|
||||||
|
Selector 是模型调用的守门员。
|
||||||
|
|
||||||
|
Regression 是模型边界的质检夹具。
|
||||||
|
|
||||||
|
Model card 是人和机器之间的共同契约。
|
||||||
|
|
||||||
|
Source / evidence 是模型不漂移的锚点。
|
||||||
|
|
||||||
|
Review bundle 是 Codex、CCRA、Owner 之间的交接机制。
|
||||||
|
|
@ -12,5 +12,18 @@ Rules:
|
||||||
- Keep session-level validation and handoff documents in `reports/`.
|
- Keep session-level validation and handoff documents in `reports/`.
|
||||||
- Do not put version numbers in filenames; record version information inside the document.
|
- Do not put version numbers in filenames; record version information inside the document.
|
||||||
- Store long-term CCRA quality gates and handoff protocols here, but do not store per-round review bundles or command logs here.
|
- Store long-term CCRA quality gates and handoff protocols here, but do not store per-round review bundles or command logs here.
|
||||||
|
- Store durable data-governance and model-invocation explanations here when they answer "how this model library should keep working", not "what happened in this review round".
|
||||||
|
- Review bundles should not include `knowledge_assets/` by default; the project owner manually syncs stable knowledge assets into GPT knowledge storage.
|
||||||
|
|
||||||
|
Current reading order:
|
||||||
|
|
||||||
|
- `00_用户背景与产品上下文.md`
|
||||||
|
- `01_核心模型地图.md`
|
||||||
|
- `02_模型卡结构规范.md`
|
||||||
|
- `03_核心模型抽取样板.md`
|
||||||
|
- `06_模型稳固性评级规则.md`
|
||||||
|
- `07_产品规划过程记录.md`
|
||||||
|
- `08_CCRA模型库MVP质量门与交接协议.md`
|
||||||
|
- `09_数据治理与模型调用机制说明.md`
|
||||||
|
|
||||||
See `docs/KNOWLEDGE_ASSET_RULES.md`.
|
See `docs/KNOWLEDGE_ASSET_RULES.md`.
|
||||||
|
|
|
||||||
|
|
@ -26,11 +26,11 @@
|
||||||
"card_file": "cards/qpi.md",
|
"card_file": "cards/qpi.md",
|
||||||
"source_article_count": 3,
|
"source_article_count": 3,
|
||||||
"source_evidence_count": 10,
|
"source_evidence_count": 10,
|
||||||
"regression_case_count": 46,
|
"regression_case_count": 52,
|
||||||
"stability_level": "B",
|
"stability_level": "B",
|
||||||
"regression_status": "pending",
|
"regression_status": "pending",
|
||||||
"status": "draft",
|
"status": "draft",
|
||||||
"last_updated": "2026-06-16"
|
"last_updated": "2026-06-17"
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -239,5 +239,5 @@
|
||||||
],
|
],
|
||||||
"productization_notes": "QPI 应作为问题回答系统的前置路由模型，用于防止系统在问题类型错误的情况下直接给答案。",
|
"productization_notes": "QPI 应作为问题回答系统的前置路由模型，用于防止系统在问题类型错误的情况下直接给答案。",
|
||||||
"version": "0.1",
|
"version": "0.1",
|
||||||
"last_updated": "2026-06-16"
|
"last_updated": "2026-06-17"
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,151 @@
|
||||||
|
# Codex 新会话交接文档：Round 03.1 评审后续
|
||||||
|
|
||||||
|
Date: 2026-06-17
|
||||||
|
|
||||||
|
Repository: `C:\Users\wangq\Documents\Codex\work-projects\the-mindscape-of-bro-tsong`
|
||||||
|
|
||||||
|
Current phase: `model_library_mvp`
|
||||||
|
|
||||||
|
## 1. 新会话目标
|
||||||
|
|
||||||
|
下一会话预计处理 GPT 对 Round 03.1 的评审结果。
|
||||||
|
|
||||||
|
请先读取本交接文档，再读取 Round 03.1 评审包和 GPT 返回意见。
|
||||||
|
|
||||||
|
## 2. 当前模型状态
|
||||||
|
|
||||||
|
不要升级生命周期状态，除非 Owner 明确要求。
|
||||||
|
|
||||||
|
当前仍为：
|
||||||
|
|
||||||
|
```text
|
||||||
|
qpi: draft / B / pending
|
||||||
|
intellectual_archaeology: draft / B / pending
|
||||||
|
```
|
||||||
|
|
||||||
|
允许使用 `draft-callable` 作为评审报告语言，但不能把它写成模型 JSON 的正式 `status`。
|
||||||
|
|
||||||
|
硬边界仍然有效：
|
||||||
|
|
||||||
|
- 不新增第三模型；
|
||||||
|
- 不升级 stable；
|
||||||
|
- 不引入 LLM selector；
|
||||||
|
- 不做完整问答系统；
|
||||||
|
- 不做 RAG / vector database；
|
||||||
|
- 不做前端、后端、用户系统或平台化扩展。
|
||||||
|
|
||||||
|
## 3. Round 03.1 已完成内容
|
||||||
|
|
||||||
|
Round 03.1 是对 Round 03 的小修补，不是新一轮大扩展。
|
||||||
|
|
||||||
|
已完成：
|
||||||
|
|
||||||
|
- 修复 selector 过度选择 QPI 的问题；
|
||||||
|
- QPI 不再能仅凭 `base_score + selection_priority` 被选中；
|
||||||
|
- 增加 direct-execution no-call signals；
|
||||||
|
- 保留 explicit analysis override；
|
||||||
|
- 新增 `scripts/run_selector_calibration_smoke.py`；
|
||||||
|
- QPI regression 从 46 条扩展到 52 条；
|
||||||
|
- aggregate regression 从 63 条扩展到 69 条；
|
||||||
|
- selector calibration inputs 从 83 条扩展到 85 条；
|
||||||
|
- `qpi_case_digests.json` 字段规范化：
|
||||||
|
- `misframing_risks` -> `misclassification_risk`
|
||||||
|
- `mixed_or_multi_perspective` -> `qpi_complexity_pattern`
|
||||||
|
- multi-perspective / inter-viewpoint case 必须有 `classification_by_viewpoint` 或 `viewpoint_summary`
|
||||||
|
- `qpi.md`、`qpi.model.json`、`content_review_report_v0.2.md` 同步 stale 状态；
|
||||||
|
- Round 03.1 review bundle 已生成；
|
||||||
|
- Round 03.1 zip 保留源路径，不再扁平化；
|
||||||
|
- Round 03.1 zip 默认不包含 `knowledge_assets/`。
|
||||||
|
|
||||||
|
## 4. 本次新补的长期资产和规则
|
||||||
|
|
||||||
|
新增长期资产：
|
||||||
|
|
||||||
|
- `knowledge_assets/09_数据治理与模型调用机制说明.md`
|
||||||
|
|
||||||
|
该文件来自 GPT 成果 `CCRA_数据治理与模型调用机制说明_v0.1.md` 的整理版，但不是原文搬运。它删除了对话式回复和临时评审上下文，只保留长期可复用的机制说明。
|
||||||
|
|
||||||
|
新增规则文件：
|
||||||
|
|
||||||
|
- `docs/FILE_TAXONOMY.md`
|
||||||
|
|
||||||
|
定位：
|
||||||
|
|
||||||
|
- `docs/FILE_TAXONOMY.md` 管全仓库文件身份；
|
||||||
|
- `docs/KNOWLEDGE_ASSET_RULES.md` 只管 `knowledge_assets/`；
|
||||||
|
- `docs/DECISIONS.md` 记录已接受的结构决策。
|
||||||
|
|
||||||
|
文件身份四类：
|
||||||
|
|
||||||
|
- canonical source of truth；
|
||||||
|
- generated / derived；
|
||||||
|
- review archive；
|
||||||
|
- temporary / local runtime。
|
||||||
|
|
||||||
|
## 5. Round 03.1 评审包位置
|
||||||
|
|
||||||
|
```text
|
||||||
|
ccra_review_bundle/round-03.1_2026-06-17_selector-no-call-regression-patch/
|
||||||
|
```
|
||||||
|
|
||||||
|
建议提交给 GPT 的读取顺序：
|
||||||
|
|
||||||
|
1. `00_OPEN_THIS_FIRST_CCRA_REVIEW_BRIEF.md`
|
||||||
|
2. `01_PATCH_MATRIX.md`
|
||||||
|
3. `02_CURRENT_ASSET_PACK.md`
|
||||||
|
4. `03_VALIDATION_AND_COMMAND_LOG.md`
|
||||||
|
5. `04_REVIEW_QUESTIONS_FOR_GPT.md`
|
||||||
|
6. `optional_raw_changed_files.zip` only if exact file inspection is needed
|
||||||
|
|
||||||
|
注意：
|
||||||
|
|
||||||
|
- `optional_raw_changed_files.zip` 已保留 source-relative paths；
|
||||||
|
- zip 中 `knowledge_assets` 条目为 0；
|
||||||
|
- Owner 会手动同步 `knowledge_assets/09_数据治理与模型调用机制说明.md` 到 GPT 知识库；
|
||||||
|
- 不要为了评审 Round 03.1 把 `knowledge_assets/` 加回 zip。
|
||||||
|
|
||||||
|
## 6. 当前验证结果
|
||||||
|
|
||||||
|
最近验证通过：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
python scripts\rebuild_indexes.py --check
|
||||||
|
python -m unittest discover -s tests -p "test*.py" -v
|
||||||
|
python scripts\validate_model_library.py
|
||||||
|
python scripts\check_card_contract.py
|
||||||
|
python scripts\run_selector_demo.py
|
||||||
|
python scripts\run_selector_regression.py
|
||||||
|
python scripts\run_selector_calibration_smoke.py
|
||||||
|
python scripts\check_model_card_sync.py
|
||||||
|
```
|
||||||
|
|
||||||
|
结果：
|
||||||
|
|
||||||
|
- Index check: PASS
|
||||||
|
- Unit tests: PASS, 17 tests
|
||||||
|
- Model library validation: PASS
|
||||||
|
- Card contract: PASS
|
||||||
|
- Selector demo: PASS
|
||||||
|
- Selector regression: PASS
|
||||||
|
- Selector calibration smoke: PASS
|
||||||
|
- Model/card sync: PASS
|
||||||
|
|
||||||
|
## 7. 新会话处理 GPT 评审结果时的建议流程
|
||||||
|
|
||||||
|
1. 先判断 GPT 结论是 pass / revise / block。
|
||||||
|
2. 如果是 revise，逐条映射到具体文件，不要扩大 scope。
|
||||||
|
3. selector 相关问题优先用 regression 或 calibration smoke 固化。
|
||||||
|
4. digest 字段问题优先改 validator，防止回漂。
|
||||||
|
5. 不要把 calibration 全部升级成 regression。
|
||||||
|
6. 不要新增模型来解决当前 QPI 或思想考古边界问题。
|
||||||
|
7. 每次改完后重新运行完整验证链。
|
||||||
|
8. 若要生成新评审包，继续放在 `ccra_review_bundle/round-03.1_...` 或新建清楚的后续小修补目录，不要覆盖旧证据。
|
||||||
|
|
||||||
|
## 8. 需要特别避免
|
||||||
|
|
||||||
|
- 不要把 GPT 长文原样放入 `knowledge_assets/`；
|
||||||
|
- 不要把 `knowledge_assets/` 放入 Round 03.1 raw zip；
|
||||||
|
- 不要创建扁平 zip；
|
||||||
|
- 不要提交 `__pycache__/` 或 `*.pyc`；
|
||||||
|
- 不要把 `draft-callable` 写成模型生命周期状态；
|
||||||
|
- 不要把 validation pass 解释为 Owner/CCRA 内容通过。
|
||||||
|
|
@ -2,6 +2,8 @@
|
||||||
|
|
||||||
Date: 2026-06-16
|
Date: 2026-06-16
|
||||||
|
|
||||||
|
Status note: this report is a pre-case-promotion review snapshot. It remains useful for the v0.2 evidence / contract / selector baseline, but its regression counts do not reflect the Round 03 / 03.1 QPI owner-reviewed case promotion. Current QPI regression counts are tracked in `tests/qpi.regression.json`, `models/model_index.json`, and the Round 03.1 review bundle.
|
||||||
|
|
||||||
## 1. 本轮修改摘要
|
## 1. 本轮修改摘要
|
||||||
|
|
||||||
本轮没有扩展第三模型，没有接完整问题回答系统，没有引入 LLM selector，也没有升级 stable。
|
本轮没有扩展第三模型，没有接完整问题回答系统，没有引入 LLM selector，也没有升级 stable。
|
||||||
|
|
@ -109,6 +111,8 @@ Coverage:
|
||||||
- 已覆盖误用、防误召回、no-call 和 pipeline gate。
|
- 已覆盖误用、防误召回、no-call 和 pipeline gate。
|
||||||
- 仍需人工审查用例真实性和遗漏边界。
|
- 仍需人工审查用例真实性和遗漏边界。
|
||||||
|
|
||||||
|
Round 03 / 03.1 update: QPI has since expanded to 52 regression cases after owner-reviewed case promotion and selector/no-call repair. This v0.2 table is retained as a historical baseline, not the current count.
|
||||||
|
|
||||||
## 6. Selector Regression 结论
|
## 6. Selector Regression 结论
|
||||||
|
|
||||||
Selector v0.2 仍为规则 selector。
|
Selector v0.2 仍为规则 selector。
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,96 @@
|
||||||
|
# Selector Calibration Smoke Report
|
||||||
|
|
||||||
|
Status: `PASS`
|
||||||
|
|
||||||
|
Command: `python scripts/run_selector_calibration_smoke.py`
|
||||||
|
|
||||||
|
Calibration inputs checked: 85
|
||||||
|
Failures: 0
|
||||||
|
|
||||||
|
## Cases
|
||||||
|
|
||||||
|
- `selector_calibration_fact_lookup_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_fact_lookup_002`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_rewrite_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_translation_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_direct_execution_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_qpi_only_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_only_002`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_insufficient_context_001`: PASS; expected=select_qpi_low_confidence; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_intra_frame_mixed_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_governance_load_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_ia_after_qpi_001`: PASS; expected=select_intellectual_archaeology; selected=['intellectual_archaeology', 'qpi']; no_call=False
|
||||||
|
- `selector_calibration_ia_after_qpi_002`: PASS; expected=select_intellectual_archaeology; selected=['intellectual_archaeology']; no_call=False
|
||||||
|
- `selector_calibration_ia_model_extraction_001`: PASS; expected=select_intellectual_archaeology; selected=['intellectual_archaeology', 'qpi']; no_call=False
|
||||||
|
- `selector_calibration_ia_model_extraction_002`: PASS; expected=select_intellectual_archaeology; selected=['intellectual_archaeology', 'qpi']; no_call=False
|
||||||
|
- `selector_calibration_false_positive_deep_word_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_false_positive_model_word_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_false_positive_philosophy_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_override_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_pipeline_001`: PASS; expected=select_qpi_reject_ia; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_pipeline_002`: PASS; expected=select_intellectual_archaeology; selected=['intellectual_archaeology', 'qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_flow_entry_point_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_no_simulation_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_local_truth_global_structure_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_stop_gate_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_intra_frame_mixed_flow_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_complexity_placement_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_psych_mechanism_ambiguity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_label_as_identity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_single_factor_totalization_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_one_time_fix_trap_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_external_authority_boundary_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_time_scale_scope_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_mismatch_diagnostics_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_one_person_issue_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_hard_resource_documentation_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_credential_continuity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_incentive_backlash_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_role_authority_mismatch_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_pattern_level_issue_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_untrusted_data_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_workaround_translation_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_org_low_cost_indicator_survival_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_ai_evidence_boundary_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_staffing_deadlock_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_policy_memory_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_strategy_myopia_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_resource_reuse_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_outcome_excuse_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_digital_governance_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_academic_bottom_line_control_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_reverse_accountability_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_credential_compliance_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_reported_metric_legitimacy_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_overpromising_pipeline_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_metric_credibility_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_asset_integration_boundary_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_authority_transfer_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_employment_transactional_education_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_target_authenticity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_equipment_solutionism_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_shortcut_debt_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_credential_integrity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_market_closure_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_micro_control_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_asset_liability_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_engineering_personnel_transition_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_compliance_retention_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_capacity_gate_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_logistics_no_call_001`: PASS; expected=no_call_or_low_priority; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_qpi_international_role_inflation_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_viewpoint_divergence_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_metric_distortion_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_pricing_trust_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_capability_institutionalization_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_international_asset_expansion_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_identity_simulation_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_accounting_reduction_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_transactional_assets_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_ranking_trap_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_talent_role_mismatch_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_hard_capacity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_safety_accountability_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_qpi_research_penalty_integrity_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
- `selector_calibration_direct_summary_no_call_001`: PASS; expected=no_call; selected=[]; no_call=True
|
||||||
|
- `selector_calibration_analysis_override_should_call_001`: PASS; expected=select_qpi; selected=['qpi']; no_call=False
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
Status: `PASS`
|
Status: `PASS`
|
||||||
|
|
||||||
Cases checked: 10
|
Cases checked: 13
|
||||||
Failures: 0
|
Failures: 0
|
||||||
|
|
||||||
## Cases
|
## Cases
|
||||||
|
|
@ -17,3 +17,6 @@ Failures: 0
|
||||||
- `case_qpi_selector_gate_fact_001`: PASS; selected=[]; no_call=True
|
- `case_qpi_selector_gate_fact_001`: PASS; selected=[]; no_call=True
|
||||||
- `case_qpi_pipeline_before_ia_001`: PASS; selected=['qpi']; no_call=False
|
- `case_qpi_pipeline_before_ia_001`: PASS; selected=['qpi']; no_call=False
|
||||||
- `case_qpi_disappointment_mismatch_diagnostics_001`: PASS; selected=['qpi']; no_call=False
|
- `case_qpi_disappointment_mismatch_diagnostics_001`: PASS; selected=['qpi']; no_call=False
|
||||||
|
- `case_qpi_international_logistics_no_call_001`: PASS; selected=[]; no_call=True
|
||||||
|
- `case_qpi_direct_summary_no_call_001`: PASS; selected=[]; no_call=True
|
||||||
|
- `case_qpi_analysis_override_should_call_001`: PASS; selected=['qpi']; no_call=False
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,101 @@
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from run_selector_demo import load_selector_rules, recommend
|
||||||
|
|
||||||
|
|
||||||
|
def read_json(path):
|
||||||
|
return json.loads(Path(path).read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def selected_model_ids(result):
|
||||||
|
return [item["model_id"] for item in result.get("selected_models", [])]
|
||||||
|
|
||||||
|
|
||||||
|
def has_ia_gate(input_text, selector_rules):
|
||||||
|
global_rules = selector_rules.get("global_rules", {})
|
||||||
|
heavy_signals = global_rules.get("ia_heavy_signals", [])
|
||||||
|
return any(signal in input_text for signal in heavy_signals) or "QPI 已判断" in input_text
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_input(root, selector_rules, item):
|
||||||
|
result = recommend(root, item["input"])
|
||||||
|
selected_ids = selected_model_ids(result)
|
||||||
|
expected = item.get("expected_selector_behavior")
|
||||||
|
errors = []
|
||||||
|
|
||||||
|
if expected in {"no_call", "no_call_or_low_priority"} and "qpi" in selected_ids:
|
||||||
|
errors.append("expected no QPI selection")
|
||||||
|
|
||||||
|
if expected in {"select_qpi", "select_qpi_low_confidence", "select_qpi_reject_ia"}:
|
||||||
|
if "qpi" not in selected_ids:
|
||||||
|
errors.append("expected QPI selection")
|
||||||
|
|
||||||
|
if expected == "select_qpi_reject_ia" and "intellectual_archaeology" in selected_ids:
|
||||||
|
errors.append("expected IA rejection")
|
||||||
|
|
||||||
|
if expected == "select_intellectual_archaeology":
|
||||||
|
if "intellectual_archaeology" not in selected_ids:
|
||||||
|
errors.append("expected IA selection")
|
||||||
|
if not has_ia_gate(item["input"], selector_rules):
|
||||||
|
errors.append("IA selected expectation lacks heavy-depth or QPI-completed gate")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"case_id": item["case_id"],
|
||||||
|
"expected_selector_behavior": expected,
|
||||||
|
"selected_models": selected_ids,
|
||||||
|
"no_call": result.get("no_call"),
|
||||||
|
"errors": errors,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def write_report(root, results):
|
||||||
|
report_path = root / "reports" / "selector_calibration_smoke_report.md"
|
||||||
|
failures = [result for result in results if result["errors"]]
|
||||||
|
lines = [
|
||||||
|
"# Selector Calibration Smoke Report",
|
||||||
|
"",
|
||||||
|
f"Status: `{'PASS' if not failures else 'FAIL'}`",
|
||||||
|
"",
|
||||||
|
"Command: `python scripts/run_selector_calibration_smoke.py`",
|
||||||
|
"",
|
||||||
|
f"Calibration inputs checked: {len(results)}",
|
||||||
|
f"Failures: {len(failures)}",
|
||||||
|
"",
|
||||||
|
"## Cases",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
for result in results:
|
||||||
|
status = "PASS" if not result["errors"] else "FAIL"
|
||||||
|
lines.append(
|
||||||
|
f"- `{result['case_id']}`: {status}; expected={result['expected_selector_behavior']}; "
|
||||||
|
f"selected={result['selected_models']}; no_call={result['no_call']}"
|
||||||
|
)
|
||||||
|
for error in result["errors"]:
|
||||||
|
lines.append(f" - {error}")
|
||||||
|
report_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||||
|
return report_path
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
root = Path(__file__).resolve().parents[1]
|
||||||
|
selector_rules = load_selector_rules(root)
|
||||||
|
calibration = read_json(root / "selector" / "selector_calibration_inputs.json")
|
||||||
|
results = [
|
||||||
|
evaluate_input(root, selector_rules, item)
|
||||||
|
for item in calibration.get("inputs", [])
|
||||||
|
]
|
||||||
|
report_path = write_report(root, results)
|
||||||
|
print(f"selector calibration smoke report written to {report_path}")
|
||||||
|
failures = [result for result in results if result["errors"]]
|
||||||
|
if failures:
|
||||||
|
for result in failures:
|
||||||
|
print(f"ERROR: {result['case_id']}: {'; '.join(result['errors'])}")
|
||||||
|
return 1
|
||||||
|
print("selector calibration smoke passed")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
|
|
@ -28,7 +28,9 @@ def has_analysis_override(user_input, global_rules):
|
||||||
|
|
||||||
|
|
||||||
def hard_no_call_hits(user_input, global_rules):
|
def hard_no_call_hits(user_input, global_rules):
|
||||||
return hit_any(user_input, global_rules.get("hard_no_call_signals", []))
|
signals = list(global_rules.get("hard_no_call_signals", []))
|
||||||
|
signals.extend(global_rules.get("direct_execution_no_call_signals", []))
|
||||||
|
return hit_any(user_input, signals)
|
||||||
|
|
||||||
|
|
||||||
def score_model(model, model_rule, global_rules, user_input, task_type="", pipeline_position=""):
|
def score_model(model, model_rule, global_rules, user_input, task_type="", pipeline_position=""):
|
||||||
|
|
@ -39,7 +41,7 @@ def score_model(model, model_rule, global_rules, user_input, task_type="", pipel
|
||||||
penalties = []
|
penalties = []
|
||||||
|
|
||||||
negative_hits = hit_any(user_input, model_rule.get("negative_triggers", [])) + hit_any(user_input, model.get("negative_triggers", []))
|
negative_hits = hit_any(user_input, model_rule.get("negative_triggers", [])) + hit_any(user_input, model.get("negative_triggers", []))
|
||||||
if negative_hits:
|
if negative_hits and not has_analysis_override(user_input, global_rules):
|
||||||
penalty_key = "model_negative_penalty_qpi" if model_id == "qpi" else "model_negative_penalty_default"
|
penalty_key = "model_negative_penalty_qpi" if model_id == "qpi" else "model_negative_penalty_default"
|
||||||
score -= weights.get(penalty_key, 0.5)
|
score -= weights.get(penalty_key, 0.5)
|
||||||
penalties.append("negative trigger: " + "、".join(sorted(set(negative_hits))[:4]))
|
penalties.append("negative trigger: " + "、".join(sorted(set(negative_hits))[:4]))
|
||||||
|
|
@ -74,6 +76,19 @@ def score_model(model, model_rule, global_rules, user_input, task_type="", pipel
|
||||||
if model_id == "qpi" and qpi_gate_hits:
|
if model_id == "qpi" and qpi_gate_hits:
|
||||||
score += weights.get("qpi_gate_bonus", 0.25)
|
score += weights.get("qpi_gate_bonus", 0.25)
|
||||||
reasons.append("QPI problem-definition gate matched")
|
reasons.append("QPI problem-definition gate matched")
|
||||||
|
if model_id == "qpi":
|
||||||
|
has_positive_signal = bool(
|
||||||
|
trigger_hits
|
||||||
|
or input_hits
|
||||||
|
or complexity_hits
|
||||||
|
or qpi_gate_hits
|
||||||
|
or has_analysis_override(user_input, global_rules)
|
||||||
|
or task_type in {"problem_definition", "question_analysis"}
|
||||||
|
or pipeline_position in {"pre_analysis", "problem_definition"}
|
||||||
|
)
|
||||||
|
if not has_positive_signal:
|
||||||
|
score -= weights.get("qpi_missing_positive_signal_penalty", 0.25)
|
||||||
|
penalties.append("QPI positive signal missing")
|
||||||
if model_id == "qpi" and "QPI 已判断" in user_input:
|
if model_id == "qpi" and "QPI 已判断" in user_input:
|
||||||
score -= weights.get("qpi_already_completed_penalty", 0.25)
|
score -= weights.get("qpi_already_completed_penalty", 0.25)
|
||||||
penalties.append("QPI already completed")
|
penalties.append("QPI already completed")
|
||||||
|
|
|
||||||
|
|
@ -69,6 +69,7 @@ EXPECTED_DOMINANT_SCARCITY_VALUES = {"data", "path_resource", "consensus_order",
|
||||||
EXPECTED_MAX_DEPTH_VALUES = {"application", "domain", "process", "purpose", "core_mechanism", "human_capability", "philosophical_bedrock", "no_call", "not_applicable"}
|
EXPECTED_MAX_DEPTH_VALUES = {"application", "domain", "process", "purpose", "core_mechanism", "human_capability", "philosophical_bedrock", "no_call", "not_applicable"}
|
||||||
EVALUATION_MODE_VALUES = {"manual", "keyword", "structured", "semantic"}
|
EVALUATION_MODE_VALUES = {"manual", "keyword", "structured", "semantic"}
|
||||||
STATUS_VALUES = {"draft", "review", "reviewed", "callable", "stable", "archived", "deprecated", "draft_pre_contract"}
|
STATUS_VALUES = {"draft", "review", "reviewed", "callable", "stable", "archived", "deprecated", "draft_pre_contract"}
|
||||||
|
QPI_COMPLEXITY_PATTERN_VALUES = {"not_mixed", "intra_frame_mixed", "inter_viewpoint_divergence"}
|
||||||
|
|
||||||
MODEL_SPECIFIC_STRUCTURED_OUTPUT_FIELDS = {
|
MODEL_SPECIFIC_STRUCTURED_OUTPUT_FIELDS = {
|
||||||
"qpi": [
|
"qpi": [
|
||||||
|
|
@ -242,6 +243,39 @@ def validate_structured_output_contract(model, label):
|
||||||
return errors
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def validate_qpi_case_digests(root):
|
||||||
|
path = root / "selector" / "qpi_case_digests.json"
|
||||||
|
if not path.exists():
|
||||||
|
return []
|
||||||
|
|
||||||
|
data, errors = read_json(path)
|
||||||
|
if errors:
|
||||||
|
return errors
|
||||||
|
if not isinstance(data, dict) or not isinstance(data.get("cases"), list):
|
||||||
|
return ["selector/qpi_case_digests.json must contain list field cases"]
|
||||||
|
|
||||||
|
errors = []
|
||||||
|
for case in data["cases"]:
|
||||||
|
case_id = case.get("case_id", "<missing>")
|
||||||
|
label = f"qpi case digest {case_id}"
|
||||||
|
if "misframing_risks" in case:
|
||||||
|
errors.append(f"{label} uses deprecated field misframing_risks; use misclassification_risk")
|
||||||
|
if "mixed_or_multi_perspective" in case:
|
||||||
|
errors.append(f"{label} uses deprecated field mixed_or_multi_perspective; use qpi_complexity_pattern")
|
||||||
|
if "misclassification_risk" not in case:
|
||||||
|
errors.append(f"{label} missing required field misclassification_risk")
|
||||||
|
pattern = case.get("qpi_complexity_pattern")
|
||||||
|
if pattern is None:
|
||||||
|
errors.append(f"{label} missing required field qpi_complexity_pattern")
|
||||||
|
elif pattern not in QPI_COMPLEXITY_PATTERN_VALUES:
|
||||||
|
errors.append(f"{label} field qpi_complexity_pattern has invalid value {pattern}")
|
||||||
|
if case.get("classification_scope") == "multi_perspective" or pattern == "inter_viewpoint_divergence":
|
||||||
|
has_viewpoint_detail = bool(case.get("classification_by_viewpoint")) or bool(case.get("viewpoint_summary"))
|
||||||
|
if not has_viewpoint_detail:
|
||||||
|
errors.append(f"{label} multi_perspective case requires classification_by_viewpoint or viewpoint_summary")
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
def load_model_index(root):
|
def load_model_index(root):
|
||||||
path = root / "models" / "model_index.json"
|
path = root / "models" / "model_index.json"
|
||||||
data, errors = read_json(path)
|
data, errors = read_json(path)
|
||||||
|
|
@ -281,6 +315,7 @@ def validate_library(root):
|
||||||
errors.extend(article_errors)
|
errors.extend(article_errors)
|
||||||
errors.extend(excerpt_errors)
|
errors.extend(excerpt_errors)
|
||||||
errors.extend(case_errors)
|
errors.extend(case_errors)
|
||||||
|
errors.extend(validate_qpi_case_digests(root))
|
||||||
|
|
||||||
for article in source_articles:
|
for article in source_articles:
|
||||||
errors.extend(require_fields(article, SOURCE_ARTICLE_REQUIRED_FIELDS, f"source article {article.get('source_id', '<missing>')}"))
|
errors.extend(require_fields(article, SOURCE_ARTICLE_REQUIRED_FIELDS, f"source article {article.get('source_id', '<missing>')}"))
|
||||||
|
|
|
||||||
|
|
@ -14,6 +14,9 @@ It should use simple matching rules:
|
||||||
- Negative trigger first
|
- Negative trigger first
|
||||||
- No-call threshold
|
- No-call threshold
|
||||||
- QPI-before-IA gate
|
- QPI-before-IA gate
|
||||||
|
- QPI positive-signal gate
|
||||||
|
- Direct-execution no-call signals
|
||||||
|
- Calibration smoke checks
|
||||||
|
|
||||||
Current files:
|
Current files:
|
||||||
|
|
||||||
|
|
@ -28,4 +31,13 @@ The executable demo is `scripts/run_selector_demo.py`.
|
||||||
|
|
||||||
Selector regression is `scripts/run_selector_regression.py`.
|
Selector regression is `scripts/run_selector_regression.py`.
|
||||||
|
|
||||||
|
Selector calibration smoke check is `scripts/run_selector_calibration_smoke.py`.
|
||||||
|
|
||||||
`qpi_case_digests.json` stores owner-reviewed QPI case digests that can feed selector calibration or later regression selection. It is not a model status upgrade and should not include unreviewed draft cases.
|
`qpi_case_digests.json` stores owner-reviewed QPI case digests that can feed selector calibration or later regression selection. It is not a model status upgrade and should not include unreviewed draft cases.
|
||||||
|
|
||||||
|
Digest field notes:
|
||||||
|
|
||||||
|
- `misclassification_risk` is the canonical digest field matching QPI's structured output contract.
|
||||||
|
- `qpi_complexity_pattern` records judgment complexity: `not_mixed`, `intra_frame_mixed`, or `inter_viewpoint_divergence`.
|
||||||
|
- `qpi_complexity_pattern=intra_frame_mixed` does not imply `classification=mixed`; `classification` is the final routing class, while `qpi_complexity_pattern` records the structure of the judgment.
|
||||||
|
- Multi-perspective / inter-viewpoint cases require `classification_by_viewpoint` or `viewpoint_summary`.
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -583,6 +583,20 @@
|
||||||
"input": "组织认为只要没有罚款、没有被约谈、没有报销损失，就说明合规风险不大。",
|
"input": "组织认为只要没有罚款、没有被约谈、没有报销损失，就说明合规风险不大。",
|
||||||
"expected_selector_behavior": "select_qpi",
|
"expected_selector_behavior": "select_qpi",
|
||||||
"expected_notes": "Owner-reviewed research case 008: punishment state versus integrity state."
|
"expected_notes": "Owner-reviewed research case 008: punishment state versus integrity state."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"case_id": "selector_calibration_direct_summary_no_call_001",
|
||||||
|
"category": "round_03_1_direct_execution_no_call",
|
||||||
|
"input": "请把下面内容总结成三点。",
|
||||||
|
"expected_selector_behavior": "no_call",
|
||||||
|
"expected_notes": "Round 03.1 direct summary request must not select QPI by default score."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"case_id": "selector_calibration_analysis_override_should_call_001",
|
||||||
|
"category": "round_03_1_analysis_override",
|
||||||
|
"input": "请把下面内容总结成三点，但不要只执行，请分析背后的问题定义和组织结构冲突。",
|
||||||
|
"expected_selector_behavior": "select_qpi",
|
||||||
|
"expected_notes": "Round 03.1 explicit analysis override should preserve QPI selection despite direct-execution phrasing."
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -25,13 +25,27 @@
|
||||||
"直接改",
|
"直接改",
|
||||||
"只改错别字",
|
"只改错别字",
|
||||||
"润色",
|
"润色",
|
||||||
"翻译",
|
"翻译这句话",
|
||||||
|
"怎么翻译",
|
||||||
|
"翻译：",
|
||||||
|
"只要一个英文",
|
||||||
"不要展开",
|
"不要展开",
|
||||||
"不要深入分析",
|
"不要深入分析",
|
||||||
"马上执行",
|
"马上执行",
|
||||||
"不用解释",
|
"不用解释",
|
||||||
"生成图片"
|
"生成图片"
|
||||||
],
|
],
|
||||||
|
"direct_execution_no_call_signals": [
|
||||||
|
"排一下",
|
||||||
|
"排床位",
|
||||||
|
"床位表",
|
||||||
|
"整理成表格",
|
||||||
|
"总结成三点",
|
||||||
|
"列出",
|
||||||
|
"格式化",
|
||||||
|
"改成",
|
||||||
|
"只要清单"
|
||||||
|
],
|
||||||
"complexity_signals": [
|
"complexity_signals": [
|
||||||
"反复",
|
"反复",
|
||||||
"冲突",
|
"冲突",
|
||||||
|
|
@ -44,7 +58,17 @@
|
||||||
"模型",
|
"模型",
|
||||||
"结构",
|
"结构",
|
||||||
"机制",
|
"机制",
|
||||||
"抽取"
|
"抽取",
|
||||||
|
"治理",
|
||||||
|
"边界",
|
||||||
|
"承诺",
|
||||||
|
"风险",
|
||||||
|
"证据",
|
||||||
|
"责任",
|
||||||
|
"信任",
|
||||||
|
"容量",
|
||||||
|
"合规",
|
||||||
|
"指标"
|
||||||
],
|
],
|
||||||
"qpi_gate_signals": [
|
"qpi_gate_signals": [
|
||||||
"缺数据",
|
"缺数据",
|
||||||
|
|
@ -54,7 +78,40 @@
|
||||||
"怎么定义",
|
"怎么定义",
|
||||||
"这是问题",
|
"这是问题",
|
||||||
"执行路径问题",
|
"执行路径问题",
|
||||||
"组织共识问题"
|
"组织共识问题",
|
||||||
|
"如何",
|
||||||
|
"判断",
|
||||||
|
"真实调用",
|
||||||
|
"上下文",
|
||||||
|
"审计边界",
|
||||||
|
"后续复用",
|
||||||
|
"维护责任",
|
||||||
|
"身份",
|
||||||
|
"触发",
|
||||||
|
"维持条件",
|
||||||
|
"行动杠杆",
|
||||||
|
"依赖风险",
|
||||||
|
"误路由",
|
||||||
|
"过度升维",
|
||||||
|
"负激励",
|
||||||
|
"成本转嫁",
|
||||||
|
"授权",
|
||||||
|
"可执行空间",
|
||||||
|
"现实冲突",
|
||||||
|
"数据边界",
|
||||||
|
"安全",
|
||||||
|
"底线控制",
|
||||||
|
"核心承诺",
|
||||||
|
"交付闭环",
|
||||||
|
"正式能力缺口",
|
||||||
|
"转移原因",
|
||||||
|
"法律审查路径",
|
||||||
|
"证书信任",
|
||||||
|
"完整链条",
|
||||||
|
"教学能力",
|
||||||
|
"科研指标",
|
||||||
|
"物理容量",
|
||||||
|
"没人牵头"
|
||||||
],
|
],
|
||||||
"ia_heavy_signals": [
|
"ia_heavy_signals": [
|
||||||
"底层假设",
|
"底层假设",
|
||||||
|
|
@ -73,6 +130,7 @@
|
||||||
"complexity_signal": 0.15,
|
"complexity_signal": 0.15,
|
||||||
"selection_priority_factor": 0.01,
|
"selection_priority_factor": 0.01,
|
||||||
"qpi_gate_bonus": 0.25,
|
"qpi_gate_bonus": 0.25,
|
||||||
|
"qpi_missing_positive_signal_penalty": 0.25,
|
||||||
"qpi_already_completed_penalty": 0.25,
|
"qpi_already_completed_penalty": 0.25,
|
||||||
"ia_heavy_bonus": 0.2,
|
"ia_heavy_bonus": 0.2,
|
||||||
"ia_qpi_completed_bonus": 0.25,
|
"ia_qpi_completed_bonus": 0.25,
|
||||||
|
|
@ -85,7 +143,7 @@
|
||||||
"models": [
|
"models": [
|
||||||
{
|
{
|
||||||
"model_id": "qpi",
|
"model_id": "qpi",
|
||||||
"base_score": 50,
|
"base_score": 25,
|
||||||
"trigger_keywords": [
|
"trigger_keywords": [
|
||||||
"问题",
|
"问题",
|
||||||
"难题",
|
"难题",
|
||||||
|
|
@ -107,13 +165,12 @@
|
||||||
],
|
],
|
||||||
"negative_triggers": [
|
"negative_triggers": [
|
||||||
"只改错别字",
|
"只改错别字",
|
||||||
"翻译",
|
|
||||||
"润色",
|
"润色",
|
||||||
"生成图片"
|
"生成图片"
|
||||||
],
|
],
|
||||||
"pipeline_position": "problem_definition",
|
"pipeline_position": "problem_definition",
|
||||||
"selection_priority": 95,
|
"selection_priority": 9,
|
||||||
"routing_notes": "作为默认前置路由模型；如果无法判断问题类型，优先调用 QPI。"
|
"routing_notes": "作为前置路由模型；只有命中问题定义、复杂性、QPI gate、任务类型或显式分析 override 时才调用 QPI，普通执行任务不得仅凭默认分调用。"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"model_id": "intellectual_archaeology",
|
"model_id": "intellectual_archaeology",
|
||||||
|
|
@ -141,10 +198,12 @@
|
||||||
"只要事实",
|
"只要事实",
|
||||||
"低风险",
|
"低风险",
|
||||||
"马上执行",
|
"马上执行",
|
||||||
"不用解释"
|
"不用解释",
|
||||||
|
"先不要思想考古",
|
||||||
|
"不要思想考古"
|
||||||
],
|
],
|
||||||
"pipeline_position": "modeling_depth_analysis",
|
"pipeline_position": "modeling_depth_analysis",
|
||||||
"selection_priority": 80,
|
"selection_priority": 7,
|
||||||
"routing_notes": "通常在 QPI 判断为中重型难题或课题后调用；遇到轻量事实检索应避免调用。"
|
"routing_notes": "通常在 QPI 判断为中重型难题或课题后调用；遇到轻量事实检索应避免调用。"
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
|
|
@ -453,6 +453,52 @@ class ValidateModelLibraryTests(unittest.TestCase):
|
||||||
errors
|
errors
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_qpi_digest_deprecated_fields_are_reported(self):
|
||||||
|
validator = load_validator()
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
self.write_minimal_library(root)
|
||||||
|
self.write_json(root, "selector/qpi_case_digests.json", {
|
||||||
|
"cases": [{
|
||||||
|
"case_id": "qpi-test-001",
|
||||||
|
"classification_scope": "subject_contextual",
|
||||||
|
"mixed_or_multi_perspective": "not_mixed",
|
||||||
|
"misframing_risks": ["violent_reduction"]
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
|
||||||
|
errors = validator.validate_library(root)
|
||||||
|
|
||||||
|
self.assertIn(
|
||||||
|
"qpi case digest qpi-test-001 uses deprecated field misframing_risks; use misclassification_risk",
|
||||||
|
errors
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"qpi case digest qpi-test-001 uses deprecated field mixed_or_multi_perspective; use qpi_complexity_pattern",
|
||||||
|
errors
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_qpi_digest_multi_perspective_requires_viewpoint_detail(self):
|
||||||
|
validator = load_validator()
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
root = Path(tmp)
|
||||||
|
self.write_minimal_library(root)
|
||||||
|
self.write_json(root, "selector/qpi_case_digests.json", {
|
||||||
|
"cases": [{
|
||||||
|
"case_id": "qpi-test-002",
|
||||||
|
"classification_scope": "multi_perspective",
|
||||||
|
"qpi_complexity_pattern": "inter_viewpoint_divergence",
|
||||||
|
"misclassification_risk": ["single_viewpoint_only"]
|
||||||
|
}]
|
||||||
|
})
|
||||||
|
|
||||||
|
errors = validator.validate_library(root)
|
||||||
|
|
||||||
|
self.assertIn(
|
||||||
|
"qpi case digest qpi-test-002 multi_perspective case requires classification_by_viewpoint or viewpoint_summary",
|
||||||
|
errors
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue