482 lines
18 KiB
Markdown
482 lines
18 KiB
Markdown
# Owner
|
||
这部分，你能否每条提供1、2条样例，除了参考内容，我主要是借鉴格式。
|
||
|
||
```text
|
||
3 条你真实工作中遇到的混合问题
|
||
2 条你觉得 QPI 可能误判的问题
|
||
2 条你觉得思想考古容易过度下潜的问题
|
||
2 条“看起来很深但其实不该调用”的问题
|
||
1 条模型抽取任务
|
||
```
|
||
|
||
---
|
||
|
||
# GPT
|
||
|
||
可以。下面这组我按 **Codex 可直接转成 `selector_calibration_inputs.json` 或人工审查表** 的格式写。重点不是内容本身，而是让 Codex 知道每条样本应包含：输入、预期路由、预期 QPI 判断、是否调用思想考古、失败信号、禁止输出。
|
||
|
||
依据当前规则，QPI 要能处理 `Question / Problem / Issue / mixed / no-call`，并输出 `scarcity_profile`、`dominant_scarcity`、`classification`、`evidence_gap` 等字段；思想考古不是默认流程，必须有 `should_call`、`recommended_max_depth`、`stop_reason`、`no_deeper_reason`，且不得默认七层全量展开。 当前 regression 也应覆盖 positive、boundary、misuse、no_call、selector_gate、pipeline，并检查误召回、混合输入证据缺口、过度下潜或过度升维。
|
||
|
||
---
|
||
|
||
## 0. 推荐给 Codex 的样本格式
|
||
|
||
先统一格式，后面每条都照这个来：
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_qpi_mixed_001",
|
||
"group": "qpi_mixed_real_work",
|
||
"input": "用户实际输入或拟真输入",
|
||
"expected_primary_model": "qpi",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "mixed",
|
||
"dominant_scarcity": "mixed",
|
||
"classification_confidence": "medium",
|
||
"scarcity_profile": {
|
||
"data_scarcity": "high",
|
||
"path_or_resource_scarcity": "medium",
|
||
"consensus_or_order_scarcity": "high"
|
||
},
|
||
"minimum_required_elements": [
|
||
"problem_owner",
|
||
"problem_source",
|
||
"time_scale",
|
||
"evidence_gap",
|
||
"misclassification_risk",
|
||
"recommended_next_step"
|
||
]
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "系统应该如何判断、路由、停止或追问。",
|
||
"failure_signal": "什么输出算失败。",
|
||
"forbidden_elements": [
|
||
"不应出现的判断、模型、层级或行动建议"
|
||
],
|
||
"notes": "这条样本主要测试什么。"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 1. 真实工作中遇到的混合问题：3 条
|
||
|
||
### 1.1 混合问题：产品模型库扩展卡住
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_qpi_mixed_real_001",
|
||
"group": "qpi_mixed_real_work",
|
||
"input": "我现在想扩展第三个核心模型，但 QPI 和思想考古还没完全稳定；一方面我缺真实样本，另一方面又担心继续停在校验会拖慢产品推进。这到底是什么问题？",
|
||
"expected_primary_model": "qpi",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "mixed",
|
||
"dominant_scarcity": "mixed",
|
||
"classification_confidence": "medium",
|
||
"scarcity_profile": {
|
||
"data_scarcity": "high",
|
||
"path_or_resource_scarcity": "medium",
|
||
"consensus_or_order_scarcity": "high"
|
||
},
|
||
"minimum_required_elements": [
|
||
"problem_owner",
|
||
"time_scale",
|
||
"scarcity_profile",
|
||
"dominant_scarcity",
|
||
"evidence_gap",
|
||
"recommended_next_step"
|
||
]
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "应识别为 mixed：缺真实 regression 样本是 data_scarcity；是否扩第三模型是路径与阶段决策；是否继续硬化还是推进，是阶段秩序与优先级问题。应建议先补 10-20 条校准样本，而不是直接扩第三模型。",
|
||
"failure_signal": "直接回答“可以扩第三模型”，或把它抽象成长期战略课题而不给 evidence_gap。",
|
||
"forbidden_elements": [
|
||
"stable",
|
||
"直接扩第三模型",
|
||
"哲学基岩"
|
||
],
|
||
"notes": "测试 QPI 是否能处理 OPC 产品推进中的混合问题。"
|
||
}
|
||
```
|
||
|
||
### 1.2 混合问题：内容资产、系统工程、个人精力同时卡住
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_qpi_mixed_real_002",
|
||
"group": "qpi_mixed_real_work",
|
||
"input": "我有很多文章和模型，但现在不知道先整理来源证据、先做模型卡，还是先让 Codex 做 selector；我一个人时间有限，又怕整理太慢导致产品迟迟不能跑起来。",
|
||
"expected_primary_model": "qpi",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "mixed",
|
||
"dominant_scarcity": "path_resource",
|
||
"classification_confidence": "medium",
|
||
"scarcity_profile": {
|
||
"data_scarcity": "medium",
|
||
"path_or_resource_scarcity": "high",
|
||
"consensus_or_order_scarcity": "medium"
|
||
},
|
||
"minimum_required_elements": [
|
||
"problem_owner",
|
||
"time_scale",
|
||
"dominant_scarcity",
|
||
"evidence_gap",
|
||
"recommended_next_step"
|
||
]
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "应识别为 mixed 偏 problem。核心不是缺理论，而是 OPC 资源约束下的路径排序。推荐下一步应是任务切片：先固定 2 个模型的 source evidence、model JSON、regression、selector，再考虑扩展。",
|
||
"failure_signal": "把它解释成创作者身份危机或知识文明问题，进入过度思想考古。",
|
||
"forbidden_elements": [
|
||
"philosophical_bedrock",
|
||
"完整知识图谱",
|
||
"一次性整理全部模型"
|
||
],
|
||
"notes": "测试系统是否能把复杂叙述收敛为可执行路径。"
|
||
}
|
||
```
|
||
|
||
### 1.3 混合问题：用户需求、模型能力、产品边界不一致
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_qpi_mixed_real_003",
|
||
"group": "qpi_mixed_real_work",
|
||
"input": "这个系统未来既要像我一样加工问题，又要管理模型卡，还可能服务外部用户。我现在不知道 MVP 到底该验证内部自用，还是直接做成面向用户的认知分析平台。",
|
||
"expected_primary_model": "qpi",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "mixed",
|
||
"dominant_scarcity": "consensus_order",
|
||
"classification_confidence": "medium",
|
||
"scarcity_profile": {
|
||
"data_scarcity": "medium",
|
||
"path_or_resource_scarcity": "medium",
|
||
"consensus_or_order_scarcity": "high"
|
||
},
|
||
"minimum_required_elements": [
|
||
"problem_owner",
|
||
"problem_source",
|
||
"time_scale",
|
||
"misclassification_risk",
|
||
"recommended_next_step"
|
||
]
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "应识别为 mixed 偏 issue：冲突来自第一使用者、产品边界和阶段目标不一致。推荐先以 Owner 自用验证模型加工链路，不进入外部平台化。",
|
||
"failure_signal": "直接给 SaaS 功能列表，或直接设计 UI。",
|
||
"forbidden_elements": [
|
||
"公共平台",
|
||
"计费系统",
|
||
"用户权限系统"
|
||
],
|
||
"notes": "测试 QPI 是否能识别产品形态冲突，而不是过早功能设计。"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 2. QPI 可能误判的问题：2 条
|
||
|
||
这类样本专门用来测 **暴力降维、恶意升维、手段错配**。QPI 必须能识别这些风险。
|
||
|
||
### 2.1 可能被误判：把组织课题暴力降维成个人执行问题
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_qpi_misjudge_001",
|
||
"group": "qpi_potential_misclassification",
|
||
"input": "团队交付总是延期，老板说就是大家执行力不够，让我出一套个人效率改进方案。",
|
||
"expected_primary_model": "qpi",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "mixed",
|
||
"dominant_scarcity": "consensus_order",
|
||
"classification_confidence": "medium",
|
||
"scarcity_profile": {
|
||
"data_scarcity": "medium",
|
||
"path_or_resource_scarcity": "medium",
|
||
"consensus_or_order_scarcity": "high"
|
||
},
|
||
"minimum_required_elements": [
|
||
"problem_source",
|
||
"misclassification_risk",
|
||
"evidence_gap",
|
||
"recommended_next_step"
|
||
]
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "应提示存在暴力降维风险：延期可能不是个人效率问题，而可能涉及目标频繁变化、资源不足、依赖不清、决策拥堵。应先补 evidence_gap。",
|
||
"failure_signal": "直接输出个人效率提升清单。",
|
||
"forbidden_elements": [
|
||
"仅归因于个人不努力",
|
||
"只给时间管理方法"
|
||
],
|
||
"notes": "测试 QPI 是否能拒绝接受甩锅式问题框架。"
|
||
}
|
||
```
|
||
|
||
### 2.2 可能被误判：把具体工程问题恶意升维成宏大课题
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_qpi_misjudge_002",
|
||
"group": "qpi_potential_misclassification",
|
||
"input": "我现在只是要决定模型库用 JSON 文件还是 SQLite，但我担心这背后是不是人类知识管理范式的根本问题。",
|
||
"expected_primary_model": "qpi",
|
||
"negative_expected_models": [
|
||
"intellectual_archaeology"
|
||
],
|
||
"expected_qpi": {
|
||
"classification": "problem",
|
||
"dominant_scarcity": "path_resource",
|
||
"classification_confidence": "high",
|
||
"scarcity_profile": {
|
||
"data_scarcity": "low",
|
||
"path_or_resource_scarcity": "high",
|
||
"consensus_or_order_scarcity": "low"
|
||
},
|
||
"minimum_required_elements": [
|
||
"misclassification_risk",
|
||
"recommended_next_step"
|
||
]
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "应识别为 problem，主要缺路径取舍。应提示存在恶意升维风险：当前阶段只需按 file-first MVP 的可维护性、校验便利性、扩展成本做技术选择。",
|
||
"failure_signal": "进入知识论、文明史或哲学基岩分析。",
|
||
"forbidden_elements": [
|
||
"philosophical_bedrock",
|
||
"人类知识范式",
|
||
"七层下钻"
|
||
],
|
||
"notes": "测试 QPI 是否能防止把工程取舍升维成无法行动的课题。"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 3. 思想考古容易过度下潜的问题：2 条
|
||
|
||
这类样本要测思想考古的 **停止门**。思想考古可以调用，但必须停在合适层级；如果继续下潜不改变判断、路径、验证方式或行动边界，就要停止。
|
||
|
||
### 3.1 应调用思想考古，但只到 purpose 层
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_ia_overdepth_001",
|
||
"group": "ia_over_depth_risk",
|
||
"input": "我想把 QPI 写成一张更稳定的模型卡。请帮我检查它到底是问题分类模型、路由模型，还是完整分析模型。",
|
||
"expected_primary_model": "intellectual_archaeology",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "problem",
|
||
"dominant_scarcity": "path_resource",
|
||
"classification_confidence": "medium"
|
||
},
|
||
"expected_ia": {
|
||
"should_call": true,
|
||
"recommended_max_depth": "purpose",
|
||
"minimum_required_elements": [
|
||
"entry_reason",
|
||
"layers_to_analyze",
|
||
"stop_reason",
|
||
"no_deeper_reason",
|
||
"action_implication"
|
||
]
|
||
},
|
||
"expected_behavior": "可以进入思想考古，但只需要下潜到 purpose 层：明确 QPI 的产品目的，是前置路由而不是完整解释系统。继续下潜到 human_capability 或 philosophical_bedrock 不会改变模型卡结构。",
|
||
"failure_signal": "默认展开七层，讨论认识论基岩。",
|
||
"forbidden_elements": [
|
||
"philosophical_bedrock",
|
||
"完整七层展开"
|
||
],
|
||
"notes": "测试思想考古的 minimum-depth，不是测试深度表达。"
|
||
}
|
||
```
|
||
|
||
### 3.2 应调用思想考古，但只到 core_mechanism 层
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_ia_overdepth_002",
|
||
"group": "ia_over_depth_risk",
|
||
"input": "思想考古这个模型容易写得很玄。我想把它改成系统可调用模型，请帮我找出它真正可复用的核心机制。",
|
||
"expected_primary_model": "intellectual_archaeology",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "problem",
|
||
"dominant_scarcity": "path_resource",
|
||
"classification_confidence": "medium"
|
||
},
|
||
"expected_ia": {
|
||
"should_call": true,
|
||
"recommended_max_depth": "core_mechanism",
|
||
"minimum_required_elements": [
|
||
"recommended_max_depth",
|
||
"stop_reason",
|
||
"no_deeper_reason",
|
||
"validation_needed"
|
||
]
|
||
},
|
||
"expected_behavior": "应下潜到 core_mechanism：思想考古的可调用机制是层级化追问、最小充分下潜、假设显性化、停止门。到这里已足以改模型卡；不需要讨论哲学基岩。",
|
||
"failure_signal": "把重点转成宏观哲学解释，而没有产出可调用机制。",
|
||
"forbidden_elements": [
|
||
"为了深刻而深刻",
|
||
"哲学基岩默认调用"
|
||
],
|
||
"notes": "测试 IA 是否能服务模型产品化，而不是自动哲学化。"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 4. “看起来很深但其实不该调用”的问题：2 条
|
||
|
||
这类样本是 selector 防误召回。当前规则明确：思想考古不应仅因为出现“底层”“模型”“哲学”等词就被召回；问题定义未完成时，应先通过 QPI。
|
||
|
||
### 4.1 出现“底层逻辑”，但其实是轻量改写
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_no_call_deep_word_001",
|
||
"group": "deep_but_no_call",
|
||
"input": "帮我把这句话改得更有底层逻辑一点：我们要先做好模型库，再扩展更多模型。不要展开分析，只改写。",
|
||
"expected_primary_model": "none",
|
||
"negative_expected_models": [
|
||
"qpi",
|
||
"intellectual_archaeology"
|
||
],
|
||
"expected_qpi": {
|
||
"classification": "no_call",
|
||
"dominant_scarcity": "unknown",
|
||
"classification_confidence": "high"
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "应识别为纯改写任务，negative trigger 优先。不得因为出现“底层逻辑”“模型库”而召回思想考古。",
|
||
"failure_signal": "开始解释模型库的底层逻辑，或调用思想考古。",
|
||
"forbidden_elements": [
|
||
"七层下钻",
|
||
"QPI 分类分析",
|
||
"思想考古"
|
||
],
|
||
"notes": "测试 negative trigger first。"
|
||
}
|
||
```
|
||
|
||
### 4.2 出现“哲学”，但其实是事实查询
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_no_call_deep_word_002",
|
||
"group": "deep_but_no_call",
|
||
"input": "帮我查一下‘哲学基岩’这个词在我上传的思想考古模型卡里有没有出现，只要告诉我有还是没有。",
|
||
"expected_primary_model": "none",
|
||
"negative_expected_models": [
|
||
"qpi",
|
||
"intellectual_archaeology"
|
||
],
|
||
"expected_qpi": {
|
||
"classification": "no_call",
|
||
"dominant_scarcity": "data",
|
||
"classification_confidence": "high"
|
||
},
|
||
"expected_ia": {
|
||
"should_call": false,
|
||
"recommended_max_depth": "not_applicable"
|
||
},
|
||
"expected_behavior": "应识别为明确事实查询或文件检索任务，不进入模型分析。可以执行检索，但不调用思想考古。",
|
||
"failure_signal": "因为出现“哲学基岩”而展开思想考古。",
|
||
"forbidden_elements": [
|
||
"深层假设分析",
|
||
"philosophical_bedrock 分析",
|
||
"七层结构"
|
||
],
|
||
"notes": "测试关键词误召回。"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 5. 模型抽取任务：1 条
|
||
|
||
模型抽取任务是思想考古可以进入的正例，因为它要把文章或代表文本抽成可复用、可边界化、可测试、可被 selector 调用的模型资产。当前模型抽取链路是：原始文章 / 代表文本 → source article record → source evidence excerpts → Markdown card → JSON model card → regression cases → selector rules / examples → indexes → validation report → CCRA review。
|
||
|
||
### 5.1 从文章中抽取一个候选核心模型
|
||
|
||
```json
|
||
{
|
||
"case_id": "calibration_model_extraction_001",
|
||
"group": "model_extraction_task",
|
||
"input": "我给你一篇关于‘认知棱镜’的文章，请不要摘要文章，而是判断里面是否存在可产品化的核心模型，并按模型卡结构抽取：核心问题、核心机制、输入、输出、适用场景、不适用场景、调用关键词、误用风险、回归测试样例。",
|
||
"expected_primary_model": "intellectual_archaeology",
|
||
"negative_expected_models": [],
|
||
"expected_qpi": {
|
||
"classification": "problem",
|
||
"dominant_scarcity": "path_resource",
|
||
"classification_confidence": "medium"
|
||
},
|
||
"expected_ia": {
|
||
"should_call": true,
|
||
"recommended_max_depth": "core_mechanism",
|
||
"minimum_required_elements": [
|
||
"entry_reason",
|
||
"layers_to_analyze",
|
||
"assumptions_by_layer",
|
||
"validation_needed",
|
||
"action_implication",
|
||
"no_deeper_reason"
|
||
]
|
||
},
|
||
"expected_behavior": "应识别为模型抽取任务，不是文章摘要。可以调用思想考古下潜到 core_mechanism，找出可复用机制、边界、误用和测试方式。若原文证据不足，应标注 evidence_gap，不得编造来源。",
|
||
"failure_signal": "只做文章摘要，或直接生成漂亮但不可追溯的模型卡。",
|
||
"forbidden_elements": [
|
||
"无来源证据的核心机制断言",
|
||
"默认 stable",
|
||
"直接扩展进核心模型库"
|
||
],
|
||
"notes": "测试模型抽取链路和 evidence discipline。"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 6. 给 Codex 的简短说明可以这样写
|
||
|
||
```text
|
||
下面 10 条不是最终内容样本，而是 selector calibration / regression 人审样本格式参考。
|
||
|
||
请不要把这些样本直接视为模型稳定证据。
|
||
请将它们放入 selector_calibration_inputs.json 或等价文件中，用于测试：
|
||
1. QPI mixed case 是否输出 scarcity_profile、dominant_scarcity、evidence_gap；
|
||
2. QPI 是否识别暴力降维、恶意升维、手段错配；
|
||
3. 思想考古是否遵守 recommended_max_depth、stop_reason、no_deeper_reason；
|
||
4. selector 是否能在轻量改写、明确事实查询、直接执行任务中 no-call；
|
||
5. 模型抽取任务是否能进入思想考古，但不能变成文章摘要或无证据模型卡。
|
||
```
|
||
|
||
我建议你实际交给 Codex 时，先用这 10 条作为格式模板，然后把其中 3-5 条替换成你最近真实遇到的问题。真实样本比拟真样本更能暴露 selector 的误召回。
|