7.2 KiB
Review Agent Regression Test Handoff
1. Objective
Run a controlled 3 x 2 regression test for two review agents:
巨人认知 / Giant Cognition
认知显影 / Cognitive Imaging
The goal is to determine whether CCPE System Lite migration improves, preserves, or degrades the original CCPE 2.0 review behavior.
This work should be continued in a new Codex session to avoid contamination from the current debugging conversation.
2. Core Hypothesis
Current working hypothesis:
For mature review agents, full template-style Lite rewrites may lose hidden review kernels.
The safer migration route may be:
original CCPE 2.0 kernel
+ minimal Lite metadata / portability wrapper
+ explicit safety and reasoning-disclosure repairs
+ regression-tested output protocol
3. Why This Regression Test Exists
The 巨人认知 Lite migration exposed a possible root cause:
Original 巨人认知 is not just one model.
It contains at least:
1. 认知架构师 role
2. 进化型生物计算架构
3. 思想考古学家 method
4. GL0-GL4 output protocol
5. Gemini-shaped style / generation habits
6. Article-context-derived worldview
The current Lite version preserved the architecture but under-specified 思想考古学家. That may explain why GL3 sometimes behaves like metaphor optimization instead of deep assumption archaeology.
4. Test Matrix
For each agent family, test:
Prompt variants:
A. original-ccpe-2
B. ccpe-system-lite
C. original-kernel-minimal-lite
Model environments:
1. ChatGPT / Codex-side model
2. Gemini 3.1 Pro
This gives:
3 prompt variants x 2 model environments = 6 conditions per agent family
2 agent families x 6 conditions = 12 condition groups
Run each condition against the same article set.
5. Article Corpus
Place full article texts in:
workbench/analysis/review-agent-regression-2026-06-02/articles/
Recommended minimum:
article-01-strong-metaphor.md
article-02-business-analysis.md
article-03-logical-argument.md
article-04-value-philosophy.md
Use full正文 rather than outline when possible. This reduces dependence on hidden context packages and makes the test more portable.
6. Prompt Inventory
Prompt variants should live under:
workbench/analysis/review-agent-regression-2026-06-02/prompts/
Recommended layout:
prompts/
giant-cognition/
original-ccpe-2.prompt.md
ccpe-system-lite.prompt.md
original-kernel-minimal-lite.prompt.md
cognitive-imaging/
original-ccpe-2.prompt.md
ccpe-system-lite.prompt.md
original-kernel-minimal-lite.prompt.md
Current known sources:
Giant Cognition Lite:
agents/lite/giant-cognition.prompt.md
Cognitive Imaging original:
workbench/raw/认知显影者1.1.md
Cognitive Imaging Lite:
agents/lite/cognitive-imaging-practitioner.prompt.md
Known missing prompt artifacts:
Giant Cognition original CCPE 2.0 prompt
Giant Cognition original-kernel-minimal-lite prompt
Cognitive Imaging original-kernel-minimal-lite prompt
The new session should reconstruct or import these before running tests.
7. Original-Kernel Minimal Lite Rule
When creating original-kernel-minimal-lite.prompt.md, do not rewrite the mature prompt into a new template.
Preserve:
Original role language
Original conceptual metaphors
Original output protocol
Original model-specific layer names
Original critique style
Original method kernels
Only minimally repair:
Front matter
Target platform / usage scenario
Hidden chain-of-thought disclosure rules
Fact / retrieval boundaries
Layer naming discipline
Explicit output validation checklist
Version notes
For 巨人认知, explicitly preserve:
进化型生物计算架构
思想考古学家
GL0-GL4
意图 / 反思双循环
批判性且建设性的语气
For 认知显影, preserve its original review kernel before applying CCPE System wrapping.
8. Test Execution Protocol
Use fresh thread isolation.
For each result:
1. Start a fresh thread/subthread.
2. Load exactly one prompt variant.
3. Provide exactly one article.
4. Ask for the standard review output.
5. Save the raw output without editing.
6. Record model, prompt variant, article id, date, and operator.
Do not mix multiple prompt variants in one test thread.
Do not show the model other variants' outputs during generation.
Do not tune prompts mid-run. If a prompt is changed, increment the prompt version and restart the condition.
9. Result File Naming
Save results under:
results/chatgpt/
results/gemini/
Use this filename pattern:
{agent-family}__{prompt-variant}__{article-id}__{model-env}.result.md
Examples:
giant-cognition__original-ccpe-2__article-01-strong-metaphor__chatgpt.result.md
giant-cognition__ccpe-system-lite__article-01-strong-metaphor__gemini.result.md
cognitive-imaging__original-kernel-minimal-lite__article-03-logical-argument__chatgpt.result.md
10. Evaluation Rubric
Use:
rubrics/review-agent-regression-rubric.md
Do not evaluate only "which output feels better." Score the specific regression dimensions:
1. Model fidelity
2. Method fidelity
3. GL3 / deep-structure performance
4. Hidden assumption detection
5. Philosophical bedrock excavation
6. Context fit
7. Concept overfitting risk
8. Output actionability
9. Naming / protocol discipline
10. Platform stability
11. ChatGPT / Codex-Side Execution
The new Codex session may use thread tools if available.
Before using thread automation, search for the relevant thread tool:
create_thread
send_message_to_thread
read_thread
list_threads
If thread tools are unavailable, run tests manually in separate fresh Codex conversations and save outputs into results/chatgpt/.
12. Gemini-Side Execution
Gemini outputs will be produced manually by the user.
Save them into:
results/gemini/
Use the same file naming convention as ChatGPT/Codex-side results.
13. Final Report
After all results are collected, produce:
reports/review-agent-regression-report.md
The report should answer:
1. Which prompt variant performs best per agent family?
2. Which prompt variant transfers best to ChatGPT/Codex?
3. Which prompt variant performs best on Gemini?
4. Does original-kernel-minimal-lite outperform full Lite rewrite?
5. Is CCPE System migration preserving the old CCPE 2.0 review kernels?
6. Should mature review agents use separate production prompts per model environment?
7. What should be changed in CCPE Forge migration rules?
14. Stop Conditions
Pause and ask the user before:
- Promoting a prompt variant to active.
- Replacing any canonical Lite prompt.
- Updating Model Index status from candidate to active.
- Creating Agent Specs, Skills, or Runtimes from these prompts.
- Deleting or archiving old CCPE 2.0 prompts.
15. Recommended Next Step
In the new session:
1. Read this handoff.
2. Confirm article files exist.
3. Copy or reconstruct prompt variants into prompts/.
4. Create original-kernel-minimal-lite variants for both agent families.
5. Run ChatGPT/Codex-side tests in fresh threads.
6. Wait for Gemini-side outputs from the user.
7. Compare all outputs and write final report.