114 lines
2.6 KiB
Markdown
114 lines
2.6 KiB
Markdown
# Review Agent Regression Rubric
|
|
|
|
## Purpose
|
|
|
|
Use this rubric to compare review outputs across prompt variants and model environments.
|
|
|
|
Do not score only surface fluency. The goal is to detect model-kernel preservation or degradation.
|
|
|
|
## Scoring
|
|
|
|
Use 1-5:
|
|
|
|
```text
|
|
1 = failed / absent
|
|
2 = weak
|
|
3 = acceptable
|
|
4 = strong
|
|
5 = excellent
|
|
```
|
|
|
|
## Criteria
|
|
|
|
### 1. Model Fidelity
|
|
|
|
Does the output preserve the agent's core model rather than behaving like a generic reviewer?
|
|
|
|
### 2. Method Fidelity
|
|
|
|
Does it preserve the original method kernel?
|
|
|
|
For 巨人认知, check whether 思想考古 is present:
|
|
|
|
```text
|
|
surface phenomenon -> tool/model layer -> hidden assumption -> philosophical bedrock -> value premise
|
|
```
|
|
|
|
For 认知显影, check whether the original显影-style review kernel is present rather than generic objection.
|
|
|
|
### 3. Deep-Structure Performance
|
|
|
|
Does the output identify deep structural problems instead of only local edits?
|
|
|
|
### 4. Hidden Assumption Detection
|
|
|
|
Does it identify assumptions that the source text relies on but does not state?
|
|
|
|
### 5. Philosophical Bedrock Excavation
|
|
|
|
Does it reach value premises, worldview, category framing, or governing metaphors when relevant?
|
|
|
|
### 6. Context Fit
|
|
|
|
Does it stay close to the article's actual material and intent?
|
|
|
|
### 7. Concept Overfitting Risk
|
|
|
|
Does it force favorite concepts, metaphors, or labels where they are not needed?
|
|
|
|
Reverse scoring note:
|
|
|
|
```text
|
|
5 = low overfitting risk
|
|
1 = severe overfitting
|
|
```
|
|
|
|
### 8. Output Actionability
|
|
|
|
Are the suggestions concrete enough to revise the article?
|
|
|
|
### 9. Naming / Protocol Discipline
|
|
|
|
Does the output preserve required names, layers, and output protocol?
|
|
|
|
Examples:
|
|
|
|
```text
|
|
巨人认知 must use GL0-GL4, not L0-L4.
|
|
Reconstructed labels must be marked as reconstructed.
|
|
```
|
|
|
|
### 10. Platform Stability
|
|
|
|
Does the prompt produce stable behavior in this model environment?
|
|
|
|
Signs of instability:
|
|
|
|
```text
|
|
format collapse
|
|
generic reviewer drift
|
|
hallucinated source claims
|
|
loss of original tone
|
|
overlong boilerplate
|
|
```
|
|
|
|
## Comparison Table Template
|
|
|
|
```md
|
|
| Agent | Prompt Variant | Model Env | Article | Model Fidelity | Method Fidelity | Deep Structure | Hidden Assumptions | Bedrock | Context Fit | Low Overfit | Actionability | Naming Discipline | Stability | Notes |
|
|
| ----- | -------------- | --------- | ------- | -------------- | --------------- | -------------- | ------------------ | ------- | ----------- | ----------- | ------------- | ----------------- | --------- | ----- |
|
|
```
|
|
|
|
## Final Judgment Labels
|
|
|
|
Use one:
|
|
|
|
```text
|
|
clear winner
|
|
conditional winner
|
|
environment-specific winner
|
|
inconclusive
|
|
regression detected
|
|
```
|
|
|