375 lines
5.7 KiB
Markdown
375 lines
5.7 KiB
Markdown
# Auditor Mode
|
|
|
|
## 1. Purpose
|
|
|
|
Auditor Mode is used to inspect, classify, and diagnose existing AI artifacts.
|
|
|
|
It does not rewrite the artifact by default.
|
|
|
|
Its main task is to answer:
|
|
|
|
```text
|
|
What is this artifact?
|
|
What does it contain?
|
|
What is strong?
|
|
What is broken?
|
|
What should be preserved?
|
|
What should be split, upgraded, simplified, or indexed?
|
|
```
|
|
|
|
## 2. When to Use Auditor Mode
|
|
|
|
Use Auditor Mode when the user asks to:
|
|
|
|
```text
|
|
Review an existing Agent
|
|
Check a prompt
|
|
Diagnose a Skill
|
|
Evaluate a Runtime
|
|
Analyze a committee workflow
|
|
Classify an artifact
|
|
Find problems in an AI assistant
|
|
Decide whether to refactor
|
|
Identify embedded models
|
|
Identify extractable Skills
|
|
```
|
|
|
|
## 3. Auditor Mode Workflow
|
|
|
|
Follow this sequence:
|
|
|
|
```text
|
|
1. Read artifact
|
|
2. Identify apparent purpose
|
|
3. Classify artifact
|
|
4. Detect embedded components
|
|
5. Assess operating mode
|
|
6. Assess depth vs automation
|
|
7. Evaluate against quality rubric
|
|
8. Identify risks
|
|
9. Identify extraction opportunities
|
|
10. Recommend target form
|
|
11. Propose files if refactored
|
|
12. Identify human decisions
|
|
```
|
|
|
|
## 4. Classification
|
|
|
|
Classify as one or more of:
|
|
|
|
```text
|
|
CCPE-Lite
|
|
CCPE-Agent
|
|
CCPE-Skill
|
|
CCPE-Runtime
|
|
Model Card
|
|
Model Index
|
|
Hybrid
|
|
```
|
|
|
|
For Hybrid artifacts, identify:
|
|
|
|
```text
|
|
Primary form
|
|
Secondary components
|
|
Embedded models
|
|
Extractable methods
|
|
Runtime node potential
|
|
Portable Lite need
|
|
```
|
|
|
|
## 5. Embedded Component Detection
|
|
|
|
Look for:
|
|
|
|
```text
|
|
Role / persona
|
|
Objective
|
|
Cognitive model
|
|
Theoretical appendix
|
|
Reusable method
|
|
Tool policy
|
|
Retrieval policy
|
|
Workflow
|
|
Report format
|
|
Evaluation rules
|
|
State rules
|
|
Runtime logic
|
|
Collaboration rules
|
|
```
|
|
|
|
## 6. Operating Mode Assessment
|
|
|
|
Determine:
|
|
|
|
```text
|
|
Expert Mode
|
|
Workshop Mode
|
|
Automation Mode
|
|
Hybrid Mode
|
|
```
|
|
|
|
Also determine Runtime orientation:
|
|
|
|
```text
|
|
None
|
|
Interactive Runtime
|
|
Automation Runtime
|
|
Hybrid Runtime
|
|
```
|
|
|
|
## 7. Depth vs Automation Assessment
|
|
|
|
Label the artifact:
|
|
|
|
```text
|
|
Depth-Oriented
|
|
Automation-Oriented
|
|
Hybrid
|
|
```
|
|
|
|
Depth-Oriented artifacts must preserve human judgment.
|
|
|
|
Automation-Oriented artifacts require authority, validation, and recovery.
|
|
|
|
Hybrid artifacts require clear separation between deep work and automated support.
|
|
|
|
## 8. Quality Rubric
|
|
|
|
Evaluate using the CCPE Quality Rubric.
|
|
|
|
Main criteria:
|
|
|
|
```text
|
|
Purpose Fit
|
|
Classification Accuracy
|
|
Structural Clarity
|
|
Boundary Precision
|
|
Capability Realism
|
|
Context Handling
|
|
Model Fidelity
|
|
Skill Reusability
|
|
Authority Clarity
|
|
Workflow Coherence
|
|
State Awareness
|
|
Output Usability
|
|
Evaluation Strength
|
|
Human-in-the-Loop Design
|
|
Runtime Safety
|
|
Portability
|
|
Maintainability
|
|
Intellectual Flavor Preservation
|
|
```
|
|
|
|
Use scores when useful:
|
|
|
|
```text
|
|
0 = Missing
|
|
1 = Weak
|
|
2 = Adequate
|
|
3 = Strong
|
|
4 = Excellent
|
|
```
|
|
|
|
Use severity labels:
|
|
|
|
```text
|
|
S = Structural blocker
|
|
A = Major issue
|
|
B = Moderate issue
|
|
C = Minor issue
|
|
```
|
|
|
|
## 9. Common Audit Findings
|
|
|
|
### 9.1 Over-Engineering
|
|
|
|
Signs:
|
|
|
|
```text
|
|
A simple expert prompt has Runtime complexity.
|
|
Too many layers obscure the task.
|
|
The artifact is hard to use in chat.
|
|
Complex authority rules exist where no tools are used.
|
|
```
|
|
|
|
Recommendation:
|
|
|
|
```text
|
|
Simplify to CCPE-Lite or Agent-Lite.
|
|
```
|
|
|
|
### 9.2 Under-Specification
|
|
|
|
Signs:
|
|
|
|
```text
|
|
A multi-agent workflow is written as one prompt.
|
|
Tools are mentioned without permissions.
|
|
No input/output contract.
|
|
No evaluation criteria.
|
|
No human decision gates.
|
|
```
|
|
|
|
Recommendation:
|
|
|
|
```text
|
|
Upgrade to Agent, Skill, or Runtime.
|
|
```
|
|
|
|
### 9.3 Model Coupling
|
|
|
|
Signs:
|
|
|
|
```text
|
|
A reusable cognitive model is trapped inside one Agent.
|
|
The same model is duplicated across prompts.
|
|
No Model Card exists.
|
|
No Model Index reference exists.
|
|
```
|
|
|
|
Recommendation:
|
|
|
|
```text
|
|
Extract Model Card.
|
|
Create or update Model Index entry.
|
|
Possibly create Skill.
|
|
```
|
|
|
|
### 9.4 Skill Burial
|
|
|
|
Signs:
|
|
|
|
```text
|
|
Stable procedure is buried in role instructions.
|
|
Multiple agents repeat the same method.
|
|
Tool use is described inconsistently.
|
|
```
|
|
|
|
Recommendation:
|
|
|
|
```text
|
|
Extract Skill.
|
|
Reference Skill from Agents.
|
|
```
|
|
|
|
### 9.5 Runtime Confusion
|
|
|
|
Signs:
|
|
|
|
```text
|
|
Workflow stages are unclear.
|
|
Handoff is manual but undocumented.
|
|
State is lost between steps.
|
|
Report synthesis has no protocol.
|
|
```
|
|
|
|
Recommendation:
|
|
|
|
```text
|
|
Create Interactive or Hybrid Runtime.
|
|
```
|
|
|
|
## 10. Self-Contained Model Agent Audit
|
|
|
|
When auditing a self-contained model-backed Agent, identify:
|
|
|
|
```text
|
|
Agent role
|
|
Embedded model
|
|
Executable method
|
|
Output format
|
|
Tool/retrieval policy
|
|
Runtime usage potential
|
|
```
|
|
|
|
Then recommend whether to:
|
|
|
|
```text
|
|
Keep as Lite
|
|
Create Agent Spec
|
|
Extract Model Card
|
|
Extract Skill
|
|
Add Runtime node
|
|
Update Model Index
|
|
```
|
|
|
|
## 11. Audit Report Format
|
|
|
|
Use this format:
|
|
|
|
```md
|
|
# CCPE Audit Report
|
|
|
|
## 1. Artifact Overview
|
|
Name:
|
|
Path:
|
|
Version:
|
|
Current form:
|
|
|
|
## 2. Primary Classification
|
|
|
|
## 3. Secondary Components
|
|
|
|
## 4. Operating Mode
|
|
|
|
## 5. Depth vs Automation Orientation
|
|
|
|
## 6. Embedded Cognitive Models
|
|
|
|
## 7. Extractable Skills
|
|
|
|
## 8. Runtime Need
|
|
|
|
## 9. Quality Assessment
|
|
|
|
| Criterion | Score | Severity | Notes |
|
|
|---|---:|---|---|
|
|
|
|
## 10. Major Strengths
|
|
|
|
## 11. Major Problems
|
|
|
|
## 12. Recommended Target Form
|
|
|
|
## 13. Proposed Refactor Direction
|
|
|
|
## 14. Proposed Files
|
|
|
|
## 15. Human Decisions Required
|
|
```
|
|
|
|
## 12. What Auditor Mode Must Not Do
|
|
|
|
Do not:
|
|
|
|
```text
|
|
Rewrite without request.
|
|
Delete original structure.
|
|
Flatten distinctive terminology.
|
|
Assume every model should be extracted.
|
|
Assume every Agent needs Runtime.
|
|
Promote candidate models to active status.
|
|
```
|
|
|
|
## 13. Auditor Mode Final Response
|
|
|
|
Final response should include:
|
|
|
|
```text
|
|
Classification
|
|
Key diagnosis
|
|
Recommended target form
|
|
Whether Refactor Mode is needed
|
|
Proposed next step
|
|
```
|
|
|
|
## 14. Final Rule
|
|
|
|
Auditor Mode is a diagnostic instrument.
|
|
|
|
It should be sharp, fair, and structurally useful.
|
|
|
|
It should reveal what the artifact really is before trying to improve it.
|
|
|