video-workbench/VIDEO_WORKBENCH.md

458 lines
26 KiB
Markdown

# VIDEO_WORKBENCH.md
## 1. Workspace Identity
This project is the local execution workspace for dimensional video, Remotion, and slide-style output.
It turns accepted GPT planning Markdown and user-provided sources into project notes, local execution plans, visual-system assets, slide/page execution folders, Remotion source projects, generated assets, renderable video files, and compact handoff material. It is optimized for practical continuation and production throughput, not heavy governance.
## 2. Collaboration Roles
```text
GPT / ChatGPT = stage 0-5 planning Markdown, outlines, visual systems, page/shot skeletons, Remotion planning briefs
User = decisions, accepted inputs, source/reference material
Codex = stage 6+ execution planning, Prompt Advisor work, Remotion implementation, visual-system materialization, iteration, local assets
```
GPT does not provide final image prompts, Codex JSON, output paths, task lists, dependency commands, render commands, complete Remotion source code, or image generation parameters. Codex should not request those from GPT as missing inputs.
For Remotion videos, accepted planning from `强哥的策划导演3.0` may provide Remotion adaptation judgment, a Remotion style-system anchor, storyboard-to-component mapping, and a Remotion Implementation Brief. The brief is a planning source, not a Codex execution package.
Codex should not rewrite GPT planning into duplicate project content. The accepted planning files belong in the project `intake/` directory, and Codex turns them into local execution decisions in `execution-plan.md`.
## 3. Project Layout
All video or slide-style work lives under `projects/`:
```text
projects/
project-id/
project.md
intake/
execution-plan.md
visual-system/
visual-system.md
refs/
iterations/
remotion/
package.json
src/
public/
slides/
slides.md
s01/
s02/
tasks/
```
Use root-level `handoff/` for repository-level, cross-project, series-level, or context-window handoff notes.
Use `docs/cases/` for Codex-side local case patterns extracted from old real GPT cases. These documents preserve reusable structure, granularity, asset layering, speaker-note style, and review dimensions. They do not restore old GPT final prompts, Codex JSON execution packages, output paths, or global text-rendering rules.
Do not split projects into `active`, `completed`, `abandoned`, `series`, or `standalone` folders. Status and series membership belong in project files because both can change during real production.
If a series needs more structure, create `projects/_series/` when the first real series needs it.
## 4. Project Notes
`projects/<project>/project.md` is a lightweight Codex-maintained note. It should record:
- project status;
- lifecycle phase from `docs/workflows/project-lifecycle.md`;
- workflow phase: new projects default to `test/audit`; switch to `production-light` only after explicit user authorization;
- medium type;
- worker profile, such as `video-slide-production`, `remotion-video-production`, or a future profile;
- source and reference links;
- which `intake/` files are GPT planning Markdown, source notes, reference notes, or background notes;
- Remotion source root when applicable, defaulting to `projects/<project>/remotion/`;
- current execution focus;
- key local execution files;
- next action.
It should not duplicate the full outline, storyboard, page design, shot design, or execution plan.
## 5. Execution Plan
Visual projects must have:
```text
projects/<project>/execution-plan.md
```
Global lifecycle rules live in:
```text
docs/workflows/project-lifecycle.md
```
This is a living Codex-maintained execution control file. It records:
- the current medium branch;
- the current lifecycle phase when it affects execution;
- the active worker profile;
- the current execution focus;
- which visual-system assets need materialization;
- for Remotion projects, source root, package manager state, composition IDs, render target, render output path, and validation commands;
- the current small-batch iteration strategy;
- which `sNN` units are in the active batch;
- the current round;
- the next decision.
It should not repeat the project ID, full intake file index, or project map from `project.md`.
## 6. Intake
`projects/<project>/intake/` holds material provided or accepted by the user:
- GPT stage 0-5 planning Markdown;
- outlines;
- storyboards, page skeletons, or shot skeletons;
- Remotion adaptation judgments, Remotion style-system anchors, Remotion storyboards, storyboard-to-component mappings, and Remotion Implementation Briefs;
- background notes;
- reference images, audio notes, or source pointers.
File naming can be agreed during the first real project. Do not create a rigid naming scheme before the real GPT outputs are known.
### Intake Heading Repair
When copied GPT / LLM intake Markdown has broken heading hierarchy, use the installed `fix-title` Skill as an external repair capability, but keep the calling protocol local to Video Workbench.
Local calling rules:
- Repair work must run through a real Agent / subagent invocation when the runtime provides one.
- The main session should pass the source file paths, `mode=artifact` or `mode=discussion`, the `fix-title` skill path, and the output directory to the worker.
- The main session should not build semantic heading plans or apply heading edits itself when an Agent carrier is available.
- For files under `projects/<project>/intake/`, output must go under:
```text
projects/<project>/intake-repair/fix-title-YYYY-MM-DD-<short-slug>/
```
The repair output package should contain:
```text
<stem>.heading-map.json
<stem>.heading-plan.json
<stem>.fixed.md
<stem>.heading-report.md
fix-title-batch-report.md
```
If repaired files replace originals, first copy the original Markdown files to:
```text
projects/<project>/intake-repair/fix-title-YYYY-MM-DD-<short-slug>/originals-before-replacement/
```
Main-session responsibilities after the worker returns:
- verify the returned files exist;
- inspect the heading map, plan, fixed copy, per-file report, and batch report;
- check that intended artifact files have one clear document title unless the report flags an intentional appended artifact;
- check that ambiguous appended artifacts are reported instead of silently merged;
- decide whether to replace originals, and write replacement notes when replacement happens.
Do not put durable intake repair output under `tmp/`. Keep reusable automation changes in `skills-vault`; Video Workbench owns only the local invocation protocol, project evidence, replacement notes, and supplier-request records.
## 7. Visual System
Visual projects use:
```text
projects/<project>/visual-system/
visual-system.md
refs/
iterations/
```
`visual-system.md` is the single record for the project's public visual system. It should cover:
- the current accepted visual system;
- materialization targets;
- trial prompts and generated reference assets;
- review notes;
- failure attribution;
- next action.
Use optional subfolders only when the project needs them:
```text
visual-system/characters/
visual-system/scenes/
visual-system/templates/
visual-system/diagrams/
visual-system/backgrounds/
visual-system/remotion/
```
Do not create a separate `makeup-still/` directory. Character anchors and identity references belong under `visual-system/`.
For Remotion projects, record the accepted Remotion style-system anchor under `visual-system/remotion/` or in `visual-system/visual-system.md` if the project is small. It should cover background systems, color and type hierarchy, component style, information density, animation language, transition motifs, chart/flow/formula handling, subtitle strategy, and reusable component suggestions. This is source authority for the Remotion implementation; it is not React/TypeScript code.
After a character anchor stage is accepted, clean the character workspace before scene generation:
- keep only current usable character references under `visual-system/characters/`;
- record the active references and their intended use in `visual-system/characters/character-reference-registry.md`;
- move historical candidates, rejected versions, prompt/review process files, previews, masks, chroma sources, and other intermediate files to `archive/characters/`;
- do not use files under `archive/characters/` as default prompt references unless a repair task explicitly reopens them.
For identity-critical shots, prompt specs must choose the person reference from the active character registry. Text description can clarify the role, but it does not replace the active character image reference.
## 8. Slides And Shots
Use `slides/` as the execution unit area for video shots, PPT-style pages, Remotion scenes, and Remotion sections.
`projects/<project>/slides/slides.md` is the unit execution fact table. It records each page, shot, Remotion scene, or Remotion section's source, batch, current prompt/spec version, current image/render/component status, review state, acceptance state, and notes. Project-level iteration strategy belongs in `execution-plan.md`.
Each `slides/sNN/` folder keeps one slide, page, shot, Remotion scene, or Remotion section together:
```text
slides/
slides.md
s01/
s01-metadata.json
s01-unit-spec.md
s01-v1-design.md # optional; required for cover/PPT/science-video/title/post-production pages
s01-v1-design-en.md # optional English companion; default design remains Chinese
s01-visual-brief.md
s01-v1-prompt.md
s01-v1-image.png
s01-v1-remotion-spec.md
s01-v1-render-check.md
s01-v1-review.md # optional; test/audit or explicit review only
s01-v2-prompt.md
s01-v2-image.png
s01-v2-review.md # optional; test/audit or explicit review only
```
All files inside `sNN/` must carry the `sNN` prefix. Iteration files use `sNN-vN-type.ext`, so files from the same version sort together.
`design` is a human-facing design specification, not an image-generation execution card. Use `sNN-vN-design.md` for cover pages, PPT pages, science-video pages, title cards, explainers, Remotion scenes with meaningful typography/layout choices, and any slide that needs later human layout, typography, text placement, or post-production. It records the design concept, intended text, placement, typography/color guidance, constraints, references, and post-production notes. `brief` stays compact and execution-facing for Codex / image generation or Remotion implementation.
`sNN-vN-design.md` is Chinese by default. If an English design companion is needed, write `sNN-vN-design-en.md` beside it. The English companion is optional and never replaces the Chinese default.
For design-gated slides, write `design` first and wait for user confirmation before writing `brief`, `prompt`, Remotion specs, or generating/rendering outputs, unless the user explicitly authorizes the whole chain. Ordinary MV shot production may skip `design` and continue directly with `brief -> prompt -> image`.
For Remotion work, `slides/sNN/` holds scene/section intent and checks. React/TypeScript source code lives under `projects/<project>/remotion/`. Do not store normal component source as append-only `sNN-vN-*` Markdown; use Git history for code changes and use `slides/sNN/` for the scene spec, component mapping, and render evidence.
New task execution must append a new slide version instead of overwriting an existing one. Resolve the next unused `vN` from existing `sNN-v*-*` files in the slide folder, and treat incomplete or superseded earlier versions as historical evidence rather than paths to reuse.
### Slide Task Orchestration
For long or context-heavy production, use local slide task orchestration:
```text
docs/workflows/slide-task-orchestration.md
docs/workflows/slide-task-templates.md
```
Project-local task records go under:
```text
projects/<project>/tasks/<task-id>/
```
Task records are for orchestration only. Durable slide artifacts remain under `slides/sNN/`.
For production work that may span multiple child sessions, keep a compact continuation file:
```text
projects/<project>/tasks/<task-id>/production-run-status.md
```
This file records the target scope, completed units, active child session, next cursor, latest callback, and stop conditions. It is the default recovery surface after parent-session compression. It is not a review report and should not duplicate prompts or image evidence.
Roles:
- Main session: receives user instructions, defines worker profile / running mode / slide scope, starts child sessions, records the parent callback target, and reads handoffs after callback.
- Child session: reads shared context and handles the assigned package. In `production-light`, it directly writes briefs, prompts/specs, generated images or Remotion code/render outputs, `slides/slides.md` updates, and a compact handoff before callback.
- Slide agent: optional worker for `test/audit`, explicit review, repair, exception handling, or intentionally isolated slide work. It handles exactly one slide when used.
Operating lines:
- `test/audit`: early project setup, core-shot proving, mechanism debugging, or failure investigation. Richer handoffs, ledgers, local review files, Agent workers, and parent inspection are allowed.
- `production-light`: normal production after core shots or the core Remotion route have reached the expected look and the user authorizes the switch in `project.md`. Keep the content line (`brief`, `prompt/spec`, generated image or Remotion source/render, `slides/slides.md`) and compact recovery handoff; omit routine review, Agent workers, packet/return files, and audit records.
Defaults:
- Worker profile for current video / MV work: `video-slide-production`.
- Worker profile for Remotion videos: `remotion-video-production`.
- Design-gated slides: cover, PPT, science-video, title, explainer, and post-production-heavy pages require `sNN-vN-design.md` before `brief`.
- Child session capacity: at most 8 ordinary production slides; reduce to 1-3 for identity-heavy, repair-heavy, core-risk, or context-expensive scopes.
- Production-light topology: main session -> child session directly.
- Test/audit topology: main session -> child session -> optional slide Agent.
- Slide agent capacity when used: exactly 1 slide.
- Execution is sequential by default; do not run child sessions or slide agents in parallel unless the user explicitly changes the rule.
- Parent orchestration is callback-only by default: after dispatch, the main session stops routine waiting; the child session wakes the parent with status, topology, resolved versions, handoff path, and image paths if complete.
- Callback is the child session's final action: all allowed project writes, `slides/slides.md` updates, narrow required checks, and the compact handoff must be complete before callback; after a successful callback the child must not write or verify additional project files.
- Polling is a fallback only when callback tooling is unavailable or the user explicitly requests polling for a named run.
- Production-light review mode: `none`.
- Future `ppt-page-production` work is reserved and may use `docs/cases/`, but it must not inherit video-slide rules blindly.
The user controls task running mode:
- `prompt only`: create briefs and prompts; do not generate images.
- `prompt and generate`: create briefs and prompts, then generate images for the task's recorded slide scope.
- `remotion plan only`: create Remotion execution specs and mappings; do not write source code.
- `remotion implement`: write or edit Remotion source and run narrow source checks; do not render final MP4 unless the task also authorizes rendering.
- `remotion render`: write or edit source as needed, run narrow source checks, and render the target MP4.
Task-level generation or Remotion render instructions count as user approval for the named scope and mode. In production-light, generated images are recorded as production-pass outputs without per-image user review or routine final verification; the user can request an append-only retry later if a result is not usable. Remotion source/render work is code work and must run the task's narrow checks before closeout. Core, repair, identity-heavy, otherwise high-risk slides, or uncertain Remotion animation systems may still pause if the task plan says they require sampling or explicit review.
For unattended production, the parent session repeats the callback loop from `docs/workflows/project-lifecycle.md`: update `production-run-status.md`, dispatch one child session, wait for callback, reload the minimal project/task state, then dispatch the next package if no stop condition is present. Exact parent context/token usage is not required for continuation because Codex may auto-compress the parent session; file-backed state is the authority.
Visual-system rules:
- `visual-system/` is the production authority.
- Style continuity must use `visual-system/style/style-prompt-master.md`.
- Character continuity must use `visual-system/characters/character-reference-registry.md` and active character reference images.
- Visible character identity cannot be generated from text-only prompts when identity matters.
- Slide prompts must use the `gpt-image-2` Skill Advisor / host-native workflow.
- Remotion implementation must use the installed Remotion Skill before code edits. Read `C:\Users\wangq\.agents\skills\remotion\SKILL.md`, then read relevant `rules/*.md` files for the task area.
Remotion project rules:
- Default source root: `projects/<project>/remotion/`.
- Default deliverable: source code plus renderable MP4.
- One-off project-specific Remotion source belongs in Video Workbench. Reusable products, reusable templates, deployable systems, and business software belong in `work-projects` unless the user explicitly keeps a prototype local.
- The user has standing task-level permission for Remotion tasks to use `npx create-video`, npm dependency installation, and Remotion render commands when needed. Normal sandbox/network approval requests still apply at command time.
- Keep source code, package files, and source assets in Git by default. Keep `node_modules`, caches, build outputs, and rendered media ignored by default.
- Use `projects/<project>/remotion/public/` for assets referenced with Remotion `staticFile()` unless a task records a different local convention.
- Use `projects/<project>/renders/` or `projects/<project>/exports/` for MP4 outputs unless the task records another render path.
- For Remotion code, normal source edits are not append-only `sNN-vN-*` artifacts; use Git history and code review. Append-only rules still apply to Markdown control artifacts, scene specs, render checks, and generated media receipts.
Review rules:
- Production-light review defaults to `none`.
- Test/audit review defaults to local-only unless the task says otherwise.
- Do not call Minimax MCP or other external vision/review services unless the user explicitly authorizes that task.
- Generated production slides can be tracked in `tasks/<task-id>/task-log.md` and `slides/slides.md` instead of a full per-slide review.
- Remotion render checks are source/render validation, not external visual review. Record the command, composition ID, output path, and result in the compact handoff or a `sNN-vN-render-check.md` file when the check is scene-specific.
## 9. Investigations
Use `investigations/` for research reports, experiments, comparisons, draft workflow proposals, and other reviewable evidence.
`tmp/` is deprecated and should not receive new durable work. Short-lived scratch material should either be deleted or promoted into `investigations/` when it becomes useful evidence.
`garden-gpt-image-2/` remains reserved for tool-level image prompt or workflow traces when that path is used.
## 10. Word Budget
Use the default standard:
```text
1 minute = about 220 Chinese characters of voiceover
10 minutes = about 2200 Chinese characters
```
Adjust only when the user specifies a different narration speed.
## 11. Asset Policy
Image and audio generation can be automated when available skills or APIs are appropriate.
Video generation from images remains manual or semi-manual for now because output reliability varies. Remotion is a separate programmatic video path: use it when the accepted plan calls for React/TypeScript animation, scene composition, captions, charts, diagrams, or renderable MP4 output from source code.
Editing and publishing are human-controlled unless explicitly automated later.
Codex is the default prompt owner for final image prompts in this workspace, but final prompts must be produced through the `gpt-image-2` Skill workflow rather than direct freehand drafting. GPT planning Markdown remains a source specification, not a final prompt.
For project-bound image prompts, Codex must:
- run the `gpt-image-2` mode check before prompt work;
- read the relevant Skill template or reference file;
- save the rendered prompt/spec as a reviewable project file;
- list the source files actually used;
- wait for user approval before image generation.
A task-level user instruction that records `prompt and generate` plus the slide scope counts as generation approval for that task scope. A task recorded as `prompt only` never authorizes image generation.
Direct prompts may exist only as clearly labeled scratch or non-generation-ready drafts. They must be rebuilt through the `gpt-image-2` Skill workflow before use.
### Reference-Capable Image Generation
The current and future default success path for project-bound reference generation is:
```text
gpt-image-2 Skill Advisor prompt/spec
→ load the real reference image into the conversation context
→ call host image_gen with the loaded image assigned an explicit role
→ copy the selected output into the project or investigation
→ record prompt, input reference path, output path, and compact handoff/status
```
This route was validated in:
```text
investigations/2026-06-24-s14-reference-capable-generation/
```
Use this path instead of API/Garden image generation for normal Video Workbench work. Do not reopen the "can references work" question for ordinary project-bound image generation unless the host image tool is unavailable, the reference file cannot be loaded, or the user explicitly changes the policy.
For every reference-capable generation:
- the prompt must still be produced through the `gpt-image-2` Skill workflow;
- the actual image reference must be loaded into the conversation context before generation;
- the prompt must name the loaded image role, such as `Image 1: identity reference`, `Image 1: style reference`, or `Image 1: edit target`;
- merely writing a local path in the prompt is not enough;
- the output must be copied from the host-generated default location into the project, investigation, or other durable workspace path;
- the compact handoff or task status must record whether the generation used the verified host reference-by-visible-context path; a slide-agent return receipt is required only when the task actually uses a slide Agent in test/audit, repair, exception, or explicit review mode.
Reference selection defaults:
| Shot or asset need | Default reference choice |
| --- | --- |
| Near shot, face, half-profile, expression, identity-heavy frame | Use the matching crop reference first. |
| Distant shot, landscape-led frame, aerial / wide scene, low identity burden | Use the accepted character anchor board or project-approved distant reference first. |
| Medium action, back view, walking pose, prop continuity | Use the matching crop, transparent slice, or anchor board according to the visible direction. |
| High identity fidelity project with many close shots | Consider creating higher-resolution crops through image-to-image from the accepted anchor board before production. |
For `在路上`-style landscape-led MV projects, a crop set derived from the accepted character anchor board and made transparent is sufficient by default when the goal is consistency rather than high-fidelity portrait likeness.
## 12. Capability-Request Mode
When a required capability is missing, create a supplier ticket and pause the dependent work.
Tickets go to:
```text
requirements/ccpe/
requirements/skills-vault/
```
Use `requirements/ccpe/` for local work agents, CCPE-Lite artifacts, agent specs, runtime rules, model cards, invocation contracts, and governed cognitive assets.
Use `requirements/skills-vault/` for deterministic automation tools, scripts, exporters, batch processors, and installable automation skills.
## 13. Text Encoding
All local project documents are UTF-8 unless a file explicitly declares another encoding.
When using PowerShell to read Markdown, JSON, YAML, text, handoff, prompt, or discussion files, use explicit UTF-8:
```powershell
Get-Content -LiteralPath <path> -Encoding UTF8
Select-String -Path <path> -Pattern <pattern> -Encoding UTF8
```
Do not wait for mojibake or a failed first read before applying UTF-8.
## 14. CCPE Consumption
Video Workbench consumes CCPE through the single external interface:
```text
C:\Users\wangq\Documents\Codex\ccpe-system\Interface.md
```
When preparing CCPE integration or invocation work, read `Interface.md` first and follow it to:
```text
C:\Users\wangq\Documents\Codex\ccpe-system\runtimes\hybrid\ccpe-agent-consumer-setup.md
```
Do not duplicate canonical CCPE definitions here. Do not hardcode Agent, Lite, Skill, Runtime, or protocol paths as default entrypoints. Use `ccpe-consumption/` for interface notes and future project-local invocation setup records.
Reusable or canonical expert agents belong in CCPE. Project-local slide task orchestration, child-session records, and slide-agent packets belong in Video Workbench and are not formal CCPE participant output. If this workflow is later packaged as deployable software, production/business agent implementation belongs in the target software project, not in Video Workbench.