video-workbench/docs/workflows/slide-task-orchestration.md

523 lines
30 KiB
Markdown

# Slide Task Orchestration
## Status
- Status: active workflow rule.
- Scope: local Video Workbench slide, scene, page, and Remotion production tasks.
- First profile: `video-slide-production`.
- Remotion profile: `remotion-video-production`.
In this workflow, `slide` is the historical execution unit. A unit may be a video frame unit, a PPT page, a Remotion scene, a Remotion section, or another visual page-like unit. Current rules implement image-led video/MV slide production and Remotion video production.
## Goals
- Keep the main session small enough to survive long production work.
- Keep style and character continuity anchored in `visual-system/`.
- Keep user review focused on real decisions, not every generated image.
- Keep task evidence light but sufficient for continuation after context compression.
- Keep mandatory write guards inside child sessions, so the main orchestration session can stay lightweight in production.
- Separate test / audit work from normal production so routine production does not pay test-level token cost.
## Worker Profiles
Current profile:
```text
video-slide-production
```
This profile uses:
- `visual-system/` as the production authority;
- `visual-system/style/style-prompt-master.md` for style modules and style prompt language;
- `visual-system/characters/character-reference-registry.md` for active character references;
- `slides/` for slide artifacts;
- `gpt-image-2` Skill Advisor / host-native prompt workflow.
For Video Workbench, the `gpt-image-2` Skill is used as the prompt/spec Advisor. Normal project-bound image generation does not need to prove Garden, API, multipart upload, or local `gpt-image-2` script execution. The expected generation path is:
```text
gpt-image-2 Advisor prompt/spec
-> load the real reference image into visible conversation context when references are required
-> call host built-in image generation with the loaded image named as Image 1 / Image 2
-> copy the host output into slides/sNN/
-> record prompt/spec evidence, loaded-reference evidence, host output path, and compact handoff/status
```
Garden/API use is expected to be `no` unless the user explicitly changes the project policy.
Remotion profile:
```text
remotion-video-production
```
Use this profile when accepted planning material, the user, or the task says the video should be implemented in Remotion, when Codex writes or edits Remotion code, or when the expected deliverable is source code plus a renderable MP4.
This profile uses:
- accepted Remotion planning material in `intake/` as source specification, not as code;
- `visual-system/` and the Remotion style-system anchor as production authority;
- `slides/slides.md` as the scene/section fact table when the project decomposes into units;
- `slides/sNN/` for scene specs, component mapping notes, and render-check receipts;
- `projects/<project>/remotion/` as the default source root;
- `C:\Users\wangq\.agents\skills\remotion\SKILL.md` and task-relevant Remotion Skill rule files before code edits;
- narrow code/render validation, such as typecheck/build when available and `npx remotion still` or the task's render command.
For Remotion work, `gpt-image-2` is used only when the Remotion project separately needs generated still assets. Remotion implementation itself is React/TypeScript source work, not image prompt work.
Default Remotion source and output paths:
```text
projects/<project>/remotion/ # source
projects/<project>/renders/ # rendered MP4s or still checks unless overridden
projects/<project>/exports/ # final export package when needed
```
One-off project-specific Remotion source belongs in Video Workbench. Reusable Remotion products, reusable templates, deployable systems, and business software belong in `work-projects` unless the user explicitly keeps a prototype local.
Reserved future profile:
```text
ppt-page-production
```
Future PPT tasks may use `docs/cases/` as an additional case-pattern knowledge base and may need a different page brief, prompt, and review structure. Do not apply `video-slide-production` rules blindly to PPT work; design that profile when the first real PPT task needs it.
Every task plan must record the active profile, for example:
```text
Worker Profile: video-slide-production
```
or:
```text
Worker Profile: remotion-video-production
```
## Operating Lines
Use two lines:
```text
test/audit
production-light
```
`test/audit` is for early project setup, first visual-system proving, core-shot trials, workflow debugging, failure investigation, or explicit user-requested review. It may use richer handoffs, local review files, Agent workers, packets, returns, ledgers, parent-side inspection, completion gates, and longer narrative logs.
`production-light` is for normal production after core shots or the core Remotion route have reached the expected look. It is the default production line. It keeps only:
- content artifacts: `sNN-vN-brief.md`, `sNN-vN-prompt.md` or `sNN-vN-remotion-spec.md`, generated image or Remotion source/render output when allowed, and `slides/slides.md`;
- compact recovery records: child-session handoff and one-line task log entries.
In production-light, review mode defaults to:
```text
review: none
```
Generated images are recorded as production-pass outputs. They do not need per-image user review before the workflow continues. If the user later finds a result unusable, run an append-only repair or retry as a new version.
Project phase is recorded in `projects/<project>/project.md`. New projects default to `test/audit`. Once the user explicitly authorizes formal production, switch the project phase to `production-light`; future runs then load the production-light workflow by default. A named session or slide scope can still override back to `test/audit` for local diagnosis.
## Directory Model
Use project-local `tasks/` for orchestration records:
```text
projects/<project-id>/
tasks/
<task-id>/
task-plan.md
task-log.md
production-run-status.md # for multi-package production or compression-safe continuation
session-01/
child-session-handoff.md
# production-light default: no packets/returns
# test/audit or exception scopes may add:
packets/
s05-v1-slide-agent-packet.md
returns/
s05-v1-slide-agent-return.md
slides/
s05/
s05-v1-brief.md
s05-v1-prompt.md
s05-v1-image.png
s05-v1-remotion-spec.md
s05-v1-render-check.md
s05-v1-review.md # test/audit or explicit review only
remotion/
package.json
src/
public/
renders/
# generated render outputs ignored by default
```
Rules:
- `tasks/<task-id>/` records orchestration and agent communication.
- `slides/sNN/` remains the durable home for slide artifacts.
- In `slides/sNN/`, Markdown control/script artifacts are Git-tracked by default; generated image, audio, and video files are asset-layer outputs ignored by default and referenced from control files by path, status, and hash when needed.
- `remotion/` is the default source root for project-specific Remotion videos. Source code, package files, and source assets are Git-tracked by default.
- Remotion dependencies, caches, build outputs, and rendered media are ignored by default. Record render paths and status in control files instead of committing generated media unless the user promotes a named artifact.
- `intake/` and `visual-system/` are source-authority areas and should stay in Git, including approved reference images.
- `archive/` is for discarded images and discarded documents by default. Do not add archive contents to Git unless the user explicitly promotes a specific item back into a source-authority area.
- Task records link to slide artifacts by path and should not duplicate full prompts.
- `production-run-status.md` is the compact continuation surface for multi-package production. It records scope, completed units, active child session, next cursor, latest callback, and stop conditions.
- Task records do not own the slide version number. Before each slide run, resolve the next unused `sNN-vN-*` version from the existing files in `slides/sNN/`, then write only that new version.
- Existing slide artifacts are historical evidence even when incomplete or superseded. Do not overwrite them to make a later task look cleaner.
- Suggested task id pattern: `YYYY-MM-DD-<short-purpose>`.
## Artifact Version And Write Policy
Default policy:
- New task execution appends a new slide version. It does not reuse or overwrite an existing `vN`.
- The next version is the next unused integer after all existing files matching `sNN-v*-*` in the slide folder, regardless of whether earlier versions have missing brief, prompt, image, or review files.
- Incomplete earlier versions should be recorded as incomplete, broad-run, interrupted, superseded, or invalid as appropriate. Do not backfill a missing prompt or brief in a way that implies it was the original generation prompt.
- Every child session or slide-agent packet must state `Overwrite allowed: no` unless the user explicitly authorizes an overwrite repair for named files.
- Before writing brief, prompt, image, review, packet, return, or handoff files, the writing agent must check whether each target file already exists. If a target exists and overwrite is not explicitly authorized, the agent must stop and report an artifact collision instead of writing.
- An approved overwrite repair must preserve or explicitly cite the prior file state before replacement. Routine production and workflow tests should not use overwrite repair.
This rule applies even when the previous run was malformed. Malformed output is still evidence about the workflow and should be superseded by a new version, not silently replaced.
Remotion source files under `projects/<project>/remotion/` are ordinary code, not append-only generated artifacts. Edit them in place under Git while respecting the user's uncommitted changes. Do not delete or replace an existing Remotion scaffold just to get a clean project; inspect it and work with it. Append-only versioning still applies to scene specs, design docs, render-check receipts, generated media receipts, packets, returns, and handoffs.
## Session Roles
### Main Session
The main session is the only session that talks with the user by default.
Responsibilities:
- receive the user instruction;
- define the worker profile, running mode, slide scope, and review expectations;
- read the project phase from `project.md`, unless the user explicitly overrides it for a named session/scope;
- create or update `task-plan.md` only with the active scope and dispatch/callback status needed for continuation;
- split work into sequential child sessions;
- start child sessions one at a time and name them with a user-facing `子会话:...` title when the tool surface supports thread naming;
- record the parent callback target for the child session;
- read the child session's short completion handoff after callback;
- ask the user only for blockers, core decisions, or requested sampling;
- update `task-log.md` with compact status.
- update `production-run-status.md` after each callback when the task may continue across multiple child sessions.
The main session should not write per-slide briefs, final prompts, images, or routine reviews by default. In production-light it should also not redo routine child-session checks unless the child session reports a problem.
Production orchestration default:
- Launch or continue exactly one child session for the approved scope.
- Record the parent thread id and require callback-only completion unless the task plan explicitly declares polling fallback.
- After dispatch, stop routine waiting. Do not inspect the child thread for routine completion.
- In `production-light`, instruct the child session to work directly. Do not start Agent/SubAgent workers, do not write packet files, do not write return receipt files, and do not run a final verification pass.
- The child session must write `child-session-handoff.md` before sending any callback.
- The child session must complete all allowed production-light writes, `slides/slides.md` updates, compact handoff writing, and any narrow file-existence checks before callback.
- The child session sends a concise callback to the parent only as its final action. After a successful callback, it must not write or verify additional project files.
- The callback payload must include status, topology, resolved versions, handoff path, image paths if complete, and any problems or user decisions needed.
- If a package would exceed 8 ordinary production slides, split it into smaller packages unless the task plan explicitly defines a custom package size.
- Consume only the handoff's status, topology, review mode, resolved versions, image paths, and needs-user field by default.
- Continue to the next approved slide/scope when status is `complete` and the handoff reports no problems.
- For unattended production, reload the minimal project/task state after callback and continue dispatching packages until the target scope is complete or a stop condition is present.
- Open detailed prompt, return, ledger, or image evidence only when the handoff reports `blocked`, `failed`, `warning`, artifact collision, duplicate worker, user-requested audit, or explicit sampling/review.
Polling fallback is allowed only when callback tooling is unavailable or the user explicitly requests polling for a named run. If fallback polling is used, the task plan must record the reason, the status-check schedule, and the timeout / blocker policy before dispatch.
### Child Session
The child session is the small-batch context holder.
Default capacity:
- maximum 8 ordinary production slides per child session;
- fewer slides for identity-heavy, repair-heavy, core-risk, or context-expensive work;
- sequential only, not parallel.
This maximum is a default child-session chunk size, not a project-wide production cap. After the workflow is proven stable, the main session may dispatch additional child sessions or a broader user-approved scope while preserving the per-child-session capacity rule unless the task plan explicitly changes it.
The 8-slide default is based on accepted production-light compression validation in `projects/2026-06-23-在路上/tasks/2026-06-25-8slide-compression-validation/`: eight images accepted into the production pass by user rule, 174K tokens, 67% context, and 38m37s elapsed. The callback-final contract was verified in the child thread tail. The earlier 5-slide validation remains historical capacity evidence and exposed the callback-order defect, so child prompts must still state that callback is the final action.
Responsibilities:
- read project rules and the task plan;
- read shared project context for assigned slides;
- read active `visual-system/` files needed for assigned slides;
- resolve the next unused slide version and write one new `sNN-vN-brief.md` per assigned slide;
- in `production-light`, write `sNN-vN-prompt.md`, generate `sNN-vN-image.png`, and update `slides/slides.md` directly;
- for `remotion-video-production`, read the installed Remotion Skill and relevant rule files before code edits, write scene specs or component mapping notes as needed, update `projects/<project>/remotion/`, run narrow source/render checks, record composition IDs and render paths, and update `slides/slides.md` or task status directly;
- in `production-light`, do not create slide-agent packets, do not start Agent/SubAgent workers, do not write return receipt files, and do not run a routine final verification pass;
- in `production-light`, finish all project writes and narrow required checks before callback; do not continue writing after callback succeeds;
- in `test/audit`, repair, or exception scopes, create one slide-agent packet per slide and start exactly one slide agent per slide, sequentially;
- in `test/audit`, repair, or exception scopes, verify expected files and run the required completion gate before returning;
- run instruction review only when the task plan or packet requires review; production-light defaults to no review;
- write `child-session-handoff.md` with a compact parent-facing summary;
- send the parent callback only after `child-session-handoff.md` exists.
For Remotion work, the narrow source/render checks recorded in the task plan are part of producing code and MP4 output. They are required validation, not a routine visual-review pass.
If a test/audit run needs an Agent and the runtime does not expose an Agent tool inside the child session, record this as a runtime fallback requirement in `child-session-handoff.md`. The main session may then start the slide agent as a fallback, but that run must be reported as a fallback orchestration test, not as a full child-session-owned slide-agent test.
### Slide Agent
The slide agent is the optional single-slide worker for `test/audit`, repair, exception handling, explicit review, or intentionally isolated slide work. It is not the default production-light carrier.
Default capacity:
- one slide per invocation;
- no ranges;
- no batches.
Responsibilities:
- read the packet and brief;
- follow the `gpt-image-2` Skill Advisor / host-native prompt workflow;
- for Remotion packets, follow the installed Remotion Skill and task-relevant rule files instead of the `gpt-image-2` workflow;
- preserve visual-system reference choices in `sNN-vN-prompt.md`;
- write the new-version `sNN-vN-prompt.md`;
- generate an image only when the packet allows generation;
- write or edit Remotion code only when the packet explicitly uses `remotion-video-production`;
- write `sNN-vN-review.md` only when required;
- write a compact return receipt.
Lifecycle rule:
- A slide agent is one-shot.
- It receives exactly one immutable packet.
- It owns exactly one slide/version.
- It writes exactly one compact return.
- After the return is recorded, the agent is terminal / closed.
- Closed agents must not be continued, reused, or assigned follow-up work.
- Any repair, retry, or next slide must launch a new agent with a new packet and, when writing slide artifacts, a new append-only version.
The slide agent must not:
- overwrite existing slide artifacts unless the packet explicitly authorizes overwrite repair for named files;
- write the brief in the default workflow;
- process a second slide;
- replace style-master guidance with ad hoc style text;
- replace active character image references with text-only identity description when character continuity matters;
- use archived style or character references unless the packet explicitly reopens them;
- invent Remotion component style outside the accepted Remotion style-system anchor unless the task explicitly asks for a new pilot;
- call Minimax MCP or external review tools by default.
## Running Modes
### Prompt And Generate
Use when the user authorizes generation for the task scope.
Flow:
```text
child session writes brief
-> child session writes prompt
-> if visible character identity matters, load required character reference image(s)
-> child session calls host image generation through the visible-context reference path
-> child session updates slides/slides.md
-> child session writes compact handoff and callbacks parent
```
For this workflow, "load required character reference image(s)" means the actual local image is made visible to the generation context before host image generation. A local filesystem path written in a prompt is traceability only and does not count as loading the image.
When a slide agent is used, it must not treat "Garden/API not used" as a failure. The expected evidence is that the Advisor prompt/spec was prepared and the host image-generation path was used with the required visible-context reference.
### Prompt Only
Use when the user wants prompt sampling, external cross-checking, or delayed generation.
Flow:
```text
child session writes brief
-> child session writes prompt
-> child session writes compact handoff and callbacks parent
```
Generation can happen later through a separate task.
### Remotion Plan Only
Use when the user wants Codex to translate accepted Remotion planning into local execution specs without writing source code.
Flow:
```text
child session reads intake + visual-system Remotion anchor
-> child session writes or updates scene specs / component mapping notes
-> child session updates slides/slides.md or task status
-> child session writes compact handoff and callbacks parent
```
No source code, npm command, or render command is allowed in this mode.
### Remotion Implement
Use when the user authorizes Remotion source work but does not request a final MP4 render.
Flow:
```text
child session reads Remotion Skill and relevant rules
-> child session creates or updates projects/<project>/remotion/
-> child session writes or updates scene specs when needed
-> child session runs narrow source checks recorded in the task plan
-> child session records source paths, composition IDs, check result, and handoff
```
### Remotion Render
Use when the user authorizes source work plus a renderable MP4.
Flow:
```text
child session reads Remotion Skill and relevant rules
-> child session creates or updates projects/<project>/remotion/
-> child session runs narrow source checks
-> child session runs the target Remotion still/render command
-> child session records composition ID, output MP4 path, check result, and handoff
```
The user has given standing task-level permission for Remotion tasks to use `npx create-video`, npm dependency installation, and Remotion render commands when needed. The agent must still request sandbox or network approval at command time when required.
## Authorization Rule
The user gives the main session concrete instructions for:
- running mode;
- slide scope;
- generation permission;
- Remotion implementation or render permission;
- operating line;
- review or sampling expectations.
If the task says `prompt only`, no image generation is allowed.
If the task says `prompt and generate`, the task plan must record the authorized slide scope and exceptions. That task-level instruction is the generation approval for the named scope. In production-light, generated images are accepted into the production pass without per-image user review. Core, repair, identity-heavy, or otherwise high-risk slides may still require prompt-only handling or an explicit pause when the task plan says so.
If the task says `remotion plan only`, no source edits or render commands are allowed. If the task says `remotion implement`, source edits and narrow source checks are allowed but final MP4 rendering is not required. If the task says `remotion render`, source edits, checks, and the target MP4 render are authorized for the named scope.
## Visual-System Requirements
`visual-system/` is the production authority.
For every slide:
- use active visual-system files, not memory reconstruction;
- choose style from `visual-system/style/style-prompt-master.md`;
- record style module and active style references in the brief and prompt;
- choose character references from `visual-system/characters/character-reference-registry.md` when a visible character matters.
For every Remotion scene or section:
- use the accepted Remotion style-system anchor from `visual-system/remotion/` or `visual-system/visual-system.md`;
- preserve component style, information density, animation language, transition motifs, chart/flow/formula handling, and subtitle strategy from the accepted planning;
- map GPT-provided component suggestions to real React/TypeScript components during Codex implementation;
- read `C:\Users\wangq\.agents\skills\remotion\SKILL.md` and the relevant Remotion Skill rule files before code edits;
- do not use CSS transitions or CSS animations for timeline-critical motion; use Remotion frame-based animation patterns from the Skill.
For visible character slides:
- text-only identity prompting is not sufficient;
- the brief must name the character reference image(s);
- the prompt must assign visible-context roles such as `Image 1: identity reference`;
- if the active generation path cannot load the required character image reference, the agent must block instead of generating a text-only identity-critical image.
## Document Structures
Stable templates live in:
```text
docs/workflows/slide-task-templates.md
```
Required per-slide artifacts:
```text
sNN-vN-design.md # required for cover/PPT/science-video/title/post-production-heavy slides
sNN-vN-design-en.md # optional English companion; default design remains Chinese
sNN-vN-brief.md
sNN-vN-prompt.md
sNN-vN-image.png # only when generation is allowed and succeeds
sNN-vN-remotion-spec.md # for Remotion scene/component intent when applicable
sNN-vN-render-check.md # for scene-specific Remotion still/render evidence when applicable
sNN-vN-review.md # only for test/audit or explicit review
```
`design` is the human-facing page or cover design spec. It is for user review and later layout / typography / post-production. It may include visual concept, intended text, text placement, font/color guidance, case references, design constraints, and post-production notes. It is not the prompt and should not be treated as a runnable image-generation artifact.
`sNN-vN-design.md` is Chinese by default. Use `sNN-vN-design-en.md` only as an optional English companion when needed.
`brief` is the universal execution card. It is written by the child session.
`prompt` is an attachment to the brief. It should be runnable or easy to copy into another platform for cross-validation.
`remotion-spec` is an execution-facing scene/component mapping for Remotion. It records source basis, composition/component target, text and data to render, animation intent, timing, assets, acceptance criteria, and which Remotion Skill rule files were read. It is not a replacement for React/TypeScript source.
`render-check` records the command, composition ID, frame or render target, output path, and result for Remotion validation.
`review` is absent by default in production-light. Create it only for test/audit, core trials, identity-heavy sampling, repair targets, explicit user requests, uncertain failures, or durable reference acceptance.
When review is required or suppressed, the task plan or slide-agent packet should say so explicitly. Production-light tasks default to `review: none`; test/audit tasks default to local review unless the task plan disables it.
Write `design` before `brief` for cover pages, PPT pages, science-video pages, title cards, explainers, or any slide that needs human text/layout/post-production decisions. Pause for user confirmation after `design` unless the task explicitly authorizes continuing through `brief`, `prompt`, and generation. Ordinary MV shot slides can skip `design` and continue directly through `brief -> prompt -> image`.
## Production-Light Closeout
Production-light uses a minimal closeout, not a full audit gate.
The child session must:
- assigned slide versions were resolved by scanning existing `slides/sNN/sNN-v*-*` files;
- no target file was overwritten unless an explicit overwrite repair authorized that exact path;
- each generated slide has a project image path recorded in `slides/slides.md` and the compact handoff;
- each Remotion task records source root, composition ID, check command, check result, and render output path when rendering was authorized;
- review files are absent when review mode is `none`;
- the child session handoff exists before callback.
Do not run a routine final verification pass in production-light. Do not compute hashes, build file inventories, inspect generated images, or write return receipts unless the task explicitly asks for that evidence or a problem occurs. For Remotion, the task's narrow source/render checks are required and should stay limited to the declared composition or render target.
If any production-light closeout item fails, return `blocked`, `failed`, or `warning` and state what the main session should inspect. The main session then leaves lightweight production mode for that item and performs targeted inspection only.
## Test / Audit Completion Gate
Use the fuller completion gate only in `test/audit`, repair, explicit review, or exception scopes. The child session should confirm:
- append-only version resolution and no overwrite;
- expected brief, packet, prompt, image when generation is allowed, and return files;
- expected Remotion source files, composition registration, render-check receipt, and MP4 output when Remotion rendering is allowed;
- review files are present when review was required and absent when review was disabled;
- exactly one owning slide agent for that slide/version when Agents are used;
- each slide agent return is recorded and the agent lifecycle is terminal / closed;
- required visible-context image references were loaded or the run blocked before generation;
- generated project images have recorded project paths and source paths; hashes are optional unless requested;
- Remotion MP4 outputs have recorded project paths; hashes are optional unless requested;
- the child session handoff contains a compact parent summary.
## Review Rule
Production-light default review is `none`. Test/audit default review is local-only.
- Do not call Minimax MCP by default.
- Do not transmit private project images to external services unless the user explicitly authorizes it for that task.
- Completeness checking is a test/audit concern. Production-light uses only minimal closeout.
- Instruction / visual review is optional and depends on the task plan.
- If review mode is `none`, do not write `sNN-vN-review.md`; the handoff, and the return when a slide Agent was used, must still state that review was intentionally suppressed.
- For character-visible slides, the prompt or handoff should state the required character image reference role.
- For style-sensitive slides, the prompt should use the active style module from the style master.
- For host-native generation, production-light records only what is useful for continuation:
- `gpt-image-2` Advisor prompt/spec evidence;
- loaded visible-context image role such as `Image 1`;
- copied project image path.
Routine non-core generated slides can be tracked in:
```text
tasks/<task-id>/task-log.md
slides/slides.md
```