Does the LoRA read the proxy?
Six scenes. Each starts from a released first frame (for its style) and everything after frame 0 is ours: the geometry, the actions, the camera, and the caption compiled from the same simulation. MiniMax-H3 then generates each scene twice with the same anchor, proxy video, caption and seed — once in its native reference mode, once with the CWM LoRA. The only variable is whether the model was taught to read depth and semantics. Where a fourth video is shown, it is LynnReal-Omni (a 32B H3-architecture model with native frame-aligned controls) given the same proxy as a control video, zero-shot: no LoRA, the same anchor, the same caption content rewritten into its six-section prompt.
Board and drive off
He walks up the boulevard; a shuttle comes down it, halts beside him; he climbs in; it pulls away with him inside. Anchor: released example 40.
A wagon passes, the camera eases back
He runs up the shoulder; a wagon comes down the causeway on his right and passes; the boom lengthens from 3.75 to 6.5 m as it does. Anchor: example 19.
Turn away from the river
He runs along the water's edge, then breaks right; the follow rig swings round behind him and the river leaves the frame. Anchor: example 4.
Stop, and the camera orbits
He walks to the reliquary, stands for two seconds while the camera swings 40° round him, then walks on. Anchor: example 30.
Two leaps
He runs across the apron and leaps the curb twice; the caption carries the leaps because the JUMP state is read from the simulation. Anchor: example 25.
Stop, push in, pull back
He runs up the road, stops at the drums while the camera pushes in from 4 m to 2.2 m, then runs on as it drops back. Anchor: example 13.
Long takes, the authors' way
Board and drive off — 4 windows, 394 frames (16.4 s)
The same scene carried on: 4 windows of 124 frames with 90 new frames each, Retake34 overlap pinned, one caption compiled per window, stitched with the overlap dropped.
Two leaps — 4 windows, 394 frames (16.4 s)
The same scene carried on: 4 windows of 124 frames with 90 new frames each, Retake34 overlap pinned, one caption compiled per window, stitched with the overlap dropped.
Streaming, chunk by chunk
Boulevard, 540p, six chunks of 22 frames
The release has no entry that takes a frame-aligned control and streams, so this borrows its anchored structure-edit recipe: each chunk is a video-edit conditioned on the anchor and that chunk's proxy, with the last frame of the previous chunk pinned into the first latent slot. 21 new frames (0.88 s) per chunk, four denoiser steps, one H200, one resident process. Seams are as continuous as the frames inside a chunk; what drifts is texture: the road coarsens chunk by chunk because each chunk restarts from one frame. Carrying the release's latent history softens it; longer chunks nearly remove it, and one 124-frame call is both the cleanest and, per frame, the fastest.
| configuration | conditioning s | generate s | of which DiT (4 steps) | decode s | peak GB | × slower than real time |
|---|---|---|---|---|---|---|
| as released: cuDNN attention, official VAE, every module staged to CPU between phases | 41.8 | 37.3 | 14.6 | 0.54 | 78 | 90× |
| as released, lightweight VAE | 40.9 | 35.6 | 14.6 | 0.34 | 75 | 87× |
| FlashAttention-3 | 40.3 | 36.1 | 13.4 | 0.51 | 78 | 87× |
| FlashAttention-3, lightweight VAE | 40.5 | 34.5 | 13.4 | 0.32 | 75 | 86× |
| FlashAttention-3, nothing staged, text encoder pinned on the GPU | 2.5 | 18.0 | 13.4 | 0.52 | 136 | 23× |
| same, lightweight VAE | 2.4 | 17.8 | 13.4 | 0.32 | 133 | 23× |
| same, Qwen vision blocks dropped from the text context (946 rows instead of 8692) | 2.3 | 12.6 | 8.0 | 0.51 | 136 | 17× |
| same as pinned, plus the release's bounded FramePack latent history | 2.5 | 19.1 | 14.4 | 0.52 | 136 | 25× |
| pinned, 39-frame chunks | 2.9 | 24.4 | 18.1 | 1.02 | 136 | 17× |
| pinned, 56-frame chunks | 3.4 | 31.5 | 23.3 | 1.53 | 137 | 15× |
| no chunking: the whole 124 frames in one call | 5.0 | 64.6 | 48.9 | 3.66 | 141 | 14× |
Made with examples/gallery.py and docs/mkgallery.py. Videos re-encoded for the web at 672 px; the conditions themselves are 336×192 float32 depth + uint8 semantic ids.