GitHub - antirez/h3.c: MiniMax H3 inference engine for Mac computers

h3-metal Native MiniMax-H3 inference for Apple Silicon. The project is being built as a sequence of working vertical slices: deterministic host/model metadata first, then portable Metal block parity, prompt encoding, prompt-to-video/audio, and first/last-frame conditioning and then ordered references. Prompt-to-video/audio, first/last-frame conditioning, and ordered Ref2VA image/video/audio references work end to end. The current work is incremental H3-specific Metal performance and memory optimization on M3 Max and M5 Max. Tutorial 1. Build and inspect the model The examples assume that the Hugging Face snapshot is in ./MiniMax-H3 and that FFmpeg and FFprobe are available on PATH. make -j8 mkdir -p outputs ./h3 --info -d ./MiniMax-H3 --info checks the model layout and prints the selected Metal device without mapping all weights or generating media. Run ./h3 --help for the complete CLI reference. Without -p, the same binary starts an Iris-style interactive session: ./h3 -d ./MiniMax-H3 --width 512 --height 512 --steps 6 Type a prompt to generate a numbered video. The session keeps the exact BF16 prompt conditioning, prepared DiT, and video decoder in memory, so repeating a prompt with another seed avoids loading and encoding them again. Useful commands are !status, !seed random, !seconds 2, !show, !save output.mp4, and !cache. Use !help for the full, short list. First/last-frame conditioning is persistent in the session: h3> !first opening.png h3> !last ending.png h3> The camera moves slowly around the subject. Use !first clear or !last clear to remove an anchor. Generated videos are written to the session directory printed at startup. For a general Ref2VA conditioning image, use !ref-image PATH instead. Images are appended in order and exposed to the model as <Picture 1>, <Picture 2>, and so on; filenames have no meaning to the model. h3> !ref-image person.png h3> Make the person shown in Picture 1 wave to the camera. !refs lists the current order, !ref-remove N removes one entry, and !refs clear removes them all. Ref2VA references cannot be mixed with !first/!last anchors. 2. Make a first fast video Start with the validated balanced preset. It generates 22 frames at 24 fps (about 0.92 seconds), displays the evolving middle-video frame after every denoising transition in a supported graphical terminal, and prints phase timings: ./h3 --profile \ -d ./MiniMax-H3 \ -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur, soft footsteps and wind." \ --width 512 --height 512 \ --frames 22 --steps 20 \ --layers 45 --reuse 2 \ --show \ -o outputs/fox-fast.mp4 This is deliberately not the most aggressive configuration: --steps 20 performs the default 20 denoising passes. --reuse 2 computes 11 fresh denoiser velocities instead of all 20 and extrapolates the skipped transitions. --layers 45 runs 45 of the 50 transformer blocks, reducing both time and unified-memory use. --show is optional. It supports Kitty/Ghostty and iTerm2/WezTerm/Konsole graphical protocols. It loads a resident preview VAE, displays one representative middle-video frame after every Euler transition, and then displays all final frames. Display dimensions default to 2x so the image has its intended logical size on macOS Retina screens; use --zoom 1 on a non-HiDPI display. This adds preview decode time and roughly 10 GiB of temporary model residency; runs without --show are unchanged. --profile is optional and does not select a different generation path. The first process invocation also pays model loading and filesystem-cache costs. Compare performance using repeated runs, and alternate variants when the machines are warming up because this workload is sensitive to thermal throttling. For a very short iteration, request four denoising passes directly: ./h3 --profile \ -d ./MiniMax-H3 \ -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur." \ --width 512 --height 512 --frames 22 \ --steps 4 --layers 50 --reuse 1 \ --show \ -o outputs/fox-four-step.mp4 --steps N always means exactly N denoising passes. Four through seven passes use the same schedule that won the low-budget comparison; increasing from 4 to 7 progressively improves detail and motion. Keep --reuse 1 at such small budgets so every requested pass runs the model. --show displays one preview after each pass. Several tail-heavy schedules were evaluated because most visible cleanup happens late in a long run. They preserved too few early composition updates and produced woven texture, weak motion, or clipped colors. The retained mode uses the released linear base grid with one terminal point. On the 512-square, 22-frame fox test, the selected four-pass result had 0.556 full-video SSIM against a 29-pass reference; an independent surfer test measured 0.547. The four-pass denoise took about 3.5 seconds on M5 Max, versus 26.4 seconds for the reference. 3. Move toward reference quality Change one control at a time when evaluating quality. First restore all layers, then all denoiser evaluations, and finally raise the default 20-pass schedule to the slower 50-pass reference: ./h3 --profile \ -d ./MiniMax-H3 \ -p "A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur, soft footsteps and wind." \ --width 512 --height 512 \ --frames 22 --steps 50 \ --layers 50 --reuse 1 \ -o outputs/fox-close.mp4 The defaults are --steps 20 --layers 50 --reuse 1; keep --steps 50 explicit for this close path. It performs 50 complete 50-block denoiser forwards and is much more expensive than the default, but is the right oracle when a fast mode changes the subject, anatomy, motion, or composition. Numerical pixel identity with MLX is not expected because the random-number and execution engines differ; the depicted content and motion should agree. 4. Choose a speed/quality preset These controls are independent unless noted otherwise: Control Slow reference Default Aggressive Main impact Denoising passes --steps 50 --steps 20 --steps 4..7 The number always names actual denoising passes. Whole denoiser reuse --reuse 1 --reuse 2 --reuse 3 At 20 steps: 20, 11, or 8 fresh DiT evaluations. Active DiT blocks --layers 50 --layers 45 --layers 40 Fewer blocks reduce compute and resident transformer weights. Core residual reuse --core-reuse 1 --core-reuse 4 --core-reuse 6 Refreshes patch/head work every step but runs the expensive core less often. Token reduction off optional --token-reduction Pairs horizontal video tokens inside middle blocks; faster but may change composition. Internal canvas output size 384x384 for 512 square output 320x320 Runs DiT/VAE smaller, then upscales with vImage. On M5, --use-int8-row-fc2 uses one activation scale per FC2 row and a single full-width TensorOps product. It is optional because it is less numerically conservative than grouped int8. It reduced complete denoiser forwards by about 2.6% in reciprocal tests. Matched four-step fox and surfer videos kept the same subjects, setting, and motion (full-video SSIM 0.919 and 0.828). In the interactive session, use !int8-row-fc2 on. --reuse and --core-reuse are mutually exclusive. Layer thinning can be combined with either one. To make the first command faster while keeping its output resolution, add token reduction: ./h3 --profile \ -d ./MiniMax-H3 \ -p "A surfer riding inside a sharp blue ocean wave, one rider and one white board, realistic spray." \ --width 512 --height 512 --frames 22 --steps 20 \ --layers 45 --reuse 2 --token-reduction \ -o outputs/surfer-fast.mp4 At the validated 512 square shape, token reduction cut the 45 layers + reuse 2 denoise profile from 16.69 to 12.60 seconds on the IT M5 Max. Independent fox and surfer renders stayed coherent, but composition can diverge more from the close path. For an aggressive preview, render internally at 320 square and upscale to the requested 512 square output: ./h3 --profile \ -d ./MiniMax-H3 \ -p "A red fox walking through snow, realistic, tracking shot." \ --width 512 --height 512 \ --render-width 320 --render-height 320 \ --frames 22 --steps 20 --layers 40 --reuse 3 \ -o outputs/fox-aggressive.mp4 This combination produced a clean, recognizable 22-frame fox in validation, but loses fine detail and can change framing. Do not add --token-reduction to both --layers 40 and --reuse 3: that tested combination produced color ringing, outlines, and ghosted limbs. As an alternative to whole-velocity reuse, this keeps the timestep-dependent patch and output heads fresh at every transition: ./h3 --profile \ -d ./MiniMax-H3 \ -p "A surfer riding a blue ocean wave." \ --width 512 --height 512 --frames 22 --steps 20 \ --layers 45 --core-reuse 4 \ -o outputs/surfer-core-reuse.mp4 Use --core-reuse 6 only as an aggressive preview. Values above 6 are not exposed because validation lost subject fidelity. 5. Pick resolution and duration Width and height must each be multiples of 32, at least 32, and their product must not exceed 768 * 1344 pixels. Those are mechanical limits, not a promise that every tiny canvas has good model quality. H3-Base is a 768p model. Canvas Current guidance 512x512 Safest development size; repeatedly validated with multiple prompts. 768x768 Validated close-quality square output; substantially more expensive. 1344x768, 768x1344 Released 768p-class landscape/portrait limit. 1024x768, 768x1024 Valid 4:3 and 3:4 768p-class canvases. 384x384 internal to 512x512 Validated fast-quality scaling point. 320x320 internal to 512x512 Validated aggressive scaling point. 256x256 Native fast-preview canvas with automatic low-resolution RoPE adaptation. For a fast native 256-square preview: ./h3 -d ./MiniMax-H3 \ -p "A red fox walks through fresh snow in a pine forest." \ --width 256 --height 256 \ --frames 22 --steps 20 \ --layers 50 --reuse 1 \ -o outputs/fox-256.mp4 At 256 square, H3 has only an 8x8 effective spatial-token grid,