Teaching Automatic1111 to Speak Metal on an M1

I use Draw Things a lot on Apple hardware, and one thing has always bothered me about Automatic1111: it feels slower than it should.Not unusably slow. Just slow enough that you notice it.On my M3 Pro, a short five-step DPM++ SDE generation in Automatic1111 was typically landing somewhere around 8–10 seconds. Draw Things had already shown me that Stable Diffusion on Apple Silicon could feel much more immediate than that.So I wanted to see how much of that gap was actually necessary.GitHub - dmikey/stable-diffusion-webui-metal: a fine tuned automatic1111 for Apple Silicon.a fine tuned automatic1111 for Apple Silicon. Contribute to dmikey/stable-diffusion-webui-metal development by creating an account on GitHub.GitHubdmikeyThere was one important constraint: I did not want to replace Automatic1111.I wanted the same WebUI, checkpoints, LoRAs, samplers, extensions, API, prompt syntax, and general workflow. I wasn't interested in converting everything to Core ML and building another inference engine around it. The goal was much narrower:How fast can Automatic1111 get if we make the parts that matter behave more like native Apple software?The answer, at least for the workloads I'm running, is quite a bit faster.The same class of generation that was taking roughly 8–10 seconds on my M3 Pro is now generally landing between 3 and 7 seconds. And 13-20, now landing 8-10 on my M1 Mac Mini.Those are observed ranges across my current workloads, not a controlled benchmark claiming a universal 2x improvement. There is also an important distinction between the runtime improvements and NGMS, which actually reduces the amount of guidance work being performed.Still, the difference in actual use is substantial.More interesting than the final number, though, was what it took to get there.It wasn't one optimization.Start with the workload, not the benchmarkThe workload I cared about was pretty specific:Stable Diffusion 1.xDPM++ SDEKarras5 stepsCFG around 1.15384×640 and 512×512FP16 UNet on MPSFP32 VAE by defaultThat specificity matters.Early on, DPM++ 2M looked like an easy way to shave off time. It was faster, but it didn't produce the result I wanted from the short schedule.That isn't an optimization. It's a different workload.This became the rule for basically everything that followed: if an optimization looks great in isolation but doesn't make the actual generation faster while preserving the result I'm trying to produce, it doesn't count.Attention was the obvious place to start.PyTorch's MPS backend has gotten substantially better, but there are still Stable Diffusion attention shapes where going directly to Metal makes sense.The mistake would have been treating a custom Metal implementation as universally faster.It isn't.Instead, I added a Metal Flash Attention path specifically for the SD 1.x shapes where it actually won in testing.The router looks conceptually like this:if inference and fp16_mps and query_tokens >= 192 and head_dim in (40, 80, 160): return metal_flash_attention(q, k, v) return pytorch_sdpa(q, k, v) There are additional checks around masks, training, dropout, tensor layout, grouped-query attention, and supported types, but that's the basic idea.Metal is not the default because Metal sounds faster. It gets the operation when we've measured that shape and it deserves it.Everything else goes back through PyTorch.That fallback is important. Automatic1111 supports far more configurations than my five-step SD 1.x workflow. I didn't want a faster fork that only worked if nobody touched anything.The kernel wasn't the whole problemGetting attention into Metal helped, but it exposed something more interesting.The native extension was committing the MPS command buffer after every attention call.Stable Diffusion calls attention over and over inside every UNet evaluation. With a short five-step generation, repeatedly submitting tiny chunks of work starts becoming a meaningful part of the total runtime.So instead of treating the Metal kernel like its own little application, I integrated it into PyTorch's current MPS stream.The extension ends PyTorch's current kernel coalescing, encodes the Metal Flash Attention operation into the current command buffer, and then lets the rest of the PyTorch MPS work continue from there.The explicit commit after every attention call went away.This ended up being one of the more important lessons from the entire project.The fastest kernel still loses if you submit the command buffer after every call.At these generation times, overhead matters. You're no longer just optimizing how quickly the GPU can multiply matrices. You're optimizing how often Python, PyTorch, MPSGraph, and Metal have to coordinate with each other.There was also a wonderfully obvious reminder not to trust the timer: one of the early versions produced a green image.It was fast.It was also green.The Metal path now runs an isolated attention-plus-projection correctness test before the WebUI enables it.Unified memory changes the rulesThe next problem was memory.Apple Silicon doesn't have a discrete pile of VRAM sitting next to system RAM. The GPU and the rest of the machine are competing for the same physical memory.That makes some traditional GPU assumptions fairly bad ones.An attention matrix can technically fit in memory and still be a terrible idea if macOS is under pressure, the allocator starts thrashing, or the machine begins swapping.So instead of using a fixed VRAM threshold, the fork estimates the cost of native attention against both total and currently available memory.Conceptually:attention_bytes = batch × heads × query_tokens × key_tokens × element_size estimated_peak = attention_bytes × 2.5 budget = min( 10% of total memory, 20% of currently available memory, 1.5 GiB ) If the estimated peak fits inside that budget, native SDPA can run.If it doesn't, the request goes through the memory-bounded sub-quadratic path instead.The chunk size for that fallback is dynamic too. An 8 GB Mac shouldn't make the same decision as a 32 GB Mac, and neither should behave as though Chrome, Xcode, or whatever else is running doesn't exist.I don't count this as a blanket speed improvement. It's mostly about keeping performance predictable and avoiding the cases where an ostensibly fast operation causes enough memory pressure to make the whole generation slower.Stop keeping every attention chunk aroundI also changed how the sub-quadratic fallback handles K/V chunks.The existing approach computes partial attention results, keeps the numerator, normalization weight, and maximum for each chunk, then stacks everything together at the end.That's unnecessary.Instead, the fork maintains a running maximum, normalization sum, and weighted output. Each new K/V chunk gets merged into that running state and can then be discarded.The recurrence is basically:new_max = max(running_max, chunk_max) running_scale = exp(running_max - new_max) chunk_scale = exp(chunk_max - new_max) running_values = running_values × running_scale + chunk_values × chunk_scale running_weights = running_weights × running_scale + chunk_weights × chunk_scale This is the same general online-softmax idea that makes Flash Attention memory efficient.Memory now scales around the current chunk instead of accumulating every partial result until the end.I tested the forward results against PyTorch SDPA and also tested gradients in float64. Again, the goal wasn't just to make something clever. It had to be a safe fallback.Some MPS workarounds have outlived the bugsThere was another category of optimization that was much less glamorous: deleting old workarounds.Apple's PyTorch backend has changed a lot.Automatic1111 accumulated defensive behavior for older MPS implementations, including cloning torch.narrow() results and pushing LayerNorm through FP32.Those fixes made sense when the underlying MPS bugs existed. On newer versions of PyTorch, they can just become copies, allocations, conversions, and memory traffic.So those behaviors are now gated by runtime version rather than applied indiscriminately.There's still an A1111_MPS_FORCE_LEGACY_OPS=1 escape hatch if somebody needs the old behavior.I also enabled PYTORCH_MPS_PREFER_METAL=1 because direct Metal matrix multiplication tested better for the SD 1.x projection sizes I was targeting, and removed the default sampling upcast so more of the short sampling path stays in FP16.That last change is a real tradeoff. FP16 reduction order and removing the upcast can affect same-seed output.I'm fine with that for this workflow, but it shouldn't be presented as free performance.Fusing GroupNorm and SiLUOnce the unnecessary work was reduced, I went looking for operations that were both necessary and repeated constantly.GroupNorm followed by SiLU is everywhere in the SD 1.x UNet.Normally those are separate PyTorch operations. That means separate dispatches and an intermediate activation that gets written out and then immediately read back.So I wrote a fused Metal kernel.For compatible FP16 inference tensors, one 256-thread Metal threadgroup handles each batch/group pair. The kernel accumulates the sum and squared sum in FP32, reduces those into mean and variance, applies normalization and the affine parameters, applies SiLU, and writes the FP16 result.One dispatch. No intermediate activation.If the tensor isn't compatible, we're training, gradients are enabled, the dtype is wrong, or the native path fails, it goes straight back to:F.silu(norm(input_tensor))