How Muse Glimmer Fits an Agent on Your Device

Meta pitches Muse Glimmer as an agent that runs on your device: autonomous, multimodal, no cloud required. That is an engineering problem as much as a product claim: fit a capable 30B-class model, a long working history, and a perception stack into consumer hardware. The answer turns out to be a memory hierarchy disguised as a 30B Transformer. Its model card is direct about the goal: Muse Glimmer is “purpose-built for autonomous agentic tasks on consumer hardware,” and it runs “without requiring cloud infrastructure or network access.” The promise is demanding because an agent's workload is long-lived. Hours of history and tool transcripts stay resident, screenshots and documents get reread mid-task, and all of it has to fit inside the 24 or 32 GB envelopes Meta names for its quantized releases. Muse Glimmer is a roughly 30-billion-parameter, decoder-only multimodal model: a vision encoder, a projector, and a dense language model. In BF16 the checkpoint weighs about 55 GiB, which would overflow both of those envelopes before a single token of context, so part of the answer is easy to name: Meta ships roughly four-bit quantized variants that bring the language model below 20 GB. The compressed model still has to share the card with a 131,072-token context, a resident vision tower, and a speculative-decoding drafter, and none of them get smaller when the language model does. The rest of the answer is architectural: where the model spends memory, and what kind of information each layer carries. Muse Glimmer is built around a deliberate division of labor. In most layers, attention is local: positioned by RoPE and bounded to a 2,048-token window. In every fourth layer, attention opens to the entire context but drops RoPE, retrieving primarily by content. Only the attention alternates; the rest of every block is identical. Thirty-two query heads provide a rich set of retrieval behaviors, while only two key/value heads are stored in the KV cache. On the visual side, a large ViT performs expensive perception once, compresses neighboring patches four-to-one, and hands the result to the language decoder as ordinary tokens. Taken together, the parts form a hierarchical memory system: local layers construct ordered, context-rich representations;global layers search those representations over the full sequence;the KV cache stores a very narrow memory trace for each active sequence. Per-sequence state is tiny by design, so nearly all the memory a running instance needs is the model's parameters. That is why weight quantization pays off so unusually well here. Once those fixed weights are compressed, the freed memory can be turned into longer contexts, larger batches, a resident perception tower, or a speculative-decoding drafter. Where the 55 GiB sits Here is the breakdown, summed from the released tensor shapes: ComponentApproximate parametersBF16 storage52 text Transformer blocks25.165B46.87 GiBInput token embedding1.345B2.50 GiBUntied language-model head1.345B2.50 GiBVision tower1.853B3.45 GiBVision-to-text bridge69.2M0.13 GiBTotal29.777B55.46 GiB Because Muse Glimmer is dense, every generated token passes through all 52 text blocks. There are no routed experts waiting unused in memory. This gives predictable execution, but at low batch sizes it also makes decoding heavily dependent on repeatedly reading a very large set of weights. Every fourth layer sees everything The 52 text layers follow a strict repeating schedule: There are therefore 39 sliding-attention layers and 13 full-attention layers. The local window is 2,048 tokens. A local layer at position can directly read only the recent interval ending at . But local receptive fields compound with depth. Ignoring boundary effects, three stacked causal windows expose a token indirectly to roughly 1 + 3 × (2048 − 1) = 6,142 positions: 6,141 predecessors plus the token itself. The global layer that follows does not receive raw isolated tokens; it receives representations that already summarize several thousand tokens of ordered local structure. One way to read the four-layer cycle: the first local layer builds immediate lexical and syntactic relationships, the next two combine them into progressively larger local structures, and the closing full layer retrieves relevant summaries from anywhere in the context. The division is soft, of course. Local layers carry global information forward in their residual streams, and global layers can attend locally. Still, the masks impose a strong prior: most computation refines nearby structure, while occasional layers handle long-range communication. This is much cheaper than making all 52 layers global, especially for the KV cache. Long-context compute is another matter: during prefill, the 13 full-attention layers still do quadratic attention work in the sequence length. FlashAttention-like kernels avoid materializing the full attention matrix, but they do not erase the dot products. Muse Glimmer makes 131K context memory-feasible; it does not make a 131K prefill equivalent to a 4K prefill. Global attention without RoPE First, a reminder of what the full layers are giving up: RoPE rotates each query and key by an angle that grows with its token index, and because the two rotations compose when the vectors are scored against each other, the attention logit ends up depending only on the relative displacement . That composition is how a Transformer normally feels distance. Muse Glimmer uses RoPE with in every local layer. In each full-attention layer, however, the per-layer RoPE theta is zero, and the implementation passes no positional embedding into attention - the configuration called NoPE. The full layers therefore score Q–K compatibility without a direct rotary position term. Order still reaches these layers through two doors. First, the causal mask tells position that it may read only positions at or before . Second, every global key and value has already passed through three RoPE-equipped local layers. A vector representing the word “bank” near “river” is different from one representing “bank” near “loan,” and both vectors encode the local order that produced them. Later global layers also receive residual states that have been modified by earlier global layers. So the precise statement is that global layers have no direct positional term in their Q–K score, but they operate on position-aware, locally contextualized representations. Why might this help at 131K tokens? A conventional global RoPE layer must interpret relative rotations over distances from one token to more than one hundred thousand. Long-distance retrieval can become entangled with phase behavior far outside the distances that dominate ordinary language. A NoPE global layer instead behaves more like a content-addressable memory: a relevant item does not become intrinsically harder to match merely because it is 80,000 tokens away. The architecture places exact ordering where it is most valuable, inside bounded local windows, and asks the global layers a different question: not how nearby pieces are arranged, but which contextualized piece anywhere in memory answers the current need. There is a trade-off. Two genuinely similar distant occurrences are harder to distinguish by absolute position when global scoring has no explicit positional term. Muse Glimmer mitigates that by making each occurrence carry its surrounding local context, but the ambiguity cannot disappear completely; the design favors robust semantic retrieval over precise global coordinate matching. Thirty-two queries, two memories Muse Glimmer uses grouped-query attention with 32 query heads and only two key/value heads, so sixteen query heads share each K/V bank. The asymmetry lines up with what generation actually pays for. Queries exist only for the token currently being produced and are discarded immediately, so keeping 32 distinct ways of asking costs compute but no standing memory. Keys and values are different: every token within a layer's attention span must stay resident, the full history in a global layer and the last 2,048 tokens in a local one. Cutting KV heads to two attacks the only per-token state that persists, which is why the choice has such a large memory effect. The memory arithmetic follows directly. In BF16 or FP16, each token in each layer stores one key vector and one value vector per KV head, at two bytes per number: So Muse Glimmer uses 1 KiB of KV cache per token per layer, counting K and V together. For a sequence of length , the 13 global layers hold all tokens while the 39 local layers cap out at their 2,048-token window, so the theoretical active KV cache is approximately At the configured 131,072-token context length: the 13 global layers consume about 1.625 GiB;the 39 local layers consume about 78 MiB once their windows are full;total active BF16 KV is about 1.70 GiB per sequence. This calculation assumes the serving engine actually evicts or circularly reuses cache entries for sliding layers. A static implementation that allocates full-length storage for every layer will not realize the full benefit. Cache quantization, page sizes, fragmentation, and runtime workspaces also change real measurements. The architectural point survives all of those caveats: per-sequence memory is deliberately narrow, which shifts the bottleneck toward the fixed model weights. Inside one decoder block So far the story has been about how layers see the context and what each sequence costs. One level down, each block reads and writes a residual stream of width 6,656, adding two updates per token: one from gated grouped-query attention, one from a SwiGLU feed-forward network. Attention uses five projections: the usual query, key, and value maps, an output map that returns the attention result to the residual stream, and a fifth matrix that computes a gate over that result (described below). Their widths: