Rotary GPU: Exploring Local Execution Paths for Large Mixture-of-Experts Models Under Limited GPU Memory

View PDF HTML (experimental) Abstract:Large language models have achieved remarkable capabilities through scaling, and this paper does not challenge that. It instead investigates a different question: once large models already exist, can they become more accessible to environments with substantially smaller hardware resources? The motivation came from deployment concerns rather than architecture research. Many organizations operate under hardware, budget, security, or closed-network constraints that limit access to large accelerator clusters, and as models continue to improve, deployment accessibility may matter as much as capability itself. This paper presents Rotary GPU, an exploratory execution approach derived from a previously disclosed rotary-based accelerator residency concept. A public validation was conducted using a Qwen3.6-35B-A3B-class Mixture-of-Experts model executed locally on a consumer laptop with an RTX 4060 Laptop GPU containing 8 GB of VRAM. Under the primary configuration, the system generated 2048 output tokens while maintaining approximately 6.3 GB of VRAM usage and an observed decode throughput of 21.06 tokens per second. The goal is not to replace data-center infrastructure but to explore whether some capabilities of large models can be brought closer to environments where such infrastructure is unavailable. The results should be read as exploratory rather than definitive, but they suggest deployment accessibility deserves continued investigation as these models evolve.

Comments: 10 pages, 3 figures. Also archived at Zenodo (DOI: https://doi.org/10.5281/zenodo.20406471). Related to Korean Patent Publication KR 10-2026-0070380

Subjects: Performance (cs.PF); Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC) ACM classes: C.1.4; I.2.7

Cite as: arXiv:2605.29135 [cs.PF]   (or arXiv:2605.29135v1 [cs.

PF] for this version)   https://doi.org/10.48550/arXiv.2605.29135 arXiv-issued DOI via DataCite (pending registration) Related DOI: https://doi.org/10.5281/zenodo.20406471

DOI(s) linking to related resources

Submission history From: Myeong Jun Jo [view email] [v1] Wed, 27 May 2026 21:57:36 UTC (12 KB)