GLM-5.2 - How to Run Locally | Unsloth Documentation

⌘CtrlkModelsGLM-5.2 - How to Run LocallyRun the new GLM-5.2 model by Z.ai on local hardware!GLM-5.2 is Z.ai’s new open model, delivering SOTA performance across long-horizon coding, reasoning, and agentic tasks. With 744B parameters, 40B active parameters, and a 1M context window, it can now be run locally using Unsloth Dynamic GGUFs. GLM-5.2 is the strongest open model to date, performing on par with Claude 4.8 Opus, GPT-5.5, and Gemini 3.1 Pro across Artificial Analysis and many other benchmarks.The full model requires 1.51TB of disk space, while Unsloth Dynamic 2-bit GGUF reduces this to 239GB (-84% size) by upcasting important layers to 8 or 16-bit. Dynamic 1-bit lowers further to 217GB (-86%). Thanks Z.ai for giving Unsloth day-zero access. GLM-5.2-GGUFRun GLM-5.2 TutorialsQuantization Results ⚙ Usage GuideThe 2-bit dynamic quant UD-IQ2_M uses 239GB of disk space - this can directly fit on a 256GB unified memory Mac and works well in a 1x24GB GPU and 256GB of RAM with MoE offloading. The 1-bit quant will fit on a 223GB RAM and 8-bit requires 810GB RAM.Table: Inference hardware requirements (units = total memory: RAM + VRAM, or unified memory)223 GB245 GB290-360 GB372-475 GB570 GB810 GBFor best performance, make sure your total available memory, including VRAM and system RAM, exceeds the quantized model file size by a comfortable margin.GLM-5.2 has 3 thinking modes. Non-thinking and Thinking in two modes: High + Max.

Use Max Thinking for complicated tasks. In Unsloth Studio you can easily toggle High + Max Thinking and non-Thinking with a UI.Use these settings for most use cases:temperature = 1.0temperature = 1.0top_p = 0.95top_p = 1.0Maximum context window: 1,048,576.GLM 5.2 uses thinking mode by default. And supports reasoning_effort as "high", "max" or disabled thinking. To disable thinking, use --chat-template-kwargs '{"enable_thinking":false}'If you're on Windows Powershell, use: --chat-template-kwargs "{\"enable_thinking\":false}"Use 'true' and 'false' interchangeably.You can also use --reasoning on or --reasoning off in llama.cpp as well now!We also ran KLD (KL Divergence) to gauge the accuracy of our quantizations of GLM-5.2-GGUF. In general, dynamic 4-bit UD-Q4_K_XL and dynamic 5-bit UD-Q5_K_XL are generally lossless, and smaller quants also work great!On pure top-1% accuracy, dynamic 1-bit gets around 76.2% accuracy yet being 86% smaller! Dynamic 2-bit gets around 82% accuracy whilst being 84% smaller.99.9% KLD is also generally good - there is a larger uplift from 4bit onwards though, so for massive out of distribution tasks, dynamic 4-bit is probably best.The mean KLD generally follows a clear monotonic trend vs disk space, and shows even at 1-bit GLM 5.2 works well!You can now run GLM-5.2 in llama.cpp and Unsloth Studio. We will be utilizing the 239GB UD-IQ2_M quant for best results in terms of accessbility and accuracy.GLM-5.2 can run in Unsloth Studio, an open-source web UI for local AI. Unsloth Studio automatically offloads to RAM and detects multiGPU setups.

With Unsloth Studio, you can run models locally on MacOS, Windows, Linux and:Search, download, run GGUFs and safetensor modelsFast CPU + GPU inference via llama.cppInstall and Launch UnslothTo install, run in your terminal:MacOS, Linux, WSL:Windows PowerShell:Launch UnslothMacOS, Linux, WSL and Windows:Then open http://127.0.0.1:8888 (or your specific URL) in your browser.Launch Unsloth securely with HTTPS and CloudflareNEW! Unsloth now provides a secure way to launch Studio over HTTPS through a free Cloudflare tunnel. Use the below (works in Windows, Mac & Linux):Search and download GLM-5.2Unsloth Studio automatically offloads to RAM and detects multiGPU setups. On first launch you will need to create a password to secure your account and sign in again later.Then go to the Studio Chat tab and search for GLM-5.2 in the search bar and download your desired model and quant. Ensure you have enough compute the run the model.Run GLM-5.2Inference parameters should be auto-set when using Unsloth Studio, however you can still change it manually. You can also edit the context length, chat template and other settings.For more information, you can view our Unsloth Studio inference guide.Example of 2-bit GLM-5.2 running with tool-calling and SVG gen.For this guide we'll be running the UD-IQ2_M quant which will require at least 245GB RAM. Feel free to change quantization type. For these tutorials, we will using llama.cpp for fast local inference. GGUF: GLM-5.2-GGUF Obtain the latest llama.cpp on GitHub here. You can follow the build instructions below as well. Change -DGGML_CUDA=ON to -DGGML_CUDA=OFF if you don't have a GPU or just want CPU inference. For Apple Mac / Metal devices, set -DGGML_CUDA=OFF then continue as usual - Metal support is on by default.You can now use llama.cpp directly to load and download models, just like ollama run.

First, select the quantization type you want like UD-IQ2_M. Also use export LLAMA_CACHE="unsloth/GLM-5.2-GGUF" to force llama.cpp to save to a specific location. Note this download process might be very slow, so it's probably best to use the manual download process in the next section.If you want to download the model manually (much faster!), we can download the model via the code below (after installing pip install huggingface_hub). If downloads get stuck, see: Hugging Face Hub, XET debuggingIf you want to use the dynamic 1bit, then do:Then run the model in conversation mode. Use unsloth/GLM-5.2-GGUF/UD-IQ2_M/GLM-5.2-UD-IQ2_M-00001-of-00006.gguf for 2bit or unsloth/GLM-5.2-GGUF/UD-IQ1_S/GLM-5.2-UD-IQ1_S-00001-of-00006.gguf for 1bit.When you launch llama-cli, you will see:Then after prompting it to make a short Flappy Bird game, we get:With the full conversation and game below:Full game in HTMLFull conversationAnd the game has sound and works wonderfully! Reminder this was a 1-bit quantization and it worked well!📐Long context via KV Cache quantizationTo utilize long context in llama.cpp, we need to employ KV cache quantization to reduce memory usage. Recently llama.cpp added higher accuracy tricks to KV cache quantization - see and other PRs!Currently, these KV cache dtypes are supported:By default f16 is used. If you use q4_0 which is around 4.5 bits per weight, you can extend around 16 / 4.5 = 3.5x longer context lengths! So if you model used to support 10K, 35K can be in reach! q4_1 is probably better since you also get a shifting parameter, and is 5 bits per weight - so 3.2x longer contexts.

Use it like below:You can view further below for GLM-5.2 benchmarks in table format:ReasoningHLE40.549.8*41.4*453141.43737.7HLE (w/ Tools)54.757.9*52.2*51.4*52.353.5-48.2CritPt20.920.927.117.74.613.43.712.9AIME 202699.295.798.398.295.397-94.6HMMT Nov. 202594.496.596.594.8949584.494.4HMMT Feb. 202692.596.796.787.382.697.184.495.2IMOAnswerBench91.083.5-8183.890-89.8GPQA-Diamond91.293.693.694.386.2909390.1CodingSWE-bench Pro62.169.258.654.258.460.65955.4NL2Repo48.969.750.733.442.747.242.135.5DeepSWE46.25870101818208ProgramBench63.771.970.839.550.9--47.8Terminal Bench 2.1 (Terminus-2)81.085847463.5756564Terminal Bench 2.1 (Best Reported

Harness)82.778.983.470.769---FrontierSWE (Dominance)74.475.172.639.630.5--29.0PostTrainBench34.337.228.421.620.1---SWE-Marathon13.026.012.04.01.0---AgenticMCP-Atlas (Public Set)76.877.875.369.271.876.474.273.6Tool-Decathlon48.259.955.648.840.7--52.8Last updated 6 hours agoWas this helpful?Recommended Settings📈 Quantization analysisRun GLM-5.2 Tutorials:🦥 Run GLM-5.2 in Unsloth Studio🦙 Run GLM-5.2 in llama.cpp📐Long context via KV Cache quantization📊 BenchmarksWas this helpful?curl -fsSL https://unsloth.ai/install.sh | shirm https://unsloth.ai/install.ps1 | iexunsloth studio -H 0.0.0.0 -p 8888unsloth studio --secureapt-get update apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y git clone https://github.com/ggml-org/llama.cpp cmake llama.cpp -B llama.cpp/build \ -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split cp llama.cpp/build/bin/llama-* llama.cppexport LLAMA_CACHE="unsloth/GLM-5.2-GGUF" ./llama.cpp/llama-cli \ -hf unsloth/GLM-5.2-GGUF:UD-IQ2_M \ --temp 1.0 \ --top-p 0.95 \ --min-p 0.01hf

download unsloth/GLM-5.2-GGUF \ --local-dir unsloth/GLM-5.2-GGUF \ --include "*UD-IQ2_M*" # Use "*UD-Q8_K_XL*" for near full precisionhf download unsloth/GLM-5.2-GGUF \ --local-dir unsloth/GLM-5.2-GGUF \ --include "*UD-IQ1_S*"./llama.cpp/llama-cli \ --model unsloth/GLM-5.2-GGUF/UD-IQ2_M/GLM-5.2-UD-IQ2_M-00001-of-00006.gguf \ --temp 1.0 \ --top-p 0.95 \ --min-p 0.01<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no"> <title>Sunset Flier</title> <link href="https://fonts.googleapis.com/css2?family=Press+Start+2P&family=Fred:wght@400;700&display=swap" rel="stylesheet"> <style> :root { --sunset-1: #ff6b6b; --sunset-2: #feca50; --sunset-3: #ff9ff3; --dusk: #36306b; --night: #1a1746; --accent: #ffd93b; --coral: #ff6b6b; --pipe: #4a902b; --pipe-dark: #2d5a1a; } * { margin: 0; padding: 0; box-sizing: border-box; } html, body { height: 100%; width: 100%; overflow: hidden; background: var(--night); font-family: 'Fred', sans-serif; -webkit-user-select: none; user-select: none; touch-action: manipulation; }

#game-wrap { position: relative; width: 100vw; height: 100vh; display: flex; justify-content: center; align-items: center; background: linear-gradient(180deg, #1a1746 0%, #36306b 40%, #ff6b6b 70%, #feca50 100%); } #game-frame { position: relative; width: min(100vw, 480px); height: min(100vh, 720px); max-height: 100vh; box-shadow: 0 30px 80px rgba(0,0,0,0.6), inset 0 0 0 1px rgba(255,255,255,0.05); overflow: hidden; background: linear-gradient(180deg, #4a3a8e 0%, #ff6b6b 60%, #feca50 100%); } canvas { position: absolute; inset: 0; width: 100%; height: 100%; display: block; } .overlay { position: absolute; inset: 0; display: flex; flex-direction: column; justify-content: center; align-items: center; pointer-events: none; z-index: 10; transition: opacity 0.3s ease; } .overlay.hidden { opacity: 0; pointer-events: none; } .overlay.visible { opacity: 1; pointer-events: auto; } .panel { background: rgba(26, 23, 70, 0.85); border: 3px solid var(--accent); border-radius: 16px; padding: 28px 36px; text-align: center; color: #fff; box-shadow: 0 12px 0 rgba(0,0,0,0.3), 0 0 40px rgba(255, 217, 59, 0.4); backdrop-filter: