Needle 3 - 8-29 MB foundation model for tiny devices | Cactus

Ask for two things and you get two calls in order; ask for something no tool covers and you get an empty list, not a guess.Structured extractionDeclare a shape, hand over messy text, get typed fields back: an invoice, a booking, a notification, a form. The decode grammar guarantees the output parses. Extraction generalised well to classification problems too.Text embeddingThe same model returns a vector for a sentence, so an app can search, match and route locally: find the note you mean, pick the tool closest to a request, collapse duplicate alerts.What that looks like in a product:Smart homeGo from pressing buttons to talking to the house. "Dim the bedroom and lock up" becomes two calls, executed offline, with no hub round trip.RobotsGive a vacuum or a small robot nuanced instructions: "clean the kitchen but leave the bedroom", "go back to the dock when you are done". Each becomes a sequence of moves it can execute.PhonesAn assistant that acts on the device instead of answering: make an album from last weekend's photos, open a site, dim the screen, find the lease in your files.WearablesRead a notification into structured data on the wrist: a card charge into merchant, amount and date; a message into a reply; a complaint into a sentiment flag.AR glassesNavigation and nearby search from a short request, with no phone or network in the loop.AutomotiveClimate, media, navigation and calls from requests in the cabin, with the tool set pinned so it survives a long drive's worth of conversation.ComputersPlain-English control of the machine in front of you: draft the mail, start the timer, copy the address, open the tab.Search and matchingEmbeddings that never leave the device: semantic search over notes, messages and documents; a query matched to the closest of hundreds of tools; near-duplicate alerts merged on a watch.ModelIntelligence laddering. Every layer of Needle 3 is a sub-network with monotonically increasing capacity. Developers can choose the right size from the 2-layer (2L) subnetwork to 20 layers (20L). Each subnetwork is amenable to fine-tuning, such that 4L can match DeepSeek V4 Flash when tuned on downstream tasks for one epoch. Intelligence laddering produces 9 to 29 MB CQ2-bit binaries and supports a wide range of tiny devices.InputsText prompts, plus tool definitions or an extraction schemaOutputsStructured JSON with tool calls or extractionsModel29-121M Laddered Simple Attention Networks, CQ2 quantisationTraining360B tokens of proprietary structured datasetSpeed400-4k tokens/s decode and 1-10k tokens/s prefill on a Raspberry Pi 5Get startedInstall the Python package. The inference engine is fetched once from Hugging Face and cached; there is nothing else to build.Needle reads your tool descriptions to decide what to call and how to fill arguments, so describing them well is the whole game.Simple: decorate a function. The signature gives the argument types, the docstring is the tool description, and run() completes the loop: the model picks the call, Needle executes your function, feeds the result back, and returns the final response with the executed tool results attached as results.Route by pattern: when a description cannot enumerate every phrasing, give a tool triggers, regular expressions matched against each request. A match restricts the decode to the matched tools and requires a call, so the request reaches the tool you named instead of being refused or misrouted, and the call ships even below the confidence floor. A match restricts the whole turn, so a catch-all should exclude the nouns other tools own, e.g. ^(?![\s\S]*\b(lights?|doors?)\b)[\s\S]*\b(turn|switch)\b[\s\S]*\b(on|off)\b; then "switch the fan on and dim the kitchen lights" still reaches both tools.Extraction: to pull structured data out of text, declare the shape and call extract(). Pass a Pydantic model and you get a typed object back.Every turn returns one JSON object:Confidence gating and routing: every response carries a confidence score from a calibrated head, and the engine already applies a floor of 0.1. Below it, the call is withheld into suppressed_calls and function_calls is empty. Above it, the score is yours to route on: act at once when it is high, show the call and ask when it is middling, and treat an empty result as a refusal. A tool with triggers always produces a call for a matching request, so the score is what tells you whether to run it or confirm it.Writing tools: the model reads a schema literally, so a narrow tool with a plain description beats a broad one. One tool per action, described by the actions it covers ("Turn a room's lights on or off") rather than a category. Name enum options after what a user says (action: ["increase", "decrease"]) and keep synonyms in the description. Give a required argument a default when a request may leave it out; a required argument with no default and no evidence in the request is withheld rather than guessed. Put value formats in descriptions ("City, ST", "e.g. T-1042"). Add triggers to intents that must always reach a tool, and keep the toolset per turn small, since every extra tool is a chance to misroute.Fine-tune: the Python package is the quick path. LoRA on the frozen base at the full 20 layers, then a 4-bit .cact of any subnetwork that runs on the same engine.Cactus PlatformNeedle was designed to be customised. Its capacity is a ladder, and a subnetwork as small as 2 layers, fine-tuned on one product's tools, runs optimally on devices far smaller than the full model needs. Constraining the capacity to a narrow, well-defined task is what lets it reach frontier-level accuracy there: fine-tuning on DroidCall lifts every subnetwork by 18 to 36 points, and from 4 layers up the tuned subnetwork passes DeepSeek V4 Flash, starting at 29M parameters (Figure 3).The Cactus Platform is the full path: Cactus datasets, the 2-bit quantisation behind the shipped model, evaluation design and tracking, full-depth fine-tunes and dataset management, all on our infrastructure and training pipeline, no need to build your own.DeployEvery deployment target ships a prebuilt engine under 1 MB that loads the needle3.cact weights at start.