GitHub - itsmeadarsh2008/opencode-senses: The vision plugin for OpenCode that truly understands images. Inspect, read, and reason about any screenshot or picture with deeper understanding than any other plugin — fully local, private, and free.

Vision for text-only OpenCode models — fully local, private, free, and slightly smug about it. Senses adds a vision layer to OpenCode so any image becomes useful to your text-only coding model: screenshots get exact OCR, objects get located, colors get measured, and everything comes back as structured evidence the model can reason over. No API keys. No hidden cost. No pictures of your desktop leaving the machine — we promise, scouts' honor, we checked the source. The text model reasons. Senses perceives, grounds, verifies, and fetches snacks. Table of contents ✨ Features ⚙ How it works 🧰 Requirements 📦 Installation 🚀 Quick start ✍ Usage 🔧 The tools 🔁 Workflows 🎛 Configuration 🔒 Privacy 🩹 Troubleshooting 🛠 Development More 📜 License Open source stays alive on donations, not vibes. Every coffee funds the next vision feature. Features Sight for text-only models — attach any image and the model sees it: a structured scene read, a caption, and exact OCR are auto-injected into your message before the model responds. 13 grounded tools — inspect, OCR, detect, point, segment, crop, zoom, colors, diff, annotate, metadata, reverse search, status. All return normalized, source-grounded evidence, the way the vision gods intended. Web images supported everywhere — any tool accepts an https:// URL as a path; the image is downloaded verbatim (original type and bytes preserved) and cached locally. Recovery-grade analysis — senses_zoom upscales regions and re-reads small text the model misses at full-image scale; senses_colors gives deterministic pixel ground truth the model can't hallucinate, even if it wanted to. Reverse image search without API keys — perceptual-hash search across your local files, plus Yandex upload search (Google Lens optional, its majesty commands it). Prompt-injection hardened — everything the model reads from an image is wrapped in an explicit untrusted data guard. A screenshot screaming "ignore previous instructions" is treated as evidence, not a career move. Zero-install runtime — auto-provisions its own Python venv + Moondream weights on first use. It even picks up its own room. Free forever — Moondream 2 runs on your own GPU. 6 GB VRAM is plenty; that's roughly the size of a good sandwich, and it will not be sharing. How it works %%{init: {"theme": "base", "themeVariables": {"fontFamily": "'JetBrains Mono', 'Fira Code', 'Cascadia Mono', Menlo, Consolas, monospace", "fontSize": "13px", "primaryColor": "#111111", "primaryTextColor": "#ffffff", "primaryBorderColor": "#333333", "lineColor": "#6b6b6b", "textColor": "#e5e5e5", "edgeLabelBackground": "#111111", "clusterBkg": "#111111", "clusterBorder": "#333333"}}}%% flowchart LR A["OpenCode<br/>(text-only model)"] B["OpenCode Senses Plugin<br/>(TypeScript, in-session)"] C["Python Runtime<br/>(python/runtime.py)"] D["Structured evidence<br/>(OCR, layout, bboxes, colors)"] A -->|"attaches an image"| B B -->|"auto-inject evidence"| A B <-->|"stdio JSON-RPC"| C C -->|"Moondream 2 via Photon/kestrel<br/>(local GPU inference)"| D D -.->|"senses_* tools &mdash; inspect, ocr, detect, point, segment,<br/>crop, zoom, colors, diff, annotate, metadata, reverse, status"| B classDef opencode fill:#3f3f46,stroke:#52525b,stroke-width:2px,color:#ffffff classDef plugin fill:#ef4444,stroke:#7f1d1d,stroke-width:2px,color:#ffffff classDef runtime fill:#3b82f6,stroke:#1e40af,stroke-width:2px,color:#ffffff classDef evidence fill:#22c55e,stroke:#14532d,stroke-width:2px,color:#ffffff class A opencode class B plugin class C runtime class D evidence Loading The plugin (TypeScript, runs inside the OpenCode session) spawns the Python runtime, exposes the senses_* tools, and auto-inspects images the moment they are attached. It is punctual because it respects you. The runtime (python/runtime.py) is a line-delimited JSON-RPC server over stdio. It owns the model lifecycle: lazy load, warm cache, explicit unload. It knows when to call it a day. Vision model: Moondream 2 by default — fits a 6 GB GPU comfortably (peaks at ~4.5 GB). Moondream 3.1 (9B) is supported for larger cards; check your wallet before enabling. Requirements OS: Linux x86_64/aarch64, Windows AMD64, or macOS. GPU: an NVIDIA GPU (Ampere or newer) or Apple Silicon (M-series). 6 GB VRAM is enough for the default model. Python 3.10–3.14 — or uv, which Senses uses to bootstrap everything (including a managed Python) automatically. macOS/Windows ship their own runtime support. Bun only needed to build from source — npm users don't need it. No API keys. None. Ever. (The only exceptions are optional finetunes, and the one key to your heart, which is not required for setup.)