GitHub - vixhal-baraiya/microgpt-c: The most atomic way to train and inference a GPT in pure, dependency-free C
Pangram verdict · v3.3
We believe that this text is a mix of AI and human-written content.
AI likelihood · overall
MixedArticle text · 255 words · 4 segments analyzed
The most atomic way to train and inference a GPT in pure, dependency-free C. A character-level transformer with forward pass, backprop, Adam and sampling, in one C file with nothing beyond libc.
It trains on ~32k names in a couple of seconds and generates new ones. Build and run make run Or run it directly, on any corpus with one item per line: ./microgpt data/names.txt Builds on macOS, Linux and Windows (MSYS2), on ARM64 with NEON and x86-64 with AVX2. The Makefile picks the flags for the host. step 5000 / 20000 | loss 2.6036 (avg 2.2940) step 10000 / 20000 | loss 1.9639 (avg 2.2564) step 15000 / 20000 | loss 2.7007 (avg 2.2151) step 20000 / 20000 | loss 2.3463 (avg 2.2201) inference sample 1: kayley sample 2: maria sample 3: arana sample 4: shayan sample 5: jayden sample 6: saria sample 7: kaylen sample 8: amari sample 9: alina sample 10: mailyn c fp32+NEON 10168430 tok/sec Notes The model has 4192 parameters and generalises rather than memorises.
Trained on 20000 of the 32033 names, it scores 2.2054 nats per character on those and 2.2039 on the 12033 it never saw, beating an interpolated trigram that has nearly five times as many parameters. Training and inference use separate forward passes. gpt_forward stores activations for backprop; gpt_forward_infer is a specialised single-token path whose logits match it to within fp32 rounding. docs/PERFORMANCE.md covers how that path works and what limits it.
machine backend tok/sec Apple M5 Pro NEON 10,168,430 AMD Ryzen 5 5600H AVX2 6,927,775