DOOM in the kernel, or fibers in eBPF

The kernel's JIT compiles the bytecode to machine code, which runs whenever the hook fires. But before the program can run, the verifier must accept it. That is where the constraints above come from: code the verifier cannot prove safe is rejected. This check, rather than the bytecode itself, is what makes DOOM inside eBPF look impossible.The project is called BPF Capsule. It is a compiler and runtime for large C programs inside ordinary BPF, with no kernel patches and no separate virtual machine in userspace. The oldest supported target profile is Linux 5.15. A profile determines which kernel capabilities the compiler may use. I have loaded and run the programs on both x86-64 and arm64.Nobody needs games in the kernel, of course. But complex application logic is useful there: parsing packets, for example, or keeping statistics about them. When such a program does not fit eBPF's constraints, it has to be simplified and rewritten by hand until the verifier is satisfied. Capsule explores another path: it takes C, C++, or no_std Rust code and transforms it into a shape that stock Linux accepts.DOOM is not the application here but a stress test for that approach. Lua, QuickJS, SQLite, zlib, wasm3, llama2.c, no_std Rust, and CPython 3.14 run on the same scheme today, and Lua and Python inspect live packets straight from XDP. This article follows the road from a hand-trimmed port through a slow interpreter to regions and fibers—and measures what they cost at run time.You can try it with one command on any supported kernel.

Then I cut out everything the kernel disliked, lied to it about pointers, and forced loops into one special form. When even that stopped scaling, I wrote a virtual machine inside eBPF. The current machine of regions, fibers, and a software stack grew out of it.These approaches broke one after another, and every failure suggested what had to be built next.

It sounds simple: write C, run clang -target bpf, and get an object the kernel can load.In practice, “write C” means writing in two rather different languages at once. LLVM understands one. The Linux verifier understands the other.I became intimately familiar with that boundary while working on Perforator. That is where I accumulated enough frustration with the current BPF stack to go this far.Before loading a program, the verifier symbolically executes it. For every register it tracks not only a value or range, but a meaning: an ordinary number (SCALAR_VALUE), a pointer to the stack, packet data, a map value, or a bpf_arena. It explores branches, merges states, and proves two things: every memory access is allowed, and every execution path eventually terminates.That creates constraints an ordinary program barely notices:r1 through r5 are all the argument registers in the classic ABI;the call graph must be acyclic, and call depth is limited;only 512 bytes of stack are available along a call chain;one loaded program may contain no more than 256 BPF functions;after processing roughly a million instructions, the verifier gives up.That last limit is not an execution-time limit. Even a short loop can exhaust the budget if the analyzer must revisit it with enough distinct states. A finite loop is legal in itself; the problem starts when the kernel cannot prove its bound or has to enumerate too many possibilities.Memory is more entertaining still. To the CPU, a pointer is ultimately just a number. To the verifier, it is a number with a biography. It may know that r10 - 8 points into a valid BPF stack slot, or that data + n remains within a packet after a check against data_end. Store that pointer as ordinary 64 bits in a map and load it back, and the CPU gets the same address while the verifier gets a number with no right to be dereferenced.Normal C programs constantly put pointers in structures, pass those structures through several functions, and load the pointers much later. Somewhere along that route, the verifier loses the proof.A recent LLVM exampleWriting a valid bounds check in C is not enough: the kernel sees the code after optimization. Here is a real fragment of packet-processing BPF code:size_t at = offset + index; asm volatile("" : "+r"(at)); at &= PACKET_CAPACITY - 1; if (data + at + 1 > data_end) return -1; byte = data[at];The mask bounds at, and the following comparison proves the packet boundary. Late in the pipeline, however, LLVM can express the index again in terms of the original offset and index loaded from a resumable loop frame. Both forms mean the same thing to the CPU. In one form, the old verifier in the supported Linux 5.15 profile sees a bounded index next to the access; in the other, it loses the proof it needs. The empty inline assembly is not needed by the CPU and emits no BPF instruction. It exists to make LLVM preserve the exact data dependency the kernel understands.This is the unpleasant third language between C and the machine: sometimes a program must not only be safe, but carry its safety proof through the optimizer in a recognizable shape.First, produce any BPF at allBefore involving the kernel, there is an intermediate step: compile DOOM to BPF and run the object in a userspace virtual machine. With no verifier, code generation bugs can be separated from failures to prove safety.I based the experiment on PureDOOM, a port that packages the whole engine into one C header and exposes a short embedding interface. It is convenient for an experiment like this while leaving DOOM itself almost entirely ordinary C.Even without the verifier, arbitrary C does not become BPF by itself. The classic ABI has nowhere to put a sixth argument, and BPF has neither floating-point operations nor indirect calls. Large structure returns, variable-size memcpy, and some 128-bit arithmetic need lowering as well. I also had to extend uBPF's program counter and add missing instructions, sections, and ELF relocations.BPF globals do not become ordinary process memory: .data and .bss become map values, while ELF relocations tell the loader which map and which offset each address in the code refers to.After those changes, DOOM ran in uBPF. The stack was not a fundamental obstacle there: the VM's frame size and total memory reserve can simply be increased. This proved LLVM could emit working BPF code, but said nothing about whether a real kernel would accept it.DOOM had been run in a userspace BPF machine before. One example is Flying the nest — a BPF port of Doom, which used its own νBPF VM. Inside a VM, you can change the machine's rules. My goal was different: an object accepted by the ordinary Linux verifier and executed by the ordinary in-kernel BPF JIT.Porting with scissorsThe next step was loading the program into a real kernel. The freedom of uBPF ended there: I could not increase the 512-byte frame, call depth, or verifier budget. The first attempt was as direct as possible—take PureDOOM and delete everything that did not fit.