Pushing the limits of RISC-V emulation

2026-07-26TL;DRThis post explores how close a RISC-V program can get to bare-metal performance on a non-RISC-V machine. The key is an ahead-of-time recompiler that connects generated basic blocks with tail calls and uses Clang’s preserve_none calling convention to keep hot guest state in host registers.

During my time at Axiom, proof generation has scaled from a single CPU to clusters of GPUs by distributing the workload across multiple nodes. However, the first step remains unchanged: we still have to run the program and record its execution. The proving phase is embarrassingly parallel and scales well with additional hardware, whereas execution is inherently sequential. As proving continues to scale, execution speed increasingly becomes the bottleneck, and this post is about making that step faster. When I started looking into execution, I kept coming back to a simple question: how fast should a program run? OpenVM runs RISC-V programs compiled from normal languages like Rust, so I could compile the same program natively for my machine and use that as a baseline. On an M3 MacBook, the native arm64 program finished in about 100 milliseconds, while the same program took three or four seconds through our interpreter. I expected the virtual machine to be slower, since running through it necessarily adds overhead, but not by orders of magnitude. The fastest way to run a program is to get out of its way and let the hardware execute it directly. A virtual machine cannot simply hand the program to the hardware because it has to maintain control over the execution and bridge the gap between the guest2 and the host. Instead, it either interprets the program or translates it into instructions the host can understand. The rest of this post explores how far we can push this idea, starting with a basic interpreter and progressively removing the layers of overhead between the guest and the host. The interpreter baseline A straightforward way to run RISC-V3 code is with an interpreter that loops through the classic fetch-decode-execute cycle. The problem is that guest instructions are often tiny. An add or a shift may only perform a single arithmetic operation, while the interpreter has to fetch the instruction, decode its fields, select the right case in the switch, and maintain its own execution state along the way. The interpreter can end up spending more time managing instructions than executing them.

switch-casefetchdecodedispatch + executefetchdecodedispatch + execute Example: a decoded RISC-V instructionAn R-type instruction, such as add rd, rs1, rs2, uses a fixed layout for its fields:The opcode identifies the instruction’s encoding class. The funct3 and funct7 fields further narrow this down to the specific operation within that class, while rd, rs1, and rs2 identify the registers involved.For example, add x5, x6, x7, which computes x5 = x6 + x7, fills in these fields as follows:Here rd, rs1, and rs2 refer to registers x5, x6, and x7 respectively. The opcode, funct3, and funct7 fields together select the add operation.A sub instruction uses the same opcode and funct3, but changes funct7 from 0000000 to 0100000. Predecoding ahead of time Decoding the same instructions every time they execute is unnecessary. Program code is usually static4, so we can decode each instruction once up front and reuse the result. This is predecoding. For each instruction we store what it does, its operands, and a pointer to the small function (handler) that executes it. Decoding is no longer part of the hot loop. What remains is dispatch through an indirect call to the handler pointer, followed by a return back to the interpreter loop for every instruction. predecodedfetchdispatchexecutefetchdispatchexecute Note: The switch interpreter jumps directly to the instruction implementation, while the predecoded interpreter requires a function call and return for every instruction. Dispatch via computed goto Predecoding removed the cost of decoding. The remaining bottleneck is the indirect dispatch. Every instruction still has to return to a shared loop so the interpreter can select what runs next. A classic fix for this is a computed goto5. Instead of returning to a shared loop, each handler looks up the next opcode’s label in a table and jumps directly to it with a goto *. The predecoded representation is simpler here than before. There is no handler pointer, just the same Instruction from the first section, decoded once into an array indexed by pc / 4. Each opcode gets its own label as a dispatch target. Each instruction still has to be fetched and dispatched, but the interpreter no longer performs a function call and return for every handler.