Making stinkarm stink way less, or more?

1Its been a while, but about half a year ago I wrote an article about implementing a userspace armv7 emulator from scratch, meaning I implemented:ARMASM 1 .section .rodata 2msg: 3 .asciz "Hello, world!\n" 4 5 .section .text 6 .global _start 7_start: 8 ldr r0, =1 9 ldr r1, =msg 10 mov r2, #14 11 mov r7, #4 12 svc #0 13 14 mov r0, #0 15 mov r7, #1 16 svc #0Or as a list:elf(32) parsing, validation and interpretationdecoding of a very small subset of armv7 instructions (only 3)executing said instructions, even conditional ones 🤓translating memory access from the guest into the hostsyscall forwarding (from armv7 to x86)syscall sandboxing (only a restricted syscall subset) and denying syscall executionDo read Building a Minimal Viable Armv7 Emulator from Scratch, since this post doesnt go as deep into detail as the previous one (It’s my first article in 3 months I had enough motivation for writing :O). This is partially an update, partially my toughts on decoding and emulating armv7-a and also a bit of a devlog.Overly complex host to guest mem translationOn the first article, ~aengelke on lobste.rs, had some comments, the one resonating the most was:[…] The Mem indirection seems pretty inefficient. When emulating 32-bit platforms on a 64-bit system, just mmap a 4 GiB region, the translation then becomes a single addition. Otherwise, having a small hash table of recently translated address regions can avoid more expensive searches – memory accesses have a very high locality. The number of mappings is usually small, so binary search over a sorted array is simpler than a B-tree. […]So now i figured, why not improve on my implementation a bit, first with replacing the complex allocation region based tracking with just allocating a 4gig slab in memory for the guest, mapping the process regions there and handing out pointers into that region to the guest.So, previously the memory translation worked as follows:One takes a binary tree map of a guest starting addr to its host segmentRUST1struct MappedSegment { 2 host_ptr: *mut u8, 3 len: u32, 4} 5 6pub struct Mem { 7 maps: BTreeMap<u32, MappedSegment>, 8}On ask for a region handout, specifically on mapping ELF segments with a starting addr, map_region is called:RUST 1// in stinkarm::elf::pheader::Pheader::map: 2 3// record mapping in guest memory table, so CPU can translate guest vaddr to host pointer 4guest_mem.map_region(self.vaddr, len, segment_ptr); 5 6// in stinkarm::mem::Mem: 7 8pub fn map_region(&mut self, guest_addr: u32, len: u32, host_ptr: *mut u8) { 9 self.maps 10 .insert(guest_addr, MappedSegment { host_ptr, len }); 11}Since the cpu needs to fetch an instruction, there is read_u32, calling translate:RUST 1/// translate a guest addr to a host addr we can write and read from 2pub fn translate(&self, guest_addr: u32) -> Option<*mut u8> { 3 // Find the greatest key <= guest_addr. 4 let (&base, seg) = self.maps.range(..=guest_addr).next_back()?; 5 if guest_addr < base.wrapping_add(seg.len) { 6 let offset = guest_addr.wrapping_sub(base); 7 Some(unsafe { seg.host_ptr.add(offset as usize) }) 8 } else { 9 None 10 } 11} 12 13pub fn read_u32(&self, guest_addr: u32) -> Option<u32> { 14 let ptr = self.translate(guest_addr)?; 15 unsafe { Some(u32::from_le(*(ptr as *const u32))) } 16} 17 18 19// in stinkarm::cpu::Cpu: 20 21pub fn step(&mut self) -> Result<bool, err::Err> { 22 let Some(word) = self.mem.read_u32(self.pc()) else { 23 return Ok(false); 24 }; 25 26 // [...] 27}Of course this totally unnecessary work, we dont need to keep track of every mapping/allocation/region by walking their ranges, we only need to make sure the R/W interaction request is within bounds. Thus the new implementation is:One takes a pointer and a size:RUST1pub struct Mem { 2 ptr: NonNull<u8>, 3 len: usize, 4}When asked to map ELF segments, stinkarm::mem::Mem::map_region is called:RUST 1// in stinkarm::elf::pheader::Pheader::map: 2guest_mem.map_region(self.vaddr, file_slice)?; 3 4// in stinkarm::mem::Mem: 5 6pub fn map_region(&mut self, guest_addr: u32, data: &[u8]) -> Result<(), String> { 7 let dst = self 8 .get_slice_mut(guest_addr, data.len()) 9 .ok_or_else(|| format!("guest region out of bounds at {guest_addr:#010x}"))?; 10 dst.copy_from_slice(data); 11 Ok(()) 12}When cpu requests a dword for decoding, it does so by invoking stinkarm::mem::Mem::read32, just as before, only this time with bounds checks:RUST 1pub fn read_u32(&self, guest_addr: u32) -> Option<u32> { 2 let bytes = self.get_slice(guest_addr, 4)?; 3 Some(u32::from_le_bytes(bytes.try_into().unwrap())) 4} 5 6fn get_slice(&self, guest_addr: u32, len: usize) -> Option<&[u8]> { 7 if !self.in_bounds(guest_addr, len) { 8 return None; 9 } 10 11 Some(unsafe { std::slice::from_raw_parts(self.ptr.as_ptr().add(guest_addr as usize), len) }) 12}Hardening the existing implementationI also noticed I have a lot of stuff that (even with the small surface of just the write.2 and exit.2 syscalls, ldr, mov and svc) could enable translating untrusted guest adresses into host mem access.Preventing this via checking the validity of the address passed to write.2, we do this while translating guest addresses to host memory space in stinkarm::mem::Mem with a in_bounds call inside the translate_range call:RUST 1const NULL_PAGE_SIZE: u32 = 0x1000; 2 3impl Mem { 4 fn in_bounds(&self, guest_addr: u32, len: usize) -> bool { 5 if guest_addr < NULL_PAGE_SIZE { 6 return false; 7 } 8 9 let start = guest_addr as usize; 10 let Some(end) = start.checked_add(len) else { 11 return false; 12 }; 13 14 end <= self.len 15 } 16 17 pub fn translate_range(&self, guest_addr: u32, len: usize) -> Option<*mut u8> { 18 if !self.in_bounds(guest_addr, len) { 19 return None; 20 } 21 22 Some(self.ptr.as_ptr().wrapping_add(guest_addr as usize)) 23 } 24}I added multiple tests for making sure I correctly catch writing a null pointer, writing out of guest memory and loading elf segments at 0x0:ARMASM 1 .section .rodata 2msg: 3 .ascii "ignored" 4 5 .section .text 6 .global _start 7_start: 8 mov r0, #1 9 mov r1, #0 10 mov r2, #7 11 mov r7, #4 12 svc #0 13 14 mov r0, #0 15 mov r7, #1 16 svc #0ARMASM 1 .section .rodata 2msg: 3 .ascii "ignored" 4 5 .section .text 6 .global _start 7_start: 8 mov r0, #1 9 ldr r1, =0x08000000 10 mov r2, #7 11 mov r7, #4 12 svc #0 13 14 mov r0, #0 15 mov r7, #1 16 svc #0To DSL or notPreviously I hardcoded every opcode and its fields to decode them into a rust representation, now only the opcode is subject to decoding. This is achived with a nice looking compiletime constant list of patterns:RUST 1const DECODE_RULES: &[ArmRule] = &[ 2 arm_rule!(Svc { 3 bits(27..24 = 0b1111), 4 }), 5 arm_rule!(Branch { 6 bits(27..25 = 0b101), 7 }), 8 // LDR literal: `ldr Rt, [pc, #imm12]`.

19 arm_rule!(MovImm { 20 bits(27..25 = 0b001), 21 bits(24..21 = Op::Mov as u32), 22 }), 23];If youre interested in ARMv7 instruction encoding I can recommend the ARM® Architecture Reference Manual ARMv7-A and ARMv7-R editionThe macro itself builds a bit pattern that can then be used with a simple AND bit instruction to detect:RUST 1macro_rules!