To Save C, We Must Save ABI

March 13, 2022 After that first Firebrand of an article on Application Binary Interface (ABI) Stability, I’m not sure anyone expected this to be the title of the next one, huh? It seems especially bad, given this title is in direct contradiction to a wildly popular C++ Weekly Jason Turner did on the exact same subject: Not only is Jason 110% thoroughly correct in his take, I deeply and fervently agree with him. My last article on the subject of ABI - spookily titled “Binary Banshees and Digital Demons” - also displayed how implementers not only back-change the standard library to fit the standard (and not the other way around) when they can get away with it, but also that occasionally threaten the existence of newly introduced features using ABI as a cudgel. But, if I’ve got such a violent hatred for ABI Stability and all of its implications, why would I claim we need to save it? Should it not be utterly destroyed and routed from this earth? Is it not the anti-human entity that I claimed it was in my last article? Could it be that I was infected by Big Business™ and Big MoneyⓇ and now I’m here to shill out for ABI Stability? Perhaps I’ve on-the-low joined a standard library effort and I’m here as a psychological operation to condition everyone to believing that ABI is good. Or maybe I’ve just finally lost my marbles and we can all start ignoring everything I write! (Un?)Fortunately, none of that has happened. My marbles are all still there, I haven’t been bought out, and the only standard library I’m working on is my own, locked away in a private repository on a git server in some RAID storage somewhere. But, what I have realized steadily is that no matter how much I agitate and evangelize and etc. etc. for a better standard library, and no matter how many bit containers I write that run circles around MSVC STL’s purely because I get to use 64-bit numbers for my bit operations while they’re stuck on 32-bit for Binary Compatibility reasons, these systems aren’t going to change their tune just for li’l old me. Or Jason Turner. Or anyone else, really, who’s fed up with losing performance and design space to legacy choices when we quite literally were just not smart enough to be making permanent decisions like this. This doesn’t mean we need to give up, however. After all, there’s more than one way to break an ABI: Silliness aside, it is important to make sure everyone is up to speed on what an “ABI” really is. Let’s look at ABI - this time, from the C side - and what it prevents us from fixing. The Monster - Application Binary Interface Application Binary Interface, which we will be colloquially referring to as ABI because that’s a whole mouthful to say, is the invisible contract you sign every time you make a structure or write a function in C or C++ code and actually use it to do anything. In particular, it is the assumptions the compiler makes about how exactly the bit-for-bit representation and the usage of the computer’s actual hardware resources when it negotiate things that lie outside of a singular routine. This includes things like: the position, order, and layout of members in a struct/class; the argument types and return type of single function (C++-only: and any relevant overloaded functions); the “special members” on a given class (C++-only); the hierarchy and ordering of virtual functions (C++-only); and more. Because this article focuses on C, we won’t be worrying too much about the C++ portions of ABI. C also has much simpler ways of doing things, so it effectively boils down to two things that matter the most: the position, order, and layout of members in a struct; and, the argument types and return type of a function. Of course, because C++ consumes the entire C standard library into itself nearly wholesale with very little modifications, C’s ABI problems become C++’s ABI problems. In fact, because C undergirds way too much software, it is effectively everyone’s problem what C decides to do with itself. How exactly can ABI manifest in C? Well, let’s give a quick example: The C ABI C’s ABI is “simple”, in that there is effectively a one-to-one correspondence between a function you write, and the symbol that gets vomited out into your binary. As an example, if I were to declare a function do_stuff, that took a long long parameter and returned a long long value to use, the code might look like this: #include <limits.h> extern long long do_stuff(long long value); int main () { long long x = do_stuff(-LLONG_MAX); /* wow cool stuff with x ! */ return 0; } and the resulting assembly for an x86_64 target would end up looking something like this: main: movabs rdi, -9223372036854775807 sub rsp, 8 call do_stuff xor eax, eax add rsp, 8 ret This seems about right for a 64-bit number passed in a single register before a function call is made. Now, let’s see what happens if we change the argument from long long, which is a 64-bit number in this case, to something like __int128_t: #include <limits.h> extern __int128_t do_stuff(__int128_t value); int main () { __int128_t x = do_stuff(-LLONG_MAX); return 0; } Just a type change! Shouldn’t change the assembly too much, right? main: sub rsp, 8 mov rsi, -1 movabs rdi, -9223372036854775807 call do_stuff xor eax, eax add rsp, 8 ret … Ah, there were a few changes. Most notably, we’re not only touching the rdi register, we’re messing with rsi too. This shows us, already, that without even seeing the inside of the definition of do_stuff and how it works, the compiler has forged a contract between itself and the people who write the definition of do_stuff. For the long long version, they expect only 1 register to be used - and it HAS to be rdi - on x86_64 (64-bit) computers. For the __int128_t version, they expect 2 registers to be used - rsi AND rdi - to be used to contain all 128 bits. It sets this up knowing that whoever is providing the definition of do_stuff is going to use the exact same convention, down to the registers in your CPU. This is not a source code-level contract: it is one forged by the compiler, on your behalf, with other compilers and other machines. This is the Application Binary Interface. We note that the problem we highlight is very specific to C and most C-like ABIs. As an example, here is the same main with the __int128_t-based do_stuff’s assembly in C++: main: push rax movabs rdi, -9223372036854775807 mov rsi, -1 call _Z8do_stuffn xor eax, eax pop rcx ret This _Z8do_stuffn is a way of describing that there’s a do_stuff function that takes an __int128_t argument. Because the argument type gets beaten up into some weird letters and injected into the final function name, the C++ linker can’t be confused about which symbol it likes, compared to the C one. This is called name mangling. This post won’t be calling for C to embrace name mangling - no implementation will do that (except for Clang and its [[overloadable]] attribute) - which does make what we’re describing substantially easier to go over. Still, how precarious can C’s direct/non-mangled symbols be, really? Right now, we see that the call in the assembly for the C-compiled code only has one piece of information: the name of the function. It just calls do_stuff. As long as it can find a symbol in the code named do_stuff, it’s gonna call do_stuff. So, well, let’s implement do_stuff! ABI from the Other Side The first version is the long long one, right? We’ll make it a simple function: checks if it’s negative and returns a specific number (0), otherwise it doesn’t do anything. Here’s our .c file containing the definition of do_stuff: long long do_stuff (long long value) { if (value < 0) { return 0; } return value; } It’s kind of like a clamp, but only for negative numbers. Either way, let’s check what this bad boy puts out: do_stuff: xor eax, eax test rdi, rdi cmovns rax, rdi ret Ooh la la, fancy! We even get to see a cmovns! But, all in all, this assembly is just testing the value of rdi, which is good! It’s then handing it back to the other side in rax. We don’t see rax in the code with main because the compiler optimized away our store to x. Still, the fact that we’re using rax for the return is also part of the Application Binary Interface (e.g., not just the parameters, but the return type matters). The compilers chose the same interpretation on both the inside of the do_stuff function and the outside of the do_stuff function. What does it look like for an __int128_t? Here’s our updated .c file: __int128_t do_stuff (__int128_t value) { if (value < 0) { return 0; } return value; } And the assembly: do_stuff: mov rdx, rsi xor esi, esi xor ecx, ecx mov rax, rdi cmp rdi, rsi mov rdi, rdx sbb rdi, rcx cmovl rax, rsi cmovl rdx, rcx ret … Oof. That’s a LOT of changes. We see both rsi and rdi being used, we’re using a cmp (compare) on rsi and rdi, and we’re using a Subtract with Borrow (sbb) to get the right computation into the rcx register. Not only that, but instead of just using the rax register for the return (from cmovl), we’re also applying that to the rdx register too (with a similar cmovl). So there’s an expectation of 2 registers containing the return value, not just one! So we’ve clearly got two different expectations for each set of functions. But… well, I mean, come on. Can it really break? How bad would it be if I created an application that compiled with the 64-bit version initially, but was somehow mistakenly linked with the 128-bit version through bad linker shenanigans or other trickery? Causing Problems (On Purpose) Let’s see what happens when we break ABI. Our function isn’t even that complex; the breakage is probably minor at best! So, here’s our 2 .c files: main.c: #include <limits.h> extern long long do_stuff(long long value); int main() { long long x = do_stuff(-LLONG_MAX); /* wow cool stuff with x ! */ if (x != 0) return 1; return 0; } do_stuff.c: __int128_t do_stuff(__int128_t value) { if (value < 0) { return 0; } return value; } Now, let’s build it, with Clang + MSVC