Hello, HellGates

IntroductionIn the Summer of 2025 I published a crackme called HellGates.Then I published it in November crackmes.one.Basically I designed a custom 32-bit CPU encrypted & bit addressable (not byte addressable) in VHDL, synthesized it down to a gate-level netlist with multiple layers of obfuscation, anti-tamper, timing checks & anti-debug. It was designed for humans, but apparently worked well against LLMs too, until now.For a year, nobody solved it.Not the humans, they spent weeks/months on it and gave up.Not the LLMs, Claude failed, ChatGPT failed, and DeepSeek failed after days or weeks of work guided by hints I gave to people, ending with the model declaring the challenge “computationally infeasible with available resources.”Then, in September 2026, GPT-6 solved it in under 20-30 minutes on SRE-Bench.Thank you Zhuo Zhang for making this possible !The short version: the crypto protecting the CPU’s state was lazily done and was very weak; there was a side-channel/differential analysis attack used by GPT-6, which made the runs replayable very easily after all obfuscation layers were removed.It gave hints on the encryption used, decrypted registers, re-encrypted registers, ran the netlist properly and dumped all the decrypted 1GB (data.bin) memory.The details of the challenge are here:The virtual CPU architecture.The program I used in the virtual CPU.The obfuscations on host side I used.The toolchains.CPU architectureGeneral information:The architecture contains 16 32-bit general purpose registers, and some special registers:type special_registers is record overflow_flag : boolean; condition_flag : boolean; program_counter : cpu_address_type; key_modifiers : special_key_modifiers_type; index_for_key_modifier : cpu_word_index_type; tea_pseudo_random_state : tea_integer_type; end record special_registers; type registers_record is record general : register_array; special : special_registers; end record registers_record; overflow_flag is when any type of integer overflow happened, or division by zero.The operations were all signed integer operations and contained your basic ALU operations. (add, subtract, multiply …)condition_flag is used for branching between different locations.Example:IsEqual R2, 0 // condition_flag=1 Branch @loc ... instructions when condition_flag=0 ... loc: ... instructions when condition_flag=1 ... Now you may ask: what are key_modifiers/index_for_key_modifier ?I will explain the memory architecture a bit later below.Now all opcodes: -- Integer operations -- constant opcode_type_or : opcode_type := "00001"; constant opcode_type_and : opcode_type := "00010"; constant opcode_type_not : opcode_type := "00011"; constant opcode_type_add : opcode_type := "00100"; constant opcode_type_substract : opcode_type := "00101"; constant opcode_type_division : opcode_type := "00110"; constant opcode_type_multiply : opcode_type := "00111"; constant opcode_type_sla : opcode_type := "01000"; constant opcode_type_sra : opcode_type := "01001"; constant opcode_type_sll : opcode_type := "01010"; constant opcode_type_srl : opcode_type := "01011"; constant opcode_type_rol : opcode_type := "01100"; constant opcode_type_ror : opcode_type := "01101"; -- Memory operations -- constant opcode_type_read : opcode_type := "01110"; constant opcode_type_write : opcode_type := "01111"; -- Branch operations -- constant opcode_type_is_bigger : opcode_type := "10000"; constant opcode_type_is_lower : opcode_type := "10001"; constant opcode_type_is_equal : opcode_type := "10010"; constant opcode_type_had_integer_overflow : opcode_type := "10011"; -- Jumping, branches, set -- constant opcode_type_jump : opcode_type := "10100"; constant opcode_type_branch : opcode_type := "10101"; constant opcode_type_set : opcode_type := "10110"; -- Expanding instructions -- constant opcode_type_xor : opcode_type := "10111"; Okay not much obfuscation here, I could have used chained instruction encryption, shuffling the opcodes and a microcode engine, but I thought it was overkill at that time. (which it was, but now, not sure)The memory architecture:When you are doing a CPU that runs on encrypted memory, you need to be careful, you can’t simply just fetch bits from memory at any bit address, otherwise you’ll just decrypt/encrypt garbage when you try to read/write an integer/instruction, so your CPU becomes useless.This is where it differs from non-encrypted memory because you can read/write directly at any bit address (in theory, most popular CPUs don’t do that of course).It needs read/write memory in an aligned address.A word in my context is the number of bits (and only that number) that the CPU can read/write into a specific slot index in RAM.On my architecture the word size is 64 bits, which is, as you guessed, the block size of the encryption method I’ve used.However this is harder to write the code into compared to plaintext memory, because you can’t simply read a word at a specific index and decode an instruction.Except if you’ve made your instruction size, integer size, word size all the same size and ALSO make the code impossible to jump to a specific bit address, but only to an address aligned to the word size, by design (the same applies for reading/writing integer to memory).For example 0x10000 would be fine, but what if you try to read an instruction at 0x10001 ? Your instruction is split in two parts (C++ example, most people are probably more familiar with this):// Instruction is between 0x10001 and 0x10041 // Let's imagine the first word is at 0x10000, second at 0x10040 // Okay let's start to read until 0x10040 static constexpr uint64_t WORD_SIZE_IN_BITS = 64; uint64_t read_bits = 0; uint64_t wanted_address = 0x10001; uint64_t bit_offset = wanted_address - (wanted_address % WORD_SIZE_IN_BITS); uint64_t word_index = (wanted_address - bit_offset) / WORD_SIZE_IN_BITS; // 0x400 bit_array_t word_bits = read_word_bits(word_index); instruction_bit_array_t instruction; for (uint64_t i = bit_offset; i < WORD_SIZE_IN_BITS; i++) { if (read_bits < sizeof(instruction)) { instruction[read_bits] = word_bits[i]; read_bits++; } } // read_bits is now 63, not 64 ! // So we miss one bit. We need to read the next word to get it bit_offset = 0; word_index++; word_bits = read_word_bits(word_index); for (uint64_t i = bit_offset; i < WORD_SIZE_IN_BITS; i++) { if (read_bits < sizeof(instruction)) { instruction[read_bits] = word_bits[i]; read_bits++; } } // Now the instruction is being completely fetched ! Safe to read. Now the VHDL code is a bit more complex than this, but you have to remember that the instructions/operations on integers are being split into words, and so we need multiple word indexes.Memory encryption:Alright, now let’s talk about memory encryption. The algorithm I used to encrypt was TEA.It was very easy to implement from the C reference, but generates a lot of logic gates because of the number of operations and rounds.To be simple, there are 2 layers of encryption.One is for encrypting the words, where each word has its own set of keys which are dynamically generated based on CPU state (this is what key modifiers are about).Second, the word keys generated are themselves encrypted (KEK) so it couldn’t be retrieved by simply reading memory.The KEK for each word was also dynamically generated (deterministic though), based on a nonce between word_index and a static key, so if a wrong index is picked to decrypt a specific word, it will output garbage.This was also used to defend against side-channel attacks, so a word would surely never be encrypted the same way. Maybe overkill, but that’s what I did.On top of that, all word indexes were permuted to look random access on memory, so word_index at 0, would be something random like 0x2281FD.This is why there is a 1GB data.bin: the virtual CPU’s program is scattered all around the 1GB of data, mixed with entropy data!This was a nice trick to obfuscate the program, it was not a simple “static key” to find in the netlist. Unfortunately, GPT-6 didn’t even need to see this to decrypt memory.The key modifiers are basically just randomly encrypted generated keys. Their indexes are also permuted, hence why ‘index’ for key modifiers. They output to different memory.The weakness (the side-channel):Here a sample of the code how the CPU encrypted its state (cpu_integer_type is just 32-bit signed integer):type fake_array is array(2 downto 0) of cpu_integer_type; variable fake_values : fake_array := (others => (others => '0')); for i in internal_registers.general'range loop internal_registers.general(i) := internal_registers.general(i) + fake_values(i mod fake_values'length); internal_registers.general(i) := internal_registers.general(i) xor (cpu_integer_type(internal_registers.special.tea_pseudo_random_state) + i + 1); end loop; fake_values:for i in fake_values'range loop fake_values(i) := fake_values(i) + cpu_integer_type(internal_registers.general(i mod internal_registers.general'length)) + cpu_integer_type(internal_registers.special.tea_pseudo_random_state + i + 1); end loop; As you can see it was clearly weak crypto.I actually left this on purpose because if I used real cryptography, the logic gates count would have increased to a larger number and I thought nobody would see the side-channel anyway and so moved on because I needed to test my design rapidly after synthesis.I was wrong, this mistake cost me a lot: it made GPT-6 partially solve it by just calling the netlist with the proper CPU state to decrypt memory. Even though it was probably a difficult differential analysis for a human, GPT-6 did it and predicted exactly what needed to be done to discover it.And you can do it too now !The programI’ve made my own assembly language in ANTLR, and a compiler for it. This is the current CTF program, written in hgasm (HellGatesAssembly):#define DMA_ADDRESS_CHARACTER 0x80001000 #define DMA_ADDRESS_RECEIVED_CHARACTER 0x80001008 #define DMA_ADDRESS_LCD_CLEAR 0x80001FF0 #define DMA_ADDRESS_LCD 0x80002000 #define