Decoding the NEC V20 Microcode

reenigne's decoding of the 8088 microcode in 2020 opened the doors for extremely accurate emulation of the 8088 CPU.  Although I had added support for the NEC V20 in MartyPC, the V20 core was not cycle-accurate in terms of the V20's actual timings.  It was an 8088 in a V20's clothing - a copy-paste of my 8088 core with V20 instructions tacked on.This was not an ideal case, but the prospect of making my V20 core cycle-exact without the microcode seemed like it might be a discouraging slog of trial and error.So why not get the microcode, then?I recently commissioned InfoSecDJ to take die photography of an NEC V20 CPU (actually a second-source V20 fabricated by Sharp, but a V20 nonetheless). He did an excellent job.The NEC V20 CPU die - InfoSecDJThis photomosaic is extremely high resolution - 5.6 Gigapixels to be exact, an astonishing 70478x80672 resolution - too large to even fit in the JPEG image format!You can see the entire thing at full resolution here.The rectangular region just below the center of the die is the main microcode ROM.The V20 Microcode ROM blockThe ROM array is 258x116, containing 29,928 bits. It's evenly divisible by 29, which we know is the microcode word length of the V20, so that's a good sign.  But it implies 1032 microcode words, when we were only expecting 1024.  That's a bit odd, isn't it?  We'll figure out the reason for that a bit later on.Here's a close-in crop of the ROM array:Microcode bits, zoomed inThe bright, horizontally running traces are part of the chip's metal layer. The yellowish dots are interconnects that connect the metal layer with the polysilicon layer beneath it.  Note the vertical bars behind the metal layer - and note that there is occasionally a gap in the polysilicon on either side of each interconnect.  These gaps form a transistor - with the presence of a transistor indicating a 1 bit. I'll highlight the 1 bits to make that a bit easier to see.Seeing this got me very excited - if we can visually identify the bits in the ROM, then we can extract the ROM contents.  Just one slight problem - there are 29,928 bits in the array.  That would be a tad bit tedious to extract by hand.Luckily, there are tools available for this task.  I loaded up the ROM image in MaskRomTool by Travis Goodspeed.Defining bit locations with MaskRomToolMaskRomTool allowed me to draw the rows and columns that defined the locations of the bits.  Unfortunately, I found that its bit-detection methods were based on thresholding, and there wasn't enough of a difference in contrast between a bit and a non-bit to make this an effective detection mechanism.  Notice the bit histogram is very compressed toward the far axis.  This was not going to work.  Perhaps the thresholding technique would have worked better without the bright metal layer, but I didn't want to ask InfoSec DJ to attempt removing it. Another approach was needed.Fortunately, we could use MaskRomTool to export our defined bit positions to JSON format.  I used this exported JSON file to write a Python script that extracted a square bitmap centered on each bit position, and saved it with the bit's logical column and row number in the filename. This gave me a 105MB ZIP file of little 42x42 pixel PNG files, each containing either a bit or a not-bit. The idea behind doing this was to train a convolutional neural network to identify the bits as either 0 or 1.If this sounds out of your depth, I had no prior experience in training CNNs before this project, and I completed it in a single afternoon. Modern Python frameworks such as PyTorch make it that easy.There are some good CNN tutorials out there, many of them focused on the classic problem of handwriting recognition, but we have an even simpler task. We just have to sort images into two buckets, 0s or 1s. The problem is quite literally "Hotdog or Not Hot Dog."Before we can train our CNN, we have to have a training data set. So I created a quick and dirty Python/tkinter script so I could quickly sort bits by eye into buckets as either 0s or 1s.The quick and dirty classification toolShown here is a '1' bit. Can you spot it by the transistor behind the metal layer? The buttons ended up being extraneous - you just need to hit either 1 or 0 on your keyboard to classify the bit.  In theory, you could do this 29,928 times and you'd have the job done in a few hours. I had originally intended for this to be my backup method in case the CNN training didn't work out - I had a few friends willing to volunteer to help, and the JSON logs that the "Bit Voter" produces can be merged to support distributed work with consensus. Fortunately, this was not needed.Ultimately, I classified a little over 1,000 bits manually this way. Once they were sorted into two directories, we could now attempt to train a CNN model using the sorted images as input.I won't go into great detail about how to write a CNN here - the source will be on my GitHub if you're  curious. I used the excellent PyTorch library. which made the whole process a lot simpler than I could have ever imagined.This is what a training run looks like. [Epoch 01] train: loss=0.6945 acc=0.7273 f1=0.0164 | val: loss=0.6924 acc=0.7876 f1=0.0000 val precision=0.0000 recall=0.0000 cm=[[178, 0], [48, 0]] [Epoch 02] train: loss=0.6860 acc=0.7151 f1=0.3826 | val: loss=0.6257 acc=0.7965 f1=0.0729 val precision=0.5000 recall=0.0394 cm=[[178, 0], [46, 2]] [Epoch 03] train: loss=0.3751 acc=0.8914 f1=0.7213 | val: loss=0.3900 acc=0.7655 f1=0.6327 val precision=0.4661 recall=1.0000 cm=[[125, 53], [0, 48]] [Epoch 04] train: loss=0.1159 acc=0.9523 f1=0.9139 | val: loss=0.0495 acc=0.9912 f1=0.9773 val precision=0.9773 recall=0.9773 cm=[[177, 1], [1, 47]] [Epoch 05] train: loss=0.0251 acc=0.9945 f1=0.9888 | val: loss=0.0460 acc=0.9867 f1=0.9744 val precision=0.9514 recall=1.0000 cm=[[175, 3], [0, 48]] [Epoch 06] train: loss=0.0319 acc=0.9933 f1=0.9802 | val: loss=0.0438 acc=0.9823 f1=0.9659 val precision=0.9350 recall=1.0000 cm=[[174, 4], [0, 48]] [Epoch 07] train: loss=0.0185 acc=0.9945 f1=0.9212 | val: loss=0.0274 acc=0.9956 f1=0.9891 val precision=0.9792 recall=1.0000 cm=[[177, 1], [0, 48]] [Epoch 08] train: loss=0.0141 acc=0.9956 f1=0.9913 | val: loss=0.0271 acc=0.9956 f1=0.9891 val precision=0.9792 recall=1.0000 cm=[[177, 1], [0, 48]] [Epoch 09] train: loss=0.0101 acc=0.9978 f1=0.9940 | val: loss=0.0447 acc=0.9867 f1=0.9735 val precision=0.9488 recall=1.0000 cm=[[175, 3], [0, 48]] [Epoch 10] train: loss=0.0110 acc=0.9967 f1=0.9907 | val: loss=0.0437 acc=0.9912 f1=0.9773 val precision=0.9773 recall=0.9773 cm=[[177, 1], [1, 47]] Early stopping: no val F1 improvement >= 0.0 for 3 epoch(s). Best val F1: 0.9891