Running a simple MIPS emulator in Python for ALI M3801 firmware (Hello World)

Here I will show my first attempt at building an emulator for the ALI M3801 microprocessor based on off-the-shelf Unicorn and Capstone modules. The developed program will load the contents of the Flash memory and execute it similarly to a real physical CPU, although it will not be without modifications and fixes, as Unicorn/Capstone do not implement the full logic of a particular SoC or its peripherals. In addition, the whole thing will be able to correctly handle sending data over the UART, i.e. it will emulate the register responsible for transmitting bytes over the hardware serial bus. In this way, we will get the same messages in the console that running a programme on a real ALI chip would show.

Tools used
Ghidra is an advanced reverse engineering (SRE) tool being developed by the NSA. It allows decompilation of MIPS machine code into pseudo-C code, making it much easier to analyse and understand firmware logic.
Unicorn is a lightweight, multi-platform framework for CPU emulation based on QEMU. In the project it serves as the main engine for emulating MIPS32 instructions in Little Endian mode, although in practice much of the operation (including memory access) is handled separately in my code anyway.
Capstone is an advanced disassembly engine supporting multiple architectures. It is used here to convert machine code into readable assembler instructions, which is essential for tracing and debugging them. It allows the user to see exactly what the processor is currently executing.
I will do the project in the Pyhon language.

Firmware used for demonstration
To simplify the workflow I used a ready-made 'hello world' on ALI found on GitHub - michal4132/ali_sdk . I presented this project previously here:
How to compile and run your own firmware for ALI M3801 and other tuner chips?
This firmware is characterised by the absence of precompiled modules (so-called "blobs"), which makes it easier to analyse and compare the source code to the results from the emulator. For this reason, we will use it here.

NOTE
The topic assumes a basic knowledge of terminology and I will not explain here how the processor works. I will focus here on just presenting the construction of my simple emulator.

Importing firmware in Ghidr
Open Ghidra, create a new project, do File->Import File:

Before importing the file, you need to configure it properly.

Set the architecture: MIPS Little-Endian. MIPS is a RISC architecture and Little-Endian specifies the order in which bytes are stored in memory.

Set base address: afc00000. The base address is the fixed starting address of the memory area. It serves as a reference point for further addressing of data or registers, without a correct base address the jump instructions would go to the wrong place.

When opened, Ghidra gives us two views. The first time we have to wait for the decompilation to finish, but after that everything works smoothly.
The first view is directly the bytes of the opened file but mapped to the base address - hence the addresses start with AFC. Next to them we have the decompiled commands with their arguments.
The second view is C pseudo-code, which tries to show what the function would do in C - as much as it can. A lot of information is lost when compiling, so we don't even have variable names or the exact syntax here as it was in the source code.

In this case we have the source code of the decompiled program, so we can compare and check. The startup routine is partly written in assembler and partly in C:
start.S:

Continued in C:
entry.c:

You can easily compare the two files:

You can see here, for example, that FUN_afc00afc(0xffffffff,0xffffffff,1); is uart_attach



Deassembly in Python
Let's start with the simplest one. This example shows a simple disassembly of a binary code after a given address in Python. The binary is loaded into memory at the specified base address, without running or emulating instruction execution. The Unicorn engine is only used here to map the memory, while Capstone reads the bytes from under the specified address and translates them into MIPS assembler instructions. This allows the contents of the ROM code to be quickly previewed in human-readable form and compared with what Ghidra shows.

As a test, I read the first few dozen instructions. Virtually everything agrees:

The only difference is li (in Ghidra) which the Python program shows as addiu. This is because li is not an actual MIPS processor instruction, but an assembler pseudo-instruction. In firmware, there is an actual addiu (or sometimes lui + ori), which Capstone shows explicitly, while Ghidra simplifies the notation to li for readability.

First steps with the emulator
Now you can go one step further and start executing the instructions. In this example, register operations, calculations and conditional jumps will already be running. This time Unicorn will already be performing operations, although, as I found out shortly afterwards, not everything will work correctly. But one step at a time:

I put a limit on the program's executed instructions and compared the emulation's "footprint" with what is seen in Ghidra. The basics match, the jumps also match:




Loops with emulator
Similarly, loops also work. First we have loops copying data into RAM. I added a counter to the emulator to show how many times (globally) an instruction has been executed, this allows us to visualise a little better what is happening:

Let's consider the first loop from this firmware:

The loop section (afc00100 - afc00108) repeats 64 times.
This is the loop from entry.S:

This can be compared to the source code:

This is what the end of the loop looks like:


Subsequent loops:




Stopping at an instruction
Another useful mechanism that I have decided to implement is to stop the program on a command with a given address. This allows me to easily check if the executed program reaches a certain point, whose address I find in Ghidra. You could say that this is a simple breakpoint, like in a debugger. For the moment, it is enough for me to define the STOP_INSTR variable in the code.

This is where Ghidra came in handy again. There I selected the address at which I want to stop the execution of the commands (afc04adc) and then verified the program trace to make sure everything was correct. This is very convenient and useful for testing and verifying that the program is running correctly.





Fine UART initialization fix

The program, prepared in this way, was already reaching uart_set_mode, but was showing an access error when trying to write data to the UART register.

These registers were not mapped to memory:

I had to add their mapping:

After this change, the emulator gets as far as 0xAFC04B04, which is where the text data will be sent:

What's more, the function itself from the display also executes. I set the endpoint right after it, and there are no errors.


Interrogating the text display
Unfortunately, full support for text display would require emulation of the UART along with reading its register used to send data. For now, we'll keep it simple and just capture the printf function itself. In Ghidra, it is easy to trace it because its argument is a character string:

We can intercept its call and artificially skip its execution:

Result:


Well, yes, but now printf-style formatting of variables doesn't work. No wonder, our Python function displays blindly. Maybe it's better to look at the printf source code:

The formatting can be emulated and we plug in fwrite though. Here, however, was a problem that took me a long time, but I will keep it to a minimum for you. It seems that the execution of the sb/sw commands , i.e. the instructions responsible for writing to memory, is not working.
I have implemented their manual execution:

And this is what the hook on fwrite looks like:

Result:

Agrees with the one from the CPU:

All text, after turning off showing instructions:

Not too bad, even operations on floating point numbers work.

Faster UART emulation
Capturing kprintf or there fwrite is nice for testing, but not at all practical. The address of these functions can probably change with each compilation. It is true that at compile time you can force a function to have a given address, but I wouldn't expect that here.
The UART needs to be handled better - you need to know where the hardware UART register is and it's from there that you read the data.
Fortunately we already have this information - it can be found in many SDKs on GitHub.
The UART addresses are:

The register for the character is:

However, let us focus on the posting itself. We can easily conclude that all we need to do is capture the write to this address and display it as output from the UART.
This is where a small technical problem arose, because as it turned out, Unicorn does not execute some of the commands correctly, so I had to implement them manually:


Only then are the operations executed.

Eventually the UART sends the data, but something is wrong. The data is repeated three times. An explanation of this will be found below:


The firmware checks to see if the UART acknowledges the sending of the data and, if not, performs the transmission again. All in a loop, in a blocking manner. So we still need to include the transmission acknowledgement flag.
To do this, we need to simulate bit 0x20 of SCI_16550_ULSR. We can do this as soon as a byte is sent. Very simple:

Full code:

As of now, the UART is sending data correctly.


Problem to be solved in the next topic
The main problem that is still to be solved is the 16-bit MIPS commands, these occur sporadically on the original upload from ALI:

And moments later:

The emulator used does not seem to support this, so there will be further combinations.

Summary Summary
This managed to run Hello world completely as if the target CPU was doing it - no shortcuts or simplifications. My program emulates the base of the ALI M3801 and is able to show what would be sent via UART 1.
The whole thing turned out to be more difficult than I thought, as I had to reimplement some of the commands myself to get the read/write to work correctly, and on addresses as MIPS sees them - Unicorn does not implement KSEG0/KSEG1 segmentation and masks the address to 29 bits, treating it as a physical address. This is well demonstrated in this example:

The code shows 0xa00026a0 and the emulator wants access to 0x000026A0. Maybe I should tweak this to hold the physical conversion, but that's in the next section. Initially I thought a triple mapping into the same memory section would suffice:

but the operations weren't performing anyway - at this point it's not clear to me what I was doing wrong.

Follow up soon, all suggestions welcome - this is my first approach to emulation.
Here's a little preview of the next topic: