Building a Linux GPU Driver for the M4 Mac Mini in One Month
Building a Linux GPU Driver for the M4 Mac Mini in One Month. I Came, I Prompted, I Left Part 2: Building a GPU Driver From Scratch in One MonthPrevious blog post: https://codyho. dev/blog/hypervisor-macbook-neo/What We DidTL;DR: Niklas and I built a fully OpenGL ES 3.
What happened
0 compliant GPU driver for the M4 Mac Mini and MacBook Neo in about a month, a process which normally takes years. Here is Chrome and Firefox running WebGL on the M4 Mac Mini with working compositing:Most importantly, the driver is fast enough to run Minecraft at 200fps:Building this driver involved reverse engineering the AGX’s (Apple’s name for the GPU) incredibly complicated firmware ABI and user-space components. Building the Kernel DriverMoving from a Python prototype driver to a fully featured Linux driver took three days, and one of those days was almost totally wasted because Codex, for some reason I still do not understand, chose to tackle partial renders first (by far the hardest task) instead of doing compute first (the easiest task).
Our goal was to implement conformant OpenGL (and soon, Vulkan) drivers for the M4 Mac Mini and MacBook Neo. Normally, building a GPU driver is an endeavor that takes years; our goal was to do it in days. In those weeks we have:Reverse engineered the M4, A18 Pro, and (mostly) M5 user space using only live probing, discovering hardware-supported features and instructions not emitted by Apple’s driverBuilt a fully working user-space driver, including a new custom IR/shader compiler, command stream builder, and many more componentsReverse engineered, from scratch, the full AGX firmware ABI using traces from the hypervisor I previously builtImplemented a full Linux kernel driver for said firmware ABIThroughout this process, we have not looked at any Apple binaries, only hardware traces (from our hypervisor) and shaders we built ourselves.
Throughout my time building this driver, it became clear that Alyssa Rosenzweig and the others who built the M1/M2 driver are utter wizards– hats off to them!
The wider picture
DeliverablesMesa: https://github. com/niklassheth/mesaLinux Kernel Driver: https://github. com/GravityLinux/linux/gravity-m4User-Space RE Documentation (horrible pile of LLM slop, but functional): Cody NiklasRemaining WorkVulkan 1. The Linux kernel driver will be an even bigger problem, since the M1/M2 driver is not yet upstream, and practically we are not in a position to change this. Now the goal became to actually do something useful with it, and what better target than writing a GPU driver.
This mirrors the split in all modern GPU drivers: the kernel is responsible for interfacing with the firmware, allocating buffers, and managing scheduling, while the actual contents of those buffers and what is being scheduled are opaque. Kernel SpaceOn Apple Silicon, the kernel driver does not interface directly with the hardware. That means that the first step to a kernel driver is not talking to hardware, it’s figuring out the firmware ABI. The firmware ABI was by far the most annoying part of this project, because rather than doing the sane thing of coming up with a reasonable ABI with nice interfaces, Apple essentially took a regular kernel driver, cut it in half, and then put half of it in the AGX and called it firmware, with the other half of the kernel driver communicating using shared structs in memory.
For an idea of how complicated the ABI is, this is what the shared memory tree looks like on the M1/M2:Asahi Lina famously figured all of this out over grueling 12-hour days to build the M1/M2 kernel driver, an amazing technical accomplishment.
What has been reported
Unfortunately, the A18 Pro firmware ABI (I started my RE work on the MacBook Neo and later pivoted to the M4 Mac Mini) is significantly more complicated than the already very complicated M1 firmware ABI:What the F@! #, Apple. In these cases, there are two options, and the driver needs to support both: either increase the size of the TVB, or perform a partial render, ie, render part of the geometry, then reload the buffer with the rest of the triangles, and finish the partial render.
These partial renders turned out to be very, very finicky, even more so than the rest of the work because they essentially mean adding save and resume to the GPU driver. At all high level, the entire process was, basically:Rewrite the existing drm-shim in Rust following the exact same pattern; this gives us a synchronous Rust driver. For the first phase, I had Claude look at every possible Metal program it could find and trying to build a disassembler, assembler, and understand the format of all the other descriptors/command streams/etc required for the GPU driver.
I had Claude enumerate everything, including stuff Linux can’t use (like tessellation) for completeness. At this point, Niklas finished his drm-shim for the M4 Mac Mini and joined me for the second phase of user-space RE. We had two different approaches to actually finishing the user-space driver:My approach was to prioritize hardware RE, and focus on just figuring out how the hardware and all the instructions worked.
What happens next
His time was split between building and testing Mesa, and performing RE. This included:A native single-instruction 64-bit addAnisotropy to 128x (Metal caps at 16x)A new mode of the matrix unit7-bit immediate support for uniform_movMesa DevelopmentThere are a few things that massively work in our favor when building out user-space graphics. Most notably, the Khronos compatibility test suite (CTS) is already an exhaustive corpus of tests our driver must pass. This is what the modern OpenGL stack on Linux looks like:All we have to do is translate between Gallium, Mesa’s internal API, and AGX hardware semantics.
As a bonus, this compiler can be reused for a future Vulkan driver. We want our driver to be as good as the best graphics drivers in the world. We also expect significant skepticism given that this is likely the first ever fully LLM-written GPU driver, and that our code will be held to a higher standard than human-written code.
The report has been compiled by The Daily Waves using information reported across codyho.dev, theguardian.com. Details are presented according to the information available at the time of publication and may change as authorities, organisers or other relevant parties provide updates.