Exploring Mach-O, Part 4
This is part 4 of a 4 part series exploring the structure of the Mach-O file format. Here are links to part 1, part 2, and part 3.
When we started this series we didn’t know anything about Mach-O other than some vague idea that it’s a binary file format used by macOS. By now, we have a much better understanding of how Mach-O is laid out and how the operating system uses the information in a Mach-O file to run a program.
We learned that every Mach-O file has a 32 byte header right at the beginning.
The first 4 bytes of the header (and thus, the first 4 bytes of every Mach-O)
file are a magic number that indicates that the file is, in fact, encoded
using Mach-O. Much to our delight, we found that this magic number is
0xfeedface on 32-bit systems, and 0xfeedfacf on 64-bit systems.
This header tells us a lot more about our file too, including the type of CPU it is compiled for and the size of the load commands that come directly after it.
After the header come a sequence of load commands. Every load command
shares two fields in common: the command type (which we encoded as a Command
enum in Zig) and the total size in bytes of the command. Each command is parsed
slightly differently, and some commands (such as the segment_commands) have
more data structures that come directly after them.
There are a lot of different load commands that a Mach-O file can contain: our
Command enum has 54 different values. Not every Mach-O file contains all of
these commands, of course. We primarily investigated the Segment commands,
which are an important concept for understanding how a Mach-O file is laid out.
Each segment can have zero or more sections, and in our tiny exit program we
found 3 segments: __PAGEZERO, __TEXT, and __LINKEDIT. More complex
programs will have more segments, notably a __DATA segment, which our program
lacks.
Immediately following the load commands are the segments. We found that our
file contains a large block of contiguous zeros, and we discovered that this is
because the header and load commands share the first segment with the __TEXT
segment. The sections within the __TEXT segment are aligned to the end of the
segment, and because segments must be page aligned, there can often be large
chunks of unused space between the end of the load commands and the start of
the first section in the __TEXT segment. This problem is “worse” on systems
with larger page sizes, such as ARM64 (which uses a 16 KiB page size rather
than the standard 4 KiB pages on x86).
We learned how to use Zig to parse binary file formats, such as using packed structs to represent bit fields and validating enum values using the
std.meta.intToEnum standard library function. We also explored some cool
parts of Zig’s comptime features to make an elegant and easy-to-use API for our
parser.
But wait, there’s more
If you followed along then you know there is a lot we didn’t cover. There are still a lot of load commands we didn’t even talk about, and who knows what kind of goodies those involve.
We also didn’t talk about universal binaries. Apple has transitioned between
ISAs twice: first from PowerPC to Intel x86, and then again to ARM. Because of
this, they figured out a long time ago how to combine two or more binary
formats into a single file. Apple refers to this as a “universal binary”, which
is a packaged archive of multiple Mach-O files. Universal binaries have their
own format, including their own header and magic number (spoiler alert: the
universal binary magic number is 0xcafebabe. Is that even better than
0xfeedface?).
The point is, there are many rabbit holes for you to follow if you’re interested. I hope you do, and if you do, I hope you write about!
Thanks for reading!