The kernel is responsible for managing core operating system functions such as start-up, memory management, device abstractions, and process isolation. For it to function, the kernel needs to interact with the rest of the operating system which is collectively referred to as “userspace”. Executables in userspace use “syscalls” to tell the kernel what to do. These syscalls are very low-level and usually not called directly by a process. Instead, an abstraction layer is provided by the standard C library, or Libc.

Linux is unique among operating systems due to the fact that the Kernel and Libc are developed independently. Linux is maintained by creator Linus Torvalds and a community of contributors. Glibc, the most popular Libc for Linux, is maintained by the GNU project. As a result, Linux has a strong separation between Syscalls and Libc.

Linux does not tie itself to any specific Libc. Even though Glibc is used in almost all distros, many alternatives are available. For instance, Musl provides a more lightweight version of Glibc, while Bionic is the Libc used in the Android operating system. In addition, multiple versions of each of these Libc’s can be used on any one kernel, even at the same time. This can become very common when using multiple Nixpkgs versions at one time.

To accomplish this, Linux provides a stable list of syscalls that it has maintained across many versions. This is specified for i386 at arch/x86/entry/syscalls/syscall_32.tbl in the kernel tree. The syscalls specified here are the interface through which the Libc communicates with the kernel. As a result, applications built in 1992 can run on a modern kernel, provided it comes with copies of all its libraries¹.

The macOS Libc is called libSystem. It is available on all macOS systems at /usr/lib/libSystem.B.dylib. This library is the main interface that binary compatibility is maintained in macOS. Unlike Linux, macOS maintains a stable interface in libSystem that all executables are expected to link to. This interface is guaranteed by Apple to be stable between versions.

In Nixpkgs, we maintain this compatibility through a list of symbols that are exported by libSystem. This is a simple text list and is available for viewing at NixOS/nixpkgs/pkgs/os-specific/darwin/apple-source-releases/Libsystem/system_c_symbols. The symbol list is created by listing symbols (nm) on the minimum macOS version that we support (for my PR, 10.12). We do some linking tricks to ensure that everything that we build in Nixpkgs only contains those symbols. This means that we can reproducibly build on newer versions of macOS, while maintaining compatibility with older macOS versions. Unfortunately, newer symbols introduced in later versions cannot be used even on systems that have those symbols.

A side effect of macOS design, is that fully static executables are not supported in macOS as they are on Linux. Without a stable syscall interface, there is nothing to provide compatibility between versions. As a result, Apple does not support this type of linking².

There is no mandated reason why we need to use libSystem directly. In fact, some languages like Go have attempted to instead use the syscall interface directly. There is no reason why this couldn’t work, however, upgrades between versions will almost certainly break binaries. Go eventually abandoned this scheme in time for Go 1.12 (proposed by Nixpkgs macOS contributor copumpkin!)

Some other examples may be useful. They mostly fall on one side or the other of the Syscall / Libc divide —

• FreeBSD - breaks syscall compatibility between major releases, should use Libc for longterm binary compatibility.
• OpenBSD - similarly, changes syscall interface, perhaps even more often than FreeBSD³.
• NetBSD - apparently has maintained syscall compatibility since 1992. ⁴
• Windows, Solaris, Fuchsia - I cannot find any information on these and how they handle binary compatibility.

As a side note, this difference can be clearly seen in how we specify target systems. The LLVM triple is a 3 or 4-part string specifying what we want to build for. The parts of the triple correspond to:

<cpu>-<vendor>-<kernel>-<abi>

• <cpu> — the CPU architecture that we are building for. Examples include x86_64, aarch64, armv7l, etc.
• <vendor> — an arbitrary string specifying the vendor for the toolchain. In Nixpkgs, this should always be unknown.
• <kernel> — the kernel to build for (linux).
• <abi> — the kernel ABI to use. On Linux, this corresponds to the Libc we are using (gnu for Glibc, musl for Musl, android for Bionic).

When building for Linux, we can build for any version of Linux at one time. No version information is required. In addition, we must specify what “ABI” we want to use. In Nix, this is not very important because the Libc is provided by the closure. In fact, Nix has its own version of the LLVM triple called a Nix system tuple that omits the <abi> portion altogether! It corresponds to <cpu>-<kernel> from the LLVM triple.

In comparison, when building for BSDs, we must specify which version of the kernel we are building for. In addition, we leave off the <abi> portion, because there is only one Libc available for these platforms. They are even included in the same tree as the kernel. Examples of BSD triples include,

• aarch64-apple-darwin16.0.0
• x86_64-unknown-freebsd12.0
• i386-unknown-openbsd5.8
• armv7l-unknown-netbsd7.99