tinylinker

A small, cross-platform linker written in plain C.

/* pick a target at runtime, not at build time */
tl_State *s = tl_new(TL_TARGET_ARM64_MACHO);

tl_set_output_type(s, TL_OUTPUT_EXE);
tl_add_file(s, "main.o");
tl_add_library(s, "System");
tl_output_file(s, "a.out");

Overview

tinylinker reads relocatable objects, static archives, shared libraries and GNU ld scripts, and writes an executable, a shared library, or a merged relocatable object, in ELF, PE/COFF or Mach-O. It can also link straight into memory and hand you function pointers, which suits a compiler runtime doing JIT-style execution.

The whole library is one static archive of under a megabyte, or a shared object exporting 41 symbols. It has no dependencies beyond libc.

Targets

Every supported architecture and object-format pair is a separate backend compiled into the library. You pick one at runtime, so a single binary cross-links for all of them.

ArchitectureELFPE/COFFMach-O
i386✓✓no
x86_64✓✓✓
arm✓nono
arm64✓✓✓
riscv64✓nono

The backend that matches the host is additionally able to link into memory and run the result. Every other backend cross-links only, which is the normal case for a compiler targeting something other than the machine it runs on.

$ tinylinker --list-targets
i386-elf
x86_64-elf
arm-elf
arm64-elf
riscv64-elf
i386-pe
x86_64-pe
arm64-pe
x86_64-macho
arm64-macho     (native)

You can trim the library to the targets you actually need. Building only x86_64-elf and arm64-macho takes the static archive from 912 KB to 213 KB.

Formats

Each target reads and writes its own object format.

TargetReadsWrites
*-elfELF .o, ar archives, .so, ld scriptsELF executable, .so, .o
*-machoMach-O MH_OBJECT, ar archives, .dylib, .tbdMach-O executable, .dylib, .o
*-peCOFF .obj, ar archives, .dll, .defPE executable, .dll, COFF .obj

Every target additionally accepts ELF objects, whatever its own format is. A compiler runtime that already produces ELF can carry on doing so, whatever it is targeting.

Archives are read in both the System V layout and the BSD one macOS writes, including its __.SYMDEF index and the way it stores long member names.

On macOS this means clang -c foo.c and tinylinker -o foo foo.o -lSystem is all it takes. The result is ad-hoc signed by the linker itself, because arm64 macOS will not run an unsigned binary, and that holds wherever the link was done: a Mach-O produced on Linux runs on macOS.

Getting started

Requires xmake and a C compiler; clang is the default.

git clone https://github.com/nickglenn/tinylinker
cd tinylinker
task build

or, without Task:

xmake f --toolchain=clang
xmake

Useful configuration:

xmake f --toolchain=gcc                    # or msvc / clang-cl
xmake f --kind=shared                      # shared library instead of static
xmake f -m debug

The sources are self-contained enough to build with nothing but a compiler, which is what the container tests do:

clang -O2 -std=c11 -DTL_VERSION='"0.1.0"' \
      -Iinclude -Isrc -Isrc/core \
      src/tinylinker.c src/backends/b_*.c cli/main.c -o tinylinker

The API

One header, tinylinker.h. A link is: choose a target, set options, choose an output type, add inputs, produce output. Every declaration in it is written up in the API reference, which is generated from the header itself.

#include <tinylinker.h>

tl_State *s = tl_new(TL_TARGET_X86_64_ELF);

/* Settings first: some of them decide what the output type does. */
tl_set_flags(s, TL_F_STATIC, 1);
tl_set_str(s, TL_STR_ENTRY, "_start");

/* Then the output type, which is what creates the sections.
   Nothing may be added as input before this point. */
tl_set_output_type(s, TL_OUTPUT_EXE);

tl_add_library_path(s, "/usr/lib");
tl_add_file(s, "main.o");
tl_add_library(s, "c");

tl_output_file(s, "a.out");
tl_delete(s);

tl_add_file() takes an object, a static archive, a shared library or a GNU ld script, and works out which it is by looking at the file rather than at its name.

The four output types are TL_OUTPUT_EXE, TL_OUTPUT_DLL, TL_OUTPUT_OBJ (the equivalent of ld -r) and TL_OUTPUT_MEMORY.

Settings

Everything the linker can be told is reachable as a flag, a string or a number, so an embedder that already has its own command line translates into these rather than assembling option strings.

tl_set_flags(s, TL_F_STATIC | TL_F_NOSTDLIB, 1);   /* on  */
tl_set_flags(s, TL_F_RDYNAMIC, 0);                 /* off */

tl_set_str(s, TL_STR_SONAME, "libthing.so.1");
tl_set_str(s, TL_STR_RPATH, "$ORIGIN/../lib");
tl_set_num(s, TL_NUM_TEXT_ADDR, 0x400000);

unsigned f = tl_get_flags(s);

tl_set_options() still parses ld-style text, which is what you want for -Wl, arguments arriving from a compiler driver.

Diagnostics

Errors and warnings go to stderr unless you install a callback. Nothing in the library calls exit(), and a failed link returns a negative value rather than taking the process down with it.

static void on_diagnostic(void *opaque, const char *msg)
{
    struct my_state *st = opaque;
    st->errors++;
    log_error("%s", msg);
}

tl_set_error_func(s, &my_state, on_diagnostic);

tl_error_count() reports how many errors a state has seen so far.

Enumerating targets

A front end usually needs to know what it can ask for.

tl_Target list[TL_TARGET__COUNT];
int n = tl_targets(list, TL_TARGET__COUNT);

for (int i = 0; i < n; i++)
    printf("%s%s\n", tl_target_name(list[i]),
           tl_target_is_native(list[i]) ? "  (native)" : "");

tl_Target t = tl_target_from_name("arm64-macho");   /* by name */
tl_Target host = tl_target_native();                /* this machine */

TL_TARGET_HOST is the machine the library is running on, and the only target whose output can be linked into memory and run rather than only written out.

Linking into memory

For a compiler runtime, the interesting mode is the one that never touches the filesystem. Undefined symbols resolve against the host process and against any shared library you add, and you can inject your own pointers.

tl_State *s = tl_new(TL_TARGET_HOST);

tl_set_output_type(s, TL_OUTPUT_MEMORY);

/* Let the linked code call back into the host. */
tl_add_symbol(s, "host_log", host_log);

tl_add_file(s, "plugin.o");
tl_relocate(s);

void (*init)(void) = tl_get_symbol(s, "plugin_init");
init();

tl_run() goes one step further and calls the entry point directly, returning its exit status. tl_list_symbols() walks everything the image defines.

In-memory linking only works on the backend that matches the host, since you cannot execute arm64 code on an x86-64 machine. Check with tl_target_is_native(); tl_relocate() returns -1 on a cross target rather than producing something unrunnable.

A single header

tinylinker.h is the whole linker packed into one file. Copy it in and define TINYLINKER_IMPLEMENTATION in exactly one translation unit.

#define TINYLINKER_IMPLEMENTATION
#include "tinylinker.h"

That unit carries the 64-bit core, which links for every 64-bit target: x86_64, arm64 and riscv64 ELF, x86_64 and arm64 PE, and x86_64 and arm64 Mach-O. A link picks its architecture and object format when tl_new() creates it.

The 32-bit targets come from a second core. The core relies on each of its roughly 490 internal names being static to its own unit, and the two cores define the same names, so a project wanting i386 and arm compiles the 32-bit core in a unit of its own and names it in the unit holding the dispatcher:

/* link.c */                          /* link32.c */
#define TINYLINKER_IMPLEMENTATION     #define TINYLINKER_BACKEND
#define TINYLINKER_ALSO_CORE32        #define TINYLINKER_CORE32
#include "tinylinker.h"               #include "tinylinker.h"

Regenerate with task amalgamate after changing the sources. The test suite packs a fresh copy, builds the driver against nothing else, and links and runs a program with it.

The implementation compiles without warnings under -std=c99 -Wall -Wextra, so it can be built with the including project’s own flags, and it undefines every macro it defined once it ends. A unit that already declares the short integer names i8 to u64, isize, usize, f32 and f64 defines TL_HAVE_SHORT_TYPES before the include, since C99 refuses the same typedef twice.

The command line

tinylinker is a thin ld-like driver over the library. It exists to exercise the library from a shell and from the test harness; everything it does is available through the API.

tinylinker --target=x86_64-elf -o a.out crt1.o main.o -lc
tinylinker -r -o merged.o a.o b.o
tinylinker --run main.o -lc          # link in memory and run
tinylinker --list-targets

It understands the linker-relevant subset of the usual options: -o, -L, -l, -B, -c, -r, -shared, -static, -soname, -rdynamic and -nostdlib. Anything in -Wl,... passes straight through, which is where -e, -rpath, -Ttext, --oformat and --whole-archive live.

How it works

Upstream, the linker core selected its target with compile-time #ifdefs, so one build of it could only ever be one linker. Here a link carries its architecture as a table of functions and constants and its object format as a field, both chosen when tl_new() creates it, so every architecture and format of a pointer width compiles into the same core.

The pointer width is still fixed at build time: it picks the ELF class, and with it the layout of every ELF structure the core touches. So there are two cores, and they would collide on every internal name but for a detail of the source layout. Under ONE_SOURCE the core’s ST_FUNC and ST_DATA markers expand to static, so the entire core compiles as one translation unit with no external linkage. Each core is a small file that sets the width and includes the whole of it, exporting only a vtable per target.

/* src/backends/b_core64.c, in full */
#define PTR_SIZE 8

#include "backend.inc"

The two cores therefore coexist in one library without colliding, and tl_new() is a table lookup. The dispatcher in src/tinylinker.c is the only code in the project that knows nothing of either width.

Thread-local variables work on every target. ELF carries them in PT_TLS and the runtime resolves them from the thread pointer; Mach-O instead builds a descriptor per variable which dyld rewrites at load time, so a reference names the descriptor rather than the storage and the initial values sit in a template each thread copies. The two models have almost nothing in common beyond the __thread that produced them.

All four models are handled. Initial exec is what a compiler emits for an extern __thread by default, and it reaches the variable through a GOT slot holding an offset from the thread pointer. The linker fills that slot itself rather than leaving a relocation for the loader, since the offset is into a block the linker laid out.

Global dynamic is the indirect one: a reference names a descriptor rather than a position, and calls a resolver to turn it into an address. What that costs a linker depends on the architecture. On RISC-V the descriptor is built – this module’s number, and the offset within its block less the 0x800 that the ABI expects __tls_get_addr to add back – and the call is left alone for libc to answer. On x86-64 and arm64 the sequence is rewritten into the local-exec one instead, which is what every other linker does in an executable, and the call goes with it: nothing asks for __tls_get_addr at all.

Either way the answer is fixed to this image, so a thread-local symbol this link does not define is refused rather than resolved to something wrong: a __thread variable in a library loaded with dlopen is still out of reach, since only the loader knows where that ends up.

A code generator can also build an object in memory, through tl_add_section(), tl_define_symbol() and tl_add_reloc(), and write it with tl_output_file(). On ELF and Mach-O such an object can hold thread-local data in TL_SEC_TDATA and TL_SEC_TBSS; on Mach-O the generator writes each variable’s descriptor itself, into a TL_SEC_DATA section named .thread_vars. On x86_64 and arm64 it can also load a word from the GOT: a symbol’s address with the TL_REL_GOT_* kinds, which reaches one a shared library defines, and a thread-local variable’s locator with the TL_REL_TLS_* kinds. On ELF that locator is the initial-exec offset; on Mach-O it is the variable’s descriptor. COFF has neither form and refuses both.

Benchmarks

Numbers from one machine, which is the only honest way to give them:

Apple M4 Max, macOS 26.3.1, 16 cores, 64 GiB
Apple ld   ld-1267            LLD 22.1.7
GNU ld     2.42               LLD 17.0.6      mold 2.31.0   (Alpine 3.20, linux/arm64)

Each workload is generated once, compiled once, then linked repeatedly; only the link is timed. Five runs of fifteen timed links each, and the figure is the mean of the per-run medians, with the spread across runs beside it.

The reference linkers are handed the argument list the compiler driver would have used, with only the binary swapped, so every one is asked for the same program from the same objects. Timing clang instead would have added about twelve milliseconds of driver startup to their column and flattered ours.

macOS, Mach-O:

workload objects symbols tinylinker spread Apple ld LLD
tiny 1 12 4.19 ms 3.5% 17.21 ms 19.31 ms
small 16 768 4.70 ms 5.1% 18.00 ms 20.43 ms
medium 96 9,216 9.99 ms 3.6% 22.76 ms 27.20 ms
wide 24 18,432 12.90 ms 1.7% 23.09 ms 30.44 ms
large 320 46,080 31.37 ms 2.2% 34.98 ms 51.11 ms

Linux, ELF, in a container on the same machine:

workload objects symbols tinylinker spread GNU ld LLD mold
tiny 1 12 0.84 ms 3.8% 4.37 ms 18.19 ms 6.92 ms
small 16 768 1.89 ms 2.4% 7.77 ms 18.83 ms 8.27 ms
medium 96 9,216 10.21 ms 3.2% 40.71 ms 32.60 ms 25.47 ms
wide 24 18,432 8.00 ms 3.6% 39.56 ms 27.40 ms 13.94 ms
large 320 46,080 42.08 ms 1.4% 169.49 ms 66.89 ms 87.33 ms

What these numbers are not

The lead is real but it is mostly not doing things, and the comparison is only worth as much as that admission.

The trend is more informative than any single row. On macOS the advantage falls from 4.3x on one object to 1.1x on three hundred: most of it is startup, and Apple’s ld catches up once there is real work. On Linux it holds up better against mold and LLD than that, but the same shape is there.

task bench                          # this machine, against the linkers on it
cd tests && go run . -bench-linux -reps 15 -runs 5   # inside Linux, against mold
task bench:compare -- before:after  # one record against another

Limitations

Worth knowing before you hit them.

Testing

task test runs a Go harness with two lanes.

The native lane builds inputs with the host compiler, links them for every compiled-in target, and verifies the result by parsing its headers with Go’s debug/elf, debug/macho and debug/pe, so the same assertions hold on a host with no binutils at all. Whatever the host can execute, it executes, including the in-memory path.

The container lane runs the linker’s Linux output on real Linux, rebuilds the library from source inside a container with nothing but clang, links a hosted program against the container’s libc, cross-links every target in an image whose compiler has all of them, and runs a PE executable under Wine. Wine is the only way to find out whether the COFF reader got its relocations right, since the other two formats are checked by running their output natively or on Linux.

task test                # both lanes
task test:native         # host lane only
task test:containers     # container lane only
task test:list           # what would run here

Containers can only stand in for the host OS when that OS is Linux: there is no such thing as a macOS container, and on Windows a Linux container is what Docker Desktop gives you for this kind of work. The harness prints which case applies, and says when one lane covered ground another had to skip, so a run is never quietly narrower than it looks.

107 passed, 0 failed, 6 skipped
note: the host compiler cannot produce input for riscv64-elf; the container
lane cross-linked them instead