Overview
tinylinker reads relocatable objects, static archives, shared libraries and GNU ld scripts, and writes an executable, a shared library, or a merged relocatable object, in ELF, PE/COFF or Mach-O. It can also link straight into memory and hand you function pointers, which suits a compiler runtime doing JIT-style execution.
The whole library is one static archive of under a megabyte, or a shared object exporting 41 symbols. It has no dependencies beyond libc.
Targets
Every supported architecture and object-format pair is a separate backend compiled into the library. You pick one at runtime, so a single binary cross-links for all of them.
| Architecture | ELF | PE/COFF | Mach-O |
|---|---|---|---|
i386 | ✓ | ✓ | no |
x86_64 | ✓ | ✓ | ✓ |
arm | ✓ | no | no |
arm64 | ✓ | ✓ | ✓ |
riscv64 | ✓ | no | no |
The backend that matches the host is additionally able to link into memory and run the result. Every other backend cross-links only, which is the normal case for a compiler targeting something other than the machine it runs on.
$ tinylinker --list-targets
i386-elf
x86_64-elf
arm-elf
arm64-elf
riscv64-elf
i386-pe
x86_64-pe
arm64-pe
x86_64-macho
arm64-macho (native)
You can trim the library to the targets you actually need. Building only
x86_64-elf and arm64-macho takes the static archive from 912 KB to 213 KB.
Formats
Each target reads and writes its own object format.
| Target | Reads | Writes |
|---|---|---|
*-elf | ELF .o, ar archives, .so, ld scripts | ELF executable, .so, .o |
*-macho | Mach-O MH_OBJECT, ar archives, .dylib, .tbd | Mach-O executable, .dylib, .o |
*-pe | COFF .obj, ar archives, .dll, .def | PE executable, .dll, COFF .obj |
Every target additionally accepts ELF objects, whatever its own format is. A compiler runtime that already produces ELF can carry on doing so, whatever it is targeting.
Archives are read in both the System V layout and the BSD one macOS writes,
including its __.SYMDEF index and the way it stores long member names.
On macOS this means clang -c foo.c and
tinylinker -o foo foo.o -lSystem is all it takes. The
result is ad-hoc signed by the linker itself, because arm64 macOS will
not run an unsigned binary, and that holds wherever the link was done:
a Mach-O produced on Linux runs on macOS.
Getting started
Requires xmake and a C compiler; clang is the default.
git clone https://github.com/nickglenn/tinylinker
cd tinylinker
task build
or, without Task:
xmake f --toolchain=clang
xmake
Useful configuration:
xmake f --toolchain=gcc # or msvc / clang-cl
xmake f --kind=shared # shared library instead of static
xmake f -m debug
The sources are self-contained enough to build with nothing but a compiler, which is what the container tests do:
clang -O2 -std=c11 -DTL_VERSION='"0.1.0"' \
-Iinclude -Isrc -Isrc/core \
src/tinylinker.c src/backends/b_*.c cli/main.c -o tinylinker
The API
One header, tinylinker.h. A link is: choose a target, set options, choose an
output type, add inputs, produce output. Every declaration in it is written up
in the API reference, which is generated from the header itself.
#include <tinylinker.h>
tl_State *s = tl_new(TL_TARGET_X86_64_ELF);
/* Settings first: some of them decide what the output type does. */
tl_set_flags(s, TL_F_STATIC, 1);
tl_set_str(s, TL_STR_ENTRY, "_start");
/* Then the output type, which is what creates the sections.
Nothing may be added as input before this point. */
tl_set_output_type(s, TL_OUTPUT_EXE);
tl_add_library_path(s, "/usr/lib");
tl_add_file(s, "main.o");
tl_add_library(s, "c");
tl_output_file(s, "a.out");
tl_delete(s);
tl_add_file() takes an object, a static archive, a shared library or a GNU ld
script, and works out which it is by looking at the file rather than at its
name.
The four output types are TL_OUTPUT_EXE, TL_OUTPUT_DLL, TL_OUTPUT_OBJ
(the equivalent of ld -r) and TL_OUTPUT_MEMORY.
Settings
Everything the linker can be told is reachable as a flag, a string or a number, so an embedder that already has its own command line translates into these rather than assembling option strings.
tl_set_flags(s, TL_F_STATIC | TL_F_NOSTDLIB, 1); /* on */
tl_set_flags(s, TL_F_RDYNAMIC, 0); /* off */
tl_set_str(s, TL_STR_SONAME, "libthing.so.1");
tl_set_str(s, TL_STR_RPATH, "$ORIGIN/../lib");
tl_set_num(s, TL_NUM_TEXT_ADDR, 0x400000);
unsigned f = tl_get_flags(s);
tl_set_options() still parses ld-style text, which is what you want for
-Wl, arguments arriving from a compiler driver.
Diagnostics
Errors and warnings go to stderr unless you install a callback. Nothing in
the library calls exit(), and a failed link returns a negative value rather
than taking the process down with it.
static void on_diagnostic(void *opaque, const char *msg)
{
struct my_state *st = opaque;
st->errors++;
log_error("%s", msg);
}
tl_set_error_func(s, &my_state, on_diagnostic);
tl_error_count() reports how many errors a state has seen so far.
Enumerating targets
A front end usually needs to know what it can ask for.
tl_Target list[TL_TARGET__COUNT];
int n = tl_targets(list, TL_TARGET__COUNT);
for (int i = 0; i < n; i++)
printf("%s%s\n", tl_target_name(list[i]),
tl_target_is_native(list[i]) ? " (native)" : "");
tl_Target t = tl_target_from_name("arm64-macho"); /* by name */
tl_Target host = tl_target_native(); /* this machine */
TL_TARGET_HOST is the machine the library is running on, and the only
target whose output can be linked into memory and run rather than only
written out.
Linking into memory
For a compiler runtime, the interesting mode is the one that never touches the filesystem. Undefined symbols resolve against the host process and against any shared library you add, and you can inject your own pointers.
tl_State *s = tl_new(TL_TARGET_HOST);
tl_set_output_type(s, TL_OUTPUT_MEMORY);
/* Let the linked code call back into the host. */
tl_add_symbol(s, "host_log", host_log);
tl_add_file(s, "plugin.o");
tl_relocate(s);
void (*init)(void) = tl_get_symbol(s, "plugin_init");
init();
tl_run() goes one step further and calls the entry point directly, returning
its exit status. tl_list_symbols() walks everything the image defines.
In-memory linking only works on the backend that matches the host, since
you cannot execute arm64 code on an x86-64 machine. Check with
tl_target_is_native(); tl_relocate() returns
-1 on a cross target rather than producing something
unrunnable.
A single header
tinylinker.h is the whole linker packed into one file. Copy it in and define
TINYLINKER_IMPLEMENTATION in exactly one translation unit.
#define TINYLINKER_IMPLEMENTATION
#include "tinylinker.h"
That unit carries the 64-bit core, which links for every 64-bit target:
x86_64, arm64 and riscv64 ELF, x86_64 and arm64 PE, and x86_64 and arm64
Mach-O. A link picks its architecture and object format when tl_new()
creates it.
The 32-bit targets come from a second core. The core relies on each of its roughly 490 internal names being static to its own unit, and the two cores define the same names, so a project wanting i386 and arm compiles the 32-bit core in a unit of its own and names it in the unit holding the dispatcher:
/* link.c */ /* link32.c */
#define TINYLINKER_IMPLEMENTATION #define TINYLINKER_BACKEND
#define TINYLINKER_ALSO_CORE32 #define TINYLINKER_CORE32
#include "tinylinker.h" #include "tinylinker.h"
Regenerate with task amalgamate after changing the sources. The test suite
packs a fresh copy, builds the driver against nothing else, and links and runs
a program with it.
The implementation compiles without warnings under -std=c99 -Wall -Wextra, so
it can be built with the including project’s own flags, and it undefines every
macro it defined once it ends. A unit that already declares the short integer
names i8 to u64, isize, usize, f32 and f64 defines
TL_HAVE_SHORT_TYPES before the include, since C99 refuses the same typedef
twice.
The command line
tinylinker is a thin ld-like driver over the library. It exists to exercise
the library from a shell and from the test harness; everything it does is
available through the API.
tinylinker --target=x86_64-elf -o a.out crt1.o main.o -lc
tinylinker -r -o merged.o a.o b.o
tinylinker --run main.o -lc # link in memory and run
tinylinker --list-targets
It understands the linker-relevant subset of the usual options: -o, -L,
-l, -B, -c, -r, -shared, -static, -soname, -rdynamic and
-nostdlib. Anything in -Wl,... passes straight through, which is where
-e, -rpath, -Ttext, --oformat and --whole-archive live.
How it works
Upstream, the linker core selected its target with compile-time #ifdefs,
so one build of it could only ever be one linker. Here a link carries its
architecture as a table of functions and constants and its object format as a
field, both chosen when tl_new() creates it, so every architecture and
format of a pointer width compiles into the same core.
The pointer width is still fixed at build time: it picks the ELF class, and
with it the layout of every ELF structure the core touches. So there are two
cores, and they would collide on every internal name but for a detail of the
source layout. Under ONE_SOURCE the core’s ST_FUNC and ST_DATA markers
expand to static, so the entire core compiles as one translation unit with no
external linkage. Each core is a small file that sets the width and includes
the whole of it, exporting only a vtable per target.
/* src/backends/b_core64.c, in full */
#define PTR_SIZE 8
#include "backend.inc"
The two cores therefore coexist in one library without colliding, and
tl_new() is a table lookup. The dispatcher in src/tinylinker.c is the only
code in the project that knows nothing of either width.
Thread-local variables work on every target. ELF carries them in PT_TLS and
the runtime resolves them from the thread pointer; Mach-O instead builds a
descriptor per variable which dyld rewrites at load time, so a reference names
the descriptor rather than the storage and the initial values sit in a
template each thread copies. The two models have almost nothing in common
beyond the __thread that produced them.
All four models are handled. Initial exec is what a compiler emits for an
extern __thread by default, and it reaches the variable through a GOT slot
holding an offset from the thread pointer. The linker fills that slot itself
rather than leaving a relocation for the loader, since the offset is into a
block the linker laid out.
Global dynamic is the indirect one: a reference names a descriptor rather than
a position, and calls a resolver to turn it into an address. What that costs a
linker depends on the architecture. On RISC-V the descriptor is built – this
module’s number, and the offset within its block less the 0x800 that the ABI
expects __tls_get_addr to add back – and the call is left alone for libc to
answer. On x86-64 and arm64 the sequence is rewritten into the local-exec one
instead, which is what every other linker does in an executable, and the call
goes with it: nothing asks for __tls_get_addr at all.
Either way the answer is fixed to this image, so a thread-local symbol this
link does not define is refused rather than resolved to something wrong: a
__thread variable in a library loaded with dlopen is still out of reach,
since only the loader knows where that ends up.
A code generator can also build an object in memory, through
tl_add_section(), tl_define_symbol() and tl_add_reloc(), and write it with
tl_output_file(). On ELF and Mach-O such an object can hold thread-local
data in TL_SEC_TDATA and TL_SEC_TBSS; on Mach-O the generator writes each
variable’s descriptor itself, into a TL_SEC_DATA section named
.thread_vars. On x86_64 and arm64 it can also load a word from the GOT: a
symbol’s address with the TL_REL_GOT_* kinds, which reaches one a shared
library defines, and a thread-local variable’s locator with the TL_REL_TLS_*
kinds. On ELF that locator is the initial-exec offset; on Mach-O it is the
variable’s descriptor. COFF has neither form and refuses both.
Benchmarks
Numbers from one machine, which is the only honest way to give them:
Apple M4 Max, macOS 26.3.1, 16 cores, 64 GiB
Apple ld ld-1267 LLD 22.1.7
GNU ld 2.42 LLD 17.0.6 mold 2.31.0 (Alpine 3.20, linux/arm64)
Each workload is generated once, compiled once, then linked repeatedly; only the link is timed. Five runs of fifteen timed links each, and the figure is the mean of the per-run medians, with the spread across runs beside it.
The reference linkers are handed the argument list the compiler driver would
have used, with only the binary swapped, so every one is asked for the same
program from the same objects. Timing clang instead would have added about
twelve milliseconds of driver startup to their column and flattered ours.
macOS, Mach-O:
| workload | objects | symbols | tinylinker | spread | Apple ld | LLD |
|---|---|---|---|---|---|---|
| tiny | 1 | 12 | 4.19 ms | 3.5% | 17.21 ms | 19.31 ms |
| small | 16 | 768 | 4.70 ms | 5.1% | 18.00 ms | 20.43 ms |
| medium | 96 | 9,216 | 9.99 ms | 3.6% | 22.76 ms | 27.20 ms |
| wide | 24 | 18,432 | 12.90 ms | 1.7% | 23.09 ms | 30.44 ms |
| large | 320 | 46,080 | 31.37 ms | 2.2% | 34.98 ms | 51.11 ms |
Linux, ELF, in a container on the same machine:
| workload | objects | symbols | tinylinker | spread | GNU ld | LLD | mold |
|---|---|---|---|---|---|---|---|
| tiny | 1 | 12 | 0.84 ms | 3.8% | 4.37 ms | 18.19 ms | 6.92 ms |
| small | 16 | 768 | 1.89 ms | 2.4% | 7.77 ms | 18.83 ms | 8.27 ms |
| medium | 96 | 9,216 | 10.21 ms | 3.2% | 40.71 ms | 32.60 ms | 25.47 ms |
| wide | 24 | 18,432 | 8.00 ms | 3.6% | 39.56 ms | 27.40 ms | 13.94 ms |
| large | 320 | 46,080 | 42.08 ms | 1.4% | 169.49 ms | 66.89 ms | 87.33 ms |
What these numbers are not
The lead is real but it is mostly not doing things, and the comparison is only worth as much as that admission.
- No link-time optimisation. An object built with
-fltois LLVM bitcode, and this linker refuses it. LTO would mean linking against libLTO and driving its symbol-resolution protocol; the library is under a megabyte and libLTO is not. - No symbol table in the output. On the
largeworkload the.textsections agree to within 0.02% — 2,088,004 bytes against mold’s 2,088,384 — but mold also writes a 1.1 MB.symtaband a 475 KB.strtab. That is 1.6 MB of output this linker never produces, and not producing it is part of why it finishes sooner. The 1.2 MB.eh_frameused to be on that list and is not any more: the frames are carried through now, and the figures above were measured with them, which cost between five and eight per cent. Nothing here does identical code folding either. - Single threaded. mold’s whole design is parallelism, and this is one thread. On a machine with more cores than this one the gap narrows.
- The workload is generated, not a real program: units that call into each other, with a fixed ratio of functions to data. It exercises symbol resolution and relocation at scale, not the shape of any particular build.
The trend is more informative than any single row. On macOS the advantage
falls from 4.3x on one object to 1.1x on three hundred: most of it is startup,
and Apple’s ld catches up once there is real work. On Linux it holds up
better against mold and LLD than that, but the same shape is there.
task bench # this machine, against the linkers on it
cd tests && go run . -bench-linux -reps 15 -runs 5 # inside Linux, against mold
task bench:compare -- before:after # one record against another
Limitations
Worth knowing before you hit them.
- Thread-local storage from a shared library. A thread-local reference
needs the variable to be defined in the image being linked, because the
position it resolves to is one this linker chose. A
__threadvariable that comes from a.soopened withdlopenis an error rather than a wrong address: only the loader knows where that ends up. All four models resolve for a variable the link does define. - No unwind tables on PE. COFF
.pdataand.xdataare dropped on input, because the entries in them are image-relative addresses and this linker has no relocation that means “distance from the image base”. ELF and Mach-O both carry their frame descriptions through and can be unwound; PE cannot, so C++ exceptions and precise stack walking do not survive there. - Mach-O
__compact_unwindis dropped. The__unwind_infoindex that a loader reads it through is a two-level compressed table this linker does not build. Unwinding still works:__eh_frameis kept, and libunwind falls back to it when there is no compact table, which is whatcorpus/unwindchecks. The fallback is slower to look up, not less correct. - COFF has no GOT. An ELF object using
R_X86_64_GOTPCRELorR_AARCH64_ADR_GOT_PAGEcannot be re-emitted as a COFF object by-r, because COFF has nothing that means the same thing. - A symbol difference across sections needs a final address. Two symbols
in different output sections are subtracted once the image is laid out,
which an executable or a library gets and a
-robject does not. So that case is still refused for relocatable output. - No compiler runtime is shipped. Helpers such as
__divdi3are not provided. A missing helper surfaces as an ordinary unresolved-symbol error. If you have one, pointtl_set_lib_path()at the directory holding it; the file is looked for under the name TinyCC gave it,libtcc1.a, which is the only established convention for such a thing and the last place that name survives here. DefineTLI_RUNTIME_LIBto look for another. - Standalone
-b/-btneed a constructor you supply, because building one would take a compiler. The runtime context is still emitted and exported as__rt_info; hand it to__bt_init()yourself. In-memory linking is unaffected.
Testing
task test runs a Go harness with two lanes.
The native lane builds inputs with the host compiler, links them for every
compiled-in target, and verifies the result by parsing its headers with Go’s
debug/elf, debug/macho and debug/pe, so the same assertions hold on a
host with no binutils at all. Whatever the host can execute, it executes,
including the in-memory path.
The container lane runs the linker’s Linux output on real Linux, rebuilds the library from source inside a container with nothing but clang, links a hosted program against the container’s libc, cross-links every target in an image whose compiler has all of them, and runs a PE executable under Wine. Wine is the only way to find out whether the COFF reader got its relocations right, since the other two formats are checked by running their output natively or on Linux.
task test # both lanes
task test:native # host lane only
task test:containers # container lane only
task test:list # what would run here
Containers can only stand in for the host OS when that OS is Linux: there is no such thing as a macOS container, and on Windows a Linux container is what Docker Desktop gives you for this kind of work. The harness prints which case applies, and says when one lane covered ground another had to skip, so a run is never quietly narrower than it looks.
107 passed, 0 failed, 6 skipped
note: the host compiler cannot produce input for riscv64-elf; the container
lane cross-linked them instead