Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

LLVM Target Customization for capOS

Deep research report on creating custom LLVM/Rust/Go targets for a capability-based OS.

Status as of 2026-04-22: capOS still builds kernel and userspace with x86_64-unknown-none plus linker-script/build flags. A checked-in x86_64-unknown-capos custom target does not exist yet. Since this report was first written, PT_TLS parsing, userspace TLS block setup, FS-base save/restore, the VirtualMemory capability, and a #[thread_local] QEMU smoke have landed. Thread creation, a user-controlled FS-base syscall, futexes, a timer capability, and a Go port remain future work.

Table of Contents

  1. Custom OS Target Triple
  2. Calling Conventions
  3. Relocations
  4. TLS (Thread-Local Storage) Models
  5. Rust Target Specification
  6. Go Runtime Requirements
  7. Relevance to capOS

1. Custom OS Target Triple

Target Triple Format

LLVM target triples follow the format <arch>-<vendor>-<os> or <arch>-<vendor>-<os>-<env>:

  • arch: x86_64, aarch64, riscv64gc, etc.
  • vendor: unknown, apple, pc, etc. (often unknown for custom OSes)
  • os: linux, none, redox, hermit, fuchsia, etc.
  • env (optional): gnu, musl, eabi, etc.

For capOS, the eventual userspace target triple should be x86_64-unknown-capos. The kernel should keep using a freestanding target (x86_64-unknown-none) unless a kernel-specific target file becomes useful for build hygiene.

What LLVM Needs

LLVM’s target description consists of:

  1. Target machine: Architecture (instruction set, register file, calling conventions). x86_64 already exists in LLVM.
  2. Object format: ELF, COFF, Mach-O. capOS uses ELF.
  3. Relocation model: static, PIC, PIE, dynamic-no-pic.
  4. Code model: small, kernel, medium, large.
  5. OS-specific ABI details: Stack alignment, calling convention defaults, TLS model, exception handling mechanism.

LLVM does NOT need kernel-level knowledge of your OS. It needs to know how to generate correct object code for the target environment. The OS name in the triple primarily affects:

  • Default calling convention selection
  • Default relocation model
  • TLS model selection
  • Object file format and flags
  • C library assumptions (relevant for C compilation, less for Rust no_std)

Creating a New OS in LLVM (Upstream Path)

To add capos as a recognized OS in LLVM itself:

  1. Add the OS to llvm/include/llvm/TargetParser/Triple.h (the OSType enum)
  2. Add string parsing in llvm/lib/TargetParser/Triple.cpp
  3. Define ABI defaults in the relevant target (llvm/lib/Target/X86/)
  4. Update Clang’s driver for the new OS (clang/lib/Driver/ToolChains/, clang/lib/Basic/Targets/)

This is significant upstream work and not necessary initially. The pragmatic path is using Rust’s custom target JSON mechanism (see Section 5).

What Other OSes Do

OSLLVM statusApproach
RedoxUpstream in Rust; no dedicated LLVM OS enum in current LLVMFull triple x86_64-unknown-redox, Tier 2 in Rust
HermitUpstream in LLVM and Rustx86_64-unknown-hermit, Tier 3, unikernel
FuchsiaUpstream in LLVM and Rustx86_64-unknown-fuchsia, Tier 2
TheseusCustom target JSONUses x86_64-unknown-theseus JSON spec, not upstream
Blog OS (phil-opp)Custom target JSONUses JSON target spec, targets x86_64-unknown-none base
seL4/RobigaliaCustom target JSONModified from x86_64-unknown-none

Recommendation for capOS: keep the kernel on x86_64-unknown-none. Introduce a userspace-only custom target JSON when cfg(target_os = "capos") or toolchain packaging becomes valuable. Do not upstream a capos OS triple until the userspace ABI is stable.


2. Calling Conventions

LLVM Calling Conventions

LLVM supports numerous calling conventions. The ones relevant to capOS:

CCLLVM IDDescriptionRelevance
C0Default C calling convention (System V AMD64 ABI on x86_64)Primary for interop
Fast8Optimized for internal use, passes in registersRust internal use
Cold9Rarely-called functions, callee-save heavyError paths
GHC10Glasgow Haskell Compiler, everything in registersNot relevant
HiPE11Erlang HiPE, similar to GHCNot relevant
WebKit JS12JavaScript JITNot relevant
AnyReg13Dynamic register allocationJIT compilers
PreserveMost14Caller saves almost nothingInterrupt handlers
PreserveAll15Caller saves nothingContext switches
Swift16Swift self/error registersNot relevant
CXX_FAST_TLS17C++ TLS access optimizationTLS wrappers
X86_StdCall64Windows stdcallNot relevant
X86_FastCall65Windows fastcallNot relevant
X86_RegCall95Register-based callingPerformance-critical code
X86_INTR83x86 interrupt handlerIDT handlers
Win6479Windows x64 calling conventionNot relevant

System V AMD64 ABI (The Default for capOS)

On x86_64, the System V AMD64 ABI (CC 0, “C”) is the standard:

  • Integer args: RDI, RSI, RDX, RCX, R8, R9
  • Float args: XMM0-XMM7
  • Return: RAX (integer), XMM0 (float)
  • Caller-saved: RAX, RCX, RDX, RSI, RDI, R8-R11, XMM0-XMM15
  • Callee-saved: RBX, RBP, R12-R15
  • Stack alignment: 16-byte at call site
  • Red zone: 128 bytes below RSP (unavailable in kernel mode)

capOS already uses this convention – the syscall handler in kernel/src/arch/x86_64/syscall.rs maps syscall registers to System V registers before calling syscall_handler.

Customizing for a New OS Target

For a custom OS, calling convention customization is usually minimal:

  1. Kernel code: Disable the red zone (capOS already does this via x86_64-unknown-none which sets "disable-redzone": true). The red zone is unsafe in interrupt/syscall contexts.

  2. Userspace code: Standard System V ABI is fine. The red zone is safe in userspace.

  3. Syscall convention: This is an OS design choice, not an LLVM CC. capOS uses: RAX=syscall number, RDI-R9=args (matching System V for easy dispatch). Linux uses a slightly different register mapping (R10 instead of RCX for arg4, because SYSCALL clobbers RCX).

  4. Interrupt handlers: Use X86_INTR (CC 83) or manual save/restore. capOS currently uses manual asm stubs.

Cross-Language Interop Implications

LanguagesConventionNotes
Rust <-> RustRust ABI (unstable)Internal to a crate, not stable across crates
Rust <-> Cextern "C" (System V)Stable, well-defined. Used for libcapos API
Rust <-> GoComplex (see Section 6)Go has its own internal ABI (ABIInternal)
C <-> Goextern "C" via cgoGo’s cgo bridge, heavy overhead
Any <-> KernelSyscall conventionRegister-based, OS-defined, not a CC

Key point: The System V AMD64 ABI is the lingua franca. All languages can produce extern "C" functions. capOS should standardize on System V for all cross-language boundaries and capability invocations.

Go’s internal ABI (ABIInternal, using R14 as the g register) is different from System V. Go functions called from outside Go must go through a trampoline. This is handled by the Go runtime, not something capOS needs to solve at the LLVM level.


3. Relocations

LLVM Relocation Models

ModelFlagDescription
static-relocation-model=staticAll addresses resolved at link time. No GOT/PLT.
pic-relocation-model=picPosition-independent code. Uses GOT for globals, PLT for calls.
dynamic-no-pic-relocation-model=dynamic-no-picLike static but with dynamic linking support (macOS legacy).
ropi-relocation-model=ropiRead-only position-independent (ARM embedded).
rwpi-relocation-model=rwpiRead-write position-independent (ARM embedded).
ropi-rwpi-relocation-model=ropi-rwpiBoth ROPI and RWPI (ARM embedded).

Code Models (x86_64)

ModelFlagAddress RangeUse Case
small-code-model=small0 to 2GBUserspace default
kernel-code-model=kernelTop 2GB (negative 32-bit)Higher-half kernel
medium-code-model=mediumCode in low 2GB, data anywhereLarge data sets
large-code-model=largeNo assumptionsMaximum flexibility, worst performance

What capOS Currently Uses

From .cargo/config.toml:

[target.x86_64-unknown-none]
rustflags = ["-C", "link-arg=-Tkernel/linker-x86_64.ld", "-C", "code-model=kernel", "-C", "relocation-model=static"]
  • Kernel: code-model=kernel + relocation-model=static. Correct for a higher-half kernel at 0xffffffff80000000. All kernel symbols are in the top 2GB of virtual address space, so 32-bit sign-extended addressing works.

  • Init/demos/capos-rt userspace: The standalone userspace crates also target x86_64-unknown-none, pass -Crelocation-model=static, and select their linker scripts through per-crate build.rs files. The binaries are loaded at 0x200000. The pinned local toolchain (rustc 1.97.0-nightly, LLVM 22.1.2) prints x86_64-unknown-none with llvm-target = "x86_64-unknown-none-elf", code-model = "kernel", soft-float ABI, inline stack probes, and static PIE-capable defaults. A future x86_64-unknown-capos userspace target should set code-model = "small" explicitly instead of inheriting the freestanding kernel-oriented default.

Kernel vs. Userspace Requirements

Kernel:

  • Static relocations, kernel code model.
  • No PIC overhead needed – the kernel is loaded at a known address.
  • The linker script places everything in the higher half.
  • This is the correct and standard approach (Linux kernel does the same).

Userspace (current – static binaries):

  • Static relocations. A future custom userspace target should choose the small code model explicitly.
  • Simple, no runtime relocator needed.
  • Binary is loaded at a fixed address (0x200000).
  • Works perfectly for single-binary-per-address-space.

Userspace (future – if shared libraries or ASLR desired):

  • PIE (Position-Independent Executable) = PIC + static linking.
  • Requires a dynamic loader or kernel-side relocator.
  • Enables ASLR (Address Space Layout Randomization) for security.
  • Adds GOT indirection overhead (typically < 5% performance impact).

Position-Independent Code in a Capability Context

PIC/PIE is relevant to capOS for several reasons:

  1. ASLR: PIE enables loading binaries at random addresses, making ROP attacks harder. Even in a capability system, defense-in-depth matters.

  2. Shared libraries: If capOS ever supports shared objects (e.g., a shared libcapos.so), PIC is required for the shared library.

  3. WASI/Wasm: Not relevant – Wasm has its own memory model.

  4. Multiple instances: With static linking, two instances of the same binary can share read-only pages (text, rodata) if loaded at the same address. PIC/PIE allows sharing even at different addresses (copy-on-write for the GOT).

Recommendation for capOS: Keep static relocation for now. Consider PIE for userspace when implementing ASLR (after threading and IPC are stable). The kernel should remain static forever.


4. TLS (Thread-Local Storage) Models

LLVM TLS Models

LLVM supports four TLS models, in order from most dynamic to most constrained:

ModelDescriptionRuntime RequirementPerformance
general-dynamicAny module, any timeFull __tls_get_addr via dynamic linkerSlowest (function call per access)
local-dynamicSame module, any time__tls_get_addr for module base, then offsetSlow (one call per module per thread)
initial-execOnly modules loaded at startupGOT slot populated by dynamic linkerFast (one memory load)
local-execMain executable onlyDirect FS/GS offset, known at link timeFastest (single instruction)

How TLS Works on x86_64

On x86_64, TLS is accessed via the FS segment register:

  1. The OS sets the FS base address for each thread (via MSR_FS_BASE or arch_prctl(ARCH_SET_FS)).
  2. TLS variables are accessed as offsets from FS base:
    • local-exec: mov %fs:OFFSET, %rax (offset known at link time)
    • initial-exec: mov %fs:0, %rax; mov GOT_OFFSET(%rax), %rcx; mov %fs:(%rcx), %rdx
    • general-dynamic: call __tls_get_addr (returns pointer to TLS block)

Which Model for capOS?

Kernel:

  • The kernel does not use compiler TLS. Current TLS support is for loaded userspace ELF images only.
  • For SMP: per-CPU data via GS segment register (the standard approach). Set MSR_GS_BASE on each CPU to point to a PerCpu struct. swapgs on kernel entry switches between user and kernel GS base.
  • Kernel TLS model: Not applicable (per-CPU data is accessed via GS, not the compiler’s TLS mechanism).

Userspace (static binaries, no dynamic linker):

  • local-exec is the only correct choice. There’s no dynamic linker to resolve TLS relocations, so general-dynamic and initial-exec won’t work.
  • Implemented for the current single-threaded process model: the ELF parser records PT_TLS, the loader maps a Variant II TLS block plus TCB self pointer, and the scheduler saves/restores FS base on context switch.
  • Still missing for future threading and Go: a syscall or capability-authorized operation equivalent to arch_prctl(ARCH_SET_FS) so a runtime can set each OS thread’s FS base itself.

Userspace (with dynamic linker, future):

  • initial-exec for the main executable and preloaded libraries.
  • general-dynamic for dlopen()-loaded libraries.
  • Requires implementing __tls_get_addr in the dynamic linker.

TLS Initialization Sequence

For a statically-linked userspace binary with local-exec TLS:

1. Kernel creates thread
2. Kernel allocates TLS block (size from ELF TLS program header)
3. Kernel copies .tdata (initialized TLS) into TLS block
4. Kernel zeros .tbss (uninitialized TLS) in TLS block
5. Kernel sets FS base = TLS block address (writes MSR_FS_BASE)
6. Thread starts executing; %fs:OFFSET accesses TLS directly

The ELF file contains two TLS sections:

  • .tdata (PT_TLS segment, initialized thread-local data)
  • .tbss (zero-initialized thread-local data, like .bss but per-thread)

The PT_TLS program header tells the loader:

  • Virtual address and file offset of .tdata
  • p_memsz = total TLS size (including .tbss)
  • p_filesz = size of .tdata only
  • p_align = required alignment

FS/GS Base Register Usage Plan

RegisterUsed ByPurpose
FSUserspace threadsThread-local storage (set per-thread by kernel)
GSKernel (via swapgs)Per-CPU data (set per-CPU during boot)

This is the standard Linux convention and what Go expects (Go uses arch_prctl(ARCH_SET_FS) to set the FS base for each OS thread).

What capOS Has and Still Needs

  1. Implemented: parse PT_TLS in capos-lib/src/elf.rs.
  2. Implemented: allocate/map a TLS block during process image load in kernel/src/spawn.rs.
  3. Implemented: copy .tdata, zero .tbss, and write the TCB self pointer for the current Variant II static TLS layout.
  4. Implemented: save/restore FS base through kernel/src/sched.rs and kernel/src/arch/x86_64/tls.rs.
  5. Still needed: arch_prctl(ARCH_SET_FS) equivalent for Go settls() and future multi-threaded userspace.

5. Rust Target Specification

How Custom Targets Work

Rust supports custom targets via JSON specification files. The workflow:

  1. Create a <target-name>.json file
  2. Pass it to rustc: --target path/to/x86_64-unknown-capos.json
  3. Use with cargo via -Zbuild-std to build core/alloc/std from source

Target lookup priority:

  1. Built-in target names
  2. File path (if the target string contains / or .json)
  3. RUST_TARGET_PATH environment variable directories

The Rust target JSON schema is explicitly unstable. Generate examples from the pinned compiler with rustc -Z unstable-options --print target-spec-json and validate against that same compiler’s target-spec-json-schema before checking in a target file.

Viewing Existing Specs

# Print the JSON spec for a built-in target:
rustc +nightly -Z unstable-options --target=x86_64-unknown-none --print target-spec-json

# Print the JSON schema for all available fields:
rustc +nightly -Z unstable-options --print target-spec-json-schema

Example: x86_64-unknown-capos Kernel Target

Based on the current x86_64-unknown-none target, with capOS-specific adjustments. This is a sketch; regenerate from the pinned rustc schema before using it.

{
    "llvm-target": "x86_64-unknown-none-elf",
    "metadata": {
        "description": "capOS kernel (x86_64)",
        "tier": 3,
        "host_tools": false,
        "std": false
    },
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
    "arch": "x86_64",
    "cpu": "x86-64",
    "target-endian": "little",
    "target-pointer-width": 64,
    "target-c-int-width": 32,
    "os": "none",
    "env": "",
    "vendor": "unknown",
    "linker-flavor": "gnu-lld",
    "linker": "rust-lld",
    "pre-link-args": {
        "gnu-lld": ["-Tkernel/linker-x86_64.ld"]
    },
    "features": "-mmx,-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-avx,-avx2,+soft-float",
    "disable-redzone": true,
    "panic-strategy": "abort",
    "code-model": "kernel",
    "relocation-model": "static",
    "rustc-abi": "softfloat",
    "executables": true,
    "exe-suffix": "",
    "has-thread-local": false,
    "position-independent-executables": false,
    "static-position-independent-executables": false,
    "plt-by-default": false,
    "max-atomic-width": 64,
    "stack-probes": { "kind": "inline" }
}

Example: x86_64-unknown-capos Userspace Target

{
    "llvm-target": "x86_64-unknown-none-elf",
    "metadata": {
        "description": "capOS userspace (x86_64)",
        "tier": 3,
        "host_tools": false,
        "std": false
    },
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128",
    "arch": "x86_64",
    "cpu": "x86-64",
    "target-endian": "little",
    "target-pointer-width": 64,
    "target-c-int-width": 32,
    "os": "capos",
    "env": "",
    "vendor": "unknown",
    "linker-flavor": "gnu-lld",
    "linker": "rust-lld",
    "pre-link-args": {
        "gnu-lld": ["-Tinit/linker.ld"]
    },
    "features": "-mmx,-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-avx,-avx2,+soft-float",
    "disable-redzone": false,
    "panic-strategy": "abort",
    "code-model": "small",
    "relocation-model": "static",
    "rustc-abi": "softfloat",
    "executables": true,
    "exe-suffix": "",
    "has-thread-local": true,
    "position-independent-executables": false,
    "static-position-independent-executables": false,
    "max-atomic-width": 64,
    "plt-by-default": false,
    "stack-probes": { "kind": "inline" },
    "tls-model": "local-exec"
}

Key JSON Fields

FieldPurposeTypical Values
llvm-targetLLVM triple for code generationx86_64-unknown-none-elf (reuse existing backend)
osOS name (affects cfg(target_os = "..."))"none", "capos", "linux"
archArchitecture name"x86_64", "aarch64"
data-layoutLLVM data layout stringCopy from same-arch target
linker-flavorWhich linker to use"gnu-lld", "gcc", "msvc"
linkerLinker binary"rust-lld", "ld.lld"
featuresCPU features to enable/disableDisable SIMD/FPU until context switching saves that state
disable-redzoneDisable System V red zonetrue for kernel, false for userspace
code-modelLLVM code model"kernel", "small"
relocation-modelLLVM relocation model"static", "pic"
panic-strategyHow to handle panics"abort", "unwind"
has-thread-localEnable #[thread_local]true for userspace now that PT_TLS/FS base works
tls-modelDefault TLS model"local-exec" for static binaries
max-atomic-widthLargest atomic type (bits)64 for x86_64
pre-link-argsArguments passed to linker before user argsLinker script path
position-independent-executablesGenerate PIE by defaultfalse for now
exe-suffixExecutable file extension"" for ELF
stack-probesStack overflow detection mechanism{"kind": "inline"} in the current freestanding x86_64 spec

no_std vs std Support Path

Current state: capOS uses no_std + alloc. This works with any target, including x86_64-unknown-none.

Path to std support (what Redox, Hermit, and Fuchsia did):

  1. Phase 1: Custom target with os: "capos" (current report). Use -Zbuild-std=core,alloc to build core and alloc. No std.

  2. Phase 2: Add capOS to Rust’s std library. This requires:

    • Adding mod capos under library/std/src/sys/ with OS-specific implementations of: filesystem, networking, threads, time, stdio, process spawning, etc.
    • Each of these maps to capOS capabilities
    • Use cfg(target_os = "capos") throughout std
    • Build with -Zbuild-std=std
  3. Phase 3: Upstream the target (optional). Submit the target spec and std implementations to the Rust project. Requires sustained maintenance.

What Redox did: Redox implemented a full POSIX-like userspace (relibc) and added std support by implementing the sys module in terms of relibc syscalls. This made Redox a Tier 2 target with pre-built std artifacts.

What Hermit did: Hermit is a unikernel, so std is implemented directly in terms of Hermit’s kernel-level APIs. Tier 3, community maintained.

What Fuchsia did: Fuchsia implemented std using Fuchsia’s native zircon syscalls (handles, channels, VMOs – similar in spirit to capabilities). Tier 2.

Recommendation for capOS: Stay on no_std + alloc with the custom target JSON. std support is a large effort that should wait until the syscall surface is stable and threading works. When the time comes, Fuchsia’s approach (std over native capability syscalls) is the best model, since Fuchsia’s handle-based API is conceptually close to capOS’s capabilities.

Other OS Projects Reference

OSTargetTierstdApproach
Redoxx86_64-unknown-redox2Yesrelibc (custom libc) over Redox syscalls
Hermitx86_64-unknown-hermit3Yesstd directly over kernel API
Fuchsiax86_64-unknown-fuchsia2Yesstd over zircon handles (capability-like)
Theseusx86_64-unknown-theseusN/ANoCustom JSON, no_std, research OS
Blog OSCustom JSONN/ANoBased on x86_64-unknown-none
MOROSCustom JSONN/ANoSimple hobby OS

6. Go Runtime Requirements

Go’s Runtime Architecture

Go’s runtime is essentially a userspace operating system. It manages goroutine scheduling, garbage collection, memory allocation, and I/O multiplexing. The runtime interfaces with the actual OS through a narrow set of functions that each GOOS must implement.

Minimum OS Interface for a Go Port

Based on analysis of runtime/os_linux.go, runtime/os_plan9.go, and runtime/os_js.go, here is the minimum interface:

Tier 1: Absolute Minimum (single-threaded, like GOOS=js)

These functions are needed for “Hello, World!”:

func osinit()                                    // OS initialization
func write1(fd uintptr, p unsafe.Pointer, n int32) int32  // stdout/stderr output
func exit(code int32)                            // process termination
func usleep(usec uint32)                         // sleep (can be no-op initially)
func readRandom(r []byte) int                    // random data (for maps, etc.)
func goenvs()                                    // environment variables
func mpreinit(mp *m)                             // pre-init new M on parent thread
func minit()                                     // init new M on its own thread
func unminit()                                   // undo minit
func mdestroy(mp *m)                             // destroy M resources

Plus memory management (in runtime/mem_*.go):

func sysAllocOS(n uintptr) unsafe.Pointer        // allocate memory (mmap)
func sysFreeOS(v unsafe.Pointer, n uintptr)       // free memory (munmap)
func sysReserveOS(v unsafe.Pointer, n uintptr) unsafe.Pointer  // reserve VA range
func sysMapOS(v unsafe.Pointer, n uintptr)        // commit reserved pages
func sysUsedOS(v unsafe.Pointer, n uintptr)       // mark as used
func sysUnusedOS(v unsafe.Pointer, n uintptr)     // mark as unused (madvise)
func sysFaultOS(v unsafe.Pointer, n uintptr)      // remove access
func sysHugePageOS(v unsafe.Pointer, n uintptr)   // hint: use huge pages

Tier 2: Multi-threaded (real goroutines)

func newosproc(mp *m)                            // create OS thread (clone)
func exitThread(wait *atomic.Uint32)             // exit current thread
func futexsleep(addr *uint32, val uint32, ns int64)  // futex wait
func futexwakeup(addr *uint32, cnt uint32)        // futex wake
func settls()                                     // set FS base for TLS
func nanotime1() int64                            // monotonic nanosecond clock
func walltime() (sec int64, nsec int32)           // wall clock time
func osyield()                                    // sched_yield

Tier 3: Full Runtime (signals, profiling, network poller)

func sigaction(sig uint32, new *sigactiont, old *sigactiont)
func signalM(mp *m, sig int)                      // send signal to thread
func setitimer(mode int32, new *itimerval, old *itimerval)
func netpollopen(fd uintptr, pd *pollDesc) uintptr
func netpoll(delta int64) (gList, int32)
func netpollBreak()

Linux Syscalls Used by Go Runtime (Complete List)

From runtime/sys_linux_amd64.s:

Syscall#Go WrappercapOS Equivalent
read0runtime.readStore cap
write1runtime.write1Console cap
close3runtime.closefdCap drop
mmap9runtime.sysMmapVirtualMemory cap
munmap11runtime.sysMunmapVirtualMemory.unmap
brk12runtime.sbrk0VirtualMemory cap
rt_sigaction13runtime.rt_sigactionSignal cap (future)
rt_sigprocmask14runtime.rtsigprocmaskSignal cap (future)
sched_yield24runtime.osyieldsys_yield
mincore27runtime.mincoreVirtualMemory.query
madvise28runtime.madviseFuture VirtualMemory decommit/query semantics, or unmap/remap policy
nanosleep35runtime.usleepTimer cap
setitimer38runtime.setitimerTimer cap
getpid39runtime.getpidProcess info
clone56runtime.cloneThread cap
exit60runtime.exitsys_exit
sigaltstack131runtime.sigaltstackNot needed initially
arch_prctl158runtime.settlssys_arch_prctl (set FS base)
gettid186runtime.gettidThread info
futex202runtime.futexsys_futex
sched_getaffinity204runtime.sched_getaffinityCPU info
timer_create222runtime.timer_createTimer cap
timer_settime223runtime.timer_settimeTimer cap
timer_delete226runtime.timer_deleteTimer cap
clock_gettime228runtime.nanotime1Timer cap
exit_group231runtime.exitsys_exit
tgkill234runtime.tgkillThread signal (future)
openat257runtime.openNamespace cap
pipe2293runtime.pipe2IPC cap

Go’s TLS Model

Go uses arch_prctl(ARCH_SET_FS, addr) to set the FS segment base for each OS thread. The convention:

  • FS base points to the thread’s m.tls array
  • Goroutine pointer g is stored at -8(FS) (ELF TLS convention)
  • In Go’s ABIInternal, R14 is cached as the g register for performance
  • On signal entry or thread start, g is loaded from TLS into R14

Go does NOT use the compiler’s TLS mechanisms (no __thread or thread_local!). It manages TLS entirely in its own runtime via the FS register.

For capOS, this means the kernel needs:

  1. arch_prctl(ARCH_SET_FS) equivalent syscall
  2. The kernel must save/restore FS base on context switch
  3. Each thread’s FS base must be independently settable

Adding GOOS=capos to Go

Files that need to be created/modified in a Go fork:

src/runtime/
    os_capos.go           // osinit, newosproc, futexsleep, etc.
    os_capos_amd64.go     // arch-specific OS functions
    sys_capos_amd64.s     // syscall wrappers in assembly
    mem_capos.go          // sysAlloc/sysFree/etc. over VirtualMemory cap
    signal_capos.go       // signal stubs (no real signals initially)
    stubs_capos.go        // misc stubs
    netpoll_capos.go      // network poller (stub initially)
    defs_capos.go         // OS-level constants
    vdso_capos.go         // VDSO stubs (no VDSO)

src/syscall/
    syscall_capos.go      // Go's syscall package
    zsyscall_capos_amd64.go

src/internal/platform/
    (modifications to supported.go, zosarch.go)

src/cmd/dist/
    (modifications to add capOS to known OS list)

Estimated: ~2000-3000 lines for Phase 1 (single-threaded).

Feasibility Assessment

FeatureDifficultyBlocked On
Hello World (write + exit)EasyConsole capability plus exit syscall
Memory allocator (mmap)MediumVirtualMemory capability exists; Go glue and any missing query/decommit semantics remain
Single-threaded goroutines (M=1)MediumVirtualMemory cap + timer
Multi-threaded (real threads)HardKernel thread support, futex, runtime-controlled FS base
Network pollerHardAsync cap invocation, networking stack
Signal-based preemptionHardSignal delivery mechanism
Full stdlibVery HardPOSIX layer or native cap wrappers

7. Relevance to capOS

Practical Scope of Work

Phase 1: Custom Target JSON (Low effort, high value)

What: Create a userspace x86_64-unknown-capos.json target spec. Keep the kernel on x86_64-unknown-none unless a kernel JSON proves useful.

Why: Replaces the current approach of using x86_64-unknown-none with rustflags overrides. Makes the build cleaner, enables cfg(target_os = "capos") for conditional compilation, and is the foundation for everything else.

Effort: 1-2 hours for an initial file, plus recurring maintenance because Rust target JSON fields are not stable.

Blockers: None. Not required for the current no_std runtime path.

Phase 2: TLS Support (mostly landed, required for Go)

What: Parse PT_TLS from ELF, allocate per-thread TLS blocks, set FS base on context switch, add arch_prctl-equivalent syscall.

Why: Required for Go runtime (Go’s settls() sets FS base), for Rust #[thread_local] in userspace, and for C’s __thread.

Current state: PT_TLS parsing, static TLS mapping, FS-base context-switch state, and a Rust #[thread_local] smoke are implemented. Remaining work is the runtime-controlled FS-base operation and the thread model that makes it per-thread rather than per-process.

Blockers: Thread support for the multi-threaded case.

Phase 3: VirtualMemory Capability (implemented baseline, required for Go)

What: Implement the VirtualMemory capability interface. The current schema has map, unmap, and protect; Go may need decommit/query semantics later.

Why: Go’s memory allocator (sysAlloc, sysReserve, sysMap, etc.) needs mmap-like functionality. This is the single biggest kernel-side requirement for Go.

Current state: VirtualMemoryCap implements map/unmap/protect over the existing page-table code with ownership tracking and quota checks. Go-specific work still has to map runtime sysAlloc/sysReserve/sysMap expectations onto that interface.

Blockers: None for the baseline capability; timer/futex/threading still block useful Go.

Phase 4: Futex Operation (Low-medium effort, required for Go threading)

What: Implement futex(WAIT) and futex(WAKE) as a fast capability-authorized kernel operation.

Why: Go’s runtime synchronization (lock_futex.go) is built on futexes. The entire goroutine scheduler depends on futex-based sleeping.

Effort: ~100-200 lines for the first private-futex path. A wait queue keyed by address-space + userspace address is enough initially.

Blockers: Futex wait-queue design and, for full Go threading, the thread scheduler.

Phase 5: Kernel Threading (High effort, required for Go GOMAXPROCS>1)

What: Multiple threads per process sharing address space and cap table.

Why: Go’s newosproc() creates OS threads via clone(). Without real threads, Go is limited to GOMAXPROCS=1.

Effort: ~500-800 lines. Major scheduler extension.

Blockers: Scheduler, per-CPU data, SMP support.

Biggest Blockers for Go

In priority order after the 2026-04-22 TLS and VirtualMemory work:

  1. Timer / monotonic clock – Go’s scheduler needs nanotime() for goroutine scheduling decisions. Without a timer, Go cannot preempt goroutines or manage timeouts.

  2. Runtime-controlled FS base – Go calls arch_prctl(ARCH_SET_FS) on every new thread. capOS can load static ELF TLS today, but Go still needs a way to set the runtime’s own TLS base.

  3. Futex – Go’s M:N scheduler depends on futex for sleeping/waking OS threads. Without futex, Go falls back to spin-waiting (wasteful) or simply cannot block.

  4. Thread creation – Required for GOMAXPROCS > 1. Phase 1 Go can work single-threaded.

  5. Go runtime port glue – map sysAlloc/write1/exit/random/env/time to capOS runtime and capabilities.

Biggest Blockers for C

C is much simpler than Go:

  1. Linker and toolchain setup – Need a cross-compilation toolchain targeting capOS (Clang with the custom target, or GCC cross-compiler).
  2. libcapos.a with C headers – Rust library with extern "C" API.
  3. musl integration (optional) – For full libc, replace musl’s __syscall() with capability invocations.
1. Custom userspace target JSON          [optional build hygiene]
     |
2. VirtualMemory capability              [done: baseline map/unmap/protect]
     |
3. TLS support (PT_TLS, FS base)         [done for static ELF processes]
     |
4. Futex authority cap + measured ABI    [extends scheduler]
     |
5. Timer capability (monotonic clock)    [extends PIT/HPET driver]
     |
6. Go Phase 1: minimal GOOS=capos       [single-threaded, M=1]
     |
7. Kernel threading                      [major scheduler work]
     |
8. Go Phase 2: multi-threaded           [GOMAXPROCS>1, concurrent GC]
     |
9. C toolchain + libcapos               [parallel with Go work]
     |
10. Go Phase 3: network poller          [depends on networking stack]

Steps 1-5 are kernel prerequisites. Step 6 is the Go fork. Steps 7-10 are incremental improvements that can proceed in parallel.

Key Architectural Decisions for capOS

  1. Keep x86_64-unknown-none for kernel, x86_64-unknown-capos for userspace. The kernel does not benefit from a custom OS target (it’s freestanding). Userspace benefits from cfg(target_os = "capos").

  2. Use local-exec TLS model for static binaries. No dynamic linker means no general-dynamic or initial-exec TLS. local-exec is zero-overhead.

  3. Implement FS base save/restore early. Both Go and Rust #[thread_local] need it. It’s a small addition to context switch code.

  4. VirtualMemory cap stays on the Go critical path. The baseline exists; the Go port still needs exact runtime allocator semantics and any missing query/decommit behavior.

  5. Futex is the synchronization primitive. Both Go and any future pthreads implementation need futex. Keep authority capability-based, but measure whether the hot path should use a compact transport operation rather than generic Cap’n Proto method dispatch.

  6. Signals can be deferred. Go can start with cooperative-only preemption (no SIGURG). Signal delivery is complex and can come much later.