Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Debug, Trace, and Profiling Authority: Prior-Art Survey

Survey of how existing systems scope and gate debug, trace, and profiling access. Each section states the verified fact and the lesson it carries for a capability OS.

1. GDB Remote Serial Protocol (gdbstub)

The GDB Remote Serial Protocol (RSP) is the wire protocol between a GDB client and a gdbstub running on or alongside the target. A stub exposes the target’s entire register file and address space to the connected client via a small set of packet types:

  • g/G — read and write all general-purpose registers.
  • p/P — read and write individual registers.
  • m/M/X — read and write arbitrary memory ranges.
  • Z/z — set and clear software breakpoints, hardware breakpoints, and hardware watchpoints (read, write, or access).
  • s/c — single-step and continue execution.

Feature negotiation (qSupported) lets client and stub advertise extensions, but the baseline packet set already provides full read/write authority over the target’s memory and execution state.

Lesson for capOS. A DebugSession capability is not a read-only observer — it is a read/write authority over the target’s registers, memory, and control flow. Attaching the stub to a process is itself the high-privilege act; the session object must be issued by an explicit grant (e.g., a ProcessSpawner debug grant or a ThreadControl-derived debug capability) rather than derived from any lesser handle. The gdbstub pattern shows that the session boundary is the right chokepoint: once a client holds the session capability the protocol can proceed without further kernel checks.

2. Linux ptrace and the Yama LSM

Ambient-authority problem

Linux ptrace originally allowed any process to PTRACE_ATTACH to any other process running under the same UID that was marked as dumpable. The kernel docs summarize the risk: “a single user is able to examine the memory and running state of any of their processes. For example, if one application (e.g. Pidgin) was compromised, it would be possible for an attacker to attach to other running processes (e.g. Firefox, SSH sessions, GPG agent, etc) to extract additional credentials and continue to expand the scope of their attack.”

Yama ptrace_scope levels

The Yama Linux Security Module adds a sysctl kernel.yama.ptrace_scope with four levels to progressively restrict this ambient authority:

LevelBehaviour
0Classic: PTRACE_ATTACH to any same-UID dumpable process.
1Restricted: only descendants (or processes that have called prctl(PR_SET_TRACER, ...)) may be attached.
2Admin-only: only processes holding CAP_SYS_PTRACE may attach.
3No attach: PTRACE_ATTACH and PTRACE_TRACEME are blocked system-wide; the setting is irreversible once applied.

Most Linux distributions now ship with level 1 as the default, but level 0 remains the kernel default if Yama is not loaded.

Lesson for capOS. Yama exists solely because ambient-authority ptrace is a privilege-escalation footgun. The correct model is the inverse: no process should be able to attach to another without an explicit, pre-granted capability. In capOS terms, DebugSession attach must require a pre-issued debug capability (analogous to level 3 everywhere, not level 0 with an opt-out). The parent process or init can hold a ThreadControl-derived debug grant; a RestrictedLauncher can be configured to never issue one. There is no ambient fallback.

3. Linux perf_events and eBPF gating

perf_event_paranoid

/proc/sys/kernel/perf_event_paranoid is the sysctl controlling what unprivileged processes may sample:

ValueEffect
-1No scope or access restrictions; most permissive.
≥ 0Raw tracepoints blocked for unprivileged users.
≥ 1CPU-level (system-wide) profiling blocked; per-process only.
≥ 2Kernel profiling blocked; user-space events only.

Debian-based distributions additionally define 4 (block all perf for unprivileged users) and use it as the distro default.

CAP_PERFMON and CAP_BPF

Linux 5.8 introduced CAP_PERFMON to separate performance-monitoring authority from the broad CAP_SYS_ADMIN. Holding CAP_PERFMON lets a process bypass perf_event_paranoid scope checks. Similarly, CAP_BPF gates loading BPF programs that have performance or tracing implications (e.g., kprobes, uprobes, perf maps); attaching BPF to a kprobe tracepoint requires CAP_PERFMON or CAP_SYS_ADMIN.

The split reflects the principle of least privilege: a profiling daemon should not require CAP_SYS_ADMIN merely to sample hardware counters.

Lesson for capOS. Read-only sampling (hardware counters, ring buffer) is a distinct authority from read/write debugging. capOS should issue a Sampler capability (read-only, non-interrupting, no memory write) separately from a DebugSession (register/memory read-write, breakpoints). The sampler does not stop the target and transfers no writable authority; the perf/CAP_PERFMON split is the prior-art justification for keeping these two surfaces apart.

4. Fuchsia / Zircon: handle-scoped debug authority

debug_agent and zxdb

On Fuchsia the debugger is split into two components: debug_agent, a component running on the target that holds process handles and communicates with the kernel, and zxdb, the developer-facing client that connects to debug_agent over a socket. The fuchsia.debugger FIDL library defines the boundary:

  • DebugAgent — core protocol; AttachTo accepts a name pattern and FilterType to select which processes to attach to.
  • ProcessInfoIterator, AttachedProcessIterator — read access to thread and process state.
  • Launcher — creates new DebugAgent instances.

The debug_agent acquires process handles from the kernel by being granted them through the Zircon job/process handle tree. Zircon’s handle model means that process operations (reading memory, setting breakpoints, receiving exceptions) all require the caller to hold a process handle with the appropriate ZX_RIGHT_* bits. A process that does not hold a handle to another process cannot inspect or modify it, regardless of UID. The zxdb UI can inspect handle tables of attached processes and displays their ZX_RIGHT_READ/ZX_RIGHT_WRITE/ZX_RIGHT_INSPECT/ZX_RIGHT_SIGNAL rights to the developer.

Exception delivery in Zircon is also capability-scoped: zx_task_create_exception_channel creates a channel on a task (thread, process, or job) object; the caller must hold that task handle. The resulting channel is read-only and can only receive exception messages, not issue commands, which means observing crashes requires the task handle but does not by itself grant write authority.

Lesson for capOS. Fuchsia demonstrates that a production debugger can be built entirely on object-handle authority without ambient attach. The debug_agent component acts like a bounded debug authority domain: it holds process handles for the processes it is authorized to debug, and zxdb interacts only through the FIDL protocol that debug_agent exposes. The capOS equivalent is a DebugSession capability issued per target process, scoped to a running session, with a separate read-only ExceptionChannel cap for crash observation.

5. seL4: TCB capability and debug-build gate

Hardware debug API

seL4 exposes hardware breakpoints, watchpoints, and single-stepping to userspace via TCB object methods, but only when the kernel is built with KernelDebugBuild (equivalently, HardwareDebugAPI=1 in the CMake config). The available TCB invocations are:

  • seL4_TCB_SetBreakpoint — configure a breakpoint or watchpoint (virtual address, access type: read/write/exec, size).
  • seL4_TCB_GetBreakpoint — read the current configuration.
  • seL4_TCB_UnsetBreakpoint — disable a slot.
  • seL4_TCB_ConfigureSingleStepping — break on every N-th instruction.

Each invocation takes a capability to the target TCB. Only a holder of that TCB capability can manipulate the thread’s debug registers.

Debug-only kernel syscalls

KernelDebugBuild also enables:

  • seL4_DebugSnapshot — outputs a CapDL dump of the current kernel capability state to the serial console.
  • seL4_DebugDumpScheduler — dumps TCB addresses, thread names, instruction pointers, priorities, and scheduler states.

These syscalls expose global kernel state and are intentionally excluded from verified (proof) builds where the information flow would violate the formal security model.

Lesson for capOS. seL4 gates per-thread debug authority on possession of the TCB capability, which is the right model. capOS’s DebugSession should similarly be derived from ThreadControl so that only the process or entity that holds ThreadControl for a thread can open a debug session on it. The seL4_DebugSnapshot pattern also shows that a system-wide cap-table snapshot is a separate, higher-privilege operation from per-thread debug access; in capOS a read-only CapTableSnapshot authority can be issued for audit purposes without granting register/memory write access.

6. Genode: capability-session GDB monitor

Architecture

Genode implements user-level debugging via a GDB monitor component that interposes between a target application and its parent. The GDB monitor:

  1. Intercepts session requests from the target before they reach the parent.
  2. Provides local virtual implementations of the CPU service, RM (region-map/address-space) service, and ROM service, wrapping the real core implementations.
  3. Exposes a gdbserver protocol endpoint over a terminal session (TCP or UART).

This gives the GDB monitor “full control over all threads and memory objects (dataspace) and the address space of the target.” The monitor holds real capabilities to the target’s CPU and address-space sessions; the target’s own session handles are virtualized stubs that forward to the monitor.

Capability session scoping

Genode’s Cpu_session interface allows retrieving and modifying thread register and execution state. The API comment in the framework explicitly notes that these operations are “primarily designated for realizing user-level debuggers.” Because the monitor interposes the CPU session, it holds the same authority the parent would hold, but the target holds only stubs — the target cannot see or touch its own debug registers directly.

Lesson for capOS. The Genode monitor pattern reinforces that debugging authority flows from capability delegation, not from process identity. The interposition model also clarifies the ring-trace design decision: debug_tap in capOS captures SQE/CQE ring records passively and does not require interposing a CPU session, which keeps ring-trace authority weaker and non-interrupting by construction. A full DebugSession (register read/write, breakpoints) requires explicit session acquisition from the parent or init, matching the Genode monitor’s explicit CPU-session grant.

Applicability to capOS

The cross-system survey points to a consistent set of design invariants:

  1. DebugSession attach is an explicit, audited capability grant, not ambient. The anti-pattern is Linux ptrace at level 0; Yama level 3 is the correct default posture. In capOS no process inherits the ability to debug another: a DebugSession is derived from ThreadControl, issued by the process’s parent or init, and recorded in the audit log.

  2. Read-only cap-table snapshots transfer no authority. seL4’s seL4_DebugSnapshot is a separate, opt-in, debug-build-only facility. In capOS a CapTableSnapshot cap can be issued for audit visibility without granting any write access to the observed process.

  3. Ring-trace builds on debug_tap and does not stop the target. perf/CAP_PERFMON shows that sampling is a distinct authority class from full debugging. capOS debug_tap ring records are append-only, non-interrupting, and do not feed back into the target’s execution — matching the sampler authority class, not the DebugSession class.

  4. Sampler does not stop the target. Hardware performance counter sampling (CAP_PERFMON semantics) and ring-record sampling (debug_tap) are passive read surfaces. A DebugSession that can set breakpoints, modify registers, or write memory is a distinct, higher-privilege capability and must not be conflated with passive tracing.

  5. Exception observation is weaker than debug write authority. Zircon’s zx_task_create_exception_channel returns a read-only channel. capOS should provide a similar ExceptionObserver capability (receive crash notifications, no write access) independent of DebugSession.

Sources