Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Userspace Runtime

The userspace runtime owns the repeated mechanics that every service needs: bootstrap validation, heap initialization, typed capability lookup, ring submission, completion matching, application exception decoding, and handle lifetime.

  • Go VirtualMemory Contract defines the caller-buffer reserve, commit, and decommit methods allocator paths need.
  • Programming Languages summarizes current native Rust support and planned language-runtime tracks.
  • Memory Management documents the implemented kernel VirtualMemory and MemoryObject behavior.
  • Go Runtime is the owning language runtime proposal; LLVM Target records the Go runtime OS hooks that drive this work.

Current Behavior

Runtime-owned _start receives only launch_parameters_len. Executable process construction always maps a writable/NX ring page at RING_VADDR and a read-only/NX CapSet page at CAPSET_VADDR; a process with no bootstrap grants receives a valid CapSet header with count = 0. The synthetic CPL0 idle process maps neither page because it never loads its userspace address space or enters _start. On x86_64, RDI carries launch_parameters_len and RSI, RDX, RCX, and R8 are zero. A zero length means no launch envelope; a nonzero bounded length selects the read-only mapping at LAUNCH_PARAMETERS_VADDR. _start initializes a fixed heap, validates the CapSet header, installs an emergency Console panic path when available, calls capos_rt_main(runtime), and exits with the returned code.

Launch parameters retain one entry register rather than adding an always-mapped descriptor page solely to make _start argument-free. The envelope length is variable payload metadata, not an assertion that a fixed mapping exists, and the kernel already maps envelope pages only when data is present. A separate descriptor page would add a frame and mapping to every process to remove that one data-bearing register without changing authority or decoding behavior.

Process identity is not part of the entry ABI. Code that needs an opaque identity must receive an explicit process-local capability. libcapos-posix derives its shim pid from the contextId returned by ProcessIdentity.info on its process_identity grant. Real scheduling contexts and process-identity objects draw from one kernel-global context-id allocator, so an identity pair cannot alias a live scheduling context. ProcessIdentity is a separate, info-only interface with no create, bind, revoke, or notification-drain methods, so the grant conveys no scheduling authority. Every normal POSIX system manifest grants the object, and the recording shim adds a fresh object to each spawned child. The spawner returns that same read-only capability to the parent as a result capability; execve() queries it before returning, so the pid used by waitpid() equals the child’s getpid() value. If the result capability cannot be queried, the parent terminates the spawned child instead of installing an unverifiable handle.

capos-rt resolves and caches the current process identity during _start, before application code can lend out the runtime ring client. Consequently, getpid() does not perform a ring operation and cannot fail because another caller temporarily owns that client. A missing grant remains distinguishable from a malformed or unreadable named grant: missing authority reaches the POSIX refusal path and exits 125 on getpid(), while an invalid named grant fails runtime startup with exit 122. The capability is installed in the CapSet before the process enters userspace, independently of whether a typed launch envelope is present.

The Runtime lends out at most one RuntimeRingClient at a time. The client wraps the raw ring page, keeps request buffers alive until completions are matched, handles out-of-order completions, packs copy-transfer descriptors, and parses result-cap records. Owned runtime handles queue CAP_OP_RELEASE when the last local reference is dropped; the release queue flushes when a ring client is borrowed or dropped, or when code calls Runtime::flush_releases() explicitly. RingClient::submit_pipelined_call_batch reserves an AnswerId, publishes an answer-allocating CALL and one dependent CALL under a single SQ-tail store, and maps the selected result-cap record index to pipeline_field. Kernel-served answers complete in the submitting drain. Endpoint answers may cross drains; the client can wait for both completions with one cap_enter(min_complete=2). When the endpoint RETURNs, the kernel wakes the submitting thread with a private retry result that the cap_enter wrapper consumes and re-enters on, so the dependent completes through the caller’s own frozen batch with no userspace round trip.

Design

The runtime separates non-owning bootstrap references from owned local handles. CapSet entries produce typed Capability<T> values only when the interface ID matches the requested type, and the same manifest-order CapSet entries remain available for diagnostic and shell surfaces that need to list or inspect what a process was actually granted. Result-cap adoption performs the same interface check before producing OwnedCapability<T>.

Typed clients are thin wrappers over the ring client. They encode Cap’n Proto params, submit CALL SQEs, wait for a matching CQE, decode transport errors, and decode kernel-produced CapException payloads into client errors. Endpoint servers can use submit_endpoint_return_exception() to return a serialized CapException to the original caller over the same endpoint RETURN path. The handwritten TimerClient exposes monotonic now reads and sleep calls over the same completion-matching path. The handwritten VirtualMemoryClient exposes map, reserve, commit, decommit, unmap, and protect calls for runtime heap/arena allocation over anonymous user pages. It has both the ordinary allocation-backed async methods and synchronous caller-buffer methods for allocator growth paths that cannot allocate while asking the kernel for more memory. This matches the reserve/commit/decommit surface specified in Go VirtualMemory Contract. The handwritten ThreadControlClient exposes current-process FS-base reads and updates for runtimes that need to swap a language-managed TLS base after process startup.

The 7.1.0 threading contract keeps one runtime-owned ring and the runtime’s single CQ-consumer invariant for process-scoped calls. With the opt-in reactor feature, ProcessRingReactor owns that ring client: one runtime thread submits and drains its CQ, matches process-scoped call completions by user_data, and publishes them into fixed wait slots. Each slot carries a monotonically changing generation plus the exact call identity, so a late completion for a retired wait cannot wake a reused slot. The waiting caller submits its compact PARK on its own per-thread ring, releases reactor state before cap_enter, and consumes only that PARK completion; it never drains the runtime ring. The runtime drainer submits UNPARK after publishing the process-scoped completion and reclaims the internal wake CQE itself.

The bridge accepts only calls whose behavior is independent of the kernel’s caller_thread context. ThreadControl, endpoint calls, and any other thread-context-sensitive operation stay on the caller’s own ring; dispatching them through the runtime ring would apply the operation to the drainer thread.

Each caller issues a same-process ring kick after publishing new SQ work, so the elected drainer can block in cap_enter(1, WAIT_FOREVER) instead of polling. Under the scheduler lock, the kernel arms the cap-enter continuation and then rechecks both CQ readiness and pending SQ work before publishing the blocked state. Publication before that recheck makes the syscall retry and drain the SQ; publication after it wakes the blocked drainer with the same private retry result. The kernel supplies the constructor with the drainer’s thread ID, avoiding a userspace copy of kernel TID allocation policy. This keeps the existing generation and single-CQ-owner contract unchanged. ThreadControlClient methods apply to the calling thread’s FS base and therefore do not use the reactor bridge. ThreadControl.exitThread and the raw exit(code) syscall both terminate the current thread; the process exits when its last live thread exits.

The 7.2.3 park slice adds a process-local ParkSpace marker type and compact CAP_OP_PARK / CAP_OP_UNPARK operations. Each reactor caller retains one ReactorThreadRing over its current thread ring across waits. The wrapper owns only PARK traffic and retains failed wait IDs until their CQEs are reclaimed, so a later PARK cannot alias a stale completion. The reactor’s process/runtime ring remains reserved for process-scoped calls and UNPARK. This separation is required by the kernel’s per-thread ring endpoints and prevents a blocked caller from holding the runtime ring client.

Future generated clients should preserve this split: transport lifetime and completion matching belong in the runtime, while interface-specific encoding belongs in generated or handwritten client wrappers.

Invariants

  • Every executable process has ring and CapSet pages at RING_VADDR and CAPSET_VADDR; an empty CapSet remains a mapped, valid zero-entry page.
  • A zero launch-parameter length means absent; a nonzero bounded length selects the fixed read-only launch-parameter mapping.
  • The CapSet header magic/version must validate before lookup.
  • CapSet handles are non-owning unless explicitly adopted.
  • Only one runtime ring client may be live at a time for a process.
  • Until Ring v2, multithreaded generic client waits must flow through a runtime reactor/demux path rather than letting multiple threads consume the process CQ directly.
  • Park wait must not hold the live runtime ring client while the kernel parks the current thread.
  • Reactor calls must not depend on caller_thread; thread-context operations use the current thread’s ring directly.
  • Wait-slot readiness is valid only when slot generation and call user_data both match. Completion delivery and timeout retirement are one locked choice, and late retired CQEs are reclaimed without waking a reused slot.
  • Request params and result buffers must outlive their matching CQE.
  • A result cap can be consumed only once and only with the expected interface ID.
  • Promise placeholders must map to sideband result-cap record indexes, not schema field paths.
  • Dropping the final owned handle queues exactly one local CAP_OP_RELEASE; Runtime::flush_releases() forces queued releases and reports rejected kernel release results.
  • Release flushing treats stale or already-removed caps as non-fatal cleanup.

Code Map

  • capos-rt/src/entry.rs - _start, Runtime, bootstrap validation, single-owner ring token, release queue flushing.
  • capos-rt/src/alloc.rs - fixed userspace heap initialization.
  • capos-rt/src/capset.rs - typed CapSet lookup and manifest-order iteration wrappers.
  • capos-rt/src/ring.rs - ring client, pending calls, completion matching, copy-transfer packing, result-cap parsing.
  • capos-rt/src/reactor.rs - process/runtime-ring CQ ownership, generation- checked wait slots, per-thread PARK waits, and UNPARK routing.
  • capos-rt/src/client.rs - Console, TerminalSession, BootPackage, ProcessSpawner, ProcessHandle, VirtualMemory, Timer, ThreadControl, ThreadSpawner, and ThreadHandle clients, and exception decoding.
  • capos-rt/src/lib.rs - typed capability marker types and owned handle reference counting.
  • capos-rt/src/panic.rs - emergency Console output path.
  • capos-rt/src/syscall.rs - raw syscall instructions and public syscall wrappers, including the hostile smoke probe for the removed ambient write syscall.
  • targets/x86_64-unknown-capos.json - userspace target specification.
  • tools/check-userspace-runtime-surface.sh - source check that keeps runtime primitives owned by capos-rt.
  • init/src/main.rs, capos-rt/src/bin/smoke.rs, and shell/src/main.rs - current runtime users.

Validation

  • make capos-rt-check builds the runtime smoke binary against targets/x86_64-unknown-capos.json, matching the booted userspace target.
  • make init-capos-build, make demos-capos-build, make shell-capos-build, and make capos-rt-capos-build expose focused custom-target build wrappers for the current userspace crates and runtime smoke binary.
  • tools/check-userspace-runtime-surface.sh verifies init, demos, and shell do not define _start, panic handlers, global allocators, raw syscall instructions, or entry-point macros outside capos-rt.
  • make run-smoke validates runtime entry, typed Console calls, exception decoding, owned handle release, result-cap parsing through IPC, and clean process exit.
  • make run-spawn validates ProcessSpawnerClient, ProcessHandleClient, VirtualMemoryClient, TimerClient, ThreadControlClient, ThreadSpawnerClient, ThreadHandleClient, result-cap adoption, and release behavior under init spawning. The single-thread-runtime child proves the first runtime-shaped checkpoint over caller-buffer VirtualMemory calls and Timer; the thread-lifecycle child proves in-process create, self-join rejection, join, detach, last-thread exitThread, and private ParkSpace wait/wake correctness.
  • make test-shell validates CapSet iteration, capability inspection, typed application-error decoding, guest session metadata, exact-grant spawning, ProcessHandle waits, and stale-handle release behavior in the focused shell-launch proof manifest.
  • make test-terminal validates TerminalSessionClient writes, bounded line reads, hidden-echo input handling, and structured cancellation in the focused terminal proof manifest.
  • cd capos-rt && cargo test --lib --target x86_64-unknown-linux-gnu covers host-testable runtime invariants when run explicitly.

Open Work

  • Add generated client bindings after the schema surface stabilizes.
  • Define runtime and ring ABI semantics for promise chains deeper than one hop.
  • Add typed ParkSpace clients with runtime-owned user_data demultiplexing.
  • Define release behavior for queued handles when a process exits before the release queue flushes.