Self-Hosted Task Backend

Detailed decomposition for hosting the multi-agent development task backend on capOS itself. Today that backend is loopyard, a companion host-side project (PostgreSQL-backed; not part of this repository): it computes runnable task sets from typed dependencies, validates status transitions, checks conflict domains, and issues lock leases with generation fencing, all enforced inside the database so no client can corrupt task state, with CLI/MCP/REST as thin transports over one API surface. The long-term intent — recorded in Stateful Task and Job Graphs and capOS-Hosted Agent Swarms — is for capOS to serve that coordination role in-system through typed capability services instead of an external SQL server.

Visible Outcome

A capOS instance serves the task backend for the multi-agent development workflow that builds capOS: host-side workers resolve runnable tasks, acquire fenced lock leases, report status transitions, and receive conflict reports from a capOS-hosted coordinator service, speaking the same task-source and lock wire contracts that loopyard serves today. loopyard’s schema and invariants are the reference semantics the capOS services must reproduce, and task state must be exportable/importable in both directions for migration.

Gap Inventory

What exists and what is missing, largest gap first:

1. Transactional structured record store. The coordinator needs record-level atomic multi-record commits, secondary lookups, and constraint-style invariant enforcement. Current durable storage is below that level: the userspace Store/Namespace service persists a whole-state snapshot (CAPOSUS1), the kernel persistent_store fixture is a disk-backed CAPOSST1 Store, and writable_fs is a single-writer CAPOSWF1 filesystem. Crash-safe A/B superblock commits and torn-grow mount recovery are landed building blocks one level down. Missing: a write-ahead-log or equivalent record store over BlockDevice with atomic multi-record transactions.
2. The coordinator service itself. A userspace service owning task records, statuses-as-data with validated transitions, dependency edges with cycle rejection, runnable-set computation, conflict-domain checks, and lock leases with expiry and generation fencing — the Stateful Task and Job Graphs Stage E operator task surface and Stage G agent workflows, with invariants enforced in the service the way loopyard enforces them in PL/pgSQL. Nothing is implemented.
3. Public ingress and TLS. Host-side workers need to reach the backend. Private GCE self-hosted Web UI serving is proven; public exposure and TLS remain gated by the explicit on-hold cloud-gce-public-self-hosted-webui-ingress-tls task. TLS building blocks (capos-tls, the ACME http-01 solver) exist.
4. A reusable multi-client API service layer. The remote-session Web UI already runs a persistent accept/recv/send/close loop with per-connection deadlines and slow-client bounds over the Phase C userspace stack, but as a demo-grade single flow. The backend needs a reusable HTTP/JSON service layer (a natural consumer for the capos-service lifecycle crate, which currently has none in-tree) plus readiness-driven multiplexing of concurrent client connections over the landed poll/select readiness bridge.
5. Remote actor authentication. loopyard’s principal/agent-session actor model maps naturally onto user_session, credential_store, and authority_broker, but API-token issuance and per-request actor attribution for programmatic clients do not exist; the Web UI login is demo-grade.
6. Client transport. Either a REST/JSON adapter (matching loopyard’s thin-transport shape; fastest for the vibe-loop command contracts) or the capos SDK remote transport (capOS SDK And Dual Transport); the transitional host-backend remote transport (slice 4a there) can carry typed clients before the full capnp-rpc upgrade.
7. Time quality and data portability. WallClock serves UTC over a fixed boot base, so lease expiry and timestamps are not comparable across reboots without time synchronization. Export/import tooling is needed for migration to and from the host backend and for backups.

Sequencing

Phases ordered so each lands on local QEMU evidence before touching the gaps behind it:

1. In-memory coordinator local proof (dispatchable now; no dependency on ingress or storage): coordinator service plus client demo proving runnable-set, transition validation, conflict domains, and fenced leases in QEMU. Root task record: task-backend-coordinator-inmemory-local-proof.
2. API surface local proof: HTTP/JSON adapter over the userspace network stack exposing the coordinator to a host-side client speaking the task-source/lock command contracts against local QEMU.
3. Durable record store: the transactional record store over BlockDevice, then switch the coordinator’s persistence from in-memory state to it, with reboot and crash-recovery proofs.
4. Actors, time, and portability: API-token authentication bound to the session/credential surface, time synchronization for cross-reboot lease and timestamp comparability, and export/import against the host backend’s data.
5. Live deployment: serve the backend behind the public ingress/TLS milestone once that separate track is authorized and closed.

Phases 1 and 2 can proceed against in-memory state while phase 3 is designed; the coordinator’s storage seam should keep the record-store swap mechanical.

Design Grounding

• Stateful Task and Job Graphs — the work-graph substrate this track exercises (Stages E and G) and the prior-art analysis, including loopyard.
• capOS-Hosted Agent Swarms — AgentTask, ResourceLease, ResourceVersion, and ConflictReport shapes the coordinator should converge toward.
• Task State and Agent Telemetry — the current file-per-task ledger the development workflow uses inside this repository.
• capOS SDK And Dual Transport — the remote transport option for typed clients.