# x2APIC and APIC Virtualization

Research note for the SMP Phase C LAPIC/IPI decision. The goal is to decide
how x2APIC should fit after the current LAPIC/IPI implementation work and to
record which virtualization facts affect that choice.

## Existing Local Research

Before adding this note, `docs/research/` contained:

- `capnp-error-handling.md`
- `completion-ring-threading.md`
- `eros-capros-coyotos.md`
- `genode.md`
- `ix-on-capos-hosting.md`
- `llvm-target.md`
- `os-error-handling.md`
- `out-of-kernel-scheduling.md`
- `pingora.md`
- `plan9-inferno.md`
- `sel4.md`
- `small-llm-survey.md`
- `zircon.md`

None of those files directly cover APIC/x2APIC or KVM APIC virtualization.

## Sources Checked

- Intel, [Intel 64 and IA-32 Architectures Software Developer's Manuals](https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html),
  Vol. 3 APIC and APIC virtualization chapters.
- Intel, [xAPIC Deprecation Plan](https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/xapic-deprecation-plan.html),
  updated 2025-09-18.
- Intel, [CPUID Enumeration and Architectural MSRs](https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html),
  updated 2025-05-12.
- QEMU, [QEMU / KVM CPU model configuration](https://www.qemu.org/docs/master/system/qemu-cpu-models.html).
- QEMU, [Paravirtualized KVM features](https://www.qemu.org/docs/master/system/i386/kvm-pv.html).
- Linux kernel documentation, [KVM API](https://www.kernel.org/doc/html/latest/virt/kvm/api.html).

Local verification:

- Host command `qemu-system-x86_64 --version` reported QEMU 8.2.2.
- Host command `qemu-system-x86_64 -cpu help` listed `x2apic` as a recognized
  CPUID feature.
- The active `workplan/smp-phase-c-lapic-ipi` worktree at commits `1de4976`
  and `dcda022` implements an xAPIC MMIO LAPIC foundation, not x2APIC: BSP
  LAPIC MMIO mapping, LAPIC timer vector 48, IPI vector 49, LAPIC EOI, AP LAPIC
  initialization, and PIT/PIC fallback.

## x2APIC Findings

x2APIC is still the forward-looking LAPIC backend for later hardware and VM
coverage:

- It avoids mapping the local APIC MMIO page and uses architectural MSRs for
  local APIC register access.
- It supports wider APIC IDs than xAPIC's 8-bit destination model, which keeps
  the CPU-id/LAPIC-id split introduced by the SMP proposal relevant on larger
  systems and VMs.
- Intel's current public guidance says x2APIC is required above 255 cores,
  newer Intel client families default to x2APIC, and legacy xAPIC can become
  unavailable or locked out after firmware or system software enters x2APIC.
- The local capOS dependency set already has `x86_64` MSR access. A manual
  minimal x2APIC backend for EOI, ICR/IPI, spurious vector, LVT timer, timer
  initial count, divide config, and current APIC ID should be small enough to
  avoid adding another architecture crate when that backend is implemented.

The current Phase C LAPIC/IPI branch already uses xAPIC MMIO and should not be
blocked or reworked only to satisfy this research note. The right planning
shape is:

1. Land and review the existing xAPIC MMIO LAPIC timer/IPI foundation.
2. Continue Phase C with TLB shootdown over the existing vector 49 IPI path.
3. Add an x2APIC backend as a later compatibility and scaling follow-up.
4. Once both backends exist, select x2APIC when `CPUID.01H:ECX.x2APIC` is
   available or when firmware has already enabled/locked x2APIC, and keep xAPIC
   MMIO as the fallback for QEMU and older hardware that only expose xAPIC.

## Virtualization Findings

Virtualization is relevant to validation and future performance, not to the
guest-visible correctness contract:

- QEMU/KVM can expose x2APIC through CPU model feature selection. capOS tests
  should make that explicit by extending the current QEMU model to
  `-cpu qemu64,+smep,+smap,+rdrand,+x2apic`, or by using another named CPU
  model with `+x2apic`, instead of relying on the host or accelerator default.
- KVM exposes APIC state through its own API and has x2APIC-specific handling
  for 32-bit APIC IDs. That matters to the VMM, but a capOS guest should use
  the architectural x2APIC interface.
- QEMU/KVM paravirtual features such as `kvm-pv-eoi`, `kvm-pv-ipi`, and
  `kvm-pv-tlb-flush` are optional accelerations. They should not be part of
  the first LAPIC/IPI or TLB-shootdown proof because they would make correctness
  depend on a Linux/KVM-specific host contract.
- APIC virtualization features such as APICv or AMD AVIC are VMM-side
  acceleration mechanisms. capOS should not require or detect them before it
  has a stable architectural x2APIC path.

The practical QEMU proof target is therefore:

1. Boot the current xAPIC MMIO LAPIC branch with `-smp 2`.
2. Prove LAPIC timer ticks on vector 48 and IPI delivery on vector 49.
3. Keep KVM paravirtual APIC/TLB/IPI features disabled or ignored for the first
   correctness proof.
4. Add a separate x2APIC QEMU/KVM proof only when the x2APIC backend exists,
   using an explicit CPU model such as
   `-cpu qemu64,+smep,+smap,+rdrand,+x2apic`.

## capOS Recommendation

Treat the active xAPIC MMIO branch as the current Phase C LAPIC/IPI foundation.
Do not convert it to x2APIC before TLB shootdown and scheduler ownership work.
Track x2APIC as a later backend required for newer/high-core systems and some
firmware states. Defer KVM paravirtual APIC/TLB/IPI features until after the
architectural LAPIC timer, IPI, and TLB-shootdown path is stable.