x2APIC and APIC Virtualization

Research note for the SMP Phase C LAPIC/IPI decision. The goal is to decide how x2APIC should fit after the current LAPIC/IPI implementation work and to record which virtualization facts affect that choice.

Existing Local Research

Before adding this note, docs/research/ contained:

• capnp-error-handling.md
• completion-ring-threading.md
• eros-capros-coyotos.md
• genode.md
• ix-on-capos-hosting.md
• llvm-target.md
• os-error-handling.md
• out-of-kernel-scheduling.md
• pingora.md
• plan9-inferno.md
• sel4.md
• small-llm-survey.md
• zircon.md

None of those files directly cover APIC/x2APIC or KVM APIC virtualization.

Sources Checked

• Intel, Intel 64 and IA-32 Architectures Software Developer’s Manuals, Vol. 3 APIC and APIC virtualization chapters.
• Intel, xAPIC Deprecation Plan, updated 2025-09-18.
• Intel, CPUID Enumeration and Architectural MSRs, updated 2025-05-12.
• QEMU, QEMU / KVM CPU model configuration.
• QEMU, Paravirtualized KVM features.
• Linux kernel documentation, KVM API.

Local verification:

• Host command qemu-system-x86_64 --version reported QEMU 8.2.2.
• Host command qemu-system-x86_64 -cpu help listed x2apic as a recognized CPUID feature.
• The active workplan/smp-phase-c-lapic-ipi worktree at commits 1de4976 and dcda022 implements an xAPIC MMIO LAPIC foundation, not x2APIC: BSP LAPIC MMIO mapping, LAPIC timer vector 48, IPI vector 49, LAPIC EOI, AP LAPIC initialization, and PIT/PIC fallback.

x2APIC Findings

x2APIC is still the forward-looking LAPIC backend for later hardware and VM coverage:

• It avoids mapping the local APIC MMIO page and uses architectural MSRs for local APIC register access.
• It supports wider APIC IDs than xAPIC’s 8-bit destination model, which keeps the CPU-id/LAPIC-id split introduced by the SMP proposal relevant on larger systems and VMs.
• Intel’s current public guidance says x2APIC is required above 255 cores, newer Intel client families default to x2APIC, and legacy xAPIC can become unavailable or locked out after firmware or system software enters x2APIC.
• The local capOS dependency set already has x86_64 MSR access. A manual minimal x2APIC backend for EOI, ICR/IPI, spurious vector, LVT timer, timer initial count, divide config, and current APIC ID should be small enough to avoid adding another architecture crate when that backend is implemented.

The current Phase C LAPIC/IPI branch already uses xAPIC MMIO and should not be blocked or reworked only to satisfy this research note. The right planning shape is:

1. Land and review the existing xAPIC MMIO LAPIC timer/IPI foundation.
2. Continue Phase C with TLB shootdown over the existing vector 49 IPI path.
3. Add an x2APIC backend as a later compatibility and scaling follow-up.
4. Once both backends exist, select x2APIC when CPUID.01H:ECX.x2APIC is available or when firmware has already enabled/locked x2APIC, and keep xAPIC MMIO as the fallback for QEMU and older hardware that only expose xAPIC.

Virtualization Findings

Virtualization is relevant to validation and future performance, not to the guest-visible correctness contract:

• QEMU/KVM can expose x2APIC through CPU model feature selection. capOS tests should make that explicit by extending the current QEMU model to -cpu qemu64,+smep,+smap,+rdrand,+x2apic, or by using another named CPU model with +x2apic, instead of relying on the host or accelerator default.
• KVM exposes APIC state through its own API and has x2APIC-specific handling for 32-bit APIC IDs. That matters to the VMM, but a capOS guest should use the architectural x2APIC interface.
• QEMU/KVM paravirtual features such as kvm-pv-eoi, kvm-pv-ipi, and kvm-pv-tlb-flush are optional accelerations. They should not be part of the first LAPIC/IPI or TLB-shootdown proof because they would make correctness depend on a Linux/KVM-specific host contract.
• APIC virtualization features such as APICv or AMD AVIC are VMM-side acceleration mechanisms. capOS should not require or detect them before it has a stable architectural x2APIC path.

The practical QEMU proof target is therefore:

1. Boot the current xAPIC MMIO LAPIC branch with -smp 2.
2. Prove LAPIC timer ticks on vector 48 and IPI delivery on vector 49.
3. Keep KVM paravirtual APIC/TLB/IPI features disabled or ignored for the first correctness proof.
4. Add a separate x2APIC QEMU/KVM proof only when the x2APIC backend exists, using an explicit CPU model such as -cpu qemu64,+smep,+smap,+rdrand,+x2apic.

capOS Recommendation

Treat the active xAPIC MMIO branch as the current Phase C LAPIC/IPI foundation. Do not convert it to x2APIC before TLB shootdown and scheduler ownership work. Track x2APIC as a later backend required for newer/high-core systems and some firmware states. Defer KVM paravirtual APIC/TLB/IPI features until after the architectural LAPIC timer, IPI, and TLB-shootdown path is stable.