Returning from an NMI which hits guest context needs special casing in FRED
mode just like it does in IDT mode.
Break nmi_exit_to_guest() out of handle_ist_exception(), and use it in
entry_FRED_R3() also.
Expand the comment a little, and invert the conditional jump to
compat_restore_all_guest() to avoid needing an #else clause for CONFIG_PV32.
Fixes: 87cfcbe9f0 ("x86/pv: Guest exception handling in FRED mode")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Relase-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Prior to commit 9c20d3c591 ("x86/IDT: Make idt_tables[] be per_cpu(idt)"),
the global idt_tables[] was always safe to use for CPUs in any state.
However, not-yet-onlined CPUs (e.g. MADT with more entries than exist in
practice) or offlined CPUs (e.g. xen-hptool) have their per-cpu pointer
poisoned to detect incorrect uses. machine_kexec() trips over the posion when
clobbering #MC entry paths. Update the comment to discuss parked CPUs, and to
remove discussion which isn't particularly relevant to the actions at hand.
This fixes a fatal #GP (non-canonical memory reference) when trying to enter
the crash kernel.
Fixes: 9c20d3c591 ("x86/IDT: Make idt_tables[] be per_cpu(idt)")
Reported-by: Lin Liu <Lin.Liu01@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
With FRED just as with IDT, it's unsafe to run the exception handlers after
switching stack.
To remove this unsafe window, %cr4 needs clearing earlier. In turn, we may
need to switch to PCID 0 earlier too in order to be able to clear CR4.PCIDE.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
After switching stack, it is not safe to run any exception handlers, because
attempts to access the cpu_info block are out-of-bounds and will generate wild
accesses.
Invalidating the IDT in the common path means there's no need to do so again
in the 32bit path, so drop compat_mode_idt entirely.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
When FRED is active, it is not possible to hook NMIs like this.
NMI hooking in the crash path has undergone several revisions since its
introduction. Notably since commit e7f147bf4a ("x86/crash: Drop manual
hooking of exception_table[]") we use the regular nmi_callback()
infrastructure.
Instead of asserting that we don't enter do_nmi_crash() on the crashing CPU,
tolerate it and return early. It's a marginally longer codepath but behaves
the same and is compatible with FRED.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The configuration uses a mix of dashes and underscores, which is irritating to
develop for. Switch to using the underscore form consistently; it is the more
common form and it has the benefit that it allows splitting on dashes to work
sensibly.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Perform standard syntax cleanup and make it a non-root container. Switch yajl
for json-c given the deprecation of the former. Drop dev86 which is an
x86-only dependency, and QEMU dependencies as we don't build QEMU in this
environment any more.
When updating the job names, also rename some for consistency so the arm64
fragment comes before the compiler.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Perform standard syntax cleanup and make it a non-root container. Switch yajl
for json-c given the deprecation of the former.
Add an x86_64 suffix for naming consistency with everything else.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
This container is tied to gitlab-runner environment in the RPis driving the
test systems, not a specific version of Alpine. Intentionally give it a
generic name so it need not change in the future.
Switch to Alpine 3.24 right away, as it doesn't interact with the 3.18 builds
under test.
The container needs to remain a root container. By no longer using the
arm64v8 build container for dual-purpose, we can finally make the build
containers be non-root.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
GCC 15.2 with Alpine Linux 3.24 fails with -Werror=nonnull, complaining that
we're calling bzero(NULL, 128).
This is a legitimate diagnostic. xenalyze has it's own error() function
shadowing the standard library one, and can in principle return when p is
NULL.
Extend the check in error() with ERR_MAX_TOLERABLE to short circuit the
variable tolerance check.
Suggested-by: Anthony PERARD <anthony.perard@vates.tech>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
In order to avoid serialising the testing on both boards, the runner
configuration is being adjusted.
Have the .xilinx-arm64 and .xilinx-x86_64 templates choose the board directly
using a more specific tag.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Victor Lira <victorm.lira@amd.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The container is tied to the runner, not a version of Ubuntu. Intentionally
give it a generic name so it need not change in the future.
Apply standard cleanup to the dockerfile, except that it must continue to be a
root container to drive real hardware. Explicitly install ca-certificates to
compensate for --no-install-recommends.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Rename it to have an x86_64 suffix, updating the build job names and scheduled
rebuild task.
Apply standard cleanups. Trim the package list down to what is actually
needed. Archlinux's base-devel contains most libraries, but some of the
listed packages have never been dependenices for Xen, and a lot are QEMU
dependenices which aren't useful owing to the lack of the ninja package.
This shrinks the container from 533MB to 380MB.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
This reverts commit 3c4e804607.
This reverts commit 957f0f0b73.
This reverts commit 2bc9bc1978.
This reverts commit 03ebee6ed8.
This reverts commit a9720605b6.
These were committed without a maintainer ack, and therefore in contravention
of policy.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
The default version of python is 3.13, so drop the 3.11 overrides.
Swap yajl for json-c, given the deprecation of the former. Add lz4.
bin86/dev86 are no longer available.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Swap yajl for json-c, given the deprecation of the former. Add lz4.
Add 26.04 GCC/Clang debug/non-debug jobs. Drop the 24.04 debug jobs to keep
the overall job count down.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Ocaml 4.08 and earlier had the compiler package depend on graphics, and
therefore on X. Ocaml 4.09 and later dropped this dependency.
Debian and Ubuntu versions with Ocaml 4.08 and earlier (which are Debian
Bullseye/11, and Ubuntu Focal/20.04 and earlier) had ocaml-nox packages with
this dependency stripped, which we use to keep the size of the containers
down.
In newer versions of Debian and Ubuntu, ocaml-nox is just a transitional
package referring back to ocaml. Ubuntu Resolute/26.04 has finally removed
this transitional package. For all versions where ocaml-nox is just a
transitional package, swap to the ocaml package.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Right now lz4, unlike every other compression scheme, unconditionally uses
Xen's unsafe decompressor. Make it consistent with all other compression
schemes by using liblz4.
The unsafe decompression is still required for the MiniOS build, so rename
xg_dom_decompress_lz4.c to xg_dom_decompress_unsafe_lz4.c and drop the
non-MiniOS content.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
As with other compression libraries, group liblz4 into ZLIB_{CFLAGS,LIBS}.
Add the packages to the Debian Trixie build containers for coverage.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
It is erroneous to check current like this; most commonly, introspection is
dom0 issuing hypercalls against domU.
The use of vm_event_is_enabled() is only for the IS_ENABLED(CONFIG_VM_EVENT)
short circut, so just use that directly.
Reported-by: Hady Azzam <hadyazzam@protonmail.com>
Fixes: b18e38e42d ("xen/mem_access: wrap memory access when VM_EVENT=n")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Tested-by: Hady Azzam <hadyazzam@protonmail.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Christian has just left Citrix, and Dave almost a decade ago.
Andrii (a XAPI committer) has worked on oxenstored before and has agreed to
step up as a maintainer, and Guillaume wishes to get involved and learn too.
In practice I do a lot of the bindings work, so lets make things official.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Andrii Sultanov <andriy.sultanov@vates.tech>
Acked-by: Guillaume Thouvenin <guillaume.thouvenin@vates.tech>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
FRED is now believed to be complete for AMD systems, and has had its tyres
kicked by both XenServer and AMD. Enable FRED by default on capable AMD
systems (Zen6 and later).
Support on Intel is still not yet complete. Leave it as tech preview and not
security supported.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
This removes qemu-system-aarch64-6.0.0-*-export and fixes the outstanding
TODOs about efi-virtio.rom, now that the QEMU in use has it's /usr/share/
properly packaged.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Exactly as per the Bookworm container, but additionally with the ipxe-qemu and
qemu-system-aarch64 packages. These will be used to remove the export jobs.
Switch qemu-arm{32,64} test jobs to use this container, and add two new build
tests.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
The binaries/ directory is a composition from the artefacts, and also used as
a working directory for most of the tests. If the very first artefact is from
a root container, then the test must also be a root container to use it as a
working directory.
For arm64, the existing linux-arm64 artefact suffices. For arm32, pull in the
microcode-x86 artefact as it's the smallest available. This bodge can be
removed when all build containers have become non-root.
For the qemu-xtf-dom0less-arm64-*-xen-version jobs, use *arm64-test-needs
ahead of alpine-3.18-gcc-* (as it's a root container), and to deduplicate
the *-export dependency.
This will allow us to change containers to being non-root one at a time,
rather than all in one go.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Commit 23e2479ff9 ("xen/domain: introduce DOMID_ANY") broke at least:
* xenstored domains
* The Python bindings
* The Ocaml bindings (i.e XAPI, and therefore XenServer/XCP-ng)
by causing Xen to reject 0 (the wildcard sentinel since the start of Xen) and
DOMID_INVALID (since 4.21) without adjusting the callers of xc_domain_create().
Under other circumstances I'd revert, but xenstored has gained a dependent use
of DOMID_ANY.
Fix up the remaining in-tree callers of xc_domain_create(), which covers
init-xenstore-domain and the python bindings.
The Ocaml C bindings, like libxc itself, expose this ABI detail to their
caller, but in this case the caller is still internal to the bindings as a
whole. Adjust the default value in the internal transformation, adding
domid_any for legibility.
Leave a very clear message in CHANGELOG.md that all external callers need
altering.
Fixes: 23e2479ff9 ("xen/domain: introduce DOMID_ANY")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Pau Ruiz Safont <pau.safont@vates.tech>
In FRED mode, the SYSCALL instruction does not modify %rcx/%r11. All current
software using SYSCALL expects the pre-FRED behaviour and spills %rcx/%r11
around the invocation, which is why FRED not doing this goes largely
unnoticed.
However, consider the following migration scenario:
* VM suspends. Hypercall, so SYSCALL, %rcx/%r11 left unmodified
* VM moves to a non-FRED system
* Xen resumes the VM with a real SYSRET instruction
Instead of resuming at the instruction following the SYSCALL instruction, the
VM is resumed at whatever dead value was in %rcx.
In FRED mode, manually adjust %rcx/%r11 when SYSCALL is used and when SYSRET
would have been used.
Regarding the choice of instructions in eretu_exit_to_guest(), a branch would
be a context dependent 50/50 split (i.e. increased chance of mispredict), and
only saves one instruction. The CMOVs read the same cacheline that ERETU is
about to process, so are as close to free as we can reasonably get.
Fixes: 76193ef47d ("x86/pv: System call handling in FRED mode")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Eclair reports that _MSC_VER is undefined, a MISRA Rule 20.9 violation.
This could be fixed by inserting an "&& defined(_MSC_VER)" clause, but the
whole block is entirely useless (it comments out the include of stdint) and
users pull in xen/types.h transitively.
Delete the whole block, and include xen/types.h to make it standalone.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
libxl_nocpuid.c needs the same treatment as libxl_cpuid.c
Fixes: f6c6f2679d ("libxl: Convert libxl__object_to_json() to json-c")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
XENMAPSPACE_gmfn_range was found to be buggy with overlapping ranges, and the
fix is sufficiently far from clear that yours truly decided a test was in
order.
XENMAPSPACE_gmfn_range is exposed by xendevicemodel_relocate_memory() so take
the opportunity to create a better example for others to copy.
The test works as follows. Populate 2M of RAM in the guest, marking the pages
so they can be identified later. Then construct four relocation test
cases (overlapping vs non-overlapping, forwards vs backwards) using the marks
in the guest pages to confirm that the P2M was adjusted correctly.
The use of XEN_DOMCTL_CDF_iommu forces xenmem_add_to_physmap() to limit the
iteration count to a maximum of 16. Use this to force a continuation and
exercise more of the logic.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
... ahead of extending it with a second test.
Rename run_tests() to test_paging_mempool_size() and have it return int so the
new run_tests() can decide whether to run subsequent tests or not.
Comment the parameters to xc_domain_populate_physmap_exact() to make it
clearer what's going on, and remove the page added to the physmap when the
test is done.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Under SVM, there are two copies of guest CR2. One is v->arch.hvm.guest_cr[2]
and one is in the VMCB.
Xen doesn't intercept CR2 accesses, so this mostly goes unnoticed; hardware
loads and saves the guest CR2 in the VMCB across VMRUN/VMExit.
For HAP guests (where #PF is not intercepted, and therefore we don't typically
inject #PF either), this causes the guest CR2 value to be lost on migrate. As
migration is cooperative and not done from the #PF handler, this also goes
unnoticed by guests.
It also means that an emulated MOV-from-CR2 reads a stale value.
Reported-by: Stefano Stabellini <sstabellini@kernel.org>
Fixes: d1bd157fbc ("Big merge the HVM full-virtualisation abstractions.")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Teddy Astie <teddy.astie@vates.tech>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Gitlab CI reported a crash on boot on Alder Lake hardware. The bug is years
old, making it an incredibly rare occurance:
(XEN) *** DOUBLE FAULT ***
(XEN) ----[ Xen-4.22-unstable x86_64 debug=y ubsan=y Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e008:[<ffff82d04077bbc4>] arch/x86/setup.c#reinit_bsp_stack+0xfa/0x160
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor
(XEN) rax: 0000000000000007 rbx: ffff83049a4b0000 rcx: 00000000000006a2
(XEN) rdx: 0000000000000000 rsi: 0000000000000000 rdi: 0000000000000000
(XEN) rbp: ffff83049a4b7f00 rsp: ffff83049a4b7ef8 r8: ffff830497e47000
(XEN) r9: 00000000ffffffff r10: 00000000900c2121 r11: 000000009a392956
(XEN) r12: ffff830497e47000 r13: ffff830497e49f40 r14: 0000000000000000
(XEN) r15: ffff82d0407dad10 cr0: 0000000080050033 cr4: 0000000000f526e0
(XEN) cr3: 0000000043c16000 cr2: fffffffffffffffc
(XEN) fsb: 0000000000000000 gsb: 0000000000000000 gss: 0000000000000000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen code around <ffff82d04077bbc4> (arch/x86/setup.c#reinit_bsp_stack+0xfa/0x160):
(XEN) 00 b9 a2 06 00 00 0f 30 <80> 3d 71 26 f1 ff 00 74 3e 48 8d 93 f8 5f 00 00
(XEN) Valid stack range: ffff83049a4b6000-ffff83049a4b8000, sp=ffff83049a4b7ef8, tss.rsp0=ffff83049a4b7fb0
(XEN) No stack overflow detected. Skipping stack trace.
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) DOUBLE FAULT -- system shutdown
(XEN) ****************************************
This is on the instruction boundary after enabling CET (writing MSR_S_CET) and
prior to establishing SSP. Despite identifying this as a critical window
where any fault was deadly (the CPU tries to push a shadow stack frame at 0,
hence the CR2 value wrapping around to the top of the address space), I
clearly forgot that this meant interrupts too, which are enabled.
Disable interrupts during the critical period.
Fixes: b60ab42db2 ("x86/shstk: Activate Supervisor Shadow Stacks")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Teddy Astie <teddy.astie@vates.tech>
lr_mask doesn't have bits set beyond the hardware limit, with the upper bits
remaining zero. Therefore, for_each_set_bit() is a better option.
For ARM64, bloat-o-meter reports:
Function old new delta
vgic_sync_from_lrs 208 168 -40
but this doesn't highlight that it also removes a call to find_next_bit() from
each loop iteration.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Acked-by: Julien Grall <julien@xen.org>
Use IS_ENABLED() rather than #ifdef to give the compiler visibility into the
block, which in turn removes the #ifdef from the variable block.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Daniel P. Smith <dpsmith@apertussolutions.com>
The new toolchain baseline knows XGETBV, VPXOR and VPOR.
For the other cases using .byte, annotate the toolchain minima.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Teddy Astie <teddy.astie@vates.tech>
The 0x90 is a ModRM byte and technically needs following by an imm32.
Switch to 0xc0 which encodes two %eax operands, and is a complete instruction.
No practical change.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
In real hardware, accesses to the registers cannot fail. The error paths are
just an artefact of the hook functions needing to return something.
The best effort unwind is also something that doesn't exist in real hardware,
and makes the logic more complicated to follow. Instead, use an
ASSERT_UNREACHABLE() with a fallback of injecting #DF. Hitting this path is
an error in Xen.
While adjusting, remove {read,write}_segment() and use {read,write}_msr() to
access MSR_GS_BASE. There's no need to access the other parts of the GS
segment, and this is less work behind the scenes.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Following the removal of XEN_SYSCTL_get_cpu_levelling_caps, asm/cpuid.h
doesn't need to include public/sysctl.h, but several other header files were
picking up their includes transitively through asm/cpuid.h. Untangle them.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Now that the LCAP_* constants are single bits, we can simplify the expressions
using them.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
This hypercall is an addition of mine from commit 67528a3f06 ("x86/cpu:
Sysctl and common infrastructure for levelling context switching", 2016), but
it never got wired into any toolstacks. In the meantime, how we handle CPUID
for guests has evolved substantially.
In order to reuse the AMD levelling infrasturcture for boot time quirks,
levelling_caps is going to have to change. While it's probably safe to expose
this difference, it's safer still to make it an internal detail.
When re-plummbing the LCAP_* constants, turn them all into single bits.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Daniel P. Smith <dpsmith@apertussolutions.com>
This makes the correctness of future changes more obvious.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
While the comment is reasonable, clearing SEP as the only action for an
unknown CPU is useless. Drop the infrastructure.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
These were plausibly useful for debugging when there were 4 words in
x86_capability[], but it's currently 22 words and growing.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Right now, xstate_alloc_save_area() leaves both XSTATE_BV and FTW clear. This
causes a difference in behaviour between FXRSTOR and XRSTOR.
Switch to using using XSTATE's idea of initial configuration which will behave
the same even on pre-XSAVE hardware. Expand the comment to explain why.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The controls for cpu-bugs were ultimately not taken. As the paths are
documented fully below, pick only a small subset for the example. Include
cpupool/ as it exists now.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
LASS is a feature designed in the aftermath of Meltdown, which codifies the
user/kernel split in address space in order to block entire classes of
sidechanels. Specifically, it allows the CPU to terminate any memory access
into the wrong half of the address space based on linear address alone and
without needing a pagewalk.
It is available in CPUs starting with Intel Sierra Forrest (2023).
For now, allow LASS to be opted in to, but leave it off by default. Some work
in the emulator is going to be needed to enable it by default.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The patch introducing these constants predate the feature being renamed.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Having two spellings of the constants makes following the code unnecessarily
difficult. Reduce it to one spelling.
This is an API change in the public headers, but one for the better.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Family 0x1a covers both Zen5 and Zen6, but the latter is not believed to be
vulnerable to entrysign.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
With the GCC toolchain baseline now at 5.1, gcc_3_4.o (covering 3.4 through
4.6) will never get chosen. Drop it.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
For the ARM build only, Eclair reports a R8.4 violation because do_argo_op()
cannot see its declaration. This means that x86 is picking hypercall-defs.h
up transitively while ARM is not. Include xen/hypercall.h explicitly.
Eclair also reports a R8.3 violation because of arg3 and arg4 differing in
name with a raw_ prefix. Because hypercall-defs.h generates both do_argo_op()
and compat_argo_op() from a single description, it's not possible to simply
rename to raw_ in the declaration, as that would force doing the same rename
in compat_argo_op().
In do_argo_op(), drop the split parameter handling, and perform the 32bit
range check via an explicit cast. While adjusting the surrounding logic, drop
unnecessary casts to void * for already pointer arguments in argo_printk().
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
In FRED mode, ERET is stricter than IRET about flags. Notably this means:
* The vm86 bit (bit 17) and IOPL (bits 12,13) must be clear.
* The sticky-1 reserved bit (bit 2) must be set, so dom0_construct() needs to
set X86_EFLAGS_MBS in order for a PV dom0 to start.
* All other reserved bits must be clear.
Xen has been overly lax with reserved bit handling. Adjust
arch_set_info_guest*() and hypercall_iret() which consume flags to clamp the
reserved bits for all guest types.
This is a minor ABI change, but by the same argument as commit
9f892f84c2 ("x86/domctl: Stop using XLAT_cpu_user_regs()"); the reserved
bits would get clamped like this naturally by hardware when the vCPU is run.
The handling of vm86 is also different. Guests under 32bit Xen really could
use vm86 mode, but Long Mode disallows vm86 mode and IRET simply ignores the
bit. Xen's behaviour for a PV32 guest trying to use vm86 mode under a 64bit
Xen is to arrange to deliver #GP at the target of the IRET, rather than to
fail the IRET itself.
However there's no filtering in arch_set_info_guest() itself, and it can't
arrange to queue a #GP at the target, so do the next best thing and fail the
hypercall. This is not expected to create an issue for PV guests, as the
result of such an arch_set_info_guest() previously would be to run supposedly
Real Mode code as Protected Mode code.
This allows PV guests to start when Xen is using FRED mode.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Under FRED, entry_from_pv() handles everything, even system call instructions.
This means more of our logic is written in C now, rather than assembly.
In order to facilitate this, introduce pv_inject_callback(), which reuses
struct trap_bounce infrastructure to inject the syscall/sysenter callbacks.
This in turns requires some !PV compatibility for pv_inject_callback() and
pv_hypercall() which can both be ASSERT_UNREACHABLE().
For each of INT $N, SYSCALL and SYSENTER, FRED gives us interrupted context
which was previously lost. As the guest can't see FRED, Xen has to lose state
in the same way to maintain the prior behaviour.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
ERETU can fault for guest reasons, and like IRET needs special handling to
forward the error into the guest.
As this is largely written in C, take the opportunity to better classify the
sources of error, and in particular, not forward errors that are actually
Xen's fault into the guest, opting for a domain crash instead.
Because ERETU does not enable NMIs if it faults, a corner case exists if an
NMI was taken while in guest context, and the ERETU back out faults. Recovery
must involve an ERETS with the interrupted context's NMI flag.
See the comments for full details.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Under FRED, entry_from_pv() handles everything. To start with, implement
exception handling in the same manner as entry_from_xen(), although we can
unconditionally enable interrupts after the async/fatal events.
After entry_from_pv() returns, test_all_events() needs to run to perform
exception and interrupt injection. Split entry_FRED_R3() into two and
introduce eretu_exit_to_guest() as the latter half, coming unilaterally from
restore_all_guest().
For all of this, there is a slightly complicated relationship with CONFIG_PV.
entry_FRED_R3() must exist irrespective of CONFIG_PV, because it's the
entrypoint registered with hardware. For simplicity, entry_from_pv() is
always called, but it collapses into fatal_trap() in the !PV case.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When FRED is active, hardware automatically swaps GS when changing privilege,
and the SWAPGS instruction is disallowed.
For native OSes using GS as the thread local pointer this is a massive
improvement on the pre-FRED architecture, but under Xen it makes handling PV
guests more complicated. Specifically, it means that GS_BASE and GS_SHADOW
are the opposite way around in FRED mode, as opposed to IDT mode.
This leads to the following changes:
* In load_segments(), we already load both GSes. Account for FRED in the
SWAP() condition and avoid the path with SWAGS.
* In save_segments(), we need to read GS_SHADOW rather than GS_BASE.
* In toggle_guest_mode(), we need to emulate SWAPGS.
* In {read,write}_msr() which access the live registers, GS_SHADOW and
GS_BASE need swapping.
* In do_set_segment_base(), merge the SEGBASE_GS_{USER,KERNEL} cases and
take FRED into account when choosing which base to update.
SEGBASE_GS_USER_SEL was already an LKGS invocation (decades before FRED)
so under FRED needs to be just a MOV %gs. Simply skip the SWAPGSes.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
With the shadow stack and exception handling adjustements in place, we can now
activate FRED when appropriate. Note that opt_fred is still disabled by
default until more development is complete.
Introduce init_fred() to set up all the MSRs relevant for FRED. FRED uses
MSR_STAR (entries from Ring3 only), and MSR_FRED_SSP_SL0 aliases MSR_PL0_SSP
when CET-SS is active. Otherwise, they're all new MSRs.
Also introduce init_fred_tss(). At this juncture we need a TSS set up, even
if it is mostly unused. Reinsert the BUILD_BUG_ON() checking the size of the
TSS against 0x67, this time with a more precise comment.
With init_fred() existing, load_system_tables() and legacy_syscall_init()
should only be used when setting up IDT delivery. Insert ASSERT()s to this
effect, and adjust the various init functions to make this property true.
The FRED initialisation path still needs to write to all system table
registers at least once, even if only to invalidate them. Per the
documentation, percpu_early_traps_init() is responsible for switching off the
boot GDT, which also needs doing even in FRED mode.
Finally, set CR4.FRED in traps_init()/percpu_early_traps_init().
Xen can now boot in FRED mode and run a PVH dom0. PV guests still need more
work before they can be run under FRED.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Under FRED, there's one entrypoint from Ring 3, and one from Ring 0.
FRED gives us a good stack (even for SYSCALL/SYSENTER), and a unified event
frame on the stack with all flags cleared, meaning that all software needs to
do is spill the GPRs with a line of PUSHes. Introduce PUSH_AND_CLEAR_GPRS for
this purpose, along with the matching POP_GPRS.
Introduce entry_FRED_R0() which to a first approximation is complete for all
event handling within Xen.
entry_FRED_R0() needs deriving from entry_FRED_R3(), so introduce a basic
Ring3 handler. There is more work required to make the return-to-guest path
work under FRED.
Also introduce entry_from_{xen,pv}() to be the C level handlers. By simply
copying regs->fred_ss.vector into regs->entry_vector, we can reuse all the
existing fault handlers.
Extend fatal_trap() to render the event type including name, when FRED is
active. This is slightly complicated, because X86_ET_OTHER must not use
vector_name() or SYSCALL and SYSENTER get rendered as #BP and #DB.
This is sufficient to handle all interrupts and exceptions encountered during
development, including plenty of Double Faults.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
FRED doesn't use Supervisor Shadow Stack tokens. This means that:
1) memguard_guard_stack() should not write Supervisor Shadow Stack Tokens.
2) cpu_has_bug_shstk_fracture is no longer relevant when deciding whether to
enable Shadow Stacks or not.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
We wish to make use of opt_fred earlier on boot, which involves moving
traps_init() earlier, but this comes with several ordering complications.
The feature word containing FRED needs collecting in early_cpu_init(), and
legacy_syscall_init() cannot be moved that early because it relies on the
stubs being allocated, yet must be called ahead of cpu_init() so the SYSCALL
linkage MSRs are set up before being cached.
Delaying legacy_syscall_init() is easy enough based on a system_state check.
Reuse bsp_traps_reinit() to cause a call to legacy_syscall_init() to occur at
the same point as previously.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Lets just say this took an unreasonable amount of time and effort to track
down, when trying to move traps_init() earlier during boot.
When the SYSCALL linkage MSRs are not configured ahead of _svm_cpu_up() on the
BSP, context switch clobbers the later-set-up linkage with the 0's cached
here. Amongst other problems, this causes PV guest to enter at 0 in
supervisor mode on the user stack.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Commit 564d261687 ("x86/ctxt-switch: Document and improve GDT handling") put
the initialisation of {,compat_}gdt_l1e into traps_init() but this wasn't a
great choice. Instead, put it in smp_prepare_cpus() which performs the BSP
preparation of variables normally set up by cpu_smpboot_alloc() for APs.
This removes an implicit dependency that prevents traps_init() moving earlier
than move_xen() in the boot sequence.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
... seeing as I've had to thoroughly reverse engineer it for FRED and make
tweaks in places.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
For INT $N instructions (besides $0x80 for which there is a dedicated fast
path), handling is mostly fault-based because of DPL0 gates in the IDT. This
means that when the guest kernel allows the instruction too, Xen must
increment %rip to the end of the instruction before passing a trap to the
guest kernel.
When an INT $N instruction has a prefix, it's longer than two bytes, and Xen
will deliver the "trap" with %rip pointing into the middle of the instruction.
Introduce a new pv_emulate_sw_interrupt() which uses x86_insn_length() to
determine the instruction length, rather than assuming two.
This is a change in behaviour for PV guests, but the prior behaviour cannot
reasonably be said to be intentional.
This change does not affect the INT $0x80 fastpath. Prefixed INT $N
instructions occur almost exclusively in test code or exploits, and INT $0x80
appears to be the only user-usable interrupt gate in contemporary PV guests.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Multiple downstream distros have tried passing discrete CPIO archives and
tripped over this not working. It turns out to be easy to support, so do so.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Teddy Astie <teddy.astie@vates.tech>
sh_destroy_shadow() has a spelling of fallthrough which doesn't match the
permitted pattern. Convert it to a real fallthrough (psedo)keyword.
Elsewhere, insert breaks at the end of the default statements.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
ECLAIR reports an unevaluated side effect in a sizeof() expression (R13.6
violation).
write_atomic() does take care to evaluates each parameter only once, but
expression is complicated to follow. Rewrite it to less resemble an entry
from an obfuscation contest.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
ECLAIR reports that tboot_gen_domain_integrity() is casting away
constness (Rule 11.8 violation).
vmac_set_key() doesn't modify the user_key parameter, so it can become const,
removing the need to cast.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
A range of functions can't see their declarations, requiring the inclusion of
appropriate headers.
For pv/mm.h and pv/traps.h, this requires adjustments to become stand-alone.
In pv/mm.h, swap xen/lib.h for xen/bug.h now that ASSERT_UNREACHABLE() has
moved. For pv/traps.h, pv_trap_callback_registered() needs to see a complete
struct vcpu, and xen/sched.h is not something appropriate to pull in
conditionally.
stack-protector.c can't see the declaration of __stack_chk_guard, and fixing
that reveals that stack-protector.h needs get_cycles() too.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
MISRA Rule 8.2 says that parameters must all have names. Rule 8.3 says that
the names and types must match between declaration and definition.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Use U where necessary (Misra Rule 7.2) and uppercase L (Misra Rule 7.3).
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
All of these are simple cases of using typeof() to avoid multiple parameter
evaluation. Using auto avoids multiple textural expansion also.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Bertrand Marquis <bertrand.marquis@arm.com>
There's no use creating a typed copy of a macro argument, simply to use it to
create a second typed copy. Remove the indirection, halving the number of
local variables created in scope.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Now that the type handling is entirely uniform, it's easier to see that
__declare_arg_3() can use __declare_arg_2() and so on, just like the larger
__declare_arg_$()'s already do.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Andrew Cooper