All arches truncate the cpu_khz without taking into account the less
significant digits. Instead use DIV_ROUND() when scaling from Hz to kHz
to get as more accurate kHz value.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 986707b461
master date: 2026-04-16 15:28:32 +0200
When running HVM guest in native TSC mode avoid using the recalculated vTSC
scaling factors based on the cpu_khz value. Using the kHz based frequency
leads to the TSC scaling values possibly not being the same as the ones
used by the per CPU cpu_time->tsc_scale field, which introduces skew
between the guest and Xen's calculations of the system time.
On a 2gHz system, where the frequency is possibly detected as 1999999999Hz
(note this is a worse-case scenario), the cpu_khz variable will be set to
1999999kHz, and hence 999Hz cycles will be not accounted for per second.
Over a second (the time synchronization period), this leads to a skew of:
cycles * 1 / (Hz freq) = 999 / 1999999999 = 499,5ns
So far this has gone unnoticed because the time synchronization rendezvous
forces the update of the tsc_timestamp and system_time fields in the vCPU
time info area, and hence the skew only accumulates up to the rendezvous
period. Attempting to remove the rendezvous causes the skew to grow
unbounded.
Fix by using the native TSC scaling values (as used by Xen) when the guest
TSC is not scaled.
Fixes: eab8a90be7 ("x86/time: scale host TSC in pvclock properly")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 305c6b901a
master date: 2026-04-16 15:28:32 +0200
The per-channel ENABLE bit is to solely be driven by hpet_enable_channel()
and hpet_msi_{,un}mask(). It doesn't need setting immediately. Except for
the (possible) channel put in legacy mode we don't do so during boot
either.
Instead reset ->arch.cpu_mask, to avoid msi_compose_msg() yielding an
all-zero message (when the passed in CPU mask has no online CPUs). Nothing
would later call msi_compose_msg() / hpet_msi_write(), and hence nothing
would later produce a well-formed message template in
hpet_events[].msi.msg.
Fixes: 15aa6c6748 ("amd iommu: use base platform MSI implementation")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: c4d5119195
master date: 2026-04-16 10:44:16 +0200
With an expiry value on the order of 10 hours, and with a bus scale value
of 256k (as supplied by qemu), the (signed) multiplication will be UB. As
we've checked that the value is positive, we mean unsigned multiplication
anyway. Yet let's play safe against even larger expiry and bus scale
values, leveraging the compiler builtin that there is for this purpose.
While there also drop the stray cast from the actual TMICT write.
Fixes: 9062553a0d ("added time and accurate timer support")
Fixes: b95beb1858 ("x86: Clean up APIC local timer handling")
Reported-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
master commit: 4f14d97a62
master date: 2026-04-16 10:29:54 +0200
A killed timer will ignore further set_timer() calls, and hence won't be
re-armed unless it's initialized again. Use stop_timer() instead of
kill_timer() in time_suspend(), so that the set_timer() call in
time_resume() successfully re-arms the timer. Otherwise time calibration
is no longer scheduled (and executed) after resuming from S3 suspend.
Fixes: 6d90db1a2c ("x86: rendezvous-based local time calibration")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
master commit: 2d670d258b
master date: 2026-04-10 14:31:10 +0200
If the user has set baudrate as "auto" (use whatever was configured) but
Xen cannot find any set baudrate use a default fallback baudrate of
115200.
Otherwise the console will very likely end up being non-functional, as
further values are derived from the baudrate (like the transmit timeout),
and using -1 will result in screwed parameters being derived from the
baudrate.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
master commit: 12e50da43d
master date: 2026-04-10 14:30:50 +0200
The supported features ELF notes was tested only if the dom0 was
PV. Factor out a function to check ELF notes and reuse it even
for PVH.
Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 44429b6931
master date: 2026-04-09 21:04:58 +0100
We should not prematurely increment the pointer, to avoid inadvertently
skipping a NUL terminator.
Fixes: 6d9199bd0f ("x86-64: enable hypervisor output on VESA frame buffer")
Reported-by: Kamil Frankowicz <kamil.frankowicz@cert.pl>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 3bcce461a3
master date: 2026-04-07 13:54:37 +0200
Advancing past a character without knowing whether that's the NUL char is
at risk of reading beyond the buffer.
Fixes: 25de1f6920 ("Allow arch-specific defaults to be specified for ns16550")
Reported-by: Kamil Frankowicz <kamil.frankowicz@cert.pl>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 2c43b5611f
master date: 2026-04-07 13:54:19 +0200
strsep(), unless passed a 1st argument which itself points at a NULL
pointer, won't return NULL. Instead if the separator(s) looked for isn't
found, that pointed-to field would be updated to hold NULL. Hence the
important check in get_token() is for token to be non-NULL after the call
(which get_token()'s callers rely upon), while the checking of param_name
is left there just in case.
Fixes: 97fd49a7e0 ("ns16550: add support for UART parameters to be specifed with name-value pairs")
Reported-by: Kamil Frankowicz <kamil.frankowicz@cert.pl>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: b27d3b45c7
master date: 2026-04-07 13:53:49 +0200
We can easily avoid the risk of wrapping UINT_MAX <-> 0 by applying a
check against the compile-time-constant maximum number of colors.
Additionally the overflow checks suffered from an off-by-1, as the parsed
ranges are inclusive (e.g. end == start being possible, requiring 1 array
slot, while availability of 0 slots was checked in that case).
Fixes: 6cdea3444e ("xen/arm: add Dom0 cache coloring support")
Reported-by: Mykola Kvach <xakep.amatop@gmail.com>
Reported-by: Kamil Frankowicz <kamil.frankowicz@cert.pl>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
xen/common: llc-coloring: Fix off-by-one in parse_color_config()
The check uses (*num_colors + (end - start + 1)) >= max_num_colors, which
rejects a configuration where exactly max_num_colors colors are specified.
For example, if max_num_colors is 4 and *num_colors is 0, a range "0-3"
gives (end - start + 1) = 4, and (0 + 4) >= 4 is true, incorrectly
returning -EINVAL.
Fix this by switching the overflow condition to the state before commit
cba8a584de that regressed the behavior (i.e. don't add 1).
Fixes: cba8a584de ("llc-coloring: improve checking while parsing")
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
master commit: cba8a584de
master date: 2026-04-07 13:52:44 +0200
master commit: dc28f531e6
master date: 2026-04-10 12:14:05 +0200
The message emitted by pre_parse() pretty clearly states the original
intention. Yet what it said wasn't done, and would have been unfriendly to
the user. Hence accesses past the allocated buffer were possible. Insert a
terminating NUL immediately past the data read, to then drop the no longer
applicable message.
NB: The iscntrl() check of just the last byte is more strict than what
pre_parse() would accept without issuing its prior message, yet I'd like
to keep the new logic reasonably simple. Config files shouldn't be huge,
and we shouldn't be _that_ short of memory (or we'd fail elsewhere pretty
soon).
Fixes: bf6501a62e ("x86-64: EFI boot code")
Reported-by: Kamil Frankowicz <kamil.frankowicz@cert.pl>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
master commit: df75f77092
master date: 2026-04-07 08:59:15 +0200
Right now, xstate_alloc_save_area() leaves both XSTATE_BV and FTW clear. This
causes a difference in behaviour between FXRSTOR and XRSTOR.
Switch to using using XSTATE's idea of initial configuration which will behave
the same even on pre-XSAVE hardware. Expand the comment to explain why.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 639de2aafa
master date: 2026-03-30 15:48:13 +0100
Read the existing PCI command register and only add the required bits to
it, as to avoid clearing bits that might be possibly set by the firmware
already, which might put the device into a non-working state.
Xen being the owner of the PCI device it's legitimately allowed to change
command registers bits. Be careful however in case buggy devices require
certain command register bits to be enabled, while not strictly required
from Xen's usage.
Fixes: f2ff5d6628 ("ns16550: enable PCI serial card usage")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: ebf5439725
master date: 2026-03-30 16:43:14 +0200
Auto-discovered serial PCI devices when using dev=amt|pci won't get
->ps_bdf_enable, and as such some of the logic (like making sure the
respective BARs are enabled) won't be applied to them.
Fix by unconditionally setting ->ps_bdf_enable for all PCI serial devices,
and removing the special case that was done in some places by checking
whether the ->bar was set. This also allows simplifying the logic in
pci_serial_early_init().
Fixes: 9738db88f6 ("xen: Automatically find serial port on PCI/PCIe and AMT devices.")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 4bf3336062
master date: 2026-03-30 16:43:14 +0200
Commit 19ab8356ab ("tools: remove support for running a guest with
qemu-traditional") made passing --dm_version to mk_dsdt mandatory on
x86 but didn't fix all the invocations of it.
The previous default was qemu-xen-traditional so with that removed set
the dm_version to qemu-xen for this invocation.
Without this change, Xen fails to build on x86 when using
--enable-rombios.
Fixes: 19ab8356ab ("tools: remove support for running a guest with qemu-traditional")
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: ea12f9d257
master date: 2026-03-27 18:56:42 +0000
When using key pairs for the uart configuration (com1 and com2 command line
options), report an error if the passed device=<type> is not recognized
instead of silently ignoring it.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 550ee7c198
master date: 2026-03-27 14:52:40 +0100
By initializing major_frame in a653sched_init(), an idle period was
inadvertently introduced into the default schedule. Account for the
initial value of major_frame when populating the default schedule.
Fixes: f2927d8343 ("xen/arinc653: fix delay in the start of major frame")
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
master commit: 6d412c0599
master date: 2026-03-26 10:49:11 +0100
Incrementing ->sched_index between bounds check and array access may
result in accessing one past the array when that is fully filled
(->num_schedule_entries == ARINC653_MAX_DOMAINS_PER_SCHEDULE).
Fixes: 22787f2e10 ("ARINC 653 scheduler")
Reported-by: Kamil Frankowicz <kamil.frankowicz@cert.pl>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Stewart Hildebrand <stewart@stew.dk>
master commit: 5a1121633f
master date: 2026-03-26 10:47:41 +0100
When that code still lived in a653sched_init(), it was redundant with the
earlier memset() / xzalloc(). Once moved, the full structure field needs
setting, as dom_handle_cmp() uses memcmp() and arinc653_sched_set() may
have clobbered entries. Rather than implying the handle to be all zero,
copy the handle out of the domain structure.
Fixes: 9f0c658bae ("arinc: add cpu-pool support to scheduler")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Stewart Hildebrand <stewart@stew.dk>
master commit: 2a0ee29f5e
master date: 2026-03-26 10:47:08 +0100
MCE init for APs was broken when CPU feature re-checking was added. At the
same time make sure we don't bypass setup_doitm() (and whatever else may
be added to the bottom of identify_cpu()).
Fixes: bb502a8ca5 ("x86: check feature flags after resume")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: 5ebd4dd141
master date: 2026-03-24 12:05:42 +0100
Most (if not all) Intel CPUs starting from Sandy Bridge have up to 8
event selectors per core, which could be halved per hyperthread.
However, current PV emulation logic doesn't support up to 8 event selector,
leading to errors when trying to access them, hence, preventing Linux from
driving the vPMU correctly.
Make sure up to MSR_P6_EVNTSEL(7) is usable, which is the same upper bound as
used in VMX code.
The check if the event selector actually exist for the hardware is done in
core2_vpmu_do_{rdmsr,wrmsr}, hence we're not allowing to access non-existent
MSRs.
Fixes: 27c5541986 ("x86/VPMU: add support for PMU register handling on PV guests")
Signed-off-by: Teddy Astie <teddy.astie@vates.tech>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 4e9d9e7cc9
master date: 2026-03-24 12:04:46 +0100
Hans reports that Xen no longer builds on Debian unstable/sid:
Assembler messages:
{standard input}:474: Error: unknown or missing system register name at operand 1 -- `msr TEECR32_EL1,x0'
{standard input}:480: Error: unknown or missing system register name at operand 1 -- `msr TEEHBR32_EL1,x0'
{standard input}:488: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEECR32_EL1'
{standard input}:494: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEEHBR32_EL1'
make[5]: *** [Rules.mk:249: arch/arm/domain.o] Error 1
This turns out to be an intentional change in binutils. ThumbEE was dropped
ahead of ARM v8 (i.e. AArch64).
Xen supports ARM v7+virt extensions so in principle we could #ifdef
CONFIG_ARM_32 to keep it working. However, there was apparently no use of
ThumbEE outside of demo code, so simply drop it.
On ThumbEE capable hardware, unconditionally trap ThumbEE instructions, and
drop the context switching logic for TEE{CR,HBR}32.
Reported-by: Hans van Kranenburg <hans@knorrie.org>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
xen/arm: Set ThumbEE as not present in PFR0
Force ThumbEE support to not available in the version of the PFR0
register value we present to guest.
Xen does not support ThumbEE and will trap all access to ThumbEE
registers so do not report it being supported if the hardware supports
it.
Fixes: 5bbe1fe413 ("ARM: Drop ThumbEE support")
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Tested-by: Luca Fancellu <luca.fancellu@arm.com>
master commit: 5bbe1fe413
master date: 2025-12-05 18:06:19 +0000
master commit: 1ecb5946bd
master date: 2026-01-05 10:33:03 +0000
This reverts commit 6228ea8acd and its
prereq 15cd2b8f1b ("xen/mm: remove
aliasing of PGC_need_scrub over PGC_allocated"). Further fixes are
needed before these can be considered for backporting.
Family 0x1a covers both Zen5 and Zen6, but the latter is not believed to be
vulnerable to entrysign.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit bd15fdedaf)
In FRED mode, ERET is stricter than IRET about flags. Notably this means:
* The vm86 bit (bit 17) and IOPL (bits 12,13) must be clear.
* The sticky-1 reserved bit (bit 2) must be set, so dom0_construct() needs to
set X86_EFLAGS_MBS in order for a PV dom0 to start.
* All other reserved bits must be clear.
Xen has been overly lax with reserved bit handling. Adjust
arch_set_info_guest*() and hypercall_iret() which consume flags to clamp the
reserved bits for all guest types.
This is a minor ABI change, but by the same argument as commit
9f892f84c2 ("x86/domctl: Stop using XLAT_cpu_user_regs()"); the reserved
bits would get clamped like this naturally by hardware when the vCPU is run.
The handling of vm86 is also different. Guests under 32bit Xen really could
use vm86 mode, but Long Mode disallows vm86 mode and IRET simply ignores the
bit. Xen's behaviour for a PV32 guest trying to use vm86 mode under a 64bit
Xen is to arrange to deliver #GP at the target of the IRET, rather than to
fail the IRET itself.
However there's no filtering in arch_set_info_guest() itself, and it can't
arrange to queue a #GP at the target, so do the next best thing and fail the
hypercall. This is not expected to create an issue for PV guests, as the
result of such an arch_set_info_guest() previously would be to run supposedly
Real Mode code as Protected Mode code.
This allows PV guests to start when Xen is using FRED mode.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit dbd936248f)
Under FRED, entry_from_pv() handles everything, even system call instructions.
This means more of our logic is written in C now, rather than assembly.
In order to facilitate this, introduce pv_inject_callback(), which reuses
struct trap_bounce infrastructure to inject the syscall/sysenter callbacks.
This in turns requires some !PV compatibility for pv_inject_callback() and
pv_hypercall() which can both be ASSERT_UNREACHABLE().
For each of INT $N, SYSCALL and SYSENTER, FRED gives us interrupted context
which was previously lost. As the guest can't see FRED, Xen has to lose state
in the same way to maintain the prior behaviour.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 76193ef47d)
ERETU can fault for guest reasons, and like IRET needs special handling to
forward the error into the guest.
As this is largely written in C, take the opportunity to better classify the
sources of error, and in particular, not forward errors that are actually
Xen's fault into the guest, opting for a domain crash instead.
Because ERETU does not enable NMIs if it faults, a corner case exists if an
NMI was taken while in guest context, and the ERETU back out faults. Recovery
must involve an ERETS with the interrupted context's NMI flag.
See the comments for full details.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 99edc6c2a0)
Under FRED, entry_from_pv() handles everything. To start with, implement
exception handling in the same manner as entry_from_xen(), although we can
unconditionally enable interrupts after the async/fatal events.
After entry_from_pv() returns, test_all_events() needs to run to perform
exception and interrupt injection. Split entry_FRED_R3() into two and
introduce eretu_exit_to_guest() as the latter half, coming unilaterally from
restore_all_guest().
For all of this, there is a slightly complicated relationship with CONFIG_PV.
entry_FRED_R3() must exist irrespective of CONFIG_PV, because it's the
entrypoint registered with hardware. For simplicity, entry_from_pv() is
always called, but it collapses into fatal_trap() in the !PV case.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 87cfcbe9f0)
When FRED is active, hardware automatically swaps GS when changing privilege,
and the SWAPGS instruction is disallowed.
For native OSes using GS as the thread local pointer this is a massive
improvement on the pre-FRED architecture, but under Xen it makes handling PV
guests more complicated. Specifically, it means that GS_BASE and GS_SHADOW
are the opposite way around in FRED mode, as opposed to IDT mode.
This leads to the following changes:
* In load_segments(), we already load both GSes. Account for FRED in the
SWAP() condition and avoid the path with SWAGS.
* In save_segments(), we need to read GS_SHADOW rather than GS_BASE.
* In toggle_guest_mode(), we need to emulate SWAPGS.
* In {read,write}_msr() which access the live registers, GS_SHADOW and
GS_BASE need swapping.
* In do_set_segment_base(), merge the SEGBASE_GS_{USER,KERNEL} cases and
take FRED into account when choosing which base to update.
SEGBASE_GS_USER_SEL was already an LKGS invocation (decades before FRED)
so under FRED needs to be just a MOV %gs. Simply skip the SWAPGSes.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 9ea7d7f939)
With the shadow stack and exception handling adjustements in place, we can now
activate FRED when appropriate. Note that opt_fred is still disabled by
default until more development is complete.
Introduce init_fred() to set up all the MSRs relevant for FRED. FRED uses
MSR_STAR (entries from Ring3 only), and MSR_FRED_SSP_SL0 aliases MSR_PL0_SSP
when CET-SS is active. Otherwise, they're all new MSRs.
Also introduce init_fred_tss(). At this juncture we need a TSS set up, even
if it is mostly unused. Reinsert the BUILD_BUG_ON() checking the size of the
TSS against 0x67, this time with a more precise comment.
With init_fred() existing, load_system_tables() and legacy_syscall_init()
should only be used when setting up IDT delivery. Insert ASSERT()s to this
effect, and adjust the various init functions to make this property true.
The FRED initialisation path still needs to write to all system table
registers at least once, even if only to invalidate them. Per the
documentation, percpu_early_traps_init() is responsible for switching off the
boot GDT, which also needs doing even in FRED mode.
Finally, set CR4.FRED in traps_init()/percpu_early_traps_init().
Xen can now boot in FRED mode and run a PVH dom0. PV guests still need more
work before they can be run under FRED.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit bbfb30f649)
Under FRED, there's one entrypoint from Ring 3, and one from Ring 0.
FRED gives us a good stack (even for SYSCALL/SYSENTER), and a unified event
frame on the stack with all flags cleared, meaning that all software needs to
do is spill the GPRs with a line of PUSHes. Introduce PUSH_AND_CLEAR_GPRS for
this purpose, along with the matching POP_GPRS.
Introduce entry_FRED_R0() which to a first approximation is complete for all
event handling within Xen.
entry_FRED_R0() needs deriving from entry_FRED_R3(), so introduce a basic
Ring3 handler. There is more work required to make the return-to-guest path
work under FRED.
Also introduce entry_from_{xen,pv}() to be the C level handlers. By simply
copying regs->fred_ss.vector into regs->entry_vector, we can reuse all the
existing fault handlers.
Extend fatal_trap() to render the event type including name, when FRED is
active. This is slightly complicated, because X86_ET_OTHER must not use
vector_name() or SYSCALL and SYSENTER get rendered as #BP and #DB.
This is sufficient to handle all interrupts and exceptions encountered during
development, including plenty of Double Faults.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 0e9cc04453)
FRED doesn't use Supervisor Shadow Stack tokens. This means that:
1) memguard_guard_stack() should not write Supervisor Shadow Stack Tokens.
2) cpu_has_bug_shstk_fracture is no longer relevant when deciding whether to
enable Shadow Stacks or not.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 7634f56625)
We wish to make use of opt_fred earlier on boot, which involves moving
traps_init() earlier, but this comes with several ordering complications.
The feature word containing FRED needs collecting in early_cpu_init(), and
legacy_syscall_init() cannot be moved that early because it relies on the
stubs being allocated, yet must be called ahead of cpu_init() so the SYSCALL
linkage MSRs are set up before being cached.
Delaying legacy_syscall_init() is easy enough based on a system_state check.
Reuse bsp_traps_reinit() to cause a call to legacy_syscall_init() to occur at
the same point as previously.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 8e4775d118)
Lets just say this took an unreasonable amount of time and effort to track
down, when trying to move traps_init() earlier during boot.
When the SYSCALL linkage MSRs are not configured ahead of _svm_cpu_up() on the
BSP, context switch clobbers the later-set-up linkage with the 0's cached
here. Amongst other problems, this causes PV guest to enter at 0 in
supervisor mode on the user stack.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit c2f15b0fff)
Commit 564d261687 ("x86/ctxt-switch: Document and improve GDT handling") put
the initialisation of {,compat_}gdt_l1e into traps_init() but this wasn't a
great choice. Instead, put it in smp_prepare_cpus() which performs the BSP
preparation of variables normally set up by cpu_smpboot_alloc() for APs.
This removes an implicit dependency that prevents traps_init() moving earlier
than move_xen() in the boot sequence.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit a1e6afbf54)
Under FRED, the SWAPGS instruction is disallowed. Therefore we must use the
MSR path instead.
read_registers() is in the show_registers() path, so this change allows Xen to
render it's current state without suffering #UD (and recursing until the stack
guard page is hit).
All hardware with FRED is expected to have some kind of non-serialising access
to these registers.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 3ff1030ea7)
FRED and IDT differ by a Supervisor Token on the base of the shadow stack.
This means that switch_stack_and_jump() needs to discard one extra word when
FRED is active.
Fix a typo in the parameter name, which should be shstk_base.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 40fbf97177)
FRED provides %cr2 in the the stack frame, avoiding the need to read %cr2
manually.
Rename do_page_fault() to handle_PF(), and update it to take cr2, still named
addr for consistency.
Introduce a new handle_PF_IDT() which reads %cr2 and conditionally re-enables
interrupts.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 4153eb7660)
FRED provides PENDING_DBG in the the stack frame, avoiding the need to read
%dr6 manually.
Rename do_debug() to handle_DB(), and update it to take a dbg field using
positive polarity.
Introduce a new handle_DB_IDT() which reads %dr6.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit d9974d7860)
This is really a rearrangement to make adding FRED support easier.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 524f3fa35e)
Under FRED, SETSSBSY is disallowed, and we want to be setting up FRED prior to
setting up shadow stacks. As we still need Supervisor Tokens in IDT mode, we
need mode-specific logic to establish SSP.
In FRED mode, write a Restore Token, RSTORSSP it, and discard the resulting
Previous-SSP token.
No change outside of FRED mode.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 3037d8b7eb)
It turns out that using the higher level helpers adjacent like this leads to
terrible code generation. Due to -fno-strict-aliasing, the store into state
invalidates the read_cr4() address calculation (which is really cpu_info->cr4
under the hood), meaning that it can't be hoisted.
As a result we get "locate the top of stack block, get cr4, and see if
FSGSBASE is set" repeated 3 times, and an unreasonable number of basic blocks.
Hoist the calculation manually, which results in two basic blocks.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 42a0709f37)
... and rdmsr() while here.
Most of these accesses are in fastpaths and do not need serialising behaviour,
but the write side is serialising on all Intel hardware as well as older AMD
hardware.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 6b448a92a3)
Right now they're inline in {read,write}_gs_shadow(), but we're going to need
to use these elsewhere to support FRED.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 9900679fa9)
Mirroring the cleanup to rdmsr(), do the same to wrmsr(). It now has the same
API as wrmsrl(), but we'll want to drop that wrapper in due course.
It's telling that almost all remaining users pass in 0. Most are converted
directly to WRMSRNS, but a few are not.
MSR_VIRT_SPEC_CTRL is unconditionally intercepted and is orders of magnitude
more expensive than just serialising. In disable_lapic_nmi_watchdog(), the P4
case won't run on hardware which has anything more than plain WRMSR.
For CTR_WRITE() in op_model_athlon.c there is a logical change in behaviour,
but it's fixing a bug. Peformance counters typically get written to -(count)
as they generate an interrupt on overflow. The performance counters even in
the K8 were 48 bits wide, and this wrmsr() not being a wrmsrl() appears to
have been an oversight in commit b5103d692a ("x86 oprofile: use
rdmsrl/wrmsrl") which converted all other users, and appears to be the last
time there was an attempt to unify the MSR APIs.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 389347a75f)
Roger Pau Monné
Jan Beulich
Frediano Ziglio
Andrew Cooper
Ross Lagerwall
Stewart Hildebrand
Teddy Astie