Commit Graph
46427 Commits
Author SHA1 Message Date
Roger Pau Monné aba830fc0d x86/viridian: avoid NULL pointer dereference in update_reference_tsc()
The function is only called when the MSR has the enabled bit set, but even
then the page might not be mapped because the guest provided gfn is not
suitable.

Prevent a NULL pointer dereference in update_reference_tsc() by checking
whether the page is mapped.

This is CVE-2025-27466 / part of XSA-472.

Fixes: 386b336522 ('viridian: use viridian_map/unmap_guest_page() for reference tsc page')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 5776a2e9db
master date: 2025-09-09 14:11:09 +0200
2025-09-09 14:25:01 +02:00
Roger Pau Monné 51190865a4 x86/suspend: unconditionally raise a timer softirq on resume
The current code to restore the timer state on resume is incomplete.  While
the local APIC Initial Count Register is saved and restored across
suspension (even if possibly no longer accurate since it's not adjusted to
account for the time spent in suspension), the TSC deadline MSR is not
saved and restored, hence hosts using the TSC deadline timer will likely
get stuck when resuming from suspension.

The lack of restoring of the TSC deadline MSR was mitigated by the raising
of a timer softirq in mwait_idle_with_hints() if the timer had expired,
previous to commit 3faf0866a3, which removed
that logic.

This patch fixes the usage of the TSC deadline timer with suspension, by
unconditionally raising a timer softirq on resume, that will take care of
rearming the hardware timer.  Given that a timer softirq will be
unconditionally risen, there's no need to save and restore the APIC Initial
Count Register anymore either.

Note that secondary processors don't need this special treatment when
resuming, since they are offlined before suspension and brought back up
during resume, the first timer that gets setup will trigger a timer softirq
unconditionally, for example from sched_migrate_timers() that gets called
for each secondary processor.

Link: https://github.com/QubesOS/qubes-issues/issues/10110
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Fixes: fd1291a826 ('X86: Prefer TSC-deadline timer in Xen')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
master commit: 757abedabb
master date: 2025-08-27 09:33:12 +0200
2025-09-03 14:09:11 +02:00
Jan Beulich 25a125c457 x86/idle: undo use of MONITOR/MWAIT mnemonics
While the change is fine on staging, where the toolchain baseline was
moved, we need to remain compatible with older gas on stable branches
for now.

Fixes: fa254938f0 ("x86/idle: Move monitor()/mwait() wrappers into cpu-idle.c")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
2025-07-09 12:19:27 +02:00
Andrew Cooper 13d3020ca1 x86/spec-ctrl: Mitigate Transitive Scheduler Attacks
TSA affects AMD Fam19h CPUs (Zen3 and 4 microarchitectures).

Three new CPUID bits have been defined.  Two (TSA_SQ_NO and TSA_L1_NO)
indicate that the system is unaffected, and must be synthesised by Xen on
unaffected parts to date.

A third new bit indicates that VERW now has a flushing side effect.  Xen must
synthesise this bit on affected systems based on microcode version.  As with
other VERW-based flushing features, VERW_CLEAR needs OR-ing across a resource
pool, and guests which have seen it can safely migrate in.

This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 578b34adb1)
2025-07-08 15:00:28 +01:00
Andrew Cooper d8cab42238 x86/idle: Rearrange VERW and MONITOR in mwait_idle_with_hints()
In order to mitigate TSA, Xen will need to issue VERW before going idle.

On AMD CPUs, the VERW scrubbing side effects cancel an active MONITOR, causing
the MWAIT to exit without entering an idle state.  Therefore the VERW must be
ahead of MONITOR.

Split spec_ctrl_enter_idle() in two and allow the VERW aspect to be handled
separately.  While adjusting, update a stale comment concerning MSBDS; more
issues have been mitigated using VERW since it was written.

By moving VERW earlier, it is ahead of the determination of whether to go
idle.  We can't move the check on softirq_pending (for correctness reasons),
but we can duplicate it earlier as a best effort attempt to skip the
speculative overhead.

This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 01429bbb0a)
2025-07-07 22:52:11 +01:00
Andrew Cooper 41b62f8a96 x86/ucode: Digests for TSA microcode
AMD are releasing microcode for TSA, so extend the known-provenance list with
their hashes.  These were produced before the remediation of the microcode
signature issues (the entrysign vulnerability), so can be OS-loaded on
out-of-date firmware.

Include an off-by-default check for the sorted-ness of patch_digests[].  It's
not worth running generally under SELF_TESTS, but is useful when editing the
digest list.

This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit a8708ea8bd)
2025-07-07 22:52:11 +01:00
Andrew Cooper 82c302a62d x86/cpu-policy: Infrastructure for CPUID leaf 0x80000021.ecx
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 6e3c3cea09)
2025-07-07 22:52:11 +01:00
Andrew Cooper b6266b20f1 x86/cpu-policy: Rearrange guest_common_*_feature_adjustments()
Turn the if()s into switch()es, as we're going to need AMD sections.

Move the RTM adjustments into the Intel section, where they ought to live.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 7c5025394d)
2025-07-07 22:52:11 +01:00
Andrew Cooper cd46db24e5 x86/xen-cpuid: Fix backports of new features
Xen 4.18 doesn't automatically generate feature names like Xen 4.19 does, and
these hunks were missed on prior security fixes.

Fixes: 8bced9a15c ("x86/spec-ctrl: Support for SRSO_U/S_NO and SRSO_MSR_FIX")
Fixes: f132c82fa6 ("x86/spec-ctrl: Synthesise ITS_NO to guests on unaffected hardware")
Fixes: dba0556612 ("x86/spec-ctrl: Support Intel's new PB-OPT")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
2025-07-07 22:52:11 +01:00
Andrew Cooper 9c0becc30e x86/idle: Fix buggy "x86/mwait-idle: enable interrupts before C1 on Xeons"
The check of this_softirq_pending must be performed with irqs disabled, but
this property was broken by an attempt to optimise entry/exit latency.

Commit c227233ad64c in Linux (which we copied into Xen) was fixed up by
edc8fc01f608 in Linux, which we have so far missed.

Going to sleep without waking on interrupts is nonsensical outside of
play_dead(), so overload this to select between two possible MWAITs, the
second using the STI shadow to cover MWAIT for exactly the same reason as we
do in safe_halt().

Fixes: b17e0ec72e ("x86/mwait-idle: enable interrupts before C1 on Xeons")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 9b0f0f6e23)
2025-07-07 22:52:11 +01:00
Andrew Cooper 9548f45217 x86/idle: Implement a new MWAIT IPI-elision algorithm
In order elide IPIs, we must be able to identify whether a target CPU is in
MWAIT at the point it is woken up.  i.e. the store to wake it up must also
identify the state.

Create a new in_mwait variable beside __softirq_pending, so we can use a
CMPXCHG to set the softirq while also observing the status safely.  Implement
an x86 version of arch_pend_softirq() which does this.

In mwait_idle_with_hints(), advertise in_mwait, with an explanation of
precisely what it means.  X86_BUG_MONITOR can be accounted for simply by not
advertising in_mwait.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 3e0bc4b503)
2025-07-07 22:52:11 +01:00
Andrew Cooper 2ec55f622c xen/softirq: Rework arch_skip_send_event_check() into arch_set_softirq()
x86 is the only architecture wanting an optimisation here, but the
test_and_set_bit() is a store into the monitored line (i.e. will wake up the
target) and, prior to the removal of the broken IPI-elision algorithm, was
racy, causing unnecessary IPIs to be sent.

To do this in a race-free way, the store to the monited line needs to also
sample the status of the target in one atomic action.  Implement a new arch
helper with different semantics; to make the softirq pending and decide about
IPIs together.  For now, implement the default helper.  It will be overridden
by x86 in a subsequent change.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit b473e5e212)
2025-07-07 22:52:11 +01:00
Andrew Cooper df49dab697 x86/idle: Convert force_mwait_ipi_wakeup to X86_BUG_MONITOR
We're going to want alternative-patch based on it.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit b0ca0f93f4)
2025-07-07 22:52:11 +01:00
Andrew Cooper 709a223389 x86/idle: Drop incorrect smp_mb() in mwait_idle_with_hints()
With the recent simplifications, it becomes obvious that smp_mb() isn't the
right barrier.  Strictly speaking, MONITOR is ordered as a load, but smp_rmb()
isn't correct either, as this only pertains to local ordering.  All we need is
a compiler barrier().

Merge the barier() into the monitor() itself, along with an explantion.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit e7710dd843)
2025-07-07 22:52:11 +01:00
Andrew Cooper 8b348e9a21 x86/idle: Remove broken MWAIT implementation
cpuidle_wakeup_mwait() is a TOCTOU race.  The cpumask_and() sampling
cpuidle_mwait_flags can take a arbitrary period of time, and there's no
guarantee that the target CPUs are still in MWAIT when writing into
mwait_wakeup(cpu).

The consequence of the race is that we'll fail to IPI certain targets.  Also,
there's no guarantee that mwait_idle_with_hints() will raise a TIMER_SOFTIRQ
on it's way out.

The fundamental bug is that the "in_mwait" variable needs to be in the
monitored line, and not in a separate cpuidle_mwait_flags variable, in order
to do this in a race-free way.

Arranging to fix this while keeping the old implementation is prohibitive, so
strip the current one out in order to implement the new one cleanly.  In the
interim, this causes IPIs to be used unconditionally which is safe albeit
suboptimal.

Fixes: 3d521e933e ("cpuidle: mwait on softirq_pending & remove wakeup ipis")
Fixes: 1adb34ea84 ("CPUIDLE: re-implement mwait wakeup process")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 3faf0866a3)
2025-07-07 22:52:11 +01:00
Andrew Cooper 60820cd4fd x86/cpu-policy: Fix handling of leaf 0x80000021
When support was originally introduced, ebx, ecx and edx were reserved and
should have been zeroed in recalculate_misc() to avoid leaking into guests.

Since then, fields have been added into ebx.  Guests can't load microcode, so
shouldn't see ucode_size, and while in principle we do want to support larger
RAP sizes in guests, virtualising this for guests depends on AMD procuding any
official documentation for ERAPS, which is long overdue and with no ETA.

This patch will cause a difference in guests on Zen5 CPUs, but as the main
ERAPS feature is hidden, guests should be ignoring the rap_size field too.

Fixes: e9b4fe2636 ("x86/cpuid: support LFENCE always serialising CPUID bit")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 10dc35c516)
2025-07-07 22:52:11 +01:00
Andrew Cooper dcc9c3ae8f x86/cpu-policy: Simplify logic in guest_common_default_feature_adjustments()
For features which are unconditionally set in the max policies, making the
default policy to match the host can be done with a conditional clear.

This is simpler than the unconditional clear, conditional set currently
performed.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 30f8fed68f)
2025-07-07 22:52:11 +01:00
Andrew Cooper 43c513c319 Revert part of "x86/mwait-idle: disable IBRS during long idle"
Most of the patch (handling of CPUIDLE_FLAG_IBRS) is fine, but the
adjustements to mwait_idle() are not; spec_ctrl_enter_idle() does more than
just alter MSR_SPEC_CTRL.IBRS.

The only reason this doesn't need an XSA is because the unconditional
spec_ctrl_{enter,exit}_idle() in mwait_idle_with_hints() were left unaltered,
and thus the MWAIT remained properly protected.

There (would have been) two problems.  In the ibrs_disable (== deep C) case:

 * On entry, VERW and RSB-stuffing are architecturally skipped.
 * On exit, there's a branch crossing the WRMSR which reinstates the
   speculative safety for indirect branches.

All this change did was double up the expensive operations in the deep C case,
and fail to optimise the intended case.

I have an idea of how to plumb this more nicely, but it requires larger
changes to legacy IBRS handling to not make spec_ctrl_enter_idle() vulnerable
in other ways.  In the short term, simply take out the perf hit.

Fixes: 08acdf9a26 ("x86/mwait-idle: disable IBRS during long idle")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 07d7163334)
2025-07-07 22:52:11 +01:00
Andrew Cooper 8fc76b187c x86/idle: Remove MFENCEs for CLFLUSH_MONITOR
Commit 48d32458bc ("x86, idle: add barriers to CLFLUSH workaround") was
inherited from Linux and added MFENCEs around the AAI65 errata fix.

The SDM now states:

  Executions of the CLFLUSH instruction are ordered with respect to each
  other and with respect to writes, locked read-modify-write instructions,
  and fence instructions[1].

with footnote 1 reading:

  Earlier versions of this manual specified that executions of the CLFLUSH
  instruction were ordered only by the MFENCE instruction.  All processors
  implementing the CLFLUSH instruction also order it relative to the other
  operations enumerated above.

I.e. the MFENCEs came about because of an incorrect statement in the SDM.

The Spec Update (no longer available on Intel's website) simply says "issue a
CLFLUSH", with no mention of MFENCEs.

As this erratum is specific to Intel, it's fine to remove the the MFENCEs; AMD
CPUs of a similar vintage do sport otherwise-unordered CLFLUSHs.

Move the feature bit into the BUG range (rather than FEATURE), and move the
workaround into monitor() itself.

The erratum check itself must use setup_force_cpu_cap().  It needs activating
if any CPU needs it, not if all of them need it.

Fixes: 48d32458bc ("x86, idle: add barriers to CLFLUSH workaround")
Fixes: 96d1b237ae ("x86/Intel: work around Xeon 7400 series erratum AAI65")
Link: https://web.archive.org/web/20090219054841/http://download.intel.com/design/xeon/specupdt/32033601.pdf
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit f77ef34435)
2025-07-07 22:52:10 +01:00
Andrew Cooper e9d112ea10 x86/idle: Move monitor()/mwait() wrappers into cpu-idle.c
They're not used by any other translation unit, so shouldn't live in
asm/processor.h, which is included almost everywhere.

Our new toolchain baseline knows the MONITOR/MWAIT instructions, so use them
directly rather than using raw hex.

Change the hint/extention parameters from long to int.  They're specified to
remain 32bit operands even 64-bit mode.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 61e10fc28c)
2025-07-07 22:52:10 +01:00
Andrew Cooper 460ece3bd9 x86/cpufeature: Reposition cpu_has_{lfence_dispatch,nscb}
LFENCE_DISPATCH used to be a synthetic feature, but was given a real CPUID bit
by AMD.  The define wasn't moved when this was changed.

NSCB has always been a real CPUID bit, and was misplaced when introduced in
the synthetic block alongside LFENCE_DISPATCH.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 6a039b0500)
2025-07-07 22:52:10 +01:00
Andrew Cooper 438bb1285f x86/emul: Fix extable registration in invoke_stub()
For exception recovery in the stubs, the registered address for fixup is the
return address of the CALL entering the stub.

In invoke_stub(), the '.Lret%=:' label is the wrong side of the 'post'
parameter.  The 'post' parameter is non-empty in cases where the arithmetic
flags of the operation need recovering.

Split the line to separate 'pre' and 'post', making it more obvious that the
return address label was in the wrong position.

However, in the case that an exception did occur, we want to skip 'post' as
it's logically part of the operation which had already failed.  Therefore, add
a new skip label and use that for the exception recovery path.

This is XSA-470 / CVE-2025-27465

Fixes: 79903e50db ("x86emul: catch exceptions occurring in stubs")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit b9f8311975)
2025-07-01 12:58:12 +01:00
Jan Beulich 0b8f769d00 update Xen version to 4.18.5 RELEASE-4.18.5 2025-05-15 10:07:19 +02:00
Jan Beulich ca12e75354 x86/IRQ: constrain creator-domain-ID assertion
If init_one_irq_desc() fails, ->arch.creator_domid won't be set to the
expected value, and hence the assertion may trigger. Limit it to just
the success case of that function call.

Fixes: 92d9101eab ("x86: allow stubdom access to irq created for msi")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: d7127e06f6
master date: 2025-05-14 11:00:00 +0200
2025-05-15 09:09:10 +02:00
Roger Pau Monné 562af755b1 x86/pv: fix MMUEXT_FLUSH_CACHE to flush all pCPU caches
The implementation of MMUEXT_FLUSH_CACHE is bogus, as it doesn't account to
flush the cache of any previous pCPU where the current vCPU might have run,
and hence is likely to not work as expected.

Fix this by resorting to use the same logic as MMUEXT_FLUSH_CACHE_GLOBAL,
which will be correct in all cases.

Adjust MMUEXT_FLUSH_CACHE_GLOBAL return code in case cache flush is not
permitted for the domain to use -EACCES instead of -EINVAL, as that's more
accurate and also matches the error code used by MMUEXT_FLUSH_CACHE.

Fixes: 8e90e37e6d ("Fix WBINVD by adding a new hypercall.")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
master commit: bbaea9878b
master date: 2025-05-13 19:43:48 +0200
2025-05-15 09:08:54 +02:00
Andrew Cooper dba0556612 x86/spec-ctrl: Support Intel's new PB-OPT
In IPU 2025.2 (May 2025), Intel have released an alternative mitigation for a
prior security issue (SA-00982) on Sapphire and Emerald Rapids CPUs.

Intel suggest that certain workloads will benefit from using the alternative
mode.  This can be selected by booting with `spec-ctrl=ibpb-alt`.

https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/cpuid-enumeration-and-architectural-msrs.html
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 5873740e41
master date: 2025-05-13 14:49:12 +0100
2025-05-15 09:08:30 +02:00
Jan Beulich 0bf71d5c22 cpufreq: don't leave stale statistics pointer
Error paths of cpufreq_statistic_init() correctly free the base
structure pointer, but the per-CPU variable would still hold it, mis-
guiding e.g. cpufreq_statistic_update(). Defer installing of the pointer
there until the structure was fully populated.

Fixes: 755af07edb ("x86/cpufreq: don't use static array for large per-CPU data structures")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: a1ce987411
master date: 2025-04-30 08:47:49 +0200
2025-05-13 14:01:00 +02:00
Jan Beulich a150ecceee x86emul: avoid UB shifts in FLDENV/FRSTOR handling
16-bit quantities, no matter whether expressed as uint16_t or as
bitfield, will be promoted to plain int before doing any arithmetic on
them. Shifting such values by 16 will therefore shift into the sign bit,
which is UB if that bit becomes set. To account for all reads and all
writes accessing opposite members of the same union, introduce yet more
local variables to reduce the shift counts to 12.

Fixes: be55ed744e ("x86emul: support FLDENV and FRSTOR")
Reported-by: Fabian Specht <f.specht@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
master commit: d00fc019b6
master date: 2025-04-30 08:46:21 +0200
2025-05-13 14:00:41 +02:00
Andrew Cooper 545f2d528c xen/link: Include .debug_str_offsets in DWARF2_DEBUG_SECTIONS
Building Xen with Clang-17 yields the following warning:

  ld: warning: orphan section `.debug_str_offsets' from `prelink.o' being placed in section `.debug_str_offsets'
  ld: ./.xen.efi.0xffff82d040000000.0:/4: section below image base
  ld: ./.xen.efi.0xffff82d040000000.1:/4: section below image base
  ld: warning: orphan section `.debug_str_offsets' from `prelink.o' being placed in section `.debug_str_offsets'
  ld: xen.efi:/4: section below image base

Set the alignment to 4 as it holds 4-byte values, despite the fact that Clang
appears to only use 1.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: a88b99300a
master date: 2025-04-28 19:46:46 +0100
2025-05-13 14:00:26 +02:00
Jan Beulich 96b9bb0afd x86: allow non-BIGMEM configs to boot on >= 16Tb systems
While frame table setup, directmap init, and boot allocator population
respect all intended bounds, the logic passing memory to the heap
allocator which wasn't passed to the boot allocator fails to respect
max_{pdx,pfn}. This then typically triggers the BUG() in
free_heap_pages() after checking page state, because of hitting a struct
page_info instance which was set to all ~0.

Of course all the memory above the 16Tb boundary is still going to
remain unused; using it requires BIGMEM=y. And of course this fix
similarly ought to help BIGMEM=y configurations on >= 123Tb systems
(where all the memory beyond that boundary continues to be unused).

Fixes: bac2000063 ("x86-64: reduce range spanned by 1:1 mapping and frame table indexes")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: 64d32b75e6
master date: 2023-12-19 13:47:38 +0100
2025-05-13 14:00:00 +02:00
Andrew Cooper f132c82fa6 x86/spec-ctrl: Synthesise ITS_NO to guests on unaffected hardware
It is easier to express feature word 17 in terms of word 16 + [32, 64) as
that's how the layout is given in documentation.

This is part of XSA-469 / CVE-2024-28956

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit f6042f38e6)
2025-05-12 17:30:56 +01:00
Jan Beulich cf4c900be3 x86/thunk: Build Xen with Return Thunks
The Indirect Target Selection speculative vulnerability means that indirect
branches (including RETs) are unsafe when in the first half of a cacheline.

In order to mitigate this, build with return thunks and arrange for
__x86_return_thunk to be (mis)aligned in the same manner as
__x86_indirect_thunk_* so the RET instruction is placed in a safe location.

place_ret() needs to conditionally emit JMP __x86_return_thunk instead of RET.

This is part of XSA-469 / CVE-2024-28956

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit afcb4a06c7)
2025-05-12 17:30:56 +01:00
Andrew Cooper b51f2c8fcd x86/stubs: Introduce place_ret() to abstract away raw 0xc3's
The Indirect Target Selection speculative vulnerability means that indirect
branches (including RETs) are unsafe when in the first half of a cacheline.
This means it's not safe for logic using the stubs to write raw 0xc3's.

Introduce place_ret() which, for now, writes a raw 0xc3 but will contain
additional logic when return thunks are in use.

stub_selftest() doesn't strictly need to be converted as they only run on
boot, but doing so gets us a partial test of place_ret() too.

No functional change.

This is part of XSA-469 / CVE-2024-28956

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 2eb1132f79)
2025-05-12 17:30:55 +01:00
Andrew Cooper d992f242c1 x86/thunk: (Mis)align the RETs in clear_bhb_loops() to mitigate ITS
The Indirect Target Selection speculative vulnerability means that indirect
branches (including RETs) are unsafe when in the first half of a cacheline.

clear_bhb_loops() has a precise layout of branches.  The alignment for
performance cause the RETs to always be in an unsafe position, and converting
those to return thunks changes the branching pattern.  While such a conversion
is believed to be safe, clear_bhb_loops() is also a performance-relevant
fastpath, so (mis)align the RETs to be in a safe position.

No functional change.

This is part of XSA-469 / CVE-2024-28956

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit c0db07031a)
2025-05-12 17:30:55 +01:00
Jan Beulich 74738ce473 x86/thunk: (Mis)align __x86_indirect_thunk_* to mitigate ITS
The Indirect Target Selection speculative vulnerability means that indirect
branches (including RETs) are unsafe when in the first half of a cacheline.

Arrange for __x86_indirect_thunk_* to always be in the second half.

This is part of XSA-469 / CVE-2024-28956

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit d293cc9da9)
2025-05-12 17:30:55 +01:00
Andrew Cooper 7b506c53ac x86/guest: Remove use of the Xen hypercall_page
In order to protect against ITS, Xen needs to start using return thunks.
Therefore the advice in XSA-466 becomes relevant, and the hypercall_page needs
to be removed.

Implement early_hypercall(), with infrastructure to figure out the correct
instruction on first use.  Use ALTERNATIVE()s to result in inline hypercalls,
including the ALT_NOT() form so we only need a single synthetic feature bit.

No overall change.

This is part of XSA-469 / CVE-2024-28956

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit ef30ffe0a0)
2025-05-12 17:30:55 +01:00
Andrew Cooper 232be2140a x86/alternative: Support replacements when a feature is not present
Use the top bit of a->cpuid to express inverted polarity.  This requires
stripping the top bit back out when performing the sanity checks.

Despite only being used once, create a replace boolean to express the decision
more clearly in _apply_alternatives().

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 328ed39c59)
2025-05-12 17:30:55 +01:00
Roger Pau Monné a9754615cb xen: remove -N from the linker command line
It's unclear why -N is being used in the first place.  It was added by
commit 4676bbf96d back in 2002 without any justification.

When building a PE image it's actually detrimental to forcefully set the
.text section as writable.  The GNU LD man page contains the following
warning regarding the -N option:

> Note: Although a writable text section is allowed for PE-COFF targets, it
> does not conform to the format specification published by Microsoft.

Remove the usage of -N uniformly on all architectures, assuming that the
addition was simply done as a copy and paste of the original x86 linking
rune.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Julien Grall <jgrall@amazon.com>
master commit: d444763f8c
master date: 2025-04-23 16:12:25 +0200
2025-04-29 12:11:06 +02:00
Roger Pau Monné 62513194e1 x86/intel: workaround several MONITOR/MWAIT errata
There are several errata on Intel regarding the usage of the MONITOR/MWAIT
instructions, all having in common that stores to the monitored region
might not wake up the CPU.

Fix them by forcing the sending of an IPI for the affected models.

The Ice Lake issue has been reproduced internally on XenServer hardware,
and the fix does seem to prevent it.  The symptom was APs getting stuck in
the idle loop immediately after bring up, which in turn prevented the BSP
from making progress.  This would happen before the watchdog was
initialized, and hence the whole system would get stuck.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 4aae4452ef
master date: 2025-04-23 16:12:25 +0200
2025-04-29 12:10:46 +02:00
Jan Beulich d980886fd7 compat/memory: avoid UB shifts in XENMEM_exchange handling
Add an early basic check, yielding the same error code as the more
thorough one the main handler would produce.

Fixes: b8a7efe852 ("Enable compatibility mode operation for HYPERVISOR_memory_op")
Reported-by: Manuel Andreas <manuel.andreas@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 560c51be8f
master date: 2025-04-22 11:25:23 +0200
2025-04-29 12:10:30 +02:00
Jan Beulich a677964cbe x86emul: also clip repetition count for STOS
Like MOVS, INS, and OUTS, STOS also has a special purpose hook, where
the hook function may legitimately have the same expectation as to the
request not straddling address space start/end.

Fixes: 5dfe4aa4ee ("x86_emulate: Do not request emulation of REP instructions beyond the")
Reported-by: Fabian Specht <f.specht@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 8c5636b6c8
master date: 2025-04-22 11:24:20 +0200
2025-04-29 12:09:45 +02:00
Jan Beulich 73570ceb9b x86/HVM: update repeat count upon nested lin->phys failure
For the X86EMUL_EXCEPTION case the repeat count must be correctly
propagated back. Since for the recursive invocation we use a local
helper variable, its value needs copying to the caller's one.

While there also correct the off-by-1 range in the comment ahead of the
function (strictly speaking for the "DF set" case we'd need to put
another, different range there as well).

Fixes: 53f87c03b4 ("x86emul: generalize exception handling for rep_* hooks")
Reported-by: Manuel Andreas <manuel.andreas@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: c07b16fd6e
master date: 2025-04-17 10:01:19 +0200
2025-04-29 12:08:55 +02:00
Jan Beulich fe03eba490 include: sort $(wildcard ...) results
The order of items is stored in .*.chk.cmd, and hence variations between
how items are ordered would result in re-invocation of the checking rule
during "make install-xen" despite that already having successfully run
earlier on. The difference can become noticable when building (as non-
root) and installing (as root) use different GNU make versions: In 3.82
the sorting was deliberately undone, just for it to be restored in 4.3.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: ff835bbc80
master date: 2025-04-10 10:56:29 +0200
2025-04-29 12:08:25 +02:00
Volodymyr Babchuk 4e7c949403 xen: x86: irq: initialize irq desc in create_irq()
While building xen with GCC 14.2.1 with "-fcondition-coverage" option
or with "-Og", the compiler produces a false positive warning:

  arch/x86/irq.c: In function ‘create_irq’:
  arch/x86/irq.c:281:11: error: ‘desc’ may be used uninitialized [-Werror=maybe-uninitialized]
    281 |     ret = init_one_irq_desc(desc);
        |           ^~~~~~~~~~~~~~~~~~~~~~~
  arch/x86/irq.c:269:22: note: ‘desc’ was declared here
    269 |     struct irq_desc *desc;
        |                      ^~~~
  cc1: all warnings being treated as errors
  make[2]: *** [Rules.mk:252: arch/x86/irq.o] Error 1

While we have signed/unsigned comparison both in "for" loop and in
"if" statement, this still can't lead to use of uninitialized "desc",
as either loop will be executed at least once, or the function will
return early. So this is a clearly false positive warning due to a
bug [1] in GCC.

Initialize "desc" with NULL to make GCC happy.

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119665

Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 7a4484d90b
master date: 2025-04-08 09:40:39 +0200
2025-04-29 12:07:53 +02:00
Ahmed S. Darwish 5aa2d23844 x86/cpu: Validate CPUID leaf 0x2 EDX output
CPUID leaf 0x2 emits one-byte descriptors in its four output registers
EAX, EBX, ECX, and EDX.  For these descriptors to be valid, the most
significant bit (MSB) of each register must be clear.

Leaf 0x2 parsing at intel.c only validated the MSBs of EAX, EBX, and
ECX, but left EDX unchecked.

Validate EDX's most-significant bit as well.

Fixes: 1aa6feb63b ("Port CPU setup code from Linux 2.6")
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://lore.kernel.org/r/20250304085152.51092-3-darwi@linutronix.de

Use ARRAY_SIZE() though.

Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 1881148215c6
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: a47b44a8f0
master date: 2025-04-08 09:37:38 +0200
2025-04-29 12:07:30 +02:00
Volodymyr Babchuk 398e2a7a15 xen: vm_event: do not do vm_event_op for an invalid domain
A privileged domain can issue XEN_DOMCTL_vm_event_op with
op->domain == DOMID_INVALID. In this case vm_event_domctl()
function will get NULL as the first parameter and this will
cause hypervisor panic, as it tries to derefer this pointer.

Fix the issue by checking if valid domain is passed in.

Fixes: 48b8424945 ("xen/vm-event: Drop unused u_domctl parameter from vm_event_domctl()")
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
master commit: 6a884750f3
master date: 2025-04-08 09:36:38 +0200
2025-04-29 12:06:59 +02:00
Stewart Hildebrand 72a60dfe4e sched/null: avoid another crash after failed domU creation
The following sequence of events may lead a debug build of Xen to crash
when using the null scheduler:

1. domain creation (e.g. d1) failed due to bad configuration
2. complete_domain_destroy() was deferred
3. domain creation (e.g. d2) succeeds

At this point, d2 is running, while the zombie d1 is not fully cleaned
up:

(XEN) Online Cpus: 0-3
(XEN) Cpupool 0:
(XEN) Cpus: 0-3
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Scheduler: null Scheduler (null)
(XEN)   cpus_free = 3
(XEN) Domain info:
(XEN)   Domain: 0
(XEN)     1: [0.0] pcpu=0
(XEN)     2: [0.1] pcpu=1
(XEN)   Domain: 1
(XEN)     3: [1.0] pcpu=2
(XEN)   Domain: 2
(XEN)     4: [2.0] pcpu=2

4. complete_domain_destroy() gets called for d1 and triggers the
following:

(XEN) Xen call trace:
(XEN)    [<00000a0000322ed4>] null.c#unit_deassign+0x2d8/0xb70 (PC)
(XEN)    [<00000a000032457c>] null.c#null_unit_remove+0x670/0xba8 (LR)
(XEN)    [<00000a000032457c>] null.c#null_unit_remove+0x670/0xba8
(XEN)    [<00000a0000336404>] sched_destroy_vcpu+0x354/0x8fc
(XEN)    [<00000a0000227324>] domain.c#complete_domain_destroy+0x11c/0x49c
(XEN)    [<00000a000029fbd0>] rcupdate.c#rcu_do_batch+0x94/0x3d0
(XEN)    [<00000a00002a10c0>] rcupdate.c#__rcu_process_callbacks+0x160/0x5f4
(XEN)    [<00000a00002a1e60>] rcupdate.c#rcu_process_callbacks+0xcc/0x1b0
(XEN)    [<00000a00002a3460>] softirq.c#__do_softirq+0x1f4/0x3d8
(XEN)    [<00000a00002a37c4>] do_softirq+0x14/0x1c
(XEN)    [<00000a0000465260>] traps.c#check_for_pcpu_work+0x30/0xb8
(XEN)    [<00000a000046bb08>] leave_hypervisor_to_guest+0x28/0x198
(XEN)    [<00000a0000409c84>] entry.o#guest_sync_slowpath+0xac/0xd8
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'npc->unit == unit' failed at common/sched/null.c:383
(XEN) ****************************************

Fix by skipping unit_deassign() when the unit to be removed does not
match the pcpu's currently assigned unit.

Fixes: c2eae2614c ("sched/null: avoid crash after failed domU creation")
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
master commit: 54fe207f29
master date: 2025-04-07 12:17:31 +0200
2025-04-29 12:06:31 +02:00
Jan Beulich a2b5eafb62 x86/MTRR: hook mtrr_bp_restore() back up
Unlike stated in the offending commit's description,
load_system_tables() wasn't the only thing left to retain from the
earlier restore_rest_processor_state(). Note that MTRR state was still
reloaded via mtrr_aps_sync_end(), but that happens quite a bit later in
the resume process.

While there also do Misra-related tidying for the function itself: The
function being used from assembly only means it doesn't need to have a
declaration.

Fixes: 4304ff420e ("x86/S3: Drop {save,restore}_rest_processor_state() completely")
Reported-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: 0414dedd6f
master date: 2025-04-03 09:39:13 +0200
2025-04-29 12:05:37 +02:00
Andrew Cooper 0b999cf7e7 x86/ucode: Extend AMD digest checks to cover Zen5 CPUs
AMD have updated the SB-7033 advisory to include Zen5 CPUs.  Extend the digest
check to cover Zen5 too.

In practice, cover everything until further notice.

Observant readers may be wondering where the update to the digest list is.  At
the time of writing, no Zen5 patches are available via a verifiable channel.

Link: https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7033.html
Fixes: 630e8875ab ("x86/ucode: Perform extra SHA2 checks on AMD Fam17h/19h microcode")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit b63951467e)

x86/ucode: Extend warning about disabling digest check too

This was missed by accident.

Fixes: b63951467e ("x86/ucode: Extend AMD digest checks to cover Zen5 CPUs")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 59bb316ea8)
2025-04-11 17:27:59 +01:00
Andrew Cooper b2b0b4bce1 x86/ucode: Perform extra SHA2 checks on AMD Fam17h/19h microcode
Collisions have been found in the microcode signing algorithm used by AMD
Fam17h/19h CPUs, and now anyone can sign their own.

For more details, see:
  https://bughunters.google.com/blog/5424842357473280/zen-and-the-art-of-microcode-hacking
  https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7033.html

As a stopgap mitigation, check the digest of patches against a table of blobs
with known provenance.  These are all Fam17h and Fam19h blobs included in
linux-firwmare at the time of writing, specifically:

  https://git.kernel.org/firmware/linux-firmware/c/48bb90cceb882cab8e9ab692bc5779d3bf3a13b8

This checks can be opted out of by booting with ucode=no-digest-check, but
doing so is not recommended.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 630e8875ab)

Xen: CI fix from XSN-2

 * Add cf_check annotation to cmp_patch_id() used by bsearch().

Fixes: 630e8875ab ("x86/ucode: Perform extra SHA2 checks on AMD Fam17h/19h microcode")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 15fe2eb5f1)
2025-04-11 17:27:59 +01:00