SYMBOL_NAME_SIZE is 124, not 128. Use the symbolic name there as well,
to avoid any disconnect.
While there correct indentation on the following line as well.
Coverity ID: 1659391
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
master commit: dfabb3fee3
master date: 2025-07-31 14:44:02 +0200
ARINC653 specification requires partition scheduling to be deterministic
and periodic over time.
However, the use of current timestamp (now) as the baseline to calculate
next_major_frame and next_switch_time introduces a delay in the start of
major frame at every period, which breaks determinism and periodicity in
partition scheduling.
For example, we observe 3.5 msec of accumulated delay at the 21st major
frame with the following configuration.
Target : qemuarm64
xen version : 4.19 (43aeacff86, x86/IRQ: constrain creator-domain-ID assertion)
dom1 : 10 msec runtime
dom2 : 10 msec runtime
$ a653_sched -p Pool-arinc dom1:10 dom2:10
0.014553536 ---x d?v? runstate_change d1v0 runnable->running //1st major
frame
0.034629712 ---x d?v? runstate_change d1v0 runnable->running //2nd major
frame
<snip>
0.397747008 |||x d?v? runstate_change d1v0 runnable->running //20th
major frame
0.418066096 -||x d?v? runstate_change d1v0 runnable->running //21st
major frame
This is due to an inherent delta between the time value the scheduler timer
is programmed to be fired with and the time value the schedule function
is executed.
Another observation that breaks the deterministic behavior of partition
scheduling is a delayed execution of schedule(); It was called 14 msec
later than programmed.
1.530603952 ---x d?v? runstate_change d1v0 runnable->running
1.564956784 --|x d?v? runstate_change d1v0 runnable->running
Enforce the periodic behavior of partition scheduling by using the value
next_major_frame as the base to calculate the start of major frame and
the next domain switch time.
Per discussion with Nathan Studer, there are odd cases the first minor
frame can be also missed. In that scenario, iterate through the schedule
after resyncing the expected next major frame.
Per suggestion from Stewart Hildebrand, use a modulo operation to
calculate the start of next major frame.
Fixes: 22787f2e10 ("ARINC 653 scheduler")
Suggested-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Suggested-by: Nathan Studer <nathan.studer@dornerworks.com>
Signed-off-by: Anderson Choi <anderson.choi@boeing.com>
Reviewed-by: Stewart Hildebrand <stewart@stew.dk>
master commit: f2927d8343
master date: 2025-07-28 13:05:56 +0200
When a running unit is about to be scheduled out due to a competing unit
with the highest remaining credit, the residual credit of the previous
unit is currently ignored in csched2_runtime() because it hasn't yet
been reinserted into the runqueue.
As a result, two equally weighted, busy units can often each be granted
almost the maximum possible runtime (i.e. consuming CSCHED2_CREDIT_INIT
in one shot) when only those two are active. In broad strokes two units
switch back and forth every 10ms (CSCHED2_MAX_TIMER). In contrast, when
more than two busy units are competing, such coarse runtime allocations
are rarely seen, since at least one active unit remains in the runqueue.
To ensure consistent behavior, have csched2_runtime() take into account
the previous unit's latest credit when it still can/wants to run.
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Reviewed-by: Juergen Gross <jgross@suse.com>
master commit: ae648e9f80
master date: 2025-07-28 13:04:40 +0200
Xen currently uses an ASID scheme where:
- ASIDs are cycled where a "TLB flush" is performed
- When ASIDs wrap around, perform a full TLB flush
- In exceptional cases, stop using ASIDs
However, the TLB control mode used only flushes the current active ASID of
the logical processor. Which mean that will supply "stale" ASIDs (not flushed),
because it hasn't been actually flushed (TLB_CTRL_FLUSH_ASID only performs a
TLB flush of the ASID set in VMCB).
This affects CPUs where flush-by-asid is available (Fam15h (2011) and later).
To fix this, always flush all ASIDs even when flush-by-asid is available.
Fixes: 64b1da5a2f ("x86/svm: Use flush-by-asid when available")
Signed-off-by: Teddy Astie <teddy.astie@vates.tech>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 68871b1f9b
master date: 2025-07-23 15:23:44 +0200
While supposedly safe via enforcing a control flow change when modifying
already prefetched code, it may not really be. Afaik a request is
pending to drop the first of the two options in the SDM's "Handling
Self- and Cross-Modifying Code" section (still present there as of
version 087). Insert a serializing instruction there, and remove the
"noinline" in exchange.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
master commit: c5b06d0b1b
master date: 2025-07-23 15:17:28 +0200
smp_store_cpu_info() doesn't call identify_cpu() again during S3 resume,
hence it is wrong to zap APs' data while bringing them down (and we're
not just parking them there) during suspend.
Fixes: bb502a8ca5 ("x86: check feature flags after resume")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
master commit: 55719030b0
master date: 2025-07-17 17:55:54 +0200
NEHALEM_EX is affected by the erratum too.
Change the comment to be the full erratum text, not an interpretation of it.
Fixes: 95807bcae4 ("C6 state with EOI issue fix for some Intel processors")
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: d225205a0b
master date: 2025-07-18 13:40:12 +0100
The current p2m type-change loop in ept_vcpu_flush_pml_buffer() relies on
each call to p2m_change_type_one() taking the p2m lock, doing the change
and then dropping the lock and flushing the p2m. Instead take the p2m lock
outside of the loop, so that calls to gfn_{,un}lock() inside
p2m_change_type_one() just take the p2m lock recursively, and more
importantly, the flush is deferred until the p2m is unlocked in
ept_vcpu_flush_pml_buffer().
No functional change intended in the end result of
ept_vcpu_flush_pml_buffer(), however a possibly noticeable performance
improvement is expected.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 060e9ab6e4
master date: 2025-07-18 09:17:25 +0200
smp_store_cpu_info() doesn't call identify_cpu() again during S3 resume,
hence it is wrong to overwrite APs' data using boot_cpu_data. With
7126b7f806 ("x86/CPU: re-work populating of cpu_data[]") this now
results in a crash, as the cpu_to_socket() use in smp_store_cpu_info()
then obtains a bad socket number, much like was already observed in [1].
[1] https://lists.xen.org/archives/html/xen-devel/2025-06/msg01879.html
Fixes: bb502a8ca5 ("x86: check feature flags after resume")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: acaa69d985
master date: 2025-07-17 12:49:18 +0200
Before CPU identification has run (and it may not have run at all e.g.
when AP bringup failed altogether), cpu_data[].phys_proc_id (which is
what cpu_to_socket() resolves to) can't really be used. The use of
cpu_to_socket()'s result as an array index in cpu_smpboot_free()
therefore needs guarding, as the function will also be invoked upon AP
bringup failure, in which case CPU identification may not have run.
Without "x86/CPU: re-work populating of cpu_data[]" [1] the issue is
less pronounced: The field starts out as zero, then has the BSP value
(likely again zero) copied into it, and it is properly invalidated only
in cpu_smpboot_free(). Still it is clearly wrong to use the BSP's socket
number here.
Making the guard work with and without the above patch applied turns out
interesting: Prior to that patch, the sole invalidation done is that in
cpu_smpboot_free(). Upon a later bringup attempt, the fields invalidated
are overwritten by the BSP values again, though. Hence compare APIC IDs,
as they cannot validly be the same once CPU identification has run.
[1] https://lists.xen.org/archives/html/xen-devel/2024-02/msg00727.html
Fixes: 4824bdfdab ("x86: add socket_cpumask")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: 8c15d3d187
master date: 2025-07-17 12:47:04 +0200
SDM revision 088 points out that apparently as of quite some time ago on
Intel hardware BSF and BSR may alter all arithmetic flags, not just ZF.
Because of the inconsistency (sometimes even errata, and because
documentation doesn't look to be quite right about PF), best we can do
is simply take the flag values from what the processor produces, just
like we do for various other arithmetic insns. (Note also that AMD and
Intel have always been disagreeing on arithmetic flags other than ZF.)
To be both safe (against further anomalies) and consistent, extend this
to {L,T}ZCNT as well. (Emulating the two insns correctly even when
underlying hardware doesn't support it was perhaps nice, but yielded
guest-observable inconsistencies.)
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 37a55e69c7
master date: 2025-07-16 12:51:26 +0200
Clearwater Forest (CWF) SoC has the same C-states as Sierra Forest (SRF)
SoC. Add CWF support by re-using the SRF C-states table.
Note: it is expected that CWF C-states will have same or very similar
characteristics as SRF C-states (latency and target residency).
However, there is a possibility that the characteristics will end up
being different enough when the CWF platform development is finished.
In that case, a separate CWF C-states table will be created and populated
with the CWF-specific characteristics (latency and target residency).
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Link: https://patch.msgid.link/20241203130306.1559024-1-artem.bityutskiy@linux.intel.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git eeed4bfbe9b9
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 5c5bb8dab7
master date: 2025-07-09 11:02:35 +0200
Add Granite Rapids Xeon C-states, which are C1, C1E, C6, and C6P.
Comparing to previous Xeon Generations (e.g., Emerald Rapids), C6
requests end up only in core C6 state, and no package C-state promotion
takes place even if all cores in the package are in core C6.
C6P requests also end up in core C6, but if all cores have requested
C6P, the SoC will enter the package C6 state.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Link: https://patch.msgid.link/20240806160310.3719205-1-artem.bityutskiy@linux.intel.com
[ rjw: Changelog edits ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 370406bf5738
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 3c7186b2a8
master date: 2025-07-09 11:02:08 +0200
Add Sierra Forest SoC C-states, which are C1, C1E, C6S, and C6SP.
Sierra Forest SoC is built with modules, each module includes 4 cores
(Crestmont microarchitecture). There is one L2 cache per module, shared
between the 4 cores.
There is no core C6 state, but there is C6S state, which has module scope:
when all 4 cores request C6S, the entire module (4 cores + L2 cache)
enters the low power state.
C6SP state has package scope - when all modules in the package enter C6S,
the package enters the power state mode.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 92813fd5b156
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: edd31f0fe9
master date: 2025-07-09 11:01:50 +0200
Add Intel Grand Ridge SoC C-states, which are C1, C1E, and C6S.
The Grand Ridge SoC is built with modules, each module includes 4 cores
(Crestmont microarchitecture). There is one L2 cache per module, shared
between the 4 cores.
There is no core C6 state, but there is C6S state, which has module
scope: when all 4 cores request C6S, the entire module (4 cores + L2
cache) enters the low power state.
Package C6 is not supported by Grand Ridge SoC.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ac89d11b93cc
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: a514b54992
master date: 2025-07-09 11:01:34 +0200
Add intel_idle support for MeteorLake.
C1 and C1E states on Meteorlake are mutually exclusive, like Alderlake
and Raptorlake, but they have little latency difference with measureable
power difference, so always enable "C1E promotion" bit and expose C1E
only.
Expose C6 because it has less power compared with C1E, and smaller
latency compared with C8/C10.
Ignore C8 and expose C10, because C8 does not show latency advantage
compared with C10.
Signed-off-by: Zhang Rui <rui.zhang@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git eeae55ed9c0a
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 433ed7d4f8
master date: 2025-07-09 11:01:03 +0200
Emerald Rapids (EMR) is the next Intel Xeon processor after Sapphire
Rapids (SPR).
EMR C-states are the same as SPR C-states, and we expect that EMR
C-state characteristics (latency and target residency) will be the
same as in SPR. Therefore, add EMR support by using SPR C-states table.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 74528edfbc66
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: a689b12199
master date: 2025-07-09 11:00:44 +0200
Similar to the other other AlderLake platforms, the C1 and C1E states on
ADL-N are mutually exclusive. Only one of them can be enabled at a time.
C1E is preferred on ADL-N for better energy efficiency.
C6S is also supported on this platform. Its latency is far bigger than
C6, but really close to C8 (PC8), thus it is not exposed as a separate
state.
Suggested-by: Baieswara Reddy Sagili <baieswara.reddy.sagili@intel.com>
Suggested-by: Vinay Kumar <vinay.kumar@intel.com>
Signed-off-by: Zhang Rui <rui.zhang@intel.com>
[ rjw: Changelog edits ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 65c0c2367e9e
x86/cpu: Fix Gracemont uarch
Alderlake N is an E-core only product using Gracemont
micro-architecture. It fits the pre-existing naming scheme perfectly
fine, adhere to it.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Link: https://lore.kernel.org/r/20230807150405.686834933@infradead.org
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 882cdb06b668
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 51b956f336
master date: 2025-07-09 11:00:16 +0200
While the change is fine on staging, where the toolchain baseline was
moved, we need to remain compatible with older gas on stable branches
for now.
Fixes: fa254938f0 ("x86/idle: Move monitor()/mwait() wrappers into cpu-idle.c")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
TSA affects AMD Fam19h CPUs (Zen3 and 4 microarchitectures).
Three new CPUID bits have been defined. Two (TSA_SQ_NO and TSA_L1_NO)
indicate that the system is unaffected, and must be synthesised by Xen on
unaffected parts to date.
A third new bit indicates that VERW now has a flushing side effect. Xen
must synthesise this bit on affected systems based on microcode version.
As with other VERW-based flushing features, VERW_CLEAR needs OR-ing across
a resource pool, and guests which have seen it can safely migrate in.
This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 578b34adb1)
In order to mitigate TSA, Xen will need to issue VERW before going idle.
On AMD CPUs, the VERW scrubbing side effects cancel an active MONITOR, causing
the MWAIT to exit without entering an idle state. Therefore the VERW must be
ahead of MONITOR.
Split spec_ctrl_enter_idle() in two and allow the VERW aspect to be handled
separately. While adjusting, update a stale comment concerning MSBDS; more
issues have been mitigated using VERW since it was written.
By moving VERW earlier, it is ahead of the determination of whether to go
idle. We can't move the check on softirq_pending (for correctness reasons),
but we can duplicate it earlier as a best effort attempt to skip the
speculative overhead.
This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 01429bbb0a)
AMD are releasing microcode for TSA, so extend the known-provenance list with
their hashes. These were produced before the remediation of the microcode
signature issues (the entrysign vulnerability), so can be OS-loaded on
out-of-date firmware.
Include an off-by-default check for the sorted-ness of patch_digests[]. It's
not worth running generally under SELF_TESTS, but is useful when editing the
digest list.
This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit a8708ea8bd)
Turn the if()s into switch()es, as we're going to need AMD sections.
Move the RTM adjustments into the Intel section, where they ought to live.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 7c5025394d)
The check of this_softirq_pending must be performed with irqs disabled, but
this property was broken by an attempt to optimise entry/exit latency.
Commit c227233ad64c in Linux (which we copied into Xen) was fixed up by
edc8fc01f608 in Linux, which we have so far missed.
Going to sleep without waking on interrupts is nonsensical outside of
play_dead(), so overload this to select between two possible MWAITs, the
second using the STI shadow to cover MWAIT for exactly the same reason as we
do in safe_halt().
Fixes: b17e0ec72e ("x86/mwait-idle: enable interrupts before C1 on Xeons")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 9b0f0f6e23)
In order elide IPIs, we must be able to identify whether a target CPU is in
MWAIT at the point it is woken up. i.e. the store to wake it up must also
identify the state.
Create a new in_mwait variable beside __softirq_pending, so we can use a
CMPXCHG to set the softirq while also observing the status safely. Implement
an x86 version of arch_pend_softirq() which does this.
In mwait_idle_with_hints(), advertise in_mwait, with an explanation of
precisely what it means. X86_BUG_MONITOR can be accounted for simply by not
advertising in_mwait.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 3e0bc4b503)
x86 is the only architecture wanting an optimisation here, but the
test_and_set_bit() is a store into the monitored line (i.e. will wake up the
target) and, prior to the removal of the broken IPI-elision algorithm, was
racy, causing unnecessary IPIs to be sent.
To do this in a race-free way, the store to the monited line needs to also
sample the status of the target in one atomic action. Implement a new arch
helper with different semantics; to make the softirq pending and decide about
IPIs together. For now, implement the default helper. It will be overridden
by x86 in a subsequent change.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit b473e5e212)
With the recent simplifications, it becomes obvious that smp_mb() isn't the
right barrier. Strictly speaking, MONITOR is ordered as a load, but smp_rmb()
isn't correct either, as this only pertains to local ordering. All we need is
a compiler barrier().
Merge the barier() into the monitor() itself, along with an explantion.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit e7710dd843)
cpuidle_wakeup_mwait() is a TOCTOU race. The cpumask_and() sampling
cpuidle_mwait_flags can take a arbitrary period of time, and there's no
guarantee that the target CPUs are still in MWAIT when writing into
mwait_wakeup(cpu).
The consequence of the race is that we'll fail to IPI certain targets. Also,
there's no guarantee that mwait_idle_with_hints() will raise a TIMER_SOFTIRQ
on it's way out.
The fundamental bug is that the "in_mwait" variable needs to be in the
monitored line, and not in a separate cpuidle_mwait_flags variable, in order
to do this in a race-free way.
Arranging to fix this while keeping the old implementation is prohibitive, so
strip the current one out in order to implement the new one cleanly. In the
interim, this causes IPIs to be used unconditionally which is safe albeit
suboptimal.
Fixes: 3d521e933e ("cpuidle: mwait on softirq_pending & remove wakeup ipis")
Fixes: 1adb34ea84 ("CPUIDLE: re-implement mwait wakeup process")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 3faf0866a3)
For features which are unconditionally set in the max policies, making the
default policy to match the host can be done with a conditional clear.
This is simpler than the unconditional clear, conditional set currently
performed.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 30f8fed68f)
Most of the patch (handling of CPUIDLE_FLAG_IBRS) is fine, but the
adjustements to mwait_idle() are not; spec_ctrl_enter_idle() does more than
just alter MSR_SPEC_CTRL.IBRS.
The only reason this doesn't need an XSA is because the unconditional
spec_ctrl_{enter,exit}_idle() in mwait_idle_with_hints() were left unaltered,
and thus the MWAIT remained properly protected.
There (would have been) two problems. In the ibrs_disable (== deep C) case:
* On entry, VERW and RSB-stuffing are architecturally skipped.
* On exit, there's a branch crossing the WRMSR which reinstates the
speculative safety for indirect branches.
All this change did was double up the expensive operations in the deep C case,
and fail to optimise the intended case.
I have an idea of how to plumb this more nicely, but it requires larger
changes to legacy IBRS handling to not make spec_ctrl_enter_idle() vulnerable
in other ways. In the short term, simply take out the perf hit.
Fixes: 08acdf9a26 ("x86/mwait-idle: disable IBRS during long idle")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 07d7163334)
Commit 48d32458bc ("x86, idle: add barriers to CLFLUSH workaround") was
inherited from Linux and added MFENCEs around the AAI65 errata fix.
The SDM now states:
Executions of the CLFLUSH instruction are ordered with respect to each
other and with respect to writes, locked read-modify-write instructions,
and fence instructions[1].
with footnote 1 reading:
Earlier versions of this manual specified that executions of the CLFLUSH
instruction were ordered only by the MFENCE instruction. All processors
implementing the CLFLUSH instruction also order it relative to the other
operations enumerated above.
I.e. the MFENCEs came about because of an incorrect statement in the SDM.
The Spec Update (no longer available on Intel's website) simply says "issue a
CLFLUSH", with no mention of MFENCEs.
As this erratum is specific to Intel, it's fine to remove the the MFENCEs; AMD
CPUs of a similar vintage do sport otherwise-unordered CLFLUSHs.
Move the feature bit into the BUG range (rather than FEATURE), and move the
workaround into monitor() itself.
The erratum check itself must use setup_force_cpu_cap(). It needs activating
if any CPU needs it, not if all of them need it.
Fixes: 48d32458bc ("x86, idle: add barriers to CLFLUSH workaround")
Fixes: 96d1b237ae ("x86/Intel: work around Xeon 7400 series erratum AAI65")
Link: https://web.archive.org/web/20090219054841/http://download.intel.com/design/xeon/specupdt/32033601.pdf
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit f77ef34435)
They're not used by any other translation unit, so shouldn't live in
asm/processor.h, which is included almost everywhere.
Our new toolchain baseline knows the MONITOR/MWAIT instructions, so use them
directly rather than using raw hex.
Change the hint/extention parameters from long to int. They're specified to
remain 32bit operands even 64-bit mode.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 61e10fc28c)
LFENCE_DISPATCH used to be a synthetic feature, but was given a real CPUID bit
by AMD. The define wasn't moved when this was changed.
NSCB has always been a real CPUID bit, and was misplaced when introduced in
the synthetic block alongside LFENCE_DISPATCH.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 6a039b0500)
When support was originally introduced, ebx, ecx and edx were reserved and
should have been zeroed in recalculate_misc() to avoid leaking into guests.
Since then, fields have been added into ebx. Guests can't load microcode, so
shouldn't see ucode_size, and while in principle we do want to support larger
RAP sizes in guests, virtualising this for guests depends on AMD procuding any
official documentation for ERAPS, which is long overdue and with no ETA.
This patch will cause a difference in guests on Zen5 CPUs, but as the main
ERAPS feature is hidden, guests should be ignoring the rap_size field too.
Fixes: e9b4fe2636 ("x86/cpuid: support LFENCE always serialising CPUID bit")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 10dc35c516
master date: 2025-07-02 18:25:03 +0100
See the respective bullet point in the Caveats section of
https://gcc.gnu.org/gcc-15/changes.html.
While I'm unaware of us currently relying on the pre-gcc15 behavior,
let's still play safe and retain what unknowingly we may have been
relying upon.
According to my observations, on x86 generated code changes
- somewhere deep in modify_bars(), presumably from the struct map_data
initializer in apply_map() (a new MOVQ),
- in vpci_process_pending(), apparently again from the struct map_data
initializer (and again a new MOVQ),
- near the top of find_cpio_data(), presumably from the struct cpio_data
initializer (a MOVW changing to a MOVQ).
Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 86da14178d
master date: 2025-07-01 11:23:59 +0200
If the total returned by Xen is more than the number of elements
allocated, it means that the buffer was too small and so the data is
incomplete. Retry to get all the data.
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
master commit: 1ff7f87e8f
master date: 2025-06-11 11:07:00 +0200
xc_pm_get_pxstat() requires the caller to allocate the pt and trans_pt
buffers but then calls xc_pm_get_max_px() to determine how big they are
(and hence how much Xen will copy into them). This is susceptible to
races if xc_pm_get_max_px() changes so avoid the problem by requiring
the caller to also pass in the size of the buffers.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
master commit: de6a05a8a0
master date: 2025-06-11 11:06:45 +0200
For exception recovery in the stubs, the registered address for fixup is the
return address of the CALL entering the stub.
In invoke_stub(), the '.Lret%=:' label is the wrong side of the 'post'
parameter. The 'post' parameter is non-empty in cases where the arithmetic
flags of the operation need recovering.
Split the line to separate 'pre' and 'post', making it more obvious that the
return address label was in the wrong position.
However, in the case that an exception did occur, we want to skip 'post' as
it's logically part of the operation which had already failed. Therefore, add
a new skip label and use that for the exception recovery path.
This is XSA-470 / CVE-2025-27465
Fixes: 79903e50db ("x86emul: catch exceptions occurring in stubs")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit b9f8311975)
The 2 missed ones are: register_runstate_phys_area and
register_vcpu_time_phys_area.
Fixes: d5df44275e ("domain: introduce GADDR based runstate area registration alternative")
Fixes: 60e544a8c5 ("x86: introduce GADDR based secondary time area registration alternative")
Signed-off-by: Gang Ji <gang.ji@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: ed939591e4
master date: 2025-06-11 11:08:11 +0200
If set_px_pminfo is called a second time with a larger state_count than
the first call, calls to PMSTAT_get_pxstat will read beyond the end of
the pt and trans_pt buffers allocated in cpufreq_statistic_init() since
they would have been allocated with the original state_count.
Secondly, the states array leaks on each subsequent call of
set_px_pminfo.
Fix both these issues by ignoring subsequent calls to set_px_pminfo if
it completed successfully previously. Return success rather than an
error to avoid errors in the dom0 kernel log when reloading the
xen_acpi_processor module.
At the same time, fix a leak of the states array on error.
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 2f36bef3d8
master date: 2025-06-11 11:06:24 +0200
Check that the total number of states passed in and hence the size of
buffers is sufficient to avoid writing more than the caller has
allocated.
The interface is not explicit about whether getpx.total is expected to
be set by the caller in this case but since it is always set in
libxenctrl it seems reasonable to check it and make it explicit.
Fixes: c06a7db0c5 ("X86 and IA64: Update cpufreq statistic logic for supporting both x86 and ia64")
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
# Commit fb16c7411d
# Date 2025-06-18 09:25:09 +0200
# Author Jan Beulich <jbeulich@suse.com>
# Committer Jan Beulich <jbeulich@suse.com>
x86/pmstat: correct PMSTAT_get_pxstat buffer size checking
min(pmpt->perf.state_count, op->u.getpx.total) == op->u.getpx.total can
be expressed differently as pmpt->perf.state_count >= op->u.getpx.total.
Copying when the two are equal is fine; (partial) copying when the state
count is larger than the number of array elements that a buffer was
allocated to hold is what - as per the comment - we mean to avoid. Drop
the use of min() again, but retain its effect for the subsequent copying
from pxpt->u.pt.
Fixes: aa70996a68 ("x86/pmstat: Check size of PMSTAT_get_pxstat buffers")
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
master commit: aa70996a68
master date: 2025-06-11 11:05:42 +0200
master commit: fb16c7411d
master date: 2025-06-18 09:25:09 +0200
Only access the HVM union b_info->u.hvm on HVM guests. The union
access is not guarded, so this reads and sets the default even on
non-HVM guests. Usually this doesn't matter as PV and PVH unions are
smaller and zero-initialized, but the zero default will be re-written as
a -1 boolean. Generally, it could incorrectly set b_info->altp2m
through aliased data.
Fixes: 0291089f6e ("xen: enable altp2m at create domain domctl")
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
master commit: 01dfd25509
master date: 2025-06-02 10:31:21 +0200
x86_merge_dr6() is not a no-op when 0 is passed in; it will discard the
previously latched breakpoint bits.
The combination of do_debug()'s manual call to x86_merge_dr6() for external
debuggers, and pv_inject_DB() calling pv_inject_event(), results in two
x86_merge_dr6() calls.
Feed the same pending_dbg in the second time. This makes pv_inject_event()'s
update of dr6 effectively a no-op, retaining the correct breakpoint bits.
Fixes: db39fa4b27 ("x86/pv: Fix merging of new status bits into %dr6")
Reported-by: Manuel Bouyer <bouyer@antioche.eu.org>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Manuel Bouyer <bouyer@antioche.eu.org>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: d965e2ee07
master date: 2025-05-27 12:21:38 +0100
Paging Writeable depends on EPT so must be disabled in non-EPT guests like the
other EPT dependent features. Otherwise, VMEntry fails with bad control
state.
Drop a piece of trailing whitepsace in context.
Fixes: ff10aa9d8f ("x86: Add Support for Paging-Write Feature")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: d779e3f98c
master date: 2025-05-27 12:21:38 +0100
For once the message printed when a BAR overlaps with a non-hole regions is
not accurate on x86. While the BAR won't be mapped by the vPCI logic, it
is quite likely overlapping with a reserved region in the memory map, and
already mapped as by default all reserved regions are identity mapped in
the p2m. Avoid printing the warning message in modify_bars(), and instead
print a more lax message in the x86 implementation of pci_check_bar() to
note the current BAR position overlaps with non-hole region(s).
Secondly, when an overlap is detected the BAR 'enabled' field is not set,
hence other vPCI code that depends on it like vPCI MSI-X handling won't
function properly, as it sees the BAR as disabled, even when memory
decoding is enabled for the device and the BAR is likely mapped in the
p2m. Change the handling of BARs that overlap non-hole regions to instead
remove any overlapped regions from the rangeset, so the resulting ranges to
map just contain the hole regions. This requires introducing a new
pci_sanitize_bar_memory() that's implemented per-arch and sanitizes the
address range to add to the p2m.
For x86 pci_sanitize_bar_memory() removes any regions present in the host
memory map, for ARM this is currently left as a dummy handler to not change
existing behavior.
Ultimately the above changes should fix the vPCI MSI-X handlers not working
correctly when the BAR that contains the MSI-X table overlaps with a
non-hole region, as then the 'enabled' BAR bit won't be set and the MSI-X
traps won't handle accesses as expected.
Reported-by: Stefano Stabellini <stefano.stabellini@amd.com>
Fixes: 53d9133638 ('pci: do not disable memory decoding for devices')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Tested-by: Victor M Lira <victorm.lira@amd.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
x86/vpci: fix off-by-one in pci_sanitize_bar_memory()
rangeset_remove_range() uses inclusive ranges, and hence the end of the
range should be calculated using PFN_DOWN(), not PFN_UP().
Fixes: 4acab25a93 ('x86/vpci: fix handling of BAR overlaps with non-hole regions')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 4acab25a93
master date: 2025-05-21 18:29:55 +0200
master commit: 7ab4b392b7
master date: 2025-05-22 16:17:12 +0200
The current emulation of wb{,no}invd is bogus for PV guests: it will only
flush the current pCPU cache, without taking into account pCPUs where the
vCPU had run previously. Resort to flushing the cache on all host pCPUs to
make it correct.
Fixes: 799fed0a7c ("Priv-op emulation in Xen, for RDMSR/WRMSR/WBINVD")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
[backport: no FLUSH_CACHE_WRITEBACK]
master commit: b0f8bf8605
master date: 2025-05-20 16:35:52 +0200
Jan Beulich