With the processing done linearly (rather than recursively), checking
whether any of the features was previously seen is wrong: That would
e.g. trigger for this simple set of dependencies
X: [A, B]
A: [C]
B: [C]
(observed in reality when making AMX-AVX512 dependent upon both
AMX-TILE and AVX512F, causing XSAVE to see AMX-AVX512 twice in its list
of dependents). But checking the whole accumulated set also isn't
necessary - just checking the feature we're processing dependents of is
sufficient. We may detect a cycle later that way, but we still will
detect it. What we need to avoid is adding a feature again when we've
already seen it.
As a result, seeding "seen[]" with "feat" isn't necessary anymore.
Fixes: fe4408d180 ("xen/x86: Generate deep dependencies of features")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 3999ff0d30
master date: 2025-09-01 13:07:53 +0200
EFI code path split options from EFI LoadOptions fields in 2
pieces, first EFI options, second Xen options.
"get_argv" function is called first to get the number of arguments
in the LoadOptions, second, after allocating enough space, to
fill some "argc"/"argv" variable. However the first parsing could
be different from second as second is able to detect "--" argument
separator. So it was possible that "argc" was bigger than the
initialized portion of "argv" array, leading to potential
uninitialized pointer dereference, in particular a string like
"-- a b c" would lead to crashes.
Using EFI shell is possible to pass any kind of string in
LoadOptions.
Fixes: bf6501a62e ("x86-64: EFI boot code")
Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
master commit: 375f0dd538
master date: 2025-09-01 13:07:01 +0200
SMBIOS specification dictates that tables should have a minimal length.
This commit introduces further validation for user-input SMBIOS tables.
As per SMBIOS Reference Specification:
* Type 0: For version 2.3 and later implementations, the length is at least 14h
* Type 1: 1Bh for 2.4 and later
* Type 2: at least 08h
* Type 3: 0Dh for version 2.1 and later
* Type 11: 5h (+ strings)
* Type 22: 1Ah (+ strings)
* Type 39: a minimum of 10h
Notably, this also fixes previously incorrect check for chassis handle in
smbios_type_2_init. Chassis handle is a WORD, therefore, the condition now
correctly checks for >= 13 instead of > 13.
hvmloader currently implements version 2.4
Furthermore, this commit introduces smbios_pt_copy helper function to substitute
previously repeating pattern of locating the struct in physical memory (via
get_smbios_pt_struct), checking the length and copying it into SMBIOS region.
Fixes: 4d23036e70 ("HVM firmware passthrough SMBIOS processing")
Signed-off-by: Petr Beneš <w1benny@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
master commit: 2f7d52f016
master date: 2025-08-27 11:26:33 +0200
* `sizeof_field` returns the size of a specific struct member
* `endof_field` returns the offset to the end of the member within the struct
It will be useful in upcoming layout checks of SMBIOS structs.
Signed-off-by: Petr Beneš <w1benny@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
master commit: 17e66ca918
master date: 2025-08-27 11:26:33 +0200
The current code to restore the timer state on resume is incomplete. While
the local APIC Initial Count Register is saved and restored across
suspension (even if possibly no longer accurate since it's not adjusted to
account for the time spent in suspension), the TSC deadline MSR is not
saved and restored, hence hosts using the TSC deadline timer will likely
get stuck when resuming from suspension.
The lack of restoring of the TSC deadline MSR was mitigated by the raising
of a timer softirq in mwait_idle_with_hints() if the timer had expired,
previous to commit 3faf0866a3, which removed
that logic.
This patch fixes the usage of the TSC deadline timer with suspension, by
unconditionally raising a timer softirq on resume, that will take care of
rearming the hardware timer. Given that a timer softirq will be
unconditionally risen, there's no need to save and restore the APIC Initial
Count Register anymore either.
Note that secondary processors don't need this special treatment when
resuming, since they are offlined before suspension and brought back up
during resume, the first timer that gets setup will trigger a timer softirq
unconditionally, for example from sched_migrate_timers() that gets called
for each secondary processor.
Link: https://github.com/QubesOS/qubes-issues/issues/10110
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Fixes: fd1291a826 ('X86: Prefer TSC-deadline timer in Xen')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
master commit: 757abedabb
master date: 2025-08-27 09:33:12 +0200
There's no family check gating intel_init_ppin(), making it incorrect to
use on non Fam6 CPUs.
Emerald Rapids is the final CPU to have PPIN but lack the architectural
enumeration, so include it too.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 49e6eb744e
master date: 2025-08-25 14:04:41 +0100
Otherwise the PCI accesses to segments different than the first one done by
the IOMMU initialization code would silently fail by returning all ones.
Introduce a new helper, called pci_setup(), and move both the creation of
PCI segment 0 internal data structures, plus the parsing of ACPI MMCFG
table to it. This moves acpi_mmcfg_init() slightly earlier from
acpi_boot_init() into pci_setup().
Note that further work will be needed to support systems where access
methods to segments different than 0 is not discoverable by Xen.
Fixes: 3950f2485b ('x86/x2APIC: defer probe until after IOMMU ACPI table parsing')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: c292772b49
master date: 2025-08-21 11:57:25 +0200
Fix the incorrect condition that causes hvm_alloc_legacy_ioreq_gfn()
to return INVALID_GFN even if the HVM param was installed properly by
the toolstack.
Fixes: 3486f398a3 (' x86/hvm/ioreq: allow ioreq servers to use HVM_PARAM_[BUF]IOREQ_PFN')
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 282ed258a5
master date: 2025-08-18 10:43:52 +0200
In order to legitimately set up initial mappings past _end[], we need
to make sure that the entire mapped range is inside a RAM region.
Therefore we need to inform the bootloader (or alike) that our allocated
size is larger than just the next SECTION_ALIGN-ed boundary past _end[].
This allows dropping a command line option from the tool, which was
introduced to work around a supposed linker bug, when the problem was
really Xen's.
While adjusting adjacent code, correct the argc check to also cover the
case correctly when --notes was passed.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: 4fb075201f
master date: 2025-08-18 10:42:24 +0200
If the config file is builtin, cfg.addr will be zero but Xen
unconditionally calls FreePages() on the address.
Xen may also call FreePages() with a zero address if blexit() is called
after this point since cfg.need_to_free is not set to false.
The UEFI specification does not say whether calling FreePages() with a
zero address is allowed so let's be cautious and use cfg.need_to_free
properly.
Fixes: 8a71d50ed4 ("efi: Enable booting unified hypervisor/kernel/initrd images")
Fixes: 04be2c3a06 ("efi/boot.c: add file.need_to_free")
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by Daniel P. Smith <dpsmith@apertussolutions.com>
master commit: 7fddedd530
master date: 2025-08-12 11:40:28 +0200
The current logic in the HPET interrupt ->ack() hook will perform a local
APIC EOI ahead of enabling interrupts, possibly leading to recursion in the
interrupt handler.
Fix this by doing the local APIC EOI strictly after the window with
interrupt enabled, as that prevents the recursion, and would only allow for
interrupts with higher priority to be serviced.
Use the generic ack_nonmaskable_msi_irq() and end_nonmaskable_irq()
functions, removing the need for hpet_msi_ack().
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: 3ba523ff95 ('CPUIDLE: enable MSI capable HPET for timer broadcast')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 1db7829e56
master date: 2025-08-06 10:47:16 +0200
SYMBOL_NAME_SIZE is 124, not 128. Use the symbolic name there as well,
to avoid any disconnect.
While there correct indentation on the following line as well.
Coverity ID: 1659391
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
master commit: dfabb3fee3
master date: 2025-07-31 14:44:02 +0200
ARINC653 specification requires partition scheduling to be deterministic
and periodic over time.
However, the use of current timestamp (now) as the baseline to calculate
next_major_frame and next_switch_time introduces a delay in the start of
major frame at every period, which breaks determinism and periodicity in
partition scheduling.
For example, we observe 3.5 msec of accumulated delay at the 21st major
frame with the following configuration.
Target : qemuarm64
xen version : 4.19 (43aeacff86, x86/IRQ: constrain creator-domain-ID assertion)
dom1 : 10 msec runtime
dom2 : 10 msec runtime
$ a653_sched -p Pool-arinc dom1:10 dom2:10
0.014553536 ---x d?v? runstate_change d1v0 runnable->running //1st major
frame
0.034629712 ---x d?v? runstate_change d1v0 runnable->running //2nd major
frame
<snip>
0.397747008 |||x d?v? runstate_change d1v0 runnable->running //20th
major frame
0.418066096 -||x d?v? runstate_change d1v0 runnable->running //21st
major frame
This is due to an inherent delta between the time value the scheduler timer
is programmed to be fired with and the time value the schedule function
is executed.
Another observation that breaks the deterministic behavior of partition
scheduling is a delayed execution of schedule(); It was called 14 msec
later than programmed.
1.530603952 ---x d?v? runstate_change d1v0 runnable->running
1.564956784 --|x d?v? runstate_change d1v0 runnable->running
Enforce the periodic behavior of partition scheduling by using the value
next_major_frame as the base to calculate the start of major frame and
the next domain switch time.
Per discussion with Nathan Studer, there are odd cases the first minor
frame can be also missed. In that scenario, iterate through the schedule
after resyncing the expected next major frame.
Per suggestion from Stewart Hildebrand, use a modulo operation to
calculate the start of next major frame.
Fixes: 22787f2e10 ("ARINC 653 scheduler")
Suggested-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Suggested-by: Nathan Studer <nathan.studer@dornerworks.com>
Signed-off-by: Anderson Choi <anderson.choi@boeing.com>
Reviewed-by: Stewart Hildebrand <stewart@stew.dk>
master commit: f2927d8343
master date: 2025-07-28 13:05:56 +0200
When a running unit is about to be scheduled out due to a competing unit
with the highest remaining credit, the residual credit of the previous
unit is currently ignored in csched2_runtime() because it hasn't yet
been reinserted into the runqueue.
As a result, two equally weighted, busy units can often each be granted
almost the maximum possible runtime (i.e. consuming CSCHED2_CREDIT_INIT
in one shot) when only those two are active. In broad strokes two units
switch back and forth every 10ms (CSCHED2_MAX_TIMER). In contrast, when
more than two busy units are competing, such coarse runtime allocations
are rarely seen, since at least one active unit remains in the runqueue.
To ensure consistent behavior, have csched2_runtime() take into account
the previous unit's latest credit when it still can/wants to run.
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Reviewed-by: Juergen Gross <jgross@suse.com>
master commit: ae648e9f80
master date: 2025-07-28 13:04:40 +0200
Xen currently uses an ASID scheme where:
- ASIDs are cycled where a "TLB flush" is performed
- When ASIDs wrap around, perform a full TLB flush
- In exceptional cases, stop using ASIDs
However, the TLB control mode used only flushes the current active ASID of
the logical processor. Which mean that will supply "stale" ASIDs (not flushed),
because it hasn't been actually flushed (TLB_CTRL_FLUSH_ASID only performs a
TLB flush of the ASID set in VMCB).
This affects CPUs where flush-by-asid is available (Fam15h (2011) and later).
To fix this, always flush all ASIDs even when flush-by-asid is available.
Fixes: 64b1da5a2f ("x86/svm: Use flush-by-asid when available")
Signed-off-by: Teddy Astie <teddy.astie@vates.tech>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 68871b1f9b
master date: 2025-07-23 15:23:44 +0200
While supposedly safe via enforcing a control flow change when modifying
already prefetched code, it may not really be. Afaik a request is
pending to drop the first of the two options in the SDM's "Handling
Self- and Cross-Modifying Code" section (still present there as of
version 087). Insert a serializing instruction there, and remove the
"noinline" in exchange.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
master commit: c5b06d0b1b
master date: 2025-07-23 15:17:28 +0200
smp_store_cpu_info() doesn't call identify_cpu() again during S3 resume,
hence it is wrong to zap APs' data while bringing them down (and we're
not just parking them there) during suspend.
Fixes: bb502a8ca5 ("x86: check feature flags after resume")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
master commit: 55719030b0
master date: 2025-07-17 17:55:54 +0200
NEHALEM_EX is affected by the erratum too.
Change the comment to be the full erratum text, not an interpretation of it.
Fixes: 95807bcae4 ("C6 state with EOI issue fix for some Intel processors")
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: d225205a0b
master date: 2025-07-18 13:40:12 +0100
The current p2m type-change loop in ept_vcpu_flush_pml_buffer() relies on
each call to p2m_change_type_one() taking the p2m lock, doing the change
and then dropping the lock and flushing the p2m. Instead take the p2m lock
outside of the loop, so that calls to gfn_{,un}lock() inside
p2m_change_type_one() just take the p2m lock recursively, and more
importantly, the flush is deferred until the p2m is unlocked in
ept_vcpu_flush_pml_buffer().
No functional change intended in the end result of
ept_vcpu_flush_pml_buffer(), however a possibly noticeable performance
improvement is expected.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 060e9ab6e4
master date: 2025-07-18 09:17:25 +0200
smp_store_cpu_info() doesn't call identify_cpu() again during S3 resume,
hence it is wrong to overwrite APs' data using boot_cpu_data. With
7126b7f806 ("x86/CPU: re-work populating of cpu_data[]") this now
results in a crash, as the cpu_to_socket() use in smp_store_cpu_info()
then obtains a bad socket number, much like was already observed in [1].
[1] https://lists.xen.org/archives/html/xen-devel/2025-06/msg01879.html
Fixes: bb502a8ca5 ("x86: check feature flags after resume")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: acaa69d985
master date: 2025-07-17 12:49:18 +0200
Before CPU identification has run (and it may not have run at all e.g.
when AP bringup failed altogether), cpu_data[].phys_proc_id (which is
what cpu_to_socket() resolves to) can't really be used. The use of
cpu_to_socket()'s result as an array index in cpu_smpboot_free()
therefore needs guarding, as the function will also be invoked upon AP
bringup failure, in which case CPU identification may not have run.
Without "x86/CPU: re-work populating of cpu_data[]" [1] the issue is
less pronounced: The field starts out as zero, then has the BSP value
(likely again zero) copied into it, and it is properly invalidated only
in cpu_smpboot_free(). Still it is clearly wrong to use the BSP's socket
number here.
Making the guard work with and without the above patch applied turns out
interesting: Prior to that patch, the sole invalidation done is that in
cpu_smpboot_free(). Upon a later bringup attempt, the fields invalidated
are overwritten by the BSP values again, though. Hence compare APIC IDs,
as they cannot validly be the same once CPU identification has run.
[1] https://lists.xen.org/archives/html/xen-devel/2024-02/msg00727.html
Fixes: 4824bdfdab ("x86: add socket_cpumask")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: 8c15d3d187
master date: 2025-07-17 12:47:04 +0200
SDM revision 088 points out that apparently as of quite some time ago on
Intel hardware BSF and BSR may alter all arithmetic flags, not just ZF.
Because of the inconsistency (sometimes even errata, and because
documentation doesn't look to be quite right about PF), best we can do
is simply take the flag values from what the processor produces, just
like we do for various other arithmetic insns. (Note also that AMD and
Intel have always been disagreeing on arithmetic flags other than ZF.)
To be both safe (against further anomalies) and consistent, extend this
to {L,T}ZCNT as well. (Emulating the two insns correctly even when
underlying hardware doesn't support it was perhaps nice, but yielded
guest-observable inconsistencies.)
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 37a55e69c7
master date: 2025-07-16 12:51:26 +0200
Clearwater Forest (CWF) SoC has the same C-states as Sierra Forest (SRF)
SoC. Add CWF support by re-using the SRF C-states table.
Note: it is expected that CWF C-states will have same or very similar
characteristics as SRF C-states (latency and target residency).
However, there is a possibility that the characteristics will end up
being different enough when the CWF platform development is finished.
In that case, a separate CWF C-states table will be created and populated
with the CWF-specific characteristics (latency and target residency).
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Link: https://patch.msgid.link/20241203130306.1559024-1-artem.bityutskiy@linux.intel.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git eeed4bfbe9b9
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 5c5bb8dab7
master date: 2025-07-09 11:02:35 +0200
Add Granite Rapids Xeon C-states, which are C1, C1E, C6, and C6P.
Comparing to previous Xeon Generations (e.g., Emerald Rapids), C6
requests end up only in core C6 state, and no package C-state promotion
takes place even if all cores in the package are in core C6.
C6P requests also end up in core C6, but if all cores have requested
C6P, the SoC will enter the package C6 state.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Link: https://patch.msgid.link/20240806160310.3719205-1-artem.bityutskiy@linux.intel.com
[ rjw: Changelog edits ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 370406bf5738
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 3c7186b2a8
master date: 2025-07-09 11:02:08 +0200
Add Sierra Forest SoC C-states, which are C1, C1E, C6S, and C6SP.
Sierra Forest SoC is built with modules, each module includes 4 cores
(Crestmont microarchitecture). There is one L2 cache per module, shared
between the 4 cores.
There is no core C6 state, but there is C6S state, which has module scope:
when all 4 cores request C6S, the entire module (4 cores + L2 cache)
enters the low power state.
C6SP state has package scope - when all modules in the package enter C6S,
the package enters the power state mode.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 92813fd5b156
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: edd31f0fe9
master date: 2025-07-09 11:01:50 +0200
Add Intel Grand Ridge SoC C-states, which are C1, C1E, and C6S.
The Grand Ridge SoC is built with modules, each module includes 4 cores
(Crestmont microarchitecture). There is one L2 cache per module, shared
between the 4 cores.
There is no core C6 state, but there is C6S state, which has module
scope: when all 4 cores request C6S, the entire module (4 cores + L2
cache) enters the low power state.
Package C6 is not supported by Grand Ridge SoC.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ac89d11b93cc
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: a514b54992
master date: 2025-07-09 11:01:34 +0200
Add intel_idle support for MeteorLake.
C1 and C1E states on Meteorlake are mutually exclusive, like Alderlake
and Raptorlake, but they have little latency difference with measureable
power difference, so always enable "C1E promotion" bit and expose C1E
only.
Expose C6 because it has less power compared with C1E, and smaller
latency compared with C8/C10.
Ignore C8 and expose C10, because C8 does not show latency advantage
compared with C10.
Signed-off-by: Zhang Rui <rui.zhang@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git eeae55ed9c0a
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 433ed7d4f8
master date: 2025-07-09 11:01:03 +0200
Emerald Rapids (EMR) is the next Intel Xeon processor after Sapphire
Rapids (SPR).
EMR C-states are the same as SPR C-states, and we expect that EMR
C-state characteristics (latency and target residency) will be the
same as in SPR. Therefore, add EMR support by using SPR C-states table.
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 74528edfbc66
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: a689b12199
master date: 2025-07-09 11:00:44 +0200
Similar to the other other AlderLake platforms, the C1 and C1E states on
ADL-N are mutually exclusive. Only one of them can be enabled at a time.
C1E is preferred on ADL-N for better energy efficiency.
C6S is also supported on this platform. Its latency is far bigger than
C6, but really close to C8 (PC8), thus it is not exposed as a separate
state.
Suggested-by: Baieswara Reddy Sagili <baieswara.reddy.sagili@intel.com>
Suggested-by: Vinay Kumar <vinay.kumar@intel.com>
Signed-off-by: Zhang Rui <rui.zhang@intel.com>
[ rjw: Changelog edits ]
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 65c0c2367e9e
x86/cpu: Fix Gracemont uarch
Alderlake N is an E-core only product using Gracemont
micro-architecture. It fits the pre-existing naming scheme perfectly
fine, adhere to it.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Link: https://lore.kernel.org/r/20230807150405.686834933@infradead.org
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 882cdb06b668
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 51b956f336
master date: 2025-07-09 11:00:16 +0200
While the change is fine on staging, where the toolchain baseline was
moved, we need to remain compatible with older gas on stable branches
for now.
Fixes: fa254938f0 ("x86/idle: Move monitor()/mwait() wrappers into cpu-idle.c")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
TSA affects AMD Fam19h CPUs (Zen3 and 4 microarchitectures).
Three new CPUID bits have been defined. Two (TSA_SQ_NO and TSA_L1_NO)
indicate that the system is unaffected, and must be synthesised by Xen on
unaffected parts to date.
A third new bit indicates that VERW now has a flushing side effect. Xen
must synthesise this bit on affected systems based on microcode version.
As with other VERW-based flushing features, VERW_CLEAR needs OR-ing across
a resource pool, and guests which have seen it can safely migrate in.
This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 578b34adb1)
In order to mitigate TSA, Xen will need to issue VERW before going idle.
On AMD CPUs, the VERW scrubbing side effects cancel an active MONITOR, causing
the MWAIT to exit without entering an idle state. Therefore the VERW must be
ahead of MONITOR.
Split spec_ctrl_enter_idle() in two and allow the VERW aspect to be handled
separately. While adjusting, update a stale comment concerning MSBDS; more
issues have been mitigated using VERW since it was written.
By moving VERW earlier, it is ahead of the determination of whether to go
idle. We can't move the check on softirq_pending (for correctness reasons),
but we can duplicate it earlier as a best effort attempt to skip the
speculative overhead.
This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 01429bbb0a)
AMD are releasing microcode for TSA, so extend the known-provenance list with
their hashes. These were produced before the remediation of the microcode
signature issues (the entrysign vulnerability), so can be OS-loaded on
out-of-date firmware.
Include an off-by-default check for the sorted-ness of patch_digests[]. It's
not worth running generally under SELF_TESTS, but is useful when editing the
digest list.
This is part of XSA-471 / CVE-2024-36350 / CVE-2024-36357.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit a8708ea8bd)
Turn the if()s into switch()es, as we're going to need AMD sections.
Move the RTM adjustments into the Intel section, where they ought to live.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 7c5025394d)
The check of this_softirq_pending must be performed with irqs disabled, but
this property was broken by an attempt to optimise entry/exit latency.
Commit c227233ad64c in Linux (which we copied into Xen) was fixed up by
edc8fc01f608 in Linux, which we have so far missed.
Going to sleep without waking on interrupts is nonsensical outside of
play_dead(), so overload this to select between two possible MWAITs, the
second using the STI shadow to cover MWAIT for exactly the same reason as we
do in safe_halt().
Fixes: b17e0ec72e ("x86/mwait-idle: enable interrupts before C1 on Xeons")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 9b0f0f6e23)
In order elide IPIs, we must be able to identify whether a target CPU is in
MWAIT at the point it is woken up. i.e. the store to wake it up must also
identify the state.
Create a new in_mwait variable beside __softirq_pending, so we can use a
CMPXCHG to set the softirq while also observing the status safely. Implement
an x86 version of arch_pend_softirq() which does this.
In mwait_idle_with_hints(), advertise in_mwait, with an explanation of
precisely what it means. X86_BUG_MONITOR can be accounted for simply by not
advertising in_mwait.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 3e0bc4b503)
x86 is the only architecture wanting an optimisation here, but the
test_and_set_bit() is a store into the monitored line (i.e. will wake up the
target) and, prior to the removal of the broken IPI-elision algorithm, was
racy, causing unnecessary IPIs to be sent.
To do this in a race-free way, the store to the monited line needs to also
sample the status of the target in one atomic action. Implement a new arch
helper with different semantics; to make the softirq pending and decide about
IPIs together. For now, implement the default helper. It will be overridden
by x86 in a subsequent change.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit b473e5e212)
With the recent simplifications, it becomes obvious that smp_mb() isn't the
right barrier. Strictly speaking, MONITOR is ordered as a load, but smp_rmb()
isn't correct either, as this only pertains to local ordering. All we need is
a compiler barrier().
Merge the barier() into the monitor() itself, along with an explantion.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit e7710dd843)
cpuidle_wakeup_mwait() is a TOCTOU race. The cpumask_and() sampling
cpuidle_mwait_flags can take a arbitrary period of time, and there's no
guarantee that the target CPUs are still in MWAIT when writing into
mwait_wakeup(cpu).
The consequence of the race is that we'll fail to IPI certain targets. Also,
there's no guarantee that mwait_idle_with_hints() will raise a TIMER_SOFTIRQ
on it's way out.
The fundamental bug is that the "in_mwait" variable needs to be in the
monitored line, and not in a separate cpuidle_mwait_flags variable, in order
to do this in a race-free way.
Arranging to fix this while keeping the old implementation is prohibitive, so
strip the current one out in order to implement the new one cleanly. In the
interim, this causes IPIs to be used unconditionally which is safe albeit
suboptimal.
Fixes: 3d521e933e ("cpuidle: mwait on softirq_pending & remove wakeup ipis")
Fixes: 1adb34ea84 ("CPUIDLE: re-implement mwait wakeup process")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 3faf0866a3)
For features which are unconditionally set in the max policies, making the
default policy to match the host can be done with a conditional clear.
This is simpler than the unconditional clear, conditional set currently
performed.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 30f8fed68f)
Most of the patch (handling of CPUIDLE_FLAG_IBRS) is fine, but the
adjustements to mwait_idle() are not; spec_ctrl_enter_idle() does more than
just alter MSR_SPEC_CTRL.IBRS.
The only reason this doesn't need an XSA is because the unconditional
spec_ctrl_{enter,exit}_idle() in mwait_idle_with_hints() were left unaltered,
and thus the MWAIT remained properly protected.
There (would have been) two problems. In the ibrs_disable (== deep C) case:
* On entry, VERW and RSB-stuffing are architecturally skipped.
* On exit, there's a branch crossing the WRMSR which reinstates the
speculative safety for indirect branches.
All this change did was double up the expensive operations in the deep C case,
and fail to optimise the intended case.
I have an idea of how to plumb this more nicely, but it requires larger
changes to legacy IBRS handling to not make spec_ctrl_enter_idle() vulnerable
in other ways. In the short term, simply take out the perf hit.
Fixes: 08acdf9a26 ("x86/mwait-idle: disable IBRS during long idle")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 07d7163334)
Commit 48d32458bc ("x86, idle: add barriers to CLFLUSH workaround") was
inherited from Linux and added MFENCEs around the AAI65 errata fix.
The SDM now states:
Executions of the CLFLUSH instruction are ordered with respect to each
other and with respect to writes, locked read-modify-write instructions,
and fence instructions[1].
with footnote 1 reading:
Earlier versions of this manual specified that executions of the CLFLUSH
instruction were ordered only by the MFENCE instruction. All processors
implementing the CLFLUSH instruction also order it relative to the other
operations enumerated above.
I.e. the MFENCEs came about because of an incorrect statement in the SDM.
The Spec Update (no longer available on Intel's website) simply says "issue a
CLFLUSH", with no mention of MFENCEs.
As this erratum is specific to Intel, it's fine to remove the the MFENCEs; AMD
CPUs of a similar vintage do sport otherwise-unordered CLFLUSHs.
Move the feature bit into the BUG range (rather than FEATURE), and move the
workaround into monitor() itself.
The erratum check itself must use setup_force_cpu_cap(). It needs activating
if any CPU needs it, not if all of them need it.
Fixes: 48d32458bc ("x86, idle: add barriers to CLFLUSH workaround")
Fixes: 96d1b237ae ("x86/Intel: work around Xeon 7400 series erratum AAI65")
Link: https://web.archive.org/web/20090219054841/http://download.intel.com/design/xeon/specupdt/32033601.pdf
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit f77ef34435)
They're not used by any other translation unit, so shouldn't live in
asm/processor.h, which is included almost everywhere.
Our new toolchain baseline knows the MONITOR/MWAIT instructions, so use them
directly rather than using raw hex.
Change the hint/extention parameters from long to int. They're specified to
remain 32bit operands even 64-bit mode.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 61e10fc28c)
LFENCE_DISPATCH used to be a synthetic feature, but was given a real CPUID bit
by AMD. The define wasn't moved when this was changed.
NSCB has always been a real CPUID bit, and was misplaced when introduced in
the synthetic block alongside LFENCE_DISPATCH.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
(cherry picked from commit 6a039b0500)
Jan Beulich