Neither current nor curr_vcpu per-CPU fields accurately track which
page-tables are loaded. There are corner cases when dealing with shadow
paging failures that switch to the idle vCPU page-tables without changing
current or curr_vcpu per-CPU fields.
Introduce a new per-CPU field that attempts to track which vCPU page-tables
are loaded. Update such tracking when cr3 is changed, and do so in a
region with interrupts disabled, as to avoid handling interrupts with a
mismatch between the vCPU tracking field and the loaded page-tables.
As a result of this newly more accurate tracking the mapcache override
functionality can be removed: the dom0 PV builder was the only user of it,
and it's updated here to properly signal which vCPU page-tables are loaded
in the calls to switch_cr3_cr4().
Note the EFI page-tables have the Xen owned L4 slots copied from the idle
page-tables, so for the effects of the mapcache the EFI page-tables could
use the idle mapcache if it had one. Pass the idle vCPU in the
switch_cr3_cr4() call that switches to the runtime EFI page-tables.
There are known issues with the use of mapcache in NMI context. This patch
does not alter the behaviour.
This is CVE-2026-42488 / XSA-494.
Fixes: fb0ff49fe9 ("x86/shadow: defer releasing of PV's top-level shadow reference")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Adjust the mask calculation in case the last range is merged with the
previous one, as then the mask must be calculated from the previous range,
which the current one has been merged into.
Instead of fixing the off-by-one in place, move the calculation of the bit
change mask to the next loop, after the ranges have been merged. This
simplifies the logic by consolidating mask calculation in a single place,
possibly making it less error prone in the future.
Also add a test case that triggers the bug being fixed by this commit.
Fixes: c5c45bcbd6 ("pdx: introduce a new compression algorithm based on region offsets")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The logic to populate memnodemap in populate_memnodemap() assumes that all
ranges are aligned to the hash shift, this however is only true for the
first address in a memory affinity node. Any subsequent ranges belonging
to the same node might not be aligned to the hash shift value.
Such lack of alignment causes issues to the logic in populate_memnodemap(),
as then the tail of the range might not be properly accounted for and setup
in memnodemap. Fix this by forcing the start address of all regions to
be aligned to the hash shift; if such alignment causes a region overlap it
would always be between regions on the same node, and hence will never
cause setup issues of the memnodemap array.
Introduce two additional test cases to the user-space NUMA setup unit
testing, first test case is the native memory affinity and memory map of
the system where this issue was found, second test case is a simplification
to demonstrate the original problem more clearly.
Fixes: 1666086b00 ("x86/NUMA: improve memnode_shift calculation for multi node system")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
NUMA setup, like PDX, requires certain amount of logic to configure the
internal structures and parameters for NUMA operation. Introduce some very
basic testing that allows building and testing NUMA setup logic in as a
user-space unit test. This allows feeding synthetic memory affinity and
map to the logic, allowing to reproduce bugs that would otherwise need
access to real systems with such a configuration.
For the time being introduce a single test case, based on a known working
NUMA setup for an AMD Turin system. Also the testing after setup is
currently limited to ensuring the start and end RAM region addresses fall
into a correctly setup memory block.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Introduce __XEN__ guards to differentiate between hypervisor vs unit test
builds. Also move numa_set_node() so it's outside the __XEN__ guards. No
functional change intended.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Per PCI Firmware 3.3 specification, section 4.1.2, ECAM space must be
reserved by declaring a motherboard resource, but there's no requirement to
mention it in E820. The specification additionally states that: the
resources can optionally be returned in Int15 E820h or EFIGetMemoryMap as
reserved memory.
Recent Lenovo systems have been found to have the MMCFG region in an
E820 hole, which looks to be technically spec compliant, but is rejected
by Xen.
The logic was introduced in Linux in 2006 as 946f2ee5c731 ("[PATCH]
i386/x86-64: Check that MCFG points to an e820 reserved area"). This was
picked up by Xen when MCFG support was added in 3b35911d70 ("Enable pci
mmcfg and ATS for x86_64"). Apply an approach similar to what Linux has
done in 199f968f1484 ("x86/pci: Skip early E820 check for ECAM region") and
relax the strict reserved region checking so it's only done for firmware
manufactured prior to 2016. For firmware from 2016 and newer allow MCFG
region to reside in holes on the memory map. Note Xen is still more strict
than Linux however, as it will refuse to use MCFG regions that overlap with
memory map regions different than reserved.
When dom0 boots it can prevent access to misconfigured MCFG regions by
using the PHYSDEVOP_pci_mmcfg_reserved hypercall. This brings Xen's early
usage of MCFG (prior to ACPI AML parsing) more in line with the
implementation in Linux.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
A non-trivial amount of issues related to PVH dom0 are fixed by enabling
the `pf-fixup` command line option, and no issues have been reported as a
result of its usage. Enable the option by default to make PVH dom0 more
robust in its default configuration.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Attempting to memset the whole IOMMU MMIO region to zero is dangerous to
say the least. We don't know which registers might be there, nor which
values might be safe for those registers. On a forthcoming platform doing
the zeroing of the MMIO region does put the IOMMU in a broken state, which
is not recoverable by the IOMMU initialization procedure in Xen.
Instead just zero the control register, which mimics the current behavior
with regards to how the control register is handled, and ensures the IOMU
setup is done with the unit disabled. This approach will need revisiting
in order to support Preboot DMA Protection.
Fold map_iommu_mmio_region() into its only caller, as the function body is
just an ioremap() call after the removal of the memset().
Fixes: 0700c962ac ("Add AMD IOMMU support into hypervisor")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper@citrix.com>
With the imminent decommission of CirrusCI we will lose the limited macOS
build testing done there. Introduce two Gitlab macOS build jobs that
reproduce the ones present on CirrusCI.
According to the "Community programs" Gitlab documentation [0] we get
50,000 compute minutes per month. The arm64 job takes ~1m 15s, and the
x86_64 one ~2m. I think it's likely enough given the amount of minutes we
have available. We might need to reconsider if we add more tasks that
consume compute minutes.
[0] https://docs.gitlab.com/subscriptions/community_programs/#gitlab-for-open-source
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Uninitialized cpu_data[] entries have the apicid field set to BAD_APICID,
not boot_cpu_data.apicid. Fix the check in cpu_smpboot_free() to use the
correct condition.
Fixes: 7126b7f806 ("x86/CPU: re-work populating of cpu_data[]")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Teddy Astie <teddy.astie@vates.tech>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
ENODATA is not defined on FreeBSD:
xenpm.c:1380:17: error: use of undeclared identifier 'ENODATA'
1380 | errno = ENODATA;
| ^
1 error generated.
Fixes: edb657887a ("xenpm: Add get-core-temp subcommand")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Teddy Astie <teddy.astie@vates.tech>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
L1 present entries that require the underlying page to be freed have the
_PAGE_AVAIL0 bit set, introduce a helper to unify the checking logic into a
single place.
No functional change intended.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
All arches truncate the cpu_khz without taking into account the less
significant digits. Instead use DIV_ROUND() when scaling from Hz to kHz
to get as more accurate kHz value.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When running HVM guest in native TSC mode avoid using the recalculated vTSC
scaling factors based on the cpu_khz value. Using the kHz based frequency
leads to the TSC scaling values possibly not being the same as the ones
used by the per CPU cpu_time->tsc_scale field, which introduces skew
between the guest and Xen's calculations of the system time.
On a 2gHz system, where the frequency is possibly detected as 1999999999Hz
(note this is a worse-case scenario), the cpu_khz variable will be set to
1999999kHz, and hence 999Hz cycles will be not accounted for per second.
Over a second (the time synchronization period), this leads to a skew of:
cycles * 1 / (Hz freq) = 999 / 1999999999 = 499,5ns
So far this has gone unnoticed because the time synchronization rendezvous
forces the update of the tsc_timestamp and system_time fields in the vCPU
time info area, and hence the skew only accumulates up to the rendezvous
period. Attempting to remove the rendezvous causes the skew to grow
unbounded.
Fix by using the native TSC scaling values (as used by Xen) when the guest
TSC is not scaled.
Fixes: eab8a90be7 ("x86/time: scale host TSC in pvclock properly")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
A killed timer will ignore further set_timer() calls, and hence won't be
re-armed unless it's initialized again. Use stop_timer() instead of
kill_timer() in time_suspend(), so that the set_timer() call in
time_resume() successfully re-arms the timer. Otherwise time calibration
is no longer scheduled (and executed) after resuming from S3 suspend.
Fixes: 6d90db1a2c ("x86: rendezvous-based local time calibration")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
If the user has set baudrate as "auto" (use whatever was configured) but
Xen cannot find any set baudrate use a default fallback baudrate of
115200.
Otherwise the console will very likely end up being non-functional, as
further values are derived from the baudrate (like the transmit timeout),
and using -1 will result in screwed parameters being derived from the
baudrate.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Assigning pages to a domain make them the possible target of hypercalls
like XENMEM_decrease_reservation ahead of such pages being scrubbed in
populate_physmap() when the guest is running in PV mode. This might allow
pages to be freed ahead of being scrubbed for example, as a stubdomain
already running could target them by guessing their MFNs. It's also
possible other action could set the page type ahead of scrubbing, which
would be problematic.
Prevent the pages pending scrub from being assigned to the domain, and only
do the assign once the scrubbing has finished. This has the disadvantage
that the allocated pages will be removed from the free pool, but not yet
accounted towards the domain consumed page quota. However there can only
be one stashed page in that state, and it's maximum size is bounded by the
memop-max-order option. This is not too different from the current logic,
where assigning pages to a domain (and thus checking whether such domain
doesn't overflow it's quota) is also done after the memory has been
allocated and removed from the pool of free pages.
Fixes: 83a784a15b ("xen/mm: allow deferred scrub of physmap populate allocated pages")
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
alloc_heap_pages() will unconditionally clear PGC_need_scrub, even when
MEMF_no_scrub is requested. This is kind of expected as otherwise some
callers will assert on seeing non-expected flags set on the count_info
field.
Introduce a new MEMF bit to signal to alloc_heap_pages() that non-scrubbed
pages should keep the PGC_need_scrub bit set. This fixes returning dirty
pages from alloc_domheap_pages() without the PGC_need_scrub bit set for
populate_physmap() to consume.
With the above change alloc_domheap_pages() needs an adjustment to cope
with allocated pages possibly having the PGC_need_scrub set.
Fixes: 83a784a15b ("xen/mm: allow deferred scrub of physmap populate allocated pages")
Reported-by: Ayden Bottos <aydenbottos12@gmail.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Read the existing PCI command register and only add the required bits to
it, as to avoid clearing bits that might be possibly set by the firmware
already, which might put the device into a non-working state.
Xen being the owner of the PCI device it's legitimately allowed to change
command registers bits. Be careful however in case buggy devices require
certain command register bits to be enabled, while not strictly required
from Xen's usage.
Fixes: f2ff5d6628 ("ns16550: enable PCI serial card usage")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Auto-discovered serial PCI devices when using dev=amt|pci won't get
->ps_bdf_enable, and as such some of the logic (like making sure the
respective BARs are enabled) won't be applied to them.
Fix by unconditionally setting ->ps_bdf_enable for all PCI serial devices,
and removing the special case that was done in some places by checking
whether the ->bar was set. This also allows simplifying the logic in
pci_serial_early_init().
Fixes: 9738db88f6 ("xen: Automatically find serial port on PCI/PCIe and AMT devices.")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When using key pairs for the uart configuration (com1 and com2 command line
options), report an error if the passed device=<type> is not recognized
instead of silently ignoring it.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
In the EPT implementation, the defer flushing logic is used
unconditionally, and that would lead to paging memory being returned to the
paging pool before its references had been flushed.
Issue any pending flushes before freeing the paging memory back to the
pool.
Note AMD (NPT) and Shadow paging are not affected, as they don't implement
the deferred flushing logic.
This is XSA-480 / CVE-2026-23554
Fixes: 4a59e6bb3a ("x86/EPT: squash meaningless TLB flush")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The limitation of which tests can be build on 32bit is too strict. Move
some tests out of the 64bit explicit region, as they can also be built on
32bits.
Fixes: cb50e40337 ("test/pdx: add PDX compression unit tests")
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When building the PDX test harness as a 32bit executable the page shifts
done on unsigned long types can overflow. Instead use pfn_to_paddr(),
which casts the values to paddr_t previous to doing the shift.
Fixes: cb50e40337 ("test/pdx: add PDX compression unit tests")
Reported-by: Edwin Török <edwin.torok@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The first parameter passed to align_timer() is the timer expiration, not
the current time. Adjust the calls to align_timer() in the on-demand
governor to pass the expected timer expiration as the first parameter.
Fixes: af74e3a15a ("cpufreq: align dbs timer for better package C state residency")
Fixes: 382b95f627 ("Fix cpufreq HW-ALL coordination handle")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The logic in dbs_check_cpu() resets the sampling window even when the
sampling period is considered too small. This leads to further calls
finding an imbalance between the total window time and the idle time, as
the total window time is possibly shorter than the idle time.
Fix by resetting the sampling window start time in the same block where the
current idle time is stored. While there also prevent a duplicated call to
NOW() and instead re-use the previously fetched value.
Fixes: d6f001cb91 ("x86: Implement cpufreq ondemand policy")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
After e4c3755d4d the function efi_halt_system() is unreachable, remove it
from the file.
No functional change expected, as the function is not called.
Fixes: e4c3755d4d ("x86-64/EFI: don't call EfiResetSystem() from machine_halt()")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Introduce an extra check and comment to ensure the outer caller has
possibly taken an extra reference on the foreign page that's about to be
removed from the p2m. Otherwise the put_page() in p2m_entry_modify() won't
be safe to do ahead of the entry being removed form the p2m and any cached
states purged.
While there also replace the error codes for unreachable paths to use
EILSEQ.
No functional change intended.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Currently the -Wa,--noexecstack parameter is only passed to the compiler
when building assembly files, as the assembler doesn't add such note for
stack attributes.
However, the default addition of the .note.GNU-stack section is dependent
on the ELF target used. Adjust the passing of -Wa,--noexecstack so it's
added to CFLAGS instead of AFLAGS, and hence such section is also added to
object files generated from .c sources. This fixes the complains from the
linker about missing .note.GNU-stack section when building x86-64 Xen on
Darwin platforms.
Note we could instead pass `-z noexecstack` to the linker, but that however
would mask any possible errors or unintended mismatches from previous
steps.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
BSD grep only recognizes [[:blank:]] as a valid expression when using
extended (modern) regular expressions. Pass -E to the grep call used in
efi-nr-fixups.
Additionally, the return from `wc -l` is space padded on BSD, and hence
the content of efi-nr-fixups is " 2", not plain "2". Strip the
spaces in the Makefile using $(strip ...).
Fixes: 1be65ec4c8 ('x86/EFI: avoid use of GNU ld's --disable-reloc-section when possible')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Like we do for the FreeBSD builds, introduce some basic smoke testing of
the built binary using the XTF selftest image.
Note this is only done for the x86 build, there's no ARM support in XTF
yet.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Introduce two new build that run on macOS Cirrus-CI instances. Those build
the hypervisor using the elf cross-toolchain from Homebrew for both x86 and
arm64.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
On macOS the default encoding is UTF-8. The binary encoding used by the
grep matching pattern is not valid when processed as UTF-8. Switch the
local to C just for the grep invocation to use the binary match pattern
correctly.
Suggested-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The `-c` option to stat is not POSIX compatible, and hence prevents the
check-endbr.sh script from running reliably.
The first instance of `stat -c` can be removed by fetching the section size
from the output of objdump itself, which the script already parses to get
the VMA values.
The other two instances can be replaced by counting the lines in the
respective files. Those files contain list of addresses, so the size in
bytes is not strictly needed, we can count the number of lines instead.
Suggested-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
On x86 GNU assembler will consider '/' as the start of comment marker for
some ELF targets. This is incorrect with Xen's usage, which does use '/'
in assembly files as a mathematical operator.
The behavior of the assembler can be altered by passing the --divide
option; unconditionally pass this option when available to force the
expected behavior.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
If shadow paging has been compiled out short circuit the creation of HVM
guests that attempt to use shadow paging at arch_sanitise_domain_config().
There's no need to further build the domain when creation is doomed to fail
later on.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
Currently do_kexec_op_internal() will return 0 for unknown hypercalls. Fix
this by returning -EOPNOTSUPP instead.
While there remove the pointless initialization of ret to -EINVAL; it just
makes the code confusing.
Fixes: d046f361dc ("Xen Security Modules: XSM")
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Convert the ARM SRCARCH variable setting to use the more common 'arm.*'
expression rather 'arm\(32\|64\)', as the later doesn't work with BSD sed
for not being a basic regular expression. The usage of | alternations is
not part of basic regexp support.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
The limitation of shared_info being allocated below 4G to fit in the
start_info field only applies to 32bit PV guests. On 64bit PV guests the
start_info field is 64bits wide. HVM guests don't use start_info at all.
Drop the restriction in arch_domain_create() and instead free and
re-allocate the page from memory below 4G if needed in switch_compat(),
when the guest is set to run in 32bit PV mode.
Fixes: 3cadc0469d ("x86_64: shared_info must be allocated below 4GB as it is advertised to 32-bit guests via a 32-bit machine address field in start_info.")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Physmap population has the need to use pages as big as possible to reduce
p2m shattering. However that triggers issues when big enough pages are not
yet scrubbed, and so scrubbing must be done at allocation time. On some
scenarios with added contention the watchdog can trigger:
Watchdog timer detects that CPU55 is stuck!
----[ Xen-4.17.5-21 x86_64 debug=n Not tainted ]----
CPU: 55
RIP: e008:[<ffff82d040204c4a>] clear_page_sse2+0x1a/0x30
RFLAGS: 0000000000000202 CONTEXT: hypervisor (d0v12)
[...]
Xen call trace:
[<ffff82d040204c4a>] R clear_page_sse2+0x1a/0x30
[<ffff82d04022a121>] S clear_domain_page+0x11/0x20
[<ffff82d04022c170>] S common/page_alloc.c#alloc_heap_pages+0x400/0x5a0
[<ffff82d04022d4a7>] S alloc_domheap_pages+0x67/0x180
[<ffff82d040226f9f>] S common/memory.c#populate_physmap+0x22f/0x3b0
[<ffff82d040228ec8>] S do_memory_op+0x728/0x1970
Introduce a mechanism to preempt page scrubbing in populate_physmap(). It
relies on stashing the dirty page in the domain struct temporarily to
preempt to guest context, so the scrubbing can resume when the domain
re-enters the hypercall. The added deferral mechanism will only be used for
domain construction, and is designed to be used with a single threaded
domain builder. If the toolstack makes concurrent calls to
XENMEM_populate_physmap for the same target domain it will trash stashed
pages, resulting in slow domain physmap population.
Note a similar issue is present in increase reservation. However that
hypercall is likely to only be used once the domain is already running and
the known implementations use 4K pages. It will be deal with in a separate
patch using a different approach, that will also take care of the
allocation in populate_physmap() once the domain is running.
Fixes: 74d2e11ccf ("mm: Scrub pages in alloc_heap_pages() if needed")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Future changes will care about the state of the PGC_need_scrub flag even
when pages have the PGC_allocated set, and hence it's no longer possible to
alias both values. Also introduce PGC_need_scrub to the set of preserved
flags, so it's not dropped by assign_pages().
No functional change intended, albeit the page counter on x86 looses a bit.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The previous logic attempted to skip an IBPB in the case of vCPU returning to
a CPU on which it was the previous vCPU to run. While safe for Xen's
isolation between vCPUs, this prevents the guest kernel correctly isolation
between tasks. Consider:
1) vCPU runs on CPU A, running task 1.
2) vCPU moves to CPU B, idle gets scheduled on A. Xen skips IBPB.
3) On CPU B, guest kernel switches from task 1 to 2, issuing IBPB.
4) vCPU moves back to CPU A. Xen skips IBPB again.
Now, task 2 is running on CPU A with task 1's training still in the BTB.
Do the flush unconditionally when switching to a vCPU different than the
idle one. Note there's no need to explicitly gate the IBPB to next domain
!= idle, as the context where the IBPB is issued is subject to that
condition already unless the pCPU is going offline, at which point we don't
really care to issue an extra IBPB.
Also add a comment with the reasoning why the IBPB needs to be in
context_switch() rather than __context_switch().
This is XSA-479 / CVE-2026-23553.
Fixes: a2ed643ed7 ("x86/ctxt: Issue a speculation barrier between vcpu contexts")
Reported-by: David Kaplan <david.kaplan@amd.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The current logic splits the update of the amount of available memory in
the system (total_avail_pages) and pending claims into two separately
locked regions. This leads to a window between counters adjustments where
the result of total_avail_pages - outstanding_claims doesn't reflect the
real amount of free memory available, and can return a negative value due
to total_avail_pages having been updated ahead of outstanding_claims.
Fix by adjusting outstanding_claims and d->outstanding_pages in the same
place where total_avail_pages is updated. Note that accesses to
d->outstanding_pages is protected by the global heap_lock, just like
total_avail_pages or outstanding_claims. Add a comment to the field
declaration, and also adjust the comment at the top of
domain_set_outstanding_pages() to be clearer in that regard.
Note that failures in assign_pages() causes the claimed amount that has
been allocated to be lost, as the amount is not added back to the domain
quota once pages are freed. Given the intended usage of claims is limited
to initial physmap populate, and the current failure paths in
assign_pages() would lead to the domain being destroyed anyway, don't
add extra logic to recover the claimed amount on failure - it's just adding
complexity for no real benefit.
Fixes: 65c9792df6 ("mmu: Introduce XENMEM_claim_pages (subop of memory ops)")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
XENMAPSPACE_gmfn{_range} allows moving gfn around the guest p2m: the mfn
behind the source gfn is zapped from the origin and mapped at the
requested destination gfn. The destination p2m entries are always created
with type p2m_ram_rw.
With the current checking done in xenmem_add_to_physmap_one() it's possible
to use XENMAPSPACE_gmfn{_range} to change the type of a p2m entry. The
source gfn is only checked to be not shared, and that the underlying page
is owned by the domain.
Make the source checks more strict, by checking that the source gfn is of
type read/write RAM or logdirty. That prevents the operation from
inadvertently changing the type as part of the move.
Fixes: 3e50af3d87 ("New XENMAPSPACE_gmfn parameter for XENMEM_add_to_physmap.")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Switch the current version to 15.0-RELEASE. Sadly the 16 snapshot images
are not working, hence use the FREEBSD_CURRENT variable as a placeholder
for 15.0 until the issues with FreeBSD 16.0 snapshot images is solved.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Introduce an additional memflags field to the xc_dom_image and
xc_sr_context structures and use it to pass additional memflags to use when
populating the domain physmap.
No meaningful usages of this new field are added as part of the patch. The
only know usage will be from the XAPI domain builder, which lives in a
different repository.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
The coloring allocator ignores claims; reject the usage of the hypercall if
LLC coloring is enabled. Note that the usage of alloc_color_heap_page() is
selected at boot time, and it's a system wide decision. Once enabled all
domains will use alloc_color_heap_page() for allocations, which ignores any
outstanding claims.
The hypercall can be enabled if/when LLC coloring is adjusted to support
claims.
Fixes: 95debe51a9 ("xen: add cache coloring allocator for domains")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Add the missing index increase in the M2P clearing loop, otherwise the loop
keeps pointlessly setting the same MFN entry repeatedly. This seems to be
an oversight from the change that introduced support to process high order
pages in one go.
Fixes: 3c352011c0 ("x86/PoD: shorten certain operations on higher order ranges")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Move x86's free_domain_struct() to common code since it is shared between
architectures.
Move the x86 version of alloc_domain_struct() to common code as most of the
logic is architecture-independent. To handle the remaining architectural
differences, introduce arch_domain_struct_memflags() for x86-specific
allocation requirements.
No functional change.
Suggested-by: Jan Beulich <jbeulich@suse.com>
[Introduce an arch-specific function instead of using a weak function]
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
There's no need to place the LLVM coverage sections in the linker script if
not building with clang, or if code coverage is not enabled.
No functional change intended.
Fixes: 186c02e859 ('xen: Support LLVM raw profile versions 5, 6, 7, 8, 9, and 10')
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The usage of atomic_dec_and_test() in msixtbl_pt_unregister() is inverted:
the function will return true when the refcount reaches 0. The current
code does the opposite and calls del_msixtbl_entry() when there are still
refcounts held on the object.
However all callers of msixtbl_pt_unregister() are serialized on the domctl
lock, and hence there cannot be parallel calls to msixtbl_pt_unregister()
that could lead to double freeing of the same object.
The incorrect freeing with active msixtlb entries will result in a possible
guest visible malfunction, but no internal Xen state corruption.
While entries are leaked once the last pIRQ is unbound, the same entry
would get re-used if the device has pIRQs bound again. The guest cannot
exploit this incorrect refcount check to leak arbitrary amounts of memory
by repeatedly enabling and disabling (binding and unbinding) MSI-X entries.
Fixes: 34097f0d30 ('hvm: passthrough MSI-X mask bit acceleration')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Setting the irq descriptor target CPU mask of high priority interrupts to
contain all online CPUs is not accurate. External interrupts are
exclusively delivered using physical destination mode, and hence can only
target a single CPU. Setting the descriptor CPU mask to contain all online
CPUs makes it impossible for Xen to figure out which CPU the interrupt is
really targeting.
Instead handle high priority vectors used by external interrupts similarly
to normal vectors, keeping the target CPU mask accurate. Introduce
specific code in _assign_irq_vector() to deal with moving high priority
vectors across CPUs, this is needed at least for fixup_irqs() to be able to
evacuate those if the target CPU goes offline.
Fixes: fc0c3fa2ad ("x86/IO-APIC: fix setup of Xen internally used IRQs (take 2)")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Avoid duplicated calls to irq_to_desc() by storing the result in a local
variable. No functional change.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
All implementations use the same hook which is a wrapper around
cpumask_of(cpu). Adjust callers to no longer use such dummy mask with a
single CPU set, as the CPU is already known to the caller. This removes a
pair of usages of for_each_cpu().
No functional change intended.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Commit d0193c6d67 introduced checking for the return value of
printf_info_one_json(), but failed to remove a further check down, which
now Coverity complains is dead code.
The correct fix is to make the call to json_object_array_add() conditional
on printf_info_one_json() having been successful, so that the further call
to libxl_domain_config_dispose() is not avoided.
CID: 1667251
Fixes: d0193c6d67 ("tools/xl: check return value of printf_info_one_json() in list_domains_details()")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Otherwise it's not possible for device models to map IRQs of devices on
segments different than 0. Keep the same function prototype and pass the
segment in the high 16bits of the bus parameter, like it's done for the
hypercall itself.
Amends: 7620c0cf9a ("PCI multi-seg: add new physdevop-s")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
printf_info_one_json() won't initialize the passed jso_r parameter on
error, and hence the usage in printf_info() needs prior initialization,
otherwise an uninitialized pointer is passed to json_object_put() on
failure.
Reported by the internal XenServer Coverity instance.
Fixes: f6c6f2679d ("libxl: Convert libxl__object_to_json() to json-c")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
And return from the function early if printf_info_one_json() returns error.
Reported by internal XenServer Coverity instance.
Fixes: f6c6f2679d ("libxl: Convert libxl__object_to_json() to json-c")
Signed-off-by: Roger Pau Monné <roger.pua@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Check the return codes of json_object_object_add() calls in
printf_info_one_json().
Reported by XenServer internal Coverity instance.
Fixes: f6c6f2679d ("libxl: Convert libxl__object_to_json() to json-c")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
It's possible for libxl__json_object_to_json_object() to not set the passed
jso_out parameter, hence initialize it in libxl__json_object_to_json() to
avoid freeing an uninitialized pointer in case of failure.
Reported by XenServer internal Coverity instance.
Fixes: 75fa670e58 ("libxl: Convert libxl__json_object_to_json() to json_object")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
It's possible for json_tokener_parse_verbose() to return NULL and leave the
error parameter unset. Initialize the error variable at definition to an
existing error token to make sure it's not uninitialized in case of
failure.
Reported by XenServer internal Coverity instance.
Fixes: 7e95dab9eb ("libxl: Convert libxl__json_parse() to use json-c")
Fixes: f6c6f2679d ("libxl: Convert libxl__object_to_json() to json-c")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
I've had the luck to come across a PCI card that exposes a MSI-X capability
where the BIR of the vector and PBA tables points at a BAR that has 0 size.
This doesn't play nice with the code in vpci_make_msix_hole(), as it would
still use the address of such empty BAR (0) and attempt to carve a hole in
the p2m. This leads to errors like the one below being reported by Xen:
d0v0 0000:22:00.0: existing mapping (mfn: 181c4300 type: 0) at 0 clobbers MSIX MMIO area
And the device left unable to enable memory decoding due to the failure
reported by vpci_make_msix_hole().
Introduce checking in init_msix() to ensure the BARs containing the MSI-X
tables are usable. This requires checking that the BIR points to a
non-empty BAR, and the offset and size of the MSI-X tables can fit in the
target BAR.
This fixes booting PVH dom0 on Supermicro AS -2126HS-TN servers with AMD
EPYC 9965 processors. The broken device is:
22:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 93)
There are multiple of those integrated controllers in the system, all
broken in the same way.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Released-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
The default shell in FreeBSD is plain sh, which doesn't support the usage
of the '|&' operator:
./automation/scripts/console.exp |& sed 's/\r\+$//'
/tmp/scripts34c427adc6599db29cb91221a0939d85.sh: 16: Syntax error: "&" unexpected
Fix this by using a redirection of stderr into stdout, and then a pipe.
Fixes: a406195c15 ("automation: call expect script with redirected standard error")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
FreeBSD QEMU build requires libinotify, otherwise the build fails with:
./qemu-xen-dir-remote/meson.build:2535:15: ERROR: C shared or static library 'inotify' not found
Install the package and add the /usr/local include and libs paths
explicitly as QEMU configure options.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reading from the E9 port if the emergency console is active should return
0xe9 according to the documentation from Bochs:
https://bochs.sourceforge.io/doc/docbook/user/bochsrc.html
See `port_e9_hack` section description.
Fix Xen so it also returns the port address. OSes can use it to detect
whether the emergency console is available or not.
Fixes: d1bd157fbc ("Big merge the HVM full-virtualisation abstractions.")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Otherwise the check for the SS feature in
check_memory_type_self_snoop_errata() fails unconditionally, which leads to
X86_FEATURE_XEN_SELFSNOOP never being set.
We could also avoid this by not doing the reset_cpuinfo() for the BSP in
identify_cpu(), because SS detection uses boot_cpu_data. However that
creates an imbalance on the state of the BSP versus the APs in the
identify_cpu() code.
I've opted for the less controversial solution of populating FEATURESET_1d
in generic_identify(), as the value is already there. The same is done for
the AMD faulting probe code. While there also populate the FEATURESET_1c
element, as the value is in context.
Fixes: f2663ca2e5 ("x86/cpu/intel: Clear cache self-snoop capability in CPUs with known errata")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The reference TSC page is shared between all vCPUs, and the data stored in
the domain struct. However the handlers to set and clear it are not safe
against concurrent accesses. It's possible for two (or more) vCPUs to call
HV_X64_MSR_REFERENCE_TSC at the same time and cause the in-use reference
TSC page to be freed, while still being on the p2m. This creates a use-
after-free bug, where the page can end up mapped in another domain or used
by Xen while still being part of the original domain's p2m.
It's also possible to underflow the reference counter, as multiple
concurrent writes to HV_X64_MSR_REFERENCE_TSC can create an imbalance on
the number of put_page_and_type() calls.
Introduce a lock to protect the reference TSC domain field, thus
serializing concurrent vCPU accesses.
This is CVE-2025-58143 / part of XSA-472.
Fixes: 386b336522 ('viridian: use viridian_map/unmap_guest_page() for reference tsc page')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The function is called unconditionally, regardless of whether the SIM page
is mapped. Avoid a NULL pointer dereference in
viridian_synic_deliver_timer_msg() by checking whether the SIM page is
mapped.
This is CVE-2025-58142 / part of XSA-472.
Fixes: 26fba3c855 ('viridian: add implementation of synthetic timers')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The function is only called when the MSR has the enabled bit set, but even
then the page might not be mapped because the guest provided gfn is not
suitable.
Prevent a NULL pointer dereference in update_reference_tsc() by checking
whether the page is mapped.
This is CVE-2025-27466 / part of XSA-472.
Fixes: 386b336522 ('viridian: use viridian_map/unmap_guest_page() for reference tsc page')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
There's no need for PDX compression given the memory map provided by Xen
to guests is contiguous, turn off the feature by default in the PV shim
defconfig.
Reported-by: Jan Beulich <jbeulich@suse.com>
Fixes: c5c45bcbd6 ("pdx: introduce a new compression algorithm based on region offsets")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The current code to restore the timer state on resume is incomplete. While
the local APIC Initial Count Register is saved and restored across
suspension (even if possibly no longer accurate since it's not adjusted to
account for the time spent in suspension), the TSC deadline MSR is not
saved and restored, hence hosts using the TSC deadline timer will likely
get stuck when resuming from suspension.
The lack of restoring of the TSC deadline MSR was mitigated by the raising
of a timer softirq in mwait_idle_with_hints() if the timer had expired,
previous to commit 3faf0866a3, which removed
that logic.
This patch fixes the usage of the TSC deadline timer with suspension, by
unconditionally raising a timer softirq on resume, that will take care of
rearming the hardware timer. Given that a timer softirq will be
unconditionally risen, there's no need to save and restore the APIC Initial
Count Register anymore either.
Note that secondary processors don't need this special treatment when
resuming, since they are offlined before suspension and brought back up
during resume, the first timer that gets setup will trigger a timer softirq
unconditionally, for example from sched_migrate_timers() that gets called
for each secondary processor.
Link: https://github.com/QubesOS/qubes-issues/issues/10110
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Fixes: fd1291a826 ('X86: Prefer TSC-deadline timer in Xen')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Otherwise the PCI accesses to segments different than the first one done by
the IOMMU initialization code would silently fail by returning all ones.
Introduce a new helper, called pci_setup(), and move both the creation of
PCI segment 0 internal data structures, plus the parsing of ACPI MMCFG
table to it. This moves acpi_mmcfg_init() slightly earlier from
acpi_boot_init() into pci_setup().
Note that further work will be needed to support systems where access
methods to segments different than 0 is not discoverable by Xen.
Fixes: 3950f2485b ('x86/x2APIC: defer probe until after IOMMU ACPI table parsing')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
With the appearance of Intel Sierra Forest and Granite Rapids it's now
possible to get a production x86 host with the following memory map:
SRAT: Node 0 PXM 0 [0000000000000000, 000000007fffffff]
SRAT: Node 0 PXM 0 [0000000100000000, 000000807fffffff]
SRAT: Node 1 PXM 1 [0000063e80000000, 000006be7fffffff]
SRAT: Node 2 PXM 2 [00000c7e80000000, 00000cfe7fffffff]
SRAT: Node 3 PXM 3 [000012be80000000, 0000133e7fffffff]
This is from a four socket Granite Rapids system, with each node having
512GB of memory. The total amount of RAM on the system is 2TB, but without
enabling CONFIG_BIGMEM the last range is not accessible, as it's above the
16TB boundary covered by the frame table. Sierra Forest and Granite Rapids
are socket compatible, however Sierra Forest only supports 2 socket
configurations, while Granite Rapids can go up to 8 sockets.
Note that while the memory map is very sparse, it couldn't be compressed
using the current PDX_MASK compression algorithm, which relies on all
ranges having a shared zeroed region of bits that can be removed.
The memory map presented above has the property of all regions being
similarly spaced between each other, and all having also a similar size.
Use a lookup table to store the offsets to translate from/to PFN and PDX
spaces. Such table is indexed based on the input PFN or PDX to translated.
The example PFN layout about would get compressed using the following:
PFN compression using PFN lookup table shift 29 and PDX region size 0x10000000
range 0 [0000000000000, 0x0000807ffff] PFN IDX 0 : 0000000000000
range 1 [0x00063e80000, 0x0006be7ffff] PFN IDX 3 : 0x00053e80000
range 2 [0x000c7e80000, 0x000cfe7ffff] PFN IDX 6 : 0x000a7e80000
range 3 [0x0012be80000, 0x00133e7ffff] PFN IDX 9 : 0x000fbe80000
Note how the tow ranges belonging to node 0 get merged into a single PDX
region by the compression algorithm.
The default size of lookup tables currently set in Kconfig is 64 entries,
and the example memory map consumes 10 entries. Such memory map is from a
4 socket Granite Rapids host, which in theory supports up to 8 sockets
according to Intel documentation. Assuming the layout of a 8 socket system
is similar to the 4 socket one, it would require 21 lookup table entries to
support it, way below the current default of 64 entries.
The valid range of lookup table size is currently restricted from 1 to 512
elements in Kconfig.
An extra array is used to keep track of the base PFN for each translated
range. Non used slots are set to ~0UL, so that in mfn_valid() the mfn <
base check always fails, thus reporting the mfn as invalid.
Introduce __init_or_pdx_mask and use it on some shared functions between
PDX mask and offset compression, as otherwise some code becomes unreachable
after boot if PDX offset compression is used. Mark the code as __init in
that case, so it's pruned after boot.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Otherwise Coverity complains about possibly shifting an integer more than
31 bits.
This also reverts the previous attempt to fix this Coverity reported
issue, commit 4dd3230290.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Coverity ID: 1662707
Fixes: cb50e40337 ('test/pdx: add PDX compression unit tests')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Coverity complains that:
In expression 0x1000 << bottom_shift, left shifting by more than 31 bits
has undefined behavior. The shift amount, bottom_shift, is as much as 63.
Cast PAGE_SIZE to paddr_t so it has the right width.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Coverity ID: 1662707
Fixes: bac2000063 ('x86-64: reduce range spanned by 1:1 mapping and frame table indexes')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Move fill_mask(), pdx_region_mask() and pdx_init_mask() to the
!CONFIG_PDX_NONE section in preparation of them also being used by a newly
added PDX compression.
No functional change intended.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce a set of unit tests for PDX compression. The unit tests contains
both real and crafted memory maps that are then compressed using the
selected PDX algorithm. Note the build system for the unit tests has been
done in a way to support adding new compression algorithms easily. That
requires generating a new test-pdx-<compress> executable that's build with
the selected PDX compression enabled.
Currently the only generated executable is test-pdx-mask that tests PDX
mask compression.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
There are four performance critical PDX conversion helpers that do the PFN
to/from PDX and the physical addresses to/from directmap offsets
translations.
In the absence of an active PDX compression, those functions would still do
the calculations needed, just to return the same input value as no
translation is in place and hence PFN and PDX spaces are identity mapped.
To reduce the overhead of having to do the pointless calculations allow
architectures to implement the translation helpers in a per-arch header.
Rename the existing conversion functions to add a trailing _xlate suffix,
so that the per-arch headers can define the non suffixed versions.
Currently only x86 implements meaningful custom handlers to short circuit
the translation when not active, using asm goto. Other architectures use
generic macros that map the non-xlate to the xlate variants to keep the
previous behavior.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Introduce a command line option to allow disabling PDX compression. The
disabling is done by turning pfn_pdx_add_region() into a no-op, so when
attempting to initialize the selected compression algorithm the array of
ranges to compress is empty.
Signed-off-by: Roger Pau Monné <roger.pau@cloud.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The current setup (pdx_init_mask() and pdx_region_mask()) and init
(pfn_pdx_hole_setup()) PDX compression functions are tailored to the
existing PDX compression algorithm.
In preparation for introducing a new compression algorithm convert the
setup and init functions to more generic interfaces that aren't tied to the
compression in-use. To accomplish this introduce a function that registers
all the PFN RAM ranges, plus an init function.
This has the downside of requiring a static array to store such ranges
ahead of being processed by the setup function, however it's the only way
to pass all the possible information to the different compression setup
functions without using per-compression specific setup functions.
Per-arch compression setup also need to be adjusted to use the new
interface. There's a slight ordering adjustment, in that after PDX
compression setup the caller will check whether all the RAM regions are
properly covered by the newly setup compression, otherwise compression is
disabled by resetting to the initial values.
No functional change intended in the resulting PDX compression values.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com> # ARM
Rename the current CONFIG_PDX_COMPRESSION to CONFIG_PDX_MASK_COMPRESSION,
and make it part of the PDX compression choice block, in preparation for
adding further PDX compression algorithms.
The PDX compression defaults should still be the same for all
architectures, however the choice block cannot be protected under EXPERT
and still have a default choice being unconditionally selected. As a
result, the new "PDX (Page inDeX) compression" item will be unconditionally
visible in Kconfig, even on architectures like x86 that previously had no
way to enable PDX compression.
As part of this preparation work to introduce new PDX compressions, adjust
some of the comments on pdx.h to note they apply to a specific PDX
compression. Also shuffle function prototypes and dummy implementations
around to make it easier to introduce a new PDX compression. Note all
PDX compression implementations are expected to provide a
pdx_is_region_compressible() that takes the same set of arguments.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
The current logic in the HPET interrupt ->ack() hook will perform a local
APIC EOI ahead of enabling interrupts, possibly leading to recursion in the
interrupt handler.
Fix this by doing the local APIC EOI strictly after the window with
interrupt enabled, as that prevents the recursion, and would only allow for
interrupts with higher priority to be serviced.
Use the generic ack_nonmaskable_msi_irq() and end_nonmaskable_irq()
functions, removing the need for hpet_msi_ack().
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: 3ba523ff95 ('CPUIDLE: enable MSI capable HPET for timer broadcast')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Clang UBSAN reports:
UBSAN: Undefined behaviour in drivers/char/ns16550.c:124:49
applying non-zero offset 0000000000000001 to null pointer
And
UBSAN: Undefined behaviour in drivers/char/ns16550.c:142:49
applying non-zero offset 0000000000000001 to null pointer
Move calculation of the MMIO address so it's only done if the MMIO is
mapped by Xen, otherwise the address is not consumed anyway because IO port
access is used instead.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The current strtoll() implementation in hvmloader requires hex number to be
prefixed with 0x, otherwise strtoll() won't parse them correctly even when
calling the function with base == 16.
Fix this by not unconditionally setting the base to 10 when the string is
not 0 prefixed, this also allows parsing octal numbers not 0 prefixed.
While there also handle '0X' as a valid hex number prefix, together with
'0x'.
No functional change intended to the existing call sites.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The current p2m type-change loop in ept_vcpu_flush_pml_buffer() relies on
each call to p2m_change_type_one() taking the p2m lock, doing the change
and then dropping the lock and flushing the p2m. Instead take the p2m lock
outside of the loop, so that calls to gfn_{,un}lock() inside
p2m_change_type_one() just take the p2m lock recursively, and more
importantly, the flush is deferred until the p2m is unlocked in
ept_vcpu_flush_pml_buffer().
No functional change intended in the end result of
ept_vcpu_flush_pml_buffer(), however a possibly noticeable performance
improvement is expected.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
As a result of the addition of a new field in libxl domain build info
structure the golang types need to be regnerated, this was missing as part
of 22650d6054.
Regenerate the headers now.
Reported-by: Juergen Gross <jgross@suse.com>
Fixes: 22650d6054 ('x86/hvmloader: select xen platform pci MMIO BAR UC or WB MTRR cache attribute')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Nick Rosbrook <enr0n@ubuntu.com>
When not using CONFIG_BIGMEM there are some restrictions in the address
width for allocations of the domain structure, as it's PDX truncated to 32
bits it's stashed into page_info structure for domain allocated pages.
The current logic to calculate this limit is based on the internals of the
PDX compression used, which is not strictly required. Instead simplify the
logic to rely on the existing PDX to PFN conversion helpers used elsewhere.
This has the added benefit of allowing alternative PDX compression
algorithms to be implemented without requiring to change the calculation of
the domain structure allocation boundary.
As a side effect introduce pdx_to_paddr() conversion macro and use it.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The Xen platform PCI device (vendor ID 0x5853) exposed to x86 HVM guests
doesn't have the functionality of a traditional PCI device. The exposed
MMIO BAR is used by some guests (including Linux) as a safe place to map
foreign memory, including the grant table itself.
Traditionally BARs from devices have the uncacheable (UC) cache attribute
from the MTRR, to ensure correct functionality of such devices. hvmloader
mimics this behavior and sets the MTRR attributes of both the low and high
PCI MMIO windows (where BARs of PCI devices reside) as UC in MTRR.
This however causes performance issues for users of the Xen platform PCI
device BAR, as for the purposes of mapping remote memory there's no need to
use the UC attribute. On Intel systems this is worked around by using
iPAT, that allows the hypervisor to force the effective cache attribute of
a p2m entry regardless of the guest PAT value. AMD however doesn't have an
equivalent of iPAT, and guest PAT values are always considered.
Linux commit:
41925b105e34 xen: replace xen_remap() with memremap()
Attempted to mitigate this by forcing mappings of the grant-table to use
the write-back (WB) cache attribute. However Linux memremap() takes MTRRs
into account to calculate which PAT type to use, and seeing the MTRR cache
attribute for the region being UC the PAT also ends up as UC, regardless of
the caller having requested WB.
As a workaround to allow current Linux to map the grant-table as WB using
memremap() introduce an xl.cfg option (xen_platform_pci_bar_uc=0) that can
be used to select whether the Xen platform PCI device BAR will have the UC
attribute in MTRR. Such workaround in hvmloader should also be paired with
a fix for Linux so it attempts to change the MTRR of the Xen platform PCI
device BAR to WB by itself.
Overall, the long term solution would be to provide the guest with a safe
range in the guest physical address space where mappings to foreign pages
can be created.
Some vif throughput performance figures provided by Anthoine from a 8
vCPUs, 4GB of RAM HVM guest(s) running on AMD hardware:
Without this patch:
vm -> dom0: 1.1Gb/s
vm -> vm: 5.0Gb/s
With the patch:
vm -> dom0: 4.5Gb/s
vm -> vm: 7.0Gb/s
Reported-by: Anthoine Bourgeois <anthoine.bourgeois@vates.tech>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Oleksii Kurochko<oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com> # hvmloader
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
After some recent change the order of MTRR vs PCI initialization is
inverted. MTRR will get initialization ahead of PCI scanning and sizing of
MMIO regions. As a result when setting up MTRRs the MMIO window below 4GB
will always have the same size, and there will be no window above 4GB.
This results in malformed and incomplete MTRRs being setup.
Fix by making sure PCI is initialized ahead of MTRR, also add a comment to
notice the ordering dependency.
Fixes: 2c3dffbaa3 ('tools/hvmloader: Replace LAPIC_ID() with cpu_to_apicid[]')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthoine Bourgeois <anthoine.bourgeois@gmail.com>
With the current AP bring up code, Xen can get stuck indefinitely if an AP
freezes during boot after the 'callin' step. Introduce a 5s timeout while
waiting for APs to finish startup.
On failure of an AP to complete startup, send an NMI to trigger the
printing of a stack backtrace on the stuck AP and panic on the BSP.
This patch was done while investigating the issue caused by Intel erratum
ICX143. It wasn't helpful in that case, but it's still and improvement
when debugging AP bring up related issues.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Roger Pau Monne