Add the missing index increase in the M2P clearing loop, otherwise the loop
keeps pointlessly setting the same MFN entry repeatedly. This seems to be
an oversight from the change that introduced support to process high order
pages in one go.
Fixes: 3c352011c0 ("x86/PoD: shorten certain operations on higher order ranges")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Use a define to centralize the common scheduler data in the per-arch
linker scripts. This is in preparation for marking it KEEP(). While
moving, switch to using POINTER_ALIGN.
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Rule 11.1 states as following: "Conversions shall not be performed
between a pointer to a function and any other type."
This deviation from Rule 11.1 relies on both ABI definitions and compiler
implementations supported by Xen. The System V x86_64 ABI and the AArch64
ELF ABI define consistent and compatible representations (i.e., having
the same size and memory layout) for (void *), unsigned long, and function
pointers, enabling safe conversions between these types without data loss
or corruption. Additionally, GCC and Clang, faithfully implement the ABI
specifications, ensuring that the generated machine code conforms to these
guarantees. Developers must note that this behavior is not universal and
depends on platform-specific ABIs and compiler implementations.
Configure Eclair to avoid reporting violations for conversions from
unsigned long or (void *) to a function pointer.
Add a compile-time assertion into the file 'xen/common/version.c' to
confirm this conversion compatibility across X86 and ARM platforms
(assuming this file is common for them).
References:
- System V x86_64 ABI: https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-ABI/abi.pdf?job=build
- AArch64 ELF ABI: https://github.com/ARM-software/abi-aa/releases
- GCC: https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html
- Clang: https://clang.llvm.org/docs/CrossCompilation.html
Signed-off-by: Dmytro Prokopchuk <dmytro_prokopchuk1@epam.com>
Reviewed-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
[stefano: remove special powerpc check]
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
The hv_compat_vstart variable is hidden behind CONFIG_PV32 but lives in
arch_domain. Moving it into pv_domain is an obvious improvement.
Its value however is less obvious, and is a mess.
In !PV32 builds, it's uniformly treated as 0, but in PV32 builds it's ~0U (HVM
guests), __HYPERVISOR_COMPAT_VIRT_START (PV guests), or custom (PV32 dom0).
This seems to work because users are guarded behind is_pv32_{domain,vcpu}().
Simplify things by leaving it as 0 in PV32 builds for most domains,
initialising it in only in switch_compat() when a domain becomes 32bit PV.
dom0_construct() adjusts the value after calling switch_compat().
Suggested-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Prior to commit 02e78311cd ("x86/domctl: Make XEN_DOMCTL_set_address_size
singleshot", Xen 4.9, 2016), it was possible for domains to switch to being
compat, and back again. Since then however, becoming compat is a singleton
action that can't be undone.
From the context it's clear to see the is_pv_32bit_domain() check is
redundant, and from the singleton nature and being the only place setting
physaddr_bitsize, there's no need to check it for being 0.
No functional change.
Co-developed-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Grygorii Strashko <grygorii_strashko@epam.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The variable, when compiled in, is never 0, but because the writes into it are
hidden behind the HYPERVISOR_COMPAT_VIRT_START() macro, it has a very good
impression of being 0.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Grygorii Strashko <grygorii_strashko@epam.com>
Linux jumps to the target image rather than calling it. Switch to using JMP,
and drop the trailing UD2s.
Linux does have a mode, named CONFIG_KEXEC_JUMP, where the target image can be
returned from, but that involves extra metadata and setting up a stack within
the target image which Xen doesn't support at the moment.
No functional change.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
With symbols-dummy.c gone, there's only on party using this header. Fold
its contents back into symbols.c.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Introduce arch/x86/lib/, and make it the home for the somewhat misplaced
x86-specific file that lived in the arch-independent lib/.
Introduce ARCH_LIBS-y as a make variable, to arrange for arch-specific
libraries to (generally) come ahead of generic one(s) when linking. Should
any library be intended to come after the generic one(s), it can be
appended to $(ALL_LIBS-y).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
MISRA C Rule 2.1 states: "A project shall not contain unreachable code".
In certain build configurations the following functions 'prepare_acpi()'
and 'gicv3_its_setup_collection()' are defined as inline functions and
contain the macro 'BUG()'. This resulted in violations due to these
functions became non-returning.
To ensure compliance with MISRA C Rule 2.1 remove inline function
implementations and their 'BUG()'-based unreachable code. Provide
unconditional function declarations for 'gicv3_its_setup_collection()'
and 'prepare_acpi()'. Rely on the compiler's DCE to remove unused function
calls in builds where these configs 'CONFIG_ACPI' or 'CONFIG_HAS_ITS' are
not enabled.
Signed-off-by: Dmytro Prokopchuk <dmytro_prokopchuk1@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Move x86's free_domain_struct() to common code since it is shared between
architectures.
Move the x86 version of alloc_domain_struct() to common code as most of the
logic is architecture-independent. To handle the remaining architectural
differences, introduce arch_domain_struct_memflags() for x86-specific
allocation requirements.
No functional change.
Suggested-by: Jan Beulich <jbeulich@suse.com>
[Introduce an arch-specific function instead of using a weak function]
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
The vcpu_kick() declaration is duplicated across multiple
architecture-specific event.h headers (ARM, x86, PPC).
Remove the redundant declarations and move vcpu_kick() into
the common xen/include/xen/sched.h header.
Drop the definition of vcpu_kick() from ppc/include/asm/event.h,
as it is already provided in ppc/stubs.c.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
It is sufficient to use HGATP*_MODE_MASK and HGATP*_VMID_MASK without
the corresponding *_SHIFT definitions.
Rename HGATP{32,64}_PPN to HGATP{32,64}_PPN_MASK to more accurately
describe their purpose. The top-level HGATP_PPN and related aliases are
updated accordingly.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
On x86, the opening if of call_handlers_*() is indented 4 spaces which
breaks alignment. The alignment is broken by printf(" "), but
that actually aligns ARM's opening "if ( likely() )". Move the indent
to the if that needs it.
This results in x86 hypercall-defs.h changes like:
#define call_handlers_hvm64(num, ret, a1, a2, a3, a4, a5) \
({ \
uint64_t mask = (num) > 63 ? 0 : 1ULL << (num); \
- if ( likely(mask & ((1ULL << __HYPERVISOR_vcpu_op) | (1ULL << __HYPERVISOR_event_channel_op))) ) \
+ if ( likely(mask & ((1ULL << __HYPERVISOR_vcpu_op) | (1ULL << __HYPERVISOR_event_channel_op))) ) \
{ \
if ( (num) == __HYPERVISOR_vcpu_op ) \
ARM's hypercall-defs.h does not change.
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
In particular when linking with lld, which converts hidden symbols to
local ones, the ELF symbol table can change in unhelpful ways between the
first two linking passes, resulting in the .rodata contributions to change
between the 2nd and 3rd pass. That, however, renders our embedded symbol
table pretty much unusable; the recently introduced self-test may then
also fail. (Another difference between compiling a C file and assembling
the generated ones is that - with -fdata-sections in use - the .rodata
contributions move between passes 1 and 2, when we'd prefer them not to.)
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
In preparation to do away with symbols-dummy, re-number the assembly and
object files used, for the numbers to match the next passes real output.
This is to make 0 available to use for what now is handled by
symbols-dummy.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
It's soon going to be needed in a second location.
Right now it's misleading saying that nothing else would be cleared. It's
missing the more important point that SYSCALLs are treated like all other
interrupts and exceptions, and undergo normal flags handling there.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Right now we have two IRET instructions that can fault for guest reasons, and
the pre exception table gives handle_exception as the fixup for both.
Instead, we can have compat_restore_all_guest() use restore_all_guest()'s IRET
which gives us just a single position to handle specially.
In exception_with_ints_disabled(), remove search_pre_exception_table() and use
a simpler check. Explain how the recovery works, because this isn't the first
time I've had to reverse engineer it for my own understanding.
The reference to iret_to_guest highlights that any checking here is specific
to CONFIG_PV, so exclude it in !PV builds.
Later in exception_with_ints_disabled(), it suffices to load %ecx rather than
%rcx, and remove a stray semi-colon from the REP MOVSQ.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
FRED provides %cr2 in the the stack frame, avoiding the need to read %cr2
manually.
Rename do_page_fault() to handle_PF(), and update it to take cr2, still named
addr for consistency.
Introduce a new handle_PF_IDT() which reads %cr2 and conditionally re-enables
interrupts.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
FRED provides PENDING_DBG in the the stack frame, avoiding the need to read
%dr6 manually.
Rename do_debug() to handle_DB(), and update it to take a dbg field using
positive polarity.
Introduce a new handle_DB_IDT() which reads %dr6.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
This is really a rearrangement to make adding FRED support easier.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Under FRED, SETSSBSY is disallowed, and we want to be setting up FRED prior to
setting up shadow stacks. As we still need Supervisor Tokens in IDT mode, we
need mode-specific logic to establish SSP.
In FRED mode, write a Restore Token, RSTORSSP it, and discard the resulting
Previous-SSP token.
No change outside of FRED mode.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
It turns out that using the higher level helpers adjacent like this leads to
terrible code generation. Due to -fno-strict-aliasing, the store into state
invalidates the read_cr4() address calculation (which is really cpu_info->cr4
under the hood), meaning that it can't be hoisted.
As a result we get "locate the top of stack block, get cr4, and see if
FSGSBASE is set" repeated 3 times, and an unreasonable number of basic blocks.
Hoist the calculation manually, which results in two basic blocks.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
... and rdmsr() while here.
Most of these accesses are in fastpaths and do not need serialising behaviour,
but the write side is serialising on all Intel hardware as well as older AMD
hardware.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Right now they're inline in {read,write}_gs_shadow(), but we're going to need
to use these elsewhere to support FRED.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The var parameter is later on expanded and referenced by LIST_HEAD_INIT
and shall be enclosed in parentheses. This resolves MISRA C R20.7
violation.
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Inside do_xen_version, there are two other places using local variables
named "params". Rename hypfs dir "params" to "params_dir". This resolves
MISRA C R5.3 violation.
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
TRC_PV_HYPERCALL_V2_ARG_MASK represents unsigned type, so according to
MISRA C R7.2. U suffix should be applied. Fix the violation.
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Keep the pkg-config lookup for SYSTEMD_SLEEP_DIR, but fall back to
"$(prefix)/lib/systemd/system-sleep/" if the variable remains unset.
This avoids reintroducing a build-time dependency on systemd-devel while
still honoring the path provided by pkg-config when present.
Fixes: e54a6cd6a1 ("systemd: Add hooks to stop/start xen-watchdog on suspend/resume")
Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Hans reports that Xen no longer builds on Debian unstable/sid:
Assembler messages:
{standard input}:474: Error: unknown or missing system register name at operand 1 -- `msr TEECR32_EL1,x0'
{standard input}:480: Error: unknown or missing system register name at operand 1 -- `msr TEEHBR32_EL1,x0'
{standard input}:488: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEECR32_EL1'
{standard input}:494: Error: unknown or missing system register name at operand 2 -- `mrs x0,TEEHBR32_EL1'
make[5]: *** [Rules.mk:249: arch/arm/domain.o] Error 1
This turns out to be an intentional change in binutils. ThumbEE was dropped
ahead of ARM v8 (i.e. AArch64).
Xen supports ARM v7+virt extensions so in principle we could #ifdef
CONFIG_ARM_32 to keep it working. However, there was apparently no use of
ThumbEE outside of demo code, so simply drop it.
On ThumbEE capable hardware, unconditionally trap ThumbEE instructions, and
drop the context switching logic for TEE{CR,HBR}32.
Reported-by: Hans van Kranenburg <hans@knorrie.org>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Prior to introducing GICv3 eSPI support, writes to these registers were
ignored. Now the behavior is changed and we inject fault to the guests.
According to documentation, eSPI registers are treated as RES0, when
GICD_TYPER.ESPI is 0 (this would be the case when CONFIG_GICV3_ESPI=n or
nr_espis is 0). Restore the previous behavior by checking whether eSPI
is in use or not.
Fixes: 93eecb2c4b ("xen/arm: vgic-v3: add emulation of GICv3.1 eSPI registers")
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
The following analysis jobs are performed:
- eclair-{x86_64,ARM64}: analyze Xen using the default configuration for
that architecture; runs on runners tagged `eclair-analysis'.
- eclair-{x86-64,ARM64}-safety: analyze Xen using the configuration for
safety, which is more restricted; runs on runners tagged
`eclair-analysis-safety`.
- eclair-{x86_64,ARM64}-testing: analyze Xen using the default
configuration for the purposes of testing new runner updates; runs on
runners tagged `eclair-analysis-testing`.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Future work will need to interact with the TPM, which requires calculating
digests for all active hash banks. Introduce an implementation in lib/,
partially derived from Trenchboot which itself is derived from Linux.
Signed-off-by: Krystian Hebel <krystian.hebel@3mdeb.com>
Signed-off-by: Sergii Dmytruk <sergii.dmytruk@3mdeb.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
In sha2_256_transform(), make state be an array of 8 rather than a pointer,
which lets the compiler do slightly more checking.
For better or worse, the canonical API in Linux is just the algorithm name.
As we're intending to import more from Linux in this area, drop the digest
suffix before we gain more users.
Compile sha2-256.o for all architectures. It's not very much code, and it
will be omitted at link time.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
We have two different functions explaining that DE_CFG is Core-scoped and that
writes are racy but happen to be safe. This is only true when there's one of
them.
Introduce amd_init_de_cfg() to be the singular function which writes to
DE_CFG, modelled after the logic we already have for BP_CFG.
While reworking amd_check_zenbleed() into a simple predicate used by
amd_init_de_cfg(), fix the microcode table. The 'good_rev' was specific to an
individual stepping and not valid to be matched by model, let alone a range.
The only CPUs incorrectly matched that I can locate appear to be
pre-production, and probably didn't get Zenbleed microcode.
Rework amd_init_lfence() to be amd_init_lfence_dispatch() with only the
purpose of configuring X86_FEATURE_LFENCE_DISPATCH in the case that it needs
synthesising. Run it on the BSP only and use setup_force_cpu_cap() to prevent
the bit disappearing on a subseuqent CPUID rescan.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
A XenServer feature in the process of being upstreamed is to be able to
re-calculate the guest CPU Policies at runtime, e.g. after a microcode load
and/or livepatch to expose new functionality. Right now, upstream Xen only
rescans the Raw CPU Policy on microcode load.
One complication with recalculating the guest policies is that BTC_NO is
handled differently to other $FOO_NO bits, by using __set_bit() rather than
setup_force_cpu_cap().
For consistency, switch it to using setup_force_cpu_cap(). This doesn't
matter for upstream Xen right now, but it will ease upstreaming the feature.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
This function can only be reached from EPT-related code which is inherently
HAP. Thus it is not useful to check for shadow_paging (or lack of HAP) there.
Moreover, it is an error to call this function in the non-EPT cases.
Not a functional change.
Signed-off-by: Teddy Astie <teddy.astie@vates.tech>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
In particular when linking with lld, which converts hidden symbols to
local ones, the ELF symbol table can change in unhelpful ways between the
first two linking passes, resulting in the .rodata contributions to change
between the 2nd and 3rd pass. That, however, renders our embedded symbol
table pretty much unusable; the recently introduced self-test may then
also fail. (Another difference between compiling a C file and assembling
the generated ones is that - with -fdata-sections in use - the .rodata
contributions move between passes 1 and 2, when we'd prefer them not to.)
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
In preparation to do away with symbols-dummy, re-number the assembly and
object files used, for the numbers to match the next passes real output.
This is to make 0 available to use for what now is handled by
symbols-dummy.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
In particular when linking with lld, which converts hidden symbols to
local ones, the ELF symbol table can change in unhelpful ways between the
first two linking passes, resulting in the .rodata contributions to change
between the 2nd and 3rd pass. That, however, renders our embedded symbol
table pretty much unusable; the recently introduced self-test may then
also fail. (Another difference between compiling a C file and assembling
the generated ones is that - with -fdata-sections in use - the .rodata
contributions move between passes 1 and 2, when we'd prefer them not to.)
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Timothy Pearson <tpearson@raptorengineering.com>
In preparation to do away with symbols-dummy, re-number the assembly and
object files used, for the numbers to match the next passes real output.
This is to make 0 available to use for what now is handled by
symbols-dummy.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Timothy Pearson <tpearson@raptorengineering.com>
make_memory_node() does not rely on any Arm-specific functionality and is
fully reused by RISC-V, making it a good candidate to move into common code.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Tested-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
The current implementation of make_chosen_node() does not contain any
architecture-specific logic. Therefore, move it from arch-specific
files to common code.
At this stage, there is no need to introduce an arch_make_chosen_node(),
as no architecture-specific customization is required.
This change avoids duplication and simplifies future maintenance for
architectures like RISC-V and ARM.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Tested-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
As noted in the command line documentation, these are both deprecated since
Xen 4.7 (2016), and are not fully effective on AMD CPUs starting from 2011.
Not realised at the time of writing the docs was that their use is also
incompatible with certain errata workarounds which edit the CPUID MSRs after
the levelling defaults are calculated.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Roger Pau Monné