When building the PDX test harness as a 32bit executable the page shifts
done on unsigned long types can overflow. Instead use pfn_to_paddr(),
which casts the values to paddr_t previous to doing the shift.
Fixes: cb50e40337 ("test/pdx: add PDX compression unit tests")
Reported-by: Edwin Török <edwin.torok@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Since 4dbcb06536, the vGICv2 CPU interface is mapped in a deferred
manner. On domains with XEN_DOMCTL_CDF_trap_unmapped_accesses unset on
GICv2 hardware, the vGICv2 CPU interface fails to be mapped. A visible
symptom is that a domU gets stuck at:
[ 0.177983] smp: Bringing up secondary CPUs ...
Move the 2nd check_p2m earlier so it's prioritized over try_handle_mmio.
Fixes: 980aff4e8f ("xen/arm: Add way to disable traps on accesses to unmapped addresses")
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
struct cpuinfo_x86
.x86 => .family
.x86_vendor => .vendor
.x86_model => .model
.x86_mask => .stepping
No functional change.
This work is part of making Xen safe for Intel family 18/19.
Signed-off-by: Kevin Lampis <kevin.lampis@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
This NR_CPUS-dimensioned array is likely unused on most installations.
Therefore it is especially wasteful for it to consume more space than
really needed. Allocate it dynamically ahead of registering the hooks.
Further the array having all fields set to -1 is actually useless. Nothing
relies on it, and core_parking_remove() doesn't restore the sentinel for
vacated slots. Drop the initializers altogether, rather than replacing
them.
Also take the opportunity and update an adjacent variable's type, where
a fixed-width type was pretty clearly inappropriate to use.
Finally drop the redundant initializer from core_parking_init().
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Under FRED, entry_from_pv() handles everything, even system call instructions.
This means more of our logic is written in C now, rather than assembly.
In order to facilitate this, introduce pv_inject_callback(), which reuses
struct trap_bounce infrastructure to inject the syscall/sysenter callbacks.
This in turns requires some !PV compatibility for pv_inject_callback() and
pv_hypercall() which can both be ASSERT_UNREACHABLE().
For each of INT $N, SYSCALL and SYSENTER, FRED gives us interrupted context
which was previously lost. As the guest can't see FRED, Xen has to lose state
in the same way to maintain the prior behaviour.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
ERETU can fault for guest reasons, and like IRET needs special handling to
forward the error into the guest.
As this is largely written in C, take the opportunity to better classify the
sources of error, and in particular, not forward errors that are actually
Xen's fault into the guest, opting for a domain crash instead.
Because ERETU does not enable NMIs if it faults, a corner case exists if an
NMI was taken while in guest context, and the ERETU back out faults. Recovery
must involve an ERETS with the interrupted context's NMI flag.
See the comments for full details.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Under FRED, entry_from_pv() handles everything. To start with, implement
exception handling in the same manner as entry_from_xen(), although we can
unconditionally enable interrupts after the async/fatal events.
After entry_from_pv() returns, test_all_events() needs to run to perform
exception and interrupt injection. Split entry_FRED_R3() into two and
introduce eretu_exit_to_guest() as the latter half, coming unilaterally from
restore_all_guest().
For all of this, there is a slightly complicated relationship with CONFIG_PV.
entry_FRED_R3() must exist irrespective of CONFIG_PV, because it's the
entrypoint registered with hardware. For simplicity, entry_from_pv() is
always called, but it collapses into fatal_trap() in the !PV case.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When FRED is active, hardware automatically swaps GS when changing privilege,
and the SWAPGS instruction is disallowed.
For native OSes using GS as the thread local pointer this is a massive
improvement on the pre-FRED architecture, but under Xen it makes handling PV
guests more complicated. Specifically, it means that GS_BASE and GS_SHADOW
are the opposite way around in FRED mode, as opposed to IDT mode.
This leads to the following changes:
* In load_segments(), we already load both GSes. Account for FRED in the
SWAP() condition and avoid the path with SWAGS.
* In save_segments(), we need to read GS_SHADOW rather than GS_BASE.
* In toggle_guest_mode(), we need to emulate SWAPGS.
* In {read,write}_msr() which access the live registers, GS_SHADOW and
GS_BASE need swapping.
* In do_set_segment_base(), merge the SEGBASE_GS_{USER,KERNEL} cases and
take FRED into account when choosing which base to update.
SEGBASE_GS_USER_SEL was already an LKGS invocation (decades before FRED)
so under FRED needs to be just a MOV %gs. Simply skip the SWAPGSes.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
clang would duplicate the loop body and end up with a double definition
of the symbol:
```
/tmp/test_x86_emulator-0f3576.s:27823: Error: symbol `vmovsh_to_mem' is already defined
/tmp/test_x86_emulator-0f3576.s:27825: Error: symbol `.Lvmovsh_to_mem_end' is already defined
```
Until a better solution is found: reduce optimizations in the test runner.
Using -Os might also work, but we can't rely on the size optimization
always avoiding the duplication of asm blocks.
This is test code, not performance critical code, and -O0 is more future
proof.
However for debugging -Og is recommended over -O0, and this still avoids
the duplicate label problem.
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The scheduler may call this function to force synchronization of given
vCPU's state. RISC-V does not support lazy context switching, so nothing
is done in sync_vcpu_execstate() and sync_local_execstate().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
As the tasklet subsystem is now initialized, it is necessary to implement
sync_local_execstate(), since it is invoked when something calls
tasklet_softirq_action(), which is registered in tasklet_subsys_init().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce timer_interrupt() to process IRQ_S_TIMER interrupts.
The handler disables further timer interrupts by clearing
SIE.STIE and raises TIMER_SOFTIRQ so the generic timer subsystem
can perform its processing.
Update do_trap() to dispatch IRQ_S_TIMER to this new handler.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement reprogram_timer() on RISC-V using the standard SBI timer call.
The privileged architecture only defines machine-mode timer interrupts
(using mtime/mtimecmp). Therefore, timer services for S/HS/VS mode must
be provided by M-mode via SBI calls. SSTC (Supervisor-mode Timer Control)
is optional and is not supported on the boards available to me, so the
only viable approach today is to program the timer through SBI.
reprogram_timer() enables/disables the supervisor timer interrupt and
programs the next timer deadline using sbi_set_timer(). If the SBI call
fails, the code panics, because sbi_set_timer() is expected to return
either 0 or -ENOSUPP (this has been stable from early OpenSBI versions to
the latest ones). The SBI spec does not define a standard negative error
code for this call, and without SSTC there is no alternative method to
program the timer, so the SBI timer call must be available.
reprogram_timer() currently returns int for compatibility with the
existing prototype. While it might be cleaner to return bool, keeping the
existing signature avoids premature changes in case sbi_set_timer() ever
needs to return other values (based on which we could try to avoid
panic-ing) in the future.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce a function pointer for sbi_set_timer(), since different OpenSBI
versions may implement the TIME extension with different extension IDs
and/or function IDs.
If the TIME extension is not available, fall back to the legacy timer
mechanism. This is useful when Xen runs as a guest under another Xen,
because the TIME extension is not currently virtualised and therefore
will not appear as available.
Despite of the fact that sbi_set_timer_v01 is introduced and used as
fall back, SBI v0.1 still isn't fully supported (with the current SBI
calls usage, sbi_rfence_v01 should be introduced too), so panic()
in sbi_init() isn't removed.
The sbi_set_timer() pointer will be used by reprogram_timer() to program
Xen’s physical timer as without SSTC extension there is no any other
option except SBI call to do that as only M-timer is available for us.
Use dprintk() for all the cases to print that a speicifc SBI extension
is available as it isn't really necessary in case of release builds.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Add handling of the SBI_EXT_0_1_SET_TIMER function ID to the legacy
extension ecall handler. The handler now programs the vCPU’s virtual
timer via vtimer_set_timer() and returns SBI_SUCCESS.
This enables guests using the legacy SBI timer interface to schedule
timer events correctly.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce vtimer_ctxt_switch_from() and vtimer_ctxt_switch_to() to handle
virtual timer state across vCPU context switches.
At present, vtimer_ctxt_switch_from() is a no-op because the RISC-V SSTC
extension, which provides a virtualization-aware timer, is not yet
supported. Xen therefore relies the virtual (SBI-based) timer.
The virtual timer uses Xen's internal timer infrastructure and must be
associated with the pCPU on which the vCPU is currently running so that
timer events can be delivered efficiently. As a result, vtimer_ctxt_switch_to()
migrates the timer to the target pCPU when a vCPU is scheduled in.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Add a RISC-V implementation of vcpu_kick(), which unblocks the target
vCPU and sends an event check IPI if the vCPU was running on another
processor. This mirrors the behavior of Arm and enables proper vCPU
wakeup handling on RISC-V.
Remove the stub implementation from stubs.c, as it is now provided by
arch/riscv/domain.c.
Since vcpu_kick() calls perfc_incr(vcpu_kick), add perfcounter for
vcpu_kick to handle the case when CONFIG_PERF_COUNTERS=y. Although
CONFIG_PERF_COUNTERS is not enabled by default, it can be enabled,
for example, by randconfig what will lead to CI build issues.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Lay the groundwork for guest timer support by introducing a per-vCPU
virtual timer backed by Xen’s common timer infrastructure.
The virtual timer is programmed in response to the guest SBI
sbi_set_timer() call and injects a virtual supervisor timer interrupt
into the vCPU when it expires.
While a dedicated struct vtimer is not strictly required at present,
it is expected to become necessary once SSTC support is introduced.
In particular, it will need to carry additional state such as whether
SSTC is enabled, the next compare value (e.g. for the VSTIMECMP CSR)
to be saved and restored across context switches, and time delta state
(e.g. HTIMEDELTA) required for use cases such as migration. Introducing
struct vtimer now avoids a later refactoring.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
This patch is based on Linux kernel 6.16.0.
Add the consumer side (vcpu_flush_interrupts()) of the lockless pending
interrupt tracking introduced in part 1 (for producers). According, to the
design only one consumer is possible, and it is vCPU itself.
vcpu_flush_interrupts() is expected to be ran (as guests aren't ran now due
to the lack of functionality) before the hypervisor returns control to the
guest.
Producers may set bits in irqs_pending_mask without a lock. Clearing bits in
irqs_pending_mask is performed only by the consumer via xchg() (with aquire
semantics). The consumer must not write to irqs_pending and must not act on
bits that are not set in the mask. Otherwise, extra synchronization should
be provided.
The worst thing which could happen with such approach is that a new pending
bit will be set to irqs_pending bitmap during update of hvip variable in
vcpu_flush_interrupt() but it isn't problem as the new pending bit won't
be lost and just be proceded during the next flush.
As AIA specs introduced hviph register which would want to be updated when
guest related AIA code vcpu_update_hvip() is introduced instead of just
open-code it in vcpu_flush_interrupts().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Based on Linux kernel v6.16.0.
Note that smp_wmb() is used instead of smp_mb__before_atomic() as what
we want to guarantee that if a bit in irqs_pending_mask is obversable
that the correspondent bit in irqs_pending is observable too.
Add lockless tracking of pending vCPU interrupts using atomic bitops.
Two bitmaps are introduced:
- irqs_pending — interrupts currently pending for the vCPU
- irqs_pending_mask — bits that have changed in irqs_pending
The design follows a multi-producer, single-consumer model, where the
consumer is the vCPU itself. Producers may set bits in
irqs_pending_mask without a lock. Clearing bits in irqs_pending_mask is
performed only by the consumer via xchg(). The consumer must not write
to irqs_pending and must not act on bits that are not set in the mask.
Otherwise, extra synchronization should be provided.
On RISC-V interrupts are not injected via guest registers, so pending
interrupts must be recorded in irqs_pending (using the new
vcpu_{un}set_interrupt() helpers) and flushed to the guest by updating
HVIP before returning control to the guest. The consumer side is
implemented in a follow-up patch.
A barrier between updating irqs_pending and setting the corresponding
mask bit in vcpu_set_interrupt()/vcpu_unset_interrupt() guarantees
that if the consumer observes a mask bit set, the corresponding pending
bit is also visible. This prevents missed interrupts during the flush.
It is possible that a guest could have pending bit in the hardware
register without being marked pending in irq_pending bitmap as:
According to the RISC-V ISA specification:
Bits hip.VSSIP and hie.VSSIE are the interrupt-pending and
interrupt-enable bits for VS-level software interrupts. VSSIP in hip
is an alias (writable) of the same bit in hvip.
Additionally:
When bit 2 of hideleg is zero, vsip.SSIP and vsie.SSIE are read-only
zeros. Else, vsip.SSIP and vsie.SSIE are aliases of hip.VSSIP and
hie.VSSIE.
This means the guest may modify vsip.SSIP, which implicitly updates
hip.VSSIP and the bit being written with 1 would also trigger an interrupt
as according to the RISC-V spec:
These conditions for an interrupt trap to occur must be evaluated in a
bounded amount of time from when an interrupt becomes, or ceases to be,
pending in sip, and must also be evaluated immediately following the
execution of an SRET instruction or an explicit write to a CSR on which
these interrupt trap conditions expressly depend (including sip, sie and
sstatus).
What means that IRQ_VS_SOFT must be synchronized separately, what is done
in vcpu_sync_interrupts(). Note, also, that IRQ_PMU_OVF would want to be
synced for the similar reason as IRQ_VS_SOFT, but isn't sync-ed now as
PMU isn't supported now.
For the remaining VS-level interrupt types (IRQ_VS_TIMER and
IRQ_VS_EXT), the specification states they cannot be modified by the guest
and are read-only because of:
Bits hip.VSEIP and hie.VSEIE are the interrupt-pending and interrupt-enable
bits for VS-level external interrupts. VSEIP is read-only in hip, and is
the logical-OR of these interrupt sources:
• bit VSEIP of hvip;
• the bit of hgeip selected by hstatus.VGEIN; and
• any other platform-specific external interrupt signal directed to
VS-level.
Bits hip.VSTIP and hie.VSTIE are the interrupt-pending and interrupt-enable
bits for VS-level timer interrupts. VSTIP is read-only in hip, and is the
logical-OR of hvip.VSTIP and any other platform-specific timer interrupt
signal directed to VS-level.
and
When bit 10 of hideleg is zero, vsip.SEIP and vsie.SEIE are read-only zeros.
Else, vsip.SEIP and vsie.SEIE are aliases of hip.VSEIP and hie.VSEIE.
When bit 6 of hideleg is zero, vsip.STIP and vsie.STIE are read-only zeros.
Else, vsip.STIP and vsie.STIE are aliases of hip.VSTIP and hie.VSTIE.
and also,
Bits sip.SEIP and sie.SEIE are the interrupt-pending and interrupt-enable
bits for supervisor-level external interrupts. If implemented, SEIP is
read-only in sip, and is set and cleared by the execution environment,
typically through a platform-specific interrupt controller.
Bits sip.STIP and sie.STIE are the interrupt-pending and interrupt-enable
bits for supervisor-level timer interrupts. If implemented, STIP is
read-only in sip, and is set and cleared by the execution environment
Thus, for these interrupt types, it is sufficient to use vcpu_set_interrupt()
and vcpu_unset_interrupt(), and flush them during the call of
vcpu_flush_interrupts() (which is introduced in follow up patch).
vcpu_sync_interrupts(), which is called just before entering the VM,
slightly bends the rule that the irqs_pending bit must be written
first, followed by updating the corresponding bit in irqs_pending_mask.
However, it still respects the core guarantee that the producer never
clears the mask and only writes to irqs_pending if it is the one that
flipped the corresponding mask bit from 0 to 1.
Moreover, since the consumer won't run concurrently because
vcpu_sync_interrupts() and the consumer path are going to be invoked
sequentially immediately before VM entry, it is safe to slightly relax
this ordering rule in vcpu_sync_interrupts().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce vcpu_csr_init() to initialise hypervisor CSRs that control
vCPU execution and virtualization behaviour before the vCPU is first
scheduled.
The function configures trap and interrupt delegation to VS-mode by
setting the appropriate bits in the hedeleg and hideleg registers,
initializes hstatus so that execution enters VS-mode when control is
passed to the guest, and restricts guest access to hardware performance
counters by initializing hcounteren, as unrestricted access would
require additional handling in Xen.
When the Smstateen and SSAIA extensions are available, access to AIA
CSRs and IMSIC guest interrupt files is enabled by setting the
corresponding bits in hstateen0, avoiding unnecessary traps into Xen
(note that SVSLCT(Supervisor Virtual Select) name is used intead of
CSRIND as OpenSBI uses such name and riscv_encoding.h is mostly based
on it).
If the Svpbmt extension is supported, the PBMTE bit is set in
henvcfg to allow its use for VS-stage address translation. Guest
access to the ENVCFG CSR is also enabled by setting ENVCFG bit in
hstateen0, as a guest may need to control certain characteristics of
the U-mode (VU-mode when V=1) execution environment.
For CSRs that may contain read-only bits (e.g. hedeleg, hideleg,
hstateen0), to the written value a correspondent mask is applied to
avoid divergence between the software state and the actual CSR
contents.
As hstatus is not part of struct arch_vcpu (it already resides in
struct cpu_user_regs), introduce vcpu_guest_cpu_user_regs() to provide
a uniform way to access hstatus and other guest CPU user registers.
This establishes a consistent and well-defined initial CSR state for
vCPUs prior to their first context switch.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Some hypervisor CSRs expose optional functionality and may not implement
all architectural bits. Writing unsupported bits can either be ignored
or raise an exception depending on the platform.
Detect the set of writable bits for selected hypervisor CSRs at boot and
store the resulting masks for later use. This allows safely programming
these CSRs during vCPU context switching and avoids relying on hardcoded
architectural assumptions.
Use csr_read()&csr_write() instead of csr_swap()+all ones mask as some
CSR registers have WPRI fields which should be preserved during write
operation.
Also, ro_one struct is introduced to cover the cases when a bit in CSR
register (at the momemnt, it is only hstateen0) may be r/o-one to have
hypervisor view of register seen by guest correct.
Masks are calculated at the moment only for hedeleg, henvcfg, hideleg,
hstateen0 registers as only them are going to be used in the follow up
patch.
If the Smstateen extension is not implemented, hstateen0 cannot be read
because the register is considered non-existent. Instructions that attempt
to access a CSR that is not implemented or not visible in the current mode
are reserved and will raise an illegal-instruction exception.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
When Dom0 informs us about MMCFG usability, this may change whether
extended capabilities are available (accessible) for devices. Zap what
might be on record, and re-initialize things.
No synchronization is added for the case where devices may already be in
use. That'll need sorting when (a) DomU support was added and (b) DomU-s
may run already while Dom0 / hwdom still boots (dom0less, Hyperlaunch).
vpci_cleanup_capabilities() also shouldn't have used
pci_find_ext_capability(), as already when the function was introduced
extended config space may not have been (properly) accessible anymore,
no matter whether it was during init. Extended capability cleanup hooks
need to cope with being called when the respective capability doesn't
exist (and hence the corresponding ->init() hook was never called).
Fixes: 70e6dace74 ("vpci: Use cleanup to free capability resource during deassign")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
The label used upon the function failing is wrong. Instead of correcting
the label, move the invocation up a little, to also avoid it altogether
for the idle domain (where ->vmtrace_size would be zero, and hence the
function would bail right away anyway).
Fixes: 217dd79ee2 ("xen/domain: Add vmtrace_size domain creation parameter")
Reported-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Using plain (signed) int variables as array indexes can be unhelpful on at
least x86, where the compiler may see the need to insert sign-extension
insns (strictly speaking it should be able to avoid that when the loop
continuation condition says >= 0, but that's not generally the case even
with gcc15).
Observed effects with gcc15 (will of course vary with compiler version and
level of optimization):
- on x86, one less preserved register in use, yet due to sub-optimal
choice of register variables still a small code size increase (%r12
isn't a good choice when it's used for base-without-index addressing, as
it requires a SIB byte which other registers wouldn't require),
- on Arm64 code size decreases, albeit that's eaten up by padding which is
being inserted ahead of a few labels,
- on Arm32 code size increases for a reason I didn't fully understand (my
ability to read Arm assembly is still somewhat limited).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
A hotplugged PCI device may be added uninitialized. In particular,
memory decoding might be disabled and the BARs might be zeroed. In this
case, the BARs will not be mapped in p2m. However, vpci_make_msix_hole()
unconditionally attempts to punch holes in p2m, leading to init_msix()
failing:
(XEN) d0v0 0000:01:00.0: existing mapping (mfn: 1c1880 type: 0) at 0 clobbers MSIX MMIO area
(XEN) d0 0000:01:00.0: init legacy cap 17 fail rc=-17, mask it
vpci_make_msix_hole() should only attempt to punch holes if the BARs
containing the MSI-X/PBA tables are mapped in p2m. Introduce a helper
for checking if a BAR is enabled, and add a check for the situation
inside vpci_make_msix_hole().
As a result of the newly introduced checks in vpci_make_msix_hole(),
move the call to vpci_make_msix_hole() within modify_decoding() to after
setting ->enabled.
Fixes: ee2eb6849d ("vpci: Refactor REGISTER_VPCI_INIT")
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
We cannot assume extended config space to (still) be accessible when
cleaning up. Necessary values need caching instead. In fact, as the caller
also cannot look up extended capabilities, the cleanup function needs to
cope with being called when there's no ReBAR capability at all.
As kind of a side effect nbars being 0 (which init_rebar() doesn't
explicitly reject) no longer results in a bogus call to
vpci_remove_registers().
Fixes: ee459aeac0 ("vpci/rebar: Implement cleanup function for Rebar")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Fixes this `-fsanitize=undefined` error:
```
test_x86_emulator.c:614:12: runtime error: null pointer passed as argument 1, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior test_x86_emulator.c:614:12
```
Although this is more of a grey area: I don't see anything in the
standard that'd forbid calling `memcpy` with NULL and 0, but `glibc`
does specify it as non-null, which allows the compiler to make
optimizations assuming it never is NULL, so this is undefined behaviour
only on glibc.
Best to avoid the potential undefined behaviour though.
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Fixes this `-fsanitize=undefined` error:
```
test_x86_emulator.c:1103:32: runtime error: left shift of 573785183 by 2 places cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior test_x86_emulator.c:1103:32
```
Signed-off-by: Edwin Török <edwin.torok@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
ticks_to_ns() and ns_to_ticks() are not architecture-specific, so provide a
common implementation that is more resilient to overflow than the historical
Arm version. This is not a practical issue for Arm, as the latest ARM ARM
that timer frequency should be fixed at 1 GHz and older platforms used much
lower rates, which is shy of 32-bit overflow. As the helpers are declared
as static inline, they should not affect x86, which does not use them.
On Arm, these helpers were historically implemented as out-of-line functions
because the counter frequency was originally defined as static and unavailable
to headers [1]. Later changes [2] removed this restriction, but the helpers
remained unchanged. Now they can be implemented as static inline without any
issues.
Centralising the helpers avoids duplication and removes subtle differences
between architectures while keeping the implementation simple.
Drop redundant <asm/time.h> includes where <xen/time.h> already pulls it in.
No functional change is intended.
[1] ddee56dc29 arm: driver for the generic timer for ARMv7
[2] 096578b4e4 xen: move XEN_SYSCTL_physinfo, XEN_SYSCTL_numainfo and
XEN_SYSCTL_topologyinfo to common code
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Bertrand Marquis <bertrand.marquis@arm.com>
... both for when the functions are invoked and where they live in source.
Don't invoke them directly in vpci_init_header(), but instead first thing
in vpci_init_capabilities().
Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Let's keep capability handling together. Start with moving
vpci_init_capabilities() and its helpers, plus splitting out of its
cleanup counterpart.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Before adding more private stuff to xen/vpci.h, split it up. In order to
be able to include the private header first in a CU, the per-arch struct
decls also need to move (to new asm/vpci.h files).
While adjusting the test harness'es Makefile, also switch the pre-existing
header symlink-ing rule to a pattern one.
Apart from in the test harness code, things only move; no functional
change intended.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
With the shadow stack and exception handling adjustements in place, we can now
activate FRED when appropriate. Note that opt_fred is still disabled by
default until more development is complete.
Introduce init_fred() to set up all the MSRs relevant for FRED. FRED uses
MSR_STAR (entries from Ring3 only), and MSR_FRED_SSP_SL0 aliases MSR_PL0_SSP
when CET-SS is active. Otherwise, they're all new MSRs.
Also introduce init_fred_tss(). At this juncture we need a TSS set up, even
if it is mostly unused. Reinsert the BUILD_BUG_ON() checking the size of the
TSS against 0x67, this time with a more precise comment.
With init_fred() existing, load_system_tables() and legacy_syscall_init()
should only be used when setting up IDT delivery. Insert ASSERT()s to this
effect, and adjust the various init functions to make this property true.
The FRED initialisation path still needs to write to all system table
registers at least once, even if only to invalidate them. Per the
documentation, percpu_early_traps_init() is responsible for switching off the
boot GDT, which also needs doing even in FRED mode.
Finally, set CR4.FRED in traps_init()/percpu_early_traps_init().
Xen can now boot in FRED mode and run a PVH dom0. PV guests still need more
work before they can be run under FRED.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Under FRED, there's one entrypoint from Ring 3, and one from Ring 0.
FRED gives us a good stack (even for SYSCALL/SYSENTER), and a unified event
frame on the stack with all flags cleared, meaning that all software needs to
do is spill the GPRs with a line of PUSHes. Introduce PUSH_AND_CLEAR_GPRS for
this purpose, along with the matching POP_GPRS.
Introduce entry_FRED_R0() which to a first approximation is complete for all
event handling within Xen.
entry_FRED_R0() needs deriving from entry_FRED_R3(), so introduce a basic
Ring3 handler. There is more work required to make the return-to-guest path
work under FRED.
Also introduce entry_from_{xen,pv}() to be the C level handlers. By simply
copying regs->fred_ss.vector into regs->entry_vector, we can reuse all the
existing fault handlers.
Extend fatal_trap() to render the event type including name, when FRED is
active. This is slightly complicated, because X86_ET_OTHER must not use
vector_name() or SYSCALL and SYSENTER get rendered as #BP and #DB.
This is sufficient to handle all interrupts and exceptions encountered during
development, including plenty of Double Faults.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
FRED doesn't use Supervisor Shadow Stack tokens. This means that:
1) memguard_guard_stack() should not write Supervisor Shadow Stack Tokens.
2) cpu_has_bug_shstk_fracture is no longer relevant when deciding whether to
enable Shadow Stacks or not.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
We wish to make use of opt_fred earlier on boot, which involves moving
traps_init() earlier, but this comes with several ordering complications.
The feature word containing FRED needs collecting in early_cpu_init(), and
legacy_syscall_init() cannot be moved that early because it relies on the
stubs being allocated, yet must be called ahead of cpu_init() so the SYSCALL
linkage MSRs are set up before being cached.
Delaying legacy_syscall_init() is easy enough based on a system_state check.
Reuse bsp_traps_reinit() to cause a call to legacy_syscall_init() to occur at
the same point as previously.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Lets just say this took an unreasonable amount of time and effort to track
down, when trying to move traps_init() earlier during boot.
When the SYSCALL linkage MSRs are not configured ahead of _svm_cpu_up() on the
BSP, context switch clobbers the later-set-up linkage with the 0's cached
here. Amongst other problems, this causes PV guest to enter at 0 in
supervisor mode on the user stack.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Commit 564d261687 ("x86/ctxt-switch: Document and improve GDT handling") put
the initialisation of {,compat_}gdt_l1e into traps_init() but this wasn't a
great choice. Instead, put it in smp_prepare_cpus() which performs the BSP
preparation of variables normally set up by cpu_smpboot_alloc() for APs.
This removes an implicit dependency that prevents traps_init() moving earlier
than move_xen() in the boot sequence.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
... seeing as I've had to thoroughly reverse engineer it for FRED and make
tweaks in places.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
For INT $N instructions (besides $0x80 for which there is a dedicated fast
path), handling is mostly fault-based because of DPL0 gates in the IDT. This
means that when the guest kernel allows the instruction too, Xen must
increment %rip to the end of the instruction before passing a trap to the
guest kernel.
When an INT $N instruction has a prefix, it's longer than two bytes, and Xen
will deliver the "trap" with %rip pointing into the middle of the instruction.
Introduce a new pv_emulate_sw_interrupt() which uses x86_insn_length() to
determine the instruction length, rather than assuming two.
This is a change in behaviour for PV guests, but the prior behaviour cannot
reasonably be said to be intentional.
This change does not affect the INT $0x80 fastpath. Prefixed INT $N
instructions occur almost exclusively in test code or exploits, and INT $0x80
appears to be the only user-usable interrupt gate in contemporary PV guests.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Multiple downstream distros have tried passing discrete CPIO archives and
tripped over this not working. It turns out to be easy to support, so do so.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Teddy Astie <teddy.astie@vates.tech>
The first parameter passed to align_timer() is the timer expiration, not
the current time. Adjust the calls to align_timer() in the on-demand
governor to pass the expected timer expiration as the first parameter.
Fixes: af74e3a15a ("cpufreq: align dbs timer for better package C state residency")
Fixes: 382b95f627 ("Fix cpufreq HW-ALL coordination handle")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The logic in dbs_check_cpu() resets the sampling window even when the
sampling period is considered too small. This leads to further calls
finding an imbalance between the total window time and the idle time, as
the total window time is possibly shorter than the idle time.
Fix by resetting the sampling window start time in the same block where the
current idle time is stored. While there also prevent a duplicated call to
NOW() and instead re-use the previously fetched value.
Fixes: d6f001cb91 ("x86: Implement cpufreq ondemand policy")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
After e4c3755d4d the function efi_halt_system() is unreachable, remove it
from the file.
No functional change expected, as the function is not called.
Fixes: e4c3755d4d ("x86-64/EFI: don't call EfiResetSystem() from machine_halt()")
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Package "bridge-utils" have been removed from the official repos, it
contained `brctl`. That utility is used by network hotplug scripts,
but they already check if it is available and switch to use "iproute2"
(`ip`) when not.
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
If we start a guest with 'cpus="all"' and without 'vcpus' or
'maxvcpus' (or sets them to 0), we execute parse_vcpu_affinity() with
`num_cpus=0`. This malloc "b_info->vcpu_hard_affinity" with a buffer
of size 0, which is implementation defined, and we still initialise
the "first" bitmap of this allocation, which mean we have a buffer
overflow.
On Alpine Linux, this result in a segv when the buffer is being
disposed of.
Since libxl will assume there's at least 1 vcpu, we default to 1 in
xl as well. (libxl sets max_vcpus to 1 if unset, and allocate
avail_vcpus if its size is 0.)
Link: https://gitlab.alpinelinux.org/alpine/aports/-/issues/17809
Fixes: a5dbdcf674 ("libxl/xl: push VCPU affinity pinning down to libxl")
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
As per the standard this is UB, i.e. we're building on an extension in
the compilers we use (for gcc, see "The C Preprocessor" section
"Directives Within Macro Arguments"). Misra C:2012 rule 20.6 disallows
this altogether, though.
Combine the fix with some tidying: Fold the two level-1 checks, adjust in
particular comment style, and move the unlikely() use to a single inner
part of the conditional.
No functional change intended.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Roger Pau Monne