do_trap() consulted the exception table only for CAUSE_ILLEGAL_INSTRUCTION,
which covers csr_read_safe() but not a probing access faulting with a page
fault cause: it would reach do_unexpected_trap() instead of its fixup.
There is no such access yet, the A/D update scheme probe will be the first.
Move the lookup ahead of the cause switch, and only do it for exceptions
taken in Xen context. Both other cases must skip it:
- Traps taken from the guest: sepc is then a guest address, which the
guest can point at an address listed in the exception table, making Xen
act on an entry fully under guest control. Entries are matched by exact
address, so this needs no more than a numerical collision.
- Interrupts: an interrupt taken at an address listed in the table would
otherwise be "fixed up" as if the access itself had faulted, silently
skipping it.
Returning early skips check_for_pcpu_work(), which is correct: that only
runs for traps taken from the guest.
Cache the "trap came from the guest" test in a local as it is now used
three times.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
[Baptiste: drop the parts depending on the vAPLIC/vIMSIC series, adapt
the commit message]
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The device tree Xen builds for dom0/hwdom or for a dom0less domain is
only ever consumed by the guest, so a mistake in it surfaces as an
obscure failure inside the domain, far away from the code which
produced the tree.
Add device_tree_dump(). The output follows dtc's format so that a dump
can be diffed against the source DTS or fed back to dtc; the value
formatting heuristics are ported from dtc's utilfdt_print_data() for
that reason. The walk itself goes through libfdt rather than decoding
the tag stream by hand as dtc's fdtdump does.
Dumping a whole tree is only useful while debugging and costs both code
size and a lot of console output, so it is put under its own option,
CONFIG_DEVICE_TREE_DUMP. It is separate from CONFIG_DEVICE_TREE_DEBUG
because the generated tree is often all that is needed, without the rest
of the device tree debug messages.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Everything discard_initial_modules() touches is already common code:
bootinfo.modules, using_static_heap and fw_unreserved_regions(). Move it
out of Arm into common/device-tree/bootinfo.c, next to the boot allocator
handling of the same memory, and declare it in xen/bootinfo.h, so that
other device-tree based architectures can reuse it instead of growing a
copy of their own.
The arch-specific remove_early_mappings() is not moved along with it:
how the early FDT mapping is torn down is up to the architecture. As the
DTB is one of the boot modules being released, the mapping has to go once
they are discarded, so call it from start_xen() right after
discard_initial_modules(), which keeps the common function free of any
arch dependency.
While moving it, use unsigned int for the loop counter and replace the
"goto out" by a plain return, as there is nothing left to do at the label.
No functional change.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
vstimecmp is a 64-bit CSR independently of XLEN, which is why it is
written with csr_write64(). On RV32 that macro splits the value into
the vstimecmp/vstimecmph pair, so passing ULONG_MAX (0xffffffff there)
writes all ones to the low half and zero to the high half, leaving the
CSR at 0x00000000ffffffff rather than at its maximum. A VS-timer irq
would then become pending as soon as (time + htimedelta) reaches 2^32,
which is exactly what the code is trying to avoid.
Use ~0 instead: csr_write64() converts its argument to uint64_t, and the
conversion of the signed -1 sign-extends to all ones, so both halves get
0xffffffff. On RV64 the result is equal to ULONG_MAX, so no functional
change there.
Fixes: 25e0327306 ("xen/riscv: allow Xen to use SSTC while hiding it from guests")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
vmid_handle_vmenter() reports that no flush is needed when VMIDs are
unavailable (e.g. vmid=off, or hardware with no more than one VMID bit).
Every domain then runs under VMID 0 with nothing flushed in between, so as
soon as a hart runs more than one domain, a domain entered there can use
the G-stage translations left behind by the domain which ran before it.
Report that a flush is needed on every entry instead.
Update the comment in p2m_handle_vmenter() accordingly: without VMIDs the
VS-stage TLB doesn't need a separate flush either, as the G-stage flush now
done on every entry already covers it.
While at it, spell the other early return as a bool literal.
Fixes: bff3b9ea46 ("xen/riscv: introduce VMID allocation and manegement")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
kernel_image64_probe() never set info->type, so every domain kept the
DOMAIN_32BIT value a zeroed KERNEL_INFO_INIT leaves behind, even though
Xen on RISC-V is only built for RV64 and accepts nothing but a 64-bit
Image.
Nothing consumes d->type on RISC-V yet, so there is no functional change.
Fixes: f6a20eda92 ("xen/riscv: add Linux kernel loading support")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
Both constants were inherited from OpenSBI and their names are misleading.
The length of an instruction is encoded in its low bits: bits[1:0] != 11
means 16 bits, otherwise bits[4:2] != 111 means 32 bits, and anything else
is 48 bits or wider. 0x3 and 0x1c are the patterns compared against in
these two tests, not masks selecting a 16-bit or a 32-bit instruction. As
each constant is needed in one place only, once INSN_IS_16BIT() and
INSN_IS_32BIT() encapsulate the tests, drop them and spell the values out
in those two macros instead.
While at it, make INSN_LEN() cover the case both macros already knew about:
an encoding of 48 bits or wider now yields 0 rather than being silently
taken for a 32-bit instruction. No ratified extension defines an instruction
of such a length, so rather than open-coding a decoder for encodings which
cannot legitimately occur, leave it to the caller to treat the 0 as an
illegal instruction.
No functional change: the only present caller applies INSN_LEN() to a bug
frame, i.e. to an instruction emitted by Xen itself, which is either 16 or
32 bits wide.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
Enable dom0less support for RISC-V by selecting HAS_DOM0LESS and
providing the minimal architecture hooks required by the common
dom0less infrastructure.
Add stub implementations for architecture-specific helpers used when
building domains from the device tree. These allow the generic
dom0less code to build and let a basic DomU be constructed on RISC-V.
construct_hwdom() and make_hypervisor_node() are still stubs returning
an error: Dom0/hwdom construction isn't supported yet, and the
hypervisor node generation (needed by domains with
DOM0LESS_ENHANCED_NO_XS set) is not implemented. Both are marked with
a TODO and are not reached by the currently supported configurations.
Provide missing helpers and definitions required by the domain
construction code, including domain bitness helpers and the
p2m_set_allocation() prototype.
Additionally define the guest magic memory region (GUEST_MAGIC_BASE /
GUEST_MAGIC_SIZE) in asm/guest-layout.h. The base is arbitrary; the
only constraint is that the region must not overlap guest RAM or the
emulated device regions. It is placed in the unused gap below
GUEST_RAM0_BASE (0x80000000); the constraints are documented next to
the #define-s.
A separate region for grant tables will be introduced at the same time as
the introduction of the grant table for RISC-V.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Wire up the missing early-boot initialization steps in start_xen().
The scheduler must be initialized prior to do_initcalls() because
cpupool_create_pool() is called during initcalls; without it,
BUG_ON(IS_ERR(pool)) is triggered inside cpupool_create_pool().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement init_intc_phandle() to read phandle of interrupt controller
node and save it in kernel->phandle_intc for the future usage during
creation of guest interrupt controller node.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
dom0less device passthrough requires granting guest domains access to
device interrupts. Introduce map_device_irqs_to_domain() to enumerate
a DT node's interrupt properties, skipping those not owned by
the primary interrupt controller (as at the moment I haven't seen usages
of it), and map_irq_to_domain() to grant domain access and configure
Xen's interrupt descriptor accordingly. Sharing IRQ between domains is
rejected.
Both map_irq_to_domain() and map_device_irqs_to_domain() are marked
__overlay_init, mirroring Arm: without CONFIG_OVERLAY_DTB this expands to
__init, so the functions are init-only and need no XSM check; with
CONFIG_OVERLAY_DTB they become runtime-callable, but the only runtime
entry point is dt_overlay_domctl(), which performs the XSM checks at the
domctl layer. RISC-V does not wire up DT overlay yet, so today these are
strictly __init; if/when overlay support is added, the domctl-level XSM
gating must be added together with it, as on Arm.
route_irq_to_guest() and release_guest_irq() manage irq_desc ownership
for guest-assigned interrupts. Each assignment carries a small irq_guest
structure as irqaction::dev_id, recording the owning domain and virtual
IRQ number which is 1:1 mapped to physical IRQ number. A per-domain
vIRQ allocation bitmap (used_irqs in struct vintc), managed by
vintc_reserve_virq(), prevents the same vIRQ being claimed twice.
Host and guest interrupts may differ in some operations, EOI timing in
particular: a host IRQ is completed once Xen's handler runs, whereas a
passthrough IRQ must defer the physical completion until the guest issues
its own EOI, otherwise a still-asserted level line would immediately
retrigger and storm. Hence a separate guest hw_irq_controller instance,
aplic_guest_irq_type, rather than an alias of the host one.
Its callbacks are BUG_ON("unimplemented") for now. The only interrupt
controller configuration supported so far is APLIC in MSI mode together
with IMSIC, where guest interrupts are delivered by hardware straight to
the guest's interrupt file, bypassing do_IRQ() and thereby Xen entirely.
The _IRQ_GUEST branch in do_IRQ() is left as BUG() for the same reason.
Real callbacks, .end() in particular, become necessary once a platform
without direct IMSIC delivery has to be supported, where Xen traps the
interrupt and injects it into the guest itself.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
booting.txt listed only H, Zbb, Zihintpause and Svpbmt as required, while
required_extensions[] in cpufeature.c also checks for I, M, A, Zicsr,
Zifencei and, with CONFIG_RISCV_ISA_C=y, C. As the panic message printed
for a missing extension points to booting.txt, document the missing ones.
Also add a section pointing to riscv_isa_ext[] for the optional extensions
Xen recognises, and a comment above required_extensions[] to keep it in
sync with the document.
No functional change.
Suggested-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Signed-off-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
These definitions are required for correct decoding of APLIC MMIO
accesses and target configuration, and will be used by both the
physical and virtual APLIC implementations.
While adding them, rearrange the header in the style of x86's
asm/msr-index.h: a register's offset is immediately followed by the
definitions of that register's fields, with the blocks sorted by
offset. This makes the relation between a register and its fields
obvious from the layout alone, so no comment is needed to express it.
No functional change is intended by this patch; it only centralises
hardware definitions that were previously missing.
Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Guests require a Device Tree description of the interrupt controller
topology. Add support for creating an APLIC node when building the
guest DT.
Provide stub for imsic_make_dt_node() it will be introduced properly
in follow-up patch.
The value chosen for GUEST_APLIC_S_BASE is based on QEMU one.
DT-building functions are marked __init because domain creation happens at
boot time, before the init sections are freed. In a typical deployment
libxl creates the interrupt controller node in userspace and hands the
complete FDT to Xen, so these functions are only called during early
domain construction.
Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Guests using the IMSIC interrupt controller require a corresponding
Device Tree description.
Add support for generating an IMSIC node when building the guest DT.
This allows guests to discover and use the IMSIC interrupt controller.
The value choosen for GUEST_IMSIC_S_BASE is an address which is typically
used for IMSIC and QEMU.
DT-building functions are marked __init because domain creation happens at
boot time, before the init sections are freed. In a typical deployment
libxl creates the interrupt controller node in userspace and hands the
complete FDT to Xen, so these functions are only called during early
domain construction.
Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Add common helpers domain_vintc_init() and domain_vintc_deinit() to
allocate and deallocate a virtual interrupt controller (vINTC)
structure and initialize basic virtual interrupt controller registers.
domain_vintc_deinit() isn't called at the moment as arch_domain_destroy()
is implemented as stub at the moment.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
At the current development stage, only domain vINTC init and deinit
operations are required, so implement those first.
Initialize vAPLIC's domaincfg to with the interrupt-enable bit set and
MSI delivery mode selected as the current solution is exepcted to have
always IMSIC, and initialize vintc->ops.
Other operations such as emulate_load(), emulate_store(), and is_access()
will be needed once guests are running and MMIO accesses to APLIC MMIO
range must be handled. These will be introduced separately later.
Introduce a structure to describe a virtual interrupt controller (vINTC)
and a vintc_ops structure, which provides operations to emulate load and
store accesses to interrupt controller MMIOs and to check whether a given
address falls within the MMIO range of a specific virtual interrupt
controller.
Note that already existed init_ops field in struct vintc will be init-ed
for APLIC in the follow up patch.
The vAPLIC implementation of these operations will be provided later
once guests can be run and these operations are actually needed.
Introduce these structures here as they are required for the implementation
of domain_vaplic_init() and domain_vaplic_alloc(). Also, introduce
vaplic_init() and init vintc_ops->vcpu_init() with it.
Co-developed-by: Romain Caritey <Romain.Caritey@microchip.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Each vCPU interacting with the IMSIC requires state to track the
associated guest interrupt file and its backing context.
Introduce a per-vCPU structure to hold IMSIC-related state, including
the guest interrupt file identifier and the CPU providing the backing
VS-file. Access to the guest file identifier is protected by a lock.
Initialize this structure during vCPU setup and store it in arch_vcpu.
The initial state marks the VS-file as software-backed until it becomes
associated with a physical CPU.
Add helper to retrieve the guest interrupt file identifier:
- vcpu_guest_file_id() is going to be used during update of APLIC's
target register with the pair of information <guest_file_id, cpu_id>
(to have MSI delivery mode work properly) when guest is trying to
access vAPLIC's target register.
It will be used in the follow up patches.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
aia_init() is going to contain all the logic related to AIA initialization.
At the moment, it only checks whether the SSAIA extension is available,
and if so, sets is_aia_usable (which indicates more than just the
availability of the extension) to true; it also signifies that the necessary
components (to be introduced in follow-up patches) have been initialized.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce a RISC-V specific function to create an interrupt controller
Device Tree node for DomU domains during dom0less build.
Add make_intc_domU_node() to the dom0less build path and wire it to
a new generic helper, intc_make_domu_dt_node(), which delegates DT
node creation to the active interrupt controller implementation via
vintc_init_ops.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce intc_hw_init_ops structure to avoid risky mix of init
function and non-init function.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
No RISC-V-specific nodes need to be created at the moment,
so make_arch_nodes() is implemented to simply return 0.
It is placed in dom0less-build.c as make_arch_nodes() is
only used in the dom0less code path. In the future, it will
be extended to create an emulated UART node.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Generally, in DT for RISC-V there is a document which describes a timer
node (riscv,timer.yaml or sifive,clint.yaml), but the Linux timer driver
is declared with TIMER_OF_DECLARE(riscv_timer, "riscv", ...).
It matches the CPU node (compatible "riscv"), not the timer node itself.
It then calls of_find_compatible_node(NULL, NULL, "riscv,timer") only to
read the optional riscv,timer-cannot-wake-cpu property.
Since Xen does not care about that property for now, make_timer_node() is
implemented to return 0, as no timer node needs to be created for RISC-V
guests.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement make_cpus_node() to create cpus node for a guest domain.
This function is going to be use by common dom0less code during
construction domain.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce build_guest_isa_str() to generate the riscv,isa string to be
passed to the guest via the Device Tree riscv,isa property.
Introduce the per-domain guest ISA bitmap, populated during domain
creation by calling init_guest_isa().
Introduce struct riscv_isa_ext_entry with a new guest_flags field
to filter out ISA extensions that should not be exposed to guests:
- f/d/q/v: FPU and vector context save/restore are not yet implemented
for guests.
- Z*inx are not exposed either: they aren't in riscv_isa_ext[], so they
can never be set in riscv_isa and thus never reach a guest, and no
current hardware/guest-OS advertises or expects them. Supporting them
would be cheaper than F/D/Q (FP values stay in integer registers Xen
already context-switches), but is left as future work.
- h: Nested virtualisation is not supported.
- sstc: Xen owns the supervisor timer; guests must use SBI.
- svade: Xen manages hardware A/D bit updates in stage-2 page tables.
- svpbmt: Page-based memory types are not yet wired up in stage-2 code.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement construct_domain() function for RISC-V, which performs initial setup
for the domain's first vCPU, loads the kernel, initrd, and device tree,
and sets up guest CPU registers for boot.
It also creates additional vCPUs up to max_vcpus and assigns the device tree
address and boot cpuid in registers.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
On architectures that run guests in dom0less mode without the PV ABI
(currently RISC-V), no shared_info page is allocated and d->shared_info
remains NULL throughout the domain lifetime. Several places in common
code access d->shared_info through the shared_info() macro or directly,
causing UBSAN null-pointer errors on such architectures.
Rather than adding runtime NULL guards that are logically unreachable
on x86 and Arm (where shared_info is always allocated), introduce a new
Kconfig symbol CONFIG_HAS_SHARED_INFO selected by x86 and Arm.
On !HAS_SHARED_INFO the shared_info() macro expands to a dereference
of shared_info_absent, an extern pointer that is declared but
intentionally never defined. Any use of shared_info() that is not
dead-code-eliminated will therefore cause a link-time failure, making
missed guards impossible to overlook.
The 2L event-channel ops call shared_info() and must not be compiled on
architectures without a shared_info page, so event_2l.o is gated on
CONFIG_HAS_SHARED_INFO. On such architectures evtchn_init() installs the
FIFO ops as a placeholder instead, so that a later guest opt-in to the
FIFO ABI via EVTCHNOP_init_control has no special-casing to do; if FIFO
support itself is also unavailable (!CONFIG_EVTCHN_FIFO), a dedicated
no-op evtchn_port_ops_none table is installed instead, so that
d->evtchn_port_ops is never NULL. evtchn_fifo_word_from_port() is
guarded against uninitialised d->evtchn_fifo so the FIFO ops are safe
before evtchn_fifo_init_control() is called by the guest.
With CONFIG_HAS_SHARED_INFO=n all vCPUs fall back to the global
dummy_vcpu_info, so writes through vcpu_info() could leak data between
vCPUs. Reviewing the write paths in common code: the write in
map_guest_area() stores the constant ~0 so nothing serious would happen
if it were leaked; the event_2l.c paths are not compiled on
!HAS_SHARED_INFO, as event_2l.o is gated on CONFIG_HAS_SHARED_INFO; the
write in vcpu_info_populate() targets the new mapping buffer, not
dummy_vcpu_info.
Outside common code, the remaining writes are x86 PV-specific, for which
CONFIG_HAS_SHARED_INFO=y. No code changes are needed.
Finally, struct domain's shared_info field itself is gated on
CONFIG_HAS_SHARED_INFO, as it would otherwise be a permanently NULL
pointer: every user of it is either arch code for an architecture that
selects HAS_SHARED_INFO, or common code already guarded by the same
Kconfig symbol.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Michal Orzel <michal.orzel@amd.com> #Arm
Until now every architecture carried its own notion of max_init_domid:
Arm defined a real variable (declared in asm/setup.h, defined in
setup.c), while ppc, riscv and x86 each provided a "#define
max_init_domid (0)" stub in their asm/setup.h. This duplicated the same
declaration across all arches and placed a purely dom0less concept in
arch setup headers.
Now that the dom0less build code lives in common (xen/common/
device-tree/dom0less-build.c sets max_init_domid, and the console
serial-input switcher reads it), there is no reason for the symbol to be
per-arch. Provide a single declaration in <xen/dom0less-build.h>, with
the !CONFIG_DOM0LESS_BOOT stub kept there as well, so there is one source
of truth and the arch headers no longer need to mention it. Update
console.c to include <xen/dom0less-build.h> for the declaration instead
of relying on asm/setup.h, and drop the now unneeded <asm/setup.h>
include.
Place the definition in xen/common/domid.c rather than in dom0less-
build.c. The latter is built as dom0less-build.init.o, i.e. the whole
object is relocated into the .init.* sections and freed after boot,
whereas max_init_domid must outlive boot because it is read at runtime
by the console serial-input switcher. domid.c is always linked (obj-y)
and resides in regular (non-init) sections, so it is a correct home for
the variable. It is marked __ro_after_init since it is only updated
while creating boot-time domains and read-only afterwards, and guarded
by CONFIG_DOM0LESS_BOOT as domid.c itself is unconditional.
While at it, make <xen/dom0less-build.h> self-contained: it includes
<public/xen.h>, which uses the fixed-width types provided by
<xen/types.h>, so include the latter explicitly rather than relying on
the includer having pulled it in first. This becomes necessary as soon
as <xen/dom0less-build.h> comes first in an alphabetically sorted
include list, as it now does in domid.c.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
imsic_init() indexes msi[] by the Xen CPU id hartid_to_cpuid() returns,
but that array is allocated with one entry per parent IRQ of the IMSIC
node, and the only range check compares the index against
num_possible_cpus(). Neither matches the array, and the check comes
after the first access:
- hartid_to_cpuid() returns NR_CPUS when the hart isn't one Xen brought
up, and msi[NR_CPUS].base_addr is read before that is noticed;
- an IMSIC node listing fewer parents than Xen has CPUs makes every
index past nr_parent_irqs go past the end of the allocation, which the
num_possible_cpus() check lets through.
Size the array by nr_cpu_ids, which is what it is indexed by,
and move the range check ahead of the first msi[] access.
Fixes: c9bd8b322e ("xen/riscv: imsic_init() implementation")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
asm/riscv_encoding.h already provides INSN_16BIT_MASK and INSN_LEN(), and
emulate.c uses them, so the tree carried two spellings of the same thing
which could drift apart. COMPRESSED_INSN_MASK never had a user.
No functional change.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
As nothing is using pregs in upstream or downstream ports of RISC-V drop
it from struct cpu_user_regs. Once it will really be needed re-introduce
it.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
The p2m_freelist is used to allocate pages for the P2M. To initialize
this list, domain_p2m_set_allocation() may be called from construct_domU()
in the common Dom0less code, so RISC-V provides an implementation and
enables CONFIG_ARCH_PAGING_MEMPOOL unconditionally.
Additionally, implement arch_{set,get}_paging_mempool_size(). They are
not directly used yet, but are required to support the
XEN_DOMCTL_{get,set}_paging_mempool_size hypercalls.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
p2m_set_allocation() uses a bool *preempted out-argument that overloads two
meanings. When non-NULL, the value written back (true) duplicates information
already carried by the -ERESTART return code — pure redundancy, which the
caller-side ASSERT(preempted == (rc == -ERESTART)) only documents. Separately,
a NULL pointer is an implicit calling convention meaning "preemption is not
permitted in this context".
Replace the pointer with a plain bool can_preempt that explicitly controls
whether the preemption check runs, making the NULL-to-suppress convention
type-safe and self-documenting, and rely on the -ERESTART return code alone to
report that preemption occurred.
Since p2m_set_allocation() is called by the common dom0less build code,
move its declaration from the ARM-specific asm/p2m.h to xen/p2m-common.h.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
desc->status is only set once during setup_irq(), but interrupts can be
enabled/disabled at runtime, so update it in the corresponding callbacks.
For the purposes of the FENCE instruction, CSR read accesses are
classified as device input (I) and CSR write accesses as device output
(O), while the barriers used by spin locks (fence rw,rw) only order
normal memory accesses. An explicit wmb() (fence ow,ow) is therefore
added in aplic_irq_{enable,disable}() to order the desc->status update
with respect to the IMSIC CSR write.
Fixes: d4676a1398 ("xen/riscv: implementation of aplic and imsic operations")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Embedding the symbol table can shift sections and flip relaxation
decisions, changing code size and thus the set of emitted tools/symbols
(e.g. gap end markers). Re-link with a regenerated table when its
size differs from the previous pass.
This takes the same remedy as commit 35de7285d5 ("Arm: do a 4th
linking pass if necessary"), though the underlying cause differs:
here no symbol (dis)appears from the ELF symbol table; instead the
set of end markers emitted by tools/symbols legitimately changes,
because want_symbol_end() depends on code layout, and layout differs
between passes due to linker relaxation.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
arch_domain_create() and arch_sanitise_domain_config() are prerequisites for
domain_create().
arch_sanitise_domain_config() currently returns 0, as there is no specific
work required at this stage.
arch_domain_create() performs basic initialization, such as setting up the P2M
and initializing of next unused phandle.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce HAS_STATIC_MEMORY so that STATIC_MEMORY can be enabled or
disabled on a per-architecture basis. ARM selects the new flag; RISC-V
does not, so CONFIG_STATIC_MEMORY is unavailable on RISC-V and
randconfig builds no longer require an explicit STATIC_MEMORY=n override
to avoid a compilation error.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
As map_device_irqs_to_domain() is used unconditionally by common part of
dom0less code, move the prototype to a common header.
fdt-domain-build.h is chosen as map_device_irqs_to_domain() could be
also called indirectly in Arm's DOM0-related code and DT overlay feature.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Rename the enum to intc_variant and the structure member from hw_version
to hw_variant to better reflect that these values select between
different controller variants, not versions of the same one.
Requested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
fdt_property() unconditionally calls memcpy(ptr, val, len) even when
len is zero and val is NULL. This is a legitimate calling convention
for adding empty FDT properties such as "interrupt-controller", which
carry no payload.
However, compilers that treat memcpy as nonnull on its pointer arguments
will fire UBSAN before observing that len is zero.
Guard the memcpy() with a check on len so it is skipped entirely when
there is no payload to copy, bringing the code in line with the
nonnull contract.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Origin: git://git.kernel.org/pub/scm/utils/dtc/dtc.git f57e7df35df4
Fixes: f0ea065580 ("libfdt: add version 1.3.0")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The `j` instruction (JAL x0) used in switch_stack_and_jump() is a
J-type instruction with only a ±1MB range, and that this can
be exceeded in some configurations, causing a linker error:
relocation truncated to fit: R_RISCV_JAL against `<symbol>'
Replace `j` with `jr` (JALR x0) via an explicit register, which has
unlimited range.
Found in a downstream branch when UBSAN instrumentation was enabled.
Note that the `tail` instruction looks more natural here, but `jr` is
chosen instead to avoid depending on how the assembler expands `tail`
and which scratch register it uses (`t1` in GAS), which would need to
be listed in the clobber section of `asm volatile`.
Fixes: e66003e7be ("xen/riscv: introduce setup_initial_pages")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Baptiste Le Duc <baptiste.le-duc@vates.tech>
Acked-by: Jan Beulich <jbeulich@suse.com>
Everything is ready to enable DOMAIN_BUILD_HELPER which are necessary
for dom0less common code. So enable it.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The dom0less solution uses defined RAM banks as compile-time constants,
so introduce macros to describe guest RAM banks.
The reason for 2 banks is that there is typically always a use case for
low memory under 4 GB, but the bank under 4 GB ends up being small because
there are other things under 4 GB it can conflict with (interrupt
controller, PCI BARs, etc.). So a second bank is added above that MMIO
region (starting at 8 GiB) to provide the remaining RAM; the gap between
the two banks also exercises code paths handling discontiguous memory.
For Sv32 guests (34-bit GPA, 16 GiB addressable), bank0 provides 2 GB
(2–4 GB) and the first 8 GB of bank1 (8–16 GB) is accessible.
Extended regions are useful for RISC-V: they could be used to provide a
"space" for Linux to map grant mappings.
Despite the fact that for every guest MMU mode the GPA could be up
to 56 bits wide (except Sv32 whose GPA is 34 bits), the combined size
of both banks is limited to 1018 GB as it is more than enough for most
use cases.
Add inclusion of asm/guest-layout.h to asm/domain.h to make dom0less
common code build happy.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
common/device-tree/domain-build.c uses p2m_gpa_bits to determine the
upper bound of the GPA space when searching for unused regions, so it
must be defined when CONFIG_DOMAIN_BUILD_HELPERS=y.
The variable is initialised to PADDR_BITS and narrowed in p2m_init() to
the GPA width of the selected G-stage mode, allowing an external entity
(e.g. an IOMMU) to restrict it further if needed.
p2m_gpa_bits is a global rather than a per-domain value, which is
acceptable for now because all domains are required to use the same
G-stage MMU mode, as dom0less common code allocates it per all
domains.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Rework G-stage mode handling to make the selected mode descriptor
reusable outside of p2m initialization, both for filling CPU nodes in
the device tree passed to dom0less guests and for per-domain G-stage
mode selection at domain creation time.
Promote gstage_modes[] from a local __initconst variable inside
gstage_mode_detect() to a file-scope static const array, and convert
max_gstage_mode from an embedded struct (assigned by value) to a global
const pointer into gstage_modes[]. This allows referencing both the mode
identifier and the mode name after init without copying the descriptor.
Remove get_max_supported_mode(); its callers now dereference
max_gstage_mode->mode directly.
Change struct p2m_domain::mode from an embedded gstage_mode_desc to a
const pointer into gstage_modes[], so each domain shares the descriptor
rather than carrying its own copy.
Adjust the gstage_modes[] entries in three ways:
- Use lowercase names without the "x4" suffix (e.g. "sv39" instead of
"Sv39x4"). The DT mmu-type binding [1] does not include the suffix,
so the name can now be passed directly to the guest without
transformation. The suffix is appended only in the diagnostic
printk, where it remains informative.
- Use "none" for Bare mode (HGATP_MODE_OFF) to match the DT binding.
- Change paging_levels to represent the root page-table level index
(i.e. total paging levels minus one) rather than the total count.
P2M_ROOT_LEVEL() now returns the correct VPN index directly, without
requiring callers to subtract one or use hardcoded offsets.
Add gstage_mode to xen_arch_domainconfig so the toolstack can
request a specific G-stage mode at domain creation time. Introduce
find_gstage_mode() to resolve a mode descriptor by HGATP_MODE_*,
capping the result at max_gstage_mode to prevent requesting a mode
the hardware does not support. Update p2m_init() to accept a
xen_domctl_createdomain pointer and call find_gstage_mode()
instead of hardcoding Sv39x4.
Add arch_parse_dom0less_node() in a new dom0less-build.c to read the
"mmu-type" DT property from a guest domain node and store it in
boot_domain::create_cfg.arch.gstage_mode, falling back to maximum
supported mode when the property is absent.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/riscv/cpus.yaml?h=v6.19-rc3#n82
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce support for loading a Linux kernel Image which is got by
uncompressing of Image.gz on RISC-V.
kernel_image_load() and place_modules() currently call panic() on
failure rather than returning an error. This is because the common
kernel_load() in common/device-tree/kernel.c does not expect a
return code. Handling errors gracefully would require a separate
refactor.
The implementation is based on the Xen Arm kernel loading code.
text_offset is available for both 32-bit and 64-bit Image, so fix
that.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce copy_to_guest_phys() for RISC-V, based on the Arm implementation.
Add a generic copy_guest() helper for copying to and from guest physical
(and potentially virtual addresses in the future), and implement
translate_get_page() to translate a guest physical address into a struct
page_info via the domain p2m.
Compared to the Arm code:
- Drop COPY_flush_dcache(), as no such use cases exist on RISC-V.
- Do not implement the linear mapping case, which is currently unused.
- Use PAGE_OFFSET() to initialize the local offset variable in copy_guest().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The IPA terminology is Arm-specific, so rename p2m_ipa_bits to
p2m_gpa_bits to use architecture-neutral naming in
xen/common/device-tree/ code.
No functional changes.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
domain_use_host_layout() is not architecture-specific and may be needed
on x86 [1]. Replace the ARM-specific macro in asm/domain.h with a common
static inline in a new dedicated header, xen/domain-layout.h.
xen/domain.h would be the natural home, but placing it there would
require including xen/paging.h (for paging_mode_translate()) and
xen/sched.h (for is_hardware_domain()), which would introduce circular
dependencies. A separate header that callers opt into avoids this.
Adjust the implementation to take paging_mode_translate() into account
so it works correctly for all architectures, including x86. Some extra
details about implementation [2] and [3].
To avoid the following compilation issue:
In file included from ./include/xen/paging.h:4,
from ./include/xen/domain-layout.h:6,
from common/device-tree/domain-build.c:4:
./arch/riscv/include/asm/paging.h:17:48: error: 'struct page_info'
declared inside parameter list will not be visible outside of this
definition or declaration [-Werror]
17 | void paging_free_page(struct domain *d, struct page_info *pg);
add the forward declaration of struct page_info to RISC-V's asm/paging.h.
[1] https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2602161038120.359097@ubuntu-linux-20-04-desktop/
[2] https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2602271742400.3148344@ubuntu-linux-20-04-desktop/
[3] https://lore.kernel.org/xen-devel/alpine.DEB.2.22.394.2602271750190.3148344@ubuntu-linux-20-04-desktop/
Suggested-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
Since the implementation of fw_unreserved_regions() is in common code, move
its declaration to xen/bootinfo.h.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
The helper kernel_zimage_probe() is referenced from common code
(xen/common/device-tree/kernel.c), but its name is tied to the zImage
format which is specific to Arm (from architectures supported by Xen).
Other architectures supported by Xen, such as RISC-V, do not use the
zImage format and instead rely on other kernel image types (e.g. Image
or compressed Image variants: Image.gz, etc). Using "zimage" in the
name is therefore misleading in architecture-independent code.
Rename kernel_zimage_probe() to kernel_image_probe() and update the
associated structure field from "zimage" to "image" to reflect that the
code handles generic kernel images rather than the zImage format
specifically.
No functional change intended.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
Widen the len argument of copy_to_guest_phys_flush_dcache() and the
copy_to_guest_phys_cb typedef from unsigned int to unsigned long, as
the function can be used to copy large blobs such as the initrd which
may exceed 4GB. Update the internal copy_guest() len argument to
unsigned long accordingly.
Change the type for local variable size in copy_guest() to avoid
compilation error because of type mismatch.
raw_* wrappers above copy_guest() keep returning unsigned long to
avoid type narrowing; it is not an issue for raw_*'s len argument
to remain 'unsigned int' since the assignment to copy_guest()'s wider
unsigned long parameter is safe and there is no raw_* users who
are using a value bigger than what can fit into 'unsigned int'.
Fixes: 2986481b3d ("xen/arm: guest_copy: Extend the prototype to pass the vCPU")
Fixes: 5302bd490b ("xen/arm: Introduce copy_to_guest_phys_flush_dcache")
Fixes: d07b7369aa ("xen/common: dom0less: introduce common domain-build.c")
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
The MMIO alignment check in imsic_init() incorrectly uses cpu as
the index into mmios[]. The loop index should be used instead.
Otherwise, the alignment check may be performed on the wrong MMIO
address.
Fix the index used in the alignment check and corresponding printk.
Fixes: c9bd8b322e ("xen/riscv: imsic_init() implementation")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
As domain type is part of common code now there is no any reason
to have architecture-specific set_domain_type() functions so
it is dropped.
Change the guard around access of kinfo->type to CONFIG_HAS_DOMAIN_TYPE
for consistency. Also, drop and add some parentheses to be aligned
with the similar if() below.
x86 with CONFIG_64BIT=y shouldn't use is_{32,64}bit_domain() as
x86 doesn't have support of CONFIG_HAS_DOMAIN_TYPE. Since x86_32 Xen no
longer builds, the fallback is currently only relevant for arm32.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
There are two reasons of this change:
1. Align prototype with what other make_*_node() are passed.
2. A follow-up RISC-V patch will call get_next_free_phandle() inside
make_cpus_node(), requiring mutable access to kinfo->free_phandle.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
There are cases where it is necessary to know the next available phandle
number in order to generate phandles for guest device nodes.
When a partial FDT (pfdt) is provided, special care is needed during
initialization of next_phandle, as the pfdt may already contain a dummy
interrupt controller node with a phandle assigned to it. next_phandle
must therefore be initialized to one past the highest phandle already
present in the pfdt, to avoid collisions.
Since next_phandle may be needed for the very first guest node generated,
domain_handle_dtb_boot_module() is moved earlier in prepare_dtb_domU().
The new call site also aligns better with the existing comment stating
that domain_handle_dtb_boot_module() must be called before the rest of
the device tree is generated.
Introduce alloc_phandle() to ensure that phandles allocated for guest
nodes do not overlap the Xen-reserved phandle range. This helper will
be used by subsequent patches (by RISC-V at the moment).
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
OpenSBI currently does not advertise the SSTC extension via the device
tree, so if SSTC support is detected by Xen the riscv_isa bitmap is updated
manually. Furthermore, removing the "sstc" string from riscv,isa is not
a reliable way to disable SSTC, because OpenSBI probes support by
attempting to access CSR_STIMECMP.
Introduce a runtime probe in Xen to determine whether SSTC is available.
The probe attempts to read CSR_STIMECMP using csr_read_safe(). If the
access succeeds, SSTC is considered available; if a trap occurs, it is
treated as unsupported.
When SSTC is detected, Xen may use it internally to program timers.
However, the extension is not exposed to guests because the required
context switch handling for the SSTC CSRs is not yet implemented.
Note: clearing RISCV_ISA_EXT_sstc from the DTS riscv,isa property is
deferred to a follow-up patch. Also, the corresponding HENVCFG bit is
not set so guests fall back to the SBI timer interface. Timer requests
are then handled by Xen via the usual SBI interception path.
Introduce set_xen_timer() to abstract how the timer is programmed,
either via the SSTC extension or an SBI call.
Drop sbi_set_timer() as it is more than enough to have only introduced
set_xen_timer().
Drop "SBI v0.2 TIME extension detected" message to avoid confusion
which set timer function is really used.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The function is implemented out-of-line rather than as a static inline,
to avoid header ordering issues where struct domain is incomplete when
asm/p2m.h is included, leading to build failures:
In file included from ./arch/riscv/include/asm/domain.h:10,
from ./include/xen/domain.h:16,
from ./include/xen/sched.h:11,
from ./include/xen/event.h:12,
from common/cpu.c:3:
./arch/riscv/include/asm/p2m.h: In function 'get_page_from_gfn':
./arch/riscv/include/asm/p2m.h:50:33: error: invalid use of undefined type 'struct domain'
50 | #define p2m_get_hostp2m(d) (&(d)->arch.p2m)
| ^~
./arch/riscv/include/asm/p2m.h:180:38: note: in expansion of macro 'p2m_get_hostp2m'
180 | return p2m_get_page_from_gfn(p2m_get_hostp2m(d), _gfn(gfn), t);
| ^~~~~~~~~~~~~~~
make[2]: *** [Rules.mk:253: common/cpu.o] Error 1
make[1]: *** [build.mk:72: common] Error 2
make: *** [Makefile:623: xen] Error 2
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Accessing some CSRs may trap when the corresponding extension is not
implemented or enabled. Introduce csr_read_safe() which attempts to
read a CSR and relies on the exception table mechanism to safely recover
if the access faults.
This helper allows Xen to probe CSR availability without taking a fatal
trap and will be used for feature detection during early boot as we
can't always rely on what is in riscv,isa string in DTS.
While touching the header, reorder the include directives to follow the
usual Xen style.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Introduce exception table handling for RISC-V so faults from selected
instructions can be recovered via fixup handlers instead of being
treated as fatal.
Add the RISC-V exception table format, sorting at boot to allow binary
search used furthuer, and lookup from the trap handler. Update the
linker script to emit the .ex_table section using introduced common
EX_TABLE macro shared with other architectures.
Also, reduce __start___ex_table alignment from 8 to 4 bytes to
match the natural alignment of struct exception_table_entry,
which contains two int32_t fields.
Add inclusion of asm/extable.h to asm/bug.h to deal with compilation
issue of common/virtual_region.c, which require declaration of
__start___ex_table and __stop___ex_table.
This implementation is based on Linux 6.16.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
There is no reason to use _UL() in define-s sitting in C file hence use UL
suffix instead.
Drop 3d argument of INIT_CSR_MASK() and INIT_RO_ONE_MASK() to reduce risk
of incomplete editing after copy-and-paste, or other typo-ing.
Use _VALID_ infix instead of _AVAIL_ as the mask identifies architecturally
defined bits, not bits available for software use.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The scheduler may call this function to force synchronization of given
vCPU's state. RISC-V does not support lazy context switching, so nothing
is done in sync_vcpu_execstate() and sync_local_execstate().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
As the tasklet subsystem is now initialized, it is necessary to implement
sync_local_execstate(), since it is invoked when something calls
tasklet_softirq_action(), which is registered in tasklet_subsys_init().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce timer_interrupt() to process IRQ_S_TIMER interrupts.
The handler disables further timer interrupts by clearing
SIE.STIE and raises TIMER_SOFTIRQ so the generic timer subsystem
can perform its processing.
Update do_trap() to dispatch IRQ_S_TIMER to this new handler.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement reprogram_timer() on RISC-V using the standard SBI timer call.
The privileged architecture only defines machine-mode timer interrupts
(using mtime/mtimecmp). Therefore, timer services for S/HS/VS mode must
be provided by M-mode via SBI calls. SSTC (Supervisor-mode Timer Control)
is optional and is not supported on the boards available to me, so the
only viable approach today is to program the timer through SBI.
reprogram_timer() enables/disables the supervisor timer interrupt and
programs the next timer deadline using sbi_set_timer(). If the SBI call
fails, the code panics, because sbi_set_timer() is expected to return
either 0 or -ENOSUPP (this has been stable from early OpenSBI versions to
the latest ones). The SBI spec does not define a standard negative error
code for this call, and without SSTC there is no alternative method to
program the timer, so the SBI timer call must be available.
reprogram_timer() currently returns int for compatibility with the
existing prototype. While it might be cleaner to return bool, keeping the
existing signature avoids premature changes in case sbi_set_timer() ever
needs to return other values (based on which we could try to avoid
panic-ing) in the future.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce a function pointer for sbi_set_timer(), since different OpenSBI
versions may implement the TIME extension with different extension IDs
and/or function IDs.
If the TIME extension is not available, fall back to the legacy timer
mechanism. This is useful when Xen runs as a guest under another Xen,
because the TIME extension is not currently virtualised and therefore
will not appear as available.
Despite of the fact that sbi_set_timer_v01 is introduced and used as
fall back, SBI v0.1 still isn't fully supported (with the current SBI
calls usage, sbi_rfence_v01 should be introduced too), so panic()
in sbi_init() isn't removed.
The sbi_set_timer() pointer will be used by reprogram_timer() to program
Xen’s physical timer as without SSTC extension there is no any other
option except SBI call to do that as only M-timer is available for us.
Use dprintk() for all the cases to print that a speicifc SBI extension
is available as it isn't really necessary in case of release builds.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Add handling of the SBI_EXT_0_1_SET_TIMER function ID to the legacy
extension ecall handler. The handler now programs the vCPU’s virtual
timer via vtimer_set_timer() and returns SBI_SUCCESS.
This enables guests using the legacy SBI timer interface to schedule
timer events correctly.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce vtimer_ctxt_switch_from() and vtimer_ctxt_switch_to() to handle
virtual timer state across vCPU context switches.
At present, vtimer_ctxt_switch_from() is a no-op because the RISC-V SSTC
extension, which provides a virtualization-aware timer, is not yet
supported. Xen therefore relies the virtual (SBI-based) timer.
The virtual timer uses Xen's internal timer infrastructure and must be
associated with the pCPU on which the vCPU is currently running so that
timer events can be delivered efficiently. As a result, vtimer_ctxt_switch_to()
migrates the timer to the target pCPU when a vCPU is scheduled in.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Add a RISC-V implementation of vcpu_kick(), which unblocks the target
vCPU and sends an event check IPI if the vCPU was running on another
processor. This mirrors the behavior of Arm and enables proper vCPU
wakeup handling on RISC-V.
Remove the stub implementation from stubs.c, as it is now provided by
arch/riscv/domain.c.
Since vcpu_kick() calls perfc_incr(vcpu_kick), add perfcounter for
vcpu_kick to handle the case when CONFIG_PERF_COUNTERS=y. Although
CONFIG_PERF_COUNTERS is not enabled by default, it can be enabled,
for example, by randconfig what will lead to CI build issues.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Lay the groundwork for guest timer support by introducing a per-vCPU
virtual timer backed by Xen’s common timer infrastructure.
The virtual timer is programmed in response to the guest SBI
sbi_set_timer() call and injects a virtual supervisor timer interrupt
into the vCPU when it expires.
While a dedicated struct vtimer is not strictly required at present,
it is expected to become necessary once SSTC support is introduced.
In particular, it will need to carry additional state such as whether
SSTC is enabled, the next compare value (e.g. for the VSTIMECMP CSR)
to be saved and restored across context switches, and time delta state
(e.g. HTIMEDELTA) required for use cases such as migration. Introducing
struct vtimer now avoids a later refactoring.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
This patch is based on Linux kernel 6.16.0.
Add the consumer side (vcpu_flush_interrupts()) of the lockless pending
interrupt tracking introduced in part 1 (for producers). According, to the
design only one consumer is possible, and it is vCPU itself.
vcpu_flush_interrupts() is expected to be ran (as guests aren't ran now due
to the lack of functionality) before the hypervisor returns control to the
guest.
Producers may set bits in irqs_pending_mask without a lock. Clearing bits in
irqs_pending_mask is performed only by the consumer via xchg() (with aquire
semantics). The consumer must not write to irqs_pending and must not act on
bits that are not set in the mask. Otherwise, extra synchronization should
be provided.
The worst thing which could happen with such approach is that a new pending
bit will be set to irqs_pending bitmap during update of hvip variable in
vcpu_flush_interrupt() but it isn't problem as the new pending bit won't
be lost and just be proceded during the next flush.
As AIA specs introduced hviph register which would want to be updated when
guest related AIA code vcpu_update_hvip() is introduced instead of just
open-code it in vcpu_flush_interrupts().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Based on Linux kernel v6.16.0.
Note that smp_wmb() is used instead of smp_mb__before_atomic() as what
we want to guarantee that if a bit in irqs_pending_mask is obversable
that the correspondent bit in irqs_pending is observable too.
Add lockless tracking of pending vCPU interrupts using atomic bitops.
Two bitmaps are introduced:
- irqs_pending — interrupts currently pending for the vCPU
- irqs_pending_mask — bits that have changed in irqs_pending
The design follows a multi-producer, single-consumer model, where the
consumer is the vCPU itself. Producers may set bits in
irqs_pending_mask without a lock. Clearing bits in irqs_pending_mask is
performed only by the consumer via xchg(). The consumer must not write
to irqs_pending and must not act on bits that are not set in the mask.
Otherwise, extra synchronization should be provided.
On RISC-V interrupts are not injected via guest registers, so pending
interrupts must be recorded in irqs_pending (using the new
vcpu_{un}set_interrupt() helpers) and flushed to the guest by updating
HVIP before returning control to the guest. The consumer side is
implemented in a follow-up patch.
A barrier between updating irqs_pending and setting the corresponding
mask bit in vcpu_set_interrupt()/vcpu_unset_interrupt() guarantees
that if the consumer observes a mask bit set, the corresponding pending
bit is also visible. This prevents missed interrupts during the flush.
It is possible that a guest could have pending bit in the hardware
register without being marked pending in irq_pending bitmap as:
According to the RISC-V ISA specification:
Bits hip.VSSIP and hie.VSSIE are the interrupt-pending and
interrupt-enable bits for VS-level software interrupts. VSSIP in hip
is an alias (writable) of the same bit in hvip.
Additionally:
When bit 2 of hideleg is zero, vsip.SSIP and vsie.SSIE are read-only
zeros. Else, vsip.SSIP and vsie.SSIE are aliases of hip.VSSIP and
hie.VSSIE.
This means the guest may modify vsip.SSIP, which implicitly updates
hip.VSSIP and the bit being written with 1 would also trigger an interrupt
as according to the RISC-V spec:
These conditions for an interrupt trap to occur must be evaluated in a
bounded amount of time from when an interrupt becomes, or ceases to be,
pending in sip, and must also be evaluated immediately following the
execution of an SRET instruction or an explicit write to a CSR on which
these interrupt trap conditions expressly depend (including sip, sie and
sstatus).
What means that IRQ_VS_SOFT must be synchronized separately, what is done
in vcpu_sync_interrupts(). Note, also, that IRQ_PMU_OVF would want to be
synced for the similar reason as IRQ_VS_SOFT, but isn't sync-ed now as
PMU isn't supported now.
For the remaining VS-level interrupt types (IRQ_VS_TIMER and
IRQ_VS_EXT), the specification states they cannot be modified by the guest
and are read-only because of:
Bits hip.VSEIP and hie.VSEIE are the interrupt-pending and interrupt-enable
bits for VS-level external interrupts. VSEIP is read-only in hip, and is
the logical-OR of these interrupt sources:
• bit VSEIP of hvip;
• the bit of hgeip selected by hstatus.VGEIN; and
• any other platform-specific external interrupt signal directed to
VS-level.
Bits hip.VSTIP and hie.VSTIE are the interrupt-pending and interrupt-enable
bits for VS-level timer interrupts. VSTIP is read-only in hip, and is the
logical-OR of hvip.VSTIP and any other platform-specific timer interrupt
signal directed to VS-level.
and
When bit 10 of hideleg is zero, vsip.SEIP and vsie.SEIE are read-only zeros.
Else, vsip.SEIP and vsie.SEIE are aliases of hip.VSEIP and hie.VSEIE.
When bit 6 of hideleg is zero, vsip.STIP and vsie.STIE are read-only zeros.
Else, vsip.STIP and vsie.STIE are aliases of hip.VSTIP and hie.VSTIE.
and also,
Bits sip.SEIP and sie.SEIE are the interrupt-pending and interrupt-enable
bits for supervisor-level external interrupts. If implemented, SEIP is
read-only in sip, and is set and cleared by the execution environment,
typically through a platform-specific interrupt controller.
Bits sip.STIP and sie.STIE are the interrupt-pending and interrupt-enable
bits for supervisor-level timer interrupts. If implemented, STIP is
read-only in sip, and is set and cleared by the execution environment
Thus, for these interrupt types, it is sufficient to use vcpu_set_interrupt()
and vcpu_unset_interrupt(), and flush them during the call of
vcpu_flush_interrupts() (which is introduced in follow up patch).
vcpu_sync_interrupts(), which is called just before entering the VM,
slightly bends the rule that the irqs_pending bit must be written
first, followed by updating the corresponding bit in irqs_pending_mask.
However, it still respects the core guarantee that the producer never
clears the mask and only writes to irqs_pending if it is the one that
flipped the corresponding mask bit from 0 to 1.
Moreover, since the consumer won't run concurrently because
vcpu_sync_interrupts() and the consumer path are going to be invoked
sequentially immediately before VM entry, it is safe to slightly relax
this ordering rule in vcpu_sync_interrupts().
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce vcpu_csr_init() to initialise hypervisor CSRs that control
vCPU execution and virtualization behaviour before the vCPU is first
scheduled.
The function configures trap and interrupt delegation to VS-mode by
setting the appropriate bits in the hedeleg and hideleg registers,
initializes hstatus so that execution enters VS-mode when control is
passed to the guest, and restricts guest access to hardware performance
counters by initializing hcounteren, as unrestricted access would
require additional handling in Xen.
When the Smstateen and SSAIA extensions are available, access to AIA
CSRs and IMSIC guest interrupt files is enabled by setting the
corresponding bits in hstateen0, avoiding unnecessary traps into Xen
(note that SVSLCT(Supervisor Virtual Select) name is used intead of
CSRIND as OpenSBI uses such name and riscv_encoding.h is mostly based
on it).
If the Svpbmt extension is supported, the PBMTE bit is set in
henvcfg to allow its use for VS-stage address translation. Guest
access to the ENVCFG CSR is also enabled by setting ENVCFG bit in
hstateen0, as a guest may need to control certain characteristics of
the U-mode (VU-mode when V=1) execution environment.
For CSRs that may contain read-only bits (e.g. hedeleg, hideleg,
hstateen0), to the written value a correspondent mask is applied to
avoid divergence between the software state and the actual CSR
contents.
As hstatus is not part of struct arch_vcpu (it already resides in
struct cpu_user_regs), introduce vcpu_guest_cpu_user_regs() to provide
a uniform way to access hstatus and other guest CPU user registers.
This establishes a consistent and well-defined initial CSR state for
vCPUs prior to their first context switch.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Some hypervisor CSRs expose optional functionality and may not implement
all architectural bits. Writing unsupported bits can either be ignored
or raise an exception depending on the platform.
Detect the set of writable bits for selected hypervisor CSRs at boot and
store the resulting masks for later use. This allows safely programming
these CSRs during vCPU context switching and avoids relying on hardcoded
architectural assumptions.
Use csr_read()&csr_write() instead of csr_swap()+all ones mask as some
CSR registers have WPRI fields which should be preserved during write
operation.
Also, ro_one struct is introduced to cover the cases when a bit in CSR
register (at the momemnt, it is only hstateen0) may be r/o-one to have
hypervisor view of register seen by guest correct.
Masks are calculated at the moment only for hedeleg, henvcfg, hideleg,
hstateen0 registers as only them are going to be used in the follow up
patch.
If the Smstateen extension is not implemented, hstateen0 cannot be read
because the register is considered non-existent. Instructions that attempt
to access a CSR that is not implemented or not visible in the current mode
are reserved and will raise an illegal-instruction exception.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
ticks_to_ns() and ns_to_ticks() are not architecture-specific, so provide a
common implementation that is more resilient to overflow than the historical
Arm version. This is not a practical issue for Arm, as the latest ARM ARM
that timer frequency should be fixed at 1 GHz and older platforms used much
lower rates, which is shy of 32-bit overflow. As the helpers are declared
as static inline, they should not affect x86, which does not use them.
On Arm, these helpers were historically implemented as out-of-line functions
because the counter frequency was originally defined as static and unavailable
to headers [1]. Later changes [2] removed this restriction, but the helpers
remained unchanged. Now they can be implemented as static inline without any
issues.
Centralising the helpers avoids duplication and removes subtle differences
between architectures while keeping the implementation simple.
Drop redundant <asm/time.h> includes where <xen/time.h> already pulls it in.
No functional change is intended.
[1] ddee56dc29 arm: driver for the generic timer for ARMv7
[2] 096578b4e4 xen: move XEN_SYSCTL_physinfo, XEN_SYSCTL_numainfo and
XEN_SYSCTL_topologyinfo to common code
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Bertrand Marquis <bertrand.marquis@arm.com>
Introduce helpers to manage VS-stage and G-stage translation state during
vCPU context switches.
As VSATP and HGATP cannot be updated atomically, clear VSATP on context
switch-out to prevent speculative VS-stage translations from being associated
with an incorrect VMID. On context switch-in, restore HGATP and VSATP in the
required order.
Add p2m_handle_vmenter() to perform VMID management and issue TLB flushes
only when required (e.g. on VMID reuse or generation change).
This provides the necessary infrastructure for correct p2m context switching
on RISC-V.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
At the moment, all code and data in setup.c are marked with __init or
__initdata, which is intended to remain this way. Leverage this by using
setup.init.o instead of setup.o in Makefile.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce architecture-specific functions to create and destroy VCPUs.
Note that arch_vcpu_create() currently returns -EOPNOTSUPP, as the virtual
timer and interrupt controller are not yet implemented.
Add callee-saved registers to struct arch_vcpu which are used to preserve
Xen’s own execution context when switching between vCPU stacks.
It is going to be used in the following way (pseudocode):
context_switch(prev_vcpu, next_vcpu):
...
/* Switch from previous stack to the next stack. */
__context_switch(prev_vcpu, next_vcpu);
...
schedule_tail(prev_vcpu):
Save and restore vCPU's CSRs.
The Xen-saved context allows __context_switch() to switch execution
from the previous vCPU’s stack to the next vCPU’s stack and later resume
execution on the original stack when switching back.
During vCPU creation, the Xen-saved context is going to be initialized
with:
- SP pointing to the newly allocated vCPU stack.
- RA pointing to a helper that performs final vCPU setup before
transferring control to the guest.
As part of this change, add continue_new_vcpu(), which will be used after
the first context_switch() of a new vCPU. Since this functionality is not
yet implemented, continue_new_vcpu() is currently provided as a stub.
The prev argument is going to be set by RISC-V ABI (prev will be stored in
a0) when __context_swtich() will be introduced and called from
context_switch().
Update the STACK_SIZE definition.
Introduce struct cpu_info to store per-vCPU state that lives at the top
of the vCPU's stack.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce flush_tlb_guest_local() to perform a local TLB flush of the guest's
address space for the current hart. This leverages the RISC-V HFENCE.VVMA
instruction, which is used to invalidate translations in the VS-stage of
address translation.
As for RISC-V binutils >= 2.39 is choosen, we can use hfence.vvma mnemonics
instead of defining hfence.vvma using .insn.
Although it would be possible to use sbi_remote_hfence_vvma() for this purpose,
it is unnecessary in this context since the flush is required only on the
local hart. Using the SBI call would introduce additional overhead without
benefit, resulting in unnecessary performance loss.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
If the Smstateen extension is not implemented, the hstateen0 CSR is
considered non-existent. Any attempt to access it will raise an
illegal-instruction exception.
Guard the hstateen0 dump with a runtime check for Smstateen support to
avoid triggering traps when dumping CSRs on systems that do not
implement this extension.
Fixes: 3babc8d2e5 ("xen/riscv: dump GPRs and CSRs on unexpected traps")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Currently, an interrupt cause which is not explicitly handled is silently
ignored, and execution resumes without reporting the fault. This is
incorrect and do_unexpected_trap() should be called in the case of
unhandled interrupt.
Fixes: a8b85fabf6 ("xen/riscv: add external interrupt handling for hypervisor mode")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
RISC-V SMP support is not yet implemented, but smp_send_event_check_mask()
is required by common code and vcpu_kick(), which is introduced later.
Provide a temporary stub implementation that asserts the mask only targets
CPU0.
cpumask_subset() is used instead of cpumask_equal() because some callers
(e.g. cpumask_raise_softirq() or cpu_raise_softirq_batch_finish()) may
legitimately pass an empty mask, which would otherwise cause false
failures.
The BUG_ON() ensures that attempts to use this function with multiple CPUs
are caught early once SMP support is introduced.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Provide additional context when an unexpected exception occurs by dumping
the relevant Supervisor, Virtual Supervisor (VS), and Hypervisor CSRs,
along with the general-purpose registers associated with the trap.
Dumping VS-mode CSRs in addition to host CSRs is beneficial when analysing
VS-mode traps. VSCAUSE, VSEPC, VSTVAL, and related VS state are required to
properly diagnose unexpected guest traps and potential hypervisor
misconfiguration.
For example, on an illegal-instruction exception the hardware may record
the faulting instruction in VSTVAL. If VSTVAL is zero, VSEPC should always
be inspected, and can be used together with objdump to identify the
faulting instruction. Dumping VSCAUSE is also useful when the guest does
not report it, or when the hypervisor redirects a trap to the guest using
VSCAUSE, VSTATUS, and VSTVEC, allowing verification that a subsequent trap
is not caused by incorrect VS state.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce tracking of metadata page entries usage and if all of them are
p2m_invalid then free them.
Intermediate P2M page tables are allocated with MEMF_no_owner, so we are free
to repurpose struct page_info fields for them. Since page_info.u.* is not
used for such pages, introduce a used_entries counter in struct page_info
to track how many metadata entries are in use for a given intermediate P2M
page table.
The counter is updated in p2m_set_type() when metadata entries transition
between p2m_invalid and a valid external type. When the last metadata entry
is cleared (used_entries == 0), the associated metadata page is freed and
returned to the P2M pool.
Refactor metadata page freeing into a new helper, p2m_free_metadata_page(),
as the same logic is needed both when tearing down a P2M table and when
all metadata entries become p2m_invalid in p2m_set_type(). As part of this
refactoring, move the declaration of p2m_free_page() earlier to satisfy the
new helper.
Additionally, implement page_set_tlbflush_timestamp() for RISC-V instead of
BUGing, as it is invoked when returning memory to the domheap.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
RISC-V's PTE has only two available bits that can be used to store the P2M
type. This is insufficient to represent all the current RISC-V P2M types.
Therefore, some P2M types must be stored outside the PTE bits.
To address this, a metadata table is introduced to store P2M types that
cannot fit in the PTE itself. Not all P2M types are stored in the
metadata table—only those that require it.
The metadata table is linked to the intermediate page table via the
`struct page_info`'s v.md.metadata field of the corresponding intermediate
page.
Such pages are allocated with MEMF_no_owner, which allows us to use
the v field for the purpose of storing the metadata table.
To simplify the allocation and linking of intermediate and metadata page
tables, `p2m_{alloc,free}_table()` functions are implemented.
These changes impact `p2m_split_superpage()`, since when a superpage is
split, it is necessary to update the metadata table of the new
intermediate page table — if the entry being split has its P2M type set
to `p2m_ext_storage` in its `P2M_TYPES` bits. In addition to updating
the metadata of the new intermediate page table, the corresponding entry
in the metadata for the original superpage is invalidated.
Also, update p2m_{get,set}_type to work with P2M types which don't fit
into PTE bits.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce helper functions for safely querying the P2M (physical-to-machine)
mapping:
- add p2m_read_lock(), p2m_read_unlock(), and p2m_is_locked() for managing
P2M lock state.
- Implement p2m_get_entry() to retrieve mapping details for a given GFN,
including MFN, page order, and validity.
- Introduce p2m_get_page_from_gfn() to convert a GFN into a page_info
pointer, acquiring a reference to the page if valid.
- Introduce get_page().
Implementations are based on Arm's functions with some minor modifications:
- p2m_get_entry():
- Reverse traversal of page tables, as RISC-V uses the opposite level
numbering compared to Arm.
- Removed the return of p2m_access_t from p2m_get_entry() since
mem_access_settings is not introduced for RISC-V.
- Updated BUILD_BUG_ON() to check using the level 0 mask, which corresponds
to Arm's THIRD_MASK.
- Replaced open-coded bit shifts with the BIT() macro.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
As {alloc,free}_domain_struct() are used only within domain.c,
they can be declared static and their declarations removed
from xen/domain.h.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
alloc_vcpu_struct() and free_vcpu_struct() contain little
architecture-specific logic and are suitable for sharing across
architectures. Move both helpers to common code.
To support the remaining architectural differences, introduce
arch_vcpu_struct_memflags(), allowing architectures to override the
memory flags passed to alloc_xenheap_pages(). This is currently needed
by x86, which may require MEMF_bits(32) for HVM guests using shadow
paging.
The ARM implementation of alloc/free_vcpu_struct() is removed and
replaced by the common version. Stub implementations are also dropped
from PPC and RISC-V.
Now that the size of struct vcpu for Arm64 is smaller than PAGE_SIZE,
MAX_PAGES_PER_VCPU is no longer needed and is removed.
Finally, make alloc_vcpu_struct() and free_vcpu_struct() static to
common/domain.c, as they are no longer used outside common code.
No functional changes.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
When CONFIG_NEW_VGIC=y and CONFIG_ARM_64=y, the size of struct vcpu
exceeds one page, which requires allocating two pages and led to the
introduction of MAX_PAGES_PER_VCPU.
To remove the need for MAX_PAGES_PER_VCPU, the vgic member of NEW_VGIC's
struct vgic_cpu member private_irqs is changed to a pointer to struct
vgic_irq.
As a result, the size of struct vcpu for Arm64 is reduced to 2176 bytes
in the case when CONFIG_ARM_64=y and CONFIG_NEW_VGIC=y, compared to 3840
bytes (without these changes and with CONFIG_ARM_64=y) and 4736 bytes
(without these changes and with both CONFIG_ARM_64=y and CONFIG_NEW_VGIC=y).
Note that all numbers are based on defconfig with the mentioned options
enabled or disabled as specified.
Since the private_irqs member is now a pointer, vcpu_vgic_init() and
vcpu_vgic_free() are updated to allocate and free private_irqs instance.
As struct vcpu now fits into one page, drop MAX_PAGES_PER_VCPU.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
Add support of virtual SBI base extension calls for RISC-V guests, delegating
hardware-specific queries to the underlying SBI and handling version and
firmware ID queries directly.
The changes include:
1. Define new SBI base extension function IDs (SBI_EXT_BASE_GET_MVENDORID,
SBI_EXT_BASE_GET_MARCHID, SBI_EXT_BASE_GET_MIMPID).
2. Introduce XEN_SBI_VER_MAJOR, XEN_SBI_VER_MINOR for imeplenataion of
SBI_EXT_BASE_GET_SPEC_VERSION.
4. Introduce SBI_XEN_IMPID to implement SBI_EXT_BASE_GET_IMP_ID.
5. Implement handling of SBI base extension functions, including version,
firmware ID, and machine-specific queries.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Use XFREE() instead of xfree() so that vcpu_vgic_free() can be idempotent.
With XFREE(), vgic_vcpu->private_irqs is set to NULL, so calling
vcpu_vgic_free() a second time is not an issue.
Update the prototype of vcpu_vgic_free() to return void to satisfy MISRA
Rule 17.7, since the return value of vcpu_vgic_free() is not used by any
callers.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Michal Orzel <michal.orzel@amd.com>
Add support for legacy SBI extensions (version 0.1) in Xen for guest
domains.
The changes include:
1. Define all legacy SBI extension IDs (0x0 to 0x8) for better clarity and
completeness.
2. Implement handling of legacy SBI extensions, starting with support for
SBI_EXT_0_1_CONSOLE_PUTCHAR. SBI_EXT_0_1_CONSOLE_GETCHAR is marked as
not supported as legacy SBI console related stuff is expected to be used
only for early debugging of guest.
The implementation uses the existing virtual SBI framework to handle legacy
SBI ecalls, ensuring compatibility with older SBI specifications in
RISC-V guests.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce support for handling virtual SBI extensions in Xen.
The changes include:
- Added new vsbi/core.c and vsbi.h files to implement virtual SBI extension
handling.
- Modified traps.c to handle CAUSE_VIRTUAL_SUPERVISOR_ECALL by calling
vsbi_handle_ecall() when the trap originates from VS-mode.
- Updated xen.lds.S to include a new .vsbi.exts section for virtual SBI
extension data.
- Updated Makefile to include the new vsbi/ directory in the build.
- Add hstatus register to struct cpu_user_regs as it is needed for
a check that CAUSE_VIRTUAL_SUPERVISOR_ECALL happens from VS-mode.
Also, add storing/restoring of hstatus register in handle_trap().
- Introduce vsbi_find_extension() to check if vsbi extension is supported
by Xen. For now it is called only inside vsbi/core.c, but in future
it is going to be called from other files.
- Introduce check_vsbi_ext_ranges() to check if there EIDs ranges
overlapping between extensions.
The implementation allows for registration and handling of SBI
extensions via a new vsbi_ext structure and ".vsbi.exts" section,
enabling extensible virtual SBI support for RISC-V guests.
Note: All EIDs are printed in the format #%#lx and all FIDs in #%lu, as
the SBI spec uses these formats. Printing them this way makes it easier
to search for them in the SBI spec.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement the mfn_valid() macro to verify whether a given MFN is valid by
checking that it falls within the range [start_page, max_page).
These bounds are initialized based on the start and end addresses of RAM.
As part of this patch, start_page is introduced and initialized with the
PFN of the first RAM page.
Also, initialize pdx_group_valid() by calling set_pdx_range() when
memory banks are being mapped.
Also, after providing a non-stub implementation of the mfn_valid() macro,
the following compilation errors started to occur:
riscv64-linux-gnu-ld: prelink.o: in function `alloc_heap_pages':
/build/xen/common/page_alloc.c:1054: undefined reference to `page_is_offlinable'
riscv64-linux-gnu-ld: /build/xen/common/page_alloc.c:1035: undefined reference to `page_is_offlinable'
riscv64-linux-gnu-ld: prelink.o: in function `reserve_offlined_page':
/build/xen/common/page_alloc.c:1151: undefined reference to `page_is_offlinable'
riscv64-linux-gnu-ld: ./.xen-syms.0: hidden symbol `page_is_offlinable' isn't defined
riscv64-linux-gnu-ld: final link failed: bad value
make[2]: *** [arch/riscv/Makefile:28: xen-syms] Error 1
To resolve these errors, the following functions have also been introduced,
based on their Arm counterparts:
- page_get_owner_and_reference() and its variant to safely acquire a
reference to a page and retrieve its owner.
- Implement page_is_offlinable() to return false for RISC-V.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement put_page(), as it will be used by p2m_put_*-related code.
Although CONFIG_STATIC_MEMORY has not yet been introduced for RISC-V,
a stub for PGC_static is added to avoid cluttering the code of
put_page() with #ifdefs.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Add support for down large memory mappings ("superpages") in the RISC-V
p2m mapping so that smaller, more precise mappings ("finer-grained entries")
can be inserted into lower levels of the page table hierarchy.
To implement that the following is done:
- Introduce p2m_split_superpage(): Recursively shatters a superpage into
smaller page table entries down to the target level, preserving original
permissions and attributes.
- p2m_set_entry() updated to invoke superpage splitting when inserting
entries at lower levels within a superpage-mapped region.
This implementation is based on the ARM code, with modifications to the part
that follows the BBM (break-before-make) approach, some parts are simplified
as according to RISC-V spec:
It is permitted for multiple address-translation cache entries to co-exist
for the same address. This represents the fact that in a conventional
TLB hierarchy, it is possible for multiple entries to match a single
address if, for example, a page is upgraded to a superpage without first
clearing the original non-leaf PTE’s valid bit and executing an SFENCE.VMA
with rs1=x0, or if multiple TLBs exist in parallel at a given level of the
hierarchy. In this case, just as if an SFENCE.VMA is not executed between
a write to the memory-management tables and subsequent implicit read of the
same address: it is unpredictable whether the old non-leaf PTE or the new
leaf PTE is used, but the behavior is otherwise well defined.
In contrast to the Arm architecture, where BBM is mandatory and failing to
use it in some cases can lead to CPU instability, RISC-V guarantees
stability, and the behavior remains safe — though unpredictable in terms of
which translation will be used.
Additionally, the page table walk logic has been adjusted, as ARM uses the
opposite level numbering compared to RISC-V.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Implement the p2m_next_level() function, which enables traversal and dynamic
allocation of intermediate levels (if necessary) in the RISC-V
p2m (physical-to-machine) page table hierarchy.
To support this, the following helpers are introduced:
- page_to_p2m_table(): Constructs non-leaf PTEs pointing to next-level page
tables with correct attributes.
- p2m_alloc_page(): Allocates page table pages, supporting both hardware and
guest domains.
- p2m_create_table(): Allocates and initializes a new page table page and
installs it into the hierarchy.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
This patch adds the initial logic for constructing PTEs from MFNs in the RISC-V
p2m subsystem. It includes:
- Implementation of p2m_pte_from_mfn(): Generates a valid PTE using the
given MFN, p2m_type_t, including permission encoding and PBMT attribute
setup.
- New helper p2m_set_permission(): Encodes access rights (r, w, x) into the
PTE based on both p2m type and access permissions.
- p2m_set_type(): Stores the p2m type in PTE's bits. The storage of types,
which don't fit PTE bits, will be implemented separately later.
- Add detection of Svade extension to properly handle a possible page-fault
if A and D bits aren't set.
PBMT type encoding support:
- Introduces an enum pbmt_type_t to represent the PBMT field values.
- Maps types like p2m_mmio_direct_dev to p2m_mmio_direct_io, others default
to pbmt_pma.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Oleksii Kurochko