The Xen platform PCI device (vendor ID 0x5853) exposed to x86 HVM guests
doesn't have the functionality of a traditional PCI device. The exposed
MMIO BAR is used by some guests (including Linux) as a safe place to map
foreign memory, including the grant table itself.
Traditionally BARs from devices have the uncacheable (UC) cache attribute
from the MTRR, to ensure correct functionality of such devices. hvmloader
mimics this behavior and sets the MTRR attributes of both the low and high
PCI MMIO windows (where BARs of PCI devices reside) as UC in MTRR.
This however causes performance issues for users of the Xen platform PCI
device BAR, as for the purposes of mapping remote memory there's no need to
use the UC attribute. On Intel systems this is worked around by using
iPAT, that allows the hypervisor to force the effective cache attribute of
a p2m entry regardless of the guest PAT value. AMD however doesn't have an
equivalent of iPAT, and guest PAT values are always considered.
Linux commit:
41925b105e34 xen: replace xen_remap() with memremap()
Attempted to mitigate this by forcing mappings of the grant-table to use
the write-back (WB) cache attribute. However Linux memremap() takes MTRRs
into account to calculate which PAT type to use, and seeing the MTRR cache
attribute for the region being UC the PAT also ends up as UC, regardless of
the caller having requested WB.
As a workaround to allow current Linux to map the grant-table as WB using
memremap() introduce an xl.cfg option (xen_platform_pci_bar_uc=0) that can
be used to select whether the Xen platform PCI device BAR will have the UC
attribute in MTRR. Such workaround in hvmloader should also be paired with
a fix for Linux so it attempts to change the MTRR of the Xen platform PCI
device BAR to WB by itself.
Overall, the long term solution would be to provide the guest with a safe
range in the guest physical address space where mappings to foreign pages
can be created.
Some vif throughput performance figures provided by Anthoine from a 8
vCPUs, 4GB of RAM HVM guest(s) running on AMD hardware:
Without this patch:
vm -> dom0: 1.1Gb/s
vm -> vm: 5.0Gb/s
With the patch:
vm -> dom0: 4.5Gb/s
vm -> vm: 7.0Gb/s
Reported-by: Anthoine Bourgeois <anthoine.bourgeois@vates.tech>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Oleksii Kurochko<oleksii.kurochko@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com> # hvmloader
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
We can now easily use __SIZEOF_LONG__ instead. For this to also work in
the test harness, move hvmloader's STR() to common-macros.h.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
We are missing a way to detect whether a user provided a value for
nr_spis equal to 0 or did not provide any value (default is also 0) which
can cause issues when calculated nr_spis is > 0 and the value from domain
config is 0. Fix it by setting default value for nr_spis to newly added
LIBXL_NR_SPIS_DEFAULT i.e. UINT32_MAX (max supported nr of SPIs is 960
anyway).
Fixes: 55d62b8d46 ("tools/arm: Reject configuration with incorrect nr_spis value")
Reported-by: Luca Fancellu <luca.fancellu@arm.com>
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
In order to have a stable interface in user land for using stable
domctl and possibly later sysctl interfaces, add a new library
libxenmanage.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
This was found by the lintian tool (Debian package checker) during
packaging xen for Debian.
Signed-off-by: Maximilian Engelhardt <maxi@daemonizer.de>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
... which converts binary content to hex automatically.
Update libxl to match. No API/ABI change.
This removes a latent libxl bug for cases when the buildid is longer than 4092
bytes.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
... which uses XENVER_extraversion2.
In order to do this sensibly, use manual hypercall buffer handling. Not only
does this avoid an extra bounce buffer (we need to strip the xen_varbuf_t
header anyway), it's also shorter and easlier to follow.
Update libxl and the ocaml stubs to match. No API/ABI change in either.
With this change made, `xl info` can now correctly access a >15 char
extraversion:
# xl info xen_version
4.18-unstable+REALLY LONG EXTRAVERSION
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
Add a new "llc_colors" parameter that defines the LLC color assignment for
a domain. The user can specify one or more color ranges using the same
syntax used everywhere else for color config described in the
documentation.
The parameter is defined as a list of strings that represent the color
ranges.
Documentation is also added.
Golang bindings are regenerated.
Based on original work from: Luca Miccio <lucmiccio@gmail.com>
Signed-off-by: Carlo Nonato <carlo.nonato@minervasys.tech>
Signed-off-by: Marco Solieri <marco.solieri@minervasys.tech>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Instead of having to include xenctrl.h for getting definitions of cpu
barriers, add a dedicated header for that purpose.
Switch the xen-9pfsd daemon to use the new header instead of xenctrl.h.
This is in preparation of making Xenstore independent from libxenctrl.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
The FRED spec architecturalises the Event Type encoding, previously exposed
only in VMCB/VMCS fields.
Introduce the constants in x86-defns.h, making them a bit more concise, and
retire enum x86_event_type.
Take the opportunity to introduce X86_ET_OTHER. It's absence appears to be a
bug in Introspection's Monitor Trap Flag support, when considering VECTORING
events during another VMExit.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When dom0 is PVH, and passthrough a device to dumU, xl will
use the gsi number of device to do a pirq mapping, see
pci_add_dm_done->xc_physdev_map_pirq, but the gsi number is
got from file /sys/bus/pci/devices/<sbdf>/irq, that confuses
irq and gsi, they are in different space and are not equal,
so it will fail when mapping.
To solve this issue, to get the real gsi and add a new function
xc_physdev_map_pirq_gsi to get a free pirq for gsi.
Note: why not use current function xc_physdev_map_pirq, because
it doesn't support to allocate a free pirq, what's more, to
prevent changing it and affecting its callers, so add
xc_physdev_map_pirq_gsi.
Besides, PVH dom0 doesn't have PIRQs flag, it doesn't do
PHYSDEVOP_map_pirq for each gsi. So grant function callstack
pci_add_dm_done->XEN_DOMCTL_irq_permission will fail at function
domain_pirq_to_irq. And old hypercall XEN_DOMCTL_irq_permission
requires passing in pirq, it is not suitable for PVH dom0 that
doesn't have PIRQs to grant irq permission.
To solve this issue, use the another hypercall
XEN_DOMCTL_gsi_permission to grant the permission of irq(
translate from gsi) to dumU when dom0 has no PIRQs.
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Chen Jiqian <Jiqian.Chen@amd.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
On PVH dom0, when passthrough a device to domU, QEMU and xl tools
want to use gsi number to do pirq mapping, see QEMU code
xen_pt_realize->xc_physdev_map_pirq, and xl code
pci_add_dm_done->xc_physdev_map_pirq, but in current codes, the gsi
number is got from file /sys/bus/pci/devices/<sbdf>/irq, that is
wrong, because irq is not equal with gsi, they are in different
spaces, so pirq mapping fails.
And in current codes, there is no method to get gsi for userspace.
For above purpose, add new function to get gsi, and the
corresponding ioctl is implemented on linux kernel side.
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Chen Jiqian <Jiqian.Chen@amd.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Despite its name, libxl_xen_console_read_line() does not read a line,
it fills the buffer with as many characters as fit. Update the
documentation to reflect the real behaviour of the function. Rename
line_r to avoid confusion since it is a pointer to an array of
characters.
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Introduce a --force option to xen-ucode, in order to bypass certain checks
when applying microcode.
Update libxc's xc_microcode_update() to take an extra parameter, and to use
the new XENPF_microcode_update2 hypercall instead.
Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
This was clearly written by a C programmer, rather than a python programmer.
Drop all the useless semi-colons.
The very final line of the script simply references f.close, rather than
calling the function. Switch to using a with: statement, as python does care
about unclosed files if you enable enough warnings.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to
attach (in the future also detach) devices from the provided DT overlay
to domains. Support this by introducing a new "xl dt-overlay" command
and related documentation, i.e. "xl dt-overlay attach. Slightly rework
the command option parsing logic.
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Currently, the number of SPIs allocated to the domain is only
configurable for Dom0less DomUs. Xen domains are supposed to be
platform agnostics and therefore the numbers of SPIs for libxl
guests should not be based on the hardware.
Introduce a new xl config entry for Arm to provide a method for
user to decide the number of SPIs. This would help to avoid
bumping the `config->arch.nr_spis` in libxl everytime there is a
new platform with increased SPI numbers.
Update the doc and the golang bindings accordingly.
Signed-off-by: Henry Wang <xin.wang2@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
... in order to avoid linking against the whole of libsystemd.
Only minimal changes to the upstream copy, to function as a drop-in
replacement for sd_notify() and as a header-only library.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses". Therefore, some
macro definitions should gain additional parentheses to ensure that all
current and future users will be safe with respect to expansions that
can possibly alter the semantics of the passed-in macro parameter.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Introduce a xen-livepatch tool --force option, that's propagated into the
hyerpvisor for livepatch operations. The intention is for the option to be
used to bypass some checks that would otherwise prevent the patch from being
loaded.
Re purpose the pad field in xen_sysctl_livepatch_op to be a flags field that
applies to all livepatch operations. The flag is currently only set by the
hypercall wrappers for the XEN_SYSCTL_LIVEPATCH_UPLOAD operation, as that's so
far the only one where it will be used initially. Other uses can be added as
required.
Note that helpers would set the .pad field to 0, that's been removed since the
structure is already zero initialized at definition.
No functional usages of the new flag introduced in this patch.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Consolidate to a single set of common macros for tools.
MB() will gain another use in libelf, so this movement makes it
available.
Requested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Add a 9pfs device to Xenstore stubdom in order to allow it to do e.g.
logging into a dom0 file.
Use the following parameters for the new device:
- tag = "Xen"
- type = "xen_9pfsd"
- path = "/var/lib/xen/xenstore"
- security-model = "none"
For now don't limit allowed file space or number of files.
Add a new libxl function for adding it similar to the function for
adding the console device.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Make the backend type of 9pfs PV devices configurable. The default is
"qemu" with the related Xenstore backend-side directory being "9pfs".
Add another type "xen_9pfsd" with the related Xenstore backend-side
directory "xen_9pfs".
As additional security features it is possible to specify:
- "max-space" for limiting the maximum space consumed on the filesystem
in MBs
- "max-files" for limiting the maximum number of files in the
filesystem
- "max-open-files" for limiting the maximum number of concurrent open
files
For convenience "auto-delete" is available to let the backend delete the
oldest file of the guest in case otherwise "max-space" or "max-files"
would be violated.
The xen-9pfsd daemon will be started by libxenlight automatically when
the first "xen_9pfs" device is being created.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: George Dunlap <george.dunlap@cloud.com> # Golang bits
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Allow administrators to control whether Xen grant mappings for
the virtio disk devices should be used. By default (when new
parameter is not specified), the existing behavior is retained
(we enable grants if backend-domid != 0).
Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
It's located in libxl_domain_build_info, not libxl_domain_create_info.
Fixes: 45ba9a7d76 ("tools/[lib]xl: Add vmtrace_buf_size parameter")
Signed-off-by: Petr Beneš <w1benny@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
The HVM pirq feature allows routing interrupts from both physical and emulated
devices over event channels, this was done a performance improvement. However
its usage is fully undocumented, and the only reference implementation is in
Linux. It defeats the purpose of local APIC hardware virtualization, because
when using it interrupts avoid the usage of the local APIC altogether.
It has also been reported to not work properly with certain devices, at least
when using some AMD GPUs Linux attempts to route interrupts over event
channels, but Xen doesn't correctly detect such routing, which leads to the
hypervisor complaining with:
(XEN) d15v0: Unsupported MSI delivery mode 7 for Dom15
When MSIs are attempted to be routed over event channels the entry delivery
mode is set to ExtINT, but Xen doesn't detect such routing and attempts to
inject the interrupt following the native MSI path, and the ExtINT delivery
mode is not supported.
Disable HVM PIRQs by default and provide a per-domain option in xl.cfg to
enable such feature. Also for backwards compatibility keep the feature enabled
for any resumed domains that don't have an explicit selection.
Note that the only user of the feature (Linux) is also able to handle native
interrupts fine, as the feature was already not used if Xen reported local APIC
hardware virtualization active.
Link: https://github.com/QubesOS/qubes-issues/issues/7971
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Much like the device model depriv mode, add the same kind of support for the
bootloader. Such feature allows passing a UID as a parameter for the
bootloader to run as, together with the bootloader itself taking the necessary
actions to isolate.
Note that the user to run the bootloader as must have the right permissions to
access the guest disk image (in read mode only), and that the bootloader will
be run in non-interactive mode when restricted.
If enabled bootloader restrict mode will attempt to re-use the user(s) from the
QEMU depriv implementation if no user is provided on the configuration file or
the environment. See docs/features/qemu-deprivilege.pandoc for more
information about how to setup those users.
Bootloader restrict mode is not enabled by default as it requires certain
setup to be done first (setup of the user(s) to use in restrict mode).
This is part of XSA-443 / CVE-2023-34325
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
xc_dt_overlay() sends the device tree binary overlay, size of .dtbo and overlay
operation type i.e. add or remove to xen.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Add xc_set_cpufreq_cppc to allow calling xen_systctl_pm_op
SET_CPUFREQ_CPPC.
Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Add a union and struct so that most of the scaling variables of struct
xen_get_cpufreq_para are within in a binary-compatible layout. This
allows cppc_para to live in the larger union and use uint32_ts - struct
xen_cppc_para will be 10 uint32_t's.
The new scaling struct is 3 * uint32_t + 16 bytes CPUFREQ_NAME_LEN + 4 *
uint32_t for xen_ondemand = 11 uint32_t. That means the old size is
retained, int32_t turbo_enabled doesn't move and it's binary compatible.
The out-of-context memcpy() in xc_get_cpufreq_para() now handles the
copying of the fields removed there.
Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Add -e argument to xl console and pass to new escape_character argument
of libxl_console_exec.
Introduce a new API version to support this new argument and advertise
the new functionality in libxl.h
In libxl_console_exec, there are currently two call sites to execl,
which uses varargs, in order to support optionally passing
'start-notify-fd' to the console client. In order to support passing
the 'escape' argument optionally too, refactor to instead have a single
call site to execv, which has the same behavior but takes an array of
arguments.
If -e is not specified, --escape is not passed to the console client and
the existing value (^]) is used as a default.
Update the xl docs.
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Currently libxl_cpuid_policy_list is an opaque type to the users of
libxl, and internally it's an array of xc_xend_cpuid objects.
Change the type to instead be a structure that contains one array for
CPUID policies, in preparation for it also holding another array for
MSR policies.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Like it's done with CPUID, introduce support for passing MSR values to
xc_cpuid_apply_policy(). The chosen format for expressing MSR policy
data matches the current one used for CPUID. Note that existing
callers of xc_cpuid_apply_policy() can pass NULL as the value for the
newly introduced 'msr' parameter in order to preserve the same
functionality, and in fact that's done in libxl on this patch.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Commit 56a7aaa16b introduced some SVE related code that is protected by
'#if defined(__aarch64__)', the issue is that this doesn't take into
consideration when the toolstack is compiled for an arm32 Dom0 running on
an arm64 platform, it should be able to create SVE enabled guests but with
the current code it's not.
So fix the issue by compiling the code when the toolstack is compiled for
both arm32 and arm64.
Fixes: 56a7aaa16b ("tools: add physinfo arch_capabilities handling for Arm")
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Currently libxl and the x86-emulator tests carry their own versions. Factor
those out into the common macros header so every library can make use of
it. This is required so the following patch can add this macro to a header
used both in Xen and tools/libs.
No functional change.
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Instead of including the same small C source in multiple binaries from
2 source directories, use a header file with inline functions as a
replacement.
As some of the functions are exported by libxenstore, rename the inline
functions from xs_*() do xenstore_*() and add xs_*() wrappers to
libxenstore.
With that no sources required to build libxenstore are left in
tools/xenstore, so the file COPYING can be removed now.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
xs_daemon_tdb() in xs_lib.c is no longer used at all, so it can be
removed. xs_domain_dev() and xs_write_all() are not used by xenstored,
so they can be moved to tools/libs/store/xs.c.
xs_daemon_rootdir() is used by xenstored only and it only calls
xs_daemon_rundir(), so replace its use cases with xs_daemon_rundir()
and remove it from xs_lib.c.
xs_daemon_socket_ro() is needed in libxenstore only, so move it to
tools/libs/store/xs.c.
Move functions used by xenstore-client only to xenstore_client.c.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
MASK_EXTR() and MASK_INSR() are a matching pair. Keep them together.
Drop the pair from x86-emulate.h which includes common-macros.h.
Fixes: 56a7aaa16b ("tools: add physinfo arch_capabilities handling for Arm")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jason Andryuk <jandryuk@gmail.com>
On Arm, the SVE vector length is encoded in arch_capabilities field
of struct xen_sysctl_physinfo, make use of this field in the tools
when building for arm.
Create header arm-arch-capabilities.h to handle the arch_capabilities
field of physinfo for Arm.
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
clang complains about the signed type:
implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
The potential ABI change in libxenvchan is covered by the Xen version based SONAME.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Roger Pau Monne