The EFI_GET_TIME implementation is well known to be broken for many firmware
implementations, for Xen the result on such implementations are:
----[ Xen-4.19-unstable x86_64 debug=y Tainted: C ]----
CPU: 0
RIP: e008:[<0000000062ccfa70>] 0000000062ccfa70
[...]
Xen call trace:
[<0000000062ccfa70>] R 0000000062ccfa70
[<00000000732e9a3f>] S 00000000732e9a3f
[<ffff82d04034f34f>] F arch/x86/time.c#get_cmos_time+0x1b3/0x26e
[<ffff82d04045926f>] F init_xen_time+0x28/0xa4
[<ffff82d040454bc4>] F __start_xen+0x1ee7/0x2578
[<ffff82d040203334>] F __high_start+0x94/0xa0
Pagetable walk from 0000000062ccfa70:
L4[0x000] = 000000207ef1c063 ffffffffffffffff
L3[0x001] = 000000005d6c0063 ffffffffffffffff
L2[0x116] = 8000000062c001e3 ffffffffffffffff (PSE)
****************************************
Panic on CPU 0:
FATAL PAGE FAULT
[error_code=0011]
Faulting linear address: 0000000062ccfa70
****************************************
Swap the preference to default to CMOS first, and EFI later, in an attempt to
use EFI_GET_TIME as a last resort option only. Note that Linux for example
doesn't allow calling the get_time method, and instead provides a dummy handler
that unconditionally returns EFI_UNSUPPORTED on x86-64.
Such change in the preferences requires some re-arranging of the function
logic, so that panic messages with workaround suggestions are suitably printed.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-By: Oleksii Kurochko<oleksii.kurochko@gmail.com>
Release-Acked-by: Oleksii Kurochko<oleksii.kurochko@gmail.com>
It's definitely worth mentioning as one of the most notable feature on
Arm this release.
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Decompressing the kernel is necessary to inspect the ELF notes, but the
dombuilder will gunzip() secondary modules too. Specifically gunzip(), no
other decompression algorithms.
This may have been necessary in the dim and distant past, but it is broken
today. Linux specifically supports concatenating CPIO fragments of differing
compressions, and any attempt to interpret it with a single algorithm may
corrupt later parts.
This was an unexpected discovery while trying to test Xen's gunzip()
logic (Xen as a PVH guest, with a gzipped XTF kernel as dom0).
Interpreting secondary modules should be left as an exercise to the guest.
This reduces work done in dom0.
This is not expected to cause a practical difference to guests these days.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The xl subcommands are generic, but only built when
LIBXL_HAVE_NO_SUSPEND_RESUME is false. Currently that is only x86, so
list them there.
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Oleksii Kurochko<oleksii.kurochko@gmail.com>
There are features in the forthcoming patches which are dependent on
MPU. For eg fixed start address.
Also, some of the Xen features (eg STATIC_MEMORY) will be selected
by the MPU configuration.
Thus, this patch introduces a choice between MMU and MPU for the type
of memory management system. By default, MMU is selected.
MPU is now gated by UNSUPPORTED.
Update SUPPORT.md to state that the support for MPU is experimental.
Also updated CHANGELOG.md as well.
Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
With the introduction of mixed x2APIC mode (using cluster addressing for
IPIs and physical for external interrupts) the use of pure cluster mode
doesn't have any benefit.
Remove the mode itself, leaving only the code required for logical
addressing when sending IPIs.
Resolves: https://gitlab.com/xen-project/xen/-/issues/189
Signed-off-by: Matthew Barnes <matthew.barnes@cloud.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Pass xen-ucode flags to do low level checks on microcode version and utilize
it to allow for microcode downgrade or reapply the same version of the
microcode. ucode_force is required to be passed to a low level Intel and AMD
for version checks to be done.
While adding ucode_force, opt_ucode_allow_same was removed. Remove
opt_ucode_allow_same from documentation.
Update CHANGELOG.md for opt_ucode_allow_same removal.
Signed-off-by: Fouad Hilly <fouad.hilly@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The current logic to chose the preferred reboot method is based on the mode Xen
has been booted into, so if the box is booted from UEFI, the preferred reboot
method will be to use the ResetSystem() run time service call.
However, that method seems to be widely untested, and quite often leads to a
result similar to:
Hardware Dom0 shutdown: rebooting machine
----[ Xen-4.18-unstable x86_64 debug=y Tainted: C ]----
CPU: 0
RIP: e008:[<0000000000000017>] 0000000000000017
RFLAGS: 0000000000010202 CONTEXT: hypervisor
[...]
Xen call trace:
[<0000000000000017>] R 0000000000000017
[<ffff83207eff7b50>] S ffff83207eff7b50
[<ffff82d0403525aa>] F machine_restart+0x1da/0x261
[<ffff82d04035263c>] F apic_wait_icr_idle+0/0x37
[<ffff82d040233689>] F smp_call_function_interrupt+0xc7/0xcb
[<ffff82d040352f05>] F call_function_interrupt+0x20/0x34
[<ffff82d04033b0d5>] F do_IRQ+0x150/0x6f3
[<ffff82d0402018c2>] F common_interrupt+0x132/0x140
[<ffff82d040283d33>] F arch/x86/acpi/cpu_idle.c#acpi_idle_do_entry+0x113/0x129
[<ffff82d04028436c>] F arch/x86/acpi/cpu_idle.c#acpi_processor_idle+0x3eb/0x5f7
[<ffff82d04032a549>] F arch/x86/domain.c#idle_loop+0xec/0xee
****************************************
Panic on CPU 0:
FATAL TRAP: vector = 6 (invalid opcode)
****************************************
Which in most cases does lead to a reboot, however that's unreliable.
Change the default reboot preference to prefer ACPI over UEFI if available and
not in reduced hardware mode.
This is in line to what Linux does, so it's unlikely to cause issues on current
and future hardware, since there's a much higher chance of vendors testing
hardware with Linux rather than Xen.
Add a special case for one Acer model that does require being rebooted using
ResetSystem(). See Linux commit 0082517fa4bce for rationale.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Do as was decided in Lisbon. Reportedly Xen hasn't been working very
well on those processors anyway.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
PVH dom0 is functionally very similar to PVH domU except for the domain
builder and the added set of hypercalls available to it.
The main concern with declaring it "Supported" is the lack of some features
when compared to classic PV dom0, hence switch it's status to supported with
caveats. List the known missing features, there might be more features missing
or not working as expected apart from the ones listed.
Note there's some (limited) PVH dom0 testing on both osstest and gitlab.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Iterate over the p2m up to the maximum recorded gfn and remove any foreign
mappings, in order to drop the underlying page references and thus don't keep
extra page references if a domain is destroyed while still having foreign
mappings on it's p2m.
The logic is similar to the one used on Arm.
Note that foreign mappings cannot be created by guests that have altp2m or
nested HVM enabled, as p2ms different than the host one are not currently
scrubbed when destroyed in order to drop references to any foreign maps.
It's unclear whether the right solution is to take an extra reference when
foreign maps are added to p2ms different than the host one, or just rely on the
host p2m already having a reference. The mapping being removed from the host
p2m should cause it to be dropped on all domain p2ms.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
There are no more users, and we want to disuade people from introducing new
users just for sd_notify() and friends. Drop the dependency.
We still want the overall --with{,out}-systemd to gate the generation of the
service/unit/mount/etc files.
Rerun autogen.sh, and mark the dependency as removed in the build containers.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
With lock handling now allowing up to 16384 cpus (spinlocks can handle
65535 cpus, rwlocks can handle 16384 cpus), raise the allowed limit for
the number of cpus to be configured to 16383.
The new limit is imposed by IOMMU_CMD_BUFFER_MAX_ENTRIES and
QINVAL_MAX_ENTRY_NR required to be larger than 2 * CONFIG_NR_CPUS.
Add a support limit of physical CPUs to SUPPORT.md (4096 on x86, 128
on ARM).
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
xl/libxl only applies vkb=[] to PV & PVH guests. HVM gets only a single
vkb by default, but that can be disabled by the vkb_device boolean.
Notably the HVM vkb cannot be configured, so feature-abs-pointer or the
backend-type cannot be specified.
Re-arrange the logic so that vkb=[] is handled regardless of domain
type. If vkb is empty or unspecified, follow the vkb_device boolean for
HVMs. Nothing changes for PVH & PV. HVMs can now get a configured vkb
instead of just the default one.
The chance for regression is an HVM config with
vkb=["$something"]
vkb_device=false
Which would now get a vkb.
This is useful for vGlass which provides a VKB to HVMs. vGlass wants to
specify feature-abs-pointer, but that is racily written by vGlass
instead of coming through the xl.cfg. Unhelpfully, Linux xen-kbdfront
reads the backend nodes without checking that the backend is in
InitWait.
Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
Signed-off-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Add an entry to CHANGELOG.md regarding the new xen-9pfsd daemon.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
In 13y of working on Xen, I've never seen seen it used. The implementation
was introduced (commit b69f92f301, Jul 28 2004) with known issues such as:
/* Resuming after we've stopped used to work, but more through luck
than any actual intention. It doesn't at the moment. */
which appear to have gone unfixed for the 20 years since.
Nowadays there are more robust ways of inspecting crashed state, such as a
kexec crash kernel, or running Xen in a VM.
This will allow us to clean up some hooks around the codebase which are
proving awkward for other tasks.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Char.lowercase got removed in OCaml 5.0 (it has been deprecated since 2014),
and doesn't build any more.
Char.lowercase_ascii has existed since OCaml 4.03, so that is the new
minimum version for oxenstored.
However, OCaml 4.05 is the oldest new-enough version found in common distros,
so pick this as a baseline.
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
[Update CHANGELOG.md]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
The current loop that iterates from 0 to the maximum RAM address in order to
setup the IOMMU mappings is highly inefficient, and it will get worse as the
amount of RAM increases. It's also not accounting for any reserved regions
past the last RAM address.
Instead of iterating over memory addresses, iterate over the memory map regions
and use a rangeset in order to keep track of which ranges need to be identity
mapped in the hardware domain physical address space.
On an AMD EPYC 7452 with 512GiB of RAM, the time to execute
arch_iommu_hwdom_init() in nanoseconds is:
x old
+ new
N Min Max Median Avg Stddev
x 5 2.2364154e+10 2.338244e+10 2.2474685e+10 2.2622409e+10 4.2949869e+08
+ 5 1025012 1033036 1026188 1028276.2 3623.1194
Difference at 95.0% confidence
-2.26214e+10 +/- 4.42931e+08
-99.9955% +/- 9.05152e-05%
(Student's t, pooled s = 3.03701e+08)
Execution time of arch_iommu_hwdom_init() goes down from ~22s to ~0.001s.
Note there's a change for HVM domains (ie: PVH dom0) that get switched to
create the p2m mappings using map_mmio_regions() instead of
p2m_add_identity_entry(), so that ranges can be mapped with a single function
call if possible. Note that the interface of map_mmio_regions() doesn't
allow creating read-only mappings, but so far there are no such mappings
created for PVH dom0 in arch_iommu_hwdom_init().
No change intended in the resulting mappings that a hardware domain gets.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The HVM pirq feature allows routing interrupts from both physical and emulated
devices over event channels, this was done a performance improvement. However
its usage is fully undocumented, and the only reference implementation is in
Linux. It defeats the purpose of local APIC hardware virtualization, because
when using it interrupts avoid the usage of the local APIC altogether.
It has also been reported to not work properly with certain devices, at least
when using some AMD GPUs Linux attempts to route interrupts over event
channels, but Xen doesn't correctly detect such routing, which leads to the
hypervisor complaining with:
(XEN) d15v0: Unsupported MSI delivery mode 7 for Dom15
When MSIs are attempted to be routed over event channels the entry delivery
mode is set to ExtINT, but Xen doesn't detect such routing and attempts to
inject the interrupt following the native MSI path, and the ExtINT delivery
mode is not supported.
Disable HVM PIRQs by default and provide a per-domain option in xl.cfg to
enable such feature. Also for backwards compatibility keep the feature enabled
for any resumed domains that don't have an explicit selection.
Note that the only user of the feature (Linux) is also able to handle native
interrupts fine, as the feature was already not used if Xen reported local APIC
hardware virtualization active.
Link: https://github.com/QubesOS/qubes-issues/issues/7971
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
This was deprecated in commit 10c88f1c18 ("tools: Install pv bootloaders in
libexec rather than /usr/bin") and caused to warn in commit c31d6a7ee2
("libxl: Warn that /usr/bin/pygrub is deprecated"), both in 2012.
New versions of RPM warn against absolute symlinks, so take out fully, 11
years later.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Flexible arrays in public headers can be problematic with some
compilers.
With XEN_FLEX_ARRAY_DIM there is a mechanism available to deal with
this issue, but care must be taken to not change the affected structs
in an incompatible way.
So bump __XEN_LATEST_INTERFACE_VERSION__ and introduce a new macro
XENPV_FLEX_ARRAY_DIM which will be XENPV_FLEX_ARRAY_DIM with the
interface version being new enough and "1" (the value used today in
the affected headers) when the interface version is an old one.
Replace the arr[1] instances (this includes the ones seen to be
problematic in recent Linux kernels [1]) with arr[XENPV_FLEX_ARRAY_DIM]
in order to avoid compilation errors.
[1]: https://bugzilla.kernel.org/show_bug.cgi?id=217693
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Henry Wang <Henry.Wang@arm.com> # CHANGELOG
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
In order to build caml-stubdom, it must be explicitly enabled via
"configure --enable-caml-stubdom". The build process is failing due to
stubdom/ocaml.patch failing to apply. Since the patched file has been
modified in 2014 the last time, it seems nobody cares for caml-stubdom
since at least then.
Remove caml-stubdom from the build system.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
Acked-by: Henry Wang <Henry.Wang@arm.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The current implementation of x2APIC requires to either use Cluster Logical or
Physical mode for all interrupts. However the selection of Physical vs Logical
is not done at APIC setup, an APIC can be addressed both in Physical or Logical
destination modes concurrently.
Introduce a new x2APIC mode called Mixed, which uses Logical Cluster mode for
IPIs, and Physical mode for external interrupts, thus attempting to use the
best method for each interrupt type.
Using Physical mode for external interrupts allows more vectors to be used, and
interrupt balancing to be more accurate.
Using Logical Cluster mode for IPIs allows fewer accesses to the ICR register
when sending those, as multiple CPUs can be targeted with a single ICR register
write.
A simple test calling flush_tlb_all() 10000 times on a tight loop on AMD EPYC
9754 with 512 CPUs gives the following figures in nano seconds:
x mixed
+ phys
* cluster
N Min Max Median Avg Stddev
x 25 3.5131328e+08 3.5716441e+08 3.5410987e+08 3.5432659e+08 1566737.4
+ 12 1.231082e+09 1.238824e+09 1.2370528e+09 1.2357981e+09 2853892.9
Difference at 95.0% confidence
8.81472e+08 +/- 1.46849e+06
248.774% +/- 0.96566%
(Student's t, pooled s = 2.05985e+06)
* 11 3.5099276e+08 3.5561459e+08 3.5461234e+08 3.5415668e+08 1415071.9
No difference proven at 95.0% confidence
So Mixed has no difference when compared to Cluster mode, and Physical mode is
248% slower when compared to either Mixed or Cluster modes with a 95%
confidence.
Note that Xen uses Cluster mode by default, and hence is already using the
fastest way for IPI delivery at the cost of reducing the amount of vectors
available system-wide.
Make the newly introduced mode the default one.
Note the printing of the APIC addressing mode done in connect_bsp_APIC() has
been removed, as with the newly introduced mixed mode this would require more
fine grained printing, or else would be incorrect. The addressing mode can
already be derived from the APIC driver in use, which is printed by different
helpers.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Henry Wang <Henry.Wang@arm.com>
Collect all x86 and ARM changes together instead of having them scattered.
Tweak grammar as necessary.
No change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Henry Wang <Henry.Wang@arm.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
Compared to "xenbits.xen.org", "xenbits.xenproject.org" appeared
later as a name, with the intention of becoming the canonical one.
Therefore, this commit unifies all the links to use "xenproject"
in the links.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The script heavily relies on the urlgrabber python module, which doesn't seem
to be packaged by all distros; it's missing from newer Debian versions at
least.
Also the usage of the commands module has been deprecated since Python 2.6, and
removed in Python 3, so the code would need to be re-written to not rely on
urlgrabber and the commands modules.
Arguably the purpose of the xenpvnetboot bootloader can also be achieved with
an isolated script that fetches the kernel and ramdisk before attempting to
launch the domain, without having to run in libxl context. There are no xl.cfg
options consumed by the bootloader apart from the base location and the
subpaths of the kernel and ramdisk to fetch.
Any interested parties in keeping such functionality are free to submit an
updated script that works with Python 3.
Resolves: xen-project/xen#172
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
XENFEAT_runstate_phys_area is exposed to all architectures, while
XENFEAT_vcpu_time_phys_area is currently only implemented for x86, and hence
the feature flag is also only exposed on x86.
Additionally add dummy guards with TODOs in the respective hypercall
implementations, to signal the intention to control the availability of those
hypercalls on a guest-by-guest basis from the toolstack.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
The feature is not commonly used, and we don't have hardware to test it,
not in OSSTest, not in Gitlab, and not even ad-hoc manually by community
members. We could use QEMU to test it, but even that it is known not to
work on our end.
Also take the opportunity to rename the feature to "ACPI CPU Hotplug"
for clarity.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Introduce a config option where the user can enable support for adding/removing
device tree nodes using a device tree binary overlay.
Update SUPPORT.md and CHANGELOG.md to state the Device Tree Overlays support for
Arm.
Signed-off-by: Vikram Garhwal <vikram.garhwal@amd.com>
Acked-by: Henry Wang <Henry.Wang@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
This option is particularly dubious as Xen does not use Protection Keys, owing
to the sharing of pagetables with PV guests. All this option does is hide PKU
by default from HVM guests, and is therefore redundant with the more generic
cpuid=no-pku.
The variable ought to be in __initdata given it's single user, but deleting it
entirely looks to be a better course of action.
Visibility of PKU to guests should be handled via its vm.cfg file.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Henry Wang <Henry.Wang@arm.com>
Arm now can use the "dom0=" Xen command line option and the support
for guests running SVE instructions is added, put entries in the
changelog.
Mention the "Tech Preview" status and add an entry in SUPPORT.md
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: Henry Wang <Henry.Wang@arm.com> # CHANGELOG
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Oleksii Kurochko