At the moment, we have two locations selecting restricted permissions, not
very far apart on boot, dependent on opposite answers from using_2M_mapping().
The later location however can shatter superpages if needed, while the former
cannot.
Collect together all the permission adjustments at the slightly later point in
boot, as we likely need to shatter a superpage to support __ro_after_init.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The calculation in __start_xen() for xen_virt_end is an opencoding of
ROUNDUP(_end, 2M). This is __2M_rwdata_end as provided by the linker script.
This corrects the bound calculations in arch_livepatch_init() and
update_xen_mappings() to not enforce 2M alignment when Xen is not compiled
with CONFIG_XEN_ALIGN_2M.
Furthermore, since 52975142d1 ("x86/boot: Create the l2_xenmap[] mappings
dynamically"), there have not been extraneous mappings to delete, meaning that
the call to destroy_xen_mappings() has been a no-op.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
multiboot_ptr should be in __initdata - it is only used on the BSP path.
Furthermore, the .align 8 then .long means that stack_start is misaligned.
Move both into setup.c, which lets the compiler handle the details correctly,
as well as providing proper debug information for them.
Declare stack_start in setup.h and avoid extern-ing it locally in smpboot.c.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The first loop relocates all reachable non-leaf entries in idle_pg_table[],
which includes l2_xenmap[511]'s reference to l1_fixmap_x[].
The second loop relocates only leaf mappings in l2_xenmap[], so update the
skip condition to be opposite to the first loop.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
It has been 4 years since the default load address changed from 1M to 2M, and
_stext ceased residing in l2_xenmap[0]. We should not be inserting an unused
mapping.
To ensure we don't create mappings accidentally, loop from 0 and obey
_PAGE_PRESENT on all entries.
Fixes: 7ed93f3a0d ("x86: change default load address from 1 MiB to 2 MiB")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
GENMASK(30, 21) should be 0x7fe00000. Fixed this in the comment
in bitops.h.
Signed-off-by: Ayan Kumar Halder <ayankuma@xilinx.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
[Tweak text, to put an end to any further bikeshedding]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
I have just forward-ported the CHANGELOG.md updates from the
stable-4.16 branch. But we need a new section for work in this
release cycle.
Signed-off-by: Ian Jackson <iwj@xenproject.org>
Document some of the relevant changes during the 4.16 release cycle.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
(cherry picked from commit e2544a28be)
Currently, the code used to handle and possibly load from the filesystem
modules defined in the DT is allocating and closing the filesystem handle
for each module to be loaded.
To improve the performance, the filesystem handle pointer is passed
through the call stack, requested when it's needed only once and closed
if it was allocated.
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Update libfdt to v1.6.1 of libfdt taken from git://github.com/dgibson/dtc.
This update is done to support device tree overlays.
A few minor changes are done to make it compatible with Xen:
fdt_overlay.c: overlay_fixup_phandle()
Replace strtoul() with simple_strtoul() as strtoul() is not available in
Xen lib and included lib.h.
Change char *endptr to const char *endptr. This change is required for
using simple_strtoul().
libfdt_env.h:
Remaining Xen changes to libfdt_env.h carried over from existing
libfdt (v1.4.0)
Signed-off-by: Vikram Garhwal <fnu.vikram@xilinx.com>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Tested-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
NMI hooking in the crash path has undergone several revisions since its
introduction. What we have now is not sufficiently different from the regular
nmi_callback() mechanism to warrant special casing.
Use set_nmi_callback() directly, and do away with patching a read-only data
structure via a read-write alias. This also means that the
vmx_vmexit_handler() can and should call do_nmi() directly, rather than
indirecting through the exception table to pick up the crash path hook.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The unconditional nmi_callback() call in do_nmi() calls dummy_nmi_callback()
in all cases other than for a few specific and rare tasks (alternative
patching, microcode loading, etc).
Indirect calls are expensive under retpoline, so rearrange the logic to use
NULL as the default, and skip the call entirely in the common case.
While rearranging the code, fold the exit paths.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
This causes NMIs, #DB and #MC to be counted, rather than being reported as 0.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
This is a simple comma separated list, so use the normal form.
* Don't cease processing subsequent elements on an error
* Do report -EINVAL for things like `dom0_nodes=4foo`
* Don't opencode the cmdline_strcmp() helper
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
TLB flushing is a hotpath, and function pointer calls are
expensive (especially under retpoline) for what amounts to an identity
transform on the data. Just pass the vcpu_bitmap bitmap directly.
As we use NULL for all rather than none, introduce a flush_vcpu() helper to
avoid the risk of logical errors from opencoding the expression. This also
means the viridian callers can avoid writing an all-ones bitmap for the
flushing logic to consume.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Function pointers are expensive, and the raw parameter is a constant at the
root of all call trees, meaning that it predicts very well with local branch
history.
Furthermore, the knock-on effects are quite impressive.
$ ../scripts/bloat-o-meter xen-syms-before xen-syms-after
add/remove: 0/4 grow/shrink: 3/9 up/down: 459/-823 (-364)
Function old new delta
__ioapic_write_entry 73 286 +213
__ioapic_read_entry 75 276 +201
save_IO_APIC_setup 182 227 +45
eoi_IO_APIC_irq 241 229 -12
disable_IO_APIC 296 280 -16
mask_IO_APIC_setup 272 240 -32
__io_apic_write 46 - -46
__io_apic_read 46 - -46
io_apic_set_pci_routing 985 930 -55
__io_apic_eoi.part 223 161 -62
io_apic_write 69 - -69
io_apic_read 69 - -69
restore_IO_APIC_setup 325 253 -72
ioapic_guest_write 1413 1333 -80
clear_IO_APIC_pin 447 343 -104
setup_IO_APIC 5148 4988 -160
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
There is no need for this to be an indirect jump at all. Execution always
returns to a specific location, so use a direct jump instead.
Use a named label for the jump. As both 1 and 2 have disappeared as labels,
rename 3 to skip to better describe its purpose.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
There are several cases where the act of interrupting a remote processor has
the required side effect. Explicitly allow NULL function pointers so the
calling code doesn't have to provide a stub implementation.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
CONFIG_ACPI_INTERPRETER does not get defined anywhere, the enclosed code
wouldn't build, and the default-to-phys logic works differently anyway
(see genapic/bigsmp.c:probe_bigsmp()).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The name hasn't been appropriate for a long time: It covers not only
command line overrides, but also x2APIC pre-enabled state.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The function does nothing but return success. Simply treat absence of a
probe hook to mean just this. This then eliminates the (purely
theoretical at this point) risk of trying to call through
apic_x2apic_{cluster,phys}'s respective NULL pointers.
While doing this also eliminate generic_apic_probe()'s "changed"
variable: apic_probe[]'s default entry will now be used unconditionally
in yet more obvious a way, such that separately setting genapic from
apic_default is (hopefully) no longer justified. Yet that was the main
purpose of the variable.
To help prove that apic_default's probe() hook doesn't get used
elsewhere, further make apic_probe[] static at this occasion.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The hook functions have been empty for a very long time, if not
(according to git history) forever. Ditch them alongside the then empty
mach_mpparse.h instances and the then unused APICFUNC() macro.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The hook functions have been empty forever (x2APIC) or issuing merely a
printk() for a long time (xAPIC). Since that printk() is (a) generally
useful (i.e. also in the x2APIC case) and (b) would better only be
issued once the final APIC driver to use was determined, move (and
generalize) it into connect_bsp_APIC().
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Xen already collects CPUID.0x80000007.edx by default, meaning that we can
refer to per-cpu data directly. This also avoids the need IPI the onlining
CPU to identify whether Core Performance Boost is available.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Currently, each feature_detect() (called on CPU add) hook for both cpufreq
drivers duplicates cpu_has_aperfmperf in a per-cpu datastructure, and edits
cpufreq_driver.getavg to point at get_measured_perf().
As all parts of this are vendor-neutral, drop the function pointer and
duplicated boolean, and call get_measured_perf() directly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
powernow_register_driver() is currently written with a K&R type definition;
I'm surprised that compilers don't object to a mismatch with its declaration,
which is written in an ANSI-C compatible way.
Furthermore, its sole caller is cpufreq_driver_init() which is a pre-smp
initcall. There are no other online CPUs, and even if there were, checking
the BSP's CPUID data $N times is pointless. Simplify registration to only
look at the BSP.
While at it, drop obviously unused includes. Also rewrite the expression in
cpufreq_driver_init() for clarity.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The current xsm_fixup_ops() is just shy of a full page when compiled, and very
fragile to NULL function pointer errors.
Address both of these issues with a minor piece of structure (ab)use.
Introduce dummy_ops, and fix up the provided xsm_ops pointer by treating both
as an array of unsigned longs.
The compiled size improvement speaks for itself:
$ ../scripts/bloat-o-meter xen-syms-before xen-syms-after
add/remove: 1/0 grow/shrink: 0/1 up/down: 712/-3897 (-3185)
Function old new delta
dummy_ops - 712 +712
xsm_fixup_ops 3987 90 -3897
and there is an additional safety check that will make it obvious during
development if there is an issue with the fallback handling.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
With alternative_call() capable of handling compound types, the three
remaining hooks can be optimised at boot time too.
Fixes: 164a0b9653 ("xsm: refactor xsm_ops handling")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Replace the conditional operator in ALT_CALL_ARG(), which was intended
to limit usable types to scalar ones, by a size check. Some restriction
here is necessary to make sure we don't violate the ABI's calling
conventions, but limiting to scalar types was both too restrictive
(disallowing e.g. guest handles) and too permissive (allowing e.g.
__int128_t).
Note that there was some anomaly with that conditional operator anyway:
Something - I don't recall what - made it impossible to omit the middle
operand.
Code-generation-wise this has the effect of removing certain zero- or
sign-extending in some altcall invocations. This ought to be fine as the
ABI doesn't require sub-sizeof(int) values to be extended, except when
passed through an ellipsis. No functions subject to altcall patching has
a variable number of arguments, though.
Requested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Unfortunately this triggers -Werror=sizeof-array-argument on some versions of
GCC, so alter xsm_{alloc,free}_security_evtchns() to use a pointer rather than
array parameter.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
OSSTest has identified a 3rd regression caused by this change. Migration
between Xen 4.15 and 4.16 on the nocera pair of machines (AMD Opteron 4133)
fails with:
xc: error: Failed to set CPUID policy: leaf 00000000, subleaf ffffffff, msr ffffffff (22 = Invalid argument): Internal error
xc: error: Restore failed (22 = Invalid argument): Internal error
which is a safety check to prevent resuming the guest when the CPUID data has
been truncated. The problem is caused by shrinking of the max policies, which
is an ABI that needs handling compatibly between different versions of Xen.
Furthermore, shrinking of the default policies also breaks things in some
cases, because certain cpuid= settings in a VM config file which used to work
will now be refused. Also external toolstacks that attempt to set the CPUID
policy from a featureset might now see some filled leaves not reachable due to
the shrinking done to the default domain policy before applying the
featureset.
This reverts commit 540d911c28, as well as the
partial fix attempt in 81da2b544c (which added
one new case where cpuid= settings might not apply correctly) and restores the
same behaviour as Xen 4.15.
Fixes: 540d911c28 ("x86/CPUID: shrink max_{,sub}leaf fields according to actual leaf contents")
Fixes: 81da2b544c ("x86/cpuid: prevent shrinking migrated policies max leaves")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
As of commit 6773b1a758 ("VT-d: Don't assume register-based
invalidation is always supported") we don't (try to) use register based
invalidation anymore when that's not supported by hardware. Hence
there's also no point in the respective check, avoiding pointless IOMMU
initialization failure. After all the spec (version 3.3 at the time of
writing) doesn't say what the respective Extended Capability Register
field would contain in such a case.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
All our present implementation requires is that the range fully fits
in a single page. No need to exclude the case of the last register
extending right to the end of that page.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Bit 0 of SAGAW in the capability register has become reserved at or
before spec version 2.2. Treat it as such. Replace the effective open-
coding of find_first_set_bit(). Adjust local variable types.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Both hvcall_flush_ex() and hvcall_ipi_ex() update "size" without
subsequently using the value; future compilers may warn about such.
Alongside dropping the updates, shrink the variables' scopes to
demonstrate that there are no outer scope uses.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
Both the wrong use of HV_STATUS_* and the return type of
hv_vpset_to_vpmask() can lead to viridian_hypercall()'s
ASSERT_UNREACHABLE() triggering when translating error codes from Xen
to Viridian representation.
Fixes: b4124682db ("viridian: add ExProcessorMasks variants of the flush hypercalls")
Fixes: 9afa867d42 ("viridian: add ExProcessorMasks variant of the IPI hypercall")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul@xen.org>
The designated maintainer email address for the remus entry is
bouncing, so remove it and declare the entry as orphaned as there's no
other maintainer.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <iwj@xenproject.org>
While domain_context_mapping() invokes domain_context_unmap() in a sub-
case of handling DEV_TYPE_PCI when encountering an error, thus avoiding
a leak, individual calls to domain_context_mapping_one() aren't
similarly covered. Such a leak might persist until domain destruction.
Leverage that these cases can be recognized by pdev being non-NULL.
Fixes: dec403cc66 ("VT-d: fix iommu_domid for PCI/PCIx devices assignment")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Merely setting bit 0 in the bitmap is insufficient, as then Dom0 will
still have DID 0 allocated to it, because of the zero-filling of
domid_map[]. Set slot 0 to DOMID_INVALID to keep DID 0 from getting
used.
Fixes: b9c20c7878 ("VT-d: per-iommu domain-id")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
The quarantine domain doesn't count as a DomU, as it won't itself
trigger any bad behavior. The workaround only needs enabling when an
actual DomU is about to gain control of a device. This then also means
enabling of the workaround can be deferred until immediately ahead of
the call to domain_context_mapping(). While there also stop open-coding
is_hardware_domain().
Fixes: 319f9a0ba9 ("passthrough: quarantine PCI devices")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
cap_super_page_val() and cap_super_offset() are unused (apart from the
latter using the former). I don't see how cap_super_offset() can be
useful in its current shape: cap_super_page_val()'s result is not an
lvalue and hence can't have its address taken. Plus a user would have
to check the capability register field is non-zero, for
find_first_bit() (or find_first_set_bit(), if suitably corrected) to be
valid in the first place. Yet as per the spec when the field is non-zero
the low bit would always be set, so the result would be independent of
the actual value the field holds.
Further zap cap_sps_512gb() and cap_sps_1tb(). While earlier versions
of the spec had things spelled out that way, the current version marks
the two bits as reserved. And "48-bit offset to page frame" wasn't in
line with 1Tb pages anyway - clearly 256Tb pages would have been meant
here.
Finally properly parenthesize parameter uses in the remaining two
macros.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Instead of listing each single file with .s or .i suffixes in
.gitignore use pattern based entries. Restrict those to the xen
directory as we have e.g. tools/libs/stat/bindings/swig/xenstat.i in
our tree.
Below xen the pattern based entries are fine, as we have pattern rules
for creating *.s and *.i files in xen/Rules.mk.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
hvm_memory_op() should take an unsigned long as cmd, like
do_memory_op().
As hvm_memory_op() is basically just calling do_memory_op() (or
compat_memory_op()) passing through the parameters the cmd parameter
should have no smaller size than that of the called functions.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
It's not really needed and has been misleading me more than once to try
and spot its "actual" use(s). It should really have been dropped when
the 32-bit specific logic was purged from here.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
M2P and PoD stats need to remain in sync with P2M; if an update succeeds
only partially, respective adjustments need to be made. If updates get
made before the call, they may also need undoing upon complete failure
(i.e. including the single-page case).
Log-dirty state would better also be kept in sync.
Note that the change to set_typed_p2m_entry() may not be strictly
necessary (due to the order restriction enforced near the top of the
function), but is being kept here to be on the safe side.
This is CVE-2021-28705 and CVE-2021-28709 / XSA-389.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Andrew Cooper
Ayan Kumar Halder
Ian Jackson
Roger Pau Monne
Luca Fancellu
Vikram Garhwal
Jan Beulich
Juergen Gross