The new helpers make it easier to read the code by abstracting the way to
set/get an MFN from/to an LPAE entry. The helpers are using "walk" as the
bits are common across different LPAE stages.
At the same time, use the new helpers to replace the various open-coding
place.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
A top level "make build", as used e.g. by osstest, wants to build all
"all" targets in enabled tools subdirectories, which by default also
includes the emulator test harness. The use of, in particular, AVX512
insns in, again in particular, test_x86_emulator.c causes this build to
fail though when the compiler is not new enough. Take a big hammer and
suppress the default harness build altogether when any of the extensions
used is not supported by the specified (or defaulted to) compiler.
Leave the "run" target alone though: While some of the test code blobs
may fail to build with older compilers, as long as the main executable
can be built some limited testing can still be done.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Now that idle scrub is the default option, all memory is marked as dirty
and alloc_domheap_pages() will do eager scrubbing by default. This can
lead to longer Dom0 construction and potentially to a watchdog timeout,
especially on older H/W (e.g. Harpertown).
Pass MEMF_no_scrub to optimise this process since there is little point
in scrubbing memory for Dom0.
Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Load balancing, when happening, at the end of a "scheduler epoch", can
trigger vcpu migration, which in its turn may call runq_tickle(). If the
cpu where this happens was idle, but we're now going to schedule a vcpu
on it, let's update the runq's idle cpus mask accordingly _before_ doing
load balancing.
Not doing that, in fact, may cause runq_tickle() to think that the cpu
is still idle, and tickle it to go pick up a vcpu from the runqueue,
which might be wrong/unideal.
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Currently randconfig tests are more likely to fail than to succeed
because of a bug in xen's build system: symbols-dummy.o's dependency
is wrong, which causes it to not get rebuild between runs, which
eventually causes linking to fail. There may also be other corner
cases we haven't discovered.
The fix is not straightforward. For now, make sure the tree is cleaned
properly between builds so we don't see random failures in Gitlab CI.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
For doing load balancing between runqueues, we check the load of each
runqueue, select the one more "distant" than our own load, and then take
the proper runq lock and attempt vcpu migrations.
If we fail to take such lock, we try again, and the idea was to give up
and bail if, during the checking phase, we can't take the lock of any
runqueue (check the comment near to the 'goto retry;', in the middle of
balance_load())
However, the variable that controls the "give up and bail" part, is not
reset upon retries. Therefore, provided we did manage to check the load of
at least one runqueue during the first pass, if we can't get any runq lock,
we don't bail, but we try again taking the lock of that same runqueue
(and that may even more than once).
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Move p2m_mem_access_sanity_check() from the asm-x86/mem_access.h
header, where it currently is declared inline, to
arch/x86/mm/mem_access.c. This allows source code that includes it
directly, or indirectly (such as xen/mem_access.h), to not worry
about also including sched.h for is_hvm_domain(). Including
xen/mem_access.h is useful for code wanting to use p2m_access_t.
Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
For domain heap pages assigned to a domain dropping the page reference
tied to PGC_allocated may not drop the last reference, as otherwise the
test_and_clear_bit() might already act on an unowned page.
Work around this where possible, but the need to acquire extra page
references is a fair hint that references should have been acquired in
other places instead.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
In particular sh_oos_audit() has become stale due to changes elsewhere,
and the need for adjustment was not noticed because both "full audit"
flags are off in both release and debug builds. Switch away from pre-
processor conditionals, thus exposing the code to the compiler at all
times. This obviously requires correcting the accumulated issues with
the so far hidden code.
Note that shadow_audit_tables() now also gains an effect with "full
entry audit" mode disabled; the prior code structure suggests that this
was originally intended anyway.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
To mitigate Spectre v2, Xen has been fixed with a software fix, namely
using retpoline sequences generated by the compiler. This way, indirect
branches are protected against the attack.
However, the retpoline sequence comes with a slow down. To make up for
this, we propose to avoid jump tables in the first place. Without the
retpoline sequences, this code would be less efficient. However, when
retpoline is enabled, this actually results in a slight performance
improvement.
This change might become irrelevant once the compiler starts avoiding
jump tables in case retpolines are used:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86952
Reported-by: Julian Stecklina <jsteckli@amazon.de>
Reported-by: Pawel Wieczorkiewicz <wipawel@amazon.de>
Signed-off-by: Norbert Manthey <nmanthey@amazon.de>
Acked-by: Jan Beulich <jbeulich@suse.com>
...and re-name them to iommu_map/unmap() since they no longer necessarily
operate on a single page.
The P2M code currently contains many loops to deal with the fact that,
while it may be require to handle page orders greater than 0, the
IOMMU map and unmap functions do not.
This patch adds a page_order parameter to those functions and implements
the necessary loops within. This allows the P2M code to be substantially
simplified.
This patch also adds emacs boilerplate to xen/iommu.h to avoid tabbing
problem.
NOTE: This patch does not modify the underlying vendor IOMMU
implementations to deal with more than a single page at once.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
This patch introduces a new test stage into the pipeline and provides
a simple QEMU based smoke test.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
... so that it can be passed on to test stage.
Note that xen is only extracted for x86_64 build since others may not
have that. Use a directory to account for possibly different file
names on Arm.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
This aids troubleshooting when we notice a failure in the default
configuration.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
This file gets installed to the host system.
This patch cleans it up: 1. remove things that don't exist anymore; 2.
change xm to xl; 3. fix xen-devel list address; 4. add things that are
missing; 5. delete trailing whitespaces.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Without this change valgrind can not decide what variant of
xen_domctl_createdomain is provided as input.
Fixes commit 4a83497635 ("xen/domctl: Merge set_max_evtchn into createdomain")
Fixes commit a903bf5233 ("tools: Pass grant table limits to XEN_DOMCTL_set_gnttab_limits")
Fixes commit ae8b8bc599 ("xen/domctl: Remove XEN_DOMCTL_set_gnttab_limits")
Fixes commit 4737fa52ce ("tools: Pass max_vcpus to XEN_DOMCTL_createdomain")
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Jan Beulich <jbeulich@suse.com>
This eliminates a separate case block here, and allows to get away with
fewer new ones when adding AVX512 vector shifts.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Note: vpadd* / vpsub* et al are put at seemingly the wrong slot of the
big switch(). This is in anticipation of adding e.g. vpunpck* to those
groups (see the legacy/VEX encoded case labels nearby to support this).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Include VPTEST{,N}M{B,D,Q,W} as once again possibly used by the compiler
for comparison against all-zero vectors.
Also table entries for a few more insns get their .d8s field set right
away, again in order to not split and later re-combine the groups.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Plus vpternlog{d,q} as being extensively used by the compiler, in order
to facilitate test enabling in the harness as soon as possible. Also the
twobyte_table[] entries for a few more insns get their .d8s field set
right away, in order to not split and later re-combine the groups.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
V{,U}COMIS{S,D} to follow later.
Also correct the AVX counterpart's comment.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The more generic AVX512 implementation allows quite a bit of insn-
specific code to be dropped/shared.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
In order to be able to verify the 32-bit variant builds and runs,
introduce a respective target (and the necessary other adjustments).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Besides the already existing tests (which are going to be extended once
respective ISA extension support is complete), let's also ensure for
every individual insn that their Disp8 scaling (and memory access width)
are correct.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Note: SDM Vol 2 rev 067 is not really consistent about EVEX.L'L for LIG
insns - the only place where this is made explicit is a table in
the section titled "Vector Length Orthogonality": While they
tolerate 0, 1, and 2, a value of 3 uniformly leads to #UD.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
This abstracts away the debug/release coverage using both
ASSERT_UNREACHABLE() and a return value of X86EMUL_UNHANDLEABLE.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
This is to avoid it overlapping the linear_pt_count field needed for PV
domains. Introduce a separate, HVM-only pagetable_dying field to replace
the sole one left in the upper 16 bits.
Note that the accesses to ->shadow_flags in shadow_{pro,de}mote() get
switched to non-atomic, non-bitops operations, as {test,set,clear}_bit()
are not allowed on uint16_t fields and hence their use would have
required ugly casts. This is fine because all updates of the field ought
to occur with the paging lock held, and other updates of it use |= and
&= as well (i.e. using atomic operations here didn't really guard
against potentially racing updates elsewhere).
This is part of XSA-280.
Reported-by: Prgmr.com Security <security@prgmr.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
In preparation of reducing struct page_info's shadow_flags field to 16
bits, lower the bit positions used for SHF_out_of_sync and
SHF_oos_may_write.
Instead of also adjusting the open coded use in _get_page_type(),
introduce shadow_prepare_page_type_change() to contain knowledge of the
bit positions to shadow code.
This is part of XSA-280.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
If the L1e update hasn't occured, the flush cannot do anything useful. This
skips the potentially expensive vcpumask_to_pcpumask() conversion, and
broadcast TLB shootdown.
More importantly however, we might be in the error path due to a bad va
parameter from the guest, and this should not propagate into the TLB flushing
logic. The INVPCID instruction for example raises #GP for a non-canonical
address.
This is XSA-279.
Reported-by: Matthew Daley <mattd@bugfuzz.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
c/s 7867181b2 "x86/PoD: correctly handle non-order-0 decrease-reservation
requests" introduced an early exit in guest_remove_page() for unexpected p2m
types. However, get_gfn_query() internally takes the p2m lock, and must be
matched with a put_gfn() call later.
Fix the erroneous comment beside the declaration of get_gfn_query().
This is XSA-277.
Reported-by: Paul Durrant <paul.durrant@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Passing MEMF_no_refcount to alloc_domheap_pages() will allocate, as
expected, a page that is assigned to the specified domain but is not
accounted for in tot_pages. Unfortunately there is no logic for tracking
such allocations and avoiding any adjustment to tot_pages when the page
is freed.
The only caller of alloc_domheap_pages() that passes MEMF_no_refcount is
hvm_alloc_ioreq_mfn() so this patch removes use of the flag from that
call-site to avoid the possibility of a domain using an ioreq server as
a means to adjust its tot_pages and hence allocate more memory than it
should be able to.
However, the reason for using the flag in the first place was to avoid
the allocation failing if the emulator domain is already at its maximum
memory limit. Hence this patch switches to allocating memory from the
target domain instead of the emulator domain. There is already an extra
memory allowance of 2MB (LIBXL_HVM_EXTRA_MEMORY) applied to HVM guests,
which is sufficient to cover the pages required by the supported
configuration of a single IOREQ server for QEMU. (Stub-domains do not,
so far, use resource mapping). It also also the case the QEMU will have
mapped the IOREQ server pages before the guest boots, hence it is not
possible for the guest to inflate its balloon to consume these pages.
Reported-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
The code does not take a page reference in hvm_alloc_ioreq_mfn(), only a
type reference. This can lead to a situation where a malicious domain with
XSM_DM_PRIV can engineer a sequence as follows:
- create IOREQ server: no pages as yet.
- acquire resource: page allocated, total 0.
- decrease reservation: -1 ref, total -1.
This will cause Xen to hit a BUG_ON() in free_domheap_pages().
This patch fixes the issue by changing the call to get_page_type() in
hvm_alloc_ioreq_mfn() to a call to get_page_and_type(). This change
in turn requires an extra put_page() in hvm_free_ioreq_mfn() in the case
that _PGC_allocated is still set (i.e. a decrease reservation has not
occurred) to avoid the page being leaked.
This is part of XSA-276.
Reported-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
The logic is not fit for this purpose, so simply disable its use until
it can be fixed / replaced. Note that this re-enables merging for the
table creation case, which was disabled as a (perhaps unintended) side
effect of the earlier "amd/iommu: fix flush checks". It relies on no
page getting mapped more than once (with different properties) in this
process, as that would still be beyond what the merging logic can cope
with. But arch_iommu_populate_page_table() guarantees this afaict.
This is part of XSA-275.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Flush checking for AMD IOMMU didn't check whether the previous entry
was present, or whether the flags (writable/readable) changed in order
to decide whether a flush should be executed.
Fix this by taking the writable/readable/next-level fields into account,
together with the present bit.
Along these lines the flushing in amd_iommu_map_page() must not be
omitted for PV domains. The comment there was simply wrong: Mappings may
very well change, both their addresses and their permissions. Ultimately
this should honor iommu_dont_flush_iotlb, but to achieve this
amd_iommu_ops first needs to gain an .iotlb_flush hook.
Also make clear_iommu_pte_present() static, to demonstrate there's no
caller omitting the (subsequent) flush.
This is part of XSA-275.
Reported-by: Paul Durrant <paul.durrant@citrix.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Since we have introduced arm64 variants, we'd better start tagging the
old ones.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
Use the "extends" keyword introduced in 11.3 to reduce repetition in
jobs. More importantly, this helps us better organise the properties
of jobs.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
Versions of GCC before 5 can't compile cpuid.c, and fail with the rather cryptic:
In file included from lib/x86/cpuid.c:3:0:
lib/x86/cpuid.c: In function ‘x86_cpuid_policy_fill_native’:
include/xen/lib/x86/cpuid.h:25:5: error: inconsistent operand constraints in an ‘asm’
asm ( "cpuid"
^
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54232 for more details.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
This is a followup to c/s 96f235c26 which fulfils the remaining TODO item.
First of all, the pre-existing SVM code has a bug. The value in
msrs->dr_mask[] may be stale, as we allow direct access to these MSRs.
Resolve this in guest_rdmsr() by reading directly from hardware in the
affected case.
With the reading/writing logic moved to the common guest_{rd,wr}msr()
infrastructure, the migration logic can be simplified. The PV migration logic
drops all of its special casing, and SVM's entire {init,save,load}_msr()
infrastructure becomes unnecessary.
The resulting diffstat shows quite how expensive the PV special cases where in
arch_do_domctl().
add/remove: 0/3 grow/shrink: 4/6 up/down: 465/-1494 (-1029)
Function old new delta
guest_rdmsr 252 484 +232
guest_wrmsr 653 822 +169
msrs_to_send 8 48 +40
hvm_load_cpu_msrs 489 513 +24
svm_init_msr 21 - -21
hvm_save_cpu_msrs 365 343 -22
read_msr 1089 1001 -88
write_msr 1829 1689 -140
svm_msr_read_intercept 1124 970 -154
svm_load_msr 195 - -195
svm_save_msr 196 - -196
svm_msr_write_intercept 1461 1265 -196
arch_do_domctl 9581 9099 -482
Total: Before=3314610, After=3313581, chg -0.03%
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Julien Grall