There's no guarantee that e.g. a switch() control expression's memory
operand(s) get(s) read just once. Guard against the compiler producing
"unexpected" code by sprinkling around some ACCESS_ONCE().
I'm leaving alone opt_conswitch[]: It gets accessed in quite a few
places anyway, and an intermediate change won't have any severe effect
afaict.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Add a Kconfig option to select no specific platform support.
Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Reviewed-by: Andrii Anisov <andrii_anisov@epam.com>
Compile platform code that doesn't have its own specific kconfig option
only if ALL32_PLAT or ALL64_PLAT depending on the architecture. The
benefit is that choosing one of the platforms available as a menu
option allows the user not to build other unnecessary platform code.
Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Reviewed-by: Andrii Anisov <andrii_anisov@epam.com>
Make the bufdev and non-bufdev messages distinct, and always print the
non-constant argument (ie, the size).
This assists diagnosis.
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Hans van Kranenburg <hans@knorrie.org>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This is very dangerous from a security point of view, because a missing entry
will cause L2's action to be interpreted as L1's action.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Now that nvmx_handle_vmx_insn() performs all VT-x instruction checks, there is
no need for redundant checking in vmx_inst_check_privilege(). Remove it, and
take out the vmxon_check boolean which was plumbed through decode_vmx_inst().
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
This is a stopgap solution until the toolstack side of initialisation can be
sorted out, but it does result in the nvmx_vcpu_in_vmx() predicate working
correctly even when nested virt hasn't been enabled for the domain.
Update nvmx_handle_vmx_insn() to include the in-vmx mode check (for all
instructions other than VMXON) to complete the set of #UD checks.
In addition, sanity check that the nested vmexit handler has worked correctly,
and that we are only providing emulation of the VT-x instructions to L1
guests.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Reusing debugreg[5] for the PV emulated IO breakpoint information is confusing
to read. Instead, introduce a dr7_emul field in pv_vcpu for the purpose.
With the PV emulation out of the way, debugreg[4,5] are entirely unused and
don't need to be stored.
Rename debugreg[0..3] to dr[0..3] to reduce code volume, but keep them as an
array because their behaviour is identical and this helps simplfy some of the
PV handling. Introduce dr6 and dr7 fields to replace debugreg[6,7] which
removes the storage for debugreg[4,5].
In arch_get_info_guest(), handle the merging of emulated dr7 state alongside
all other dr handling, rather than much later.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
No functional change (as curr->arch.debugreg[5] is zero when DE is clear), but
this change simplifies the following patch.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Backport commit 3714ce1d6655098ee69ede632883e5874d67e4ab
"iommu/arm-smmu: Disable stalling faults for all endpoints" from the
Linux kernel. This works-around Erratum #842869.
Original commit message:
Enabling stalling faults can result in hardware deadlock on poorly
designed systems, particularly those with a PCI root complex upstream of
the SMMU.
Although it's not really Linux's job to save hardware integrators from
their own misfortune, it *is* our job to stop userspace (e.g. VFIO
clients) from hosing the system for everybody else, even if they might
already be required to have elevated privileges.
Given that the fault handling code currently executes entirely in IRQ
context, there is nothing that can sensibly be done to recover from
things like page faults anyway, so let's rip this code out for now and
avoid the potential for deadlock.
Cc: <stable@vger.kernel.org>
Fixes: 48ec83bcbcf5 ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Reported-by: Matt Evans <matt.evans@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Credit2 was declared "supported" in 4.8, and as of 4.10 had two other
critical features implemented (soft affinity / NUMA and caps).
Why change the default?
The code is better: more predictable, less jitter, easier to determine
how modifications will affect overall behavior, easier in the future
to make load-balancing behavior more subtle (e.g., taking into account
the cost of powering up extra cores, &c).
Overall performance compared to Credit1 is somewhat of a mixed bag.
Unfortunately most of what I have are tests using XenServer's internal
perf testing system, so I can't share the raw data (via links anyway).
Here is a summary of data from an internal e-mail Dario sent in the
past:
* DVDbench: On underloaded systems, credit2 outperformed credit1 by
about 4%. On overloaded systems, credit2 underperformed by about 3%.
* On a range of tests (unixbench, lmbench, &c), credit and credit2
perform within 5% of each other (up and down).
* Credit2 fairly consistently beats credit for TCP-style workloads.
* Credit2 is sometimes equal to, sometimes 5-15% worse than, credit for
synthetic CPU workloads (e.g., Dhrystone).
* On LoginVSI, credit2 fairly consistently outperforms credit by about 10%.
Credit2, like credit, has a number of workloads / setups for which
performance could be improved. Personally I think networking and
partially-loaded systems is going to be more representative of what
Xen is actually used for; so I think credit2 is on the whole the
better scheduler to use by default. And in any case, making those
improvements on credit2 will be easier than on credit.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Dario Faggioli <dfaggioli@suse.com>
The levels are absolute numbers such that IOMMU_PAGING_MODE_LEVEL_X
evaluates to X (for the valid range of 0 - 7) so simply use numbers in
the code.
No functional change.
NOTE: This patch also adds emacs boilerplate to amd-iommu-defs.h
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Brian Woods <brian.woods@amd.com>
To allow for some code sharing where possible, copy VEX.L into EVEX.LR
even for VEX (or XOP) encoded insns. Make operand size determination
use this right away, at the same time adding consistency checks for the
EVEX scalar insn cases (the non-scalar ones aren't uniform enough for
the checking to be done in a central place like this).
Note that the broadcast case is not handled here, but will be taken care
of elsewhere (in just a single place rather than at least two).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
This function is called by both PV and HVM. Unfortunately the code is
very convoluted. We can reason that code between the call to
hvm_set_info_guest and out label is PV only. Put that portion under
CONFIG_PV.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Start by putting hypercall handlers which are supposed to be PV only
under CONFIG_PV. Shuffle some code around to avoid introducing
excessive numbers of CONFIG_PV.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Fix an inverted pair of checks, drop an incorrect instance of #UD
raising for non-64-bit mode, and add further generic checks.
Note: Despite what SDM Vol 2 rev 067 states, EVEX.V' is _not_ ignored
outside of 64-bit mode when the field does not encode a register.
Just like EVEX.VVVV is required to be 0b1111 in that case, EVEX.V'
is required to be 1 there.
Also rename the bcst field to br, as #UD generation for individual insns
will need to consider both of its possible meanings.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
In preparation for sensible to-boolean conversion on AVX512, wrap
another abstraction function around the present to_bool(<x> == <y>), to
get rid of the open-coded == (which will get in the way of using
built-in functions instead). For the future AVX512 use scalar operands
can't be used then anymore: Use (vec_t){} when the operand is zero,
and broadcast (if available) otherwise (assume pre-AVX512 when broadcast
is not available, in which case a plain scalar is still fine).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
These are all VEX encoded, so the EVEX decoding logic continues to
remain unused at this point.
The new testcase is deliberately coded in assembly, as a C one would
have become almost unreadable due to the overwhelming amount of
__builtin_...() that would need to be used. After all the compiler has
no underlying type (yet) that could be operated on without builtins,
other than the vector types used for "normal" SIMD insns.
Note that outside of 64-bit mode and despite the SDM not currently
saying so, VEX.W is ignored for the KMOV{D,Q} encodings to/from GPRs,
just like e.g. for the similar VMOV{D,Q}.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
This is not used (and probably was never meant to be) by the tool stack.
Limiting it to the current domain in particular allows to eliminate a
bogus use of vCPU 0 in pagetable_dying().
Remove the now unnecessary domain/vCPU parameters from the wrapper/hook
functions at the same time.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
It's dead code in that case.
We could go further, as we don't really need the 2- and 3-level walk
code in PV mode, but to drop their compilation requires quite a bit of
disentangling of shadow mode code.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
c/s ac6a4500b "vvmx: set vmxon_region_pa of vcpu out of VMX operation to an
invalid address" was a real bugfix as described, but has a very subtle bug
which results in all VT-x instructions being usable by a guest.
The toolstack constructs a guest by issuing:
XEN_DOMCTL_createdomain
XEN_DOMCTL_max_vcpus
and optionally later, HVMOP_set_param to enable nested virt.
As a result, the call to nvmx_vcpu_initialise() in hvm_vcpu_initialise()
(which is what makes the above patch look correct during review) is actually
dead code. In practice, nvmx_vcpu_initialise() first gets called when nested
virt is enabled, which is typically never.
As a result, the zeroed memory of struct vcpu causes nvmx_vcpu_in_vmx() to
return true before nested virt is enabled for the guest.
Fixing the order of initialisation is a work in progress for other reasons,
but not viable for security backports.
A compounding factor is that the vmexit handlers for all instructions, other
than VMXON, pass 0 into vmx_inst_check_privilege()'s vmxop_check parameter,
which skips the CR4.VMXE check. (This is one of many reasons why nested virt
isn't a supported feature yet.)
However, the overall result is that when nested virt is not enabled by the
toolstack (i.e. the default configuration for all production guests), the VT-x
instructions (other than VMXON) are actually usable, and Xen very quickly
falls over the fact that the nvmx structure is uninitialised.
In order to fail safe in the supported case, re-implement all the VT-x
instruction handling using a single function with a common prologue, covering
all the checks which should cause #UD or #GP faults. This deliberately
doesn't use any state from the nvmx structure, in case there are other lurking
issues.
This is XSA-278
Reported-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Sergey Dyasli <sergey.dyasli@citrix.com>
In particular, initialising %dr6 with the value 0 is buggy, because on
hardware supporting Transactional Memory, it will cause the sticky RTM bit to
be asserted, even though a debug exception from a transaction hasn't actually
been observed.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
In particular, initialising %dr6 with the value 0 is buggy, because on
hardware supporting Transactional Memory, it will cause the sticky RTM bit to
be asserted, even though a debug exception from a transaction hasn't actually
been observed.
Introduce arch_vcpu_regs_init() to set various architectural defaults, and
reuse this in the hvm_vcpu_reset_state() path.
Architecturally, %edx's init state contains the processors model information,
and 0xf looks to be a remnant of the old Intel processors. We clearly have no
software which cares, seeing as it is wrong for the last decade's worth of
Intel hardware and for all other vendors, so lets use the value 0 for
simplicity.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
In particular, initialising %dr6 with the value 0 is buggy, because on
hardware supporting Transactional Memory, it will cause the sticky RTM bit to
be asserted, even though a debug exception from a transaction hasn't actually
been observed.
Move X86_DR6_DEFAULT into x86-defns.h along with the other architectural
register constants, and introduce a new X86_DR7_DEFAULT. Use the existing
write_debugreg() helper, rather than opencoded inline assembly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
In certain scenarios, NMIs might be disabled during Xen boot process.
Such situation will cause alternative_instructions() to:
panic("Timed out waiting for alternatives self-NMI to hit\n");
This bug was originally seen when using Tboot to boot Xen 4.11
To prevent this from happening, enable NMIs during cpu_init() and
during __start_xen() for BSP.
Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
ARM Dom0 creation was broken by the said commit because ARM neither
provided XEN_DOMCTL_CDF_hvm_guest nor had CONFIG_PV set.
Set XEN_DOMCTL_CDF_hvm_guest flag for ARM Dom0 to fix the issue. Also
set XEN_DOMCTL_CDF_hap while at it.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
The format identifier is consistent with Linux. The code is adapted from
bitmap_scn{,list}printf() but cleaned up.
This change allows all callers to avoid needing a secondary buffer to render a
cpumask/nodemask into.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: <jbeulich@suse.com>
The code snippet is to switch between SYS_DECS_trap_gate and
SYS_DESC_irq_gate depending on whether XPTI is used. When PV is
disabled there is no need to switch.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
When PV is disabled those symbols won't be available. It is impossible
for Xen to hit #DB there.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
This is a bit more complicated than the HVM case because system
domains have PV guest type. Leave them like that.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Commit 54ec59f6b0 "xen/arm: vgic-v3: Don't create empty re-distributor
regions" breaks compilation when using the new vGIC.
This is because the field nr_regions is not existing in the vgic
structure. For simplicity, as vGICv3 is not yet imported, disable GICv3.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Since commit 2c257bd6 "x86/hvm: remove default ioreq server (again)" the
GFNs allocated by the toolstack and set in HVM_PARAM_IOREQ_PFN and
HVM_PARAM_BUFIOREQ_PFN have been unused. This patch allows them to be used
by (non-default) ioreq servers.
While in the area, also make sure HVM_PARAM_[BUF]IOREQ_PFN can only be set
once. These parameters should have always been in the 'set once' category
but this has, so far, not been enforced.
NOTE: This fixes a compatibility issue. A guest created on a version of
Xen that pre-dates the initial ioreq server implementation and then
migrated in will currently fail to resume because its migration
stream will lack values for HVM_PARAM_IOREQ_SERVER_PFN and
HVM_PARAM_NR_IOREQ_SERVER_PAGES *unless* the system has an
emulator domain that uses direct resource mapping (which depends
on the version of privcmd it happens to have) in which case it
will not require use of GFNs for the ioreq server shared
pages.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
These fields have existed since the SVM code was first introduced.
The earliest reference I can find is c/s d1bd157fbc which is unforunately a
rebase & squash of a separate dev tree. Looking a the commit message, I'm
guessing it was introduced by:
> user: twoller@xen-trw1.site
> date: Tue Dec 13 19:49:53 2005 -0500
> files: ... xen/include/asm-x86/svm_vmcb.h ...
> description:
> Add SVM base files to repository.
Anyway, the AMD SDM has no mention of PDPE fields in the VMCB and marks this
part of the VMCB as reserved. The manual does explicitly say that 32bit PAE
paging may read the PDPE fields from memory rather from the CPU registers.
Chances are very good that this is a vestigial remnent of an early design.
Xen doesn't use the fields at all, except to copy them on virtual
vmentry/vmexit.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
When using shadow paging, EFER.NX is a Xen controlled bit, and is required by
the shadow pagefault handler to distinguish instruction fetches from data
accesses.
This can be observed by a guest which has NX and SMEP clear but SMAP active by
attempting to execute code on a user mapping. The first attempt to build the
target shadow will #PF so is handled by the shadow code, but when walking the
the guest pagetables, the lack of PFEC_insn_fetch being signalled causes the
shadow code to mistake the instruction fetch for a data fetch, and believe
that it is a real guest fault. As a result, the guest receives #PF[-d-srP]
for an action which should complete successfully.
The suspicious-looking gymnastics with LME is actually a subtle corner case
with shadow paging. When dropping out of Long Mode, a guests choice of LME
and Xen's choice of CR0.PG cause hardware to operate in Long Mode, but the
shadow code to operate in 2-on-3 mode.
In addition to describing this corner case in the SVM side, extend the comment
for the same fix on the VT-x side. (I have a suspicion that I've just worked
out why VT-x doesn't tolerate LMA != LME when Unrestricted Guest is clear.)
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
The name of the "with_gla" flag is confusing; it has nothing to do
with the existence or lack thereof of a faulting GLA, but rather where
the fault originated. The npfec.kind value is always valid, and
should thus be propagated, regardless of whether gla_valid is set or
not.
In particular, gla_valid will never be set on AMD systems; but
npfec.kind will still be valid and should still be propagated.
Signed-off-by: Alexandru Isaila <aisaila@bitdefender.com>
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Alexandru Isaila <aisaila@bitdefender.com>
Acked-by: Razvan Cojocaru <rcojocaru@bitdefender.com>
Currently there is a subop for setting the memaccess of a page, but not
for consulting it. The new HVMOP_altp2m_get_mem_access adds this
functionality.
Both altp2m get/set mem access functions use the struct
xen_hvm_altp2m_mem_access which has now dropped the `set' part and has
been renamed from xen_hvm_altp2m_set_mem_access.
Signed-off-by: Adrian Pop <apop@bitdefender.com>
Signed-off-by: Razvan Cojocaru <rcojocaru@bitdefender.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
This hypercall is PV only on x86. Provide a stub for it when
!CONFIG_PV.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Provide declarations for hypercall_page_initialise_ring*_kernel, make
sure DCE work as expected.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Jan Beulich
Stefano Stabellini
Ian Jackson
Andrew Cooper
Stefano Stabellini
George Dunlap
Paul Durrant
Wei Liu
Sergey Dyasli
Julien Grall
Alexander Schulz
Roger Pau Monne
Razvan Cojocaru