Commit cb563d7665 ("xen/sched: support core scheduling for moving
cpus to/from cpupools") introduced a regression when trying to remove
an offline cpu from a cpupool, as the system would crash in this
situation.
Fix that by testing the cpu to be online.
Fixes: cb563d7665 ("xen/sched: support core scheduling for moving cpus to/from cpupools")
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Dario Faggioli <dfaggioli@suse.com>
Currently it might be not obvious which scheduling mode (e.g. core-
scheduling) is being used by the scheduler. Alleviate this by printing
additional information about the selected granularity per-cpupool.
Note: per-cpupool granularity selection is not implemented yet. Every
cpupool gets its granularity from the single global value.
Take this opportunity to introduce struct sched_gran_name array and
refactor sched_select_granularity().
Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Acked-by: Dario Faggioli <dfaggioli@suse.com>
AMD Fam15h processors introduced the flush-by-asid feature, for more fine
grain flushing purposes.
Flushing everything including ASID 0 (i.e. Xen context) is an an unnecesserily
large hammer, and never necessary in the context of guest TLBs needing
invalidating.
When available, use TLB_CTRL_FLUSH_ASID in preference to TLB_CTRL_FLUSH_ALL.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Rework the vmcbcleanbits_t definitons to use bool, drop 'fields' from the
namespace, position the comments in an unambiguous position, and include the
bit position.
In svm_vmexit_handler(), don't bother conditionally writing ~0 or 0 based on
hardware support. The field was entirely unused and ignored on older
hardware (and we're already setting reserved cleanbits anyway).
In nsvm_vmcb_prepare4vmrun(), simplify the logic massively by dropping the
vcleanbit_set() macro using a vmcbcleanbits_t local variable which only gets
filled in the case that clean bits were valid previously. Fix up the style on
impacted lines.
No practical change in behaviour.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Recent changes no longer allow modification of AFLAGS. The needed
conversion was apparently missed in 2740d96efd ("xen/build: have the
root Makefile generates the CFLAGS").
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Julien Grall <jgrall@amazon.com>
First of all avoid excessive conversions. copy_{from,to}_guest(), for
example, work fine with all of XEN_GUEST_HANDLE{,_64,_PARAM}().
Further
- do_physdev_op_compat() didn't use the param form for its parameter,
- {hap,shadow}_track_dirty_vram() wrongly used the param form,
- compat processor Px logic failed to check compatibility of native and
compat structures not further converted.
As this eliminates all users of guest_handle_from_param() and as there's
no real need to allow for conversions in both directions, drop the
macros as well.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Julien Grall <jgrall@amazon.com>
The path provided by EXTRA_PREFIX should be added to the search path
of the configure script, like it's done in Config.mk. Not doing so
makes the search path for configure differ from the search path used
by the build.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
[ wei: run autogen.sh ]
Acked-by: Wei Liu <wl@xen.org>
The value returned from CPUID is the maximum number for virtual
processors supported by Hyper-V. It could be larger than the maximum
number of virtual processors configured.
Stash the configured number into a variable and use it in calculations.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
To fix CPUID handling, libxl__build_pre() is going to have to distinguish
between a brand new VM vs one which is being migrated-in/resumed.
Transcribe dcs->restore_fd into dbs->restore in initiate_domain_create()
only (specifically avoiding the stubdom state in libxl__spawn_stub_dm()).
While tweaking initiate_domain_create(), make a new dbs alias and simplify
later code, and drop the local restore_fd alias as the new dbs->restore is
more intuitive in context.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Updating microcode is less error prone when caches have been flushed and
depending on what exactly the microcode is updating. For example, some of the
issues around certain Broadwell parts can be addressed by doing a full cache
flush.
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
[Linux commit 91df9fdf51492aec9fed6b4cbd33160886740f47, ported to Xen]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
This allows the AP boot assembly use per-cpu variables, and brings the
semantics closer to that of the BSP, which can use per-cpu variables from the
start of day.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
stack++ can go into the next page and unmap_domain_page() will unmap the
wrong one, causing mapcache and memory corruption. Fix.
Signed-off-by: Hongyan Xia <hongyxia@amazon.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Clang 3.5 doesn't do enough dead-code-elimination to drop the compat_gdt
reference, resulting in a linker failure:
hidden symbol `per_cpu__compat_gdt' isn't defined
Drop the local variable, and move the evaluation of this_cpu(compat_gdt) to
within the guarded region.
Reported-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Several of these in particular haven't been pruned since the logic was all
part of arch/x86/traps.c
Some adjustments to header files are required to avoid compile errors:
* emulate.h needs xen/sched.h because gdt_ldt_desc_ptr() uses v->vcpu_id.
* mmconfig.h needs to forward declare acpi_table_header.
* shadow.h and trace.h need to have uint*_t in scope before including the Xen
public headers. For shadow.h, reorder the includes. For trace.h, include
types.h
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
PVHv1 dom0 was given access to all PHYSDEVOP hypercalls, and such
restriction was not removed when PVHv1 code was removed. As a result
the switch in hvm_physdev_op was more complicated than required, and
relied on PVHv2 dom0 not having PIRQ support in order to prevent
access to some PV specific PHYSDEVOPs.
Fix this by moving the default case to the bottom of the switch, since
there's no need for any fall through now. Also remove the hardware
domain check, as all the not explicitly listed PHYSDEVOPs are
forbidden for HVM domains.
Finally tighten the condition to allow usage of
PHYSDEVOP_pci_mmcfg_reserved: apart from having vPCI enabled it should
only be used by the hardware domain. Note that the code in
do_physdev_op is already restricting the call to privileged domains
only, but it can be further restricted to the hardware domain only, as
other privileged domains don't have access to MMCFG regions anyway.
Overall no functional change should arise from this change.
Reported-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Several insns were missed when this function was first added. As far as
insns already supported by the emulator go - SMSW and {,V}STMXCSR were
wrongly considered r/o insns so far.
Insns like the VMX, SVM, or CET-SS ones, PTWRITE, or AMD's new SNP ones
are intentionally not covered just yet. VMPTRST is put there just to
complete the respective group.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Along the lines of what the not reverted part of 3c4b2eef49 ("x86:
refine link time stub area related assertion") did, we need to transform
the absolute HV_HCALL_PAGE into the image base relative hv_hcall_page
(or else there'd be no need for two distinct symbols). Otherwise
mkreloc, as used for generating the base relocations of xen.efi, will
spit out warnings like "Difference at .text:0009b74f is 0xc0000000
(expected 0x40000000)". As long as the offending relocations are PC
relative ones, the generated binary is correct afaict, but if there ever
was the absolute address stored, xen.efi would miss a fixup for it.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wl@xen.org>
These are more helpful if they point at the address where the relocated
value starts, rather than at the specific byte of the difference.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
When doing an assisted flush on HAP the purpose of the
on_selected_cpus is just to trigger a vmexit on remote CPUs that are
in guest context, and hence just using is_vcpu_dirty_cpu is too lax,
also check that the vCPU is running. Due to the lazy context switching
done by Xen dirty_cpu won't always be cleared when the guest vCPU is
not running, and hence relying on is_running allows more fine grained
control of whether the vCPU is actually running.
I've measured the time of the non-local branch of flush_area_mask
inside the shim running with 32vCPUs over 100000 executions and
averaged the result on a large Westmere system (80 ways total). The
figures where fetched during the boot of a SLES 11 PV guest. The
results are as follow (less is better):
Non assisted flush with x2APIC: 112406ns
Assisted flush without this patch: 820450ns
Assisted flush with this patch: 8330ns
While there also pass NULL as the data parameter of on_selected_cpus,
the dummy handler doesn't consume the data in any way.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Quite a few of the items are used by xenoprof.c only, so move them there
to limit their visibility as well as the amount of re-building needed in
case of changes. Also drop the inclusion of the public header there.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wl@xen.org>
Both is_primary and domain_ready are only ever written to. Drop both
fields and restrict structure visibility to just the one involved CU.
While doing so (and just for starters) make "is_compat" properly bool.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Wei Liu <wl@xen.org>
Buffer pages should be shared with "ignored" or "active" guests only
(besides, obviously, the primary profiling domain). Hence domain type
should be set to "ignored" before unsharing from the primary domain
(which implies even a previously "passive" domain may then access its
buffers, albeit that's not very useful unless it gets promoted to
"active" subsequently), i.e. such that no further writes of records to
the buffer would occur, and (at least for consistency) also before
sharing it (with the calling domain) from the XENOPROF_get_buffer path.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Wei Liu <wl@xen.org>
With the max base leaf using 0, this one should be using the extended
leaf counterpart thereof, rather than some arbitrary extended leaf.
Fixes: 588a966a57 ("libx86: Introduce x86_cpu_policies_are_compatible()")
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
During a VM fork we copy the shared_info page; however, we also need to ensure
that the page is mapped into the same GFN in the fork as its in the parent.
Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
XEN_DOMCTL_destroydomain creates a continuation if domain_kill -ERESTARTs.
In that scenario, it is possible to receive multiple _pirq_guest_unbind
calls for the same pirq from domain_kill, if the pirq has not yet been
removed from the domain's pirq_tree, as:
domain_kill()
-> domain_relinquish_resources()
-> pci_release_devices()
-> pci_clean_dpci_irq()
-> pirq_guest_unbind()
-> __pirq_guest_unbind()
Avoid recurring invocations of pirq_guest_unbind() by removing the pIRQ
from the tree being iterated after the first call there. In case such a
removed entry still has a softirq outstanding, record it and re-check
upon re-invocation.
Note that pirq_cleanup_check() gets relaxed beyond what's strictly
needed here, to avoid introducing an asymmetry there between HVM and PV
guests.
Reported-by: Varad Gautam <vrd@amazon.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Varad Gautam <vrd@amazon.de>
Reviewed-by: Paul Durrant <paul@xen.org>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Now that we don't properly hook things up into the page tables anymore
we also don't need to set aside an address range. Drop it, using
compat_idle_pg_table_l2[] simply (explicitly) from slot 0.
While doing the re-arrangement, which is accompanied by the dropping or
replacing of some local variables, restrict the scopes of some further
ones at the same time.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Wei Liu <wl@xen.org>
Older binutils complains with:
trampoline.S:95: Error: junk `ul&0xffffffff' after expression
Use an assembly-safe constant.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
We have a global variable where the necessary L2 table is recorded; no
need to inspect L4 and L3 tables (and this way a few less places will
eventually need adjustment when we want to support 5-level page tables).
Also avoid setting up the L3 entry, as the address range never gets used
anyway (it'll be dropped altogether in a subsequent patch).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Hongyan Xia <hongyxia@amazon.com>
Reviewed-by: Wei Liu <wl@xen.org>
This will cause all SYSCALL/SYSRET instructions to suffer #UD rather than
following the MSR_{L,C}STAR pointers, allowing us to drop the star_enter()
panic helper, allowing us to clean up the IST stacks in a subsequent patch.
Drop the now-dead conditional SYSENTER logic in the middle of
subarch_percpu_traps_init().
In addition, vmx_restore_host_msrs() need not restore any host
state. (Regarding the asymmetric changes, VT-x automatically restores
SYSENTER state on vmexit, and SVM restores both SYSCALL/SYSENTER state with
the VMSAVE/VMLOAD instructions.)
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
There is no need for the Compat GDT if there are no 32bit PV guests. This
saves 4k per online CPU
Bloat-o-meter reports the following savings in Xen itself:
add/remove: 0/3 grow/shrink: 1/4 up/down: 7/-4612 (-4605)
Function old new delta
cpu_smpboot_free 1249 1256 +7
per_cpu__compat_gdt_l1e 8 - -8
per_cpu__compat_gdt 8 - -8
init_idt_traps 442 420 -22
load_system_tables 414 364 -50
trap_init 444 280 -164
cpu_smpboot_callback 1255 991 -264
boot_compat_gdt 4096 - -4096
Total: Before=3062726, After=3058121, chg -0.15%
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wl@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
... and move arch.is_32bit_pv into the pv union while at it.
Adjust the impacted code to use true/false, dropping the hunk
pv_domain_initialise() which is storing 0 into an already zeroed
datastructure.
Bloat-o-meter reports the following net savings with some notable differences
highlighted:
add/remove: 4/6 grow/shrink: 5/76 up/down: 1955/-18792 (-16837)
Function old new delta
...
pv_vcpu_initialise 411 158 -253
guest_cpuid 1837 1584 -253
pv_hypercall 579 297 -282
check_descriptor 427 130 -297
_get_page_type 5915 5202 -713
arch_get_info_guest 2225 1195 -1030
context_switch 3831 2635 -1196
dom0_construct_pv 10284 8939 -1345
arch_set_info_guest 5564 3267 -2297
Total: Before=3079563, After=3062726, chg -0.55%
In principle, DOMAIN_is_32bit_pv should be based on CONFIG_PV32, but the
assembly code is going to need further untangling before that becomes easy to
do. For now, use CONFIG_PV as missed accidentally by c/s ec651bd246 "x86:
make entry point code build when !CONFIG_PV".
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wl@xen.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
This is the start of some performance and security-hardening improvements,
based on the fact that 32bit PV guests are few and far between these days.
Ring1 is full of architectural corner cases, such as counting as supervisor
from a paging point of view. This accounts for a substantial performance hit
on processors from the last 8 years (adjusting SMEP/SMAP on every privilege
transition), and the gap is only going to get bigger with new hardware
features.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wl@xen.org>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
There is no need to save/restore FS/GS/XCR0 state. It will be handled
suitably on the context switch away from the idle.
The CR4 restoration in restore_rest_processor_state() was actually fighting
later code in enter_state() which tried to keep CR4.MCE clear until everything
was set up. Delete the intermediate restoration, and defer final restoration
until after MCE is reconfigured.
Restoring PAT can be done earlier, and ideally before paging is enabled. By
moving it into the trampoline during the setup for 64bit, the call can be
dropped from cpu_init(). The EFI path boot path doesn't disable paging, so
make the adjustment when switching onto Xen's pagetables.
The only remaing piece of restoration is load_system_tables(), so suspend.c
can be deleted in its entirety.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The GNTTABOP_cache_flush hypercall has a wrong test for hypercall
continuation, the test today is:
if ( rc > 0 || opaque_out != 0 )
Unfortunately this will be true even in case of an error (rc < 0),
possibly leading to very long lasting hypercalls (times of more
than an hour have been observed in a test case).
Correct the test condition to result in false with rc < 0 and set
opaque_out only if no error occurred, to be on the safe side.
Partially-suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Implement L0 assisted TLB flush for Xen on Hyper-V. It takes advantage
of several hypercalls:
* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST
* HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX
* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE
* HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX
Pick the most efficient hypercall available.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Paul Durrant <pdurrant@amazon.com>
Implement a basic hook for L0 assisted TLB flush. The hook needs to
check if prerequisites are met. If they are not met, it returns an error
number to fall back to native flushes.
Introduce a new variable to indicate if hypercall page is ready.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Paul Durrant <pdurrant@amazon.com>
Hyper-V's L0 assisted flush has fine-grained control over what gets
flushed. We need all the flags available to make the best decisions
possible.
No functional change because Xen's implementation doesn't care about
what is passed to it.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Paul Durrant <pdurrant@amazon.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Use Xen's L0 HVMOP_flush_tlbs hypercall in order to perform flushes.
This greatly increases the performance of TLB flushes when running
with a high amount of vCPUs as a Xen guest, and is specially important
when running in shim mode.
The following figures are from a PV guest running `make -j32 xen` in
shim mode with 32 vCPUs and HAP.
Using x2APIC and ALLBUT shorthand:
real 4m35.973s
user 4m35.110s
sys 36m24.117s
Using L0 assisted flush:
real 1m2.596s
user 4m34.818s
sys 5m16.374s
The implementation adds a new hook to hypervisor_ops so other
enlightenments can also implement such assisted flush just by filling
the hook.
Note that the Xen implementation completely ignores the dirty CPU mask
and the linear address passed in, and always performs a global TLB
flush on all vCPUs. This is a limitation of the hypercall provided by
Xen. Also note that local TLB flushes are not performed using the
assisted TLB flush, only remote ones.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Wei Liu <wl@xen.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The TLB clock is helpful when running Xen on bare metal because when
doing a TLB flush each CPU is IPI'ed and can keep a timestamp of the
last flush.
This is not the case however when Xen is running virtualized, and the
underlying hypervisor provides mechanism to assist in performing TLB
flushes: Xen itself for example offers a HVMOP_flush_tlbs hypercall in
order to perform a TLB flush without having to IPI each CPU. When
using such mechanisms it's no longer possible to keep a timestamp of
the flushes on each CPU, as they are performed by the underlying
hypervisor.
Offer a boolean in order to signal Xen that the timestamped TLB
shouldn't be used. This avoids keeping the timestamps of the flushes,
and also forces NEED_FLUSH to always return true.
No functional change intended, as this change doesn't introduce any
user that disables the timestamped TLB.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Wei Liu <wl@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Introduce a specific flag to request a HVM guest linear TLB flush,
which is an ASID/VPID tickle that forces a guest linear to guest
physical TLB flush for all HVM guests.
This was previously unconditionally done in each pre_flush call, but
that's not required: HVM guests not using shadow don't require linear
TLB flushes as Xen doesn't modify the pages tables the guest runs on
in that case (ie: when using HAP). Note that shadow paging code
already takes care of issuing the necessary flushes when the shadow
page tables are modified.
In order to keep the previous behavior modify all shadow code TLB
flushes to also flush the guest linear to physical TLB if the guest is
HVM. I haven't looked at each specific shadow code TLB flush in order
to figure out whether it actually requires a guest TLB flush or not,
so there might be room for improvement in that regard.
Also perform ASID/VPID flushes when modifying the p2m tables as it's a
requirement for AMD hardware. Finally keep the flush in
switch_cr3_cr4, as it's not clear whether code could rely on
switch_cr3_cr4 also performing a guest linear TLB flush. A following
patch can remove the ASID/VPID tickle from switch_cr3_cr4 if found to
not be necessary.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
Juergen Gross
Sergey Dyasli
Andrew Cooper
Jan Beulich
Roger Pau Monne
Tamas K Lengyel
Wei Liu
Ashok Raj
Hongyan Xia
Wei Liu
Wei Liu