With all other pieces in place, it is now safe to restore the CPUID and MSR
data in the migration stream, rather than discarding them and using the higher
level toolstacks compatibility logic.
While this is a small patch, it has large implications for migrated/resumed
domains. Most obviously, the CPU family/model/stepping data,
cache/tlb/etc. will no longer change behind the guests back.
Another change is the interpretation of the Xend cpuid strings. The 'k'
option is not a sensible thing to have ever supported, and 's' is how how the
stream will end up behaving.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Pre Xen-4.14 streams will not contain any CPUID/MSR information. There is
nothing libxc can do about this, and will have to rely on the higher level
toolstack to provide backwards compatibility.
To facilitate this, extend the static_data_done() callback, highlighting the
missing information, and modify libxl to use it. At the libxc level, this
requires an arch-specific hook which, for now, always reports CPUID and MSR as
missing. This will be adjusted in a later change.
No overall functional change - this is just plumbing.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
With the destination side now able to understand X86_{CPUID,MSR}_DATA
records (and compatibly handle their absense), update the sending logic to
obtain and forward this data from Xen.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
For now, the data are just stashed, and discarded at the end.
A future change will restore the data, once libxl has been adjusted to avoid
clobbering the data.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Introduce a new static_data() hook which is responsible for writing out
any static data records. The HVM side continues to be a no-op, while
the PV side moves write_x86_pv_info() into this earlier hook. The the
common code writes out a STATIC_DATA_END record, and the stream version
is bumped to 3.
Update convert-legacy-stream to write a v3 stream, because this will
bypass the compatibly logic in libxc.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
A v3 stream can compatibly read a v2 stream by inferring the position of the
STATIC_DATA_END record.
v2 compatibility is only needed for x86. No other architectures exist yet,
but they will have a minimum of v3 when introduced.
The x86 HVM compatibility point being in handle_page_data() (which is common
code) is a bit awkward. However, as the two compatibility points are subtly
different, and it is (intentionally) not possible to call into arch specific
code from common code (except via the ops hooks), use some #ifdef-ary and
opencode the check, rather than make handle_page_data() a per-arch helper.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Higher level toolstacks may wish to know when the static data is complete, so
introduce a restore_callback for the purpose.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Migration data can be split into two parts - that which is invariant of
guest execution, and that which is not. Separate these two with the
STATIC_DATA_END record.
The short term, we want to move the x86 CPU Policy data into the stream.
In the longer term, we want to provisionally send the static data only
to the destination as a more robust compatibility check. In both cases,
we will want a callback into the higher level toolstack.
Mandate the presence of the STATIC_DATA_END record, and declare this v3,
along with instructions for how to compatibly interpret a v2 stream.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Migration v3 is in the process of being introduced, meaning that the code has
to cope with both versions. Use an explicit 2 for now.
For the verify-stream-v2 and convert-legacy-stream scripts, update text to say
"v2 (or later)". What matters is the distinction vs legacy streams.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
While it should have been this way from the beginning, not doing so will
become an actual problem with PVH Dom0. The interface change is binary
compatible, but requires tools side producers to be re-built.
Drop the bogus/unnecessary page alignment restriction on the input
buffer at the same time.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
xc_cpuid_apply_policy() is gaining extra parameters to untangle CPUID
complexity in Xen. While an improvement in general, it does have the
unfortunate side effect of duplicating some settings across multiple
parameters.
Rearrange the logic to only consider 'pae' if no explicit featureset is
provided. This reduces the complexity for callers who have already provided a
pae setting in the featureset.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Paul Durrant <pdurrant@amzn.com>
Acked-by: Wei Liu <wl@xen.org>
At this moment a guest can call vmfunc to change the altp2m view. This
should be limited in order to avoid any unwanted view switch.
The new xc_altp2m_set_visibility() solves this by making views invisible
to vmfunc.
This is done by having a separate arch.altp2m_working_eptp that is
populated and made invalid in the same places as altp2m_eptp. This is
written to EPTP_LIST_ADDR.
The views are made in/visible by marking them with INVALID_MFN or
copying them back from altp2m_eptp.
To have consistency the visibility also applies to
p2m_switch_domain_altp2m_by_id().
The usage of this hypercall is aimed at dom0 having a logic with a number of views
created and at some time there is a need to be sure that only some of the views
can be switched, saving the rest and making them visible when the time
is right.
Note: If altp2m mode is set to mixed the guest is able to change the view
visibility and then call vmfunc.
Signed-off-by: Alexandru Isaila <aisaila@bitdefender.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Acked-by: Wei Liu <wl@xen.org>
libxc defines XGS_POLICY_ABORT for precopy policy to signal that migration
should be aborted (eg. if the estimated pause time is too huge for the
instance). Default simple precopy policy never returns that, but it could be
overriden with a custom one.
Signed-off-by: Andrew Panyakin <apanyaki@amazon.com>
Acked-by: Wei Liu <wl@xen.org>
[wei: fix coding style issue]
The parameter 'params' of xc_set_parameters() should never be modified.
So mark it as const.
Signed-off-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
OCaml is using a string to describe the parameter 'keys' of
xc_send_debug_keys(). Since Ocaml 4.06.01, String_val() will return a
const char * when using -safe-string. This will result to a build
failure because xc_send_debug_keys() expects a char *.
The function should never modify the parameter 'keys' and therefore the
parameter should be const. Unfortunately, this is not directly possible
because DECLARE_HYPERCALL_BOUNCE() is expecting a non-const variable.
A new macro DECLARE_HYPERCALL_BOUNCE_IN() is introduced and will take
care of const parameter. The first user will be xc_send_debug_keys() but
this can be used in more place in the future.
Reported-by: Dario Faggioli <dfaggioli@suse.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
For now, write the same content for both. Update the users of the
initialisers to use the new name, and extend xen-cpuid to dump both default
and max featuresets.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Drop XC_FEATUREMASK_DEEP_FEATURES. It isn't used by any callers, and unlike
the other static masks, won't be of interest to anyone without other pieces of
cpuid-autogen.h
In xc_get_static_cpu_featuremask(), use a 2d array instead of individually
named variables, and drop the switch statement completely.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Xen has never acted upon the value of HVM_PARAM_PAE_ENABLED, contrary perhaps
to expectations based on how other boolean fields work.
It was only ever used as a non-standard calling convention for
xc_cpuid_apply_policy() but that has been fixed now.
Purge its use, and any possible confusion over its behaviour, by having Xen
reject any attempts to use it. Forgo setting it up in libxl's
hvm_set_conf_params(). The only backwards compatibility necessary is to have
the HVM restore stream discard it if found.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The local xc_hvm_param_deprecated_check() in libxc tries to guess Xen's
behaviour for the MEMORY_EVENT params, but is wrong for the get side, where
Xen would return 0 (which is also a bug). Delete the helper.
In Xen, perform the checks in hvm_allow_set_param(), rather than
hvm_set_param(), and actually implement checks on the get side so the
hypercall doesn't return successfully with 0 as an answer.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wl@xen.org>
HVM_PARAM_PAE_ENABLED is set and consumed by the toolstack only. It is in
practice a complicated and non-standard way of passing a boolean parameter
into xc_cpuid_apply_policy().
This is silly. Pass PAE as a regular parameter instead.
In libxl__cpuid_legacy(), leave a rather better explaination of why only HVM
guests have a choice in PAE setting.
No change in how a guest is constructed.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The written ABI states that %es will be set up, but libxc doesn't do so. In
practice, it breaks `rep movs` inside guests before they reload %es.
The written ABI doesn't mention %ss, but libxc does set it up. Having %ds
different to %ss is obnoxous to work with, as different registers have
different implicit segments.
Modify the spec to state that %ss is set up as a flat read/write segment.
This a) matches the Multiboot 1 spec, b) matches what is set up in practice,
and c) is the more sane behaviour for guests to use.
Fixes: 68e1183411 ('libxc: introduce a xc_dom_arch for hvm-3.0-x86_32 guests')
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wl@xen.org>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
This reverts commit 7b3c5b70a3 and
re-enables the usage of x2APIC with nested virtualization.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
It turns out that a bug (since forever) in Xen causes XSAVE records to have
non-architectural behaviour on xsave-capable hardware, when a PV guest has not
touched the state.
In such a case, the data record returned from Xen is 2*uint64_t, both claiming
the (illegitimate) state of %xcr0 and %xcr0_accum being 0.
Adjust the bound in handle_x86_pv_vcpu_blob() to cope with this.
Fixes: 2a62c22715 "libxc/restore: Fix data auditing in handle_x86_pv_vcpu_blob()"
Reported-by: Igor Druzhinin <igor.druzhinin@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
By default the sve bits are not set.
This patch adds a new hypercall, xc_altp2m_set_supress_ve_multi(),
to set a range of sve bits.
The core function, p2m_set_suppress_ve_multi(), does not break in case
of a error and it is doing a best effort for setting the bits in the
given range. A check for continuation is made in order to have
preemption on large ranges.
The gfn of the first error is stored in
xen_hvm_altp2m_suppress_ve_multi.first_error_gfn and the error code is
stored in xen_hvm_altp2m_suppress_ve_multi.first_error.
If no error occurred the values will be 0.
Signed-off-by: Alexandru Isaila <aisaila@bitdefender.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Petre Pircalabu <ppircalabu@bitdefender.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
With the 32 PAE build of Xen gone, 32bit PV guests' top level pagetables no
longer behave exactly like PAE in hardware.
They should have A/D bits set, for the same performance reasons as apply to
other levels. This brings the domain builder in line with how Xen constructs
a 32bit dom0.
As a purely code improvement, make use of range notation to initialise
identical values in adjacent array elements.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wl@xen.org>
We are shortly going to want to introduce some common x86 fields, so having
x86_pv and x86_hvm as the top level objects is a problem. Insert a
surrounding struct x86 and drop the x86 prefix from the pv/hvm objects.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The code has devating from the prevailing style in many ways. Adjust spacing,
indentation, position of operators, layout of multiline comments, removal of
superfluous comments, constness, trailing commas, and use of unqualified
'unsigned'.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Originally, 'checkpointed' was a boolean signalling the difference between a
plain and a Remus stream. COLO was added later, but several bits of code
retained boolean-style logic. While correct, it is confusing to follow.
Additionally, XC_MIG_STREAM_NONE means "no checkpoints" but reads as "no
stream".
Consolidate all the logic on the term 'stream_type', and rename STREAM_NONE
to STREAM_PLAIN. Re-position the stream_type variable so it isn't
duplicated in both the save and restore unions.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
During migration, we buffer several blobs of data which ultimately need
handing back to Xen at an appropriate time.
Currently, this is all handled in an ad-hoc manner, but more blobs are soon
going to be added. Introduce xc_sr_blob to encapsulate a ptr/size pair, and
update_blob() to handle the memory management aspects.
Switch the HVM_CONTEXT and the four PV_VCPU_* blobs over to this new
infrastructure. No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
xc_dom_p2m() and dom->p2m_host[] implement a linear transform for translated
domains, but waste a substantial chunk of RAM doing so.
ARM literally never reads dom->p2m_host[] (because of the xc_dom_translated()
short circuit in xc_dom_p2m()). Drop it all.
x86 HVM does use dom->p2m_host[] for xc_domain_populate_physmap_exact() calls
when populating 4k pages. Reuse the same tactic from 2M/1G ranges and use an
on-stack array instead. Drop the memory allocation.
x86 PV guests do use dom->p2m_host[] as a non-identity transform. Rename the
field to pv_p2m to make it clear it is PV-only.
No change in the constructed guests.
Reported-by: Varad Gautam <vrd@amazon.de>
Reported-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
In-guest p2m's are a concept specific to x86 PV guests. alloc_p2m_list() is
the only hook which initialises dom->p2m_guest, making
xc_dom_update_guest_p2m() a nop for non-PV guests.
Move p2m_guest into xc_dom_image_x86 and adjust alloc_p2m_list() to match.
Drop xc_dom_update_guest_p2m() entirely.
One caller, move_l3_below_4G(), only uses it to modify a single entry, so
rewriting the whole guest p2m is wasteful - opencode the single update
instead. The other caller is common code. Instead, move the logic into the
setup_pgtables() hooks, which know their own sizeof_pfn and can do away with
the switch statement.
No change in the constructed guests.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
Currently, the setup_pgtable() hook is optional, but alloc_pgtable() hook is
not. Both are specific to x86 PV guests, and stubbed in various ways by the
dombuilders for translated guests (x86 HVM, ARM).
Make alloc_pgtables() optional, and drop all the stubs for translated guest
types.
No change in the constructed guests.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Julien Grall <julien@xen.org>
The two xc_dom_params structures for PV pagetables are never modified and can
live in .rodata. Reduce their scope to the alloc_pgtable_*() functions which
construct xc_dom_image_x86 appropriately.
Rename {alloc,setup}_pgtables() to {alloc,setup}_pgtables_pv() to highlight
that they are PV only, and drop some _x86() suffixes from static helpers.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
XCFLAGS_CHECKPOINT_COMPRESS has been unused since c/s b15bc4345 (2015),
XCFLAGS_HVM since c/s 9e8672f1c (2013), and XCFLAGS_STDVGA since c/s
087d43326 (2007). Drop the constants, and code which sets them.
The separate hvm parameter (appeared in c/s d11bec8a1, 2007 and ultimately
redundant with XCFLAGS_HVM), is used for sanity checking and debug printing,
then discarded and replaced with Xen's idea of whether the domain is PV or
HVM.
Rearrange the logic in xc_domain_save() to ask Xen sightly earlier, and use a
consistent idea of 'hvm' throughout. Removing this parameter removes the
final user of libxl's dss->hvm, so drop that field as well.
Update the doxygen comment to be accurate.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@citrix.com>
There are issues as reported by osstest when Xen is running nested on
itself and the L1 Xen is using x2APIC. While those are being
investigated, disable announcing the x2APIC feature in CPUID when nested
HVM mode is enabled.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
The hvm and pae parameters are a remnant of legacy migration. They have 0
passed in from libxl_stream_read.c's process_record(), and are discarded in
xc_domain_restore().
While dropping these, update the doxygen comment to be accurate, and simplify
the other hvm vs pv handling in xc_domain_restore().
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Despite as suggested in c/s 685e922d6f, not all HVM_PARAMs are handled
in the same way. HVM_PARAM_PAE_ENABLED is a toolstack-only value, and
the xc_cpuid_apply_policy() used to be the only consumer.
Reinstate the old behaviour (mad as it is) to avoid regressions.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
x86 PVH is completely obsolete - it was intended for legacy PVH before that
idea was abandoned. There was an RFC series for ARM in 2015, but there is
plenty of outstanding work which hasn't been done yet.
No functional change. New types can be (re)introduced with the code which
actually implements them.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Julien Grall <julien@xen.org>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
This record is specific to x86, and should have had a prefix to being with.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
vcpu_guest_context_any_t is currently allocated on the stack, and copied from
a mutable buffer which is freed immediately after its use here. Mutate the
buffer in place instead of duplicating it.
The code is as it is due to how it was developed. Originally,
process_vcpu_basic() operated on a const pointer from the X86_VCPU_BASIC
record, but during upstreaming, the addition of Remus support required
buffering of X86_VCPU_BASIC records each checkpoint.
By the time process_vcpu_basic() runs, we are commited to completing state
restoration and unpausing the guest.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
There have been no users of the xc_compression_*() interface since Migration
v2 replaced legacy migration (2015, c/s b15bc4345).
It would need adjusting to fit into migration v2, and can be pulled out of git
history if someone wants to resurrect it in the future.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The current logic only works by chance, in that XSAVE records also tend to be
a multiple of 128. Implement the missing logic for XSAVE.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
handle_x86_pv_info() has a subtle bug. It uses an 'else if' chain with a
clause in the middle which doesn't exit unconditionally. In practice, this
means that when restoring a 32bit PV guest, later sanity checks are skipped.
Rework the logic a little to be simpler. There are exactly two valid
combinations of fields in X86_PV_INFO, so factor this out and check them all
in one go, before making adjustments to the current domain.
Once adjustments have been completed successfully, sanity check the result
against the X86_PV_INFO settings in one go, rather than piece-wise.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Andrew Cooper
Juergen Gross
Jan Beulich
Tamas K Lengyel
Alexandru Isaila
Andrew Panyakin
Julien Grall
Roger Pau Monné
Wei Liu
Konrad Rzeszutek Wilk