Pre Xen-4.14 streams will not contain any CPUID/MSR information. There is
nothing libxc can do about this, and will have to rely on the higher level
toolstack to provide backwards compatibility.
To facilitate this, extend the static_data_done() callback, highlighting the
missing information, and modify libxl to use it. At the libxc level, this
requires an arch-specific hook which, for now, always reports CPUID and MSR as
missing. This will be adjusted in a later change.
No overall functional change - this is just plumbing.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
A v3 stream can compatibly read a v2 stream by inferring the position of the
STATIC_DATA_END record.
v2 compatibility is only needed for x86. No other architectures exist yet,
but they will have a minimum of v3 when introduced.
The x86 HVM compatibility point being in handle_page_data() (which is common
code) is a bit awkward. However, as the two compatibility points are subtly
different, and it is (intentionally) not possible to call into arch specific
code from common code (except via the ops hooks), use some #ifdef-ary and
opencode the check, rather than make handle_page_data() a per-arch helper.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Higher level toolstacks may wish to know when the static data is complete, so
introduce a restore_callback for the purpose.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The code has devating from the prevailing style in many ways. Adjust spacing,
indentation, position of operators, layout of multiline comments, removal of
superfluous comments, constness, trailing commas, and use of unqualified
'unsigned'.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Originally, 'checkpointed' was a boolean signalling the difference between a
plain and a Remus stream. COLO was added later, but several bits of code
retained boolean-style logic. While correct, it is confusing to follow.
Additionally, XC_MIG_STREAM_NONE means "no checkpoints" but reads as "no
stream".
Consolidate all the logic on the term 'stream_type', and rename STREAM_NONE
to STREAM_PLAIN. Re-position the stream_type variable so it isn't
duplicated in both the save and restore unions.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
XCFLAGS_CHECKPOINT_COMPRESS has been unused since c/s b15bc4345 (2015),
XCFLAGS_HVM since c/s 9e8672f1c (2013), and XCFLAGS_STDVGA since c/s
087d43326 (2007). Drop the constants, and code which sets them.
The separate hvm parameter (appeared in c/s d11bec8a1, 2007 and ultimately
redundant with XCFLAGS_HVM), is used for sanity checking and debug printing,
then discarded and replaced with Xen's idea of whether the domain is PV or
HVM.
Rearrange the logic in xc_domain_save() to ask Xen sightly earlier, and use a
consistent idea of 'hvm' throughout. Removing this parameter removes the
final user of libxl's dss->hvm, so drop that field as well.
Update the doxygen comment to be accurate.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@citrix.com>
The hvm and pae parameters are a remnant of legacy migration. They have 0
passed in from libxl_stream_read.c's process_record(), and are discarded in
xc_domain_restore().
While dropping these, update the doxygen comment to be accurate, and simplify
the other hvm vs pv handling in xc_domain_restore().
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Mixed throughout libxc are uint32_t, int, and domid_t for domid parameters.
With a signed type, and an explicitly 16-bit type, it is exceedingly difficult
to construct an INVALID_DOMID constant which works with all of them. (The
main problem being that domid_t gets unconditionally zero extended when
promoted to int for arithmatic.)
Libxl uses uint32_t consistently everywhere, so alter libxc to match.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Release-acked-by: Julien Grall <julien.grall@linaro.org>
[ wei: fix compilation error in libxl ]
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
This Patch allows a migration precopy policy to be specified.
The precopy phase of the xc_domain_save() live migration algorithm has
historically been implemented to run until either a) (almost) no pages
are dirty or b) some fixed, hard-coded maximum number of precopy
iterations has been exceeded. This policy and its implementation are
less than ideal for a few reasons:
- the logic of the policy is intertwined with the control flow of the
mechanism of the precopy stage
- it can't take into account facts external to the immediate
migration context, such external state transfer state, interactive
user input, or the passage of wall-clock time.
- it does not permit the user to change their mind, over time, about
what to do at the end of the precopy (they get an unconditional
transition into the stop-and-copy phase of the migration)
To permit callers to implement arbitrary higher-level policies governing
when the live migration precopy phase should end, and what should be
done next:
- add a precopy_policy() callback to the xc_domain_save() user-supplied
callbacks
- during the precopy phase of live migrations, consult this policy after
each batch of pages transmitted and take the dictated action, which
may be to a) abort the migration entirely, b) continue with the
precopy, or c) proceed to the stop-and-copy phase.
- provide an implementation of the old policy, used when
precopy_policy callback is not provided.
Signed-off-by: Jennifer Herbert <Jennifer.Herbert@citrix.com>
Signed-off-by: Joshua Otto <jtotto@uwaterloo.ca>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
a. call callbacks resume/checkpoint/suspend while secondary vm
status is consistent with primary
b. send dirty pfn list to primary when checkpoint under colo
c. send store gfn and console gfn to xl before resuming secondary vm
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
We will do the following things again and again:
1. Suspend primary vm
a. Suspend primary vm
b. do postsuspend
c. Read CHECKPOINT_SVM_SUSPENDED sent by secondary
2. Checkpoint
a. Write emulator xenstore data and emulator context
b. Write checkpoint end record
3. Resume primary vm
a. Read CHECKPOINT_SVM_READY from slave
b. Do presume
c. Resume primary vm
d. Read CHECKPOINT_SVM_RESUMED from slave
4. Wait a new checkpoint
a. Wait a new checkpoint(not implemented)
b. Send CHECKPOINT_NEW to slave
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Secondary vm is running in colo mode. So we will do
the following things again and again:
1. Resume secondary vm
a. Send CHECKPOINT_SVM_READY to master.
b. If it is not the first resume, call libxl__checkpoint_devices_preresume().
c. If it is the first resume(resume right after live migration),
- call libxl__xc_domain_restore_done() to build the secondary vm.
- enable secondary vm's logdirty.
- call libxl__domain_resume() to resume secondary vm.
- call libxl__checkpoint_devices_setup() to setup checkpoint devices.
d. Send CHECKPOINT_SVM_RESUMED to master.
2. Wait a new checkpoint
a. Call libxl__checkpoint_devices_commit().
b. Read CHECKPOINT_NEW from master.
3. Suspend secondary vm
a. Suspend secondary vm.
b. Call libxl__checkpoint_devices_postsuspend().
c. Send CHECKPOINT_SVM_SUSPENDED to master.
4. Checkpoint
a. Read emulator xenstore data and emulator context
b. REC_TYPE_CHECKPOINT_END
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
In COLO mode, both VMs are running, and are considered in sync if the
visible network traffic is identical. After some time, they fall out of
sync.
At this point, the two VMs have definitely diverged. Lets call the
primary dirty bitmap set A, while the secondary dirty bitmap set B.
Sets A and B are different.
Under normal migration, the page data for set A will be sent from the
primary to the secondary.
However, the set difference B - A (the one in B but not in A, lets
call this C) is out-of-date on the secondary (with respect to the
primary) and will not be sent by the primary (to secondary), as it
was not memory dirtied by the primary. The secondary needs C page data
to reconstruct an exact copy of the primary at the checkpoint.
The secondary cannot calculate C as it doesn't know A. Instead, the
secondary must send B to the primary, at which point the primary
calculates the union of A and B (lets call this D) which is all the
pages dirtied by both the primary and the secondary, and sends all page
data covered by D.
In the general case, D is a superset of both A and B. Without the
backchannel dirty bitmap, a COLO checkpoint can't reconstruct a valid
copy of the primary.
We transfer the dirty bitmap on libxc side, so we need to introduce back
channel to libxc.
Note: it is different from the paper. We change the original design to
the current one, according to our following concerns:
1. The original design needs extra memory on Secondary host. When there's
multiple backups on one host, the memory cost is high.
2. The memory cache code will be another 1k+, it will make the review
more time consuming.
Note: this patch merely adds new parameters to various prototypes and
functions. The new parameters are used in later patch called
"libxc/restore: send dirty pfn list to primary when checkpoint under
COLO".
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
xc_domain_save() and xc_domain_restore's parameter will use this type,
so it should be public.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Similar to the previous xentoollog case this is not allowed. Switch to
a forward decl of the struct and use of it in the APIs.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
libxenevtchn will provide a stable API and ABI for accessing the
evtchn device.
The functions are moved into the xenevtchn namespace to make a clean
break from libxc and avoid ambiguity regarding which interfaces are
stable.
All in-tree users are updated to use the new names.
Upon request (via #define XC_WANT_COMPAT_EVTCHN_API) libxenctrl will
provide a compat API for the old names. This is used by qemu-xen for
the time being. qemu-xen-traditional is updated in lockstep.
This leaves a few event channel related functions which go via privcmd
(EVTCHNOP) rather than ioctls on the /dev/xen/evtchn device in
libxenctrl. Specifically:
- xc_evtchn_alloc_unbound
- xc_evtchn_reset
- xc_evtchn_status
Note that xc_evtchn_alloc_unbound's functionality is also provided by
xenevtchn_bind_unbound_port() (née xc_evtchn_bind_unbound_port) and is
probably redundant.
These functions do not appear to be needed by qemu-dm, qemu-pv
(provision of device model to HVM guests and PV backends respectively)
or by libvchan suggesting they are not needed by non-toolstack uses of
event channels. QEMU does use these in hw/xenpv/xen_domainbuild.c but
that is a "toolstack use".
The new library uses a version script to ensure that only expected
symbols are exported and to version them such that ABI guarantees can
be kept in the future.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ ijc -- updated MINIOS_UPSTREAM_REVISION and QEMU_TRADITIONAL_REVISION ]
xc_get_bit_size() is being used by the unused python wrapper
xc.getBitSize() only. Remove the wrapper and xc_get_bit_size().
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
In tools/libxc/xc_dom_compat_linux.c xc_linux_build() is the only
domain building function used by an in-tree component (qemu-xen) which
is really necessary.
Remove the other domain building functions and the unused python
wrapper xc.linux_build() referencing one of the to be removed
functions.
Suggested-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
As recommended in http://www.gnu.org/licenses/gpl-howto.en.html.
This is the result of:
$ git grep -El Mass\|Temple\|Franklin | xargs ./fsf.pl
Where fsf.pl is:
#!/usr/bin/perl -w -pi.bak -0777
my $repl = 'If not, see <http://www.gnu.org/licenses/>.';
my $br = qr/(?:\s*\n\s*(?:[\*\#]|\/\/|\.\\" )?\s*|\s+)/;
my $inwt = qr/[Ii]f${br}not,${br}write${br}(?:to${br})?the${br}Free${br}Software${br}Foundation,(?:${br}Inc\.,)?/;
my $mass = qr/675${br}Mass${br}Ave,?${br}Cambridge,?${br}MA${br}02139,?${br}USA,?\.?/;
my $franklin = qr/51${br}Franklin${br}St(?:reet)?(?:,${br}| - )Fifth${br}Floor,?${br}Boston,?${br}MA,?${br}02110-1301,?${br}USA,?\.?/;
my $temple = qr/59${br}Temple${br}Place(?:,${br}| - )Suite${br}330,?${br}Boston,?${br}MA,?${br}021110?-1307,?${br}USA,?\.?/;
s|$inwt$br$mass|$repl|m;
s|$inwt$br$franklin|$repl|m;
s|$inwt$br$temple|$repl|m;
The only remaining mentions of these addresses are in COPYING files which I
haven't touched.
Some of the changed files are imports from elsewhere, however
filtering them out is tricky, I think it is tolerable to have these
files be modified here and then perhaps reverted on the next sync,
since it's only 1-2 lines and obvious what is going on.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Update the libxc spec to indicate more sternly that TOOLSTACK records
should no longer be used.
Also, trim further toolstack infrastructure which should have gone in
c/s 39bf4e9 "tools/libxl: Drop all knowledge of toolstack callbacks"
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
And call it when a checkpoint record is found in the libxc stream.
Some parts of this patch have been based on patches from the COLO
series.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
---
v3: Named constants for the API
v2: Borrow sufficient fragments from several COLO patches to get
BROKEN_CHANNEL and checkpoint failover to function.
The old code checks the callbacks "postcopy & checkpoint", if the
callbacks exists, it will call them. However this is unreliable, so
add this flag to explicitly indicate a checkpointed stream in the new
code. This is backward compatible with the legacy migration just don't
know this flag and will ignore it.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Yang Hongyang <yanghy@cn.fujitsu.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
[ ijc -- added discussion of legacy compat ]
For testing purposes, the environmental variable "XG_MIGRATION_V2" allows the
two save/restore codepaths to coexist, and have a runtime switch.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
The algorithm is more or less the same as the one used for PV guest.
Libxc gets hold of the mapping of vnode to pnode and size of each vnode
then allocate memory accordingly.
And then the function returns low memory end, high memory end and mmio
start to caller. Libxl needs those values to construct vmemranges for
that guest.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
From libxc's point of view, it only needs to know vnode to pnode mapping
and size of each vnode to allocate memory accordingly. Add these fields
to xc_dom structure.
The caller might not pass in vNUMA information. In that case, a dummy
layout is generated for the convenience of libxc's allocation code. The
upper layer (libxl etc) still sees the domain has no vNUMA
configuration.
Note that for this patch on PV x86 guest can have multiple regions of
ram allocated.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Also moves xc_dom.h to include as it is used often by other xen tools.
Use the new include subdirectories to build Xen tools, qemu-xen and
stubdoms.
Add the old libxc include path to the programs that need it to build,
on a case by case basis and commeting that they shouldn't require
internal libxc headers to build.
[ And: update QEMU_TRADITIONAL_REVISION to corresponding qemu patch
- Ian jackson ]
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Andrew Cooper
Jennifer Herbert
Wei Liu
Wen Congyang
Ian Campbell
Juergen Gross
Roger Pau Monne
Stefano Stabellini