Instead of sending dirty pages of guest memory as-is, use a simple compression
algorithm that sends a RLE-encoded XOR of the page against its last sent copy.
A small LRU cache is used to hold recently dirtied pages. Pagetable pages are
sent as-is, as they are canonicalized at sender side and uncanonicalized at
receiver.
[ Fixed up a conflict in sg_save_restore.h. I had to increase the
ID values used from -11 and -12 to -12 and -13 because -11 had
been taken by ..._HVM_VIRIDIAN in the meantime. -iwj ]
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca>
Acked-by: Brendan Cully <brendan@cs.ubc.ca>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
xc_domain_save/restore currently pay no attention to
HVM_PARAM_VIRIDIAN which results in an HVM domain running a recent
version on Windows (post-Vista) locking up on a domain restore due to
EOIs (done via a viridian MSR write) being silently dropped. This
patch adds an extra save entry for the viridian parameter and also
adds code in the viridian kernel module to catch attempted use of
viridian functionality when the HVM parameter has not been set.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Committed-by: Keir Fraser <keir@xen.org>
To detect presence of superpages on the receiver side, we need
to have strings of sequential pfns sent across on the first iteration
through the memory. However, as we go through the memory, more and
more of it will be marked dirty, making it wasteful to send those pages.
This patch introduces a new PFINFO type, "XALLOC". Like PFINFO_XTAB, it
indicates that there is no corresponding page present in the subsquent
page buffer. However, unlike PFINFO_XTAB, it contains a pfn which should be
allocated.
This new type is only used for migration; but it's placed in
xen/public/domctl.h so that the value isn't reused.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
When receiving pages, look for contiguous 2-meg aligned regions and
attempt to allocate a superpage for that region, falling back to
4k pages if the allocation fails.
(Minor conflict fixed up. -iwj)
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
20544:ad9d75d74bd5 and 20545:cc7d66ba0dad seemingly intended to change these
global static variables into stack variables but didn't remove the static
qualifier.
Also zero the entire struct once with memset rather than clearing fields
piecemeal in two different places.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Acked-by: Vincent Hanquez <vincent.hanquez@eu.citrix.com>
xc_domain_restore.c: In function 'xc_domain_restore':
xc_domain_restore.c:1090:18: error: variable 'prev_pc' set but not
used [-Werror=unused-but-set-variable]
Signed-off-by: Olaf Hering <olaf@aepfle.de>
xc_domain_restore.c:apply_batch function makes two passes over the pfn_types
array in pagebuf to allocate the needed MFNs. The curbatch parameter to this
function specifies the array offset in pfn_types, from where the current scan
should begin. But this variable is not taken into account (index always starts
at 0) during the two passes. While this [bug] does not manifest itsef during
save/restore or live migration, under Remus, xc_domain_restore fails due to
corrupt guest page tables.
(This appears to have been broken by 21588:6c3d8aec202d which reverted
two changesets from before Remus support was added and hence
reintroduced some none-Remus compatible bits.)
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
This makes the interface consistent with the changes made to the main
interface in 21483:779c0ef9682c.
Also fix some references to "struct xc_interface" which should have
been simply "xc_interface" in tools/xenpaging, and update QEMU_TAG to
pull in the corresponding qemu change.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
I have tested the patch in the following senarios:
1> Non-xsave platform
2> Xsave-capable platform, guest does not support xsave, xen support xsave
3> Xsave-capable platform, guest does support xsave, xen supports xsave
4> Guest (non-xsave) saved on platform without xsave, restored on a
Xsave-capable system.
All passed.
Signed-off-by: Shan Haitao <haitao.shan@intel.com>
Signed-off-by: Han Weidong <weidong.han@intel.com>
In the default "QemuDeviceModelRecord"-style HVM tail of the migration
protocol the size of the qemu save record is unknown at the receiver
and therefore it must read until EOF. This is not compatible with the
xl migration protocol which contains a post-migration handshake and
therefore cannot close the socket on the sending end.
What is required is an explicit length field for the save record,
which the "RemusDeviceModelState"-style HVM tail includes.
Rather than overload the "RemusDeviceModelState" name for a non-Remus
use case (on off chance that they need to diverge for some reason in
the future) introduce a third style called "DeviceModelRecord0002"
which is identical to current "RemusDeviceModelState"-style.
Hopefully the inclusion of a number here will allow easier extension
in the future without needing to come up with increasingly less
helpful names!
Also propagate errors from xc_domain_save and
libxl__domain_suspend_common to callers.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
On restore:
region_mfn is passed to xc_map_foreign_range and
xc_map_foreign_bulk. In both cases the buffer is accessed from the
ioctl handler in the kernel and not from any hypercall. Therefore
normal copy_{to,from}_user handling in the kernel will cope with any
faulting access.
p2m_batch is passed to xc_domain_memory_populate_physmap which takes
care of bouncing the buffer already.
On save:
pfn_type is passed to xc_map_foreign_bulk which does not need locking
as per region_mfn above.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson.citrix.com>
Currently the wrappers for these hypercalls swallow partial success
and return failure to the caller.
In order to use these functions more widely instead of open-coding
uses of XENMEM_* and xc_memory_op add variants which return the actual
hypercall result.
Therefore add the following functions:
xc_domain_increase_reservation
xc_domain_decrease_reservation
xc_domain_populate_physmap
and implement the existing semantics using these new functions as
xc_domain_increase_reservation_exact
xc_domain_decrease_reservation_exact
xc_domain_populate_physmap_exact
replacing the existing xc_domain_memory_* functions.
Use these new functions to replace all open coded uses of
XENMEM_increase_reservation, XENMEM_decrease_reservation and
XENMEM_populate_physmap.
Also rename xc_domain_memory_*_pod_target to xc_domain_*_pod_target
for consistency.
Temporarily add a compatibility macro for
xc_domain_memory_populate_physmap to allow time for qemu to catch up.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Not actually used here but useful to confirm that a handle is passed
down to each location where it will be required once we switch to
hypercall buffers.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
In particular this causes us to turn O_NONBLOCK back off if we set it.
The caller may continue to use the fd for it's own protocol needs and
may not be prepared to have it become non-blocking.
This probably only effects Remus now after my previous patch to signal
the last checkpoint, since a regular migration will no longer set the
fd non-blocking.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Brendan Cully <brendan@cs.ubc.ca>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
When the restore code sees this notification it will restore the
currently in-progress checkpoint when it completes.
This allows the restore end to finish up without waiting for a
spurious timeout on the receive fd and thereby avoids unnecessary
error logging in the case of a successful migration or restore.
In the normal migration or restore case the first checkpoint is always
the last. For a rolling checkpoint (such as Remus) the notification is
currently unused but could be used in the future for example to
provide a controlled failover for reasons other than error
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Brendan Cully <brendan@cs.ubc.ca>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reverse engineered from the code, likely contains inaccuracies but I
think provides a base to work from.
Add symbolic names for the minus-flags.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1282671421 -3600
# Node ID d1dd29a470ef1b9d2c77478a123326036dfe90bb
# Parent d7a4adad9c328decbd384d87b23001aea8951b86
tools/libxc, tools/libelf: Relicense under LGPL v2.1
Relicense these two libraries under LGPL v2.1 only except where
individual files already included the "or later" provision.
Copyright holders have been contacted by Stephen Spector and have all
agreed this change.
Removed tools/libxc/ia64/aclinux.h since it appeared to be
unused. There is a separate, more up to date, copy in
xen/include/acpi/platform/aclinux.h which does appear to be used.
Clarify the license of MiniOS privcmd.h under the same terms as other
tools/include/xen-sys headers.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stephen Spector <stephen.spector@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
The aforementioned patch missed several calls to read_exact, causing spurious
timeouts under Remus.
Also add a couple of additional error reports to rdexact.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Creates a shared memory ring buffer and event channel in HVM domains
for passing debug messages from PV drivers on HVM guests
The console is used by windows pv drivers to send debug data to xenconsoled
Signed-off-by: Owen Smith <owen.smith@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
This disables superpage restore support, but should gain us acceptable
performance when restoring a domain using a pv_ops dom0 kernel. This
is because single-page allocations will be batched rather than issued
singly.
Signed-off-by: Keir Fraser <keir.fraser@citrix.com>
Previously, the code was inconsistent: some calls to PERROR passed \n
and some did not. With the new logging arrangements, passing \n is
definitely incorrect.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
* Make "read_exact" in libxc always set errno.
* Rename "read_exact" macro in xc_domain_restore.c (which shadows
real function) to RDEXACT and change all callers.
* Make RDEXACT anamorphically use xch for error reporting rather than
* stderr.
* Call PERROR rather than ERROR when appropriate, so that log messages
include errno.
* Save errno in noncached_write so that its errno value is always
* right.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
oubuf and batch restore write messages should be sent with level
XTL_DEBUG so that they don't disturb progress output even with -v.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
This patch eliminate the global variables in libxenctrl (used for
logging and error reporting).
Instead the information which was in the global variables is now in a
new xc_interface* opaque structure, which xc_interface open returns
instead of the raw file descriptor; furthermore, logging is done via
xentoollog.
There are three new parameters to xc_interface_open to control the
logging, but existing callers can just pass "0" for all three to get
the old behaviour.
All libxc callers have been adjusted accordingly.
Also update QEMU_TAG for corresponding qemu change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
500ms is aggressive enough to trigger split-brain under fairly
ordinary workloads, particularly for HVM. The long-term fix is to
integrate with a real HA monitor like linux HA.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
Allow certain performance-critical hypercall wrappers to register data
buffers via a new interface which allows them to be 'bounced' into a
pre-mlock'ed page-sized per-thread data area. This saves the cost of
mlock/munlock on every such hypercall, which can be very expensive on
modern kernels.
Signed-off-by: Keir Fraser <keir.fraser@citrix.com>
0b138a019292 was a little too ambitious replacing xc_map_foreign_batch
with xc_map_foreign_pages in xc_domain_restore. With HVM, some of the
mappings are expected to fail (as "XTAB" pages).
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
Replace all calls to xc_map_foreign_batch() where the caller doesn't
look at the passed in array to check for errors by calls to
xc_map_foreign_pages(). Replace all remaining calls by such to the
newly introduced xc_map_foreign_bulk().
As a sideband modification (needed while writing the patch to ensure
they're unused) eliminate unused parameters to
uncanonicalize_pagetable() and xc_map_foreign_batch_single(). Also
unmap live_p2m_frame_list earlier in map_and_save_p2m_table(),
reducing the peak amount of virtual address space required.
All supported OSes other than Linux continue to use the old ioctl for
the time being.
Also change libxc's MAJOR to 4.0 to reflect the API change.
Signed-off-by: Jan Beulich <jbeulich@novell.com>
(NOTE: pvrdtscp mode not finished yet, but all other
modes have been tested so sooner seemed better than
later to submit this fairly major patch so we can get
more mileage on it before next release.)
New tsc_mode config option supercedes tsc_native and
offers a more intelligent default and an additional
option for intelligent apps running on PV domains
("pvrdtscp").
For PV domains, default mode will determine if the initial
host has a "safe"** TSC (meaning it is always synchronized
across all physical CPUs). If so, all domains will
execute all rdtsc instructions natively; if not,
all domains will emulate all rdtsc instructions but
providing the TSC hertz rate of the initial machine.
After being restored or live-migrated, all PV domains will
emulate all rdtsc instructions. Hence, this default mode
guarantees correctness while providing native performance
in most conditions.
For PV domains, tsc_mode==1 will always emulate rdtsc
and tsc_mode==2 will never emulate rdtsc. For tsc_mode==3,
rdtsc will never be emulated, but information is provided
through pvcpuid instructions and rdtscp instructions
so that an app can obtain "safe" pvclock-like TSC information
across save/restore and live migration. (Will be completed in
a follow-on patch.)
For HVM domains, the default mode and "always emulate"
mode do the same as tsc_native==0; the other two modes
do the same as tsc_native==1. (HVM domains since 3.4
have implemented a tsc_mode=default-like functionality,
but also can preserve native TSC across save/restore
and live-migration IFF the initial and target machines
have a common TSC cycle rate.)
** All newer AMD machines, and Nehalem and future Intel
machines have "Invariant TSC"; many newer Intel machines
have "Constant TSC" and do not support deep-C sleep states;
these and all single-processor machines are "safe".
Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
The tail containing the final PFN table, VCPU contexts and
shared_info_page is buffered, then the read loop is restarted.
After the first pass, incoming pages are buffered until the next tail
is read, completing a new consistent checkpoint. At this point, the
memory changes are applied and the loop begins again. When the fd read
fails, the tail buffer is processed.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
This patch try to allocate 2M pages on target side based on analysis
of pfn sequence sent from source side for HVM migration.
The algorithm is: If pseudo-phys page is not yet populated in target
domain, AND it is first page of a 2MB extent, AND no other pages in
that extent are yet populated, AND the next pages in the save-image
stream populate that extent in order, THEN allocate a super page. If
the next 511 pages (to make the 2MB extent) are split across a batch
boundary, we have to optimistically allocate a super page in this
batch, and then break it into several 4K pages in the next batch,
which is speculative.
This patch is also friendly to PV guest migration.
Signed-Off-By: Zhai Edwin <edwin.zhai@intel.com>
Attached patch implements save/restore/migration/livemigration
for transcendent memory ("tmem"). Without this patch, domains
using tmem may in some cases lose data when doing save/restore
or migrate/livemigrate. Also included in this patch is
support for a new (privileged) hypercall for authorizing
domains to share pools; this provides the foundation to
accomodate upstream linux requests for security for shared
pools.
Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Since the shared info layout is fixed, guests are required to use
VCPUOP_register_vcpu_info prior to booting any vCPU beyond the
traditional limit of 32.
MAX_VIRT_CPUS, being an implemetation detail of the hypervisor, is no
longer being exposed in the public headers.
The tools changes are clearly incomplete (and done only so things
would
build again), and the current state of the tools (using scalar
variables all over the place to represent vCPU bitmaps) very likely
doesn't permit booting DomU-s with more than the traditional number of
vCPU-s. Testing of the extended functionality was done with Dom0 (96
vCPU-s, as well as 128 vCPU-s out of which the kernel elected - by way
of a simple kernel side patch - to use only some, resulting in a
sparse
bitmap).
ia64 changes only to make things build, and build-tested only (and the
tools part only as far as the build would go without encountering
unrelated problems in the blktap code).
Signed-off-by: Jan Beulich <jbeulich@novell.com>
This patch adds the option "superpages" to the domain configuration
file. If it is set, the domain is populated using 2M pages.
This code does not support fallback to small pages. If the domain can
not be created with 2M pages, the create will fail.
The patch also includes support for saving and restoring domains with
the superpage flag set. However, if a domain has freed small pages
within its physical page array and then extended the array, the
restore will fill in those freed pages. It will then attempt to
allocate more than its memory limit and will fail. This is
significant because apparently Linux does this during boot, thus a
freshly booted Linux image can not be saved and restored successfully.
Signed-off-by: Dave McCracken <dcm@mccr.org>
guest pfn width
(i.e. 32on64 domain 0 and 64 bit guest domain).
Otherwise P2M entries which were INVALID_P2M_ENTRY
(==0xffffffffffffffff) become 0xffffffff after a migrate.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Shriram Rajagopalan
Paul Durrant
George Dunlap
Ian Campbell
Olaf Hering
Stefano Stabellini
Anthony Perard
Keir Fraser
Brendan Cully
Stefano Stabellini
Keir Fraser