Instead of sending dirty pages of guest memory as-is, use a simple compression
algorithm that sends a RLE-encoded XOR of the page against its last sent copy.
A small LRU cache is used to hold recently dirtied pages. Pagetable pages are
sent as-is, as they are canonicalized at sender side and uncanonicalized at
receiver.
[ Fixed up a conflict in sg_save_restore.h. I had to increase the
ID values used from -11 and -12 to -12 and -13 because -11 had
been taken by ..._HVM_VIRIDIAN in the meantime. -iwj ]
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca>
Acked-by: Brendan Cully <brendan@cs.ubc.ca>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
xc_domain_save/restore currently pay no attention to
HVM_PARAM_VIRIDIAN which results in an HVM domain running a recent
version on Windows (post-Vista) locking up on a domain restore due to
EOIs (done via a viridian MSR write) being silently dropped. This
patch adds an extra save entry for the viridian parameter and also
adds code in the viridian kernel module to catch attempted use of
viridian functionality when the HVM parameter has not been set.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Committed-by: Keir Fraser <keir@xen.org>
.. as there is no need to keep that internal header file anymore.
We export two functions xc_domain_[set|get]_memory_map which
depend on the 'struct e820entry' defined in 'xc_e820.h'.
We move the contents of the 'xc_e820.h' to the 'xenctrl.h' fixing
compiler errors when applications outside the Xen tree are trying
to compile against the libraries.
Tested-by: Daniel Castro <evil.dani@gmail.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Bitmaps are used in save/restore, xenpaging and blktap2. Merge the code into a
private xc_bitops.h file. All users are single threaded, so locking is not an
issue. The array of bits is handled as volatile because the x86 save/restore
code passes the bitmap to the hypervisor which in turn modifies the bitmap.
blktap2 uses a private bitmap. There was a possible overflow in the
bitmap_size() function, the remainder was not considered.
ia64 save/restore uses a bitmap to send the number of vcpus to the host.
x86 save/restore uses a bitmap to track dirty pages. This bitmap is shared with
the hypervisor. An unused function count_bits() was removed and a new
bitmap_size() function is now used.
xenpaging uses 3 private bitmaps to track the gfns which are in paged-out
state. It had a copy of some Linux bitops.h, which is now obsolete. Also the
BITS_PER_LONG macro was hardcoded to 64 which made it impossible to run 32bit
tools on a 64bit host. Wether this works at all has to be tested, yet.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
To detect presence of superpages on the receiver side, we need
to have strings of sequential pfns sent across on the first iteration
through the memory. However, as we go through the memory, more and
more of it will be marked dirty, making it wasteful to send those pages.
This patch introduces a new PFINFO type, "XALLOC". Like PFINFO_XTAB, it
indicates that there is no corresponding page present in the subsquent
page buffer. However, unlike PFINFO_XTAB, it contains a pfn which should be
allocated.
This new type is only used for migration; but it's placed in
xen/public/domctl.h so that the value isn't reused.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
On the first time through (when pfns are mostly allocated on
the receiving side), try to keep superpages together in the same
batch by ending a batch early if we see the first page of a
potential superpage and there isn't enough room in the batch
for a full superpage.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
As a precursor to making non-static.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
so drop it.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
It doesn't do any ratelimiting...
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
The ifdef was added in 2005 (7702:b3c2bc39d815) but, as far as I can see, was
never enabled by default. Dropping it will help untangle some macros redefining
functions etc.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
20544:ad9d75d74bd5 and 20545:cc7d66ba0dad seemingly intended to change these
global static variables into stack variables but didn't remove the static
qualifier.
Also zero the entire struct once with memset rather than clearing fields
piecemeal in two different places.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Acked-by: Vincent Hanquez <vincent.hanquez@eu.citrix.com>
With some help from Olaf, I've finally got to the bottom of an issue I
came across while trying to implement save/restore in the libvirt
libxenlight driver. After issuing the save operation, the saved domain
was not being cleaned up properly and left in this state from xl's
perspective
xen33:# xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 6821 8 r----- 122.5
(null) 2 2 2 --pssd 10.8
Checking the libvirtd /proc/$pid/maps I found this
7f3798984000-7f3798b86000 r--s 00002000 00:03 4026532097 /proc/xen/privcmd
So not all all pages belonging to the domain were unmapped from
libvirtd. In tools/libxc/xc_domain_save.c we found that P2M_FL_ENTRIES
were being mapped but only P2M_FLL_ENTRIES were being unmapped. The
attached patch changes the unmapping to use the same P2M_FL_ENTRIES
macro. I'm not too familiar with this code though so posting here for
review.
I suspect this was not noticed before since most (all?) processes doing
save terminate after the save and are not long-running like libvirtd.
Ian Campbell writes:
> Looks like I introduced this in 18558:ccf0205255e1, sorry!
>
> I guess it is also wrong in the error path out of map_and_save_p2m_table
> and so we also need [another hunk].
This change should be backported to relevant earlier trees. -iwj
From: Jim Fehlig <jfehlig@novell.com>
From: Ian Campbell <Ian.Campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Olaf Hering <olaf@aepfle.de>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
The domain save code will read an HVM param, and if it's not zero,
make an entry for it. However, if the hypercall fails for any reason,
the data may not be written, and the value for the previous parameter
may be written in the save file as the parameter that failed.
Initialize the value to zero before each hypercall, so that in case of
a failure, no value will be written.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
I have tested the patch in the following senarios:
1> Non-xsave platform
2> Xsave-capable platform, guest does not support xsave, xen support xsave
3> Xsave-capable platform, guest does support xsave, xen supports xsave
4> Guest (non-xsave) saved on platform without xsave, restored on a
Xsave-capable system.
All passed.
Signed-off-by: Shan Haitao <haitao.shan@intel.com>
Signed-off-by: Han Weidong <weidong.han@intel.com>
On restore:
region_mfn is passed to xc_map_foreign_range and
xc_map_foreign_bulk. In both cases the buffer is accessed from the
ioctl handler in the kernel and not from any hypercall. Therefore
normal copy_{to,from}_user handling in the kernel will cope with any
faulting access.
p2m_batch is passed to xc_domain_memory_populate_physmap which takes
care of bouncing the buffer already.
On save:
pfn_type is passed to xc_map_foreign_bulk which does not need locking
as per region_mfn above.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson.citrix.com>
The callback return convention follows xc_domain_save and therefore
returns 0 on success and >0 on error, correct the check accordingly.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Move the function into struct save_callbacks with the others and add
the void *closure to the callback arguments.
Add and propagate an error return code from the callback.
Use this in libxl to pass the save context to
libxl__domain_suspend_common_switch_qemu_logdirty allowing us to reuse
the parent's xenstore handle, gc context etc.
Also add an apparently missing libxl__free_all to
libxl__domain_suspend_common.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Not actually used here but useful to confirm that a handle is passed
down to each location where it will be required once we switch to
hypercall buffers.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
When the restore code sees this notification it will restore the
currently in-progress checkpoint when it completes.
This allows the restore end to finish up without waiting for a
spurious timeout on the receive fd and thereby avoids unnecessary
error logging in the case of a successful migration or restore.
In the normal migration or restore case the first checkpoint is always
the last. For a rolling checkpoint (such as Remus) the notification is
currently unused but could be used in the future for example to
provide a controlled failover for reasons other than error
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Brendan Cully <brendan@cs.ubc.ca>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
In xc_domain_save ensure that we signal completion at the end of each
batch.
This ensures that the next logged line starts on a new line. e.g. instead of:
Savefile contains xl domain config
xc: Saving memory: iter 3 (last sent 0 skipped 0): 0/32768 0%migration target: Transfer complete, requesting permission to start domain.
migration sender: Target has acknowledged transfer.
what is desired is:
Savefile contains xl domain config
xc: Saving memory: iter 0 (last sent 0 skipped 0): 32768/32768 100%
xc: Saving memory: iter 1 (last sent 32576 skipped 192): 32768/32768 100%
xc: Saving memory: iter 2 (last sent 217 skipped 0): 32768/32768 100%
xc: Saving memory: iter 3 (last sent 0 skipped 0): 32768/32768 100%
migration target: Transfer complete, requesting permission to start domain.
migration sender: Target has acknowledged transfer.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reverse engineered from the code, likely contains inaccuracies but I
think provides a base to work from.
Add symbolic names for the minus-flags.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1282671421 -3600
# Node ID d1dd29a470ef1b9d2c77478a123326036dfe90bb
# Parent d7a4adad9c328decbd384d87b23001aea8951b86
tools/libxc, tools/libelf: Relicense under LGPL v2.1
Relicense these two libraries under LGPL v2.1 only except where
individual files already included the "or later" provision.
Copyright holders have been contacted by Stephen Spector and have all
agreed this change.
Removed tools/libxc/ia64/aclinux.h since it appeared to be
unused. There is a separate, more up to date, copy in
xen/include/acpi/platform/aclinux.h which does appear to be used.
Clarify the license of MiniOS privcmd.h under the same terms as other
tools/include/xen-sys headers.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stephen Spector <stephen.spector@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Creates a shared memory ring buffer and event channel in HVM domains
for passing debug messages from PV drivers on HVM guests
The console is used by windows pv drivers to send debug data to xenconsoled
Signed-off-by: Owen Smith <owen.smith@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
As discussed in the thread starting at
http://lists.xensource.com/archives/html/xen-devel/2010-05/msg01383.html,
don't save Xen heap pages in order to avoid overallocation when the
domain gets restored, as those pages would get (temporarily) backed
with normal RAM pages by the restore code.
This requires making DOMCTL_getpageframeinfo{2,3} usable for HVM
guests, meaning that the input to these must be treated as GMFNs.
Signed-off-by: Jan Beulich <jbeulich@novell.com>
Previously, the code was inconsistent: some calls to PERROR passed \n
and some did not. With the new logging arrangements, passing \n is
definitely incorrect.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
* Make "read_exact" in libxc always set errno.
* Rename "read_exact" macro in xc_domain_restore.c (which shadows
real function) to RDEXACT and change all callers.
* Make RDEXACT anamorphically use xch for error reporting rather than
* stderr.
* Call PERROR rather than ERROR when appropriate, so that log messages
include errno.
* Save errno in noncached_write so that its errno value is always
* right.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
oubuf and batch restore write messages should be sent with level
XTL_DEBUG so that they don't disturb progress output even with -v.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
This patch eliminate the global variables in libxenctrl (used for
logging and error reporting).
Instead the information which was in the global variables is now in a
new xc_interface* opaque structure, which xc_interface open returns
instead of the raw file descriptor; furthermore, logging is done via
xentoollog.
There are three new parameters to xc_interface_open to control the
logging, but existing callers can just pass "0" for all three to get
the old behaviour.
All libxc callers have been adjusted accordingly.
Also update QEMU_TAG for corresponding qemu change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
There is already a "write_exact" function declared in xc_private.h.
So rename this macro to to wrexact.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
The final, flushing call to discard_file_cache also discards any
errors from fsync. Call fsync explicitly before leaving, to check if
all VM memory actually made it to the disk.
Signed-off-by: Daniel Stodden <daniel.stodden@citrix.com>
Just like the kernel, the fallback implementation of
xc_map_foreign_bulk() should clear the error indication array upon
success.
Also, a few allocations were needlessly using calloc() instead of
malloc().
Finally, in xc_domain_save() allocate the error indicator array once
(along with the other arrays) instead of using realloc() (without
error checking) in the loop body.
Signed-off-by: Jan Beulich <jbeulich@novell.com>
Allow certain performance-critical hypercall wrappers to register data
buffers via a new interface which allows them to be 'bounced' into a
pre-mlock'ed page-sized per-thread data area. This saves the cost of
mlock/munlock on every such hypercall, which can be very expensive on
modern kernels.
Signed-off-by: Keir Fraser <keir.fraser@citrix.com>
Due to recent changes related to xc_map_foreign_bulk, xc_domain_save
segfaults because it tries to use pfn_err without allocating it first.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
To support wider than 28-bit MFNs, add XEN_DOMCTL_getpageframeinfo3
(with the type replacing the passed in MFN rather than getting or-ed
into it) to properly back xc_get_pfn_type_batch().
With xc_get_pfn_type_batch() only used internally to libxc, move its
prototype from xenctrl.h to xc_private.h.
This also fixes a couple of bugs in pre-existing code:
- the failure path for init_mem_info() leaked minfo->pfn_type,
- one error path of the XEN_DOMCTL_getpageframeinfo2 handler used
put_domain() where rcu_unlock_domain() was meant, and
- the XEN_DOMCTL_getpageframeinfo2 handler could call
xsm_getpageframeinfo() with an invalid struct page_info pointer.
Signed-off-by: Jan Beulich <jbeulich@novell.com>
Replace all calls to xc_map_foreign_batch() where the caller doesn't
look at the passed in array to check for errors by calls to
xc_map_foreign_pages(). Replace all remaining calls by such to the
newly introduced xc_map_foreign_bulk().
As a sideband modification (needed while writing the patch to ensure
they're unused) eliminate unused parameters to
uncanonicalize_pagetable() and xc_map_foreign_batch_single(). Also
unmap live_p2m_frame_list earlier in map_and_save_p2m_table(),
reducing the peak amount of virtual address space required.
All supported OSes other than Linux continue to use the old ioctl for
the time being.
Also change libxc's MAJOR to 4.0 to reflect the API change.
Signed-off-by: Jan Beulich <jbeulich@novell.com>
(NOTE: pvrdtscp mode not finished yet, but all other
modes have been tested so sooner seemed better than
later to submit this fairly major patch so we can get
more mileage on it before next release.)
New tsc_mode config option supercedes tsc_native and
offers a more intelligent default and an additional
option for intelligent apps running on PV domains
("pvrdtscp").
For PV domains, default mode will determine if the initial
host has a "safe"** TSC (meaning it is always synchronized
across all physical CPUs). If so, all domains will
execute all rdtsc instructions natively; if not,
all domains will emulate all rdtsc instructions but
providing the TSC hertz rate of the initial machine.
After being restored or live-migrated, all PV domains will
emulate all rdtsc instructions. Hence, this default mode
guarantees correctness while providing native performance
in most conditions.
For PV domains, tsc_mode==1 will always emulate rdtsc
and tsc_mode==2 will never emulate rdtsc. For tsc_mode==3,
rdtsc will never be emulated, but information is provided
through pvcpuid instructions and rdtscp instructions
so that an app can obtain "safe" pvclock-like TSC information
across save/restore and live migration. (Will be completed in
a follow-on patch.)
For HVM domains, the default mode and "always emulate"
mode do the same as tsc_native==0; the other two modes
do the same as tsc_native==1. (HVM domains since 3.4
have implemented a tsc_mode=default-like functionality,
but also can preserve native TSC across save/restore
and live-migration IFF the initial and target machines
have a common TSC cycle rate.)
** All newer AMD machines, and Nehalem and future Intel
machines have "Invariant TSC"; many newer Intel machines
have "Constant TSC" and do not support deep-C sleep states;
these and all single-processor machines are "safe".
Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Shriram Rajagopalan
Paul Durrant
Konrad Rzeszutek Wilk
Olaf Hering
George Dunlap
Ian Campbell
Jim Fehlig
Anthony Perard
Keir Fraser
Stefano Stabellini
Keir Fraser