libxenctrl expects to be able to simply run the save or restore
operation synchronously. This won't work well in a process which is
trying to handle multiple domains.
The options are:
- Block such a whole process (eg, the whole of libvirt) while
migration completes (or until it fails).
- Create a thread to run xc_domain_save and xc_domain_restore on.
This is quite unpalatable. Multithreaded programming is error
prone enough without generating threads in libraries, particularly
if the thread does some very complex operation.
- Fork and run the operation in the child without execing. This is
no good because we would need to negotiate with the caller about
fds we would inherit (and we might be a very large process).
- Fork and exec a helper.
Of these options the latter is the most palatable.
Consequently:
* A new helper program libxl-save-helper (which does both save and
restore). It will be installed in /usr/lib/xen/bin. It does not
link against libxl, only libxc, and its error handling does not
need to be very advanced. It does contain a plumbing through of
the logging interface into the callback stream.
* A small ad-hoc protocol between the helper and libxl which allows
log messages and the libxc callbacks to be passed up and down.
Protocol doc comment is in libxl_save_helper.c.
* To avoid a lot of tedium the marshalling boilerplate (stubs for the
helper and the callback decoder for libxl) is generated with a
small perl script.
* Implement new functionality to spawn the helper, monitor its
output, provide responses, and check on its exit status.
* The functions libxl__xc_domain_restore_done and
libxl__xc_domain_save_done now turn out to want be called in the
same place. So make their state argument a void* so that the two
functions are type compatible.
The domain save path still writes the qemu savefile synchronously.
This will need to be fixed in a subsequent patch.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Change the internal and external APIs for domain save (suspend) to be
capable of asynchronous operation. The implementation remains
synchronous. The interfaces surrounding device model saving are still
synchronous.
Public API changes:
* libxl_domain_save takes an ao_how.
* libxl_domain_remus_start takes an ao_how. If the
libxl_domain_remus_info is NULL, we abort rather than returning an
error.
* The `suspend_callback' function passed to libxl_domain_save is
never called by the existing implementation in libxl. Abolish it.
* libxl_domain_save takes its flags parameter as an argument.
Thus libxl_domain_suspend_info is abolished.
* XL_SUSPEND_* flags renamed to LIBXL_SAVE_*.
* Callers in xl updated.
Internal code restructuring:
* libxl__domain_suspend_state member types and names rationalised.
* libxl__domain_suspend renamed from libxl__domain_suspend_common.
(_common here actually meant "internal function").
* libxl__domain_suspend takes a libxl__domain_suspend_state, which
where the parameters to the operation are filled in by the caller.
* xc_domain_save is now called via libxl__xc_domain_save which can
itself become asynchronous.
* Consequently, libxl__domain_suspend is split into two functions at
the callback boundary; the second half is
libxl__xc_domain_save_done.
* libxl__domain_save_device_model is now called by the actual
implementation rather than by the public wrapper. It is already in
its proper place in the domain save execution sequence. So
officially make it part of that execution sequence, renaming it to
domain_save_device_model.
* Effectively, rewrite the public wrapper functions
libxl_domain_suspend and libxl_domain_remus_start.
* Remove a needless #include <xenctrl.h>
* libxl__domain_suspend aborts on unexpected domain types rather
than mysteriously returning EINVAL.
* struct save_callbacks moved from the stack to the dss.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
We are going to arrange that libxl, instead of calling
xc_domain_restore, calls a stub function which forks and execs a
helper program, so that restore can be asynchronous rather than
blocking the whole toolstack.
This stub function will be called libxl__xc_domain_restore.
However, its prospective call site is unsuitable for a function which
needs to make a callback, and is buried in two nested single-call-site
functions which are logically part of the domain creation procedure.
So we first abolish those single-call-site functions, integrate their
contents into domain creation in their proper temporal order, and
break out libxl__xc_domain_restore ready for its reimplementation.
No functional change - just the following reorganisation:
* Abolish libxl__domain_restore_common, as it had only one caller.
Move its contents into (what was) domain_restore.
* There is a new stage function domcreate_rebuild_done containing what
used to be the bulk of domcreate_bootloader_done, since
domcreate_bootloader_done now simply starts the restore (or does the
rebuild) and arranges to call the next stage.
* Move the contents of domain_restore into its correct place in the
domain creation sequence. We put it inside
domcreate_bootloader_done, which now either calls
libxl__xc_domain_restore which will call the new function
domcreate_rebuild_done, or calls domcreate_rebuild_done directly.
* Various general-purpose local variables (`i' etc.) and convenience
alias variables need to be shuffled about accordingly.
* Consequently libxl__toolstack_restore needs to gain external linkage
as it is now in a different file to its user.
* Move the xc_domain_save callbacks struct from the stack into
libxl__domain_create_state.
In general the moved code remains almost identical. Two returns in
what used to be libxl__domain_restore_common have been changed to set
the return value and "goto out", and the call sites for the abolished
and new functions have been adjusted.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Preparatory work for making domain suspend asynchronous:
* Rename `struct suspendinfo' to `libxl__domain_suspend_state'
and move it to libxl_internal.h.
* Rename variables `si' to `dss'.
* Change the stack-allocated state and callbacks from
struct suspendinfo si;
struct save_callbacks callbacks;
struct restore_callbacks callbacks;
to
libxl__domain_suspend_state dss[1];
struct save_callbacks callbacks[1];
struct restore_callbacks callbacks[1];
so that it may be referred to as a pointer variable everywhere.
* Rename the variable `flags' (in libxl__domain_suspend_state) to
`xcflags', to help distinguish it from the other `flags' which is
passed in from the calling application in libxl_domain_suspend_info.
Abolish the local variable in libxl__domain_suspend_common, as it
can use the one in the dss.
* Move the prototypes of suspend-related functions in libxl_internal.h
to after the definition of the state struct.
* Replace several ctx variables with gc variables and
consequently references to ctx with CTX. Change references
to `dss->gc' in the functional code to simply `gc'.
* Use LOG* rather than LIBXL__LOG* in a number of places.
* In libxl__domain_save_device_model use `rc' instead of `ret'.
* Introduce and use `gc' and `domid' in
libxl__domain_suspend_common_callback.
* Wrap some long lines.
* Add an extra pair of parens for clarity in a flag test.
* Remove two pointless casts from void* to a struct*.
No functional change whatsoever.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
In xc_domain_save the local variable `ob' is initialised to NULL.
There are then various startup actions. Some of these `goto out' on
failure; for example the call to callbacks->switch_qemu_logdirty on
l.978. However, out is used both by success and error paths. So it
attempts (l.2043) to flush the current output buffer. If ob has not
yet been assigned a non-NULL value, this segfaults. So make the call
to outbuf_flush conditional on ob.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
In current implementation, it uses integer to record current avail
cpus and this only allows user to specify 31 vcpus. In following
patch, it uses cpumap instead integer which make more sense than
before. Also there is no limit to the max vcpus.
Signed-off-by: Yang Zhang <yang.z.zhang@Intel.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Currently, libxl_cpumap_alloc()allocate the cpumap with size of number
of physical cpus. In some place, we may want to allocate specific size
of cpumap. This patch allow to pass a argument to specific the size
that you want to allocate. If pass 0, it means the size is equal to
number of physical cpus.
Signed-off-by: Yang Zhang <yang.z.zhang@Intel.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Testing has shown that doing sweeps for superpages slows down boot
significantly, but does not result in a significantly higher number of
superpages after boot. Early sweeping for 4k pages causes superpages
to be broken up unnecessarily.
Only sweep if we're really out of memory.
v2:
- Move unrelated code-motion hunk to another patch
v3:
- Remove now-unused reclaim_super from pod struct
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Committed-by: Tim Deegan <tim@xen.org>
When demand-populating pages due to guest accesses, check recently populated
pages to see if we can reclaim them for the cache. This should keep the PoD
cache filled when the start-of-day scrubber is going through.
The number 128 was chosen by experiment. Windows does its page
scrubbing in parallel; while a small nubmer like 4 works well for
single VMs, it breaks down as multiple vcpus are scrubbing different
pages in parallel. Increasing to 128 works well for higher numbers of
vcpus.
v2:
- Wrapped some long lines
- unsigned int for index, unsigned long for array
v3:
- Use PAGE_ORDER_2M instead of 9
- Removed inappropriate use of p2m_pod_zero_check_superpage() return value
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Committed-by: Tim Deegan <tim@xen.org>
Windows balloon drivers can typically only get 4k pages from the kernel,
and so hand them back at that level. Try to regain superpages by checking
the superpage frame that the 4k page is in to see if we can reclaim the whole
thing for the PoD cache.
This also modifies p2m_pod_zero_check_superpage() to return SUPERPAGE_PAGES on
success.
v2:
- Rewritten to simply to the check as in demand-fault case, without needing
to know that the p2m entry is a superpage.
- Also, took out the re-writing of the reclaim loop, leaving it optimized for
4k pages (by far the most common case), and simplifying the patch.
v3:
- Add SoB
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Committed-by: Tim Deegan <tim@xen.org>
25479:61dfb3da56b0 added a .PRECIOUS which causes these files to be
left over more often. They should have been ignored already, though.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
msitranslate is known to cause problems with some device drivers,
because it sets the real device in MSI mode while making the guest think
is actually in legacy interrupts mode. Some drivers are able to spot this
inconsistency and break (Nvidia drivers for example).
Disable msitranslate by default.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Only x86 currently has a struct vcpu field arch.gdbsx_vcpu_event. But
as the whole function domain_pause_for_debugger() is pointless to be
compiled when there's no arch support, simply introduce another HAS_*
macro, enabled only on x86.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Keir Fraser <keir@xen.org>
In its current shape, if Xen unshares a foreign gfn successfully while building
a foreign writable map, it is left with a reference to the old shared page in
the "target" var.
Instead, push unsharing request down on the initial get_page_from_gfn call,
which will DTRT.
This allows for greatly simplifying the unshare related condition handling,
removing ugly comments and s86_64 ifdef-ery.
Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
Acked-by: Tim Deegan <tim@xen.org>
Committed-by: Tim Deegan <tim@xen.org>
Since the emulator now supports a few 256-bit memory operations, this
array needs to follow (and the comments should, too).
To limit growth, re-order the mmio_large_write_* fields so that the
two mmio_large_*_bytes fields end up adjacent to each other.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
There's no apparent reason not to permit this, and since we don't
support out-of-source-tree builds, the least overhead way of doing
multiple, differently configured (perhaps different architecture)
builds from a single source tree is to create symlinked build trees.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Causing a build failure when trying to use it:
xxx: error: expected ';' before ')' token
xxx: error: expected statement before ')' token
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
For some reason we were holding the lock over only the unmaps at the end of
the function, rather than for the whole walk.
We might want to be more clever in the future, but for now lets just lock for
the whole walk+create process.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
This needs to be done for all cpus. The allocations require smp_prepare_cpus
to be called a bit later on.
In a previous version of this patch these maps were being zeroed (instead of
setting the CPU itself in them). This in turn causes cpumask_first to return
NR_CPUS, which in turn was causing default_vcpu0_location to misbehave and
read off the end of its cnt array.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
XEN_DOMCTL_max_vcpus cannot reduce max_vcpus and therefore we can't create a
smaller guest.
The limit of 8 (due to GIC limits) should be expressed in MAX_VIRT_CPUS.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
For most traps we can do this as soon as we have saved the necessary state.
For IRQs and FIQs we must wait until we have acked the interrupt with the GIC.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
We align the stack before calling into C code but we weren't undoing this on
return.
Collapse continue_(non)idle_domain into continue_new_vcpu.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
Include helpers for dumping hypervisor walks and guest p2m walks.
Useful for debug but not actually used in this patch.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
As was already pointed out months ago (see
http://lists.xen.org/archives/html/xen-devel/2012-02/msg00826.html),
gcc 4.7.x (imo validly) refuses to take both -mcpu=cortex-a15 and
-march=armv7-a due to conflicting feature sets causing amibiguity in
instruction selection. Since the former implies the latter, just use
the former (and drop the -march=).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Fixes crashes with Windows guests when shadow-on-nested is used.
Signed-off-by: Christoph Egger <Christoph.Egger@amd.com>
Acked-by: Tim Deegan <tim@xen.org>
2205914617cb does its job in correcting the "wrong domain being
considered" issue introduced by 9d1fd58ff602. Unfortunately, when
dealing (again!) with the sedf scheduler, it is required for the
vCPUs of a domain to have been allocated and setup already (in
the hypervisor), when the first call to libxl_domain_sched_params_get()
happens, and that is not true.
This fixes that by avoiding calling that function at all, as we
only need to know which scheduler the domain is running under,
and that is provided by libxl__domain_scheduler() which is safe
to be called there.
While at it, also improve a bit the comments about the whole
sedf parameter validation and mangling process.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
When attempting to use AMD's extension to access the extended PCI config
space, only the low byte of the register number was being passed to XSM.
Include the correct value of the register if this feature is enabled;
otherwise, bits 24-30 of port cf8 are reserved, so disallow the invalid
access.
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Don't fail the permission check except when the MSR can't be read.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Committed-by: Jan Beulich <jbeulich@suse.com>
Recent Dom0 kernels want to disable PCI MSI on all devices, yet doing
so on AMD IOMMUs (which get represented by a PCI device) disables part
of the functionality set up by the hypervisor.
Add a mechanism to mark certain PCI devices as having write protected
config spaces (both through port based [method 1] accesses and, for
x86-64, mmconfig), and use that for AMD's IOMMUs.
Note that due to ptwr_do_page_fault() being run first, there'll be a
MEM_LOG() issued for each such mmconfig based write attempt. If that's
undesirable, the order of the calls in fixup_page_fault() would need
to be swapped.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Wei Wang <wei.wang2@amd.com>
Acked-by: Keir Fraser <keir@xen.org>
This was previously done by xl itself however the domain was not
created at that point so there was no domid to check. This happened to
work on first boot because xl's global domid was initialised to zero
so we would (incorrectly) validate the new domain to be against
domain0. On reboot though we would try to use the old domain's id and
fail.
sched_params_valid is moved and gains a gc+domid parameters and
s/ctx/CTX/. The call is placed after
libxl__domain_build_info_setdefault in the create path, because
set_defaults doesn't have access to the domid and there are other
callers which don't even have a domid to give it.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
These additions did not fulfill their purpose - they checked hypervisor
config space accesses instead of guest (Dom0) ones.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Ian Jackson
Yang Zhang
George Dunlap
Ian Campbell
Stefano Stabellini
Dario Faggioli
Jan Beulich
Andres Lagar-Cavilla
Dario Faggioli
Christoph Egger
Daniel De Graaf