The call site is enclosed by x86 define guards.
Without this patch:
[ 334s] xc_resume.c:112:12: error: 'xc_domain_resume_hvm' defined but not used [-Werror=unused-function]
[ 334s] static int xc_domain_resume_hvm(xc_interface *xch, uint32_t domid)
Reported-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
The patches adding Xen tools support for paravirtualized USB devices
(pvUSB) omitted documenting the introduced Xenstore paths.
Add the paths to docs/misc/xenstore-paths.markdown
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
The hyperlinks for the different I/O protocols are wrong.
Correct them.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
The code in hvm/hvm.c related to handling I/O emulation using the ioreq
server framework is large and mostly self-contained.
This patch separates the ioreq server code into a new hvm/ioreq.c source
module and accompanying asm-x86/hvm/ioreq.h header file. There is no
intended functional change, only code movement.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Add XEN_DOMCTL_SCHEDOP_getvcpuinfo and _putvcpuinfo hypercalls
to independently get and set the scheduling parameters of each
vCPU of a domain.
Also fix a bug in XEN_DOMCTL_SCHEDOP_getinfo, where PERIOD and
BUDGET are not divided by MICROSECS(1) before being retruned
to the caller.
Signed-off-by: Chong Li <chong.li@wustl.edu>
Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>
Signed-off-by: Sisu Xi <xisisu@gmail.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Dario Faggioli <dario.faggioli@citrix.com>
This was introduced by 97ee1f (~5 years ago), but was probably never
surfaced because most people used regular files as CDROM images, so the PHY
backend was actually never selected. A year ago this was changed, and now
regular RAW files are also handled by the PHY backend, which has made this
bug suface.
Fix it by allowing empty disks to use the PHY backend, skipping the stat
tests.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reported-by: Alex Braunegg <alex.braunegg@gmail.com>
Current implementation of elf_load_bsdsyms is broken when loading inside of
a HVM guest, because it assumes elf_memcpy_safe is able to write into guest
memory space, which it is not.
Take the oportunity to do some cleanup and properly document how
elf_{parse/load}_bsdsyms works. The new implementation uses elf_load_image
when dealing with data that needs to be copied to the guest memory space.
Also reduce the number of section headers copied to the minimum necessary.
This patch also removes the duplication of code found in the libxc ELF
loader, since the libelf symtab/strtab loading code will also handle this
case without having to duplicate it.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
And use it as the default value for the VGA kind. This allows libxl to set
it to the default value later on when the domain type is known. For HVM
guests the default value is LIBXL_VGA_INTERFACE_TYPE_CIRRUS while for
HVMlite the default value is LIBXL_VGA_INTERFACE_TYPE_NONE.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
(i.e., domain creation and destruction) so the
trace will show properly decoded info, rather
than just a bunch of hex codes.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Before this patch:
1. Suspend
a. PVHVM and PV: we use the same way to suspend the guest (send the
suspend request to the guest). If the guest doesn't support evtchn, the
xenstore variant will be used, suspending the guest via XenBus control
node.
b. Pure HVM: we call xc_domain_shutdown(..., SHUTDOWN_suspend) to suspend
the guest.
2. Resume:
a. Fast path (fast=1)
Do not change the guest state. We call libxl__domain_resume(.., 1) which
calls xc_domain_resume(..., 1 /* fast=1*/) to resume the guest.
PV: Modify the return code to 1, and than call the domctl:
XEN_DOMCTL_resumedomain
PVHVM: same with PV
Pure HVM: Do nothing in modify_returncode, and than call the domctl:
XEN_DOMCTL_resumedomain
b. Slow path (fast=0)
Used when the guest's state have been changed. Will call
libxl__domain_resume(..., 0) to resume the guest.
PV: Update start info, and reset all secondary CPU states. Than call
the domctl: XEN_DOMCTL_resumedomain
PVHVM: Can not be resumed. You will get the following error message:
"Cannot resume uncooperative HVM guests"
Pure HVM: Same with PVHVM
After this patch:
1. Suspend
Unchanged.
2. Resume
a. Fast path
Unchanged.
b. Slow path
PV: Unchanged.
PVHVM: Call XEN_DOMCTL_resumedomain to resume the guest. Because we
don't modify the return code, PV drivers will disconnect
and reconnect.
The guest ends up doing the XENMAPSPACE_shared_info
XENMEM_add_to_physmap hypercall and resetting all of its CPU
states to point to the shared_info (except the ones past 32).
That is the Linux kernel does that - regardless whether the
SCHEDOP_shutdown:SHUTDOWN_suspend returns 1 or not.
Pure HVM: Call XEN_DOMCTL_resumedomain to resume the guest.
Under COLO, we will update the guest's state (modify memory, CPU registers,
device state etc). In this case, we cannot use the fast path to resume it.
Keep the return code 0, and use the slow path to resume the guest.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
[ wei: reformat commit message a bit ]
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Add cmdline switches to 'xl migrate-receive' command to specify
a domain-specific hotplug script to setup COLO proxy.
Add a new config var 'colo.default.agentscript' to xl.conf, that
allows the user to override the default global script used to
setup COLO proxy.
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Use qemu block replication as our block replication solution.
Note that guest must be paused before starting COLO, otherwise,
the disk won't be consistent between primary and secondary.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
We will use qemu block replication, and qemu provides some qmp commands
to prepare replication, start replication, get replication error, and
stop replication. Introduce new API to execute these qmp commands.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Add a new option -c to the command 'xl remus'. If you want
to use COLO HA instead of Remus HA, please use -c option.
Update man pages to reflect the addition of a new option to
'xl remus' command.
Also add a new option --colo to the internal command 'xl migrate-receive'.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
a. call callbacks resume/checkpoint/suspend while secondary vm
status is consistent with primary
b. send dirty pfn list to primary when checkpoint under colo
c. send store gfn and console gfn to xl before resuming secondary vm
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
We will do the following things again and again:
1. Suspend primary vm
a. Suspend primary vm
b. do postsuspend
c. Read CHECKPOINT_SVM_SUSPENDED sent by secondary
2. Checkpoint
a. Write emulator xenstore data and emulator context
b. Write checkpoint end record
3. Resume primary vm
a. Read CHECKPOINT_SVM_READY from slave
b. Do presume
c. Resume primary vm
d. Read CHECKPOINT_SVM_RESUMED from slave
4. Wait a new checkpoint
a. Wait a new checkpoint(not implemented)
b. Send CHECKPOINT_NEW to slave
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
No functional changes and this cleanup will make the later
patch called "primary vm suspend/resume/checkpoint code" not
too complicated.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Secondary vm is running in colo mode. So we will do
the following things again and again:
1. Resume secondary vm
a. Send CHECKPOINT_SVM_READY to master.
b. If it is not the first resume, call libxl__checkpoint_devices_preresume().
c. If it is the first resume(resume right after live migration),
- call libxl__xc_domain_restore_done() to build the secondary vm.
- enable secondary vm's logdirty.
- call libxl__domain_resume() to resume secondary vm.
- call libxl__checkpoint_devices_setup() to setup checkpoint devices.
d. Send CHECKPOINT_SVM_RESUMED to master.
2. Wait a new checkpoint
a. Call libxl__checkpoint_devices_commit().
b. Read CHECKPOINT_NEW from master.
3. Suspend secondary vm
a. Suspend secondary vm.
b. Call libxl__checkpoint_devices_postsuspend().
c. Send CHECKPOINT_SVM_SUSPENDED to master.
4. Checkpoint
a. Read emulator xenstore data and emulator context
b. REC_TYPE_CHECKPOINT_END
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
This is used by primay to read records sent by secondary.
Note: The function libxl__stream_read_checkpoint_state() will be used
in later patches called "secondary vm suspend/resume/checkpoint code" and
"primary vm suspend/resume/checkpoint code".
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Add back channel support to write stream. If the write stream is
a back channel stream, this means the write stream is used by
Secondary to send some records back.
Note: The function libxl__stream_write_checkpoint_state() will be used
in later patches called "secondary vm suspend/resume/checkpoint code" and
"primary vm suspend/resume/checkpoint code".
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
read_record() could be used by primary to read dirty bitmap
record sent by secondary under COLO.
When used by xc save side, we need to pass the backchannel fd
instead of ctx->fd to read_record(), so we added a fd param to
it.
No functional changes.
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
In COLO mode, both VMs are running, and are considered in sync if the
visible network traffic is identical. After some time, they fall out of
sync.
At this point, the two VMs have definitely diverged. Lets call the
primary dirty bitmap set A, while the secondary dirty bitmap set B.
Sets A and B are different.
Under normal migration, the page data for set A will be sent from the
primary to the secondary.
However, the set difference B - A (the one in B but not in A, lets
call this C) is out-of-date on the secondary (with respect to the
primary) and will not be sent by the primary (to secondary), as it
was not memory dirtied by the primary. The secondary needs C page data
to reconstruct an exact copy of the primary at the checkpoint.
The secondary cannot calculate C as it doesn't know A. Instead, the
secondary must send B to the primary, at which point the primary
calculates the union of A and B (lets call this D) which is all the
pages dirtied by both the primary and the secondary, and sends all page
data covered by D.
In the general case, D is a superset of both A and B. Without the
backchannel dirty bitmap, a COLO checkpoint can't reconstruct a valid
copy of the primary.
We transfer the dirty bitmap on libxc side, so we need to introduce back
channel to libxc.
Note: it is different from the paper. We change the original design to
the current one, according to our following concerns:
1. The original design needs extra memory on Secondary host. When there's
multiple backups on one host, the memory cost is high.
2. The memory cache code will be another 1k+, it will make the review
more time consuming.
Note: this patch merely adds new parameters to various prototypes and
functions. The new parameters are used in later patch called
"libxc/restore: send dirty pfn list to primary when checkpoint under
COLO".
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
In COLO mode, secondary needs to send the following data to primary:
1. In libxl
Secondary sends the following CHECKPOINT_CONTEXT to primary:
CHECKPOINT_SVM_SUSPENDED, CHECKPOINT_SVM_READY and CHECKPOINT_SVM_RESUMED
2. In libxc
Secondary sends the dirty pfn list to primary
But the io_fd only can be written in primary, and only can be read in
secondary. Save recv_fd in domain_suspend_state, and send_fd in
domain_create_state. Extend libxl_domain_create_restore API, add a
send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate
the API change.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Secondary vm is running in COLO mode, we need to send secondary
vm's dirty page information to primary host at checkpoint, so we
have to enable qemu logdirty on secondary.
libxl__domain_suspend_common_switch_qemu_logdirty() is to enable
qemu logdirty. But it uses libxl__domain_save_state, and calls
libxl__xc_domain_saverestore_async_callback_done() before exits.
This can not be used for secondary vm.
Update libxl__domain_suspend_common_switch_qemu_logdirty() to
introduce a new API libxl__domain_common_switch_qemu_logdirty().
This API only uses libxl__logdirty_switch, and calls
lds->callback before exits. This new API will be used by the patch:
secondary vm suspend/resume/checkpoint codes
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
In normal migration, the qemu state is passed to qemu as a parameter.
With COLO, secondary vm is running. So we will do the following steps
at every checkpoint:
1. suspend both primary vm and secondary vm
2. sync the state
3. resume both primary vm and secondary vm
Primary will send qemu's state in step2, and secondary's qemu should
read it and restore the state before it is resumed. We can not pass the
state to qemu as a parameter because secondary QEMU is already started
at this point, so we introduce libxl__qmp_restore() to do it.
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Commit d275ec9c introduce ssize_t but not include relevant header,
it will cause compile errors just like below:
./include/xenctrl.h:1485: error: expected '=', ',', ';', 'asm' or
'__attribute__' before 'xc_version_len'
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Forwarding entire batches to the device model when an individual
iteration of them got rejected by internal device emulation handlers
with X86EMUL_UNHANDLEABLE is wrong: The device model would then handle
all iterations, without the internal handler getting to see any past
the one it returned failure for. This causes misbehavior in at least
the MSI-X and VGA code, which want to see all such requests for
internal tracking/caching purposes. But note that this does not apply
to buffered I/O requests.
This in turn means that the condition in hvm_process_io_intercept() of
when to crash the domain was wrong: Since X86EMUL_UNHANDLEABLE can
validly be returned by the individual device handlers, we mustn't
blindly crash the domain if such occurs on other than the initial
iteration. Instead we need to distinguish hvm_copy_*_guest_phys()
failures from device specific ones, and then the former need to always
be fatal to the domain (i.e. also on the first iteration), since
otherwise we again would end up forwarding a request to qemu which the
internal handler didn't get to see.
The adjustment should be okay even for stdvga's MMIO handling:
- if it is not caching then the accept function would have failed so we
won't get into hvm_process_io_intercept(),
- if it issued the buffered ioreq then we only get to the p->count
reduction if hvm_send_ioreq() actually encountered an error (in which
we don't care about the request getting split up).
Also commit 4faffc41d ("x86/hvm: limit reps to avoid the need to handle
retry") went too far in removing code from hvm_process_io_intercept():
When there were successfully handled iterations, the function should
continue to return success with a clipped repeat count.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
I have made many modifications to this code over the past few years
so I'm probably the one most familiar with it.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Commit 57844631 "save APIC assist vector" added an extra field to the
viridian vcpu context save record. This field was only a uint8_t and
so an extra _pad field was also added to pad up to the next 64-bit
boundary.
This patch makes sure that _pad field is zeroed on save and checked
for zero on restore. This prevents a potential leak of information
from the stack and a compatibility check against future use of the
space occupied by the _pad field.
The _pad field is zeroed as a side effect of making use of a C99 struct
initializer for the other fields. This patch also modifies the domain
context save code to use the same mechanism.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
We change the xen_version libxc code to use the new hypercall.
Which of course means every user in the code base has to
be changed over.
It is important to note that the xc_version_op has a different
return semantic than the previous one. It returns negative
values on error (like the old one), but it also returns
an positive value on success (unlike the old one). The positive
value is the number of bytes copied in.
Note that both Ocaml and xenstat use tabs instead of four
spaces so they look quite odd.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Andrew Cooper <andrew.cooper3@citrix.com> [for the Ocaml stubs]
Acked-by: George Dunlap <george.dunlap@eu.citrix.com> [xenctx bits]
Acked-by: Wei Liu <wei.liu2@citrix.com>
This hypercall mirrors the XENVER_ in that it has similar functionality.
However it is designed differently:
- No compat layer. The data structures are the same size on 32
as on 64-bit.
- The hypercall accepts three arguments - the command, pointer to
an buffer, and the length of the buffer.
- Each sub-ops can be "probed" for size by returning the size of
buffer that will be needed - if the buffer is NULL.
- Subops can complete even if the buffer is too small - truncated
data will be filled and hypercall will return -ENOBUFS.
- VERSION_commandline, VERSION_changeset are privileged.
- There is no XENVER_compile_info equivalent.
- The hypercall can return -EPERM and toolstack/OSes are expected
to deal with. However there are three subops: XEN_VERSION_version,
XEN_VERSION_platform_parameters and XEN_VERSION_get_features
that will always return an value as guests cannot survive without them.
While we combine some of the common code between XENVER_ and VERSION_
take the liberty of moving pae_extended_cr3 in x86 area.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> [XSM bits]
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Commit 363ae55c8 used an OCaml feature called record field punning. This broke
the build on compilers prior to OCaml 3.12.0.
This patch makes no semantic change but now uses backwards-compatible syntax.
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reported-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Tested-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Wei Liu
Juergen Gross
Paul Durrant
Chong Li
Roger Pau Monne
Dario Faggioli
Olaf Hering
Konrad Rzeszutek Wilk
Wen Congyang
Changlong Xie
Jan Beulich
Jonathan Davies
Shannon Zhao