read_record() could be used by primary to read dirty bitmap
record sent by secondary under COLO.
When used by xc save side, we need to pass the backchannel fd
instead of ctx->fd to read_record(), so we added a fd param to
it.
No functional changes.
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
In COLO mode, both VMs are running, and are considered in sync if the
visible network traffic is identical. After some time, they fall out of
sync.
At this point, the two VMs have definitely diverged. Lets call the
primary dirty bitmap set A, while the secondary dirty bitmap set B.
Sets A and B are different.
Under normal migration, the page data for set A will be sent from the
primary to the secondary.
However, the set difference B - A (the one in B but not in A, lets
call this C) is out-of-date on the secondary (with respect to the
primary) and will not be sent by the primary (to secondary), as it
was not memory dirtied by the primary. The secondary needs C page data
to reconstruct an exact copy of the primary at the checkpoint.
The secondary cannot calculate C as it doesn't know A. Instead, the
secondary must send B to the primary, at which point the primary
calculates the union of A and B (lets call this D) which is all the
pages dirtied by both the primary and the secondary, and sends all page
data covered by D.
In the general case, D is a superset of both A and B. Without the
backchannel dirty bitmap, a COLO checkpoint can't reconstruct a valid
copy of the primary.
We transfer the dirty bitmap on libxc side, so we need to introduce back
channel to libxc.
Note: it is different from the paper. We change the original design to
the current one, according to our following concerns:
1. The original design needs extra memory on Secondary host. When there's
multiple backups on one host, the memory cost is high.
2. The memory cache code will be another 1k+, it will make the review
more time consuming.
Note: this patch merely adds new parameters to various prototypes and
functions. The new parameters are used in later patch called
"libxc/restore: send dirty pfn list to primary when checkpoint under
COLO".
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
In COLO mode, secondary needs to send the following data to primary:
1. In libxl
Secondary sends the following CHECKPOINT_CONTEXT to primary:
CHECKPOINT_SVM_SUSPENDED, CHECKPOINT_SVM_READY and CHECKPOINT_SVM_RESUMED
2. In libxc
Secondary sends the dirty pfn list to primary
But the io_fd only can be written in primary, and only can be read in
secondary. Save recv_fd in domain_suspend_state, and send_fd in
domain_create_state. Extend libxl_domain_create_restore API, add a
send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate
the API change.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Secondary vm is running in COLO mode, we need to send secondary
vm's dirty page information to primary host at checkpoint, so we
have to enable qemu logdirty on secondary.
libxl__domain_suspend_common_switch_qemu_logdirty() is to enable
qemu logdirty. But it uses libxl__domain_save_state, and calls
libxl__xc_domain_saverestore_async_callback_done() before exits.
This can not be used for secondary vm.
Update libxl__domain_suspend_common_switch_qemu_logdirty() to
introduce a new API libxl__domain_common_switch_qemu_logdirty().
This API only uses libxl__logdirty_switch, and calls
lds->callback before exits. This new API will be used by the patch:
secondary vm suspend/resume/checkpoint codes
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
In normal migration, the qemu state is passed to qemu as a parameter.
With COLO, secondary vm is running. So we will do the following steps
at every checkpoint:
1. suspend both primary vm and secondary vm
2. sync the state
3. resume both primary vm and secondary vm
Primary will send qemu's state in step2, and secondary's qemu should
read it and restore the state before it is resumed. We can not pass the
state to qemu as a parameter because secondary QEMU is already started
at this point, so we introduce libxl__qmp_restore() to do it.
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Commit d275ec9c introduce ssize_t but not include relevant header,
it will cause compile errors just like below:
./include/xenctrl.h:1485: error: expected '=', ',', ';', 'asm' or
'__attribute__' before 'xc_version_len'
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Forwarding entire batches to the device model when an individual
iteration of them got rejected by internal device emulation handlers
with X86EMUL_UNHANDLEABLE is wrong: The device model would then handle
all iterations, without the internal handler getting to see any past
the one it returned failure for. This causes misbehavior in at least
the MSI-X and VGA code, which want to see all such requests for
internal tracking/caching purposes. But note that this does not apply
to buffered I/O requests.
This in turn means that the condition in hvm_process_io_intercept() of
when to crash the domain was wrong: Since X86EMUL_UNHANDLEABLE can
validly be returned by the individual device handlers, we mustn't
blindly crash the domain if such occurs on other than the initial
iteration. Instead we need to distinguish hvm_copy_*_guest_phys()
failures from device specific ones, and then the former need to always
be fatal to the domain (i.e. also on the first iteration), since
otherwise we again would end up forwarding a request to qemu which the
internal handler didn't get to see.
The adjustment should be okay even for stdvga's MMIO handling:
- if it is not caching then the accept function would have failed so we
won't get into hvm_process_io_intercept(),
- if it issued the buffered ioreq then we only get to the p->count
reduction if hvm_send_ioreq() actually encountered an error (in which
we don't care about the request getting split up).
Also commit 4faffc41d ("x86/hvm: limit reps to avoid the need to handle
retry") went too far in removing code from hvm_process_io_intercept():
When there were successfully handled iterations, the function should
continue to return success with a clipped repeat count.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
I have made many modifications to this code over the past few years
so I'm probably the one most familiar with it.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Commit 57844631 "save APIC assist vector" added an extra field to the
viridian vcpu context save record. This field was only a uint8_t and
so an extra _pad field was also added to pad up to the next 64-bit
boundary.
This patch makes sure that _pad field is zeroed on save and checked
for zero on restore. This prevents a potential leak of information
from the stack and a compatibility check against future use of the
space occupied by the _pad field.
The _pad field is zeroed as a side effect of making use of a C99 struct
initializer for the other fields. This patch also modifies the domain
context save code to use the same mechanism.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
We change the xen_version libxc code to use the new hypercall.
Which of course means every user in the code base has to
be changed over.
It is important to note that the xc_version_op has a different
return semantic than the previous one. It returns negative
values on error (like the old one), but it also returns
an positive value on success (unlike the old one). The positive
value is the number of bytes copied in.
Note that both Ocaml and xenstat use tabs instead of four
spaces so they look quite odd.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Andrew Cooper <andrew.cooper3@citrix.com> [for the Ocaml stubs]
Acked-by: George Dunlap <george.dunlap@eu.citrix.com> [xenctx bits]
Acked-by: Wei Liu <wei.liu2@citrix.com>
This hypercall mirrors the XENVER_ in that it has similar functionality.
However it is designed differently:
- No compat layer. The data structures are the same size on 32
as on 64-bit.
- The hypercall accepts three arguments - the command, pointer to
an buffer, and the length of the buffer.
- Each sub-ops can be "probed" for size by returning the size of
buffer that will be needed - if the buffer is NULL.
- Subops can complete even if the buffer is too small - truncated
data will be filled and hypercall will return -ENOBUFS.
- VERSION_commandline, VERSION_changeset are privileged.
- There is no XENVER_compile_info equivalent.
- The hypercall can return -EPERM and toolstack/OSes are expected
to deal with. However there are three subops: XEN_VERSION_version,
XEN_VERSION_platform_parameters and XEN_VERSION_get_features
that will always return an value as guests cannot survive without them.
While we combine some of the common code between XENVER_ and VERSION_
take the liberty of moving pae_extended_cr3 in x86 area.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov> [XSM bits]
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Commit 363ae55c8 used an OCaml feature called record field punning. This broke
the build on compilers prior to OCaml 3.12.0.
This patch makes no semantic change but now uses backwards-compatible syntax.
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reported-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Tested-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
It needs to map platform or amba device mmio to Dom0 on ARM. But when
booting with ACPI, it can't get the mmio region in Xen due to lack of
AML interpreter to parse DSDT table. Therefore, let Dom0 call a
hypercall to map mmio region when it adds the devices.
Here we add a new map space like the XEN_DOMCTL_memory_mapping to map
mmio region for Dom0. Also add a helper to combine the
xsm_add_to_physmap and XENMAPSPACE_dev_mmio space check together.
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Acked-by: Julien Grall <julien.grall@arm.com>
Store the event-channel interrupt number and flag in HVM parameter
HVM_PARAM_CALLBACK_IRQ. Then Dom0 could get it through hypercall
HVMOP_get_param.
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Firstly it permits full MMIO capabilities for Dom0. Then deny MMIO
access of Xen used devices, such as UART, GIC, SMMU. Currently, it only
denies the MMIO access of UART and GIC regions. For other Xen used
devices it could be added later when they are supported.
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Interrupt information is described in DSDT and is not available at the
time of booting. Check if the interrupt is permitted to access and set
the interrupt type, route it to guest dynamically only for SPI
and Dom0.
Signed-off-by: Parth Dixit <parth.dixit@linaro.org>
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Allow DOM0 to use all SPIs but the ones used by Xen. Then when Dom0
configures the interrupt, it could set the interrupt type and route it
to Dom0.
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Create a DT for Dom0 for ACPI-case only. DT contains minimal required
information such as Dom0 bootargs, initrd, efi description table and
address of uefi memory table.
Also document this device tree bindings of "hypervisor" and
"hypervisor/uefi" node.
Signed-off-by: Naresh Bhat <naresh.bhat@linaro.org>
Signed-off-by: Parth Dixit <parth.dixit@linaro.org>
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Copy and modify XSDT table before passing it to Dom0. Replace the entry
value of the copied table. Add a new entry for STAO table as well. And
keep entry value of other reused tables unchanged.
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Copy main MADT table contents and distributor subtable from physical
ACPI MADT table. Make other subtables through the callback of
gic_hw_ops.
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Estimate the memory required for loading acpi/efi tables in Dom0. Make
the length of each table aligned with 64bit. Alloc the pages to store
the new created EFI and ACPI tables and free these pages when
destroying domain.
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Reviewed-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Julien Grall <julien.grall@arm.com>
Update my email address.
Remove myself from STUB DOMAINS, MINI-OS and TOOLSTACK, where I haven't
been active recently.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
* 'pin' of https://github.com/jgross1/xen:
libxl: add force option for xl vcpu-pin
libxl: print message how to recover from xl cpupool-cpu-remove errors
libxc: do some retries in xc_cpupool_removecpu() for EBUSY case
All patches have Acked-by and Reviewed-by tags.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Commit 78c5f59e "x86/hvm/viridian: save APIC assist vector" changed
the name of a field in the viridian vcpu save record. Unfortunately this
record has a decode function in xen-hvmctx and so it no longer builds.
This patch fixes the field name in xen-hvmctx and also adds a decode of
the additional field that was added to the save record.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
There should be 6 instead of 7 arguments now for tmem_control()
. which was done in commit 54a51b1766
"tmem: Remove xc_tmem_control mystical arg3" which missed
this change.
Signed-off-by: Zhigang Wang <zhigang.x.wang@oracle.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This wasn't a good idea after all - make them unavailable except for
legacy code using an older interface version.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
They're not really useful past the building stage and only needlessly
increase binary file sizes.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Recursive locks know their current owner, and since we use the function
solely to determine whether a particular lock is being held by the
current CPU (which so far has been an imprecise check), make actually
check the owner for recusrively acquired locks.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Quan Xu <quan.xu@intel.com>
Acked-by: Tim Deegan <tim@xen.org>
Previous patch using all available features calculate comp_offsets.
This is wrong.This patch fix this bug by calculating the comp_offset
based on xcomp_bv of current guest.
Also, the comp_offset should take alignment into consideration.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Shuai Ruan <shuai.ruan@linux.intel.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Other than the controllers supported so far, multiple port Pericom
boards map all of their ports via BAR0, which requires a number of
adjustments: Instead of tracking "max_bars" we now flag whether all
ports use BAR0, and whether to expect a port-I/O or MMIO resource. As
a result pci_uart_config() now gets handed a port index, which it then
maps into a BAR index or an offset into BAR0 depending on the bar0
flag.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Tim Deegan <tim@xen.org>
Subsequent changes will want to use this pointer.
This makes the enable_ro structure member redundant, so it gets dropped
at once.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Tested-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Tim Deegan <tim@xen.org>
This new delivery type which is for ARM shares the same value with
HVM_PARAM_CALLBACK_TYPE_VECTOR which is for x86.
val[15:8] is flag: val[7:0] is a PPI.
To the flag, bit 8 stands the interrupt mode is edge(1) or level(0) and
bit 9 stands the interrupt polarity is active low(1) or high(0).
Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Commit a6f2cdb6 "keep APIC assist page mapped..." introduced a page
leak because it relied on viridian_vcpu_deinit() always being called
to release the page mapping. This does not happen in the case a normal
domain shutdown.
This patch fixes the problem by introducing a new function,
viridian_domain_deinit(), which will iterate through the vCPUs and
release any page mappings still present.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
If any vcpu has a pending APIC assist when the domain is suspended
then the vector needs to be saved. If this is not done then it's
possible for the vector to remain pending in the vlapic ISR
indefinitely after resume.
This patch adds code to save the APIC assist vector value in the
viridian vcpu save record. This means that the record is now zero-
extended on load and, because this implies a loaded value of
zero means nothing is pending (for backwards compatibility with
hosts not implementing APIC assist), the rest of the viridian APIC
assist code is adjusted to treat a zero value in this way. A
check has therefore been added to viridian_start_apic_assist() to
prevent the enlightenment being used for vectors < 0x10 (which
are illegal for an APIC).
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Wen Congyang
Changlong Xie
Jan Beulich
Paul Durrant
Konrad Rzeszutek Wilk
Jonathan Davies
Shannon Zhao
Parth Dixit
Stefano Stabellini
Zhigang Wang
Shuai Ruan