While building a VM, HVM domain builder provides struct hvm_info_table{}
to help hvmloader. Currently it includes two fields to construct guest
e820 table by hvmloader, low_mem_pgend and high_mem_pgend. So we should
check them to fix any conflict with RDM.
RMRR can reside in address space beyond 4G theoretically, but we never
see this in real world. So in order to avoid breaking highmem layout
we don't solve highmem conflict. Note this means highmem rmrr could still
be supported if no conflict.
But in the case of lowmem, RMRR probably scatter the whole RAM space.
Especially multiple RMRR entries would worsen this to lead a complicated
memory layout. And then its hard to extend hvm_info_table{} to work
hvmloader out. So here we're trying to figure out a simple solution to
avoid breaking existing layout. So when a conflict occurs,
#1. Above a predefined boundary (2G)
- move lowmem_end below reserved region to solve conflict;
#2. Below a predefined boundary (2G)
- Check strict/relaxed policy.
"strict" policy leads to fail libxl. Note when both policies
are specified on a given region, 'strict' is always preferred.
"relaxed" policy issue a warning message and also mask this entry INVALID
to indicate we shouldn't expose this entry to hvmloader.
Note later we need to provide a parameter to set that predefined boundary
dynamically.
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
v13a: Change `flag' to `flags' in libxl__xc_device_get_rdm.
No functional change. [ Suggested by Tiejun Chen. ]
v13: Mechanical changes to deal with changes to patch 01/
XENMEM_reserved_device_memory_map.
This patch introduces user configurable parameters to specify RDM
resource and according policies,
Global RDM parameter:
rdm = "strategy=host,policy=strict/relaxed"
Per-device RDM parameter:
pci = [ 'sbdf, rdm_policy=strict/relaxed' ]
Global RDM parameter, "strategy", allows user to specify reserved regions
explicitly, Currently, using 'host' to include all reserved regions reported
on this platform which is good to handle hotplug scenario. In the future
this parameter may be further extended to allow specifying random regions,
e.g. even those belonging to another platform as a preparation for live
migration with passthrough devices. By default this isn't set so we don't
check all rdms. Instead, we just check rdm specific to a given device if
you're assigning this kind of device. Note this option is not recommended
unless you can make sure any conflict does exist.
'strict/relaxed' policy decides how to handle conflict when reserving RDM
regions in pfn space. If conflict exists, 'strict' means an immediate error
so VM can't keep running, while 'relaxed' allows moving forward with a
warning message thrown out.
Default per-device RDM policy is same as default global RDM policy as being
'relaxed'. And the per-device policy would override the global policy like
others.
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Checked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Try to avoid placing PCI BARs over RMRRs:
- If mmio_hole_size is not specified, and the existing MMIO range has
RMRRs in it, and there is space to expand the hole in lowmem without
moving more memory, then make the MMIO hole as large as possible.
- When placing RMRRs, find the next RMRR higher than the current base
in the lowmem mmio hole. If it overlaps, skip ahead of it and find
the next one.
This certainly won't work in all cases, but it should work in a
significant number of cases. Additionally, users should be able to
work around problems by setting mmio_hole_size larger in the guest
config.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
Now we get this map layout by call XENMEM_memory_map then
save them into one global variable memory_map[]. It should
include lowmem range, rdm range and highmem range. Note
rdm range and highmem range may not exist in some cases.
And here we need to check if any reserved memory conflicts with
[RESERVED_MEMORY_DYNAMIC_START, RESERVED_MEMORY_DYNAMIC_END).
This range is used to allocate memory in hvmloder level, and
we would lead hvmloader failed in case of conflict since its
another rare possibility in real world.
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
RMRR reserved regions must be setup in the pfn space with an identity
mapping to reported mfn. However existing code has problem to setup
correct mapping when VT-d shares EPT page table, so lead to problem
when assigning devices (e.g GPU) with RMRR reported. So instead, this
patch aims to setup identity mapping in p2m layer, regardless of
whether EPT is shared or not. And we still keep creating VT-d table.
And we also need to introduce a pair of helper to create/clear this
sort of identity mapping as follows:
set_identity_p2m_entry():
If the gfn space is unoccupied, we just set the mapping. If space
is already occupied by desired identity mapping, do nothing.
Otherwise, failure is returned.
clear_identity_p2m_entry():
We just define macro to wrapper guest_physmap_remove_page() with
a returning value as necessary.
CC: Tim Deegan <tim@xen.org>
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Yang Zhang <yang.z.zhang@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
This is a prerequisite for punching holes into HVM and PVH guests' P2M
to allow passing through devices that are associated with (on VT-d)
RMRRs.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v12a: Move interface structure union member to the end, while moving
the whole public header block into a __XEN__ / __XEN_TOOLS__
conditional block.
v12: Restore changes as much as possible to my original version, fixing
a few issues that got introduced after handing it over. Unionize
new public memop interface structure to allow for non-PCI to be
supported later on. Check flags to have all currently undefined
flags clear. Refine adjustments to xen/pci.h.
Commit 372900faf8 ("x86/MSI-X: reduce fiddling with control register
during restore") introduced de-references of pdev before it gets
checked against NULL. Instead of deferring the de-references, drop
the pointless check - both call sites do that check already.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
This being an in-guest component, it shouldn't get to see (and even
less so use) tools-only public interfaces.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
... to hook up pci_conf_write_intercept() even for Dom0 not using
method 1 accesses for the base part of PCI device config space.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
... by monitoring writes to the mask register.
This allows reverting the main effect of the XSA-129 patches in qemu.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Rather than disabling and enabling MSI-X once per vector, do it just
once per device.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
As done in Linux by f598282f51 ("PCI: Fix the NIU MSI-X problem in a
better way") and its broken predecessor, make sure we don't access the
MSI-X table without having enabled MSI-X first, using the mask-all flag
instead to prevent interrupts from occurring.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
When a device gets detached from a guest, pciback will clear its
command register, thus disabling both memory and I/O decoding. The
disabled memory decoding, however, has an effect on the MSI-X table
accesses the hypervisor does: These won't have the intended effect
anymore. Even worse, for PCIe devices (but not SR-IOV virtual
functions) such accesses may (will?) be treated as Unsupported
Requests, causing respective errors to be surfaced, potentially in the
form of NMIs that may be fatal to the hypervisor or Dom0 is different
ways. Hence rather than carrying out these accesses, we should avoid
them where we can, and use alternative (e.g. PCI config space based)
mechanisms to achieve at least the same effect.
At this time it continues to be unclear whether this is fixing an
actual bug or is rather just working around bogus (but apparently
common) system behavior.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Host uses of the bits will be added subsequently, and must not be
overridden by guests (including Dom0, namely when acting on behalf of
a guest).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
This is to be used by MSI code, and later to also be hooked up to
MMCFG accesses by Dom0.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Some DOMCTLs returned non-POSIX error codes, replace them with POSIX
compilant values instead. EBADRQC and EBADSLT are replaced by EDOM, while
EUSERS is replaced with EOVERFLOW.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Replace two EDOM uses.
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
PSR was using EBADSLT and EUSERS which are not POSIX error codes, replace
them with ENOTSOCK and EOVERFLOW respectively.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
In fact, right now, the following (legitimate)
configuration:
vcpus = '4'
vnuma = [ [ "pnode=0","size=512","vcpus=0,1","vdistances=10,20" ],
[ "pnode=1","size=512","vcpus=2,3","vdistances=20,10" ] ]
Produces the following error:
# xl create /etc/xen/test.cfg
Parsing config from /etc/xen/test.cfg
xl: maxvcpus < vcpu
That is because, we only process the first element of the
"vcpus=" list (of each vnode specification). Therefore,
in the above case, we only see 2 vcpus, out of 4, being
assigned to the vnodes, and hence the error.
What we need is either a multidimentional array, or a
bitmap, to temporary store the vcpus of a vnode, while
parsing the vnuma config entry. Let's use the latter,
which happens to also make it easier to copy the outcome
of the parsing to its final destination in b_info, if
everything goes ok.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
cpu_to_socket() can't give correct socket value in CPU_PREPARE notifier
as at that time phys_proc_id has not yet been initialized (the value is
its default 0 in this case) which is incorrect for sockets other than
socket 0.
cos_to_cbm now is pre-allocated in CPU_PREPARE notifier and then consumed
in CPU_STARTING notifier.
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
htobe*() and be*toh() don't exist there. While replacing the 32-bit
ones with hton() and ntoh() would be possible, there wouldn't be an
obvious replacement for the 64-bit ones. Hence just take what current
glibc (2.21) has (assuming __bswap_*() exists, which it does back to
at least 2.4 according to my checking).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
xl subcommands ought all to take -h. def_getopt and hence
SWITCH_FOREACH_OPT already handles 'h' by calling helpstr. None of
the call sites see the 'h'.
In this patch:
* Change SWITCH_FOREACH_OPT to always add a "h" to the short opts
string, using string concatenation.
* Remove the now-redundant h's from some existing option strings.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This limit of 11 has been in this function since it was written, but
serves no purpose. The extra arguments are fed one by one to
parse_nic_config, and it is possible to have as many as you like.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This adjust commit a49077e5 "Fix segfaults from `xl psr-cat-cbm-set`
command line handling":
* Do not use the constant `required_argument' here (we simply use 1
everywhere else).
* Fix the minimum required arguments argument to SWITCH_FOREACH_OPT.
Leave the separate check on optind, because it checks for too many as
well as too few arguments.
(There are many things in xl which fail to check for too many
arguments. I do not intend to drain that swamp now: I started but
decided a complete overhaul of most of xl's command line argument
processing would be best.)
This is just a code cleanup with no ultimate functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Chao Peng <chao.p.peng@linux.intel.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Applying the series out of order led to d72befc35f "tools/libxl:
Identify copywhat in stream v2 datacopiers" unintentionally reverting
part of 21d9b079e5 "tools/libxl: Initialise the fd of the unused
half of a datacopier".
Put this back.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Call libxl_cpupoolinfo_init at the beginning. Change two returns to
goto out so that libxl_cpupoolinfo_dispose is called in failure path.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
nr_mods is set in add_boot_module() to the number of module
array elements used. This function also ensures that nr_mods
never exceeds MAX_MODULES (the size of the array). When looping
through the array, the correct maximum index is "nr_mods-1",
not "nr_mods". If the array is full, using the latter will in
fact access beyond the end of the array.
This was done correctly in boot_module_find_by_kind() and
consider_modules() but incorrectly in discard_initial_modules()
and next_module().
Signed-off-by: Chris Brand <chris.brand@broadcom.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
These result in log messages such as:
(XEN) d0v0: vGICD: RAZ on reserved register offset 0x00000c
(XEN) d0v0: vGICR: write r2 offset 0x000180
(XEN) not found<G><3>traps.c:2417:d0v0 HSR=0x93820046 pc=0xffffffc000322bfc gva=0xffffff8000090180 gpa=0x0000008d110180
Fix this by rewording without a \n in the middle. Also add one at the end.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Refactoring Remus setup by introducing libxl__remus_setup API.
All Remus setup work are done in this function.
There is a subtle behavioural change here, which is that if anything
which is now done in _setup fails then the result is a call to
dss->callback( ..,..,ERROR_FAIL) rather than _start returning
AO_CREATE_FAIL(ERROR_FAIL). But this is correct and intended.
Also remove the libxl__ prefix for static functions.
Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
There are 2 remus checkpoint callbacks(save/restore), currently, they
both called libxl__remus_domain_checkpoint_callback in different
files, so it is ok. But in the following patch, we will move all of the
remus callback code into a separate file, the name should be different.
So rename them to:
libxl__remus_domain_{save/restore}_checkpoint_callback
Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Move domain suspend code into a separate file libxl_dom_suspend.c.
Add an API libxl__domain_suspend() which wraps the static
function domain_suspend_callback_common() for internal use.
Export the existing API libxl__domain_suspend_callback() used by
libxc to suspend the guest during migration.
Note that the newly added file libxl_dom_suspend.c is used for
suspend/resume code.
Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
The suspend/save terminology used by libxc is more consistent.
"suspend" refers to quiescing the VM, so pausing qemu, making a
remote_shutdown(SHUTDOWN_suspend) hypercall etc.
"save" refers to the actions involved in actually shuffling the
state of the VM, so xc_domain_save() etc.
libxl currently uses "suspend" to encapsulate both. The patch
Rename libxl__domain_suspend() to libxl__domain_save() since it
actually refers to shuffling the state of the VM.
This results in some strangeness in that some functions called *save*
are now passed a struct called *suspend*, this is temporary and is all
fixed up later by the refactoring of the suspend_state.
Signed-off-by: Yang Hongyang <yanghy@cn.fujitsu.com>
Some comments, commit messages:
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Tiejun Chen
George Dunlap
Jan Beulich
Ian Campbell
Roger Pau Monné
Dario Faggioli
Chao Peng
Ian Jackson
Andrew Cooper
Wei Liu
Chris (Christopher) Brand
Yang Hongyang