Originally there was only one counter to keep track of pages. It was
used erroneously to keep track of how many pages were mapped and how
many pages needed to be sent. In the end munmap(2) always had 0 as the
length argument, which resulted in leaking the mapping.
This problem was discovered on 32bit toolstack because 32bit applications
have notably smaller address space. In fact this bug affects 64bit
toolstack too.
Use a separate counter to keep track of the number of mapped pages to
solve this problem.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
If xenconsole is run with stdin closed or redirected to /dev/null,
console_loop() will return immediately due to failure to read from
STDIN_FILENO. This patch tests if stdin and stdout are both connected to
a TTY and, if not, xenconsole will not attempt to read from stdin or
modify stdout terminal attributes.
Existing behaviour when xenconsole is run from a terminal does not
change.
This allows for non-interactive use, eg. running "xl create -c" under
systemd or piping the output of "xl console" to another command.
Signed-off-by: Martin Lucina <martin@lucina.net>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Add the remaining routines required to support enabling the alternate
p2m functionality.
Signed-off-by: Ed White <edmund.h.white@intel.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Fix off-by-one in various checks against MAX_ALTP2M. Adjust error code
in p2m_destroy_altp2m_by_id(). Cosmetic adjustments.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
Add a flag to indicate that a memory event occurred in an alternate p2m
and a field containing the p2m index. Allow any event response to switch
to a different alternate p2m using the same flag and field.
Modify p2m_mem_access_check() to handle alternate p2m's.
Signed-off-by: Ed White <edmund.h.white@intel.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com> for the x86 bits.
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Tamas K Lengyel <tlengyel@novetta.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
The existing ept_set_entry() and ept_get_entry() routines are extended
to optionally set/get suppress_ve. Passing -1 will set suppress_ve on
new p2m entries, or retain suppress_ve flag on existing entries.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Signed-off-by: Ravi Sahita <ravi.sahita@intel.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: George Dunlap <george.dunlap@eu.citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
Also adjust the caller in set_identity_p2m_entry().
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Add the basic data structures needed to support alternate p2m's and
the functions to initialise them and tear them down.
Although Intel hardware can handle 512 EPTP's per hardware thread
concurrently, only 10 per domain are supported in this patch for
performance reasons.
This change also splits the p2m lock into one lock type for altp2m's
and another type for all other p2m's. The purpose of this is to place
the altp2m list lock between the types, so the list lock can be
acquired whilst holding the host p2m lock.
Signed-off-by: Ed White <edmund.h.white@intel.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Cosmetic adjustments.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
As implemented here, only supported on platforms with VMX HAP.
By default this functionality is force-disabled, it can be enabled
by specifying altp2m=1 on the Xen command line.
Signed-off-by: Ed White <edmund.h.white@intel.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
In preparation for selectively enabling #VE in a later patch, set
suppress #VE on all EPTE's.
Suppress #VE should always be the default condition for two reasons:
it is generally not safe to deliver #VE into a guest unless that guest
has been modified to receive it; and even then for most EPT violations only
the hypervisor is able to handle the violation.
Signed-off-by: Ed White <edmund.h.white@intel.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Jun Nakajima <jun.nakajima@intel.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
For use on codepaths which would need to use domain_pause() but might be in
the target domain's context. In the case that the target domain is in
context, all other vcpus are paused.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@eu.citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
Since 25652f232c "tools/libxl: detect and avoid conflicts with RDM"
the build is broken for x86_32 and arm32 with:
libxl_dm.c: In function ‘libxl__domain_device_construct_rdm’:
libxl_dm.c:349:13: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 8 has type ‘uint64_t’ [-Werror=format=]
LOG(ERROR, "RDM conflict at 0x%lx.\n", d_config->rdms[i].start);
^
libxl_dm.c:352:13: error: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 8 has type ‘uint64_t’ [-Werror=format=]
LOG(WARN, "Ignoring RDM conflict at 0x%lx.\n",
Use PRIx64 for these.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Cc: Tiejun Chen <tiejun.chen@intel.com>
Pid is a null terminated string allocated by asprintf. It should be
freed after use.
Also fixed a coding style problem while I was there.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The DOMCTLs {,un}bind_pt_irq are using uint32_t for the machine_irq
while the helper is using uint8_t.
Currently on ARM, we are supporting SPIs whose irq number can go up to
1019 which doesn't fit in an uint8_t. The helpers xc_domain_bind_pt_spi
and xc_domain_unbint_pt_spi are correctly taking an uint16_t so the
libxc was truncating without noticing the user which may end up to
route the wrong IRQ.
Fix the problem by widening the machine_irq parameter in
xc_domain_*bind_pt_irq_int.
Note that XEN_DOMCTL_irq_permission has the same problem but it's not
used at the moment on ARM. So we can defer the changes after the release
of Xen 4.7.
Reported-by: Iurii Konovalenko <iurii.konovalenko@globallogic.com>
Signed-off-by: Julien Grall <julien.grall@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
In setup_mm(), the value passed as xenheap_megabytes gets
converted to pages and passed to setup_xenheap_mappings(),
which in turn passes it to create_32mb_mappings(), which
contains an ASSERT that the value passed is a multiple of
32MB. So specifying any value that is not an integer multiple
of 32 will cause Xen to hit this assert and fail to boot.
Signed-off-by: Chris Brand <chris.brand@broadcom.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
If systemd is configured for use AND you are building oxenstored, the C
systemd library must be linked in to the systemd.cxma library.
Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
And this time, do it right. In fact, a similar change was
attempted in 93be8285a7 ("cpupools: update domU's node-affinity
on the cpupool_unassign_cpu() path"). But that was buggy, and got
reverted with 8395b67ab0.
However, even though reverting was the right thing to do, it
remains true that:
- calling the function is better done in the cpupool cpu removal
code, even if just for simmetry with the cpupool cpu adding path;
- it is not necessary to call it during cpu teardown (for suspend
or shutdown) code as we either are going down and will never
come up (shutdown) or, when coming up, we want everything to be
as before the tearing down process started, and so we would just
undo any update made during the process.
- calling it from the teardown path is not only unnecessary, but
it can trigger an ASSERT(), in case we get, during the process,
to remove the last online pcpu of a domain's node affinity:
(XEN) Assertion '!cpumask_empty(dom_cpumask)' failed at domain.c:466
(XEN) ----[ Xen-4.6-unstable x86_64 debug=y Tainted: C ]----
... ... ...
(XEN) Xen call trace:
(XEN) [<ffff82d0801055b9>] domain_update_node_affinity+0x113/0x240
(XEN) [<ffff82d08012e676>] cpu_disable_scheduler+0x334/0x3f2
(XEN) [<ffff82d08018bb8d>] __cpu_disable+0x313/0x36e
(XEN) [<ffff82d080101424>] take_cpu_down+0x34/0x3b
(XEN) [<ffff82d080130ad9>] stopmachine_action+0x70/0x99
(XEN) [<ffff82d08013274f>] do_tasklet_work+0x78/0xab
(XEN) [<ffff82d080132a85>] do_tasklet+0x5e/0x8a
(XEN) [<ffff82d08016478c>] idle_loop+0x56/0x6b
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 12:
(XEN) Assertion '!cpumask_empty(dom_cpumask)' failed at domain.c:466
(XEN) ****************************************
Therefore, for all these reasons, move the call from
cpu_disable_schedule() to cpupool_unassign_cpu_helper().
While there, add some sanity checking (in the latter function), and
make sure that scanning the domain list is done with domlist_read_lock
held, at least when the system is 'live'.
I re-tested the scenario described in here:
http://permalink.gmane.org/gmane.comp.emulators.xen.devel/235310
which is what led to the revert of 93be8285a7, and that is
working ok after this commit.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Juergen Gross <jgross@suse.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
The function is called both when we want to remove a cpu
from a cpupool, and during cpu teardown, for suspend or
shutdown. If, however, the boot cpu (cpu 0, most of the
times) is not present in the default cpupool, during
suspend or shutdown, Xen crashes like this:
root@Zhaman:~# xl cpupool-cpu-remove Pool-0 0
root@Zhaman:~# shutdown -h now
(XEN) ----[ Xen-4.6-unstable x86_64 debug=y Tainted: C ]----
...
(XEN) Xen call trace:
(XEN) [<ffff82d0801238de>] _csched_cpu_pick+0x156/0x61f
(XEN) [<ffff82d080123db5>] csched_cpu_pick+0xe/0x10
(XEN) [<ffff82d08012de3c>] vcpu_migrate+0x18e/0x321
(XEN) [<ffff82d08012e4f8>] cpu_disable_scheduler+0x1cf/0x2ac
(XEN) [<ffff82d08018bb8d>] __cpu_disable+0x313/0x36e
(XEN) [<ffff82d080101424>] take_cpu_down+0x34/0x3b
(XEN) [<ffff82d08013097a>] stopmachine_action+0x70/0x99
(XEN) [<ffff82d0801325f0>] do_tasklet_work+0x78/0xab
(XEN) [<ffff82d080132926>] do_tasklet+0x5e/0x8a
(XEN) [<ffff82d08016478c>] idle_loop+0x56/0x6b
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 15:
(XEN) Assertion 'cpu < nr_cpu_ids' failed at ...URCES/xen/xen/xen.git/xen/include/xen/cpumask.h:97
(XEN) ****************************************
There also are problems when we try to suspend or shutdown
with a cpupool configured with just one cpu (no matter, in
this case, whether that is the boot cpu or not):
root@Zhaman:~# xl create /etc/xen/test.cfg
root@Zhaman:~# xl cpupool-migrate test Pool-1
root@Zhaman:~# xl cpupool-list -c
Name CPU list
Pool-0 0,1,2,3,4,5,6,7,8,9,10,11,13,14,15
Pool-1 12
root@Zhaman:~# shutdown -h now
(XEN) ----[ Xen-4.6-unstable x86_64 debug=y Tainted: C ]----
(XEN) CPU: 12
...
(XEN) Xen call trace:
(XEN) [<ffff82d08018bb91>] __cpu_disable+0x317/0x36e
(XEN) [<ffff82d080101424>] take_cpu_down+0x34/0x3b
(XEN) [<ffff82d08013097a>] stopmachine_action+0x70/0x99
(XEN) [<ffff82d0801325f0>] do_tasklet_work+0x78/0xab
(XEN) [<ffff82d080132926>] do_tasklet+0x5e/0x8a
(XEN) [<ffff82d08016478c>] idle_loop+0x56/0x6b
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 12:
(XEN) Xen BUG at smpboot.c:895
(XEN) ****************************************
In both cases, the problem is the scheduler not being able
to:
- move all the vcpus to the boot cpu (as the boot cpu is
not in the cpupool), in the former;
- move the vcpus away from a cpu at all (as that is the
only one cpu in the cpupool), in the latter.
Solution is to distinguish, inside cpu_disable_scheduler(),
the two cases of cpupool manipulation and teardown. For
cpupool manipulation, it is correct to ask the scheduler to
take an action, as pathological situation (like there not
being any cpu in the pool where to send vcpus) are taken
care of (i.e., forbidden!) already. For suspend and shutdown,
we don't want the scheduler to be involved at all, as the
final goal is pretty simple: "send all the vcpus to the
boot cpu ASAP", so we just go for it.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
This patch parses to enable user configurable parameters to specify
RDM resource and according policies which are defined previously,
Global RDM parameter:
rdm = "strategy=host,policy=strict/relaxed"
Per-device RDM parameter:
pci = [ 'sbdf, rdm_policy=strict/relaxed' ]
Default per-device RDM policy is same as default global RDM policy as being
'relaxed'. And the per-device policy would override the global policy like
others.
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Currently we're intending to cover this kind of devices
with shared RMRR simply since the case of shared RMRR is
a rare case according to our previous experiences. But
late we can group these devices which shared rmrr, and
then allow all devices within a group to be assigned to
same domain.
CC: Yang Zhang <yang.z.zhang@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
USB RMRR may conflict with guest BIOS region. In such case, identity
mapping setup is simply skipped in previous implementation. Now we
can handle this scenario cleanly with new policy mechanism so previous
hack code can be removed now.
CC: Yang Zhang <yang.z.zhang@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
While building a VM, HVM domain builder provides struct hvm_info_table{}
to help hvmloader. Currently it includes two fields to construct guest
e820 table by hvmloader, low_mem_pgend and high_mem_pgend. So we should
check them to fix any conflict with RDM.
RMRR can reside in address space beyond 4G theoretically, but we never
see this in real world. So in order to avoid breaking highmem layout
we don't solve highmem conflict. Note this means highmem rmrr could still
be supported if no conflict.
But in the case of lowmem, RMRR probably scatter the whole RAM space.
Especially multiple RMRR entries would worsen this to lead a complicated
memory layout. And then its hard to extend hvm_info_table{} to work
hvmloader out. So here we're trying to figure out a simple solution to
avoid breaking existing layout. So when a conflict occurs,
#1. Above a predefined boundary (2G)
- move lowmem_end below reserved region to solve conflict;
#2. Below a predefined boundary (2G)
- Check strict/relaxed policy.
"strict" policy leads to fail libxl. Note when both policies
are specified on a given region, 'strict' is always preferred.
"relaxed" policy issue a warning message and also mask this entry INVALID
to indicate we shouldn't expose this entry to hvmloader.
Note later we need to provide a parameter to set that predefined boundary
dynamically.
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
v13a: Change `flag' to `flags' in libxl__xc_device_get_rdm.
No functional change. [ Suggested by Tiejun Chen. ]
v13: Mechanical changes to deal with changes to patch 01/
XENMEM_reserved_device_memory_map.
This patch introduces user configurable parameters to specify RDM
resource and according policies,
Global RDM parameter:
rdm = "strategy=host,policy=strict/relaxed"
Per-device RDM parameter:
pci = [ 'sbdf, rdm_policy=strict/relaxed' ]
Global RDM parameter, "strategy", allows user to specify reserved regions
explicitly, Currently, using 'host' to include all reserved regions reported
on this platform which is good to handle hotplug scenario. In the future
this parameter may be further extended to allow specifying random regions,
e.g. even those belonging to another platform as a preparation for live
migration with passthrough devices. By default this isn't set so we don't
check all rdms. Instead, we just check rdm specific to a given device if
you're assigning this kind of device. Note this option is not recommended
unless you can make sure any conflict does exist.
'strict/relaxed' policy decides how to handle conflict when reserving RDM
regions in pfn space. If conflict exists, 'strict' means an immediate error
so VM can't keep running, while 'relaxed' allows moving forward with a
warning message thrown out.
Default per-device RDM policy is same as default global RDM policy as being
'relaxed'. And the per-device policy would override the global policy like
others.
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Checked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Try to avoid placing PCI BARs over RMRRs:
- If mmio_hole_size is not specified, and the existing MMIO range has
RMRRs in it, and there is space to expand the hole in lowmem without
moving more memory, then make the MMIO hole as large as possible.
- When placing RMRRs, find the next RMRR higher than the current base
in the lowmem mmio hole. If it overlaps, skip ahead of it and find
the next one.
This certainly won't work in all cases, but it should work in a
significant number of cases. Additionally, users should be able to
work around problems by setting mmio_hole_size larger in the guest
config.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
Now we get this map layout by call XENMEM_memory_map then
save them into one global variable memory_map[]. It should
include lowmem range, rdm range and highmem range. Note
rdm range and highmem range may not exist in some cases.
And here we need to check if any reserved memory conflicts with
[RESERVED_MEMORY_DYNAMIC_START, RESERVED_MEMORY_DYNAMIC_END).
This range is used to allocate memory in hvmloder level, and
we would lead hvmloader failed in case of conflict since its
another rare possibility in real world.
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
RMRR reserved regions must be setup in the pfn space with an identity
mapping to reported mfn. However existing code has problem to setup
correct mapping when VT-d shares EPT page table, so lead to problem
when assigning devices (e.g GPU) with RMRR reported. So instead, this
patch aims to setup identity mapping in p2m layer, regardless of
whether EPT is shared or not. And we still keep creating VT-d table.
And we also need to introduce a pair of helper to create/clear this
sort of identity mapping as follows:
set_identity_p2m_entry():
If the gfn space is unoccupied, we just set the mapping. If space
is already occupied by desired identity mapping, do nothing.
Otherwise, failure is returned.
clear_identity_p2m_entry():
We just define macro to wrapper guest_physmap_remove_page() with
a returning value as necessary.
CC: Tim Deegan <tim@xen.org>
CC: Keir Fraser <keir@xen.org>
CC: Jan Beulich <jbeulich@suse.com>
CC: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Yang Zhang <yang.z.zhang@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
This is a prerequisite for punching holes into HVM and PVH guests' P2M
to allow passing through devices that are associated with (on VT-d)
RMRRs.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v12a: Move interface structure union member to the end, while moving
the whole public header block into a __XEN__ / __XEN_TOOLS__
conditional block.
v12: Restore changes as much as possible to my original version, fixing
a few issues that got introduced after handing it over. Unionize
new public memop interface structure to allow for non-PCI to be
supported later on. Check flags to have all currently undefined
flags clear. Refine adjustments to xen/pci.h.
Wei Liu
Ian Campbell
Tamas K Lengyel
Martin Lucina
Ravi Sahita
Ed White
George Dunlap
Andrew Cooper
Ian Jackson
Julien Grall
Chris (Christopher) Brand
Jonathan Creekmore
Dario Faggioli
Tiejun Chen
Jan Beulich