When parsing bitmap objects JSON parser will create libxl_bitmap map of the
smallest size needed.
This can cause problems when saved image file specifies CPU affinity. For
example, if 'vcpu_hard_affinity' in the saved image has only the first CPU
specified, just a single byte will be allocated and libxl_bitmap->size will be
set to 1.
This will result in assertion in libxl_set_vcpuaffinity()->libxl_bitmap_copy()
since the destination bitmap is created for maximum number of CPUs.
We could allocate that bitmap of the same size as the source, however, it is
later passed to xc_vcpu_setaffinity() which expects it to be sized to the max
number of CPUs
To fix this issue, introduce an internal function to allowing copying between
bitmaps of different sizes. Note that this function is only used in
libxl_set_vcpuaffinity at the moment. Though NUMA placement logic invoke
libxl_bitmap_copy as well there's no need to replace those invocations. NUMA
placement logic comes into effect when no vcpu / node pinning is provided, so
it always operates on bitmap of the same sizes (that is, size of maximum
number of cpus /nodes).
Reported-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Tested-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
* Add options whose code has been committed (without a patch to this file).
* Remove options which have been deleted (without a patch to this file).
* Tweak some formatting for consistency.
* Nuke some trailing whitespace.
I believe this document now identifies the exact set of command line options
accepted by Xen, for both x86 and ARM architectures.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Currently when VPMU is enabled on a system both HVM and PVH VPCUs will
initialize their VPMUs, including setting up vpmu_ops. As result even
though VPMU will not work for PVH guests (APIC is not supported there),
the guest may decide to perform a write to a PMU MSR. This will cause a
call to is_vlapic_lvtpc_enabled() which will crash the hypervisor, e.g.:
(XEN) Xen call trace:
(XEN) [<ffff82d0801ca06f>] is_vlapic_lvtpc_enabled+0x13/0x22
(XEN) [<ffff82d0801e2a15>] core2_vpmu_do_wrmsr+0x415/0x589
(XEN) [<ffff82d0801cedaa>] vpmu_do_wrmsr+0x2a/0x33
(XEN) [<ffff82d0801dd648>] vmx_msr_write_intercept+0x268/0x557
(XEN) [<ffff82d0801bcd2e>] hvm_msr_write_intercept+0x36c/0x39b
(XEN) [<ffff82d0801e0a0e>] vmx_vmexit_handler+0x1082/0x185b
(XEN) [<ffff82d0801e74c1>] vmx_asm_vmexit_handler+0x41/0xc0
If we prevent VPMU from being initialized on PVH guests we will avoid
those accesses.
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
While it is generally wrong to cross region boundaries when dealing
with MMIO accesses of repeated string instructions (currently only
MOVS) as that would do things a guest doesn't expect (leaving aside
that none of these regions would normally be accessed with repeated
string instructions in the first place), this is even more of a problem
for all virtual MSI-X page accesses (both msixtbl_{read,write}() can be
made dereference NULL "entry" pointers this way) as well as undersized
(1- or 2-byte) LAPIC writes (causing vlapic_read_aligned() to access
space beyond the one memory page set up for holding LAPIC register
values).
Since those functions validly assume to be called only with addresses
their respective checking functions indicated to be okay, it is generic
code that needs to be fixed to clip the repetition count.
To be on the safe side (and consistent), also do the same for buffered
I/O intercepts, even if their only client (stdvga) doesn't put the
hypervisor at risk (i.e. "only" guest misbehavior would result).
This is CVE-2014-8867 / XSA-112.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
HVM/PVH guests can otherwise trigger the final BUG_ON() in that
function by entering 64-bit mode, setting the high halves of affected
registers to non-zero values, leaving 64-bit mode, and issuing a
hypercall that might get preempted and hence become subject to
continuation argument translation (HYPERVISOR_memory_op being the only
one possible for HVM, PVH also having the option of using
HYPERVISOR_mmuext_op). This issue got introduced when HVM code was
switched to use compat_memory_op() - neither that nor
hypercall_xlat_continuation() were originally intended to be used by
other than PV guests (which can't enter 64-bit mode and hence have no
way to alter the high halves of 64-bit registers).
This is CVE-2014-8866 / XSA-111.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
No-one (including me) paid attention during review that these
structures don't adhere to the naming requirements of the public
interface: Consistently use xen_ prefixes at least for all new
additions.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
The libxc xc_dom_* infrastructure uses a very simple malloc memory pool which
is freed by xc_dom_release. However the various xc_try_*_decode routines (other
than the gzip one) just use plain malloc/realloc and therefore the buffer ends
up leaked.
The memory pool currently supports mmap'd buffers as well as a directly
allocated buffers, however the try decode routines make use of realloc and do
not fit well into this model. Introduce a concept of an external memory block
to the memory pool and provide an interface to register such memory.
The mmap_ptr and mmap_len fields of the memblock tracking struct lose their
mmap_ prefix since they are now also used for external memory blocks.
We are only seeing this now because the gzip decoder doesn't leak and it's only
relatively recently that kernels in the wild have switched to better
compression.
This is https://bugs.debian.org/767295
Reported by: Gedalya <gedalya@gedalya.net>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Currently we only establish specific mappings for pcie0, which is
used on the Mustang platform. However at least McDivitt uses pcie3.
So wire up all the others, based on whether the corresponding DT node
is marked as available.
This results in no change for Mustang.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Pranavkumar Sawargaonkar <pranavkumar@linaro.org>
Return proper error codes on failure so that scripts can tell whether
the command completed properly or not.
This is not a proper fix, since it fails to call
libxl_device_disk_dispose() on the error path. But a proper fix
requires some refactoring, so given where we are in the release
process, it's better to have a fix that is simple and obvious, and do
the refactoring once the next development window opens up.
Signed-off-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
This extends commit 5283b310 ("x86/HVM: only kill guest when unknown VM
exit occurred in guest kernel mode") to a few more cases, including the
failed VM entry one that XSA-110 was needed to be issued for.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Instead properly fail requests that shouldn't be issued on foreign
domains or - for MMUEXT_{CLEAR,COPY}_PAGE - extend the existing
operation to work that way.
In the course of doing this the need to always clear "okay" even when
wanting an error code other than -EINVAL became unwieldy, so the
respective logic is being adjusted at once, together with a little
other related cleanup.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
MMU_MACHPHYS_UPDATE, not manipulating page tables, shouldn't ignore
a bad page table domain being specified.
Also pt_owner can't be NULL when reaching the "out" label, so the
respective check can be dropped.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Commit 4ca6f9f0 ("x86/cpuidle: publish new states only after fully
initializing them") resulted in the state counter to be incremented
for C1 despite that using a fixed table entry (and the statically
initialized counter value already accounting for it and C0).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
commit f6dd295381 "dpci: replace tasklet
with softirq" used the 'masked' as an two-bit state mechanism
(STATE_SCHED, STATE_RUN) to communicate between 'raise_softirq_for' and
'dpci_softirq' to determine whether the 'struct hvm_pirq_dpci' can be
re-scheduled.
However it ignored the 'pt_irq_guest_eoi' was not adhering to the proper
dialogue and was not using locked cmpxchg or test_bit operations and
ended setting 'state' set to zero. That meant 'raise_softirq_for' was
free to schedule it while the 'struct hvm_pirq_dpci'' was still on an
per-cpu list causing an list corruption.
The code would trigger the following path causing list corruption:
\-timer_softirq_action
pt_irq_time_out calls pt_pirq_softirq_cancel sets state to 0.
pirq_dpci is still on dpci_list.
\- dpci_sofitrq
while (!list_emptry(&our_list))
list_del, but has not yet done 'entry->next = LIST_POISON1;'
[interrupt happens]
raise_softirq checks state which is zero. Adds pirq_dpci to the dpci_list.
[interrupt is done, back to dpci_softirq]
finishes the entry->next = LIST_POISON1;
.. test STATE_SCHED returns true, so executes the hvm_dirq_assist.
ends the loop, exits.
\- dpci_softirq
while (!list_emtpry)
list_del, but ->next already has LIST_POISON1 and we blow up.
An alternative solution was proposed (adding STATE_ZOMBIE and making
pt_irq_time_out use the cmpxchg protocol on 'state'), which fixed the above
issue but had an fatal bug. It would miss interrupts that are to be scheduled!
This patch brings back the 'masked' boolean which is used as an
communication channel between 'hvm_do_IRQ_dpci', 'hvm_dirq_assist' and
'pt_irq_guest_eoi'. When we have an interrupt we set 'masked'. Anytime
'hvm_dirq_assist' or 'pt_irq_guest_eoi' executes - it clears it.
The 'state' is left as a seperate mechanism to provide an mechanism between
'raise_sofitrq' and 'softirq_dpci' to communicate the state of the
'struct hvm_dirq_pirq'.
However since we have now two seperate machines we have to deal with an
cancellations and outstanding interrupt being serviced: 'pt_irq_destroy_bind'
is called while an 'hvm_dirq_assist' is just about to service.
The 'pt_irq_destroy_bind' takes the lock first and kills the timer - and
the moment it releases the spinlock, 'hvm_dirq_assist' thunders in and calls
'set_timer' hitting an ASSERT.
By clearing the 'masked' in the 'pt_irq_destroy_bind' we take care of that
scenario by inhibiting 'hvm_dirq_assist' to call the 'set_timer'.
In the 'pt_irq_create_bind' - in the error cases we could be seeing
an softirq scheduled right away and being serviced (though stuck at
the spinlock). The 'pt_irq_create_bind' fails in 'pt_pirq_softirq_reset'
to change the 'state' (as the state is in 'STATE_RUN', not 'STATE_SCHED').
'pt_irq_create_bind' continues on with setting '->flag=0' and unlocks the lock.
'hvm_dirq_assist' grabs the lock and continues one. Since 'flag = 0' and
'digl_list' is empty, it thunders through the 'hvm_dirq_assist' not doing
anything until it hits 'set_timer' which is undefined for MSI. Adding
in 'masked=0' for the MSI case fixes that.
The legacy interrupt one does not need it as there is no chance of
do_IRQ being called at that point.
Reported-by: Sander Eikelenboom <linux@eikelenboom.it>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Any domain which can pass the XSM check against a translated guest can cause a
page reference to be leaked.
While shuffling the order of checks, drop the quite-pointless MEM_LOG(). This
brings the check in line with similar checks in the vicinity.
Discovered while reviewing the XSA-109/110 followup series.
This is XSA-113.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
For 'dom0_max_vcpus' and 'hvm_debug', markdown was interpreting the text as
regular text, and reflowing it as a regular paragraph, leading to a single
line as output. Reformat them as code blocks inside blockquote blocks, which
causes them to take their precise whitespace layout.
For 'psr', the bullet point was incorrectly delineated from paragraph text,
causing it to be reflowed. Alter the formatting to include the CMT-specific
options as sub-bullets of the overall CMT resource.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Release-acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
The region assigned to PCIE0, according to the docs, is 0x0e000000000 to
0x10000000000. They make no distinction between PCI CFG and PCI IO mem within
this range (in fact, I'm not sure that isn't up to the driver).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@linaro.org>
The callers pass the end as the pfn immediately *after* the last page to be
mapped, therefore adding one is incorrect and causes an additional page to be
mapped.
At the same time correct the printing of the mfn values, zero-padding them to
16 digits as for a paddr when they are frame numbers is just confusing.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@linaro.org>
EARLY_PRINTK_BAUD doesn't do anything unless EARLY_PRINTK_INIT_UART is set.
Furthermore only the pl011 driver implements the init routine at all, so the
entries which use any other UART driver and specified a BAUD were doubly wrong.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@linaro.org>
Some versions of markdown (specifically the one in Debian Wheezy, currently
used to generate
http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html) seem to be
confused by nested lists in the middle of multi-paragraph parent list entries
as seen in the com1,com2 entry.
The effect is that the "Default" section of all following entries are replace
by some sort of hash or checksum (at least, a string of 32 random seeming hex
digits).
Workaround this issue by making the decriptions of the DPS options a nested
list, moving the existing nested list describing the options for S into a third
level list. This seems to avoid the issue, and is arguably better formatting in
its own right (at least its not a regression IMHO)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
mkdeb previously set the package architecture to be 'amd64' for anything other than
XEN_TARGET_ARCH=x86_32. This patch attempts to correctly map the architecture
from XEN_TARGET_ARCH to the Debian architecture names for x86 and ARM
architectures.
Signed-off-by: Clark Laughlin <clark.laughlin@linaro.org>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
The device parameter of libxl_device_<type>_add is an in/out parameter.
Unspecified fields are filled in with appropriate values for the created
device when the function returns. Document this behaviour.
Signed-off-by: Euan Harris <euan.harris@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The existence check is to make sure a device is not added to a guest
multiple times.
PCI device backend path has different rules from vif, disk etc. For
example:
/local/domain/0/backend/pci/9/0/dev-1/0000:03:10.1
/local/domain/0/backend/pci/9/0/key-1/0000:03:10.1
/local/domain/0/backend/pci/9/0/dev-2/0000:03:10.2
/local/domain/0/backend/pci/9/0/key-2/0000:03:10.2
The devid for PCI devices is hardcoded 0. libxl__device_exists only
checks up to /local/.../9/0 so it always returns true even the device is
assignable.
Remove invocation of libxl__device_exists. We're sure at this point that
the PCI device is assignable (hence no xenstore entry or JSON entry).
The check is done before hand. For HVM guest it's done by calling
xc_test_assign_device and for PV guest it's done by calling
pciback_dev_is_assigned.
Reported-by: Li, Liang Z <liang.z.li@intel.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Konrad Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Document that:
- the existing code is not all confirming yet
- code should conform
- we will sometimes accept patches with nonconforming elements if
they don't make matters worse.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Privilege level checks were basically missing for the CS case, the
only check that was done (RPL == DPL for nonconforming segments)
was solely covering a single special case (return to non-conforming
segment).
Additionally in long mode the L bit set requires the D bit to be clear,
as was recently pointed out for KVM by Nadav Amit
<namit@cs.technion.ac.il>.
Finally we also need to force the loaded selector's RPL to CPL (at
least as long as lret/retf emulation doesn't support privilege level
changes).
This is CVE-2014-8595 / XSA-110.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
paging_write_guest_entry() and paging_cmpxchg_guest_entry() aren't
consistently supported for non-PV guests (they'd deref NULL for PVH or
non-HAP HVM ones). Don't allow respective MMU_* operations on the
page tables of such domains.
This is CVE-2014-8594 / XSA-109.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
The specification is kind of vague under what conditions
ExitBootServices() may legitimately fail, requiring the OS loader to
retry:
"If MapKey value is incorrect, ExitBootServices() returns
EFI_INVALID_PARAMETER and GetMemoryMap() with ExitBootServices() must
be called again. Firmware implementation may choose to do a partial
shutdown of the boot services during the first call to
ExitBootServices(). EFI OS loader should not make calls to any boot
service function other then GetMemoryMap() after the first call to
ExitBootServices()."
While our code guarantees the map key to be valid, there are systems
where a firmware internal notification sent while processing
ExitBootServices() reportedly results in changes to the memory map.
In that case, make a best effort second try: Avoid any boot service
calls other than the two named above, with the possible exception of
error paths. Those aren't a problem, since if we end up needing to
retry, we're hosed when something goes wrong as much as if we didn't
make the retry attempt.
For x86, a minimal adjustment to efi_arch_process_memory_map() is
needed for it to cope with potentially being called a second time.
For arm64, while efi_process_memory_map_bootinfo() is easy to verify
that it can safely be called more than once without violating spec
constraints, it's not so obvious for fdt_add_uefi_nodes(), hence a
step by step approach:
- deletion of memory nodes and memory reserve map entries: the 2nd pass
shouldn't find any as the 1st one deleted them all,
- a "chosen" node should be found as it got added in the 1st pass,
- the various "linux,uefi-*" nodes all got added during the 1st pass
and hence only their contents may get updated.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Roy Franz <roy.franz@linaro.org>
Release-acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Having these point into space not controlled by the hypervisor provides
an unnecessary attack surface. Allow architectures to override them and
utilize that override to make them non-canonical addresses (thus
causing #GP rather than #PF when dereferenced).
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
... otherwise when device add operation fails, the error message looks
like "libxl: error: libxl.c:1897:device_addrm_aocomplete: unable to (null)
device", which is not very helpful.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Some versions of Atmel TPMs provide invalid values for TPM_CAP_PROP_TIS_TIMEOUT query.
Because timeouts are invalid, every other command after tpm_get_timeouts will fail.
It is a known issue and it was fixed recently in linux kernel tpm_tis.c on 2014-07-29.
This patch does not allow timeouts to be less than standard values.
I tested it on a Dell Latitude E5520 and after making the changes I was able to start vtpmmgr-stubdom.
Signed-off-by: Emil Condrea <emilcondrea@gmail.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Commit bac6334b51 (move domain to
cpupool0 before destroying it) introduced an error in the accounting
of cpupools regarding the number of domains. The number of domains
is nor adjusted when a domain is moved to cpupool0 in kill_domain().
Correct this by introducing a cpupool function doing the move
instead of open coding it by calling sched_move_domain().
Reported-by: Dietmar Hahn <dietmar.hahn@ts.fujitsu.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Tested-by: Dietmar Hahn <dietmar.hahn@ts.fujitsu.com>
Reviewed-by: Andrew Cooper <Andrew.Cooper3@citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
The existing tasklet mechanism has a single global spinlock that is
taken every-time the global list is touched. And we use this lock quite
a lot - when we call do_tasklet_work which is called via an softirq and
from the idle loop. We take the lock on any operation on the
tasklet_list.
The problem we are facing is that there are quite a lot of tasklets
scheduled. The most common one that is invoked is the one injecting the
VIRQ_TIMER in the guest. Guests are not insane and don't set the
one-shot or periodic clocks to be in sub 1ms intervals (causing said
tasklet to be scheduled for such small intervalls).
The problem appears when PCI passthrough devices are used over many
sockets and we have an mix of heavy-interrupt guests and idle guests.
The idle guests end up seeing 1/10 of its RUNNING timeslice eaten by
the hypervisor (and 40% steal time).
The mechanism by which we inject PCI interrupts is by hvm_do_IRQ_dpci
which schedules the hvm_dirq_assist tasklet every time an interrupt is
received. The callchain is:
_asm_vmexit_handler
-> vmx_vmexit_handler
->vmx_do_extint
-> do_IRQ
-> __do_IRQ_guest
-> hvm_do_IRQ_dpci
tasklet_schedule(&dpci->dirq_tasklet);
[takes lock to put the tasklet on]
[later on the schedule_tail is invoked which is 'vmx_do_resume']
vmx_do_resume
-> vmx_asm_do_vmentry
-> call vmx_intr_assist
-> vmx_process_softirqs
-> do_softirq
[executes the tasklet function, takes the
lock again]
While on other CPUs they might be sitting in a idle loop and invoked to
deliver an VIRQ_TIMER, which also ends up taking the lock twice: first
to schedule the v->arch.hvm_vcpu.assert_evtchn_irq_tasklet (accounted
to the guests' BLOCKED_state); then to execute it - which is accounted
for in the guest's RUNTIME_state.
The end result is that on a 8 socket machine with PCI passthrough,
where four sockets are busy with interrupts, and the other sockets have
idle guests - we end up with the idle guests having around 40% steal
time and 1/10 of its timeslice (3ms out of 30 ms) being tied up taking
the lock. The latency of the PCI interrupts delieved to guest is also
hindered.
With this patch the problem disappears completly. That is removing the
lock for the PCI passthrough use-case (the 'hvm_dirq_assist' case) by
not using tasklets at all.
The patch is simple - instead of scheduling an tasklet we schedule our
own softirq - HVM_DPCI_SOFTIRQ, which will take care of running
'hvm_dirq_assist'. The information we need on each CPU is which
'struct hvm_pirq_dpci' structure the 'hvm_dirq_assist' needs to run on.
That is simple solved by threading the 'struct hvm_pirq_dpci' through a
linked list. The rule of only running one 'hvm_dirq_assist' for only
one 'hvm_pirq_dpci' is also preserved by having 'schedule_dpci_for'
ignore any subsequent calls for an domain which has already been
scheduled.
== Code details ==
Most of the code complexity comes from the '->dom' field in the
'hvm_pirq_dpci' structure. We use it for ref-counting and as such it
MUST be valid as long as STATE_SCHED bit is set. Whoever clears the
STATE_SCHED bit does the ref-counting and can also reset the '->dom'
field.
To compound the complexity, there are multiple points where the
'hvm_pirq_dpci' structure is reset or re-used. Initially (first time
the domain uses the pirq), the 'hvm_pirq_dpci->dom' field is set to
NULL as it is allocated. On subsequent calls in to 'pt_irq_create_bind'
the ->dom is whatever it had last time.
As this is the initial call (which QEMU ends up calling when the guest
writes an vector value in the MSI field) we MUST set the '->dom' to a
the proper structure (otherwise we cannot do proper ref-counting).
The mechanism to tear it down is more complex as there are three ways
it can be executed. To make it simpler everything revolves around
'pt_pirq_softirq_active'. If it returns -EAGAIN that means there is an
outstanding softirq that needs to finish running before we can continue
tearing down. With that in mind:
a) pci_clean_dpci_irq. This gets called when the guest is being
destroyed. We end up calling 'pt_pirq_softirq_active' to see if it
is OK to continue the destruction.
The scenarios in which the 'struct pirq' (and subsequently the
'hvm_pirq_dpci') gets destroyed is when:
- guest did not use the pirq at all after setup.
- guest did use pirq, but decided to mask and left it in that state.
- guest did use pirq, but crashed.
In all of those scenarios we end up calling 'pt_pirq_softirq_active'
to check if the softirq is still active. Read below on the
'pt_pirq_softirq_active' loop.
b) pt_irq_destroy_bind (guest disables the MSI). We double-check that
the softirq has run by piggy-backing on the existing
'pirq_cleanup_check' mechanism which calls 'pt_pirq_cleanup_check'.
We add the extra call to 'pt_pirq_softirq_active' in
'pt_pirq_cleanup_check'.
NOTE: Guests that use event channels unbind first the event channel
from PIRQs, so the 'pt_pirq_cleanup_check' won't be called as 'event'
is set to zero. In that case we either clean it up via the a) or c)
mechanism.
There is an extra scenario regardless of 'event' being set or not:
the guest did 'pt_irq_destroy_bind' while an interrupt was triggered
and softirq was scheduled (but had not been run). It is OK to still
run the softirq as hvm_dirq_assist won't do anything (as the flags
are set to zero). However we will try to deschedule the softirq if
we can (by clearing the STATE_SCHED bit and us doing the
ref-counting).
c) pt_irq_create_bind (not a typo). The scenarios are:
- guest disables the MSI and then enables it (rmmod and modprobe in
a loop). We call 'pt_pirq_reset' which checks to see if the
softirq has been scheduled. Imagine the 'b)' with interrupts in
flight and c) getting called in a loop.
We will spin up on 'pt_pirq_is_active' (at the start of the
'pt_irq_create_bind') with the event_lock spinlock dropped and waiting
(cpu_relax). We cannot call 'process_pending_softirqs' as it might
result in a dead-lock. hvm_dirq_assist will be executed and then the
softirq will clear 'state' which signals that that we can re-use the
'hvm_pirq_dpci' structure. In case this softirq is scheduled on a
remote CPU the softirq will run on it as the semantics behind an
softirq is that it will execute within the guest interruption.
- we hit once the error paths in 'pt_irq_create_bind' while an
interrupt was triggered and softirq was scheduled.
If the softirq is in STATE_RUN that means it is executing and we should
let it continue on. We can clear the '->dom' field as the softirq has
stashed it beforehand. If the softirq is STATE_SCHED and we are
successful in clearing it, we do the ref-counting and clear the '->dom'
field. Otherwise we let the softirq continue on and the '->dom' field
is left intact. The clearing of the '->dom' is left to a), b) or again
c) case.
Note that in both cases the 'flags' variable is cleared so
hvm_dirq_assist won't actually do anything.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When an interrupt for an PCI (or PCIe) passthrough device is to be sent
to a guest, we find the appropiate 'hvm_dirq_dpci' structure for the
interrupt (PIRQ), set a bit (masked), and schedule an tasklet.
Then the 'hvm_dirq_assist' tasklet gets called with the 'struct domain'
from where it iterates over the the radix-tree of 'hvm_dirq_dpci' (from
zero to the number of PIRQs allocated) which are masked to the guest
and calls each 'hvm_pirq_assist'. If the PIRQ has a bit set (masked) it
figures out how to inject the PIRQ to the guest.
This is inefficient and not fair as:
- We iterate starting at PIRQ 0 and up every time. That means the PCIe
devices that have lower PIRQs get to be called first.
- If we have many PCIe devices passed in with many PIRQs and if most
of the time only the highest numbered PIRQ get an interrupt (as the
initial ones are for control) we end up iterating over many PIRQs.
But we could do beter - the 'hvm_dirq_dpci' has the field for
'struct domain', which we can use instead of having to pass in the
'struct domain'.
As such this patch moves the tasklet to the 'struct hvm_dirq_dpci' and
sets the 'dom' field to the domain. We also double-check that the
'->dom' is not reset before using it.
We have to be careful with this as that means we MUST have 'dom' set
before pirq_guest_bind() is called. As such we add the
'pirq_dpci->dom = d;' to cover for such cases.
The mechanism to tear it down is more complex as there are two ways it
can be executed:
a) pci_clean_dpci_irq. This gets called when the guest is being
destroyed. We end up calling 'tasklet_kill'.
The scenarios in which the 'struct pirq' (and subsequently the
'hvm_pirq_dpci') gets destroyed is when:
- guest did not use the pirq at all after setup.
- guest did use pirq, but decided to mask and left it in that
state.
- guest did use pirq, but crashed.
In all of those scenarios we end up calling 'tasklet_kill' which
will spin on the tasklet if it is running.
b) pt_irq_destroy_bind (guest disables the MSI). We double-check that
the softirq has run by piggy-backing on the existing
'pirq_cleanup_check' mechanism which calls 'pt_pirq_cleanup_check'.
We add the extra call to 'pt_pirq_softirq_active' in
'pt_pirq_cleanup_check'.
NOTE: Guests that use event channels unbind first the event channel
from PIRQs, so the 'pt_pirq_cleanup_check' won't be called as event
is set to zero. In that case we either clean it up via the a)
mechanism. It is OK to re-use the tasklet when 'pt_irq_create_bind'
is called afterwards.
There is an extra scenario regardless of event being set or not:
the guest did 'pt_irq_destroy_bind' while an interrupt was
triggered and tasklet was scheduled (but had not been run). It is
OK to still run the tasklet as hvm_dirq_assist won't do anything
(as the flags are set to zero). As such we can exit out of
hvm_dirq_assist without doing anything.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Fix the following two issues in rtds scheduler:
1) The runq queue lock is not grabbed when rt_update_deadline is
called in rt_alloc_vdata function, which may cause race condition;
Solution: Move call to rt_update_deadline from _alloc to _insert;
Note: rt_alloc_vdata does not need grab the runq lock, because only one
cpu will allocate the rt_vcpu; before the rt_vcpu is inserted into the
runq, no more than one cpu operates on the rt_vcpu.
2) rt_vcpu_remove should grab the runq lock before remove the vcpu
from runq; otherwise, race condition may happen.
Solution: Add lock in rt_vcpu_remove().
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Meng Xu <mengxu@cis.upenn.edu>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@eu.citrix.com>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
The build of xen-4.5.0-rc2 fails if XSM_ENABLE=y due to an inconsistency
in commit fda1614 "xen/arm: Add support for GICv3 for domU" which uses
XEN_DOMCTL_configure_domain in xen/xsm/flask/hooks.c and
xen/xsm/flask/policy/access_vectors but XEN_DOMCTL_arm_configure_domain
elsewhere.
Michael Young
In fda1614 ("xen/arm: Add support for GICv3 for domU")
XEN_DOMCTL_configure_domain is used in xen/xsm/flask/hooks.c and
xen/xsm/flask/policy/access_vectors but XEN_DOMCTL_arm_configure_domain
is used elsewhere.
Signed-off-by: Michael Young <m.a.young@durham.ac.uk>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
When using pvgrub in graphical mode with vnc, the grub timeout doesn't
work: the countdown doesn't even start. With a serial terminal the
problem doesn't occur and the countdown works as expected.
It turns out that the problem is that when using a graphical terminal,
checkkey () returns 0 instead of -1 when there is no activity on the
mouse or keyboard. As a consequence grub thinks that the user typed
something and interrupts the count down.
To fix the issue simply ignore keystrokes returning 0, that is the NUL
character anyway. Add a patch to grub.patches to do that.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: Steven Haigh <netwiz@crc.id.au>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Olaf Hering