I am no longer in a position to be a Xen maintainer. Andrew has
kindly volunteered to continue maintainance of the KEXEC subsystem.
EVENT CHANNELS (FIFO-BASED ABI) will be maintained by the "Other"
hypervisor maintainers.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
When showing the CPU state (e.g., after a crash) the dump of code
around RIP is incorrect.
Incorrect:
Xen code around <ffff82d0801113cf> (...):
00 c6 c1 ee 08 48 c1 e0 <04> 03 04 f1 8b ...
^^ Uninitialized ^^ Missing 0x48
Correct:
Xen code around <ffff82d0801113cf> (...):
c6 c1 ee 08 48 c1 e0 04 <48> 03 04 f1 8b ...
When coping the bytes before RIP, the destination was off-by-one.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
Holding the p2m lock while calling ept_sync_domain() is very expensive
since it does an on_selected_cpus() call. IPIs on many socket
machines can be very slow and on_selected_cpus() is serialized.
It is safe to defer the invalidate until the p2m lock is released
except for two cases:
1. When freeing a page table page (since partial translations may be
cached).
2. When reclaiming a zero page as part of PoD.
For these cases, add p2m_tlb_flush_sync() calls which will immediately
perform the invalidate before the page is freed or reclaimed.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: George Dunlap <geroge.dunlap@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
The hardware may not write the FIP/FDP fields with a XSAVE*
instruction. e.g., with XSAVEOPT/XSAVES if the state hasn't changed
or on AMD CPUs when a floating point exception is not pending. We
need to identify this case so we can correctly apply the check for
whether to save/restore FCS/FDS.
By poisoning FIP in the saved state we can check if the hardware
writes to this field. The poison value is both: a) non-canonical; and
b) random with a vanishingly small probability of matching a value
written by the hardware (1 / (2^63) = 10^-19).
The poison value is fixed and thus knowable by a guest (or guest
userspace). This could allow the guest to cause Xen to incorrectly
detect that the field has not been written. But: a) this requires the
FIP register to be a full 64 bits internally which is not the case for
all current AMD and Intel CPUs; and b) this only allows the guest (or
a guest userspace process) to corrupt its own state (i.e., it cannot
affect the state of another guest or another user space process).
This results in smaller code with fewer branches and is more
understandable.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Intel confirmed that 64-bit {F,}XRSTOR sign-extend FIP from bit 47.
While leaving the description above intact, modify the code comment
accordingly.
Reviewed-by: Jan Beulich <jbeulich@suse.com>
If the MMIO hole is large and hvmloader needs to relocate memory to
immediately above the 4 GiB boundary, the e820 presented to the guest
will not have a RAM region above 4 GiB.
e.g., a guest with 3 GiB of memory and a 2 GiB MMIO hole will only see
2 GiB.
The required e820 memory region above 4 GiB needs to be added, and not
just filled in.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Introduce the FLUSH_VA_VALID flag to flush_area_mask() and friends to
say that it is safe to use CLFLUSH (i.e., the virtual address is still
valid).
Use this when changing the cachability of the Xen direct mappings (in
response to the guest changing the cachability of its mappings). This
significantly improves performance by avoiding an expensive WBINVD.
This fixes a performance regression introduced by
c61a6f74f8 (x86: enforce consistent
cachability of MMIO mappings), the fix for XSA-154.
e.g., A set_memory_wc() call in Linux:
before: 4097 us
after: 47 us
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The HVM parameter HVM_PARAM_X87_FIP_WIDTH to allow tools and the guest
to adjust the width of the FIP/FDP registers to be saved/restored by
the hypervisor. This is in case the hypervisor hueristics do not do
the right thing.
Add this parameter to the set saved during domain save/migrate.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The x86 architecture allows either: a) the 64-bit FIP/FDP registers to
be restored (clearing FCS and FDS); or b) the 32-bit FIP/FDP and
FCS/FDS registers to be restored (clearing the upper 32-bits).
Add a per-domain field to indicate which of these options a guest
needs. The options are: 8, 4 or 0. Where 0 indicates that the
hypervisor should automatically guess the FIP width by checking the
value of FIP/FDP when saving the state (this is the existing
behaviour).
The FIP width is initially automatic but is set explicitly in the
following cases:
- 32-bit PV guest: 4
- Newer CPUs that do not save FCS/FDS: 8
The x87_fip_width field is placed into an existing 1 byte hole in
struct arch_domain.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Fix build.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Since the NMI handler can now recognize watchdog NMIs, make
check_nmi_watchdog() only check for at least two watchdog NMIs. This
prevents false negatives caused by other processors (which may be
being power managed by the BIOS) running at reduced clock frequencies.
We check for more than one NMI since there are apparently systems
where the NMI works only once.
This will also slightly speed up boot times since we only wait the
full 10 ticks if the NMI watchdog on one or more CPUs is not working.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
atomic_compareandswap() used atomic_t as the new, old and returned
values which is less convenient than using just int.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
If a guest allocates a page and the tlbflush_timestamp on the page
indicates that a TLB flush of the previous owner is required, only the
linear and combined mappings are invalidated. The guest-physical
mappings are not invalidated.
This is currently safe because the EPT code ensures that the
guest-physical and combined mappings are invalidated /before/ the page
is freed. However, this prevents us from deferring the EPT invalidate
until after the page is freed (e.g., to defer the invalidate until the
p2m locks are released).
The TLB flush that may be done after allocating page already causes
the original guest to VMEXIT, thus on VMENTER we can do an INVEPT if
one is pending.
This means __ept_sync_domain() need not do anything and the thus the
on_selected_cpu() call does not need to wait for as long.
ept_sync_domain() now marks all PCPUs as needing to be invalidated,
including PCPUs that the domain has not run on. We still only IPI
those PCPUs that are active so this does not result in any more INVEPT
calls.
We do not attempt to track when PCPUs may have cached translations
because the only safe way to clear this per-CPU state is if
immediately after an invalidate the PCPU is not active (i.e., the PCPU
is not in d->domain_dirty_cpumask). Since we only invalidate on
VMENTER or by IPIing active PCPUs this can never happen.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Using RING_GET_REQUEST() on a shared ring is easy to use incorrectly
(i.e., by not considering that the other end may alter the data in the
shared ring while it is being inspected). Safe usage of a request
generally requires taking a local copy.
Provide a RING_COPY_REQUEST() macro to use instead of
RING_GET_REQUEST() and an open-coded memcpy(). This takes care of
ensuring that the copy is done correctly regardless of any possible
compiler optimizations.
Use a volatile source to prevent the compiler from reordering or
omitting the copy.
This is part of XSA155.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
When using the FIFO ABI a guest may close an event channel that is
still LINKED. If this port is reused, subsequent events may be lost
because they may become pending on the wrong queue.
This could be fixed by requiring guests to only close event channels
that are not linked. This is difficult since: a) irq cleanup in the
guest may be done in a context that cannot wait for the event to be
unlinked; b) the guest may attempt to rebind a PIRQ whose previous
close is still pending; and c) existing guests already have the
problematic behaviour.
Instead, simply check a port is not "busy" (i.e., it's not linked)
before reusing it.
Guests should still drain any queues for VCPUs that are being
offlined, or the port will become unusable until the VCPU is onlined
and starts processing events again.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
When using EPT, type changes are done with the following steps:
1. Set entry as invalid (misconfigured) by settings a reserved memory
type.
2. Flush all EPT and combined translations (ept_sync_domain()).
3. Fixup misconfigured entries as required (on EPT_MISCONFIG vmexits or
when explicitly setting an entry.
Since resolve_misconfig() only updates entries that were misconfigured,
there is no need to invalidate any translations since the hardware
does not cache misconfigured translations (vol 3, section 28.3.2).
Remove the unnecessary (and very expensive) ept_sync_domain() calls).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Instead of cpu_relax() while spinning and observing the ticket head,
introduce arch_lock_relax() which executes a WFE instruction. After
the ticket head is changed call arch_lock_signal() to execute an SEV
instruction (with the required DSB first) to wake any spinners.
This should improve power consumption when locks are contented and
spinning.
For consistency also move arch_lock_(acquire|release)_barrier to
asm/spinlock.h.
Booted the result on arm32 (Midway) and arm64 (Mustang). Build test
only on amd64.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
[ijc: add barrier, rename as arch_lock_*, move arch_lock_*_barrier, test]
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Commit 902d1b5c31 (x86,arm: remove
asm/spinlock.h from all architectures) inadvertantly removed an
x86-specific optimization for read_unlock*().
Re-add asm/spinlock.h to allow architectures to provide an optmized
_raw_read_unlock() and make x86 provide the previous implementation.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Now that all architecture use a common ticket lock implementation for
spinlocks, remove the architecture specific byte lock implementations.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Replace the byte locks with ticket locks. Ticket locks are: a) fair;
and b) peform better when contented since they spin without an atomic
operation.
The lock is split into two ticket values: head and tail. A locker
acquires a ticket by (atomically) increasing tail and using the
previous tail value. A CPU holds the lock if its ticket == head. The
lock is released by increasing head.
spin_lock_irq() and spin_lock_irqsave() now spin with irqs disabled
(previously, they would spin with irqs enabled if possible). This is
required to prevent deadlocks when the irq handler tries to take the
same lock with a higher ticket.
Architectures need only provide arch_fetch_and_add() and two barriers:
arch_lock_acquire_barrier() and arch_lock_release_barrier().
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The number of struct evtchn in a page must be a power of two. Under
some workloads performance is improved slightly by padding struct
evtchn to 64 bytes (a typical cache line size), thus putting the fewer
per-channel locks into each cache line.
This does not decrease the number of struct evtchn's per-page.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
When sending an event, use a new per-event channel lock to safely
validate the event channel state.
This new lock must be held when changing event channel state. Note
that the event channel lock must also be held when changing state from
ECS_FREE or it will race with a concurrent get_free_port() call.
To avoid having to take the remote event channel locks when sending to
an interdomain event channel, the local and remote channel locks are
both held when binding or closing an interdomain event channel.
This significantly increases the number of events that can be sent
from multiple VCPUs. But struct evtchn increases in size, reducing
the number that fit into a single page to 64 (instead of 128).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
notify_via_xen_event_channel() and free_xen_event_channel() had to
check if the domain was dying because they may be called while the
domain is being destroyed and the struct evtchn's are being freed.
By deferring the freeing of the struct evtchn's until all references
to the domain are dropped, these functions can rely on the channel
state being present and valid.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Freeing a xen event channel would clear xen_consumer before clearing
the channel state, leaving a window where the channel is in a funny
state (still bound but no consumer).
Move the clear of xen_consumer into free_evtchn() where the state is
also cleared.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Ditch the pointless evtchn_close() wrapper around __evtchn_close()
(renaming the latter) as well as some bogus casts of function results
to void.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
If a guest is not evenly grant mapping across its VCPUs one of the
VCPUs may run out of free maptrack entries even though other VCPUs
have many free.
If this happens, "steal" free entries from other VCPUs. We want to
steal entries such that:
a) We avoid ping-ponging stolen entries between VCPUs.
b) The number of free entries owned by each VCPUs tends (over time) to
the number it uses.
So when stealing, we select a VCPU at random (reducing (a)) and we
transfer the stolen entries to the thief VCPU (aiming for (b)).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
By keeping a count of the number of currently valid event channels,
port_is_valid() can be simplified.
d->valid_evtchns is only increased (while holding d->event_lock), so
port_is_valid() may be safely called without taking the lock (this
will be useful later).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
We're going to want to free an event channel from two places. Factor out
the code into a free_evtchn() function.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
In combination with the per-active entry locks, the grant table lock
can be made a read-write lock since the majority of cases only the
read lock is required. The grant table read lock protects against
changes to the table version or size (which are done with the write
lock held).
The write lock is also required when two active entries must be
acquired.
The double lock is still required when updating IOMMU page tables.
With the lock contention being only on the maptrack lock (unless IOMMU
updates are required), performance and scalability is improved.
Based on a patch originally by Matt Wilson <msw@amazon.com>.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Split grant table lock into two separate locks. One to protect
maptrack free list (maptrack_lock) and one for everything else (lock).
Based on a patch originally by Matt Wilson <msw@amazon.com>.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Introduce a per-active entry spin lock to protect active entry state
The grant table lock must be locked before acquiring (locking) an
active entry.
This is a step in reducing contention on the grant table lock, but
will only do so once the grant table lock is turned into a read-write
lock.
Based on a patch originally by Matt Wilson <msw@amazon.com>.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The per-domain event channel lock may suffer from contention. Add it to
the set of locks to be profiled when lock profiling is enabled.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Older GCC versions such as 4.3 cannot have initializers for the
members of anonymous structures, so initialize .head_tail instead.
Use a SPINLOCK_TICKET_INC define so this initializer is near the
spinlock_tickets_t definition (in case the structure changes requiring
changes to the initializer).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reported-and-tested-by: Jan Beulich <jbeulich@suse.com>
Now that all architecture use a common ticket lock implementation for
spinlocks, remove the architecture specific byte lock implementations.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Replace the byte locks with ticket locks. Ticket locks are: a) fair;
and b) peform better when contented since they spin without an atomic
operation.
The lock is split into two ticket values: head and tail. A locker
acquires a ticket by (atomically) increasing tail and using the
previous tail value. A CPU holds the lock if its ticket == head. The
lock is released by increasing head.
spin_lock_irq() and spin_lock_irqsave() now spin with irqs disabled
(previously, they would spin with irqs enabled if possible). This is
required to prevent deadlocks when the irq handler tries to take the
same lock with a higher ticket.
Architectures need only provide arch_fetch_and_add() and two barriers:
arch_lock_acquire_barrier() and arch_lock_release_barrier().
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
add_sized(ptr, inc) adds inc to the value at ptr using only the correct
size of loads and stores for the type of *ptr. The add is /not/ atomic.
This is needed for ticket locks to ensure the increment of the head ticket
does not affect the tail ticket.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
add_sized(ptr, inc) adds inc to the value at ptr using only the correct
size of loads and stores for the type of *ptr. The add is /not/ atomic.
This is needed for ticket locks to ensure the increment of the head ticket
does not affect the tail ticket.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Pack struct paging_domain to reduce it by 8 bytes. Thus reducing the
size of struct domain by 8 bytes.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
arch_fetch_and_add() atomically adds a value and returns the previous
value.
This generic arm implementation uses the GCC __sync_fetch_and_add()
builtin. This builtin resulted in suitable inlined asm for GCC 4.8.3
(arm64) and GCC 4.6.3 (arm32).
This is needed to implement ticket locks.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
arch_fetch_and_add() atomically adds a value and returns the previous
value.
This is needed to implement ticket locks.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Add the specification for a new migration stream format. The document
includes all the details but to summarize:
The existing (legacy) format is dependant on the word size of the
toolstack. This prevents domains from migrating from hosts running
32-bit toolstacks to hosts running 64-bit toolstacks (and vice-versa).
The legacy format lacks any version information making it difficult to
extend in compatible way.
The new format has a header (the image header) with version information,
a domain header with basic information of the domain and a stream of
records for the image data.
The format will be used for future domain types (such as on ARM).
The specification is pandoc format (an extended markdown format) and the
documentation build system is extended to support pandoc format documents.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Acquiring a page for the source or destination of a grant copy is an
expensive operation. A common use case is for two adjacent grant copy
ops to operate on either the same source or the same destination page.
Instead of always acquiring and releasing destination and source pages
for each operation, release the page once it is no longer valid for
the next op.
If either the source or destination domains changes both pages are
released as it is unlikely that either will still be valid.
XenServer's performance benchmarks show modest improvements in network
receive throughput (netback uses grant copy in the guest Rx path) and
no regressions in disk performance (using tapdisk3 which grant copies
as the backend).
Baseline Deferred Release
Interhost receive to VM 7.2 Gb/s ~9 Gbit/s
Interhost aggregate 24 Gb/s 28 Gb/s
Intrahost single stream 14 Gb/s 14 Gb/s
Intrahost aggregate 34 Gb/s 36 Gb/s
Aggregate disk write 900 MB/s 900 MB/s
Aggregate disk read 890 MB/s 890 MB/s
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Much of the grant copy operation is identical for the source and
destination buffers. Refactor the code into per-buffer functions.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
When using the FIFO-based event channels, there are no checks for the
existance of a control block when binding an event or moving it to a
different VCPU. This is because events may be bound when the ABI is
in 2-level mode (e.g., by the toolstack before the domain is started).
The guest may trigger a Xen crash in evtchn_fifo_set_pending() if:
a) the event is bound to a VCPU without a control block; or
b) VCPU 0 does not have a control block.
In case (a), Xen will crash when looking up the current queue. In
(b), Xen will crash when looking up the old queue (which defaults to a
queue on VCPU 0).
By allocating all the per-VCPU structures when enabling the FIFO ABI,
we can be sure that v->evtchn_fifo is always valid.
EVTCHNOP_init_control_block for all the other CPUs need only map the
shared control block.
A single check in evtchn_fifo_set_pending() before accessing the
control block fixes all cases where the guest has not initialized some
control blocks.
This is XSA-107.
Reported-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
If the "generation_id" option is set in the domain configuration,
generate and set a new random VM generation ID every time a domain is
created or restored.
xl lacks the infrastructure to fully track the lifecycle of VM images
as they are snapshotted and cloned (etc) so always using a new ID is
the safe option and ensures that a new one will be used where it matters.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Toolstacks may specify a VM generation ID using the u.hvm.ms_vm_genid
field in the libxl_domain_build_info structure, when creating a
domain.
The toolstack is responsible for providing the correct generation ID
according to the Microsoft specification (e.g., generating new random
ones with libxl_ms_vm_genid_generate() as appropriate when restoring).
Although the specification requires that a ACPI Notify event is raised
if the generation ID is changed, the generation ID is never changed
when the domain is in a state to receive such an event (it's either
newly created or suspended).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Cc: David Scott <dave.scott@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Make the set of MCEs that xen-mceinj can inject by data-driven instead
of a whole pile of cut-and-paste functions.
There is no functional change.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Christoph Egger <chegger@amazon.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Correctly name --page option and make it clear that it takes a
parameter. Use consistent formatting for help message.
Remove unused options.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
err() calls perror() which appends a newline.
Make Lprintf() also append a newline so err() and Lprintf() are
consistent.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Christoph Egger <chegger@amazon.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The VM generation ID support in libxc/libxl was based on a draft
specification which subsequently changed considerably. Remove much of
the code in anticipation of introducing something simpler that
conforms to the current specification from Microsoft.
Switch to using a HVM param (HVM_PARAM_VM_GENERATION_ID_ADDR) instead
of the hvmloader/generation-id-address XenStore key. This simplifies
save/restore since it only needs to transfer the value of this param.
There are no changes to the migration stream format or the public
libxl API.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
[ ijc -- updated xc_nomigrate.c for interface change ]
HVM_PARAM_VM_GENERATION_ID_ADDR is the guest physical address of the
VM Generation ID. This parameter will be written by hvmloader and read
by the toolstack when updating the gen. ID (e.g., after restoring from a
snapshot).
A HVM parameter is easier for the save/restore code to work with (than
a XenStore key).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
The random bytes are obtained from /dev/urandom and are suitable for
almost all uses (except for generating long-lived secure keys).
Documentation suggests that /dev/urandom is widely available on Unix-like
systems (such FreeBSD and NetBSD).
A public libxl_random_bytes() (or similar) could be trivially added,
if this required in the future.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
HVMOP_get_param and HVMOP_set_param take a uint32_t for the parameter
index and a uint64_t for the value. So add xc_hvm_param_get() and
xc_hvm_param_set() that take the same types.
The existing xc_get_hvm_param() and xc_set_hvm_param() are retained
for compatibility but are deprecated.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
[ ijc -- dropped one hunk against xen-access.c which didn't apply due to
intervening changes ]
Add const to parameters of libxl_uuid_*() calls where it does not
change the API.
Add libxl_uuid_byte_array_const() to return a const array.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
DECLARE_HYPERCALL_BUFFER_SHADOW() is like DECLARE_HYPERCALL_BUFFER()
except it is backed by an already allocated hypercall buffer.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
There's no need to print all the CPUs that are ok, only the ones that
got stuck.
The resulting output is either:
Testing NMI watchdog on all CPUs: 1 4 6 stuck
or
Testing NMI watchdog on all CPUs: ok
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
All callers of check_nmi_watchdog() already have local irqs enabled so
remove the unpaired local_irq_enable().
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
The LINUX (PV_OPS) section was out-dated and it's better to only have
this information in one place (Tte Linux MAINTAINERS file).
The LINUX (XCP) section was an external project that that hasn't been
maintained for years.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Commit 7113a45451 (kexec/x86: do not map
crash kernel area) causes fatal page faults when loading a crash
image. The attempt to zero the first control page allocated from the
crash region will fault as the VA return by map_domain_page() has no
mapping.
The fault will occur on non-debug builds of Xen when the crash area is
below 5 TiB (which will be most systems).
The assumption that the crash area mapping was not used is incorrect.
map_domain_page() is used when loading an image and building the
image's page tables to temporarily map the crash area, thus the
mapping is required if the crash area is in the direct map area.
Reintroduce the mapping, but only the portions of the crash area that
are within the direct map area.
Reported-by: Don Slutz <dslutz@verizon.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Tested-by: Don Slutz <dslutz@verizon.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Daniel Kiper <daniel.kiper@oracle.com>
This is really just a band aid - kexec shouldn't rely on the crash area
being always mapped when in the direct mapping range (and it didn't use
to in its previous form). That's primarily because map_domain_page()
(needed when the area is outside the direct mapping range) may be
unusable when wanting to kexec due to a crash, but also because in the
case of PFN compression the kexec range (if specified on the command
line) could fall into a hole between used memory ranges (while we're
currently only ignoring memory at the top of the physical address
space, it's pretty clear that sooner or later we will want that
selection to become more sophisticated in order to maximize the memory
made use of).
Acked-by: Jan Beulich <jbeulich@suse.com>
An event may still be the tail of a queue even if the queue is now
empty (an 'old tail' event). There is logic to handle the case when
this old tail event needs to be added to the now empty queue (by
checking for q->tail == port).
However, this does not cover all cases.
1. An old tail may be re-added simultaneously with another event.
LINKED is set on the old tail, and the other CPU may misinterpret
this as the old tail still being valid and set LINK instead of
HEAD. All events on this queue will then be lost.
2. If the old tail event on queue A is moved to a different queue B
(by changing its VCPU or priority), the event may then be linked
onto queue B. When another event is linked onto queue A it will
check the old tail, see that it is linked (but on queue B) and
overwrite the LINK field, corrupting both queues.
When an event is linked, save the vcpu id and priority of the queue it
is being linked onto. Use this when linking an event to check if it
is an unlinked old tail event. If it is an old tail event, the old
queue is empty and old_q->tail is invalidated to ensure adding another
event to old_q will update HEAD. The tail is invalidated by setting
it to 0 since the event 0 is never linked.
The old_q->lock is held while setting LINKED to avoid the race with
the test of LINKED in evtchn_fifo_set_link().
Since a event channel may move queues after old_q->lock is acquired,
we must check that we have the correct lock and retry if not. Since
changing VCPUs or priority is expected to be rare events that are
serialized in the guest, we try at most 3 times before dropping the
event. This prevents a malicious guest from repeatedly adjusting
priority to prevent another domain from acquiring old_q->lock.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Event channel ports that are reused or that were not in the initial
bucket would have a non-default priority.
Add an init evtchn_port_op hook and use this to set the priority when
an event channel is bound.
Within this new evtchn_fifo_init() call, also check if the event is
already on a queue and print a warning, as this event may have its
first event delivered on a queue with the wrong VCPU or priority.
This guest is expected to prevent this (if it cares) by not unbinding
events that are still linked.
Reported-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
If a HVM guest attempts to use the FIFO-based ABI it will not receive
any events and destroying the guest may crash Xen or trigger an assert
when attempting to unmap a control block page. This occurs because
Xen maps the wrong page for both the control blocks and the event
arrays.
In map_guest_page(), use the MFN of the guest's page and not the GFN
when calling map_domain_page_global().
Reported-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Setting a queue's READY bit for every event added to the queue
introduces a race.
If an event is added to the tail of a queue, the guest may consume the
newly added event and leave an empty queue before the READY is set.
The guest may then see a stale HEAD value and if the event at the
stale head became linked onto a different queue, the guest would
consume events from the wrong queue (corrupting it).
As noted in section 4.1.2 of the design document, only set READY if a
new HEAD is set. This ensures that if the guest sees a READY bit set
the corresponding HEAD is valid.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
nmi_shootdown_cpus() is called during a crash to park all the other
CPUs. This changes the NMI trap handlers which means there's no point
in having the watchdog still running.
This also disables the watchdog before executing any crash kexec image
and prevents the image from receiving unexpected NMIs.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
PVOps Linux as a kexec image shoots itself in the foot otherwise.
On a Core2 system, Linux declares a firmware bug and tries to invert some bits
in the performance counter register. It ends up setting the number of retired
instructions to generate another NMI to fewer instructions than the NMI
interrupt path itself, and ceases to make any useful progress.
The call to disable_lapic_nmi_watchdog() must be this late into the kexec path
to be sure that this cpu is the one which will execute the kexec image.
Otherwise there are race conditions where the NMIs might be disabled on the
wrong cpu, resulting in the kexec image still receiving NMIs.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CID 1128566
If an image source page is allocated in kimage_alloc_page() but the
machine_kexec_add_page() fails, the image may appear to load
succesfully but it will not execute. The relocation will fault
(rebooting the host) when trying to copy the source page, as it is not
mapped.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
CID 1128573
If a bad image type is supplied in a KEXECOP_unload hypercall, the
kexec_lock in kexec_swap_images() was left locked, causing a deadlock
on a subsequent image load or unload.
The kexec_lock is only required to serialize the swap operation
itself.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
A malicious or buggy guest can cause another domain to spin
indefinitely by repeatedly writing to an event word when the other
guest is trying to link a new event. The cmpxchg() in
evtchn_fifo_set_link() will repeatedly fail and the loop may never
terminate.
Fixing this requires a change to the ABI which is documented in draft
H of the design.
http://xenbits.xen.org/people/dvrabel/event-channels-H.pdf
Since a well-behaved guest only makes a limited set of state changes,
the loop can terminate early if the guest makes an invalid state
transition.
The guest may:
- clear LINKED and LINK.
- clear PENDING
- set MASKED
- clear MASKED
It is valid for the guest to mask and unmask an event at any time so
specify that it is not valid for a guest to clear MASKED if Xen is
trying to update LINK. Indicate this to the guest with an additional
BUSY bit in the event word. The guest must not clear MASKED if BUSY
is set and it should spin until BUSY is cleared.
The remaining valid writes (clear LINKED, clear PENDING, set MASKED,
clear MASKED by Xen) will limit the number of failures of the
cmpxchg() to at most 4. A clear of LINKED will also terminate the
loop early. Therefore, the loop can then be limited to at most 4
iterations.
If the buggy or malicious guest does cause the loop to exit with
LINKED set and LINK unset then that buggy guest will lose events.
Reported-by: Anthony Liguori <aliguori@amazon.com>
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
In the existing kexec hypercall, the load and unload ops depend on
internals of the Linux kernel (the page list and code page provided by
the kernel). The code page is used to transition between Xen context
and the image so using kernel code doesn't make sense and will not
work for PVH guests.
Add replacement KEXEC_CMD_kexec_load and KEXEC_CMD_kexec_unload ops
that no longer require a code page to be provided by the guest -- Xen
now provides the code for calling the image directly.
The new load op looks similar to the Linux kexec_load system call and
allows the guest to provide the image data to be loaded. The guest
specifies the architecture of the image which may be a 32-bit subarch
of the hypervisor's architecture (i.e., an EM_386 image on an
EM_X86_64 hypervisor).
The toolstack can now load images without kernel involvement. This is
required for supporting kexec when using a dom0 with an upstream
kernel.
Crash images are copied directly into the crash region on load.
Default images are copied into domheap pages and a list of source and
destination machine addresses is created. This is list is used in
kexec_reloc() to relocate the image to its destination.
The old load and unload sub-ops are still available (as
KEXEC_CMD_load_v1 and KEXEC_CMD_unload_v1) and are implemented on top
of the new infrastructure.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Don Slutz <dslutz@verizon.com>
Tested-by: Don Slutz <dslutz@verizon.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Daniel Kiper <daniel.kiper@oracle.com>
Acked-by: Keir Fraser <keir@xen.org>
Add replacement KEXEC_CMD_load and KEXEC_CMD_unload sub-ops to the
kexec hypercall. These new sub-ops allow a priviledged guest to
provide the image data to be loaded into Xen memory or the crash
region instead of guests loading the image data themselves and
providing the relocation code and metadata.
The old interface is provided to guests requesting an interface
version prior to 4.4.
Bump __XEN_LATEST_INTERFACE_VERSION__ to 0x00040400.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Don Slutz <dslutz@verizon.com>
Tested-by: Don Slutz <dslutz@verizon.com>
Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
Tested-by: Daniel Kiper <daniel.kiper@oracle.com>
Acked-by: Keir Fraser <keir@xen.org>
When the FIFO-based ABI is in use, if an event is bound when the
corresponding event array page is missing any attempt to set the event
pending will lose the event (because there is nowhere to write the
pending state).
This wasn't initially considered an issue because guests were expected
to only bind events once they had expanded the event array, however:
1. A domain may start with events already bound (by the toolstack).
2. The guest does not know what the port number will be until the
event is bound (it doesn't know how many already bound events there
are), so it does not know how many event array pages are required.
This makes it difficult to expand in advanced (the current Linux
implementation expands after binding for example).
To prevent pending events from being lost because there is no array
page, temporarily store the pending state in evtchn->pending. When an
array page is added, use this state to set the port as pending.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Add the 'max_event_channels' option to the xl configuration file to
limit the number of event channels that domain may use.
Plumb this option through to libxl via a new libxl_build_info field
and call xc_domain_set_max_evtchn() in the post build stage of domain
creation.
A new LIBXL_HAVE_BUILDINFO_EVENT_CHANNELS #define indicates that this
new field is available.
The default value of 1023 limits the domain to using the minimum
amount of global mapping pages and at most 5 xenheap pages.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Keir Fraser <keir@xen.org>
Add XEN_DOMCTL_set_max_evtchn which may be used during domain creation to
set the maximum event channel port a domain may use. This may be used to
limit the amount of Xen resources (global mapping space and xenheap) that
a domain may use for event channels.
A domain that does not have a limit set may use all the event channels
supported by the event channel ABI in use.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Acked-by: Keir Fraser <keir@xen.org>
Add the implementation for the FIFO-based event channel ABI. The new
hypercall sub-ops (EVTCHNOP_init_control, EVTCHNOP_expand_array) and
the required evtchn_ops (set_pending, unmask, etc.).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Implement EVTCHNOP_set_priority. A new set_priority hook added to
struct evtchn_port_ops will do the ABI specific validation and setup.
If an ABI does not provide a set_priority hook (as is the case of the
2-level ABI), the sub-op will return -ENOSYS.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Add the event channel hypercall sub-ops and the definitions for the
shared data structures for the FIFO-based event channel ABI.
The design document for this new ABI is available here:
http://xenbits.xen.org/people/dvrabel/event-channels-F.pdf
In summary, events are reported using a per-domain shared event array
of event words. Each event word has PENDING, LINKED and MASKED bits
and a LINK field for pointing to the next event in the event queue.
There are 16 event queues (with different priorities) per-VCPU.
Key advantages of this new ABI include:
- Support for over 100,000 events (2^17).
- 16 different event priorities.
- Improved fairness in event latency through the use of FIFOs.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Expand the number of event channels that can be supported internally
by altering now struct evtchn's are allocated.
The objects are indexed using a two level scheme of groups and buckets
(instead of only buckets). Each group is a page of bucket pointers.
Each bucket is a page-sized array of struct evtchn's.
The optimal number of evtchns per bucket is calculated at compile
time.
If XSM is not enabled, struct evtchn is 16 bytes and each bucket
contains 256, requiring only 1 group of 512 pointers for 2^17
(131,072) event channels. With XSM enabled, struct evtchn is 24
bytes, each bucket contains 128 and 2 groups are required.
For the common case of a domain with only a few event channels,
instead of requiring an additional allocation for the group page, the
first bucket is indexed directly.
As a consequence of this, struct domain shrinks by at least 232 bytes
as 32 bucket pointers are replaced with 1 bucket pointer and (at most)
2 group pointers.
[ Based on a patch from Wei Liu with improvements from Malcolm
Crossley. ]
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Instead of the MAX_EVTCHNS(d) macro, use d->max_evtchns instead. This
avoids having to repeatedly check the ABI type.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
In the output of the 'e' debug key, print some ABI specific state in
addition to the (p)ending and (m)asked bits.
For the 2-level ABI, print the state of that event's selector
bit. e.g.,
(XEN) port [p/m/s]
(XEN) 1 [0/0/1]: s=3 n=0 x=0 d=0 p=74
(XEN) 2 [0/0/1]: s=3 n=0 x=0 d=0 p=75
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
Use functions for the low-level event channel port operations
(set/clear pending, unmask, is_pending and is_masked).
Group these functions into a struct evtchn_port_op so they can be
replaced by alternate implementations (for different ABIs) on a
per-domain basis.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
The 'i' key would always use VCPU0's selector word when printing the
event channel state. Remove the incorrect output as a subsequent
change will add the (correct) information to the 'e' key instead.
When dumping domain information, printing the state of the VIRQ_DEBUG
port is redundant -- this information is available via the 'e' key.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Keir Fraser <keir@xen.org>
From: David Vrabel <david.vrabel@citrix.com>
sched_move_domain() changes v->processor for all the domain's VCPUs.
If another domain, softirq etc. triggers a simultaneous call to
vcpu_wake() (e.g., by setting an event channel as pending), then
vcpu_wake() may lock one schedule lock and try to unlock another.
vcpu_schedule_lock() attempts to handle this but only does so for the
window between reading the schedule_lock from the per-CPU data and the
spin_lock() call. This does not help with sched_move_domain()
changing v->processor between the calls to vcpu_schedule_lock() and
vcpu_schedule_unlock().
Fix the race by taking the schedule_lock for v->processor in
sched_move_domain().
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Juergen Gross <juergen.gross@ts.fujitsu.com>
Use vcpu_schedule_lock_irq() (which now returns the lock) to properly
retry the locking should the to be used lock have changed in the course
of acquiring it (issue pointed out by George Dunlap).
Add a comment explaining the state after the v->processor adjustment.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Keir Fraser <keir@xen.org>
In xl.conf, autoballoon="auto" will do the right thing for most
people. Make it the default (instead of "on").
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
autoballoon=1 is not recommened if dom0_mem was used to reduce the
amount of dom0 memory. Instead of requiring users to change xl.conf
if they do this, extend the autoballoon option with a new choice:
"auto".
With autoballoon="auto", autoballooning will be disabled if dom0_mem
was used on the Xen command line.
For consistency, accept "on" and "off" as valid autoballoon options (1
and 0 are still accepted).
The default remains "on" for now.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
David Vrabel