The existing transaction merge algorithm keeps track of the least upper bound
(longest common prefix) of all the nodes which have been read and written, and
will re-combine two stores which have disjoint upper bounds. This works well for
small transactions but causes unnecessary conflicts for ones that span a large
subtree, such as the following ones used by the xapi toolstack:
* VM start: creates /vm/... /vss/... /local/domain/...
The least upper bound of this transaction is / and so all
these transactions conflict with everything.
* Device hotplug: creates /local/domain/0/... /local/domain/n/...
The least upper bound of this transaction is /local/domain so
all these transactions conflict with each other.
If the existing merge algorithm cannot merge and commit, we attempt
a /replay/ of the failed transaction against the new store.
When we replay the requests we check whether the response sent to the client is
the same as during the first attempt at the transaction. If the responses are
all the same then the transaction replay can be committed. If any differ then
the transaction replay must be aborted and the client must retry.
This algorithm uses the intuition that the transactions made by the toolstack
are designed to be for separate domains, and should fundamentally not conflict
in the sense that they don't read or write any shared keys. By replaying the
transaction on the server side we do what the client would have to do anyway,
only we can do it quickly without allowing any other requests to interfere.
Performing 300 parallel simulated VM start and shutdowns without this code:
300 parallel starts and shutdowns: 268.92
Performing 300 parallel simulated VM start and shutdowns with this code:
300 parallel starts and shutdowns: 3.80
Signed-off-by: Dave Scott <dave@recoil.org>
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
Separate the functions which process operations that can be done as part of a
transaction. Specifically, these operations are: read, write, rm, getperms,
setperms, getdomainpath, directory, mkdir.
Also split function_of_type into two functions: one for processing the simple
operations and one for processing the rest.
This will help allow replay of transactions, allowing us to invoke the functions
that process the simple operations as part of the processing of transaction_end.
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
A list of (request, response) pairs from the operations performed within the
transaction will be useful to support transaction replay.
Since this consumes memory, the number of requests per transaction must not be
left unbounded. Hence a new quota for this is introduced. This quota, configured
via the configuration key 'quota-maxrequests', limits the size of transactions
initiated by domUs.
After the maximum number of requests has been exhausted, any further requests
will result in EQUOTA errors. The client may then choose to end the transaction;
a successful commit will result in the retention of only the prior requests.
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
Previously, the functions reply_{ack,data,data_or_ack} and input_handle_error
put the response on the wire by invoking Connection.send_{ack,reply,error}.
Instead, these functions now return a value indicating what needs to be put on
the wire, and that action is done by a send_response function called
afterwards.
This refactoring gives us a chance to store the value of the response, useful
for replaying transactions.
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
The help message did not include information about the --type parameter.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Most of XENVER_* have now an XSM check for their sub-ops.
The subop for XENVER_commandline is now a priviliged operation.
To not break guests we still return an string - but it is
just '<denied>\0'.
The XENVER_[version|platform_parameters|get_features] - will
always return an value to the guest.
The rest: XENVER_[extraversion|capabilities|page_size|
guest_handle|changeset| compile_info] behave as before -
allowed by default for all guests if using the XSM default
policy or with the dummy one. And if the system admin
wants to curtail access to some of them - they can do
that now with a non-default XSM policy.
Also we add a local variable block.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
---
Cc: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
v2: Do XSM check for all the XENVER_ ops.
- Add empty data conditions.
- Return <denied> for priv subops.
- Move extraversion from priv to normal. Drop the XSM check
for the non-priv subops.
v3:
- Add +1 for strlen(xen_deny()) to include NULL. Move changeset,
compile_info to non-priv subops.
- Remove the \0 on xen_deny()
- Add new XSM domain for xenver hypercall. Add all subops to it.
- Remove the extra line, Add Ack from Daniel
v4:
- Rename the XSM from xen_version_op to xsm_xen_version.
Prefix the types with 'xen' to distinguish it from another
hypercall performing similar operation. Removed Ack from Daniel
as it was so large. Add local variable block.
v5:
- Make XENVER_platform_parameters,get_features,version be excluded
from the XSM check per Jans' review. Add BUILD_BUG_CHECK and fix
odd line removals. Remove stray changes and fix spelling.
This patch adds code to enable the APIC assist enlightenment which,
under certain conditions, means that the guest can avoid an EOI of
the local APIC and thereby avoid a VMEXIT.
Use of the enlightenment by the hypervisor is under control of the
toolstack, and is added to the default set.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list,
usbdev-attach and usbdev-detach.
To attach a usb device to guest through pvusb, one could follow
following example:
#xl usbctrl-attach test_vm version=1 ports=8
#xl usb-list test_vm
will show the usb controllers and port usage under the domain.
#xl usbdev-attach test_vm hostbus=1 hostaddr=2
will find the first usable controller:port, and attach usb
device whose busnum is 1 and devnum is 6.
One could also specify which <controller> and which <port>.
#xl usbdev-detach test_vm 0 1
will detach USB device under controller 0 port 1.
#xl usbctrl-detach test_vm dev_id
will destroy the controller with specified dev_id. Dev_id
can be traced in usb-list info.
Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Add code to support pvusb in domain config file. One could specify
usbctrl and usb in domain's configuration file and create domain,
then usb controllers will be created and usb device would be attached
to guest automatically.
One could specify usb controllers and usb devices in config file
like this:
usbctrl=['version=2,ports=4', 'version=1, ports=4', ]
usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ]
Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Add pvusb APIs, including:
- attach/detach (create/destroy) virtual usb controller.
- attach/detach usb device
- list usb controller and usb devices
- some other helper functions
Signed-off-by: Simon Cao <caobosimon@gmail.com>
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Chunyan Liu <cyliu@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
For some device type, device removal operation needs to be
handled specially, like usbctrl, it needs to remove all usb
devices under it first, then remove usbctrl. Extend
DEFINE_DEVICE_REMOVE to support generic and custom way
For those need to be handled specially, call
DEFINE_DEVICE_REMOVE_CUSTOM, it requires user defined
libxl__initiate_device_##type##_remove. Otherwise, just
call DEFINE_DEVICE_REMOVE as before.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Chunyan Liu <cyliu@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Add a new function libxl_read_sysfs_file_contents to handle sysfs file
specially. It would be used in later pvusb work.
Signed-off-by: Chunyan Liu <cyliu@suse.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
xc_domain_save() and xc_domain_restore's parameter will use this type,
so it should be public.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
If the MMIO hole is large and hvmloader needs to relocate memory to
immediately above the 4 GiB boundary, the e820 presented to the guest
will not have a RAM region above 4 GiB.
e.g., a guest with 3 GiB of memory and a 2 GiB MMIO hole will only see
2 GiB.
The required e820 memory region above 4 GiB needs to be added, and not
just filled in.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Coverity points out that using LOG macro dereferences gc which is NULL
at that point. Use LIBXL__LOG instead.
CID: 1343291
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Commit 5644ce0142 ("x86emul: relax asm() constraints") introduced a
64-bit only instruction suffix, which breaks running the emulator test
on a 32-bit system. Mirror __OS (and _OP for completeness) to the test
wrapper source file.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Some versions of GCC complain that the 'firmware' variable can be used
uninitialized. It looks like the switch inside of the else case is just
confusing GCC.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The hunk that prints out error message can only be reached from the loop
that allocates memory. Move that hunk into the loop to avoid confusing
gcc.
Reported-by: Doug Goldstein <cardoe@cardoe.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>
Acked-by: Roger Pau Monne <roger.pau@citrix.com>
The foreign header generation blindly replaces 'uint64_t' with '__align8__
uint64_t', to get correct alignment when built as 32bit. This is correct in
most circumstances, but Clang objects to two specific uses.
* Inside a sizeof() expression
* As part of a typecast
An example error looks like:
/local/xen.git/tools/libxc/../../tools/include/xen/foreign/x86_64.h:204:44:
error: 'aligned' attribute ignored when parsing type [-Werror,-Wignored-attributes]
__align8__ uint64_t evtchn_mask[sizeof(__align8__ uint64_t) * 8];
^~~~~~~~~~
/local/xen.git/tools/libxc/../../tools/include/xen/foreign/x86_64.h:13:36:
note: expanded from macro '__align8__'
^~~~~~~~~~~
This sedary is sufficient to fix all the bad examples without touching any of
the legitimate uses, and is more simple than teaching mkheader.py how to parse
C.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
hvmloader is unhosted, and shouldn't use the system errno.h. It already has
to use Xen's errno.h for other hypercalls. The use of public/io/xs_wire.h
requires the use of un-prefixed errno values.
This fixes the build on stricter toolchains where requesting -fno-builtin does
reduce the include path as much as it can.
Reported-by: Doug Goldstein <cardoe@cardoe.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
to include the vcpu IDs, in a way that matches
how the "dom:vcpu" couple is displayed in other
events (runstate changes).
Also add the trace for TRC_SCHED_SHUTDOWN_CODE which
was missing and was done via SCHEDOP_shutdown_code hypercall.
(TRC_SCHED_SHUTDOWN trace was present).
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
Cc: George Dunlap <george.dunlap@eu.citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Olaf Hering <olaf@aepfle.de>
---
Changes from v1:
* enhanced changelog, as suggested during review.
AC_CHECK_LIB() was running gcc -Llib -lm -lutils conftest.c which on
platforms that do as needed operations by default will result in
underlinking. Instead AC_CHECK_LIB() suggests supplying the extra
libraries necessary in a 5th argument.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
If hardware support memory protect externsion, expose this feature
to guest by default. Users don't have to use a 'cpuid= ' option in
config file to turn it on.
Signed-off-by: Liang Li <liang.z.li@intel.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
bcc/ld86/as86 are necessary when we build ROMBIOS. However if we do not
build it (and are not building qemu-trad), the build requirements are
overly strict and can lead to failures.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Coverity rightly points out that qmp->buffer may not be NULL
terminated when passed to strncat.
Make the actual buffer a byte bigger than QMP_RECEIVE_BUFFER_SIZE and
always append a NULL byte.
I suspect that in practice we have not yet seen QMP messages
approaching the buffer size (4K).
Compile tested only.
CID: 1055989
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The contents of /proc/uptime is typically something like "80164.57
640617.58", so the existing 512 byte buffer is more than large enoguh,
so reduce its effective size to 511 bytes and ensure we include a
NULL.
Otherwise Coverity points out that we pass a potentially unterminated
string to strtok. In practice this likely doesn't actually cause
issues (at least on Linux) because the
string should always contain a space so we will stop parsing.
CID: 105590
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Dom0 is handled separately (via print_dom0_uptime) and the domU
variant doesn't work for dom0 since libxl_vm_get_start_time() doesn't.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
We assert that nullfd if not std{in,out,err} since that would result
in closing one of the just dup2'd fds. For this to happen
std{in,out,err} would have needed to be closed, at which point all
sorts of other things could go wrong.
CID: 1130519
It was previously hypothesised[0] that fixing 1130516 would solve this
too, but that appears to not have been the case.
Compile tested only.
[0] http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02931.html
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: andrew.cooper3@citrix.com
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
find_vec() is supposed to find the vector in the list if it exists,
choose an empty slot if it doesn't exist, and return null if all slots
are full.
However, coverity noticed that although the callers of find_vec() handle
the last condition, find_vec() itself didn't.
Check to see if we actually found an empty slot before attempting to
initialize it.
CID 1306864
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Coverity notices that if !head is true, that !N can never be true.
Don't bother checking N, since we know it has to be at least one.
CID 1354243
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Commit 7449fb36 ("migration/save: pass checkpointed_stream from libxl to
libxc ") failed to initialise rc in the error path.
Reported-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Jonathan Davies
Doug Goldstein
Konrad Rzeszutek Wilk
Paul Durrant
Chunyan Liu
Wen Congyang
Razvan Cojocaru
David Vrabel
Wei Liu
Jan Beulich
Andrew Cooper
Dario Faggioli
Liang Li
Ian Campbell
George Dunlap