ml_interface_{read,write}() would miscalculate the quantity of
data/space in the ring if it crossed the ring boundary, and incorrectly
return a short read/write.
This causes a protocol stall, as either side of the ring ends up waiting
for what they believe to be the other side needing to take the next
action.
Correct the calculations to cope with crossing the ring boundary.
In addition, correct the error detection. It is a hard error if the
producer index gets more than a ring size ahead of the consumer, or if
the consumer ever overtakes the producer.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Reviewed-by: David Scott <dave@recoil.org>
(cherry picked from commit 8a2c11f876)
(cherry picked from commit 6150df9f3f)
In Quota.merge, we merge two quota hashtables, orig_quota and mod_quota, putting
the results into dest_quota. These hashtables map domids to the number of
entries currently owned by that domain.
When mod_quota contains an entry for a domid that was not present in orig_quota
(or dest_quota), the call to get_entry caused Quota.merge to raise a Not_found
exception. This propagates back to the client as an ENOENT error, which is not
an appropriate return value from some operations, such as transaction_end.
This situation can arise when a transaction that introduces a domain (hence
calling Quota.add_entry) needs to be coalesced due to concurrent xenstore
activity.
This patch handles the merge in the case where mod_quota contains an entry not
present in orig_quota (or in dest_quota) by treating that hashtable as having
existing value 0.
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
(cherry picked from commit 82ff7cbed6)
(cherry picked from commit ba391da2e7)
The Ocaml stub to retrieve the hypervisor console ring had a few problems.
* A single 32k buffer would truncate a large console ring.
* The buffer was static and not under the protection of the Ocaml GC lock so
could be clobbered by concurrent accesses.
* Embedded NUL characters would cause caml_copy_string() (which is strlen()
based) to truncate the buffer.
The function is rewritten from scratch, using the same algorithm as the python
stubs, but uses the protection of the Ocaml GC lock to maintain a static
running total of the ring size, to avoid redundant realloc()ing in future
calls.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Dave Scott <dave.scott@eu.citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
(cherry picked from commit 1a010ca99e)
Hvmloader uses the xenstore ring and then tries to reset it back
to its initial state before booting the guest. Occasionally xenstored
will read the ring while it is being zeroed and conclude it has
been corrupted. This prevents PV drivers from loading in the guest.
This patch updates the xenstore ring protocol definition, enabling
a server to advertise additional features to the guest. One such feature
is defined: the ability to cleanly reset the ring including the
higher-level protocol, like an enhanced RESET_WATCHES for rings.
This patch implements the ring reconnection features in oxenstored
and hvmloader, fixing the bug.
This patch also defines an 'invalid' xenstore packet type and uses this
to poison the ring over a reconnect. This will make diagnosing this
bug much easier in future.
Signed-off-by: David Scott <dave.scott@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
We noticed that, if configured to use syslog as the logging backend, every
single line of access logging (via the syslog C binding) will call stat on
/etc/localtime for 3 times. The rational behind this is probably to detect any
timezone changes over time.
This is a considerable cost we'd like to avoid, given the intensiveness of our
access logging --- we log almost every xenstore status change (for good
reason). Also a running Xen host is rarely a mobile environment, so the little
benefit can hardly justify the cost.
Setting up the TZ environment varialbe can avoid stat calls.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
In previous commit, we moved from exhaustively scanning all domain connections
to only processing those have correctly notified us by events. The benefits are
not only efficiency but also correctness, because it could potentially block an
ill-behaved client and have it waiting on its own mistake. If someone makes a
mistake on this when developing a piece of code, he/she would immediately
notice the problem (as the process being blocked), so that he/she could fix it
rightaway before anything else. Note that the chances of making such mistakes
are rare in reality, because most client code would use the libxenstore library
(which has all the notification logic built in correctly) instead of having to
implement raw accessing from scratch.
On the other hand, we did notice that there were some legacy code that didn't do
the notification correctly. As some code might be still running in wild, it
would be bad if they break by this change (e.g. after an upgrade). This patch
introduces a safe net mechanism to ensure ill-behaved clients continue to work,
but still retain most of the performance benefits here.
* We add a checker to still scan all the rings periodically, so that we can
still pick up these messages at an acceptable frequency.
* Internally, we introduce an io_credit concept for domain connections. It
represents the rounds of ring scan we are going to perform on a domain
connection. For well-behaved connections, this value is changing between 0
and 1; but for connections detected as ill-behaved, we'll bump its credit
to a high value so that we'll unconditionally scan its ring for the next
$n$ rounds. This way, the client won't hiccupped by the interval between
checker's running (especially during periods when it continously interacts
with oxenstored); and oxenstored doesn't have to keep scanning these
rings indefinitely (with the credit running out), as they are usually quite
most of the time.
* We log an message when a domain connection is suspected as ill-behaved.
Enable [info] level logging if you want/need to see it in action. Note that
this information won't be accurate, as false positives are possible due to
time window (e.g. we detect a client has written to the ring and we get no
notificiation from it for the time being, but still the notification could
potentially arrive at some time later). It's no harm to give a domain
connection extra credit though.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently, upon receiving an event, oxenstored will always scan/process all
the domain connections (xs rings), disregarding which domain sent that event.
This is rather costy and inefficient. It also shadows and indulges client
for not correctly communicating with us on message/space availability.
With this patch, oxenstore will only scan/process the domain connections
that have correctly notified us by events or have IO actions leftover from
previous communication.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently in xenstore connection database, we use a hash table of
(domid -> connection) to store domain connections. This allows fast indexing
based on dom ids.
This patch adds another dimention of fast indexing that is based on eventchn
port number. This is useful when doing selective connection processing
based on the port numbers of incoming events.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently we use list to store socket connections. This is fine for smaller
number of connections. But when we scale up, traveling through a list of
hundreds or thousands of connections just to find a single one of them is very
low efficient.
This patch replaces the list with a (Unix.file_descr -> Connection.t) hash table.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
The function process_fdset_with is called on the read set connections first.
During the process, it might destroy a connection and remove it from the
connections database if some errors occur. However, a reference to the same
connection might still exist in the write set, which is awaiting to be
processed next. In this case, a Not_found error will be raised and the process
is aborted.
This patch changes the logic to ignore connections just missing from the
connection database and continue the rest part of the work.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
This allows to fall back to the original Unix.select if preferred. It could be
useful for debugging purposes too.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
To go beyond 1024 fds, we also need to raise the process limitation on max
open fds (usually defaults to 1024).
We need to know the system level max open fds so that we won't go above that.
Simply setting the limit to RLIM_INFINITY doesn't work on Linux 3.x (EPERM), a
patch on this went into the 2.x branch but not 3.x for some reason.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently, oxenstored uses Unix.select underneath, so it doesn't work properly
if given a FD number >= 1024. This is a scalability bottleneck for hosts
running large number of VMs.
To remove this limitation, we implemented a poll-based mechanism but with the
same type signature as the Unix.select currently in use. So these two functions
can be interchangeable at any stage.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Also moves xc_dom.h to include as it is used often by other xen tools.
Use the new include subdirectories to build Xen tools, qemu-xen and
stubdoms.
Add the old libxc include path to the programs that need it to build,
on a case by case basis and commeting that they shouldn't require
internal libxc headers to build.
[ And: update QEMU_TRADITIONAL_REVISION to corresponding qemu patch
- Ian jackson ]
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
OCaml is replacing its own uint32 type with the ISO C99 uint32_t type
in 4.03.0+dev. Ths patch replaces the remaining uses of uint32 with
uint32_t in the ocaml C code for xen.
Signed-off-by: Michael Young <m.a.young@durham.ac.uk>
Acked-by: David Scott <dave.scott@citrix.com>
This adds the systemd xen service / module files and integrates
support into the build system.
This goes in with AX_AVAILABLE_SYSTEMD() which will enable
systemd if development libraries have been found on your
build system. If you don't have systemd on target systems
for binaries built with systemd then the binary will not
work, you must explicitly disable systemd support if you
do not want to build systemd support.
When systemd libraries are present only systems that
have booted into systemd go through the systemd initialization,
otherwise the SysVinit is used.
These are originally based on the Fedora systemd files.
Changes made from Fedora's systemd files:
* split sockets into two files to claim different permissions
* Use /bin/sh -c exec to for a simple launcher implementation
* enables systemd socket activation for C xenstored and Ocaml
oxenstored
* use sd_notify(), so change the service to Type=notify, because of
this we remove the PIDFile specification as we don't care for it, and let
systemd do its magic for us, this also means we don't have to fork
so we use --no-fork with systemd
* defines a modules-load.d, its original source file will be shared
between systemd and old init systems
* simplify service files with ConditionVirtualization=xen which uses
the built in systemd virtualization backend detection, these
service files will not be available to start on systems that do not
boot with xen as a hypervisor
* use autoconf to replace @variable@ paths for us which piggy
backs on top of the latest autoconf changes to xen
* removes oxenstored service file in favor of a system variable which
controls which which xentored to use at run time, we avoid multiple
service files this way.
* simplifies startup to not require polling on the sockets
as initial socket management is handled by systemd, we just
take on the socket later once anything pokes at it, a simple nc -U
(as root) on any of the sockets files can activate the service for example.
Anything queued up will be sent to us once we start. Socket activation
should in theory also let us dynamically switch between xenstores but more
importantly we could upgrade xenstored while keeping all active
socket communication queued up, but in order to take advantage of
this we eventually would need to remove the requirement of not being
able to bring down the xenstored. Even though active sockets are
supported since most libxl communication doesn't triggger a check
on the unix socket first administrators are encouraged to enable
the xenstored.service to triggger an initialization of the xenstored
upon bring up. Some systems also never use unix sockets for
communication with the xenstored and as such active sockets will
not be used there.
* allow for xenstored configuration through *either* of these
configuration files:
- /etc/sysconfig/xenstored
- /etc/default/xenstored
The /etc/default/xenstored will let debian based systems do
the same, while SUSE/OpenSUSE/Fedora/RedHat can keep on chugging
with sysconfig. We leave these files all commented out by default
though given that for systemd we want to encourage not using them.
* ensures we create the run directory as most systems will likely
be using a tmpfs for run dirs for the pid files
* Some systems define the selinux context in the systemd Option for the
/var/lib/xenstored tmpfs:
Options=mode=755,context="system_u:object_r:xenstored_var_lib_t:s0"
For the upstream version we remove that and let systems specify the
context on their system /etc/default/xenstored or /etc/sysconfig/xenstored
$XENSTORED_MOUNT_CTX variable, with a default to none.
* takes advantage of the shared xendomains helper for the xendomains
service
* Add the new dom0 that gets kicked off for disk backend access into
its own systemd service associated to xen
We end up with these systemd files:
General requirements:
* proc-xen.mount
* var-lib-xenstored.mount
xenstored:
* xenstored.service
* xenstored.socket
* xenstored_ro.socket
* xenconsoled.service
* xen-qemu-dom0-disk-backend.service.in
Optional:
* xendomains.service
* xen-watchdog.service
As for integration with xen, we house keep all the systemd files
under a new directory tools/hotplug/Linux/systemd/ and will be targeted
by default when building on Linux systems if systemd development
libraries are present at build time.
The systemd files will be sanitized for meta @VARIABLES@ upon
configuration and installed upon the install target. Systems that
do not use systemd can still get systemd service unit files installed
if the build system enabled systemd support, this however does not
mandate a requirement of having systemd libraries present. Old init
scripts are always installed.
If you don't specify a prefix you will end up with the services
files under /usr/local/lib/systemd/system/ by default, and systemd
modules-load.d conf files under /usr/local/lib/modules-load.d/ which
systemd does look for (although it seems this is not documented).
Distributions are expected to provide their /usr/ prefix to end up in
the more generic location upon distribution install at
/usr/lib/systemd/system/ and /usr/lib/modules-load.d/ respectively.
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Jan Rękorajski <baggins@pld-linux.org>
Cc: M A Young <m.a.young@durham.ac.uk>
Cc: Jacek Konieczny <jajcus@jajcus.net>
Cc: xen-devel@lists.xenproject.org
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
[ ijc -- ran autogen.sh ]
This adds systemd socket activation support for the Ocaml xenstored.
Ocaml lacks systemd library support so we provide our own C helpers
as is done with other functionality lacking on Ocaml.
Active sockets enables oxenstored to be loaded only if required by a system
onto which Xen is installed on. Socket activation is handled by
systemd, once a port for a service which claims a socket is used
systemd will start the required services for it, on demand. For more
details on socket activation refer to Lennart's socket-activation
post regarding this [0].
An important difference with socket activation is that systemd will set
FD_CLOEXEC for us on the socket before giving it to us, we'll sprinkly
the Unix.set_close_on_exec for LSB init next as a separate commit.
Right now this code adds a no-op for this functionality, leaving the
enablement to be done later once systemd is properly hooked into
the build system. The socket activation is ordered in aligment with
the socket activation order passed on to systemd.
[0] http://0pointer.de/blog/projects/socket-activation2.html
Cc: David Scott <dave.scott@eu.citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Vincent Hanquez <Vincent.Hanquez@eu.citrix.com>
Acked-by: Dave Scott <Dave.Scott@citrix.com>
Acked-by: Anil Madhavapeddy <anil@recoil.org>
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
This will be used later for dynamic configuration paths on C code.
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
At a beginning of a transaction, the quotas from the global store
are duplicated and modified by the transaction. If during the
transaction, an action associated to no transaction is concurrently
executed, the quotas of the global store are updated, and then the
updates are lost when the transaction merges.
We fix this problem by keeping another copy of the quota at the
beginning of the transaction, and performing a 3-way merge between
the quotas from the transaction and the "original" copy of the quota
onto the quota of the global store.
Signed-off-by: Jerome Maloberti <jerome.maloberti@citrix.com>
Signed-off-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
If a domU has exhausted its quota we still want the toolstack in dom0 to
be able to create new nodes in places like
/local/domain/%d/control/shutdown
Without this patch, a domU which has exhausted its quota can only be
powered off, which is not as good as being able to request a clean
shutdown.
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Signed-off-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
Toolstacks may specify a VM generation ID using the u.hvm.ms_vm_genid
field in the libxl_domain_build_info structure, when creating a
domain.
The toolstack is responsible for providing the correct generation ID
according to the Microsoft specification (e.g., generating new random
ones with libxl_ms_vm_genid_generate() as appropriate when restoring).
Although the specification requires that a ACPI Notify event is raised
if the generation ID is changed, the generation ID is never changed
when the domain is in a state to receive such an event (it's either
newly created or suspended).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Cc: David Scott <dave.scott@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
by using the flag and the new cpumap arguments introduced in
the parameters of the DOMCTL_{get,set}_vcpuaffinity hypercalls.
Now, both xc_vcpu_setaffinity() and xc_vcpu_getaffinity() have
a new flag parameter, to specify whether the user wants to
set/get hard affinity, soft affinity or both. They also have
two cpumap parameters instead of only one. This way, it is
possible to set/get both hard and soft affinity at the same
time (and, in case of set, each one to its own value).
In xc_vcpu_setaffinity(), the cpumaps are IN/OUT parameters,
as it is for the corresponding arguments of the
DOMCTL_set_vcpuaffinity hypercall. What Xen puts there is the
hard and soft effective affinity, that is what Xen will actually
use for scheduling.
In-tree callers are also fixed to cope with the new interface.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The various cpuid functions are not available on ARM, so this
makes them raise an OCaml exception. Omitting the functions
completely results in a link failure in oxenstored due to the
missing symbols, so this is preferable to the much bigger patch
that would result from adding conditional compilation into the
OCaml interfaces.
With this patch, oxenstored can successfully start a domain on
Xen/ARM.
Signed-off-by: Anil Madhavapeddy <anil@recoil.org>
Acked-by: David Scott <dave.scott@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
This allows the application to pass a token to libxl in the fd/timeout
registration callbacks, which it receives back in modification or
deregistration callbacks.
It turns out that this is essential for timeout handling, in order to
identify which timeout to change on a modify event.
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
Acked-by: David Scott <dave.scott@eu.citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The original code works fine on 64-bit, but on 32-bit, the OCaml int (which is
1 bit smaller than the C int) is likely to overflow.
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ocaml has a heap lock which must be held whenever ocaml code is running. Ocaml
usually drops this lock when it enters a potentially blocking low-level
function, such as writing to a file. Libxl has its own lock, which it may
acquire when being called.
Things get interesting when libxl calls back into ocaml code. There is a risk
of ending up in a deadlock when a thread holds both locks at the same time,
then temporarily drop the ocaml lock, while another thread calls another libxl
function.
To avoid deadlocks, we drop the ocaml heap lock before entering libxl, and
reacquire it in callbacks to ocaml. This way, the ocaml heap lock is never held
together with the libxl lock, except in osevent registration callbacks, and
xentoollog callbacks. If we guarantee to not call any libxl functions inside
those callbacks, we can avoid deadlocks.
This patch handle the dropping and reacquiring of the ocaml heap lock by the
caml_enter_blocking_section and caml_leave_blocking_section functions, and
related macros. We are also careful to not call any functions that access the
ocaml heap while the ocaml heap lock is dropped. This often involves copying
ocaml values to C before dropping the ocaml lock.
The ao_how in aohow_val is now malloc'ed, just to make this function a little
easier to use.
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
Acked-by: David Scott <dave.scott@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Also:
* Reorganise toplevel OCaml functions into modules of Xenlight.
* Factor out the management of ao_how into the function aohow_val. The ao_how
is now malloc'ed, just to make this function a little easier to use.
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: David Scott <dave.scott@eu.citrix.com>
Having bindings to the low-level functions libxl_osevent_register_hooks and
related, allows to run an event loop in OCaml; either one we write ourselves,
or one that is available elsewhere.
The Lwt cooperative threads library (http://ocsigen.org/lwt/), which is quite
popular these days, has an event loop that can be easily extended to poll any
additional fds that we get from libxl. Lwt provides a "lightweight" threading
model, which does not let you run any other (POSIX) threads in your
application, and therefore excludes an event loop implemented in the C
bindings.
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
Acked-by: David Scott <dave.scott@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Allow a domain updating existing xenstore keys even if it has already reached
its max entries limit
As updating existing key won't increase the number of entries belonging to a
domain, we should avoid checking the max entries limit prematurely. The patch
addresses this issue in the following functions: write/add, mkdir, setperms.
Signed-off-by: Zheng Li <zheng.li@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Libxl functions such as libxl_domain_create_new take large structs
of configuration parameters. Often, we would like to use the default
values for many of these parameters.
The struct and keyed-union types in libxl have init functions, which
fill in the defaults for a given type. This commit provides an OCaml
interface to obtain records of defaults by calling the relevant init
function.
These default records can be used as a base to construct your own
records, and to selectively override parameters where needed.
For example, a Domain_create_info record can now be created as follows:
Xenlight.Domain_create_info.({ default ctx () with
ty = Xenlight.DOMAIN_TYPE_PV;
name = Some vm_name;
uuid = vm_uuid;
})
For types with KeyedUnion fields, such as Domain_build_info, a record
with defaults is obtained by specifying the type key:
Xenlight.Domain_build_info.default ctx ~ty:Xenlight.DOMAIN_TYPE_HVM ()
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
Acked-by: David Scott <dave.scott@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Andrew Cooper
Jonathan Davies
David Scott
Zheng Li
Olaf Hering
Stefano Stabellini
M A Young
Luis R. Rodriguez
Jerome Maloberti
Vincent Bernardoff
David Vrabel
Dario Faggioli
Don Slutz
Anil Madhavapeddy
Rob Hoes
Zheng Li