In Quota.merge, we merge two quota hashtables, orig_quota and mod_quota, putting
the results into dest_quota. These hashtables map domids to the number of
entries currently owned by that domain.
When mod_quota contains an entry for a domid that was not present in orig_quota
(or dest_quota), the call to get_entry caused Quota.merge to raise a Not_found
exception. This propagates back to the client as an ENOENT error, which is not
an appropriate return value from some operations, such as transaction_end.
This situation can arise when a transaction that introduces a domain (hence
calling Quota.add_entry) needs to be coalesced due to concurrent xenstore
activity.
This patch handles the merge in the case where mod_quota contains an entry not
present in orig_quota (or in dest_quota) by treating that hashtable as having
existing value 0.
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
If systemd is configured for use AND you are building oxenstored, the C
systemd library must be linked in to the systemd.cxma library.
Signed-off-by: Jonathan Creekmore <jonathan.creekmore@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Instead switch to an explicit -include $(XEN_ROOT)/tools/config.h to
pickup config.h.
Most places already do this, fixup the rest.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
The statement to reset nb_watches should be in del_watches, not
del_transactions.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: David Scott <dave.scott@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
[ ijc -- fix syntax error by adding a ";" to the previous line in the
new location and removing from the previous line in the old ]
Hvmloader uses the xenstore ring and then tries to reset it back
to its initial state before booting the guest. Occasionally xenstored
will read the ring while it is being zeroed and conclude it has
been corrupted. This prevents PV drivers from loading in the guest.
This patch updates the xenstore ring protocol definition, enabling
a server to advertise additional features to the guest. One such feature
is defined: the ability to cleanly reset the ring including the
higher-level protocol, like an enhanced RESET_WATCHES for rings.
This patch implements the ring reconnection features in oxenstored
and hvmloader, fixing the bug.
This patch also defines an 'invalid' xenstore packet type and uses this
to poison the ring over a reconnect. This will make diagnosing this
bug much easier in future.
Signed-off-by: David Scott <dave.scott@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
We noticed that, if configured to use syslog as the logging backend, every
single line of access logging (via the syslog C binding) will call stat on
/etc/localtime for 3 times. The rational behind this is probably to detect any
timezone changes over time.
This is a considerable cost we'd like to avoid, given the intensiveness of our
access logging --- we log almost every xenstore status change (for good
reason). Also a running Xen host is rarely a mobile environment, so the little
benefit can hardly justify the cost.
Setting up the TZ environment varialbe can avoid stat calls.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
In previous commit, we moved from exhaustively scanning all domain connections
to only processing those have correctly notified us by events. The benefits are
not only efficiency but also correctness, because it could potentially block an
ill-behaved client and have it waiting on its own mistake. If someone makes a
mistake on this when developing a piece of code, he/she would immediately
notice the problem (as the process being blocked), so that he/she could fix it
rightaway before anything else. Note that the chances of making such mistakes
are rare in reality, because most client code would use the libxenstore library
(which has all the notification logic built in correctly) instead of having to
implement raw accessing from scratch.
On the other hand, we did notice that there were some legacy code that didn't do
the notification correctly. As some code might be still running in wild, it
would be bad if they break by this change (e.g. after an upgrade). This patch
introduces a safe net mechanism to ensure ill-behaved clients continue to work,
but still retain most of the performance benefits here.
* We add a checker to still scan all the rings periodically, so that we can
still pick up these messages at an acceptable frequency.
* Internally, we introduce an io_credit concept for domain connections. It
represents the rounds of ring scan we are going to perform on a domain
connection. For well-behaved connections, this value is changing between 0
and 1; but for connections detected as ill-behaved, we'll bump its credit
to a high value so that we'll unconditionally scan its ring for the next
$n$ rounds. This way, the client won't hiccupped by the interval between
checker's running (especially during periods when it continously interacts
with oxenstored); and oxenstored doesn't have to keep scanning these
rings indefinitely (with the credit running out), as they are usually quite
most of the time.
* We log an message when a domain connection is suspected as ill-behaved.
Enable [info] level logging if you want/need to see it in action. Note that
this information won't be accurate, as false positives are possible due to
time window (e.g. we detect a client has written to the ring and we get no
notificiation from it for the time being, but still the notification could
potentially arrive at some time later). It's no harm to give a domain
connection extra credit though.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently, upon receiving an event, oxenstored will always scan/process all
the domain connections (xs rings), disregarding which domain sent that event.
This is rather costy and inefficient. It also shadows and indulges client
for not correctly communicating with us on message/space availability.
With this patch, oxenstore will only scan/process the domain connections
that have correctly notified us by events or have IO actions leftover from
previous communication.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently in xenstore connection database, we use a hash table of
(domid -> connection) to store domain connections. This allows fast indexing
based on dom ids.
This patch adds another dimention of fast indexing that is based on eventchn
port number. This is useful when doing selective connection processing
based on the port numbers of incoming events.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently we use list to store socket connections. This is fine for smaller
number of connections. But when we scale up, traveling through a list of
hundreds or thousands of connections just to find a single one of them is very
low efficient.
This patch replaces the list with a (Unix.file_descr -> Connection.t) hash table.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
The function process_fdset_with is called on the read set connections first.
During the process, it might destroy a connection and remove it from the
connections database if some errors occur. However, a reference to the same
connection might still exist in the write set, which is awaiting to be
processed next. In this case, a Not_found error will be raised and the process
is aborted.
This patch changes the logic to ignore connections just missing from the
connection database and continue the rest part of the work.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
This allows to fall back to the original Unix.select if preferred. It could be
useful for debugging purposes too.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
To go beyond 1024 fds, we also need to raise the process limitation on max
open fds (usually defaults to 1024).
We need to know the system level max open fds so that we won't go above that.
Simply setting the limit to RLIM_INFINITY doesn't work on Linux 3.x (EPERM), a
patch on this went into the 2.x branch but not 3.x for some reason.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
Currently, oxenstored uses Unix.select underneath, so it doesn't work properly
if given a FD number >= 1024. This is a scalability bottleneck for hosts
running large number of VMs.
To remove this limitation, we implemented a poll-based mechanism but with the
same type signature as the Unix.select currently in use. So these two functions
can be interchangeable at any stage.
Signed-off-by: Zheng Li <dev@zheng.li>
Reviewed-by: David Scott <dave.scott@citrix.com>
This adds the systemd xen service / module files and integrates
support into the build system.
This goes in with AX_AVAILABLE_SYSTEMD() which will enable
systemd if development libraries have been found on your
build system. If you don't have systemd on target systems
for binaries built with systemd then the binary will not
work, you must explicitly disable systemd support if you
do not want to build systemd support.
When systemd libraries are present only systems that
have booted into systemd go through the systemd initialization,
otherwise the SysVinit is used.
These are originally based on the Fedora systemd files.
Changes made from Fedora's systemd files:
* split sockets into two files to claim different permissions
* Use /bin/sh -c exec to for a simple launcher implementation
* enables systemd socket activation for C xenstored and Ocaml
oxenstored
* use sd_notify(), so change the service to Type=notify, because of
this we remove the PIDFile specification as we don't care for it, and let
systemd do its magic for us, this also means we don't have to fork
so we use --no-fork with systemd
* defines a modules-load.d, its original source file will be shared
between systemd and old init systems
* simplify service files with ConditionVirtualization=xen which uses
the built in systemd virtualization backend detection, these
service files will not be available to start on systems that do not
boot with xen as a hypervisor
* use autoconf to replace @variable@ paths for us which piggy
backs on top of the latest autoconf changes to xen
* removes oxenstored service file in favor of a system variable which
controls which which xentored to use at run time, we avoid multiple
service files this way.
* simplifies startup to not require polling on the sockets
as initial socket management is handled by systemd, we just
take on the socket later once anything pokes at it, a simple nc -U
(as root) on any of the sockets files can activate the service for example.
Anything queued up will be sent to us once we start. Socket activation
should in theory also let us dynamically switch between xenstores but more
importantly we could upgrade xenstored while keeping all active
socket communication queued up, but in order to take advantage of
this we eventually would need to remove the requirement of not being
able to bring down the xenstored. Even though active sockets are
supported since most libxl communication doesn't triggger a check
on the unix socket first administrators are encouraged to enable
the xenstored.service to triggger an initialization of the xenstored
upon bring up. Some systems also never use unix sockets for
communication with the xenstored and as such active sockets will
not be used there.
* allow for xenstored configuration through *either* of these
configuration files:
- /etc/sysconfig/xenstored
- /etc/default/xenstored
The /etc/default/xenstored will let debian based systems do
the same, while SUSE/OpenSUSE/Fedora/RedHat can keep on chugging
with sysconfig. We leave these files all commented out by default
though given that for systemd we want to encourage not using them.
* ensures we create the run directory as most systems will likely
be using a tmpfs for run dirs for the pid files
* Some systems define the selinux context in the systemd Option for the
/var/lib/xenstored tmpfs:
Options=mode=755,context="system_u:object_r:xenstored_var_lib_t:s0"
For the upstream version we remove that and let systems specify the
context on their system /etc/default/xenstored or /etc/sysconfig/xenstored
$XENSTORED_MOUNT_CTX variable, with a default to none.
* takes advantage of the shared xendomains helper for the xendomains
service
* Add the new dom0 that gets kicked off for disk backend access into
its own systemd service associated to xen
We end up with these systemd files:
General requirements:
* proc-xen.mount
* var-lib-xenstored.mount
xenstored:
* xenstored.service
* xenstored.socket
* xenstored_ro.socket
* xenconsoled.service
* xen-qemu-dom0-disk-backend.service.in
Optional:
* xendomains.service
* xen-watchdog.service
As for integration with xen, we house keep all the systemd files
under a new directory tools/hotplug/Linux/systemd/ and will be targeted
by default when building on Linux systems if systemd development
libraries are present at build time.
The systemd files will be sanitized for meta @VARIABLES@ upon
configuration and installed upon the install target. Systems that
do not use systemd can still get systemd service unit files installed
if the build system enabled systemd support, this however does not
mandate a requirement of having systemd libraries present. Old init
scripts are always installed.
If you don't specify a prefix you will end up with the services
files under /usr/local/lib/systemd/system/ by default, and systemd
modules-load.d conf files under /usr/local/lib/modules-load.d/ which
systemd does look for (although it seems this is not documented).
Distributions are expected to provide their /usr/ prefix to end up in
the more generic location upon distribution install at
/usr/lib/systemd/system/ and /usr/lib/modules-load.d/ respectively.
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Jan Rękorajski <baggins@pld-linux.org>
Cc: M A Young <m.a.young@durham.ac.uk>
Cc: Jacek Konieczny <jajcus@jajcus.net>
Cc: xen-devel@lists.xenproject.org
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
[ ijc -- ran autogen.sh ]
This adds systemd socket activation support for the Ocaml xenstored.
Ocaml lacks systemd library support so we provide our own C helpers
as is done with other functionality lacking on Ocaml.
Active sockets enables oxenstored to be loaded only if required by a system
onto which Xen is installed on. Socket activation is handled by
systemd, once a port for a service which claims a socket is used
systemd will start the required services for it, on demand. For more
details on socket activation refer to Lennart's socket-activation
post regarding this [0].
An important difference with socket activation is that systemd will set
FD_CLOEXEC for us on the socket before giving it to us, we'll sprinkly
the Unix.set_close_on_exec for LSB init next as a separate commit.
Right now this code adds a no-op for this functionality, leaving the
enablement to be done later once systemd is properly hooked into
the build system. The socket activation is ordered in aligment with
the socket activation order passed on to systemd.
[0] http://0pointer.de/blog/projects/socket-activation2.html
Cc: David Scott <dave.scott@eu.citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Vincent Hanquez <Vincent.Hanquez@eu.citrix.com>
Acked-by: Dave Scott <Dave.Scott@citrix.com>
Acked-by: Anil Madhavapeddy <anil@recoil.org>
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
This will be used later for dynamic configuration paths on C code.
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
At a beginning of a transaction, the quotas from the global store
are duplicated and modified by the transaction. If during the
transaction, an action associated to no transaction is concurrently
executed, the quotas of the global store are updated, and then the
updates are lost when the transaction merges.
We fix this problem by keeping another copy of the quota at the
beginning of the transaction, and performing a 3-way merge between
the quotas from the transaction and the "original" copy of the quota
onto the quota of the global store.
Signed-off-by: Jerome Maloberti <jerome.maloberti@citrix.com>
Signed-off-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
If a domU has exhausted its quota we still want the toolstack in dom0 to
be able to create new nodes in places like
/local/domain/%d/control/shutdown
Without this patch, a domU which has exhausted its quota can only be
powered off, which is not as good as being able to request a clean
shutdown.
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Signed-off-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
Allow a domain updating existing xenstore keys even if it has already reached
its max entries limit
As updating existing key won't increase the number of entries belonging to a
domain, we should avoid checking the max entries limit prematurely. The patch
addresses this issue in the following functions: write/add, mkdir, setperms.
Signed-off-by: Zheng Li <zheng.li@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
This fixes the issue for both C and ocaml xenstored, however only the ocaml
xenstored is vulnerable in its default configuration.
Adding a new error appears to be safe, since bit libxenstore and the Linux
driver at least treat an unknown error code as EINVAL.
This is XSA-72 / CVE-2013-4416.
Original ocaml patch by Jerome Maloberti <jerome.maloberti@citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
add check logic when read from IO ring, and if error happens,
then mark the reading connection as "bad", Unless vm reboot,
oxenstored will not handle message from this connection any more.
xs_ring_stubs.c: add a more strict check on ring reading
connection.ml, domain.ml: add getter and setter for bad flag
process.ml: if exception raised when reading from domain's ring,
mark this domain as "bad"
xenstored.ml: if a domain is marked as "bad", do not handle it.
Signed-off-by: John Liu <john.liuqiming@huawei.com>
Acked-by: David Scott <dave.scott@eu.citrix.com>
It's a common OCaml convention to add a 'type t' in a module to
represent the main "thing" that the module is about. We add an
opaque type t and to_int/of_int functions for those who really
need it, in particular:
1. to_int is needed for debug logging; and
2. both to_int and of_int are needed for anyone who communicates
a port number through xenstore.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
We now log different kinds of events at different levels. The convention
is now:
new/end_connection: Debug
coalesce: Debug
conflict: Debug
commit: Debug
regular ops: Info
start/end_transaction: Debug
error (ENOENT): Debug
error (any other): Warn
watch: Info
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
To log to files the config file should contain:
xenstored-log-file = /var/log/xenstored.log
access-log-file = /var/log/xenstored-access.log
(These two files are still the built-in defaults. The log format is
unchanged.)
To log to syslog the config file should contain:
xenstored-log-file = syslog:<facility>
access-log-file = syslog:<facility>
where <facility> is the syslog facility to use (e.g. 'daemon' 'local2')
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
This was lost in the OCaml xenstored log merge of 10/Oct/2011.
The binding isn't exported as a shared "log" library, instead it is kept
local to xenstored.
Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Otherwise at the default level we rarely log anything at all.
A completely empty log file is a good sign, but only if you know you are
looking in the right place...
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
First though:
- Move it to /etc/xen/oxenstored.conf.
- Use /var/run/xenstored.pid as default pid file
- Disable test-eagain "Randomly failed a transaction with EAGAIN. Used for
testing Xs user". Doesn't sound fun by default...
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Previous an unknown operation would be decoded as a Not_found exception which
would bubble all the way up to the try ... with surrounding the call to
main_loop where it would be logged and ignored.
This would leave the guest hanging waiting for a response to the invalid
request.
Instead introduce a specific "Invalid" operation. Higher level functionality,
such as Process.process_packet, already handles operations which are not
understood with an error reply due to the final wildcard entry in
Process.function_of_type but explicitly handle Invalid this way to make it
clear what is going on.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
The only user was oxenstored, which has had the relevant bits
merged in.
Signed-off-by: Zheng Li <zheng.li@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Acked-by: Jon Ludlam <jonathan.ludlam@eu.citrix.com>
The library was only minimally used, and was really rather redundant.
Signed-off-by: Zheng Li <zheng.li@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Acked-by: Jon Ludlam <jonathan.ludlam@eu.citrix.com>
ocamlfind does not support namespaces, so to avoid
name clashes the module names have become longer.
Additionally, the xenstore and xenbus subdirs, which
contain several modules each, have been packed into
toplevel Xenstore and Xenbus modules.
xb becomes xenbus, xc becomes xenctrl, xl becomes xenlight,
xs becomes xenstore, eventchn becomes xeneventchn and
mmap becomes xenmmap.
[ Patch modified from that submitted, to update the .hgignore, and to
cope with intervening changes to mmap_stubs.c -iwj ]
Signed-off-by: Jon Ludlam <jonathan.ludlam@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Otherwise make can search the path relative to certain standard paths
such as /usr/include (e.g., the line '-include $(XEN_ROOT)/.config' in
Config.mk suffers from this).
Signed-off-by: Keir Fraser <keir@xen.org>
Now that tools/libxc is licensed under LGPL I don't think there is any need for
an LGPL reimplementation under tools/ocaml.
For the most part the conversion to the up-to-date libxc API (xc_lib.c
essentially implemented the same interface as an older libxc) was pretty
automatic. There are some functions which appear to no longer exist in libxc
which I therefore simply removed the bindings for and a small number of
interfaces which had changed.
Many of the functions bound by the stubs have no in-tree users (which I think
is fine for a language binding) so I have no way to confirm correctness other
than by eye. I was however able to confirm that oxenstored still worked and to
build a XCP toolstack which could successfully start a PV guest.
Uses the new XC_OPENFLAG_NON_REENTRANT option to avoid potential conflicts
between pthreads and the ocaml runtime.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Vincent Hanquez <Vincent.Hanquez@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Rename Evtchn.bind_virq as Evtchn.bind_dom_exc_virq
to reflect its actual behaviour.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Previously the install target was having no effect because it ended up
calling the default target in the subdir Makefile instead of the
install target.
Resolve this by tying the tools/ocaml Makefiles into the generic
handling done by tools/Rules.mk.
Other changes arising in one way or another from this:
- Add libs/xl/META.in
- Update .hgignore for META files
- Create leading directories
- Remove existing module before installation in install targer
(worksaround what appears to be a quirk of "ocamlfind install")
- Use the globally defined $(DESTDIR)
- Move "ocamlfind printfconf destdir" to a common variable,
repurposing exising unused OCAMLDESTDIR, incorporating $(DESTDIR) at
the same time.
- Drop a few unused variabe definitions (mainly to avoid deciding if
$(DESTDIR) made sense for them or not.
- Pass -destdir to ocamlfind in uninstall target for symmetry with
install target.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Jonathan Davies
Wei Liu
Jonathan Creekmore
Ian Campbell
Olaf Hering
Andrew Cooper
David Scott
Zheng Li
Luis R. Rodriguez
Jerome Maloberti
Vincent Bernardoff
Zheng Li
Ian Jackson
=John Liu
David Scott
Jon Ludlam
Keir Fraser
Vincent Hanquez
Keir Fraser