Introduce a limit on the number of outstanding reply packets in the xenbus
queue. This limits the number of in-flight requests: when the output queue is
full we'll stop processing inputs until the output queue has room again.
To avoid a busy loop on the Unix socket we only add it to the watched input
file descriptor set if we'd be able to call `input` on it. Even though Dom0
is trusted and exempt from quotas a flood of events might cause a backlog
where events are produced faster than daemons in Dom0 can consume them, which
could lead to an unbounded queue size and OOM.
Therefore the xenbus queue limit must apply to all connections, Dom0 is not
exempt from it, although if everything works correctly it will eventually
catch up.
This prevents a malicious guest from sending more commands while it has
outstanding watch events or command replies in its input ring. However if it
can cause the generation of watch events by other means (e.g. by Dom0, or
another cooperative guest) and stop reading its own ring then watch events
would've queued up without limit.
The xenstore protocol doesn't have a back-pressure mechanism, and doesn't
allow dropping watch events. In fact, dropping watch events is known to break
some pieces of normal functionality. This leaves little choice to safely
implement the xenstore protocol without exposing the xenstore daemon to
out-of-memory attacks.
Implement the fix as pipes with bounded buffers:
* Use a bounded buffer for watch events
* The watch structure will have a bounded receiving pipe of watch events
* The source will have an "overflow" pipe of pending watch events it couldn't
deliver
Items are queued up on one end and are sent as far along the pipe as possible:
source domain -> watch -> xenbus of target -> xenstore ring/socket of target
If the pipe is "full" at any point then back-pressure is applied and we prevent
more items from being queued up. For the source domain this means that we'll
stop accepting new commands as long as its pipe buffer is not empty.
Before we try to enqueue an item we first check whether it is possible to send
it further down the pipe, by attempting to recursively flush the pipes. This
ensures that we retain the order of events as much as possible.
We might break causality of watch events if the target domain's queue is full
and we need to start using the watch's queue. This is a breaking change in
the xenstore protocol, but only for domains which are not processing their
incoming ring as expected.
When a watch is deleted its entire pending queue is dropped (no code is needed
for that, because it is part of the 'watch' type).
There is a cache of watches that have pending events that we attempt to flush
at every cycle if possible.
Introduce 3 limits here:
* quota-maxwatchevents on watch event destination: when this is hit the
source will not be allowed to queue up more watch events.
* quota-maxoustanding which is the number of responses not read from the ring:
once exceeded, no more inputs are processed until all outstanding replies
are consumed by the client.
* overflow queue on the watch event source: all watches that cannot be stored
on destination are queued up here, a single command can trigger multiple
watches (e.g. due to recursion).
The overflow queue currently doesn't have an upper bound, it is difficult to
accurately calculate one as it depends on whether you are Dom0 and how many
watches each path has registered and how many watch events you can trigger
with a single command (e.g. a commit). However these events were already
using memory, this just moves them elsewhere, and as long as we correctly
block a domain it shouldn't result in unbounded memory usage.
Note that Dom0 is not excluded from these checks, it is important that Dom0 is
especially not excluded when it is the source, since there are many ways in
which a guest could trigger Dom0 to send it watch events.
This should protect against malicious frontends as long as the backend follows
the PV xenstore protocol and only exposes paths needed by the frontend, and
changes those paths at most once as a reaction to guest events, or protocol
state.
The queue limits are per watch, and per domain-pair, so even if one
communication channel would be "blocked", others would keep working, and the
domain itself won't get blocked as long as it doesn't overflow the queue of
watch events.
Similarly a malicious backend could cause the frontend to get blocked, but
this watch queue protects the frontend as well as long as it follows the PV
protocol. (Although note that protection against malicious backends is only a
best effort at the moment)
This is part of XSA-326 / CVE-2022-42318.
Signed-off-by: Edwin Török <edvin.torok@citrix.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
The queue here would only ever hold at most one element. This will simplify
follow-up patches.
This is part of XSA-326.
Signed-off-by: Edwin Török <edvin.torok@citrix.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
paths.ml contains various paths known to configure, and currently is generated
via a Makefile rule. Simplify this and generate it through configure, similar
to how oxenstored.conf is generated from oxenstored.conf.in.
This will allow to reuse the generated file more easily with Dune.
No functional change.
Signed-off-by: Edwin Török <edvin.torok@citrix.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
They are two competiting spelling for the variable holding the path to
"tools/ocaml", $(TOPLEVEL) and $(OCAML_TOPLEVEL). The "Makefile.rules"
which is included in all ocaml Makefiles have one rule which make use
of that variable which is then sometime unset. When building
"ocaml/xenstored", make isn't capable of generating ".ocamldep.make"
because $(TOPLEVEL) isn't defined in this case.
This can fail with an error like this when paths.ml have been
regenerated:
Error: Files define.cmx and paths.cmx
make inconsistent assumptions over interface Paths
This patch fix ".ocamldep.make" rule by always spelling the variable
$(OCAML_TOPLEVEL).
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
The type of Xsraw.sync is made more precise:
from val sync : (Xenbus.Xb.t -> 'a) -> con -> string
to val sync : (Xenbus.Xb.t -> unit) -> con -> string
The first argument is enforced to return unit rather than a value that
is not used anyway.
[ No functional change. -iwj ]
Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Since 9c89dc9520 libxenstore prefers using
/dev/xen/xenbus over /proc/xen/xenbus. This makes the OCaml xenstore
library contain the same preference.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Acked-by: David Scott <dave.scott@citrix.com>
configure checks just for ocamlc, but the tools in tools/ocaml depend
also on ocamlfind. On my workstation I have just ocamlc installed, but
no ocamlfind. As a result make will fail.
Update configure.ac to check also for OCAMLFIND, update various
Makefiles and replace hardcoded ocamlfind string with $(OCAMLFIND)
Please rerun autogen.sh after applying this patch.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Campbell <ian.campbell@citrix.com>
ocamlfind does not support namespaces, so to avoid
name clashes the module names have become longer.
Additionally, the xenstore and xenbus subdirs, which
contain several modules each, have been packed into
toplevel Xenstore and Xenbus modules.
xb becomes xenbus, xc becomes xenctrl, xl becomes xenlight,
xs becomes xenstore, eventchn becomes xeneventchn and
mmap becomes xenmmap.
[ Patch modified from that submitted, to update the .hgignore, and to
cope with intervening changes to mmap_stubs.c -iwj ]
Signed-off-by: Jon Ludlam <jonathan.ludlam@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Otherwise make can search the path relative to certain standard paths
such as /usr/include (e.g., the line '-include $(XEN_ROOT)/.config' in
Config.mk suffers from this).
Signed-off-by: Keir Fraser <keir@xen.org>
Previously the install target was having no effect because it ended up
calling the default target in the subdir Makefile instead of the
install target.
Resolve this by tying the tools/ocaml Makefiles into the generic
handling done by tools/Rules.mk.
Other changes arising in one way or another from this:
- Add libs/xl/META.in
- Update .hgignore for META files
- Create leading directories
- Remove existing module before installation in install targer
(worksaround what appears to be a quirk of "ocamlfind install")
- Use the globally defined $(DESTDIR)
- Move "ocamlfind printfconf destdir" to a common variable,
repurposing exising unused OCAMLDESTDIR, incorporating $(DESTDIR) at
the same time.
- Drop a few unused variabe definitions (mainly to avoid deciding if
$(DESTDIR) made sense for them or not.
- Pass -destdir to ocamlfind in uninstall target for symmetry with
install target.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Edwin Török
Edwin Török
Anthony PERARD
Christian Lindig
Andrew Cooper
Wei Liu
Doug Goldstein
Olaf Hering
Jon Ludlam
Ian Campbell
Keir Fraser
Keir Fraser