Commit Graph
286 Commits
Author SHA1 Message Date
Christian Lindig 712082daee tools/oxenstored: Make evaluation order explicit
In Store.path_write(), Path.apply_modify() updates the node_created
reference and both the value of apply_modify() and node_created are
returned by path_write().

At least with OCaml 4.06.1 this leads to the value of node_created being
returned *before* it is updated by apply_modify().  This in turn leads
to the quota for a domain not being updated in Store.write().  Hence, a
guest can create an unlimited number of entries in xenstore.

The fix is to make evaluation order explicit.

This is XSA-272.

Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
Reviewed-by: Rob Hoes <rob.hoes@citrix.com>
(cherry picked from commit 73392c7fd1)
2018-08-14 17:27:26 +01:00
Thomas Sanders e1c62cdf78 oxenstored: trim history in the frequent_ops function
We were trimming the history of commits only at the end of each
transaction (regardless of how it ended).

Therefore if non-transactional writes were being made but no
transactions were being ended, the history would grow
indefinitely. Now we trim the history at regular intervals.

Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders 336afa82ca oxenstored transaction conflicts: improve logging
For information related to transaction conflicts, potentially frequent
logging at "info" priority has been changed to "debug" priority, and
once per two minutes there is an "info" priority summary.

Additional detailed logging has been added at "debug" priority.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders 3ee0d82af2 oxenstored: don't wake to issue no conflict-credit
In the main loop, when choosing the timeout for the select function
call, we were setting it so as to wake up to issue conflict-credit to
any domains that could accept it. When xenstore is idle, this would
mean waking up every 50ms (by default) to do no work. With this
commit, we check whether any domain is below its cap, and if not then
we set the timeout for longer (the same timeout as before the
conflict-protection feature was added).

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders 84ee808e36 oxenstored: do not commit read-only transactions
The packet telling us to end the transaction has always carried an
argument telling us whether to commit.

If the transaction made no modifications to the tree, now we ignore
that argument and do not commit: it is just a waste of effort.

This makes read-only transactions immune to conflicts, and means that
we do not need to store any of their details in the history that is
used for assigning blame for conflicts.

We count a transaction as a read-only transaction only if it contains
no operations that modified the tree.

This means that (for example) a transaction that creates a new node
then deletes it would NOT count as read-only, even though it makes no
change overall. A more sophisticated algorithm could judge the
transaction based on comparison of its initial and final states, but
this would add complexity and computational cost.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders cb778dee01 oxenstored: allow self-conflicts
We already avoid inter-domain conflicts but now allow intra-domain
conflicts.  Although there are no known practical examples of a domain
that might perform operations that conflict with its own transactions,
this is conceivable, so here we avoid changing those semantics
unnecessarily.

When a transaction commit fails with a conflict and we look through
the history of commits to see which connection(s) to blame, ignore
historical commits that were made by the same connection as the
failing commit.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@citrix.com>
2017-04-05 15:10:42 +01:00
Jonathan Davies fa0b2b9555 oxenstored: blame the connection that caused a transaction conflict
Blame each connection found to have made a commit that would cause this
transaction to fail. Each blamed connection is penalised by having its
conflict-credit decremented.

Note the change in semantics for the replay function: we no longer stop after
finding the first operation that can't be replayed. This allows us to identify
all operations that conflicted with this transaction, not just the one that
conflicted first.

Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
v1 Reviewed-by: Christian Lindig <christian.lindig@citrix.com>

Changes since v1:
 * use correct log levels for informational messages
Changes since v2:
 * fix the blame algorithm and improve logging
   (fix was reviewed by Jonathan Davies)

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
2017-04-05 15:10:42 +01:00
Jonathan Davies 9ea503220d oxenstored: track commit history
Since the list of historic activity cannot grow without bound, it is safe to use
this to track commits.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Thomas Sanders <thomas.sanders@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders c68276082a oxenstored: discard old commit-history on txn end
The history of commits is to be used for working out which historical
commit(s) (including atomic writes) caused conflicts with a
currently-failing commit of a transaction. Any commit that was made
before the current transaction started cannot be relevant. Therefore
we never need to keep history from before the start of the
longest-running transaction that is open at any given time: whenever a
transaction ends (with or without a commit) then if it was the
longest-running open transaction we can delete history up until start
of the the next-longest-running open transaction.

Some transactions might stay open for a very long time, so if any
transaction exceeds conflict_max_history_seconds then we remove it
from consideration in this context, and will not guarantee to keep
remembering about historical commits made during such a transaction.

We implement this by keeping a list of all open transactions that have
not been open too long. When a transaction ends, we remove it from the
list, along with any that have been open longer than the maximum; then
we delete any history from before the start of the longest-running
transaction remaining in the list.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Christian Lindig <christian.lindig@citrix.com>
2017-04-05 15:10:42 +01:00
Jonathan Davies 9a2c5b42ad oxenstored: only record operations with side-effects in history
There is no need to record "read" operations as they will never cause another
transaction to fail.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Thomas Sanders <thomas.sanders@citrix.com>
2017-04-05 15:10:42 +01:00
Jonathan Davies 567051b618 oxenstored: support commit history tracking
Add ability to track xenstore tree operations -- either non-transactional
operations or committed transactions.

For now, the call to actually retain commits is commented out because history
can grow without bound.

For now, we call record_commit for all non-transactional operations. A
subsequent patch will make it retain only the ones with side-effects.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Christian Lindig <christian.lindig@citrix.com>
2017-04-05 15:10:42 +01:00
Jonathan Davies 4f4596a0e9 oxenstored: add transaction info relevant to history-tracking
Specifically:
 * retain the original store (not just the root) in full transactions
 * store commit count at the time of the start of the transaction

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reviewed-by: Christian Lindig <christian.lindig@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders b795db0e3d oxenstored: ignore domains with no conflict-credit
When processing connections, skip those from domains with no remaining
conflict-credit.

Also, issue a point of conflict-credit at regular intervals, the
period being set by the configuration option "conflict-max-history-
seconds".  When issuing conflict-credit, we give a point either to
every domain at once (one each) or only to the single domain at the
front of the queue, depending on the configuration option
"conflict-rate-limit-is-aggregate".

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Christian Lindig <christian.lindig@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders 6636c70b36 oxenstored: handling of domain conflict-credit
This commit gives each domain a conflict-credit variable, which will
later be used for limiting how often a domain can cause other domain's
transaction-commits to fail.

This commit also provides functions and data for manipulating domains
and their conflict-credit, and checking whether they have credit.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Christian Lindig <christian.lindig@citrix.com>
2017-04-05 15:10:42 +01:00
Thomas Sanders f2c7ab1f47 oxenstored: comments explaining some variables
It took a while of reading and reasoning to work out what these are
for, so here are comments to make life easier for everyone reading
this code in future.

Reported-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reviewed-by: Christian Lindig <christian.lindig@citrix.com>
2017-04-05 15:10:41 +01:00
Andrew Cooper 496673a2ad tools/oxenstored: Fix transaction handling in 32bit builds
In a 32bit build, the ocaml code 'proposed_id >= 0x7fffffff' compiles to:

  8055eac:       83 fb ff                cmp    $0xffffffff,%ebx
  8055eaf:       7d 0f                   jge    8055ec0 <...+0x20>

which in C is 'proposed_id >= INT_MIN', or in other words, tautologically
true.  As a result, 32bit builds of oxenstored always try to allocate the
transaction id 1, and fall into an infinite loop of trying the next id if
transaction 1 is already in use.

Restrict the range down to 1 billion, to sit in the positive half of a 31 bit
ocaml integer.  The compiled code is now:

  8055eac:       b9 ff ff ff 7f          mov    $0x7fffffff,%ecx
  8055eb1:       39 cb                   cmp    %ecx,%ebx
  8055eb3:       7d 0b                   jge    8055ec0 <...+0x20>

which (other than non-optimal code generation because of the unnecessary use
of %ecx), isn't unconditionally true.

In principle, the check could be changed to 'proposed_id == 0x7fffffff' which
would still allow for 2 billion transaction in 32bit builds.  However, in
64bit builds, this reintroduces a risk that if proposed_id is initially
greater than 0x7fffffff, it will not be clipped suitably into range.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
2016-10-31 21:02:08 +00:00
Andrew Cooper 0897514b4b tools/oxenstored: Avoid allocating invalid transaction ids
The transaction id of 0 is reserved, meaning "not in a transaction".  It is up
to the xenstored server to allocate transaction ids.  While oxenstored starts
its ids at 1, but insufficient care is taken with truncation cases.

A 32bit oxenstored has an int with 31 bits of width, meaning that the
transaction id will wrap around to 0 after 2 billion transactions.

A 64bit oxenstored has an int with 63 bits of width, meaning that once 4
billion transactions are used, the allocated id will be truncated when written
into the uin32_t field in the ring.  This causes the client to reply with the
truncated id, breaking any further attempt to use any transactions.

Limit all transaction ids to the range between 1 and 0x7ffffffe.  This is the
best which can be done without making oxenstored depend on Stdint or Cstruct,
yet still work for 32bit builds.

Also check that the proposed new transaction id isn't currently in use.  For
the first 2 billion transactions there is no chance of a collision, and after
that, the chance is at most 20 (the default open transaction quota) in 2
billion.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: David Scott <dave@recoil.org>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
2016-10-26 12:37:45 +01:00
Juergen Gross 976ce01e10 tools: remove systemd xenstore socket definitions
On a system with systemd the xenstore sockets are created via systemd.
Remove the related configuration files in order to be able to decide
at runtime whether the sockets should be created or not. This will
enable Xen to start xenstore either via a daemon or via a stub domain.

As the xenstore domain start program will exit after it has done its
job prepare the same behaviour to be tolerated by systemd for the
xenstore daemon by specifying the appropriate flags in the service
file.

A rerun of autogen.sh is required.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-08-05 10:37:02 +01:00
Wei Liu df59218807 oxenstored: honour XEN_RUN_DIR
Move default the pid file under XEN_RUN_DIR. Note that it changes the
location from /var/run to /var/run/xen.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-07-20 16:08:28 +01:00
Wei Liu 67a5ab831d oxenstored: honour XEN_RUN_STORED in systemd C stub
Generate a _paths.h for that and add proper dependency.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-06-17 11:27:21 +01:00
Wei Liu 44e6ba4b33 oxenstored: honour XEN_RUN_STORED and XEN_CONFIG_DIR
Only contain changes to ocaml source code. C stub files will be handled
separately.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-06-17 11:27:21 +01:00
Wei Liu 0e3e738605 ocaml/libxs: honour XEN_RUN_STORED
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-06-17 11:27:21 +01:00
Wei Liu 33bcbdb330 ocaml/libxs: generate a paths.ml
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-06-17 11:27:21 +01:00
Wei Liu a4fa83e568 oxenstored: honour XEN_LOG_DIR defined by configure
We generate a corresponding constant (in lower case) in paths.ml. Use
that in source code to get rid of hard-coded path.

Note that with this patch the default location for xenstored*.log is
changed from /var/log to /var/log/xen.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-06-14 14:31:27 +01:00
Wei Liu 6b007a6df1 oxenstored: generate a paths module
That file includes all paths generated by configure. It will be used to
get rid of hard-coded paths in code.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-06-14 14:31:27 +01:00
Wei Liu c84a1fa2e6 oxenstored: honour XEN_{LOG,RUN}_DIR in oxenstored.conf
Generate oxenstored.conf with configure. This involves modifying
tools/configure.ac and rerun autogen.sh.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
2016-06-14 14:31:27 +01:00
Konrad Rzeszutek Wilk 397463f2bb ocaml/xc_get_cpu_featureset/arm: Return not implemented on ARM
... as it is not implemented on it.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
2016-05-02 10:19:34 +01:00
Roger Pau Monne 92afa655e7 oxenstored: fix error when shifting negative value
By explicitly casting it to unsigned.

Reasoning on why this is needed, provided by Andrew Cooper:

"Ocaml stores integers shifted left by one, and with the bottom bit set.

Values with the bottom bit clear are pointers into the GC'd heap. Values
with the bottom bit set are integers, and need to be shifted by 1 bit to
have calculations performed."

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
2016-04-27 14:14:18 +01:00
Konrad Rzeszutek Wilk f359869074 Revert "libxc/libxl/python/xenstat/ocaml: Use new XEN_VERSION hypercall"
This reverts commit d275ec9ca8.

As we prefer to still utilize the old XENVER_ hypercall.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2016-04-26 11:53:49 +02:00
Andrew Cooper 9e9500fa07 tools/libxc: Wire a featureset through to cpuid policy logic
Later changes (Patch titled "tools/libxc: Use featuresets rather than
guesswork") will cause the cpuid generation logic to seed their
information from a featureset.  This patch adds the infrastructure to
specify a featureset, and will obtain the appropriate defaults from Xen
if omitted.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2016-04-08 21:55:10 +01:00
Andrew Cooper 1ab68ab1ef xen+tools: Export maximum host and guest cpu featuresets via SYSCTL
And provide stubs for toolstack use.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave@recoil.org>
Acked-by: Jan Beulich <JBeulich@suse.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
2016-04-08 21:54:48 +01:00
Wen Congyang a1d8b4bcd8 tools/libxl: Add back channel to allow migration target send data back
In COLO mode, secondary needs to send the following data to primary:
1. In libxl
   Secondary sends the following CHECKPOINT_CONTEXT to primary:
   CHECKPOINT_SVM_SUSPENDED, CHECKPOINT_SVM_READY and CHECKPOINT_SVM_RESUMED
2. In libxc
   Secondary sends the dirty pfn list to primary

But the io_fd only can be written in primary, and only can be read in
secondary. Save recv_fd in domain_suspend_state, and send_fd in
domain_create_state. Extend libxl_domain_create_restore API, add a
send_fd param to it. Add LIBXL_HAVE_CREATE_RESTORE_SEND_FD to indicate
the API change.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Signed-off-by: Yang Hongyang <hongyang.yang@easystack.cn>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2016-04-01 11:07:28 +08:00
Konrad Rzeszutek Wilk d275ec9ca8 libxc/libxl/python/xenstat/ocaml: Use new XEN_VERSION hypercall
We change the xen_version libxc code to use the new hypercall.
Which of course means every user in the code base has to
be changed over.

It is important to note that the xc_version_op has a different
return semantic than the previous one. It returns negative
values on error (like the old one), but it also returns
an positive value on success (unlike the old one). The positive
value is the number of bytes copied in.

Note that both Ocaml and xenstat use tabs instead of four
spaces so they look quite odd.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Andrew Cooper <andrew.cooper3@citrix.com> [for the Ocaml stubs]
Acked-by: George Dunlap <george.dunlap@eu.citrix.com> [xenctx bits]
Acked-by: Wei Liu <wei.liu2@citrix.com>
2016-03-30 16:01:20 -04:00
Jonathan Davies 72a91f6771 oxenstored: allow compilation prior to OCaml 3.12.0
Commit 363ae55c8 used an OCaml feature called record field punning. This broke
the build on compilers prior to OCaml 3.12.0.

This patch makes no semantic change but now uses backwards-compatible syntax.

Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reported-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Tested-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
2016-03-30 12:15:13 -04:00
Jonathan Davies 5963a60d4c oxenstored: log request and response during transaction replay
During a transaction replay, the replayed requests and the new responses are
logged in the same way as the original requests and the original responses.

Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-03-24 14:29:39 +00:00
Jonathan Davies aef35b9877 oxenstored: replay transaction upon conflict
The existing transaction merge algorithm keeps track of the least upper bound
(longest common prefix) of all the nodes which have been read and written, and
will re-combine two stores which have disjoint upper bounds. This works well for
small transactions but causes unnecessary conflicts for ones that span a large
subtree, such as the following ones used by the xapi toolstack:

 * VM start: creates /vm/... /vss/... /local/domain/...
   The least upper bound of this transaction is / and so all
   these transactions conflict with everything.

 * Device hotplug: creates /local/domain/0/... /local/domain/n/...
   The least upper bound of this transaction is /local/domain so
   all these transactions conflict with each other.

If the existing merge algorithm cannot merge and commit, we attempt
a /replay/ of the failed transaction against the new store.

When we replay the requests we check whether the response sent to the client is
the same as during the first attempt at the transaction. If the responses are
all the same then the transaction replay can be committed. If any differ then
the transaction replay must be aborted and the client must retry.

This algorithm uses the intuition that the transactions made by the toolstack
are designed to be for separate domains, and should fundamentally not conflict
in the sense that they don't read or write any shared keys. By replaying the
transaction on the server side we do what the client would have to do anyway,
only we can do it quickly without allowing any other requests to interfere.

Performing 300 parallel simulated VM start and shutdowns without this code:

300 parallel starts and shutdowns: 268.92

Performing 300 parallel simulated VM start and shutdowns with this code:

300 parallel starts and shutdowns: 3.80

Signed-off-by: Dave Scott <dave@recoil.org>
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-03-24 14:29:37 +00:00
Jonathan Davies be244d8667 oxenstored: move functions that process simple operations
Separate the functions which process operations that can be done as part of a
transaction. Specifically, these operations are: read, write, rm, getperms,
setperms, getdomainpath, directory, mkdir.

Also split function_of_type into two functions: one for processing the simple
operations and one for processing the rest.

This will help allow replay of transactions, allowing us to invoke the functions
that process the simple operations as part of the processing of transaction_end.

Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-03-24 14:27:40 +00:00
Jonathan Davies 0ae7f678bc oxenstored: keep track of each transaction's operations
A list of (request, response) pairs from the operations performed within the
transaction will be useful to support transaction replay.

Since this consumes memory, the number of requests per transaction must not be
left unbounded. Hence a new quota for this is introduced. This quota, configured
via the configuration key 'quota-maxrequests', limits the size of transactions
initiated by domUs.

After the maximum number of requests has been exhausted, any further requests
will result in EQUOTA errors. The client may then choose to end the transaction;
a successful commit will result in the retention of only the prior requests.

Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-03-24 14:26:54 +00:00
Jonathan Davies 363ae55c8a oxenstored: refactor request processing
Encapsulate the request in a record that is passed from do_input to
process_packet and input_handle_error.

This will be helpful when keeping track of the requests made as part of a
transaction.

Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-03-24 14:26:52 +00:00
Jonathan Davies f8ec0bc652 oxenstored: remove some unused parameters
Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-03-24 14:25:03 +00:00
Jonathan Davies a2c5046db9 oxenstored: refactor putting response on wire
Previously, the functions reply_{ack,data,data_or_ack} and input_handle_error
put the response on the wire by invoking Connection.send_{ack,reply,error}.

Instead, these functions now return a value indicating what needs to be put on
the wire, and that action is done by a send_response function called
afterwards.

This refactoring gives us a chance to store the value of the response, useful
for replaying transactions.

Signed-off-by: Jonathan Davies <jonathan.davies@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Reviewed-by: Euan Harris <euan.harris@citrix.com>
Acked-by: David Scott <dave@recoil.org>
2016-03-24 14:24:57 +00:00
Ian Campbell 3237645813 tools: Do not expose all builds to unstable hypercall APIs by default
Currently __XEN_TOOLS__ is defined in the global CFLAGS used for tools
builds, exposing them all to the various unstable hypercall interfaces.

However there are places where we do not want this (in particular all
the stable libraries now under tools/libs), and there is a risk of
such uses slipping in unnoticed (currently there are none, but I
tripped over the mixed nature of the HVMOPs just now in some new code
I was putting together).

Instead only define __XEN_TOOLS__ for things which explicitly want the
unstable APIs. The vast majority of these are consumers of libxenctrl,
which have opted into unstable interfaces by using libxenctrl already,
so adding -D__XEN_TOOLS__ to CFLAGS_libxenctrl covers these without
needing to patch the dozen or so tools manually.

The libxc build itself, which doesn't use CFLAGS_libxenctrl, now needs
the define too. As does gdbsx's own low level hypercall binding
library xg.

hvmloader no longer needs to undefine the symbol.

The ocaml evtchn bindings don't actually need sysctl.h and therefore
don't actually need __XEN_TOOLS__ either.

qemu-xen-traditional doesn't use CFLAGS_libxenctrl and so we pass the
define there (via --extra-cflags) too.

Having removed __XEN_TOOLS__ replace it with
__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ so that in
tree code is always using the latest APIs.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
2016-02-04 15:20:40 +00:00
Ian Campbell bc8baa9673 libxenevtchn: rename evtchn_port_or_errot_t as xenevtchn_port...
Otherwise the name clashes with the libxc evtchn_compat use of the
name (which is compat, so can't really change), and triggers an error
with older gcc relating to redefining typedefs (which are not allowed
per the spec, but which are tolerated by newer gcc)

None of qemu-trad, qemu-xen or mini-os actually use this type, so this
should be fine. Note that the library API is not actually stable until
4.7 is releases.

Reported-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Tested-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2016-02-04 15:19:30 +00:00
Ian Campbell 3248691679 tools: Refactor foreign memory mapping into libxenforeignmemory
libxenforeignmemory will provide a stable API and ABI for mapping
foreign domain memory (subject to appropriate privileges).

The new library exposes an interface equivalent to
xc_map_foreign_memory_bulk, which all the other
xc_map_foreign_memory_* functions (which remain in libxc) are
implemented in terms of.

Upon request (via #define XC_WANT_COMPAT_MAP_FOREIGN_API) libxenctrl
will provide a compat API for the old names. This is used by qemu-xen
and qemu-trad as well as various in tree things (which required
de-dupping various #includes in some too to get the #define before the
first).

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ ijc -- updated MINIOS_UPSTREAM_REVISION and QEMU_TRADITIONAL_REVISION ]
2016-01-22 12:24:03 +00:00
Ian Campbell 7a3543df83 tools/libxc: Remove osdep indirection for privcmd
The alternative backend (a xen-api/xapi shim) is no longer around and
so this stuff is now just baggage which is getting in the way of
refactoring libxenctrl.

Nested virt probably suffices for this use case now.

This was the last component of the osdep infrastructure, so all the
dynamic loading etc stuff all falls away too.

As part of this I was forced to investigate the twisty
xc_map_foreign_* maze, which I have added to the
toolstack-library-apis doc in the hopes of doing something sensible.

NetBSD and Solaris now call xc_map_foreign_bulk_compat directly from
their xc_map_foreign_bulk, which could have been achieved by using
some ifdefs around a renamed function. This will fall out in the wash
when these functions move to their own library.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2016-01-22 12:22:38 +00:00
Ian Campbell b7f76a699d tools: Refactor /dev/xen/evtchn wrappers into libxenevtchn.
libxenevtchn will provide a stable API and ABI for accessing the
evtchn device.

The functions are moved into the xenevtchn namespace to make a clean
break from libxc and avoid ambiguity regarding which interfaces are
stable.

All in-tree users are updated to use the new names.

Upon request (via #define XC_WANT_COMPAT_EVTCHN_API) libxenctrl will
provide a compat API for the old names. This is used by qemu-xen for
the time being. qemu-xen-traditional is updated in lockstep.

This leaves a few event channel related functions which go via privcmd
(EVTCHNOP) rather than ioctls on the /dev/xen/evtchn device in
libxenctrl. Specifically:

 - xc_evtchn_alloc_unbound
 - xc_evtchn_reset
 - xc_evtchn_status

Note that xc_evtchn_alloc_unbound's functionality is also provided by
xenevtchn_bind_unbound_port() (née xc_evtchn_bind_unbound_port) and is
probably redundant.

These functions do not appear to be needed by qemu-dm, qemu-pv
(provision of device model to HVM guests and PV backends respectively)
or by libvchan suggesting they are not needed by non-toolstack uses of
event channels. QEMU does use these in hw/xenpv/xen_domainbuild.c but
that is a "toolstack use".

The new library uses a version script to ensure that only expected
symbols are exported and to version them such that ABI guarantees can
be kept in the future.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
[ ijc -- updated MINIOS_UPSTREAM_REVISION and QEMU_TRADITIONAL_REVISION ]
2016-01-22 12:21:07 +00:00
Ian Campbell 97d9bc6ca6 tools/libxc: Remove osdep indirection for xc_evtchn
The alternative backend (a xen-api/xapi shim) is no longer around and
so this stuff is now just baggage which is getting in the way of
refactoring libxenctrl.

Note that the intention is to move this into a separate library
shortly.

Nested virt probably suffices for this use case now.

One incorrect instance of using xc_interface where xc_evtchn (in ocaml
stubs) is removed, this used to work because they were typedefs to the
same struct, but is no longer permitted.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2016-01-22 12:19:02 +00:00
Ian Campbell 5d3dc86715 tools: Refactor "xentoollog" into its own library
In attempting to disaggregate libxenctrl I found that many of the
pieces were going to want access to this library, so split it out (as
it probably should always have been).

Various build adjustments are needed. In particular things which use
xtl_* themselves now need to explicity link against the library.

This has a nice side effect which is that users of libxl no longer
need to link against libxenctrl just to create a logger, which was
counter to the principal that applications using libxl shouldn't be
required to look behind the curtain. This means that xl no longer
links against libxenctrl.

The new library uses a version script to ensure that only expected
symbols are exported and to version them such that ABI guarantees can
be kept in the future.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
[ ijc -- dropped QEMU_TRADITIONAL_REVISION update, this had since
         progressed to 569eac99e8dd which is after 9fad9ed28583, the
         commit needed here. ]
2016-01-05 11:45:26 +00:00
Ian Campbell ec07125761 Revert "tools: Refactor "xentoollog" into its own library"
This reverts commit c7d3afbb44 which
broke the libvirt build, since libvirt uses xtl_* and hence needs
updating to link against the new library when necessary.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
2015-12-10 10:21:34 +00:00
Ian Campbell c7d3afbb44 tools: Refactor "xentoollog" into its own library
In attempting to disaggregate libxenctrl I found that many of the
pieces were going to want access to this library, so split it out (as
it probably should always have been).

Various build adjustments are needed. In particular things which use
xtl_* themselves now need to explicity link against the library.

This has a nice side effect which is that users of libxl no longer
need to link against libxenctrl just to create a logger, which was
counter to the principal that applications using libxl shouldn't be
required to look behind the curtain. This means that xl no longer
links against libxenctrl.

The new library uses a version script to ensure that only expected
symbols are exported and to version them such that ABI guarantees can
be kept in the future.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
[ ijc -- Update QEMU_TRADITIONAL_REVISION and MINIOS_UPSTREAM_REVISION ]
2015-12-09 12:37:00 +00:00