in one sub-call: XEN_SYSCTL_TMEM_OP_GET_POOLS.
These operations are used during the save process of migration.
Instead of doing 64 hypercalls lets do just one. We modify
the 'struct xen_tmem_client' structure (used in
XEN_SYSCTL_TMEM_OP_[GET|SET]_CLIENT_INFO) to have an extra field
'nr_pools'. Armed with that the code slurping up pages from the
hypervisor can allocate a big enough structure (struct tmem_pool_info)
to contain all the active pools. And then just iterate over each
one and save it in the stream.
We are also re-using one of the subcommands numbers for this,
as such the XEN_SYSCTL_INTERFACE_VERSION should be incremented
and that was done in the patch titled:
"tmem/libxc: Squash XEN_SYSCTL_TMEM_OP_[SET|SAVE].."
In the xc_tmem_[save|restore] we also added proper memory handling
of the 'buf' and 'pools'. Because of the loops and to make it as
easy as possible to review we add a goto label and for almost
all error conditions jump in it.
The include for inttypes is required for the PRId64 macro to
work (which is needed to compile this code under 32-bit).
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
That is what they are used for. Lets make it more clear.
Of all the various sub-commands, the only one that needed
semantic change is XEN_SYSCTL_TMEM_OP_SAVE_BEGIN. That in the
past used 'arg1', and now we are moving it to use 'arg'.
Since that code is only used during migration which is tied
to the toolstack it is OK to change it.
We should increment the XEN_SYSCTL_INTERFACE_VERSION because
of this, and that was fortunatly done in the patch titled:
"tmem/libxc: Squash XEN_SYSCTL_TMEM_OP_[SET|SAVE].."
While at it, also fix xc_tmem_control_oid to properly handle
the 'buf' and bounce it as appropiate.
Acked-by: Andrew cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
return values. For success they used to be 1 ([SAVE,RESTORE]_BEGIN),
0 if guest did not have any tmem (but only for SAVE_BEGIN), and
-1 for any type of failure.
And SAVE_END (which you would think would mirror SAVE_BEGIN)
had 0 for success and -1 if guest did not any tmem enabled for it.
This is confusing. Now the code will return 0 if the operation was
success. Various XEN_EXX values are returned if tmem is not enabled
or the operation could not performed.
The xc_tmem.c code only needs one place to check - where we use
SAVE_BEGIN. The place where RESTORE_BEGIN is used will have errno
with the proper error value and return will be -1, so will still
fail properly.
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Specifically:
XEN_SYSCTL_TMEM_OP_SET_[WEIGHT,COMPRESS] are now done via:
XEN_SYSCTL_TMEM_SET_CLIENT_INFO
and XEN_SYSCTL_TMEM_OP_SAVE_GET_[VERSION,MAXPOOLS,
CLIENT_WEIGHT, CLIENT_FLAGS] can now be retrieved via:
XEN_SYSCTL_TMEM_GET_CLIENT_INFO
All this information is now in 'struct xen_tmem_client' and
that is what we pass around.
We also rev up the XEN_SYSCTL_INTERFACE_VERSION as we are
re-using the value number of the deleted ones (and henceforth
the information is retrieved differently).
On the toolstack, prior to this patch, the xc_tmem_control
would use the bounce buffer only when arg1 was set and the cmd
was to list. With the 'XEN_SYSCTL_TMEM_OP_SET_[WEIGHT|COMPRESS]'
that made sense as the 'arg1' would have the value. However
for the other ones (say XEN_SYSCTL_TMEM_OP_SAVE_GET_POOL_UUID)
the 'arg1' would be the length of the 'buf'. If this
confusing don't despair, patch patch titled:
tmem/xc_tmem_control: Rename 'arg1' to 'len' and 'arg2' to arg.
takes care of that.
The acute reader of the toolstack code will discover that
we only used the bounce buffer for LIST, not for any other
subcommands that used 'buf'!?! Which means that the contents
of 'buf' would never be copied back to the calleer 'buf'!
The author is not sure how this could possibly work, perhaps Xen 4.1
(when this was introduced) was more relaxed about the bounce buffer
being enabled. Anyhow this fixes xc_tmem_control to do it for
any subcommand that has 'arg1'.
Lastly some of the checks in xc_tmem_[restore|save] are removed
as they can't ever be reached (not even sure how they could
have been reached in the original submission). One of them
is the check for the weight against -1 when in fact the
hypervisor would never have provided that value.
Now the checks are simple - as the hypercall always returns
->version and ->maxpools (which is mirroring how it was done
prior to this patch). But if one wants to check the if a guest
has any tmem activity then the patch titled
"tmem: Batch and squash XEN_SYSCTL_TMEM_OP_SAVE_GET_POOL_
[FLAGS,NPAGES,UUID] in one sub-call: XEN_SYSCTL_TMEM_OP_GET_POOLS."
adds an ->nr_pools to check for that.
Also we add the check for ->version and ->maxpools and remove
the TODO.
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
No functional change. We do this to prepare for another
entry to be added in the union. See patch titled:
"tmem/libxc: Squash XEN_SYSCTL_TMEM_OP_[SET|SAVE]"
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
It is not used by anything. Its intent was to complement
the 'weight' attribute but there hadn't been any request for this.
If there is a need to resurface it, it can be integrated back
via the XEN_SYSCTL_TMEM_SET_CLIENT_INFO introduced in
"tmem/libxc: Squash XEN_SYSCTL_TMEM_OP_[SET|SAVE].."
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
The only thing this hypercall returns is TMEM_SPEC_VERSION.
The comment around is also misleading - this call does not
do any domain operation.
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
libxencall will provide a stable API and ABI for calling hypercalls
(although those hypercalls themselves may not have a stable API). As
well as the hypercall buffer infrastructure needed in order to safely
provide pointer arguments to hypercalls.
libxenctrl encapsulates a instance of this interface, so users of that
library are not currently subjected to any actual changes. However all
hypercalls made internally by libxc now use the correct interface. It
is expected that most users of this library will be other libraries
providing a higher level interface, rather than applications directly.
Only the basic functionality to allocate hypercall safe memory is
moved, the type safe stuff and bounce buffers remain in libxc.
Note that the functionality to map foreign pages using privcmd is not
yet moved, meaning that an xc_interface will now contain two open
privcmd file descriptors. Foreign memory mapping is logically separate
functionality and will be moved into its own library.
The new library uses a version script to ensure that only expected
symbols are exported and to version them such that ABI guarantees can
be kept in the future.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
[ ijc -- updated MINIOS_UPSTREAM_REVISION and QEMU_TRADITIONAL_REVISION ]
Instead of the three member uint64_t structure.
The structure is used by the control stack for
XEN_SYSCTL_TMEM_OP_SAVE_GET_NEXT_[PAGE|INV] only so
moving it to the sysctl header.
Also modified tmemc_save_get_next_page to deal with
the new type - and converted some of the on-stack
usage of the array to use an pointer.
Further work will be to make the xen_sysctl_tmem_op have
an union with proper type for the two: ..GET_NEXT_[PAGE|INV]
operations.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
And use it almost everywhere. It is easy to use it for the
sysctl since the hypervisor and toolstack are intertwined.
But for the tmem hypercall we need to be dilligient (as it
is guest facing) so delaying that to another patch:
"tmem: Use 'struct xen_tmem_oid' for every user" to help
with bisection issues.
We also move some of the parameters on functions to be within
the right location.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Jen Beulich <jbeulich@suse.com>
The operations are to be used by an control domain to set parameters,
list pools, clients, and to be used during migration.
There is no need to have them in the tmem hypercall path.
This patch moves code without adding fixes - and in fact in
some cases makes the parameters soo long that they hurt eyes - but
that is for another patch.
Note that in regards to existing users:
- Only the control domain could call it - which meant that if
a guest called it would get -EPERM, so we are OK there.
In practice no guests called this TMEM_CONTROL command.
- The spec: https://oss.oracle.com/projects/tmem/dist/documentation/api/tmemspec-v001.pdf
mentions: "TBD [Not sure if this is really needed.]"
which is a carte blanche as any to do this!
Note: The XSM check is the same - we just move it from do_tmem_op
to do_sysctl.
We also add an 32-bit pad to make the sysctl structure have the same
exact size under 32 and 64-bit toolstacks and not worry about aligment
issues.
And the XLAT does not need to deal with the buf as it has been
moved to another structure which is 32/64 fixed.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Acked-by: Jen Beulich <jbeulich@suse.com>
It mentions it but it is never used. The hypercall interface
knows nothing of this sort of thing either. Lets just remove it.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Wei Liu <wei.liu2@citrix.com> [release + toolstack]
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
We are doing another call to set_xen_guest_handle right
after the xc_hypercall_bounce_pre (the correct place to do it).
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
As recommended in http://www.gnu.org/licenses/gpl-howto.en.html.
This is the result of:
$ git grep -El Mass\|Temple\|Franklin | xargs ./fsf.pl
Where fsf.pl is:
#!/usr/bin/perl -w -pi.bak -0777
my $repl = 'If not, see <http://www.gnu.org/licenses/>.';
my $br = qr/(?:\s*\n\s*(?:[\*\#]|\/\/|\.\\" )?\s*|\s+)/;
my $inwt = qr/[Ii]f${br}not,${br}write${br}(?:to${br})?the${br}Free${br}Software${br}Foundation,(?:${br}Inc\.,)?/;
my $mass = qr/675${br}Mass${br}Ave,?${br}Cambridge,?${br}MA${br}02139,?${br}USA,?\.?/;
my $franklin = qr/51${br}Franklin${br}St(?:reet)?(?:,${br}| - )Fifth${br}Floor,?${br}Boston,?${br}MA,?${br}02110-1301,?${br}USA,?\.?/;
my $temple = qr/59${br}Temple${br}Place(?:,${br}| - )Suite${br}330,?${br}Boston,?${br}MA,?${br}021110?-1307,?${br}USA,?\.?/;
s|$inwt$br$mass|$repl|m;
s|$inwt$br$franklin|$repl|m;
s|$inwt$br$temple|$repl|m;
The only remaining mentions of these addresses are in COPYING files which I
haven't touched.
Some of the changed files are imports from elsewhere, however
filtering them out is tricky, I think it is tolerable to have these
files be modified here and then perhaps reverted on the next sync,
since it's only 1-2 lines and obvious what is going on.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
The API returns now negative values on error and stashes
the error in errno. Fix the user of this API.
The 'xc_hypercall_bounce_pre' can fail - and if so it will
stash its errno values - no need to over-write it.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The ifdef sections are not enabled at all in tree, and their justification is
out of date now that Xen hypercall support exists upstream in valgrind.
This also removes a commented-out tweak to CFLAGS in the libxc Makefile which
is not being used, and becomes stale given this patch. In the unlikely event
that any developers were using the line, the results can be more easily
achieved by tweaking APPEND_CFLAGS in the environment.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The emacs variable to set the C style from a local variable block is
c-file-style, not c-set-style.
Signed-off-by: David Vrabel <david.vrabel@citrix.com
xc_tmem.c: In function 'xc_tmem_restore':
xc_tmem.c:393:14: error: variable 'save_max_pools' set but not used
[-Werror=unused-but-set-variable]
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Not actually used here but useful to confirm that a handle is passed
down to each location where it will be required once we switch to
hypercall buffers.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
After a great deal of discussion and review with linux
kernel developers, it appears there are "next-generation"
filesystems (such as btrfs, xfs, Lustre) that will not
be able to use tmem due to an ABI limitation... a field
that represents a unique file identifier is 64-bits in
the tmem ABI and may need to be as large as 192-bits.
So to support these guest filesystems, the tmem ABI must be
revised, from "v0" to "v1".
I *think* it is still the case that tmem is experimental
and is not used anywhere yet in production.
The tmem ABI is designed to support multiple revisions,
so the Xen tmem implementation could be updated to
handle both v0 and v1. However this is a bit
messy and would require data structures for both v0
and v1 to appear in public Xen header files.
I am inclined to update the Xen tmem implementation
to only support v1 and gracefully fail v0. This would
result in only a performance loss (as if tmem were
disabled) for newly launched tmem-v0-enabled guests,
but live-migration between old tmem-v0 Xen and new
tmem-v1 Xen machines would fail, and saved tmem-v0
guests will not be able to be restored on a tmem-v1
Xen machine. I would plan to update both pre-4.0.2
and unstable (future 4.1) to only support v1.
I believe these restrictions are reasonable at this
point in the tmem lifecycle, though they may not
be reasonable in the near future; should the tmem
ABI need to be revised from v1 to v2, I understand
backwards compatibility will be required.
Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1282671421 -3600
# Node ID d1dd29a470ef1b9d2c77478a123326036dfe90bb
# Parent d7a4adad9c328decbd384d87b23001aea8951b86
tools/libxc, tools/libelf: Relicense under LGPL v2.1
Relicense these two libraries under LGPL v2.1 only except where
individual files already included the "or later" provision.
Copyright holders have been contacted by Stephen Spector and have all
agreed this change.
Removed tools/libxc/ia64/aclinux.h since it appeared to be
unused. There is a separate, more up to date, copy in
xen/include/acpi/platform/aclinux.h which does appear to be used.
Clarify the license of MiniOS privcmd.h under the same terms as other
tools/include/xen-sys headers.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Stephen Spector <stephen.spector@citrix.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
This patch eliminate the global variables in libxenctrl (used for
logging and error reporting).
Instead the information which was in the global variables is now in a
new xc_interface* opaque structure, which xc_interface open returns
instead of the raw file descriptor; furthermore, logging is done via
xentoollog.
There are three new parameters to xc_interface_open to control the
logging, but existing callers can just pass "0" for all three to get
the old behaviour.
All libxc callers have been adjusted accordingly.
Also update QEMU_TAG for corresponding qemu change.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Attached patch implements save/restore/migration/livemigration
for transcendent memory ("tmem"). Without this patch, domains
using tmem may in some cases lose data when doing save/restore
or migrate/livemigrate. Also included in this patch is
support for a new (privileged) hypercall for authorizing
domains to share pools; this provides the foundation to
accomodate upstream linux requests for security for shared
pools.
Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
This implicitly required coverting the tmem_op structure from
anonymous to standard struct/union sub-fields, and extending the
get-fields.sh helper script to deal with typedef-ed guest handles used
as types of translated compound type fields.
Signed-off-by: Jan Beulich <jbeulich@novell.com>
Tmem, when called from a tmem-capable (paravirtualized) guest, makes
use of otherwise unutilized ("fallow") memory to create and manage
pools of pages that can be accessed from the guest either as
"ephemeral" pages or as "persistent" pages. In either case, the pages
are not directly addressible by the guest, only copied to and fro via
the tmem interface. Ephemeral pages are a nice place for a guest to
put recently evicted clean pages that it might need again; these pages
can be reclaimed synchronously by Xen for other guests or other uses.
Persistent pages are a nice place for a guest to put "swap" pages to
avoid sending them to disk. These pages retain data as long as the
guest lives, but count against the guest memory allocation.
Tmem pages may optionally be compressed and, in certain cases, can be
shared between guests. Tmem also handles concurrency nicely and
provides limited QoS settings to combat malicious DoS attempts.
Save/restore and live migration support is not yet provided.
Tmem is primarily targeted for an x86 64-bit hypervisor. On a 32-bit
x86 hypervisor, it has limited functionality and testing due to
limitations of the xen heap. Nearly all of tmem is
architecture-independent; three routines remain to be ported to ia64
and it should work on that architecture too. It is also structured to
be portable to non-Xen environments.
Tmem defaults off (for now) and must be enabled with a "tmem" xen boot
option (and does nothing unless a tmem-capable guest is running). The
"tmem_compress" boot option enables compression which takes about 10x
more CPU but approximately doubles the number of pages that can be
stored.
Tmem can be controlled via several "xm" commands and many interesting
tmem statistics can be obtained. A README and internal specification
will follow, but lots of useful prose about tmem, as well as Linux
patches, can be found at http://oss.oracle.com/projects/tmem .
Signed-off-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Konrad Rzeszutek Wilk
Ian Campbell
Andrew Cooper
David Vrabel
Olaf Hering
Keir Fraser
Keir Fraser