Commit Graph
14437 Commits
Author SHA1 Message Date
Andrew Cooper 279521aed9 libxc/migration: Drop unimplemented domain types
x86 PVH is completely obsolete - it was intended for legacy PVH before that
idea was abandoned.  There was an RFC series for ARM in 2015, but there is
plenty of outstanding work which hasn't been done yet.

No functional change.  New types can be (re)introduced with the code which
actually implements them.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Julien Grall <julien@xen.org>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-24 14:59:04 +00:00
Andrew Cooper 50ea2445f4 libxc/migration: Rename TSC_INFO to X86_TSC_INFO
This record is specific to x86, and should have had a prefix to being with.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-24 14:59:04 +00:00
Andrew Cooper e9bd648015 libxc/restore: Don't duplicate state in process_vcpu_basic()
vcpu_guest_context_any_t is currently allocated on the stack, and copied from
a mutable buffer which is freed immediately after its use here.  Mutate the
buffer in place instead of duplicating it.

The code is as it is due to how it was developed.  Originally,
process_vcpu_basic() operated on a const pointer from the X86_VCPU_BASIC
record, but during upstreaming, the addition of Remus support required
buffering of X86_VCPU_BASIC records each checkpoint.

By the time process_vcpu_basic() runs, we are commited to completing state
restoration and unpausing the guest.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-24 14:59:04 +00:00
Nick Rosbrook bb3b34be87 golang/xenlight: implement array C to Go marshaling
Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-24 09:52:12 +00:00
Nick Rosbrook 7aac825058 golang/xenlight: add error return type to Context.Cpupoolinfo
A previous commit that removed Context.CheckOpen revealed
an ineffectual assignent to err in Context.Cpupoolinfo, as
there is no error return type.

Since it appears that the intent is to return an error here,
add an error return value to the function signature.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-24 09:52:12 +00:00
Nick Rosbrook b35567eeb8 golang/xenlight: revise use of Context type
Remove the exported global context variable, 'Ctx.' Generally, it is
better to not export global variables for use through a Go package.
However, there are some exceptions that can be found in the standard
library.

Add a NewContext function instead, and remove the Open, IsOpen, and
CheckOpen functions as a result.

Also, comment-out an ineffectual assignment to 'err' inside the function
Context.CpupoolInfo so that compilation does not fail.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-24 09:52:12 +00:00
Andrew Cooper f318635451 tools/libxc: Drop unused xc_compression_*()
There have been no users of the xc_compression_*() interface since Migration
v2 replaced legacy migration (2015, c/s b15bc4345).

It would need adjusting to fit into migration v2, and can be pulled out of git
history if someone wants to resurrect it in the future.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-20 16:08:00 +00:00
Andrew Cooper 31bf4f26aa tools/libxc: Drop other examples of the 'goto x; } else if' antipattern
None of these are buggy, but the resulting code is more robust.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-20 16:08:00 +00:00
Wei Liu d93630b18c tools: bump library version numbers
Signed-off-by: Wei Liu <wl@xen.org>
2019-12-20 14:21:25 +00:00
Andrew Cooper 2a62c22715 libxc/restore: Fix data auditing in handle_x86_pv_vcpu_blob()
The current logic only works by chance, in that XSAVE records also tend to be
a multiple of 128.  Implement the missing logic for XSAVE.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-20 12:12:00 +00:00
Andrew Cooper aafae0e800 libxc/restore: Fix data auditing in handle_x86_pv_info()
handle_x86_pv_info() has a subtle bug.  It uses an 'else if' chain with a
clause in the middle which doesn't exit unconditionally.  In practice, this
means that when restoring a 32bit PV guest, later sanity checks are skipped.

Rework the logic a little to be simpler.  There are exactly two valid
combinations of fields in X86_PV_INFO, so factor this out and check them all
in one go, before making adjustments to the current domain.

Once adjustments have been completed successfully, sanity check the result
against the X86_PV_INFO settings in one go, rather than piece-wise.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-20 12:11:29 +00:00
Andrew Cooper 1430c5a8ca tools/python: Python 3 compatibility
convert-legacy-stream is only used for incomming migration from pre Xen 4.7,
and verify-stream-v2 appears to only be used by me during migration
development - it is little surprise that they missed the main converstion
effort in Xen 4.13.

Fix it all up.

Move open_file_or_fd() into a new util.py to avoid duplication, making it a
more generic wrapper around open() or fdopen().

In libxc.py, drop all long() conversion.  Python 2 will DTRT with int => long
promotion, even on 32bit builds.

In convert-legacy-stream, don't pass empty strings to write_record().  Join on
the empty argl will do the right thing.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2019-12-20 11:19:56 +00:00
Konrad Rzeszutek Wilk 25164571fc Merge branch 'livepatch.aws.v6' into staging
* livepatch.aws.v6:
  livepatch: Add metadata runtime retrieval mechanism
  livepatch: Handle arbitrary size names with the list operation
  livepatch: Add support for modules .modinfo section metadata
  livepatch: Add support for inline asm livepatching expectations
  livepatch: Add per-function applied/reverted state tracking marker
  livepatch: Do not enforce ELF_LIVEPATCH_FUNC section presence
  livepatch: Add support for apply|revert action replacement hooks
  livepatch: Implement pre-|post- apply|revert hooks
  livepatch: Export payload structure via livepatch_payload.h
  livepatch: Allow to override inter-modules buildid dependency
  livepatch: Always check hypervisor build ID upon livepatch upload

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
2019-12-19 20:16:43 -05:00
Andrew Cooper 0cd791c499 tools/python: Drop test.py
This file hasn't been touched since it was introduced in 2005 (c/s 0c6f36628)
and has a wildly obsolete shebang for Python 2.3.  Most importantly for us is
that it isn't Python 3 compatible.

Drop the file entirely.  Since the 2.3 days, automatic discovery of tests has
been included in standard functionality.  Rewrite the test rule to use
"$(PYTHON) -m unittest discover" which is equivelent.

Dropping test.py drops the only piece of ZPL-2.0 code in the tree.  Drop the
ancillary files, and adjust COPYING to match.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
Reviewed-by: Lars Kurth <lars.kurth@citrix.com>
2019-12-19 18:48:00 +00:00
Tamas K Lengyel e09aa1dbf3 tools/libxc: clean up memory sharing files
No functional changes.

Signed-off-by: Tamas K Lengyel <tamas.lengyel@intel.com>
Acked-by: Wei Liu <wl@xen.org>
[Further cleanup]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
2019-12-19 18:05:11 +00:00
Jan Beulich dea5d6dd05 x86: include the PPIN in MCE records when available
Quoting the respective Linux commit:

    Intel Xeons from Ivy Bridge onwards support a processor identification
    number set in the factory. To the user this is a handy unique number to
    identify a particular CPU. Intel can decode this to the fab/production
    run to track errors. On systems that have it, include it in the machine
    check record. I'm told that this would be helpful for users that run
    large data centers with multi-socket servers to keep track of which CPUs
    are seeing errors.

Newer AMD CPUs support this too, at different MSR numbers.

Take the opportunity and hide __MC_NMSRS from the public interface going
forward.

[Linux commit 3f5a7896a5096fd50030a04d4c3f28a7441e30a5]
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
2019-12-18 14:49:10 +01:00
Steven Haigh 0e7c69bd3c tools/hotplug: Use ip on systems where brctl is not available
Newer distros like CentOS 8 do not have brctl available. As such, we
can't use it to configure networking anymore.

This patch will fall back to 'ip' or 'bridge' commands if brctl is not
available in the working PATH.

This would be a likely backport candidate to any version expected to be
built on CentOS 8 etc.

Signed-off-by: Steven Haigh <netwiz@crc.id.au>
Acked-by: Wei Liu <wl@xen.org>
2019-12-18 11:54:33 +00:00
Andrew Cooper f50a4f6e24 libxc/restore: Fix error message for unrecognised stream version
The Expected and Got values are rendered in the wrong order.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
2019-12-17 14:03:14 +00:00
Nick Rosbrook 21cb0bdcf4 golang/xenlight: implement keyed union C to Go marshaling
Switch over union key to determine how to populate 'union' in Go struct.

Since the unions of C types cannot be directly accessed in cgo, use a
typeof trick to typedef a struct in the cgo preamble that is analagous
to each inner struct of a keyed union. For example, to define a struct
for the hvm inner struct of libxl_domain_build_info, do:

  typedef typeof(((struct libxl_domain_build_info *)NULL)->u.hvm) libxl_domain_build_info_type_union_hvm;

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:10 +00:00
Nick Rosbrook cc8e54a1a2 golang/xenlight: begin C to Go type marshaling
Begin implementation of fromC marshaling functions for generated struct
types. This includes support for converting fields that are basic
primitive types such as string and integer types, nested anonymous
structs, nested libxl structs, and libxl built-in types.

This patch does not implement conversion of arrays or keyed unions.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:09 +00:00
Nick Rosbrook d916bf3f7e golang/xenlight: remove no-longer used type MemKB
Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:08 +00:00
Nick Rosbrook 42e71a4d1b golang/xenlight: generate structs from the IDL
Add struct and keyed union generation to gengotypes.py. For keyed unions,
use a method similar to gRPC's oneof to interpret C unions as Go types.
Meaning, for a given struct with a union field, generate a struct for
each sub-struct defined in the union. Then, define an interface of one
method which is implemented by each of the defined sub-structs. For
example:

  type domainBuildInfoTypeUnion interface {
          isdomainBuildInfoTypeUnion()
  }

  type DomainBuildInfoTypeUnionHvm struct {
      // HVM-specific fields...
  }

  func (x DomainBuildInfoTypeUnionHvm) isdomainBuildInfoTypeUnion() {}

  type DomainBuildInfoTypeUnionPv struct {
      // PV-specific fields...
  }

  func (x DomainBuildInfoTypeUnionPv) isdomainBuildInfoTypeUnion() {}

  type DomainBuildInfoTypeUnionPvh struct {
      // PVH-specific fields...
  }

  func (x DomainBuildInfoTypeUnionPvh) isdomainBuildInfoTypeUnion() {}

Then, remove existing struct definitions in xenlight.go that conflict
with the generated types, and modify existing marshaling functions to
align with the new type definitions. Notably, drop "time" package since
fields of type time.Duration are now of type uint64.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:08 +00:00
Nick Rosbrook f0e2384f3f golang/xenlight: re-factor Hwcap type implementation
Re-define Hwcap as [8]uint32, and implement toC function. Also, re-name and
modify signature of toGo function to fromC.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:07 +00:00
Nick Rosbrook 685001ee94 golang/xenlight: re-factor Uuid type implementation
Re-define Uuid as [16]byte and implement fromC, toC, and String functions.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:06 +00:00
Nick Rosbrook 186f162c2a golang/xenlight: define CpuidPolicyList builtin type
Define CpuidPolicyList as a string so that libxl_cpuid_parse_config can
be used in the toC function.

For now, fromC is a no-op since libxl does not support a way to read a
policy, modify it,and then give it back to libxl.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:05 +00:00
Nick Rosbrook c730fb2a83 golang/xenlight: define EvLink builtin as empty struct
Define EvLink as empty struct as there is currently no reason the internal of
this type should be used in Go.

Implement fromC and toC functions as no-ops.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:05 +00:00
Nick Rosbrook 4993745554 golang/xenlight: define MsVmGenid builtin type
Define MsVmGenid as [int(C.LIBXL_MS_VM_GENID_LEN)]byte and implement fromC and toC functions.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:04 +00:00
Nick Rosbrook 1db777e5e4 golang/xenlight: define Mac builtin type
Define Mac as [6]byte and implement fromC, toC, and String functions.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:03 +00:00
Nick Rosbrook c80a9da01a golang/xenlight: define StringList builtin type
Define StringList as []string an implement fromC and toC functions.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:02 +00:00
Nick Rosbrook 7a05fcff7f golang/xenlight: re-name Bitmap marshaling functions
Re-name and modify signature of toGo function to fromC. The reason for
using 'fromC' rather than 'toGo' is that it is not a good idea to define
methods on the C types. Also, add error return type to Bitmap's toC function.

Finally, as code-cleanup, re-organize the Bitmap type's comments as per
Go conventions.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
--
Changes in v2:
- Use consistent variable naming for slice created from
  libxl_bitmap.
2019-12-16 18:08:01 +00:00
Nick Rosbrook 7e84aeb864 golang/xenlight: define KeyValueList as empty struct
Define KeyValueList as empty struct as there is currently no reason for
this type to be available in the Go package.

Implement fromC and toC functions as no-ops.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:01 +00:00
Nick Rosbrook fb2475aed3 golang/xenlight: define Devid type as int
Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:08:00 +00:00
Nick Rosbrook 53d6570308 golang/xenlight: define Defbool builtin type
Define Defbool as struct analagous to the C type, and define the type
'defboolVal' that represent true, false, and default defbool values.

Implement Set, Unset, SetIfDefault, IsDefault, Val, and String functions
on Defbool so that the type can be used in Go analagously to how its
used in C.

Finally, implement fromC and toC functions.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:07:59 +00:00
Nick Rosbrook 871e51d2d4 golang/xenlight: generate enum types from IDL
Introduce gengotypes.py to generate Go code the from IDL. As a first step,
implement 'enum' type generation.

As a result of the newly-generated code, remove the existing, and now
conflicting definitions in xenlight.go. In the case of the Error type,
rename the slice 'errors' to 'libxlErrors' so that it does not conflict
with the standard library package 'errors.' And, negate the values used
in 'libxlErrors' since the generated error values are negative.

Signed-off-by: Nick Rosbrook <rosbrookn@ainfosec.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
2019-12-16 18:07:59 +00:00
Pawel Wieczorkiewicz 5083e0ff93 livepatch: Add metadata runtime retrieval mechanism
Extend the livepatch list operation to fetch also payloads' metadata.
This is achieved by extending the sysctl list interface with 2 extra
guest handles:
* metadata     - an array of arbitrary size strings
* metadata_len - an array of metadata strings' lengths (uin32_t each)

Payloads' metadata is a string of arbitrary size and does not have an
upper bound limit. It may also vary in size between payloads.

In order to let the userland allocate enough space for the incoming
data add a metadata total size field to the list sysctl operation and
fill it with total size of all payloads' metadata.

Extend the libxc to handle the metadata back-to-back data transfers
as well as metadata length array data transfers.

The xen-livepatch userland tool is extended to always display the
metadata for each received module. The metadata is received with the
following format: key=value\0key=value\0...key=value\0. The format is
modified to the following one: key=value;key=value;...key=value.
The new format allows to easily parse the metadata for a given module
by a machine.

Signed-off-by: Pawel Wieczorkiewicz <wipawel@amazon.de>
Reviewed-by: Andra-Irina Paraschiv <andraprs@amazon.com>
Reviewed-by: Martin Pohlack <mpohlack@amazon.de>
Reviewed-by: Norbert Manthey <nmanthey@amazon.de>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
2019-12-13 14:45:32 +00:00
Pawel Wieczorkiewicz b145b4a39c livepatch: Handle arbitrary size names with the list operation
The payloads' name strings can be of arbitrary size (typically small
with an upper bound of XEN_LIVEPATCH_NAME_SIZE).
Current implementation of the list operation interface allows to copy
names in the XEN_LIVEPATCH_NAME_SIZE chunks regardless of its actual
size and enforces space allocation requirements on userland tools.

To unify and simplify the interface, handle the name strings of
arbitrary size by copying them in adhering chunks to the userland.
In order to let the userland allocate enough space for the incoming
data add an auxiliary interface xc_livepatch_list_get_sizes() that
provides the current number of payload entries and the total size of
all name strings. This is achieved by extending the sysctl list
interface with an extra fields: name_total_size.

The xc_livepatch_list_get_sizes() issues the livepatch sysctl list
operation with the nr field set to 0. In this mode the operation
returns the number of payload entries and calculates the total sizes
for all payloads' names.
When the sysctl operation is issued with a non-zero nr field (for
instance with a value obtained earlier with the prior call to the
xc_livepatch_list_get_sizes()) the new field name_total_size provides
the total size of actually copied data.

Extend the libxc to handle the name back-to-back data transfers.

The xen-livepatch tool is modified to start the list operation with a
call to the xc_livepatch_list_get_sizes() to obtain the actual number
of payloads as well as the necessary space for names.
The tool now always requests the actual number of entries and leaves
the preemption handling to the libxc routine. The libxc still returns
'done' and 'left' parameters with the same semantic allowing the tool
to detect anomalies and react to them. At the moment it is expected
that the tool receives the exact number of entries as requested.
The xen-livepatch tool has been also modified to handle the name
back-to-back transfers correctly.

Signed-off-by: Pawel Wieczorkiewicz <wipawel@amazon.de>
Reviewed-by: Andra-Irina Paraschiv <andraprs@amazon.com>
Reviewed-by: Bjoern Doebel <doebel@amazon.de>
Reviewed-by: Martin Pohlack <mpohlack@amazon.de>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
2019-12-13 14:45:32 +00:00
Pawel Wieczorkiewicz b274989b61 livepatch: Allow to override inter-modules buildid dependency
By default Livepatch enforces the following buildid-based dependency
chain between livepatch modules:
  1) first module depends on given hypervisor buildid
  2) every consecutive module depends on previous module's buildid
This way proper livepatch stack order is maintained and enforced.
While it is important for production livepatches it limits agility and
blocks usage of testing or debug livepatches. These kinds of livepatch
modules are typically expected to be loaded at any time irrespective
of current state of the modules stack.

To enable testing and debug livepatches allow user dynamically ignore
the inter-modules dependency. In this case only hypervisor buildid
match is verified and enforced.

To allow userland pass additional paremeters for livepatch actions
add support for action flags.
Each of the apply, revert, unload and revert action gets additional
32-bit parameter 'flags' where extra flags can be applied in a mask
form.
Initially only one flag '--nodeps' is added for the apply action.
This flag modifies the default buildid dependency check as described
above.
The global sysctl interface input flag parameter is defined with a
single corresponding flag macro:
  LIVEPATCH_ACTION_APPLY_NODEPS (1 << 0)

The userland xen-livepatch tool is modified to support the '--nodeps'
flag for apply and load commands. A general mechanism for specifying
more flags in the future for apply and other action is however added.

Signed-off-by: Pawel Wieczorkiewicz <wipawel@amazon.de>
Reviewed-by: Andra-Irina Paraschiv <andraprs@amazon.com>
Reviewed-by: Eslam Elnikety <elnikety@amazon.de>
Reviewed-by: Petre Eftime <epetre@amazon.com>
Reviewed-by: Leonard Foerster <foersleo@amazon.de>
Reviewed-by: Martin Pohlack <mpohlack@amazon.de>
Reviewed-by: Norbert Manthey <nmanthey@amazon.de>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
2019-12-13 14:45:32 +00:00
Juergen Gross 5852ca4852 build: fix tools/configure in case only python3 exists
Calling ./configure with python3 being there but no python,
tools/configure will fail. Fix that by defaulting to python and
falling back to python3 or python2.

While at it fix the use of non portable "type -p" by replacing it by
AC_PATH_PROG().

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wl@xen.org>
[ wei: run autogen.sh ]
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-12-12 14:16:43 +00:00
Julian Tuminaro 0022387cef Fix the KDD_LOG statements to use appropriate format specifier for printing uint64_t
Previous commit in kdd.c had a small issue which lead to warning/error while compiling
on 32-bit systems due to mismatch of type size while doing type cast from uint64_t to
void *

Signed-off-by: Jenish Rakholiya <rjenish@cmu.edu>
Signed-off-by: Julian Tuminaro <jtuminar@andrew.cmu.edu>
Acked-by: Tim Deegan <tim@xen.org>
2019-12-01 22:06:15 +00:00
George Dunlap f2ae59bc4b Rationalize max_grant_frames and max_maptrack_frames handling
Xen used to have single, system-wide limits for the number of grant
frames and maptrack frames a guest was allowed to create. Increasing
or decreasing this single limit on the Xen command-line would change
the limit for all guests on the system.

Later, per-domain limits for these values was created. The system-wide
limits became strict limits: domains could not be created with higher
limits, but could be created with lower limits. However, that change
also introduced a range of different "default" values into various
places in the toolstack:

- The python libxc bindings hard-coded these values to 32 and 1024,
  respectively
- The libxl default values are 32 and 1024 respectively.
- xl will use the libxl default for maptrack, but does its own default
  calculation for grant frames: either 32 or 64, based on the max
  possible mfn.

These defaults interact poorly with the hypervisor command-line limit:

- The hypervisor command-line limit cannot be used to raise the limit
  for all guests anymore, as the default in the toolstack will
  effectively override this.
- If you use the hypervisor command-line limit to *reduce* the limit,
  then the "default" values generated by the toolstack are too high,
  and all guest creations will fail.

In other words, the toolstack defaults require any change to be
effected by having the admin explicitly specify a new value in every
guest.

In order to address this, have grant_table_init treat negative values
for max_grant_frames and max_maptrack_frames as instructions to use the
system-wide default, and have all the above toolstacks default to passing
-1 unless a different value is explicitly configured.

This restores the old behavior in that changing the hypervisor command-line
option can change the behavior for all guests, while retaining the ability
to set per-guest values.  It also removes the bug that reducing the
system-wide max will cause all domains without explicit limits to fail.

NOTE: - The Ocaml bindings require the caller to always specify a value,
        and the code to start a xenstored stubdomain hard-codes these to 4
	and 128 respectively; this behavour will not be modified.

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Paul Durrant <pdurrant@amazon.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2019-11-29 21:43:49 +00:00
julian.tuminaro@gmail.com 53e9ecffde kdd.c: Add support for initial handshake in KD protocol for Win 7, 8 and 10 (64 bit)
Current implementation of find_os is based on the hard-coded values for
different Windows version. It uses the value for get the address to
start looking for DOS header in the given specified range. However, this
is not scalable to all version of Windows as it will require us to keep
adding new entries and also due to KASLR, chances of not hitting the PE
header is significant. We implement a way for 64-bit systems to use IDT
entry to get a valid exception/interrupt handler and then move back into
the memory to find the valid DOS header. Since IDT entries are protected
by PatchGuard, we think our assumption that IDT entries will not be
corrupted is valid for our purpose. Once we have the image base, we
search for the DBGKD_GET_VERSION64 structure type in .data section to
get information required for handshake.

Currently, this is a work in progress feature and current patch only
supports the handshake and memory read/write on 64-bit systems.

NOTE: This is the Updated version of the previous patch submitted
NOTE: This has currently been only tested when debugging was not enabled
on the guest Windows.

Signed-off-by: Jenish Rakholiya <rjenish@cmu.edu>
Signed-off-by: Julian Tuminaro <jtuminar@andrew.cmu.edu>
Reviewed-by: Tim Deegan <tim@xen.org>
Reviewed-by: Paul Durrant <paul@xen.org>
2019-11-29 20:49:34 +00:00
Ian Jackson 8b955796c5 Config.mk et al: Branch off Xen 4.14-unstable
Bump the version numbers everywhere and rerun autogen.sh.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
2019-11-29 14:46:34 +00:00
George Dunlap 8c79c129a6 x86: Don't increase ApicIdCoreSize past 7
Changeset ca2eee92df ("x86, hvm: Expose host core/HT topology to HVM
guests") attempted to "fake up" a topology which would induce guest
operating systems to not treat vcpus as sibling hyperthreads.  This
involved actually reporting hyperthreading as available, but giving
vcpus every other ApicId; which in turn led to doubling the ApicIds
per core by bumping the ApicIdCoreSize by one.  In particular, Ryzen
3xxx series processors, and reportedly EPYC "Rome" cpus -- have an
ApicIdCoreSize of 7; the "fake" topology increases this to 8.

Unfortunately, Windows running on modern AMD hardware -- including
Ryzen 3xxx series processors, and reportedly EPYC "Rome" cpus --
doesn't seem to cope with this value being higher than 7.  (Linux
guests have so far continued to cope.)

A "proper" fix is complicated and it's too late to fix it either for
4.13, or to backport to supported branches.  As a short-term fix,
limit this value to 7.

This does mean that a Linux guest, booted on such a system without
this change, and then migrating to a system with this change, with
more than 64 vcpus, would see an apparent topology change.  This is a
low enough risk in practice that enabling this limit unilaterally, to
allow other guests to boot without manual intervention, is worth it.

Reported-by: Steven Haigh <netwiz@crc.id.au>
Reported-by: Andreas Kinzler <hfp@posteo.de>
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
2019-11-26 10:33:52 +00:00
Oleksandr Grytsov 8f486344a0 libxl: make default path to add/remove all PV devices
Adding/removing device is handled for specific devices only: VBD, VIF,
QDISK. This commit adds default case to handle adding/removing for all PV
devices by default, except QDISK device, which requires special handling.
If any other device is required a special handling it should be done by
implementing separate case (similar to QDISK device). The default
behaviour for adding device is to wait when the backend goes to
XenbusStateInitWait and the default behaviour on removing device is to
start generic device remove procedure.

Also this commit fixes removing guest function: before the guest was
removed when all VIF and VBD devices are removed. The fix removes
guest when all created devices are removed. This is done by checking the
guest device list instead of checking num_vifs and num_vbds. num_vifs and
num_vbds variables are removed as redundant in this case.

Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wl@xen.org>
Release-acked-by: Juergen Gross <jgross@suse.com>
2019-11-25 10:43:32 +00:00
Oleksandr Grytsov c568b11da2 libxl: introduce new backend type VINPUT
There are two kind of VKBD devices: with QEMU backend and user space PV
backend. In current implementation they can't be distinguished as both use
VKBD backend type. As result, user space PV KBD backend is started and
stopped as QEMU backend. This commit adds new device kind VINPUT to be
used as backend type for user space PV KBD backend.

Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
2019-11-25 10:42:56 +00:00
Anthony PERARD 0d2791b007 configure: Fix test for python 3.8
https://docs.python.org/3.8/whatsnew/3.8.html#debug-build-uses-the-same-abi-as-release-build

> To embed Python into an application, a new --embed option must be
> passed to python3-config --libs --embed to get -lpython3.8 (link the
> application to libpython). To support both 3.8 and older, try
> python3-config --libs --embed first and fallback to python3-config
> --libs (without --embed) if the previous command fails.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
[ wei: rerun autogen.sh ]
2019-11-20 10:46:48 +00:00
Ian Jackson bad237ded7 tools/configure: Honour XEN_COMPILE_ARCH and _TARGET_ for shim
The pvshim can only be built 64-bit because the hypervisor is only
64-bit nowadays.  The hypervisor build supports XEN_COMPILE_ARCH and
XEN_TARGET_ARCH which override the information from uname.  The pvshim
build runs out of the tools/ directory but calls the hypervisor build
system.

If one runs in a Linux 32-bit userland with a 64-bit kernel, one used
to be able to set XEN_COMPILE_ARCH.  But nowadays this does not work.
configure sees the target cpu as 64-bit and tries to build pvshim.
The build prints
  echo "*** Xen x86/32 target no longer supported!"
and doesn't build anything.  Then the subsequent Makefiles try to
install the non-built pieces.

Fix this anomaly by causing configure to honour the Xen hypervisor way
of setting the target architecture.

In principle this user behaviour is not handled quite right, because
configure will still see 64-bit and so all the autoconf-based
architecture testing will see 64-bit rather than 32-bit x86.  But the
tools are in fact generally quite portable: this particular location
in configure{.ac,} is the only place in tools/ where 64-bit x86 is
treated differently from 32-bit x86, so the fix is sufficient and
correct for this use case.

It remains the case that XEN_COMPILE_ARCH or XEN_TARGET_ARCH to a
non-x86 architecture, when configure thinks things are x86, or vice
versa, will not work right.

(This is a bugfix to 8845155c83
  pvshim: make PV shim build selectable from configure
which inadvertantly deleted the logic to only build the shim for
XEN_TARGET_ARCH != x86_32.)

I have rerun autogen.sh, so this patch contains the fix to configure
as well as the source fix to configure.ac.

Fixes: 8845155c83
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Olaf Hering <olaf@aepfle.de>
CC: Roger Pau Monné <roger.pau@citrix.com>
Release-acked-by: Jürgen Groß <jgross@suse.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Wei Liu <wl@xen.org>
2019-11-20 10:11:16 +00:00
Oleksandr Grytsov 0273d8e242 libxl: gentypes: initialise array elements in json
Currently, array elements are initialized with calloc.  Which means
initialize all element fields with zero values.  If an entry is not
present in the json (which is entirely permitted), the element will be
all-bits-zero instead of the default value (which is wrong).

The fix is to initalise each array element before parsing it, using
the new libxl_C_type_do_init function.

With existing types this results in a lot of new calls like this:

      for (i=0; (t=libxl__json_array_get(x,i)); i++) {
 +            libxl_sched_params_init(&p->vcpus[i]);
              rc = libxl__sched_params_parse_json(gc, t, &p->vcpus[i]);

(indentation adjusted).  This looks right.  To check what happens with
types which have nontrivial defaults but don't have init functions (of
which we currently have none in arrays), I (Ian) experimentally added:

      ("pnode", uint32), # physical node of this node
      ("vcpus", libxl_bitmap), # vcpus in this node
 +    ("sporks", Array(MemKB, "num_sporks")),
      ])

The result was this:

          for (i=0; (t=libxl__json_array_get(x,i)); i++) {
 +                p->sporks[i] = LIBXL_MEMKB_DEFAULT;
                  rc = libxl__uint64_parse_json(gc, t, &p->sporks[i]);

where the context was added by adding "sporks" and "+" indicates a
line added by this patch, "initialise array elements in json".

Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
---
v2 [iwj]: Use libxl_C_type_do_init.
          Reword commit message and discuss spork testing.
2019-11-19 16:44:33 +00:00
Ian Jackson f710b76911 libxl: gentypes.py: Break out libxl_C_type_do_init
This is going to be the common way to initialise things.
_libxl_C_type_init remains the thing for generating the body of the
init function, and for some special cases.

No functional change with existing types: C output is identical.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
2019-11-19 16:34:15 +00:00
Ian Jackson dde31350cb libxl: gentypes.py: Break out field_pass in ..._copy_deprecated
We are going to want this in a moment.

No functional change with existing types: C output is identical.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
2019-11-19 16:33:56 +00:00