Commit 23e2479ff9 ("xen/domain: introduce DOMID_ANY") broke at least:
* xenstored domains
* The Python bindings
* The Ocaml bindings (i.e XAPI, and therefore XenServer/XCP-ng)
by causing Xen to reject 0 (the wildcard sentinel since the start of Xen) and
DOMID_INVALID (since 4.21) without adjusting the callers of xc_domain_create().
Under other circumstances I'd revert, but xenstored has gained a dependent use
of DOMID_ANY.
Fix up the remaining in-tree callers of xc_domain_create(), which covers
init-xenstore-domain and the python bindings.
The Ocaml C bindings, like libxc itself, expose this ABI detail to their
caller, but in this case the caller is still internal to the bindings as a
whole. Adjust the default value in the internal transformation, adding
domid_any for legibility.
Leave a very clear message in CHANGELOG.md that all external callers need
altering.
Fixes: 23e2479ff9 ("xen/domain: introduce DOMID_ANY")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Pau Ruiz Safont <pau.safont@vates.tech>
Remove the code in tools for running a guest with qemu-traditional.
This covers xl, libxl, libacpi, hvmloader and the related python and
go bindings.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Jan Beulich <jbeulich@suse.com> # hvmloader
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
The HVM pirq feature allows routing interrupts from both physical and emulated
devices over event channels, this was done a performance improvement. However
its usage is fully undocumented, and the only reference implementation is in
Linux. It defeats the purpose of local APIC hardware virtualization, because
when using it interrupts avoid the usage of the local APIC altogether.
It has also been reported to not work properly with certain devices, at least
when using some AMD GPUs Linux attempts to route interrupts over event
channels, but Xen doesn't correctly detect such routing, which leads to the
hypervisor complaining with:
(XEN) d15v0: Unsupported MSI delivery mode 7 for Dom15
When MSIs are attempted to be routed over event channels the entry delivery
mode is set to ExtINT, but Xen doesn't detect such routing and attempts to
inject the interrupt following the native MSI path, and the ExtINT delivery
mode is not supported.
Disable HVM PIRQs by default and provide a per-domain option in xl.cfg to
enable such feature. Also for backwards compatibility keep the feature enabled
for any resumed domains that don't have an explicit selection.
Note that the only user of the feature (Linux) is also able to handle native
interrupts fine, as the feature was already not used if Xen reported local APIC
hardware virtualization active.
Link: https://github.com/QubesOS/qubes-issues/issues/7971
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Add a local .gitignore file for tools/python.
As at least on some systems (e.g. OpenSUSE Leap 15.5) the build will
produce a tools/python/xen.egg-info directory, add it to the new
.gitignore file, too.
Use "/dir/" as pattern for ignoring a directory and its contents, as
the "/dir/*" pattern won't ignore the directory itself (git just
doesn't complain when seeing a directory without any not ignored file
in it).
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Python distutils is deprecated and is going to be removed in Python
3.12. Add support for setuptools.
Setuptools in Python 3.11 complains:
SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
Keep using setup.py anyway to build the C extension.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Commit 56a7aaa16b introduced a memory leak on the error path for a
Py_BuildValue built object that on some newly introduced error path
has not the correct reference count handling, fix that by decrementing
the refcount in these path.
Fixes: 56a7aaa16b ("tools: add physinfo arch_capabilities handling for Arm")
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Commit 56a7aaa16b introduced some SVE related code that is protected by
'#if defined(__aarch64__)', the issue is that this doesn't take into
consideration when the toolstack is compiled for an arm32 Dom0 running on
an arm64 platform, it should be able to create SVE enabled guests but with
the current code it's not.
So fix the issue by compiling the code when the toolstack is compiled for
both arm32 and arm64.
Fixes: 56a7aaa16b ("tools: add physinfo arch_capabilities handling for Arm")
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
On Arm, the SVE vector length is encoded in arch_capabilities field
of struct xen_sysctl_physinfo, make use of this field in the tools
when building for arm.
Create header arm-arch-capabilities.h to handle the arch_capabilities
field of physinfo for Arm.
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
xc_domain_getinfo() is slow and prone to races because N hypercalls are
needed to find information about N domains. xc_domain_getinfolist() finds
the same information in a single hypercall as long as a big enough buffer
is provided. Plus, xc_domain_getinfo() is disappearing on a future patch
so migrate the callers interested in more than 1 domain to the the *list()
version.
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
In order to better reflect the contents of the header and to make it
more appropriate to use it for different runtime environments like
programs, libraries, and firmware, rename the libs.h include file to
common-macros.h. Additionally add a comment pointing out the need to be
self-contained.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> # tools/python/xen/lowlevel/xc/xc.c
Acked-by: Christian Lindig <christian.lindig@cloud.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
* Add X86_{CPUID,MSR}_POLICY_FORMAT checks which were missed previously.
* Drop test_suite(). It hasn't been necessary since the Py2.3 era.
* Drop the __main__ logic. This can't be used without manually adjusting the
include path, and `make test` knows how to do the right thing.
* For `make test`, use `-v` to see which tests have been discovered and run.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Python < 3.10 by default uses 'int' type for data+size string types
(s#), unless PY_SSIZE_T_CLEAN is defined - in which case it uses
Py_ssize_t. The former behavior was removed in Python 3.10 and now it's
required to define PY_SSIZE_T_CLEAN before including Python.h, and using
Py_ssize_t for the length argument. The PY_SSIZE_T_CLEAN behavior is
supported since Python 2.5.
Adjust bindings accordingly.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
Such max version should be provided by the caller, otherwise the
bindings will default to specifying a max version of 2, which is
inline with the current defaults in the hypervisor.
Fixes: 7379f9e10a ('gnttab: allow setting max version per-domain')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <iwj@xenproject.org>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Release-Acked-by: Ian Jackson <iwj@xenproject.org>
For log-dirty operations a 64-bit field is being truncated to become an
"int" return value. Seeing the large number of arguments the present
function takes, reduce its set of parameters to that needed for all
operations not involving the log-dirty bitmap, while introducing a new
wrapper for the log-dirty bitmap operations. This new function in turn
doesn't need an "mb" parameter, but has a 64-bit return type. (Using the
return value in favor of a pointer-type parameter is left as is, to
disturb callers as little as possible.)
While altering xc_shadow_control() anyway, also adjust the types of the
last two of the remaining parameters.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Ian Jackson <iwj@xenproject.org>
Using the first element of a tuple for a format specifier fails with
python3.4 as included in SLE12:
b = b"string/%x" % (i, )
TypeError: unsupported operand type(s) for %: 'bytes' and 'tuple'
It happens to work with python 2.7 and 3.6.
To support older Py3, format as strings and explicitly encode as ASCII.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
The trailing member name[] in libxl__physmap_info is written as a
cstring into the stream. The current code does a sanity check if the
last byte is zero. This attempt fails with python3 because name[-1]
returns a type int. As a result the comparison with byte(\00) fails:
File "/usr/lib/xen/bin/convert-legacy-stream", line 347, in read_libxl_toolstack
raise StreamError("physmap name not NUL terminated")
StreamError: physmap name not NUL terminated
To handle both python variants, cast to bytearray().
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
A single slash gives a float, a double slash gives an int.
bitmap = unpack_exact("Q" * ((max_id/64) + 1))
TypeError: can't multiply sequence by non-int of type 'float'
Use future division to remain compatible with python 2.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
With the split of libraries, I've observed a number of warnings from
(old?) ld.
Instead of duplicating the additions in two places, introduce a setup.py
make variable holding all the common parts of the invocations.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Unexpectedly the environment variable which needs to be passed is
$LDSHARED and not $LD. Otherwise Python may find the build `ld` instead
of the host `ld`.
Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects
it can load at runtime, not executables.
This uses $(CC) instead of $(LD) since Python distutils appends $CFLAGS
to $LDFLAGS which breaks many linkers.
Signed-off-by: Elliott Mitchell <ehem+xen@m5p.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Switch all usages of xs_daemon_open*() and xs_domain_open() to use
xs_open() instead. While at it switch xs_daemon_close() users to
xs_close().
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wl@xen.org>
Today xenctrl_dom.h is part of libxenctrl as it is included by
xc_private.c. This seems not to be needed, so merge xenctrl_dom.h into
xenguest.h where its contents really should be.
Replace all #includes of xenctrl_dom.h by xenguest.h ones or drop them
if xenguest.h is already included.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wl@xen.org>
It is conceptually wrong for this information to exist in the hypervisor in
the first place. Only the toolstack is capable of correctly reasoning about
the non-migrateability of guests.
This hypercall has only ever existed to control the visibility of the
Invariant TSC flag to the guest. Now that we have properly disentanged that
and moved ITSC into the guests CPUID policy, delete this hypercall.
Furthermore, this fixes a corner case where Xen would override the toolstacks
choice of ITSC for a xenstore stubdomain. That said, init-xenstore-domain
doesn't currently ask for ITSC, and MiniOS doesn't know what ITSC is either,
so nothing actually changes in practice.
Bump the DOMCTL_INTERFACE_VERSION for 4.15
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Commit 7c273ffdd0 ("tools/python: drop libxenguest from setup.py")
was just wrong: there is one function from libxenguest used in the
bindings, so readd the library again.
While at it remove the unused PATH_LIBXL setting.
Fixes: 7c273ffdd0 ("tools/python: drop libxenguest from setup.py")
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
There is no reason why libxenstore is not placed in the tools/libs
directory.
The common files between libxenstore and xenstored are kept in the
tools/xenstore directory to be easily accessible by xenstore-stubdom
which needs the xenstored files to be built.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wl@xen.org>
Today tools/libxc needs to be built after tools/libs as libxenctrl is
depending on some libraries in tools/libs. This in turn blocks moving
other libraries depending on libxenctrl below tools/libs.
So carve out libxenctrl from tools/libxc and move it into
tools/libs/ctrl.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org> (stubdom parts)
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> (python parts)
Acked-by: Wei Liu <wl@xen.org>
For being able to disentangle lixenctrl and libxenguest headers
xc_dom.h will need to be public. Prepare that by renaming xc_dom.h
to xenctrl_dom.h.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Wei Liu <wl@xen.org>
There is not a single wrapper for a libxenguest function defined.
So drop libxenguest from tools/python/setup.py.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Introduce a new static_data() hook which is responsible for writing out
any static data records. The HVM side continues to be a no-op, while
the PV side moves write_x86_pv_info() into this earlier hook. The the
common code writes out a STATIC_DATA_END record, and the stream version
is bumped to 3.
Update convert-legacy-stream to write a v3 stream, because this will
bypass the compatibly logic in libxc.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Migration data can be split into two parts - that which is invariant of
guest execution, and that which is not. Separate these two with the
STATIC_DATA_END record.
The short term, we want to move the x86 CPU Policy data into the stream.
In the longer term, we want to provisionally send the static data only
to the destination as a more robust compatibility check. In both cases,
we will want a callback into the higher level toolstack.
Mandate the presence of the STATIC_DATA_END record, and declare this v3,
along with instructions for how to compatibly interpret a v2 stream.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Migration v3 is in the process of being introduced, meaning that the code has
to cope with both versions. Use an explicit 2 for now.
For the verify-stream-v2 and convert-legacy-stream scripts, update text to say
"v2 (or later)". What matters is the distinction vs legacy streams.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Whenever python scripts are install, have the shebang be modified to use
whatever PYTHON_PATH is. This is useful for system where python isn't available, or
where the package build tools prevent unversioned shebang.
INSTALL_PYTHON_PROG only looks for "#!/usr/bin/env python".
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
This allows to use install-wrap when the source scripts is in a
subdirectory.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
pygrub in xen-4.13.0 with python 3.8.2 fails with the error
Traceback (most recent call last):
File "/usr/libexec/xen/bin/pygrub", line 21, in <module>
import xen.lowlevel.xc
SystemError: bad call flags
This patch fixes mismatches in tools/python/xen/lowlevel/xc/xc.c
between the flag bits defined in pyxc_methods and the parameters passed
to the corresponding PyObject definitions.
With this patch applied pygrub works as expected.
Signed-off-by: Michael Young <m.a.young@durham.ac.uk>
Reviewed-by: Wei Liu <wl@xen.org>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
These are believed-unused, and the underlying infrastructure is about to be
rewritten completely.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
The code has devating from the prevailing style in many ways. Adjust spacing,
indentation, position of operators, layout of multiline comments, removal of
superfluous comments, constness, trailing commas, and use of unqualified
'unsigned'.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
x86 PVH is completely obsolete - it was intended for legacy PVH before that
idea was abandoned. There was an RFC series for ARM in 2015, but there is
plenty of outstanding work which hasn't been done yet.
No functional change. New types can be (re)introduced with the code which
actually implements them.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Julien Grall <julien@xen.org>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
This record is specific to x86, and should have had a prefix to being with.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
convert-legacy-stream is only used for incomming migration from pre Xen 4.7,
and verify-stream-v2 appears to only be used by me during migration
development - it is little surprise that they missed the main converstion
effort in Xen 4.13.
Fix it all up.
Move open_file_or_fd() into a new util.py to avoid duplication, making it a
more generic wrapper around open() or fdopen().
In libxc.py, drop all long() conversion. Python 2 will DTRT with int => long
promotion, even on 32bit builds.
In convert-legacy-stream, don't pass empty strings to write_record(). Join on
the empty argl will do the right thing.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
This file hasn't been touched since it was introduced in 2005 (c/s 0c6f36628)
and has a wildly obsolete shebang for Python 2.3. Most importantly for us is
that it isn't Python 3 compatible.
Drop the file entirely. Since the 2.3 days, automatic discovery of tests has
been included in standard functionality. Rewrite the test rule to use
"$(PYTHON) -m unittest discover" which is equivelent.
Dropping test.py drops the only piece of ZPL-2.0 code in the tree. Drop the
ancillary files, and adjust COPYING to match.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
Reviewed-by: Lars Kurth <lars.kurth@citrix.com>
Xen used to have single, system-wide limits for the number of grant
frames and maptrack frames a guest was allowed to create. Increasing
or decreasing this single limit on the Xen command-line would change
the limit for all guests on the system.
Later, per-domain limits for these values was created. The system-wide
limits became strict limits: domains could not be created with higher
limits, but could be created with lower limits. However, that change
also introduced a range of different "default" values into various
places in the toolstack:
- The python libxc bindings hard-coded these values to 32 and 1024,
respectively
- The libxl default values are 32 and 1024 respectively.
- xl will use the libxl default for maptrack, but does its own default
calculation for grant frames: either 32 or 64, based on the max
possible mfn.
These defaults interact poorly with the hypervisor command-line limit:
- The hypervisor command-line limit cannot be used to raise the limit
for all guests anymore, as the default in the toolstack will
effectively override this.
- If you use the hypervisor command-line limit to *reduce* the limit,
then the "default" values generated by the toolstack are too high,
and all guest creations will fail.
In other words, the toolstack defaults require any change to be
effected by having the admin explicitly specify a new value in every
guest.
In order to address this, have grant_table_init treat negative values
for max_grant_frames and max_maptrack_frames as instructions to use the
system-wide default, and have all the above toolstacks default to passing
-1 unless a different value is explicitly configured.
This restores the old behavior in that changing the hypervisor command-line
option can change the behavior for all guests, while retaining the ability
to set per-guest values. It also removes the bug that reducing the
system-wide max will cause all domains without explicit limits to fail.
NOTE: - The Ocaml bindings require the caller to always specify a value,
and the code to start a xenstored stubdomain hard-codes these to 4
and 128 respectively; this behavour will not be modified.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Paul Durrant <pdurrant@amazon.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
c-indent-level isn't considered a safe variable, and "solaris" isn't a
recognised C style. Both cause prompts when opening the files.
Fix all blocks up per CODING_STYLE
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
Release-acked-by: Juergen Gross <jgross@suse.com>
This functionality is obsolete. It was introduced by c/s 41296317a3 into
Xend, but was never exposed in libxl.
Nothing limits this to PV guests, but it makes no sense for HVM guests.
Looking through the XenServer templates, this was used to work around bugs in
the 32bit RHEL/CentOS 4.7 and 4.8 kernels (fixed in 4.9) and RHEL/CentOS/OEL
5.2 and 5.3 kernels (fixed in 5.4). RHEL 4 as a major version went out of
support in 2017, whereas the 5.2/5.3 kernels went out of support when 5.4 was
released in 2009.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Andrew Cooper