Xen used to have single, system-wide limits for the number of grant
frames and maptrack frames a guest was allowed to create. Increasing
or decreasing this single limit on the Xen command-line would change
the limit for all guests on the system.
Later, per-domain limits for these values was created. The system-wide
limits became strict limits: domains could not be created with higher
limits, but could be created with lower limits. However, that change
also introduced a range of different "default" values into various
places in the toolstack:
- The python libxc bindings hard-coded these values to 32 and 1024,
respectively
- The libxl default values are 32 and 1024 respectively.
- xl will use the libxl default for maptrack, but does its own default
calculation for grant frames: either 32 or 64, based on the max
possible mfn.
These defaults interact poorly with the hypervisor command-line limit:
- The hypervisor command-line limit cannot be used to raise the limit
for all guests anymore, as the default in the toolstack will
effectively override this.
- If you use the hypervisor command-line limit to *reduce* the limit,
then the "default" values generated by the toolstack are too high,
and all guest creations will fail.
In other words, the toolstack defaults require any change to be
effected by having the admin explicitly specify a new value in every
guest.
In order to address this, have grant_table_init treat negative values
for max_grant_frames and max_maptrack_frames as instructions to use the
system-wide default, and have all the above toolstacks default to passing
-1 unless a different value is explicitly configured.
This restores the old behavior in that changing the hypervisor command-line
option can change the behavior for all guests, while retaining the ability
to set per-guest values. It also removes the bug that reducing the
system-wide max will cause all domains without explicit limits to fail.
NOTE: - The Ocaml bindings require the caller to always specify a value,
and the code to start a xenstored stubdomain hard-codes these to 4
and 128 respectively; this behavour will not be modified.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Paul Durrant <pdurrant@amazon.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wl@xen.org>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
master commit: f2ae59bc4b
master date: 2019-11-29 21:43:49 +0000
A call site in create_domain can call it with PVH type. That site was
missed during the review of 48dab9767.
Reinstate PVH type in the switch.
Reported-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
(cherry picked from commit 860d6e158d)
Parsing the config seems to be an overkill for this particular task
and the config might simply be absent. Type returned from libxl_domain_info
should be either LIBXL_DOMAIN_TYPE_HVM or LIBXL_DOMAIN_TYPE_PV but in
that context distinction between PVH and HVM should be irrelevant.
Signed-off-by: Igor Druzhinin <igor.druzhinin@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
(cherry picked from commit 48dab9767d)
Every invocation of xl via valgrind will show three leaks.
Since libxl_bitmap_alloc uses NOGC, the caller has to free the memory
after use. And since xl_ctx_free might be called before
parse_global_config, also move the libxl_bitmap_init calls into
xl_ctx_alloc.
Also move the call to atexit() after xl_ctx_alloc, because the latter is
also called again in postfork.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Wei Liu <wei.liu2@citrix.com>
0e2c886ef ("xen: decouple HVM and IOMMU capabilities") provided a
truth table for what `xl info` would report. In order to make the
table work xen will need to report its PV capability.
Replace cap_directio with cap_pv in libxl IDL. It is safe to do so
because cap_directio has never been released. Revert to use
cap_hvm_directio to mark the availability of IOMMU, to save us from
providing a compatibility layer.
Don't bump sysctl version number because we've already done so.
Also provide a new virt_caps "pv", change "directio" to "pv_directio".
The truth table is now:
pv hvm iommu flags in xl info
0 0 x n/a
0 1 0 hvm
0 1 1 hvm hvm_directio
1 0 0 pv
1 0 1 pv pv_directio
1 1 0 pv hvm
1 1 1 pv hvm hvm_directio pv_directio
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Right now, if either an hard or soft-affinity are explicitly specified
in a domain's config file, automatic NUMA placement is skipped. However,
automatic NUMA placement affects only the soft-affinity of the domain
which is being created.
Therefore, it is ok to let it run if an hard-affinity is specified. The
semantics will be that the best placement candidate would be found,
respecting the specified hard-affinity, i.e., using only the nodes that
contain the pcpus in the hard-affinity mask.
This is particularly helpful if global xl pinning masks are defined, as
made possible by commit aa67b97ed3 ("xl.conf: Add global affinity
masks"). In fact, without this commit, defining a global affinity mask
would also mean disabling automatic placement, but that does not
necessarily have to be the case (especially in large systems).
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Currently, the toolstack is considering Arm guest always PV. However,
they are very similar to PVH because HW virtualization extension are used
and QEMU is not started. So switch Arm guest type to PVH.
To keep compatibility with toolstack creating Arm guest with PV type
(e.g libvirt), libxl will now convert those guests to PVH.
Furthermore, the default type for Arm in xl will now be PVH to allow
smooth transition for user.
Signed-off-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
HVM and IOMMU are two distinct hardware features, yet they were
bundled together in sysctl and xl's output.
Decouple them on sysctl level. On toolstack level we still need to
maintain a sensible semantics for `xl info`. Massage the information
according to the following table:
pv hvm iommu flags in xl info
0 0 0 n/a
0 0 1 n/a
0 1 0 hvm
0 1 1 hvm hvm_directio
1 0 0 NIL
1 0 1 directio
1 1 0 hvm
1 1 1 hvm hvm_directio directio
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Trying to set the number of vcpus of a domain to 0 isn't refused.
We should not allow that.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
With not all physical cpus online (e.g. with smt=0) the output of hte
vcpu affinities is wrong, as the affinity bitmaps are capped after
nr_cpus bits, instead of using max_cpu_id.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Fix another smt=0 fallout: xl -N vcpu-pin prints only parts of the
affinities as it is using the number of online cpus instead of the
maximum cpu number.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
XSA-273 involves one hyperthread being able to use Spectre-like
techniques to "spy" on another thread. The details are somewhat
complicated, but the upshot is that after all Xen-based mitigations
have been applied:
* PV guests cannot spy on sibling threads
* HVM guests can spy on sibling threads
(NB that for purposes of this vulnerability, PVH and HVM guests are
identical. Whenever this comment refers to 'HVM', this includes PVH.)
There are many possible mitigations to this, including disabling
hyperthreading entirely. But another solution would be:
* Specify some cores as PV-only, others as PV or HVM
* Allow HVM guests to only run on thread 0 of the "HVM-or-PV" cores
* Allow PV guests to run on the above cores, as well as any thread of the PV-only cores.
For example, suppose you had 16 threads across 8 cores (0-7). You
could specify 0-3 as PV-only, and 4-7 as HVM-or-PV. Then you'd set
the affinity of the HVM guests as follows (binary representation):
0000000010101010
And the affinity of the PV guests as follows:
1111111110101010
In order to make this easy, this patches introduces three "global affinity
masks", placed in xl.conf:
vm.cpumask
vm.hvm.cpumask
vm.pv.cpumask
These are parsed just like the 'cpus' and 'cpus_soft' options in the
per-domain xl configuration files. The resulting mask is AND-ed with
whatever mask results at the end of the xl configuration file.
`vm.cpumask` would be applied to all guest types, `vm.hvm.cpumask`
would be applied to HVM and PVH guest types, and `vm.pv.cpumask`
would be applied to PV guest types.
The idea would be that to implement the above mask across all your
VMs, you'd simply add the following two lines to the configuration
file:
vm.hvm.cpumask=8,10,12,14
vm.pv.cpumask=0-8,10,12,14
See xl.conf manpage for details.
This is part of XSA-273 / CVE-2018-3646.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
In the display protocol connector's id is named as unique-id. This patch renames
it in the libxl/xl code and uses XENDISPL_FIELD... definitions from the protocol
header.
Signed-off-by: Oleksandr Grytsov <oleksandr_grytsov@epam.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The exact value of cpu_khz can only be obtained via 'xl dmesg', and
therefore can be lost after some time. 'xl info' truncates the value to
full MHz. Adjust the output to show the full khz value.
This helps the host admin to track how a host has calibrated itself. The
value of cpu_khz is used during live migration for the decision if
access to TSC should be emualted.
Commit eb5277a30e ("bitkeeper revision 1.959.1.4
(40d04a87acOb29u-5Y5OxMhHvP2x9g)" gives no hint why cpu_mhz instead of
cpu_khz was chosen.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
Add pvcalls support to libxl and xl. Create the appropriate pvcalls
entries in xenstore.
Signed-off-by: Stefano Stabellini <stefano@aporeto.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Make it possible to get and set a (Credit1) scheduler's
vCPU migration delay via the SCHEDOP sysctl, from both
libxl and xl (no change needed in libxc).
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
XSA-256 forces the local APIC to always be enabled for PVH guests, so
ignore any apic option for PVH guests. Update the documentation
accordingly.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
We generally support old config formats indefinintely (see the disk
format) without emitting warnings.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
This patch implements new generic set value interfaces in libxc and libxl.
These interfaces are suitable for all allocation features. It also adds a
new MBA set value command in xl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This patch implements generic get value interfaces in libxc and libxl.
It also refactors the get value flow in xl to make it be suitable for all
allocation features. Based on that, a new MBA get value command is added in xl.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
This patch implements a new xl get HW info interface. A new argument
is added for psr-hwinfo command to get and show MBA HW info.
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This breaks ARM. It should be protected by some x86 #if. For now,
revert it, as it's not critical (and it isn't included in the
comet/vixen security patch branches published via XSA-254).
This reverts commit 63080b7043.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Julien Grall <julien.grall@linaro.org>
If the config file specifies a type (or builder), it overrides this
default. But if it doesn't, you now get a PV-in-PVH guest.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Right now, this is broken because libxl__build_device_model_args_new
is used also for the qemu run for pv guests for qdisk devices, pvfb,
etc.
We can either make this option properly HVM-specific, or make it
generic.
In principle it is a reasonable request, to make the PV qemu
deprivileged (even though it is not likely to be implemented any time
soon). So make this option generic.
We retain the name "device model" even though it is arguably
inaccurate, because the xl docs already say, for example
For a PV guest a device-model is sometimes used to provide backends
for certain PV devices
The documentation patch here is pure code motion. For ease of review
we will fix up the docs, so the wording to be right for the new
context, in the next patch.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Reported-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
v2: Change xl too, to avoid breaking the build.
Fix manpage pod syntax.
This functionality is still quite imperfect, but it will be useful in
certain restricted use cases.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This is in line with the previous behavior, setting the number of
maptrack frames to 0 will prevent driver domains from working
correctly.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
An option is provided in libxl to enable/disable SBSA vuart while
creating a guest domain.
Libxl now supports a generic vuart console and SBSA uart is a specific type.
In future support can be added for multiple vuart of different types.
User can enable SBSA vuart by adding the following line in the guest
configuration file:
vuart = "sbsa_uart"
Signed-off-by: Bhupinder Thakur <bhupinder.thakur@linaro.org>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
The new firmware option aims to provide a coherent way to set the
firmware for the different kind of guests Xen supports.
For PV guests the available firmwares are pvgrub{32|64}, and for HVM
the following are supported: bios, uefi, seabios, rombios and ovmf.
Note that uefi maps to ovmf, and bios maps to the default firmware for
each device model.
The xl.cfg man page is updated to document the new feature.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Introduce a new type option to xl configuration files in order to
specify the domain type. This supersedes the current builder option.
The new option is documented in the xl.cfg man page, and the previous
builder option is marked as deprecated.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
This is required because those options will be used by the new PVH
guest type, and thus need to be shared between PV and HVM.
Defines are added in order to signal consumers that the fields are
available.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Code movement in preparation for making the bootloader,
bootloader_args, nested_hvm and timer_mode fields shared between all
guests types. While moving the code, limit the line-length to 80
columns.
No functional change.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Add new domain config items for setting the limits for the maximum
numbers of grant table frames and maptrack frames of a domain.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Add xl.conf config items for default values of grant limits:
max_grant_frames will set the default for the maximum number of grant
frames for a domain which will take effect if the domain's config file
doesn't specify a value. If max_grant_frames isn't set in xl.conf it
will default to 32 for hosts with all memory below 16TB and to 64 for
hosts with memory above 16TB.
max_maptrack_frames will set the default for the maximum number of
maptrack frames for a domain. If max_maptrack_frames isn't set in
xl.conf it will default to 0, as normally only backend domains need
maptrack frames.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
George Dunlap
Wei Liu
Igor Druzhinin
Olaf Hering
Dario Faggioli
Ian Jackson
Julien Grall
Juergen Gross
Oleksandr Grytsov
Stefano Stabellini
Roger Pau Monne
Doug Goldstein
Yi Sun
Ian Jackson
Meng Xu
Bhupinder Thakur
Dario Faggioli