Introduce new Xen command line parameter called "vwfi", which stands for
virtual wfi. The default is "trap": Xen traps guest wfi and wfe
instructions. In the case of wfi, Xen calls vcpu_block on the guest
vcpu; in the case of guest wfe, Xen calls vcpu_yield on the guest vcpu.
The behavior can be changed by setting vwfi to "native", in that case
Xen doesn't trap neither wfi nor wfe, running them in guest context.
The result is strong reduction in irq latency (from 5000ns to 2000ns,
measured using https://github.com/edgarigl/tbm, the physical timer, and
1 pcpu dedicated to 1 vcpu). The downside is that the scheduler thinks
that the guest is busy when actually is sleeping, leading to suboptimal
scheduling decisions.
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Julien Grall <julien.grall@arm.com>
PV Calls is a paravirtualized protocol that allows the implementation of
a set of POSIX functions in a different domain. The PV Calls frontend
sends POSIX function calls to the backend, which implements them and
returns a value to the frontend and acts on the function call.
This version of the document covers networking function calls, such as
connect, accept, bind, release, listen, poll, recvmsg and sendmsg; but
the protocol is meant to be easily extended to cover different sets of
calls. Unimplemented commands return ENOTSUP.
Signed-off-by: Stefano Stabellini <stefano@aporeto.com>
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
9pfs is a network filesystem protocol developed for Plan 9. 9pfs is very
simple and describes a series of commands and responses. It is
completely independent from the communication channels, in fact many
clients and servers support multiple channels, usually called
"transports". For example the Linux client supports tcp and unix
sockets, fds, virtio and rdma.
This design document outlines the transport protocol for
9pfs payload.
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Stefano Stabellini <stefano@aporeto.com>
As some users have suggested, elaborate the usage of RMRR specification
on the command line, and provide a usage example.
Also, always treat the specified page numbers as hexadecimal values.
Signed-off-by: Venu Busireddy <venu.busireddy@oracle.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
The default dom0_mem is 128M which is not sufficient to boot a Ubuntu
based Dom0. It is not clear what a better default value could be.
Instead, loudly warn the user when dom0_mem is unspecified and wait 3
secs. Then use 512M.
Update the docs to specify that dom0_mem is required on ARM. (The
current xen-command-line document does not actually reflect the current
behavior of dom0_mem on ARM correctly.)
Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Julien Grall <julien.grall@arm.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
There have been reports that Fedora 25 uses /run instead of /var/run.
Add a --with-rundir option ito configure to be able to specify that
directory. Default is still /var/run.
A re-run of autogen.sh is required.
Signed-off-by: Juergen Gross <jgross@suse.com>
[ wei: run autogen.sh ]
Acked-by: Wei Liu <wei.liu2@citrix.com>
PVHv2 guests, unlike HVM guests, won't have the option to route interrupts
from physical or emulated devices over event channels using PIRQs. This
applies to both DomU and Dom0 PVHv2 guests.
Introduce a new XEN_X86_EMU_USE_PIRQ to notify Xen whether a HVM guest can
route physical interrupts (even from emulated devices) over event channels,
and is thus allowed to use some of the PHYSDEV ops.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Split the Dom0 builder into two different functions, one for PV (and classic
PVH), and another one for PVHv2. Introduce a new command line parameter called
'dom0' that can be used to request the creation of a PVHv2 Dom0 by setting the
'hvm' sub-option. A panic has also been added if a user tries to use dom0=hvm
until all the code is in place, then the panic will be removed.
While there mark the dom0_shadow option that was used by PV Dom0 as deprecated,
it was lacking documentation and was not functional. Point users towards
dom0=shadow instead.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The information given in the xl man page for the mem-max command is
rather brief. Expand it in order to let the reader understand what it
is really doing.
As the related libxl function libxl_domain_setmaxmem() isn't much
clearer add a comment to it explaining the desired semantics.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The documentation states that a value of '1' will cause unplug of
emulated IDE disks. This is not quite correct, as QEMU will also unplug
emulated SCSI disks at the same time.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
...as a set of hypercalls to be used by a device model.
As stated in the new docs/designs/dm_op.markdown:
"The aim of DMOP is to prevent a compromised device model from
compromising domains other then the one it is associated with. (And is
therefore likely already compromised)."
See that file for further information.
This patch simply adds the boilerplate for the hypercall.
Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
Suggested-by: Ian Jackson <ian.jackson@citrix.com>
Suggested-by: Jennifer Herbert <jennifer.herbert@citrix.com>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
On some platforms firmware fails to specify RMRR regions in ACPI tables and
thus those regions will not be mapped in dom0 or guests and may cause IO
Page Faults and prevent dom0 from booting if "iommu=dom0-strict" option is
specified on the Xen command line.
New Xen command line option rmrr allows to specify such devices and
memory regions. These regions are added to the list of RMRR defined in ACPI
if the device is present in system. As a result, additional RMRRs will be
mapped 1:1 in dom0 with correct permissions.
The above mentioned problems were discovered during the PVH work with
ThinkCentre M and Dell 5600T. No official documentation was found so far
in regards to what devices and why cause this. Experiments show that
ThinkCentre M USB devices with enabled debug port generate DMA read
transactions to the regions of memory marked reserved in host e820 map.
For Dell 5600T the device and faulting addresses are not found yet.
For detailed history of the discussion please check following threads:
http://lists.Xen.org/archives/html/xen-devel/2015-02/msg01724.htmlhttp://lists.Xen.org/archives/html/xen-devel/2015-01/msg02513.html
Format for rmrr Xen command line option:
rmrr=start<-end>=[s1]bdf1[,[s1]bdf2[,...]];start<-end>=[s2]bdf1[,[s2]bdf2[,...]]
For example, for Lenovo ThinkCentre M, use:
rmrr=0xd5d45=0:0:1d.0;0xd5d46=0:0:1a.0
If grub2 used and multiple ranges are specified, ';' should be
quoted/escaped, refer to grub2 manual for more information.
Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: Venu Busireddy <venu.busireddy@oracle.com>
Acked-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
tscmode.txt is referenced in xl.cfg(5). Convert it into a pod
formatted man page.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
pci-device-reservations is references in xl.cfg(5), convert it as a man
page in pod format. The name is now prefixed with 'xen-' to avoid
possible name conflicts.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
channel.txt is referenced in xl.cfg(5). Move it to man pages, section 7.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
vtpmmgr.txt is referenced in a man page, convert it to a man page.
The man page is named xen-vtpmmgr to avoid any conflict with other
potential vtpm docs.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
vtpm.txt is referenced in xl.cfg man page. Convert it to pod,
move it to the man folder and update the reference. The man page
is named xen-vtmp to avoid any potential conflict with other
VTPM documentation.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
docs/misc/xl-numa-placement.markdown is referenced by xl.cfg.5 man page,
move it to a man page, section 7.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Make vbd-interface a man page, section7, as this document is
referenced in other man pages (xl-disk-configuration)
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Convert xl-disk-configuration.txt from plain text file to a POD file
to get it as a man page. The references to it in the other man pages
are also updated.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Move docs/misc/xl-network-configuration.markdown to docs/man and
update the references to it in the other man pages.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Some of the docs/misc documents will need to go in man 7 section,
prepare docs/Makefile for it.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Some of the docs/misc documents are written in markdown language.
As an effort to cleanup man pages these documents will be converted into
man pages. To avoid some more conversion, add rules to the docs/Makefile
to generate man pages out of markdown files as well as pod ones.
However, pandoc doesn't know how to convert man pages links. Thus the
man links in markdown pages won't work.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The hypervisor expects the timeout from the hypercall to be in
nanoseconds, so document this correctly. Also correctly document
what happens when timeout is set to zero.
Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Qdisk supports qcow and qcow2, extend it to also support qed disk
format.
Signed-off-by: Cédric Bosdonnat <cbosdonnat@suse.com>
[ wei: regenerate libxlu_disk_l.{c,h} ]
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Benchmarks have shown that per-socket runqueues arrangement
behaves better (e.g., we achieve better load balancing)
than the current per-core default.
Here's an example (coming from
https://lists.xen.org/archives/html/xen-devel/2016-06/msg02287.html ):
|=======================================|
| XEN BUILD TIME, LOW LOAD, NO NOISE |
|---------------------------------------|
| runq=core runq=socket |
| 35.200 33.433 |
|---------------------------------------|------------------------------|
| XEN BUILD TIME, HIGH LOAD, NO NOISE | IPERF, HIGH LOAD, NO NOISE |
|---------------------------------------|------------------------------|
| runq=core runq=socket | runq=core runq=socket |
| 18.013 18.530 | 23.200 23.466 |
|---------------------------------------|------------------------------|
| XEN BUILD TIME, LOW LOAD, WITH NOISE |
|------------------------------------- |
| runq=core runq=socket |
| 45.866 39.493 |
|---------------------------------------|------------------------------|
| XEN BUILD TIME, HIGH LOAD, WITH NOISE | IPERF, HIGH LOAD, WITH NOISE |
|---------------------------------------|------------------------------|
| runq=core runq=socket | runq=core runq=socket |
| 36.840 29.080 | 19.967 21.000 |
|=======================================|==============================|
The only reason why we went for per-core, initially, was to
introduce some form of hyperthreading support. Now we have
hyperthreading support, independently from how runqueues
are organized (9bb9c7388 "xen: credit2: implement true SMT
support"), and thus we can switch to per-socket.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
It is possible for PVHv2 guests to get the hardware description from ACPI
tables, add this to the documentation also.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
On PVHv2 guests the local APIC can also be used to start APs if present.
Amend the documentation in order to reflect this.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
Credit2 is available in tree as an "Experimental" scheduler since
a few years. Recently, effort started for making it production ready
and, eventually, the new Xen's default scheduler. As a consequence of
that, it has undergone a great deal of development, testing and
benchmarking.
In fact, Credit2's much more modern (wrt Credit1) design and cleaner
code makes it a lot easier to understand what the scheduler is doing,
fix scheduling issues that may come up, and implement new and more
advanced features, in future.
In some more details:
- key features that were missing (pinning and context switching
rate-limiting) have now been implemented, and more (soft affinity,
caps and reservations) are about to come. The gap wrt Credit1 is
therefore closing. In particular, with pinning and rate-limiting
available, the scheduler can be considered usable.
- Credit2 is tested by OSSTest since long time. Furthermore, as a
part of recent efforts, stress tests and benchmarks have been run
and shown no bugs or stability issues.
- A number of different benchmarks have been run, most of them
comparing Credit2 with Credit1. Some of the results were posted on
xen-devel, some others have been illustrated during a talk at 2016
edition of Xen-Project Developer Summit. In general, performance
look promising --if not better than Credit1 already, in some of
the cases.
It therefore appears that we are ready to mark the Credit2 scheduler
as a 'Supported' feature, and ask users to look at it and try it, if
they think it suits their needs.
Of course, declaring something 'Supported' has security implications.
So here it is how the situation looks like from a security standpoint:
1) Is guest->host privilege escalation possible?
The only interfaces exposed to unprivileged guests are the SCHEDOP
hypercalls, and timers. None of those hypercalls contain any pointers,
and they don't look to contain any privilege escalation path. Also,
they're not specific to Credit2, as they're "used" by all schedulers
(ingluding the current default, Credit1), so anything about these
interfaces would be a security concern already.
2) Is guest user->guest kernel escalation possible?
The guest kernel is not really relying on anything from the scheduler
to protect itself or any data in any way.
3) Is there any information leakage?
The only information which the scheduler exposes to unprivileged
guests is the timing information. This may be able to be used for
side-channel attacks to probabilistically infer things about other
vcpus running on the same system; but this has not traditionally
been considered within the security boundary. And, again, this is
possible with all schedulers.
The control domain can issue DOMCTL_SCHEDOP and SYSCTL_SCHEDOP
hypercalls, but the involved data structures are handled in a
way that does not leak information (which would be leaked "only"
to Dom0 anyway).
4) Can a Denial-of-Service be triggered?
This is a risk, with schedulers, and one that's hard to foresee.
For instance, it _did_ happen on Credit1, in the past (a vcpu
could "game the system" by sleeping at particular times to gain
BOOST priority and monopolize 95% of the cpu). In that case, it
was possible because of the probabilistic nature of accounting
in Credit1 (which was then fixed). Well, Credit2:
- already do accurate, rather than probabilistic, accounting;
- does not have any BOOST or, in general, any way for a vcpu to
become 'more important' than the others: they're all subjected
to the same crediting algorithm.
Also note that, the accounting and the crediting algorithm are a lot
simpler than in Credit1, and hence a lot easier to understand, debug
and audit.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
docs/misc/xenstore.txt states that xenstored will use "0" as a valid
transaction id after 2^32 transactions. This is not true. Remove that
statement.
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
SMEP/SMAP is a security feature to prevent kernel executing/accessing
user address involuntarily, any such behavior will lead to a page fault.
SMEP/SMAP is open (in CR4) for both Xen and HVM guest in earlier code.
SMEP/SMAP bit set in Xen CR4 would enforce security checking for 32-bit
PV guest which will suffer unknown SMEP/SMAP page fault when guest
kernel attempt to access user address although SMEP/SMAP is close for
PV guests.
This patch introduces a new boot option value "hvm" for "sm{e,a}p", it
is going to diable SMEP/SMAP for Xen hypervisor while enable them for
HVM. In this way, 32-bit PV guest will not suffer SMEP/SMAP security
issue. Users can choose whether open SMEP/SMAP for Xen itself,
especially when they are going to run 32-bit PV guests.
Signed-off-by: He Chen <he.chen@linux.intel.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
[Fixed up command line docs]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Couple of reasons:
- It can lead to security issues (see row-hammer, KSM and such
attacks).
- Code is quite complex.
- Deduplication is good if the pages themselves are the same
but that is hardly guaranteed.
- We got some gains (if pages are deduped) but at the cost of
making code less maintainable.
- tze depends on deduplication code.
As such, deleting it.
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Wei Liu
Roger Pau Monne
Zhang Chen
Stefano Stabellini
Stefano Stabellini
Venu Busireddy
Juergen Gross
Olaf Hering
Fatih Acar
Paul Durrant
Elena Ufimtseva
Cédric Bosdonnat
Ross Lagerwall
Dario Faggioli
Meng Xu
He Chen
Konrad Rzeszutek Wilk