To unify certain common sanity checks, checks are done very early in
processing based only on the top-level type.
Unfortunately, when TRC_HVM_EMUL was introduced, it broke some of the
assumptions about how the top-level types worked. Namely, traces of
this type will show up outside of HVM contexts: in idle domains and in
PV domains.
Make an explicit exception for TRC_HVM_EMUL types in a number of places:
- Pass the record info pointer to toplevel_assert_check, so that it
can exclude TRC_HVM_EMUL records from idle and vcpu data_mode
checks
- Don't attempt to set the vcpu data_type in hvm_process for
TRC_HVM_EMUL records.
Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
A long-standing usability sub-optimality with xenalyze is the
necessity to specify `--svm-mode` when analyzing AMD processors. This
fundamentally comes about because the same trace event ID is used for
both VMX and SVM, but the contents of the trace must be interpreted
differently.
Instead, allocate separate trace events for VMX and SVM vmexits in
Xen; this will allow all readers to properly interpret the meaning of
the vmexit reason.
In xenalyze, first remove the redundant call to init_hvm_data();
there's no way to get to hvm_vmexit_process() without it being already
initialized by the set_vcpu_type call in hvm_process().
Replace this with set_hvm_exit_reson_data(), and move setting of
hvm->exit_reason_* into that function.
Modify hvm_process and hvm_vmexit_process to handle all four potential
values appropriately.
If SVM entries are encountered, set opt.svm_mode so that other
SVM-specific functionality is triggered.
Remove the `--svm-mode` command-line option, since it's now redundant.
Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
xentrace_format was always of limited utility, since trace records
across pcpus were processed out of order; it was superseded by xenalyze
over a decade ago.
But for several releases, the `formats` file it has depended on for
proper operation has not even been included in `make install` (which
generally means it doesn't get picked up by distros either); yet
nobody has seemed to complain.
Simple remove xentrace_format, and point people to xenalyze instead.
NB that there is no man page for xenalyze, so the "see also" on the
xentrace man page is simply removed for now.
Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Olaf Hering <olaf@aepfle.de>
We warn when we see data for a vcpu moving into a non-RUNNING state,
just so that people know why we're ignoring it. On full traces, this
happens only once. However, if the trace was limited to a subset of
pcpus, then this will happen every time the domain in question is
woken on that pcpu.
Add a 'delayed_init' flag to the vcpu struct to indicate when a vcpu
has experienced a delayed init. Print a warning message once when
entering the state, and once when leaving it.
Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
EIP lists are generalized across several use cases. For many of them,
it make sense to have a cycle per sample; but not really for interrupt
EIP lists. For this reason, it normally just passes 0 as for the tsc
value, which will in turn down at the bottom of update_cycles(),
update only the summary.event_count, but nothing else.
The dump_eip() function attempted to handle this by calling the generic
cycle print handler if the summary contained *any* cycles, and by collecting
and printing its own stats, based solely on counts, if not.
Unfortunately, it used the wrong element for this: it collected the
total from samples.count rather samples.event_count; in the case that
there are no cycles, this will always be zero. It then divided by
this zero value. This results in output that looked like this:
```
ffff89d29656 : 0 -nan%
ffff89d298b6 : 0 -nan%
ffff89d298c0 : 0 -nan%
```
It's better than nothing, but a lot less informative than one would
like.
Use event_count rather than count for collecting the total, and the
reporting when there are no cycles in the summary information. This results
in output that looks like this:
```
ffff89d29656 : 2 1.21%
ffff89d298b6 : 1 0.61%
ffff89d298c0 : 1 0.61%
```
Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
If a vmentry/exit arc unexpectedly doesn't have a handler, we throw an
error, and then log the information under HVM event 0; thus those
particular traces within the vmexit reason will have stats gathered,
and will show up with "(no handler)". This is useful in the event
that there are unusual paths through the hypervisor which don't have
trace points.
However, if there are more than one of these, then although we detect and warn
that this is happening, we subsequently continue to stuff data about all exits
into that one exit, even though we only show it in one place.
Instead, refator things to only allow a single exit reason to be
accumulated into any given event.
Also put a comment explaining what's going on, and how to fix it.
Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
These defines are not used by xenalyze.
NR_CPUS will not match what was used for building Xen itself.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: George Dunlap <george.dunlap@cloud.com>
Invoking the --help option of any tool should not return with an error,
if that tool does have a documented and implemented help option.
Adjust the usage() function to exit with either error or success.
Handle the existing entry in the option table to call usage accordingly.
Adjust the getopt value for help. The char '?' is returned for unknown
options. Returning 'h' instead of '?' allows to handle --help.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: George Dunlap <george.dunlap@cloud.com>
A recent xentrace highlighted an unhandled corner case in the vcpu
"start-of-day" logic, if the trace starts after the last running ->
non-running transition, but before the first non-running -> running
transition. Because start-of-day wasn't handled, vcpu_next_update()
was expecting p->current to be NULL, and tripping out with the
following error message when it wasn't:
vcpu_next_update: FATAL: p->current not NULL! (d32768dv$p, runstate RUNSTATE_INIT)
where 32768 is the DEFAULT_DOMAIN, and $p is the pcpu number.
Instead of calling vcpu_start() piecemeal throughout
sched_runstate_process(), call it at the top of the function if the
vcpu in question is still in RUNSTATE_INIT, so that we can handle all
the cases in one place.
Sketch out at the top of the function all cases which we need to
handle, and what to do in those cases. Some transitions tell us where
v is running; some transitions tell us about what is (or is not)
running on p; some transitions tell us neither.
If a transition tells us where v is now running, update its state;
otherwise leave it in INIT, in order to avoid having to deal with TSC
skew on start-up.
If a transition tells us what is or is not running on p, update
p->current (either to v or NULL). Otherwise leave it alone.
If neither, do nothing.
Reifying those rules:
- If we're continuing to run, set v to RUNNING, and use p->first_tsc
as the runstate time.
- If we're starting to run, set v to RUNNING, and use ri->tsc as the
runstate time.
- If v is being deschedled, leave v in the INIT state to avoid dealing
with TSC skew; but set p->current to NULL so that whatever is
scheduled next won't trigger the assert in vcpu_next_update().
- If a vcpu is waking up (switching from one non-runnable state to
another non-runnable state), leave v in INIT, and p in whatever
state it's in (which may be the default domain, or some other vcpu
which has already run).
While here, fix the comment above vcpu_start; it's called when the
vcpu state is INIT, not when current is the default domain.
Signed-off-by: George Dunlap <george.dunlap@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
The program is structured so that fatal errors cause exit() to be
called directly, rather than being passed up the stack; returning a
value here may mislead people into believing otherwise.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: George Dunlap <george.dunlap@cloud.com>
The output file is optional. In case it is missing, xentrace is supposed
to write to stdout - unless it is a tty, which is checked prior using it.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: George Dunlap <george.dunlap@cloud.com>
clang complains about the signed type:
implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
The potential ABI change in libxenvchan is covered by the Xen version based SONAME.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Four TRC_MEM_* records supply custom structures with tail padding, leaking
stack rubble into the trace buffer. Three of the records were fine in 32-bit
builds of Xen, due to the relaxed alignment of 64-bit integers, but
POD_SUPERPAGE_SPLITER was broken right from the outset.
We could pack the datastructures to remove the padding, but xentrace_format
has no way of rendering the upper half of a 16-bit field. Instead, expand all
16-bit fields to 32-bit.
For POD_SUPERPAGE_SPLINTER, introduce an order field as it is relevant
information, and to match DECREASE_RESERVATION, and so it doesn't require a
__packed attribute to drop tail padding.
Update xenalyze's structures to match, and introduce xentrace_format rendering
which was absent previously.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Dario Faggioli <dfaggioli@suse.com>
In order to better reflect the contents of the header and to make it
more appropriate to use it for different runtime environments like
programs, libraries, and firmware, rename the libs.h include file to
common-macros.h. Additionally add a comment pointing out the need to be
self-contained.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> # tools/python/xen/lowlevel/xc/xc.c
Acked-by: Christian Lindig <christian.lindig@cloud.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Remove "build" targets.
Use "$(TARGETS)" to list binary to be built.
Cleanup "clean" rule.
Also drop conditional install of $(BIN) and $(LIBBIN) as those two
variables are now always populated.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Henry Wang <Henry.Wang@arm.com>
The first thing XEN_SYSCTL_physinfo does is zero op->u.physinfo.
Do not copy-in. It's pointless, and most callers don't initialise their
xc_physinfo_t buffer to begin with. Remove the redundant zeroing from the
remaining callers.
Spotted by Coverity.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Chen <Wei.Chen@arm.com>
Acked-by: Anthony PERARD <anthony.perard@citrix.com>
918b8842a8 changed CPSR and SPSR to be stored as 64bit values.
This is fixing the print size in some tools to use 64bit type.
Fixes: 918b8842a8 ("arm64: Change type of hsr, cpsr, spsr_el1 to uint64_t")
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@arm.com>
Tested-by: Michal Orzel <michal.orzel@arm.com>
literal strings are not meant to be modified. So we should use const
char * rather than char * when we want to store a pointer to them.
Signed-off-by: Julien Grall <jgrall@amazon.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Use unsigned char variable, or cast to (unsigned char), for
tolower()/islower() and friends. Fix compiler error
array subscript has type 'char' [-Werror=char-subscripts]
Signed-off-by: Manuel Bouyer <bouyer@netbsd.org>
Reviewed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Release-Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
There is a bug in commit 5e4b419966 ("xen: credit2: only reset
credit on reset condition"). In fact, the aim of that commit was to
make sure that we do not perform too many credit reset operations
(which are not super cheap, and in an hot-path). But the check used
to determine whether a reset is necessary was the wrong one.
In fact, knowing just that some vCPUs have been skipped, while
traversing the runqueue (in runq_candidate()), is not enough. We
need to check explicitly whether the first vCPU in the runqueue
has a negative amount of credit.
Since a trace record is changed, this patch updates xentrace format file
and xenalyze as well
This should be backported.
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
There have been report of stalls of guest vCPUs, when Credit2 was used.
It seemed like these vCPUs were not getting scheduled for very long
time, even under light load conditions (e.g., during dom0 boot).
Investigations led to the discovery that --although rarely-- it can
happen that a vCPU manages to run for very long timeslices. In Credit2,
this means that, when runtime accounting happens, the vCPU will lose a
large quantity of credits. This in turn may lead to the vCPU having less
credits than the idle vCPUs (-2^30). At this point, the scheduler will
pick the idle vCPU, instead of the ready to run vCPU, for a few
"epochs", which often times is enough for the guest kernel to think the
vCPU is not responding and crashing.
An example of this situation is shown here. In fact, we can see d0v1
sitting in the runqueue while all the CPUs are idle, as it has
-1254238270 credits, which is smaller than -2^30 = −1073741824:
(XEN) Runqueue 0:
(XEN) ncpus = 28
(XEN) cpus = 0-27
(XEN) max_weight = 256
(XEN) pick_bias = 22
(XEN) instload = 1
(XEN) aveload = 293391 (~111%)
(XEN) idlers: 00,00000000,00000000,00000000,00000000,00000000,0fffffff
(XEN) tickled: 00,00000000,00000000,00000000,00000000,00000000,00000000
(XEN) fully idle cores: 00,00000000,00000000,00000000,00000000,00000000,0fffffff
[...]
(XEN) Runqueue 0:
(XEN) CPU[00] runq=0, sibling=00,..., core=00,...
(XEN) CPU[01] runq=0, sibling=00,..., core=00,...
[...]
(XEN) CPU[26] runq=0, sibling=00,..., core=00,...
(XEN) CPU[27] runq=0, sibling=00,..., core=00,...
(XEN) RUNQ:
(XEN) 0: [0.1] flags=0 cpu=5 credit=-1254238270 [w=256] load=262144 (~100%)
We certainly don't want, under any circumstance, this to happen.
Let's, therefore, define a minimum amount of credits a vCPU can have.
During accounting, we make sure that, for however long the vCPU has
run, it will never get to have less than such minimum amount of
credits. Then, we set the credits of the idle vCPU to an even
smaller value.
NOTE: investigations have been done about _how_ it is possible for a
vCPU to execute for so much time that its credits becomes so low. While
still not completely clear, there are evidence that:
- it only happens very rarely,
- it appears to be both machine and workload specific,
- it does not look to be a Credit2 (e.g., as it happens when
running with Credit1 as well) issue, or a scheduler issue.
This patch makes Credit2 more robust to events like this, whatever
the cause is, and should hence be backported (as far as possible).
Reported-by: Glen <glenbarney@gmail.com>
Reported-by: Tomas Mozes <hydrapolic@gmail.com>
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
On Arm64, system registers are always 64-bit including SCTLR_EL1.
However, Xen is assuming this is 32-bit because earlier revision of
Armv8 had the top 32-bit RES0 (see ARM DDI0595.b).
>From Armv8.5, some bits in [63:32] will be defined and allowed to be
modified by the guest. So we would effectively reset those bits to 0
after each context switch. This means the guest may not function
correctly afterwards.
Rather than resetting to 0 the bits [63:32], preserve them across
context switch.
Note that the corresponding register on Arm32 (i.e SCTLR) is always
32-bit. So we need to use register_t anywhere we deal the SCTLR{,_EL1}.
Outside interface is switched to use 64-bit to allow ABI compatibility
between 32-bit and 64-bit.
[Stefano: fix typo in commit message]
Signed-off-by: Julien Grall <julien.grall@arm.com>
Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
Reviewed-by: Volodymyr Babchuk <volodymyr.babchuk@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Modified to provide building of the xenalyze binary for both ARM and
x86 platforms. The xenalyze binary is now built as part of the BIN
list for both platforms.
Signed-off-by: Benjamin Sanda <ben.sanda@dornerworks.com>
Signed-off-by: Julien Grall <julien.grall@arm.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
The per-cpu buffers for Xentrace are addressed by cpu-id, but the info
array for the buffers is sized only by number of online cpus. This
might lead to crashes when using Xentrace with smt=0.
The t_info structure has to be sized based on nr_cpu_ids.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Modify the xentrace utility to allow sparse cpu list resulting in not
all possible cpus having a trace buffer allocated.
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
xen-tools/libs.h currently contains a shared BUILD_BUG_ON() implementation and
is used by some tools. Extend this to include ARRAY_SIZE and clean up all the
opencoding.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
With gcc optimization enabled by:
tools: detect appropriate debug optimization level
b435014517
-Wformat-truncation throws warnings
gcc version 7.3.0
xenalyze.c: In function 'find_symbol':
xenalyze.c:382:36: error: 'snprintf' output may be truncated before the last format character [-Werror=format-truncation=]
snprintf(name, 128, "(%s +%llx)",
^
xenalyze.c:382:5: note: 'snprintf' output between 6 and 144 bytes into a destination of size 128
snprintf(name, 128, "(%s +%llx)",
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lastname, offset);
~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Signed-off-by: John Thomson <git@johnthomson.fastmail.com.au>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Recent changes in xenalyze introduced INT_MIN without also adding the
required header, fix this by adding the header.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Release-acked-by: Julien Grall <julien.grall@linaro.org>
This commit implements the Xen part of the cap mechanism for
Credit2.
A cap is how much, in terms of % of physical CPU time, a domain
can execute at most.
For instance, a domain that must not use more than 1/4 of
one physical CPU, must have a cap of 25%; one that must not
use more than 1+1/2 of physical CPU time, must be given a cap
of 150%.
Caps are per domain, so it is all a domain's vCPUs, cumulatively,
that will be forced to execute no more than the decided amount.
This is implemented by giving each domain a 'budget', and
using a (per-domain again) periodic timer. Values of budget
and 'period' are chosen so that budget/period is equal to the
cap itself.
Budget is burned by the domain's vCPUs, in a similar way to
how credits are.
When a domain runs out of budget, its vCPUs can't run any
longer. They can gain, when the budget is replenishment by
the timer, which event happens once every period.
Blocking the vCPUs because of lack of budget happens by
means of a new (_VPF_parked) pause flag, so that, e.g.,
vcpu_runnable() still works. This is similar to what is
done in sched_rtds.c, as opposed to what happens in
sched_credit.c, where vcpu_pause() and vcpu_unpause()
(which means, among other things, more overhead).
Note that, while adding new fields to csched2_vcpu and
csched2_dom, currently existing members are being moved
around, to achieve best placement inside cache lines.
Note also that xenalyze and tools/xentrace/format are being
updated too.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
DEPS_INCLUDE is currently the same as DEPS, so no functional change.
This patch is the result of this perl rune:
git-grep -l 'include.*DEPS' | xargs perl -i -pe 'next unless m/^-?include/; s/\bDEPS\b/DEPS_INCLUDE/'
I have verified that I haven't missed anything, with this rune:
git-grep '\bDEPS\b'
Reported-by: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
DEPS_RM is currently the same as DEPS, so no functional change.
This patch is the result of two perl runes:
git-grep -l 'rm.*DEPS' | xargs perl -i~ -pe 'next unless m/^\t+rm\b/; s/\bDEPS\b/DEPS_RM/;'
git-grep -l 'RM.*DEPS' | xargs perl -i~ -pe 'next unless m/^\t+\$\(RM\)/; s/\bDEPS\b/DEPS_RM/;'
And editing tools/xenstat/libxenstat/Makefile by hand.
I verified that I didn't miss anything with this rune:
git-grep '\bDEPS\b' | grep -v include |less
Reported-by: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Soft-affinity support is usually implemented by means
of a two step "balancing loop", where:
- during the first step, we consider soft-affinity
(if the vcpu has one);
- during the second (if we get to it), we consider
hard-affinity.
In runq_tickle(), we need to do that for checking
whether we can execute the waking vCPU on an pCPU
that is idle. In fact, we want to be sure that, if
there is an idle pCPU in the vCPU's soft affinity,
we'll use it.
If there are no such idle pCPUs, though, and we
have to check non-idle ones, we can avoid the loop
and to both hard and soft-affinity in one pass.
In fact, we can we scan runqueue and compute a
"score" for each vCPU which is running on each pCPU.
The idea is, since we may have to preempt someone:
- try to make sure that the waking vCPU will run
inside its soft-affinity,
- try to preempt someone that is running outside
of its own soft-affinity.
The value of the score is added to a trace record,
so xenalyze's code and tools/xentrace/formats are
updated accordingly.
Suggested-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Running "make uninstall" does not remove all installed files, a
situation which might cause link related issues if xen is re-installed
in a different location.
In order to make uninstall correctly remove the files it is best
the process should be done recursively by mirroring each "install"
target with an "uninstall" who removes the installed files.
An exception to this rule is uninstalling the files produced by
"qemu-xen-dir-remote" and "qemu-xen-traditional-dir", which are external
to the project. These projects do not implement an "uninstall" target so
the files have to be removed manually.
Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Including whether we actually tried stealing a vCPU from
a given pCPU, or we skipped that one, because of lock
contention.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
George Dunlap
Javi Merino
Olaf Hering
Alejandro Vallejo
Andrew Cooper
Juergen Gross
Anthony PERARD
Bertrand Marquis
Julien Grall
Manuel Bouyer
Dario Faggioli
Julien Grall
Benjamin Sanda
Ian Jackson
John Thomson
Roger Pau Monne
Dario Faggioli
Ian Jackson
George Dunlap
Petre Pircalabu
Alistair Francis