Wire the per-cpupool admission-control switch through libxl
to xl sched-rtds, and document it.
Add -s/--schedparam to list or set pool-wide RTDS scheduler
parameters, and -a/--admission to enable or disable admission
control for a cpupool ("-c pool -s -a 0/1").
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
Add a per-cpupool on/off switch for RTDS admission control via
XEN_SYSCTL_scheduler_op, enabled by default, so an operator can
disable/enable enforcement for a pool
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Print a cpupool's admitted utilization against its capacity in the
'r' debug ky (xl debug-keys r), so an operator can see the total
usage of a cpupool.
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Now that we have introduced admission control for new and
removed units. Extend it to XEN_DOMCTL_SCHEDOP_putinfo and
putvcpuinfo, so growing an existing reservation via
xl sched-rtds is checked too.
putinfo sets the same (period, budget) for every unit of a
domain at once, so it tests the whole domain's utilization
delta atomically, rather than unit-by-unit, which could
spuriously reject an overall-acceptable change depending on
iteration order.
putvcpuinfo changes one unit at a time, so it just calls
rt_try_set_utilization() directly.
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
RTDS has no admission control: nothing stops the sum of all admitted
units' (budget/period) reservations in a cpupool from exceeding
what its pCPUs can actually provide. Once that happens, none of the
EDF deadline guarantees this scheduler is built around still hold
for the units sharing that pool.
Introduce admission control to prevent this: reject a reservation
whenever admitting it would push a cpupool's units over its capacity.
Track a running utilization total per cpupool, and enforce it in
rt_alloc_udata()/rt_free_udata(), the paired lifecycle hooks for a
unit's creation and destruction. This catches the default
period/budget every new unit gets.
Utilization is represented as a fixed-point value: budget is
left shifted by RTDS_UTIL_SHIFT (20 bits) and divided by period.
A plain "(budget << RTDS_UTIL_SHIFT) / period" risks overflowing
the multiply for large enough budgets. Rather than widen the
arithmetic to tolerate any input, the input itself is bounded:
rt_validate_params() rejects any budget above RTDS_MAX_BUDGET,
chosen as the largest value that can be left-shifted by
RTDS_UTIL_SHIFT without overflowing 64 bits, so the shift in
rt_unit_utilization() can never overflow.
A cpupool's capacity rt_utilization_cap() scales with the number of
scheduling resources in it. It is calculated as:
(number of sched_resources * RTDS_UTIL_SCALE * RTDS_UTIL_CAP_PCT / 100),
where RTDS_UTIL_CAP_PCT controls how much of that capacity can
actually be reserved; at 100% (its current value), all of it can be.
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
sched_move_domain() derives the number of units to rebuild from
d->max_vcpus, which is fixed at domain creation and never rolled
back if vcpu_create() fails partway through building a domain. So
d->vcpu[i] can be NULL for some i even though max_vcpus still
counts it - this happens if sched_alloc_udata() returns NULL.
The per-unit loop doesn't check for this: it sets
unit->vcpu_list = d->vcpu[unit_id] (NULL) and hands that broken
unit straight to the destination scheduler's alloc_udata(),
which assumes vcpu_list is always valid and crashes Xen when
it is not.
Reproduced by building a domain in a non-default cpupool where
vcpu creation fails partway through, then destroying it.
domain_kill() moves the domain back to the default cpupool via
sched_move_domain() before actually destroying it, crashing
inside the destination scheduler's alloc_udata() (seen in
Credit2's csched2_alloc_udata() -> is_idle_unit() -> NULL deref).
Before building a unit in sched_move_domain(), check whether all
vpcu slots belonging to that unit are populated. If any of its
vpcus is missing:
- For a dying domain, skip the unit allocation.
- For an active domain, abort the move and return -EINVAL to
prevent running with dropped vCPUs.
Fixes: 70fadc4163 ("xen/cpupool: support moving domain between cpupools with different granularity")
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
sched_init_vcpu() calls init_timer() for a vcpu's periodic_timer,
singleshot_timer and poll_timer before it can fail -- these
become live, linked into their target pCPU's per-cpu timer list
regardless of what happens next. If the sched_alloc_udata() call
further down then fails, the function frees the sched_unit via
sched_free_unit() and returns 1, but never unlinks these three
timers.
The caller, vcpu_create(), makes this worse: on sched_init_vcpu()
returning nonzero it jumps to fail_wq, skipping fail_sched and
thus sched_destroy_vcpu() -- the only function on this path that
calls kill_timer() on them. vcpu_destroy() then frees the vcpu,
and the three timers embedded in it, while they are still linked
into that shared list.
This silently corrupts that list. It only shows up later, when
something else touches a neighboring timer: sched_move_domain()
crashed with "Assertion 'entry->prev->next == entry' failed" on a
completely unrelated, valid vcpu's timer.
Call sched_destroy_vcpu() in sched_init_vcpu()'s own failure
branch instead of sched_free_unit(), so it doesn't depend
on the caller reaching sched_destroy_vcpu() to undo what it set
up itself. sched_destroy_vcpu() assumes unit->priv is set, which
is not the case here, so make it only free the udata and remove
the unit if unit->priv in non-NULL.
Fixes: d884b10778 ("Domain creation/destruction cleanups.")
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
struct scheduler currently serves two purposes: it is the static
vtable a scheduler backend defines (name, opt_name, sched_id, and
all its function pointers), and it is also the per-cpupool runtime
object scheduler_alloc() allocates. Being the same type forces
scheduler_alloc() to memcpy() the whole vtable into a fresh
allocation per cpupool, duplicating identical function pointers
across every cpupool using the same scheduler.
Split the vtable out into its own type, struct sched_ops, so it
can be shared by every cpupool using a given scheduler instead of
copied per cpupool. struct scheduler is left holding only what is
actually per-instance: a pointer to the shared sched_ops, plus
sched_data and cpupool. scheduler_alloc() now stores a pointer to
the matching sched_ops instance instead of copying its fields and
uses xzalloc() to zero-initialize the struct. Every accessor in
private.h is updated from s->field to s->ops->field to match.
Every in-tree scheduler backend (credit, credit2, rtds, arinc653,
null) is converted from struct scheduler to struct sched_ops.
Also drop the generic comment in arinc653.c.
A handful of call sites elsewhere read a scheduler's name,
opt_name or sched_id directly and are updated to go through
->ops as well.
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Several files in xen/common/sched still reference schedule.c in
their comments, which was the original name of xen/common/sched/core.c
before commit 6cb4b01c03 ("xen/sched: move schedulers and cpupool
coding to dedicated directory") renamed and moved it. Some comments
also reference functions that no longer exist.
Rework the comments to remove stale file and function references.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Acked-by: Juergen Gross <jgross@suse.com>
Commit 6cb4b01c03 ("xen/sched: move schedulers and cpupool coding to
dedicated directory") moved the scheduler files from xen/common/ to
xen/common/sched and renamed some of them, but did not update the
in-file path comments at the top of each file.
Remove the comments rather than updating them, as they are redundant
information already conveyed by the filename itself.
Suggested-by: Roger Pau Monné <roger.pau@citrix.com>
Suggested-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Furkan Caliskan <frn1furkan10@gmail.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Furkan Caliskan