xen/livepatch: Make check_for_livepatch_work() faster in the common case

When livepatching is enabled, this function is used all the time.  Really do
check the fastpath first, and annotate it likely() as this is the right answer
100% of the time (to many significant figures).  This cuts out 3 pointer
dereferences in the "nothing to do path".

However, GCC still needs some help to persuade it not to set the full stack
frame (6 spilled registers, 3 slots of locals) even on the fastpath.

Create a new check_for_livepatch_work() with the fastpath only, and make the
"new" do_livepatch_work() noinline.  This causes the fastpath to need no stack
frame, making it faster still.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
This commit is contained in:
Andrew Cooper committed 2024-01-18 21:18:21 +00:00
1 parent 86d0edda34
commit c30021be30
1 file changed
+12 -5
+12 -5
View File
@@ -1693,7 +1693,7 @@ static int livepatch_spin(atomic_t *counter, s_time_t timeout,
* The main function which manages the work of quiescing the system and
* patching code.
*/
void check_for_livepatch_work(void)
static void noinline do_livepatch_work(void)
{
#define ACTION(x) [LIVEPATCH_ACTION_##x] = #x
static const char *const names[] = {
@@ -1711,10 +1711,6 @@ void check_for_livepatch_work(void)
!is_idle_domain(current->sched_unit->domain) )
return;
/* Fast path: no work to do. */
if ( !per_cpu(work_to_do, cpu ) )
return;
smp_rmb();
/* In case we aborted, other CPUs can skip right away. */
if ( !livepatch_work.do_work )
@@ -1864,6 +1860,17 @@ void check_for_livepatch_work(void)
}
}
void check_for_livepatch_work(void)
{
unsigned int cpu = smp_processor_id();
/* Fast path: no work to do. */
if ( likely(!per_cpu(work_to_do, cpu)) )
return;
do_livepatch_work();
}
/*
* Only allow dependent payload is applied on top of the correct
* build-id.
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.