Add additional stage in console output parsing - wait for first message
from Xen. The message is defined via BOOT_MSG variable. This has two
effects:
- distinguishes failing Xen to load at all from later test failures
- resets timeout when Xen starts loading
The latter is especially relevant for hardware tests where firmware +
network boot may take some time before Xen starts booting. The two-stage
timeout is more robust solution than increasing the overall timeout.
The issue has been observed on some dom0pvh-hvm jobs, at least on
runners hw3 and hw11. This patch is a first stage before qubes-x86-64.sh
is switched to use expect in the next stage.
While at it, consistently use 'expect -re' for all matches. This
especially allows matching newlines ("\n"), which will become relevant
in the next patch. And document variables used in console.exp.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
(cherry picked from commit 7c96a27e37)
This should have been dropped too, as the authoritative copy lives in another
repo now.
Fixes: babe11b46c ("CI: Drop alpine-3.18-rootfs-export and use test-artefacts")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
(cherry picked from commit 424d20afae)
Linux 6.6.56 was already added to test-artifacts for the argo testing, and
this removes one moving part while cleaning things up.
Drop the associated export job, and dockerfile.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
(cherry picked from commit f858837664)
Change how PCI device lookup is done to handle also USB devices, in
which case get the USB controller. Instead of taking basename of the
'device' symlink, resolve the full path (example:
/sys/devices/pci0000:00/0000:00:09.0/usb4/4-7/4-7:1.0) and take the
first part after pci0000:00. Theoretically it could be a bridge, but VM
has flat PCI topology.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 930279d96c)
The network driver initializes asynchronously, and it may not be ready
yet by the time the startup script is called. This is especially the
case for USB network adapter (where the PCI device is the USB
controller) in the upcoming runner.
Don't bother about separate timeout - test timeout will cover this part
too.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 6e3fe711fb)
And start collecting qemu log earlier, so it isn't lost in case of a
timeout during domain startup.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
(cherry picked from commit b8d41a9b22)
It fails on larger initramfs (~250MB one) and sometimes even smaller
depending on memory size/memory map, let Linux do it.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 32e6d052c2)
This is Intel i7-7567U in NUC 7i7BNH. This one is an older one, with no
firmware updates (last update from 2023) and no microcode udpates
either. While this firmware supports UEFI, network boot works only in
legacy mode - thus legacy is used here (via iPXE, instead of grub2.efi).
Testing legacy boot path may be a useful thing on its own.
Add the same set of tests as on ADL runner.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 911a766f14)
Hardware runners that use legacy boot use iPXE instead of grub2. Create
boot.ipxe for those too - with exact same options.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 42beecae0b)
While activating verbose logging simplifies debugging, this causes
GitLab logs to be truncated, preventing the links to the ECLAIR
analysis database to be shown.
No functional change.
Fixes: c4392ec832 ("automation: Add ECLAIR utilities and settings")
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 78f2a5792c)
Debugging sometimes involves running specific jobs on different
versions. It's useful to easily avoid running all of the not interesting
ones (for given case) to save both time and CI resources. Doing so used
to require changing the yaml files, usually in several places.
Ease this step by adding SELECTED_JOBS_ONLY variable that takes a regex.
Note that one needs to satisfy job dependencies on their own (for
example if a test job needs a build job, that specific build job
needs to be included too).
The variable can be specified via Gitlab web UI when scheduling a
pipeline, but it can be also set when doing git push directly:
git push -o ci.variable=SELECTED_JOBS_ONLY="/job1|job2/"
More details at https://docs.gitlab.co.jp/ee/user/project/push_options.html
The variable needs to include regex for selecting jobs, including
enclosing slashes.
A coma/space separated list of jobs to select would be friendlier UX,
but unfortunately that is not supported:
https://gitlab.com/gitlab-org/gitlab/-/issues/209904 (note the proposed
workaround doesn't work for job-level CI_JOB_NAME).
On the other hand, the regex is more flexible (one can select for
example all arm32 jobs).
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 9d419afa46)
There are a bunch of tests in tools/tests/, let them run in CI.
For each subdirectory expect "make run" will run the test, and observe
its exit code. This way, adding new tests is easy, and they will be
automatically picked up.
For better visibility, log test output to junit xml format, and let
gitlab ingest it. Set SUT_ADDR variable with name/address of the system
under test, so a network can be used to extract the file. The actual
address is set using DHCP. And for the test internal network, still add
the 192.168.0.1 IP (but don't replace the DHCP-provided one).
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <stefano.stabellini@amd.com>
(cherry picked from commit fc10d6ecdd)
This allow to change the registry used for container in a single
place, and could be controlled via other mean.
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit a231538acd)
It's unclear why -N is being used in the first place. It was added by
commit 4676bbf96d back in 2002 without any justification.
When building a PE image it's actually detrimental to forcefully set the
.text section as writable. The GNU LD man page contains the following
warning regarding the -N option:
> Note: Although a writable text section is allowed for PE-COFF targets, it
> does not conform to the format specification published by Microsoft.
Remove the usage of -N uniformly on all architectures, assuming that the
addition was simply done as a copy and paste of the original x86 linking
rune.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Julien Grall <jgrall@amazon.com>
master commit: d444763f8c
master date: 2025-04-23 16:12:25 +0200
There are several errata on Intel regarding the usage of the MONITOR/MWAIT
instructions, all having in common that stores to the monitored region
might not wake up the CPU.
Fix them by forcing the sending of an IPI for the affected models.
The Ice Lake issue has been reproduced internally on XenServer hardware,
and the fix does seem to prevent it. The symptom was APs getting stuck in
the idle loop immediately after bring up, which in turn prevented the BSP
from making progress. This would happen before the watchdog was
initialized, and hence the whole system would get stuck.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 4aae4452ef
master date: 2025-04-23 16:12:25 +0200
Add an early basic check, yielding the same error code as the more
thorough one the main handler would produce.
Fixes: b8a7efe852 ("Enable compatibility mode operation for HYPERVISOR_memory_op")
Reported-by: Manuel Andreas <manuel.andreas@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Jason Andryuk <jason.andryuk@amd.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 560c51be8f
master date: 2025-04-22 11:25:23 +0200
Like MOVS, INS, and OUTS, STOS also has a special purpose hook, where
the hook function may legitimately have the same expectation as to the
request not straddling address space start/end.
Fixes: 5dfe4aa4ee ("x86_emulate: Do not request emulation of REP instructions beyond the")
Reported-by: Fabian Specht <f.specht@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: 8c5636b6c8
master date: 2025-04-22 11:24:20 +0200
For the X86EMUL_EXCEPTION case the repeat count must be correctly
propagated back. Since for the recursive invocation we use a local
helper variable, its value needs copying to the caller's one.
While there also correct the off-by-1 range in the comment ahead of the
function (strictly speaking for the "DF set" case we'd need to put
another, different range there as well).
Fixes: 53f87c03b4 ("x86emul: generalize exception handling for rep_* hooks")
Reported-by: Manuel Andreas <manuel.andreas@tum.de>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: c07b16fd6e
master date: 2025-04-17 10:01:19 +0200
Given the following rangset operation:
{ [0, 1], [4, 5] } - { [3, 4] }
The current rangeset logic will output a rangeset:
{ [0, 2], [5, 5] }
This is incorrect, and also has the undesirable property of being bogus in
a way that the resulting rangeset is expanded.
Fix this by making sure the bounds are correctly checked before modifying
the previous range.
Fixes: 484a058c48 ('Add auto-destructing per-domain rangeset data structure...')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: e118fc98e7
master date: 2025-04-11 12:20:10 +0200
The order of items is stored in .*.chk.cmd, and hence variations between
how items are ordered would result in re-invocation of the checking rule
during "make install-xen" despite that already having successfully run
earlier on. The difference can become noticable when building (as non-
root) and installing (as root) use different GNU make versions: In 3.82
the sorting was deliberately undone, just for it to be restored in 4.3.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: ff835bbc80
master date: 2025-04-10 10:56:29 +0200
While building xen with GCC 14.2.1 with "-fcondition-coverage" option
or with "-Og", the compiler produces a false positive warning:
arch/x86/irq.c: In function ‘create_irq’:
arch/x86/irq.c:281:11: error: ‘desc’ may be used uninitialized [-Werror=maybe-uninitialized]
281 | ret = init_one_irq_desc(desc);
| ^~~~~~~~~~~~~~~~~~~~~~~
arch/x86/irq.c:269:22: note: ‘desc’ was declared here
269 | struct irq_desc *desc;
| ^~~~
cc1: all warnings being treated as errors
make[2]: *** [Rules.mk:252: arch/x86/irq.o] Error 1
While we have signed/unsigned comparison both in "for" loop and in
"if" statement, this still can't lead to use of uninitialized "desc",
as either loop will be executed at least once, or the function will
return early. So this is a clearly false positive warning due to a
bug [1] in GCC.
Initialize "desc" with NULL to make GCC happy.
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119665
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
master commit: 7a4484d90b
master date: 2025-04-08 09:40:39 +0200
CPUID leaf 0x2 emits one-byte descriptors in its four output registers
EAX, EBX, ECX, and EDX. For these descriptors to be valid, the most
significant bit (MSB) of each register must be clear.
Leaf 0x2 parsing at intel.c only validated the MSBs of EAX, EBX, and
ECX, but left EDX unchecked.
Validate EDX's most-significant bit as well.
Fixes: 1aa6feb63b ("Port CPU setup code from Linux 2.6")
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://lore.kernel.org/r/20250304085152.51092-3-darwi@linutronix.de
Use ARRAY_SIZE() though.
Origin: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 1881148215c6
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
master commit: a47b44a8f0
master date: 2025-04-08 09:37:38 +0200
A privileged domain can issue XEN_DOMCTL_vm_event_op with
op->domain == DOMID_INVALID. In this case vm_event_domctl()
function will get NULL as the first parameter and this will
cause hypervisor panic, as it tries to derefer this pointer.
Fix the issue by checking if valid domain is passed in.
Fixes: 48b8424945 ("xen/vm-event: Drop unused u_domctl parameter from vm_event_domctl()")
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
master commit: 6a884750f3
master date: 2025-04-08 09:36:38 +0200
The following sequence of events may lead a debug build of Xen to crash
when using the null scheduler:
1. domain creation (e.g. d1) failed due to bad configuration
2. complete_domain_destroy() was deferred
3. domain creation (e.g. d2) succeeds
At this point, d2 is running, while the zombie d1 is not fully cleaned
up:
(XEN) Online Cpus: 0-3
(XEN) Cpupool 0:
(XEN) Cpus: 0-3
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Scheduler: null Scheduler (null)
(XEN) cpus_free = 3
(XEN) Domain info:
(XEN) Domain: 0
(XEN) 1: [0.0] pcpu=0
(XEN) 2: [0.1] pcpu=1
(XEN) Domain: 1
(XEN) 3: [1.0] pcpu=2
(XEN) Domain: 2
(XEN) 4: [2.0] pcpu=2
4. complete_domain_destroy() gets called for d1 and triggers the
following:
(XEN) Xen call trace:
(XEN) [<00000a0000322ed4>] null.c#unit_deassign+0x2d8/0xb70 (PC)
(XEN) [<00000a000032457c>] null.c#null_unit_remove+0x670/0xba8 (LR)
(XEN) [<00000a000032457c>] null.c#null_unit_remove+0x670/0xba8
(XEN) [<00000a0000336404>] sched_destroy_vcpu+0x354/0x8fc
(XEN) [<00000a0000227324>] domain.c#complete_domain_destroy+0x11c/0x49c
(XEN) [<00000a000029fbd0>] rcupdate.c#rcu_do_batch+0x94/0x3d0
(XEN) [<00000a00002a10c0>] rcupdate.c#__rcu_process_callbacks+0x160/0x5f4
(XEN) [<00000a00002a1e60>] rcupdate.c#rcu_process_callbacks+0xcc/0x1b0
(XEN) [<00000a00002a3460>] softirq.c#__do_softirq+0x1f4/0x3d8
(XEN) [<00000a00002a37c4>] do_softirq+0x14/0x1c
(XEN) [<00000a0000465260>] traps.c#check_for_pcpu_work+0x30/0xb8
(XEN) [<00000a000046bb08>] leave_hypervisor_to_guest+0x28/0x198
(XEN) [<00000a0000409c84>] entry.o#guest_sync_slowpath+0xac/0xd8
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion 'npc->unit == unit' failed at common/sched/null.c:383
(XEN) ****************************************
Fix by skipping unit_deassign() when the unit to be removed does not
match the pcpu's currently assigned unit.
Fixes: c2eae2614c ("sched/null: avoid crash after failed domU creation")
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
master commit: 54fe207f29
master date: 2025-04-07 12:17:31 +0200
Unlike stated in the offending commit's description,
load_system_tables() wasn't the only thing left to retain from the
earlier restore_rest_processor_state(). Note that MTRR state was still
reloaded via mtrr_aps_sync_end(), but that happens quite a bit later in
the resume process.
While there also do Misra-related tidying for the function itself: The
function being used from assembly only means it doesn't need to have a
declaration, but wants to be asmlinkage.
Fixes: 4304ff420e ("x86/S3: Drop {save,restore}_rest_processor_state() completely")
Reported-by: Roger Pau Monné <roger.pau@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
master commit: 0414dedd6f
master date: 2025-04-03 09:39:13 +0200
AMD have updated the SB-7033 advisory to include Zen5 CPUs. Extend the digest
check to cover Zen5 too.
In practice, cover everything until further notice.
Observant readers may be wondering where the update to the digest list is. At
the time of writing, no Zen5 patches are available via a verifiable channel.
Link: https://www.amd.com/en/resources/product-security/bulletin/amd-sb-7033.html
Fixes: 630e8875ab ("x86/ucode: Perform extra SHA2 checks on AMD Fam17h/19h microcode")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit b63951467e)
x86/ucode: Extend warning about disabling digest check too
This was missed by accident.
Fixes: b63951467e ("x86/ucode: Extend AMD digest checks to cover Zen5 CPUs")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 59bb316ea8)
A future change will need to calculate SHA2-256 digests. Introduce an
implementation in lib/, derived from Trenchboot which itself is derived from
Linux.
In order to be useful to other architectures, it is careful with endianness
and misaligned accesses as well as being more MISRA friendly, but is only
wired up for x86 in the short term.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
(cherry picked from commit 372af52441)
Xen: CI fix from XSN-2
* Add U suffix to the K[] table to fix MISRA Rule 7.2 violations.
Fixes: 372af52441 ("xen/lib: Introduce SHA2-256")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 15fe2eb5f1)
It makes no sense to request for preprocessor only output and also request
object file generation. Fix the _libxl.api-for-check target to only use
-E (preprocessor output).
Also Clang 20.0 reports an error if both options are used.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: 2862bf5b6c ('libxl: enforce prohibitions of internal callers')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Anthony PERARD <anthony.perard@vates.tech>
(cherry picked from commit a235f856e4)
The Eclair runners in GitlabCI have been update. This service is now
included, and redefining results in an error.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
(cherry picked from commit a43b0a770b)
This is AMD Zen2 (Ryzen 5 4500U specifically), in a HP Probook 445 G7.
This one has working S3, so add a test for it here.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
(cherry picked from commit debe8bf537)
Introduce a basic set of smoke tests using the XTF selftest image, and run
them on QEMU. Use the matrix keyword to create a different task for each
XTF flavor on each FreeBSD build.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Oleksii Kurochko<oleksii.kurochko@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 7973cba4db)
In preparation for adding some smoke tests that will consume those outputs.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit c9de0e2ce4)
In preparation for using the XTF selftests to smoke test the FreeBSD based
Xen builds.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit a92c0a9888)
Move the current logic to use the matrix keyword to generate a task for
each version of FreeBSD we want to build Xen on. The matrix keyword
however cannot be used in YAML aliases, so it needs to be explicitly used
inside of each task, which creates a bit of duplication. At least abstract
the FreeBSD minor version numbers to avoid repetition of image names.
Note that the full build uses matrix over an env variable instead of using
it directly in image_family. This is so that the alias can also be set
based on the FreeBSD version, in preparation for adding further tasks that
will depend on the full build having finished.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit b548a7dc4b)
Instead use env to find the location of expect.
Additionally do not use the -f flag, as it's only meaningful when passing
arguments on the command line, which we never do for console.exp. From the
expect 5.45.4 man page:
> The -f flag prefaces a file from which to read commands from. The flag
> itself is optional as it is only useful when using the #! notation (see
> above), so that other arguments may be supplied on the command line.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
In case the build fails or gets stuck, store the Kconfig file ahead of
starting the build.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 421c2bd58f)
Such timestamps can still be disabled from the Web UI using a tick box.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
(cherry picked from commit 1256159f3c)
Marek Marczykowski-Górecki
Andrew Cooper
Nicola Vetrini
Stefano Stabellini
Stefano Stabellini
Anthony PERARD
Roger Pau Monné
Jan Beulich
Volodymyr Babchuk
Ahmed S. Darwish
Stewart Hildebrand