Add test runner script qemu-xtf.sh which is allows any XTF x86 test to be
easily executed. Test runner is invoked from the qemu-smoke* jobs with the
hardcoded parameters.
Each x86 XTF job lead time is reduced a bit since only the test-related code
is built, not the entire XTF project.
Add .gitignore to avoid committing test artifacts by mistake.
Signed-off-by: Denis Mukhin <dmukhin@ford.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
This also moves executables too.
I'm not sure why xilinx-smoke-dom0-x86_64.sh was overriding PATH too, as
/usr/local is clearly in PATH given the other tests, but drop that too.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
All the x86 hardware runners are out of date, to varying degrees, in terms of
microcode.
Microcode must be uncompressed and prepended to the initrd in order for Xen to
be able to use it during early boot.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
The argo artefact is now a cpio archive and can be appended to dom?-rootfs
directly, rather than being part of of the overlay. Drop the
copy_dom?_files() functions.
xen-argo.ko has moved into the standard location, so update the insmod path.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Use the new test-artifacts which provide rootfs.cpio.gz rather than
initrd.tar.gz. rootfs.cpio.gz also has all the necessary top-level
directories, and includes the rc_verbose setting, so these modifications can
be dropped.
Having that, do not repack the whole initrd, but only pack modified
files and rely on Linux handling of concatenated archives.
This allows packing just test-related files (which includes the whole
toolstack), instead of the whole initrd.
For xilinx-smoke-dom0-x86_64.sh, this involves instructing grub not to unzip
the archive, as doing so corrupts it.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Right now, we have initrd.cpio.gz as domU, and initrd.tar.gz as the base for
dom0.
Rename initrd.cpio.gz to domU-rootfs.cpio.gz, and xen-rootfs.cpio.gz to
dom0-rootfs.cpio.gz to make it clearer which is which. Rename the VM from
test to domU.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
The previous attempt has correct diagnosis, but added -notransfer flag
in a wrong place - it should be used in the first (outer) match out of
two, not the second (inner) one.
Fixes: 1e12cbd6af ("CI: fix waiting for final test message")
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Upcoming changes won't unpack original rootfs anymore, so sed on
existing file cannot be used. Override the whole file instead.
No functional change intended.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
It appears as sometimes it takes more time for Xen even start booting,
mostly due to firmware and fetching large boot files by grub. In some
jobs the current timeout is pretty close to the actual time needed, and
sometimes (rarely for now) test fails due to timeout expiring in the
middle of dom0 booting. This will be happening more often if the
initramfs will grow (and with more complex tests).
This has been observed on some dom0pvh-hvm jobs, at least on runners hw3
and hw11.
Switch to using expect (console.exp) for more robust test output
handling. This allows waiting separately for Xen starting to boot and
then for the test to complete. For now, set both of those to 120s, which
pessimistically bumps timeout for the whole test to 240s (from 120s).
Add S3 handling to console.exp via SUSPEND_MSG + WAKEUP_CMD.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Expect normally discards initial part of its buffer after matching the
patter, before looking for the next one. If both PASSED and LOG_MSG
happen to be in the buffer at the same time, depending on their order,
only one will be matched and the waiting for the other will timeout.
Example expect -d output of this happening (parts eclipsed for brevity):
expect: does "\r\r\r\nWelcome to Alpine Linux 3.18\r\r\r\n...\r\r\r\r\n(domU) + echo 'pci test passed'\r\r\r\r\n(domU) pci test passed\r\r\r\r..." (spawn_id exp4) match regular expression "pci test passed"? Gate "pci test passed"? gate=yes re=yes
...
Gate keeper glob pattern for '\nWelcome to Alpine Linux' is '
Welcome to Alpine Linux'. Activating booster.
expect: does "'\r\r\r\r\n(domU) pci test passed\r\r\r\r\n(domU) [ ok ]\r\r\r\r\n(domU) [ ok ]\r\r\r\r\n(domU) \r\r\r\r\r\n(domU) domU Welcome to Alpine Linux 3.18\r\r\r\r\n(domU) \rKernel 6.6.56 on an x86_64 (/dev/hvc0)\r\r\r\r\n(domU) \r\r\r\r\r\n" (spawn_id exp4) match regular expression "\nWelcome to Alpine Linux"? Gate "\nWelcome to Alpine Linux"? gate=no
Fix this by using -notransfer flag to keep matched part in the buffer.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Andrew Cooper <andrew.cooper3@citix.com>
Add additional stage in console output parsing - wait for first message
from Xen. The message is defined via BOOT_MSG variable. This has two
effects:
- distinguishes failing Xen to load at all from later test failures
- resets timeout when Xen starts loading
The latter is especially relevant for hardware tests where firmware +
network boot may take some time before Xen starts booting. The two-stage
timeout is more robust solution than increasing the overall timeout.
The issue has been observed on some dom0pvh-hvm jobs, at least on
runners hw3 and hw11. This patch is a first stage before qubes-x86-64.sh
is switched to use expect in the next stage.
While at it, consistently use 'expect -re' for all matches. This
especially allows matching newlines ("\n"), which will become relevant
in the next patch. And document variables used in console.exp.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Change how PCI device lookup is done to handle also USB devices, in
which case get the USB controller. Instead of taking basename of the
'device' symlink, resolve the full path (example:
/sys/devices/pci0000:00/0000:00:09.0/usb4/4-7/4-7:1.0) and take the
first part after pci0000:00. Theoretically it could be a bridge, but VM
has flat PCI topology.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
The network driver initializes asynchronously, and it may not be ready
yet by the time the startup script is called. This is especially the
case for USB network adapter (where the PCI device is the USB
controller) in the upcoming runner.
Don't bother about separate timeout - test timeout will cover this part
too.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
And start collecting qemu log earlier, so it isn't lost in case of a
timeout during domain startup.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
It fails on larger initramfs (~250MB one) and sometimes even smaller
depending on memory size/memory map, let Linux do it.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Hardware runners that use legacy boot use iPXE instead of grub2. Create
boot.ipxe for those too - with exact same options.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
By default, the `aia` option is set to "none" which selects the SiFive PLIC for
handling wired interrupts. However, since PLIC is now considered obsolete and
will not be supported by Xen now, APLIC and IMSIC are selected instead to manage
both wired interrupts and MSIs.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Instead use env to find the location of expect.
Additionally do not use the -f flag, as it's only meaningful when passing
arguments on the command line, which we never do for console.exp. From the
expect 5.45.4 man page:
> The -f flag prefaces a file from which to read commands from. The flag
> itself is optional as it is only useful when using the #! notation (see
> above), so that other arguments may be supplied on the command line.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
There are two reasons for that:
1. In the README, GCC baseline is chosen to be 12.2, whereas Debian 11
uses GCC 10.2.1.
2. Xen requires mandatory some Z extensions, but GCC 10.2.1 does not
support Z extensions in -march, causing the compilation to fail.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
There are a bunch of tests in tools/tests/, let them run in CI.
For each subdirectory expect "make run" will run the test, and observe
its exit code. This way, adding new tests is easy, and they will be
automatically picked up.
For better visibility, log test output to junit xml format, and let
gitlab ingest it. Set SUT_ADDR variable with name/address of the system
under test, so a network can be used to extract the file. The actual
address is set using DHCP. And for the test internal network, still add
the 192.168.0.1 IP (but don't replace the DHCP-provided one).
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Stefano Stabellini <stefano.stabellini@amd.com>
With the recent fixes, Dom0less direct mapped domains can use PV
drivers. Extend the existing static-mem test with a PV network ping
tests.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Rework the container to be non-root, use heredocs for readability, and
use apt-get --no-install-recommends to keep the size down. Rename the
job to x86_32, to be consistent with XEN_TARGET_ARCH and the
naming scheme of all the other CI jobs:
${VERSION}-${ARCH}-${BUILD_NAME}
Remove build dependencies for building QEMU. The absence of ninja/meson means
that the container hasn't been able to build QEMU in years.
Remove build dependencies for the documentation as we don't have to
build it for every single arch.
This reduces the size of the container from 2.22GB to 1.32Gb.
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Rework the container to use heredocs for readability, and use
apt-get --no-install-recommends to keep the size down.
This reduces the size of the (uncompressed) container from 3.44GB to
1.97GB.
The container is left running the builds and tests as root. A
subsequent patch will make the necessary changes to the test scripts
to allow test execution as a non-root user.
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Add x86_64 hardware test that creates a Xen Argo communication
connection between two PVH domains. In the test, dom0 creates a domU and
listens for messages sent by the domU through Argo.
To accomplish this, build Xen with CONFIG_ARGO=y and create a CI test job.
Update the xilinx x86_64 test script to support the new test, and add
"sync_console" to command line to avoid an issue with console messages
being lost.
Requested-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Right now, various blocks rewrite domU_config= as a whole, even though it is
largely the same.
* dom0pvh-hvm does nothing but change the domain type to hvm
* *-pci sets the domain type, clears vif=[], appends earlyprintk=xen to the
cmdline, and adds some PCI config.
Refactor this to be domU_type (defaults to pvh), domU_vif (defaults to
xenbr0), and domU_extra_config (defaults to empty) and use these variables to
build domU_config= once.
Of note, the default domU_config= now sets cmdline=, and extra= is intended
for inclusion via domU_extra_config as necessary.
No practical change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
* List all the test_variants and summerise what's going on
* Use case rather than an if/else chain for $test_variant
* Adjust indentation inside the case block
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Fedora 29 is long out of date. Move forward 5 years to Fedora 40.
Include all the usual improvements. Rework the container to be non-root, use
heredocs for legibility, and switch to the new naming scheme.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
We spend an awful lot of CI time building QEMU, even though most changes don't
touch the subset of tools/libs/ used by QEMU. Some numbers taken at a time
when CI was otherwise quiet:
With Without
Alpine: 13m38s 6m04s
Debian 12: 10m05s 8m10s
OpenSUSE Tumbleweed: 11m40s 7m54s
Ubuntu 24.04: 14m56s 8m06s
which is a >50% improvement in wallclock time in some cases.
The only build we have that needs QEMU is alpine-3.18-gcc-debug. This is the
build deployed and used by the QubesOS ADL-* and Zen3p-* jobs.
Xilinx-x86_64 deploys it too, but is PVH-only and doesn't use QEMU.
QEMU is also built by CirrusCI for FreeBSD (fully Clang/LLVM toolchain).
This should help quite a lot with Gitlab CI capacity.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
A recent update to qemu-xen has bumped the build requirements, with Python 3.8
being the new baseline but also needing the 'ensurepip' and 'tomllib/tomli'
packages.
* Ubuntu/Debian package 'ensurepip' separately, but it can be obtained by
installing the python3-venv package.
* 'tomllib' was added to the python standard library in Python 3.11, but
previously it was a separate package named 'tomli'.
In terms of changes required to build QEMU:
* Ubuntu 24.04 (Noble) has Python 3.12 so only needs python3-venv
* Ubuntu 22.04 (Jammy) has Python 3.10 but does have a python3-tomli package
that QEMU is happy with.
* FreeBSD has Python 3.9, but Python 3.11 is available.
In terms of exclusions:
* Ubuntu 20.04 (Focal) has Python 3.8, but lacks any kind of tomli package.
* Fedora 29 (Python 3.7), OpenSUSE Leap 15.6 (Python 3.6), and Ubuntu
18.04/Bionic (Python 3.6) are now too old.
Detecting tomllib/tomli is more than can fit in build's one-liner, so break it
out into a proper script.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Check if xen.efi is bootable with an XTF dom0.
The multiboot2+EFI path is tested on hardware tests already.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
TEST_TIMEOUT is set as a CI/CD project variable, as it should be, to
match the capability and speed of the testing infrastructure.
As it turns out, TEST_TIMEOUT defined in test.yaml cannot override
TEST_TIMEOUT defined as CI/CD project variable. As a consequence, today
the TEST_TIMEOUT setting in test.yaml for the Xilinx jobs is ignored.
Instead, rename TEST_TIMEOUT to TEST_TIMEOUT_OVERRIDE in test.yaml and
check for TEST_TIMEOUT_OVERRIDE first in console.exp.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
MISRA C Rule 18.2 states: "Subtraction between pointers shall
only be applied to pointers that address elements of the same array".
Subtractions between pointer where at least one symbol is a
symbol defined by the linker are safe and thus deviated, because
the compiler cannot exploit the undefined behaviour that would
arise from violating the rules in this case.
To create an ECLAIR configuration that contains the list of
linker-defined symbols, the script "linker-symbols.sh" is used
after a build of xen (without static analysis) is performed.
The generated file "linker_symbols.ecl" is then used as part of the
static analysis configuration.
Additional changes to the ECLAIR integration are:
- perform a build of xen without static analysis during prepare.sh
- run the scripts to generated ECL configuration during the prepare.sh,
rather than analysis.sh
- export ECLAIR_PROJECT_ROOT earlier, to allow such generation
Additionally, the macro page_to_mfn performs a subtraction that is safe,
so its uses are deviated.
No functional changes.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
The test showed unreliable behavior due to unsupported console settings.
Update the baud rate used to connect to the UART.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Fixes: 95764a0817 (automation: update xilinx test scripts (tty))
This patch introduced a CI failure due to a timeout in xilinx-x86_64 test.
Change xilinx-x86_64 and xilinx-arm64 scripts to use "expect" utility
to determine test result and allow early exit from tests.
Add "expect" to xilinx container environment (dockerfile).
Rename references to "QEMU" in "qemu-key.exp" expect script to "TEST" to be
used by both QEMU and hardware tests.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Fix flaw in qemu-*.sh tests that producess a false success. The following
lines produces success despite the "expect" script producing nonzero exit
status:
set +e
...
./automation/scripts/qemu-key.exp | sed 's/\r\+$//'
(end of file)
The default exit status for a pipeline using "|" operator is that of the
rightmost command. Fix this by setting the "pipefail" option in the shell,
and removing "set +e" allowing the expect script to determine the result.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Update serial device names from ttyUSB* to test board specific names.
Update xilinx-smoke-dom0-x86_64 with new Xen command line console options,
which are now set as Gitlab CI/CD variables. Abstract the directory where
binaries are stored. Increase the timeout to match new setup.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
After commit c36efb7fce ("automation: use expect to run QEMU") we lost
the \r filtering introduced by b576497e3b ("automation: remove CR
characters from serial output"). This patch reintroduced it.
Fixes: c36efb7fce ("automation: use expect to run QEMU")
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
The expectation is that QEMU_TIMEOUT should be set as a Gitlab CI/CD
variable but if not we should be able to run the pipeline anyway.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Use expect to invoke QEMU so that we can terminate the test as soon as
we get the right string in the output instead of waiting until the
final timeout.
For timeout, instead of an hardcoding the value, use a Gitlab CI
variable "QEMU_TIMEOUT" that can be changed depending on the latest
status of the Gitlab CI runners.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
The "eclair" script calls action_push.sh even for merge request, while
instead action_pull_request.sh should be called, resulting in a job
failure with this error:
Unexpected event pull_request
Fix the script to call action_pull_request.sh appropriately.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Simone Ballarin <simone.ballarin@bugseng.com>
Add a test script and related job for running x86_64 dom0 tests.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Upgrade Yocto to a newer version. Use ext4 as image format for testing
with QEMU on ARM and ARM64 as the default is WIC and it is not available
for our xen-image-minimal target.
Also update the tar.bz2 filename for the rootfs.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
The containers are exactly as per 20.04 (Focal). However, this now brings us
to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
The oldest and newest toolchains are the most likely to find problems with new
code, so reduce the middle 3 releases (18/20/22) to just a single smoke test
each.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Denis Mukhin