When the variable is set, only build jobs that are used in tests are
scheduled. This saves a lot of time, for example when scheduling a test
pipeline with alternative linux branch.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
We'd rather have the full spelling in Kconfig. Adjusts every other
reference to the name too.
Not a functional change.
Signed-off-by: Alejandro Vallejo <alejandro.garciavallejo@amd.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
The selected jobs regex was assumed to be possibly empty in some places
and setting a default value for the regex caused the hardware jobs to
bypass some rules checks.
To fix this, adjust the rules to remove the check for empty and
instead just match the regex.
Fixes: 485ab1b5db
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
The original yocto config had a hidden default of allow_failure: true for the
manual job while the hidden default for rules is false. This causes the
stages with manual jobs to show as blocked.
Set it explicitly to true to avoid the analyze and build stages being shown as
blocked.
Fixes: 485ab1b5db
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Filtering jobs using the selected jobs regex is missing for
qemu-export + yocto jobs when running regular pipelines and eclair jobs
when running scheduled pipelines.
First, set a default value for the selected jobs regex to remove the
need to always check if the variable is empty.
Then add the missing rules to filter out those jobs.
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
With the exception of the custom IBT job, copy all Debian 12 jobs making
Debian 13 versions, then trim the Debian 12 ranconfig jobs.
Update the test jobs using Debian 12 to use 13.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Everything works fine with Debian 13. Provide two new build jobs (for a total
of 6), and update the test jobs.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Denis Mukhin <dmukhin@ford.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Everything works fine with Debian 13. Provide two new build jobs, and update
both the randconfig the test jobs.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Just like for RISCV and PPC, the earlier we enable the CI build the
better.
Signed-off-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
All the x86 hardware runners are out of date, to varying degrees, in terms of
microcode.
Microcode must be uncompressed and prepended to the initrd in order for Xen to
be able to use it during early boot.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Linux 6.6.56 was already added to test-artifacts for the argo testing, and
this removes one moving part while cleaning things up.
Drop the associated export job, and dockerfile.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Drop CentOS 7 entirely. It's way to old now.
Ubuntu 22.04 is the oldest Ubuntu with a suitable version of Clang, so swap
the 16.04 clang builds for 22.04.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Use more generic CONFIG_VM_EVENT name throughout Xen code instead of
CONFIG_MEM_ACCESS. This reflects the fact that vm_event is a higher level
feature, with mem_access & monitor depending on it.
Suggested-by: Tamas K Lengyel <tamas@tklengyel.com>
Acked-by: Tamas K Lengyel <tamas@tklengyel.com>
Signed-off-by: Sergiy Kibrik <Sergiy_Kibrik@epam.com>
There are two reasons for that:
1. In the README, GCC baseline is chosen to be 12.2, whereas Debian 11
uses GCC 10.2.1.
2. Xen requires mandatory some Z extensions, but GCC 10.2.1 does not
support Z extensions in -march, causing the compilation to fail.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Debugging sometimes involves running specific jobs on different
versions. It's useful to easily avoid running all of the not interesting
ones (for given case) to save both time and CI resources. Doing so used
to require changing the yaml files, usually in several places.
Ease this step by adding SELECTED_JOBS_ONLY variable that takes a regex.
Note that one needs to satisfy job dependencies on their own (for
example if a test job needs a build job, that specific build job
needs to be included too).
The variable can be specified via Gitlab web UI when scheduling a
pipeline, but it can be also set when doing git push directly:
git push -o ci.variable=SELECTED_JOBS_ONLY="/job1|job2/"
More details at https://docs.gitlab.co.jp/ee/user/project/push_options.html
The variable needs to include regex for selecting jobs, including
enclosing slashes.
A coma/space separated list of jobs to select would be friendlier UX,
but unfortunately that is not supported:
https://gitlab.com/gitlab-org/gitlab/-/issues/209904 (note the proposed
workaround doesn't work for job-level CI_JOB_NAME).
On the other hand, the regex is more flexible (one can select for
example all arm32 jobs).
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Also enable -fno-sanitize=alignment like x86 since support for unaligned
accesses is guaranteed by the ISA and the existing OPAL setup code
relies on it.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Shawn Anastasio <sanastasio@raptorengineering.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
RISC-V has less complicated headers, so update ubsan.c to pull in everything
it needs. Provide dump_execution_state(), and update the printk() message to
make it more obvious that it's an outstanding task.
As with commit 8ef2ac727e ("automation: enable UBSAN for debug tests"),
enable UBSAN in RISC-V testing too.
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
automation: enable UBSAN for debug tests
Enable CONFIG_UBSAN and CONFIG_UBSAN_FATAL for the ARM64 and x86_64
build jobs, with debug enabled, which are later used for Xen tests on
QEMU and/or real hardware.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
R-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
If randconfig enables coverage support the build times out due to GNU LD
taking too long. For the time being prevent coverage from being enabled in
clang randconfig job.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko<oleksii.kurochko@gmail.com>
... and hook it up for RISC-V and PPC.
On RISC-V at least, no combination of headers pulls in errno.h, so include it
explicitly.
Guard the hypercalls array declaration based on NR_hypercalls existing. This
is sufficient to get PERF_COUNTERS fully working on RISC-V and PPC, so drop
the randconfig override.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Oleksii Kurohcko <oleksii.kurochko@gmail.com>
Except setting CONFIG_QEMU_PLATFORM=y in tiny64_defconfig,
CONFIG_QEMU_PLATFORM should be fixed for RISC-V's randconfig job.
Otherwise, an expected compilation error for RISC-V's randconfig job
will occur since clean_and_invalidate_dcache_va_range() and
clean_dcache_va_range() are currently implemented only for the QEMU
platform.
Additionally, sort the EXTRA_FIXED_RANDCONFIG list alphabetically.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Fixes: f92e2709bd ("xen/riscv: implement data and instruction cache operations")
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
This allow to change the registry used for container in a single
place, and could be controlled via other mean.
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Rework the container to be non-root, use heredocs for readability, and
use apt-get --no-install-recommends to keep the size down. Rename the
job to x86_32, to be consistent with XEN_TARGET_ARCH and the
naming scheme of all the other CI jobs:
${VERSION}-${ARCH}-${BUILD_NAME}
Remove build dependencies for building QEMU. The absence of ninja/meson means
that the container hasn't been able to build QEMU in years.
Remove build dependencies for the documentation as we don't have to
build it for every single arch.
This reduces the size of the container from 2.22GB to 1.32Gb.
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Rework the container to use heredocs for readability, and use
apt-get --no-install-recommends to keep the size down.
This reduces the size of the (uncompressed) container from 3.44GB to
1.97GB.
The container is left running the builds and tests as root. A
subsequent patch will make the necessary changes to the test scripts
to allow test execution as a non-root user.
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Add x86_64 hardware test that creates a Xen Argo communication
connection between two PVH domains. In the test, dom0 creates a domU and
listens for messages sent by the domU through Argo.
To accomplish this, build Xen with CONFIG_ARGO=y and create a CI test job.
Update the xilinx x86_64 test script to support the new test, and add
"sync_console" to command line to avoid an issue with console messages
being lost.
Requested-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Victor Lira <victorm.lira@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Fedora 29 is long out of date. Move forward 5 years to Fedora 40.
Include all the usual improvements. Rework the container to be non-root, use
heredocs for legibility, and switch to the new naming scheme.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
We spend an awful lot of CI time building QEMU, even though most changes don't
touch the subset of tools/libs/ used by QEMU. Some numbers taken at a time
when CI was otherwise quiet:
With Without
Alpine: 13m38s 6m04s
Debian 12: 10m05s 8m10s
OpenSUSE Tumbleweed: 11m40s 7m54s
Ubuntu 24.04: 14m56s 8m06s
which is a >50% improvement in wallclock time in some cases.
The only build we have that needs QEMU is alpine-3.18-gcc-debug. This is the
build deployed and used by the QubesOS ADL-* and Zen3p-* jobs.
Xilinx-x86_64 deploys it too, but is PVH-only and doesn't use QEMU.
QEMU is also built by CirrusCI for FreeBSD (fully Clang/LLVM toolchain).
This should help quite a lot with Gitlab CI capacity.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
The Yocto jobs take a long time to run. We are changing Gitlab ARM64
runners and the new runners might not be able to finish the Yocto jobs
in a reasonable time.
For now, disable the Yocto jobs by turning them into "manual" trigger
(they need to be manually executed.)
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Upgrade Yocto to a newer version. Use ext4 as image format for testing
with QEMU on ARM and ARM64 as the default is WIC and it is not available
for our xen-image-minimal target.
Also update the tar.bz2 filename for the rootfs.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
The containers are exactly as per 20.04 (Focal). However, this now brings us
to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
The oldest and newest toolchains are the most likely to find problems with new
code, so reduce the middle 3 releases (18/20/22) to just a single smoke test
each.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
As with 16.04 (Xenial), with python3-setuptools included. Having this package
only in some containers was intentional; see commit bbc72a7877 ("automation:
Add python3's setuptools to some containers") for the rational.
This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 20.04-x86_64 1.06GB
registry.gitlab.com/xen-project/xen/ubuntu focal 1.57GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
As with 16.04 (Xenial), except that ninja-build package is available so QEMU
can be built.
This halves the size of the container:
registry.gitlab.com/xen-project/xen/ubuntu 18.04-x86_64 857MB
registry.gitlab.com/xen-project/xen/ubuntu bionic 1.44GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Rework the container to be non-root, use heredocs for legibility, and use
apt-get --no-install-recommends to keep the size down.
Ubuntu Xenial has no ninja-build package, so can't build the QEMU referenced
by Xen 4.16. Therefore, drop the dependencies too.
This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 16.04-x86_64 698MB
registry.gitlab.com/xen-project/xen/ubuntu xenial 1.21GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
See prior patch for most discussion.
Despite appearing to be a fixed release (and therefore not marked as permitted
failure), the dockerfile references the `leap` tag which is rolling in
practice. Switch to 15.6 explicitly, for better test stability.
Vs tumbleweed, use `zypper update` rather than dist-upgrade, and retain the
RomBIOS dependencies; bin86 and dev86.
In terms of size, this saves ~700M:
registry.gitlab.com/xen-project/xen/opensuse leap-15.6-x86_64 1.33GB
registry.gitlab.com/xen-project/xen/suse opensuse-leap 2.05GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Existing as suse:opensuse-tumbleweed is a historical quirk, and adjusted for
consistency with all the other containers.
Make it non-root, use heredocs for legibility, and use the zypper long names
for the benefit of those wondering what was being referenced or duplicated.
Trim the dependencies substantially. Testing docs isn't very interesting and
saves a lot of space. Other savings come from removing a huge pile of
optional QEMU dependencies (QEMU just needs to build the Xen parts to be
useful here, not have a full GUI environment).
Finally, there where some packages such as bc, libssh2-devel, libtasn1-devel
and nasm that I'm not aware of any reason to have had, even historically.
This shaves 1G off the size of the container:
registry.gitlab.com/xen-project/xen/opensuse tumbleweed-x86_64 1.39GB
registry.gitlab.com/xen-project/xen/suse opensuse-tumbleweed 2.35GB
Furthermore, identify which components of the build use which dependencies,
which will help managing them in the future.
Thanks to Olaf Hering for dependency fixes that have been subsumed into this
total overhaul.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The containers are both much smaller, with stable toolchains over time, and
this at least means we're not doing all testing with a single compiler.
Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH-*) and
reposition the jobs to optimise starting the smoke test.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
... in the style of debian:12-ppc64le.
Rename the jobs and reposition them later as they're not a dependency for the
smoke testing any more.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
qemu-system-ppc64/8.1.0-ppc64 was added because bullseye's QEMU didn't
understand the powernv9 machine. However bookworm's QEMU does and this is
preferable to maintaining a random build of QEMU ourselves.
Use the debian:12-ppc64le container and test the output of that build too.
Remove qemu-system-ppc64-8.1.0-ppc64-export which is unused now.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
... conforming to the new naming scheme; $DISTRO-$VERSION-$ARCH-* so the jobs
sort more coherently.
Make it non-root by default, and set XEN_TARGET_ARCH=ppc64. Include QEMU too,
which will be used subsequently.
Add build jobs too, with debian-12-ppc64le-gcc-debug specifically early as it
will be used for smoke testing shortly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Archlinux is a rolling distro. As a consequence, rebuilding the container
periodically changes the toolchain, and this affects all stable branches in
one go.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Marek Marczykowski-Górecki