Use expect to invoke QEMU so that we can terminate the test as soon as
we get the right string in the output instead of waiting until the
final timeout.
For timeout, instead of an hardcoding the value, use a Gitlab CI
variable "QEMU_TIMEOUT" that can be changed depending on the latest
status of the Gitlab CI runners.
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
master commit: c36efb7fce
master date: 2024-08-09 23:59:20 -0700
The Yocto jobs take a long time to run. We are changing Gitlab ARM64
runners and the new runners might not be able to finish the Yocto jobs
in a reasonable time.
For now, disable the Yocto jobs by turning them into "manual" trigger
(they need to be manually executed.)
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
master commit: 1c24bca387
master date: 2024-08-09 23:59:18 -0700
The containers are exactly as per 20.04 (Focal). However, this now brings us
to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
The oldest and newest toolchains are the most likely to find problems with new
code, so reduce the middle 3 releases (18/20/22) to just a single smoke test
each.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
As with 16.04 (Xenial), with python3-setuptools included. Having this package
only in some containers was intentional; see commit bbc72a7877 ("automation:
Add python3's setuptools to some containers") for the rational.
This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 20.04-x86_64 1.06GB
registry.gitlab.com/xen-project/xen/ubuntu focal 1.57GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
As with 16.04 (Xenial), except that ninja-build package is available so QEMU
can be built.
This halves the size of the container:
registry.gitlab.com/xen-project/xen/ubuntu 18.04-x86_64 857MB
registry.gitlab.com/xen-project/xen/ubuntu bionic 1.44GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Rework the container to be non-root, use heredocs for legibility, and use
apt-get --no-install-recommends to keep the size down.
Ubuntu Xenial has no ninja-build package, so can't build the QEMU referenced
by Xen 4.16. Therefore, drop the dependencies too.
This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 16.04-x86_64 698MB
registry.gitlab.com/xen-project/xen/ubuntu xenial 1.21GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
See prior patch for most discussion.
Despite appearing to be a fixed release (and therefore not marked as permitted
failure), the dockerfile references the `leap` tag which is rolling in
practice. Switch to 15.6 explicitly, for better test stability.
Vs tumbleweed, use `zypper update` rather than dist-upgrade, and retain the
RomBIOS dependencies; bin86 and dev86.
In terms of size, this saves ~700M:
registry.gitlab.com/xen-project/xen/opensuse leap-15.6-x86_64 1.33GB
registry.gitlab.com/xen-project/xen/suse opensuse-leap 2.05GB
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Existing as suse:opensuse-tumbleweed is a historical quirk, and adjusted for
consistency with all the other containers.
Make it non-root, use heredocs for legibility, and use the zypper long names
for the benefit of those wondering what was being referenced or duplicated.
Trim the dependencies substantially. Testing docs isn't very interesting and
saves a lot of space. Other savings come from removing a huge pile of
optional QEMU dependencies (QEMU just needs to build the Xen parts to be
useful here, not have a full GUI environment).
Finally, there where some packages such as bc, libssh2-devel, libtasn1-devel
and nasm that I'm not aware of any reason to have had, even historically.
This shaves 1G off the size of the container:
registry.gitlab.com/xen-project/xen/opensuse tumbleweed-x86_64 1.39GB
registry.gitlab.com/xen-project/xen/suse opensuse-tumbleweed 2.35GB
Furthermore, identify which components of the build use which dependencies,
which will help managing them in the future.
Thanks to Olaf Hering for dependency fixes that have been subsumed into this
total overhaul.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The containers are both much smaller, with stable toolchains over time, and
this at least means we're not doing all testing with a single compiler.
Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH-*) and
reposition the jobs to optimise starting the smoke test.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
For starters, they're slightly smaller:
$ docker image list <snip>
registry.gitlab.com/xen-project/xen/debian 12-riscv64 772MB
registry.gitlab.com/xen-project/xen/debian 11-riscv64 422MB
registry.gitlab.com/xen-project/xen/archlinux current-riscv64 2.32GB
They also not rolling distros, so will be more predicatable testing for the
stable trees in the future.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Reviewed-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
... in the style of debian:12-ppc64le.
Rename the jobs and reposition them later as they're not a dependency for the
smoke testing any more.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
qemu-system-ppc64/8.1.0-ppc64 was added because bullseye's QEMU didn't
understand the powernv9 machine. However bookworm's QEMU does and this is
preferable to maintaining a random build of QEMU ourselves.
Use the debian:12-ppc64le container and test the output of that build too.
Remove qemu-system-ppc64-8.1.0-ppc64-export which is unused now.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
... conforming to the new naming scheme; $DISTRO-$VERSION-$ARCH-* so the jobs
sort more coherently.
Make it non-root by default, and set XEN_TARGET_ARCH=ppc64. Include QEMU too,
which will be used subsequently.
Add build jobs too, with debian-12-ppc64le-gcc-debug specifically early as it
will be used for smoke testing shortly.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Archlinux is a rolling distro. As a consequence, rebuilding the container
periodically changes the toolchain, and this affects all stable branches in
one go.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Randconfig builds pick CONFIG_DEBUG with 50% probability. Therefore
$foo{,-debug}-randconfig are just two equivalent jobs with misleading names.
Furthermore, arm64 has randconfig jobs for both Alpine and Bookworm; Both
environments use GCC 12.2, so we have 4 equivalent jobs. Delete 3 of them.
This leaves us with 4 primary randconfig jobs:
gitlab-ci/build.yaml:381:debian-bookworm-gcc-arm32-randconfig:
gitlab-ci/build.yaml:429:alpine-3.18-gcc-arm64-randconfig:
gitlab-ci/build.yaml:495:archlinux-current-gcc-riscv64-randconfig:
gitlab-ci/build.yaml:640:debian-bookworm-gcc-randconfig:
as PPC64 doesn't want randconfig right now, and buster-gcc-ibt is a special
job with a custom compiler.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Right now, most build containers use root. Archlinux, Fedora and Yocto set up
a regular user called `user`.
For those containers, trying to containerize as root fails, because
CONTAINER_UID0=1 does nothing, whereas CONTAINER_UID0=0 forces the user away
from root.
To make CONTAINER_UID0=1 work reliably, force to root if requested.
Fixes: 17fbe6504d ("automation: introduce a new variable to control container user")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
The history on this one is complicated. The note to README was added in
commit 1f95747a4f ("Add openssl-dev and xorg-x11-dev to README") in 2007.
At the time, there was a vendered version of Qemu in xen.git with a local
modification using <X11/keysymdef.h> to access the monitor console over VNC.
The final reference to keysymdef.h was dropped in commit 85896a7c4d ("build:
add autoconf to replace custom checks in tools/check") in 2012. The next
prior mention was in 2009 with commit a8ccb671c3 ("tools: fix x11 check")
noting that x11 was not a direct dependcy of Xen; it was transitive through
SDL for Qemu for source-based distros.
It appears there may have been other unspecified dependencies on xorg,
e.g. the use of lndir by unmodified_drivers which are no longer relevant
either.
These days its only the Debian based dockerfiles which install xorg-x11, and
Qemu builds fine in these and others without x11.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CentOS 7 is fully End-of-life as of 2024-06-30, and the Yum repo configuration
points at URLs which have become non-existent.
First, start by using a heredoc RUN for legibility. It's important to use
`set -e` to offset the fact that we're no longer chaining every command
together with an &&.
Also, because we're using a single RUN command to perform all RPM operations,
we no longer need to work around the OverlayFS bug.
Adjust the CentOS-*.repo files to point at vault.centos.org. This also
involves swapping mirrorlist= for baseurl= in the yum config.
Use a minor bashism to express the dependenices more coherently, and identify
why we have certain dependencies. Some adjustments are:
* We need bzip2-devel for the dombuilder. bzip2 needs retaining stubdom or
`tar` fails to unpack the .bz2 archives.
* {lzo,lz4,ztd}-devel are new optional dependency since the last time this
package list was refreshed.
* openssl-devel hasn't been a dependency since Xen 4.6.
* We long ago ceased being able to build Qemu and SeaBIOS in this container,
so drop their dependencies too.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Commit b5739330d7 introduced the use of heredocs in the jessie/stretch
dockerfiles.
It turns out this was introduced by BuildKit in 2018 along with a
standardisation of Dockerfile syntax, and has subsequently been adopted by the
docker community.
Annotate all dockerfiles with a statement of the syntax in use, and extend
README.md details including how to activate BuildKit when it's available but
off by default.
This allows the containers to be rebuilt following commit a0e29b3163 ("CI:
Drop glibc-i386 from the build containers").
Fixes: b5739330d7 ("automation: fix jessie/stretch images to use archive.debian.org apt repos")
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Update ECLAIR configuration to take into account the deviations
agreed during the MISRA meetings.
While doing this, remove the obsolete "Set [123]" comments.
Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Remove from the ECLAIR integration scripts an unused option, which
was already ignored, and make the help texts consistent
with the rest of the scripts.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
MISRA C Rule 20.7 states: "Expressions resulting from the expansion
of macro parameters shall be enclosed in parentheses".
The local helpers GRP2 and XADD in the x86 emulator use their first
argument as the constant expression for a case label. This pattern
is deviated project-wide, because it is very unlikely to induce
developer confusion and result in the wrong control flow being
carried out.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Rule 21.2 reports identifiers reserved for the C and POSIX standard
libraries: or, and, not and xor are reserved identifiers because they
constitute alternate spellings for the corresponding operators (they are
defined as macros by iso646.h); however Xen doesn't use standard library
headers, so there is no risk of overlap.
This addresses violations arising from x86_emulate/x86_emulate.c, where
label statements named as or, and and xor appear.
No functional change.
Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Rule 13.6 states that "The operand of the `sizeof' operator shall not
contain any expression which has potential side effects".
Define service B.UNEVALEFF as an extension of Rule 13.6 to
check for unevalued side effects also for typeof and alignof operators.
Update ECLAIR configuration to deviate uses of BUILD_BUG_ON and
alternative_v?call[0-9] for both Rule 13.6 and B.UNEVALEFF.
Add service B.UNEVALEFF to the accepted.ecl guidelines to check
"violations" in the weekly analysis.
Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
MISRA C Rule 5.5 states that "Identifiers shall be distinct from macro
names".
Update ECLAIR configuration to deviate:
- macros expanding to their own name;
- clashes between macros and non-callable entities;
- clashes related to the selection of specific implementations of string
handling functions.
Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Rules 20.12 and 14.4 are now clean on ARM and x86, so they are added
to the list of clean guidelines.
Some guidelines listed in the additional clean section for ARM are also
clean on x86, so they can be removed from there.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
[stefano: remove 20.9 from commit message]
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
The DEFINE macro in asm-offsets.c (for all architectures) still generates
violations despite the file(s) being excluded from compliance, due to the
fact that in its expansion it sometimes refers entities in non-excluded files.
These corner cases are deviated by the configuration.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
This tests if QEMU works in PVH dom0. QEMU in dom0 requires enabling TUN
in the kernel, so do that too.
Add it to both x86 runners, similar to the PVH domU test.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Release-Acked-By: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Commit ea59e7d780 ("xen/bitops: Cleanup and new infrastructure ahead of
rearrangements") introduced new violations on previously clean rules 20.9 and
20.12 (clean on ARM only, right now).
The first is introduced because CONFIG_CC_IS_CLANG in xen/self-tests.h is not
defined in the configuration under analysis. Using "defined()" instead avoids
relying on the preprocessor's behaviour upon encountering an undedfined identifier
and addresses the violation.
The violation of Rule 20.12 is due to "val" being used both as an ordinary argument
in macro RUNTIME_CHECK, and as a stringification operator.
No functional change.
Fixes: ea59e7d780 ("xen/bitops: Cleanup and new infrastructure ahead of rearrangements")
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Disable unnecessary configs for two cases:
1. By utilizing EXTRA_FIXED_RANDCONFIG for randconfig builds (GitLab CI jobs).
2. By using tiny64_defconfig for non-randconfig builds.
Only configs which lead to compilation issues were disabled.
Remove lines related to disablement of configs which aren't affected
compilation:
-# CONFIG_SCHED_CREDIT is not set
-# CONFIG_SCHED_RTDS is not set
-# CONFIG_SCHED_NULL is not set
-# CONFIG_SCHED_ARINC653 is not set
-# CONFIG_TRACEBUFFER is not set
-# CONFIG_HYPFS is not set
-# CONFIG_SPECULATIVE_HARDEN_ARRAY is not set
Update argo.c to include asm/p2m.h directly, rather than on a transitive
dependency through asm/domain.h Update asm/p2m.h to include xen/errno.h,
rather than rely on it having included already.
CONFIG_XSM=n as it requires an introduction of:
* boot_module_find_by_kind()
* BOOTMOD_XSM
* struct bootmodule
* copy_from_paddr()
The mentioned things aren't introduced now.
CONFIG_BOOT_TIME_CPUPOOLS requires an introduction of cpu_physical_id() and
acpi_disabled, so it is disabled for now.
PERF_COUNTERS requires asm/perf.h and asm/perfc-defn.h, so it is
also disabled for now, as RISC-V hasn't introduced this headers yet.
LIVEPATCH isn't ready for RISC-V too and it can be overriden by randconfig,
so to avoid compilation errors for randconfig it is disabled for now.
Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
[Fix up common/argo.c rather than inserting a transitive dependency]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
The parentheses in this regular expression should be doubly
escaped because they undergo expansion twice.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
[stefano: fix commit message]
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Some MISRA C rules already have no violations in Xen, so they can be
set as clean.
Reorder the rules in tagging.ecl according to version ordering
(i.e. sort -V) and split the configuration on multiple lines for
readability.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
This rule has no more violations in the codebase, so it can be
set as clean.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
There are no more users, and we want to disuade people from introducing new
users just for sd_notify() and friends. Drop the dependency.
We still want the overall --with{,out}-systemd to gate the generation of the
service/unit/mount/etc files.
Rerun autogen.sh, and mark the dependency as removed in the build containers.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Christian Lindig <christian.lindig@cloud.com>
The count_args_ macro violates Rule 20.7, but it can't be made
compliant with Rule 20.7 without breaking its functionality. Since
it's very unlikely for this macro to be misused, it is deviated.
No functional change.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
These rules are concerned with the use of facilities provided by the
C Standard Library (qsort, bsearch for rule 21.9, and those provided
by <time.h> for rule 21.10).
Xen provides in its source code its own implementation of some of these
functions and macros, therefore a justification is provided for allowing
uses of these functions in the project.
The rules are also marked as clean as a consequence.
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Tag the rule as clean, as there are no more violations in the codebase since
93c27d54dd ("xen/arm: Fix MISRA regression on R1.1,
flexible array member not at the end").
Signed-off-by: Nicola Vetrini <nicola.vetrini@bugseng.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
To improve clarity and ease of navigation do not show reports related
to adopted code in the scheduled analysis.
Configuration options are commented out because they may be useful
in the future.
Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com>
Signed-off-by: Federico Serafini <federico.serafini@bugseng.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Adjust ECLAIR configuration for rules: R21.14, R21.15, R21.16 by taking
into account mem* macros defined in the Xen sources as if they were
equivalent to the ones in Standard Library.
Signed-off-by: Alessandro Zucchelli <alessandro.zucchelli@bugseng.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Stefano Stabellini
Andrew Cooper
Michal Orzel
Federico Serafini
Nicola Vetrini
Alessandro Zucchelli
Marek Marczykowski-Górecki
Oleksii Kurochko