Commit Graph
63 Commits
Author SHA1 Message Date
Dario Faggioli 0a6ad045c5 automation: add openSUSE Tumbleweed CI image
openSUSE comes in two flavours: Leap, which is non-rolling, and released
annualy, and Tumbleweed, which is rolling.

Reasons why it makes sense to have both (despite both being openSUSE,
package lists in dockerfiles being quite similar, etc) are:
- Leap share a lot with SUSE Linux Enterprise. So, regressions on Leap,
  not only means regressions for all openSUSE Leap users, but also helps
  prevent/catch regressions on SLE;
- Tumbleweed often has the most bleeding-edge software, so it will help
  us prevent/catch regressions with newly released versions of
  libraries, compilers, etc (e.g., at the time of writing this commit,
  some build issues, with GCC9, where discovered while trying to build
  in a Tumbleweed image).

Note that, considering the rolling nature of Tumbleweed, the container
would need to be rebuilt (e.g., periodically), even if the docker file
does not change.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-08-03 15:55:39 +01:00
Dario Faggioli d446dbb234 automation: try to keep openSUSE Leap image a little smaller
Using `--no-recommends` when updating or installing commands should
prevent non strictly necessary packages to be installed.

doing a `clean -a` after installing all the packages, should, in
theory, free more space (as opposed to using just `clean`).

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-08-03 15:55:39 +01:00
Doug Goldstein c7a99eee3d automation: add info about container pushes
To be able to push a container, users must have access and have logged
into the container registry. The docs did not explain this fully so this
documents the steps better.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
2019-08-03 15:55:39 +01:00
Dario Faggioli 107853d89d ci: install C++ in opensuse-leap CI container
The openSUSE Leap container image, built after
opensuse-leap.dockerfile was missing the gcc-c++,
which is necessary, e.g., for building OVMF.

Add it.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-08-02 16:48:00 +01:00
Roger Pau Monne 508908fd44 automation: fix typo in comment
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wl@xen.org>
2019-06-16 21:54:47 +01:00
Andrew Cooper f093199132 automation: Fix CI with the fedora container
A recent rebuild of the CI contaniers switched from Fedora 29 to 30 because
the dockerfile is targetting latest.

Unfortunately, the version of iPXE in master doesn't build with the default
GCC in Fedora 30, which is blocking all CI activity.

Switch from latest to an explicit version, to avoid future breakage.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-06-11 17:42:52 +01:00
Andrew Cooper 892e538f43 automation: Add an 'all' target for container maintenance
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-06-11 11:06:52 +01:00
Roger Pau Monne 6325f70e77 automation: add clang and lld 8 tests to gitlab
Using clang and lld 8 requires installing the packages from the
official llvm apt repositories, so modify the Debian Docker files for
stretch and unstable to add the llvm repo and install clang and lld
from it.

Also add some jobs to test building Xen with clang 8 and lld.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-06-11 11:06:52 +01:00
Wei Liu 1d1f161153 gitlab-ci: allow specifying base and tip in build test
We will soon provide this new capability to humans and automated
systems.

The default behaviour is retained: tip and base are passed by Gitlab
CI.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-05-15 11:01:06 +01:00
Wei Liu 004299fba4 gitlab-ci: avoid deleting build-each-commit-gcc.log
072a96c490 used `git clean -ffdx` which caused the log to be deleted.

Generate the log in the parent directory then move it back.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-05-13 12:45:57 +01:00
Wei Liu 072a96c490 gitlab-ci: use git clean -ffdx in build each commit test
The build script invoked is designed to run in a pristine checkout.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-08 13:28:12 +01:00
Wei Liu 7a01cfaeb0 gitlab-ci: log commit range in build test
It is easier to debug stuff when the target range is clearly visible
at the top.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
2019-04-08 11:31:28 +01:00
Wei Liu b3c2939fdd automation: fix "build each commit" test
An error was introduced while rebasing 9b8b3f30. The new test
shouldn't depend on anything, otherwise artefacts will be downloaded
from build stage and cause the script to abort.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
2019-04-05 15:40:01 +01:00
Wei Liu 9b8b3f300f automation: introduce a test to build each commit
This is added to the test stage so that its failure won't block other
things.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-05 11:15:38 +01:00
Wei Liu 1ba432fb73 automation: add a script to build newly pushed commits in Gitlab CI
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-05 11:15:36 +01:00
Wei Liu 78798fb431 automation: set ret for potential error in build-test.sh
`git rev-list` can fail if the base..tip range contains invalid
commit(s). If that happens ret never gets a chance to be set.

Set ret before hand to fix the issue.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-05 11:14:57 +01:00
Wei Liu f21a359a30 automation: allow build-test.sh to run in detached HEAD state
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-05 11:14:57 +01:00
Wei Liu bcb15f52d7 gitlab-ci: add fedora gcc build jobs
Although the image comes with clang, clang builds don't work yet.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-04 14:20:27 +01:00
Wei Liu 460837f800 automation: add Fedora image to containerize script
At the same time sort the list alphabetically.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-04 14:20:25 +01:00
Wei Liu e4c96ef233 automation: add a Fedora image
Use the latest and greatest.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-04 14:20:13 +01:00
Anthony PERARD 7f6658d9cb automation: Add Arch Linux container and build jobs
One particularity of Arch Linux, /usr/bin/python is python3.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2019-04-04 10:57:35 +01:00
Andrew Cooper 8eed571409 CI: Add a CentOS 6 container and build jobs
CentOS 6 is probably the most frequently broken build, so adding it to CI
would be a very good move.

One problem is that CentOS 6 comes with Python 2.6, and Qemu requires 2.7.
There appear to be no sensible ways to get Python 2.7 into a CentOS 6
environments, so modify the build script to skip the Qemu upstream build
instead.  Additionally, SeaBIOS requires GCC 4.6 or later, so skip it as well.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2019-03-26 15:36:19 +00:00
Andrew Cooper 1316369dca CI: Fix indentation in containerize script
The script is mostly indented with spaces, but there are three tabs.  Fix them
up to be consistent.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2019-03-25 11:34:48 +00:00
Wei Liu 75e383fa93 automation: enable building rombios with clang
Previously it is disabled because the embedded ipxe can't be built
with clang. Now that ipxe is split out we can use --with-system-ipxe
to work around the issue.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
2019-03-15 12:40:24 +00:00
Wei Liu 49704b8d23 automation: use python-dev python2.7-dev in Debian and Ubuntu
... instead of python2.7-dev.

We installed python2.7-dev because xen only worked with 2.7.

Installing python2.7-dev only gives python2.7-config, which causes
configure to fail because it wants python-config by default. Now xen
should work with 2.6 and above, we can install python-dev and let
distros pick the default python.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
2019-03-13 15:01:36 +00:00
Wei Liu 755eb6403e automation: introduce a QEMU smoke test for PVH Dom0
Make qemu-smoke-x86-64.sh take a variant argument. Make two new tests
in test.yaml.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
Release-acked-by: Juergen Gross <jgross@suse.com>
2019-02-01 15:13:25 +00:00
Wei Liu 4cf5d62642 automation: skip test stage for some branches
We skipped build stage for those branches. We want to skip test state
for those branches too.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-12-10 16:31:09 +00:00
Wei Liu 81cfc1b3c7 automation: break .gitlab-yaml into smaller files
Break out files for build jobs and test jobs. Keep the top level
.gitlab-ci.yaml small.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-12-05 22:30:14 +00:00
Wei Liu 568653e01b automation: make clean between builds
Currently randconfig tests are more likely to fail than to succeed
because of a bug in xen's build system: symbols-dummy.o's dependency
is wrong, which causes it to not get rebuild between runs, which
eventually causes linking to fail. There may also be other corner
cases we haven't discovered.

The fix is not straightforward. For now, make sure the tree is cleaned
properly between builds so we don't see random failures in Gitlab CI.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-21 18:34:27 +00:00
Wei Liu 29e284512f automation: add qemu smoke test
This patch introduces a new test stage into the pipeline and provides
a simple QEMU based smoke test.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-20 18:15:46 +00:00
Wei Liu 3c21dba189 automation: also specify xen binary as artifact on x86_64
... so that it can be passed on to test stage.

Note that xen is only extracted for x86_64 build since others may not
have that. Use a directory to account for possibly different file
names on Arm.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-20 18:15:39 +00:00
Wei Liu b9fb402ec3 automation: stash default config file for artifact extraction
This aids troubleshooting when we notice a failure in the default
configuration.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-20 16:55:10 +00:00
Wei Liu 102a437d59 automation: introduce CONTAINER_NO_PULL for containerize
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-20 16:54:55 +00:00
Andrew Cooper 219fc4840a automation: Add a 32bit Debian Jessie dockerfile
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2018-11-19 21:13:05 +00:00
Dario Faggioli 5e83981390 automation: add openSUSE Leap dockerfile.
Tracks the latest release of openSUSE Leap. At the time of writing this
patch, this is Leap 15.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2018-11-17 10:33:00 +00:00
Wei Liu 15410d8fc8 automation: lift tools build restriction in build script
The restrictions are mostly from Travis CI days when cross-compilation
was not feasible. Now we will be building natively on Arm, remove
those lines.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-13 13:48:46 +00:00
Wei Liu 8cf264cba5 automation: add debian unstable arm64v8 image
Unfortunately sharing dockerfile has proven impossible: 1) the base
image is different; 2) Arm doesn't need libc-dev-i386; 3) Arm needs
libfdt-dev.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-13 13:48:40 +00:00
Wei Liu 77533e4f7d automation: add dockerfile for Ubuntu 18.04
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
2018-11-08 18:15:24 +00:00
Wei Liu 7ef789d79f automation: build some customised configs
Introduce a new directory to put in configs we care about. Modify
build script to build with those configs.

While we only introduce x86 configs initially, provision for non-x86
configs.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-11-08 16:40:20 +00:00
Wei Liu 17fbe6504d automation: introduce a new variable to control container user
Sometimes it is handy to create a container and play with its setup
manually as root.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-09-26 09:50:17 +01:00
Wei Liu 583037acab automation: specify -j$(nproc) in build script
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-09-07 08:39:10 +01:00
Doug Goldstein 5a5b8bfec3 automation: ensure created are not owned as root
By default the container runs as the root user and since the source tree
is bind mounted into the container, any file is created and owned by the
root user which harms ergonomics when working outside of the container
environment. This maps the root user within the container to the uid of
the user outside of the container so files are not owned by root.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
2018-08-03 16:11:22 +01:00
Doug Goldstein 92ff8f0979 automation: remove dead code from containerize
This is more dead code.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
2018-08-03 16:11:22 +01:00
Doug Goldstein f437ac101f automation: drop container name from containerize
This was something that existed for some scripting support for a totally
unrelated project and when I copied this script I failed to remove it so
this removes it. Build containers for Xen are best as ephemeral
environments and should just utilizes Docker's default container naming
behavior.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
2018-08-03 16:11:22 +01:00
Doug Goldstein 7d217430d6 automation: standardize containerize env names
Standardized all the environment variable names that the containerize
script uses to start with CONTAINER_

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
2018-08-03 16:11:18 +01:00
Doug Goldstein 45205b6529 automation: add CentOS 7.x image
This image will always track the latest CentOS 7.x release. Add this
container to containerize for easy access.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
2018-07-30 16:01:09 +01:00
Wei Liu acd00a3033 automation: introduce a script for build test
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>
2018-07-27 09:32:54 +01:00
Wei Liu e66e5c52a7 automation: add debian unstable images
This will get us the latest toolchain available in Debian.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>
2018-07-27 09:28:42 +01:00
Doug Goldstein 5437bdcd48 scripts: add helper script to use Docker containers
This adds a script that can be used to do builds easily within the
defined containers under the automation directory. These containers live
in the public GitLab registry under the xen-project namespace. The
script can be executed a number of ways but the default is to drop you
at a bash shell within a Debian Stretch container at the top level of
the source tree.

Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
2018-07-20 11:27:54 +01:00
Wei Liu b1afab7fb5 automation: add SLES 11 SP4 dockerfile
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Doug Goldstein <cardoe@cardoe.com>
2018-07-18 17:32:19 +01:00