Since Python 3.12, invalid escape sequences generate SyntaxWarning. In the
future, these invalid sequences will raise a SyntaxError, so changed to using
raw string notation.
Link: https://docs.python.org/3/whatsnew/3.12.html#other-language-changes
Fixes: d8f3a67bf9 ("pygrub: further improve grub2 support")
Fixes: dd03048708 ("xen/pygrub: grub2/grub.cfg from RHEL 7 has new commands in menuentry")
Fixes: d1b93ea261 ("tools/pygrub: Make pygrub understand default entry in string format")
Fixes: 622e368758 ("Add ZFS libfsimage support patch")
Fixes: 02b26c02c7 ("xen/scripts: add cppcheck tool to the xen-analysis.py script")
Fixes: 56c0063f4e ("xen/misra: xen-analysis.py: Improve the cppcheck version check")
Signed-off-by: Ariel Otilibili <Ariel.Otilibili-Anieli@eurecom.fr>
Reviewed-by: Luca Fancellu <luca.fancellu@arm.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
This was deprecated in commit 10c88f1c18 ("tools: Install pv bootloaders in
libexec rather than /usr/bin") and caused to warn in commit c31d6a7ee2
("libxl: Warn that /usr/bin/pygrub is deprecated"), both in 2012.
New versions of RPM warn against absolute symlinks, so take out fully, 11
years later.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
The env variable must be interpreted as an integer. As it is, the override
logic simply causes an exception.
Fixes: e0342ae555 ("tools/pygrub: Deprivilege pygrub")
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
This is important in order for every mount done inside a mount namespace to
go away after the namespace itself goes away. The comment referring to
unreliability in Linux 4.19 was just wrong.
This patch sets the story straight and makes the depriv pygrub a bit more
confined should a layer of the onion be vulnerable.
Fixes: e0342ae555 ("tools/pygrub: Deprivilege pygrub")
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Add a local .gitignore file for tools/pygrub.
As at least on some systems (e.g. OpenSUSE Leap 15.5) the build will
produce a tools/pygrub/pygrub.egg-info directory, add it to the new
.gitignore file, too.
Use "/dir/" as pattern for ignoring a directory and its contents, as
the "/dir/*" pattern won't ignore the directory itself (git just
doesn't complain when seeing a directory without any not ignored file
in it).
Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Introduce a --runas=<uid> flag to deprivilege pygrub on Linux and *BSDs. It
also implicitly creates a chroot env where it drops a deprivileged forked
process. The chroot itself is cleaned up at the end.
If the --runas arg is present, then pygrub forks, leaving the child to
deprivilege itself, and waiting for it to complete. When the child exists,
the parent performs cleanup and exits with the same error code.
This is roughly what the child does:
1. Initialize libfsimage (this loads every .so in memory so the chroot
can avoid bind-mounting /{,usr}/lib*
2. Create a temporary empty chroot directory
3. Mount tmpfs in it
4. Bind mount the disk inside, because libfsimage expects a path, not a
file descriptor.
5. Remount the root tmpfs to be stricter (ro,nosuid,nodev)
6. Set RLIMIT_FSIZE to a sensibly high amount (128 MiB)
7. Depriv gid, groups and uid
With this scheme in place, the "output" files are writable (up to
RLIMIT_FSIZE octets) and the exposed filesystem is immutable and contains
the single only file we can't easily get rid of (the disk).
If running on Linux, the child process also unshares mount, IPC, and
network namespaces before dropping its privileges.
This is part of XSA-443 / CVE-2023-34325
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
This is work required in order to let pygrub operate in highly deprivileged
chroot mode. This patch adds a function that preloads every plugin, hence
ensuring that a on function exit, every shared library is loaded in memory.
The new "init" function is supposed to be used before depriv, but that's
fine because it's not acting on untrusted data.
This is part of XSA-443 / CVE-2023-34325
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
This patch allows pygrub to get ahold of every RW file descriptor it needs
early on. A later patch will clamp the filesystem it can access so it can't
obtain any others.
This is part of XSA-443 / CVE-2023-34325
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Small tidy up to ensure output_directory always has a trailing '/' to ease
concatenating paths and that `output` can only be a filename or None.
This is part of XSA-443 / CVE-2023-34325
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
There's a hypercall being issued in order to determine whether PV64 is
supported, but since Xen 4.3 that's strictly true so it's not required.
Plus, this way we can avoid mapping the privcmd interface altogether in the
depriv pygrub.
This is part of XSA-443 / CVE-2023-34325
Signed-off-by: Alejandro Vallejo <alejandro.vallejo@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Python distutils is deprecated and is going to be removed in Python
3.12. Add support for setuptools.
Setuptools in Python 3.11 complains:
SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
Keep using setup.py anyway to build the C extension.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Javi Merino <javi.merino@cloud.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
This is mandated by the Fedora packaging guidelines because it is a security
vulnerability otherwise in suid scripts. While Xen doesn't have suid scripts,
it's a very good idea generally because it prevents the users local python
environment interfering from system packaged scripts.
pygrub is the odd-script-out, being installed by distutils rather than
manually with INSTALL_PYTHON_PROG. distutils has no nice way of editing the
shebang, so arrange to use INSTALL_PYTHON_PROG for pygrub too.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Unexpectedly the environment variable which needs to be passed is
$LDSHARED and not $LD. Otherwise Python may find the build `ld` instead
of the host `ld`.
Replace $(LDFLAGS) with $(SHLIB_LDFLAGS) as Python needs shared objects
it can load at runtime, not executables.
This uses $(CC) instead of $(LD) since Python distutils appends $CFLAGS
to $LDFLAGS which breaks many linkers.
Signed-off-by: Elliott Mitchell <ehem+xen@m5p.com>
Acked-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
In Grub2ConfigFile the code to handle ${saved_entry} and ${next_entry}
sets arg = "0" but this now does nothing following c/s d1b93ea261
"tools/pygrub: Make pygrub understand default entry in string format"
which replaced arg.strip() with arg_strip in the following line. This
patch restores the previous behaviour.
Signed-off-by: Michael Young <m.a.young@durham.ac.uk>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
The original code set the default to either a string or an integer
(0) and relies on a Python 2 specific behaviour to work (integer is
allowed to be compared to string in Python 2 but not 3).
Always use integer. The caller (pygrub) already has code to handle
that.
Reported-by: M A Young <m.a.young@durham.ac.uk>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
With the help of two porting guides and cpython source code:
1. Use PyBytes to replace PyString counterparts.
2. Use PyVarObject_HEAD_INIT.
3. Remove usage of Py_FindMethod.
4. Use new module initialisation routine.
For #3, Py_FindMethod was removed, yet an alternative wasn't
documented. The code is the result of reverse-engineering cpython
commit 6116d4a1d1
https://docs.python.org/3/howto/cporting.htmlhttp://python3porting.com/cextensions.html
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Run 2to3 and pick the sensible suggestions.
Import print_function and absolute_import so 2.6 can work.
There has never been a curses.wrapper module according to 2.x and 3.x
doc, only a function, so "import curses.wrapper" is not correct. It
happened to work because 2.x implemented a (undocumented) module.
We only need to import curses to make curses.wrapper available to
pygrub.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Not sure why Python 2 never complained, but Python 3 does.
Change tabs to spaces.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
This module should be called `libxenfsimage' for the same reasons that
the C library should.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
`fsimage' is rather general. And we do not expect this library to be
very useful out of tree because of its unstable ABI.
So add the word `xen'. This will avoid naming conflicts with anyone
else's fsimage library.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
This seems to have been simply omitted. Obviously this is needed when
building and not just when installing. Passing only when installing
is ineffective.
Signed-off-by: Ian Jackson <ian.jackson@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
DEPS_INCLUDE is currently the same as DEPS, so no functional change.
This patch is the result of this perl rune:
git-grep -l 'include.*DEPS' | xargs perl -i -pe 'next unless m/^-?include/; s/\bDEPS\b/DEPS_INCLUDE/'
I have verified that I haven't missed anything, with this rune:
git-grep '\bDEPS\b'
Reported-by: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
DEPS_RM is currently the same as DEPS, so no functional change.
This patch is the result of two perl runes:
git-grep -l 'rm.*DEPS' | xargs perl -i~ -pe 'next unless m/^\t+rm\b/; s/\bDEPS\b/DEPS_RM/;'
git-grep -l 'RM.*DEPS' | xargs perl -i~ -pe 'next unless m/^\t+\$\(RM\)/; s/\bDEPS\b/DEPS_RM/;'
And editing tools/xenstat/libxenstat/Makefile by hand.
I verified that I didn't miss anything with this rune:
git-grep '\bDEPS\b' | grep -v include |less
Reported-by: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Running "make uninstall" does not remove all installed files, a
situation which might cause link related issues if xen is re-installed
in a different location.
In order to make uninstall correctly remove the files it is best
the process should be done recursively by mirroring each "install"
target with an "uninstall" who removes the installed files.
An exception to this rule is uninstalling the files produced by
"qemu-xen-dir-remote" and "qemu-xen-traditional-dir", which are external
to the project. These projects do not implement an "uninstall" target so
the files have to be removed manually.
Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
* When the caller wants sexpr output, use `repr()'
This is what Xend expects.
The returned S-expressions are now escaped and quoted by Python,
generally using '...'. Previously kernel and ramdisk were unquoted
and args was quoted with "..." but without proper escaping. This
change may break toolstacks which do not properly dequote the
returned S-expressions.
* When the caller wants "simple" output, crash if the delimiter is
contained in the returned value.
With --output-format=simple it does not seem like this could ever
happen, because the bootloader config parsers all take line-based
input from the various bootloader config files.
With --output-format=simple0, this can happen if the bootloader
config file contains nul bytes.
This is CVE-2016-9379 and CVE-2016-9380 / XSA-198.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Tested-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
pygrub (in identify_disk_image()) detects a DOS style partition table
via the presence of the 0xaa55 signature at the end of the first
sector of the disk.
However this signature is also present in whole-disk configurations
when there is an MBR on the disk. Many filesystems (e.g. ext[234])
include leading padding in their on disk format specifically to enable
this.
So if we think we have a DOS partition table but do not find any
actual partition table entries we may as well try looking at it as a
whole disk image. Worst case is we probe and find there isn't anything
there.
This was reported by Sjors Gielen in Debian bug #745419. The fix was
inspired by a patch by Adi Kriegisch in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=745419#27
Tested by genext2fs'ing my /boot into a new raw image (works) and
then:
dd if=/usr/lib/grub/i386-pc/g2ldr.mbr of=img conv=notrunc bs=512 count=1
to add an MBR (with 0xaa55 signature) to it, which after this patch
also works.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: 745419-forwarded@bugs.debian.org
As recommended in http://www.gnu.org/licenses/gpl-howto.en.html.
This is the result of:
$ git grep -El Mass\|Temple\|Franklin | xargs ./fsf.pl
Where fsf.pl is:
#!/usr/bin/perl -w -pi.bak -0777
my $repl = 'If not, see <http://www.gnu.org/licenses/>.';
my $br = qr/(?:\s*\n\s*(?:[\*\#]|\/\/|\.\\" )?\s*|\s+)/;
my $inwt = qr/[Ii]f${br}not,${br}write${br}(?:to${br})?the${br}Free${br}Software${br}Foundation,(?:${br}Inc\.,)?/;
my $mass = qr/675${br}Mass${br}Ave,?${br}Cambridge,?${br}MA${br}02139,?${br}USA,?\.?/;
my $franklin = qr/51${br}Franklin${br}St(?:reet)?(?:,${br}| - )Fifth${br}Floor,?${br}Boston,?${br}MA,?${br}02110-1301,?${br}USA,?\.?/;
my $temple = qr/59${br}Temple${br}Place(?:,${br}| - )Suite${br}330,?${br}Boston,?${br}MA,?${br}021110?-1307,?${br}USA,?\.?/;
s|$inwt$br$mass|$repl|m;
s|$inwt$br$franklin|$repl|m;
s|$inwt$br$temple|$repl|m;
The only remaining mentions of these addresses are in COPYING files which I
haven't touched.
Some of the changed files are imports from elsewhere, however
filtering them out is tricky, I think it is tolerable to have these
files be modified here and then perhaps reverted on the next sync,
since it's only 1-2 lines and obvious what is going on.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Some systems have python-config include -D_FORTIFY_SOURCE in the
CFLAGS. But -D_FORTIFY_SOURCE does not (currently) work with -O0, and
-O0 is enabled in debug builds (since 1166ecf781). As a result, on
those systems, debug builds fail.
Work around this problem as follows:
* In configure, detect -D_FORTIFY_SOURCE in $(python-config --cflags)
* If detected, set the new autoconf substitution and make variable
PY_NOOPT_CFLAGS to -O1.
* In tools/Rules.mk, where we add -O0, also add PY_NOOPT_CFLAGS
(which will override the -O0 with -O1 if required).
Overriding the -O0 is better than disabling Fortify because the
latter might have an adverse security impact. A user who wants to
disable optimisation completely even for Python and also disable
Fortify can set the environment variable
EXTRA_CFLAGS_XEN_TOOLS='-U_FORTIFY_SOURCE -O0'
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Reported-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Euan Harris <euan.harris@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: Don Slutz <dslutz@verizon.com>
The original rule to target "distclean" in tools/Rules.mk was in effect
"make clean". It should be "make distclean".
However not all Makefiles in subdirectories have distclean target
defined. So this patch also adds a bunch of distclean targets to various
Makefiles. They only depend on clean target and don't have any actions
in most cases.
With the patch applied, following command outputs 0 results:
find tools -name 'Makefile*' -exec grep -L 'distclean' {} \+ \
| grep -v ocaml | grep -v libfsimage
Ocaml and libfsimage are known to have distclean defined in a dedicated
rules file.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
c/s d1b93ea causes substantial functional regressions in pygrub's ability to
parse bootloader configuration files.
c/s d1b93ea itself changed an an interface which previously used exclusively
integers, to using strings in the case of a grub configuration with explicit
default set, along with changing the code calling the interface to require a
string. The default value for "default" remained as an integer.
As a result, any Extlinux or Lilo configuration (which drives this interface
exclusively with integers), or Grub configuration which doesn't explicitly
declare a default will die with an AttributeError when attempting to call
"self.cf.default.isdigit()" where "default" is an integer.
Sadly, this AttributeError gets swallowed by the blanket ignore in the loop
which searches partitions for valid bootloader configurations, causing the
issue to be reported as "Unable to find partition containing kernel"
We should explicitly check type of "default" in image_index() and process it
appropriately.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Changes to handle non-numeric default attributes for grub2 caused run_grub()
to attempt to index into the images list using a string. Pull out the code
that handles submenus into a new function and use that to ensure sel is
numeric.
Reported-by: David Scott <dave.scott@citrix.com>
Signed-off-by: Simon Rowe <simon.rowe@eu.citrix.com>
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Reviewed-and-tested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The current usage of LIBEXEC is bogus. It describes the directory for
private xen executables. Other places create their own, similar
libexecdir path as $prefix/lib/xen/*.
Additional two other variables are used to describe similar paths:
PRIVATE_BINDIR and PRIVATE_PREFIX
The autoconf documentation refers to libexec as a directory for
executables and stuff which is called by other programs, not by the
user.
Adjust all places that want libexecdir as a target path. LIBEXEC refers
now to the base directory. Three convenience variables are used to refer
to paths to private binaries, libs and include files.
In the systemd files LIBEXEC_BIN is substituted, so this variable has to
be present in autoconf. All other variables are expanded in Paths.mk
because they are only used in Makefiles.
Most users of LIBEXEC are updated to use LIBEXEC_BIN because that is
what they want.
Users of PRIVATE_BINDIR are updated to use LIBEXEC_BIN because that is
what they want. PRIVATE_BINDIR and PRIVATE_PREFIX usage is removed by
this patch, in favour of LIBXEC_BIN and LIBEXEC
An internal libxl function was removed. A single helper to retrieve
LIBEXEC_BIN remains.
As suggested by the autoconf documentation, configure appends the
package name to LIBEXEC to make sure the provided directory really
refers to xen. This makes sure "make uninstall" preserves the real
libexecdir.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Cc: George Dunlap <george.dunlap@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
[ ijc -- ran autogen.sh as requested, updated QEMU_TRADITIONAL_REVISION to
pickup version which uses LIBEXEC_BIN ]
Move location of temporary bootfiles from /var/run/xend/boot to
/var/run/xen/pygrub. Create the subdirectory if does not exist.
The <dir> argument --output-directory must be an existing directory.
The reason for this change is that all entrys below /var/run have to be
created at runtime in case /var/run is cleared on every boot.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Currently pygrub can only correctly parse grub2's default attribute when it is
specified as a number. If it is set to ${saved_entry} or ${next_entry} then
the first image (i.e. entry number 0) is selected. If any other value is
specified (typically this would be the string in menuentry) pygrub will crash.
This patch will allow pygrub to interpret default attribute if it is specified
as a string (note that in case of submenus only the leaf string will be
considered).
Also issue a warning if default is set to ${saved_entry} or ${next_entry}.
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Ariel Otilibili