mirror of
https://github.com/xen-project/xen.git
synced 2026-10-29 18:50:05 +01:00
Andrew Warfield
ed6af790ae
with I/O errors.
There are a number of flaws in the blktap userspace daemon when dealing
with I/O errors.
- The backends which use AIO check the io_events.res member to determine
if an I/O error occurred. Which is good. But when calling the callback
to signal completion of the I/O, they pass the io_events.res2 member
Now this seems fine at first glance[1]
"res is the usual result of an I/O operation: the number of bytes
transfered, or a negative error code. res2 is a second status
value which will be returned to the user"
Except that
"currently (2.6.0-test9), callers of aio_complete() within the
kernel always set res2 to zero."
And this hasn't changed anytime since 2.6.0, so by passing through
the status from 'res2', the callback thinks the I/O operation succeeded
even when it failed :-(
The fix is simple instead of passing 'res2', just pass
ep->res == io->u.c.nbytes ? 0 : 1
This would solve the error reporting to the guest, except that there
is a second flaw...
- The tapdisk I/O completion callback checks the status parameter
passed in, syslog's it and then returns. It never bothers to send
the I/O completion response back to the blktap kernel driver when
a failure occurrs.
Fortunately the fix for this is also simple. Instead of returning
from the callback when dealing with an error, we simply toggle the
status field for the pending response to BLKIF_RSP_ERROR and then
continue with the normal codepath. So the error eventually gets
back to the guest.
The scenario I used to discover the problem and test the patch is thus:
- In dom0 create a filesystem with only 200 MB of free space
- Create a 1 GB sparse file on this volume.
- Configure the guest so this sparse file appears as /dev/xvdb
- In the domU create a single partition on /dev/xvdb and format
it with ext3.
- In the DomU, mount /dev/xvdb1 on /mnt and then run
dd if=/dev/zero of=/mnt/data.bin bs=1GB count=1
Without this patch, the 'dd' command would succeed in writing 1 GB of data
even though the underlying disk in Dom0 was only 200 MB in size. More complex
tests of copying a whole directory heirarchy across resulted in catastrophic
data corruption of the filessytem itself. Manual fsck was needed to fixup
the filesystem & there were many very bad errors needing fixing.
With this patch applied the DomU sees the I/O failures and kernel logs
messages
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 722127
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 730327
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 738527
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 746727
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 754927
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 763127
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 771327
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 779527
Dec 1 11:02:53 dhcp-5-203 kernel: end_request: I/O error, dev xvdc, sector 792399
It will retry the I/O operation until it runs out of sectors to try, and then
fail the operation. The filesystem is not seriously damaged - ext3 journal
recovery will trivially cleanup if the guest is rebooted after the disk in
Dom0 is enlarged.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
[1] http://lwn.net/Articles/24366/
################################
__ __ _____ ___
\ \/ /___ _ __ |___ / / _ \
\ // _ \ '_ \ |_ \| | | |
/ \ __/ | | | ___) | |_| |
/_/\_\___|_| |_| |____(_)___/
################################
http://www.xensource.com/xen/about.html
What is Xen?
============
Xen is a Virtual Machine Monitor (VMM) originally developed by the
Systems Research Group of the University of Cambridge Computer
Laboratory, as part of the UK-EPSRC funded XenoServers project. Xen
is freely-distributable Open Source software, released under the GNU
GPL. Since its initial public release, Xen has grown a large
development community, spearheaded by XenSource Inc, a company created
by the original Xen development team to build enterprise products
around Xen.
The 3.0 release offers excellent performance, hardware support and
enterprise-grade features such as x86_32-PAE, x86_64, SMP guests and
live relocation of VMs. This install tree contains source for a Linux
2.6 guest; ports to Linux 2.4, NetBSD, FreeBSD and Solaris will follow
later (and are already available for previous Xen releases).
This file contains some quick-start instructions to install Xen on
your system. For full documentation, see the Xen User Manual. If this
is a pre-built release then you can find the manual at:
dist/install/usr/share/doc/xen/pdf/user.pdf
If you have a source release, then 'make -C docs' will build the
manual at docs/pdf/user.pdf.
Quick-Start Guide - Pre-Built Binary Release
============================================
[NB. Unless noted otherwise, all the following steps should be
performed with root privileges.]
1. Install the binary distribution onto your filesystem:
# sh ./install.sh
Among other things, this will install Xen and Xen-ready Linux
kernel files in /boot, kernel modules and Python packages in /lib,
and various control tools in standard 'bin' directories.
2. Configure your bootloader to boot Xen and an initial Linux virtual
machine. Note that Xen currently only works with GRUB and pxelinux
derived boot loaders: less common alternatives such as LILO are
*not* supported. You can most likely find your GRUB menu file at
/boot/grub/menu.lst: edit this file to include an entry like the
following:
title Xen 3.0 / XenLinux 2.6
kernel /boot/xen-3.0.gz console=vga
module /boot/vmlinuz-2.6-xen root=<root-dev> ro console=tty0
module /boot/initrd-2.6-xen.img
NB: Not all kernel configs need an initial ram disk (initrd), but
if you do specify one you'll need to use the 'module' grub directive
rather than 'initrd'.
The linux command line takes all the usual options, such as
root=<root-dev> to specify your usual root partition (e.g.,
/dev/hda1).
The Xen command line takes a number of optional arguments described
in the manual. The most common is 'dom0_mem=xxxM' which sets the
amount of memory to allocate for use by your initial virtual
machine (known as domain 0). Note that Xen itself reserves about
32MB memory for internal use, which is not available for allocation
to virtual machines.
3. Reboot your system and select the "Xen 3.0 / XenLinux 2.6" menu
option. After booting Xen, Linux will start and your initialisation
scripts should execute in the usual way.
Quick-Start Guide - Source Release
==================================
First, there are a number of prerequisites for building a Xen source
release. Make sure you have all the following installed, either by
visiting the project webpage or installing a pre-built package
provided by your Linux distributor:
* GCC (preferably v3.2.x or v3.3.x; older versions are unsupported)
* GNU Make
* GNU Binutils
* Development install of zlib (e.g., zlib-dev)
* Development install of Python v2.3 or later (e.g., python-dev)
* Development install of curses (e.g., libncurses-dev)
* bridge-utils package (/sbin/brctl)
* iproute package (/sbin/ip)
* hotplug or udev
[NB. Unless noted otherwise, all the following steps should be
performed with root privileges.]
1. Download and untar the source tarball file. This will be a
file named xen-unstable-src.tgz, or xen-$version-src.tgz.
You can also pull the current version from the SCMS
that is being used (Bitkeeper, scheduled to change shortly).
# tar xzf xen-unstable-src.tgz
Assuming you are using the unstable tree, this will
untar into xen-unstable. The rest of the instructions
use the unstable tree as an example, substitute the
version for unstable.
2. cd to xen-unstable (or whatever you sensibly rename it to).
The Linux, netbsd and freebsd kernel source trees are in
the $os-$version-xen-sparse directories.
On Linux:
3. For the very first build, or if you want to destroy existing
.configs and build trees, perform the following steps:
# make world
# make install
This will create and install onto the local machine. It will build
the xen binary (xen.gz), and a linux kernel and modules that can be
used in both dom0 and an unprivileged guest kernel (vmlinuz-2.6.x-xen),
the tools and the documentation.
You can override the destination for make install by setting DESTDIR
to some value.
The make command line defaults to building the kernel vmlinuz-2.6.x-xen.
You can override this default by specifying KERNELS=kernelname. For
example, you can make two kernels - linux-2.6-xen0
and linux-2.6-xenU - which are smaller builds containing only selected
modules, intended primarily for developers that don't like to wait
for a full -xen kernel to build. The -xenU kernel is particularly small,
as it does not contain any physical device drivers, and hence is
only useful for guest domains.
To make these two kernels, simply specify
KERNELS="linux-2.6-xen0 linux-2.6-xenU"
in the make command line.
If you want to build an x86_32 PAE capable xen and kernel to work
on machines with >= 4GB of memory, use XEN_TARGET_X86_PAE=y on the
make command line.
4. To rebuild an existing tree without modifying the config:
# make dist
This will build and install xen, kernels, tools, and
docs into the local dist/ directory.
You can override the destination for make install by setting DISTDIR
to some value.
make install and make dist differ in that make install does the
right things for your local machine (installing the appropriate
version of hotplug or udev scripts, for example), but make dist
includes all versions of those scripts, so that you can copy the dist
directory to another machine and install from that distribution.
5. To rebuild a kernel with a modified config:
# make linux-2.6-xen-config CONFIGMODE=menuconfig (or xconfig)
# make linux-2.6-xen-build
# make linux-2.6-xen-install
Depending on your config, you may need to use 'mkinitrd' to create
an initial ram disk, just like a native system e.g.
# depmod 2.6.16-xen
# mkinitrd -v -f --with=aacraid --with=sd_mod --with=scsi_mod initrd-2.6.16-xen.img 2.6.16-xen