This patch is the folded version of API updates, along with the associated tool
changes to ensure that the build is always consistent.
API updates:
- The source domain in the sharing calls is no longer assumed to be dom0.
- Previously, the mem sharing code would return an opaque handle to index
shared pages (and nominees) in its global hash table. By removing the hash
table, the handle becomes a version, to avoid sharing a stale version of a
page. Thus, libxc wrappers and tools need to be updated to recall the share
functions with the information needed to fetch the page (which they readily
have).
Tool updates:
The only (in-tree, that we know of) consumer of the mem sharing API is the
memshr tool. This is updated to use the new API.
Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
Signed-off-by: Adin Scannell <adin@scannell.ca>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Tim Deegan <tim@xen.org>
Use the same approach as tools/blktap2/include/libvhd.h, remove local
definitions of bswap* and include byteswap.h. Also remove the
HAVE_BYTESWAP_H ifdef, since it was not defined in this context (it's
defined by QEMU).
Signed-off-by: Roger Pau Monne <roger.pau@entel.upc.edu>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
This patch addresses some of the compile and link issues with the memshr
module.
Signed-off-by: Adin Scannell <adin@scannell.ca>
Signed-off-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
Acked-by: Ian Jackson <ian.jackson.citrix.com>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
I could have sworn I did this years ago.
IIRC the need for our own copy was due to the use of io_set_eventfd which is
not present in version 0.3.106. However it is in 0.3.107 the first version of
which was uploaded to Debian in June 2008 (I can't find a better reference for
the release date).
The necessary version is available in Debian Lenny onwards and is in at least
RHEL 6, Fedora 13 and OpenSuSE 11.3. The necessary version appears to not be
available in RHEL 5 or SLES 11 which is why I haven't simply nuked the in tree
version.
This is based on tools-system-libaio.diff from the Debian packaging although I
have made it optional (but default on).
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Bitmaps are used in save/restore, xenpaging and blktap2. Merge the code into a
private xc_bitops.h file. All users are single threaded, so locking is not an
issue. The array of bits is handled as volatile because the x86 save/restore
code passes the bitmap to the hypervisor which in turn modifies the bitmap.
blktap2 uses a private bitmap. There was a possible overflow in the
bitmap_size() function, the remainder was not considered.
ia64 save/restore uses a bitmap to send the number of vcpus to the host.
x86 save/restore uses a bitmap to track dirty pages. This bitmap is shared with
the hypervisor. An unused function count_bits() was removed and a new
bitmap_size() function is now used.
xenpaging uses 3 private bitmaps to track the gfns which are in paged-out
state. It had a copy of some Linux bitops.h, which is now obsolete. Also the
BITS_PER_LONG macro was hardcoded to 64 which made it impossible to run 32bit
tools on a 64bit host. Wether this works at all has to be tested, yet.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Committed-by: Ian Jackson <ian.jackson.citrix.com>
At the end of a checkpoint, when a new flush (of buffered disk writes)
is merged with ongoing flush, we have to make sure that none of the new
disk I/O requests overlap with ones in in progress. If it does, hold the
request and dont issue I/O until the overlapping one finishes. If we allow
the I/O to proceed, we might end up with two overlapping requests in the
disk's queue and the disk may not offer any guarantee on which one is
written first.
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
hashtable.c has already been copied, forked and modified, there
doesn't seem much point in avoiding the same for the header until
someone feels motivated to properly refactor.
Add comments to the various duplicated files cross-referencing each
other for future reference and as a barrier to forking again...
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
A fair few things under tools link against libraries which they don't
even use.
Most of this appears to come from copy-and-pasting previous Makefile
snippets and cargo-culting plus the tendency to define global $(LIBS)
even for Makefiles which build multiple separate utilities or
libraries.
Identified by comparing a build with --as-needed to one without by
looking at the NEEDED header of all ELF objects.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
In particular if binary A uses libB and libB uses libC entirely
internally then A does not need to link against libC only libB.
However when linking binary A the linker does need to have visibility
of the libraries which libB links against (libC in this example). For
out of tree uses this is achieved without fuss due because the
libraries are installed in a standard path. However in the case of
in-tree users the linker needs a hint in the form of the -rpath-link
option. Therefore a new class of build variable, $(SHLIB_FOO), is
introduced which includes the linker options needed to link against a
library which uses libFOO. The intention is that $(LDLIBS_bar) will
include the $(SHLIB_foo)s which it uses where necessary rather
requiring that users are aware of this.
For the python extensions this change appears particularly large since
previously each of python bindings were linked against the union of
all possible libraries used by all bindings instead of just what they
individually needed.
This change removes a dependency on libdl.so from nearly everything
in the system, only libxenctrl actually uses it.
In the context of xl/libxl the intention of libxl is to remove any
need for a user of libxl to know about libxenstore or libxenctrl,
however in the current build it is xl which links against those
libraries rather than libxl (which only links against libc). After
this change libxl correctly depends on the libraries it uses and xl
does not depend on libraries which it is not support to be required to
know about. Note that xl does depend on libxenctrl.so since it uses
xtl_* directly.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Otherwise make can search the path relative to certain standard paths
such as /usr/include (e.g., the line '-include $(XEN_ROOT)/.config' in
Config.mk suffers from this).
Signed-off-by: Keir Fraser <keir@xen.org>
Make the DISK_TYPE_* id numbering in tapdisk-disktypes.h contiguous.
Currently, id 8 is unallocated causing a null disk type entry in
tapdisk_disk_drivers array in tapdisk-disktypes.c. This causes the
function tapdisk_disktype_find() to return an error on encountering
disk types >7 (remus:, log:, etc.).
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca>
Acked-by: Daniel Stodden <daniel.stodden@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Fix tapdisk-disktype.c's initialization for remus' disk_info_t,
which is currently initializing the disk name with disk description.
Signed-off-by: Shriram Rajagopalan <rshriram@cs.ubc.ca>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
The former is a userspace sanitised header which contains the
definitions we need. In some distros linux/fs.h defines WRITE which
conflicts with blktaps own use of that name.
Also there is no reason to use <linux/errno.h> over the more normal
<errno.h>.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
Linker command lines are order-sensitive.
Move linker options -Lfoo -lfoo from LDFLAGS to LDLIBS and place this new
variable after the objects to link. This resolves build errors in xenpagin
and blktap with recent toolchains.
rename SHLIB_CFLAGS to SHLIB_LDFLAGS
rename LDFLAGS_* to LDLIBS_*
move LDFLAGS usage after CFLAGS in CC calls
remove stale comments in xenpaging Makefile
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
I don't think protocol specific data really belongs in this header
but since it is already there and we seem to be stuck with it let's at
least make the users explicit lest people get caught out by future new
fields moving the pad field around.
This is the Xen portion of this change. The kernel portion will be
sent separately. There is no dependency between the two.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Cc: Daniel Stodden <daniel.stodden@citrix.com>
Cc: Dongxiao Xu <dongxiao.xu@intel.com>
Tapdisk control in userspace, a replacement for the original blktap2
control stack, which had to pass a kernel space interface based on
sysfs nodes.
All tapdisk processes listen for commands on a unix stream socket. The
control library supports scanning the socket namespace for running
tapdisks, VBD minors allocated, associated images and state inquiry.
Control operations include allocating/releasing devices, spawning
tapdisks, opening/closing images, attaching disk images to
devices. disk pause/resume operations and runtime switching of disk
images.
Signed-off-by: Jake Wires <jake.wires@citrix.com>
Signed-off-by: Daniel Stodden <daniel.stodden@citrix.com>
Stop coercing drivers/disktype code into the tool stack. Make both
blktapctrl and tap-ctl transfer type/path pairs as "<type>:<path>"
strings. Remove the message.disktype integer altogether.
Signed-off-by: Daniel Stodden <daniel.stodden@citrix.com>
Signed-off-by: Jake Wires <jake.wires@citrix.com>
1. Some files use stat, mkfifo, mkdir etc. without including
sys/stat.h
2. Some programs link against libpthread without a -lpthread compile
option. The compile used to work if this library happened to be used
by one of the other libraries that was being linked against, but
Fedora 13 has stopped allowing this.
From: M A Young <m.a.young@durham.ac.uk>
Signed-off-by: Keir Fraser <keir.fraser@citrix.com>
500ms is aggressive enough to trigger split-brain under fairly
ordinary workloads, particularly for HVM. The long-term fix is to
integrate with a real HA monitor like linux HA.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
Mainline kernel support for eventfd(2) in linux aio was added between
2.6.21 and 2.6.22. Libaio after 0.3.107 has the header file, but
presently few systems support it. Neither do we rely on an up-to-date
libc6.
Instead, this patch adds a header which defines custom iocb_common
struct, and works around a potentially missing sys/eventfd.h.
Signed-off-by: Daniel Stodden <daniel.stodden@citrix.com>
Hide tapdisk support for different raw I/O interfaces behind a new
struct tio. Libaio remains to dominate the interface, requiring
everyone to dispatch iocb/ioevent structs.
Backends:
- lio: Kernel AIO via libaio.
- rwio: Canonical read/write() mode.
Misc:
- Fixes a bug in tapdisk-vbd which locks up the sync io mode.
- Wants a PERROR macro in blktaplib.h
- Removes dead code in qcow2raw to make it link again.
Signed-off-by: Daniel Stodden <daniel.stodden@citrix.com>
Signed-off-by: Jake Wires <jake.wires@citrix.com>
Move event callbacks registration into tapdisk-queue. This should also
obsoletes the dummy pollfd pipe in the synchronous I/O case.
Signed-off-by: Daniel Stodden <daniel.stodden@citrix.com>
Attached patch makes memshr optional for blktap/blktap2.
This fixes build for platforms where memshr isn't build on.
While there, make indentation consistent.
Signed-off-by: Christoph Egger <Christoph.Egger@amd.com>
detect sharable memory pages). This patch establishes unique id for each disk
image opened by tapdisk2, and stores it in shared memory region, thus making it
available to the remaining tapdisk2s.
Signed-off-by: Grzegorz Milos <Grzegorz.Milos@citrix.com>
the tapdisk2 process if sharing is enabled (tapdisk2 is not normally aware what
domain it is working for).
Signed-off-by: Grzegorz Milos <Grzegorz.Milos@citrix.com>
a single map is used to store mappings between sharing handles and disk blocks.
This is used to share pages which store data read of the same blocks on
(virtual) disk.
Note that the map is stored in a shared memory region, as it needs to be
accessed by multiple tapdisk processes. This complicates memory allocation
(malloc cannot be used), prevents poniters to be stored directly (as the shared
memory region might and is mapped at different base address) and finally pthread
locks need to be multi-process aware.
Signed-off-by: Grzegorz Milos <Grzegorz.Milos@citrix.com>
(4K) block read into 8 (512 bytes) sector reads. This is inefficient, but also
prevents sharing detector from working, as it is based on page-size reads.
Signed-off-by: Grzegorz Milos <Grzegorz.Milos@citrix.com>
If we want to check the functionality of libgcrypt, we shouldn't test
a function only exported by openssl, but instead the one actually used
in the code.
Signed-off-by: Andre Przywara <andre.przywara@amd.com>
Blktap2 port of remus disk driver. Backwards compatable with blktap1
implementation.
Signed-off-by: Ryan O'Connor <rjo@cs.ubc.ca>
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
Currently blktap2 opens a driver stack, closes it, and re-opens
it. This causes problems with our remus driver: the primary may
connect to the backup in between the first and second open.
This is a temporary fix.
Signed-off-by: Ryan O'Connor <rjo@cs.ubc.ca>
Blktap2 allows block device drivers to be layered to create more
advanced virtual block devices. However, composing a layered driver is
not exposed to the user. This patch fixes this, and allows the user to
explicitly specify a driver chain when starting a tapdisk process,
using the pipe character ('|') to explicitly seperate layers in a
blktap2 configuration string.
for example, the command:
~$ tapdisk2 -n "log:|aio:/path/to/file.img"
will create a blktap2 device where read and write requests are passed
to the 'log' driver, then forwarded to the 'aio' driver.
Signed-off-by: Ryan O'Connor <rjo@cs.ubc.ca>
Andres Lagar-Cavilla
Roger Pau Monne
Ian Campbell
Olaf Hering
Philipp Hahn
Shriram Rajagopalan
Ian Jackson
Keir Fraser
Gianni Tedesco
Stefano Stabellini
Keir Fraser