In libxl_set_memory_target when setting the new maxmem, retain the same
offset on top of the current target. In the future the offset will
include memory allocated by QEMU for rom files.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
XenServer's VM density testing uncovered a regression when moving from
sysvinit to systemd where the file descriptor limit dropped from 4096 to
1024. (XenServer had previously inserted a ulimit statement into its
initscripts.)
One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the
lost ulimit, but that is only a stopgap solution.
As Xenconsoled genuinely needs a large number of file descriptors if a large
number of domains are running, attempt to increase the limit.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
freemem_slack accounts for the amount of memory to be left free in the
system because empirical experiments seem to demonstrate that is needed
for "stability reasons".
As we don't have any actual data on these stability issues, remove it.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: mlatimer@suse.com
CC: ian.campbell@citrix.com
Acked-by: Mike Latimer <mlatimer@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Make some internal routines common so that total/local memory bandwidth
monitoring in the next patch can make use of them.
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
- space: remove space after '(' or before ')' in 'if' condition;
- indention: align function definition/call arguments;
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Provide mini-os url and revision in Config.mk
Make stubdom targets depend on mini-os-dir target. Make
subtree-force-update{,-all} depend on mini-os-dir-force-update.
Also make mktarball script generate mini-os archive.
Original mini-os directory is renamed to mini-os-intree to help reduce
patch length. That directory will be deleted in a separate patch.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
Changes in v3:
1. Put mini-os targets in main Makefile.
2. Stubdom clean and distclean don't depend on mini-os-dir.
Changes in v2:
1. Use hash in Config.mk.
When building hvmloader for Xen tools the ipxe objects are also linked
into the binary. Unfortunately the linker will place them in the order
found in the archive. Since this order is random the resulting hvmloader
binary differs when it was built from identical sources but on different
build hosts. To help with creating a reproducible binary the elements in
blib.a must simply be sorted before passing them to $(AR).
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Don't kill xenstored as part of the usual service shutdown process to
prevent hangs on shutdown where the kernel tries to unplug a VIF
after xenstored has exited.
In an ideal case with all guests cooperating, xendomains will have shut
down all guests before xenstored is killed.
However in the uncooperative case, malicious or crashed guests may still
be running after xendomains has exited and this should not block the
shutdown/reboot of dom0.
Xenstored has no state to sync to disk, and never used to be killed in
the sysvinit case; observe the warning in xencommons. Our testing has
shown regressions caused by the change in behaviour between sysvinit and
systemd when it comes to killing xenstored.
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
[ ijc -- added systemd to title ]
Coverity uses several heuristics to identify when one case statement
legitimately falls through into the next, and a comment as the final item in a
case statement is one heuristic (the assumption being that it is a
justification for the fallthrough).
Use this to perform an audit of defects and hide the legitimate fallthroughs.
There are two bugfixes identified in the audit, both minor:
* 'n' command line handling for gtracestat
* BKSPC handling in xentop
All other identified defaults are legitimate fallthoughs
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Coverity-IDs: 1055464, 1055465, 1055467, 1055468, 1055481, 1055482
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Xen Coverity Team <coverity@xen.org>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Don Koch <dkoch@verizon.com>
Avoid late build failure in openSUSE 11.4, it has just pixman-0.20:
....
[ 211s] ERROR: pixman >= 0.21.8 not present. Your options:
[ 211s] (1) Preferred: Install the pixman devel package (any recent
[ 211s] distro should have packages as Xorg needs pixman too).
[ 211s] (2) Fetch the pixman submodule, using:
[ 211s] git submodule update --init pixman
....
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
In 674ad2b (xenstore: extend the xenstore ring with a 'closing' signal)
two uses of uint32 are added to tools/ocaml/libs/xb/xs_ring_stubs.c .
As of ocaml 4.03.0+dev the uint32 type is no longer supported. This patch
replaces the uses of uint32 with uint32_t .
Signed-off-by: Michael Young <m.a.young@durham.ac.uk>
Acked-by: David Scott <dave.scott@citrix.com>
XEN_DOMCTL_destroydomain so far is being special cased in libxc to
reinvoke the operation when getting back EAGAIN. Quite a few other
domctl-s have gained continuations, so I see no reason not to use them
here too.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Hvmloader issues XENMEM_get_vnumainfo hypercall and stores the
information retrieved in scratch space for later use.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
In libxl_ctx_free:
1. Move the GC_INIT earlier, so that we can:
2. Take the lock around most of the work. This is technically
unnecessary since calling any other libxl entrypoint on a ctx being
passed to libxl_ctx_free risks undefined behaviour. But, taking
the lock allows us to much more usually spot this.
3. Increment osevent_in_hook by 1000. If we are reentered after
destroy, this will trip some of our entrypoints' asserts. It also
means that if we crash for some other reason relating to reentering
a destroyed ctx, the cause will be more obviously visible by
examining ctx->osevent_in_hook (assuming that the memory previously
used for the ctx hasn't been reused and overwritten).
4. Free the lock again. (pthread_mutex_destroy requires that the
mutex be unlocked.)
With this patch, I find that an occasional race previously reported
as:
libvirtd: libxl_internal.h:3265: libxl__ctx_unlock: Assertion `!r' failed.
is now reported as:
libvirtd: libxl_event.c:1236: libxl_osevent_occurred_fd: Assertion `!libxl__gc_owner(gc)->osevent_in_hook' failed.
Examining the call trace with gdb shows this:
(gdb) bt
#0 0xb773f424 in __kernel_vsyscall ()
#1 0xb7022871 in raise () from /lib/i386-linux-gnu/i686/nosegneg/libc.so.6
#2 0xb7025d62 in abort () from /lib/i386-linux-gnu/i686/nosegneg/libc.so.6
#3 0xb701b958 in __assert_fail () from /lib/i386-linux-gnu/i686/nosegneg/libc.so.6
#4 0xb6f00390 in libxl_osevent_occurred_fd (ctx=0xb84813a8, for_libxl=0xb84d6848, fd=31, events_ign=0, revents_ign=1) at libxl_event.c:1236
#5 0xb1b70464 in libxlDomainObjFDEventCallback () from /usr/local/lib/libvirt/connection-driver/libvirt_driver_libxl.so
#6 0xb72163b1 in virEventPollDispatchHandles () from /usr/local/lib/libvirt.so.0
#7 0xb7216be5 in virEventPollRunOnce () from /usr/local/lib/libvirt.so.0
#8 0xb7214a7e in virEventRunDefaultImpl () from /usr/local/lib/libvirt.so.0
#9 0xb77c7b98 in virNetServerRun ()
#10 0xb7771c63 in main ()
(gdb) print ctx->osevent_in_hook
$2 = 1000
(gdb)
which IMO demonstrates that libxl_osevent_occurred_fd is being called
on a destroyed ctx.
This is probably a symptom of the bug in libvirt fixed by these
patches:
https://www.redhat.com/archives/libvir-list/2015-February/msg00024.html
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Jim Fehlig <jfehlig@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
libxl__ao_inprogress needs to check (like
libxl__ao_complete_check_progress_reports) that there are no
oustanding progress callbacks.
Otherwise it might happen that we would destroy the ao while another
thread has an outstanding callback its egc report queue. The other
thread would then, in its egc_run_callbacks, touch the destroyed ao.
Instead, when this happens in libxl__ao_inprogress, simply run round
the event loop again. The thread which eventually makes the callback
will spot our poller in the ao, and notify the poller, waking us up.
This fixes an assertion failure race seen with libvirt:
libvirtd: libxl_event.c:1792: libxl__ao_complete_check_progress_reports: Assertion `ao->in_initiator' failed.
or (after "Add an assert to egc_run_callbacks")
libvirtd: libxl_event.c:1338: egc_run_callbacks: Assertion `aop->ao->magic == 0xA0FACE00ul' failed.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Jim Fehlig <jfehlig@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Break out the test in libxl__ao_complete_check_progress_reports, into
ao_work_outstanding, which reports false if either (i) the ao is still
ongoing or (ii) there is a progress report (perhaps on a different
thread's callback queue) which has yet to be reported to the
application.
No functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Jim Fehlig <jfehlig@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Check that the ao is still live when we are about to running some of
its callbacks.
This reveals an existing bug in libxl which is exercised by libvirt,
converting
libvirtd: libxl_event.c:1792: libxl__ao_complete_check_progress_reports: Assertion `ao->in_initiator' failed.
into
libvirtd: libxl_event.c:1338: egc_run_callbacks: Assertion `aop->ao->magic == 0xA0FACE00ul' failed.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Ian Campbell <ian.campbell@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Jim Fehlig <jfehlig@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Some systems have python-config include -D_FORTIFY_SOURCE in the
CFLAGS. But -D_FORTIFY_SOURCE does not (currently) work with -O0, and
-O0 is enabled in debug builds (since 1166ecf781). As a result, on
those systems, debug builds fail.
Work around this problem as follows:
* In configure, detect -D_FORTIFY_SOURCE in $(python-config --cflags)
* If detected, set the new autoconf substitution and make variable
PY_NOOPT_CFLAGS to -O1.
* In tools/Rules.mk, where we add -O0, also add PY_NOOPT_CFLAGS
(which will override the -O0 with -O1 if required).
Overriding the -O0 is better than disabling Fortify because the
latter might have an adverse security impact. A user who wants to
disable optimisation completely even for Python and also disable
Fortify can set the environment variable
EXTRA_CFLAGS_XEN_TOOLS='-U_FORTIFY_SOURCE -O0'
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Reported-by: Jan Beulich <JBeulich@suse.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Euan Harris <euan.harris@citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: Don Slutz <dslutz@verizon.com>
Both vchan_wr() and stdout_wr() should be defined with a non-empty
argument list (i.e. void). Additionally both of them as well as usage()
should be static to make clear that no other code is referencing them.
Further, statements should follow declarations.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
During domain startup, all required memory ballooning must complete
within a maximum window of 33 seconds (3 retries, 11 seconds of delay).
If not, domain creation is aborted with a 'failed to free memory' error.
In order to accommodate large domains or slower hardware (which require
substantially longer to balloon memory) the free memory process should
continue retrying if the amount of free memory is increasing on each
iteration of the loop.
Signed-off-by: Mike Latimer <mlatimer@suse.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Starting from rump kernel changeset 91d5623 ("Renaming platform macros,
app-tools and autoconf target string"), __RUMPUSER_XEN__ and __RUMPAPP__
are deleted. We are supposed to use __RUMPRUN__ instead.
We still keep __RUMPUSER_XEN__ for now in order to make xen-unstable
pass osstest push gate. I will remove __RUMPUSER_XEN__ later.
Related discussion:
http://thread.gmane.org/gmane.comp.rumpkernel.user/739
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Linux is in the process of converting their seconds representation to
64 bits, so in order to support it consistently we should follow suit
(which at some point in quite a few years we'd have to do anyway). To
represent this in struct shared_info we leverage a 32-bit hole in
x86-64's and arm's variant of the structure; for x86-32 guests the only
(reasonable) choice we have is to put the extension in struct
arch_shared_info.
A note on the conditional suppressing the xen_wc_sec_hi helper macro
definition in the ix86 case for hypervisor and tools: Neither of the
two actually need this, and its presence causes the tools to fail to
build (due to the inclusion of both the x86-64 and x86-32 variants of
the header).
As a secondary change, x86's do_platform_op() gets a pointless
initializer as well as a pointless assignment of that same variable
dropped.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
Acked-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
The Ocaml stub to retrieve the hypervisor console ring had a few problems.
* A single 32k buffer would truncate a large console ring.
* The buffer was static and not under the protection of the Ocaml GC lock so
could be clobbered by concurrent accesses.
* Embedded NUL characters would cause caml_copy_string() (which is strlen()
based) to truncate the buffer.
The function is rewritten from scratch, using the same algorithm as the python
stubs, but uses the protection of the Ocaml GC lock to maintain a static
running total of the ring size, to avoid redundant realloc()ing in future
calls.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Dave Scott <dave.scott@eu.citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: David Scott <dave.scott@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
When commit 6865e52b78, "PCI multi-seg: adjust domctl interface",
is introduced, we missed to sync that head file.
Signed-off-by: Tiejun Chen <tiejun.chen@intel.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
c/s 5b5c40c0d1 "libxc: introduce a per architecture scratch pfn for temporary
grant mapping" accidentally an issue whereby there were two paths out of
xc_core_arch_get_scratch_gpfn() which returned 0, but only one of which
assigned a value to the gpfn parameter.
xc_domain_maximum_gpfn() can validly return 0, at which point gpfn 1 is a
valid scratch page to use.
In addition, widen rc before adding 1 and possibly overflowing.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Julien Grall <julien.grall@linaro.org>
CC: Jan Beulich <JBeulich@suse.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
If the new target is relative to the current target, do not remove
videoram again: it has already been removed from the current target.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The legacy save/restore infrastructure requires several function pointers from
the toolstack (libxl or Xend in the past) in order to work, and for HVM guests
also need to be able to play around in dom0's filesystem to move the device
model save record.
Migration v2 changes some of this, but is similarly dependent on
toolstack-provided function pointers.
Someone who wishes to re-architect the interaction of moving parts for running
a domain might be in a position to re-enabled this, but for now, explicitly
fail with ENOSYS (from xc_nomigrate.c) rather than failing with an error about
a missing function pointer (or indeed falling over a NULL pointer on certain
paths).
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Attempting to build libxl causes Make to emit the following warnings
andrewcoop@andrewcoop:xen.git$ make -C tools/libxl all
...
Makefile:253: target `xenlight.pc' doesn't match the target pattern
Makefile:253: target `xlutil.pc' doesn't match the target pattern
...
because the static pattern rule is malformed. 'Makefile' as the only
prereq-pattern does not contain a pattern.
The rule ends up working because of the use of $@.in where $< should have been
used, but lacked any dependency between a $FOO.pc and its .in source file.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
At present on shutdown when using pci-passthrough with qemu-xen, qemu
closes the QMP socket before libxl is done with it causing these
errors to be logged by libxl:
Waiting for domain test (domid 1) to die [pid 11568]
Domain 1 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 1 needs to be cleaned up: destroying the domain
libxl: error: libxl_qmp.c:443:qmp_next: Socket read error: Connection reset by peer
libxl: error: libxl_qmp.c:701:libxl__qmp_initialize: Failed to connect to QMP
libxl: error: libxl_qmp.c:686:libxl__qmp_initialize: Connection error: Connection refused
libxl: error: libxl_dm.c:1588:kill_device_model: Device Model already exited
Done. Exiting now
Prevent this by using the qemu '-no-shutdown' parameter which is
described as doing:
"Don’t exit QEMU on guest shutdown, but instead only stop the emulation.
This allows for instance switching to monitor to commit changes to the disk image."
So Qemu will stop emulating, but keeps the QMP socket open and waits
for libxl to kill the qemu process when it is done, preventing the
race and resulting in this to be logged by libxl:
Waiting for domain test (domid 1) to die [pid 10859]
Domain 1 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 1 needs to be cleaned up: destroying the domain
Done. Exiting now
Signed-off-by: Sander Eikelenboom <linux@eikelenboom.it>
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Usage:
spice_streaming_video=[filter|all|off]
Specifies what streaming video setting is to be used by spice (if
given),
otherwise the qemu default will be used.
Signed-off-by: Fabio Fantoni <fabio.fantoni@m2r.biz>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Usage:
spice_image_compression=[auto_glz|auto_lz|quic|glz|lz|off]
Specifies what image compression is to be used by spice (if given),
otherwise the qemu default will be used.
Signed-off-by: Fabio Fantoni <fabio.fantoni@m2r.biz>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The original rule to target "distclean" in tools/Rules.mk was in effect
"make clean". It should be "make distclean".
However not all Makefiles in subdirectories have distclean target
defined. So this patch also adds a bunch of distclean targets to various
Makefiles. They only depend on clean target and don't have any actions
in most cases.
With the patch applied, following command outputs 0 results:
find tools -name 'Makefile*' -exec grep -L 'distclean' {} \+ \
| grep -v ocaml | grep -v libfsimage
Ocaml and libfsimage are known to have distclean defined in a dedicated
rules file.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Stefano Stabellini
Andrew Cooper
Ian Jackson
Chao Peng
Ian Campbell
Wei Liu
Olaf Hering
Ross Lagerwall
Don Slutz
Michael Young
Jan Beulich
Mike Latimer
Tiejun Chen
Sander Eikelenboom
Fabio Fantoni