There is race when we clear the STATE_SCHED in the softirq
- which allows the 'raise_softirq_for' (on another CPU or
on the one running the softirq) to schedule the dpci.
Specifically this can happen when the other CPU receives
an interrupt, calls 'raise_softirq_for', and puts the dpci
on its per-cpu list (same dpci structure). Note that
this could also happen on the same physical CPU, however
the explanation for simplicity will assume two CPUs actors.
There would be two 'dpci_softirq' running at the same time
(on different CPUs) where on one CPU it would be executing
hvm_dirq_assist (so had cleared STATE_SCHED and set STATE_RUN)
and on the other CPU it is trying to call:
if ( test_and_set_bit(STATE_RUN, &pirq_dpci->state) )
BUG();
Since STATE_RUN is already set it would end badly.
The reason we can get his with this is when an interrupt
affinity is set over multiple CPUs.
Potential solutions:
a) Instead of the BUG() we can put the dpci back on the per-cpu
list to deal with later (when the softirq are activated again).
This putting the 'dpci' back on the per-cpu list is an spin
until the bad condition clears.
b) We could also expand the test-and-set(STATE_SCHED) in raise_softirq_for
to detect for 'STATE_RUN' bit being set and schedule the dpci.
The BUG() check in dpci_softirq would be replace with a spin until
'STATE_RUN' has been cleared. The dpci would still not
be scheduled when STATE_SCHED bit was set.
c) Only schedule the dpci when the state is cleared
(no STATE_SCHED and no STATE_RUN). It would spin if STATE_RUN is set
(as it is in progress and will finish). If the STATE_SCHED is set
(so hasn't run yet) we won't try to spin and just exit.
Down-sides of the solutions:
a). Live-lock of the CPU. We could be finishing an dpci, then adding
the dpci, exiting, and the processing the dpci once more. And so on.
We would eventually stop as the TIMER_SOFTIRQ would be set, which will
cause SCHEDULER_SOFTIRQ to be set as well and we would exit this loop.
Interestingly the old ('tasklet') code used this mechanism.
If the function assigned to the tasklet was running - the softirq
that ran said function (hvm_dirq_assist) would be responsible for
putting the tasklet back on the per-cpu list. This would allow
to have an running tasklet and an 'to-be-scheduled' tasklet
at the same time.
b). is similar to a) - instead of re-entering the dpci_softirq
we are looping in the softirq waiting for the correct condition to
arrive. As it does not allow unwedging ourselves because the other
softirqs are not called - it is less preferable.
c) can cause an dead-lock if the interrupt comes in when we are
processing the dpci in the softirq - iff this happens on the same CPU.
We would be looping in on raise_softirq waiting for STATE_RUN
to be cleared, while the softirq that was to clear it - is preempted
by our interrupt handler.
As such, this patch - which implements a) is the best candidate
for this quagmire.
Reported-and-Tested-by: Sander Eikelenboom <linux@eikelenboom.it>
Reported-and-Tested-by: Malcolm Crossley <malcolm.crossley@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Most callers have this available already, and the functions don't need
any vcpu specifics.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Tim Deegan <tim@xen.org>
being serviced by the RTDS scheduler, as that is a
legit situation to be in: think, for instance, of a
newly created RTDS cpupool, with no domains migrated
to it yet.
While there:
- move the spinlock acquisition up, to effectively
protect the domain list and avoid races;
- the mask of online pCPUs was being retrieved
but then not used anywhere in the function: get
rid of that.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
We don't need to try to destroy it since it can't be already allocated at the
time we try to initialize it.
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
instead of just list of single pCPUs or NUMA node IDs, as
it happens right now.
On the other hand, after this change, strings containing
pCPUs and NUMA node ranges is supported. The syntax is the
same one supported by the "cpus" and "cpus_soft" config
switch, i.e., "4-8" or "node:1,12-18,^14".
This make things more flexible, more consistent, and also
improves error handling, as the pCPU range parsing routine
already present in xl is more reliable than just a call
to atoi().
While there, remove a redundant error check in the legacy syntax
handling (libxl_bitmap_test() already checks the index being
within the size of the bitmap).
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Juergen Gross <JGross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
in fact, right now, xl sub-commands 'cpupool-cpu-add' and
'cpupool-cpu-remove' only accept the specification of one
pCPU to be added or removed to/from a cpupool.
With this change, they can deal with ranges, like "4-8",
or "node:1,12-18,^14". The syntax is exactly the same one
that is supported by the 'vcpu-pin' subcommand, and
specifying just one pCPU still works, of course.
This make things more flexible, more consistent, and also
improves error handling, as the pCPU range parsing routine
already present in xl is more reliable than just a call
to atoi().
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Juergen Gross <JGross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
To add (removes) to (from) a cpupool all the pCPUs corresponding
to the bits that are set in the passed bitmap.
This is convenient and useful in order to implement, in xl,
the possibility of specifying ranges of pCPUs to be added
(removed) to (from) a cpupool, in the relevant commands.
While there, convert libxl_cpupool_cpu{add,remove} to use the
appropriate logging macro (LOGE()).
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Juergen Gross <JGross@suse.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The definitions of static device labels must be placed at the end of the
policy.conf before passing it to checkpolicy; the existing examples
(which are commented out) are in the wrong location. Create a new file
for device contexts which will place them in the proper location.
This also removes some directions about using the xen policy type in
checkpolicy which is no longer needed.
Reported-by: Julien Grall <julien.grall@linaro.org>
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
These APIs can be used to manipulate XLU_ConfigValue and XLU_ConfigList.
APIs introduced:
1. xlu_cfg_value_type
2. xlu_cfg_value_get_string
3. xlu_cfg_value_get_list
4. xlu_cfg_get_listitem2
Move some definitions from private header to public header as needed.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
This patches does following things:
1. Properly define a XLU_ConfigList type. Originally it was defined to
be XLU_ConfigSetting.
2. Define XLU_ConfigValue type, which can be either a string or a list
of XLU_ConfigValue.
3. ConfigSetting now references XLU_ConfigValue. Originally it only
worked with **string.
4. Properly construct list where necessary, see changes to .y file.
To achieve above changes:
1. xlu__cfg_set_mk and xlu__cfg_set_add are deleted, because they
are no more needed in the new code.
2. Introduce xlu__cfg_string_mk to make a XLU_ConfigSetting that points
to a XLU_ConfigValue that wraps a string.
3. Introduce xlu__cfg_list_mk to make a XLU_ConfigSetting that points
to XLU_ConfigValue that is a list.
4. The parser now generates XLU_ConfigValue instead of XLU_ConfigSetting
when construct values, which enables us to recursively generate list
of lists.
5. XLU_ConfigSetting is generated in xlu__cfg_set_store.
6. Adapt other functions to use new types.
No change to public API. Xl compiles without problem and 'xl create -n
guest.cfg' is valgrind clean.
This patch is needed because we're going to implement nested list
support, which requires support for list of list.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Disallow memory relocation when vNUMA is enabled, because relocated
memory ends up off node. Further more, even if we dynamically expand
node coverage in hvmloader, low memory and high memory may reside
in different physical nodes, blindly relocating low memory to high
memory gives us a sub-optimal configuration.
Introduce a function called libxl__vnuma_configured and use it.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Konrad Wilk <konrad.wilk@oracle.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Transform user supplied vNUMA configuration into libxl internal
representations then libxc representations. Check validity along the
line.
Libxc has more involvement in building vmemranges in HVM case compared
to PV case. The building of vmemranges is placed after xc_hvm_build
returns, because it relies on memory hole information provided by
xc_hvm_build.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
The algorithm is more or less the same as the one used for PV guest.
Libxc gets hold of the mapping of vnode to pnode and size of each vnode
then allocate memory accordingly.
And then the function returns low memory end, high memory end and mmio
start to caller. Libxl needs those values to construct vmemranges for
that guest.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Introduce a arch-independent routine to generate one vmemrange per
vnode. Also introduce arch-dependent routines for different
architectures because part of the process is arch-specific -- ARM has
yet have NUMA support and E820 is x86 only.
For those x86 guests who care about machine E820 map (i.e. with
e820_host=1), vnode is further split into several vmemranges to
accommodate memory holes. A few stubs for libxl_arm.c are created.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
This function is used to check whether vNUMA configuration (be it
auto-generated or supplied by user) is valid.
Define a new error code ERROR_VNUMA_CONFIG_INVALID.
The checks performed can be found in the comment of the function.
This vNUMA function (and future ones) is placed in a new file called
libxl_vnuma.c
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
A vnode consists of one or more vmemranges (virtual memory range). One
example of multiple vmemranges is that there is a hole in one vnode.
Currently we haven't exported vmemrange interface to libxl user.
Vmemranges are generated during domain build, so we have relevant
structures in domain build state.
Later if we discover we need to export the interface, those structures
can be moved to libxl_domain_build_info as well.
These new fields (along with other fields in that struct) are set to 0
at start of day so we don't need to explicitly initialise them. A
following patch which introduces an independent checking function will
need to access these fields. I don't feel very comfortable squashing
this change into that one so I didn't use a single commit.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
From libxc's point of view, it only needs to know vnode to pnode mapping
and size of each vnode to allocate memory accordingly. Add these fields
to xc_dom structure.
The caller might not pass in vNUMA information. In that case, a dummy
layout is generated for the convenience of libxc's allocation code. The
upper layer (libxl etc) still sees the domain has no vNUMA
configuration.
Note that for this patch on PV x86 guest can have multiple regions of
ram allocated.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Elena Ufimtseva <ufimtseva@gmail.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Add a new field p2m_size to keep track of the number of pages covered by
p2m. Change total_pages to p2m_size in functions which in fact need
the size of p2m.
This is needed because we are going to ditch the assumption that PV x86
has only one contiguous ram region. Originally the p2m size was always
equal to total_pages, but we will soon change that in later patch.
This patch doesn't change the behaviour of libxc.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Dario Faggioli <dario.faggioli@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Currently all in tree code doesn't set the superpage flag, I would just
remove superpage support if I can, but Konrad wants it retained for the
moment.
As I'm going to change the p2m_host array allocation, duplicate the code
snippet to allocate p2m_host array in this patch, so that we retain the
behaviour in superpage case.
This patch introduces no functional change and it will make future patch
easier to review. Also removed one stray tab while I was there.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Konrad Wilk <konrad.wilk@oracle.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Typedefs are duplicated in stubdom/vtpmmgr/tcg.h and supported compilers
do not cope with current staging branch.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Quan Xu <quan.xu@intel.com>
Tested-by: Olaf Hering <olaf@aepfle.de>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
[ ijc -- reworded subject line ]
POLLHUP|POLLIN is a valid revent to receive when there is readable data in a
pipe, but the writable fd has been closed. This occurs in migration v2 when
the legacy conversion process (which transforms the data inline) completes and
exits successfully.
In the case that there is data to read, suppress the POLLHUP. POSIX states
that the hangup state is latched[1], which means it will reoccur on subsequent
poll() calls. The datacopier is thus provided the opportunity to read until
EOF, if possible.
A POLLHUP on its own is treated exactly as before, indicating a different
error with the fd.
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/poll.html
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Currently a datacopier may source its data from an fd or local buffer, but its
destination must be an fd. For migration v2, libxl needs to read from the
migration stream into a local buffer.
Implement a "read into local buffer" mode, invoked when readbuf is set and
writefd is -1. On success, the callback passes the number of bytes read.
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
[Rewrite commit message]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Currently, a datacopier will unconditionally read until EOF on its read fd.
For migration v2, libxl needs to read records of a specific length out of the
migration stream, without reading any further data.
Introduce a parameter, bytes_to_read, which may be used to stop the datacopier
ahead of reaching EOF. If bytes_to_read is set to -1, then the datacopier will
read until EOF.
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
[Rewrite commit message]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
An individual datacopier_buf contains a static buffer of 1000 bytes.
Attempting to add prefixdata of more than 1000 bytes would overrun the buffer
and cause heap corruption.
Instead, split the prefixdata and chain together multiple datacopier buffers.
This allows for an arbitrary quantity of prefixdata to be added to a
datacopier.
Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Currently, starting a datacopier requires a valid read and write fd, but this
is a problem when purely sending data from a local buffer to a writable fd.
The prefixdata mechanism already exists and works for inserting data from a
local buffer ahead of reading from the read fd.
Make the lack of a read fd non-fatal. A datacopier with no read fd, but some
prefixdata will write the prefixdata to the write fd and complete successfully.
Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
[Rewrite commit message]
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
and more consistent. In fact, before this changes, it looks
like this:
(XEN) VCPU information and callbacks for domain 0:
(XEN) VCPU0: CPU4 [has=F] poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={4} cpu_affinity={0-15}
(XEN) cpu_soft_affinity={0-15}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
After, it looks like this:
(XEN) VCPU information and callbacks for domain 0:
(XEN) VCPU0: CPU4 [has=F] poll=0 upcall_pend=00 upcall_mask=00 dirty_cpus={4}
(XEN) cpu_hard_affinity={0-15} cpu_soft_affinity={0-15}
(XEN) pause_count=0 pause_flags=1
(XEN) No periodic timer
So, consistently _not_ put space between fields and '=',
and consistently _not_ use ',' as separator. Also, put the
info about affinity on the same, properly indented.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@eu.citrix.com>
Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>
which is called by cpu_schedule_down(), and is necessary
for resetting the spinlock pointers in schedule_data from
the RTDS global runqueue lock, back to the default _lock
fields in the struct.
Not doing so causes Xen to explode, e.g., when removing
pCPUs from an RTDS cpupool and assigning them to another
one.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Reviewed-by: Meng Xu <mengxu@cis.upenn.edu>
Reviewed-by: George Dunlap <george.dunlap@eu.citrix.com>
Make XENMEM_increase_reservation and XENMEM_populate_physmap
vNUMA-aware.
That is, if guest requests Xen to allocate memory for specific vnode,
Xen can translate vnode to pnode using vNUMA information of that guest.
XENMEMF_vnode is introduced for the guest to mark the node number is in
fact virtual node number and should be translated by Xen.
XENFEAT_memory_op_vnode_supported is introduced to indicate that Xen is
able to translate virtual node to physical node.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Hand struct domain via struct memops_arg's respective field to
construct_memop_from_reservation().
Signed-off-by: Jan Beulich <jbeulich@suse.com>
This adds support in the hypervisor and policy build toolchain for
Xen/Flask policy version 30, which adds the ability to label ARM device
tree nodes and expands the IOMEM ocontext entries to 64 bits.
Signed-off-by: Daniel De Graaf <dgdegra@tycho.nsa.gov>
Tested-by: Julien Grall <julien.grall@linaro.org>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
for events that are specific to Credit2 (as it happens for Credit1
already). Re-use Credit1's vcpu_check counter by making it generic.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: George Dunlap <george.dunlap@eu.citrix.com>
Konrad Rzeszutek Wilk
Ian Campbell
Jan Beulich
Dario Faggioli
Boris Ostrovsky
Koushik Chakravarty
Pramod Devendra
Daniel De Graaf
Wei Liu
Quan Xu
Andrew Cooper
Ross Lagerwall
Wen Congyang