assert is not async-signal-safe.
In practice the effect of calling assert there is that if the
assertion fails we might get a secondary crash, or other undesirable
behaviour from stdio (which is how assert usually reports failures).
Mention in a comment in libxl__self_pipe_wakeup that it has to be
async-signal-safe.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
In the case that fd is -1, preserve errno and don't attempt to set
CLOEXEC.
Note that the implementation can still fail, as it ignores fcntl()
errors and may not set CLOEXEC properly. Update the documentation
accordingly until it is fixed.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Wei Liu <wei.liu2@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
Fixing the fnctl() error issue involves more TUITs than I currently
have.
New in v4
The callbacks actually ignore this except for logging, but we should
log the correct pid.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
---
v4: New patch in this version of the series.
We want to have no fd events registered when we are idle. This
implies that we must be able to deregister our interest in the sigchld
self-pipe fd, not just modify to request no events.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Tested-by: Ian Campbell <ian.campbell@citrix.com>
Release-Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
This avoids the process being forked while a carefd is recorded in the
list but the actual fd has been closed. If that happened, a
subsequent libxl_postfork_child_noexec would attempt to close the fd
again. If we are lucky that results in a harmless warning; but if we
are unlucky the fd number has been reused and we close an unrelated
fd.
This race has not been observed anywhere as far as we are aware.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
CC: George Dunlap <george.dunlap@eu.citrix.com>
libxl_postfork_child_noexec would nestedly reaquire the non-recursive
"no_forking" mutex: atfork_lock uses it, as does sigchld_user_remove.
The result on Linux is that the process always deadlocks before
returning from this function.
This is used by xl's console child. So, the ultimate effect is that
xl with pygrub does not manage to connect to the pygrub console.
This behaviour was reported by Michael Young in Xen 4.4.0 RC5.
Also, the use of sigchld_user_remove in libxl_postfork_child_noexec is
not correct with SIGCHLD sharing. libxl_postfork_child_noexec is
documented to suffice if called only on one ctx. So deregistering the
ctx it's called on is not sufficient. Instead, we need a new approach
which discards the whole sigchld_user list and unconditionally removes
our SIGCHLD handler if we had one.
Prompted by this, clarify the semantics of
libxl_postfork_child_noexec. Specifically, expand on the meaning of
"quickly" by explaining what operations are not permitted; and
document the fact that the function doesn't reclaim the resources in
the ctxs.
And add a comment in libxl_postfork_child_noexec explaining the
internal concurrency situation.
This is an important bugfix. IMO the bug is a blocker for Xen 4.4.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Reported-by: M A Young <m.a.young@durham.ac.uk>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: George Dunlap <george.dunlap@eu.citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Release-acked-by: George Dunlap <george.dunlap@eu.citrix.com>
(cherry picked from commit 5be1e95318)
Use the new libxl__pipe_nonblock and _close functions, rather than
open coding the same logic. Now the pipe is nonblocking, which avoids
a race which could result in libxl deadlocking in a multithreaded
program.
Reported-by: Jim Fehlig <jfehlig@suse.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Previously, an application which had multiple libxl ctxs in multiple
threads, would have to itself plumb SIGCHLD through to each ctx.
Instead, permit multiple libxl ctxs to all share the SIGCHLD handler.
We keep a list of all the ctxs which are interested in SIGCHLD and
notify all of their self-pipes.
In more detail:
* sigchld_owner, the ctx* of the SIGCHLD owner, is replaced by
sigchld_users, a list of SIGCHLD users.
* Each ctx keeps track of whether it is on the users list, so that
libxl__sigchld_needed and libxl__sigchld_notneeded now instead of
idempotently installing and removing the handler, idempotently add
or remove the ctx from the list.
We ensure that we always have the SIGCHLD handler installed
iff the sigchld_users list is nonempty. To make this a bit
easier we make sigchld_installhandler_core and
sigchld_removehandler_core idempotent.
Specifically, the call sites for sigchld_installhandler_core and
sigchld_removehandler_core are updated to manipulate sigchld_users
and only call the install or remove functions as applicable.
* In the signal handler we walk the list of SIGCHLD users and write
to each of their self-pipes. That means that we need to arrange to
defer SIGCHLD when we are manipulating the list (to avoid the
signal handler interrupting our list manipulation); this is quite
tiresome to arrange.
The code as written will, on the first installation of the SIGCHLD
handler, firstly install the real handler, then immediately replace
it with the deferral handler. Doing it this way makes the code
clearer as it makes the SIGCHLD deferral machinery much more
self-contained (and hence easier to reason about).
* The first part of libxl__sigchld_notneeded is broken out into a new
function sigchld_user_remove (which is also needed during for
postfork). And of course that first part of the function is now
rather different, as explained above.
* sigchld_installhandler_core no longer takes the gc argument,
because it now deals with SIGCHLD for all ctxs.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Ian Campbell <Ian.Campbell@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
---
v3: Include bugfixes from "Fixup SIGCHLD sharing" patch:
* Use a mutex for defer_sigchld, to guard against concurrency
between the thread calling defer_sigchld and an instance of the
primary signal handler on another thread.
* libxl_sigchld_owner_libxl_always is incompatible with SIGCHLD
sharing. Document this correctly.
Fix "have have" error in comment.
Move removal of newly unused variables to previous patch.
v2.1: Provide feature test macro LIBXL_HAVE_SIGCHLD_SHARING
We are going to want introduce another call site in the final
substantive patch.
Pure code motion; no functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Ian Campbell <Ian.Campbell@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
---
v3: Remove now-unused variables from sigchld_installhandler_core
We are going to change these functions so that different libxl ctx's
can share a single SIGCHLD handler. Rename them now to a new name
which doesn't imply unconditional handler installation or removal.
Also note in the comments that they are idempotent.
No functional change.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Ian Campbell <Ian.Campbell@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Applications exist which want to use libxl in an event-driven mode but
which do not integrate child termination into their event system, but
instead reap all their own children synchronously.
In such an application libxl must own SIGCHLD but avoid reaping any
children that don't belong to libxl.
Provide libxl_sigchld_owner_libxl_always_selective_reap which has this
behaviour.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Ian Campbell <Ian.Campbell@citrix.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
---
v2: Document the new mode in the big "Subprocess handling" comment.
Applications exist which don't keep track of all their child processes
in a manner suitable for coherent dispatch of their termination. In
such a situation, nothing in the whole process may call wait, or
waitpid(-1,,). Doing so reaps processes belonging to other parts of
the application and there is then no way to deliver the exit status to
the right place.
To facilitate this, provide a facility for such an application to ask
libxl to call waitpid on each of its children individually.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Ian Campbell <Ian.Campbell@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
This is a simple error-handling wrapper for waitpid. We're going to
want to call waitpid somewhere else and this avoids some of the
duplication.
No functional change in this patch. (Technically, we used to check
chldmode_ours again in the EINTR case, and don't now, but that can't
have changed because we continuously hold the libxl ctx lock.)
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Cc: Ian Campbell <Ian.Campbell@citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
An application which uses libxl_osevent_register_hooks, and doesn't
use libxl_sigchld_owner_mainloop, would never properly experience the
deaths of its (libxl) children.
This is because the SIGCHLD self pipe would be handled using ad-hoc
code in beforepoll_internal and afterpoll_internal. However, this is
no good if application is using the fd registration system instead; in
that case these functions would not be called and nothing would deal
with readability of the self pipe.
Fix this as follows:
The SIGCHLD self pipe now is now dealt with by a standard libxl__ev_fd
handler, which is registered and deregistered along with the SIGCHLD
handler itself. The handler function subsumes the ad-hoc response
code removed from beforepoll_internal, and the functionality of
libxl__fork_selfpipe_woken.
This is also tidier as the SIGCHLD self pipe is no longer touched
outside libxl_fork.c other than in ctx initialisation and teardown.
(The ad-hoc arrangements for poller->wakeup_pipe in
beforepoll_internal and afterpoll_internal are OK because the
libxl__poller mechanism exists to wake up threads which are sitting
inside libxl's poll loop, so is not applicable to the application's
event loop.)
Reported-by: Bamvor Jian Zhang <bjzhang@suse.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Bamvor Jian Zhang <bjzhang@suse.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
Previously, libxl_sigchld_owner_libxl_always was mishandled. It would
result in libxl paying no attention to the sigchld self pipe.
Fix this by fixing chldmode_ours so that it returns true iff we are
supposed to be handling SIGCHLD.
Additionally, we arrange to use chldmode_ours everywhere where we are
installing/removing signal handlers and/or deciding whether to check
the self pipe, etc. This means it needs a new "creating" flag
argument for the benefit of libxl__ev_child_fork, which needs to
install the signal handler in libxl_sigchld_owner_libxl even if there
are not currently any children.
ctx->childproc_hooks->chldowner is now interpreted only by
chldmode_ours.
Reported-by: Bamvor Jian Zhang <bjzhang@suse.com>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Bamvor Jian Zhang <bjzhang@suse.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Jim Fehlig <jfehlig@suse.com>
---
v2: Get sense of chldmode test right in perhaps_removehandler (!)
We are going to want the gc for libxl__ev_fd_register.
No functional change in this patch. Simply change the argument types,
and the actual arguments from ctx to gc. Inside these functions, use
CTX (the macro which uses gc) rather than the old formal parameter
ctx.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Without this, code interrupted by SIGCHLD may experience strange
values of errno. (As far as I know this is not the cause of any
reported bugs.)
This fix should be backported in due course.
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
In 25182:6c3345d7e9d9 the practice of passing NULL to gc-using memory
allocation functions was introduced. However, the arrangements there
were not correct as committed, because the error handling and logging
depends on getting a ctx from the gc - so an allocation error would in
fact result in libxl dereferencing NULL.
Instead, provide a special dummy gc in the ctx, called `nogc_gc'. It
is marked out specially by having alloc_maxsize==-1, which is
otherwise invalid.
Functions which need to actually look into the gc use the new test
function gc_is_real (whose purpose is mainly clarity of the code) to
check whether the gc is the dummy one, and do nothing if it is. And
we provide a helper macro NOGC which uses the in-scope real gc to find
the ctx and hence the dummy gc (and which replaces the previous
#define NOGC NULL).
Change all callers which pass 0 or NULL to an allocation function to
use NOGC or &ctx->nogc_gc, as applicable in the context.
We add a comment near the definition of LIBXL_INIT_GC pointing out
that it isn't any more the only place a libxl__gc struct is
initialised, for the benefit of anyone changing the contents of gc's
in the future.
Also, actually document that libxl__ptr_add is legal with ptr==NULL,
and change a couple of calls not to check for NULL argument.
Reported-by: Bamvor Jian Zhang <bjzhang@suse.com>
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Bamvor Jian Zhang <bjzhang@suse.com>
Acked-by: Ian Campbell <Ian.Campbell@citrix.com>
Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Abolish libxl_fork. Its only callers were in xl. Its functionality
is now moved elsewhere, as follows:
* The "logging version of fork", which is what it was billed as, is now
xl_fork, which also dies on failure.
* Closing the xenstore handle in the child is now done in
libxl__ev_child_fork, which is the only remaining place where fork
is called in libxl.
* We provide a new function libxl__ev_child_xenstore_reopen for
in-libxl children to make the ctx useable for xenstore again.
* Consequently libxl__spawn_record_pid now no longer needs to mess
about with its own xenstore handle. As a bonus it can now just use
libxl__xs_write.
Also, since we have now converted all the forkers in libxl, we can
always honour the fork_replacement childproc hook - so do so.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
SIGCHLD or not; if it does, it has to tell us about deaths of our
children.
Currently there are no callers in libxl which use these facilities.
All code in libxl which forks needs to be converted and libxl_fork
needse to be be abolished.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
We introduce a new "carefd" concept, which relates to fds that we care
about not being inherited by long-lived children.
As yet we do not use this anywhere in libxl. Until all locations in
libxl which make such fds are converted, libxl__postfork may not work
entirely properly. If these locations do not use O_CLOEXEC (or use
calls for which there is no O_CLOEXEC) then multithreaded programs may
not work properly.
This introduces a new API call libxl_postfork_child_noexec which must
be called by applications which make long-running non-execing
children. Add the appropriate call to xl's postfork function.
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Ian Campbell <ian.campbell@citrix.com>
Committed-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
Ian Jackson
Andrew Cooper