In the x86 test harness and the fuzzer, and instead create a link in
the tools/include directory that can be used by all the tools.
No functional change.
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
In my automated SLE_11 builds I often see failures like that:
[ 74s] wrappers.c:5:25: error: x86-emulate.h: No such file or directory
[ 74s] make[6]: *** [wrappers.o] Error 1
Signed-off-by: Olaf Hering <olaf@aepfle.de>
Move line to where it belongs.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Use hooks, just like done for other special purpose registers.
This includes moving XCR0 checks from hvmemul_get_fpu() to the emulator
itself as well as adding support for XGETBV emulation.
For now fuzzer reads will obtain the real values (minus the fuzzing of
the hook pointer itself).
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com> [tracing parts]
Currently with the native tool chain on Debian Jessie ./test_x86_emulator
yields:
Testing AVX2 256bit single native execution...okay
Testing AVX2 256bit single 64-bit code sequence...[line 933] failed!
The bug is that libc's memcpy() in read() uses %xmm8 (specifically, in
__memcpy_sse2_unaligned()), which corrupts %ymm8 behind the back of the AVX2
test code.
Introduce wrappers (and machinery to forward calls to those wrappers)
saving/restoring FPU state around certain library calls.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-and-tested-by: Andrew Cooper <andrew.cooper3@citrix.com>
The maximum size for the input size was set to INPUT_SIZE, which is actually
the size of the data array inside the fuzz_corpus structure and so was not
abling user (or AFL) to fill in the whole structure. Changing to
sizeof(struct fuzz_corpus) correct this problem.
Signed-off-by: Paul Semel <semelpaul@gmail.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Changeset 2b1cde7783 introduced "batch mode" to afl-harness, which allowed
the handling of several inputs in sequence.
Unfortunately, it introduced a file pointer leak when the file was
larger than the maximum size. Restructure the code to always close fp
if we opened it.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Release-acked-by: Julien Grall <julien.grall@linaro.org>
Move fuzz-emul.c function prototypes into a header. Also share the
definition of the input size (rather than hard-coding it in
fuzz-emul.c).
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Finding aggregate coverage for a set of test files means running each
afl-generated test case through the harness. At the moment, this is
done by re-executing afl-harness-cov with each input file. When a
large number of test cases have been generated, this can take a
significant amonut of time; a recent test with 30k total files
generated by 4 parallel fuzzers took over 7 minutes.
The vast majority of this time is taken up with 'exec', however.
Since the harness is already designed to loop over multiple inputs for
llvm "persistent mode", just allow it to take a large number of inputs
on the same when *not* running in llvm "persistent mode".. Then the
command can be efficiently executed like this:
ls */queue/id* | xargs $path/afl-harness-cov
For the above-mentioned test on 30k files, the time to generate
coverage data was reduced from 7 minutes to under 30 seconds.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
...to generate a "normal" coverage-instrumented binary, suitable for
use with gcov or afl-cov.
This is slightly annoying because:
- Every object file needs to have been instrumented to work
effectively
- You generally want to have both an afl-instrumented binary and a
gcov-instrumented binary at the same time, but
- gcov instrumentation and afl instrumentation are mutually exclusive
So when making the `afl-cov` target, generate a second set of object
files and a second binary with the `-cov` suffix.
While we're here, remove the redundant x86-emulate.c dependency for
x86-emulate.o.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
When generating coverage output, by default gcov generates output
filenames based only on the coverage file and the "leaf" source file,
not the full path. As a result, it uses the same name for
x86_emulate.c and x86_emulate/x86_emulate.c, generally overwriting the
second (which we actually are about) with the first (which is just a
wrapper).
Rename the user-space wrapper helpers to x86-emulate.[ch], so
that it generates separate files.
There is actually an option to gcov, `--preserve-paths`, which will
cause the full path name to be included in the filename, properly
distinguishing between the two. However, given that the user-space
wrapper doesn't actually do any emulation (and the poor state of gcov
documentation making it difficult to find the option in the first
place), it seems to make more sense to rename the file anyway.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
- Print the symbolic name rather than the number
- Explicitly state when data_read() fails due to EOI
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Commit 849a1f10c9 was checked in inappropriately; review flagged up
that clearerr() was too big a hammer, as it would clear both the EOF
flag and stream errors.
Stream errors shouldn't be cleared; we only want the EOF and other
stream-related state reset. To do this, it is sufficient to fseek()
to zero.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Once feof() returns true for a stream, it will continue to return true
for that stream until clearerr() is called (or the stream is closed
and re-opened).
In llvm-clang-fast-mode, the same file descriptor is used for each
iteration of the loop, meaning that the "Input too large" check was
broken -- feof() would return true even if the fread() hadn't hit the
end of the file. The result is that AFL generates testcases of
arbitrary size.
Fix this by clearing the error after each iteration.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Commit c07574b reorganized the way fuzzing was done, explicitly
creating a structure that the input data would be copied into.
Unfortunately, the cpu register state used by the emulator is on the
stack; it's cleared, but data is never copied into it.
If we're explicitly setting an entirely new cpu_regs struct for each
new input anyway, there's no need to have two copies around anymore;
just point to the one in the data structure.
Signed-off-by: George Dunlap <george.dunlap@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
fuzz_insn_fetch() is the only data access helper where it is possible
to see offsets larger than 4Gb in 16- or 32-bit modes, as we leave the
incoming rIP untouched in the emulator itself. The check is needed here
as otherwise, after successfully fetching insn bytes, we may end up
zero-extending EIP soon after complete_insn, which collides with the
X86EMUL_EXCEPTION-conditional respective ASSERT() in
x86_emulate_wrapper(). (NB: put_rep_prefix() is what allows
complete_insn to be reached with rc set to other than X86EMUL_OKAY or
X86EMUL_DONE. See also commit 53f87c03b4 ["x86emul: generalize
exception handling for rep_* hooks"].)
Add assert()-s for all other (data) access routines, as effective
address generation in the emulator ought to guarantee in-range values.
For them to not trigger, several adjustments to the emulator's address
calculations are needed: While the DstBitBase one is really mandatory,
the specification allows for either original or new behavior for two-
part accesses. Observed behavior on real hardware, however, is for such
accesses to silently wrap at the 2^^32 boundary in other than 64-bit
mode, just like they do at the 2^^64 boundary in 64-bit mode, which our
code is now being brought in line with. While adding truncate_ea()
invocations there, also convert open coded instances of it.
Reported-by: George Dunlap <george.dunlap@citrix.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: George Dunlap <george.dunlap@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Running "make uninstall" does not remove all installed files, a
situation which might cause link related issues if xen is re-installed
in a different location.
In order to make uninstall correctly remove the files it is best
the process should be done recursively by mirroring each "install"
target with an "uninstall" who removes the installed files.
An exception to this rule is uninstalling the files produced by
"qemu-xen-dir-remote" and "qemu-xen-traditional-dir", which are external
to the project. These projects do not implement an "uninstall" target so
the files have to be removed manually.
Signed-off-by: Petre Pircalabu <ppircalabu@bitdefender.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
The amount of namespace resolution is unnecessarily large, as all code deals
in terms of struct segment_register. This removes the attr.fields part of all
references, and alters attr.bytes to just attr.
Three areas of code using initialisers for segment_register are tweaked to
compile with older versions of GCC. arch_set_info_hvm_guest() has its SEG()
macros altered to use plain comma-based initialisation, while
{rm,vm86}_{cs,ds}_attr are simplified to plain numbers which matches their
description in the manuals.
No functional change. (For some reason, the old {rm,vm86}_{cs,ds}_attr causes
GCC to create variable in .rodata, whereas the new code uses immediate
operands. As a result, vmx_{get,set}_segment_register() are slightly
shorter.)
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
c/s 92cf67888 "x86/emul: Hold x86_emulate() to strict X86EMUL_EXCEPTION
requirements" was appropriate for the hypervisor, but the fuzzer stubs didn't
conform to the stricter requirements. AFL is very quick to discover this.
Extend the fuzzing harness exception logic to raise exceptions appropriately.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
x86_emulates()'s is_branch_step() performs a speculative read of
IA32_DEBUGCTL, but doesn't squash exceptions should they arise. In reality,
this MSR is always available.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The correct prototypes for the hooks are to use enum x86_segment rather than
unsigned int. It is implementation specific as to whether this compiles.
assert() that the emulator never passes an inappropriate segment. The only
hook which may legitimately be passed x86_seg_none is invlpg().
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The fuzzing harness conditionally disables hooks to test error paths in the
emulator. However, fuzz_emulops is a static structure.
c/s 69f4633 "tools/insn-fuzz: Support AFL's afl-clang-fast mode" introduced
persistent mode, but because fuzz_emulops is static, the clobbering of hooks
accumulates over repeated input, meaning that previous corpora influence the
execution over the current corpus.
Move the partially clobbered struct x86_emulate_ops into struct fuzz_state,
which is re-initialised from full on each call to LLVMFuzzerTestOneInput()
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
AFL has a measure of stability, where it passes the same corpus into the
fuzzing harness and observes whether the execution path changes from before.
Any instability in the fuzzing harness reduces its effectiveness, as an
observed crash may not reliably be caused by the original corpus.
In preparation to fix a stability bug, introduce struct fuzz_state, allocated
on the stack and passed around via struct x86_emulate_ctxt's data parameter.
Propagate ctxt into the helpers such as maybe_fail(), so the state can be
retrieved.
Move the previously-static data_{index,num} into struct fuzz_state.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
For control-flow changes, the emulator needs to perform a zero-length
instruction fetch at the target offset. It also passes NULL for the
destination buffer, as there is no instruction stream to collect.
This trips up UBSAN when passed to memcpy(), as passing NULL is undefined
behaviour per the C spec (irrespective of passing a size of 0).
Special case these fetches in fuzz_insn_fetch() before reaching data_read().
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: George Dunlap <george.dunlap@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Long mode (or not) influences emulation behaviour in a number of cases.
Instead of reusing the ->read_msr() hook to obtain EFER.LMA, require callers
to provide it directly.
This simplifies all long mode checks during emulation to a simple boolean
read, removing embedded msr reads. It also allows for the removal of a local
variable in the sysenter emulation block, and removes a latent bug in the
syscall emulation block where rc contains a non X86EMUL_* constant for a
period of time.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
Acked-by: Tim Deegan <tim@xen.org>
Reviewed-by: Jan Beulich <JBeulich@suse.com>
With the SVM injection logic capable of doing its own emulation, there is no
need for this hardware-specific assistance in the common emulator.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Tim Deegan <tim@xen.org>
When an FPU instruction with a memory destination fails during the
memory write, it should not affect FPU register state. Due to the way
we emulate FPU (and SIMD) instructions, we can only guarantee this by
- backing out changes to the FPU register state in such a case or
- doing a descriptor read and/or page walk up front, perhaps with the
stubs accessing the actual memory location then.
The latter would require a significant change in how the emulator does
its guest memory accessing, so for now the former variant is being
chosen.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com> [hvm/emulate.c]
Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
AFL has an alternative llvm-base instrumentation mode, which has much lower
overhead than the traditional afl-gcc.
One extra ability is to chose exactly where the master process gets
initialised to, before being forked for testing. This point is chosen after
the call to LLVMFuzzerInitialize(), so the stack isn't being remapped
executable for every test.
Another extra ability is to feed multiple inputs into a single test process,
to reduce the number of fork() calls required overall. Two caveats are that if
stdin is used for data, it must be unbuffered, and if input is passed via a
command line parameter, the underlying file must be opened and closed on each
iteration.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
libfuzz can perform one-time initialisation by calling LLVMFuzzerInitialize().
Move emul_test_init() into this, to avoid repeating it on every
LLVMFuzzerTestOneInput() call.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
This is rather faster for afl-fuzz to arrange than using an explicit file
parameter. Also update the README to recommend using a tmpfs for findings_dir
which reduces disk load and is more performant.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Several `ln -sf` can race with each other and cause error like:
14:43:56 00:07:06 O: ln: cannot remove 'asm': No such file or directory
Provide dedicated targets for soft-linking directories.
Reported-by: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
The "reg" variable in fuzz_read_msr stores the real MSR index, not an
index within the fuzzer.
The rest of that function already handles things correctly. We just need
to remove the bogus check.
Spotted by Coverity.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
... so that users can know how big the initial input should be.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Change the names used in code according to numeric values. Remove the
now unused macros in x86_emualte.c and fix indentation. This in turns
requires including msr-index.h and removing duplicates in userspace
x86_emulate.c in userspace harness program.
No functional change.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Basically this patch does 's/EFLG_/X86_EFLAGS_/g' and with indentation
fixed up. And remove the duplicates in x86_emualte.c. This in turn
requires userspace test harness to include x86-defns.h. Also remove a
few duplicates in userspace harness program.
No functional change.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
Now the function can return nonzero value. Use that value as exit code
for the stub program. AFL might be able to use such information to
optimise fuzzing process.
Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Roger Pau Monne