merge with xen-unstable.hg

This commit is contained in:
awilliam@xenbuild2.aw committed 2007-03-07 12:38:28 -07:00
commit 5ac1e3b0f0
358 files changed
+11557 -6406

No files matched your search

+4 -3
View File
@@ -73,9 +73,10 @@ ACM_SECURITY ?= n
ACM_DEFAULT_SECURITY_POLICY ?= ACM_NULL_POLICY
# Optional components
XENSTAT_XENTOP ?= y
VTPM_TOOLS ?= n
XENSTAT_XENTOP ?= y
VTPM_TOOLS ?= n
LIBXENAPI_BINDINGS ?= n
XENFB_TOOLS ?= n
XENFB_TOOLS ?= n
PYTHON_TOOLS ?= y
-include $(XEN_ROOT)/.config
+2
View File
@@ -92,6 +92,8 @@ provided by your Linux distributor:
* Development install of zlib (e.g., zlib-dev)
* Development install of Python v2.3 or later (e.g., python-dev)
* Development install of curses (e.g., libncurses-dev)
* Development install of openssl (e.g., openssl-dev)
* Development install of x11 (e.g. xorg-x11-dev)
* bridge-utils package (/sbin/brctl)
* iproute package (/sbin/ip)
* hotplug or udev
+3 -3
View File
@@ -12,9 +12,9 @@ OBJDUMP = $(CROSS_COMPILE)objdump
MSGFMT = msgfmt
INSTALL = install
INSTALL_DIR = $(INSTALL) -d -m0755
INSTALL_DATA = $(INSTALL) -m0644
INSTALL_PROG = $(INSTALL) -m0755
INSTALL_DIR = $(INSTALL) -d -m0755 -p
INSTALL_DATA = $(INSTALL) -m0644 -p
INSTALL_PROG = $(INSTALL) -m0755 -p
LIB64DIR = lib64
+3 -3
View File
@@ -14,9 +14,9 @@ MSGFMT = gmsgfmt
SHELL = bash
INSTALL = ginstall
INSTALL_DIR = $(INSTALL) -d -m0755
INSTALL_DATA = $(INSTALL) -m0644
INSTALL_PROG = $(INSTALL) -m0755
INSTALL_DIR = $(INSTALL) -d -m0755 -p
INSTALL_DATA = $(INSTALL) -m0644 -p
INSTALL_PROG = $(INSTALL) -m0755 -p
LIB64DIR = lib/amd64
+231
View File
@@ -0,0 +1,231 @@
xen dump-core format
Written by Isaku Yamahata <yamahata at valinux co jp> Feb. 2007
Introduction
------------
With xm dump-core command, the guest domain's core can be created as a file.
Its format was changed to be based on ELF format because elf format is easily
extensible and handy. This document describes the new format.
In this document the new format is called new xen dump-core format,
xen dump-core format or simply dump-core format. The file of xen dump-core
format is called xen dump-core file or dump-core file.
The usual process core file includes program headers and no section header.
On the other hand the xen dump-core file includes no program headers and
some sections because of its peculiar requirements.
Reference
---------
For ELF format itself, see Tool Interface Standard(TIS) Executable and
Linking Format(ELF) Specification version 1.2.
For xen related structure, please see the xen header files.
Elf header
----------
The elf header members are set as follows
e_ident[EI_CLASS] = ELFCLASS64 = 2
e_ident[EI_OSABI] = ELFOSABI_SYSV = 0
e_type = ET_CORE = 4
ELFCLASS64 is always used independent of architecture.
e_ident[EI_DATA] and e_flags are set according to the dumping system's
architecture. Other members are set as usual.
Sections
--------
Currently the following sections are defined. Some sections are optional.
".note.Xen" section
name ".note.Xen"
type SHT_NOTE
description
This section is used as note section to store xen dump-core
file specific informations. The each detailed informations are
described in note section. This section must exist.
".xen_prstatus" section
name ".xen_prstatus"
type SHT_PROGBITS
structure array of vcpu_guest_context_t
description
This section stores the array of vcpu_guest_context_t
which is obtained by XEN_DOMCTL_getvcpucontext hypercall
when the xen dump-core file is created.
The size of array is stored in xch_nr_vcpus member of header
note descriptor in .note.Xen note section.
This section must exist.
".xen_shared_info" section
name ".xen_shared_info"
type SHT_PROGBITS
structure shared_info_t
description
This section stores the contents of shared info page
of a domain. This section is optional.
".xen_p2m" section
name ".xen_p2m"
type SHT_PROGBITS
structure array of struct xen_dumpcore_p2m
struct xen_dumpcore_p2m {
uint64_t pfn;
uint64_t gmfn;
};
description
This elements represents the frame number of the page
in .xen_pages section.
pfn: guest-specific pseudo-physical frame number
gmfn: machine physical frame number
The size of arrays is stored in xch_nr_pages member of header
note descriptor in .note.Xen note section.
There is no rule about the order. Analysis tools must no rely
on its order.
This section must exist when the domain is non auto
translated physmap mode. Currently x86 paravirtualized domain.
".xen_pfn" section
name ".xen_pfn"
type SHT_PROGBITS
structure array of uint64_t
description
This elements represents the frame number of the page
in .xen_pages section.
The size of arrays is stored in xch_nr_pages member of header
note descriptor in .note.Xen note section.
There is no rule about the order. Analysis tools must no rely
on its order.
This section must exist when the domain is auto translated
physmap mode. Currently x86 full virtualized domain and
ia64 domain.
".xen_pages" section
name ".xen_pages"
type SHT_PROGBITS
structure array of page where page is page size byte array
description
This section includes the contents of pages.
The corresponding address is described in .xen_p2m section
or .xen_pfn section.
The page size is stored in xch_page_size member of header note
descriptor in .note.Xen section.
The array size is stored in xch_nr_pages member of header note
descriptor in .note.Xen section.
This section must exist.
".xen_ia64_mapped_regs" section
name ".xen_ia64_mapped_regs"
type SHT_PROGBITS
structure array of mapped_regs_t
description
This section stores the array of mapped_regs_t.
The size of array is stored in xch_nr_vcpus member of header
note descriptor in .note.Xen note section.
This section is ia64 specific and must exist for ia64 domain.
This section must not exist for non-ia64 domain.
note section
------------
The note types are defined in xen/include/public/elfnote.h.
The note descriptors are defined in tools/libxc/xc_core.h
Currently the following note informations are defined.
elf note section
"Xen" is used as elf note name in elf note info
namesz 4
name "Xen" (null-terminated)
Descriptors
none note descriptor
type XEN_ELFNOTE_DUMPCORE_NONE = 0x2000000
structure struct xen_dumpcore_elfnote_none_desc {
/* nothing is defined */
};
description
This note descriptor is defined to just indicate that this
file is xen dump-core format without any specific information.
This note information must exist.
header note descriptor
type XEN_ELFNOTE_DUMPCORE_HEADER = 0x2000001
structure struct xen_dumpcore_elfnote_header_desc {
uint64_t xch_magic;
uint64_t xch_nr_vcpus;
uint64_t xch_nr_pages;
uint64_t xch_page_size;
};
description
This note descriptor stores basic information of the domain.
xch_magic magic number
XC_CORE_MAGIC = 0xF00FEBED for paravirtualized domain
XC_CORE_MAGIC_HVM = 0xF00FEBEE for full virtualized domain
xch_nr_vcpus the number of vcpus
xch_nr_pages the number of pages
xch_page_size guest OS's page size
This note information must exist.
xen_version descriptor
type XEN_ELFNOTE_DUMPCORE_XEN_VERSION = 0x2000002
structure struct xen_dumpcore_elfnote_xen_version_desc {
uint64_t major_version;
uint64_t minor_version;
xen_extraversion_t extra_version;
xen_compile_info_t compile_info;
xen_capabilities_info_t capabilities;
xen_changeset_info_t changeset;
xen_platform_parameters_t platform_parameters;
uint64_t pagesize;
};
description
This note descriptor stores basic information about xen
hypervisor. The each members store the result of
__HYPERVISOR_xen_version hypercall.
major_version 16msb bit of the result of XENVER_version
minor_version 16lsb bit of the result of XENVER_version
uint64_t is used to make struct
xen_dumpcore_elfnote_xen_version_desc independent
on 32bit/64bit instead of uint32_t.
extra_version the result of XENVER_extraversion
compile_info the result of XENVER_compile_info
capabilities the result of XENVER_capabilities
changeset the result of XENVER_changeset
platform_parameters
the result of XENVER_platform_parameters
pagesize the result of XENVER_pagesize
This note information must exist.
format_version descriptor
type XEN_ELFNOTE_DUMPCORE_FORMAT_VERSION = 0x2000003
structure struct xen_dumpcore_elfnote_format_version_desc {
uint64_t version;
};
description
This note descriptor stores xen dump-core format version.
The 32msb bit is major version and the 32lsb bit is minor
version.
The minor version will be incremented when the format
is changed in compatible way. e.g. new sections, new note
descriptors are added.
the major version will be incremented when the format is
changed in incompatible way.
This note information must exit. Analysis tools should check
this format version.
This note information must exist.
Format version history
----------------------
Currently only (major, minor) = (0, 1) is used.
[When the format is changed, it would be described here.]
(0, 1) update
- EI_CLASS member of elf header was changed to ELFCLASS64 independent of
architecture. This is mainly for x86_32pae.
The format version isn't bumped because analysis tools can distinguish it.
File diff suppressed because it is too large. Load diff
+1 -2
View File
@@ -4,8 +4,6 @@
# Makefile and a arch.mk.
#
pae ?= n
XEN_ROOT = ../..
include $(XEN_ROOT)/Config.mk
@@ -35,6 +33,7 @@ TARGET_ARCH_DIR := arch/$(TARGET_ARCH_FAM)
export TARGET_ARCH
export TARGET_ARCH_DIR
export TARGET_ARCH_FAM
export XEN_TARGET_X86_PAE
# This is used for architecture specific links.
# This can be overwritten from arch specific rules.
+4
View File
@@ -130,3 +130,7 @@ map_frames(unsigned long* frames, unsigned long n)
return (void*) __va(frames[0] << PAGE_SHIFT);
}
void arch_init_p2m(unsigned long max_pfn)
{
printk("Warn: p2m map not implemented.\n");
}
+1
View File
@@ -4,6 +4,7 @@
#
# include arch.mk has to be before mini-os.mk!
include arch.mk
include ../../minios.mk
+4 -6
View File
@@ -9,13 +9,11 @@ ARCH_LDFLAGS := -m elf_i386
ARCH_ASFLAGS := -m32
EXTRA_INC += $(TARGET_ARCH_FAM)/$(TARGET_ARCH)
EXTRA_SRC += arch/$(EXTRA_INC)
endif
ifeq ($(TARGET_ARCH)$(pae),x86_32y)
ARCH_CFLAGS := -DCONFIG_X86_PAE=1
ARCH_ASFLAGS := -DCONFIG_X86_PAE=1
EXTRA_INC += $(TARGET_ARCH_FAM)/$(TARGET_ARCH)
EXTRA_SRC += arch/$(EXTRA_INC)
ifeq ($(XEN_TARGET_X86_PAE),y)
ARCH_CFLAGS += -DCONFIG_X86_PAE=1
ARCH_ASFLAGS += -DCONFIG_X86_PAE=1
endif
endif
ifeq ($(TARGET_ARCH),x86_64)
+43
View File
@@ -402,6 +402,49 @@ void *map_frames(unsigned long *f, unsigned long n)
}
}
void arch_init_p2m(unsigned long max_pfn)
{
#define L1_P2M_SHIFT 10
#define L2_P2M_SHIFT 20
#define L3_P2M_SHIFT 30
#define L1_P2M_ENTRIES (1 << L1_P2M_SHIFT)
#define L2_P2M_ENTRIES (1 << (L2_P2M_SHIFT - L1_P2M_SHIFT))
#define L3_P2M_ENTRIES (1 << (L3_P2M_SHIFT - L2_P2M_SHIFT))
#define L1_P2M_MASK (L1_P2M_ENTRIES - 1)
#define L2_P2M_MASK (L2_P2M_ENTRIES - 1)
#define L3_P2M_MASK (L3_P2M_ENTRIES - 1)
unsigned long *l1_list, *l2_list, *l3_list;
unsigned long pfn;
l3_list = (unsigned long *)alloc_page();
for(pfn=0; pfn<max_pfn; pfn++)
{
if(!(pfn % (L1_P2M_ENTRIES * L2_P2M_ENTRIES)))
{
l2_list = (unsigned long*)alloc_page();
if((pfn >> L3_P2M_SHIFT) > 0)
{
printk("Error: Too many pfns.\n");
do_exit();
}
l3_list[(pfn >> L2_P2M_SHIFT)] = virt_to_mfn(l2_list);
}
if(!(pfn % (L1_P2M_ENTRIES)))
{
l1_list = (unsigned long*)alloc_page();
l2_list[(pfn >> L1_P2M_SHIFT) & L2_P2M_MASK] =
virt_to_mfn(l1_list);
}
l1_list[pfn & L1_P2M_MASK] = pfn_to_mfn(pfn);
}
HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list =
virt_to_mfn(l3_list);
HYPERVISOR_shared_info->arch.max_pfn = max_pfn;
}
void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p)
{
+1
View File
@@ -55,6 +55,7 @@ static __inline__ int get_order(unsigned long size)
void arch_init_demand_mapping_area(unsigned long max_pfn);
void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p);
void arch_init_p2m(unsigned long max_pfn_p);
void *map_frames(unsigned long *f, unsigned long n);
+2
View File
@@ -379,6 +379,8 @@ void init_mm(void)
init_page_allocator(PFN_PHYS(start_pfn), PFN_PHYS(max_pfn));
printk("MM: done\n");
arch_init_p2m(max_pfn);
arch_init_demand_mapping_area(max_pfn);
}
-6
View File
@@ -255,7 +255,6 @@ config NR_CPUS
config SCHED_SMT
bool "SMT (Hyperthreading) scheduler support"
depends on X86_HT
depends on !X86_XEN
help
SMT scheduler support improves the CPU scheduler's decision making
when dealing with Intel Pentium 4 chips with HyperThreading at a
@@ -313,11 +312,6 @@ config X86_VISWS_APIC
depends on X86_VISWS
default y
config X86_TSC
bool
depends on (MWINCHIP3D || MWINCHIP2 || MCRUSOE || MEFFICEON || MCYRIXIII || MK7 || MK6 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || M586MMX || M586TSC || MK8 || MVIAC3_2 || MGEODEGX1) && !X86_NUMAQ && !X86_XEN
default y
config X86_MCE
bool "Machine Check Exception"
depends on !(X86_VOYAGER || X86_XEN)
+1 -1
View File
@@ -311,5 +311,5 @@ config X86_OOSTORE
config X86_TSC
bool
depends on (MWINCHIP3D || MWINCHIP2 || MCRUSOE || MEFFICEON || MCYRIXIII || MK7 || MK6 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || M586MMX || M586TSC || MK8 || MVIAC3_2 || MGEODEGX1 || MGEODE_LX) && !X86_NUMAQ
depends on (MWINCHIP3D || MWINCHIP2 || MCRUSOE || MEFFICEON || MCYRIXIII || MK7 || MK6 || MPENTIUM4 || MPENTIUMM || MPENTIUMIII || MPENTIUMII || M686 || M586MMX || M586TSC || MK8 || MVIAC3_2 || MGEODEGX1 || MGEODE_LX) && !X86_NUMAQ && !X86_XEN
default y
@@ -1,389 +0,0 @@
#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/list.h>
#include <asm/alternative.h>
#include <asm/sections.h>
static int no_replacement = 0;
static int smp_alt_once = 0;
static int debug_alternative = 0;
static int __init noreplacement_setup(char *s)
{
no_replacement = 1;
return 1;
}
static int __init bootonly(char *str)
{
smp_alt_once = 1;
return 1;
}
static int __init debug_alt(char *str)
{
debug_alternative = 1;
return 1;
}
__setup("noreplacement", noreplacement_setup);
__setup("smp-alt-boot", bootonly);
__setup("debug-alternative", debug_alt);
#define DPRINTK(fmt, args...) if (debug_alternative) \
printk(KERN_DEBUG fmt, args)
#ifdef GENERIC_NOP1
/* Use inline assembly to define this because the nops are defined
as inline assembly strings in the include files and we cannot
get them easily into strings. */
asm("\t.data\nintelnops: "
GENERIC_NOP1 GENERIC_NOP2 GENERIC_NOP3 GENERIC_NOP4 GENERIC_NOP5 GENERIC_NOP6
GENERIC_NOP7 GENERIC_NOP8);
extern unsigned char intelnops[];
static unsigned char *intel_nops[ASM_NOP_MAX+1] = {
NULL,
intelnops,
intelnops + 1,
intelnops + 1 + 2,
intelnops + 1 + 2 + 3,
intelnops + 1 + 2 + 3 + 4,
intelnops + 1 + 2 + 3 + 4 + 5,
intelnops + 1 + 2 + 3 + 4 + 5 + 6,
intelnops + 1 + 2 + 3 + 4 + 5 + 6 + 7,
};
#endif
#ifdef K8_NOP1
asm("\t.data\nk8nops: "
K8_NOP1 K8_NOP2 K8_NOP3 K8_NOP4 K8_NOP5 K8_NOP6
K8_NOP7 K8_NOP8);
extern unsigned char k8nops[];
static unsigned char *k8_nops[ASM_NOP_MAX+1] = {
NULL,
k8nops,
k8nops + 1,
k8nops + 1 + 2,
k8nops + 1 + 2 + 3,
k8nops + 1 + 2 + 3 + 4,
k8nops + 1 + 2 + 3 + 4 + 5,
k8nops + 1 + 2 + 3 + 4 + 5 + 6,
k8nops + 1 + 2 + 3 + 4 + 5 + 6 + 7,
};
#endif
#ifdef K7_NOP1
asm("\t.data\nk7nops: "
K7_NOP1 K7_NOP2 K7_NOP3 K7_NOP4 K7_NOP5 K7_NOP6
K7_NOP7 K7_NOP8);
extern unsigned char k7nops[];
static unsigned char *k7_nops[ASM_NOP_MAX+1] = {
NULL,
k7nops,
k7nops + 1,
k7nops + 1 + 2,
k7nops + 1 + 2 + 3,
k7nops + 1 + 2 + 3 + 4,
k7nops + 1 + 2 + 3 + 4 + 5,
k7nops + 1 + 2 + 3 + 4 + 5 + 6,
k7nops + 1 + 2 + 3 + 4 + 5 + 6 + 7,
};
#endif
#ifdef CONFIG_X86_64
extern char __vsyscall_0;
static inline unsigned char** find_nop_table(void)
{
return k8_nops;
}
#else /* CONFIG_X86_64 */
static struct nop {
int cpuid;
unsigned char **noptable;
} noptypes[] = {
{ X86_FEATURE_K8, k8_nops },
{ X86_FEATURE_K7, k7_nops },
{ -1, NULL }
};
static unsigned char** find_nop_table(void)
{
unsigned char **noptable = intel_nops;
int i;
for (i = 0; noptypes[i].cpuid >= 0; i++) {
if (boot_cpu_has(noptypes[i].cpuid)) {
noptable = noptypes[i].noptable;
break;
}
}
return noptable;
}
#endif /* CONFIG_X86_64 */
extern struct alt_instr __alt_instructions[], __alt_instructions_end[];
extern struct alt_instr __smp_alt_instructions[], __smp_alt_instructions_end[];
extern u8 *__smp_locks[], *__smp_locks_end[];
extern u8 __smp_alt_begin[], __smp_alt_end[];
/* Replace instructions with better alternatives for this CPU type.
This runs before SMP is initialized to avoid SMP problems with
self modifying code. This implies that assymetric systems where
APs have less capabilities than the boot processor are not handled.
Tough. Make sure you disable such features by hand. */
void apply_alternatives(struct alt_instr *start, struct alt_instr *end)
{
unsigned char **noptable = find_nop_table();
struct alt_instr *a;
u8 *instr;
int diff, i, k;
DPRINTK("%s: alt table %p -> %p\n", __FUNCTION__, start, end);
for (a = start; a < end; a++) {
BUG_ON(a->replacementlen > a->instrlen);
if (!boot_cpu_has(a->cpuid))
continue;
instr = a->instr;
#ifdef CONFIG_X86_64
/* vsyscall code is not mapped yet. resolve it manually. */
if (instr >= (u8 *)VSYSCALL_START && instr < (u8*)VSYSCALL_END) {
instr -= VSYSCALL_START - (unsigned long)&__vsyscall_0;
DPRINTK("%s: vsyscall fixup: %p => %p\n",
__FUNCTION__, a->instr, instr);
}
#endif
memcpy(instr, a->replacement, a->replacementlen);
diff = a->instrlen - a->replacementlen;
/* Pad the rest with nops */
for (i = a->replacementlen; diff > 0; diff -= k, i += k) {
k = diff;
if (k > ASM_NOP_MAX)
k = ASM_NOP_MAX;
memcpy(a->instr + i, noptable[k], k);
}
}
}
#ifdef CONFIG_SMP
static void alternatives_smp_save(struct alt_instr *start, struct alt_instr *end)
{
struct alt_instr *a;
DPRINTK("%s: alt table %p-%p\n", __FUNCTION__, start, end);
for (a = start; a < end; a++) {
memcpy(a->replacement + a->replacementlen,
a->instr,
a->instrlen);
}
}
static void alternatives_smp_apply(struct alt_instr *start, struct alt_instr *end)
{
struct alt_instr *a;
for (a = start; a < end; a++) {
memcpy(a->instr,
a->replacement + a->replacementlen,
a->instrlen);
}
}
static void alternatives_smp_lock(u8 **start, u8 **end, u8 *text, u8 *text_end)
{
u8 **ptr;
for (ptr = start; ptr < end; ptr++) {
if (*ptr < text)
continue;
if (*ptr > text_end)
continue;
**ptr = 0xf0; /* lock prefix */
};
}
static void alternatives_smp_unlock(u8 **start, u8 **end, u8 *text, u8 *text_end)
{
unsigned char **noptable = find_nop_table();
u8 **ptr;
for (ptr = start; ptr < end; ptr++) {
if (*ptr < text)
continue;
if (*ptr > text_end)
continue;
**ptr = noptable[1][0];
};
}
struct smp_alt_module {
/* what is this ??? */
struct module *mod;
char *name;
/* ptrs to lock prefixes */
u8 **locks;
u8 **locks_end;
/* .text segment, needed to avoid patching init code ;) */
u8 *text;
u8 *text_end;
struct list_head next;
};
static LIST_HEAD(smp_alt_modules);
static DEFINE_SPINLOCK(smp_alt);
void alternatives_smp_module_add(struct module *mod, char *name,
void *locks, void *locks_end,
void *text, void *text_end)
{
struct smp_alt_module *smp;
unsigned long flags;
if (no_replacement)
return;
if (smp_alt_once) {
if (boot_cpu_has(X86_FEATURE_UP))
alternatives_smp_unlock(locks, locks_end,
text, text_end);
return;
}
smp = kzalloc(sizeof(*smp), GFP_KERNEL);
if (NULL == smp)
return; /* we'll run the (safe but slow) SMP code then ... */
smp->mod = mod;
smp->name = name;
smp->locks = locks;
smp->locks_end = locks_end;
smp->text = text;
smp->text_end = text_end;
DPRINTK("%s: locks %p -> %p, text %p -> %p, name %s\n",
__FUNCTION__, smp->locks, smp->locks_end,
smp->text, smp->text_end, smp->name);
spin_lock_irqsave(&smp_alt, flags);
list_add_tail(&smp->next, &smp_alt_modules);
if (boot_cpu_has(X86_FEATURE_UP))
alternatives_smp_unlock(smp->locks, smp->locks_end,
smp->text, smp->text_end);
spin_unlock_irqrestore(&smp_alt, flags);
}
void alternatives_smp_module_del(struct module *mod)
{
struct smp_alt_module *item;
unsigned long flags;
if (no_replacement || smp_alt_once)
return;
spin_lock_irqsave(&smp_alt, flags);
list_for_each_entry(item, &smp_alt_modules, next) {
if (mod != item->mod)
continue;
list_del(&item->next);
spin_unlock_irqrestore(&smp_alt, flags);
DPRINTK("%s: %s\n", __FUNCTION__, item->name);
kfree(item);
return;
}
spin_unlock_irqrestore(&smp_alt, flags);
}
void alternatives_smp_switch(int smp)
{
struct smp_alt_module *mod;
unsigned long flags;
#ifdef CONFIG_LOCKDEP
/*
* A not yet fixed binutils section handling bug prevents
* alternatives-replacement from working reliably, so turn
* it off:
*/
printk("lockdep: not fixing up alternatives.\n");
return;
#endif
if (no_replacement || smp_alt_once)
return;
BUG_ON(!smp && (num_online_cpus() > 1));
spin_lock_irqsave(&smp_alt, flags);
if (smp) {
printk(KERN_INFO "SMP alternatives: switching to SMP code\n");
clear_bit(X86_FEATURE_UP, boot_cpu_data.x86_capability);
clear_bit(X86_FEATURE_UP, cpu_data[0].x86_capability);
alternatives_smp_apply(__smp_alt_instructions,
__smp_alt_instructions_end);
list_for_each_entry(mod, &smp_alt_modules, next)
alternatives_smp_lock(mod->locks, mod->locks_end,
mod->text, mod->text_end);
} else {
printk(KERN_INFO "SMP alternatives: switching to UP code\n");
set_bit(X86_FEATURE_UP, boot_cpu_data.x86_capability);
set_bit(X86_FEATURE_UP, cpu_data[0].x86_capability);
apply_alternatives(__smp_alt_instructions,
__smp_alt_instructions_end);
list_for_each_entry(mod, &smp_alt_modules, next)
alternatives_smp_unlock(mod->locks, mod->locks_end,
mod->text, mod->text_end);
}
spin_unlock_irqrestore(&smp_alt, flags);
}
#endif
void __init alternative_instructions(void)
{
if (no_replacement) {
printk(KERN_INFO "(SMP-)alternatives turned off\n");
free_init_pages("SMP alternatives",
(unsigned long)__smp_alt_begin,
(unsigned long)__smp_alt_end);
return;
}
apply_alternatives(__alt_instructions, __alt_instructions_end);
/* switch to patch-once-at-boottime-only mode and free the
* tables in case we know the number of CPUs will never ever
* change */
#ifdef CONFIG_HOTPLUG_CPU
if (num_possible_cpus() < 2)
smp_alt_once = 1;
#else
smp_alt_once = 1;
#endif
#ifdef CONFIG_SMP
if (smp_alt_once) {
if (1 == num_possible_cpus()) {
printk(KERN_INFO "SMP alternatives: switching to UP code\n");
set_bit(X86_FEATURE_UP, boot_cpu_data.x86_capability);
set_bit(X86_FEATURE_UP, cpu_data[0].x86_capability);
apply_alternatives(__smp_alt_instructions,
__smp_alt_instructions_end);
alternatives_smp_unlock(__smp_locks, __smp_locks_end,
_text, _etext);
}
free_init_pages("SMP alternatives",
(unsigned long)__smp_alt_begin,
(unsigned long)__smp_alt_end);
} else {
alternatives_smp_save(__smp_alt_instructions,
__smp_alt_instructions_end);
alternatives_smp_module_add(NULL, "core kernel",
__smp_locks, __smp_locks_end,
_text, _etext);
alternatives_smp_switch(0);
}
#endif
}
@@ -1,774 +0,0 @@
/*
* Routines to indentify caches on Intel CPU.
*
* Changes:
* Venkatesh Pallipadi : Adding cache identification through cpuid(4)
* Ashok Raj <ashok.raj@intel.com>: Work with CPU hotplug infrastructure.
* Andi Kleen : CPUID4 emulation on AMD.
*/
#include <linux/init.h>
#include <linux/slab.h>
#include <linux/device.h>
#include <linux/compiler.h>
#include <linux/cpu.h>
#include <linux/sched.h>
#include <asm/processor.h>
#include <asm/smp.h>
#define LVL_1_INST 1
#define LVL_1_DATA 2
#define LVL_2 3
#define LVL_3 4
#define LVL_TRACE 5
struct _cache_table
{
unsigned char descriptor;
char cache_type;
short size;
};
/* all the cache descriptor types we care about (no TLB or trace cache entries) */
static struct _cache_table cache_table[] __cpuinitdata =
{
{ 0x06, LVL_1_INST, 8 }, /* 4-way set assoc, 32 byte line size */
{ 0x08, LVL_1_INST, 16 }, /* 4-way set assoc, 32 byte line size */
{ 0x0a, LVL_1_DATA, 8 }, /* 2 way set assoc, 32 byte line size */
{ 0x0c, LVL_1_DATA, 16 }, /* 4-way set assoc, 32 byte line size */
{ 0x22, LVL_3, 512 }, /* 4-way set assoc, sectored cache, 64 byte line size */
{ 0x23, LVL_3, 1024 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x25, LVL_3, 2048 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x29, LVL_3, 4096 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x2c, LVL_1_DATA, 32 }, /* 8-way set assoc, 64 byte line size */
{ 0x30, LVL_1_INST, 32 }, /* 8-way set assoc, 64 byte line size */
{ 0x39, LVL_2, 128 }, /* 4-way set assoc, sectored cache, 64 byte line size */
{ 0x3a, LVL_2, 192 }, /* 6-way set assoc, sectored cache, 64 byte line size */
{ 0x3b, LVL_2, 128 }, /* 2-way set assoc, sectored cache, 64 byte line size */
{ 0x3c, LVL_2, 256 }, /* 4-way set assoc, sectored cache, 64 byte line size */
{ 0x3d, LVL_2, 384 }, /* 6-way set assoc, sectored cache, 64 byte line size */
{ 0x3e, LVL_2, 512 }, /* 4-way set assoc, sectored cache, 64 byte line size */
{ 0x41, LVL_2, 128 }, /* 4-way set assoc, 32 byte line size */
{ 0x42, LVL_2, 256 }, /* 4-way set assoc, 32 byte line size */
{ 0x43, LVL_2, 512 }, /* 4-way set assoc, 32 byte line size */
{ 0x44, LVL_2, 1024 }, /* 4-way set assoc, 32 byte line size */
{ 0x45, LVL_2, 2048 }, /* 4-way set assoc, 32 byte line size */
{ 0x46, LVL_3, 4096 }, /* 4-way set assoc, 64 byte line size */
{ 0x47, LVL_3, 8192 }, /* 8-way set assoc, 64 byte line size */
{ 0x49, LVL_3, 4096 }, /* 16-way set assoc, 64 byte line size */
{ 0x4a, LVL_3, 6144 }, /* 12-way set assoc, 64 byte line size */
{ 0x4b, LVL_3, 8192 }, /* 16-way set assoc, 64 byte line size */
{ 0x4c, LVL_3, 12288 }, /* 12-way set assoc, 64 byte line size */
{ 0x4d, LVL_3, 16384 }, /* 16-way set assoc, 64 byte line size */
{ 0x60, LVL_1_DATA, 16 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x66, LVL_1_DATA, 8 }, /* 4-way set assoc, sectored cache, 64 byte line size */
{ 0x67, LVL_1_DATA, 16 }, /* 4-way set assoc, sectored cache, 64 byte line size */
{ 0x68, LVL_1_DATA, 32 }, /* 4-way set assoc, sectored cache, 64 byte line size */
{ 0x70, LVL_TRACE, 12 }, /* 8-way set assoc */
{ 0x71, LVL_TRACE, 16 }, /* 8-way set assoc */
{ 0x72, LVL_TRACE, 32 }, /* 8-way set assoc */
{ 0x73, LVL_TRACE, 64 }, /* 8-way set assoc */
{ 0x78, LVL_2, 1024 }, /* 4-way set assoc, 64 byte line size */
{ 0x79, LVL_2, 128 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x7a, LVL_2, 256 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x7b, LVL_2, 512 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x7c, LVL_2, 1024 }, /* 8-way set assoc, sectored cache, 64 byte line size */
{ 0x7d, LVL_2, 2048 }, /* 8-way set assoc, 64 byte line size */
{ 0x7f, LVL_2, 512 }, /* 2-way set assoc, 64 byte line size */
{ 0x82, LVL_2, 256 }, /* 8-way set assoc, 32 byte line size */
{ 0x83, LVL_2, 512 }, /* 8-way set assoc, 32 byte line size */
{ 0x84, LVL_2, 1024 }, /* 8-way set assoc, 32 byte line size */
{ 0x85, LVL_2, 2048 }, /* 8-way set assoc, 32 byte line size */
{ 0x86, LVL_2, 512 }, /* 4-way set assoc, 64 byte line size */
{ 0x87, LVL_2, 1024 }, /* 8-way set assoc, 64 byte line size */
{ 0x00, 0, 0}
};
enum _cache_type
{
CACHE_TYPE_NULL = 0,
CACHE_TYPE_DATA = 1,
CACHE_TYPE_INST = 2,
CACHE_TYPE_UNIFIED = 3
};
union _cpuid4_leaf_eax {
struct {
enum _cache_type type:5;
unsigned int level:3;
unsigned int is_self_initializing:1;
unsigned int is_fully_associative:1;
unsigned int reserved:4;
unsigned int num_threads_sharing:12;
unsigned int num_cores_on_die:6;
} split;
u32 full;
};
union _cpuid4_leaf_ebx {
struct {
unsigned int coherency_line_size:12;
unsigned int physical_line_partition:10;
unsigned int ways_of_associativity:10;
} split;
u32 full;
};
union _cpuid4_leaf_ecx {
struct {
unsigned int number_of_sets:32;
} split;
u32 full;
};
struct _cpuid4_info {
union _cpuid4_leaf_eax eax;
union _cpuid4_leaf_ebx ebx;
union _cpuid4_leaf_ecx ecx;
unsigned long size;
cpumask_t shared_cpu_map;
};
unsigned short num_cache_leaves;
/* AMD doesn't have CPUID4. Emulate it here to report the same
information to the user. This makes some assumptions about the machine:
No L3, L2 not shared, no SMT etc. that is currently true on AMD CPUs.
In theory the TLBs could be reported as fake type (they are in "dummy").
Maybe later */
union l1_cache {
struct {
unsigned line_size : 8;
unsigned lines_per_tag : 8;
unsigned assoc : 8;
unsigned size_in_kb : 8;
};
unsigned val;
};
union l2_cache {
struct {
unsigned line_size : 8;
unsigned lines_per_tag : 4;
unsigned assoc : 4;
unsigned size_in_kb : 16;
};
unsigned val;
};
static const unsigned short assocs[] = {
[1] = 1, [2] = 2, [4] = 4, [6] = 8,
[8] = 16,
[0xf] = 0xffff // ??
};
static const unsigned char levels[] = { 1, 1, 2 };
static const unsigned char types[] = { 1, 2, 3 };
static void __cpuinit amd_cpuid4(int leaf, union _cpuid4_leaf_eax *eax,
union _cpuid4_leaf_ebx *ebx,
union _cpuid4_leaf_ecx *ecx)
{
unsigned dummy;
unsigned line_size, lines_per_tag, assoc, size_in_kb;
union l1_cache l1i, l1d;
union l2_cache l2;
eax->full = 0;
ebx->full = 0;
ecx->full = 0;
cpuid(0x80000005, &dummy, &dummy, &l1d.val, &l1i.val);
cpuid(0x80000006, &dummy, &dummy, &l2.val, &dummy);
if (leaf > 2 || !l1d.val || !l1i.val || !l2.val)
return;
eax->split.is_self_initializing = 1;
eax->split.type = types[leaf];
eax->split.level = levels[leaf];
eax->split.num_threads_sharing = 0;
eax->split.num_cores_on_die = current_cpu_data.x86_max_cores - 1;
if (leaf <= 1) {
union l1_cache *l1 = leaf == 0 ? &l1d : &l1i;
assoc = l1->assoc;
line_size = l1->line_size;
lines_per_tag = l1->lines_per_tag;
size_in_kb = l1->size_in_kb;
} else {
assoc = l2.assoc;
line_size = l2.line_size;
lines_per_tag = l2.lines_per_tag;
/* cpu_data has errata corrections for K7 applied */
size_in_kb = current_cpu_data.x86_cache_size;
}
if (assoc == 0xf)
eax->split.is_fully_associative = 1;
ebx->split.coherency_line_size = line_size - 1;
ebx->split.ways_of_associativity = assocs[assoc] - 1;
ebx->split.physical_line_partition = lines_per_tag - 1;
ecx->split.number_of_sets = (size_in_kb * 1024) / line_size /
(ebx->split.ways_of_associativity + 1) - 1;
}
static int __cpuinit cpuid4_cache_lookup(int index, struct _cpuid4_info *this_leaf)
{
union _cpuid4_leaf_eax eax;
union _cpuid4_leaf_ebx ebx;
union _cpuid4_leaf_ecx ecx;
unsigned edx;
if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
amd_cpuid4(index, &eax, &ebx, &ecx);
else
cpuid_count(4, index, &eax.full, &ebx.full, &ecx.full, &edx);
if (eax.split.type == CACHE_TYPE_NULL)
return -EIO; /* better error ? */
this_leaf->eax = eax;
this_leaf->ebx = ebx;
this_leaf->ecx = ecx;
this_leaf->size = (ecx.split.number_of_sets + 1) *
(ebx.split.coherency_line_size + 1) *
(ebx.split.physical_line_partition + 1) *
(ebx.split.ways_of_associativity + 1);
return 0;
}
/* will only be called once; __init is safe here */
static int __init find_num_cache_leaves(void)
{
unsigned int eax, ebx, ecx, edx;
union _cpuid4_leaf_eax cache_eax;
int i = -1;
do {
++i;
/* Do cpuid(4) loop to find out num_cache_leaves */
cpuid_count(4, i, &eax, &ebx, &ecx, &edx);
cache_eax.full = eax;
} while (cache_eax.split.type != CACHE_TYPE_NULL);
return i;
}
unsigned int __cpuinit init_intel_cacheinfo(struct cpuinfo_x86 *c)
{
unsigned int trace = 0, l1i = 0, l1d = 0, l2 = 0, l3 = 0; /* Cache sizes */
unsigned int new_l1d = 0, new_l1i = 0; /* Cache sizes from cpuid(4) */
unsigned int new_l2 = 0, new_l3 = 0, i; /* Cache sizes from cpuid(4) */
unsigned int l2_id = 0, l3_id = 0, num_threads_sharing, index_msb;
#ifdef CONFIG_X86_HT
unsigned int cpu = (c == &boot_cpu_data) ? 0 : (c - cpu_data);
#endif
if (c->cpuid_level > 3) {
static int is_initialized;
if (is_initialized == 0) {
/* Init num_cache_leaves from boot CPU */
num_cache_leaves = find_num_cache_leaves();
is_initialized++;
}
/*
* Whenever possible use cpuid(4), deterministic cache
* parameters cpuid leaf to find the cache details
*/
for (i = 0; i < num_cache_leaves; i++) {
struct _cpuid4_info this_leaf;
int retval;
retval = cpuid4_cache_lookup(i, &this_leaf);
if (retval >= 0) {
switch(this_leaf.eax.split.level) {
case 1:
if (this_leaf.eax.split.type ==
CACHE_TYPE_DATA)
new_l1d = this_leaf.size/1024;
else if (this_leaf.eax.split.type ==
CACHE_TYPE_INST)
new_l1i = this_leaf.size/1024;
break;
case 2:
new_l2 = this_leaf.size/1024;
num_threads_sharing = 1 + this_leaf.eax.split.num_threads_sharing;
index_msb = get_count_order(num_threads_sharing);
l2_id = c->apicid >> index_msb;
break;
case 3:
new_l3 = this_leaf.size/1024;
num_threads_sharing = 1 + this_leaf.eax.split.num_threads_sharing;
index_msb = get_count_order(num_threads_sharing);
l3_id = c->apicid >> index_msb;
break;
default:
break;
}
}
}
}
/*
* Don't use cpuid2 if cpuid4 is supported. For P4, we use cpuid2 for
* trace cache
*/
if ((num_cache_leaves == 0 || c->x86 == 15) && c->cpuid_level > 1) {
/* supports eax=2 call */
int i, j, n;
int regs[4];
unsigned char *dp = (unsigned char *)regs;
int only_trace = 0;
if (num_cache_leaves != 0 && c->x86 == 15)
only_trace = 1;
/* Number of times to iterate */
n = cpuid_eax(2) & 0xFF;
for ( i = 0 ; i < n ; i++ ) {
cpuid(2, &regs[0], &regs[1], &regs[2], &regs[3]);
/* If bit 31 is set, this is an unknown format */
for ( j = 0 ; j < 3 ; j++ ) {
if ( regs[j] < 0 ) regs[j] = 0;
}
/* Byte 0 is level count, not a descriptor */
for ( j = 1 ; j < 16 ; j++ ) {
unsigned char des = dp[j];
unsigned char k = 0;
/* look up this descriptor in the table */
while (cache_table[k].descriptor != 0)
{
if (cache_table[k].descriptor == des) {
if (only_trace && cache_table[k].cache_type != LVL_TRACE)
break;
switch (cache_table[k].cache_type) {
case LVL_1_INST:
l1i += cache_table[k].size;
break;
case LVL_1_DATA:
l1d += cache_table[k].size;
break;
case LVL_2:
l2 += cache_table[k].size;
break;
case LVL_3:
l3 += cache_table[k].size;
break;
case LVL_TRACE:
trace += cache_table[k].size;
break;
}
break;
}
k++;
}
}
}
}
if (new_l1d)
l1d = new_l1d;
if (new_l1i)
l1i = new_l1i;
if (new_l2) {
l2 = new_l2;
#ifdef CONFIG_X86_HT
cpu_llc_id[cpu] = l2_id;
#endif
}
if (new_l3) {
l3 = new_l3;
#ifdef CONFIG_X86_HT
cpu_llc_id[cpu] = l3_id;
#endif
}
if (trace)
printk (KERN_INFO "CPU: Trace cache: %dK uops", trace);
else if ( l1i )
printk (KERN_INFO "CPU: L1 I cache: %dK", l1i);
if (l1d)
printk(", L1 D cache: %dK\n", l1d);
else
printk("\n");
if (l2)
printk(KERN_INFO "CPU: L2 cache: %dK\n", l2);
if (l3)
printk(KERN_INFO "CPU: L3 cache: %dK\n", l3);
c->x86_cache_size = l3 ? l3 : (l2 ? l2 : (l1i+l1d));
return l2;
}
/* pointer to _cpuid4_info array (for each cache leaf) */
static struct _cpuid4_info *cpuid4_info[NR_CPUS];
#define CPUID4_INFO_IDX(x,y) (&((cpuid4_info[x])[y]))
#ifdef CONFIG_SMP
static void __cpuinit cache_shared_cpu_map_setup(unsigned int cpu, int index)
{
struct _cpuid4_info *this_leaf, *sibling_leaf;
unsigned long num_threads_sharing;
int index_msb, i;
struct cpuinfo_x86 *c = cpu_data;
this_leaf = CPUID4_INFO_IDX(cpu, index);
num_threads_sharing = 1 + this_leaf->eax.split.num_threads_sharing;
if (num_threads_sharing == 1)
cpu_set(cpu, this_leaf->shared_cpu_map);
else {
index_msb = get_count_order(num_threads_sharing);
for_each_online_cpu(i) {
if (c[i].apicid >> index_msb ==
c[cpu].apicid >> index_msb) {
cpu_set(i, this_leaf->shared_cpu_map);
if (i != cpu && cpuid4_info[i]) {
sibling_leaf = CPUID4_INFO_IDX(i, index);
cpu_set(cpu, sibling_leaf->shared_cpu_map);
}
}
}
}
}
static void __cpuinit cache_remove_shared_cpu_map(unsigned int cpu, int index)
{
struct _cpuid4_info *this_leaf, *sibling_leaf;
int sibling;
this_leaf = CPUID4_INFO_IDX(cpu, index);
for_each_cpu_mask(sibling, this_leaf->shared_cpu_map) {
sibling_leaf = CPUID4_INFO_IDX(sibling, index);
cpu_clear(cpu, sibling_leaf->shared_cpu_map);
}
}
#else
static void __init cache_shared_cpu_map_setup(unsigned int cpu, int index) {}
static void __init cache_remove_shared_cpu_map(unsigned int cpu, int index) {}
#endif
static void free_cache_attributes(unsigned int cpu)
{
kfree(cpuid4_info[cpu]);
cpuid4_info[cpu] = NULL;
}
static int __cpuinit detect_cache_attributes(unsigned int cpu)
{
struct _cpuid4_info *this_leaf;
unsigned long j;
int retval;
cpumask_t oldmask;
if (num_cache_leaves == 0)
return -ENOENT;
cpuid4_info[cpu] = kmalloc(
sizeof(struct _cpuid4_info) * num_cache_leaves, GFP_KERNEL);
if (unlikely(cpuid4_info[cpu] == NULL))
return -ENOMEM;
memset(cpuid4_info[cpu], 0,
sizeof(struct _cpuid4_info) * num_cache_leaves);
oldmask = current->cpus_allowed;
retval = set_cpus_allowed(current, cpumask_of_cpu(cpu));
if (retval)
goto out;
/* Do cpuid and store the results */
retval = 0;
for (j = 0; j < num_cache_leaves; j++) {
this_leaf = CPUID4_INFO_IDX(cpu, j);
retval = cpuid4_cache_lookup(j, this_leaf);
if (unlikely(retval < 0))
break;
cache_shared_cpu_map_setup(cpu, j);
}
set_cpus_allowed(current, oldmask);
out:
if (retval)
free_cache_attributes(cpu);
return retval;
}
#ifdef CONFIG_SYSFS
#include <linux/kobject.h>
#include <linux/sysfs.h>
extern struct sysdev_class cpu_sysdev_class; /* from drivers/base/cpu.c */
/* pointer to kobject for cpuX/cache */
static struct kobject * cache_kobject[NR_CPUS];
struct _index_kobject {
struct kobject kobj;
unsigned int cpu;
unsigned short index;
};
/* pointer to array of kobjects for cpuX/cache/indexY */
static struct _index_kobject *index_kobject[NR_CPUS];
#define INDEX_KOBJECT_PTR(x,y) (&((index_kobject[x])[y]))
#define show_one_plus(file_name, object, val) \
static ssize_t show_##file_name \
(struct _cpuid4_info *this_leaf, char *buf) \
{ \
return sprintf (buf, "%lu\n", (unsigned long)this_leaf->object + val); \
}
show_one_plus(level, eax.split.level, 0);
show_one_plus(coherency_line_size, ebx.split.coherency_line_size, 1);
show_one_plus(physical_line_partition, ebx.split.physical_line_partition, 1);
show_one_plus(ways_of_associativity, ebx.split.ways_of_associativity, 1);
show_one_plus(number_of_sets, ecx.split.number_of_sets, 1);
static ssize_t show_size(struct _cpuid4_info *this_leaf, char *buf)
{
return sprintf (buf, "%luK\n", this_leaf->size / 1024);
}
static ssize_t show_shared_cpu_map(struct _cpuid4_info *this_leaf, char *buf)
{
char mask_str[NR_CPUS];
cpumask_scnprintf(mask_str, NR_CPUS, this_leaf->shared_cpu_map);
return sprintf(buf, "%s\n", mask_str);
}
static ssize_t show_type(struct _cpuid4_info *this_leaf, char *buf) {
switch(this_leaf->eax.split.type) {
case CACHE_TYPE_DATA:
return sprintf(buf, "Data\n");
break;
case CACHE_TYPE_INST:
return sprintf(buf, "Instruction\n");
break;
case CACHE_TYPE_UNIFIED:
return sprintf(buf, "Unified\n");
break;
default:
return sprintf(buf, "Unknown\n");
break;
}
}
struct _cache_attr {
struct attribute attr;
ssize_t (*show)(struct _cpuid4_info *, char *);
ssize_t (*store)(struct _cpuid4_info *, const char *, size_t count);
};
#define define_one_ro(_name) \
static struct _cache_attr _name = \
__ATTR(_name, 0444, show_##_name, NULL)
define_one_ro(level);
define_one_ro(type);
define_one_ro(coherency_line_size);
define_one_ro(physical_line_partition);
define_one_ro(ways_of_associativity);
define_one_ro(number_of_sets);
define_one_ro(size);
define_one_ro(shared_cpu_map);
static struct attribute * default_attrs[] = {
&type.attr,
&level.attr,
&coherency_line_size.attr,
&physical_line_partition.attr,
&ways_of_associativity.attr,
&number_of_sets.attr,
&size.attr,
&shared_cpu_map.attr,
NULL
};
#define to_object(k) container_of(k, struct _index_kobject, kobj)
#define to_attr(a) container_of(a, struct _cache_attr, attr)
static ssize_t show(struct kobject * kobj, struct attribute * attr, char * buf)
{
struct _cache_attr *fattr = to_attr(attr);
struct _index_kobject *this_leaf = to_object(kobj);
ssize_t ret;
ret = fattr->show ?
fattr->show(CPUID4_INFO_IDX(this_leaf->cpu, this_leaf->index),
buf) :
0;
return ret;
}
static ssize_t store(struct kobject * kobj, struct attribute * attr,
const char * buf, size_t count)
{
return 0;
}
static struct sysfs_ops sysfs_ops = {
.show = show,
.store = store,
};
static struct kobj_type ktype_cache = {
.sysfs_ops = &sysfs_ops,
.default_attrs = default_attrs,
};
static struct kobj_type ktype_percpu_entry = {
.sysfs_ops = &sysfs_ops,
};
static void cpuid4_cache_sysfs_exit(unsigned int cpu)
{
kfree(cache_kobject[cpu]);
kfree(index_kobject[cpu]);
cache_kobject[cpu] = NULL;
index_kobject[cpu] = NULL;
free_cache_attributes(cpu);
}
static int __cpuinit cpuid4_cache_sysfs_init(unsigned int cpu)
{
if (num_cache_leaves == 0)
return -ENOENT;
detect_cache_attributes(cpu);
if (cpuid4_info[cpu] == NULL)
return -ENOENT;
/* Allocate all required memory */
cache_kobject[cpu] = kmalloc(sizeof(struct kobject), GFP_KERNEL);
if (unlikely(cache_kobject[cpu] == NULL))
goto err_out;
memset(cache_kobject[cpu], 0, sizeof(struct kobject));
index_kobject[cpu] = kmalloc(
sizeof(struct _index_kobject ) * num_cache_leaves, GFP_KERNEL);
if (unlikely(index_kobject[cpu] == NULL))
goto err_out;
memset(index_kobject[cpu], 0,
sizeof(struct _index_kobject) * num_cache_leaves);
return 0;
err_out:
cpuid4_cache_sysfs_exit(cpu);
return -ENOMEM;
}
/* Add/Remove cache interface for CPU device */
static int __cpuinit cache_add_dev(struct sys_device * sys_dev)
{
unsigned int cpu = sys_dev->id;
unsigned long i, j;
struct _index_kobject *this_object;
int retval = 0;
retval = cpuid4_cache_sysfs_init(cpu);
if (unlikely(retval < 0))
return retval;
cache_kobject[cpu]->parent = &sys_dev->kobj;
kobject_set_name(cache_kobject[cpu], "%s", "cache");
cache_kobject[cpu]->ktype = &ktype_percpu_entry;
retval = kobject_register(cache_kobject[cpu]);
for (i = 0; i < num_cache_leaves; i++) {
this_object = INDEX_KOBJECT_PTR(cpu,i);
this_object->cpu = cpu;
this_object->index = i;
this_object->kobj.parent = cache_kobject[cpu];
kobject_set_name(&(this_object->kobj), "index%1lu", i);
this_object->kobj.ktype = &ktype_cache;
retval = kobject_register(&(this_object->kobj));
if (unlikely(retval)) {
for (j = 0; j < i; j++) {
kobject_unregister(
&(INDEX_KOBJECT_PTR(cpu,j)->kobj));
}
kobject_unregister(cache_kobject[cpu]);
cpuid4_cache_sysfs_exit(cpu);
break;
}
}
return retval;
}
static void __cpuexit cache_remove_dev(struct sys_device * sys_dev)
{
unsigned int cpu = sys_dev->id;
unsigned long i;
for (i = 0; i < num_cache_leaves; i++) {
cache_remove_shared_cpu_map(cpu, i);
kobject_unregister(&(INDEX_KOBJECT_PTR(cpu,i)->kobj));
}
kobject_unregister(cache_kobject[cpu]);
cpuid4_cache_sysfs_exit(cpu);
return;
}
static int __cpuinit cacheinfo_cpu_callback(struct notifier_block *nfb,
unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
struct sys_device *sys_dev;
sys_dev = get_cpu_sysdev(cpu);
switch (action) {
case CPU_ONLINE:
cache_add_dev(sys_dev);
break;
case CPU_DEAD:
cache_remove_dev(sys_dev);
break;
}
return NOTIFY_OK;
}
static struct notifier_block __cpuinitdata cacheinfo_cpu_notifier =
{
.notifier_call = cacheinfo_cpu_callback,
};
static int __cpuinit cache_sysfs_init(void)
{
int i;
if (num_cache_leaves == 0)
return 0;
register_hotcpu_notifier(&cacheinfo_cpu_notifier);
for_each_online_cpu(i) {
cacheinfo_cpu_callback(&cacheinfo_cpu_notifier, CPU_ONLINE,
(void *)(long)i);
}
return 0;
}
device_initcall(cache_sysfs_init);
#endif
@@ -368,6 +368,7 @@ sysexit_scrit: /**** START OF SYSEXIT CRITICAL REGION ****/
movl ESI(%esp), %esi
sysexit
14: __DISABLE_INTERRUPTS
TRACE_IRQS_OFF
sysexit_ecrit: /**** END OF SYSEXIT CRITICAL REGION ****/
push %esp
call evtchn_do_upcall
@@ -427,11 +428,13 @@ restore_nocheck:
restore_nocheck:
movl EFLAGS(%esp), %eax
testl $(VM_MASK|NMI_MASK), %eax
CFI_REMEMBER_STATE
jnz hypervisor_iret
shr $9, %eax # EAX[0] == IRET_EFLAGS.IF
GET_VCPU_INFO
andb evtchn_upcall_mask(%esi),%al
andb $1,%al # EAX[0] == IRET_EFLAGS.IF & event_mask
CFI_REMEMBER_STATE
jnz restore_all_enable_events # != 0 => enable event delivery
#endif
TRACE_IRQS_IRET
@@ -455,8 +458,8 @@ iret_exc:
.long 1b,iret_exc
.previous
#ifndef CONFIG_XEN
CFI_RESTORE_STATE
#ifndef CONFIG_XEN
ldt_ss:
larl OLDSS(%esp), %eax
jnz restore_nocheck
@@ -485,14 +488,36 @@ ldt_ss:
.align 4
.long 1b,iret_exc
.previous
CFI_ENDPROC
#else
ALIGN
restore_all_enable_events:
TRACE_IRQS_ON
__ENABLE_INTERRUPTS
scrit: /**** START OF CRITICAL REGION ****/
__TEST_PENDING
jnz 14f # process more events if necessary...
RESTORE_REGS
addl $4, %esp
CFI_ADJUST_CFA_OFFSET -4
1: iret
.section __ex_table,"a"
.align 4
.long 1b,iret_exc
.previous
14: __DISABLE_INTERRUPTS
TRACE_IRQS_OFF
jmp 11f
ecrit: /**** END OF CRITICAL REGION ****/
CFI_RESTORE_STATE
hypervisor_iret:
andl $~NMI_MASK, EFLAGS(%esp)
RESTORE_REGS
addl $4, %esp
CFI_ADJUST_CFA_OFFSET -4
jmp hypercall_page + (__HYPERVISOR_iret * 32)
#endif
CFI_ENDPROC
# perform work that needs to be done immediately before resumption
ALIGN
@@ -736,7 +761,9 @@ error_code:
# critical region we know that the entire frame is present and correct
# so we can simply throw away the new one.
ENTRY(hypervisor_callback)
RING0_INT_FRAME
pushl %eax
CFI_ADJUST_CFA_OFFSET 4
SAVE_ALL
movl EIP(%esp),%eax
cmpl $scrit,%eax
@@ -749,26 +776,13 @@ ENTRY(hypervisor_callback)
ja 11f
addl $OLDESP,%esp # Remove eflags...ebx from stack frame.
11: push %esp
CFI_ADJUST_CFA_OFFSET 4
call evtchn_do_upcall
add $4,%esp
CFI_ADJUST_CFA_OFFSET -4
jmp ret_from_intr
CFI_ENDPROC
ALIGN
restore_all_enable_events:
__ENABLE_INTERRUPTS
scrit: /**** START OF CRITICAL REGION ****/
__TEST_PENDING
jnz 14f # process more events if necessary...
RESTORE_REGS
addl $4, %esp
1: iret
.section __ex_table,"a"
.align 4
.long 1b,iret_exc
.previous
14: __DISABLE_INTERRUPTS
jmp 11b
ecrit: /**** END OF CRITICAL REGION ****/
# [How we do the fixup]. We want to merge the current stack frame with the
# just-interrupted frame. How we do this depends on where in the critical
# region the interrupted handler was executing, and so how many saved
@@ -835,6 +849,7 @@ ENTRY(failsafe_callback)
addl $16,%esp # EAX != 0 => Category 2 (Bad IRET)
jmp iret_exc
5: addl $16,%esp # EAX == 0 => Category 1 (Bad segment)
RING0_INT_FRAME
pushl $0
SAVE_ALL
jmp ret_from_exception
@@ -860,6 +875,7 @@ ENTRY(failsafe_callback)
.long 4b,9b; \
.previous
#endif
CFI_ENDPROC
ENTRY(coprocessor_error)
RING0_INT_FRAME
@@ -1187,8 +1203,11 @@ ENDPROC(arch_unwind_init_running)
#endif
ENTRY(fixup_4gb_segment)
RING0_EC_FRAME
pushl $do_fixup_4gb_segment
CFI_ADJUST_CFA_OFFSET 4
jmp error_code
CFI_ENDPROC
.section .rodata,"a"
.align 4
@@ -27,7 +27,6 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/config.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/slab.h>
@@ -9,11 +9,10 @@
#include <asm/cache.h>
#include <asm/thread_info.h>
#include <asm/asm-offsets.h>
#include <asm/dwarf2.h>
#include <xen/interface/xen.h>
#include <xen/interface/elfnote.h>
#define _PAGE_PRESENT 0x1
/*
* References to members of the new_cpu_data structure.
*/
@@ -65,14 +64,13 @@ ENTRY(startup_32)
pushl %eax # fake return address
jmp start_kernel
L6:
jmp L6 # main should never return here, but
# just in case, we know what happens.
#define HYPERCALL_PAGE_OFFSET 0x1000
.org HYPERCALL_PAGE_OFFSET
ENTRY(hypercall_page)
CFI_STARTPROC
.skip 0x1000
CFI_ENDPROC
/*
* Real beginning of normal "text" segment
@@ -206,3 +204,4 @@ ENTRY(cpu_gdt_table)
ELFNOTE(Xen, XEN_ELFNOTE_L1_MFN_VALID, .long, _PAGE_PRESENT,_PAGE_PRESENT)
#endif
ELFNOTE(Xen, XEN_ELFNOTE_LOADER, .asciz, "generic")
ELFNOTE(Xen, XEN_ELFNOTE_SUSPEND_CANCEL, .long, 1)
@@ -60,9 +60,6 @@
#include <asm/tlbflush.h>
#include <asm/cpu.h>
#include <asm/tlbflush.h>
#include <asm/cpu.h>
asmlinkage void ret_from_fork(void) __asm__("ret_from_fork");
static int hlt_counter;
@@ -104,7 +101,7 @@ EXPORT_SYMBOL(enable_hlt);
* to poll the ->work.need_resched flag instead of waiting for the
* cross-CPU IPI to arrive. Use this option with caution.
*/
static void poll_idle(void)
static void poll_idle (void)
{
local_irq_enable();
@@ -172,10 +169,7 @@ void cpu_idle(void)
__get_cpu_var(cpu_idle_state) = 0;
rmb();
idle = pm_idle;
if (!idle)
idle = xen_idle;
idle = xen_idle; /* no alternatives */
if (cpu_is_offline(cpu))
play_dead();
@@ -1847,6 +1847,9 @@ static __init int add_pcspkr(void)
struct platform_device *pd;
int ret;
if (!is_initial_xendomain())
return 0;
pd = platform_device_alloc("pcspkr", -1);
if (!pd)
return -ENOMEM;
@@ -1022,16 +1022,21 @@ void halt(void)
}
EXPORT_SYMBOL(halt);
/* No locking required. We are only CPU running, and interrupts are off. */
/* No locking required. Interrupts are disabled on all CPUs. */
void time_resume(void)
{
unsigned int cpu;
init_cpu_khz();
get_time_values_from_xen(0);
for_each_online_cpu(cpu) {
get_time_values_from_xen(cpu);
per_cpu(processed_system_time, cpu) =
per_cpu(shadow_time, 0).system_timestamp;
init_missing_ticks_accounting(cpu);
}
processed_system_time = per_cpu(shadow_time, 0).system_timestamp;
per_cpu(processed_system_time, 0) = processed_system_time;
init_missing_ticks_accounting(0);
update_wallclock();
}
@@ -30,7 +30,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/sched.h>
#include <linux/mm.h>
#include <linux/vmalloc.h>
@@ -374,8 +374,6 @@ void iounmap(volatile void __iomem *addr)
}
EXPORT_SYMBOL(iounmap);
#ifdef __i386__
void __init *bt_ioremap(unsigned long phys_addr, unsigned long size)
{
unsigned long offset, last_addr;
@@ -443,5 +441,3 @@ void __init bt_iounmap(void *addr, unsigned long size)
--nrpages;
}
}
#endif /* __i386__ */
+27 -28
View File
@@ -573,64 +573,67 @@ void make_pages_writable(void *va, unsigned int nr, unsigned int feature)
}
}
static inline int pgd_walk_set_prot(struct page *page, pgprot_t flags)
static inline void pgd_walk_set_prot(struct page *page, pgprot_t flags)
{
unsigned long pfn = page_to_pfn(page);
int rc;
if (PageHighMem(page))
return pgprot_val(flags) & _PAGE_RW
? test_and_clear_bit(PG_pinned, &page->flags)
: !test_and_set_bit(PG_pinned, &page->flags);
BUG_ON(HYPERVISOR_update_va_mapping(
(unsigned long)__va(pfn << PAGE_SHIFT),
pfn_pte(pfn, flags), 0));
return 0;
if (PageHighMem(page)) {
if (pgprot_val(flags) & _PAGE_RW)
clear_bit(PG_pinned, &page->flags);
else
set_bit(PG_pinned, &page->flags);
} else {
rc = HYPERVISOR_update_va_mapping(
(unsigned long)__va(pfn << PAGE_SHIFT),
pfn_pte(pfn, flags), 0);
if (rc)
BUG();
}
}
static int pgd_walk(pgd_t *pgd_base, pgprot_t flags)
static void pgd_walk(pgd_t *pgd_base, pgprot_t flags)
{
pgd_t *pgd = pgd_base;
pud_t *pud;
pmd_t *pmd;
int g, u, m, flush;
int g, u, m, rc;
if (xen_feature(XENFEAT_auto_translated_physmap))
return 0;
for (g = 0, flush = 0; g < USER_PTRS_PER_PGD; g++, pgd++) {
for (g = 0; g < USER_PTRS_PER_PGD; g++, pgd++) {
if (pgd_none(*pgd))
continue;
pud = pud_offset(pgd, 0);
if (PTRS_PER_PUD > 1) /* not folded */
flush |= pgd_walk_set_prot(virt_to_page(pud),flags);
pgd_walk_set_prot(virt_to_page(pud),flags);
for (u = 0; u < PTRS_PER_PUD; u++, pud++) {
if (pud_none(*pud))
continue;
pmd = pmd_offset(pud, 0);
if (PTRS_PER_PMD > 1) /* not folded */
flush |= pgd_walk_set_prot(virt_to_page(pmd),flags);
pgd_walk_set_prot(virt_to_page(pmd),flags);
for (m = 0; m < PTRS_PER_PMD; m++, pmd++) {
if (pmd_none(*pmd))
continue;
flush |= pgd_walk_set_prot(pmd_page(*pmd),flags);
pgd_walk_set_prot(pmd_page(*pmd),flags);
}
}
}
BUG_ON(HYPERVISOR_update_va_mapping(
rc = HYPERVISOR_update_va_mapping(
(unsigned long)pgd_base,
pfn_pte(virt_to_phys(pgd_base)>>PAGE_SHIFT, flags),
UVMF_TLB_FLUSH));
return flush;
UVMF_TLB_FLUSH);
if (rc)
BUG();
}
static void __pgd_pin(pgd_t *pgd)
{
if (pgd_walk(pgd, PAGE_KERNEL_RO))
kmap_flush_unused();
pgd_walk(pgd, PAGE_KERNEL_RO);
kmap_flush_unused();
xen_pgd_pin(__pa(pgd));
set_bit(PG_pinned, &virt_to_page(pgd)->flags);
}
@@ -638,8 +641,7 @@ static void __pgd_pin(pgd_t *pgd)
static void __pgd_unpin(pgd_t *pgd)
{
xen_pgd_unpin(__pa(pgd));
if (pgd_walk(pgd, PAGE_KERNEL))
kmap_flush_unused();
pgd_walk(pgd, PAGE_KERNEL);
clear_bit(PG_pinned, &virt_to_page(pgd)->flags);
}
@@ -671,9 +673,6 @@ void mm_pin_all(void)
{
struct page *page;
/* Only pgds on the pgd_list please: none hidden in the slab cache. */
kmem_cache_shrink(pgd_cache);
if (xen_feature(XENFEAT_writable_page_tables))
return;
@@ -1,8 +1,6 @@
#ifndef OPROFILE_PERFMON_H
#define OPROFILE_PERFMON_H
#include <linux/config.h>
#ifdef CONFIG_PERFMON
int __perfmon_init(void);
void __perfmon_exit(void);
@@ -4,7 +4,6 @@
* Copyright (C) 2005 Dan Magenheimer <dan.magenheimer@hp.com>
*/
#include <linux/config.h>
#include <asm/processor.h>
#include <asm/asmmacro.h>
-1
View File
@@ -15,7 +15,6 @@
* VA Linux Systems Japan K.K.
*/
#include <linux/config.h>
#include <linux/mm.h>
#include <linux/efi.h>
@@ -21,7 +21,6 @@
*
*/
#include <linux/config.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/slab.h>
@@ -303,6 +303,7 @@ xencomm_privcmd_domctl(privcmd_hypercall_t *hypercall)
case XEN_DOMCTL_hypercall_init:
case XEN_DOMCTL_arch_setup:
case XEN_DOMCTL_settimeoffset:
case XEN_DOMCTL_sendtrigger:
break;
default:
printk("%s: unknown domctl cmd %d\n", __func__, kern_op.cmd);
@@ -8,8 +8,6 @@
* Dan Magenheimer <dan.magenheimer@.hp.com>
*/
#include <linux/config.h>
#include <asm/asmmacro.h>
#include <asm/cache.h>
#include <asm/errno.h>
@@ -29,8 +29,6 @@
* Table is based upon EAS2.6 (Oct 1999)
*/
#include <linux/config.h>
#include <asm/asmmacro.h>
#include <asm/break.h>
#include <asm/ia32.h>
@@ -1,5 +1,3 @@
#include <linux/config.h>
#include <asm/cache.h>
#ifdef CONFIG_XEN
@@ -4,7 +4,6 @@
* Copyright (C) 2005 Dan Magenheimer <dan.magenheimer@hp.com>
*/
#include <linux/config.h>
#include <asm/processor.h>
#include <asm/asmmacro.h>
+1 -1
View File
@@ -450,7 +450,7 @@ config CALGARY_IOMMU
bool "IBM Calgary IOMMU support"
default y
select SWIOTLB
depends on PCI && EXPERIMENTAL
depends on PCI && !X86_64_XEN && EXPERIMENTAL
help
Support for hardware IOMMUs in IBM's xSeries x366 and x460
systems. Needed to run systems with more than 3GB of memory
@@ -61,9 +61,7 @@ ifdef CONFIG_XEN
time-y += ../../i386/kernel/time-xen.o
pci-dma-y += ../../i386/kernel/pci-dma-xen.o
microcode-$(subst m,y,$(CONFIG_MICROCODE)) := ../../i386/kernel/microcode-xen.o
intel_cacheinfo-y := ../../i386/kernel/cpu/intel_cacheinfo-xen.o
quirks-y := ../../i386/kernel/quirks-xen.o
alternative-y := ../../i386/kernel/alternative-xen.o
n-obj-xen := i8259.o reboot.o i8237.o smpboot.o trampoline.o
@@ -1,4 +1,3 @@
#include <linux/config.h>
#include <linux/console.h>
#include <linux/kernel.h>
#include <linux/init.h>
@@ -116,19 +116,21 @@ NMI_MASK = 0x80000000
CFI_ADJUST_CFA_OFFSET -(6*8)
.endm
.macro CFI_DEFAULT_STACK start=1
.macro CFI_DEFAULT_STACK start=1,adj=0
.if \start
CFI_STARTPROC simple
CFI_DEF_CFA rsp,SS+8
CFI_DEF_CFA rsp,SS+8-(\adj*ARGOFFSET)
.else
CFI_DEF_CFA_OFFSET SS+8
CFI_DEF_CFA_OFFSET SS+8-(\adj*ARGOFFSET)
.endif
.if \adj == 0
CFI_REL_OFFSET r15,R15
CFI_REL_OFFSET r14,R14
CFI_REL_OFFSET r13,R13
CFI_REL_OFFSET r12,R12
CFI_REL_OFFSET rbp,RBP
CFI_REL_OFFSET rbx,RBX
.endif
CFI_REL_OFFSET r11,R11
CFI_REL_OFFSET r10,R10
CFI_REL_OFFSET r9,R9
@@ -363,8 +365,8 @@ ENTRY(int_ret_from_sys_call)
CFI_REL_OFFSET r9,R9-ARGOFFSET
CFI_REL_OFFSET r10,R10-ARGOFFSET
CFI_REL_OFFSET r11,R11-ARGOFFSET
TRACE_IRQS_OFF
XEN_BLOCK_EVENTS(%rsi)
TRACE_IRQS_OFF
testb $3,CS-ARGOFFSET(%rsp)
jnz 1f
/* Need to set the proper %ss (not NULL) for ring 3 iretq */
@@ -534,7 +536,7 @@ END(stub_rt_sigreturn)
*/
retint_check:
CFI_DEFAULT_STACK
CFI_DEFAULT_STACK adj=1
movl threadinfo_flags(%rcx),%edx
andl %edi,%edx
CFI_REMEMBER_STATE
@@ -838,6 +840,7 @@ ENTRY(error_entry)
CFI_REL_OFFSET r15,R15
#if 0
cmpl $__KERNEL_CS,CS(%rsp)
CFI_REMEMBER_STATE
je error_kernelspace
#endif
error_call_handler:
@@ -864,7 +867,7 @@ error_exit:
TRACE_IRQS_IRETQ
jmp retint_restore_args
error_kernelspace:
#if 0
/*
* We need to re-write the logic here because we don't do iretq to
* to return to user mode. It's still possible that we get trap/fault
@@ -872,7 +875,8 @@ error_kernelspace:
* for example).
*
*/
#if 0
CFI_RESTORE_STATE
error_kernelspace:
incl %ebx
/* There are two places in the kernel that can potentially fault with
usergs. Handle them here. The exception handlers after
@@ -888,11 +892,13 @@ error_kernelspace:
cmpq $gs_change,RIP(%rsp)
je error_swapgs
jmp error_sti
#endif
#endif
CFI_ENDPROC
END(error_entry)
ENTRY(hypervisor_callback)
zeroentry do_hypervisor_callback
END(hypervisor_callback)
/*
* Copied from arch/xen/i386/kernel/entry.S
@@ -909,48 +915,66 @@ ENTRY(hypervisor_callback)
# existing activation in its critical region -- if so, we pop the current
# activation and restart the handler using the previous one.
ENTRY(do_hypervisor_callback) # do_hypervisor_callback(struct *pt_regs)
CFI_STARTPROC
# Since we don't modify %rdi, evtchn_do_upall(struct *pt_regs) will
# see the correct pointer to the pt_regs
movq %rdi, %rsp # we don't return, adjust the stack frame
11: movq %gs:pda_irqstackptr,%rax
incl %gs:pda_irqcount
cmovzq %rax,%rsp
pushq %rdi
CFI_ENDPROC
CFI_DEFAULT_STACK
11: incl %gs:pda_irqcount
movq %rsp,%rbp
CFI_DEF_CFA_REGISTER rbp
cmovzq %gs:pda_irqstackptr,%rsp
pushq %rbp # backlink for old unwinder
call evtchn_do_upcall
popq %rsp
CFI_DEF_CFA_REGISTER rsp
decl %gs:pda_irqcount
jmp error_exit
CFI_ENDPROC
END(do_hypervisor_callback)
#ifdef CONFIG_X86_LOCAL_APIC
KPROBE_ENTRY(nmi)
zeroentry do_nmi_callback
ENTRY(do_nmi_callback)
CFI_STARTPROC
addq $8, %rsp
CFI_ENDPROC
CFI_DEFAULT_STACK
call do_nmi
orl $NMI_MASK,EFLAGS(%rsp)
RESTORE_REST
XEN_BLOCK_EVENTS(%rsi)
TRACE_IRQS_OFF
GET_THREAD_INFO(%rcx)
jmp retint_restore_args
CFI_ENDPROC
.previous .text
END(nmi)
#endif
ALIGN
restore_all_enable_events:
CFI_DEFAULT_STACK adj=1
TRACE_IRQS_ON
XEN_UNBLOCK_EVENTS(%rsi) # %rsi is already set up...
scrit: /**** START OF CRITICAL REGION ****/
XEN_TEST_PENDING(%rsi)
CFI_REMEMBER_STATE
jnz 14f # process more events if necessary...
XEN_PUT_VCPU_INFO(%rsi)
RESTORE_ARGS 0,8,0
HYPERVISOR_IRET 0
CFI_RESTORE_STATE
14: XEN_LOCKED_BLOCK_EVENTS(%rsi)
XEN_PUT_VCPU_INFO(%rsi)
SAVE_REST
movq %rsp,%rdi # set the argument again
jmp 11b
CFI_ENDPROC
ecrit: /**** END OF CRITICAL REGION ****/
# At this point, unlike on x86-32, we don't do the fixup to simplify the
# code and the stack frame is more complex on x86-64.
@@ -970,8 +994,12 @@ ecrit: /**** END OF CRITICAL REGION ****/
# We distinguish between categories by comparing each saved segment register
# with its current contents: any discrepancy means we in category 1.
ENTRY(failsafe_callback)
_frame (RIP-0x30)
CFI_REL_OFFSET rcx, 0
CFI_REL_OFFSET r11, 8
movw %ds,%cx
cmpw %cx,0x10(%rsp)
CFI_REMEMBER_STATE
jne 1f
movw %es,%cx
cmpw %cx,0x18(%rsp)
@@ -984,17 +1012,26 @@ ENTRY(failsafe_callback)
jne 1f
/* All segments match their saved values => Category 2 (Bad IRET). */
movq (%rsp),%rcx
CFI_RESTORE rcx
movq 8(%rsp),%r11
CFI_RESTORE r11
addq $0x30,%rsp
CFI_ADJUST_CFA_OFFSET -0x30
movq $11,%rdi /* SIGSEGV */
jmp do_exit
CFI_RESTORE_STATE
1: /* Segment mismatch => Category 1 (Bad segment). Retry the IRET. */
movq (%rsp),%rcx
CFI_RESTORE rcx
movq 8(%rsp),%r11
CFI_RESTORE r11
addq $0x30,%rsp
CFI_ADJUST_CFA_OFFSET -0x30
pushq $0
CFI_ADJUST_CFA_OFFSET 8
SAVE_ALL
jmp error_exit
CFI_ENDPROC
#if 0
.section __ex_table,"a"
.align 8
@@ -1117,12 +1154,12 @@ END(device_not_available)
/* runs on exception stack */
KPROBE_ENTRY(debug)
INTR_FRAME
/* pushq $0
/* INTR_FRAME
pushq $0
CFI_ADJUST_CFA_OFFSET 8 */
zeroentry do_debug
/* paranoid_exit */
CFI_ENDPROC
/* paranoidexit
CFI_ENDPROC */
END(debug)
.previous .text
@@ -1144,12 +1181,12 @@ END(nmi)
#endif
KPROBE_ENTRY(int3)
INTR_FRAME
/* pushq $0
/* INTR_FRAME
pushq $0
CFI_ADJUST_CFA_OFFSET 8 */
zeroentry do_int3
/* jmp paranoid_exit1 */
CFI_ENDPROC
/* jmp paranoid_exit1
CFI_ENDPROC */
END(int3)
.previous .text
@@ -1193,9 +1230,11 @@ END(segment_not_present)
/* runs on exception stack */
ENTRY(stack_segment)
XCPT_FRAME
/* XCPT_FRAME
paranoidentry do_stack_segment */
errorentry do_stack_segment
CFI_ENDPROC
/* jmp paranoid_exit1
CFI_ENDPROC */
END(stack_segment)
KPROBE_ENTRY(general_protection)
@@ -10,7 +10,6 @@
*
* Hacked to pieces for Xen by Chris Wright.
*/
#include <linux/config.h>
#include <linux/threads.h>
#include <linux/cpumask.h>
#include <linux/string.h>
@@ -22,11 +22,9 @@
#include <asm/page.h>
#include <asm/msr.h>
#include <asm/cache.h>
#include <asm/dwarf2.h>
#include <xen/interface/elfnote.h>
#define _PAGE_PRESENT 0x1
.section .bootstrap.text, "ax", @progbits
.code64
#define VIRT_ENTRY_OFFSET 0x0
@@ -42,6 +40,7 @@ ENTRY(_start)
/* rsi is pointer to startup info structure.
pass it to C */
movq %rsi,%rdi
pushq $0 # fake return address
jmp x86_64_start_kernel
ENTRY(stext)
@@ -82,7 +81,25 @@ NEXT_PAGE(level2_kernel_pgt)
.fill 512,8,0
NEXT_PAGE(hypercall_page)
.fill 512,8,0
CFI_STARTPROC
.rept 0x1000 / 0x20
.skip 1 /* push %rcx */
CFI_ADJUST_CFA_OFFSET 8
CFI_REL_OFFSET rcx,0
.skip 2 /* push %r11 */
CFI_ADJUST_CFA_OFFSET 8
CFI_REL_OFFSET rcx,0
.skip 5 /* mov $#,%eax */
.skip 2 /* syscall */
.skip 2 /* pop %r11 */
CFI_ADJUST_CFA_OFFSET -8
CFI_RESTORE r11
.skip 1 /* pop %rcx */
CFI_ADJUST_CFA_OFFSET -8
CFI_RESTORE rcx
.align 0x20,0 /* ret */
.endr
CFI_ENDPROC
#undef NEXT_PAGE
@@ -186,3 +203,4 @@ ENTRY(empty_zero_page)
ELFNOTE(Xen, XEN_ELFNOTE_L1_MFN_VALID, .quad, _PAGE_PRESENT,_PAGE_PRESENT)
ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .asciz, "writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel")
ELFNOTE(Xen, XEN_ELFNOTE_LOADER, .asciz, "generic")
ELFNOTE(Xen, XEN_ELFNOTE_SUSPEND_CANCEL, .long, 1)
@@ -47,11 +47,11 @@ EXPORT_SYMBOL(init_task);
* on exact cacheline boundaries, to eliminate cacheline ping-pong.
*/
DEFINE_PER_CPU(struct tss_struct, init_tss) ____cacheline_internodealigned_in_smp = INIT_TSS;
#endif
/* Copies of the original ist values from the tss are only accessed during
* debugging, no special alignment required.
*/
DEFINE_PER_CPU(struct orig_ist, orig_ist);
#endif
#define ALIGN_TO_4K __attribute__((section(".data.init_task")))
@@ -117,7 +117,7 @@ void exit_idle(void)
* to poll the ->need_resched flag instead of waiting for the
* cross-CPU IPI to arrive. Use this option with caution.
*/
static void poll_idle(void)
static void poll_idle (void)
{
local_irq_enable();
@@ -179,9 +179,7 @@ void cpu_idle (void)
if (__get_cpu_var(cpu_idle_state))
__get_cpu_var(cpu_idle_state) = 0;
rmb();
idle = pm_idle;
if (!idle)
idle = xen_idle;
idle = xen_idle; /* no alternatives */
if (cpu_is_offline(smp_processor_id()))
play_dead();
enter_idle();
@@ -676,7 +676,8 @@ void __init setup_arch(char **cmdline_p)
init_memory_mapping(0, (end_pfn_map << PAGE_SHIFT));
/* dmi_scan_machine(); */
if (is_initial_xendomain())
dmi_scan_machine();
#ifdef CONFIG_ACPI_NUMA
/*
@@ -691,6 +692,14 @@ void __init setup_arch(char **cmdline_p)
contig_initmem_init(0, end_pfn);
#endif
#ifdef CONFIG_XEN
/*
* Reserve kernel, physmap, start info, initial page tables, and
* direct mapping.
*/
reserve_bootmem_generic(__pa_symbol(&_text),
(table_end << PAGE_SHIFT) - __pa_symbol(&_text));
#else
/* Reserve direct mapping */
reserve_bootmem_generic(table_start << PAGE_SHIFT,
(table_end - table_start) << PAGE_SHIFT);
@@ -699,12 +708,6 @@ void __init setup_arch(char **cmdline_p)
reserve_bootmem_generic(__pa_symbol(&_text),
__pa_symbol(&_end) - __pa_symbol(&_text));
#ifdef CONFIG_XEN
/* reserve physmap, start info and initial page tables */
reserve_bootmem_generic(__pa_symbol(&_end),
(table_start << PAGE_SHIFT) -
__pa_symbol(&_end));
#else
/*
* reserve physical page 0 - it's a special BIOS page on many boxes,
* enabling clean reboots, SMP operation, laptop functions.
@@ -1625,13 +1628,6 @@ struct seq_operations cpuinfo_op = {
.show = show_cpuinfo,
};
static int __init run_dmi_scan(void)
{
dmi_scan_machine();
return 0;
}
core_initcall(run_dmi_scan);
#if defined(CONFIG_INPUT_PCSPKR) || defined(CONFIG_INPUT_PCSPKR_MODULE)
#include <linux/platform_device.h>
static __init int add_pcspkr(void)
@@ -1639,6 +1635,9 @@ static __init int add_pcspkr(void)
struct platform_device *pd;
int ret;
if (!is_initial_xendomain())
return 0;
pd = platform_device_alloc("pcspkr", -1);
if (!pd)
return -ENOMEM;
@@ -1,60 +0,0 @@
/* Exports for assembly files.
All C exports should go in the respective C files. */
#include <linux/config.h>
#include <linux/module.h>
#include <linux/smp.h>
#include <asm/semaphore.h>
#include <asm/processor.h>
#include <asm/uaccess.h>
#include <asm/pgtable.h>
EXPORT_SYMBOL(kernel_thread);
EXPORT_SYMBOL(__down_failed);
EXPORT_SYMBOL(__down_failed_interruptible);
EXPORT_SYMBOL(__down_failed_trylock);
EXPORT_SYMBOL(__up_wakeup);
EXPORT_SYMBOL(__get_user_1);
EXPORT_SYMBOL(__get_user_2);
EXPORT_SYMBOL(__get_user_4);
EXPORT_SYMBOL(__get_user_8);
EXPORT_SYMBOL(__put_user_1);
EXPORT_SYMBOL(__put_user_2);
EXPORT_SYMBOL(__put_user_4);
EXPORT_SYMBOL(__put_user_8);
EXPORT_SYMBOL(copy_user_generic);
EXPORT_SYMBOL(copy_from_user);
EXPORT_SYMBOL(copy_to_user);
EXPORT_SYMBOL(copy_page);
EXPORT_SYMBOL(clear_page);
#ifdef CONFIG_SMP
extern void FASTCALL( __write_lock_failed(rwlock_t *rw));
extern void FASTCALL( __read_lock_failed(rwlock_t *rw));
EXPORT_SYMBOL(__write_lock_failed);
EXPORT_SYMBOL(__read_lock_failed);
#endif
/* Export string functions. We normally rely on gcc builtin for most of these,
but gcc sometimes decides not to inline them. */
#undef memcpy
#undef memset
#undef memmove
extern void * memset(void *,int,__kernel_size_t);
extern void * memcpy(void *,const void *,__kernel_size_t);
extern void * __memcpy(void *,const void *,__kernel_size_t);
EXPORT_SYMBOL(memset);
EXPORT_SYMBOL(memcpy);
EXPORT_SYMBOL(__memcpy);
EXPORT_SYMBOL(empty_zero_page);
EXPORT_SYMBOL(load_gs_index);
+124 -88
View File
@@ -102,14 +102,11 @@ static void __meminit early_make_page_readonly(void *va, unsigned int feature)
BUG();
}
void make_page_readonly(void *va, unsigned int feature)
static void __make_page_readonly(void *va)
{
pgd_t *pgd; pud_t *pud; pmd_t *pmd; pte_t pte, *ptep;
unsigned long addr = (unsigned long) va;
if (xen_feature(feature))
return;
pgd = pgd_offset_k(addr);
pud = pud_offset(pgd, addr);
pmd = pmd_offset(pud, addr);
@@ -120,17 +117,14 @@ void make_page_readonly(void *va, unsigned int feature)
xen_l1_entry_update(ptep, pte); /* fallback */
if ((addr >= VMALLOC_START) && (addr < VMALLOC_END))
make_page_readonly(__va(pte_pfn(pte) << PAGE_SHIFT), feature);
__make_page_readonly(__va(pte_pfn(pte) << PAGE_SHIFT));
}
void make_page_writable(void *va, unsigned int feature)
static void __make_page_writable(void *va)
{
pgd_t *pgd; pud_t *pud; pmd_t *pmd; pte_t pte, *ptep;
unsigned long addr = (unsigned long) va;
if (xen_feature(feature))
return;
pgd = pgd_offset_k(addr);
pud = pud_offset(pgd, addr);
pmd = pmd_offset(pud, addr);
@@ -141,7 +135,19 @@ void make_page_writable(void *va, unsigned int feature)
xen_l1_entry_update(ptep, pte); /* fallback */
if ((addr >= VMALLOC_START) && (addr < VMALLOC_END))
make_page_writable(__va(pte_pfn(pte) << PAGE_SHIFT), feature);
__make_page_writable(__va(pte_pfn(pte) << PAGE_SHIFT));
}
void make_page_readonly(void *va, unsigned int feature)
{
if (!xen_feature(feature))
__make_page_readonly(va);
}
void make_page_writable(void *va, unsigned int feature)
{
if (!xen_feature(feature))
__make_page_writable(va);
}
void make_pages_readonly(void *va, unsigned nr, unsigned int feature)
@@ -150,7 +156,7 @@ void make_pages_readonly(void *va, unsigned nr, unsigned int feature)
return;
while (nr-- != 0) {
make_page_readonly(va, feature);
__make_page_readonly(va);
va = (void*)((unsigned long)va + PAGE_SIZE);
}
}
@@ -161,7 +167,7 @@ void make_pages_writable(void *va, unsigned nr, unsigned int feature)
return;
while (nr-- != 0) {
make_page_writable(va, feature);
__make_page_writable(va);
va = (void*)((unsigned long)va + PAGE_SIZE);
}
}
@@ -208,7 +214,11 @@ static __init void *spp_getpage(void)
void *ptr;
if (after_bootmem)
ptr = (void *) get_zeroed_page(GFP_ATOMIC);
else
else if (start_pfn < table_end) {
ptr = __va(start_pfn << PAGE_SHIFT);
start_pfn++;
memset(ptr, 0, PAGE_SIZE);
} else
ptr = alloc_bootmem_pages(PAGE_SIZE);
if (!ptr || ((unsigned long)ptr & ~PAGE_MASK))
panic("set_pte_phys: cannot allocate page data %s\n", after_bootmem?"after bootmem":"");
@@ -299,36 +309,26 @@ static __init void set_pte_phys_ma(unsigned long vaddr,
pmd = (pmd_t *) spp_getpage();
make_page_readonly(pmd, XENFEAT_writable_page_tables);
set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE | _PAGE_USER));
if (pmd != pmd_offset(pud, 0)) {
printk("PAGETABLE BUG #01! %p <-> %p\n", pmd, pmd_offset(pud,0));
return;
}
}
pmd = pmd_offset(pud, vaddr);
if (pmd_none(*pmd)) {
pte = (pte_t *) spp_getpage();
make_page_readonly(pte, XENFEAT_writable_page_tables);
set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE | _PAGE_USER));
if (pte != pte_offset_kernel(pmd, 0)) {
printk("PAGETABLE BUG #02!\n");
return;
}
}
new_pte = pfn_pte_ma(phys >> PAGE_SHIFT, prot);
pte = pte_offset_kernel(pmd, vaddr);
/*
* Note that the pte page is already RO, thus we want to use
* xen_l1_entry_update(), not set_pte().
*/
xen_l1_entry_update(pte,
pfn_pte_ma(phys >> PAGE_SHIFT, prot));
pte = pte_offset_kernel(pmd, vaddr);
set_pte(pte, new_pte);
/*
* It's enough to flush this one mapping.
@@ -404,6 +404,7 @@ static inline void __set_pte(pte_t *dst, pte_t val)
static inline int make_readonly(unsigned long paddr)
{
extern char __vsyscall_0;
int readonly = 0;
/* Make new page tables read-only. */
@@ -420,25 +421,45 @@ static inline int make_readonly(unsigned long paddr)
/*
* No need for writable mapping of kernel image. This also ensures that
* page and descriptor tables embedded inside don't have writable
* mappings.
* mappings. Exclude the vsyscall area here, allowing alternative
* instruction patching to work.
*/
if ((paddr >= __pa_symbol(&_text)) && (paddr < __pa_symbol(&_end)))
if ((paddr >= __pa_symbol(&_text)) && (paddr < __pa_symbol(&_end))
&& !(paddr >= __pa_symbol(&__vsyscall_0)
&& paddr < __pa_symbol(&__vsyscall_0) + PAGE_SIZE))
readonly = 1;
return readonly;
}
#ifndef CONFIG_XEN
/* Must run before zap_low_mappings */
__init void *early_ioremap(unsigned long addr, unsigned long size)
{
return ioremap(addr, size);
unsigned long map = round_down(addr, LARGE_PAGE_SIZE);
/* actually usually some more */
if (size >= LARGE_PAGE_SIZE) {
printk("SMBIOS area too long %lu\n", size);
return NULL;
}
set_pmd(temp_mappings[0].pmd, __pmd(map | _KERNPG_TABLE | _PAGE_PSE));
map += LARGE_PAGE_SIZE;
set_pmd(temp_mappings[1].pmd, __pmd(map | _KERNPG_TABLE | _PAGE_PSE));
__flush_tlb();
return temp_mappings[0].address + (addr & (LARGE_PAGE_SIZE-1));
}
/* To avoid virtual aliases later */
__init void early_iounmap(void *addr, unsigned long size)
{
iounmap(addr);
if ((void *)round_down((unsigned long)addr, LARGE_PAGE_SIZE) != temp_mappings[0].address)
printk("early_iounmap: bad address %p\n", addr);
set_pmd(temp_mappings[0].pmd, __pmd(0));
set_pmd(temp_mappings[1].pmd, __pmd(0));
__flush_tlb();
}
#endif
static void __meminit
phys_pmd_init(pmd_t *pmd, unsigned long address, unsigned long end)
@@ -638,7 +659,7 @@ static void __init extend_init_mapping(unsigned long tables_space)
static void __init find_early_table_space(unsigned long end)
{
unsigned long puds, pmds, ptes, tables;
unsigned long puds, pmds, ptes, tables;
puds = (end + PUD_SIZE - 1) >> PUD_SHIFT;
pmds = (end + PMD_SIZE - 1) >> PMD_SHIFT;
@@ -658,12 +679,70 @@ static void __init find_early_table_space(unsigned long end)
(table_start << PAGE_SHIFT) + tables);
}
static void xen_finish_init_mapping(void)
{
unsigned long i, start, end;
/* Re-vector virtual addresses pointing into the initial
mapping to the just-established permanent ones. */
xen_start_info = __va(__pa(xen_start_info));
xen_start_info->pt_base = (unsigned long)
__va(__pa(xen_start_info->pt_base));
if (!xen_feature(XENFEAT_auto_translated_physmap)) {
phys_to_machine_mapping =
__va(__pa(xen_start_info->mfn_list));
xen_start_info->mfn_list = (unsigned long)
phys_to_machine_mapping;
}
if (xen_start_info->mod_start)
xen_start_info->mod_start = (unsigned long)
__va(__pa(xen_start_info->mod_start));
/* Destroy the Xen-created mappings beyond the kernel image as
* well as the temporary mappings created above. Prevents
* overlap with modules area (if init mapping is very big).
*/
start = PAGE_ALIGN((unsigned long)_end);
end = __START_KERNEL_map + (table_end << PAGE_SHIFT);
for (; start < end; start += PAGE_SIZE)
WARN_ON(HYPERVISOR_update_va_mapping(
start, __pte_ma(0), 0));
/* Allocate pte's for initial fixmaps from 'start_pfn' allocator. */
table_end = ~0UL;
/*
* Prefetch pte's for the bt_ioremap() area. It gets used before the
* boot-time allocator is online, so allocate-on-demand would fail.
*/
for (i = FIX_BTMAP_END; i <= FIX_BTMAP_BEGIN; i++)
__set_fixmap(i, 0, __pgprot(0));
/* Switch to the real shared_info page, and clear the dummy page. */
set_fixmap(FIX_SHARED_INFO, xen_start_info->shared_info);
HYPERVISOR_shared_info = (shared_info_t *)fix_to_virt(FIX_SHARED_INFO);
memset(empty_zero_page, 0, sizeof(empty_zero_page));
/* Set up mapping of lowest 1MB of physical memory. */
for (i = 0; i < NR_FIX_ISAMAPS; i++)
if (is_initial_xendomain())
set_fixmap(FIX_ISAMAP_BEGIN - i, i * PAGE_SIZE);
else
__set_fixmap(FIX_ISAMAP_BEGIN - i,
virt_to_mfn(empty_zero_page)
<< PAGE_SHIFT,
PAGE_KERNEL_RO);
/* Disable the 'start_pfn' allocator. */
table_end = start_pfn;
}
/* Setup the direct mapping of the physical memory at PAGE_OFFSET.
This runs before bootmem is initialized and gets pages directly from the
physical memory. To access them they are temporarily mapped. */
void __meminit init_memory_mapping(unsigned long start, unsigned long end)
{
unsigned long next;
unsigned long next;
Dprintk("init_memory_mapping\n");
@@ -702,31 +781,7 @@ void __meminit init_memory_mapping(unsigned long start, unsigned long end)
if (!after_bootmem) {
BUG_ON(start_pfn != table_end);
/* Re-vector virtual addresses pointing into the initial
mapping to the just-established permanent ones. */
xen_start_info = __va(__pa(xen_start_info));
xen_start_info->pt_base = (unsigned long)
__va(__pa(xen_start_info->pt_base));
if (!xen_feature(XENFEAT_auto_translated_physmap)) {
phys_to_machine_mapping =
__va(__pa(xen_start_info->mfn_list));
xen_start_info->mfn_list = (unsigned long)
phys_to_machine_mapping;
}
if (xen_start_info->mod_start)
xen_start_info->mod_start = (unsigned long)
__va(__pa(xen_start_info->mod_start));
/* Destroy the Xen-created mappings beyond the kernel image as
* well as the temporary mappings created above. Prevents
* overlap with modules area (if init mapping is very big).
*/
start = PAGE_ALIGN((unsigned long)_end);
end = __START_KERNEL_map + (table_end << PAGE_SHIFT);
for (; start < end; start += PAGE_SIZE)
WARN_ON(HYPERVISOR_update_va_mapping(
start, __pte_ma(0), 0));
xen_finish_init_mapping();
}
__flush_tlb_all();
@@ -805,7 +860,6 @@ size_zones(unsigned long *z, unsigned long *h,
void __init paging_init(void)
{
unsigned long zones[MAX_NR_ZONES], holes[MAX_NR_ZONES];
int i;
memory_present(0, 0, end_pfn);
sparse_init();
@@ -813,22 +867,7 @@ void __init paging_init(void)
free_area_init_node(0, NODE_DATA(0), zones,
__pa(PAGE_OFFSET) >> PAGE_SHIFT, holes);
/* Switch to the real shared_info page, and clear the
* dummy page. */
set_fixmap(FIX_SHARED_INFO, xen_start_info->shared_info);
HYPERVISOR_shared_info = (shared_info_t *)fix_to_virt(FIX_SHARED_INFO);
memset(empty_zero_page, 0, sizeof(empty_zero_page));
init_mm.context.pinned = 1;
/* Setup mapping of lower 1st MB */
for (i = 0; i < NR_FIX_ISAMAPS; i++)
if (is_initial_xendomain())
set_fixmap(FIX_ISAMAP_BEGIN - i, i * PAGE_SIZE);
else
__set_fixmap(FIX_ISAMAP_BEGIN - i,
virt_to_mfn(empty_zero_page) << PAGE_SHIFT,
PAGE_KERNEL_RO);
}
#endif
@@ -1028,11 +1067,6 @@ void __init mem_init(void)
void free_init_pages(char *what, unsigned long begin, unsigned long end)
{
#ifdef __DO_LATER__
/*
* Some pages can be pinned, but some are not. Unpinning such pages
* triggers BUG().
*/
unsigned long addr;
if (begin >= end)
@@ -1044,25 +1078,27 @@ void free_init_pages(char *what, unsigned long begin, unsigned long end)
init_page_count(virt_to_page(addr));
memset((void *)(addr & ~(PAGE_SIZE-1)),
POISON_FREE_INITMEM, PAGE_SIZE);
make_page_writable(
__va(__pa(addr)), XENFEAT_writable_page_tables);
/*
* Make pages from __PAGE_OFFSET address as well
*/
make_page_writable(
(void *)addr, XENFEAT_writable_page_tables);
if (addr >= __START_KERNEL_map) {
/* make_readonly() reports all kernel addresses. */
__make_page_writable(__va(__pa(addr)));
if (HYPERVISOR_update_va_mapping(addr, __pte(0), 0)) {
pgd_t *pgd = pgd_offset_k(addr);
pud_t *pud = pud_offset(pgd, addr);
pmd_t *pmd = pmd_offset(pud, addr);
pte_t *pte = pte_offset_kernel(pmd, addr);
xen_l1_entry_update(pte, __pte(0)); /* fallback */
}
}
free_page(addr);
totalram_pages++;
}
#endif
}
void free_initmem(void)
{
#ifdef __DO_LATER__
memset(__initdata_begin, POISON_FREE_INITDATA,
__initdata_end - __initdata_begin);
#endif
free_init_pages("unused kernel memory",
(unsigned long)(&__init_begin),
(unsigned long)(&__init_end));
@@ -1125,7 +1161,7 @@ int kern_addr_valid(unsigned long addr)
if (pgd_none(*pgd))
return 0;
pud = pud_offset_k(pgd, addr);
pud = pud_offset(pgd, addr);
if (pud_none(*pud))
return 0;
@@ -24,10 +24,13 @@ static inline void mm_walk_set_prot(void *pt, pgprot_t flags)
{
struct page *page = virt_to_page(pt);
unsigned long pfn = page_to_pfn(page);
int rc;
BUG_ON(HYPERVISOR_update_va_mapping(
(unsigned long)__va(pfn << PAGE_SHIFT),
pfn_pte(pfn, flags), 0));
rc = HYPERVISOR_update_va_mapping(
(unsigned long)__va(pfn << PAGE_SHIFT),
pfn_pte(pfn, flags), 0);
if (rc)
BUG();
}
static void mm_walk(struct mm_struct *mm, pgprot_t flags)
+1 -1
View File
@@ -787,7 +787,7 @@ static const struct file_operations mem_fops = {
.open = open_mem,
};
#else
extern struct file_operations mem_fops;
extern const struct file_operations mem_fops;
#endif
static const struct file_operations kmem_fops = {
+85 -95
View File
@@ -113,14 +113,13 @@ void __exit tpmif_exit(void);
static inline int
tx_buffer_copy(struct tx_buffer *txb, const u8 * src, int len,
tx_buffer_copy(struct tx_buffer *txb, const u8 *src, int len,
int isuserbuffer)
{
int copied = len;
if (len > txb->size) {
if (len > txb->size)
copied = txb->size;
}
if (isuserbuffer) {
if (copy_from_user(txb->data, src, copied))
return -EFAULT;
@@ -133,18 +132,20 @@ tx_buffer_copy(struct tx_buffer *txb, const u8 * src, int len,
static inline struct tx_buffer *tx_buffer_alloc(void)
{
struct tx_buffer *txb = kzalloc(sizeof (struct tx_buffer),
GFP_KERNEL);
struct tx_buffer *txb;
if (txb) {
txb->len = 0;
txb->size = PAGE_SIZE;
txb->data = (unsigned char *)__get_free_page(GFP_KERNEL);
if (txb->data == NULL) {
kfree(txb);
txb = NULL;
}
txb = kzalloc(sizeof(struct tx_buffer), GFP_KERNEL);
if (!txb)
return NULL;
txb->len = 0;
txb->size = PAGE_SIZE;
txb->data = (unsigned char *)__get_free_page(GFP_KERNEL);
if (txb->data == NULL) {
kfree(txb);
txb = NULL;
}
return txb;
}
@@ -160,37 +161,41 @@ static inline void tx_buffer_free(struct tx_buffer *txb)
/**************************************************************
Utility function for the tpm_private structure
**************************************************************/
static inline void tpm_private_init(struct tpm_private *tp)
static void tpm_private_init(struct tpm_private *tp)
{
spin_lock_init(&tp->tx_lock);
init_waitqueue_head(&tp->wait_q);
atomic_set(&tp->refcnt, 1);
}
static inline void tpm_private_put(void)
static void tpm_private_put(void)
{
if ( atomic_dec_and_test(&my_priv->refcnt)) {
tpmif_free_tx_buffers(my_priv);
kfree(my_priv);
my_priv = NULL;
}
if (!atomic_dec_and_test(&my_priv->refcnt))
return;
tpmif_free_tx_buffers(my_priv);
kfree(my_priv);
my_priv = NULL;
}
static struct tpm_private *tpm_private_get(void)
{
int err;
if (!my_priv) {
my_priv = kzalloc(sizeof(struct tpm_private), GFP_KERNEL);
if (my_priv) {
tpm_private_init(my_priv);
err = tpmif_allocate_tx_buffers(my_priv);
if (err < 0) {
tpm_private_put();
}
}
} else {
if (my_priv) {
atomic_inc(&my_priv->refcnt);
return my_priv;
}
my_priv = kzalloc(sizeof(struct tpm_private), GFP_KERNEL);
if (!my_priv)
return NULL;
tpm_private_init(my_priv);
err = tpmif_allocate_tx_buffers(my_priv);
if (err < 0)
tpm_private_put();
return my_priv;
}
@@ -379,10 +384,8 @@ static int tpmfront_probe(struct xenbus_device *dev,
return -ENOMEM;
tp->chip = init_vtpm(&dev->dev, &tvd, tp);
if (IS_ERR(tp->chip)) {
if (IS_ERR(tp->chip))
return PTR_ERR(tp->chip);
}
err = xenbus_scanf(XBT_NIL, dev->nodename,
"handle", "%i", &handle);
@@ -401,6 +404,7 @@ static int tpmfront_probe(struct xenbus_device *dev,
tpm_private_put();
return err;
}
return 0;
}
@@ -417,32 +421,36 @@ static int tpmfront_suspend(struct xenbus_device *dev)
{
struct tpm_private *tp = tpm_private_from_dev(&dev->dev);
u32 ctr;
/* lock, so no app can send */
/* Take the lock, preventing any application from sending. */
mutex_lock(&suspend_lock);
tp->is_suspended = 1;
for (ctr = 0; atomic_read(&tp->tx_busy) && ctr <= 300; ctr++) {
for (ctr = 0; atomic_read(&tp->tx_busy); ctr++) {
if ((ctr % 10) == 0)
printk("TPM-FE [INFO]: Waiting for outstanding "
"request.\n");
/*
* Wait for a request to be responded to.
*/
/* Wait for a request to be responded to. */
interruptible_sleep_on_timeout(&tp->wait_q, 100);
}
xenbus_switch_state(dev, XenbusStateClosing);
if (atomic_read(&tp->tx_busy)) {
/*
* A temporary work-around.
*/
printk("TPM-FE [WARNING]: Resetting busy flag.");
atomic_set(&tp->tx_busy, 0);
}
return 0;
}
static int tpmfront_suspend_finish(struct tpm_private *tp)
{
tp->is_suspended = 0;
/* Allow applications to send again. */
mutex_unlock(&suspend_lock);
return 0;
}
static int tpmfront_suspend_cancel(struct xenbus_device *dev)
{
struct tpm_private *tp = tpm_private_from_dev(&dev->dev);
return tpmfront_suspend_finish(tp);
}
static int tpmfront_resume(struct xenbus_device *dev)
{
struct tpm_private *tp = tpm_private_from_dev(&dev->dev);
@@ -484,6 +492,7 @@ static struct xenbus_driver tpmfront = {
.resume = tpmfront_resume,
.otherend_changed = backend_changed,
.suspend = tpmfront_suspend,
.suspend_cancel = tpmfront_suspend_cancel,
};
static void __init init_tpm_xenbus(void)
@@ -514,9 +523,8 @@ static void tpmif_free_tx_buffers(struct tpm_private *tp)
{
unsigned int i;
for (i = 0; i < TPMIF_TX_RING_SIZE; i++) {
for (i = 0; i < TPMIF_TX_RING_SIZE; i++)
tx_buffer_free(tp->tx_buffers[i]);
}
}
static void tpmif_rx_action(unsigned long priv)
@@ -536,9 +544,8 @@ static void tpmif_rx_action(unsigned long priv)
received = tx->size;
buffer = kmalloc(received, GFP_ATOMIC);
if (NULL == buffer) {
if (!buffer)
goto exit;
}
for (i = 0; i < TPMIF_TX_RING_SIZE && offset < received; i++) {
struct tx_buffer *txb = tp->tx_buffers[i];
@@ -547,9 +554,8 @@ static void tpmif_rx_action(unsigned long priv)
tx = &tp->tx->ring[i].req;
tocopy = tx->size;
if (tocopy > PAGE_SIZE) {
if (tocopy > PAGE_SIZE)
tocopy = PAGE_SIZE;
}
memcpy(&buffer[offset], txb->data, tocopy);
@@ -607,12 +613,13 @@ static int tpm_xmit(struct tpm_private *tp,
struct tx_buffer *txb = tp->tx_buffers[i];
int copied;
if (NULL == txb) {
if (!txb) {
DPRINTK("txb (i=%d) is NULL. buffers initilized?\n"
"Not transmitting anything!\n", i);
spin_unlock_irq(&tp->tx_lock);
return -EFAULT;
}
copied = tx_buffer_copy(txb, &buf[offset], count,
isuserbuffer);
if (copied < 0) {
@@ -624,25 +631,26 @@ static int tpm_xmit(struct tpm_private *tp,
offset += copied;
tx = &tp->tx->ring[i].req;
tx->addr = virt_to_machine(txb->data);
tx->size = txb->len;
DPRINTK("First 4 characters sent by TPM-FE are 0x%02x 0x%02x 0x%02x 0x%02x\n",
DPRINTK("First 4 characters sent by TPM-FE are "
"0x%02x 0x%02x 0x%02x 0x%02x\n",
txb->data[0],txb->data[1],txb->data[2],txb->data[3]);
/* get the granttable reference for this page */
/* Get the granttable reference for this page. */
tx->ref = gnttab_claim_grant_reference(&gref_head);
if (-ENOSPC == tx->ref) {
if (tx->ref == -ENOSPC) {
spin_unlock_irq(&tp->tx_lock);
DPRINTK(" Grant table claim reference failed in func:%s line:%d file:%s\n", __FUNCTION__, __LINE__, __FILE__);
DPRINTK("Grant table claim reference failed in "
"func:%s line:%d file:%s\n",
__FUNCTION__, __LINE__, __FILE__);
return -ENOSPC;
}
gnttab_grant_foreign_access_ref( tx->ref,
tp->backend_id,
(tx->addr >> PAGE_SHIFT),
0 /*RW*/);
gnttab_grant_foreign_access_ref(tx->ref,
tp->backend_id,
virt_to_mfn(txb->data),
0 /*RW*/);
wmb();
}
@@ -660,15 +668,10 @@ static int tpm_xmit(struct tpm_private *tp,
static void tpmif_notify_upperlayer(struct tpm_private *tp)
{
/*
* Notify upper layer about the state of the connection
* to the BE.
*/
if (tp->is_connected) {
vtpm_vd_status(tp->chip, TPM_VD_STATUS_CONNECTED);
} else {
vtpm_vd_status(tp->chip, TPM_VD_STATUS_DISCONNECTED);
}
/* Notify upper layer about the state of the connection to the BE. */
vtpm_vd_status(tp->chip, (tp->is_connected
? TPM_VD_STATUS_CONNECTED
: TPM_VD_STATUS_DISCONNECTED));
}
@@ -679,20 +682,16 @@ static void tpmif_set_connected_state(struct tpm_private *tp, u8 is_connected)
* should disconnect - assumption is that we will resume
* The mutex keeps apps from sending.
*/
if (is_connected == 0 && tp->is_suspended == 1) {
if (is_connected == 0 && tp->is_suspended == 1)
return;
}
/*
* Unlock the mutex if we are connected again
* after being suspended - now resuming.
* This also removes the suspend state.
*/
if (is_connected == 1 && tp->is_suspended == 1) {
tp->is_suspended = 0;
/* unlock, so apps can resume sending */
mutex_unlock(&suspend_lock);
}
if (is_connected == 1 && tp->is_suspended == 1)
tpmfront_suspend_finish(tp);
if (is_connected != tp->is_connected) {
tp->is_connected = is_connected;
@@ -710,33 +709,24 @@ static void tpmif_set_connected_state(struct tpm_private *tp, u8 is_connected)
static int __init tpmif_init(void)
{
long rc = 0;
struct tpm_private *tp;
if (is_initial_xendomain())
return -EPERM;
tp = tpm_private_get();
if (!tp) {
rc = -ENOMEM;
goto failexit;
}
if (!tp)
return -ENOMEM;
IPRINTK("Initialising the vTPM driver.\n");
if ( gnttab_alloc_grant_references ( TPMIF_TX_RING_SIZE,
&gref_head ) < 0) {
rc = -EFAULT;
goto gnttab_alloc_failed;
if (gnttab_alloc_grant_references(TPMIF_TX_RING_SIZE,
&gref_head) < 0) {
tpm_private_put();
return -EFAULT;
}
init_tpm_xenbus();
return 0;
gnttab_alloc_failed:
tpm_private_put();
failexit:
return (int)rc;
}
@@ -32,7 +32,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/sched.h>
@@ -28,7 +28,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/capability.h>
#include <linux/stat.h>
#include <linux/sysdev.h>
@@ -27,7 +27,6 @@
#ifndef __BLKIF__BACKEND__COMMON_H__
#define __BLKIF__BACKEND__COMMON_H__
#include <linux/config.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/interrupt.h>
@@ -35,7 +35,6 @@
#ifndef __XEN_DRIVERS_BLOCK_H__
#define __XEN_DRIVERS_BLOCK_H__
#include <linux/config.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/kernel.h>
@@ -335,7 +335,7 @@ static int blktap_ioctl(struct inode *inode, struct file *filp,
unsigned int cmd, unsigned long arg);
static unsigned int blktap_poll(struct file *file, poll_table *wait);
static struct file_operations blktap_fops = {
static const struct file_operations blktap_fops = {
.owner = THIS_MODULE,
.poll = blktap_poll,
.ioctl = blktap_ioctl,
@@ -27,7 +27,6 @@
#ifndef __BLKIF__BACKEND__COMMON_H__
#define __BLKIF__BACKEND__COMMON_H__
#include <linux/config.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/interrupt.h>
+1 -2
View File
@@ -8,7 +8,6 @@
* Shared /dev/zero mmaping support, Feb 2000, Kanoj Sarcar <kanoj@sgi.com>
*/
#include <linux/config.h>
#include <linux/mm.h>
#include <linux/miscdevice.h>
#include <linux/slab.h>
@@ -195,7 +194,7 @@ static int open_mem(struct inode * inode, struct file * filp)
return capable(CAP_SYS_RAWIO) ? 0 : -EPERM;
}
struct file_operations mem_fops = {
const struct file_operations mem_fops = {
.llseek = memory_lseek,
.read = read_mem,
.write = write_mem,
@@ -30,7 +30,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/errno.h>
@@ -1,4 +1,3 @@
#include <linux/config.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/sched.h>
@@ -122,29 +121,19 @@ arch_initcall(setup_vcpu_hotplug_event);
int smp_suspend(void)
{
int i, err;
int cpu, err;
lock_cpu_hotplug();
/*
* Take all other CPUs offline. We hold the hotplug mutex to
* avoid other processes bringing up CPUs under our feet.
*/
while (num_online_cpus() > 1) {
unlock_cpu_hotplug();
for_each_online_cpu(i) {
if (i == 0)
continue;
err = cpu_down(i);
if (err) {
printk(KERN_CRIT "Failed to take all CPUs "
"down: %d.\n", err);
for_each_possible_cpu(i)
vcpu_hotplug(i);
return err;
}
for_each_online_cpu(cpu) {
if (cpu == 0)
continue;
err = cpu_down(cpu);
if (err) {
printk(KERN_CRIT "Failed to take all CPUs "
"down: %d.\n", err);
for_each_possible_cpu(cpu)
vcpu_hotplug(cpu);
return err;
}
lock_cpu_hotplug();
}
return 0;
@@ -154,11 +143,6 @@ void smp_resume(void)
{
int cpu;
for_each_possible_cpu(cpu)
cpu_initialize_context(cpu);
unlock_cpu_hotplug();
for_each_possible_cpu(cpu)
vcpu_hotplug(cpu);
}
+62 -56
View File
@@ -30,7 +30,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/module.h>
#include <linux/irq.h>
#include <linux/interrupt.h>
@@ -889,11 +888,67 @@ void unmask_evtchn(int port)
}
EXPORT_SYMBOL_GPL(unmask_evtchn);
void irq_resume(void)
static void restore_cpu_virqs(int cpu)
{
struct evtchn_bind_virq bind_virq;
struct evtchn_bind_ipi bind_ipi;
int cpu, pirq, virq, ipi, irq, evtchn;
int virq, irq, evtchn;
for (virq = 0; virq < NR_VIRQS; virq++) {
if ((irq = per_cpu(virq_to_irq, cpu)[virq]) == -1)
continue;
BUG_ON(irq_info[irq] != mk_irq_info(IRQT_VIRQ, virq, 0));
/* Get a new binding from Xen. */
bind_virq.virq = virq;
bind_virq.vcpu = cpu;
if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_virq,
&bind_virq) != 0)
BUG();
evtchn = bind_virq.port;
/* Record the new mapping. */
evtchn_to_irq[evtchn] = irq;
irq_info[irq] = mk_irq_info(IRQT_VIRQ, virq, evtchn);
bind_evtchn_to_cpu(evtchn, cpu);
/* Ready for use. */
unmask_evtchn(evtchn);
}
}
static void restore_cpu_ipis(int cpu)
{
struct evtchn_bind_ipi bind_ipi;
int ipi, irq, evtchn;
for (ipi = 0; ipi < NR_IPIS; ipi++) {
if ((irq = per_cpu(ipi_to_irq, cpu)[ipi]) == -1)
continue;
BUG_ON(irq_info[irq] != mk_irq_info(IRQT_IPI, ipi, 0));
/* Get a new binding from Xen. */
bind_ipi.vcpu = cpu;
if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_ipi,
&bind_ipi) != 0)
BUG();
evtchn = bind_ipi.port;
/* Record the new mapping. */
evtchn_to_irq[evtchn] = irq;
irq_info[irq] = mk_irq_info(IRQT_IPI, ipi, evtchn);
bind_evtchn_to_cpu(evtchn, cpu);
/* Ready for use. */
unmask_evtchn(evtchn);
}
}
void irq_resume(void)
{
int cpu, pirq, irq, evtchn;
init_evtchn_cpu_bindings();
@@ -905,66 +960,17 @@ void irq_resume(void)
for (pirq = 0; pirq < NR_PIRQS; pirq++)
BUG_ON(irq_info[pirq_to_irq(pirq)] != IRQ_UNBOUND);
/* Secondary CPUs must have no VIRQ or IPI bindings. */
for_each_possible_cpu(cpu) {
if (cpu == 0)
continue;
for (virq = 0; virq < NR_VIRQS; virq++)
BUG_ON(per_cpu(virq_to_irq, cpu)[virq] != -1);
for (ipi = 0; ipi < NR_IPIS; ipi++)
BUG_ON(per_cpu(ipi_to_irq, cpu)[ipi] != -1);
}
/* No IRQ <-> event-channel mappings. */
for (irq = 0; irq < NR_IRQS; irq++)
irq_info[irq] &= ~0xFFFF; /* zap event-channel binding */
for (evtchn = 0; evtchn < NR_EVENT_CHANNELS; evtchn++)
evtchn_to_irq[evtchn] = -1;
/* Primary CPU: rebind VIRQs automatically. */
for (virq = 0; virq < NR_VIRQS; virq++) {
if ((irq = per_cpu(virq_to_irq, 0)[virq]) == -1)
continue;
BUG_ON(irq_info[irq] != mk_irq_info(IRQT_VIRQ, virq, 0));
/* Get a new binding from Xen. */
bind_virq.virq = virq;
bind_virq.vcpu = 0;
if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_virq,
&bind_virq) != 0)
BUG();
evtchn = bind_virq.port;
/* Record the new mapping. */
evtchn_to_irq[evtchn] = irq;
irq_info[irq] = mk_irq_info(IRQT_VIRQ, virq, evtchn);
/* Ready for use. */
unmask_evtchn(evtchn);
for_each_possible_cpu(cpu) {
restore_cpu_virqs(cpu);
restore_cpu_ipis(cpu);
}
/* Primary CPU: rebind IPIs automatically. */
for (ipi = 0; ipi < NR_IPIS; ipi++) {
if ((irq = per_cpu(ipi_to_irq, 0)[ipi]) == -1)
continue;
BUG_ON(irq_info[irq] != mk_irq_info(IRQT_IPI, ipi, 0));
/* Get a new binding from Xen. */
bind_ipi.vcpu = 0;
if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_ipi,
&bind_ipi) != 0)
BUG();
evtchn = bind_ipi.port;
/* Record the new mapping. */
evtchn_to_irq[evtchn] = irq;
irq_info[irq] = mk_irq_info(IRQT_IPI, ipi, evtchn);
/* Ready for use. */
unmask_evtchn(evtchn);
}
}
void __init xen_init_IRQ(void)
@@ -31,7 +31,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/module.h>
#include <linux/sched.h>
#include <linux/mm.h>
@@ -518,25 +517,26 @@ int gnttab_suspend(void)
static int gnttab_map(unsigned int start_idx, unsigned int end_idx)
{
struct xen_add_to_physmap xatp;
unsigned int i;
unsigned int i = end_idx;
/* Loop backwards, so that the first hypercall has the largest index,
* ensuring that the table will grow only once.
*/
for (i = end_idx; i >= start_idx; i--) {
do {
xatp.domid = DOMID_SELF;
xatp.idx = i;
xatp.space = XENMAPSPACE_grant_table;
xatp.gpfn = (resume_frames >> PAGE_SHIFT) + i;
if (HYPERVISOR_memory_op(XENMEM_add_to_physmap, &xatp))
BUG();
}
} while (i-- > start_idx);
return 0;
}
int gnttab_resume(void)
{
struct xen_add_to_physmap xatp;
unsigned int i, max_nr_gframes, nr_gframes;
unsigned int max_nr_gframes, nr_gframes;
nr_gframes = nr_grant_frames;
max_nr_gframes = max_nr_grant_frames();
@@ -7,7 +7,6 @@
* published by the Free Software Foundation.
*/
#include <linux/config.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kobject.h>
@@ -1,4 +1,3 @@
#define __KERNEL_SYSCALLS__
#include <linux/version.h>
#include <linux/kernel.h>
#include <linux/mm.h>
@@ -7,6 +6,7 @@
#include <linux/reboot.h>
#include <linux/sysrq.h>
#include <linux/stringify.h>
#include <linux/stop_machine.h>
#include <asm/irq.h>
#include <asm/mmu_context.h>
#include <xen/evtchn.h>
@@ -18,6 +18,7 @@
#include <xen/gnttab.h>
#include <xen/xencons.h>
#include <xen/cpu_hotplug.h>
#include <xen/interface/vcpu.h>
#if defined(__i386__) || defined(__x86_64__)
@@ -58,27 +59,11 @@ EXPORT_SYMBOL(machine_restart);
EXPORT_SYMBOL(machine_halt);
EXPORT_SYMBOL(machine_power_off);
/* Ensure we run on the idle task page tables so that we will
switch page tables before running user space. This is needed
on architectures with separate kernel and user page tables
because the user page table pointer is not saved/restored. */
static void switch_idle_mm(void)
{
struct mm_struct *mm = current->active_mm;
if (mm == &init_mm)
return;
atomic_inc(&init_mm.mm_count);
switch_mm(mm, &init_mm, current);
current->active_mm = &init_mm;
mmdrop(mm);
}
static void pre_suspend(void)
{
HYPERVISOR_shared_info = (shared_info_t *)empty_zero_page;
clear_fixmap(FIX_SHARED_INFO);
HYPERVISOR_update_va_mapping(fix_to_virt(FIX_SHARED_INFO),
__pte_ma(0), 0);
xen_start_info->store_mfn = mfn_to_pfn(xen_start_info->store_mfn);
xen_start_info->console.domU.mfn =
@@ -88,6 +73,7 @@ static void pre_suspend(void)
static void post_suspend(int suspend_cancelled)
{
int i, j, k, fpp;
unsigned long shinfo_mfn;
extern unsigned long max_pfn;
extern unsigned long *pfn_to_mfn_frame_list_list;
extern unsigned long *pfn_to_mfn_frame_list[];
@@ -97,10 +83,15 @@ static void post_suspend(int suspend_cancelled)
pfn_to_mfn(xen_start_info->store_mfn);
xen_start_info->console.domU.mfn =
pfn_to_mfn(xen_start_info->console.domU.mfn);
} else {
#ifdef CONFIG_SMP
cpu_initialized_map = cpumask_of_cpu(0);
#endif
}
set_fixmap(FIX_SHARED_INFO, xen_start_info->shared_info);
shinfo_mfn = xen_start_info->shared_info >> PAGE_SHIFT;
HYPERVISOR_update_va_mapping(fix_to_virt(FIX_SHARED_INFO),
pfn_pte_ma(shinfo_mfn, PAGE_KERNEL), 0);
HYPERVISOR_shared_info = (shared_info_t *)fix_to_virt(FIX_SHARED_INFO);
memset(empty_zero_page, 0, PAGE_SIZE);
@@ -130,12 +121,87 @@ static void post_suspend(int suspend_cancelled)
#endif
int __xen_suspend(void)
static int take_machine_down(void *p_fast_suspend)
{
int fast_suspend = *(int *)p_fast_suspend;
int suspend_cancelled, err, cpu;
extern void time_resume(void);
if (fast_suspend) {
preempt_disable();
} else {
for (;;) {
err = smp_suspend();
if (err)
return err;
xenbus_suspend();
preempt_disable();
if (num_online_cpus() == 1)
break;
preempt_enable();
xenbus_suspend_cancel();
}
}
mm_pin_all();
local_irq_disable();
preempt_enable();
gnttab_suspend();
pre_suspend();
/*
* This hypercall returns 1 if suspend was cancelled or the domain was
* merely checkpointed, and 0 if it is resuming in a new domain.
*/
suspend_cancelled = HYPERVISOR_suspend(virt_to_mfn(xen_start_info));
post_suspend(suspend_cancelled);
gnttab_resume();
if (!suspend_cancelled) {
irq_resume();
#ifdef __x86_64__
/*
* Older versions of Xen do not save/restore the user %cr3.
* We do it here just in case, but there's no need if we are
* in fast-suspend mode as that implies a new enough Xen.
*/
if (!fast_suspend) {
struct mmuext_op op;
op.cmd = MMUEXT_NEW_USER_BASEPTR;
op.arg1.mfn = pfn_to_mfn(__pa(__user_pgd(
current->active_mm->pgd)) >> PAGE_SHIFT);
if (HYPERVISOR_mmuext_op(&op, 1, NULL, DOMID_SELF))
BUG();
}
#endif
}
time_resume();
local_irq_enable();
if (fast_suspend && !suspend_cancelled) {
/*
* In fast-suspend mode the APs may not be brought back online
* when we resume. In that case we do it here.
*/
for_each_online_cpu(cpu) {
if (cpu == 0)
continue;
cpu_set_initialized(cpu);
err = HYPERVISOR_vcpu_op(VCPUOP_up, cpu, NULL);
BUG_ON(err);
}
}
return suspend_cancelled;
}
int __xen_suspend(int fast_suspend)
{
int err, suspend_cancelled;
extern void time_resume(void);
BUG_ON(smp_processor_id() != 0);
BUG_ON(in_interrupt());
@@ -147,41 +213,21 @@ int __xen_suspend(void)
}
#endif
err = smp_suspend();
if (err)
/* If we are definitely UP then 'slow mode' is actually faster. */
if (num_possible_cpus() == 1)
fast_suspend = 0;
if (fast_suspend) {
xenbus_suspend();
err = stop_machine_run(take_machine_down, &fast_suspend, 0);
} else {
err = take_machine_down(&fast_suspend);
}
if (err < 0)
return err;
xenbus_suspend();
preempt_disable();
mm_pin_all();
local_irq_disable();
preempt_enable();
gnttab_suspend();
pre_suspend();
/*
* This hypercall returns 1 if suspend was cancelled or the domain was
* merely checkpointed, and 0 if it is resuming in a new domain.
*/
suspend_cancelled = HYPERVISOR_suspend(virt_to_mfn(xen_start_info));
post_suspend(suspend_cancelled);
gnttab_resume();
if (!suspend_cancelled)
irq_resume();
time_resume();
switch_idle_mm();
local_irq_enable();
suspend_cancelled = err;
if (!suspend_cancelled) {
xencons_resume();
xenbus_resume();
@@ -189,7 +235,8 @@ int __xen_suspend(void)
xenbus_suspend_cancel();
}
smp_resume();
if (!fast_suspend)
smp_resume();
return err;
return 0;
}
+14 -4
View File
@@ -24,13 +24,16 @@ MODULE_LICENSE("Dual BSD/GPL");
/* Ignore multiple shutdown requests. */
static int shutting_down = SHUTDOWN_INVALID;
/* Can we leave APs online when we suspend? */
static int fast_suspend;
static void __shutdown_handler(void *unused);
static DECLARE_WORK(shutdown_work, __shutdown_handler, NULL);
#ifdef CONFIG_XEN
int __xen_suspend(void);
int __xen_suspend(int fast_suspend);
#else
#define __xen_suspend() (void)0
#define __xen_suspend(fast_suspend) 0
#endif
static int shutdown_process(void *__unused)
@@ -44,7 +47,8 @@ static int shutdown_process(void *__unused)
if ((shutting_down == SHUTDOWN_POWEROFF) ||
(shutting_down == SHUTDOWN_HALT)) {
if (call_usermodehelper("/sbin/poweroff", poweroff_argv, envp, 0) < 0) {
if (call_usermodehelper("/sbin/poweroff", poweroff_argv,
envp, 0) < 0) {
#ifdef CONFIG_XEN
sys_reboot(LINUX_REBOOT_MAGIC1,
LINUX_REBOOT_MAGIC2,
@@ -61,7 +65,9 @@ static int shutdown_process(void *__unused)
static int xen_suspend(void *__unused)
{
__xen_suspend();
int err = __xen_suspend(fast_suspend);
if (err)
printk(KERN_ERR "Xen suspend failed (%d)\n", err);
shutting_down = SHUTDOWN_INVALID;
return 0;
}
@@ -193,6 +199,10 @@ static int setup_shutdown_watcher(struct notifier_block *notifier,
{
int err;
xenbus_scanf(XBT_NIL, "control",
"platform-feature-multiprocessor-suspend",
"%d", &fast_suspend);
err = register_xenbus_watch(&shutdown_watch);
if (err)
printk(KERN_ERR "Failed to set shutdown watcher\n");
@@ -6,7 +6,6 @@
*/
#include <linux/module.h>
#include <linux/config.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/mm.h>
@@ -48,7 +47,7 @@ cpumask_t cpu_online_map;
EXPORT_SYMBOL(cpu_online_map);
cpumask_t cpu_possible_map;
EXPORT_SYMBOL(cpu_possible_map);
static cpumask_t cpu_initialized_map;
cpumask_t cpu_initialized_map;
struct cpuinfo_x86 cpu_data[NR_CPUS] __cacheline_aligned;
EXPORT_SYMBOL(cpu_data);
@@ -186,7 +185,7 @@ static void cpu_bringup_and_idle(void)
cpu_idle();
}
void cpu_initialize_context(unsigned int cpu)
static void cpu_initialize_context(unsigned int cpu)
{
vcpu_guest_context_t ctxt;
struct task_struct *idle = idle_task(cpu);
@@ -196,7 +195,7 @@ void cpu_initialize_context(unsigned int cpu)
struct Xgt_desc_struct *gdt_descr = &per_cpu(cpu_gdt_descr, cpu);
#endif
if (cpu == 0)
if (cpu_test_and_set(cpu, cpu_initialized_map))
return;
memset(&ctxt, 0, sizeof(ctxt));
@@ -418,10 +417,7 @@ int __devinit __cpu_up(unsigned int cpu)
if (rc)
return rc;
if (!cpu_isset(cpu, cpu_initialized_map)) {
cpu_set(cpu, cpu_initialized_map);
cpu_initialize_context(cpu);
}
cpu_initialize_context(cpu);
if (num_online_cpus() == 1)
alternatives_smp_switch(1);
@@ -1,5 +1,4 @@
#include <linux/config.h>
#include <linux/proc_fs.h>
#include <xen/xen_proc.h>
@@ -7,7 +7,6 @@
* published by the Free Software Foundation.
*/
#include <linux/config.h>
#include <linux/err.h>
#include <linux/kernel.h>
#include <linux/module.h>
@@ -31,7 +31,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/sched.h>
@@ -407,7 +406,7 @@ static int evtchn_release(struct inode *inode, struct file *filp)
return 0;
}
static struct file_operations evtchn_fops = {
static const struct file_operations evtchn_fops = {
.owner = THIS_MODULE,
.read = evtchn_read,
.write = evtchn_write,
@@ -29,7 +29,6 @@
#ifndef __NETIF__BACKEND__COMMON_H__
#define __NETIF__BACKEND__COMMON_H__
#include <linux/config.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/interrupt.h>
@@ -100,9 +99,21 @@ typedef struct netif_st {
struct net_device *dev;
struct net_device_stats stats;
unsigned int carrier;
wait_queue_head_t waiting_to_free;
} netif_t;
/*
* Implement our own carrier flag: the network stack's version causes delays
* when the carrier is re-enabled (in particular, dev_activate() may not
* immediately be called, which can cause packet loss; also the etherbridge
* can be rather lazy in activating its port).
*/
#define netback_carrier_on(netif) ((netif)->carrier = 1)
#define netback_carrier_off(netif) ((netif)->carrier = 0)
#define netback_carrier_ok(netif) ((netif)->carrier)
#define NET_TX_RING_SIZE __RING_SIZE((netif_tx_sring_t *)0, PAGE_SIZE)
#define NET_RX_RING_SIZE __RING_SIZE((netif_rx_sring_t *)0, PAGE_SIZE)
@@ -121,7 +132,8 @@ int netif_map(netif_t *netif, unsigned long tx_ring_ref,
void netif_xenbus_init(void);
#define netif_schedulable(dev) (netif_running(dev) && netif_carrier_ok(dev))
#define netif_schedulable(netif) \
(netif_running((netif)->dev) && netback_carrier_ok(netif))
void netif_schedule_work(netif_t *netif);
void netif_deschedule_work(netif_t *netif);
@@ -66,16 +66,19 @@ static void __netif_down(netif_t *netif)
static int net_open(struct net_device *dev)
{
netif_t *netif = netdev_priv(dev);
if (netif_carrier_ok(dev))
if (netback_carrier_ok(netif)) {
__netif_up(netif);
netif_start_queue(dev);
}
return 0;
}
static int net_close(struct net_device *dev)
{
netif_t *netif = netdev_priv(dev);
if (netif_carrier_ok(dev))
if (netback_carrier_ok(netif))
__netif_down(netif);
netif_stop_queue(dev);
return 0;
}
@@ -138,8 +141,6 @@ netif_t *netif_alloc(domid_t domid, unsigned int handle)
return ERR_PTR(-ENOMEM);
}
netif_carrier_off(dev);
netif = netdev_priv(dev);
memset(netif, 0, sizeof(*netif));
netif->domid = domid;
@@ -148,6 +149,8 @@ netif_t *netif_alloc(domid_t domid, unsigned int handle)
init_waitqueue_head(&netif->waiting_to_free);
netif->dev = dev;
netback_carrier_off(netif);
netif->credit_bytes = netif->remaining_credit = ~0UL;
netif->credit_usec = 0UL;
init_timer(&netif->credit_timeout);
@@ -285,7 +288,7 @@ int netif_map(netif_t *netif, unsigned long tx_ring_ref,
netif_get(netif);
rtnl_lock();
netif_carrier_on(netif->dev);
netback_carrier_on(netif);
if (netif_running(netif->dev))
__netif_up(netif);
rtnl_unlock();
@@ -302,9 +305,10 @@ err_rx:
void netif_disconnect(netif_t *netif)
{
if (netif_carrier_ok(netif->dev)) {
if (netback_carrier_ok(netif)) {
rtnl_lock();
netif_carrier_off(netif->dev);
netback_carrier_off(netif);
netif_carrier_off(netif->dev); /* discard queued packets */
if (netif_running(netif->dev))
__netif_down(netif);
rtnl_unlock();
@@ -45,7 +45,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/inetdevice.h>
@@ -38,7 +38,10 @@
#include <xen/balloon.h>
#include <xen/interface/memory.h>
/*#define NETBE_DEBUG_INTERRUPT*/
/*define NETBE_DEBUG_INTERRUPT*/
/* extra field used in struct page */
#define netif_page_index(pg) (*(long *)&(pg)->mapping)
struct netbk_rx_meta {
skb_frag_t frag;
@@ -231,7 +234,7 @@ static inline int netbk_queue_full(netif_t *netif)
static void tx_queue_callback(unsigned long data)
{
netif_t *netif = (netif_t *)data;
if (netif_schedulable(netif->dev))
if (netif_schedulable(netif))
netif_wake_queue(netif->dev);
}
@@ -242,7 +245,7 @@ int netif_be_start_xmit(struct sk_buff *skb, struct net_device *dev)
BUG_ON(skb->dev != dev);
/* Drop the packet if the target domain has no receive buffers. */
if (unlikely(!netif_schedulable(dev) || netbk_queue_full(netif)))
if (unlikely(!netif_schedulable(netif) || netbk_queue_full(netif)))
goto drop;
/*
@@ -352,7 +355,7 @@ static u16 netbk_gop_frag(netif_t *netif, struct netbk_rx_meta *meta,
copy_gop->flags = GNTCOPY_dest_gref;
if (PageForeign(page)) {
struct pending_tx_info *src_pend =
&pending_tx_info[page->index];
&pending_tx_info[netif_page_index(page)];
copy_gop->source.domid = src_pend->netif->domid;
copy_gop->source.u.ref = src_pend->req.gref;
copy_gop->flags |= GNTCOPY_source_gref;
@@ -681,7 +684,7 @@ static void net_rx_action(unsigned long unused)
}
if (netif_queue_stopped(netif->dev) &&
netif_schedulable(netif->dev) &&
netif_schedulable(netif) &&
!netbk_queue_full(netif))
netif_wake_queue(netif->dev);
@@ -739,7 +742,7 @@ static void add_to_net_schedule_list_tail(netif_t *netif)
spin_lock_irq(&net_schedule_list_lock);
if (!__on_net_schedule_list(netif) &&
likely(netif_schedulable(netif->dev))) {
likely(netif_schedulable(netif))) {
list_add_tail(&netif->list, &net_schedule_list);
netif_get(netif);
}
@@ -1327,7 +1330,7 @@ static void netif_page_release(struct page *page)
/* Ready for next use. */
init_page_count(page);
netif_idx_release(page->index);
netif_idx_release(netif_page_index(page));
}
irqreturn_t netif_be_int(int irq, void *dev_id, struct pt_regs *regs)
@@ -1337,7 +1340,7 @@ irqreturn_t netif_be_int(int irq, void *dev_id, struct pt_regs *regs)
add_to_net_schedule_list_tail(netif);
maybe_schedule_tx_action();
if (netif_schedulable(netif->dev) && !netbk_queue_full(netif))
if (netif_schedulable(netif) && !netbk_queue_full(netif))
netif_wake_queue(netif->dev);
return IRQ_HANDLED;
@@ -1457,7 +1460,7 @@ static int __init netback_init(void)
for (i = 0; i < MAX_PENDING_REQS; i++) {
page = mmap_pages[i];
SetPageForeign(page, netif_page_release);
page->index = i;
netif_page_index(page) = i;
}
pending_cons = 0;
@@ -338,9 +338,7 @@ static void connect(struct backend_info *be)
xenbus_switch_state(dev, XenbusStateConnected);
/* May not get a kick from the frontend, so start the tx_queue now. */
if (!netbk_can_queue(be->netif->dev))
netif_wake_queue(be->netif->dev);
netif_wake_queue(be->netif->dev);
}
@@ -29,7 +29,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/module.h>
#include <linux/version.h>
#include <linux/kernel.h>
@@ -155,6 +154,7 @@ struct netfront_info {
unsigned int irq;
unsigned int copying_receiver;
unsigned int carrier;
/* Receive-ring batched refills. */
#define RX_MIN_TARGET 8
@@ -193,6 +193,15 @@ struct netfront_rx_info {
struct netif_extra_info extras[XEN_NETIF_EXTRA_TYPE_MAX - 1];
};
/*
* Implement our own carrier flag: the network stack's version causes delays
* when the carrier is re-enabled (in particular, dev_activate() may not
* immediately be called, which can cause packet loss).
*/
#define netfront_carrier_on(netif) ((netif)->carrier = 1)
#define netfront_carrier_off(netif) ((netif)->carrier = 0)
#define netfront_carrier_ok(netif) ((netif)->carrier)
/*
* Access macros for acquiring freeing slots in tx_skbs[].
*/
@@ -591,26 +600,6 @@ static int send_fake_arp(struct net_device *dev)
return dev_queue_xmit(skb);
}
static int network_open(struct net_device *dev)
{
struct netfront_info *np = netdev_priv(dev);
memset(&np->stats, 0, sizeof(np->stats));
spin_lock(&np->rx_lock);
if (netif_carrier_ok(dev)) {
network_alloc_rx_buffers(dev);
np->rx.sring->rsp_event = np->rx.rsp_cons + 1;
if (RING_HAS_UNCONSUMED_RESPONSES(&np->rx))
netif_rx_schedule(dev);
}
spin_unlock(&np->rx_lock);
netif_start_queue(dev);
return 0;
}
static inline int netfront_tx_slot_available(struct netfront_info *np)
{
return ((np->tx.req_prod_pvt - np->tx.rsp_cons) <
@@ -627,6 +616,26 @@ static inline void network_maybe_wake_tx(struct net_device *dev)
netif_wake_queue(dev);
}
static int network_open(struct net_device *dev)
{
struct netfront_info *np = netdev_priv(dev);
memset(&np->stats, 0, sizeof(np->stats));
spin_lock(&np->rx_lock);
if (netfront_carrier_ok(np)) {
network_alloc_rx_buffers(dev);
np->rx.sring->rsp_event = np->rx.rsp_cons + 1;
if (RING_HAS_UNCONSUMED_RESPONSES(&np->rx))
netif_rx_schedule(dev);
}
spin_unlock(&np->rx_lock);
network_maybe_wake_tx(dev);
return 0;
}
static void network_tx_buf_gc(struct net_device *dev)
{
RING_IDX cons, prod;
@@ -634,7 +643,7 @@ static void network_tx_buf_gc(struct net_device *dev)
struct netfront_info *np = netdev_priv(dev);
struct sk_buff *skb;
BUG_ON(!netif_carrier_ok(dev));
BUG_ON(!netfront_carrier_ok(np));
do {
prod = np->tx.sring->rsp_prod;
@@ -704,7 +713,7 @@ static void network_alloc_rx_buffers(struct net_device *dev)
int nr_flips;
netif_rx_request_t *req;
if (unlikely(!netif_carrier_ok(dev)))
if (unlikely(!netfront_carrier_ok(np)))
return;
/*
@@ -935,7 +944,7 @@ static int network_start_xmit(struct sk_buff *skb, struct net_device *dev)
spin_lock_irq(&np->tx_lock);
if (unlikely(!netif_carrier_ok(dev) ||
if (unlikely(!netfront_carrier_ok(np) ||
(frags > 1 && !xennet_can_sg(dev)) ||
netif_needs_gso(dev, skb))) {
spin_unlock_irq(&np->tx_lock);
@@ -1025,7 +1034,7 @@ static irqreturn_t netif_int(int irq, void *dev_id, struct pt_regs *ptregs)
spin_lock_irqsave(&np->tx_lock, flags);
if (likely(netif_carrier_ok(dev))) {
if (likely(netfront_carrier_ok(np))) {
network_tx_buf_gc(dev);
/* Under tx_lock: protects access to rx shared-ring indexes. */
if (RING_HAS_UNCONSUMED_RESPONSES(&np->rx))
@@ -1300,7 +1309,7 @@ static int netif_poll(struct net_device *dev, int *pbudget)
spin_lock(&np->rx_lock);
if (unlikely(!netif_carrier_ok(dev))) {
if (unlikely(!netfront_carrier_ok(np))) {
spin_unlock(&np->rx_lock);
return 0;
}
@@ -1318,7 +1327,7 @@ static int netif_poll(struct net_device *dev, int *pbudget)
work_done = 0;
while ((i != rp) && (work_done < budget)) {
memcpy(rx, RING_GET_RESPONSE(&np->rx, i), sizeof(*rx));
memset(extras, 0, sizeof(extras));
memset(extras, 0, sizeof(rinfo.extras));
err = xennet_get_responses(np, &rinfo, rp, &tmpq,
&pages_flipped);
@@ -1745,7 +1754,7 @@ static int network_connect(struct net_device *dev)
* domain a kick because we've probably just requeued some
* packets.
*/
netif_carrier_on(dev);
netfront_carrier_on(np);
notify_remote_via_irq(np->irq);
network_tx_buf_gc(dev);
network_alloc_rx_buffers(dev);
@@ -1990,7 +1999,7 @@ static struct net_device * __devinit create_netdev(struct xenbus_device *dev)
np->netdev = netdev;
netif_carrier_off(netdev);
netfront_carrier_off(np);
return netdev;
@@ -2024,7 +2033,7 @@ static void netif_disconnect_backend(struct netfront_info *info)
/* Stop old i/f to prevent errors whilst we rebuild the state. */
spin_lock_irq(&info->tx_lock);
spin_lock(&info->rx_lock);
netif_carrier_off(info->netdev);
netfront_carrier_off(info);
spin_unlock(&info->rx_lock);
spin_unlock_irq(&info->tx_lock);
@@ -239,17 +239,12 @@ static void free_root_bus_devs(struct pci_bus *bus)
{
struct pci_dev *dev;
down_write(&pci_bus_sem);
while (!list_empty(&bus->devices)) {
dev = container_of(bus->devices.next, struct pci_dev, bus_list);
up_write(&pci_bus_sem);
dev = container_of(bus->devices.next, struct pci_dev,
bus_list);
dev_dbg(&dev->dev, "removing device\n");
pci_remove_bus_device(dev);
down_write(&pci_bus_sem);
}
up_write(&pci_bus_sem);
}
void pcifront_free_roots(struct pcifront_device *pdev)
@@ -6,7 +6,6 @@
* Copyright (c) 2002-2004, K A Fraser, B Dragovic
*/
#include <linux/config.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/slab.h>
@@ -249,7 +248,7 @@ static int privcmd_enforce_singleshot_mapping(struct vm_area_struct *vma)
}
#endif
static struct file_operations privcmd_file_ops = {
static const struct file_operations privcmd_file_ops = {
.ioctl = privcmd_ioctl,
.mmap = privcmd_mmap,
};
@@ -2,10 +2,9 @@
* drivers/xen/tpmback/common.h
*/
#ifndef __NETIF__BACKEND__COMMON_H__
#define __NETIF__BACKEND__COMMON_H__
#ifndef __TPM__BACKEND__COMMON_H__
#define __TPM__BACKEND__COMMON_H__
#include <linux/config.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/interrupt.h>
@@ -629,7 +629,7 @@ static unsigned int vtpm_op_poll(struct file *file,
return flags;
}
static struct file_operations vtpm_ops = {
static const struct file_operations vtpm_ops = {
.owner = THIS_MODULE,
.llseek = no_llseek,
.open = vtpm_op_open,
-1
View File
@@ -1,4 +1,3 @@
#include <linux/config.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/slab.h>
@@ -137,6 +137,17 @@ int xb_write(const void *data, unsigned len)
return 0;
}
int xb_data_to_read(void)
{
struct xenstore_domain_interface *intf = xen_store_interface;
return (intf->rsp_cons != intf->rsp_prod);
}
int xb_wait_for_data_to_read(void)
{
return wait_event_interruptible(xb_waitq, xb_data_to_read());
}
int xb_read(void *data, unsigned len)
{
struct xenstore_domain_interface *intf = xen_store_interface;
@@ -147,9 +158,7 @@ int xb_read(void *data, unsigned len)
unsigned int avail;
const char *src;
rc = wait_event_interruptible(
xb_waitq,
intf->rsp_cons != intf->rsp_prod);
rc = xb_wait_for_data_to_read();
if (rc < 0)
return rc;
@@ -191,8 +200,20 @@ int xb_read(void *data, unsigned len)
/* Set up interrupt handler off store event channel. */
int xb_init_comms(void)
{
struct xenstore_domain_interface *intf = xen_store_interface;
int err;
if (intf->req_prod != intf->req_cons)
printk(KERN_ERR "XENBUS request ring is not quiescent "
"(%08x:%08x)!\n", intf->req_cons, intf->req_prod);
if (intf->rsp_prod != intf->rsp_cons) {
printk(KERN_WARNING "XENBUS response ring is not quiescent "
"(%08x:%08x): fixing up\n",
intf->rsp_cons, intf->rsp_prod);
intf->rsp_cons = intf->rsp_prod;
}
if (xenbus_irq)
unbind_from_irqhandler(xenbus_irq, &xb_waitq);
@@ -37,6 +37,8 @@ int xb_init_comms(void);
/* Low level routines. */
int xb_write(const void *data, unsigned len);
int xb_read(void *data, unsigned len);
int xb_data_to_read(void);
int xb_wait_for_data_to_read(void);
int xs_input_avail(void);
extern struct xenstore_domain_interface *xen_store_interface;
extern int xen_store_evtchn;
@@ -32,7 +32,6 @@
* IN THE SOFTWARE.
*/
#include <linux/config.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/uio.h>
@@ -174,17 +173,22 @@ static ssize_t xenbus_dev_write(struct file *filp,
void *reply;
char *path, *token;
struct watch_adapter *watch, *tmp_watch;
int err;
int err, rc = len;
if ((len + u->len) > sizeof(u->u.buffer))
return -EINVAL;
if ((len + u->len) > sizeof(u->u.buffer)) {
rc = -EINVAL;
goto out;
}
if (copy_from_user(u->u.buffer + u->len, ubuf, len) != 0)
return -EFAULT;
if (copy_from_user(u->u.buffer + u->len, ubuf, len) != 0) {
rc = -EFAULT;
goto out;
}
u->len += len;
if (u->len < (sizeof(u->u.msg) + u->u.msg.len))
return len;
if ((u->len < sizeof(u->u.msg)) ||
(u->len < (sizeof(u->u.msg) + u->u.msg.len)))
return rc;
msg_type = u->u.msg.type;
@@ -202,14 +206,17 @@ static ssize_t xenbus_dev_write(struct file *filp,
case XS_SET_PERMS:
if (msg_type == XS_TRANSACTION_START) {
trans = kmalloc(sizeof(*trans), GFP_KERNEL);
if (!trans)
return -ENOMEM;
if (!trans) {
rc = -ENOMEM;
goto out;
}
}
reply = xenbus_dev_request_and_reply(&u->u.msg);
if (IS_ERR(reply)) {
kfree(trans);
return PTR_ERR(reply);
rc = PTR_ERR(reply);
goto out;
}
if (msg_type == XS_TRANSACTION_START) {
@@ -232,8 +239,10 @@ static ssize_t xenbus_dev_write(struct file *filp,
case XS_UNWATCH:
path = u->u.buffer + sizeof(u->u.msg);
token = memchr(path, 0, u->u.msg.len);
if (token == NULL)
return -EILSEQ;
if (token == NULL) {
rc = -EILSEQ;
goto out;
}
token++;
if (msg_type == XS_WATCH) {
@@ -252,7 +261,8 @@ static ssize_t xenbus_dev_write(struct file *filp,
err = register_xenbus_watch(&watch->watch);
if (err) {
free_watch_adapter(watch);
return err;
rc = err;
goto out;
}
list_add(&watch->list, &u->watches);
@@ -266,7 +276,6 @@ static ssize_t xenbus_dev_write(struct file *filp,
&u->watches, list) {
if (!strcmp(watch->token, token) &&
!strcmp(watch->watch.node, path))
break;
{
unregister_xenbus_watch(&watch->watch);
list_del(&watch->list);
@@ -279,11 +288,13 @@ static ssize_t xenbus_dev_write(struct file *filp,
break;
default:
return -EINVAL;
rc = -EINVAL;
break;
}
out:
u->len = 0;
return len;
return rc;
}
static int xenbus_dev_open(struct inode *inode, struct file *filp)
@@ -343,7 +354,7 @@ static unsigned int xenbus_dev_poll(struct file *file, poll_table *wait)
return 0;
}
static struct file_operations xenbus_dev_file_ops = {
static const struct file_operations xenbus_dev_file_ops = {
.read = xenbus_dev_read,
.write = xenbus_dev_write,
.open = xenbus_dev_open,
@@ -77,11 +77,22 @@ struct xs_handle {
spinlock_t reply_lock;
wait_queue_head_t reply_waitq;
/*
* Mutex ordering: transaction_mutex -> watch_mutex -> request_mutex.
* response_mutex is never taken simultaneously with the other three.
*/
/* One request at a time. */
struct mutex request_mutex;
/* Protect xenbus reader thread against save/restore. */
struct mutex response_mutex;
/* Protect transactions against save/restore. */
struct rw_semaphore suspend_mutex;
struct rw_semaphore transaction_mutex;
/* Protect watch (de)register against save/restore. */
struct rw_semaphore watch_mutex;
};
static struct xs_handle xs_state;
@@ -150,22 +161,6 @@ static void *read_reply(enum xsd_sockmsg_type *type, unsigned int *len)
return body;
}
/* Emergency write. */
void xenbus_debug_write(const char *str, unsigned int count)
{
struct xsd_sockmsg msg = { 0 };
msg.type = XS_DEBUG;
msg.len = sizeof("print") + count + 1;
mutex_lock(&xs_state.request_mutex);
xb_write(&msg, sizeof(msg));
xb_write("print", sizeof("print"));
xb_write(str, count);
xb_write("", 1);
mutex_unlock(&xs_state.request_mutex);
}
void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg)
{
void *ret;
@@ -173,7 +168,7 @@ void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg)
int err;
if (req_msg.type == XS_TRANSACTION_START)
down_read(&xs_state.suspend_mutex);
down_read(&xs_state.transaction_mutex);
mutex_lock(&xs_state.request_mutex);
@@ -189,7 +184,7 @@ void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg)
if ((req_msg.type == XS_TRANSACTION_END) ||
((req_msg.type == XS_TRANSACTION_START) &&
(msg->type == XS_ERROR)))
up_read(&xs_state.suspend_mutex);
up_read(&xs_state.transaction_mutex);
return ret;
}
@@ -440,11 +435,11 @@ int xenbus_transaction_start(struct xenbus_transaction *t)
{
char *id_str;
down_read(&xs_state.suspend_mutex);
down_read(&xs_state.transaction_mutex);
id_str = xs_single(XBT_NIL, XS_TRANSACTION_START, "", NULL);
if (IS_ERR(id_str)) {
up_read(&xs_state.suspend_mutex);
up_read(&xs_state.transaction_mutex);
return PTR_ERR(id_str);
}
@@ -469,7 +464,7 @@ int xenbus_transaction_end(struct xenbus_transaction t, int abort)
err = xs_error(xs_single(t, XS_TRANSACTION_END, abortstr, NULL));
up_read(&xs_state.suspend_mutex);
up_read(&xs_state.transaction_mutex);
return err;
}
@@ -602,7 +597,7 @@ int register_xenbus_watch(struct xenbus_watch *watch)
sprintf(token, "%lX", (long)watch);
down_read(&xs_state.suspend_mutex);
down_read(&xs_state.watch_mutex);
spin_lock(&watches_lock);
BUG_ON(find_watch(token));
@@ -618,7 +613,7 @@ int register_xenbus_watch(struct xenbus_watch *watch)
spin_unlock(&watches_lock);
}
up_read(&xs_state.suspend_mutex);
up_read(&xs_state.watch_mutex);
return err;
}
@@ -632,7 +627,7 @@ void unregister_xenbus_watch(struct xenbus_watch *watch)
sprintf(token, "%lX", (long)watch);
down_read(&xs_state.suspend_mutex);
down_read(&xs_state.watch_mutex);
spin_lock(&watches_lock);
BUG_ON(!find_watch(token));
@@ -645,7 +640,7 @@ void unregister_xenbus_watch(struct xenbus_watch *watch)
"XENBUS Failed to release watch %s: %i\n",
watch->node, err);
up_read(&xs_state.suspend_mutex);
up_read(&xs_state.watch_mutex);
/* Cancel pending watch events. */
spin_lock(&watch_events_lock);
@@ -668,8 +663,10 @@ EXPORT_SYMBOL_GPL(unregister_xenbus_watch);
void xs_suspend(void)
{
down_write(&xs_state.suspend_mutex);
down_write(&xs_state.transaction_mutex);
down_write(&xs_state.watch_mutex);
mutex_lock(&xs_state.request_mutex);
mutex_lock(&xs_state.response_mutex);
}
void xs_resume(void)
@@ -677,21 +674,25 @@ void xs_resume(void)
struct xenbus_watch *watch;
char token[sizeof(watch) * 2 + 1];
mutex_unlock(&xs_state.response_mutex);
mutex_unlock(&xs_state.request_mutex);
up_write(&xs_state.transaction_mutex);
/* No need for watches_lock: the suspend_mutex is sufficient. */
/* No need for watches_lock: the watch_mutex is sufficient. */
list_for_each_entry(watch, &watches, list) {
sprintf(token, "%lX", (long)watch);
xs_watch(watch->node, token);
}
up_write(&xs_state.suspend_mutex);
up_write(&xs_state.watch_mutex);
}
void xs_suspend_cancel(void)
{
mutex_unlock(&xs_state.response_mutex);
mutex_unlock(&xs_state.request_mutex);
up_write(&xs_state.suspend_mutex);
up_write(&xs_state.watch_mutex);
up_write(&xs_state.transaction_mutex);
}
static int xenwatch_handle_callback(void *data)
@@ -753,27 +754,46 @@ static int process_msg(void)
char *body;
int err;
/*
* We must disallow save/restore while reading a xenstore message.
* A partial read across s/r leaves us out of sync with xenstored.
*/
for (;;) {
err = xb_wait_for_data_to_read();
if (err)
return err;
mutex_lock(&xs_state.response_mutex);
if (xb_data_to_read())
break;
/* We raced with save/restore: pending data 'disappeared'. */
mutex_unlock(&xs_state.response_mutex);
}
msg = kmalloc(sizeof(*msg), GFP_KERNEL);
if (msg == NULL)
return -ENOMEM;
if (msg == NULL) {
err = -ENOMEM;
goto out;
}
err = xb_read(&msg->hdr, sizeof(msg->hdr));
if (err) {
kfree(msg);
return err;
goto out;
}
body = kmalloc(msg->hdr.len + 1, GFP_KERNEL);
if (body == NULL) {
kfree(msg);
return -ENOMEM;
err = -ENOMEM;
goto out;
}
err = xb_read(body, msg->hdr.len);
if (err) {
kfree(body);
kfree(msg);
return err;
goto out;
}
body[msg->hdr.len] = '\0';
@@ -782,7 +802,8 @@ static int process_msg(void)
&msg->u.watch.vec_size);
if (IS_ERR(msg->u.watch.vec)) {
kfree(msg);
return PTR_ERR(msg->u.watch.vec);
err = PTR_ERR(msg->u.watch.vec);
goto out;
}
spin_lock(&watches_lock);
@@ -806,7 +827,9 @@ static int process_msg(void)
wake_up(&xs_state.reply_waitq);
}
return 0;
out:
mutex_unlock(&xs_state.response_mutex);
return err;
}
static int xenbus_thread(void *unused)
@@ -835,7 +858,9 @@ int xs_init(void)
init_waitqueue_head(&xs_state.reply_waitq);
mutex_init(&xs_state.request_mutex);
init_rwsem(&xs_state.suspend_mutex);
mutex_init(&xs_state.response_mutex);
init_rwsem(&xs_state.transaction_mutex);
init_rwsem(&xs_state.watch_mutex);
/* Initialize the shared memory rings to talk to xenstored */
err = xb_init_comms();
@@ -6,7 +6,6 @@
* documentation.
*/
#include <linux/config.h>
#include <linux/mm.h>
#include <asm/cache.h>
#include <asm/io.h>
@@ -33,7 +33,6 @@
#ifndef __HYPERVISOR_H__
#define __HYPERVISOR_H__
#include <linux/config.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/version.h>
@@ -57,7 +56,11 @@
extern shared_info_t *HYPERVISOR_shared_info;
#define vcpu_info(cpu) (HYPERVISOR_shared_info->vcpu_info + (cpu))
#ifdef CONFIG_SMP
#define current_vcpu_info() vcpu_info(smp_processor_id())
#else
#define current_vcpu_info() vcpu_info(0)
#endif
#ifdef CONFIG_X86_32
extern unsigned long hypervisor_virt_start;
@@ -20,6 +20,14 @@
#define LARGE_PAGE_SIZE (1UL << PMD_SHIFT)
#ifdef __KERNEL__
/*
* Need to repeat this here in order to not include pgtable.h (which in turn
* depends on definitions made here), but to be able to use the symbolic
* below. The preprocessor will warn if the two definitions aren't identical.
*/
#define _PAGE_PRESENT 0x001
#ifndef __ASSEMBLY__
#include <linux/string.h>
@@ -29,13 +37,6 @@
#include <xen/interface/xen.h>
#include <xen/features.h>
/*
* Need to repeat this here in order to not include pgtable.h (which in turn
* depends on definitions made here), but to be able to use the symbolic
* below. The preprocessor will warn if the two definitions aren't identical.
*/
#define _PAGE_PRESENT 0x001
#define arch_free_page(_page,_order) \
({ int foreign = PageForeign(_page); \
if (foreign) \
@@ -139,7 +140,11 @@ typedef struct { unsigned long pgprot; } pgprot_t;
static inline unsigned long pgd_val(pgd_t x)
{
unsigned long ret = x.pgd;
#ifdef CONFIG_XEN_COMPAT_030002
if (ret) ret = machine_to_phys(ret) | _PAGE_PRESENT;
#else
if (ret & _PAGE_PRESENT) ret = machine_to_phys(ret);
#endif
return ret;
}
#define HPAGE_SHIFT 22
@@ -221,8 +226,6 @@ extern int page_is_ram(unsigned long pagenr);
((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0 ) | \
VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC)
#define __HAVE_ARCH_GATE_AREA 1
#include <asm-generic/memory_model.h>
#include <asm-generic/page.h>
@@ -1,8 +1,6 @@
#ifndef _ASM_SWIOTLB_H
#define _ASM_SWIOTLB_H 1
#include <linux/config.h>
/* SWIOTLB interface */
extern dma_addr_t swiotlb_map_single(struct device *hwdev, void *ptr, size_t size,
@@ -7,8 +7,6 @@
* when communicating with Xen or other guest OSes running on other CPUs.
*/
#include <linux/config.h>
#ifdef HAVE_XEN_PLATFORM_COMPAT_H
#include <xen/platform-compat.h>
#endif
@@ -137,7 +137,7 @@ __asm__ __volatile__ ("movw %%dx,%1\n\t" \
})
#define write_cr4(x) \
__asm__ __volatile__("movl %0,%%cr4": :"r" (x));
__asm__ __volatile__("movl %0,%%cr4": :"r" (x))
/*
* Clear and set 'TS' bit respectively
@@ -46,7 +46,6 @@ extern int running_on_xen;
#endif /* CONFIG_XEN */
#if defined(CONFIG_XEN) || defined(CONFIG_VMX_GUEST)
#include <linux/config.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/version.h>
@@ -1,8 +1,6 @@
#ifndef _ASM_SWIOTLB_H
#define _ASM_SWIOTLB_H 1
#include <linux/config.h>
/* SWIOTLB interface */
extern dma_addr_t swiotlb_map_single(struct device *hwdev, void *ptr, size_t size,
@@ -7,8 +7,6 @@
* when communicating with Xen or other guest OSes running on other CPUs.
*/
#include <linux/config.h>
#define ADDR (*(volatile long *) addr)
static __inline__ void synch_set_bit(int nr, volatile void * addr)
@@ -53,6 +53,11 @@ enum fixed_addresses {
#define NR_FIX_ISAMAPS 256
FIX_ISAMAP_END,
FIX_ISAMAP_BEGIN = FIX_ISAMAP_END + NR_FIX_ISAMAPS - 1,
__end_of_permanent_fixed_addresses,
/* temporary boot-time mappings, used before ioremap() is functional */
#define NR_FIX_BTMAPS 16
FIX_BTMAP_END = __end_of_permanent_fixed_addresses,
FIX_BTMAP_BEGIN = FIX_BTMAP_END + NR_FIX_BTMAPS - 1,
__end_of_fixed_addresses
};
@@ -150,8 +150,10 @@ static inline void __iomem * ioremap (unsigned long offset, unsigned long size)
return __ioremap(offset, size, 0);
}
extern void *early_ioremap(unsigned long addr, unsigned long size);
extern void early_iounmap(void *addr, unsigned long size);
extern void *bt_ioremap(unsigned long addr, unsigned long size);
extern void bt_iounmap(void *addr, unsigned long size);
#define early_ioremap bt_ioremap
#define early_iounmap bt_iounmap
/*
* This one maps high address device memory and turns off caching for that area.
@@ -10,7 +10,6 @@
* <tomsoft@informatik.tu-chemnitz.de>
*/
#include <linux/config.h>
#include <linux/sched.h>
/* include comes from machine specific directory */
#include "irq_vectors.h"
@@ -403,19 +403,6 @@ static inline int pmd_large(pmd_t pte) {
/* to find an entry in a page-table-directory. */
#define pud_index(address) (((address) >> PUD_SHIFT) & (PTRS_PER_PUD-1))
#define pud_offset(pgd, address) ((pud_t *) pgd_page(*(pgd)) + pud_index(address))
static inline pud_t *__pud_offset_k(pud_t *pud, unsigned long address)
{
return pud + pud_index(address);
}
/* Find correct pud via the hidden fourth level page level: */
/* This accesses the reference page table of the boot cpu.
Other CPUs get synced lazily via the page fault handler. */
static inline pud_t *pud_offset_k(pgd_t *pgd, unsigned long address)
{
return pud_offset(pgd_offset_k(address), address);
}
/* PMD - Level 2 access */
#define pmd_page_kernel(pmd) ((unsigned long) __va(pmd_val(pmd) & PTE_MASK))
@@ -246,11 +246,13 @@ DECLARE_PER_CPU(struct tss_struct,init_tss);
extern struct cpuinfo_x86 boot_cpu_data;
#ifndef CONFIG_X86_NO_TSS
/* Save the original ist values for checking stack pointers during debugging */
struct orig_ist {
unsigned long ist[7];
};
DECLARE_PER_CPU(struct orig_ist, orig_ist);
#endif
#ifdef CONFIG_X86_VSMP
#define ARCH_MIN_TASKALIGN (1 << INTERNODE_CACHE_SHIFT)
@@ -252,14 +252,14 @@
#define PageForeign(page) test_bit(PG_foreign, &(page)->flags)
#define SetPageForeign(page, dtor) do { \
set_bit(PG_foreign, &(page)->flags); \
(page)->mapping = (void *)dtor; \
(page)->index = (long)(dtor); \
} while (0)
#define ClearPageForeign(page) do { \
clear_bit(PG_foreign, &(page)->flags); \
(page)->mapping = NULL; \
(page)->index = 0; \
} while (0)
#define PageForeignDestructor(page) \
( (void (*) (struct page *)) (page)->mapping )(page)
( (void (*) (struct page *)) (page)->index )(page)
struct page; /* forward declaration */
@@ -1,18 +1,18 @@
#ifndef __XEN_CPU_HOTPLUG_H__
#define __XEN_CPU_HOTPLUG_H__
#include <linux/config.h>
#include <linux/kernel.h>
#include <linux/cpumask.h>
#if defined(CONFIG_HOTPLUG_CPU)
#if defined(CONFIG_X86)
void cpu_initialize_context(unsigned int cpu);
#if defined(CONFIG_X86) && defined(CONFIG_SMP)
extern cpumask_t cpu_initialized_map;
#define cpu_set_initialized(cpu) cpu_set(cpu, cpu_initialized_map)
#else
#define cpu_initialize_context(cpu) ((void)0)
#define cpu_set_initialized(cpu) ((void)0)
#endif
#if defined(CONFIG_HOTPLUG_CPU)
int cpu_up_check(unsigned int cpu);
void init_xenbus_allowed_cpumask(void);
int smp_suspend(void);
@@ -2,7 +2,6 @@
#ifndef __ASM_XEN_DRIVER_UTIL_H__
#define __ASM_XEN_DRIVER_UTIL_H__
#include <linux/config.h>
#include <linux/vmalloc.h>
/* Allocate/destroy a 'vmalloc' VM area. */
@@ -34,7 +34,6 @@
#ifndef __ASM_EVTCHN_H__
#define __ASM_EVTCHN_H__
#include <linux/config.h>
#include <linux/interrupt.h>
#include <asm/hypervisor.h>
#include <asm/ptrace.h>
Loaded 100 of 358 files, more files were not shown because too many files have changed in this diff. Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.