Implement support for PCI devices in the SMMU driver. Trigger iommu-map
parsing when new PCI device is added. Add checks to assign/deassign
functions to ensure PCI devices are handled correctly. Implement basic
quarantining.
All pci devices are automatically assigned to hardware domain if it exists
to ensure it can probe them.
TODO:
Implement scratch page quarantining support.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Setting CONFIG_PCI_PASSTHROUGH=y will enable PCI passthrough on ARM,
even though the feature is not yet complete in the current upstream
codebase. The purpose of this is to make it easier to enable the
necessary configs (HAS_PCI, HAS_VPCI) for testing and development of PCI
passthrough on ARM, and to build it in CI.
Since PCI passthrough on ARM is still work in progress at this time,
make it depend on EXPERT.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
When ITS is enabled and PCI devices that are behind an SMMU generate an
MSI interrupt, SMMU fault will be observed as there is currently no
mapping in p2m table for the ITS translation register (GITS_TRANSLATER).
A mapping is required in the iommu page tables so that the device can
generate the MSI interrupt writing to the GITS_TRANSLATER register.
The GITS_TRANSLATER register is a 32-bit register, and there is nothing
else in a page containing it, so map that page.
Add new host_addr parameter to vgic_v3_its_init_virtual to prepare the
foundation for DomU MSI support where guest doorbell address can differ
for the host's. Note that the 1:1 check in arm_iommu_map_page remains
for now, as virtual ITSes are currently only created for hwdom where the
doorbell mapping is always 1:1.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Current code skip the mapping for PCI bridge MMIO region to dom0 when
pci_passthrough_enabled flag is set. Mapping should be skip when
has_vpci(d) is enabled for the domain, as we need to skip the mapping
only when VPCI handler are registered for ECAM.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Acked-by: Julien Grall <jgrall@amazon.com>
When ACPI is enabled and the system booted with ACPI, BUG() is observed
after merging the static event channel series. As there is no DT when
booted with ACPI there will be no chosen node because of that
"BUG_ON(chosen == NULL)" will be hit.
(XEN) Xen BUG at arch/arm/domain_build.c:3578
Move call to alloc_static_evtchn() under acpi_disabled check to fix the
issue.
Fixes: 1fe16b3ed7 (xen/arm: introduce xen-evtchn dom0less property)
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
[stefano: minor spelling fix in commit message]
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Reviewed-by: Ayan Kumar Halder <ayankuma@amd.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
is_memory_hole was implemented for x86 and not for ARM when introduced.
Replace is_memory_hole call to pci_check_bar as function should check
if device BAR is in defined memory range. Also, add an implementation
for ARM which is required for PCI passthrough.
On x86, pci_check_bar will call is_memory_hole which will check if BAR
is not overlapping with any memory region defined in the memory map.
On ARM, pci_check_bar will go through the host bridge ranges and check
if the BAR is in the range of defined ranges.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Modify pci_find_host_bridge_node argument to const pdev to avoid
converting the dev to pdev in pci_find_host_bridge_node and also
constify the return.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Introduce a new sub-node under /chosen node to establish static event
channel communication between domains on dom0less systems.
An event channel will be created beforehand to allow the domains to
send notifications to each other.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Introduce a new "xen,enhanced" dom0less property value "no-xenstore" to
disable xenstore interface for dom0less guests.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Static event channel support will be added for dom0less domains. Modify
evtchn_bind_interdomain to support static evtchn.
It is necessary to have access to the evtchn_bind_interdomain function
to do that, so make evtchn_bind_interdomain global and also make it
__must_check.
Currently evtchn_bind_interdomain() always allocates the next available
local port. Static event channel support for dom0less domains requires
allocating a specified port. Modify the evtchn_bind_interdomain to
accept the port number as an argument and allocate the specified port
if available. If the port number argument is zero, the next available
port will be allocated.
Currently evtchn_bind_interdomain() finds the local domain from
"current->domain" pointer. evtchn_bind_interdomain() will be called from
the XEN to create static event channel during domain creation.
"current" pointer is not valid at that time, therefore modify the
evtchn_bind_interdomain() to pass domain as an argument.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Currently evtchn_alloc_unbound() always allocates the next available
port. Static event channel support for dom0less domains requires
allocating a specified port.
Modify the evtchn_alloc_unbound() to accept the port number as an
argument and allocate the specified port if available. If the port
number argument is zero, the next available port will be allocated.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Restrict the maximum number of evtchn supported for domUs to avoid
allocating a large amount of memory in Xen.
Set the default value of max_evtchn_port to 1023. The value of 1023
should be sufficient for guests because on ARM we don't bind physical
interrupts to event channels. The only use of the evtchn port is
inter-domain communications. Another reason why we choose the value
of 1023 is to follow the default behavior of libxl.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>
Acked-by: Julien Grall <jgrall@amazon.com>
When devices are deassigned/assigned, SMMU global fault is observed
because SMEs are freed in detach function and not allocated again when
the device is assigned back to the guest.
Don't free the SMEs when devices are deassigned, set the s2cr to type
fault. This way the SMMU will generate a fault if a DMA access is done
by a device not assigned to a guest.
Remove the arm_smmu_master_free_smes() as this is not needed anymore,
arm_smmu_write_s2cr() will be used to set the s2cr to type fault.
Fixes: 0435784cc7 ("xen/arm: smmuv1: Intelligent SMR allocation")
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
When Xen boots on the platform that implements the GIC 600, ITS
MAPC_LPI_OFF uncorrectable command error issue is observed.
As per the GIC-600 TRM (Revision: r1p6) MAPC_LPI_OFF command error can
be reported if the MAPC command has tried to map a collection to a core
that does not have LPIs enabled. The definition of GICR.EnableLPIs
also suggests enabling the LPIs before sending any ITS command that
involves LPIs
0b0 LPI support is disabled. Any doorbell interrupt generated as a
result of a write to a virtual LPI register must be discarded,
and any ITS translation requests or commands involving LPIs in
this Redistributor are ignored.
0b1 LPI support is enabled.
To fix the MAPC command error issue, enable the LPIs using
GICR_CTLR.EnableLPIs before mapping the collection table.
gicv3_enable_lpis() is using writel_relaxed(), write to the GICR_CTLR
register may not be visible before gicv3_its_setup_collection() send the
MAPC command. Use wmb() after writel_relaxed() to make sure register
write to enable LPIs is visible.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
This commit is adding Spectre BHB handling to Xen on Arm.
The commit is introducing new alternative code to be executed during
exception entry:
- SMCC workaround 3 call
- loop workaround (with 8, 24 or 32 iterations)
- use of new clearbhb instruction
Cpuerrata is modified by this patch to apply the required workaround for
CPU affected by Spectre BHB when CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR is
enabled.
To do this the system previously used to apply smcc workaround 1 is
reused and new alternative code to be copied in the exception handler is
introduced.
To define the type of workaround required by a processor, 4 new cpu
capabilities are introduced (for each number of loop and for smcc
workaround 3).
When a processor is affected, enable_spectre_bhb_workaround is called
and if the processor does not have CSV2 set to 3 or ECBHB feature (which
would mean that the processor is doing what is required in hardware),
the proper code is enabled at exception entry.
In the case where workaround 3 is not supported by the firmware, we
enable workaround 1 when possible as it will also mitigate Spectre BHB
on systems without CSV2.
This is part of XSA-398 / CVE-2022-23960.
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <julien@xen.org>
libxl will create an emulated PCI device tree node in the device tree to
enable the guest OS to discover the virtual PCI during guest boot.
Emulated PCI device tree node will only be created when there is any
device assigned to guest.
A new area has been reserved in the arm guest physical map at
which the VPCI bus is declared in the device tree (reg and ranges
parameters of the node).
Note that currently we are using num_pcidevs instead of
c_info->passthrough to decide whether to create a vPCI DT node.
This will be insufficient if and when ARM does PCI hotplug.
Add this note inside libxl_create.c where c_info->passthrough
is set.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Michal Orzel <michal.orzel@arm.com>
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Julien Grall <jgrall@amazon.com>
Acked-by: Ian Jackson <iwj@xenproject.org>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
The existing VPCI support available for X86 is adapted for Arm.
When the device is added to XEN via the hyper call
“PHYSDEVOP_pci_device_add”, VPCI handler for the config space
access is added to the Xen to emulate the PCI devices config space.
A MMIO trap handler for the PCI ECAM space is registered in XEN
so that when guest is trying to access the PCI config space,XEN
will trap the access and emulate read/write using the VPCI and
not the real PCI hardware.
For Dom0less systems scan_pci_devices() would be used to discover the
PCI device in XEN and VPCI handler will be added during XEN boots.
This patch is also doing some small fixes to fix compilation errors on
arm32 of vpci and prevent 64bit accesses on 32bit:
- use %zu instead of lu in header.c for print
- prevent 64bit accesses in vpci_access_allowed
- ifdef out using CONFIG_64BIT handling of len 8 in
vpci_ecam_{read/write}
TODO: currently vpci_add_handlers is marked as __hwdom_init, but on ARM
vpci_add_handlers can be called after boot from
PHYSDEVOP_pci_device_add. Consider removing __hwdom_init.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
[stefano: add TODO item to commit message]
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Acked-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
If the property is not present in the device tree node for host bridge,
XEN while creating the dtb for hwdom will create this property and
assigns the already allocated segment to the host bridge
so that XEN and linux will have the same segment for the host bridges.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Introduce XEN_DOMCTL_CDF_vpci flag to enable VPCI support in XEN.
Reject the use of this new flag for x86 as VPCI is not supported for
DOMU guests for x86.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
[stefano: drop _XEN_DOMCTL_CDF_vpci]
Signed-off-by: Stefano Stabellini <stefano.stabellini@xilinx.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Christian Lindig <christian.lindig@citrix.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
XEN during boot will read the PCI device tree node “reg” property
and will map the PCI config space to the XEN memory.
As of now only "pci-host-ecam-generic" compatible board is supported.
"linux,pci-domain" device tree property assigns a fixed PCI domain
number to a host bridge, otherwise an unstable (across boots) unique
number will be assigned by Linux. XEN access the PCI devices based on
Segment:Bus:Device:Function. A Segment number in the XEN is same as a
domain number in Linux. Segment number and domain number has to be in
sync to access the correct PCI devices.
XEN will read the “linux,pci-domain” property from the device tree node
and configure the host bridge segment number accordingly. If this
property is not available XEN will allocate the unique segment number
to the host bridge.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Hardware domain is in charge of doing the PCI enumeration and will
discover the PCI devices and then will communicate to XEN via hyper
call PHYSDEVOP_pci_device_add(..) to add the PCI devices in XEN.
Also implement PHYSDEVOP_pci_device_remove(..) to remove the PCI device.
As most of the code for PHYSDEVOP_pci_device_* is the same between x86
and ARM, move the code to a common file to avoid duplication.
There are other PHYSDEVOP_pci_device_* operations to add PCI devices.
Currently implemented PHYSDEVOP_pci_device_remove(..) and
PHYSDEVOP_pci_device_add(..) only as those are minimum required to
support PCI passthrough on ARM.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
pci_init(..) will be called during xen startup to initialize and probe
the PCI host-bridge driver.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Prototypes declared in asm/pci.h that take argument of type pci_sbdf_t
are included in xen/pci.h before defining pci_sbdf_t.
Include asm/pci.h after pci_sbdf_t in xen/pci.h to fix the issue.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
APEI not supported on ARM yet move the code under CONFIG_X86 flag to
gate the code for ARM.
This patch is the preparatory work to enable HAS_PCI on ARM to avoid
compilation error on ARM.
prelink.o: In function `pcie_aer_get_firmware_first’:
drivers/passthrough/pci.c:1251: undefined reference to `apei_hest_parse'
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Based Linux commit 41e5c0f81d3e676d671d96a0a1fafb27abfbd9d7
Import the Linux helper of_get_pci_domain_nr. This function will try to
obtain the host bridge domain number by finding a property called
"linux,pci-domain" of the given device node.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3
Import the Linux helper of_property_read_u32_array. This function find
and read an array of 32 bit integers from a property.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Based Linux commit a67e9472da423ec47a3586920b526ebaedf25fc3
Import the Linux helper of_property_read_variable_u32_array. This
function find and read an array of 32 bit integers from a property,
with bounds on the minimum and maximum array size.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
In a follow-up we will enable PCI support in Xen on Arm (i.e select
HAS_PCI).
The generic code expects the arch to implement a few functions:
arch_iommu_use_permitted()
arch_pci_clean_pirqs()
Note that this is not yet sufficient to enable HAS_PCI and will be
addressed in follow-ups.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
On Arm, the initial plan is to only support GICv3 ITS which doesn't
require us to manage the MSIs because the HW will protect against
spoofing. Move the code under CONFIG_HAS_PCI_MSI flag to gate the code
for ARM.
No functional change intended.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Daniel P. Smith <dpsmith@apertussolutions.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
Switch from kzalloc_array(..) to devm_kcalloc(..) when allocating the
SMR to make code coherent.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Backport commit e19898077cfb642fe151ba22981e795c74d9e114
"iommu/arm-smmu: Set privileged attribute to 'default' instead of
'unprivileged'"
Original commit message:
Currently the driver sets all the device transactions privileges
to UNPRIVILEGED, but there are cases where the iommu masters wants
to isolate privileged supervisor and unprivileged user.
So don't override the privileged setting to unprivileged, instead
set it to default as incoming and let it be controlled by the
pagetable settings.
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Sricharan R <sricharan@codeaurora.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Tested-by: Stefano Stabellini <sstabellini@kernel.org>
SMR allocation should be based on the number of supported stream
matching register for each SMMU device.
Issue introduced by commit 5e08586afb
when backported the patches from Linux to XEN to fix the stream match
conflict issue when two devices have the same stream-id.
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Tested-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Move iommu_update_ire_from_msi(..) from passthrough/pci.c to
xen/iommu.h and wrap it under CONFIG_X86 as it is referenced in x86
code only to avoid compilation error for other architecture when
HAS_PCI is enabled.
No functional change intended.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Revert the code that associates the group pointer with the S2CR as this
code causing an issue when the SMMU device has more than one master
device with same stream-id. This issue is introduced by commit
0435784cc7 ("xen/arm: smmuv1: Intelligent SMR allocation”
Reverting the code will not impact to use of SMMU if two devices use the
same stream-id but each device will be in a separate group. This is the same
behaviour before the code is merged.
Fixes: 0435784cc7 ("xen/arm: smmuv1: Intelligent SMR allocation”
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
PCI ATS code is common for all architecture, move code to common
directory to be usable for other architectures.
No functional change intended.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Backport 588888a7399db352d2b1a41c9d5b3bf0fd482390
"iommu/arm-smmu: Intelligent SMR allocation" from the Linux kernel
This patch fix the stream match conflict issue when two devices have the
same stream-id.
Only difference while applying this patch with regard to Linux patch are
as follows:
1. Spinlock is used in place of mutex when attaching a device to the
SMMU via arm_smmu_master_alloc_smes(..) function call.Replacing the
mutex with spinlock is fine here as we are configuring the hardware
via registers and it is very fast.
2. move iommu_group_alloc(..) function call in arm_smmu_add_device(..)
function from the start of the function to the end.
Original commit message:
iommu/arm-smmu: Intelligent SMR allocation
Stream Match Registers are one of the more awkward parts of the SMMUv2
architecture; there are typically never enough to assign one to each
stream ID in the system, and configuring them such that a single ID
matches multiple entries is catastrophically bad - at best, every
transaction raises a global fault; at worst, they go *somewhere*.
To address the former issue, we can mask ID bits such that a single
register may be used to match multiple IDs belonging to the same device
or group, but doing so also heightens the risk of the latter problem
(which can be nasty to debug).
Tackle both problems at once by replacing the simple bitmap allocator
with something much cleverer. Now that we have convenient in-memory
representations of the stream mapping table, it becomes straightforward
to properly validate new SMR entries against the current state, opening
the door to arbitrary masking and SMR sharing.
Another feature which falls out of this is that with IDs shared by
separate devices being automatically accounted for, simply associating a
group pointer with the S2CR offers appropriate group allocation almost
for free, so hook that up in the process.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien GralL <jgrall@amazon.com>
Backport commit d3097e39302083d58922a3d1032d7d59a63d263d
"iommu/arm-smmu: Add a stream map entry iterator" from the Linux kernel.
This patch is the preparatory work to fix the stream match conflict
when two devices have the same stream-id.
Original commit message:
iommu/arm-smmu: Add a stream map entry iterator
We iterate over the SMEs associated with a master config quite a lot in
various places, and are about to do so even more. Let's wrap the idiom
in a handy iterator macro before the repetition gets out of hand.
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Backport commit 8e8b203eabd8b9e96d02d6339e4abce3e5a7ea4b
"iommu/arm-smmu: Keep track of S2CR state" from the Linux kernel.
This patch is the preparatory work to fix the stream match conflict
when two devices have the same stream-id.
Original commit message:
iommu/arm-smmu: Keep track of S2CR state
Making S2CRs first-class citizens within the driver with a high-level
representation of their state offers a neat solution to a few problems:
Firstly, the information about which context a device's stream IDs are
associated with is already present by necessity in the S2CR. With that
state easily accessible we can refer directly to it and obviate the need
to track an IOMMU domain in each device's archdata (its earlier purpose
of enforcing correct attachment of multi-device groups now being handled
by the IOMMU core itself).
Secondly, the core API now deprecates explicit domain detach and expects
domain attach to move devices smoothly from one domain to another; for
SMMUv2, this notion maps directly to simply rewriting the S2CRs assigned
to the device. By giving the driver a suitable abstraction of those
S2CRs to work with, we can massively reduce the overhead of the current
heavy-handed "detach, free resources, reallocate resources, attach"
approach.
Thirdly, making the software state hardware-shaped and attached to the
SMMU instance once again makes suspend/resume of this register group
that much simpler to implement in future.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Backport commit 1f3d5ca43019bff1105838712d55be087d93c0da
"iommu/arm-smmu: Consolidate stream map entry state" from the Linux
kernel.
This patch is the preparatory work to fix the stream match conflict
when two devices have the same stream-id.
Original commit message:
iommu/arm-smmu: Consolidate stream map entry state
In order to consider SMR masking, we really want to be able to validate
ID/mask pairs against existing SMR contents to prevent stream match
conflicts, which at best would cause transactions to fault unexpectedly,
and at worst lead to silent unpredictable behaviour. With our SMMU
instance data holding only an allocator bitmap, and the SMR values
themselves scattered across master configs hanging off devices which we
may have no way of finding, there's essentially no way short of digging
everything back out of the hardware. Similarly, the thought of power
management ops to support suspend/resume faces the exact same problem.
By massaging the software state into a closer shape to the underlying
hardware, everything comes together quite nicely; the allocator and the
high-level view of the data become a single centralised state which we
can easily keep track of, and to which any updates can be validated in
full before being synchronised to the hardware itself.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Backport commit 21174240e4f4439bb8ed6c116cdbdc03eba2126e
"iommu/arm-smmu: Handle stream IDs more dynamically" from the Linux
ernel.
This patch is the preparatory work to fix the stream match conflict
when two devices have the same stream-id.
Original commit message:
iommu/arm-smmu: Handle stream IDs more dynamically
Rather than assuming fixed worst-case values for stream IDs and SMR
masks, keep track of whatever implemented bits the hardware actually
reports. This also obviates the slightly questionable validation of SMR
fields in isolation - rather than aborting the whole SMMU probe for a
hardware configuration which is still architecturally valid, we can
simply refuse masters later if they try to claim an unrepresentable ID
or mask (which almost certainly implies a DT error anyway).
Acked-by: Will Deacon <will.deacon@arm.com>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
SMMUv3 driver does not handle multiple StreamId if the master device
supports more than one StreamID.
This bug was introduced when the driver was ported from Linux to XEN.
dt_device_set_protected(..) should be called from add_device(..) not
from the dt_xlate(..).
Move dt_device_set_protected(..) from dt_xlate(..) to add_device().
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
SMMUv3 driver does not support ACPI device probe.If APCI is enabled with
SMMUv3 driver compiler will throw an error.
Disable SMMUv3 driver when ACPI is enabled in kconfig to fix compilation
error.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Add support for ARM architected SMMUv3 implementation. It is based on
the Linux SMMUv3 driver.
Driver is currently supported as Tech Preview.
Major differences with regard to Linux driver are as follows:
2. Only Stage-2 translation is supported as compared to the Linux driver
that supports both Stage-1 and Stage-2 translations.
3. Use P2M page table instead of creating one as SMMUv3 has the
capability to share the page tables with the CPU.
4. Tasklets are used in place of threaded IRQ's in Linux for event queue
and priority queue IRQ handling.
5. Latest version of the Linux SMMUv3 code implements the commands queue
access functions based on atomic operations implemented in Linux.
Atomic functions used by the commands queue access functions are not
implemented in XEN therefore we decided to port the earlier version
of the code. Atomic operations are introduced to fix the bottleneck
of the SMMU command queue insertion operation. A new algorithm for
inserting commands into the queue is introduced, which is lock-free
on the fast-path.
Consequence of reverting the patch is that the command queue
insertion will be slow for large systems as spinlock will be used to
serializes accesses from all CPUs to the single queue supported by
the hardware. Once the proper atomic operations will be available in
XEN the driver can be updated.
6. Spin lock is used in place of mutex when attaching a device to the
SMMU, as there is no blocking locks implementation available in XEN.
This might introduce latency in XEN. Need to investigate before
driver is out for tech preview.
7. PCI ATS functionality is not supported, as there is no support
available in XEN to test the functionality. Code is not tested and
compiled. Code is guarded by the flag CONFIG_PCI_ATS.
8. MSI interrupts are not supported as there is no support available in
XEN to request MSI interrupts. Code is not tested and compiled. Code
is guarded by the flag CONFIG_MSI.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Replace all Linux device tree handling function with the XEN
functions.
Replace all Linux ktime function with the XEN time functions.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Backport commit df561f6688fef775baa341a0f5d960becd248b11
"treewide: Use fallthrough pseudo-keyword" from Linux kernel.
Replace the existing /* fall through */ comments and its variants with
the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary
fall-through markings when it is the case.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
-Wimplicit-fallthrough warns when a switch case falls through. Warning
can be suppress by either adding a /* fallthrough */ comment, or by
using a null statement: __attribute__ ((fallthrough))
Define the pseudo keyword 'fallthrough' for the ability to convert the
various case block /* fallthrough */ style comments to null statement
"__attribute__((__fallthrough__))"
In C mode, GCC supports the __fallthrough__ attribute since 7.1,
the same time the warning and the comment parsing were introduced.
fallthrough devolves to an empty "do {} while (0)" if the compiler
version (any version less than gcc 7) does not support the attribute.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Import the Linux helper of_property_match_string. This function searches
a string list property and returns the index of a specific string value.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Linux SMMUv3 driver supports both Stage-1 and Stage-2 translations.
As of now only Stage-2 translation support has been tested.
Once Stage-1 translation support is tested this patch can be added.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
XArray is not implemented in XEN revert the patch that introduce the
XArray code in SMMUv3 driver.
XArray is added in preparation for sharing some ASIDs with the CPU,
As XEN support only Stage-2 translation, ASID is used for Stage-1
translation there is no consequences of reverting this patch for XEN.
Once XArray is implemented in XEN this patch can be added in XEN if XEN
supports Stage-1 translation.
Reverted the commit 0299a1a81ca056e79c1a7fb751f936ec0d5c7afe
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Linux SMMUv3 code implements the commands-queue insertion based on
atomic operations implemented in Linux. Atomic functions used by the
commands-queue insertion are not implemented in XEN therefore revert the
patch that implemented the commands-queue insertion based on atomic
operations.
Reverted the other patches also that are implemented based on the code
that introduced the atomic-operations.
Atomic operations are introduced in the patch "iommu/arm-smmu-v3: Reduce
contention during command-queue insertion" that fixed the bottleneck of
the SMMU command queue insertion operation. A new algorithm for
inserting commands into the queue is introduced in this patch, which is
lock-free on the fast-path.
Consequence of reverting the patch is that the command queue insertion
will be slow for large systems as spinlock will be used to serializes
accesses from all CPUs to the single queue supported by the hardware.
Once the proper atomic operations will be available in XEN the driver
can be updated.
Following commits are reverted in this patch:
1. "iommu/arm-smmu-v3: Add SMMUv3.2 range invalidation support"
commit 6a481a95d4c198a2dd0a61f8877b92a375757db8.
2. "iommu/arm-smmu-v3: Batch ATC invalidation commands"
commit 9e773aee8c3e1b3ba019c5c7f8435aaa836c6130.
3. "iommu/arm-smmu-v3: Batch context descriptor invalidation"
commit edd0351e7bc49555d8b5ad8438a65a7ca262c9f0.
4. "iommu/arm-smmu-v3: Add command queue batching helpers
commit 4ce8da453640147101bda418640394637c1a7cfc.
5. "iommu/arm-smmu-v3: Fix ATC invalidation ordering wrt main TLBs"
commit 353e3cf8590cf182a9f42e67993de3aca91e8090.
6. "iommu/arm-smmu-v3: Defer TLB invalidation until ->iotlb_sync()"
commit 2af2e72b18b499fa36d3f7379fd010ff25d2a984.
7. "iommu/arm-smmu-v3: Reduce contention during command-queue insertion"
commit 587e6c10a7ce89a5924fdbeff2ec524fbd6a124b.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Based on tag Linux 5.8.18 commit ab435ce49bd1d02e33dfec24f76955dc1196970b
Directory structure change for the SMMUv3 driver starting from
Linux 5.9, to revert the patches smoothly using the "git revert" command
we decided to choose Linux 5.8.18.
Only difference between latest stable Linux 5.9.12 and Linux 5.8.18
SMMUv3 driver is the use of the "fallthrough" keyword. This patch will
be merged once "fallthrough" keyword implementation is available in XEN.
It's a copy of the Linux SMMUv3 driver. Xen specific code has not
been added yet and code has not been compiled.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Acked-by: Julien Grall <jgrall@amazon.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
The NS16550 driver is assuming that NS16550 PCI card are usable if the
architecture supports PCI (i.e. CONFIG_HAS_PCI=y). However, the code is
very x86 focus and will fail to build on Arm (/!\ it is not all the
errors):
ns16550.c: In function ‘ns16550_init_irq’:
ns16550.c:726:21: error: implicit declaration of function ‘create_irq’;
did you mean ‘release_irq’? [-Werror=implicit-function-declaration]
uart->irq = create_irq(0, false);
^~~~~~~~~~
release_irq
ns16550.c:726:21: error: nested extern declaration of ‘create_irq’
[-Werror=nested-externs]
ns16550.c: In function ‘ns16550_init_postirq’:
ns16550.c:768:33: error: ‘mmio_ro_ranges’ undeclared (first use in this
function); did you mean ‘mmio_handler’?
rangeset_add_range(mmio_ro_ranges, uart->io_base,
^~~~~~~~~~~~~~
mmio_handler
ns16550.c:768:33: note: each undeclared identifier is reported only once
for each function it appears in
ns16550.c:780:20: error: variable ‘msi’ has initializer but incomplete
type
struct msi_info msi = {
^~~~~~~~
Enabling support for NS16550 PCI card on Arm would require more plumbing
in addition to fixing the compilation error.
Arm systems tend to have platform UART available such as NS16550, PL011.
So there are limited reasons to get NS16550 PCI support for now on Arm.
Guard all remaining PCI code that is not under x86 flag with CONFIG_X86.
No functional change intended.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Signed-off-by: Julien Grall <jgrall@amazon.com>
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
If mem-sharing, mem-paging, or log-dirty functionality is not enabled
for architecture when HAS_PCI is enabled, the compiler will throw an
error.
Move code to x86 specific file to fix compilation error.
Also, modify the code to use likely() in place of unlikley() for each
condition to make code more optimized.
No functional change intended.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
passthrough/pci.c file is common for all architecture, but there is x86
specific code in this file.
Move x86 specific code to the drivers/passthrough/io.c file to avoid
compilation error for other architecture.
As drivers/passthrough/io.c is compiled only for x86 move it to
x86 directory and rename it to hvm.c.
No functional change intended.
Signed-off-by: Rahul Singh <rahul.singh@arm.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
Rahul Singh