Merge branch 'remotes/lorenzo/pci/endpoint'

- Use notification chain instead of EPF linkup ops for EPC events (Kishon Vijay Abraham I) - Protect concurrent allocation in endpoint outbound address region (Kishon Vijay Abraham I) - Protect concurrent access to pci_epf_ops (Kishon Vijay Abraham I) - Assign function number for each PF in endpoint core (Kishon Vijay Abraham I) - Refactor endpoint mode core initialization (Vidya Sagar) - Add API to notify when core initialization completes (Vidya Sagar) - Add test framework support to defer core initialization (Vidya Sagar) - Update Tegra SoC ABI header to support uninitialization of UPHY PLL when in endpoint mode without reference clock (Vidya Sagar) - Add DT and driver support for Tegra194 PCIe endpoint nodes (Vidya Sagar) - Add endpoint test support for DMA data transfer (Kishon Vijay Abraham I) - Print throughput information in endpoint test (Kishon Vijay Abraham I) - Use streaming DMA APIs for endpoint test buffer allocation (Kishon Vijay Abraham I) - Add endpoint test command line option for DMA (Kishon Vijay Abraham I) - When stopping a controller via configfs, clear endpoint "start" entry to prevent WARN_ON (Kunihiko Hayashi) - Update endpoint ->set_msix() to pay attention to MSI-X BAR Indicator and offset when finding MSI-X tables (Kishon Vijay Abraham I) - MSI-X tables are in local memory, not in the PCI address space. Update pcie-designware-ep to account for this (Kishon Vijay Abraham I) - Allow AM654 PCIe Endpoint to raise MSI-X interrupts (Kishon Vijay Abraham I) - Avoid using module parameter to determine irqtype for endpoint test (Kishon Vijay Abraham I) - Add ioctl to clear IRQ for endpoint test (Kishon Vijay Abraham I) - Add endpoint test 'e' option to clear IRQ (Kishon Vijay Abraham I) - Bump limit on number of endpoint test devices from 10 to 10,000 (Kishon Vijay Abraham I) - Use full pci-endpoint-test name in request_irq() for easier profiling (Kishon Vijay Abraham I) - Reduce log level of -EPROBE_DEFER error messages to debug (Thierry Reding) * remotes/lorenzo/pci/endpoint: misc: pci_endpoint_test: remove duplicate macro PCI_ENDPOINT_TEST_STATUS PCI: tegra: Print -EPROBE_DEFER error message at debug level misc: pci_endpoint_test: Use full pci-endpoint-test name in request_irq() misc: pci_endpoint_test: Fix to support > 10 pci-endpoint-test devices tools: PCI: Add 'e' to clear IRQ misc: pci_endpoint_test: Add ioctl to clear IRQ misc: pci_endpoint_test: Avoid using module parameter to determine irqtype PCI: keystone: Allow AM654 PCIe Endpoint to raise MSI-X interrupt PCI: dwc: Fix dw_pcie_ep_raise_msix_irq() to get correct MSI-X table address PCI: endpoint: Fix ->set_msix() to take BIR and offset as arguments misc: pci_endpoint_test: Add support to get DMA option from userspace tools: PCI: Add 'd' command line option to support DMA misc: pci_endpoint_test: Use streaming DMA APIs for buffer allocation PCI: endpoint: functions/pci-epf-test: Print throughput information PCI: endpoint: functions/pci-epf-test: Add DMA support to transfer data PCI: endpoint: Fix clearing start entry in configfs PCI: tegra: Add support for PCIe endpoint mode in Tegra194 dt-bindings: PCI: tegra: Add DT support for PCIe EP nodes in Tegra194 soc/tegra: bpmp: Update ABI header PCI: pci-epf-test: Add support to defer core initialization PCI: dwc: Add API to notify core initialization completion PCI: endpoint: Add notification for core init completion PCI: dwc: Refactor core initialization code for EP mode PCI: endpoint: Add core init notifying feature PCI: endpoint: Assign function number for each PF in EPC core PCI: endpoint: Protect concurrent access to pci_epf_ops with mutex PCI: endpoint: Fix for concurrent memory allocation in OB address region PCI: endpoint: Replace spinlock with mutex PCI: endpoint: Use notification chain mechanism to notify EPC events to EPF

Bjorn Helgaas committed 6 years ago

Commit b16f2ab2 parents cc36a45 e48ba3e17 files · +1655 −308Browse files

Unified · Split

17 files

Documentation/devicetree/bindings/pci/nvidia,tegra194-pcie.txt +99 −26
drivers/misc/pci_endpoint_test.c +179 −34
drivers/pci/controller/dwc/Kconfig +26 −3
drivers/pci/controller/dwc/pci-keystone.c +4 −1
drivers/pci/controller/dwc/pcie-designware-ep.c +85 −59
drivers/pci/controller/dwc/pcie-designware.h +12 −0
drivers/pci/controller/dwc/pcie-tegra194.c +695 −17
drivers/pci/endpoint/functions/pci-epf-test.c +356 −46
drivers/pci/endpoint/pci-ep-cfs.c +6 −22
drivers/pci/endpoint/pci-epc-core.c +75 −62
drivers/pci/endpoint/pci-epc-mem.c +8 −2
drivers/pci/endpoint/pci-epf-core.c +13 −22
include/linux/pci-epc.h +22 −5
include/linux/pci-epf.h +25 −4
include/soc/tegra/bpmp-abi.h +9 −1
include/uapi/linux/pcitest.h +8 −0
tools/pci/pcitest.c +33 −4

Documentation/devicetree/bindings/pci/nvidia,tegra194-pcie.txt+99−26

@@ -1,11 +1,11 @@
1	1	NVIDIA Tegra PCIe controller (Synopsys DesignWare Core based)
2	2
3		−This PCIe host controller is based on the Synopsis Designware PCIe IP
	3	+This PCIe controller is based on the Synopsis Designware PCIe IP
4	4	and thus inherits all the common properties defined in designware-pcie.txt.
	5	+Some of the controller instances are dual mode where in they can work either
	6	+in root port mode or endpoint mode but one at a time.
5	7
6	8	Required properties:
7		−- compatible: For Tegra19x, must contain "nvidia,tegra194-pcie".
8		−- device_type: Must be "pci"
9	9	- power-domains: A phandle to the node that controls power to the respective
10	10	PCIe controller and a specifier name for the PCIe controller. Following are
11	11	the specifiers for the different PCIe controllers
@@ -32,6 +32,32 @@ Required properties: · 20 unmodified lines
32	32	entry for each entry in the interrupt-names property.
33	33	- interrupt-names: Must include the following entries:
34	34	"intr": The Tegra interrupt that is asserted for controller interrupts
	35	+- clocks: Must contain an entry for each entry in clock-names.
	36	+ See ../clocks/clock-bindings.txt for details.
	37	+- clock-names: Must include the following entries:
	38	+ - core
	39	+- resets: Must contain an entry for each entry in reset-names.
	40	+ See ../reset/reset.txt for details.
	41	+- reset-names: Must include the following entries:
	42	+ - apb
	43	+ - core
	44	+- phys: Must contain a phandle to P2U PHY for each entry in phy-names.
	45	+- phy-names: Must include an entry for each active lane.
	46	+ "p2u-N": where N ranges from 0 to one less than the total number of lanes
	47	+- nvidia,bpmp: Must contain a pair of phandle to BPMP controller node followed
	48	+ by controller-id. Following are the controller ids for each controller.
	49	+ 0: C0
	50	+ 1: C1
	51	+ 2: C2
	52	+ 3: C3
	53	+ 4: C4
	54	+ 5: C5
	55	+- vddio-pex-ctl-supply: Regulator supply for PCIe side band signals
	56	+
	57	+RC mode:
	58	+- compatible: Tegra19x must contain "nvidia,tegra194-pcie"
	59	+- device_type: Must be "pci" for RC mode
	60	+- interrupt-names: Must include the following entries:
35	61	"msi": The Tegra interrupt that is asserted when an MSI is received
36	62	- bus-range: Range of bus numbers associated with this controller
37	63	- #address-cells: Address representation for root ports (must be 3)
@@ -60,27 +86,15 @@ Required properties: · 22 unmodified lines
60	86	- interrupt-map-mask and interrupt-map: Standard PCI IRQ mapping properties
61	87	Please refer to the standard PCI bus binding document for a more detailed
62	88	explanation.
63		−- clocks: Must contain an entry for each entry in clock-names.
64		− See ../clocks/clock-bindings.txt for details.
65		−- clock-names: Must include the following entries:
66		− - core
67		−- resets: Must contain an entry for each entry in reset-names.
68		− See ../reset/reset.txt for details.
69		−- reset-names: Must include the following entries:
70		− - apb
71		− - core
72		−- phys: Must contain a phandle to P2U PHY for each entry in phy-names.
73		−- phy-names: Must include an entry for each active lane.
74		− "p2u-N": where N ranges from 0 to one less than the total number of lanes
75		−- nvidia,bpmp: Must contain a pair of phandle to BPMP controller node followed
76		− by controller-id. Following are the controller ids for each controller.
77		− 0: C0
78		− 1: C1
79		− 2: C2
80		− 3: C3
81		− 4: C4
82		− 5: C5
83		−- vddio-pex-ctl-supply: Regulator supply for PCIe side band signals
	89	+
	90	+EP mode:
	91	+In Tegra194, Only controllers C0, C4 & C5 support EP mode.
	92	+- compatible: Tegra19x must contain "nvidia,tegra194-pcie-ep"
	93	+- reg-names: Must include the following entries:
	94	+ "addr_space": Used to map remote RC address space
	95	+- reset-gpios: Must contain a phandle to a GPIO controller followed by
	96	+ GPIO that is being used as PERST input signal. Please refer to pci.txt
	97	+ document.
84	98
85	99	Optional properties:
86	100	- pinctrl-names: A list of pinctrl state names.
@@ -104,6 +118,8 @@ Optional properties: · 17 unmodified lines
104	118	specified in microseconds
105	119	- nvidia,aspm-l0s-entrance-latency-us: ASPM L0s entrance latency to be
106	120	specified in microseconds
	121	+
	122	+RC mode:
107	123	- vpcie3v3-supply: A phandle to the regulator node that supplies 3.3V to the slot
108	124	if the platform has one such slot. (Ex:- x16 slot owned by C5 controller
109	125	in p2972-0000 platform).
@@ -111,11 +127,18 @@ Optional properties: · 1 unmodified line
111	127	if the platform has one such slot. (Ex:- x16 slot owned by C5 controller
112	128	in p2972-0000 platform).
113	129
	130	+EP mode:
	131	+- nvidia,refclk-select-gpios: Must contain a phandle to a GPIO controller
	132	+ followed by GPIO that is being used to enable REFCLK to controller from host
	133	+
	134	+NOTE:- On Tegra194's P2972-0000 platform, only C5 controller can be enabled to
	135	+operate in the endpoint mode because of the way the platform is designed.
	136	+
114	137	Examples:
115	138	=========
116	139
117		−Tegra194:
118		−--------
	140	+Tegra194 RC mode:
	141	+-----------------
119	142
120	143	pcie@14180000 {
121	144	compatible = "nvidia,tegra194-pcie", "snps,dw-pcie";
@@ -169,3 +192,53 @@ Tegra194: · 47 unmodified lines
169	192	<&p2u_hsio_5>;
170	193	phy-names = "p2u-0", "p2u-1", "p2u-2", "p2u-3";
171	194	};
	195	+
	196	+Tegra194 EP mode:
	197	+-----------------
	198	+
	199	+ pcie_ep@141a0000 {
	200	+ compatible = "nvidia,tegra194-pcie-ep", "snps,dw-pcie-ep";
	201	+ power-domains = <&bpmp TEGRA194_POWER_DOMAIN_PCIEX8A>;
	202	+ reg = <0x00 0x141a0000 0x0 0x00020000 /* appl registers (128K) */
	203	+ 0x00 0x3a040000 0x0 0x00040000 /* iATU_DMA reg space (256K) */
	204	+ 0x00 0x3a080000 0x0 0x00040000 /* DBI reg space (256K) */
	205	+ 0x1c 0x00000000 0x4 0x00000000>; /* Address Space (16G) */
	206	+ reg-names = "appl", "atu_dma", "dbi", "addr_space";
	207	+
	208	+ num-lanes = <8>;
	209	+ num-ib-windows = <2>;
	210	+ num-ob-windows = <8>;
	211	+
	212	+ pinctrl-names = "default";
	213	+ pinctrl-0 = <&clkreq_c5_bi_dir_state>;
	214	+
	215	+ clocks = <&bpmp TEGRA194_CLK_PEX1_CORE_5>;
	216	+ clock-names = "core";
	217	+
	218	+ resets = <&bpmp TEGRA194_RESET_PEX1_CORE_5_APB>,
	219	+ <&bpmp TEGRA194_RESET_PEX1_CORE_5>;
	220	+ reset-names = "apb", "core";
	221	+
	222	+ interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>; /* controller interrupt */
	223	+ interrupt-names = "intr";
	224	+
	225	+ nvidia,bpmp = <&bpmp 5>;
	226	+
	227	+ nvidia,aspm-cmrt-us = <60>;
	228	+ nvidia,aspm-pwr-on-t-us = <20>;
	229	+ nvidia,aspm-l0s-entrance-latency-us = <3>;
	230	+
	231	+ vddio-pex-ctl-supply = <&vdd_1v8ao>;
	232	+
	233	+ reset-gpios = <&gpio TEGRA194_MAIN_GPIO(GG, 1) GPIO_ACTIVE_LOW>;
	234	+
	235	+ nvidia,refclk-select-gpios = <&gpio_aon TEGRA194_AON_GPIO(AA, 5)
	236	+ GPIO_ACTIVE_HIGH>;
	237	+
	238	+ phys = <&p2u_nvhs_0>, <&p2u_nvhs_1>, <&p2u_nvhs_2>,
	239	+ <&p2u_nvhs_3>, <&p2u_nvhs_4>, <&p2u_nvhs_5>,
	240	+ <&p2u_nvhs_6>, <&p2u_nvhs_7>;
	241	+
	242	+ phy-names = "p2u-0", "p2u-1", "p2u-2", "p2u-3", "p2u-4",
	243	+ "p2u-5", "p2u-6", "p2u-7";
	244	+ };

drivers/misc/pci_endpoint_test.c+179−34

@@ -17,6 +17,7 @@ · 16 unmodified lines
17	17	#include <linux/mutex.h>
18	18	#include <linux/random.h>
19	19	#include <linux/slab.h>
	20	+#include <linux/uaccess.h>
20	21	#include <linux/pci.h>
21	22	#include <linux/pci_ids.h>
22	23
@@ -64,6 +65,9 @@ · 41 unmodified lines
64	65	#define PCI_ENDPOINT_TEST_IRQ_TYPE 0x24
65	66	#define PCI_ENDPOINT_TEST_IRQ_NUMBER 0x28
66	67
	68	+#define PCI_ENDPOINT_TEST_FLAGS 0x2c
	69	+#define FLAG_USE_DMA BIT(0)
	70	+
67	71	#define PCI_DEVICE_ID_TI_AM654 0xb00c
68	72
69	73	#define is_am654_pci_dev(pdev) \
@@ -98,11 +102,13 @@ struct pci_endpoint_test { · 28 unmodified lines
98	102	struct completion irq_raised;
99	103	int last_irq;
100	104	int num_irqs;
	105	+ int irq_type;
101	106	/* mutex to protect the ioctls */
102	107	struct mutex mutex;
103	108	struct miscdevice miscdev;
104	109	enum pci_barno test_reg_bar;
105	110	size_t alignment;
	111	+ const char *name;
106	112	};
107	113
108	114	struct pci_endpoint_test_data {
@@ -157,6 +163,7 @@ static void pci_endpoint_test_free_irq_vectors(struct pci_endpoint_test *test) · 48 unmodified lines
157	163	struct pci_dev *pdev = test->pdev;
158	164
159	165	pci_free_irq_vectors(pdev);
	166	+ test->irq_type = IRQ_TYPE_UNDEFINED;
160	167	}
161	168
162	169	static bool pci_endpoint_test_alloc_irq_vectors(struct pci_endpoint_test *test,
@@ -191,6 +198,8 @@ static bool pci_endpoint_test_alloc_irq_vectors(struct pci_endpoint_test *test, · 28 unmodified lines
191	198	irq = 0;
192	199	res = false;
193	200	}
	201	+
	202	+ test->irq_type = type;
194	203	test->num_irqs = irq;
195	204
196	205	return res;
@@ -218,7 +227,7 @@ static bool pci_endpoint_test_request_irq(struct pci_endpoint_test *test) · 21 unmodified lines
218	227	for (i = 0; i < test->num_irqs; i++) {
219	228	err = devm_request_irq(dev, pci_irq_vector(pdev, i),
220	229	pci_endpoint_test_irqhandler,
221		− IRQF_SHARED, DRV_MODULE_NAME, test);
	230	+ IRQF_SHARED, test->name, test);
222	231	if (err)
223	232	goto fail;
224	233	}
@@ -315,11 +324,16 @@ static bool pci_endpoint_test_msi_irq(struct pci_endpoint_test *test, · 90 unmodified lines
315	324	return false;
316	325	}
317	326
318		−static bool pci_endpoint_test_copy(struct pci_endpoint_test *test, size_t size)
	327	+static bool pci_endpoint_test_copy(struct pci_endpoint_test *test,
	328	+ unsigned long arg)
319	329	{
	330	+ struct pci_endpoint_test_xfer_param param;
320	331	bool ret = false;
321	332	void *src_addr;
322	333	void *dst_addr;
	334	+ u32 flags = 0;
	335	+ bool use_dma;
	336	+ size_t size;
323	337	dma_addr_t src_phys_addr;
324	338	dma_addr_t dst_phys_addr;
325	339	struct pci_dev *pdev = test->pdev;
@@ -330,25 +344,46 @@ static bool pci_endpoint_test_copy(struct pci_endpoint_test *test, size_t size) · 4 unmodified lines
330	344	dma_addr_t orig_dst_phys_addr;
331	345	size_t offset;
332	346	size_t alignment = test->alignment;
	347	+ int irq_type = test->irq_type;
333	348	u32 src_crc32;
334	349	u32 dst_crc32;
	350	+ int err;
335	351
	352	+ err = copy_from_user(&param, (void __user *)arg, sizeof(param));
	353	+ if (err) {
	354	+ dev_err(dev, "Failed to get transfer param\n");
	355	+ return false;
	356	+ }
	357	+
	358	+ size = param.size;
336	359	if (size > SIZE_MAX - alignment)
337	360	goto err;
338	361
	362	+ use_dma = !!(param.flags & PCITEST_FLAGS_USE_DMA);
	363	+ if (use_dma)
	364	+ flags \|= FLAG_USE_DMA;
	365	+
339	366	if (irq_type < IRQ_TYPE_LEGACY \|\| irq_type > IRQ_TYPE_MSIX) {
340	367	dev_err(dev, "Invalid IRQ type option\n");
341	368	goto err;
342	369	}
343	370
344		− orig_src_addr = dma_alloc_coherent(dev, size + alignment,
345		− &orig_src_phys_addr, GFP_KERNEL);
	371	+ orig_src_addr = kzalloc(size + alignment, GFP_KERNEL);
346	372	if (!orig_src_addr) {
347	373	dev_err(dev, "Failed to allocate source buffer\n");
348	374	ret = false;
349	375	goto err;
350	376	}
351	377
	378	+ get_random_bytes(orig_src_addr, size + alignment);
	379	+ orig_src_phys_addr = dma_map_single(dev, orig_src_addr,
	380	+ size + alignment, DMA_TO_DEVICE);
	381	+ if (dma_mapping_error(dev, orig_src_phys_addr)) {
	382	+ dev_err(dev, "failed to map source buffer address\n");
	383	+ ret = false;
	384	+ goto err_src_phys_addr;
	385	+ }
	386	+
352	387	if (alignment && !IS_ALIGNED(orig_src_phys_addr, alignment)) {
353	388	src_phys_addr = PTR_ALIGN(orig_src_phys_addr, alignment);
354	389	offset = src_phys_addr - orig_src_phys_addr;
@@ -364,15 +399,21 @@ static bool pci_endpoint_test_copy(struct pci_endpoint_test *test, size_t size) · 9 unmodified lines
364	399	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_UPPER_SRC_ADDR,
365	400	upper_32_bits(src_phys_addr));
366	401
367		− get_random_bytes(src_addr, size);
368	402	src_crc32 = crc32_le(~0, src_addr, size);
369	403
370		− orig_dst_addr = dma_alloc_coherent(dev, size + alignment,
371		− &orig_dst_phys_addr, GFP_KERNEL);
	404	+ orig_dst_addr = kzalloc(size + alignment, GFP_KERNEL);
372	405	if (!orig_dst_addr) {
373	406	dev_err(dev, "Failed to allocate destination address\n");
374	407	ret = false;
375		− goto err_orig_src_addr;
	408	+ goto err_dst_addr;
	409	+ }
	410	+
	411	+ orig_dst_phys_addr = dma_map_single(dev, orig_dst_addr,
	412	+ size + alignment, DMA_FROM_DEVICE);
	413	+ if (dma_mapping_error(dev, orig_dst_phys_addr)) {
	414	+ dev_err(dev, "failed to map destination buffer address\n");
	415	+ ret = false;
	416	+ goto err_dst_phys_addr;
376	417	}
377	418
378	419	if (alignment && !IS_ALIGNED(orig_dst_phys_addr, alignment)) {
@@ -392,6 +433,7 @@ static bool pci_endpoint_test_copy(struct pci_endpoint_test *test, size_t size) · 13 unmodified lines
392	433	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_SIZE,
393	434	size);
394	435
	436	+ pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_FLAGS, flags);
395	437	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_TYPE, irq_type);
396	438	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_NUMBER, 1);
397	439	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_COMMAND,
@@ -399,24 +441,34 @@ static bool pci_endpoint_test_copy(struct pci_endpoint_test *test, size_t size) · 1 unmodified line
399	441
400	442	wait_for_completion(&test->irq_raised);
401	443
	444	+ dma_unmap_single(dev, orig_dst_phys_addr, size + alignment,
	445	+ DMA_FROM_DEVICE);
	446	+
402	447	dst_crc32 = crc32_le(~0, dst_addr, size);
403	448	if (dst_crc32 == src_crc32)
404	449	ret = true;
405	450
406		− dma_free_coherent(dev, size + alignment, orig_dst_addr,
407		− orig_dst_phys_addr);
	451	+err_dst_phys_addr:
	452	+ kfree(orig_dst_addr);
408	453
409		−err_orig_src_addr:
410		− dma_free_coherent(dev, size + alignment, orig_src_addr,
411		− orig_src_phys_addr);
	454	+err_dst_addr:
	455	+ dma_unmap_single(dev, orig_src_phys_addr, size + alignment,
	456	+ DMA_TO_DEVICE);
	457	+
	458	+err_src_phys_addr:
	459	+ kfree(orig_src_addr);
412	460
413	461	err:
414	462	return ret;
415	463	}
416	464
417		−static bool pci_endpoint_test_write(struct pci_endpoint_test *test, size_t size)
	465	+static bool pci_endpoint_test_write(struct pci_endpoint_test *test,
	466	+ unsigned long arg)
418	467	{
	468	+ struct pci_endpoint_test_xfer_param param;
419	469	bool ret = false;
	470	+ u32 flags = 0;
	471	+ bool use_dma;
420	472	u32 reg;
421	473	void *addr;
422	474	dma_addr_t phys_addr;
@@ -426,24 +478,47 @@ static bool pci_endpoint_test_write(struct pci_endpoint_test *test, size_t size) · 3 unmodified lines
426	478	dma_addr_t orig_phys_addr;
427	479	size_t offset;
428	480	size_t alignment = test->alignment;
	481	+ int irq_type = test->irq_type;
	482	+ size_t size;
429	483	u32 crc32;
	484	+ int err;
	485	+
	486	+ err = copy_from_user(&param, (void __user *)arg, sizeof(param));
	487	+ if (err != 0) {
	488	+ dev_err(dev, "Failed to get transfer param\n");
	489	+ return false;
	490	+ }
430	491
	492	+ size = param.size;
431	493	if (size > SIZE_MAX - alignment)
432	494	goto err;
433	495
	496	+ use_dma = !!(param.flags & PCITEST_FLAGS_USE_DMA);
	497	+ if (use_dma)
	498	+ flags \|= FLAG_USE_DMA;
	499	+
434	500	if (irq_type < IRQ_TYPE_LEGACY \|\| irq_type > IRQ_TYPE_MSIX) {
435	501	dev_err(dev, "Invalid IRQ type option\n");
436	502	goto err;
437	503	}
438	504
439		− orig_addr = dma_alloc_coherent(dev, size + alignment, &orig_phys_addr,
440		− GFP_KERNEL);
	505	+ orig_addr = kzalloc(size + alignment, GFP_KERNEL);
441	506	if (!orig_addr) {
442	507	dev_err(dev, "Failed to allocate address\n");
443	508	ret = false;
444	509	goto err;
445	510	}
446	511
	512	+ get_random_bytes(orig_addr, size + alignment);
	513	+
	514	+ orig_phys_addr = dma_map_single(dev, orig_addr, size + alignment,
	515	+ DMA_TO_DEVICE);
	516	+ if (dma_mapping_error(dev, orig_phys_addr)) {
	517	+ dev_err(dev, "failed to map source buffer address\n");
	518	+ ret = false;
	519	+ goto err_phys_addr;
	520	+ }
	521	+
447	522	if (alignment && !IS_ALIGNED(orig_phys_addr, alignment)) {
448	523	phys_addr = PTR_ALIGN(orig_phys_addr, alignment);
449	524	offset = phys_addr - orig_phys_addr;
@@ -453,8 +528,6 @@ static bool pci_endpoint_test_write(struct pci_endpoint_test *test, size_t size) · 3 unmodified lines
453	528	addr = orig_addr;
454	529	}
455	530
456		− get_random_bytes(addr, size);
457		−
458	531	crc32 = crc32_le(~0, addr, size);
459	532	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_CHECKSUM,
460	533	crc32);
@@ -466,6 +539,7 @@ static bool pci_endpoint_test_write(struct pci_endpoint_test *test, size_t size) · 5 unmodified lines
466	539
467	540	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_SIZE, size);
468	541
	542	+ pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_FLAGS, flags);
469	543	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_TYPE, irq_type);
470	544	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_NUMBER, 1);
471	545	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_COMMAND,
@@ -477,15 +551,24 @@ static bool pci_endpoint_test_write(struct pci_endpoint_test *test, size_t size) · 5 unmodified lines
477	551	if (reg & STATUS_READ_SUCCESS)
478	552	ret = true;
479	553
480		− dma_free_coherent(dev, size + alignment, orig_addr, orig_phys_addr);
	554	+ dma_unmap_single(dev, orig_phys_addr, size + alignment,
	555	+ DMA_TO_DEVICE);
	556	+
	557	+err_phys_addr:
	558	+ kfree(orig_addr);
481	559
482	560	err:
483	561	return ret;
484	562	}
485	563
486		−static bool pci_endpoint_test_read(struct pci_endpoint_test *test, size_t size)
	564	+static bool pci_endpoint_test_read(struct pci_endpoint_test *test,
	565	+ unsigned long arg)
487	566	{
	567	+ struct pci_endpoint_test_xfer_param param;
488	568	bool ret = false;
	569	+ u32 flags = 0;
	570	+ bool use_dma;
	571	+ size_t size;
489	572	void *addr;
490	573	dma_addr_t phys_addr;
491	574	struct pci_dev *pdev = test->pdev;
@@ -494,24 +577,44 @@ static bool pci_endpoint_test_read(struct pci_endpoint_test *test, size_t size) · 2 unmodified lines
494	577	dma_addr_t orig_phys_addr;
495	578	size_t offset;
496	579	size_t alignment = test->alignment;
	580	+ int irq_type = test->irq_type;
497	581	u32 crc32;
	582	+ int err;
	583	+
	584	+ err = copy_from_user(&param, (void __user *)arg, sizeof(param));
	585	+ if (err) {
	586	+ dev_err(dev, "Failed to get transfer param\n");
	587	+ return false;
	588	+ }
498	589
	590	+ size = param.size;
499	591	if (size > SIZE_MAX - alignment)
500	592	goto err;
501	593
	594	+ use_dma = !!(param.flags & PCITEST_FLAGS_USE_DMA);
	595	+ if (use_dma)
	596	+ flags \|= FLAG_USE_DMA;
	597	+
502	598	if (irq_type < IRQ_TYPE_LEGACY \|\| irq_type > IRQ_TYPE_MSIX) {
503	599	dev_err(dev, "Invalid IRQ type option\n");
504	600	goto err;
505	601	}
506	602
507		− orig_addr = dma_alloc_coherent(dev, size + alignment, &orig_phys_addr,
508		− GFP_KERNEL);
	603	+ orig_addr = kzalloc(size + alignment, GFP_KERNEL);
509	604	if (!orig_addr) {
510	605	dev_err(dev, "Failed to allocate destination address\n");
511	606	ret = false;
512	607	goto err;
513	608	}
514	609
	610	+ orig_phys_addr = dma_map_single(dev, orig_addr, size + alignment,
	611	+ DMA_FROM_DEVICE);
	612	+ if (dma_mapping_error(dev, orig_phys_addr)) {
	613	+ dev_err(dev, "failed to map source buffer address\n");
	614	+ ret = false;
	615	+ goto err_phys_addr;
	616	+ }
	617	+
515	618	if (alignment && !IS_ALIGNED(orig_phys_addr, alignment)) {
516	619	phys_addr = PTR_ALIGN(orig_phys_addr, alignment);
517	620	offset = phys_addr - orig_phys_addr;
@@ -528,6 +631,7 @@ static bool pci_endpoint_test_read(struct pci_endpoint_test *test, size_t size) · 10 unmodified lines
528	631
529	632	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_SIZE, size);
530	633
	634	+ pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_FLAGS, flags);
531	635	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_TYPE, irq_type);
532	636	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_NUMBER, 1);
533	637	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_COMMAND,
@@ -535,15 +639,26 @@ static bool pci_endpoint_test_read(struct pci_endpoint_test *test, size_t size) · 1 unmodified line
535	639
536	640	wait_for_completion(&test->irq_raised);
537	641
	642	+ dma_unmap_single(dev, orig_phys_addr, size + alignment,
	643	+ DMA_FROM_DEVICE);
	644	+
538	645	crc32 = crc32_le(~0, addr, size);
539	646	if (crc32 == pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_CHECKSUM))
540	647	ret = true;
541	648
542		− dma_free_coherent(dev, size + alignment, orig_addr, orig_phys_addr);
	649	+err_phys_addr:
	650	+ kfree(orig_addr);
543	651	err:
544	652	return ret;
545	653	}
546	654
	655	+static bool pci_endpoint_test_clear_irq(struct pci_endpoint_test *test)
	656	+{
	657	+ pci_endpoint_test_release_irq(test);
	658	+ pci_endpoint_test_free_irq_vectors(test);
	659	+ return true;
	660	+}
	661	+
547	662	static bool pci_endpoint_test_set_irq(struct pci_endpoint_test *test,
548	663	int req_irq_type)
549	664	{
@@ -555,7 +670,7 @@ static bool pci_endpoint_test_set_irq(struct pci_endpoint_test *test, · 5 unmodified lines
555	670	return false;
556	671	}
557	672
558		− if (irq_type == req_irq_type)
	673	+ if (test->irq_type == req_irq_type)
559	674	return true;
560	675
561	676	pci_endpoint_test_release_irq(test);
@@ -567,12 +682,10 @@ static bool pci_endpoint_test_set_irq(struct pci_endpoint_test *test, · 5 unmodified lines
567	682	if (!pci_endpoint_test_request_irq(test))
568	683	goto err;
569	684
570		− irq_type = req_irq_type;
571	685	return true;
572	686
573	687	err:
574	688	pci_endpoint_test_free_irq_vectors(test);
575		− irq_type = IRQ_TYPE_UNDEFINED;
576	689	return false;
577	690	}
578	691
@@ -616,6 +729,9 @@ static long pci_endpoint_test_ioctl(struct file *file, unsigned int cmd, · 37 unmodified lines
616	729	case PCITEST_GET_IRQTYPE:
617	730	ret = irq_type;
618	731	break;
	732	+ case PCITEST_CLEAR_IRQ:
	733	+ ret = pci_endpoint_test_clear_irq(test);
	734	+ break;
619	735	}
620	736
621	737	ret:
@@ -633,7 +749,7 @@ static int pci_endpoint_test_probe(struct pci_dev *pdev, · 11 unmodified lines
633	749	{
634	750	int err;
635	751	int id;
636		− char name[20];
	752	+ char name[24];
637	753	enum pci_barno bar;
638	754	void __iomem *base;
639	755	struct device *dev = &pdev->dev;
@@ -652,6 +768,7 @@ static int pci_endpoint_test_probe(struct pci_dev *pdev, · 12 unmodified lines
652	768	test->test_reg_bar = 0;
653	769	test->alignment = 0;
654	770	test->pdev = pdev;
	771	+ test->irq_type = IRQ_TYPE_UNDEFINED;
655	772
656	773	if (no_msi)
657	774	irq_type = IRQ_TYPE_LEGACY;
@@ -667,6 +784,12 @@ static int pci_endpoint_test_probe(struct pci_dev *pdev, · 9 unmodified lines
667	784	init_completion(&test->irq_raised);
668	785	mutex_init(&test->mutex);
669	786
	787	+ if ((dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(48)) != 0) &&
	788	+ dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)) != 0) {
	789	+ dev_err(dev, "Cannot set DMA mask\n");
	790	+ return -EINVAL;
	791	+ }
	792	+
670	793	err = pci_enable_device(pdev);
671	794	if (err) {
672	795	dev_err(dev, "Cannot enable PCI device\n");
@@ -684,9 +807,6 @@ static int pci_endpoint_test_probe(struct pci_dev *pdev, · 11 unmodified lines
684	807	if (!pci_endpoint_test_alloc_irq_vectors(test, irq_type))
685	808	goto err_disable_irq;
686	809
687		− if (!pci_endpoint_test_request_irq(test))
688		− goto err_disable_irq;
689		−
690	810	for (bar = 0; bar < PCI_STD_NUM_BARS; bar++) {
691	811	if (pci_resource_flags(pdev, bar) & IORESOURCE_MEM) {
692	812	base = pci_ioremap_bar(pdev, bar);
@@ -716,12 +836,21 @@ static int pci_endpoint_test_probe(struct pci_dev *pdev, · 23 unmodified lines
716	836	}
717	837
718	838	snprintf(name, sizeof(name), DRV_MODULE_NAME ".%d", id);
	839	+ test->name = kstrdup(name, GFP_KERNEL);
	840	+ if (!test->name) {
	841	+ err = -ENOMEM;
	842	+ goto err_ida_remove;
	843	+ }
	844	+
	845	+ if (!pci_endpoint_test_request_irq(test))
	846	+ goto err_kfree_test_name;
	847	+
719	848	misc_device = &test->miscdev;
720	849	misc_device->minor = MISC_DYNAMIC_MINOR;
721	850	misc_device->name = kstrdup(name, GFP_KERNEL);
722	851	if (!misc_device->name) {
723	852	err = -ENOMEM;
724		− goto err_ida_remove;
	853	+ goto err_release_irq;
725	854	}
726	855	misc_device->fops = &pci_endpoint_test_fops,
727	856
@@ -736,6 +865,12 @@ static int pci_endpoint_test_probe(struct pci_dev *pdev, · 8 unmodified lines
736	865	err_kfree_name:
737	866	kfree(misc_device->name);
738	867
	868	+err_release_irq:
	869	+ pci_endpoint_test_release_irq(test);
	870	+
	871	+err_kfree_test_name:
	872	+ kfree(test->name);
	873	+
739	874	err_ida_remove:
740	875	ida_simple_remove(&pci_endpoint_test_ida, id);
741	876
@@ -744,7 +879,6 @@ err_iounmap: · 2 unmodified lines
744	879	if (test->bar[bar])
745	880	pci_iounmap(pdev, test->bar[bar]);
746	881	}
747		− pci_endpoint_test_release_irq(test);
748	882
749	883	err_disable_irq:
750	884	pci_endpoint_test_free_irq_vectors(test);
@@ -770,6 +904,7 @@ static void pci_endpoint_test_remove(struct pci_dev *pdev) · 19 unmodified lines
770	904
771	905	misc_deregister(&test->miscdev);
772	906	kfree(misc_device->name);
	907	+ kfree(test->name);
773	908	ida_simple_remove(&pci_endpoint_test_ida, id);
774	909	for (bar = 0; bar < PCI_STD_NUM_BARS; bar++) {
775	910	if (test->bar[bar])
@@ -783,6 +918,12 @@ static void pci_endpoint_test_remove(struct pci_dev *pdev) · 7 unmodified lines
783	918	pci_disable_device(pdev);
784	919	}
785	920
	921	+static const struct pci_endpoint_test_data default_data = {
	922	+ .test_reg_bar = BAR_0,
	923	+ .alignment = SZ_4K,
	924	+ .irq_type = IRQ_TYPE_MSI,
	925	+};
	926	+
786	927	static const struct pci_endpoint_test_data am654_data = {
787	928	.test_reg_bar = BAR_2,
788	929	.alignment = SZ_64K,
@@ -790,8 +931,12 @@ static const struct pci_endpoint_test_data am654_data = { · 1 unmodified line
790	931	};
791	932
792	933	static const struct pci_device_id pci_endpoint_test_tbl[] = {
793		− { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_DRA74x) },
794		− { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_DRA72x) },
	934	+ { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_DRA74x),
	935	+ .driver_data = (kernel_ulong_t)&default_data,
	936	+ },
	937	+ { PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_DRA72x),
	938	+ .driver_data = (kernel_ulong_t)&default_data,
	939	+ },
795	940	{ PCI_DEVICE(PCI_VENDOR_ID_FREESCALE, 0x81c0) },
796	941	{ PCI_DEVICE_DATA(SYNOPSYS, EDDA, NULL) },
797	942	{ PCI_DEVICE(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_AM654),

drivers/pci/controller/dwc/Kconfig+26−3

@@ -248,14 +248,37 @@ config PCI_MESON · 247 unmodified lines
248	248	implement the driver.
249	249
250	250	config PCIE_TEGRA194
251		− tristate "NVIDIA Tegra194 (and later) PCIe controller"
	251	+ tristate
	252	+
	253	+config PCIE_TEGRA194_HOST
	254	+ tristate "NVIDIA Tegra194 (and later) PCIe controller - Host Mode"
252	255	depends on ARCH_TEGRA_194_SOC \|\| COMPILE_TEST
253	256	depends on PCI_MSI_IRQ_DOMAIN
254	257	select PCIE_DW_HOST
255	258	select PHY_TEGRA194_P2U
	259	+ select PCIE_TEGRA194
	260	+ help
	261	+ Enables support for the PCIe controller in the NVIDIA Tegra194 SoC to
	262	+ work in host mode. There are two instances of PCIe controllers in
	263	+ Tegra194. This controller can work either as EP or RC. In order to
	264	+ enable host-specific features PCIE_TEGRA194_HOST must be selected and
	265	+ in order to enable device-specific features PCIE_TEGRA194_EP must be
	266	+ selected. This uses the DesignWare core.
	267	+
	268	+config PCIE_TEGRA194_EP
	269	+ tristate "NVIDIA Tegra194 (and later) PCIe controller - Endpoint Mode"
	270	+ depends on ARCH_TEGRA_194_SOC \|\| COMPILE_TEST
	271	+ depends on PCI_ENDPOINT
	272	+ select PCIE_DW_EP
	273	+ select PHY_TEGRA194_P2U
	274	+ select PCIE_TEGRA194
256	275	help
257		− Say Y here if you want support for DesignWare core based PCIe host
258		− controller found in NVIDIA Tegra194 SoC.
	276	+ Enables support for the PCIe controller in the NVIDIA Tegra194 SoC to
	277	+ work in host mode. There are two instances of PCIe controllers in
	278	+ Tegra194. This controller can work either as EP or RC. In order to
	279	+ enable host-specific features PCIE_TEGRA194_HOST must be selected and
	280	+ in order to enable device-specific features PCIE_TEGRA194_EP must be
	281	+ selected. This uses the DesignWare core.
259	282
260	283	config PCIE_UNIPHIER
261	284	bool "Socionext UniPhier PCIe controllers"

drivers/pci/controller/dwc/pci-keystone.c+4−1

@@ -959,6 +959,9 @@ static int ks_pcie_am654_raise_irq(struct dw_pcie_ep *ep, u8 func_no, · 958 unmodified lines
959	959	case PCI_EPC_IRQ_MSI:
960	960	dw_pcie_ep_raise_msi_irq(ep, func_no, interrupt_num);
961	961	break;
	962	+ case PCI_EPC_IRQ_MSIX:
	963	+ dw_pcie_ep_raise_msix_irq(ep, func_no, interrupt_num);
	964	+ break;
962	965	default:
963	966	dev_err(pci->dev, "UNKNOWN IRQ type\n");
964	967	return -EINVAL;
@@ -970,7 +973,7 @@ static int ks_pcie_am654_raise_irq(struct dw_pcie_ep *ep, u8 func_no, · 5 unmodified lines
970	973	static const struct pci_epc_features ks_pcie_am654_epc_features = {
971	974	.linkup_notifier = false,
972	975	.msi_capable = true,
973		− .msix_capable = false,
	976	+ .msix_capable = true,
974	977	.reserved_bar = 1 << BAR_0 \| 1 << BAR_1,
975	978	.bar_fixed_64bit = 1 << BAR_0,
976	979	.bar_fixed_size[2] = SZ_1M,

drivers/pci/controller/dwc/pcie-designware-ep.c+85−59

@@ -18,6 +18,15 @@ void dw_pcie_ep_linkup(struct dw_pcie_ep *ep) · 17 unmodified lines
18	18
19	19	pci_epc_linkup(epc);
20	20	}
	21	+EXPORT_SYMBOL_GPL(dw_pcie_ep_linkup);
	22	+
	23	+void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep)
	24	+{
	25	+ struct pci_epc *epc = ep->epc;
	26	+
	27	+ pci_epc_init_notify(epc);
	28	+}
	29	+EXPORT_SYMBOL_GPL(dw_pcie_ep_init_notify);
21	30
22	31	static void __dw_pcie_ep_reset_bar(struct dw_pcie *pci, enum pci_barno bar,
23	32	int flags)
@@ -125,6 +134,7 @@ static void dw_pcie_ep_clear_bar(struct pci_epc *epc, u8 func_no, · 101 unmodified lines
125	134
126	135	dw_pcie_disable_atu(pci, atu_index, DW_PCIE_REGION_INBOUND);
127	136	clear_bit(atu_index, ep->ib_window_map);
	137	+ ep->epf_bar[bar] = NULL;
128	138	}
129	139
130	140	static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no,
@@ -158,6 +168,7 @@ static int dw_pcie_ep_set_bar(struct pci_epc *epc, u8 func_no, · 27 unmodified lines
158	168	dw_pcie_writel_dbi(pci, reg + 4, 0);
159	169	}
160	170
	171	+ ep->epf_bar[bar] = epf_bar;
161	172	dw_pcie_dbi_ro_wr_dis(pci);
162	173
163	174	return 0;
@@ -269,7 +280,8 @@ static int dw_pcie_ep_get_msix(struct pci_epc *epc, u8 func_no) · 105 unmodified lines
269	280	return val;
270	281	}
271	282
272		−static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts)
	283	+static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts,
	284	+ enum pci_barno bir, u32 offset)
273	285	{
274	286	struct dw_pcie_ep *ep = epc_get_drvdata(epc);
275	287	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
@@ -278,12 +290,22 @@ static int dw_pcie_ep_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts) · 2 unmodified lines
278	290	if (!ep->msix_cap)
279	291	return -EINVAL;
280	292
	293	+ dw_pcie_dbi_ro_wr_en(pci);
	294	+
281	295	reg = ep->msix_cap + PCI_MSIX_FLAGS;
282	296	val = dw_pcie_readw_dbi(pci, reg);
283	297	val &= ~PCI_MSIX_FLAGS_QSIZE;
284	298	val \|= interrupts;
285		− dw_pcie_dbi_ro_wr_en(pci);
286	299	dw_pcie_writew_dbi(pci, reg, val);
	300	+
	301	+ reg = ep->msix_cap + PCI_MSIX_TABLE;
	302	+ val = offset \| bir;
	303	+ dw_pcie_writel_dbi(pci, reg, val);
	304	+
	305	+ reg = ep->msix_cap + PCI_MSIX_PBA;
	306	+ val = (offset + (interrupts * PCI_MSIX_ENTRY_SIZE)) \| bir;
	307	+ dw_pcie_writel_dbi(pci, reg, val);
	308	+
287	309	dw_pcie_dbi_ro_wr_dis(pci);
288	310
289	311	return 0;
@@ -409,55 +431,41 @@ int dw_pcie_ep_raise_msix_irq(struct dw_pcie_ep *ep, u8 func_no, · 119 unmodified lines
409	431	u16 interrupt_num)
410	432	{
411	433	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
	434	+ struct pci_epf_msix_tbl *msix_tbl;
412	435	struct pci_epc *epc = ep->epc;
413		− u16 tbl_offset, bir;
414		− u32 bar_addr_upper, bar_addr_lower;
415		− u32 msg_addr_upper, msg_addr_lower;
	436	+ struct pci_epf_bar *epf_bar;
416	437	u32 reg, msg_data, vec_ctrl;
417		− u64 tbl_addr, msg_addr, reg_u64;
418		− void __iomem *msix_tbl;
	438	+ unsigned int aligned_offset;
	439	+ u32 tbl_offset;
	440	+ u64 msg_addr;
419	441	int ret;
	442	+ u8 bir;
420	443
421	444	reg = ep->msix_cap + PCI_MSIX_TABLE;
422	445	tbl_offset = dw_pcie_readl_dbi(pci, reg);
423	446	bir = (tbl_offset & PCI_MSIX_TABLE_BIR);
424	447	tbl_offset &= PCI_MSIX_TABLE_OFFSET;
425	448
426		− reg = PCI_BASE_ADDRESS_0 + (4 * bir);
427		− bar_addr_upper = 0;
428		− bar_addr_lower = dw_pcie_readl_dbi(pci, reg);
429		− reg_u64 = (bar_addr_lower & PCI_BASE_ADDRESS_MEM_TYPE_MASK);
430		− if (reg_u64 == PCI_BASE_ADDRESS_MEM_TYPE_64)
431		− bar_addr_upper = dw_pcie_readl_dbi(pci, reg + 4);
432		−
433		− tbl_addr = ((u64) bar_addr_upper) << 32 \| bar_addr_lower;
434		− tbl_addr += (tbl_offset + ((interrupt_num - 1) * PCI_MSIX_ENTRY_SIZE));
435		− tbl_addr &= PCI_BASE_ADDRESS_MEM_MASK;
436		−
437		− msix_tbl = ioremap(ep->phys_base + tbl_addr,
438		− PCI_MSIX_ENTRY_SIZE);
439		− if (!msix_tbl)
440		− return -EINVAL;
441		−
442		− msg_addr_lower = readl(msix_tbl + PCI_MSIX_ENTRY_LOWER_ADDR);
443		− msg_addr_upper = readl(msix_tbl + PCI_MSIX_ENTRY_UPPER_ADDR);
444		− msg_addr = ((u64) msg_addr_upper) << 32 \| msg_addr_lower;
445		− msg_data = readl(msix_tbl + PCI_MSIX_ENTRY_DATA);
446		− vec_ctrl = readl(msix_tbl + PCI_MSIX_ENTRY_VECTOR_CTRL);
	449	+ epf_bar = ep->epf_bar[bir];
	450	+ msix_tbl = epf_bar->addr;
	451	+ msix_tbl = (struct pci_epf_msix_tbl )((char )msix_tbl + tbl_offset);
447	452
448		− iounmap(msix_tbl);
	453	+ msg_addr = msix_tbl[(interrupt_num - 1)].msg_addr;
	454	+ msg_data = msix_tbl[(interrupt_num - 1)].msg_data;
	455	+ vec_ctrl = msix_tbl[(interrupt_num - 1)].vector_ctrl;
449	456
450	457	if (vec_ctrl & PCI_MSIX_ENTRY_CTRL_MASKBIT) {
451	458	dev_dbg(pci->dev, "MSI-X entry ctrl set\n");
452	459	return -EPERM;
453	460	}
454	461
455		− ret = dw_pcie_ep_map_addr(epc, func_no, ep->msi_mem_phys, msg_addr,
	462	+ aligned_offset = msg_addr & (epc->mem->page_size - 1);
	463	+ ret = dw_pcie_ep_map_addr(epc, func_no, ep->msi_mem_phys, msg_addr,
456	464	epc->mem->page_size);
457	465	if (ret)
458	466	return ret;
459	467
460		− writel(msg_data, ep->msi_mem);
	468	+ writel(msg_data, ep->msi_mem + aligned_offset);
461	469
462	470	dw_pcie_ep_unmap_addr(epc, func_no, ep->msi_mem_phys);
463	471
@@ -492,19 +500,54 @@ static unsigned int dw_pcie_ep_find_ext_capability(struct dw_pcie *pci, int cap) · 28 unmodified lines
492	500	return 0;
493	501	}
494	502
495		−int dw_pcie_ep_init(struct dw_pcie_ep *ep)
	503	+int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep)
496	504	{
	505	+ struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
	506	+ unsigned int offset;
	507	+ unsigned int nbars;
	508	+ u8 hdr_type;
	509	+ u32 reg;
497	510	int i;
	511	+
	512	+ hdr_type = dw_pcie_readb_dbi(pci, PCI_HEADER_TYPE);
	513	+ if (hdr_type != PCI_HEADER_TYPE_NORMAL) {
	514	+ dev_err(pci->dev,
	515	+ "PCIe controller is not set to EP mode (hdr_type:0x%x)!\n",
	516	+ hdr_type);
	517	+ return -EIO;
	518	+ }
	519	+
	520	+ ep->msi_cap = dw_pcie_find_capability(pci, PCI_CAP_ID_MSI);
	521	+
	522	+ ep->msix_cap = dw_pcie_find_capability(pci, PCI_CAP_ID_MSIX);
	523	+
	524	+ offset = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR);
	525	+ if (offset) {
	526	+ reg = dw_pcie_readl_dbi(pci, offset + PCI_REBAR_CTRL);
	527	+ nbars = (reg & PCI_REBAR_CTRL_NBAR_MASK) >>
	528	+ PCI_REBAR_CTRL_NBAR_SHIFT;
	529	+
	530	+ dw_pcie_dbi_ro_wr_en(pci);
	531	+ for (i = 0; i < nbars; i++, offset += PCI_REBAR_CTRL)
	532	+ dw_pcie_writel_dbi(pci, offset + PCI_REBAR_CAP, 0x0);
	533	+ dw_pcie_dbi_ro_wr_dis(pci);
	534	+ }
	535	+
	536	+ dw_pcie_setup(pci);
	537	+
	538	+ return 0;
	539	+}
	540	+EXPORT_SYMBOL_GPL(dw_pcie_ep_init_complete);
	541	+
	542	+int dw_pcie_ep_init(struct dw_pcie_ep *ep)
	543	+{
498	544	int ret;
499		− u32 reg;
500	545	void *addr;
501		− u8 hdr_type;
502		− unsigned int nbars;
503		− unsigned int offset;
504	546	struct pci_epc *epc;
505	547	struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
506	548	struct device *dev = pci->dev;
507	549	struct device_node *np = dev->of_node;
	550	+ const struct pci_epc_features *epc_features;
508	551
509	552	if (!pci->dbi_base \|\| !pci->dbi_base2) {
510	553	dev_err(dev, "dbi_base/dbi_base2 is not populated\n");
@@ -563,13 +606,6 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) · 52 unmodified lines
563	606	if (ep->ops->ep_init)
564	607	ep->ops->ep_init(ep);
565	608
566		− hdr_type = dw_pcie_readb_dbi(pci, PCI_HEADER_TYPE);
567		− if (hdr_type != PCI_HEADER_TYPE_NORMAL) {
568		− dev_err(pci->dev, "PCIe controller is not set to EP mode (hdr_type:0x%x)!\n",
569		− hdr_type);
570		− return -EIO;
571		− }
572		−
573	609	ret = of_property_read_u8(np, "max-functions", &epc->max_functions);
574	610	if (ret < 0)
575	611	epc->max_functions = 1;
@@ -587,23 +623,13 @@ int dw_pcie_ep_init(struct dw_pcie_ep *ep) · 11 unmodified lines
587	623	dev_err(dev, "Failed to reserve memory for MSI/MSI-X\n");
588	624	return -ENOMEM;
589	625	}
590		− ep->msi_cap = dw_pcie_find_capability(pci, PCI_CAP_ID_MSI);
591		−
592		− ep->msix_cap = dw_pcie_find_capability(pci, PCI_CAP_ID_MSIX);
593		−
594		− offset = dw_pcie_ep_find_ext_capability(pci, PCI_EXT_CAP_ID_REBAR);
595		− if (offset) {
596		− reg = dw_pcie_readl_dbi(pci, offset + PCI_REBAR_CTRL);
597		− nbars = (reg & PCI_REBAR_CTRL_NBAR_MASK) >>
598		− PCI_REBAR_CTRL_NBAR_SHIFT;
599	626
600		− dw_pcie_dbi_ro_wr_en(pci);
601		− for (i = 0; i < nbars; i++, offset += PCI_REBAR_CTRL)
602		− dw_pcie_writel_dbi(pci, offset + PCI_REBAR_CAP, 0x0);
603		− dw_pcie_dbi_ro_wr_dis(pci);
	627	+ if (ep->ops->get_features) {
	628	+ epc_features = ep->ops->get_features(ep);
	629	+ if (epc_features->core_init_notifier)
	630	+ return 0;
604	631	}
605	632
606		− dw_pcie_setup(pci);
607		−
608		− return 0;
	633	+ return dw_pcie_ep_init_complete(ep);
609	634	}
	635	+EXPORT_SYMBOL_GPL(dw_pcie_ep_init);

drivers/pci/controller/dwc/pcie-designware.h+12−0

@@ -233,6 +233,7 @@ struct dw_pcie_ep { · 232 unmodified lines
233	233	phys_addr_t msi_mem_phys;
234	234	u8 msi_cap; /* MSI capability offset */
235	235	u8 msix_cap; /* MSI-X capability offset */
	236	+ struct pci_epf_bar *epf_bar[PCI_STD_NUM_BARS];
236	237	};
237	238
238	239	struct dw_pcie_ops {
@@ -411,6 +412,8 @@ static inline int dw_pcie_allocate_domains(struct pcie_port *pp) · 172 unmodified lines
411	412	#ifdef CONFIG_PCIE_DW_EP
412	413	void dw_pcie_ep_linkup(struct dw_pcie_ep *ep);
413	414	int dw_pcie_ep_init(struct dw_pcie_ep *ep);
	415	+int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep);
	416	+void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep);
414	417	void dw_pcie_ep_exit(struct dw_pcie_ep *ep);
415	418	int dw_pcie_ep_raise_legacy_irq(struct dw_pcie_ep *ep, u8 func_no);
416	419	int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, u8 func_no,
@@ -428,6 +431,15 @@ static inline int dw_pcie_ep_init(struct dw_pcie_ep *ep) · 11 unmodified lines
428	431	return 0;
429	432	}
430	433
	434	+static inline int dw_pcie_ep_init_complete(struct dw_pcie_ep *ep)
	435	+{
	436	+ return 0;
	437	+}
	438	+
	439	+static inline void dw_pcie_ep_init_notify(struct dw_pcie_ep *ep)
	440	+{
	441	+}
	442	+
431	443	static inline void dw_pcie_ep_exit(struct dw_pcie_ep *ep)
432	444	{
433	445	}

drivers/pci/controller/dwc/pcie-tegra194.c+695−17

@@ -11,6 +11,7 @@ · 10 unmodified lines
11	11	#include <linux/debugfs.h>
12	12	#include <linux/delay.h>
13	13	#include <linux/gpio.h>
	14	+#include <linux/gpio/consumer.h>
14	15	#include <linux/interrupt.h>
15	16	#include <linux/iopoll.h>
16	17	#include <linux/kernel.h>
@@ -53,6 +54,7 @@ · 36 unmodified lines
53	54	#define APPL_INTR_EN_L0_0_LINK_STATE_INT_EN BIT(0)
54	55	#define APPL_INTR_EN_L0_0_MSI_RCV_INT_EN BIT(4)
55	56	#define APPL_INTR_EN_L0_0_INT_INT_EN BIT(8)
	57	+#define APPL_INTR_EN_L0_0_PCI_CMD_EN_INT_EN BIT(15)
56	58	#define APPL_INTR_EN_L0_0_CDM_REG_CHK_INT_EN BIT(19)
57	59	#define APPL_INTR_EN_L0_0_SYS_INTR_EN BIT(30)
58	60	#define APPL_INTR_EN_L0_0_SYS_MSI_INTR_EN BIT(31)
@@ -60,19 +62,26 @@ · 1 unmodified line
60	62	#define APPL_INTR_STATUS_L0 0xC
61	63	#define APPL_INTR_STATUS_L0_LINK_STATE_INT BIT(0)
62	64	#define APPL_INTR_STATUS_L0_INT_INT BIT(8)
	65	+#define APPL_INTR_STATUS_L0_PCI_CMD_EN_INT BIT(15)
	66	+#define APPL_INTR_STATUS_L0_PEX_RST_INT BIT(16)
63	67	#define APPL_INTR_STATUS_L0_CDM_REG_CHK_INT BIT(18)
64	68
65	69	#define APPL_INTR_EN_L1_0_0 0x1C
66	70	#define APPL_INTR_EN_L1_0_0_LINK_REQ_RST_NOT_INT_EN BIT(1)
	71	+#define APPL_INTR_EN_L1_0_0_RDLH_LINK_UP_INT_EN BIT(3)
	72	+#define APPL_INTR_EN_L1_0_0_HOT_RESET_DONE_INT_EN BIT(30)
67	73
68	74	#define APPL_INTR_STATUS_L1_0_0 0x20
69	75	#define APPL_INTR_STATUS_L1_0_0_LINK_REQ_RST_NOT_CHGED BIT(1)
	76	+#define APPL_INTR_STATUS_L1_0_0_RDLH_LINK_UP_CHGED BIT(3)
	77	+#define APPL_INTR_STATUS_L1_0_0_HOT_RESET_DONE BIT(30)
70	78
71	79	#define APPL_INTR_STATUS_L1_1 0x2C
72	80	#define APPL_INTR_STATUS_L1_2 0x30
73	81	#define APPL_INTR_STATUS_L1_3 0x34
74	82	#define APPL_INTR_STATUS_L1_6 0x3C
75	83	#define APPL_INTR_STATUS_L1_7 0x40
	84	+#define APPL_INTR_STATUS_L1_15_CFG_BME_CHGED BIT(1)
76	85
77	86	#define APPL_INTR_EN_L1_8_0 0x44
78	87	#define APPL_INTR_EN_L1_8_BW_MGT_INT_EN BIT(2)
@@ -103,8 +112,12 @@ · 24 unmodified lines
103	112	#define APPL_INTR_STATUS_L1_18_CDM_REG_CHK_CMP_ERR BIT(1)
104	113	#define APPL_INTR_STATUS_L1_18_CDM_REG_CHK_LOGIC_ERR BIT(0)
105	114
	115	+#define APPL_MSI_CTRL_1 0xAC
	116	+
106	117	#define APPL_MSI_CTRL_2 0xB0
107	118
	119	+#define APPL_LEGACY_INTX 0xB8
	120	+
108	121	#define APPL_LTR_MSG_1 0xC4
109	122	#define LTR_MSG_REQ BIT(15)
110	123	#define LTR_MST_NO_SNOOP_SHIFT 16
@@ -205,6 +218,13 @@ · 94 unmodified lines
205	218	#define AMBA_ERROR_RESPONSE_CRS_OKAY_FFFFFFFF 1
206	219	#define AMBA_ERROR_RESPONSE_CRS_OKAY_FFFF0001 2
207	220
	221	+#define MSIX_ADDR_MATCH_LOW_OFF 0x940
	222	+#define MSIX_ADDR_MATCH_LOW_OFF_EN BIT(0)
	223	+#define MSIX_ADDR_MATCH_LOW_OFF_MASK GENMASK(31, 2)
	224	+
	225	+#define MSIX_ADDR_MATCH_HIGH_OFF 0x944
	226	+#define MSIX_ADDR_MATCH_HIGH_OFF_MASK GENMASK(31, 0)
	227	+
208	228	#define PORT_LOGIC_MSIX_DOORBELL 0x948
209	229
210	230	#define CAP_SPCIE_CAP_OFF 0x154
@@ -223,6 +243,13 @@ · 12 unmodified lines
223	243	#define GEN3_CORE_CLK_FREQ 250000000
224	244	#define GEN4_CORE_CLK_FREQ 500000000
225	245
	246	+#define LTR_MSG_TIMEOUT (100 * 1000)
	247	+
	248	+#define PERST_DEBOUNCE_TIME (5 * 1000)
	249	+
	250	+#define EP_STATE_DISABLED 0
	251	+#define EP_STATE_ENABLED 1
	252	+
226	253	static const unsigned int pcie_gen_freq[] = {
227	254	GEN1_CORE_CLK_FREQ,
228	255	GEN2_CORE_CLK_FREQ,
@@ -260,6 +287,8 @@ struct tegra_pcie_dw { · 31 unmodified lines
260	287	struct dw_pcie pci;
261	288	struct tegra_bpmp *bpmp;
262	289
	290	+ enum dw_pcie_device_mode mode;
	291	+
263	292	bool supports_clkreq;
264	293	bool enable_cdm_check;
265	294	bool link_state;
@@ -283,6 +312,16 @@ struct tegra_pcie_dw { · 17 unmodified lines
283	312	struct phy **phys;
284	313
285	314	struct dentry *debugfs;
	315	+
	316	+ /* Endpoint mode specific */
	317	+ struct gpio_desc *pex_rst_gpiod;
	318	+ struct gpio_desc *pex_refclk_sel_gpiod;
	319	+ unsigned int pex_rst_irq;
	320	+ int ep_state;
	321	+};
	322	+
	323	+struct tegra_pcie_dw_of_data {
	324	+ enum dw_pcie_device_mode mode;
286	325	};
287	326
288	327	static inline struct tegra_pcie_dw to_tegra_pcie(struct dw_pcie pci)
@@ -339,8 +378,9 @@ static void apply_bad_link_workaround(struct pcie_port *pp) · 50 unmodified lines
339	378	}
340	379	}
341	380
342		−static irqreturn_t tegra_pcie_rp_irq_handler(struct tegra_pcie_dw *pcie)
	381	+static irqreturn_t tegra_pcie_rp_irq_handler(int irq, void *arg)
343	382	{
	383	+ struct tegra_pcie_dw *pcie = arg;
344	384	struct dw_pcie *pci = &pcie->pci;
345	385	struct pcie_port *pp = &pci->pp;
346	386	u32 val, tmp;
@@ -411,11 +451,121 @@ static irqreturn_t tegra_pcie_rp_irq_handler(struct tegra_pcie_dw *pcie) · 64 unmodified lines
411	451	return IRQ_HANDLED;
412	452	}
413	453
414		−static irqreturn_t tegra_pcie_irq_handler(int irq, void *arg)
	454	+static void pex_ep_event_hot_rst_done(struct tegra_pcie_dw *pcie)
	455	+{
	456	+ u32 val;
	457	+
	458	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L0);
	459	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_0_0);
	460	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_1);
	461	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_2);
	462	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_3);
	463	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_6);
	464	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_7);
	465	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_8_0);
	466	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_9);
	467	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_10);
	468	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_11);
	469	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_13);
	470	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_14);
	471	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_15);
	472	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_17);
	473	+ appl_writel(pcie, 0xFFFFFFFF, APPL_MSI_CTRL_2);
	474	+
	475	+ val = appl_readl(pcie, APPL_CTRL);
	476	+ val \|= APPL_CTRL_LTSSM_EN;
	477	+ appl_writel(pcie, val, APPL_CTRL);
	478	+}
	479	+
	480	+static irqreturn_t tegra_pcie_ep_irq_thread(int irq, void *arg)
415	481	{
416	482	struct tegra_pcie_dw *pcie = arg;
	483	+ struct dw_pcie *pci = &pcie->pci;
	484	+ u32 val, speed;
	485	+
	486	+ speed = dw_pcie_readw_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKSTA) &
	487	+ PCI_EXP_LNKSTA_CLS;
	488	+ clk_set_rate(pcie->core_clk, pcie_gen_freq[speed - 1]);
417	489
418		− return tegra_pcie_rp_irq_handler(pcie);
	490	+ /* If EP doesn't advertise L1SS, just return */
	491	+ val = dw_pcie_readl_dbi(pci, pcie->cfg_link_cap_l1sub);
	492	+ if (!(val & (PCI_L1SS_CAP_ASPM_L1_1 \| PCI_L1SS_CAP_ASPM_L1_2)))
	493	+ return IRQ_HANDLED;
	494	+
	495	+ /* Check if BME is set to '1' */
	496	+ val = dw_pcie_readl_dbi(pci, PCI_COMMAND);
	497	+ if (val & PCI_COMMAND_MASTER) {
	498	+ ktime_t timeout;
	499	+
	500	+ /* 110us for both snoop and no-snoop */
	501	+ val = 110 \| (2 << PCI_LTR_SCALE_SHIFT) \| LTR_MSG_REQ;
	502	+ val \|= (val << LTR_MST_NO_SNOOP_SHIFT);
	503	+ appl_writel(pcie, val, APPL_LTR_MSG_1);
	504	+
	505	+ /* Send LTR upstream */
	506	+ val = appl_readl(pcie, APPL_LTR_MSG_2);
	507	+ val \|= APPL_LTR_MSG_2_LTR_MSG_REQ_STATE;
	508	+ appl_writel(pcie, val, APPL_LTR_MSG_2);
	509	+
	510	+ timeout = ktime_add_us(ktime_get(), LTR_MSG_TIMEOUT);
	511	+ for (;;) {
	512	+ val = appl_readl(pcie, APPL_LTR_MSG_2);
	513	+ if (!(val & APPL_LTR_MSG_2_LTR_MSG_REQ_STATE))
	514	+ break;
	515	+ if (ktime_after(ktime_get(), timeout))
	516	+ break;
	517	+ usleep_range(1000, 1100);
	518	+ }
	519	+ if (val & APPL_LTR_MSG_2_LTR_MSG_REQ_STATE)
	520	+ dev_err(pcie->dev, "Failed to send LTR message\n");
	521	+ }
	522	+
	523	+ return IRQ_HANDLED;
	524	+}
	525	+
	526	+static irqreturn_t tegra_pcie_ep_hard_irq(int irq, void *arg)
	527	+{
	528	+ struct tegra_pcie_dw *pcie = arg;
	529	+ struct dw_pcie_ep *ep = &pcie->pci.ep;
	530	+ int spurious = 1;
	531	+ u32 val, tmp;
	532	+
	533	+ val = appl_readl(pcie, APPL_INTR_STATUS_L0);
	534	+ if (val & APPL_INTR_STATUS_L0_LINK_STATE_INT) {
	535	+ val = appl_readl(pcie, APPL_INTR_STATUS_L1_0_0);
	536	+ appl_writel(pcie, val, APPL_INTR_STATUS_L1_0_0);
	537	+
	538	+ if (val & APPL_INTR_STATUS_L1_0_0_HOT_RESET_DONE)
	539	+ pex_ep_event_hot_rst_done(pcie);
	540	+
	541	+ if (val & APPL_INTR_STATUS_L1_0_0_RDLH_LINK_UP_CHGED) {
	542	+ tmp = appl_readl(pcie, APPL_LINK_STATUS);
	543	+ if (tmp & APPL_LINK_STATUS_RDLH_LINK_UP) {
	544	+ dev_dbg(pcie->dev, "Link is up with Host\n");
	545	+ dw_pcie_ep_linkup(ep);
	546	+ }
	547	+ }
	548	+
	549	+ spurious = 0;
	550	+ }
	551	+
	552	+ if (val & APPL_INTR_STATUS_L0_PCI_CMD_EN_INT) {
	553	+ val = appl_readl(pcie, APPL_INTR_STATUS_L1_15);
	554	+ appl_writel(pcie, val, APPL_INTR_STATUS_L1_15);
	555	+
	556	+ if (val & APPL_INTR_STATUS_L1_15_CFG_BME_CHGED)
	557	+ return IRQ_WAKE_THREAD;
	558	+
	559	+ spurious = 0;
	560	+ }
	561	+
	562	+ if (spurious) {
	563	+ dev_warn(pcie->dev, "Random interrupt (STATUS = 0x%08X)\n",
	564	+ val);
	565	+ appl_writel(pcie, val, APPL_INTR_STATUS_L0);
	566	+ }
	567	+
	568	+ return IRQ_HANDLED;
419	569	}
420	570
421	571	static int tegra_pcie_dw_rd_own_conf(struct pcie_port *pp, int where, int size,
@@ -884,8 +1034,26 @@ static void tegra_pcie_set_msi_vec_num(struct pcie_port *pp) · 462 unmodified lines
884	1034	pp->num_vectors = MAX_MSI_IRQS;
885	1035	}
886	1036
	1037	+static int tegra_pcie_dw_start_link(struct dw_pcie *pci)
	1038	+{
	1039	+ struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
	1040	+
	1041	+ enable_irq(pcie->pex_rst_irq);
	1042	+
	1043	+ return 0;
	1044	+}
	1045	+
	1046	+static void tegra_pcie_dw_stop_link(struct dw_pcie *pci)
	1047	+{
	1048	+ struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
	1049	+
	1050	+ disable_irq(pcie->pex_rst_irq);
	1051	+}
	1052	+
887	1053	static const struct dw_pcie_ops tegra_dw_pcie_ops = {
888	1054	.link_up = tegra_pcie_dw_link_up,
	1055	+ .start_link = tegra_pcie_dw_start_link,
	1056	+ .stop_link = tegra_pcie_dw_stop_link,
889	1057	};
890	1058
891	1059	static struct dw_pcie_host_ops tegra_pcie_dw_host_ops = {
@@ -986,6 +1154,40 @@ static int tegra_pcie_dw_parse_dt(struct tegra_pcie_dw *pcie) · 94 unmodified lines
986	1154	pcie->enable_cdm_check =
987	1155	of_property_read_bool(np, "snps,enable-cdm-check");
988	1156
	1157	+ if (pcie->mode == DW_PCIE_RC_TYPE)
	1158	+ return 0;
	1159	+
	1160	+ /* Endpoint mode specific DT entries */
	1161	+ pcie->pex_rst_gpiod = devm_gpiod_get(pcie->dev, "reset", GPIOD_IN);
	1162	+ if (IS_ERR(pcie->pex_rst_gpiod)) {
	1163	+ int err = PTR_ERR(pcie->pex_rst_gpiod);
	1164	+ const char *level = KERN_ERR;
	1165	+
	1166	+ if (err == -EPROBE_DEFER)
	1167	+ level = KERN_DEBUG;
	1168	+
	1169	+ dev_printk(level, pcie->dev,
	1170	+ dev_fmt("Failed to get PERST GPIO: %d\n"),
	1171	+ err);
	1172	+ return err;
	1173	+ }
	1174	+
	1175	+ pcie->pex_refclk_sel_gpiod = devm_gpiod_get(pcie->dev,
	1176	+ "nvidia,refclk-select",
	1177	+ GPIOD_OUT_HIGH);
	1178	+ if (IS_ERR(pcie->pex_refclk_sel_gpiod)) {
	1179	+ int err = PTR_ERR(pcie->pex_refclk_sel_gpiod);
	1180	+ const char *level = KERN_ERR;
	1181	+
	1182	+ if (err == -EPROBE_DEFER)
	1183	+ level = KERN_DEBUG;
	1184	+
	1185	+ dev_printk(level, pcie->dev,
	1186	+ dev_fmt("Failed to get REFCLK select GPIOs: %d\n"),
	1187	+ err);
	1188	+ pcie->pex_refclk_sel_gpiod = NULL;
	1189	+ }
	1190	+
989	1191	return 0;
990	1192	}
991	1193
@@ -1017,6 +1219,34 @@ static int tegra_pcie_bpmp_set_ctrl_state(struct tegra_pcie_dw *pcie, · 25 unmodified lines
1017	1219	return tegra_bpmp_transfer(pcie->bpmp, &msg);
1018	1220	}
1019	1221
	1222	+static int tegra_pcie_bpmp_set_pll_state(struct tegra_pcie_dw *pcie,
	1223	+ bool enable)
	1224	+{
	1225	+ struct mrq_uphy_response resp;
	1226	+ struct tegra_bpmp_message msg;
	1227	+ struct mrq_uphy_request req;
	1228	+
	1229	+ memset(&req, 0, sizeof(req));
	1230	+ memset(&resp, 0, sizeof(resp));
	1231	+
	1232	+ if (enable) {
	1233	+ req.cmd = CMD_UPHY_PCIE_EP_CONTROLLER_PLL_INIT;
	1234	+ req.ep_ctrlr_pll_init.ep_controller = pcie->cid;
	1235	+ } else {
	1236	+ req.cmd = CMD_UPHY_PCIE_EP_CONTROLLER_PLL_OFF;
	1237	+ req.ep_ctrlr_pll_off.ep_controller = pcie->cid;
	1238	+ }
	1239	+
	1240	+ memset(&msg, 0, sizeof(msg));
	1241	+ msg.mrq = MRQ_UPHY;
	1242	+ msg.tx.data = &req;
	1243	+ msg.tx.size = sizeof(req);
	1244	+ msg.rx.data = &resp;
	1245	+ msg.rx.size = sizeof(resp);
	1246	+
	1247	+ return tegra_bpmp_transfer(pcie->bpmp, &msg);
	1248	+}
	1249	+
1020	1250	static void tegra_pcie_downstream_dev_to_D0(struct tegra_pcie_dw *pcie)
1021	1251	{
1022	1252	struct pcie_port *pp = &pcie->pci.pp;
@@ -1427,8 +1657,396 @@ fail_pm_get_sync: · 404 unmodified lines
1427	1657	return ret;
1428	1658	}
1429	1659
	1660	+static void pex_ep_event_pex_rst_assert(struct tegra_pcie_dw *pcie)
	1661	+{
	1662	+ u32 val;
	1663	+ int ret;
	1664	+
	1665	+ if (pcie->ep_state == EP_STATE_DISABLED)
	1666	+ return;
	1667	+
	1668	+ /* Disable LTSSM */
	1669	+ val = appl_readl(pcie, APPL_CTRL);
	1670	+ val &= ~APPL_CTRL_LTSSM_EN;
	1671	+ appl_writel(pcie, val, APPL_CTRL);
	1672	+
	1673	+ ret = readl_poll_timeout(pcie->appl_base + APPL_DEBUG, val,
	1674	+ ((val & APPL_DEBUG_LTSSM_STATE_MASK) >>
	1675	+ APPL_DEBUG_LTSSM_STATE_SHIFT) ==
	1676	+ LTSSM_STATE_PRE_DETECT,
	1677	+ 1, LTSSM_TIMEOUT);
	1678	+ if (ret)
	1679	+ dev_err(pcie->dev, "Failed to go Detect state: %d\n", ret);
	1680	+
	1681	+ reset_control_assert(pcie->core_rst);
	1682	+
	1683	+ tegra_pcie_disable_phy(pcie);
	1684	+
	1685	+ reset_control_assert(pcie->core_apb_rst);
	1686	+
	1687	+ clk_disable_unprepare(pcie->core_clk);
	1688	+
	1689	+ pm_runtime_put_sync(pcie->dev);
	1690	+
	1691	+ ret = tegra_pcie_bpmp_set_pll_state(pcie, false);
	1692	+ if (ret)
	1693	+ dev_err(pcie->dev, "Failed to turn off UPHY: %d\n", ret);
	1694	+
	1695	+ pcie->ep_state = EP_STATE_DISABLED;
	1696	+ dev_dbg(pcie->dev, "Uninitialization of endpoint is completed\n");
	1697	+}
	1698	+
	1699	+static void pex_ep_event_pex_rst_deassert(struct tegra_pcie_dw *pcie)
	1700	+{
	1701	+ struct dw_pcie *pci = &pcie->pci;
	1702	+ struct dw_pcie_ep *ep = &pci->ep;
	1703	+ struct device *dev = pcie->dev;
	1704	+ u32 val;
	1705	+ int ret;
	1706	+
	1707	+ if (pcie->ep_state == EP_STATE_ENABLED)
	1708	+ return;
	1709	+
	1710	+ ret = pm_runtime_get_sync(dev);
	1711	+ if (ret < 0) {
	1712	+ dev_err(dev, "Failed to get runtime sync for PCIe dev: %d\n",
	1713	+ ret);
	1714	+ return;
	1715	+ }
	1716	+
	1717	+ ret = tegra_pcie_bpmp_set_pll_state(pcie, true);
	1718	+ if (ret) {
	1719	+ dev_err(dev, "Failed to init UPHY for PCIe EP: %d\n", ret);
	1720	+ goto fail_pll_init;
	1721	+ }
	1722	+
	1723	+ ret = clk_prepare_enable(pcie->core_clk);
	1724	+ if (ret) {
	1725	+ dev_err(dev, "Failed to enable core clock: %d\n", ret);
	1726	+ goto fail_core_clk_enable;
	1727	+ }
	1728	+
	1729	+ ret = reset_control_deassert(pcie->core_apb_rst);
	1730	+ if (ret) {
	1731	+ dev_err(dev, "Failed to deassert core APB reset: %d\n", ret);
	1732	+ goto fail_core_apb_rst;
	1733	+ }
	1734	+
	1735	+ ret = tegra_pcie_enable_phy(pcie);
	1736	+ if (ret) {
	1737	+ dev_err(dev, "Failed to enable PHY: %d\n", ret);
	1738	+ goto fail_phy;
	1739	+ }
	1740	+
	1741	+ /* Clear any stale interrupt statuses */
	1742	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L0);
	1743	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_0_0);
	1744	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_1);
	1745	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_2);
	1746	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_3);
	1747	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_6);
	1748	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_7);
	1749	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_8_0);
	1750	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_9);
	1751	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_10);
	1752	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_11);
	1753	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_13);
	1754	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_14);
	1755	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_15);
	1756	+ appl_writel(pcie, 0xFFFFFFFF, APPL_INTR_STATUS_L1_17);
	1757	+
	1758	+ /* configure this core for EP mode operation */
	1759	+ val = appl_readl(pcie, APPL_DM_TYPE);
	1760	+ val &= ~APPL_DM_TYPE_MASK;
	1761	+ val \|= APPL_DM_TYPE_EP;
	1762	+ appl_writel(pcie, val, APPL_DM_TYPE);
	1763	+
	1764	+ appl_writel(pcie, 0x0, APPL_CFG_SLCG_OVERRIDE);
	1765	+
	1766	+ val = appl_readl(pcie, APPL_CTRL);
	1767	+ val \|= APPL_CTRL_SYS_PRE_DET_STATE;
	1768	+ val \|= APPL_CTRL_HW_HOT_RST_EN;
	1769	+ appl_writel(pcie, val, APPL_CTRL);
	1770	+
	1771	+ val = appl_readl(pcie, APPL_CFG_MISC);
	1772	+ val \|= APPL_CFG_MISC_SLV_EP_MODE;
	1773	+ val \|= (APPL_CFG_MISC_ARCACHE_VAL << APPL_CFG_MISC_ARCACHE_SHIFT);
	1774	+ appl_writel(pcie, val, APPL_CFG_MISC);
	1775	+
	1776	+ val = appl_readl(pcie, APPL_PINMUX);
	1777	+ val \|= APPL_PINMUX_CLK_OUTPUT_IN_OVERRIDE_EN;
	1778	+ val \|= APPL_PINMUX_CLK_OUTPUT_IN_OVERRIDE;
	1779	+ appl_writel(pcie, val, APPL_PINMUX);
	1780	+
	1781	+ appl_writel(pcie, pcie->dbi_res->start & APPL_CFG_BASE_ADDR_MASK,
	1782	+ APPL_CFG_BASE_ADDR);
	1783	+
	1784	+ appl_writel(pcie, pcie->atu_dma_res->start &
	1785	+ APPL_CFG_IATU_DMA_BASE_ADDR_MASK,
	1786	+ APPL_CFG_IATU_DMA_BASE_ADDR);
	1787	+
	1788	+ val = appl_readl(pcie, APPL_INTR_EN_L0_0);
	1789	+ val \|= APPL_INTR_EN_L0_0_SYS_INTR_EN;
	1790	+ val \|= APPL_INTR_EN_L0_0_LINK_STATE_INT_EN;
	1791	+ val \|= APPL_INTR_EN_L0_0_PCI_CMD_EN_INT_EN;
	1792	+ appl_writel(pcie, val, APPL_INTR_EN_L0_0);
	1793	+
	1794	+ val = appl_readl(pcie, APPL_INTR_EN_L1_0_0);
	1795	+ val \|= APPL_INTR_EN_L1_0_0_HOT_RESET_DONE_INT_EN;
	1796	+ val \|= APPL_INTR_EN_L1_0_0_RDLH_LINK_UP_INT_EN;
	1797	+ appl_writel(pcie, val, APPL_INTR_EN_L1_0_0);
	1798	+
	1799	+ reset_control_deassert(pcie->core_rst);
	1800	+
	1801	+ if (pcie->update_fc_fixup) {
	1802	+ val = dw_pcie_readl_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF);
	1803	+ val \|= 0x1 << CFG_TIMER_CTRL_ACK_NAK_SHIFT;
	1804	+ dw_pcie_writel_dbi(pci, CFG_TIMER_CTRL_MAX_FUNC_NUM_OFF, val);
	1805	+ }
	1806	+
	1807	+ config_gen3_gen4_eq_presets(pcie);
	1808	+
	1809	+ init_host_aspm(pcie);
	1810	+
	1811	+ /* Disable ASPM-L1SS advertisement if there is no CLKREQ routing */
	1812	+ if (!pcie->supports_clkreq) {
	1813	+ disable_aspm_l11(pcie);
	1814	+ disable_aspm_l12(pcie);
	1815	+ }
	1816	+
	1817	+ val = dw_pcie_readl_dbi(pci, GEN3_RELATED_OFF);
	1818	+ val &= ~GEN3_RELATED_OFF_GEN3_ZRXDC_NONCOMPL;
	1819	+ dw_pcie_writel_dbi(pci, GEN3_RELATED_OFF, val);
	1820	+
	1821	+ /* Configure N_FTS & FTS */
	1822	+ val = dw_pcie_readl_dbi(pci, PORT_LOGIC_ACK_F_ASPM_CTRL);
	1823	+ val &= ~(N_FTS_MASK << N_FTS_SHIFT);
	1824	+ val \|= N_FTS_VAL << N_FTS_SHIFT;
	1825	+ dw_pcie_writel_dbi(pci, PORT_LOGIC_ACK_F_ASPM_CTRL, val);
	1826	+
	1827	+ val = dw_pcie_readl_dbi(pci, PORT_LOGIC_GEN2_CTRL);
	1828	+ val &= ~FTS_MASK;
	1829	+ val \|= FTS_VAL;
	1830	+ dw_pcie_writel_dbi(pci, PORT_LOGIC_GEN2_CTRL, val);
	1831	+
	1832	+ /* Configure Max Speed from DT */
	1833	+ if (pcie->max_speed && pcie->max_speed != -EINVAL) {
	1834	+ val = dw_pcie_readl_dbi(pci, pcie->pcie_cap_base +
	1835	+ PCI_EXP_LNKCAP);
	1836	+ val &= ~PCI_EXP_LNKCAP_SLS;
	1837	+ val \|= pcie->max_speed;
	1838	+ dw_pcie_writel_dbi(pci, pcie->pcie_cap_base + PCI_EXP_LNKCAP,
	1839	+ val);
	1840	+ }
	1841	+
	1842	+ pcie->pcie_cap_base = dw_pcie_find_capability(&pcie->pci,
	1843	+ PCI_CAP_ID_EXP);
	1844	+ clk_set_rate(pcie->core_clk, GEN4_CORE_CLK_FREQ);
	1845	+
	1846	+ val = (ep->msi_mem_phys & MSIX_ADDR_MATCH_LOW_OFF_MASK);
	1847	+ val \|= MSIX_ADDR_MATCH_LOW_OFF_EN;
	1848	+ dw_pcie_writel_dbi(pci, MSIX_ADDR_MATCH_LOW_OFF, val);
	1849	+ val = (lower_32_bits(ep->msi_mem_phys) & MSIX_ADDR_MATCH_HIGH_OFF_MASK);
	1850	+ dw_pcie_writel_dbi(pci, MSIX_ADDR_MATCH_HIGH_OFF, val);
	1851	+
	1852	+ ret = dw_pcie_ep_init_complete(ep);
	1853	+ if (ret) {
	1854	+ dev_err(dev, "Failed to complete initialization: %d\n", ret);
	1855	+ goto fail_init_complete;
	1856	+ }
	1857	+
	1858	+ dw_pcie_ep_init_notify(ep);
	1859	+
	1860	+ /* Enable LTSSM */
	1861	+ val = appl_readl(pcie, APPL_CTRL);
	1862	+ val \|= APPL_CTRL_LTSSM_EN;
	1863	+ appl_writel(pcie, val, APPL_CTRL);
	1864	+
	1865	+ pcie->ep_state = EP_STATE_ENABLED;
	1866	+ dev_dbg(dev, "Initialization of endpoint is completed\n");
	1867	+
	1868	+ return;
	1869	+
	1870	+fail_init_complete:
	1871	+ reset_control_assert(pcie->core_rst);
	1872	+ tegra_pcie_disable_phy(pcie);
	1873	+fail_phy:
	1874	+ reset_control_assert(pcie->core_apb_rst);
	1875	+fail_core_apb_rst:
	1876	+ clk_disable_unprepare(pcie->core_clk);
	1877	+fail_core_clk_enable:
	1878	+ tegra_pcie_bpmp_set_pll_state(pcie, false);
	1879	+fail_pll_init:
	1880	+ pm_runtime_put_sync(dev);
	1881	+}
	1882	+
	1883	+static irqreturn_t tegra_pcie_ep_pex_rst_irq(int irq, void *arg)
	1884	+{
	1885	+ struct tegra_pcie_dw *pcie = arg;
	1886	+
	1887	+ if (gpiod_get_value(pcie->pex_rst_gpiod))
	1888	+ pex_ep_event_pex_rst_assert(pcie);
	1889	+ else
	1890	+ pex_ep_event_pex_rst_deassert(pcie);
	1891	+
	1892	+ return IRQ_HANDLED;
	1893	+}
	1894	+
	1895	+static int tegra_pcie_ep_raise_legacy_irq(struct tegra_pcie_dw *pcie, u16 irq)
	1896	+{
	1897	+ /* Tegra194 supports only INTA */
	1898	+ if (irq > 1)
	1899	+ return -EINVAL;
	1900	+
	1901	+ appl_writel(pcie, 1, APPL_LEGACY_INTX);
	1902	+ usleep_range(1000, 2000);
	1903	+ appl_writel(pcie, 0, APPL_LEGACY_INTX);
	1904	+ return 0;
	1905	+}
	1906	+
	1907	+static int tegra_pcie_ep_raise_msi_irq(struct tegra_pcie_dw *pcie, u16 irq)
	1908	+{
	1909	+ if (unlikely(irq > 31))
	1910	+ return -EINVAL;
	1911	+
	1912	+ appl_writel(pcie, (1 << irq), APPL_MSI_CTRL_1);
	1913	+
	1914	+ return 0;
	1915	+}
	1916	+
	1917	+static int tegra_pcie_ep_raise_msix_irq(struct tegra_pcie_dw *pcie, u16 irq)
	1918	+{
	1919	+ struct dw_pcie_ep *ep = &pcie->pci.ep;
	1920	+
	1921	+ writel(irq, ep->msi_mem);
	1922	+
	1923	+ return 0;
	1924	+}
	1925	+
	1926	+static int tegra_pcie_ep_raise_irq(struct dw_pcie_ep *ep, u8 func_no,
	1927	+ enum pci_epc_irq_type type,
	1928	+ u16 interrupt_num)
	1929	+{
	1930	+ struct dw_pcie *pci = to_dw_pcie_from_ep(ep);
	1931	+ struct tegra_pcie_dw *pcie = to_tegra_pcie(pci);
	1932	+
	1933	+ switch (type) {
	1934	+ case PCI_EPC_IRQ_LEGACY:
	1935	+ return tegra_pcie_ep_raise_legacy_irq(pcie, interrupt_num);
	1936	+
	1937	+ case PCI_EPC_IRQ_MSI:
	1938	+ return tegra_pcie_ep_raise_msi_irq(pcie, interrupt_num);
	1939	+
	1940	+ case PCI_EPC_IRQ_MSIX:
	1941	+ return tegra_pcie_ep_raise_msix_irq(pcie, interrupt_num);
	1942	+
	1943	+ default:
	1944	+ dev_err(pci->dev, "Unknown IRQ type\n");
	1945	+ return -EPERM;
	1946	+ }
	1947	+
	1948	+ return 0;
	1949	+}
	1950	+
	1951	+static const struct pci_epc_features tegra_pcie_epc_features = {
	1952	+ .linkup_notifier = true,
	1953	+ .core_init_notifier = true,
	1954	+ .msi_capable = false,
	1955	+ .msix_capable = false,
	1956	+ .reserved_bar = 1 << BAR_2 \| 1 << BAR_3 \| 1 << BAR_4 \| 1 << BAR_5,
	1957	+ .bar_fixed_64bit = 1 << BAR_0,
	1958	+ .bar_fixed_size[0] = SZ_1M,
	1959	+};
	1960	+
	1961	+static const struct pci_epc_features*
	1962	+tegra_pcie_ep_get_features(struct dw_pcie_ep *ep)
	1963	+{
	1964	+ return &tegra_pcie_epc_features;
	1965	+}
	1966	+
	1967	+static struct dw_pcie_ep_ops pcie_ep_ops = {
	1968	+ .raise_irq = tegra_pcie_ep_raise_irq,
	1969	+ .get_features = tegra_pcie_ep_get_features,
	1970	+};
	1971	+
	1972	+static int tegra_pcie_config_ep(struct tegra_pcie_dw *pcie,
	1973	+ struct platform_device *pdev)
	1974	+{
	1975	+ struct dw_pcie *pci = &pcie->pci;
	1976	+ struct device *dev = pcie->dev;
	1977	+ struct dw_pcie_ep *ep;
	1978	+ struct resource *res;
	1979	+ char *name;
	1980	+ int ret;
	1981	+
	1982	+ ep = &pci->ep;
	1983	+ ep->ops = &pcie_ep_ops;
	1984	+
	1985	+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, "addr_space");
	1986	+ if (!res)
	1987	+ return -EINVAL;
	1988	+
	1989	+ ep->phys_base = res->start;
	1990	+ ep->addr_size = resource_size(res);
	1991	+ ep->page_size = SZ_64K;
	1992	+
	1993	+ ret = gpiod_set_debounce(pcie->pex_rst_gpiod, PERST_DEBOUNCE_TIME);
	1994	+ if (ret < 0) {
	1995	+ dev_err(dev, "Failed to set PERST GPIO debounce time: %d\n",
	1996	+ ret);
	1997	+ return ret;
	1998	+ }
	1999	+
	2000	+ ret = gpiod_to_irq(pcie->pex_rst_gpiod);
	2001	+ if (ret < 0) {
	2002	+ dev_err(dev, "Failed to get IRQ for PERST GPIO: %d\n", ret);
	2003	+ return ret;
	2004	+ }
	2005	+ pcie->pex_rst_irq = (unsigned int)ret;
	2006	+
	2007	+ name = devm_kasprintf(dev, GFP_KERNEL, "tegra_pcie_%u_pex_rst_irq",
	2008	+ pcie->cid);
	2009	+ if (!name) {
	2010	+ dev_err(dev, "Failed to create PERST IRQ string\n");
	2011	+ return -ENOMEM;
	2012	+ }
	2013	+
	2014	+ irq_set_status_flags(pcie->pex_rst_irq, IRQ_NOAUTOEN);
	2015	+
	2016	+ pcie->ep_state = EP_STATE_DISABLED;
	2017	+
	2018	+ ret = devm_request_threaded_irq(dev, pcie->pex_rst_irq, NULL,
	2019	+ tegra_pcie_ep_pex_rst_irq,
	2020	+ IRQF_TRIGGER_RISING \|
	2021	+ IRQF_TRIGGER_FALLING \| IRQF_ONESHOT,
	2022	+ name, (void *)pcie);
	2023	+ if (ret < 0) {
	2024	+ dev_err(dev, "Failed to request IRQ for PERST: %d\n", ret);
	2025	+ return ret;
	2026	+ }
	2027	+
	2028	+ name = devm_kasprintf(dev, GFP_KERNEL, "tegra_pcie_%u_ep_work",
	2029	+ pcie->cid);
	2030	+ if (!name) {
	2031	+ dev_err(dev, "Failed to create PCIe EP work thread string\n");
	2032	+ return -ENOMEM;
	2033	+ }
	2034	+
	2035	+ pm_runtime_enable(dev);
	2036	+
	2037	+ ret = dw_pcie_ep_init(ep);
	2038	+ if (ret) {
	2039	+ dev_err(dev, "Failed to initialize DWC Endpoint subsystem: %d\n",
	2040	+ ret);
	2041	+ return ret;
	2042	+ }
	2043	+
	2044	+ return 0;
	2045	+}
	2046	+
1430	2047	static int tegra_pcie_dw_probe(struct platform_device *pdev)
1431	2048	{
	2049	+ const struct tegra_pcie_dw_of_data *data;
1432	2050	struct device *dev = &pdev->dev;
1433	2051	struct resource *atu_dma_res;
1434	2052	struct tegra_pcie_dw *pcie;
@@ -1440,6 +2058,8 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev) · 5 unmodified lines
1440	2058	int ret;
1441	2059	u32 i;
1442	2060
	2061	+ data = of_device_get_match_data(dev);
	2062	+
1443	2063	pcie = devm_kzalloc(dev, sizeof(*pcie), GFP_KERNEL);
1444	2064	if (!pcie)
1445	2065	return -ENOMEM;
@@ -1449,19 +2069,37 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev) · 3 unmodified lines
1449	2069	pci->ops = &tegra_dw_pcie_ops;
1450	2070	pp = &pci->pp;
1451	2071	pcie->dev = &pdev->dev;
	2072	+ pcie->mode = (enum dw_pcie_device_mode)data->mode;
1452	2073
1453	2074	ret = tegra_pcie_dw_parse_dt(pcie);
1454	2075	if (ret < 0) {
1455		− dev_err(dev, "Failed to parse device tree: %d\n", ret);
	2076	+ const char *level = KERN_ERR;
	2077	+
	2078	+ if (ret == -EPROBE_DEFER)
	2079	+ level = KERN_DEBUG;
	2080	+
	2081	+ dev_printk(level, dev,
	2082	+ dev_fmt("Failed to parse device tree: %d\n"),
	2083	+ ret);
1456	2084	return ret;
1457	2085	}
1458	2086
1459	2087	ret = tegra_pcie_get_slot_regulators(pcie);
1460	2088	if (ret < 0) {
1461		− dev_err(dev, "Failed to get slot regulators: %d\n", ret);
	2089	+ const char *level = KERN_ERR;
	2090	+
	2091	+ if (ret == -EPROBE_DEFER)
	2092	+ level = KERN_DEBUG;
	2093	+
	2094	+ dev_printk(level, dev,
	2095	+ dev_fmt("Failed to get slot regulators: %d\n"),
	2096	+ ret);
1462	2097	return ret;
1463	2098	}
1464	2099
	2100	+ if (pcie->pex_refclk_sel_gpiod)
	2101	+ gpiod_set_value(pcie->pex_refclk_sel_gpiod, 1);
	2102	+
1465	2103	pcie->pex_ctl_supply = devm_regulator_get(dev, "vddio-pex-ctl");
1466	2104	if (IS_ERR(pcie->pex_ctl_supply)) {
1467	2105	ret = PTR_ERR(pcie->pex_ctl_supply);
@@ -1557,24 +2195,49 @@ static int tegra_pcie_dw_probe(struct platform_device *pdev) · 89 unmodified lines
1557	2195	return -ENODEV;
1558	2196	}
1559	2197
1560		− ret = devm_request_irq(dev, pp->irq, tegra_pcie_irq_handler,
1561		− IRQF_SHARED, "tegra-pcie-intr", pcie);
1562		− if (ret) {
1563		− dev_err(dev, "Failed to request IRQ %d: %d\n", pp->irq, ret);
1564		− return ret;
1565		− }
1566		−
1567	2198	pcie->bpmp = tegra_bpmp_get(dev);
1568	2199	if (IS_ERR(pcie->bpmp))
1569	2200	return PTR_ERR(pcie->bpmp);
1570	2201
1571	2202	platform_set_drvdata(pdev, pcie);
1572	2203
1573		− ret = tegra_pcie_config_rp(pcie);
1574		− if (ret && ret != -ENOMEDIUM)
1575		− goto fail;
1576		− else
1577		− return 0;
	2204	+ switch (pcie->mode) {
	2205	+ case DW_PCIE_RC_TYPE:
	2206	+ ret = devm_request_irq(dev, pp->irq, tegra_pcie_rp_irq_handler,
	2207	+ IRQF_SHARED, "tegra-pcie-intr", pcie);
	2208	+ if (ret) {
	2209	+ dev_err(dev, "Failed to request IRQ %d: %d\n", pp->irq,
	2210	+ ret);
	2211	+ goto fail;
	2212	+ }
	2213	+
	2214	+ ret = tegra_pcie_config_rp(pcie);
	2215	+ if (ret && ret != -ENOMEDIUM)
	2216	+ goto fail;
	2217	+ else
	2218	+ return 0;
	2219	+ break;
	2220	+
	2221	+ case DW_PCIE_EP_TYPE:
	2222	+ ret = devm_request_threaded_irq(dev, pp->irq,
	2223	+ tegra_pcie_ep_hard_irq,
	2224	+ tegra_pcie_ep_irq_thread,
	2225	+ IRQF_SHARED \| IRQF_ONESHOT,
	2226	+ "tegra-pcie-ep-intr", pcie);
	2227	+ if (ret) {
	2228	+ dev_err(dev, "Failed to request IRQ %d: %d\n", pp->irq,
	2229	+ ret);
	2230	+ goto fail;
	2231	+ }
	2232	+
	2233	+ ret = tegra_pcie_config_ep(pcie, pdev);
	2234	+ if (ret < 0)
	2235	+ goto fail;
	2236	+ break;
	2237	+
	2238	+ default:
	2239	+ dev_err(dev, "Invalid PCIe device type %d\n", pcie->mode);
	2240	+ }
1578	2241
1579	2242	fail:
1580	2243	tegra_bpmp_put(pcie->bpmp);
@@ -1593,6 +2256,8 @@ static int tegra_pcie_dw_remove(struct platform_device *pdev) · 12 unmodified lines
1593	2256	pm_runtime_put_sync(pcie->dev);
1594	2257	pm_runtime_disable(pcie->dev);
1595	2258	tegra_bpmp_put(pcie->bpmp);
	2259	+ if (pcie->pex_refclk_sel_gpiod)
	2260	+ gpiod_set_value(pcie->pex_refclk_sel_gpiod, 0);
1596	2261
1597	2262	return 0;
1598	2263	}
@@ -1697,9 +2362,22 @@ static void tegra_pcie_dw_shutdown(struct platform_device *pdev) · 98 unmodified lines
1697	2362	__deinit_controller(pcie);
1698	2363	}
1699	2364
	2365	+static const struct tegra_pcie_dw_of_data tegra_pcie_dw_rc_of_data = {
	2366	+ .mode = DW_PCIE_RC_TYPE,
	2367	+};
	2368	+
	2369	+static const struct tegra_pcie_dw_of_data tegra_pcie_dw_ep_of_data = {
	2370	+ .mode = DW_PCIE_EP_TYPE,
	2371	+};
	2372	+
1700	2373	static const struct of_device_id tegra_pcie_dw_of_match[] = {
1701	2374	{
1702	2375	.compatible = "nvidia,tegra194-pcie",
	2376	+ .data = &tegra_pcie_dw_rc_of_data,
	2377	+ },
	2378	+ {
	2379	+ .compatible = "nvidia,tegra194-pcie-ep",
	2380	+ .data = &tegra_pcie_dw_ep_of_data,
1703	2381	},
1704	2382	{},
1705	2383	};

drivers/pci/endpoint/functions/pci-epf-test.c+356−46

@@ -8,6 +8,7 @@ · 7 unmodified lines
8	8
9	9	#include <linux/crc32.h>
10	10	#include <linux/delay.h>
	11	+#include <linux/dmaengine.h>
11	12	#include <linux/io.h>
12	13	#include <linux/module.h>
13	14	#include <linux/slab.h>
@@ -39,6 +40,8 @@ · 25 unmodified lines
39	40	#define STATUS_SRC_ADDR_INVALID BIT(7)
40	41	#define STATUS_DST_ADDR_INVALID BIT(8)
41	42
	43	+#define FLAG_USE_DMA BIT(0)
	44	+
42	45	#define TIMER_RESOLUTION 1
43	46
44	47	static struct workqueue_struct *kpcitest_workqueue;
@@ -47,7 +50,11 @@ struct pci_epf_test { · 2 unmodified lines
47	50	void *reg[PCI_STD_NUM_BARS];
48	51	struct pci_epf *epf;
49	52	enum pci_barno test_reg_bar;
	53	+ size_t msix_table_offset;
50	54	struct delayed_work cmd_handler;
	55	+ struct dma_chan *dma_chan;
	56	+ struct completion transfer_complete;
	57	+ bool dma_supported;
51	58	const struct pci_epc_features *epc_features;
52	59	};
53	60
@@ -61,6 +68,7 @@ struct pci_epf_test_reg { · 7 unmodified lines
61	68	u32 checksum;
62	69	u32 irq_type;
63	70	u32 irq_number;
	71	+ u32 flags;
64	72	} __packed;
65	73
66	74	static struct pci_epf_header test_header = {
@@ -72,13 +80,156 @@ static struct pci_epf_header test_header = { · 5 unmodified lines
72	80
73	81	static size_t bar_size[] = { 512, 512, 1024, 16384, 131072, 1048576 };
74	82
	83	+static void pci_epf_test_dma_callback(void *param)
	84	+{
	85	+ struct pci_epf_test *epf_test = param;
	86	+
	87	+ complete(&epf_test->transfer_complete);
	88	+}
	89	+
	90	+/**
	91	+ * pci_epf_test_data_transfer() - Function that uses dmaengine API to transfer
	92	+ * data between PCIe EP and remote PCIe RC
	93	+ * @epf_test: the EPF test device that performs the data transfer operation
	94	+ * @dma_dst: The destination address of the data transfer. It can be a physical
	95	+ * address given by pci_epc_mem_alloc_addr or DMA mapping APIs.
	96	+ * @dma_src: The source address of the data transfer. It can be a physical
	97	+ * address given by pci_epc_mem_alloc_addr or DMA mapping APIs.
	98	+ * @len: The size of the data transfer
	99	+ *
	100	+ * Function that uses dmaengine API to transfer data between PCIe EP and remote
	101	+ * PCIe RC. The source and destination address can be a physical address given
	102	+ * by pci_epc_mem_alloc_addr or the one obtained using DMA mapping APIs.
	103	+ *
	104	+ * The function returns '0' on success and negative value on failure.
	105	+ */
	106	+static int pci_epf_test_data_transfer(struct pci_epf_test *epf_test,
	107	+ dma_addr_t dma_dst, dma_addr_t dma_src,
	108	+ size_t len)
	109	+{
	110	+ enum dma_ctrl_flags flags = DMA_CTRL_ACK \| DMA_PREP_INTERRUPT;
	111	+ struct dma_chan *chan = epf_test->dma_chan;
	112	+ struct pci_epf *epf = epf_test->epf;
	113	+ struct dma_async_tx_descriptor *tx;
	114	+ struct device *dev = &epf->dev;
	115	+ dma_cookie_t cookie;
	116	+ int ret;
	117	+
	118	+ if (IS_ERR_OR_NULL(chan)) {
	119	+ dev_err(dev, "Invalid DMA memcpy channel\n");
	120	+ return -EINVAL;
	121	+ }
	122	+
	123	+ tx = dmaengine_prep_dma_memcpy(chan, dma_dst, dma_src, len, flags);
	124	+ if (!tx) {
	125	+ dev_err(dev, "Failed to prepare DMA memcpy\n");
	126	+ return -EIO;
	127	+ }
	128	+
	129	+ tx->callback = pci_epf_test_dma_callback;
	130	+ tx->callback_param = epf_test;
	131	+ cookie = tx->tx_submit(tx);
	132	+ reinit_completion(&epf_test->transfer_complete);
	133	+
	134	+ ret = dma_submit_error(cookie);
	135	+ if (ret) {
	136	+ dev_err(dev, "Failed to do DMA tx_submit %d\n", cookie);
	137	+ return -EIO;
	138	+ }
	139	+
	140	+ dma_async_issue_pending(chan);
	141	+ ret = wait_for_completion_interruptible(&epf_test->transfer_complete);
	142	+ if (ret < 0) {
	143	+ dmaengine_terminate_sync(chan);
	144	+ dev_err(dev, "DMA wait_for_completion_timeout\n");
	145	+ return -ETIMEDOUT;
	146	+ }
	147	+
	148	+ return 0;
	149	+}
	150	+
	151	+/**
	152	+ * pci_epf_test_init_dma_chan() - Function to initialize EPF test DMA channel
	153	+ * @epf_test: the EPF test device that performs data transfer operation
	154	+ *
	155	+ * Function to initialize EPF test DMA channel.
	156	+ */
	157	+static int pci_epf_test_init_dma_chan(struct pci_epf_test *epf_test)
	158	+{
	159	+ struct pci_epf *epf = epf_test->epf;
	160	+ struct device *dev = &epf->dev;
	161	+ struct dma_chan *dma_chan;
	162	+ dma_cap_mask_t mask;
	163	+ int ret;
	164	+
	165	+ dma_cap_zero(mask);
	166	+ dma_cap_set(DMA_MEMCPY, mask);
	167	+
	168	+ dma_chan = dma_request_chan_by_mask(&mask);
	169	+ if (IS_ERR(dma_chan)) {
	170	+ ret = PTR_ERR(dma_chan);
	171	+ if (ret != -EPROBE_DEFER)
	172	+ dev_err(dev, "Failed to get DMA channel\n");
	173	+ return ret;
	174	+ }
	175	+ init_completion(&epf_test->transfer_complete);
	176	+
	177	+ epf_test->dma_chan = dma_chan;
	178	+
	179	+ return 0;
	180	+}
	181	+
	182	+/**
	183	+ * pci_epf_test_clean_dma_chan() - Function to cleanup EPF test DMA channel
	184	+ * @epf: the EPF test device that performs data transfer operation
	185	+ *
	186	+ * Helper to cleanup EPF test DMA channel.
	187	+ */
	188	+static void pci_epf_test_clean_dma_chan(struct pci_epf_test *epf_test)
	189	+{
	190	+ dma_release_channel(epf_test->dma_chan);
	191	+ epf_test->dma_chan = NULL;
	192	+}
	193	+
	194	+static void pci_epf_test_print_rate(const char *ops, u64 size,
	195	+ struct timespec64 *start,
	196	+ struct timespec64 *end, bool dma)
	197	+{
	198	+ struct timespec64 ts;
	199	+ u64 rate, ns;
	200	+
	201	+ ts = timespec64_sub(end, start);
	202	+
	203	+ /* convert both size (stored in 'rate') and time in terms of 'ns' */
	204	+ ns = timespec64_to_ns(&ts);
	205	+ rate = size * NSEC_PER_SEC;
	206	+
	207	+ /* Divide both size (stored in 'rate') and ns by a common factor */
	208	+ while (ns > UINT_MAX) {
	209	+ rate >>= 1;
	210	+ ns >>= 1;
	211	+ }
	212	+
	213	+ if (!ns)
	214	+ return;
	215	+
	216	+ /* calculate the rate */
	217	+ do_div(rate, (uint32_t)ns);
	218	+
	219	+ pr_info("\n%s => Size: %llu bytes\t DMA: %s\t Time: %llu.%09u seconds\t"
	220	+ "Rate: %llu KB/s\n", ops, size, dma ? "YES" : "NO",
	221	+ (u64)ts.tv_sec, (u32)ts.tv_nsec, rate / 1024);
	222	+}
	223	+
75	224	static int pci_epf_test_copy(struct pci_epf_test *epf_test)
76	225	{
77	226	int ret;
	227	+ bool use_dma;
78	228	void __iomem *src_addr;
79	229	void __iomem *dst_addr;
80	230	phys_addr_t src_phys_addr;
81	231	phys_addr_t dst_phys_addr;
	232	+ struct timespec64 start, end;
82	233	struct pci_epf *epf = epf_test->epf;
83	234	struct device *dev = &epf->dev;
84	235	struct pci_epc *epc = epf->epc;
@@ -117,8 +268,26 @@ static int pci_epf_test_copy(struct pci_epf_test *epf_test) · 32 unmodified lines
117	268	goto err_dst_addr;
118	269	}
119	270
120		− memcpy(dst_addr, src_addr, reg->size);
	271	+ ktime_get_ts64(&start);
	272	+ use_dma = !!(reg->flags & FLAG_USE_DMA);
	273	+ if (use_dma) {
	274	+ if (!epf_test->dma_supported) {
	275	+ dev_err(dev, "Cannot transfer data using DMA\n");
	276	+ ret = -EINVAL;
	277	+ goto err_map_addr;
	278	+ }
	279	+
	280	+ ret = pci_epf_test_data_transfer(epf_test, dst_phys_addr,
	281	+ src_phys_addr, reg->size);
	282	+ if (ret)
	283	+ dev_err(dev, "Data transfer failed\n");
	284	+ } else {
	285	+ memcpy(dst_addr, src_addr, reg->size);
	286	+ }
	287	+ ktime_get_ts64(&end);
	288	+ pci_epf_test_print_rate("COPY", reg->size, &start, &end, use_dma);
121	289
	290	+err_map_addr:
122	291	pci_epc_unmap_addr(epc, epf->func_no, dst_phys_addr);
123	292
124	293	err_dst_addr:
@@ -140,10 +309,14 @@ static int pci_epf_test_read(struct pci_epf_test *epf_test) · 15 unmodified lines
140	309	void __iomem *src_addr;
141	310	void *buf;
142	311	u32 crc32;
	312	+ bool use_dma;
143	313	phys_addr_t phys_addr;
	314	+ phys_addr_t dst_phys_addr;
	315	+ struct timespec64 start, end;
144	316	struct pci_epf *epf = epf_test->epf;
145	317	struct device *dev = &epf->dev;
146	318	struct pci_epc *epc = epf->epc;
	319	+ struct device *dma_dev = epf->epc->dev.parent;
147	320	enum pci_barno test_reg_bar = epf_test->test_reg_bar;
148	321	struct pci_epf_test_reg *reg = epf_test->reg[test_reg_bar];
149	322
@@ -169,12 +342,44 @@ static int pci_epf_test_read(struct pci_epf_test *epf_test) · 19 unmodified lines
169	342	goto err_map_addr;
170	343	}
171	344
172		− memcpy_fromio(buf, src_addr, reg->size);
	345	+ use_dma = !!(reg->flags & FLAG_USE_DMA);
	346	+ if (use_dma) {
	347	+ if (!epf_test->dma_supported) {
	348	+ dev_err(dev, "Cannot transfer data using DMA\n");
	349	+ ret = -EINVAL;
	350	+ goto err_dma_map;
	351	+ }
	352	+
	353	+ dst_phys_addr = dma_map_single(dma_dev, buf, reg->size,
	354	+ DMA_FROM_DEVICE);
	355	+ if (dma_mapping_error(dma_dev, dst_phys_addr)) {
	356	+ dev_err(dev, "Failed to map destination buffer addr\n");
	357	+ ret = -ENOMEM;
	358	+ goto err_dma_map;
	359	+ }
	360	+
	361	+ ktime_get_ts64(&start);
	362	+ ret = pci_epf_test_data_transfer(epf_test, dst_phys_addr,
	363	+ phys_addr, reg->size);
	364	+ if (ret)
	365	+ dev_err(dev, "Data transfer failed\n");
	366	+ ktime_get_ts64(&end);
	367	+
	368	+ dma_unmap_single(dma_dev, dst_phys_addr, reg->size,
	369	+ DMA_FROM_DEVICE);
	370	+ } else {
	371	+ ktime_get_ts64(&start);
	372	+ memcpy_fromio(buf, src_addr, reg->size);
	373	+ ktime_get_ts64(&end);
	374	+ }
	375	+
	376	+ pci_epf_test_print_rate("READ", reg->size, &start, &end, use_dma);
173	377
174	378	crc32 = crc32_le(~0, buf, reg->size);
175	379	if (crc32 != reg->checksum)
176	380	ret = -EIO;
177	381
	382	+err_dma_map:
178	383	kfree(buf);
179	384
180	385	err_map_addr:
@@ -192,10 +397,14 @@ static int pci_epf_test_write(struct pci_epf_test *epf_test) · 11 unmodified lines
192	397	int ret;
193	398	void __iomem *dst_addr;
194	399	void *buf;
	400	+ bool use_dma;
195	401	phys_addr_t phys_addr;
	402	+ phys_addr_t src_phys_addr;
	403	+ struct timespec64 start, end;
196	404	struct pci_epf *epf = epf_test->epf;
197	405	struct device *dev = &epf->dev;
198	406	struct pci_epc *epc = epf->epc;
	407	+ struct device *dma_dev = epf->epc->dev.parent;
199	408	enum pci_barno test_reg_bar = epf_test->test_reg_bar;
200	409	struct pci_epf_test_reg *reg = epf_test->reg[test_reg_bar];
201	410
@@ -224,7 +433,38 @@ static int pci_epf_test_write(struct pci_epf_test *epf_test) · 22 unmodified lines
224	433	get_random_bytes(buf, reg->size);
225	434	reg->checksum = crc32_le(~0, buf, reg->size);
226	435
227		− memcpy_toio(dst_addr, buf, reg->size);
	436	+ use_dma = !!(reg->flags & FLAG_USE_DMA);
	437	+ if (use_dma) {
	438	+ if (!epf_test->dma_supported) {
	439	+ dev_err(dev, "Cannot transfer data using DMA\n");
	440	+ ret = -EINVAL;
	441	+ goto err_map_addr;
	442	+ }
	443	+
	444	+ src_phys_addr = dma_map_single(dma_dev, buf, reg->size,
	445	+ DMA_TO_DEVICE);
	446	+ if (dma_mapping_error(dma_dev, src_phys_addr)) {
	447	+ dev_err(dev, "Failed to map source buffer addr\n");
	448	+ ret = -ENOMEM;
	449	+ goto err_dma_map;
	450	+ }
	451	+
	452	+ ktime_get_ts64(&start);
	453	+ ret = pci_epf_test_data_transfer(epf_test, phys_addr,
	454	+ src_phys_addr, reg->size);
	455	+ if (ret)
	456	+ dev_err(dev, "Data transfer failed\n");
	457	+ ktime_get_ts64(&end);
	458	+
	459	+ dma_unmap_single(dma_dev, src_phys_addr, reg->size,
	460	+ DMA_TO_DEVICE);
	461	+ } else {
	462	+ ktime_get_ts64(&start);
	463	+ memcpy_toio(dst_addr, buf, reg->size);
	464	+ ktime_get_ts64(&end);
	465	+ }
	466	+
	467	+ pci_epf_test_print_rate("WRITE", reg->size, &start, &end, use_dma);
228	468
229	469	/*
230	470	* wait 1ms inorder for the write to complete. Without this delay L3
@@ -232,6 +472,7 @@ static int pci_epf_test_write(struct pci_epf_test *epf_test) · 1 unmodified line
232	472	*/
233	473	usleep_range(1000, 2000);
234	474
	475	+err_dma_map:
235	476	kfree(buf);
236	477
237	478	err_map_addr:
@@ -360,14 +601,6 @@ reset_handler: · 122 unmodified lines
360	601	msecs_to_jiffies(1));
361	602	}
362	603
363		−static void pci_epf_test_linkup(struct pci_epf *epf)
364		−{
365		− struct pci_epf_test *epf_test = epf_get_drvdata(epf);
366		−
367		− queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
368		− msecs_to_jiffies(1));
369		−}
370		−
371	604	static void pci_epf_test_unbind(struct pci_epf *epf)
372	605	{
373	606	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
@@ -376,6 +609,7 @@ static void pci_epf_test_unbind(struct pci_epf *epf) · 2 unmodified lines
376	609	int bar;
377	610
378	611	cancel_delayed_work(&epf_test->cmd_handler);
	612	+ pci_epf_test_clean_dma_chan(epf_test);
379	613	pci_epc_stop(epc);
380	614	for (bar = 0; bar < PCI_STD_NUM_BARS; bar++) {
381	615	epf_bar = &epf->bar[bar];
@@ -424,11 +658,90 @@ static int pci_epf_test_set_bar(struct pci_epf *epf) · 42 unmodified lines
424	658	return 0;
425	659	}
426	660
	661	+static int pci_epf_test_core_init(struct pci_epf *epf)
	662	+{
	663	+ struct pci_epf_test *epf_test = epf_get_drvdata(epf);
	664	+ struct pci_epf_header *header = epf->header;
	665	+ const struct pci_epc_features *epc_features;
	666	+ struct pci_epc *epc = epf->epc;
	667	+ struct device *dev = &epf->dev;
	668	+ bool msix_capable = false;
	669	+ bool msi_capable = true;
	670	+ int ret;
	671	+
	672	+ epc_features = pci_epc_get_features(epc, epf->func_no);
	673	+ if (epc_features) {
	674	+ msix_capable = epc_features->msix_capable;
	675	+ msi_capable = epc_features->msi_capable;
	676	+ }
	677	+
	678	+ ret = pci_epc_write_header(epc, epf->func_no, header);
	679	+ if (ret) {
	680	+ dev_err(dev, "Configuration header write failed\n");
	681	+ return ret;
	682	+ }
	683	+
	684	+ ret = pci_epf_test_set_bar(epf);
	685	+ if (ret)
	686	+ return ret;
	687	+
	688	+ if (msi_capable) {
	689	+ ret = pci_epc_set_msi(epc, epf->func_no, epf->msi_interrupts);
	690	+ if (ret) {
	691	+ dev_err(dev, "MSI configuration failed\n");
	692	+ return ret;
	693	+ }
	694	+ }
	695	+
	696	+ if (msix_capable) {
	697	+ ret = pci_epc_set_msix(epc, epf->func_no, epf->msix_interrupts,
	698	+ epf_test->test_reg_bar,
	699	+ epf_test->msix_table_offset);
	700	+ if (ret) {
	701	+ dev_err(dev, "MSI-X configuration failed\n");
	702	+ return ret;
	703	+ }
	704	+ }
	705	+
	706	+ return 0;
	707	+}
	708	+
	709	+static int pci_epf_test_notifier(struct notifier_block *nb, unsigned long val,
	710	+ void *data)
	711	+{
	712	+ struct pci_epf *epf = container_of(nb, struct pci_epf, nb);
	713	+ struct pci_epf_test *epf_test = epf_get_drvdata(epf);
	714	+ int ret;
	715	+
	716	+ switch (val) {
	717	+ case CORE_INIT:
	718	+ ret = pci_epf_test_core_init(epf);
	719	+ if (ret)
	720	+ return NOTIFY_BAD;
	721	+ break;
	722	+
	723	+ case LINK_UP:
	724	+ queue_delayed_work(kpcitest_workqueue, &epf_test->cmd_handler,
	725	+ msecs_to_jiffies(1));
	726	+ break;
	727	+
	728	+ default:
	729	+ dev_err(&epf->dev, "Invalid EPF test notifier event\n");
	730	+ return NOTIFY_BAD;
	731	+ }
	732	+
	733	+ return NOTIFY_OK;
	734	+}
	735	+
427	736	static int pci_epf_test_alloc_space(struct pci_epf *epf)
428	737	{
429	738	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
430	739	struct device *dev = &epf->dev;
431	740	struct pci_epf_bar *epf_bar;
	741	+ size_t msix_table_size = 0;
	742	+ size_t test_reg_bar_size;
	743	+ size_t pba_size = 0;
	744	+ bool msix_capable;
432	745	void *base;
433	746	int bar, add;
434	747	enum pci_barno test_reg_bar = epf_test->test_reg_bar;
@@ -437,13 +750,25 @@ static int pci_epf_test_alloc_space(struct pci_epf *epf) · 2 unmodified lines
437	750
438	751	epc_features = epf_test->epc_features;
439	752
440		− if (epc_features->bar_fixed_size[test_reg_bar])
	753	+ test_reg_bar_size = ALIGN(sizeof(struct pci_epf_test_reg), 128);
	754	+
	755	+ msix_capable = epc_features->msix_capable;
	756	+ if (msix_capable) {
	757	+ msix_table_size = PCI_MSIX_ENTRY_SIZE * epf->msix_interrupts;
	758	+ epf_test->msix_table_offset = test_reg_bar_size;
	759	+ /* Align to QWORD or 8 Bytes */
	760	+ pba_size = ALIGN(DIV_ROUND_UP(epf->msix_interrupts, 8), 8);
	761	+ }
	762	+ test_reg_size = test_reg_bar_size + msix_table_size + pba_size;
	763	+
	764	+ if (epc_features->bar_fixed_size[test_reg_bar]) {
	765	+ if (test_reg_size > bar_size[test_reg_bar])
	766	+ return -ENOMEM;
441	767	test_reg_size = bar_size[test_reg_bar];
442		− else
443		− test_reg_size = sizeof(struct pci_epf_test_reg);
	768	+ }
444	769
445		− base = pci_epf_alloc_space(epf, test_reg_size,
446		− test_reg_bar, epc_features->align);
	770	+ base = pci_epf_alloc_space(epf, test_reg_size, test_reg_bar,
	771	+ epc_features->align);
447	772	if (!base) {
448	773	dev_err(dev, "Failed to allocated register space\n");
449	774	return -ENOMEM;
@@ -492,14 +817,11 @@ static int pci_epf_test_bind(struct pci_epf *epf) · 42 unmodified lines
492	817	{
493	818	int ret;
494	819	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
495		− struct pci_epf_header *header = epf->header;
496	820	const struct pci_epc_features *epc_features;
497	821	enum pci_barno test_reg_bar = BAR_0;
498	822	struct pci_epc *epc = epf->epc;
499		− struct device *dev = &epf->dev;
500	823	bool linkup_notifier = false;
501		− bool msix_capable = false;
502		− bool msi_capable = true;
	824	+ bool core_init_notifier = false;
503	825
504	826	if (WARN_ON_ONCE(!epc))
505	827	return -EINVAL;
@@ -507,8 +829,7 @@ static int pci_epf_test_bind(struct pci_epf *epf) · 1 unmodified line
507	829	epc_features = pci_epc_get_features(epc, epf->func_no);
508	830	if (epc_features) {
509	831	linkup_notifier = epc_features->linkup_notifier;
510		− msix_capable = epc_features->msix_capable;
511		− msi_capable = epc_features->msi_capable;
	832	+ core_init_notifier = epc_features->core_init_notifier;
512	833	test_reg_bar = pci_epc_get_first_free_bar(epc_features);
513	834	pci_epf_configure_bar(epf, epc_features);
514	835	}
@@ -516,38 +837,28 @@ static int pci_epf_test_bind(struct pci_epf *epf) · 1 unmodified line
516	837	epf_test->test_reg_bar = test_reg_bar;
517	838	epf_test->epc_features = epc_features;
518	839
519		− ret = pci_epc_write_header(epc, epf->func_no, header);
520		− if (ret) {
521		− dev_err(dev, "Configuration header write failed\n");
522		− return ret;
523		− }
524		−
525	840	ret = pci_epf_test_alloc_space(epf);
526	841	if (ret)
527	842	return ret;
528	843
529		− ret = pci_epf_test_set_bar(epf);
530		− if (ret)
531		− return ret;
532		−
533		− if (msi_capable) {
534		− ret = pci_epc_set_msi(epc, epf->func_no, epf->msi_interrupts);
535		− if (ret) {
536		− dev_err(dev, "MSI configuration failed\n");
	844	+ if (!core_init_notifier) {
	845	+ ret = pci_epf_test_core_init(epf);
	846	+ if (ret)
537	847	return ret;
538		− }
539	848	}
540	849
541		− if (msix_capable) {
542		− ret = pci_epc_set_msix(epc, epf->func_no, epf->msix_interrupts);
543		− if (ret) {
544		− dev_err(dev, "MSI-X configuration failed\n");
545		− return ret;
546		− }
547		− }
	850	+ epf_test->dma_supported = true;
548	851
549		− if (!linkup_notifier)
	852	+ ret = pci_epf_test_init_dma_chan(epf_test);
	853	+ if (ret)
	854	+ epf_test->dma_supported = false;
	855	+
	856	+ if (linkup_notifier) {
	857	+ epf->nb.notifier_call = pci_epf_test_notifier;
	858	+ pci_epc_register_notifier(epc, &epf->nb);
	859	+ } else {
550	860	queue_work(kpcitest_workqueue, &epf_test->cmd_handler.work);
	861	+ }
551	862
552	863	return 0;
553	864	}
@@ -580,7 +891,6 @@ static int pci_epf_test_probe(struct pci_epf *epf) · 26 unmodified lines
580	891	static struct pci_epf_ops ops = {
581	892	.unbind = pci_epf_test_unbind,
582	893	.bind = pci_epf_test_bind,
583		− .linkup = pci_epf_test_linkup,
584	894	};
585	895
586	896	static struct pci_epf_driver test_driver = {

drivers/pci/endpoint/pci-ep-cfs.c+6−22

@@ -29,7 +29,6 @@ struct pci_epc_group { · 28 unmodified lines
29	29	struct config_group group;
30	30	struct pci_epc *epc;
31	31	bool start;
32		− unsigned long function_num_map;
33	32	};
34	33
35	34	static inline struct pci_epf_group to_pci_epf_group(struct config_item item)
@@ -58,6 +57,7 @@ static ssize_t pci_epc_start_store(struct config_item item, const char page, · 22 unmodified lines
58	57
59	58	if (!start) {
60	59	pci_epc_stop(epc);
	60	+ epc_group->start = 0;
61	61	return len;
62	62	}
63	63
@@ -89,37 +89,22 @@ static int pci_epc_epf_link(struct config_item *epc_item, · 25 unmodified lines
89	89	struct config_item *epf_item)
90	90	{
91	91	int ret;
92		− u32 func_no = 0;
93	92	struct pci_epf_group *epf_group = to_pci_epf_group(epf_item);
94	93	struct pci_epc_group *epc_group = to_pci_epc_group(epc_item);
95	94	struct pci_epc *epc = epc_group->epc;
96	95	struct pci_epf *epf = epf_group->epf;
97	96
98		− func_no = find_first_zero_bit(&epc_group->function_num_map,
99		− BITS_PER_LONG);
100		− if (func_no >= BITS_PER_LONG)
101		− return -EINVAL;
102		−
103		− set_bit(func_no, &epc_group->function_num_map);
104		− epf->func_no = func_no;
105		−
106	97	ret = pci_epc_add_epf(epc, epf);
107	98	if (ret)
108		− goto err_add_epf;
	99	+ return ret;
109	100
110	101	ret = pci_epf_bind(epf);
111		− if (ret)
112		− goto err_epf_bind;
	102	+ if (ret) {
	103	+ pci_epc_remove_epf(epc, epf);
	104	+ return ret;
	105	+ }
113	106
114	107	return 0;
115		−
116		−err_epf_bind:
117		− pci_epc_remove_epf(epc, epf);
118		−
119		−err_add_epf:
120		− clear_bit(func_no, &epc_group->function_num_map);
121		−
122		− return ret;
123	108	}
124	109
125	110	static void pci_epc_epf_unlink(struct config_item *epc_item,
@@ -134,7 +119,6 @@ static void pci_epc_epf_unlink(struct config_item *epc_item, · 8 unmodified lines
134	119
135	120	epc = epc_group->epc;
136	121	epf = epf_group->epf;
137		− clear_bit(epf->func_no, &epc_group->function_num_map);
138	122	pci_epf_unbind(epf);
139	123	pci_epc_remove_epf(epc, epf);
140	124	}

drivers/pci/endpoint/pci-epc-core.c+75−62

@@ -120,7 +120,6 @@ const struct pci_epc_features pci_epc_get_features(struct pci_epc epc, · 119 unmodified lines
120	120	u8 func_no)
121	121	{
122	122	const struct pci_epc_features *epc_features;
123		− unsigned long flags;
124	123
125	124	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions)
126	125	return NULL;
@@ -128,9 +127,9 @@ const struct pci_epc_features pci_epc_get_features(struct pci_epc epc, · 1 unmodified line
128	127	if (!epc->ops->get_features)
129	128	return NULL;
130	129
131		− spin_lock_irqsave(&epc->lock, flags);
	130	+ mutex_lock(&epc->lock);
132	131	epc_features = epc->ops->get_features(epc, func_no);
133		− spin_unlock_irqrestore(&epc->lock, flags);
	132	+ mutex_unlock(&epc->lock);
134	133
135	134	return epc_features;
136	135	}
@@ -144,14 +143,12 @@ EXPORT_SYMBOL_GPL(pci_epc_get_features); · 7 unmodified lines
144	143	*/
145	144	void pci_epc_stop(struct pci_epc *epc)
146	145	{
147		− unsigned long flags;
148		−
149	146	if (IS_ERR(epc) \|\| !epc->ops->stop)
150	147	return;
151	148
152		− spin_lock_irqsave(&epc->lock, flags);
	149	+ mutex_lock(&epc->lock);
153	150	epc->ops->stop(epc);
154		− spin_unlock_irqrestore(&epc->lock, flags);
	151	+ mutex_unlock(&epc->lock);
155	152	}
156	153	EXPORT_SYMBOL_GPL(pci_epc_stop);
157	154
@@ -164,7 +161,6 @@ EXPORT_SYMBOL_GPL(pci_epc_stop); · 6 unmodified lines
164	161	int pci_epc_start(struct pci_epc *epc)
165	162	{
166	163	int ret;
167		− unsigned long flags;
168	164
169	165	if (IS_ERR(epc))
170	166	return -EINVAL;
@@ -172,9 +168,9 @@ int pci_epc_start(struct pci_epc *epc) · 1 unmodified line
172	168	if (!epc->ops->start)
173	169	return 0;
174	170
175		− spin_lock_irqsave(&epc->lock, flags);
	171	+ mutex_lock(&epc->lock);
176	172	ret = epc->ops->start(epc);
177		− spin_unlock_irqrestore(&epc->lock, flags);
	173	+ mutex_unlock(&epc->lock);
178	174
179	175	return ret;
180	176	}
@@ -193,7 +189,6 @@ int pci_epc_raise_irq(struct pci_epc *epc, u8 func_no, · 12 unmodified lines
193	189	enum pci_epc_irq_type type, u16 interrupt_num)
194	190	{
195	191	int ret;
196		− unsigned long flags;
197	192
198	193	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions)
199	194	return -EINVAL;
@@ -201,9 +196,9 @@ int pci_epc_raise_irq(struct pci_epc *epc, u8 func_no, · 1 unmodified line
201	196	if (!epc->ops->raise_irq)
202	197	return 0;
203	198
204		− spin_lock_irqsave(&epc->lock, flags);
	199	+ mutex_lock(&epc->lock);
205	200	ret = epc->ops->raise_irq(epc, func_no, type, interrupt_num);
206		− spin_unlock_irqrestore(&epc->lock, flags);
	201	+ mutex_unlock(&epc->lock);
207	202
208	203	return ret;
209	204	}
@@ -219,7 +214,6 @@ EXPORT_SYMBOL_GPL(pci_epc_raise_irq); · 9 unmodified lines
219	214	int pci_epc_get_msi(struct pci_epc *epc, u8 func_no)
220	215	{
221	216	int interrupt;
222		− unsigned long flags;
223	217
224	218	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions)
225	219	return 0;
@@ -227,9 +221,9 @@ int pci_epc_get_msi(struct pci_epc *epc, u8 func_no) · 1 unmodified line
227	221	if (!epc->ops->get_msi)
228	222	return 0;
229	223
230		− spin_lock_irqsave(&epc->lock, flags);
	224	+ mutex_lock(&epc->lock);
231	225	interrupt = epc->ops->get_msi(epc, func_no);
232		− spin_unlock_irqrestore(&epc->lock, flags);
	226	+ mutex_unlock(&epc->lock);
233	227
234	228	if (interrupt < 0)
235	229	return 0;
@@ -252,7 +246,6 @@ int pci_epc_set_msi(struct pci_epc *epc, u8 func_no, u8 interrupts) · 16 unmodified lines
252	246	{
253	247	int ret;
254	248	u8 encode_int;
255		− unsigned long flags;
256	249
257	250	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions \|\|
258	251	interrupts > 32)
@@ -263,9 +256,9 @@ int pci_epc_set_msi(struct pci_epc *epc, u8 func_no, u8 interrupts) · 4 unmodified lines
263	256
264	257	encode_int = order_base_2(interrupts);
265	258
266		− spin_lock_irqsave(&epc->lock, flags);
	259	+ mutex_lock(&epc->lock);
267	260	ret = epc->ops->set_msi(epc, func_no, encode_int);
268		− spin_unlock_irqrestore(&epc->lock, flags);
	261	+ mutex_unlock(&epc->lock);
269	262
270	263	return ret;
271	264	}
@@ -281,7 +274,6 @@ EXPORT_SYMBOL_GPL(pci_epc_set_msi); · 9 unmodified lines
281	274	int pci_epc_get_msix(struct pci_epc *epc, u8 func_no)
282	275	{
283	276	int interrupt;
284		− unsigned long flags;
285	277
286	278	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions)
287	279	return 0;
@@ -289,9 +281,9 @@ int pci_epc_get_msix(struct pci_epc *epc, u8 func_no) · 1 unmodified line
289	281	if (!epc->ops->get_msix)
290	282	return 0;
291	283
292		− spin_lock_irqsave(&epc->lock, flags);
	284	+ mutex_lock(&epc->lock);
293	285	interrupt = epc->ops->get_msix(epc, func_no);
294		− spin_unlock_irqrestore(&epc->lock, flags);
	286	+ mutex_unlock(&epc->lock);
295	287
296	288	if (interrupt < 0)
297	289	return 0;
@@ -305,13 +297,15 @@ EXPORT_SYMBOL_GPL(pci_epc_get_msix); · 7 unmodified lines
305	297	* @epc: the EPC device on which MSI-X has to be configured
306	298	* @func_no: the endpoint function number in the EPC device
307	299	* @interrupts: number of MSI-X interrupts required by the EPF
	300	+ * @bir: BAR where the MSI-X table resides
	301	+ * @offset: Offset pointing to the start of MSI-X table
308	302	*
309	303	* Invoke to set the required number of MSI-X interrupts.
310	304	*/
311		−int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts)
	305	+int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts,
	306	+ enum pci_barno bir, u32 offset)
312	307	{
313	308	int ret;
314		− unsigned long flags;
315	309
316	310	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions \|\|
317	311	interrupts < 1 \|\| interrupts > 2048)
@@ -320,9 +314,9 @@ int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts) · 2 unmodified lines
320	314	if (!epc->ops->set_msix)
321	315	return 0;
322	316
323		− spin_lock_irqsave(&epc->lock, flags);
324		− ret = epc->ops->set_msix(epc, func_no, interrupts - 1);
325		− spin_unlock_irqrestore(&epc->lock, flags);
	317	+ mutex_lock(&epc->lock);
	318	+ ret = epc->ops->set_msix(epc, func_no, interrupts - 1, bir, offset);
	319	+ mutex_unlock(&epc->lock);
326	320
327	321	return ret;
328	322	}
@@ -339,17 +333,15 @@ EXPORT_SYMBOL_GPL(pci_epc_set_msix); · 10 unmodified lines
339	333	void pci_epc_unmap_addr(struct pci_epc *epc, u8 func_no,
340	334	phys_addr_t phys_addr)
341	335	{
342		− unsigned long flags;
343		−
344	336	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions)
345	337	return;
346	338
347	339	if (!epc->ops->unmap_addr)
348	340	return;
349	341
350		− spin_lock_irqsave(&epc->lock, flags);
	342	+ mutex_lock(&epc->lock);
351	343	epc->ops->unmap_addr(epc, func_no, phys_addr);
352		− spin_unlock_irqrestore(&epc->lock, flags);
	344	+ mutex_unlock(&epc->lock);
353	345	}
354	346	EXPORT_SYMBOL_GPL(pci_epc_unmap_addr);
355	347
@@ -367,7 +359,6 @@ int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, · 11 unmodified lines
367	359	phys_addr_t phys_addr, u64 pci_addr, size_t size)
368	360	{
369	361	int ret;
370		− unsigned long flags;
371	362
372	363	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions)
373	364	return -EINVAL;
@@ -375,9 +366,9 @@ int pci_epc_map_addr(struct pci_epc *epc, u8 func_no, · 1 unmodified line
375	366	if (!epc->ops->map_addr)
376	367	return 0;
377	368
378		− spin_lock_irqsave(&epc->lock, flags);
	369	+ mutex_lock(&epc->lock);
379	370	ret = epc->ops->map_addr(epc, func_no, phys_addr, pci_addr, size);
380		− spin_unlock_irqrestore(&epc->lock, flags);
	371	+ mutex_unlock(&epc->lock);
381	372
382	373	return ret;
383	374	}
@@ -394,8 +385,6 @@ EXPORT_SYMBOL_GPL(pci_epc_map_addr); · 10 unmodified lines
394	385	void pci_epc_clear_bar(struct pci_epc *epc, u8 func_no,
395	386	struct pci_epf_bar *epf_bar)
396	387	{
397		− unsigned long flags;
398		−
399	388	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions \|\|
400	389	(epf_bar->barno == BAR_5 &&
401	390	epf_bar->flags & PCI_BASE_ADDRESS_MEM_TYPE_64))
@@ -404,9 +393,9 @@ void pci_epc_clear_bar(struct pci_epc *epc, u8 func_no, · 2 unmodified lines
404	393	if (!epc->ops->clear_bar)
405	394	return;
406	395
407		− spin_lock_irqsave(&epc->lock, flags);
	396	+ mutex_lock(&epc->lock);
408	397	epc->ops->clear_bar(epc, func_no, epf_bar);
409		− spin_unlock_irqrestore(&epc->lock, flags);
	398	+ mutex_unlock(&epc->lock);
410	399	}
411	400	EXPORT_SYMBOL_GPL(pci_epc_clear_bar);
412	401
@@ -422,7 +411,6 @@ int pci_epc_set_bar(struct pci_epc *epc, u8 func_no, · 9 unmodified lines
422	411	struct pci_epf_bar *epf_bar)
423	412	{
424	413	int ret;
425		− unsigned long irq_flags;
426	414	int flags = epf_bar->flags;
427	415
428	416	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions \|\|
@@ -437,9 +425,9 @@ int pci_epc_set_bar(struct pci_epc *epc, u8 func_no, · 8 unmodified lines
437	425	if (!epc->ops->set_bar)
438	426	return 0;
439	427
440		− spin_lock_irqsave(&epc->lock, irq_flags);
	428	+ mutex_lock(&epc->lock);
441	429	ret = epc->ops->set_bar(epc, func_no, epf_bar);
442		− spin_unlock_irqrestore(&epc->lock, irq_flags);
	430	+ mutex_unlock(&epc->lock);
443	431
444	432	return ret;
445	433	}
@@ -460,7 +448,6 @@ int pci_epc_write_header(struct pci_epc *epc, u8 func_no, · 14 unmodified lines
460	448	struct pci_epf_header *header)
461	449	{
462	450	int ret;
463		− unsigned long flags;
464	451
465	452	if (IS_ERR_OR_NULL(epc) \|\| func_no >= epc->max_functions)
466	453	return -EINVAL;
@@ -468,9 +455,9 @@ int pci_epc_write_header(struct pci_epc *epc, u8 func_no, · 1 unmodified line
468	455	if (!epc->ops->write_header)
469	456	return 0;
470	457
471		− spin_lock_irqsave(&epc->lock, flags);
	458	+ mutex_lock(&epc->lock);
472	459	ret = epc->ops->write_header(epc, func_no, header);
473		− spin_unlock_irqrestore(&epc->lock, flags);
	460	+ mutex_unlock(&epc->lock);
474	461
475	462	return ret;
476	463	}
@@ -487,7 +474,8 @@ EXPORT_SYMBOL_GPL(pci_epc_write_header); · 10 unmodified lines
487	474	*/
488	475	int pci_epc_add_epf(struct pci_epc epc, struct pci_epf epf)
489	476	{
490		− unsigned long flags;
	477	+ u32 func_no;
	478	+ int ret = 0;
491	479
492	480	if (epf->epc)
493	481	return -EBUSY;
@@ -495,16 +483,30 @@ int pci_epc_add_epf(struct pci_epc epc, struct pci_epf epf) · 1 unmodified line
495	483	if (IS_ERR(epc))
496	484	return -EINVAL;
497	485
498		− if (epf->func_no > epc->max_functions - 1)
499		− return -EINVAL;
	486	+ mutex_lock(&epc->lock);
	487	+ func_no = find_first_zero_bit(&epc->function_num_map,
	488	+ BITS_PER_LONG);
	489	+ if (func_no >= BITS_PER_LONG) {
	490	+ ret = -EINVAL;
	491	+ goto ret;
	492	+ }
500	493
	494	+ if (func_no > epc->max_functions - 1) {
	495	+ dev_err(&epc->dev, "Exceeding max supported Function Number\n");
	496	+ ret = -EINVAL;
	497	+ goto ret;
	498	+ }
	499	+
	500	+ set_bit(func_no, &epc->function_num_map);
	501	+ epf->func_no = func_no;
501	502	epf->epc = epc;
502	503
503		− spin_lock_irqsave(&epc->lock, flags);
504	504	list_add_tail(&epf->list, &epc->pci_epf);
505		− spin_unlock_irqrestore(&epc->lock, flags);
506	505
507		− return 0;
	506	+ret:
	507	+ mutex_unlock(&epc->lock);
	508	+
	509	+ return ret;
508	510	}
509	511	EXPORT_SYMBOL_GPL(pci_epc_add_epf);
510	512
@@ -517,15 +519,14 @@ EXPORT_SYMBOL_GPL(pci_epc_add_epf); · 6 unmodified lines
517	519	*/
518	520	void pci_epc_remove_epf(struct pci_epc epc, struct pci_epf epf)
519	521	{
520		− unsigned long flags;
521		−
522	522	if (!epc \|\| IS_ERR(epc) \|\| !epf)
523	523	return;
524	524
525		− spin_lock_irqsave(&epc->lock, flags);
	525	+ mutex_lock(&epc->lock);
	526	+ clear_bit(epf->func_no, &epc->function_num_map);
526	527	list_del(&epf->list);
527	528	epf->epc = NULL;
528		− spin_unlock_irqrestore(&epc->lock, flags);
	529	+ mutex_unlock(&epc->lock);
529	530	}
530	531	EXPORT_SYMBOL_GPL(pci_epc_remove_epf);
531	532
@@ -539,19 +540,30 @@ EXPORT_SYMBOL_GPL(pci_epc_remove_epf); · 7 unmodified lines
539	540	*/
540	541	void pci_epc_linkup(struct pci_epc *epc)
541	542	{
542		− unsigned long flags;
543		− struct pci_epf *epf;
544		−
545	543	if (!epc \|\| IS_ERR(epc))
546	544	return;
547	545
548		− spin_lock_irqsave(&epc->lock, flags);
549		− list_for_each_entry(epf, &epc->pci_epf, list)
550		− pci_epf_linkup(epf);
551		− spin_unlock_irqrestore(&epc->lock, flags);
	546	+ atomic_notifier_call_chain(&epc->notifier, LINK_UP, NULL);
552	547	}
553	548	EXPORT_SYMBOL_GPL(pci_epc_linkup);
554	549
	550	+/**
	551	+ * pci_epc_init_notify() - Notify the EPF device that EPC device's core
	552	+ * initialization is completed.
	553	+ * @epc: the EPC device whose core initialization is completeds
	554	+ *
	555	+ * Invoke to Notify the EPF device that the EPC device's initialization
	556	+ * is completed.
	557	+ */
	558	+void pci_epc_init_notify(struct pci_epc *epc)
	559	+{
	560	+ if (!epc \|\| IS_ERR(epc))
	561	+ return;
	562	+
	563	+ atomic_notifier_call_chain(&epc->notifier, CORE_INIT, NULL);
	564	+}
	565	+EXPORT_SYMBOL_GPL(pci_epc_init_notify);
	566	+
555	567	/**
556	568	* pci_epc_destroy() - destroy the EPC device
557	569	* @epc: the EPC device that has to be destroyed
@@ -610,8 +622,9 @@ __pci_epc_create(struct device dev, const struct pci_epc_ops ops, · 52 unmodified lines
610	622	goto err_ret;
611	623	}
612	624
613		− spin_lock_init(&epc->lock);
	625	+ mutex_init(&epc->lock);
614	626	INIT_LIST_HEAD(&epc->pci_epf);
	627	+ ATOMIC_INIT_NOTIFIER_HEAD(&epc->notifier);
615	628
616	629	device_initialize(&epc->dev);
617	630	epc->dev.class = pci_epc_class;

drivers/pci/endpoint/pci-epc-mem.c+8−2

@@ -79,6 +79,7 @@ int __pci_epc_mem_init(struct pci_epc *epc, phys_addr_t phys_base, size_t size, · 78 unmodified lines
79	79	mem->page_size = page_size;
80	80	mem->pages = pages;
81	81	mem->size = size;
	82	+ mutex_init(&mem->lock);
82	83
83	84	epc->mem = mem;
84	85
@@ -122,7 +123,7 @@ void __iomem pci_epc_mem_alloc_addr(struct pci_epc epc, · 37 unmodified lines
122	123	phys_addr_t *phys_addr, size_t size)
123	124	{
124	125	int pageno;
125		− void __iomem *virt_addr;
	126	+ void __iomem *virt_addr = NULL;
126	127	struct pci_epc_mem *mem = epc->mem;
127	128	unsigned int page_shift = ilog2(mem->page_size);
128	129	int order;
@@ -130,15 +131,18 @@ void __iomem pci_epc_mem_alloc_addr(struct pci_epc epc, · 1 unmodified line
130	131	size = ALIGN(size, mem->page_size);
131	132	order = pci_epc_mem_get_order(mem, size);
132	133
	134	+ mutex_lock(&mem->lock);
133	135	pageno = bitmap_find_free_region(mem->bitmap, mem->pages, order);
134	136	if (pageno < 0)
135		− return NULL;
	137	+ goto ret;
136	138
137	139	*phys_addr = mem->phys_base + ((phys_addr_t)pageno << page_shift);
138	140	virt_addr = ioremap(*phys_addr, size);
139	141	if (!virt_addr)
140	142	bitmap_release_region(mem->bitmap, pageno, order);
141	143
	144	+ret:
	145	+ mutex_unlock(&mem->lock);
142	146	return virt_addr;
143	147	}
144	148	EXPORT_SYMBOL_GPL(pci_epc_mem_alloc_addr);
@@ -164,7 +168,9 @@ void pci_epc_mem_free_addr(struct pci_epc *epc, phys_addr_t phys_addr, · 19 unmodified lines
164	168	pageno = (phys_addr - mem->phys_base) >> page_shift;
165	169	size = ALIGN(size, mem->page_size);
166	170	order = pci_epc_mem_get_order(mem, size);
	171	+ mutex_lock(&mem->lock);
167	172	bitmap_release_region(mem->bitmap, pageno, order);
	173	+ mutex_unlock(&mem->lock);
168	174	}
169	175	EXPORT_SYMBOL_GPL(pci_epc_mem_free_addr);
170	176

drivers/pci/endpoint/pci-epf-core.c+13−22

@@ -20,26 +20,6 @@ static DEFINE_MUTEX(pci_epf_mutex); · 19 unmodified lines
20	20	static struct bus_type pci_epf_bus_type;
21	21	static const struct device_type pci_epf_type;
22	22
23		−/**
24		− * pci_epf_linkup() - Notify the function driver that EPC device has
25		− * established a connection with the Root Complex.
26		− * @epf: the EPF device bound to the EPC device which has established
27		− * the connection with the host
28		− *
29		− * Invoke to notify the function driver that EPC device has established
30		− * a connection with the Root Complex.
31		− */
32		−void pci_epf_linkup(struct pci_epf *epf)
33		−{
34		− if (!epf->driver) {
35		− dev_WARN(&epf->dev, "epf device not bound to driver\n");
36		− return;
37		− }
38		−
39		− epf->driver->ops->linkup(epf);
40		−}
41		−EXPORT_SYMBOL_GPL(pci_epf_linkup);
42		−
43	23	/**
44	24	* pci_epf_unbind() - Notify the function driver that the binding between the
45	25	* EPF device and EPC device has been lost
@@ -55,7 +35,9 @@ void pci_epf_unbind(struct pci_epf *epf) · 9 unmodified lines
55	35	return;
56	36	}
57	37
	38	+ mutex_lock(&epf->lock);
58	39	epf->driver->ops->unbind(epf);
	40	+ mutex_unlock(&epf->lock);
59	41	module_put(epf->driver->owner);
60	42	}
61	43	EXPORT_SYMBOL_GPL(pci_epf_unbind);
@@ -69,6 +51,8 @@ EXPORT_SYMBOL_GPL(pci_epf_unbind); · 7 unmodified lines
69	51	*/
70	52	int pci_epf_bind(struct pci_epf *epf)
71	53	{
	54	+ int ret;
	55	+
72	56	if (!epf->driver) {
73	57	dev_WARN(&epf->dev, "epf device not bound to driver\n");
74	58	return -EINVAL;
@@ -77,7 +61,11 @@ int pci_epf_bind(struct pci_epf *epf) · 2 unmodified lines
77	61	if (!try_module_get(epf->driver->owner))
78	62	return -EAGAIN;
79	63
80		− return epf->driver->ops->bind(epf);
	64	+ mutex_lock(&epf->lock);
	65	+ ret = epf->driver->ops->bind(epf);
	66	+ mutex_unlock(&epf->lock);
	67	+
	68	+ return ret;
81	69	}
82	70	EXPORT_SYMBOL_GPL(pci_epf_bind);
83	71
@@ -99,6 +87,7 @@ void pci_epf_free_space(struct pci_epf epf, void addr, enum pci_barno bar) · 15 unmodified lines
99	87	epf->bar[bar].phys_addr);
100	88
101	89	epf->bar[bar].phys_addr = 0;
	90	+ epf->bar[bar].addr = NULL;
102	91	epf->bar[bar].size = 0;
103	92	epf->bar[bar].barno = 0;
104	93	epf->bar[bar].flags = 0;
@@ -135,6 +124,7 @@ void pci_epf_alloc_space(struct pci_epf epf, size_t size, enum pci_barno bar, · 30 unmodified lines
135	124	}
136	125
137	126	epf->bar[bar].phys_addr = phys_addr;
	127	+ epf->bar[bar].addr = space;
138	128	epf->bar[bar].size = size;
139	129	epf->bar[bar].barno = bar;
140	130	epf->bar[bar].flags \|= upper_32_bits(size) ?
@@ -214,7 +204,7 @@ int __pci_epf_register_driver(struct pci_epf_driver *driver, · 73 unmodified lines
214	204	if (!driver->ops)
215	205	return -EINVAL;
216	206
217		− if (!driver->ops->bind \|\| !driver->ops->unbind \|\| !driver->ops->linkup)
	207	+ if (!driver->ops->bind \|\| !driver->ops->unbind)
218	208	return -EINVAL;
219	209
220	210	driver->driver.bus = &pci_epf_bus_type;
@@ -272,6 +262,7 @@ struct pci_epf pci_epf_create(const char name) · 51 unmodified lines
272	262	device_initialize(dev);
273	263	dev->bus = &pci_epf_bus_type;
274	264	dev->type = &pci_epf_type;
	265	+ mutex_init(&epf->lock);
275	266
276	267	ret = dev_set_name(dev, "%s", name);
277	268	if (ret) {

include/linux/pci-epc.h+22−5

@@ -53,7 +53,8 @@ struct pci_epc_ops { · 52 unmodified lines
53	53	phys_addr_t addr);
54	54	int (set_msi)(struct pci_epc epc, u8 func_no, u8 interrupts);
55	55	int (get_msi)(struct pci_epc epc, u8 func_no);
56		− int (set_msix)(struct pci_epc epc, u8 func_no, u16 interrupts);
	56	+ int (set_msix)(struct pci_epc epc, u8 func_no, u16 interrupts,
	57	+ enum pci_barno, u32 offset);
57	58	int (get_msix)(struct pci_epc epc, u8 func_no);
58	59	int (raise_irq)(struct pci_epc epc, u8 func_no,
59	60	enum pci_epc_irq_type type, u16 interrupt_num);
@@ -71,6 +72,7 @@ struct pci_epc_ops { · 11 unmodified lines
71	72	* @bitmap: bitmap to manage the PCI address space
72	73	* @pages: number of bits representing the address region
73	74	* @page_size: size of each page
	75	+ * @lock: mutex to protect bitmap
74	76	*/
75	77	struct pci_epc_mem {
76	78	phys_addr_t phys_base;
@@ -78,6 +80,8 @@ struct pci_epc_mem { · 1 unmodified line
78	80	unsigned long *bitmap;
79	81	size_t page_size;
80	82	int pages;
	83	+ /* mutex to protect against concurrent access for memory allocation*/
	84	+ struct mutex lock;
81	85	};
82	86
83	87	/**
@@ -88,7 +92,9 @@ struct pci_epc_mem { · 4 unmodified lines
88	92	* @mem: address space of the endpoint controller
89	93	* @max_functions: max number of functions that can be configured in this EPC
90	94	* @group: configfs group representing the PCI EPC device
91		− * @lock: spinlock to protect pci_epc ops
	95	+ * @lock: mutex to protect pci_epc ops
	96	+ * @function_num_map: bitmap to manage physical function number
	97	+ * @notifier: used to notify EPF of any EPC events (like linkup)
92	98	*/
93	99	struct pci_epc {
94	100	struct device dev;
@@ -97,8 +103,10 @@ struct pci_epc { · 2 unmodified lines
97	103	struct pci_epc_mem *mem;
98	104	u8 max_functions;
99	105	struct config_group *group;
100		− /* spinlock to protect against concurrent access of EP controller */
101		− spinlock_t lock;
	106	+ /* mutex to protect against concurrent access of EP controller */
	107	+ struct mutex lock;
	108	+ unsigned long function_num_map;
	109	+ struct atomic_notifier_head notifier;
102	110	};
103	111
104	112	/**
@@ -113,6 +121,7 @@ struct pci_epc { · 8 unmodified lines
113	121	*/
114	122	struct pci_epc_features {
115	123	unsigned int linkup_notifier : 1;
	124	+ unsigned int core_init_notifier : 1;
116	125	unsigned int msi_capable : 1;
117	126	unsigned int msix_capable : 1;
118	127	u8 reserved_bar;
@@ -141,6 +150,12 @@ static inline void epc_get_drvdata(struct pci_epc epc) · 22 unmodified lines
141	150	return dev_get_drvdata(&epc->dev);
142	151	}
143	152
	153	+static inline int
	154	+pci_epc_register_notifier(struct pci_epc epc, struct notifier_block nb)
	155	+{
	156	+ return atomic_notifier_chain_register(&epc->notifier, nb);
	157	+}
	158	+
144	159	struct pci_epc *
145	160	__devm_pci_epc_create(struct device dev, const struct pci_epc_ops ops,
146	161	struct module *owner);
@@ -151,6 +166,7 @@ void devm_pci_epc_destroy(struct device dev, struct pci_epc epc); · 4 unmodified lines
151	166	void pci_epc_destroy(struct pci_epc *epc);
152	167	int pci_epc_add_epf(struct pci_epc epc, struct pci_epf epf);
153	168	void pci_epc_linkup(struct pci_epc *epc);
	169	+void pci_epc_init_notify(struct pci_epc *epc);
154	170	void pci_epc_remove_epf(struct pci_epc epc, struct pci_epf epf);
155	171	int pci_epc_write_header(struct pci_epc *epc, u8 func_no,
156	172	struct pci_epf_header *hdr);
@@ -165,7 +181,8 @@ void pci_epc_unmap_addr(struct pci_epc *epc, u8 func_no, · 8 unmodified lines
165	181	phys_addr_t phys_addr);
166	182	int pci_epc_set_msi(struct pci_epc *epc, u8 func_no, u8 interrupts);
167	183	int pci_epc_get_msi(struct pci_epc *epc, u8 func_no);
168		−int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts);
	184	+int pci_epc_set_msix(struct pci_epc *epc, u8 func_no, u16 interrupts,
	185	+ enum pci_barno, u32 offset);
169	186	int pci_epc_get_msix(struct pci_epc *epc, u8 func_no);
170	187	int pci_epc_raise_irq(struct pci_epc *epc, u8 func_no,
171	188	enum pci_epc_irq_type type, u16 interrupt_num);

include/linux/pci-epf.h+25−4

@@ -15,6 +15,11 @@ · 14 unmodified lines
15	15
16	16	struct pci_epf;
17	17
	18	+enum pci_notify_event {
	19	+ CORE_INIT,
	20	+ LINK_UP,
	21	+};
	22	+
18	23	enum pci_barno {
19	24	BAR_0,
20	25	BAR_1,
@@ -55,13 +60,10 @@ struct pci_epf_header { · 34 unmodified lines
55	60	* @bind: ops to perform when a EPC device has been bound to EPF device
56	61	* @unbind: ops to perform when a binding has been lost between a EPC device
57	62	* and EPF device
58		− * @linkup: ops to perform when the EPC device has established a connection with
59		− * a host system
60	63	*/
61	64	struct pci_epf_ops {
62	65	int (bind)(struct pci_epf epf);
63	66	void (unbind)(struct pci_epf epf);
64		− void (linkup)(struct pci_epf epf);
65	67	};
66	68
67	69	/**
@@ -92,10 +94,12 @@ struct pci_epf_driver { · 24 unmodified lines
92	94	/**
93	95	* struct pci_epf_bar - represents the BAR of EPF device
94	96	* @phys_addr: physical address that should be mapped to the BAR
	97	+ * @addr: virtual address corresponding to the @phys_addr
95	98	* @size: the size of the address space present in BAR
96	99	*/
97	100	struct pci_epf_bar {
98	101	dma_addr_t phys_addr;
	102	+ void *addr;
99	103	size_t size;
100	104	enum pci_barno barno;
101	105	int flags;
@@ -112,6 +116,8 @@ struct pci_epf_bar { · 10 unmodified lines
112	116	* @epc: the EPC device to which this EPF device is bound
113	117	* @driver: the EPF driver to which this EPF device is bound
114	118	* @list: to add pci_epf as a list of PCI endpoint functions to pci_epc
	119	+ * @nb: notifier block to notify EPF of any EPC events (like linkup)
	120	+ * @lock: mutex to protect pci_epf_ops
115	121	*/
116	122	struct pci_epf {
117	123	struct device dev;
@@ -125,6 +131,22 @@ struct pci_epf { · 7 unmodified lines
125	131	struct pci_epc *epc;
126	132	struct pci_epf_driver *driver;
127	133	struct list_head list;
	134	+ struct notifier_block nb;
	135	+ /* mutex to protect against concurrent access of pci_epf_ops */
	136	+ struct mutex lock;
	137	+};
	138	+
	139	+/**
	140	+ * struct pci_epf_msix_tbl - represents the MSIX table entry structure
	141	+ * @msg_addr: Writes to this address will trigger MSIX interrupt in host
	142	+ * @msg_data: Data that should be written to @msg_addr to trigger MSIX interrupt
	143	+ * @vector_ctrl: Identifies if the function is prohibited from sending a message
	144	+ * using this MSIX table entry
	145	+ */
	146	+struct pci_epf_msix_tbl {
	147	+ u64 msg_addr;
	148	+ u32 msg_data;
	149	+ u32 vector_ctrl;
128	150	};
129	151
130	152	#define to_pci_epf(epf_dev) container_of((epf_dev), struct pci_epf, dev)
@@ -154,5 +176,4 @@ void pci_epf_alloc_space(struct pci_epf epf, size_t size, enum pci_barno bar, · 23 unmodified lines
154	176	void pci_epf_free_space(struct pci_epf epf, void addr, enum pci_barno bar);
155	177	int pci_epf_bind(struct pci_epf *epf);
156	178	void pci_epf_unbind(struct pci_epf *epf);
157		−void pci_epf_linkup(struct pci_epf *epf);
158	179	#endif /* __LINUX_PCI_EPF_H */

include/soc/tegra/bpmp-abi.h+9−1

@@ -1,6 +1,6 @@
1	1	/* SPDX-License-Identifier: GPL-2.0-only */
2	2	/*
3		− * Copyright (c) 2014-2018, NVIDIA CORPORATION. All rights reserved.
	3	+ * Copyright (c) 2014-2020, NVIDIA CORPORATION. All rights reserved.
4	4	*/
5	5
6	6	#ifndef _ABI_BPMP_ABI_H_
@@ -2119,6 +2119,7 @@ enum { · 2112 unmodified lines
2119	2119	CMD_UPHY_PCIE_LANE_MARGIN_STATUS = 2,
2120	2120	CMD_UPHY_PCIE_EP_CONTROLLER_PLL_INIT = 3,
2121	2121	CMD_UPHY_PCIE_CONTROLLER_STATE = 4,
	2122	+ CMD_UPHY_PCIE_EP_CONTROLLER_PLL_OFF = 5,
2122	2123	CMD_UPHY_MAX,
2123	2124	};
2124	2125
@@ -2151,6 +2152,11 @@ struct cmd_uphy_pcie_controller_state_request { · 26 unmodified lines
2151	2152	uint8_t enable;
2152	2153	} __ABI_PACKED;
2153	2154
	2155	+struct cmd_uphy_ep_controller_pll_off_request {
	2156	+ /** @brief EP controller number, valid: 0, 4, 5 */
	2157	+ uint8_t ep_controller;
	2158	+} __ABI_PACKED;
	2159	+
2154	2160	/**
2155	2161	* @ingroup UPHY
2156	2162	* @brief Request with #MRQ_UPHY
@@ -2165,6 +2171,7 @@ struct cmd_uphy_pcie_controller_state_request { · 8 unmodified lines
2165	2171	* \|CMD_UPHY_PCIE_LANE_MARGIN_STATUS \| \|
2166	2172	* \|CMD_UPHY_PCIE_EP_CONTROLLER_PLL_INIT \|cmd_uphy_ep_controller_pll_init_request \|
2167	2173	* \|CMD_UPHY_PCIE_CONTROLLER_STATE \|cmd_uphy_pcie_controller_state_request \|
	2174	+ * \|CMD_UPHY_PCIE_EP_CONTROLLER_PLL_OFF \|cmd_uphy_ep_controller_pll_off_request \|
2168	2175	*
2169	2176	*/
2170	2177
@@ -2178,6 +2185,7 @@ struct mrq_uphy_request { · 7 unmodified lines
2178	2185	struct cmd_uphy_margin_control_request uphy_set_margin_control;
2179	2186	struct cmd_uphy_ep_controller_pll_init_request ep_ctrlr_pll_init;
2180	2187	struct cmd_uphy_pcie_controller_state_request controller_state;
	2188	+ struct cmd_uphy_ep_controller_pll_off_request ep_ctrlr_pll_off;
2181	2189	} __UNION_ANON;
2182	2190	} __ABI_PACKED;
2183	2191

include/uapi/linux/pcitest.h+8−0

@@ -19,5 +19,13 @@ · 18 unmodified lines
19	19	#define PCITEST_MSIX _IOW('P', 0x7, int)
20	20	#define PCITEST_SET_IRQTYPE _IOW('P', 0x8, int)
21	21	#define PCITEST_GET_IRQTYPE _IO('P', 0x9)
	22	+#define PCITEST_CLEAR_IRQ _IO('P', 0x10)
	23	+
	24	+#define PCITEST_FLAGS_USE_DMA 0x00000001
	25	+
	26	+struct pci_endpoint_test_xfer_param {
	27	+ unsigned long size;
	28	+ unsigned char flags;
	29	+};
22	30
23	31	#endif /* __UAPI_LINUX_PCITEST_H */

tools/pci/pcitest.c+33−4

@@ -30,14 +30,17 @@ struct pci_test { · 29 unmodified lines
30	30	int irqtype;
31	31	bool set_irqtype;
32	32	bool get_irqtype;
	33	+ bool clear_irq;
33	34	bool read;
34	35	bool write;
35	36	bool copy;
36	37	unsigned long size;
	38	+ bool use_dma;
37	39	};
38	40
39	41	static int run_test(struct pci_test *test)
40	42	{
	43	+ struct pci_endpoint_test_xfer_param param;
41	44	int ret = -EINVAL;
42	45	int fd;
43	46
@@ -74,6 +77,15 @@ static int run_test(struct pci_test *test) · 30 unmodified lines
74	77	fprintf(stdout, "%s\n", irq[ret]);
75	78	}
76	79
	80	+ if (test->clear_irq) {
	81	+ ret = ioctl(fd, PCITEST_CLEAR_IRQ);
	82	+ fprintf(stdout, "CLEAR IRQ:\t\t");
	83	+ if (ret < 0)
	84	+ fprintf(stdout, "FAILED\n");
	85	+ else
	86	+ fprintf(stdout, "%s\n", result[ret]);
	87	+ }
	88	+
77	89	if (test->legacyirq) {
78	90	ret = ioctl(fd, PCITEST_LEGACY_IRQ, 0);
79	91	fprintf(stdout, "LEGACY IRQ:\t");
@@ -102,7 +114,10 @@ static int run_test(struct pci_test *test) · 22 unmodified lines
102	114	}
103	115
104	116	if (test->write) {
105		− ret = ioctl(fd, PCITEST_WRITE, test->size);
	117	+ param.size = test->size;
	118	+ if (test->use_dma)
	119	+ param.flags = PCITEST_FLAGS_USE_DMA;
	120	+ ret = ioctl(fd, PCITEST_WRITE, &param);
106	121	fprintf(stdout, "WRITE (%7ld bytes):\t\t", test->size);
107	122	if (ret < 0)
108	123	fprintf(stdout, "TEST FAILED\n");
@@ -111,7 +126,10 @@ static int run_test(struct pci_test *test) · 2 unmodified lines
111	126	}
112	127
113	128	if (test->read) {
114		− ret = ioctl(fd, PCITEST_READ, test->size);
	129	+ param.size = test->size;
	130	+ if (test->use_dma)
	131	+ param.flags = PCITEST_FLAGS_USE_DMA;
	132	+ ret = ioctl(fd, PCITEST_READ, &param);
115	133	fprintf(stdout, "READ (%7ld bytes):\t\t", test->size);
116	134	if (ret < 0)
117	135	fprintf(stdout, "TEST FAILED\n");
@@ -120,7 +138,10 @@ static int run_test(struct pci_test *test) · 2 unmodified lines
120	138	}
121	139
122	140	if (test->copy) {
123		− ret = ioctl(fd, PCITEST_COPY, test->size);
	141	+ param.size = test->size;
	142	+ if (test->use_dma)
	143	+ param.flags = PCITEST_FLAGS_USE_DMA;
	144	+ ret = ioctl(fd, PCITEST_COPY, &param);
124	145	fprintf(stdout, "COPY (%7ld bytes):\t\t", test->size);
125	146	if (ret < 0)
126	147	fprintf(stdout, "TEST FAILED\n");
@@ -153,7 +174,7 @@ int main(int argc, char **argv) · 26 unmodified lines
153	174	/* set default endpoint device */
154	175	test->device = "/dev/pci-endpoint-test.0";
155	176
156		− while ((c = getopt(argc, argv, "D:b:m:x:i:Ilhrwcs:")) != EOF)
	177	+ while ((c = getopt(argc, argv, "D:b:m:x:i:deIlhrwcs:")) != EOF)
157	178	switch (c) {
158	179	case 'D':
159	180	test->device = optarg;
@@ -194,9 +215,15 @@ int main(int argc, char **argv) · 34 unmodified lines
194	215	case 'c':
195	216	test->copy = true;
196	217	continue;
	218	+ case 'e':
	219	+ test->clear_irq = true;
	220	+ continue;
197	221	case 's':
198	222	test->size = strtoul(optarg, NULL, 0);
199	223	continue;
	224	+ case 'd':
	225	+ test->use_dma = true;
	226	+ continue;
200	227	case 'h':
201	228	default:
202	229	usage:
@@ -208,7 +235,9 @@ usage: · 5 unmodified lines
208	235	"\t-m <msi num> MSI test (msi number between 1..32)\n"
209	236	"\t-x <msix num> \tMSI-X test (msix number between 1..2048)\n"
210	237	"\t-i <irq type> \tSet IRQ type (0 - Legacy, 1 - MSI, 2 - MSI-X)\n"
	238	+ "\t-e Clear IRQ\n"
211	239	"\t-I Get current IRQ type configured\n"
	240	+ "\t-d Use DMA\n"
212	241	"\t-l Legacy IRQ test\n"
213	242	"\t-r Read buffer test\n"
214	243	"\t-w Write buffer test\n"

Merge branch 'remotes/lorenzo/pci/endpoint'

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code