History

Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma

Pull rdma updates from Jason Gunthorpe: - Various minor code cleanups and fixes for hns, iser, cxgb4, hfi1, rxe, erdma, mana_ib - Prefetch supprot for rxe ODP - Remove memory window support from hns as new device FW is no longer support it - Remove qib, it is very old and obsolete now, Cornelis wishes to restructure the hfi1/qib shared layer - Fix a race in destroying CQs where we can still end up with work running because the work is cancled before the driver stops triggering it - Improve interaction with namespaces: * Follow the devlink namespace for newly spawned RDMA devices * Create iopoib net devces in the parent IB device's namespace * Allow CAP_NET_RAW checks to pass in user namespaces - A new flow control scheme for IB MADs to try and avoid queue overflows in the network - Fix 2G message sizes in bnxt_re - Optimize mkey layout for mlx5 DMABUF - New "DMA Handle" concept to allow controlling PCI TPH and steering tags * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (71 commits) RDMA/siw: Change maintainer email address RDMA/mana_ib: add support of multiple ports RDMA/mlx5: Refactor optional counters steering code RDMA/mlx5: Add DMAH support for reg_user_mr/reg_user_dmabuf_mr IB: Extend UVERBS_METHOD_REG_MR to get DMAH RDMA/mlx5: Add DMAH object support RDMA/core: Introduce a DMAH object and its alloc/free APIs IB/core: Add UVERBS_METHOD_REG_MR on the MR object net/mlx5: Add support for device steering tag net/mlx5: Expose IFC bits for TPH PCI/TPH: Expose pcie_tph_get_st_table_size() RDMA/mlx5: Fix incorrect MKEY masking RDMA/mlx5: Fix returned type from _mlx5r_umr_zap_mkey() RDMA/mlx5: remove redundant check on err on return expression RDMA/mana_ib: add additional port counters RDMA/mana_ib: Fix DSCP value in modify QP RDMA/efa: Add CQ with external memory support RDMA/core: Add umem "is_contiguous" and "start_dma_addr" helpers RDMA/uverbs: Add a common way to create CQ with umem RDMA/mlx5: Optimize DMABUF mkey page size ...

Linus Torvalds committed 1 year ago

Commit 7ce4de12 parents 2c8c9aa ee23592153 files · +2869 −49167Browse files

Unified · Split

153 files
  1. .mailmap +1 −0
  2. MAINTAINERS +1 −7
  3. drivers/infiniband/Kconfig +0 −1
  4. drivers/infiniband/core/Makefile +1 −0
  5. drivers/infiniband/core/cm.c +41 −6
  6. drivers/infiniband/core/counters.c +1 −1
  7. drivers/infiniband/core/cq.c +10 −2
  8. drivers/infiniband/core/device.c +45 −2
  9. drivers/infiniband/core/mad.c +362 −106
  10. drivers/infiniband/core/mad_priv.h +73 −3
  11. drivers/infiniband/core/mad_rmpp.c +24 −17
  12. drivers/infiniband/core/nldev.c +13 −11
  13. drivers/infiniband/core/rdma_core.c +29 −0
  14. drivers/infiniband/core/rdma_core.h +1 −0
  15. drivers/infiniband/core/restrack.c +2 −0
  16. drivers/infiniband/core/uverbs_cmd.c +7 −6
  17. drivers/infiniband/core/uverbs_std_types_cq.c +81 −6
  18. drivers/infiniband/core/uverbs_std_types_dmah.c +145 −0
  19. drivers/infiniband/core/uverbs_std_types_mr.c +170 −2
  20. drivers/infiniband/core/uverbs_std_types_qp.c +1 −1
  21. drivers/infiniband/core/uverbs_uapi.c +1 −0
  22. drivers/infiniband/core/verbs.c +4 −1
  23. drivers/infiniband/hw/Makefile +0 −1
  24. drivers/infiniband/hw/bnxt_re/ib_verbs.c +9 −1
  25. drivers/infiniband/hw/bnxt_re/ib_verbs.h +2 −0
  26. drivers/infiniband/hw/bnxt_re/qplib_fp.c +14 −14
  27. drivers/infiniband/hw/bnxt_re/qplib_fp.h +3 −0
  28. drivers/infiniband/hw/bnxt_re/qplib_sp.c +1 −1
  29. drivers/infiniband/hw/bnxt_re/qplib_sp.h +1 −0
  30. drivers/infiniband/hw/cxgb4/device.c +1 −2
  31. drivers/infiniband/hw/cxgb4/iw_cxgb4.h +1 −0
  32. drivers/infiniband/hw/cxgb4/mem.c +5 −1
  33. drivers/infiniband/hw/efa/efa.h +5 −0
  34. drivers/infiniband/hw/efa/efa_admin_cmds_defs.h +16 −1
  35. drivers/infiniband/hw/efa/efa_com_cmd.c +35 −18
  36. drivers/infiniband/hw/efa/efa_com_cmd.h +10 −1
  37. drivers/infiniband/hw/efa/efa_main.c +1 −0
  38. drivers/infiniband/hw/efa/efa_verbs.c +77 −14
  39. drivers/infiniband/hw/erdma/erdma_verbs.c +7 −2
  40. drivers/infiniband/hw/erdma/erdma_verbs.h +2 −1
  41. drivers/infiniband/hw/hfi1/affinity.c +37 −59
  42. drivers/infiniband/hw/hns/hns_roce_device.h +2 −19
  43. drivers/infiniband/hw/hns/hns_roce_hem.c +5 −13
  44. drivers/infiniband/hw/hns/hns_roce_hw_v2.c +64 −70
  45. drivers/infiniband/hw/hns/hns_roce_hw_v2.h +7 −9
  46. drivers/infiniband/hw/hns/hns_roce_main.c +7 −25
  47. drivers/infiniband/hw/hns/hns_roce_mr.c +6 −114
  48. drivers/infiniband/hw/hns/hns_roce_qp.c +2 −2
  49. drivers/infiniband/hw/irdma/verbs.c +9 −0
  50. drivers/infiniband/hw/mana/counters.c +76 −2
  51. drivers/infiniband/hw/mana/counters.h +18 −0
  52. drivers/infiniband/hw/mana/device.c +66 −54
  53. drivers/infiniband/hw/mana/main.c +10 −3
  54. drivers/infiniband/hw/mana/mana_ib.h +30 −0
  55. drivers/infiniband/hw/mana/mr.c +8 −0
  56. drivers/infiniband/hw/mana/qp.c +1 −1
  57. drivers/infiniband/hw/mlx4/mlx4_ib.h +1 −0
  58. drivers/infiniband/hw/mlx4/mr.c +4 −0
  59. drivers/infiniband/hw/mlx5/Makefile +1 −0
  60. drivers/infiniband/hw/mlx5/counters.c +26 −4
  61. drivers/infiniband/hw/mlx5/counters.h +0 −13
  62. drivers/infiniband/hw/mlx5/cq.c +15 −4
  63. drivers/infiniband/hw/mlx5/devx.c +5 −1
  64. drivers/infiniband/hw/mlx5/dmah.c +54 −0
  65. drivers/infiniband/hw/mlx5/dmah.h +23 −0
  66. drivers/infiniband/hw/mlx5/fs.c +63 −58
  67. drivers/infiniband/hw/mlx5/fs.h +6 −2
  68. drivers/infiniband/hw/mlx5/ib_rep.c +2 −1
  69. drivers/infiniband/hw/mlx5/main.c +11 −2
  70. drivers/infiniband/hw/mlx5/mlx5_ib.h +75 −24
  71. drivers/infiniband/hw/mlx5/mr.c +92 −24
  72. drivers/infiniband/hw/mlx5/odp.c +25 −7
  73. drivers/infiniband/hw/mlx5/umr.c +279 −22
  74. drivers/infiniband/hw/mlx5/umr.h +12 −1
  75. drivers/infiniband/hw/mthca/mthca_provider.c +5 −1
  76. drivers/infiniband/hw/ocrdma/ocrdma_verbs.c +5 −1
  77. drivers/infiniband/hw/ocrdma/ocrdma_verbs.h +2 −1
  78. drivers/infiniband/hw/qedr/verbs.c +5 −1
  79. drivers/infiniband/hw/qedr/verbs.h +2 −1
  80. drivers/infiniband/hw/qib/Kconfig +0 −17
  81. drivers/infiniband/hw/qib/Makefile +0 −17
  82. drivers/infiniband/hw/qib/qib.h +0 −1492
  83. drivers/infiniband/hw/qib/qib_6120_regs.h +0 −977
  84. drivers/infiniband/hw/qib/qib_7220.h +0 −149
  85. drivers/infiniband/hw/qib/qib_7220_regs.h +0 −1496
  86. drivers/infiniband/hw/qib/qib_7322_regs.h +0 −3163
  87. drivers/infiniband/hw/qib/qib_common.h +0 −798
  88. drivers/infiniband/hw/qib/qib_debugfs.c +0 −274
  89. drivers/infiniband/hw/qib/qib_debugfs.h +0 −45
  90. drivers/infiniband/hw/qib/qib_diag.c +0 −906
  91. drivers/infiniband/hw/qib/qib_driver.c +0 −798
  92. drivers/infiniband/hw/qib/qib_eeprom.c +0 −271
  93. drivers/infiniband/hw/qib/qib_file_ops.c +0 −2401
  94. drivers/infiniband/hw/qib/qib_fs.c +0 −549
  95. drivers/infiniband/hw/qib/qib_iba6120.c +0 −3533
  96. drivers/infiniband/hw/qib/qib_iba7220.c +0 −4596
  97. drivers/infiniband/hw/qib/qib_iba7322.c +0 −8475
  98. drivers/infiniband/hw/qib/qib_init.c +0 −1782
  99. drivers/infiniband/hw/qib/qib_intr.c +0 −241
  100. drivers/infiniband/hw/qib/qib_mad.c +0 −2450
  101. drivers/infiniband/hw/qib/qib_mad.h +0 −300
  102. drivers/infiniband/hw/qib/qib_pcie.c +0 −598
  103. drivers/infiniband/hw/qib/qib_pio_copy.c +0 −64
  104. drivers/infiniband/hw/qib/qib_qp.c +0 −454
  105. drivers/infiniband/hw/qib/qib_qsfp.c +0 −549
  106. drivers/infiniband/hw/qib/qib_qsfp.h +0 −188
  107. drivers/infiniband/hw/qib/qib_rc.c +0 −2131
  108. drivers/infiniband/hw/qib/qib_ruc.c +0 −314
  109. drivers/infiniband/hw/qib/qib_sd7220.c +0 −1445
  110. drivers/infiniband/hw/qib/qib_sdma.c +0 −999
  111. drivers/infiniband/hw/qib/qib_sysfs.c +0 −731
  112. drivers/infiniband/hw/qib/qib_twsi.c +0 −502
  113. drivers/infiniband/hw/qib/qib_tx.c +0 −566
  114. drivers/infiniband/hw/qib/qib_uc.c +0 −521
  115. drivers/infiniband/hw/qib/qib_ud.c +0 −583
  116. drivers/infiniband/hw/qib/qib_user_pages.c +0 −137
  117. drivers/infiniband/hw/qib/qib_user_sdma.c +0 −1470
  118. drivers/infiniband/hw/qib/qib_user_sdma.h +0 −52
  119. drivers/infiniband/hw/qib/qib_verbs.c +0 −1705
  120. drivers/infiniband/hw/qib/qib_verbs.h +0 −398
  121. drivers/infiniband/hw/qib/qib_wc_ppc64.c +0 −62
  122. drivers/infiniband/hw/qib/qib_wc_x86_64.c +0 −150
  123. drivers/infiniband/hw/usnic/usnic_ib_verbs.c +4 −0
  124. drivers/infiniband/hw/usnic/usnic_ib_verbs.h +1 −0
  125. drivers/infiniband/hw/vmw_pvrdma/pvrdma_mr.c +5 −0
  126. drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h +1 −0
  127. drivers/infiniband/sw/rdmavt/mr.c +5 −0
  128. drivers/infiniband/sw/rdmavt/mr.h +1 −0
  129. drivers/infiniband/sw/rdmavt/vt.c +1 −1
  130. drivers/infiniband/sw/rxe/rxe.c +7 −0
  131. drivers/infiniband/sw/rxe/rxe_loc.h +12 −0
  132. drivers/infiniband/sw/rxe/rxe_odp.c +175 −17
  133. drivers/infiniband/sw/rxe/rxe_verbs.c +4 −0
  134. drivers/infiniband/sw/siw/siw_qp_tx.c +16 −6
  135. drivers/infiniband/sw/siw/siw_verbs.c +6 −1
  136. drivers/infiniband/sw/siw/siw_verbs.h +2 −1
  137. drivers/infiniband/ulp/ipoib/ipoib_main.c +2 −0
  138. drivers/infiniband/ulp/iser/iscsi_iser.c +1 −5
  139. drivers/net/ethernet/mellanox/mlx5/core/Makefile +5 −0
  140. drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h +0 −5
  141. drivers/net/ethernet/mellanox/mlx5/core/lib/st.c +164 −0
  142. drivers/net/ethernet/mellanox/mlx5/core/main.c +2 −0
  143. drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +9 −0
  144. drivers/pci/tph.c +6 −5
  145. include/linux/cpumask.h +12 −0
  146. include/linux/mlx5/driver.h +25 −0
  147. include/linux/mlx5/mlx5_ifc.h +12 −2
  148. include/linux/pci-tph.h +1 −0
  149. include/rdma/ib_umem.h +21 −4
  150. include/rdma/ib_verbs.h +63 −2
  151. include/rdma/restrack.h +4 −0
  152. include/uapi/rdma/efa-abi.h +2 −1
  153. include/uapi/rdma/ib_user_ioctl_cmds.h +36 −0
.mailmap+1−0
@@ -138,6 +138,7 @@ Benjamin Poirier <benjamin.poirier@gmail.com> <bpoirier@suse.de> · 137 unmodified lines
138Benjamin Tissoires <bentiss@kernel.org> <benjamin.tissoires@gmail.com>138Benjamin Tissoires <bentiss@kernel.org> <benjamin.tissoires@gmail.com>
139Benjamin Tissoires <bentiss@kernel.org> <benjamin.tissoires@redhat.com>139Benjamin Tissoires <bentiss@kernel.org> <benjamin.tissoires@redhat.com>
140Benno Lossin <lossin@kernel.org> <benno.lossin@proton.me>140Benno Lossin <lossin@kernel.org> <benno.lossin@proton.me>
141Bernard Metzler <bernard.metzler@linux.dev> <bmt@zurich.ibm.com>
141Bingwu Zhang <xtex@aosc.io> <xtexchooser@duck.com>142Bingwu Zhang <xtex@aosc.io> <xtexchooser@duck.com>
142Bingwu Zhang <xtex@aosc.io> <xtex@xtexx.eu.org>143Bingwu Zhang <xtex@aosc.io> <xtex@xtexx.eu.org>
143Bjorn Andersson <andersson@kernel.org> <bjorn@kryo.se>144Bjorn Andersson <andersson@kernel.org> <bjorn@kryo.se>
MAINTAINERS+1−7
@@ -20370,12 +20370,6 @@ S: Maintained · 20369 unmodified lines
20370F: drivers/firmware/qemu_fw_cfg.c20370F: drivers/firmware/qemu_fw_cfg.c
20371F: include/uapi/linux/qemu_fw_cfg.h20371F: include/uapi/linux/qemu_fw_cfg.h
20372 20372
20373QIB DRIVER
20374M: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
20375L: linux-rdma@vger.kernel.org
20376S: Supported
20377F: drivers/infiniband/hw/qib/
20378
20379QLOGIC QL41xxx FCOE DRIVER20373QLOGIC QL41xxx FCOE DRIVER
20380M: Saurav Kashyap <skashyap@marvell.com>20374M: Saurav Kashyap <skashyap@marvell.com>
20381M: Javed Hasan <jhasan@marvell.com>20375M: Javed Hasan <jhasan@marvell.com>
@@ -23222,7 +23216,7 @@ S: Maintained · 2840 unmodified lines
23222F: drivers/leds/leds-net48xx.c23216F: drivers/leds/leds-net48xx.c
23223 23217
23224SOFT-IWARP DRIVER (siw)23218SOFT-IWARP DRIVER (siw)
23225M: Bernard Metzler <bmt@zurich.ibm.com>23219M: Bernard Metzler <bernard.metzler@linux.dev>
23226L: linux-rdma@vger.kernel.org23220L: linux-rdma@vger.kernel.org
23227S: Supported23221S: Supported
23228F: drivers/infiniband/sw/siw/23222F: drivers/infiniband/sw/siw/
drivers/infiniband/Kconfig+0−1
@@ -92,7 +92,6 @@ source "drivers/infiniband/hw/mlx5/Kconfig" · 91 unmodified lines
92source "drivers/infiniband/hw/mthca/Kconfig"92source "drivers/infiniband/hw/mthca/Kconfig"
93source "drivers/infiniband/hw/ocrdma/Kconfig"93source "drivers/infiniband/hw/ocrdma/Kconfig"
94source "drivers/infiniband/hw/qedr/Kconfig"94source "drivers/infiniband/hw/qedr/Kconfig"
95source "drivers/infiniband/hw/qib/Kconfig"
96source "drivers/infiniband/hw/usnic/Kconfig"95source "drivers/infiniband/hw/usnic/Kconfig"
97source "drivers/infiniband/hw/vmw_pvrdma/Kconfig"96source "drivers/infiniband/hw/vmw_pvrdma/Kconfig"
98source "drivers/infiniband/sw/rdmavt/Kconfig"97source "drivers/infiniband/sw/rdmavt/Kconfig"
drivers/infiniband/core/Makefile+1−0
@@ -33,6 +33,7 @@ ib_umad-y := user_mad.o · 32 unmodified lines
33ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \33ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
34 rdma_core.o uverbs_std_types.o uverbs_ioctl.o \34 rdma_core.o uverbs_std_types.o uverbs_ioctl.o \
35 uverbs_std_types_cq.o \35 uverbs_std_types_cq.o \
36 uverbs_std_types_dmah.o \
36 uverbs_std_types_flow_action.o uverbs_std_types_dm.o \37 uverbs_std_types_flow_action.o uverbs_std_types_dm.o \
37 uverbs_std_types_mr.o uverbs_std_types_counters.o \38 uverbs_std_types_mr.o uverbs_std_types_counters.o \
38 uverbs_uapi.o uverbs_std_types_device.o \39 uverbs_uapi.o uverbs_std_types_device.o \
drivers/infiniband/core/cm.c+41−6
@@ -161,6 +161,7 @@ struct cm_counter_attribute { · 160 unmodified lines
161struct cm_port {161struct cm_port {
162 struct cm_device *cm_dev;162 struct cm_device *cm_dev;
163 struct ib_mad_agent *mad_agent;163 struct ib_mad_agent *mad_agent;
164 struct ib_mad_agent *rep_agent;
164 u32 port_num;165 u32 port_num;
165 atomic_long_t counters[CM_COUNTER_GROUPS][CM_ATTR_COUNT];166 atomic_long_t counters[CM_COUNTER_GROUPS][CM_ATTR_COUNT];
166};167};
@@ -274,7 +275,8 @@ static inline void cm_deref_id(struct cm_id_private *cm_id_priv) · 107 unmodified lines
274 complete(&cm_id_priv->comp);275 complete(&cm_id_priv->comp);
275}276}
276 277
277static struct ib_mad_send_buf *cm_alloc_msg(struct cm_id_private *cm_id_priv)278static struct ib_mad_send_buf *
279cm_alloc_msg_agent(struct cm_id_private *cm_id_priv, bool rep_agent)
278{280{
279 struct ib_mad_agent *mad_agent;281 struct ib_mad_agent *mad_agent;
280 struct ib_mad_send_buf *m;282 struct ib_mad_send_buf *m;
@@ -286,7 +288,8 @@ static struct ib_mad_send_buf *cm_alloc_msg(struct cm_id_private *cm_id_priv) · 5 unmodified lines
286 return ERR_PTR(-EINVAL);288 return ERR_PTR(-EINVAL);
287 289
288 read_lock(&cm_id_priv->av.port->cm_dev->mad_agent_lock);290 read_lock(&cm_id_priv->av.port->cm_dev->mad_agent_lock);
289 mad_agent = cm_id_priv->av.port->mad_agent;291 mad_agent = rep_agent ? cm_id_priv->av.port->rep_agent :
292 cm_id_priv->av.port->mad_agent;
290 if (!mad_agent) {293 if (!mad_agent) {
291 m = ERR_PTR(-EINVAL);294 m = ERR_PTR(-EINVAL);
292 goto out;295 goto out;
@@ -315,6 +318,11 @@ out: · 22 unmodified lines
315 return m;318 return m;
316}319}
317 320
321static struct ib_mad_send_buf *cm_alloc_msg(struct cm_id_private *cm_id_priv)
322{
323 return cm_alloc_msg_agent(cm_id_priv, false);
324}
325
318static void cm_free_msg(struct ib_mad_send_buf *msg)326static void cm_free_msg(struct ib_mad_send_buf *msg)
319{327{
320 if (msg->ah)328 if (msg->ah)
@@ -323,13 +331,14 @@ static void cm_free_msg(struct ib_mad_send_buf *msg) · 2 unmodified lines
323}331}
324 332
325static struct ib_mad_send_buf *333static struct ib_mad_send_buf *
326cm_alloc_priv_msg(struct cm_id_private *cm_id_priv, enum ib_cm_state state)334cm_alloc_priv_msg_rep(struct cm_id_private *cm_id_priv, enum ib_cm_state state,
335 bool rep_agent)
327{336{
328 struct ib_mad_send_buf *msg;337 struct ib_mad_send_buf *msg;
329 338
330 lockdep_assert_held(&cm_id_priv->lock);339 lockdep_assert_held(&cm_id_priv->lock);
331 340
332 msg = cm_alloc_msg(cm_id_priv);341 msg = cm_alloc_msg_agent(cm_id_priv, rep_agent);
333 if (IS_ERR(msg))342 if (IS_ERR(msg))
334 return msg;343 return msg;
335 344
@@ -344,6 +353,12 @@ cm_alloc_priv_msg(struct cm_id_private *cm_id_priv, enum ib_cm_state state) · 8 unmodified lines
344 return msg;353 return msg;
345}354}
346 355
356static struct ib_mad_send_buf *
357cm_alloc_priv_msg(struct cm_id_private *cm_id_priv, enum ib_cm_state state)
358{
359 return cm_alloc_priv_msg_rep(cm_id_priv, state, false);
360}
361
347static void cm_free_priv_msg(struct ib_mad_send_buf *msg)362static void cm_free_priv_msg(struct ib_mad_send_buf *msg)
348{363{
349 struct cm_id_private *cm_id_priv = msg->context[0];364 struct cm_id_private *cm_id_priv = msg->context[0];
@@ -2295,7 +2310,7 @@ int ib_send_cm_rep(struct ib_cm_id *cm_id, · 1945 unmodified lines
2295 goto out;2310 goto out;
2296 }2311 }
2297 2312
2298 msg = cm_alloc_priv_msg(cm_id_priv, IB_CM_REP_SENT);2313 msg = cm_alloc_priv_msg_rep(cm_id_priv, IB_CM_REP_SENT, true);
2299 if (IS_ERR(msg)) {2314 if (IS_ERR(msg)) {
2300 ret = PTR_ERR(msg);2315 ret = PTR_ERR(msg);
2301 goto out;2316 goto out;
@@ -4380,9 +4395,22 @@ static int cm_add_one(struct ib_device *ib_device) · 2078 unmodified lines
4380 goto error2;4395 goto error2;
4381 }4396 }
4382 4397
4398 port->rep_agent = ib_register_mad_agent(ib_device, i,
4399 IB_QPT_GSI,
4400 NULL,
4401 0,
4402 cm_send_handler,
4403 NULL,
4404 port,
4405 0);
4406 if (IS_ERR(port->rep_agent)) {
4407 ret = PTR_ERR(port->rep_agent);
4408 goto error3;
4409 }
4410
4383 ret = ib_modify_port(ib_device, i, 0, &port_modify);4411 ret = ib_modify_port(ib_device, i, 0, &port_modify);
4384 if (ret)4412 if (ret)
4385 goto error3;4413 goto error4;
4386 4414
4387 count++;4415 count++;
4388 }4416 }
@@ -4397,6 +4425,8 @@ static int cm_add_one(struct ib_device *ib_device) · 8 unmodified lines
4397 write_unlock_irqrestore(&cm.device_lock, flags);4425 write_unlock_irqrestore(&cm.device_lock, flags);
4398 return 0;4426 return 0;
4399 4427
4428error4:
4429 ib_unregister_mad_agent(port->rep_agent);
4400error3:4430error3:
4401 ib_unregister_mad_agent(port->mad_agent);4431 ib_unregister_mad_agent(port->mad_agent);
4402error2:4432error2:
@@ -4410,6 +4440,7 @@ error1: · 7 unmodified lines
4410 4440
4411 port = cm_dev->port[i-1];4441 port = cm_dev->port[i-1];
4412 ib_modify_port(ib_device, port->port_num, 0, &port_modify);4442 ib_modify_port(ib_device, port->port_num, 0, &port_modify);
4443 ib_unregister_mad_agent(port->rep_agent);
4413 ib_unregister_mad_agent(port->mad_agent);4444 ib_unregister_mad_agent(port->mad_agent);
4414 ib_port_unregister_client_groups(ib_device, i,4445 ib_port_unregister_client_groups(ib_device, i,
4415 cm_counter_groups);4446 cm_counter_groups);
@@ -4439,12 +4470,14 @@ static void cm_remove_one(struct ib_device *ib_device, void *client_data) · 23 unmodified lines
4439 4470
4440 rdma_for_each_port (ib_device, i) {4471 rdma_for_each_port (ib_device, i) {
4441 struct ib_mad_agent *mad_agent;4472 struct ib_mad_agent *mad_agent;
4473 struct ib_mad_agent *rep_agent;
4442 4474
4443 if (!rdma_cap_ib_cm(ib_device, i))4475 if (!rdma_cap_ib_cm(ib_device, i))
4444 continue;4476 continue;
4445 4477
4446 port = cm_dev->port[i-1];4478 port = cm_dev->port[i-1];
4447 mad_agent = port->mad_agent;4479 mad_agent = port->mad_agent;
4480 rep_agent = port->rep_agent;
4448 ib_modify_port(ib_device, port->port_num, 0, &port_modify);4481 ib_modify_port(ib_device, port->port_num, 0, &port_modify);
4449 /*4482 /*
4450 * We flush the queue here after the going_down set, this4483 * We flush the queue here after the going_down set, this
@@ -4458,8 +4491,10 @@ static void cm_remove_one(struct ib_device *ib_device, void *client_data) · 7 unmodified lines
4458 */4491 */
4459 write_lock(&cm_dev->mad_agent_lock);4492 write_lock(&cm_dev->mad_agent_lock);
4460 port->mad_agent = NULL;4493 port->mad_agent = NULL;
4494 port->rep_agent = NULL;
4461 write_unlock(&cm_dev->mad_agent_lock);4495 write_unlock(&cm_dev->mad_agent_lock);
4462 ib_unregister_mad_agent(mad_agent);4496 ib_unregister_mad_agent(mad_agent);
4497 ib_unregister_mad_agent(rep_agent);
4463 ib_port_unregister_client_groups(ib_device, i,4498 ib_port_unregister_client_groups(ib_device, i,
4464 cm_counter_groups);4499 cm_counter_groups);
4465 }4500 }
drivers/infiniband/core/counters.c+1−1
@@ -461,7 +461,7 @@ static struct ib_qp *rdma_counter_get_qp(struct ib_device *dev, u32 qp_num) · 460 unmodified lines
461 return NULL;461 return NULL;
462 462
463 qp = container_of(res, struct ib_qp, res);463 qp = container_of(res, struct ib_qp, res);
464 if (qp->qp_type == IB_QPT_RAW_PACKET && !capable(CAP_NET_RAW))464 if (qp->qp_type == IB_QPT_RAW_PACKET && !rdma_dev_has_raw_cap(dev))
465 goto err;465 goto err;
466 466
467 return qp;467 return qp;
drivers/infiniband/core/cq.c+10−2
@@ -317,13 +317,18 @@ EXPORT_SYMBOL(__ib_alloc_cq_any); · 316 unmodified lines
317 */317 */
318void ib_free_cq(struct ib_cq *cq)318void ib_free_cq(struct ib_cq *cq)
319{319{
320 int ret;320 int ret = 0;
321 321
322 if (WARN_ON_ONCE(atomic_read(&cq->usecnt)))322 if (WARN_ON_ONCE(atomic_read(&cq->usecnt)))
323 return;323 return;
324 if (WARN_ON_ONCE(cq->cqe_used))324 if (WARN_ON_ONCE(cq->cqe_used))
325 return;325 return;
326 326
327 if (cq->device->ops.pre_destroy_cq) {
328 ret = cq->device->ops.pre_destroy_cq(cq);
329 WARN_ONCE(ret, "Disable of kernel CQ shouldn't fail");
330 }
331
327 switch (cq->poll_ctx) {332 switch (cq->poll_ctx) {
328 case IB_POLL_DIRECT:333 case IB_POLL_DIRECT:
329 break;334 break;
@@ -340,7 +345,10 @@ void ib_free_cq(struct ib_cq *cq) · 10 unmodified lines
340 345
341 rdma_dim_destroy(cq);346 rdma_dim_destroy(cq);
342 trace_cq_free(cq);347 trace_cq_free(cq);
343 ret = cq->device->ops.destroy_cq(cq, NULL);348 if (cq->device->ops.post_destroy_cq)
349 cq->device->ops.post_destroy_cq(cq);
350 else
351 ret = cq->device->ops.destroy_cq(cq, NULL);
344 WARN_ONCE(ret, "Destroy of kernel CQ shouldn't fail");352 WARN_ONCE(ret, "Destroy of kernel CQ shouldn't fail");
345 rdma_restrack_del(&cq->res);353 rdma_restrack_del(&cq->res);
346 kfree(cq->wc);354 kfree(cq->wc);
drivers/infiniband/core/device.c+45−2
@@ -145,6 +145,33 @@ bool rdma_dev_access_netns(const struct ib_device *dev, const struct net *net) · 144 unmodified lines
145}145}
146EXPORT_SYMBOL(rdma_dev_access_netns);146EXPORT_SYMBOL(rdma_dev_access_netns);
147 147
148/**
149 * rdma_dev_has_raw_cap() - Returns whether a specified rdma device has
150 * CAP_NET_RAW capability or not.
151 *
152 * @dev: Pointer to rdma device whose capability to be checked
153 *
154 * Returns true if a rdma device's owning user namespace has CAP_NET_RAW
155 * capability, otherwise false. When rdma subsystem is in legacy shared network,
156 * namespace mode, the default net namespace is considered.
157 */
158bool rdma_dev_has_raw_cap(const struct ib_device *dev)
159{
160 const struct net *net;
161
162 /* Network namespace is the resource whose user namespace
163 * to be considered. When in shared mode, there is no reliable
164 * network namespace resource, so consider the default net namespace.
165 */
166 if (ib_devices_shared_netns)
167 net = &init_net;
168 else
169 net = read_pnet(&dev->coredev.rdma_net);
170
171 return ns_capable(net->user_ns, CAP_NET_RAW);
172}
173EXPORT_SYMBOL(rdma_dev_has_raw_cap);
174
148/*175/*
149 * xarray has this behavior where it won't iterate over NULL values stored in176 * xarray has this behavior where it won't iterate over NULL values stored in
150 * allocated arrays. So we need our own iterator to see all values stored in177 * allocated arrays. So we need our own iterator to see all values stored in
@@ -557,6 +584,8 @@ static void rdma_init_coredev(struct ib_core_device *coredev, · 406 unmodified lines
557/**584/**
558 * _ib_alloc_device - allocate an IB device struct585 * _ib_alloc_device - allocate an IB device struct
559 * @size:size of structure to allocate586 * @size:size of structure to allocate
587 * @net: network namespace device should be located in, namespace
588 * must stay valid until ib_register_device() is completed.
560 *589 *
561 * Low-level drivers should use ib_alloc_device() to allocate &struct590 * Low-level drivers should use ib_alloc_device() to allocate &struct
562 * ib_device. @size is the size of the structure to be allocated,591 * ib_device. @size is the size of the structure to be allocated,
@@ -564,7 +593,7 @@ static void rdma_init_coredev(struct ib_core_device *coredev, · 1 unmodified line
564 * ib_dealloc_device() must be used to free structures allocated with593 * ib_dealloc_device() must be used to free structures allocated with
565 * ib_alloc_device().594 * ib_alloc_device().
566 */595 */
567struct ib_device *_ib_alloc_device(size_t size)596struct ib_device *_ib_alloc_device(size_t size, struct net *net)
568{597{
569 struct ib_device *device;598 struct ib_device *device;
570 unsigned int i;599 unsigned int i;
@@ -581,7 +610,15 @@ struct ib_device *_ib_alloc_device(size_t size) · 10 unmodified lines
581 return NULL;610 return NULL;
582 }611 }
583 612
584 rdma_init_coredev(&device->coredev, device, &init_net);613 /* ib_devices_shared_netns can't change while we have active namespaces
614 * in the system which means either init_net is passed or the user has
615 * no idea what they are doing.
616 *
617 * To avoid breaking backward compatibility, when in shared mode,
618 * force to init the device in the init_net.
619 */
620 net = ib_devices_shared_netns ? &init_net : net;
621 rdma_init_coredev(&device->coredev, device, net);
585 622
586 INIT_LIST_HEAD(&device->event_handler_list);623 INIT_LIST_HEAD(&device->event_handler_list);
587 spin_lock_init(&device->qp_open_list_lock);624 spin_lock_init(&device->qp_open_list_lock);
@@ -2671,6 +2708,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops) · 2083 unmodified lines
2671 SET_DEVICE_OP(dev_ops, add_sub_dev);2708 SET_DEVICE_OP(dev_ops, add_sub_dev);
2672 SET_DEVICE_OP(dev_ops, advise_mr);2709 SET_DEVICE_OP(dev_ops, advise_mr);
2673 SET_DEVICE_OP(dev_ops, alloc_dm);2710 SET_DEVICE_OP(dev_ops, alloc_dm);
2711 SET_DEVICE_OP(dev_ops, alloc_dmah);
2674 SET_DEVICE_OP(dev_ops, alloc_hw_device_stats);2712 SET_DEVICE_OP(dev_ops, alloc_hw_device_stats);
2675 SET_DEVICE_OP(dev_ops, alloc_hw_port_stats);2713 SET_DEVICE_OP(dev_ops, alloc_hw_port_stats);
2676 SET_DEVICE_OP(dev_ops, alloc_mr);2714 SET_DEVICE_OP(dev_ops, alloc_mr);
@@ -2691,6 +2729,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops) · 14 unmodified lines
2691 SET_DEVICE_OP(dev_ops, create_ah);2729 SET_DEVICE_OP(dev_ops, create_ah);
2692 SET_DEVICE_OP(dev_ops, create_counters);2730 SET_DEVICE_OP(dev_ops, create_counters);
2693 SET_DEVICE_OP(dev_ops, create_cq);2731 SET_DEVICE_OP(dev_ops, create_cq);
2732 SET_DEVICE_OP(dev_ops, create_cq_umem);
2694 SET_DEVICE_OP(dev_ops, create_flow);2733 SET_DEVICE_OP(dev_ops, create_flow);
2695 SET_DEVICE_OP(dev_ops, create_qp);2734 SET_DEVICE_OP(dev_ops, create_qp);
2696 SET_DEVICE_OP(dev_ops, create_rwq_ind_table);2735 SET_DEVICE_OP(dev_ops, create_rwq_ind_table);
@@ -2698,6 +2737,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops) · 1 unmodified line
2698 SET_DEVICE_OP(dev_ops, create_user_ah);2737 SET_DEVICE_OP(dev_ops, create_user_ah);
2699 SET_DEVICE_OP(dev_ops, create_wq);2738 SET_DEVICE_OP(dev_ops, create_wq);
2700 SET_DEVICE_OP(dev_ops, dealloc_dm);2739 SET_DEVICE_OP(dev_ops, dealloc_dm);
2740 SET_DEVICE_OP(dev_ops, dealloc_dmah);
2701 SET_DEVICE_OP(dev_ops, dealloc_driver);2741 SET_DEVICE_OP(dev_ops, dealloc_driver);
2702 SET_DEVICE_OP(dev_ops, dealloc_mw);2742 SET_DEVICE_OP(dev_ops, dealloc_mw);
2703 SET_DEVICE_OP(dev_ops, dealloc_pd);2743 SET_DEVICE_OP(dev_ops, dealloc_pd);
@@ -2763,8 +2803,10 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops) · 59 unmodified lines
2763 SET_DEVICE_OP(dev_ops, modify_srq);2803 SET_DEVICE_OP(dev_ops, modify_srq);
2764 SET_DEVICE_OP(dev_ops, modify_wq);2804 SET_DEVICE_OP(dev_ops, modify_wq);
2765 SET_DEVICE_OP(dev_ops, peek_cq);2805 SET_DEVICE_OP(dev_ops, peek_cq);
2806 SET_DEVICE_OP(dev_ops, pre_destroy_cq);
2766 SET_DEVICE_OP(dev_ops, poll_cq);2807 SET_DEVICE_OP(dev_ops, poll_cq);
2767 SET_DEVICE_OP(dev_ops, port_groups);2808 SET_DEVICE_OP(dev_ops, port_groups);
2809 SET_DEVICE_OP(dev_ops, post_destroy_cq);
2768 SET_DEVICE_OP(dev_ops, post_recv);2810 SET_DEVICE_OP(dev_ops, post_recv);
2769 SET_DEVICE_OP(dev_ops, post_send);2811 SET_DEVICE_OP(dev_ops, post_send);
2770 SET_DEVICE_OP(dev_ops, post_srq_recv);2812 SET_DEVICE_OP(dev_ops, post_srq_recv);
@@ -2793,6 +2835,7 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops) · 22 unmodified lines
2793 SET_OBJ_SIZE(dev_ops, ib_ah);2835 SET_OBJ_SIZE(dev_ops, ib_ah);
2794 SET_OBJ_SIZE(dev_ops, ib_counters);2836 SET_OBJ_SIZE(dev_ops, ib_counters);
2795 SET_OBJ_SIZE(dev_ops, ib_cq);2837 SET_OBJ_SIZE(dev_ops, ib_cq);
2838 SET_OBJ_SIZE(dev_ops, ib_dmah);
2796 SET_OBJ_SIZE(dev_ops, ib_mw);2839 SET_OBJ_SIZE(dev_ops, ib_mw);
2797 SET_OBJ_SIZE(dev_ops, ib_pd);2840 SET_OBJ_SIZE(dev_ops, ib_pd);
2798 SET_OBJ_SIZE(dev_ops, ib_qp);2841 SET_OBJ_SIZE(dev_ops, ib_qp);
drivers/infiniband/core/mad.c+362−106
@@ -210,6 +210,29 @@ int ib_response_mad(const struct ib_mad_hdr *hdr) · 209 unmodified lines
210}210}
211EXPORT_SYMBOL(ib_response_mad);211EXPORT_SYMBOL(ib_response_mad);
212 212
213#define SOL_FC_MAX_DEFAULT_FRAC 4
214#define SOL_FC_MAX_SA_FRAC 32
215
216static int get_sol_fc_max_outstanding(struct ib_mad_reg_req *mad_reg_req)
217{
218 if (!mad_reg_req)
219 /* Send only agent */
220 return mad_recvq_size / SOL_FC_MAX_DEFAULT_FRAC;
221
222 switch (mad_reg_req->mgmt_class) {
223 case IB_MGMT_CLASS_CM:
224 return mad_recvq_size / SOL_FC_MAX_DEFAULT_FRAC;
225 case IB_MGMT_CLASS_SUBN_ADM:
226 return mad_recvq_size / SOL_FC_MAX_SA_FRAC;
227 case IB_MGMT_CLASS_SUBN_LID_ROUTED:
228 case IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE:
229 return min(mad_recvq_size, IB_MAD_QP_RECV_SIZE) /
230 SOL_FC_MAX_DEFAULT_FRAC;
231 default:
232 return 0;
233 }
234}
235
213/*236/*
214 * ib_register_mad_agent - Register to send/receive MADs237 * ib_register_mad_agent - Register to send/receive MADs
215 *238 *
@@ -391,13 +414,17 @@ struct ib_mad_agent *ib_register_mad_agent(struct ib_device *device, · 175 unmodified lines
391 spin_lock_init(&mad_agent_priv->lock);414 spin_lock_init(&mad_agent_priv->lock);
392 INIT_LIST_HEAD(&mad_agent_priv->send_list);415 INIT_LIST_HEAD(&mad_agent_priv->send_list);
393 INIT_LIST_HEAD(&mad_agent_priv->wait_list);416 INIT_LIST_HEAD(&mad_agent_priv->wait_list);
394 INIT_LIST_HEAD(&mad_agent_priv->done_list);
395 INIT_LIST_HEAD(&mad_agent_priv->rmpp_list);417 INIT_LIST_HEAD(&mad_agent_priv->rmpp_list);
418 INIT_LIST_HEAD(&mad_agent_priv->backlog_list);
396 INIT_DELAYED_WORK(&mad_agent_priv->timed_work, timeout_sends);419 INIT_DELAYED_WORK(&mad_agent_priv->timed_work, timeout_sends);
397 INIT_LIST_HEAD(&mad_agent_priv->local_list);420 INIT_LIST_HEAD(&mad_agent_priv->local_list);
398 INIT_WORK(&mad_agent_priv->local_work, local_completions);421 INIT_WORK(&mad_agent_priv->local_work, local_completions);
399 refcount_set(&mad_agent_priv->refcount, 1);422 refcount_set(&mad_agent_priv->refcount, 1);
400 init_completion(&mad_agent_priv->comp);423 init_completion(&mad_agent_priv->comp);
424 mad_agent_priv->sol_fc_send_count = 0;
425 mad_agent_priv->sol_fc_wait_count = 0;
426 mad_agent_priv->sol_fc_max =
427 recv_handler ? get_sol_fc_max_outstanding(mad_reg_req) : 0;
401 428
402 ret2 = ib_mad_agent_security_setup(&mad_agent_priv->agent, qp_type);429 ret2 = ib_mad_agent_security_setup(&mad_agent_priv->agent, qp_type);
403 if (ret2) {430 if (ret2) {
@@ -1055,6 +1082,180 @@ int ib_send_mad(struct ib_mad_send_wr_private *mad_send_wr) · 651 unmodified lines
1055 return ret;1082 return ret;
1056}1083}
1057 1084
1085static void handle_queued_state(struct ib_mad_send_wr_private *mad_send_wr,
1086 struct ib_mad_agent_private *mad_agent_priv)
1087{
1088 if (mad_send_wr->state == IB_MAD_STATE_WAIT_RESP) {
1089 mad_agent_priv->sol_fc_wait_count--;
1090 list_move_tail(&mad_send_wr->agent_list,
1091 &mad_agent_priv->backlog_list);
1092 } else {
1093 expect_mad_state(mad_send_wr, IB_MAD_STATE_INIT);
1094 list_add_tail(&mad_send_wr->agent_list,
1095 &mad_agent_priv->backlog_list);
1096 }
1097}
1098
1099static void handle_send_state(struct ib_mad_send_wr_private *mad_send_wr,
1100 struct ib_mad_agent_private *mad_agent_priv)
1101{
1102 if (mad_send_wr->state == IB_MAD_STATE_INIT) {
1103 list_add_tail(&mad_send_wr->agent_list,
1104 &mad_agent_priv->send_list);
1105 } else {
1106 expect_mad_state2(mad_send_wr, IB_MAD_STATE_WAIT_RESP,
1107 IB_MAD_STATE_QUEUED);
1108 list_move_tail(&mad_send_wr->agent_list,
1109 &mad_agent_priv->send_list);
1110 }
1111
1112 if (mad_send_wr->is_solicited_fc) {
1113 if (mad_send_wr->state == IB_MAD_STATE_WAIT_RESP)
1114 mad_agent_priv->sol_fc_wait_count--;
1115 mad_agent_priv->sol_fc_send_count++;
1116 }
1117}
1118
1119static void handle_wait_state(struct ib_mad_send_wr_private *mad_send_wr,
1120 struct ib_mad_agent_private *mad_agent_priv)
1121{
1122 struct ib_mad_send_wr_private *temp_mad_send_wr;
1123 struct list_head *list_item;
1124 unsigned long delay;
1125
1126 expect_mad_state3(mad_send_wr, IB_MAD_STATE_SEND_START,
1127 IB_MAD_STATE_WAIT_RESP, IB_MAD_STATE_CANCELED);
1128 if (mad_send_wr->state == IB_MAD_STATE_SEND_START &&
1129 mad_send_wr->is_solicited_fc) {
1130 mad_agent_priv->sol_fc_send_count--;
1131 mad_agent_priv->sol_fc_wait_count++;
1132 }
1133
1134 list_del_init(&mad_send_wr->agent_list);
1135 delay = mad_send_wr->timeout;
1136 mad_send_wr->timeout += jiffies;
1137
1138 if (delay) {
1139 list_for_each_prev(list_item,
1140 &mad_agent_priv->wait_list) {
1141 temp_mad_send_wr = list_entry(
1142 list_item,
1143 struct ib_mad_send_wr_private,
1144 agent_list);
1145 if (time_after(mad_send_wr->timeout,
1146 temp_mad_send_wr->timeout))
1147 break;
1148 }
1149 } else {
1150 list_item = &mad_agent_priv->wait_list;
1151 }
1152
1153 list_add(&mad_send_wr->agent_list, list_item);
1154}
1155
1156static void handle_early_resp_state(struct ib_mad_send_wr_private *mad_send_wr,
1157 struct ib_mad_agent_private *mad_agent_priv)
1158{
1159 expect_mad_state(mad_send_wr, IB_MAD_STATE_SEND_START);
1160 mad_agent_priv->sol_fc_send_count -= mad_send_wr->is_solicited_fc;
1161}
1162
1163static void handle_canceled_state(struct ib_mad_send_wr_private *mad_send_wr,
1164 struct ib_mad_agent_private *mad_agent_priv)
1165{
1166 not_expect_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
1167 if (mad_send_wr->is_solicited_fc) {
1168 if (mad_send_wr->state == IB_MAD_STATE_SEND_START)
1169 mad_agent_priv->sol_fc_send_count--;
1170 else if (mad_send_wr->state == IB_MAD_STATE_WAIT_RESP)
1171 mad_agent_priv->sol_fc_wait_count--;
1172 }
1173}
1174
1175static void handle_done_state(struct ib_mad_send_wr_private *mad_send_wr,
1176 struct ib_mad_agent_private *mad_agent_priv)
1177{
1178 if (mad_send_wr->is_solicited_fc) {
1179 if (mad_send_wr->state == IB_MAD_STATE_SEND_START)
1180 mad_agent_priv->sol_fc_send_count--;
1181 else if (mad_send_wr->state == IB_MAD_STATE_WAIT_RESP)
1182 mad_agent_priv->sol_fc_wait_count--;
1183 }
1184
1185 list_del_init(&mad_send_wr->agent_list);
1186}
1187
1188void change_mad_state(struct ib_mad_send_wr_private *mad_send_wr,
1189 enum ib_mad_state new_state)
1190{
1191 struct ib_mad_agent_private *mad_agent_priv =
1192 mad_send_wr->mad_agent_priv;
1193
1194 switch (new_state) {
1195 case IB_MAD_STATE_INIT:
1196 break;
1197 case IB_MAD_STATE_QUEUED:
1198 handle_queued_state(mad_send_wr, mad_agent_priv);
1199 break;
1200 case IB_MAD_STATE_SEND_START:
1201 handle_send_state(mad_send_wr, mad_agent_priv);
1202 break;
1203 case IB_MAD_STATE_WAIT_RESP:
1204 handle_wait_state(mad_send_wr, mad_agent_priv);
1205 if (mad_send_wr->state == IB_MAD_STATE_CANCELED)
1206 return;
1207 break;
1208 case IB_MAD_STATE_EARLY_RESP:
1209 handle_early_resp_state(mad_send_wr, mad_agent_priv);
1210 break;
1211 case IB_MAD_STATE_CANCELED:
1212 handle_canceled_state(mad_send_wr, mad_agent_priv);
1213 break;
1214 case IB_MAD_STATE_DONE:
1215 handle_done_state(mad_send_wr, mad_agent_priv);
1216 break;
1217 }
1218
1219 mad_send_wr->state = new_state;
1220}
1221
1222static bool is_solicited_fc_mad(struct ib_mad_send_wr_private *mad_send_wr)
1223{
1224 struct ib_rmpp_mad *rmpp_mad;
1225 u8 mgmt_class;
1226
1227 if (!mad_send_wr->timeout)
1228 return 0;
1229
1230 rmpp_mad = mad_send_wr->send_buf.mad;
1231 if (mad_send_wr->mad_agent_priv->agent.rmpp_version &&
1232 (ib_get_rmpp_flags(&rmpp_mad->rmpp_hdr) & IB_MGMT_RMPP_FLAG_ACTIVE))
1233 return 0;
1234
1235 mgmt_class =
1236 ((struct ib_mad_hdr *)mad_send_wr->send_buf.mad)->mgmt_class;
1237 return mgmt_class == IB_MGMT_CLASS_CM ||
1238 mgmt_class == IB_MGMT_CLASS_SUBN_ADM ||
1239 mgmt_class == IB_MGMT_CLASS_SUBN_LID_ROUTED ||
1240 mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE;
1241}
1242
1243static bool mad_is_for_backlog(struct ib_mad_send_wr_private *mad_send_wr)
1244{
1245 struct ib_mad_agent_private *mad_agent_priv =
1246 mad_send_wr->mad_agent_priv;
1247
1248 if (!mad_send_wr->is_solicited_fc || !mad_agent_priv->sol_fc_max)
1249 return false;
1250
1251 if (!list_empty(&mad_agent_priv->backlog_list))
1252 return true;
1253
1254 return mad_agent_priv->sol_fc_send_count +
1255 mad_agent_priv->sol_fc_wait_count >=
1256 mad_agent_priv->sol_fc_max;
1257}
1258
1058/*1259/*
1059 * ib_post_send_mad - Posts MAD(s) to the send queue of the QP associated1260 * ib_post_send_mad - Posts MAD(s) to the send queue of the QP associated
1060 * with the registered client1261 * with the registered client
@@ -1080,9 +1281,7 @@ int ib_post_send_mad(struct ib_mad_send_buf *send_buf, · 19 unmodified lines
1080 if (ret)1281 if (ret)
1081 goto error;1282 goto error;
1082 1283
1083 if (!send_buf->mad_agent->send_handler ||1284 if (!send_buf->mad_agent->send_handler) {
1084 (send_buf->timeout_ms &&
1085 !send_buf->mad_agent->recv_handler)) {
1086 ret = -EINVAL;1285 ret = -EINVAL;
1087 goto error;1286 goto error;
1088 }1287 }
@@ -1118,15 +1317,19 @@ int ib_post_send_mad(struct ib_mad_send_buf *send_buf, · 29 unmodified lines
1118 mad_send_wr->max_retries = send_buf->retries;1317 mad_send_wr->max_retries = send_buf->retries;
1119 mad_send_wr->retries_left = send_buf->retries;1318 mad_send_wr->retries_left = send_buf->retries;
1120 send_buf->retries = 0;1319 send_buf->retries = 0;
1121 /* Reference for work request to QP + response */1320 change_mad_state(mad_send_wr, IB_MAD_STATE_INIT);
1122 mad_send_wr->refcount = 1 + (mad_send_wr->timeout > 0);
1123 mad_send_wr->status = IB_WC_SUCCESS;
1124 1321
1125 /* Reference MAD agent until send completes */1322 /* Reference MAD agent until send completes */
1126 refcount_inc(&mad_agent_priv->refcount);1323 refcount_inc(&mad_agent_priv->refcount);
1127 spin_lock_irqsave(&mad_agent_priv->lock, flags);1324 spin_lock_irqsave(&mad_agent_priv->lock, flags);
1128 list_add_tail(&mad_send_wr->agent_list,1325 mad_send_wr->is_solicited_fc = is_solicited_fc_mad(mad_send_wr);
1129 &mad_agent_priv->send_list);1326 if (mad_is_for_backlog(mad_send_wr)) {
1327 change_mad_state(mad_send_wr, IB_MAD_STATE_QUEUED);
1328 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
1329 return 0;
1330 }
1331
1332 change_mad_state(mad_send_wr, IB_MAD_STATE_SEND_START);
1130 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);1333 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
1131 1334
1132 if (ib_mad_kernel_rmpp_agent(&mad_agent_priv->agent)) {1335 if (ib_mad_kernel_rmpp_agent(&mad_agent_priv->agent)) {
@@ -1138,7 +1341,7 @@ int ib_post_send_mad(struct ib_mad_send_buf *send_buf, · 5 unmodified lines
1138 if (ret < 0) {1341 if (ret < 0) {
1139 /* Fail send request */1342 /* Fail send request */
1140 spin_lock_irqsave(&mad_agent_priv->lock, flags);1343 spin_lock_irqsave(&mad_agent_priv->lock, flags);
1141 list_del(&mad_send_wr->agent_list);1344 change_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
1142 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);1345 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
1143 deref_mad_agent(mad_agent_priv);1346 deref_mad_agent(mad_agent_priv);
1144 goto error;1347 goto error;
@@ -1746,7 +1949,19 @@ ib_find_send_mad(const struct ib_mad_agent_private *mad_agent_priv, · 601 unmodified lines
1746 */1949 */
1747 (is_direct(mad_hdr->mgmt_class) ||1950 (is_direct(mad_hdr->mgmt_class) ||
1748 rcv_has_same_gid(mad_agent_priv, wr, wc)))1951 rcv_has_same_gid(mad_agent_priv, wr, wc)))
1749 return (wr->status == IB_WC_SUCCESS) ? wr : NULL;1952 return (wr->state != IB_MAD_STATE_CANCELED) ? wr : NULL;
1953 }
1954
1955 list_for_each_entry(wr, &mad_agent_priv->backlog_list, agent_list) {
1956 if ((wr->tid == mad_hdr->tid) &&
1957 rcv_has_same_class(wr, wc) &&
1958 /*
1959 * Don't check GID for direct routed MADs.
1960 * These might have permissive LIDs.
1961 */
1962 (is_direct(mad_hdr->mgmt_class) ||
1963 rcv_has_same_gid(mad_agent_priv, wr, wc)))
1964 return (wr->state != IB_MAD_STATE_CANCELED) ? wr : NULL;
1750 }1965 }
1751 1966
1752 /*1967 /*
@@ -1765,17 +1980,55 @@ ib_find_send_mad(const struct ib_mad_agent_private *mad_agent_priv, · 12 unmodified lines
1765 (is_direct(mad_hdr->mgmt_class) ||1980 (is_direct(mad_hdr->mgmt_class) ||
1766 rcv_has_same_gid(mad_agent_priv, wr, wc)))1981 rcv_has_same_gid(mad_agent_priv, wr, wc)))
1767 /* Verify request has not been canceled */1982 /* Verify request has not been canceled */
1768 return (wr->status == IB_WC_SUCCESS) ? wr : NULL;1983 return (wr->state != IB_MAD_STATE_CANCELED) ? wr : NULL;
1769 }1984 }
1770 return NULL;1985 return NULL;
1771}1986}
1772 1987
1988static void
1989process_backlog_mads(struct ib_mad_agent_private *mad_agent_priv)
1990{
1991 struct ib_mad_send_wr_private *mad_send_wr;
1992 struct ib_mad_send_wc mad_send_wc = {};
1993 unsigned long flags;
1994 int ret;
1995
1996 spin_lock_irqsave(&mad_agent_priv->lock, flags);
1997 while (!list_empty(&mad_agent_priv->backlog_list) &&
1998 (mad_agent_priv->sol_fc_send_count +
1999 mad_agent_priv->sol_fc_wait_count <
2000 mad_agent_priv->sol_fc_max)) {
2001 mad_send_wr = list_entry(mad_agent_priv->backlog_list.next,
2002 struct ib_mad_send_wr_private,
2003 agent_list);
2004 change_mad_state(mad_send_wr, IB_MAD_STATE_SEND_START);
2005 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2006 ret = ib_send_mad(mad_send_wr);
2007 if (ret) {
2008 spin_lock_irqsave(&mad_agent_priv->lock, flags);
2009 deref_mad_agent(mad_agent_priv);
2010 change_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
2011 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2012 mad_send_wc.send_buf = &mad_send_wr->send_buf;
2013 mad_send_wc.status = IB_WC_LOC_QP_OP_ERR;
2014 mad_agent_priv->agent.send_handler(
2015 &mad_agent_priv->agent, &mad_send_wc);
2016 }
2017
2018 spin_lock_irqsave(&mad_agent_priv->lock, flags);
2019 }
2020
2021 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2022}
2023
1773void ib_mark_mad_done(struct ib_mad_send_wr_private *mad_send_wr)2024void ib_mark_mad_done(struct ib_mad_send_wr_private *mad_send_wr)
1774{2025{
1775 mad_send_wr->timeout = 0;2026 mad_send_wr->timeout = 0;
1776 if (mad_send_wr->refcount == 1)2027 if (mad_send_wr->state == IB_MAD_STATE_WAIT_RESP ||
1777 list_move_tail(&mad_send_wr->agent_list,2028 mad_send_wr->state == IB_MAD_STATE_QUEUED)
1778 &mad_send_wr->mad_agent_priv->done_list);2029 change_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
2030 else
2031 change_mad_state(mad_send_wr, IB_MAD_STATE_EARLY_RESP);
1779}2032}
1780 2033
1781static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv,2034static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv,
@@ -1784,6 +2037,7 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, · 2 unmodified lines
1784 struct ib_mad_send_wr_private *mad_send_wr;2037 struct ib_mad_send_wr_private *mad_send_wr;
1785 struct ib_mad_send_wc mad_send_wc;2038 struct ib_mad_send_wc mad_send_wc;
1786 unsigned long flags;2039 unsigned long flags;
2040 bool is_mad_done;
1787 int ret;2041 int ret;
1788 2042
1789 INIT_LIST_HEAD(&mad_recv_wc->rmpp_list);2043 INIT_LIST_HEAD(&mad_recv_wc->rmpp_list);
@@ -1832,6 +2086,7 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, · 42 unmodified lines
1832 }2086 }
1833 } else {2087 } else {
1834 ib_mark_mad_done(mad_send_wr);2088 ib_mark_mad_done(mad_send_wr);
2089 is_mad_done = (mad_send_wr->state == IB_MAD_STATE_DONE);
1835 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);2090 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
1836 2091
1837 /* Defined behavior is to complete response before request */2092 /* Defined behavior is to complete response before request */
@@ -1841,10 +2096,13 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, · 3 unmodified lines
1841 mad_recv_wc);2096 mad_recv_wc);
1842 deref_mad_agent(mad_agent_priv);2097 deref_mad_agent(mad_agent_priv);
1843 2098
1844 mad_send_wc.status = IB_WC_SUCCESS;2099 if (is_mad_done) {
1845 mad_send_wc.vendor_err = 0;2100 mad_send_wc.status = IB_WC_SUCCESS;
1846 mad_send_wc.send_buf = &mad_send_wr->send_buf;2101 mad_send_wc.vendor_err = 0;
1847 ib_mad_complete_send_wr(mad_send_wr, &mad_send_wc);2102 mad_send_wc.send_buf = &mad_send_wr->send_buf;
2103 ib_mad_complete_send_wr(mad_send_wr,
2104 &mad_send_wc);
2105 }
1848 }2106 }
1849 } else {2107 } else {
1850 mad_agent_priv->agent.recv_handler(&mad_agent_priv->agent, NULL,2108 mad_agent_priv->agent.recv_handler(&mad_agent_priv->agent, NULL,
@@ -2172,30 +2430,11 @@ static void adjust_timeout(struct ib_mad_agent_private *mad_agent_priv) · 321 unmodified lines
2172static void wait_for_response(struct ib_mad_send_wr_private *mad_send_wr)2430static void wait_for_response(struct ib_mad_send_wr_private *mad_send_wr)
2173{2431{
2174 struct ib_mad_agent_private *mad_agent_priv;2432 struct ib_mad_agent_private *mad_agent_priv;
2175 struct ib_mad_send_wr_private *temp_mad_send_wr;
2176 struct list_head *list_item;
2177 unsigned long delay;2433 unsigned long delay;
2178 2434
2179 mad_agent_priv = mad_send_wr->mad_agent_priv;2435 mad_agent_priv = mad_send_wr->mad_agent_priv;
2180 list_del(&mad_send_wr->agent_list);
2181
2182 delay = mad_send_wr->timeout;2436 delay = mad_send_wr->timeout;
2183 mad_send_wr->timeout += jiffies;2437 change_mad_state(mad_send_wr, IB_MAD_STATE_WAIT_RESP);
2184
2185 if (delay) {
2186 list_for_each_prev(list_item, &mad_agent_priv->wait_list) {
2187 temp_mad_send_wr = list_entry(list_item,
2188 struct ib_mad_send_wr_private,
2189 agent_list);
2190 if (time_after(mad_send_wr->timeout,
2191 temp_mad_send_wr->timeout))
2192 break;
2193 }
2194 } else {
2195 list_item = &mad_agent_priv->wait_list;
2196 }
2197
2198 list_add(&mad_send_wr->agent_list, list_item);
2199 2438
2200 /* Reschedule a work item if we have a shorter timeout */2439 /* Reschedule a work item if we have a shorter timeout */
2201 if (mad_agent_priv->wait_list.next == &mad_send_wr->agent_list)2440 if (mad_agent_priv->wait_list.next == &mad_send_wr->agent_list)
@@ -2229,32 +2468,28 @@ void ib_mad_complete_send_wr(struct ib_mad_send_wr_private *mad_send_wr, · 27 unmodified lines
2229 } else2468 } else
2230 ret = IB_RMPP_RESULT_UNHANDLED;2469 ret = IB_RMPP_RESULT_UNHANDLED;
2231 2470
2232 if (mad_send_wc->status != IB_WC_SUCCESS &&2471 if (mad_send_wr->state == IB_MAD_STATE_CANCELED)
2233 mad_send_wr->status == IB_WC_SUCCESS) {2472 mad_send_wc->status = IB_WC_WR_FLUSH_ERR;
2234 mad_send_wr->status = mad_send_wc->status;2473 else if (mad_send_wr->state == IB_MAD_STATE_SEND_START &&
2235 mad_send_wr->refcount -= (mad_send_wr->timeout > 0);2474 mad_send_wr->timeout) {
2236 }2475 wait_for_response(mad_send_wr);
2237
2238 if (--mad_send_wr->refcount > 0) {
2239 if (mad_send_wr->refcount == 1 && mad_send_wr->timeout &&
2240 mad_send_wr->status == IB_WC_SUCCESS) {
2241 wait_for_response(mad_send_wr);
2242 }
2243 goto done;2476 goto done;
2244 }2477 }
2245 2478
2246 /* Remove send from MAD agent and notify client of completion */2479 /* Remove send from MAD agent and notify client of completion */
2247 list_del(&mad_send_wr->agent_list);2480 if (mad_send_wr->state != IB_MAD_STATE_DONE)
2481 change_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
2248 adjust_timeout(mad_agent_priv);2482 adjust_timeout(mad_agent_priv);
2249 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);2483 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2250 2484
2251 if (mad_send_wr->status != IB_WC_SUCCESS)2485 if (ret == IB_RMPP_RESULT_INTERNAL) {
2252 mad_send_wc->status = mad_send_wr->status;
2253 if (ret == IB_RMPP_RESULT_INTERNAL)
2254 ib_rmpp_send_handler(mad_send_wc);2486 ib_rmpp_send_handler(mad_send_wc);
2255 else2487 } else {
2488 if (mad_send_wr->is_solicited_fc)
2489 process_backlog_mads(mad_agent_priv);
2256 mad_agent_priv->agent.send_handler(&mad_agent_priv->agent,2490 mad_agent_priv->agent.send_handler(&mad_agent_priv->agent,
2257 mad_send_wc);2491 mad_send_wc);
2492 }
2258 2493
2259 /* Release reference on agent taken when sending */2494 /* Release reference on agent taken when sending */
2260 deref_mad_agent(mad_agent_priv);2495 deref_mad_agent(mad_agent_priv);
@@ -2396,40 +2631,53 @@ static bool ib_mad_send_error(struct ib_mad_port_private *port_priv, · 135 unmodified lines
2396 return true;2631 return true;
2397}2632}
2398 2633
2634static void clear_mad_error_list(struct list_head *list,
2635 enum ib_wc_status wc_status,
2636 struct ib_mad_agent_private *mad_agent_priv)
2637{
2638 struct ib_mad_send_wr_private *mad_send_wr, *n;
2639 struct ib_mad_send_wc mad_send_wc;
2640
2641 mad_send_wc.status = wc_status;
2642 mad_send_wc.vendor_err = 0;
2643
2644 list_for_each_entry_safe(mad_send_wr, n, list, agent_list) {
2645 mad_send_wc.send_buf = &mad_send_wr->send_buf;
2646 mad_agent_priv->agent.send_handler(&mad_agent_priv->agent,
2647 &mad_send_wc);
2648 deref_mad_agent(mad_agent_priv);
2649 }
2650}
2651
2399static void cancel_mads(struct ib_mad_agent_private *mad_agent_priv)2652static void cancel_mads(struct ib_mad_agent_private *mad_agent_priv)
2400{2653{
2401 unsigned long flags;2654 unsigned long flags;
2402 struct ib_mad_send_wr_private *mad_send_wr, *temp_mad_send_wr;2655 struct ib_mad_send_wr_private *mad_send_wr, *temp_mad_send_wr;
2403 struct ib_mad_send_wc mad_send_wc;
2404 struct list_head cancel_list;2656 struct list_head cancel_list;
2405 2657
2406 INIT_LIST_HEAD(&cancel_list);2658 INIT_LIST_HEAD(&cancel_list);
2407 2659
2408 spin_lock_irqsave(&mad_agent_priv->lock, flags);2660 spin_lock_irqsave(&mad_agent_priv->lock, flags);
2409 list_for_each_entry_safe(mad_send_wr, temp_mad_send_wr,2661 list_for_each_entry_safe(mad_send_wr, temp_mad_send_wr,
2410 &mad_agent_priv->send_list, agent_list) {2662 &mad_agent_priv->send_list, agent_list)
2411 if (mad_send_wr->status == IB_WC_SUCCESS) {2663 change_mad_state(mad_send_wr, IB_MAD_STATE_CANCELED);
2412 mad_send_wr->status = IB_WC_WR_FLUSH_ERR;
2413 mad_send_wr->refcount -= (mad_send_wr->timeout > 0);
2414 }
2415 }
2416 2664
2417 /* Empty wait list to prevent receives from finding a request */2665 /* Empty wait & backlog list to prevent receives from finding request */
2418 list_splice_init(&mad_agent_priv->wait_list, &cancel_list);2666 list_for_each_entry_safe(mad_send_wr, temp_mad_send_wr,
2419 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);2667 &mad_agent_priv->wait_list, agent_list) {
2420 2668 change_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
2421 /* Report all cancelled requests */2669 list_add_tail(&mad_send_wr->agent_list, &cancel_list);
2422 mad_send_wc.status = IB_WC_WR_FLUSH_ERR;2670 }
2423 mad_send_wc.vendor_err = 0;
2424 2671
2425 list_for_each_entry_safe(mad_send_wr, temp_mad_send_wr,2672 list_for_each_entry_safe(mad_send_wr, temp_mad_send_wr,
2426 &cancel_list, agent_list) {2673 &mad_agent_priv->backlog_list, agent_list) {
2427 mad_send_wc.send_buf = &mad_send_wr->send_buf;2674 change_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
2428 list_del(&mad_send_wr->agent_list);2675 list_add_tail(&mad_send_wr->agent_list, &cancel_list);
2429 mad_agent_priv->agent.send_handler(&mad_agent_priv->agent,
2430 &mad_send_wc);
2431 deref_mad_agent(mad_agent_priv);
2432 }2676 }
2677
2678 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2679 /* Report all cancelled requests */
2680 clear_mad_error_list(&cancel_list, IB_WC_WR_FLUSH_ERR, mad_agent_priv);
2433}2681}
2434 2682
2435static struct ib_mad_send_wr_private*2683static struct ib_mad_send_wr_private*
@@ -2451,6 +2699,13 @@ find_send_wr(struct ib_mad_agent_private *mad_agent_priv, · 15 unmodified lines
2451 &mad_send_wr->send_buf == send_buf)2699 &mad_send_wr->send_buf == send_buf)
2452 return mad_send_wr;2700 return mad_send_wr;
2453 }2701 }
2702
2703 list_for_each_entry(mad_send_wr, &mad_agent_priv->backlog_list,
2704 agent_list) {
2705 if (&mad_send_wr->send_buf == send_buf)
2706 return mad_send_wr;
2707 }
2708
2454 return NULL;2709 return NULL;
2455}2710}
2456 2711
@@ -2468,16 +2723,16 @@ int ib_modify_mad(struct ib_mad_send_buf *send_buf, u32 timeout_ms) · 11 unmodified lines
2468 struct ib_mad_agent_private, agent);2723 struct ib_mad_agent_private, agent);
2469 spin_lock_irqsave(&mad_agent_priv->lock, flags);2724 spin_lock_irqsave(&mad_agent_priv->lock, flags);
2470 mad_send_wr = find_send_wr(mad_agent_priv, send_buf);2725 mad_send_wr = find_send_wr(mad_agent_priv, send_buf);
2471 if (!mad_send_wr || mad_send_wr->status != IB_WC_SUCCESS) {2726 if (!mad_send_wr || mad_send_wr->state == IB_MAD_STATE_CANCELED) {
2472 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);2727 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2473 return -EINVAL;2728 return -EINVAL;
2474 }2729 }
2475 2730
2476 active = (!mad_send_wr->timeout || mad_send_wr->refcount > 1);2731 active = ((mad_send_wr->state == IB_MAD_STATE_SEND_START) ||
2477 if (!timeout_ms) {2732 (mad_send_wr->state == IB_MAD_STATE_EARLY_RESP) ||
2478 mad_send_wr->status = IB_WC_WR_FLUSH_ERR;2733 (mad_send_wr->state == IB_MAD_STATE_QUEUED && timeout_ms));
2479 mad_send_wr->refcount -= (mad_send_wr->timeout > 0);2734 if (!timeout_ms)
2480 }2735 change_mad_state(mad_send_wr, IB_MAD_STATE_CANCELED);
2481 2736
2482 mad_send_wr->send_buf.timeout_ms = timeout_ms;2737 mad_send_wr->send_buf.timeout_ms = timeout_ms;
2483 if (active)2738 if (active)
@@ -2589,6 +2844,11 @@ static int retry_send(struct ib_mad_send_wr_private *mad_send_wr) · 105 unmodified lines
2589 mad_send_wr->send_buf.retries++;2844 mad_send_wr->send_buf.retries++;
2590 2845
2591 mad_send_wr->timeout = msecs_to_jiffies(mad_send_wr->send_buf.timeout_ms);2846 mad_send_wr->timeout = msecs_to_jiffies(mad_send_wr->send_buf.timeout_ms);
2847 if (mad_send_wr->is_solicited_fc &&
2848 !list_empty(&mad_send_wr->mad_agent_priv->backlog_list)) {
2849 change_mad_state(mad_send_wr, IB_MAD_STATE_QUEUED);
2850 return 0;
2851 }
2592 2852
2593 if (ib_mad_kernel_rmpp_agent(&mad_send_wr->mad_agent_priv->agent)) {2853 if (ib_mad_kernel_rmpp_agent(&mad_send_wr->mad_agent_priv->agent)) {
2594 ret = ib_retry_rmpp(mad_send_wr);2854 ret = ib_retry_rmpp(mad_send_wr);
@@ -2606,26 +2866,25 @@ static int retry_send(struct ib_mad_send_wr_private *mad_send_wr) · 11 unmodified lines
2606 } else2866 } else
2607 ret = ib_send_mad(mad_send_wr);2867 ret = ib_send_mad(mad_send_wr);
2608 2868
2609 if (!ret) {2869 if (!ret)
2610 mad_send_wr->refcount++;2870 change_mad_state(mad_send_wr, IB_MAD_STATE_SEND_START);
2611 list_add_tail(&mad_send_wr->agent_list,2871
2612 &mad_send_wr->mad_agent_priv->send_list);
2613 }
2614 return ret;2872 return ret;
2615}2873}
2616 2874
2617static void timeout_sends(struct work_struct *work)2875static void timeout_sends(struct work_struct *work)
2618{2876{
2619 struct ib_mad_send_wr_private *mad_send_wr, *n;2877 struct ib_mad_send_wr_private *mad_send_wr;
2620 struct ib_mad_agent_private *mad_agent_priv;2878 struct ib_mad_agent_private *mad_agent_priv;
2621 struct ib_mad_send_wc mad_send_wc;2879 struct list_head timeout_list;
2622 struct list_head local_list;2880 struct list_head cancel_list;
2881 struct list_head *list_item;
2623 unsigned long flags, delay;2882 unsigned long flags, delay;
2624 2883
2625 mad_agent_priv = container_of(work, struct ib_mad_agent_private,2884 mad_agent_priv = container_of(work, struct ib_mad_agent_private,
2626 timed_work.work);2885 timed_work.work);
2627 mad_send_wc.vendor_err = 0;2886 INIT_LIST_HEAD(&timeout_list);
2628 INIT_LIST_HEAD(&local_list);2887 INIT_LIST_HEAD(&cancel_list);
2629 2888
2630 spin_lock_irqsave(&mad_agent_priv->lock, flags);2889 spin_lock_irqsave(&mad_agent_priv->lock, flags);
2631 while (!list_empty(&mad_agent_priv->wait_list)) {2890 while (!list_empty(&mad_agent_priv->wait_list)) {
@@ -2643,25 +2902,22 @@ static void timeout_sends(struct work_struct *work) · 11 unmodified lines
2643 break;2902 break;
2644 }2903 }
2645 2904
2646 list_del_init(&mad_send_wr->agent_list);2905 if (mad_send_wr->state == IB_MAD_STATE_CANCELED)
2647 if (mad_send_wr->status == IB_WC_SUCCESS &&2906 list_item = &cancel_list;
2648 !retry_send(mad_send_wr))2907 else if (retry_send(mad_send_wr))
2908 list_item = &timeout_list;
2909 else
2649 continue;2910 continue;
2650 2911
2651 list_add_tail(&mad_send_wr->agent_list, &local_list);2912 change_mad_state(mad_send_wr, IB_MAD_STATE_DONE);
2913 list_add_tail(&mad_send_wr->agent_list, list_item);
2652 }2914 }
2653 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2654 2915
2655 list_for_each_entry_safe(mad_send_wr, n, &local_list, agent_list) {2916 spin_unlock_irqrestore(&mad_agent_priv->lock, flags);
2656 if (mad_send_wr->status == IB_WC_SUCCESS)2917 process_backlog_mads(mad_agent_priv);
2657 mad_send_wc.status = IB_WC_RESP_TIMEOUT_ERR;2918 clear_mad_error_list(&timeout_list, IB_WC_RESP_TIMEOUT_ERR,
2658 else2919 mad_agent_priv);
2659 mad_send_wc.status = mad_send_wr->status;2920 clear_mad_error_list(&cancel_list, IB_WC_WR_FLUSH_ERR, mad_agent_priv);
2660 mad_send_wc.send_buf = &mad_send_wr->send_buf;
2661 mad_agent_priv->agent.send_handler(&mad_agent_priv->agent,
2662 &mad_send_wc);
2663 deref_mad_agent(mad_agent_priv);
2664 }
2665}2921}
2666 2922
2667/*2923/*
drivers/infiniband/core/mad_priv.h+73−3
@@ -95,13 +95,16 @@ struct ib_mad_agent_private { · 94 unmodified lines
95 95
96 spinlock_t lock;96 spinlock_t lock;
97 struct list_head send_list;97 struct list_head send_list;
98 unsigned int sol_fc_send_count;
98 struct list_head wait_list;99 struct list_head wait_list;
99 struct list_head done_list;100 unsigned int sol_fc_wait_count;
100 struct delayed_work timed_work;101 struct delayed_work timed_work;
101 unsigned long timeout;102 unsigned long timeout;
102 struct list_head local_list;103 struct list_head local_list;
103 struct work_struct local_work;104 struct work_struct local_work;
104 struct list_head rmpp_list;105 struct list_head rmpp_list;
106 unsigned int sol_fc_max;
107 struct list_head backlog_list;
105 108
106 refcount_t refcount;109 refcount_t refcount;
107 union {110 union {
@@ -118,6 +121,32 @@ struct ib_mad_snoop_private { · 10 unmodified lines
118 struct completion comp;121 struct completion comp;
119};122};
120 123
124enum ib_mad_state {
125 /* MAD is in the making and is not yet in any list */
126 IB_MAD_STATE_INIT,
127 /* MAD is in backlog list */
128 IB_MAD_STATE_QUEUED,
129 /*
130 * MAD was sent to the QP and is waiting for completion
131 * notification in send list.
132 */
133 IB_MAD_STATE_SEND_START,
134 /*
135 * MAD send completed successfully, waiting for a response
136 * in wait list.
137 */
138 IB_MAD_STATE_WAIT_RESP,
139 /*
140 * Response came early, before send completion notification,
141 * in send list.
142 */
143 IB_MAD_STATE_EARLY_RESP,
144 /* MAD was canceled while in wait or send list */
145 IB_MAD_STATE_CANCELED,
146 /* MAD processing completed, MAD in no list */
147 IB_MAD_STATE_DONE
148};
149
121struct ib_mad_send_wr_private {150struct ib_mad_send_wr_private {
122 struct ib_mad_list_head mad_list;151 struct ib_mad_list_head mad_list;
123 struct list_head agent_list;152 struct list_head agent_list;
@@ -132,8 +161,6 @@ struct ib_mad_send_wr_private { · 8 unmodified lines
132 int max_retries;161 int max_retries;
133 int retries_left;162 int retries_left;
134 int retry;163 int retry;
135 int refcount;
136 enum ib_wc_status status;
137 164
138 /* RMPP control */165 /* RMPP control */
139 struct list_head rmpp_list;166 struct list_head rmpp_list;
@@ -143,8 +170,48 @@ struct ib_mad_send_wr_private { · 3 unmodified lines
143 int seg_num;170 int seg_num;
144 int newwin;171 int newwin;
145 int pad;172 int pad;
173
174 enum ib_mad_state state;
175
176 /* Solicited MAD flow control */
177 bool is_solicited_fc;
146};178};
147 179
180static inline void expect_mad_state(struct ib_mad_send_wr_private *mad_send_wr,
181 enum ib_mad_state expected_state)
182{
183 if (IS_ENABLED(CONFIG_LOCKDEP))
184 WARN_ON(mad_send_wr->state != expected_state);
185}
186
187static inline void expect_mad_state2(struct ib_mad_send_wr_private *mad_send_wr,
188 enum ib_mad_state expected_state1,
189 enum ib_mad_state expected_state2)
190{
191 if (IS_ENABLED(CONFIG_LOCKDEP))
192 WARN_ON(mad_send_wr->state != expected_state1 &&
193 mad_send_wr->state != expected_state2);
194}
195
196static inline void expect_mad_state3(struct ib_mad_send_wr_private *mad_send_wr,
197 enum ib_mad_state expected_state1,
198 enum ib_mad_state expected_state2,
199 enum ib_mad_state expected_state3)
200{
201 if (IS_ENABLED(CONFIG_LOCKDEP))
202 WARN_ON(mad_send_wr->state != expected_state1 &&
203 mad_send_wr->state != expected_state2 &&
204 mad_send_wr->state != expected_state3);
205}
206
207static inline void
208not_expect_mad_state(struct ib_mad_send_wr_private *mad_send_wr,
209 enum ib_mad_state wrong_state)
210{
211 if (IS_ENABLED(CONFIG_LOCKDEP))
212 WARN_ON(mad_send_wr->state == wrong_state);
213}
214
148struct ib_mad_local_private {215struct ib_mad_local_private {
149 struct list_head completion_list;216 struct list_head completion_list;
150 struct ib_mad_private *mad_priv;217 struct ib_mad_private *mad_priv;
@@ -222,4 +289,7 @@ void ib_mark_mad_done(struct ib_mad_send_wr_private *mad_send_wr); · 71 unmodified lines
222void ib_reset_mad_timeout(struct ib_mad_send_wr_private *mad_send_wr,289void ib_reset_mad_timeout(struct ib_mad_send_wr_private *mad_send_wr,
223 unsigned long timeout_ms);290 unsigned long timeout_ms);
224 291
292void change_mad_state(struct ib_mad_send_wr_private *mad_send_wr,
293 enum ib_mad_state new_state);
294
225#endif /* __IB_MAD_PRIV_H__ */295#endif /* __IB_MAD_PRIV_H__ */
drivers/infiniband/core/mad_rmpp.c+24−17
@@ -608,16 +608,20 @@ static void abort_send(struct ib_mad_agent_private *agent, · 607 unmodified lines
608 goto out; /* Unmatched send */608 goto out; /* Unmatched send */
609 609
610 if ((mad_send_wr->last_ack == mad_send_wr->send_buf.seg_count) ||610 if ((mad_send_wr->last_ack == mad_send_wr->send_buf.seg_count) ||
611 (!mad_send_wr->timeout) || (mad_send_wr->status != IB_WC_SUCCESS))611 (!mad_send_wr->timeout) ||
612 (mad_send_wr->state == IB_MAD_STATE_CANCELED))
612 goto out; /* Send is already done */613 goto out; /* Send is already done */
613 614
614 ib_mark_mad_done(mad_send_wr);615 ib_mark_mad_done(mad_send_wr);
616 if (mad_send_wr->state == IB_MAD_STATE_DONE) {
617 spin_unlock_irqrestore(&agent->lock, flags);
618 wc.status = IB_WC_REM_ABORT_ERR;
619 wc.vendor_err = rmpp_status;
620 wc.send_buf = &mad_send_wr->send_buf;
621 ib_mad_complete_send_wr(mad_send_wr, &wc);
622 return;
623 }
615 spin_unlock_irqrestore(&agent->lock, flags);624 spin_unlock_irqrestore(&agent->lock, flags);
616
617 wc.status = IB_WC_REM_ABORT_ERR;
618 wc.vendor_err = rmpp_status;
619 wc.send_buf = &mad_send_wr->send_buf;
620 ib_mad_complete_send_wr(mad_send_wr, &wc);
621 return;625 return;
622out:626out:
623 spin_unlock_irqrestore(&agent->lock, flags);627 spin_unlock_irqrestore(&agent->lock, flags);
@@ -684,7 +688,8 @@ static void process_rmpp_ack(struct ib_mad_agent_private *agent, · 60 unmodified lines
684 }688 }
685 689
686 if ((mad_send_wr->last_ack == mad_send_wr->send_buf.seg_count) ||690 if ((mad_send_wr->last_ack == mad_send_wr->send_buf.seg_count) ||
687 (!mad_send_wr->timeout) || (mad_send_wr->status != IB_WC_SUCCESS))691 (!mad_send_wr->timeout) ||
692 (mad_send_wr->state == IB_MAD_STATE_CANCELED))
688 goto out; /* Send is already done */693 goto out; /* Send is already done */
689 694
690 if (seg_num > mad_send_wr->send_buf.seg_count ||695 if (seg_num > mad_send_wr->send_buf.seg_count ||
@@ -709,21 +714,24 @@ static void process_rmpp_ack(struct ib_mad_agent_private *agent, · 18 unmodified lines
709 struct ib_mad_send_wc wc;714 struct ib_mad_send_wc wc;
710 715
711 ib_mark_mad_done(mad_send_wr);716 ib_mark_mad_done(mad_send_wr);
717 if (mad_send_wr->state == IB_MAD_STATE_DONE) {
718 spin_unlock_irqrestore(&agent->lock, flags);
719 wc.status = IB_WC_SUCCESS;
720 wc.vendor_err = 0;
721 wc.send_buf = &mad_send_wr->send_buf;
722 ib_mad_complete_send_wr(mad_send_wr, &wc);
723 return;
724 }
712 spin_unlock_irqrestore(&agent->lock, flags);725 spin_unlock_irqrestore(&agent->lock, flags);
713
714 wc.status = IB_WC_SUCCESS;
715 wc.vendor_err = 0;
716 wc.send_buf = &mad_send_wr->send_buf;
717 ib_mad_complete_send_wr(mad_send_wr, &wc);
718 return;726 return;
719 }727 }
720 if (mad_send_wr->refcount == 1)728 if (mad_send_wr->state == IB_MAD_STATE_WAIT_RESP)
721 ib_reset_mad_timeout(mad_send_wr,729 ib_reset_mad_timeout(mad_send_wr,
722 mad_send_wr->send_buf.timeout_ms);730 mad_send_wr->send_buf.timeout_ms);
723 spin_unlock_irqrestore(&agent->lock, flags);731 spin_unlock_irqrestore(&agent->lock, flags);
724 ack_ds_ack(agent, mad_recv_wc);732 ack_ds_ack(agent, mad_recv_wc);
725 return;733 return;
726 } else if (mad_send_wr->refcount == 1 &&734 } else if (mad_send_wr->state == IB_MAD_STATE_WAIT_RESP &&
727 mad_send_wr->seg_num < mad_send_wr->newwin &&735 mad_send_wr->seg_num < mad_send_wr->newwin &&
728 mad_send_wr->seg_num < mad_send_wr->send_buf.seg_count) {736 mad_send_wr->seg_num < mad_send_wr->send_buf.seg_count) {
729 /* Send failure will just result in a timeout/retry */737 /* Send failure will just result in a timeout/retry */
@@ -731,7 +739,7 @@ static void process_rmpp_ack(struct ib_mad_agent_private *agent, · 1 unmodified line
731 if (ret)739 if (ret)
732 goto out;740 goto out;
733 741
734 mad_send_wr->refcount++;742 change_mad_state(mad_send_wr, IB_MAD_STATE_SEND_START);
735 list_move_tail(&mad_send_wr->agent_list,743 list_move_tail(&mad_send_wr->agent_list,
736 &mad_send_wr->mad_agent_priv->send_list);744 &mad_send_wr->mad_agent_priv->send_list);
737 }745 }
@@ -890,7 +898,6 @@ int ib_send_rmpp_mad(struct ib_mad_send_wr_private *mad_send_wr) · 152 unmodified lines
890 mad_send_wr->newwin = init_newwin(mad_send_wr);898 mad_send_wr->newwin = init_newwin(mad_send_wr);
891 899
892 /* We need to wait for the final ACK even if there isn't a response */900 /* We need to wait for the final ACK even if there isn't a response */
893 mad_send_wr->refcount += (mad_send_wr->timeout == 0);
894 ret = send_next_seg(mad_send_wr);901 ret = send_next_seg(mad_send_wr);
895 if (!ret)902 if (!ret)
896 return IB_RMPP_RESULT_CONSUMED;903 return IB_RMPP_RESULT_CONSUMED;
@@ -912,7 +919,7 @@ int ib_process_rmpp_send_wc(struct ib_mad_send_wr_private *mad_send_wr, · 15 unmodified lines
912 return IB_RMPP_RESULT_INTERNAL; /* ACK, STOP, or ABORT */919 return IB_RMPP_RESULT_INTERNAL; /* ACK, STOP, or ABORT */
913 920
914 if (mad_send_wc->status != IB_WC_SUCCESS ||921 if (mad_send_wc->status != IB_WC_SUCCESS ||
915 mad_send_wr->status != IB_WC_SUCCESS)922 mad_send_wr->state == IB_MAD_STATE_CANCELED)
916 return IB_RMPP_RESULT_PROCESSED; /* Canceled or send error */923 return IB_RMPP_RESULT_PROCESSED; /* Canceled or send error */
917 924
918 if (!mad_send_wr->timeout)925 if (!mad_send_wr->timeout)
drivers/infiniband/core/nldev.c+13−11
@@ -255,7 +255,7 @@ EXPORT_SYMBOL(rdma_nl_put_driver_u64_hex); · 254 unmodified lines
255 255
256bool rdma_nl_get_privileged_qkey(void)256bool rdma_nl_get_privileged_qkey(void)
257{257{
258 return privileged_qkey || capable(CAP_NET_RAW);258 return privileged_qkey;
259}259}
260EXPORT_SYMBOL(rdma_nl_get_privileged_qkey);260EXPORT_SYMBOL(rdma_nl_get_privileged_qkey);
261 261
@@ -1469,10 +1469,11 @@ static const struct nldev_fill_res_entry fill_entries[RDMA_RESTRACK_MAX] = { · 1207 unmodified lines
1469 1469
1470};1470};
1471 1471
1472static int res_get_common_doit(struct sk_buff *skb, struct nlmsghdr *nlh,1472static noinline_for_stack int
1473 struct netlink_ext_ack *extack,1473res_get_common_doit(struct sk_buff *skb, struct nlmsghdr *nlh,
1474 enum rdma_restrack_type res_type,1474 struct netlink_ext_ack *extack,
1475 res_fill_func_t fill_func)1475 enum rdma_restrack_type res_type,
1476 res_fill_func_t fill_func)
1476{1477{
1477 const struct nldev_fill_res_entry *fe = &fill_entries[res_type];1478 const struct nldev_fill_res_entry *fe = &fill_entries[res_type];
1478 struct nlattr *tb[RDMA_NLDEV_ATTR_MAX];1479 struct nlattr *tb[RDMA_NLDEV_ATTR_MAX];
@@ -2263,10 +2264,10 @@ err: · 784 unmodified lines
2263 return ret;2264 return ret;
2264}2265}
2265 2266
2266static int stat_get_doit_default_counter(struct sk_buff *skb,2267static noinline_for_stack int
2267 struct nlmsghdr *nlh,2268stat_get_doit_default_counter(struct sk_buff *skb, struct nlmsghdr *nlh,
2268 struct netlink_ext_ack *extack,2269 struct netlink_ext_ack *extack,
2269 struct nlattr *tb[])2270 struct nlattr *tb[])
2270{2271{
2271 struct rdma_hw_stats *stats;2272 struct rdma_hw_stats *stats;
2272 struct nlattr *table_attr;2273 struct nlattr *table_attr;
@@ -2356,8 +2357,9 @@ err: · 83 unmodified lines
2356 return ret;2357 return ret;
2357}2358}
2358 2359
2359static int stat_get_doit_qp(struct sk_buff *skb, struct nlmsghdr *nlh,2360static noinline_for_stack int
2360 struct netlink_ext_ack *extack, struct nlattr *tb[])2361stat_get_doit_qp(struct sk_buff *skb, struct nlmsghdr *nlh,
2362 struct netlink_ext_ack *extack, struct nlattr *tb[])
2361 2363
2362{2364{
2363 static enum rdma_nl_counter_mode mode;2365 static enum rdma_nl_counter_mode mode;
drivers/infiniband/core/rdma_core.c+29−0
@@ -1019,3 +1019,32 @@ void uverbs_finalize_object(struct ib_uobject *uobj, · 1018 unmodified lines
1019 WARN_ON(true);1019 WARN_ON(true);
1020 }1020 }
1021}1021}
1022
1023/**
1024 * rdma_uattrs_has_raw_cap() - Returns whether a rdma device linked to the
1025 * uverbs attributes file has CAP_NET_RAW
1026 * capability or not.
1027 *
1028 * @attrs: Pointer to uverbs attributes
1029 *
1030 * Returns true if a rdma device's owning user namespace has CAP_NET_RAW
1031 * capability, otherwise false.
1032 */
1033bool rdma_uattrs_has_raw_cap(const struct uverbs_attr_bundle *attrs)
1034{
1035 struct ib_uverbs_file *ufile = attrs->ufile;
1036 struct ib_ucontext *ucontext;
1037 bool has_cap = false;
1038 int srcu_key;
1039
1040 srcu_key = srcu_read_lock(&ufile->device->disassociate_srcu);
1041 ucontext = ib_uverbs_get_ucontext_file(ufile);
1042 if (IS_ERR(ucontext))
1043 goto out;
1044 has_cap = rdma_dev_has_raw_cap(ucontext->device);
1045
1046out:
1047 srcu_read_unlock(&ufile->device->disassociate_srcu, srcu_key);
1048 return has_cap;
1049}
1050EXPORT_SYMBOL(rdma_uattrs_has_raw_cap);
drivers/infiniband/core/rdma_core.h+1−0
@@ -156,6 +156,7 @@ extern const struct uapi_definition uverbs_def_obj_counters[]; · 155 unmodified lines
156extern const struct uapi_definition uverbs_def_obj_cq[];156extern const struct uapi_definition uverbs_def_obj_cq[];
157extern const struct uapi_definition uverbs_def_obj_device[];157extern const struct uapi_definition uverbs_def_obj_device[];
158extern const struct uapi_definition uverbs_def_obj_dm[];158extern const struct uapi_definition uverbs_def_obj_dm[];
159extern const struct uapi_definition uverbs_def_obj_dmah[];
159extern const struct uapi_definition uverbs_def_obj_flow_action[];160extern const struct uapi_definition uverbs_def_obj_flow_action[];
160extern const struct uapi_definition uverbs_def_obj_intf[];161extern const struct uapi_definition uverbs_def_obj_intf[];
161extern const struct uapi_definition uverbs_def_obj_mr[];162extern const struct uapi_definition uverbs_def_obj_mr[];
drivers/infiniband/core/restrack.c+2−0
@@ -100,6 +100,8 @@ static struct ib_device *res_to_dev(struct rdma_restrack_entry *res) · 99 unmodified lines
100 return container_of(res, struct rdma_counter, res)->device;100 return container_of(res, struct rdma_counter, res)->device;
101 case RDMA_RESTRACK_SRQ:101 case RDMA_RESTRACK_SRQ:
102 return container_of(res, struct ib_srq, res)->device;102 return container_of(res, struct ib_srq, res)->device;
103 case RDMA_RESTRACK_DMAH:
104 return container_of(res, struct ib_dmah, res)->device;
103 default:105 default:
104 WARN_ONCE(true, "Wrong resource tracking type %u\n", res->type);106 WARN_ONCE(true, "Wrong resource tracking type %u\n", res->type);
105 return NULL;107 return NULL;
drivers/infiniband/core/uverbs_cmd.c+7−6
@@ -741,7 +741,7 @@ static int ib_uverbs_reg_mr(struct uverbs_attr_bundle *attrs) · 740 unmodified lines
741 }741 }
742 742
743 mr = pd->device->ops.reg_user_mr(pd, cmd.start, cmd.length, cmd.hca_va,743 mr = pd->device->ops.reg_user_mr(pd, cmd.start, cmd.length, cmd.hca_va,
744 cmd.access_flags,744 cmd.access_flags, NULL,
745 &attrs->driver_udata);745 &attrs->driver_udata);
746 if (IS_ERR(mr)) {746 if (IS_ERR(mr)) {
747 ret = PTR_ERR(mr);747 ret = PTR_ERR(mr);
@@ -1312,9 +1312,9 @@ static int create_qp(struct uverbs_attr_bundle *attrs, · 564 unmodified lines
1312 1312
1313 switch (cmd->qp_type) {1313 switch (cmd->qp_type) {
1314 case IB_QPT_RAW_PACKET:1314 case IB_QPT_RAW_PACKET:
1315 if (!capable(CAP_NET_RAW))1315 if (!rdma_uattrs_has_raw_cap(attrs))
1316 return -EPERM;1316 return -EPERM;
1317 break;1317 fallthrough;
1318 case IB_QPT_RC:1318 case IB_QPT_RC:
1319 case IB_QPT_UC:1319 case IB_QPT_UC:
1320 case IB_QPT_UD:1320 case IB_QPT_UD:
@@ -1451,7 +1451,7 @@ static int create_qp(struct uverbs_attr_bundle *attrs, · 130 unmodified lines
1451 }1451 }
1452 1452
1453 if (attr.create_flags & IB_QP_CREATE_SOURCE_QPN) {1453 if (attr.create_flags & IB_QP_CREATE_SOURCE_QPN) {
1454 if (!capable(CAP_NET_RAW)) {1454 if (!rdma_uattrs_has_raw_cap(attrs)) {
1455 ret = -EPERM;1455 ret = -EPERM;
1456 goto err_put;1456 goto err_put;
1457 }1457 }
@@ -1877,7 +1877,8 @@ static int modify_qp(struct uverbs_attr_bundle *attrs, · 419 unmodified lines
1877 attr->path_mig_state = cmd->base.path_mig_state;1877 attr->path_mig_state = cmd->base.path_mig_state;
1878 if (cmd->base.attr_mask & IB_QP_QKEY) {1878 if (cmd->base.attr_mask & IB_QP_QKEY) {
1879 if (cmd->base.qkey & IB_QP_SET_QKEY &&1879 if (cmd->base.qkey & IB_QP_SET_QKEY &&
1880 !rdma_nl_get_privileged_qkey()) {1880 !(rdma_nl_get_privileged_qkey() ||
1881 rdma_uattrs_has_raw_cap(attrs))) {
1881 ret = -EPERM;1882 ret = -EPERM;
1882 goto release_qp;1883 goto release_qp;
1883 }1884 }
@@ -3225,7 +3226,7 @@ static int ib_uverbs_ex_create_flow(struct uverbs_attr_bundle *attrs) · 1341 unmodified lines
3225 if (cmd.comp_mask)3226 if (cmd.comp_mask)
3226 return -EINVAL;3227 return -EINVAL;
3227 3228
3228 if (!capable(CAP_NET_RAW))3229 if (!rdma_uattrs_has_raw_cap(attrs))
3229 return -EPERM;3230 return -EPERM;
3230 3231
3231 if (cmd.flow_attr.flags >= IB_FLOW_ATTR_FLAGS_RESERVED)3232 if (cmd.flow_attr.flags >= IB_FLOW_ATTR_FLAGS_RESERVED)
drivers/infiniband/core/uverbs_std_types_cq.c+81−6
@@ -64,15 +64,21 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)( · 63 unmodified lines
64 struct ib_ucq_object *obj = container_of(64 struct ib_ucq_object *obj = container_of(
65 uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_CQ_HANDLE),65 uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_CQ_HANDLE),
66 typeof(*obj), uevent.uobject);66 typeof(*obj), uevent.uobject);
67 struct ib_uverbs_completion_event_file *ev_file = NULL;
67 struct ib_device *ib_dev = attrs->context->device;68 struct ib_device *ib_dev = attrs->context->device;
68 int ret;69 struct ib_umem_dmabuf *umem_dmabuf;
69 u64 user_handle;
70 struct ib_cq_init_attr attr = {};70 struct ib_cq_init_attr attr = {};
71 struct ib_cq *cq;
72 struct ib_uverbs_completion_event_file *ev_file = NULL;
73 struct ib_uobject *ev_file_uobj;71 struct ib_uobject *ev_file_uobj;
72 struct ib_umem *umem = NULL;
73 u64 buffer_length;
74 u64 buffer_offset;
75 struct ib_cq *cq;
76 u64 user_handle;
77 u64 buffer_va;
78 int buffer_fd;
79 int ret;
74 80
75 if (!ib_dev->ops.create_cq || !ib_dev->ops.destroy_cq)81 if ((!ib_dev->ops.create_cq && !ib_dev->ops.create_cq_umem) || !ib_dev->ops.destroy_cq)
76 return -EOPNOTSUPP;82 return -EOPNOTSUPP;
77 83
78 ret = uverbs_copy_from(&attr.comp_vector, attrs,84 ret = uverbs_copy_from(&attr.comp_vector, attrs,
@@ -112,9 +118,66 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)( · 33 unmodified lines
112 INIT_LIST_HEAD(&obj->comp_list);118 INIT_LIST_HEAD(&obj->comp_list);
113 INIT_LIST_HEAD(&obj->uevent.event_list);119 INIT_LIST_HEAD(&obj->uevent.event_list);
114 120
121 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_VA)) {
122
123 ret = uverbs_copy_from(&buffer_va, attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_VA);
124 if (ret)
125 goto err_event_file;
126
127 ret = uverbs_copy_from(&buffer_length, attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_LENGTH);
128 if (ret)
129 goto err_event_file;
130
131 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_FD) ||
132 uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_OFFSET) ||
133 !ib_dev->ops.create_cq_umem) {
134 ret = -EINVAL;
135 goto err_event_file;
136 }
137
138 umem = ib_umem_get(ib_dev, buffer_va, buffer_length, IB_ACCESS_LOCAL_WRITE);
139 if (IS_ERR(umem)) {
140 ret = PTR_ERR(umem);
141 goto err_event_file;
142 }
143 } else if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_FD)) {
144
145 ret = uverbs_get_raw_fd(&buffer_fd, attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_FD);
146 if (ret)
147 goto err_event_file;
148
149 ret = uverbs_copy_from(&buffer_offset, attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_OFFSET);
150 if (ret)
151 goto err_event_file;
152
153 ret = uverbs_copy_from(&buffer_length, attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_LENGTH);
154 if (ret)
155 goto err_event_file;
156
157 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_VA) ||
158 !ib_dev->ops.create_cq_umem) {
159 ret = -EINVAL;
160 goto err_event_file;
161 }
162
163 umem_dmabuf = ib_umem_dmabuf_get_pinned(ib_dev, buffer_offset, buffer_length,
164 buffer_fd, IB_ACCESS_LOCAL_WRITE);
165 if (IS_ERR(umem_dmabuf)) {
166 ret = PTR_ERR(umem_dmabuf);
167 goto err_event_file;
168 }
169 umem = &umem_dmabuf->umem;
170 } else if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_OFFSET) ||
171 uverbs_attr_is_valid(attrs, UVERBS_ATTR_CREATE_CQ_BUFFER_LENGTH) ||
172 !ib_dev->ops.create_cq) {
173 ret = -EINVAL;
174 goto err_event_file;
175 }
176
115 cq = rdma_zalloc_drv_obj(ib_dev, ib_cq);177 cq = rdma_zalloc_drv_obj(ib_dev, ib_cq);
116 if (!cq) {178 if (!cq) {
117 ret = -ENOMEM;179 ret = -ENOMEM;
180 ib_umem_release(umem);
118 goto err_event_file;181 goto err_event_file;
119 }182 }
120 183
@@ -128,7 +191,8 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)( · 7 unmodified lines
128 rdma_restrack_new(&cq->res, RDMA_RESTRACK_CQ);191 rdma_restrack_new(&cq->res, RDMA_RESTRACK_CQ);
129 rdma_restrack_set_name(&cq->res, NULL);192 rdma_restrack_set_name(&cq->res, NULL);
130 193
131 ret = ib_dev->ops.create_cq(cq, &attr, attrs);194 ret = umem ? ib_dev->ops.create_cq_umem(cq, &attr, umem, attrs) :
195 ib_dev->ops.create_cq(cq, &attr, attrs);
132 if (ret)196 if (ret)
133 goto err_free;197 goto err_free;
134 198
@@ -180,6 +244,17 @@ DECLARE_UVERBS_NAMED_METHOD( · 45 unmodified lines
180 UVERBS_OBJECT_ASYNC_EVENT,244 UVERBS_OBJECT_ASYNC_EVENT,
181 UVERBS_ACCESS_READ,245 UVERBS_ACCESS_READ,
182 UA_OPTIONAL),246 UA_OPTIONAL),
247 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_CQ_BUFFER_VA,
248 UVERBS_ATTR_TYPE(u64),
249 UA_OPTIONAL),
250 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_CQ_BUFFER_LENGTH,
251 UVERBS_ATTR_TYPE(u64),
252 UA_OPTIONAL),
253 UVERBS_ATTR_RAW_FD(UVERBS_ATTR_CREATE_CQ_BUFFER_FD,
254 UA_OPTIONAL),
255 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_CQ_BUFFER_OFFSET,
256 UVERBS_ATTR_TYPE(u64),
257 UA_OPTIONAL),
183 UVERBS_ATTR_UHW());258 UVERBS_ATTR_UHW());
184 259
185static int UVERBS_HANDLER(UVERBS_METHOD_CQ_DESTROY)(260static int UVERBS_HANDLER(UVERBS_METHOD_CQ_DESTROY)(
drivers/infiniband/core/uverbs_std_types_dmah.c+145−0
@@ -0,0 +1,145 @@
1// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
2/*
3 * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
4 */
5
6#include "rdma_core.h"
7#include "uverbs.h"
8#include <rdma/uverbs_std_types.h>
9#include "restrack.h"
10
11static int uverbs_free_dmah(struct ib_uobject *uobject,
12 enum rdma_remove_reason why,
13 struct uverbs_attr_bundle *attrs)
14{
15 struct ib_dmah *dmah = uobject->object;
16 int ret;
17
18 if (atomic_read(&dmah->usecnt))
19 return -EBUSY;
20
21 ret = dmah->device->ops.dealloc_dmah(dmah, attrs);
22 if (ret)
23 return ret;
24
25 rdma_restrack_del(&dmah->res);
26 kfree(dmah);
27 return 0;
28}
29
30static int UVERBS_HANDLER(UVERBS_METHOD_DMAH_ALLOC)(
31 struct uverbs_attr_bundle *attrs)
32{
33 struct ib_uobject *uobj =
34 uverbs_attr_get(attrs, UVERBS_ATTR_ALLOC_DMAH_HANDLE)
35 ->obj_attr.uobject;
36 struct ib_device *ib_dev = attrs->context->device;
37 struct ib_dmah *dmah;
38 int ret;
39
40 dmah = rdma_zalloc_drv_obj(ib_dev, ib_dmah);
41 if (!dmah)
42 return -ENOMEM;
43
44 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_ALLOC_DMAH_CPU_ID)) {
45 ret = uverbs_copy_from(&dmah->cpu_id, attrs,
46 UVERBS_ATTR_ALLOC_DMAH_CPU_ID);
47 if (ret)
48 goto err;
49
50 if (!cpumask_test_cpu(dmah->cpu_id, current->cpus_ptr)) {
51 ret = -EPERM;
52 goto err;
53 }
54
55 dmah->valid_fields |= BIT(IB_DMAH_CPU_ID_EXISTS);
56 }
57
58 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_ALLOC_DMAH_TPH_MEM_TYPE)) {
59 dmah->mem_type = uverbs_attr_get_enum_id(attrs,
60 UVERBS_ATTR_ALLOC_DMAH_TPH_MEM_TYPE);
61 dmah->valid_fields |= BIT(IB_DMAH_MEM_TYPE_EXISTS);
62 }
63
64 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_ALLOC_DMAH_PH)) {
65 ret = uverbs_copy_from(&dmah->ph, attrs,
66 UVERBS_ATTR_ALLOC_DMAH_PH);
67 if (ret)
68 goto err;
69
70 /* Per PCIe spec 6.2-1.0, only the lowest two bits are applicable */
71 if (dmah->ph & 0xFC) {
72 ret = -EINVAL;
73 goto err;
74 }
75
76 dmah->valid_fields |= BIT(IB_DMAH_PH_EXISTS);
77 }
78
79 dmah->device = ib_dev;
80 dmah->uobject = uobj;
81 atomic_set(&dmah->usecnt, 0);
82
83 rdma_restrack_new(&dmah->res, RDMA_RESTRACK_DMAH);
84 rdma_restrack_set_name(&dmah->res, NULL);
85
86 ret = ib_dev->ops.alloc_dmah(dmah, attrs);
87 if (ret) {
88 rdma_restrack_put(&dmah->res);
89 goto err;
90 }
91
92 uobj->object = dmah;
93 rdma_restrack_add(&dmah->res);
94 uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_ALLOC_DMAH_HANDLE);
95 return 0;
96err:
97 kfree(dmah);
98 return ret;
99}
100
101static const struct uverbs_attr_spec uverbs_dmah_mem_type[] = {
102 [TPH_MEM_TYPE_VM] = {
103 .type = UVERBS_ATTR_TYPE_PTR_IN,
104 UVERBS_ATTR_NO_DATA(),
105 },
106 [TPH_MEM_TYPE_PM] = {
107 .type = UVERBS_ATTR_TYPE_PTR_IN,
108 UVERBS_ATTR_NO_DATA(),
109 },
110};
111
112DECLARE_UVERBS_NAMED_METHOD(
113 UVERBS_METHOD_DMAH_ALLOC,
114 UVERBS_ATTR_IDR(UVERBS_ATTR_ALLOC_DMAH_HANDLE,
115 UVERBS_OBJECT_DMAH,
116 UVERBS_ACCESS_NEW,
117 UA_MANDATORY),
118 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_DMAH_CPU_ID,
119 UVERBS_ATTR_TYPE(u32),
120 UA_OPTIONAL),
121 UVERBS_ATTR_ENUM_IN(UVERBS_ATTR_ALLOC_DMAH_TPH_MEM_TYPE,
122 uverbs_dmah_mem_type,
123 UA_OPTIONAL),
124 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_ALLOC_DMAH_PH,
125 UVERBS_ATTR_TYPE(u8),
126 UA_OPTIONAL));
127
128DECLARE_UVERBS_NAMED_METHOD_DESTROY(
129 UVERBS_METHOD_DMAH_FREE,
130 UVERBS_ATTR_IDR(UVERBS_ATTR_FREE_DMA_HANDLE,
131 UVERBS_OBJECT_DMAH,
132 UVERBS_ACCESS_DESTROY,
133 UA_MANDATORY));
134
135DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_DMAH,
136 UVERBS_TYPE_ALLOC_IDR(uverbs_free_dmah),
137 &UVERBS_METHOD(UVERBS_METHOD_DMAH_ALLOC),
138 &UVERBS_METHOD(UVERBS_METHOD_DMAH_FREE));
139
140const struct uapi_definition uverbs_def_obj_dmah[] = {
141 UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_DMAH,
142 UAPI_DEF_OBJ_NEEDS_FN(dealloc_dmah),
143 UAPI_DEF_OBJ_NEEDS_FN(alloc_dmah)),
144 {}
145};
drivers/infiniband/core/uverbs_std_types_mr.c+170−2
@@ -238,7 +238,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_DMABUF_MR)( · 237 unmodified lines
238 return ret;238 return ret;
239 239
240 mr = pd->device->ops.reg_user_mr_dmabuf(pd, offset, length, iova, fd,240 mr = pd->device->ops.reg_user_mr_dmabuf(pd, offset, length, iova, fd,
241 access_flags,241 access_flags, NULL,
242 attrs);242 attrs);
243 if (IS_ERR(mr))243 if (IS_ERR(mr))
244 return PTR_ERR(mr);244 return PTR_ERR(mr);
@@ -266,6 +266,135 @@ static int UVERBS_HANDLER(UVERBS_METHOD_REG_DMABUF_MR)( · 21 unmodified lines
266 return ret;266 return ret;
267}267}
268 268
269static int UVERBS_HANDLER(UVERBS_METHOD_REG_MR)(
270 struct uverbs_attr_bundle *attrs)
271{
272 struct ib_uobject *uobj =
273 uverbs_attr_get_uobject(attrs, UVERBS_ATTR_REG_MR_HANDLE);
274 struct ib_pd *pd =
275 uverbs_attr_get_obj(attrs, UVERBS_ATTR_REG_MR_PD_HANDLE);
276 u32 valid_access_flags = IB_ACCESS_SUPPORTED;
277 u64 length, iova, fd_offset = 0, addr = 0;
278 struct ib_device *ib_dev = pd->device;
279 struct ib_dmah *dmah = NULL;
280 bool has_fd_offset = false;
281 bool has_addr = false;
282 bool has_fd = false;
283 u32 access_flags;
284 struct ib_mr *mr;
285 int fd;
286 int ret;
287
288 ret = uverbs_copy_from(&iova, attrs, UVERBS_ATTR_REG_MR_IOVA);
289 if (ret)
290 return ret;
291
292 ret = uverbs_copy_from(&length, attrs, UVERBS_ATTR_REG_MR_LENGTH);
293 if (ret)
294 return ret;
295
296 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_ADDR)) {
297 ret = uverbs_copy_from(&addr, attrs,
298 UVERBS_ATTR_REG_MR_ADDR);
299 if (ret)
300 return ret;
301 has_addr = true;
302 }
303
304 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_FD_OFFSET)) {
305 ret = uverbs_copy_from(&fd_offset, attrs,
306 UVERBS_ATTR_REG_MR_FD_OFFSET);
307 if (ret)
308 return ret;
309 has_fd_offset = true;
310 }
311
312 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_FD)) {
313 ret = uverbs_get_raw_fd(&fd, attrs,
314 UVERBS_ATTR_REG_MR_FD);
315 if (ret)
316 return ret;
317 has_fd = true;
318 }
319
320 if (has_fd) {
321 if (!ib_dev->ops.reg_user_mr_dmabuf)
322 return -EOPNOTSUPP;
323
324 /* FD requires offset and can't come with addr */
325 if (!has_fd_offset || has_addr)
326 return -EINVAL;
327
328 if ((fd_offset & ~PAGE_MASK) != (iova & ~PAGE_MASK))
329 return -EINVAL;
330
331 valid_access_flags = IB_ACCESS_LOCAL_WRITE |
332 IB_ACCESS_REMOTE_READ |
333 IB_ACCESS_REMOTE_WRITE |
334 IB_ACCESS_REMOTE_ATOMIC |
335 IB_ACCESS_RELAXED_ORDERING;
336 } else {
337 if (!has_addr || has_fd_offset)
338 return -EINVAL;
339
340 if ((addr & ~PAGE_MASK) != (iova & ~PAGE_MASK))
341 return -EINVAL;
342 }
343
344 if (uverbs_attr_is_valid(attrs, UVERBS_ATTR_REG_MR_DMA_HANDLE)) {
345 dmah = uverbs_attr_get_obj(attrs,
346 UVERBS_ATTR_REG_MR_DMA_HANDLE);
347 if (IS_ERR(dmah))
348 return PTR_ERR(dmah);
349 }
350
351 ret = uverbs_get_flags32(&access_flags, attrs,
352 UVERBS_ATTR_REG_MR_ACCESS_FLAGS,
353 valid_access_flags);
354 if (ret)
355 return ret;
356
357 ret = ib_check_mr_access(ib_dev, access_flags);
358 if (ret)
359 return ret;
360
361 if (has_fd)
362 mr = pd->device->ops.reg_user_mr_dmabuf(pd, fd_offset, length,
363 iova, fd, access_flags,
364 dmah, attrs);
365 else
366 mr = pd->device->ops.reg_user_mr(pd, addr, length, iova,
367 access_flags, dmah, NULL);
368
369 if (IS_ERR(mr))
370 return PTR_ERR(mr);
371
372 mr->device = pd->device;
373 mr->pd = pd;
374 mr->type = IB_MR_TYPE_USER;
375 mr->uobject = uobj;
376 atomic_inc(&pd->usecnt);
377 if (dmah) {
378 mr->dmah = dmah;
379 atomic_inc(&dmah->usecnt);
380 }
381 rdma_restrack_new(&mr->res, RDMA_RESTRACK_MR);
382 rdma_restrack_set_name(&mr->res, NULL);
383 rdma_restrack_add(&mr->res);
384 uobj->object = mr;
385
386 uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_REG_MR_HANDLE);
387
388 ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_LKEY,
389 &mr->lkey, sizeof(mr->lkey));
390 if (ret)
391 return ret;
392
393 ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_MR_RESP_RKEY,
394 &mr->rkey, sizeof(mr->rkey));
395 return ret;
396}
397
269DECLARE_UVERBS_NAMED_METHOD(398DECLARE_UVERBS_NAMED_METHOD(
270 UVERBS_METHOD_ADVISE_MR,399 UVERBS_METHOD_ADVISE_MR,
271 UVERBS_ATTR_IDR(UVERBS_ATTR_ADVISE_MR_PD_HANDLE,400 UVERBS_ATTR_IDR(UVERBS_ATTR_ADVISE_MR_PD_HANDLE,
@@ -362,6 +491,44 @@ DECLARE_UVERBS_NAMED_METHOD( · 90 unmodified lines
362 UVERBS_ATTR_TYPE(u32),491 UVERBS_ATTR_TYPE(u32),
363 UA_MANDATORY));492 UA_MANDATORY));
364 493
494DECLARE_UVERBS_NAMED_METHOD(
495 UVERBS_METHOD_REG_MR,
496 UVERBS_ATTR_IDR(UVERBS_ATTR_REG_MR_HANDLE,
497 UVERBS_OBJECT_MR,
498 UVERBS_ACCESS_NEW,
499 UA_MANDATORY),
500 UVERBS_ATTR_IDR(UVERBS_ATTR_REG_MR_PD_HANDLE,
501 UVERBS_OBJECT_PD,
502 UVERBS_ACCESS_READ,
503 UA_MANDATORY),
504 UVERBS_ATTR_IDR(UVERBS_ATTR_REG_MR_DMA_HANDLE,
505 UVERBS_OBJECT_DMAH,
506 UVERBS_ACCESS_READ,
507 UA_OPTIONAL),
508 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_IOVA,
509 UVERBS_ATTR_TYPE(u64),
510 UA_MANDATORY),
511 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_LENGTH,
512 UVERBS_ATTR_TYPE(u64),
513 UA_MANDATORY),
514 UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_REG_MR_ACCESS_FLAGS,
515 enum ib_access_flags,
516 UA_MANDATORY),
517 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_ADDR,
518 UVERBS_ATTR_TYPE(u64),
519 UA_OPTIONAL),
520 UVERBS_ATTR_PTR_IN(UVERBS_ATTR_REG_MR_FD_OFFSET,
521 UVERBS_ATTR_TYPE(u64),
522 UA_OPTIONAL),
523 UVERBS_ATTR_RAW_FD(UVERBS_ATTR_REG_MR_FD,
524 UA_OPTIONAL),
525 UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_LKEY,
526 UVERBS_ATTR_TYPE(u32),
527 UA_MANDATORY),
528 UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_REG_MR_RESP_RKEY,
529 UVERBS_ATTR_TYPE(u32),
530 UA_MANDATORY));
531
365DECLARE_UVERBS_NAMED_METHOD_DESTROY(532DECLARE_UVERBS_NAMED_METHOD_DESTROY(
366 UVERBS_METHOD_MR_DESTROY,533 UVERBS_METHOD_MR_DESTROY,
367 UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_MR_HANDLE,534 UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_MR_HANDLE,
@@ -376,7 +543,8 @@ DECLARE_UVERBS_NAMED_OBJECT( · 8 unmodified lines
376 &UVERBS_METHOD(UVERBS_METHOD_DM_MR_REG),543 &UVERBS_METHOD(UVERBS_METHOD_DM_MR_REG),
377 &UVERBS_METHOD(UVERBS_METHOD_MR_DESTROY),544 &UVERBS_METHOD(UVERBS_METHOD_MR_DESTROY),
378 &UVERBS_METHOD(UVERBS_METHOD_QUERY_MR),545 &UVERBS_METHOD(UVERBS_METHOD_QUERY_MR),
379 &UVERBS_METHOD(UVERBS_METHOD_REG_DMABUF_MR));546 &UVERBS_METHOD(UVERBS_METHOD_REG_DMABUF_MR),
547 &UVERBS_METHOD(UVERBS_METHOD_REG_MR));
380 548
381const struct uapi_definition uverbs_def_obj_mr[] = {549const struct uapi_definition uverbs_def_obj_mr[] = {
382 UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_MR,550 UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_MR,
drivers/infiniband/core/uverbs_std_types_qp.c+1−1
@@ -133,7 +133,7 @@ static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)( · 132 unmodified lines
133 device = xrcd->device;133 device = xrcd->device;
134 break;134 break;
135 case IB_UVERBS_QPT_RAW_PACKET:135 case IB_UVERBS_QPT_RAW_PACKET:
136 if (!capable(CAP_NET_RAW))136 if (!rdma_uattrs_has_raw_cap(attrs))
137 return -EPERM;137 return -EPERM;
138 fallthrough;138 fallthrough;
139 case IB_UVERBS_QPT_RC:139 case IB_UVERBS_QPT_RC:
drivers/infiniband/core/uverbs_uapi.c+1−0
@@ -631,6 +631,7 @@ static const struct uapi_definition uverbs_core_api[] = { · 630 unmodified lines
631 UAPI_DEF_CHAIN(uverbs_def_obj_cq),631 UAPI_DEF_CHAIN(uverbs_def_obj_cq),
632 UAPI_DEF_CHAIN(uverbs_def_obj_device),632 UAPI_DEF_CHAIN(uverbs_def_obj_device),
633 UAPI_DEF_CHAIN(uverbs_def_obj_dm),633 UAPI_DEF_CHAIN(uverbs_def_obj_dm),
634 UAPI_DEF_CHAIN(uverbs_def_obj_dmah),
634 UAPI_DEF_CHAIN(uverbs_def_obj_flow_action),635 UAPI_DEF_CHAIN(uverbs_def_obj_flow_action),
635 UAPI_DEF_CHAIN(uverbs_def_obj_intf),636 UAPI_DEF_CHAIN(uverbs_def_obj_intf),
636 UAPI_DEF_CHAIN(uverbs_def_obj_mr),637 UAPI_DEF_CHAIN(uverbs_def_obj_mr),
drivers/infiniband/core/verbs.c+4−1
@@ -2223,7 +2223,7 @@ struct ib_mr *ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, · 2222 unmodified lines
2223 }2223 }
2224 2224
2225 mr = pd->device->ops.reg_user_mr(pd, start, length, virt_addr,2225 mr = pd->device->ops.reg_user_mr(pd, start, length, virt_addr,
2226 access_flags, NULL);2226 access_flags, NULL, NULL);
2227 2227
2228 if (IS_ERR(mr))2228 if (IS_ERR(mr))
2229 return mr;2229 return mr;
@@ -2262,6 +2262,7 @@ int ib_dereg_mr_user(struct ib_mr *mr, struct ib_udata *udata) · 32 unmodified lines
2262{2262{
2263 struct ib_pd *pd = mr->pd;2263 struct ib_pd *pd = mr->pd;
2264 struct ib_dm *dm = mr->dm;2264 struct ib_dm *dm = mr->dm;
2265 struct ib_dmah *dmah = mr->dmah;
2265 struct ib_sig_attrs *sig_attrs = mr->sig_attrs;2266 struct ib_sig_attrs *sig_attrs = mr->sig_attrs;
2266 int ret;2267 int ret;
2267 2268
@@ -2272,6 +2273,8 @@ int ib_dereg_mr_user(struct ib_mr *mr, struct ib_udata *udata) · 4 unmodified lines
2272 atomic_dec(&pd->usecnt);2273 atomic_dec(&pd->usecnt);
2273 if (dm)2274 if (dm)
2274 atomic_dec(&dm->usecnt);2275 atomic_dec(&dm->usecnt);
2276 if (dmah)
2277 atomic_dec(&dmah->usecnt);
2275 kfree(sig_attrs);2278 kfree(sig_attrs);
2276 }2279 }
2277 2280
drivers/infiniband/hw/Makefile+0−1
@@ -1,6 +1,5 @@
1# SPDX-License-Identifier: GPL-2.01# SPDX-License-Identifier: GPL-2.0
2obj-$(CONFIG_INFINIBAND_MTHCA) += mthca/2obj-$(CONFIG_INFINIBAND_MTHCA) += mthca/
3obj-$(CONFIG_INFINIBAND_QIB) += qib/
4obj-$(CONFIG_INFINIBAND_CXGB4) += cxgb4/3obj-$(CONFIG_INFINIBAND_CXGB4) += cxgb4/
5obj-$(CONFIG_INFINIBAND_EFA) += efa/4obj-$(CONFIG_INFINIBAND_EFA) += efa/
6obj-$(CONFIG_INFINIBAND_IRDMA) += irdma/5obj-$(CONFIG_INFINIBAND_IRDMA) += irdma/
drivers/infiniband/hw/bnxt_re/ib_verbs.c+9−1
@@ -4235,6 +4235,7 @@ free_mr: · 4234 unmodified lines
4235 4235
4236struct ib_mr *bnxt_re_reg_user_mr(struct ib_pd *ib_pd, u64 start, u64 length,4236struct ib_mr *bnxt_re_reg_user_mr(struct ib_pd *ib_pd, u64 start, u64 length,
4237 u64 virt_addr, int mr_access_flags,4237 u64 virt_addr, int mr_access_flags,
4238 struct ib_dmah *dmah,
4238 struct ib_udata *udata)4239 struct ib_udata *udata)
4239{4240{
4240 struct bnxt_re_pd *pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd);4241 struct bnxt_re_pd *pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd);
@@ -4242,6 +4243,9 @@ struct ib_mr *bnxt_re_reg_user_mr(struct ib_pd *ib_pd, u64 start, u64 length, · 1 unmodified line
4242 struct ib_umem *umem;4243 struct ib_umem *umem;
4243 struct ib_mr *ib_mr;4244 struct ib_mr *ib_mr;
4244 4245
4246 if (dmah)
4247 return ERR_PTR(-EOPNOTSUPP);
4248
4245 umem = ib_umem_get(&rdev->ibdev, start, length, mr_access_flags);4249 umem = ib_umem_get(&rdev->ibdev, start, length, mr_access_flags);
4246 if (IS_ERR(umem))4250 if (IS_ERR(umem))
4247 return ERR_CAST(umem);4251 return ERR_CAST(umem);
@@ -4255,6 +4259,7 @@ struct ib_mr *bnxt_re_reg_user_mr(struct ib_pd *ib_pd, u64 start, u64 length, · 7 unmodified lines
4255struct ib_mr *bnxt_re_reg_user_mr_dmabuf(struct ib_pd *ib_pd, u64 start,4259struct ib_mr *bnxt_re_reg_user_mr_dmabuf(struct ib_pd *ib_pd, u64 start,
4256 u64 length, u64 virt_addr, int fd,4260 u64 length, u64 virt_addr, int fd,
4257 int mr_access_flags,4261 int mr_access_flags,
4262 struct ib_dmah *dmah,
4258 struct uverbs_attr_bundle *attrs)4263 struct uverbs_attr_bundle *attrs)
4259{4264{
4260 struct bnxt_re_pd *pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd);4265 struct bnxt_re_pd *pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd);
@@ -4263,6 +4268,9 @@ struct ib_mr *bnxt_re_reg_user_mr_dmabuf(struct ib_pd *ib_pd, u64 start, · 2 unmodified lines
4263 struct ib_umem *umem;4268 struct ib_umem *umem;
4264 struct ib_mr *ib_mr;4269 struct ib_mr *ib_mr;
4265 4270
4271 if (dmah)
4272 return ERR_PTR(-EOPNOTSUPP);
4273
4266 umem_dmabuf = ib_umem_dmabuf_get_pinned(&rdev->ibdev, start, length,4274 umem_dmabuf = ib_umem_dmabuf_get_pinned(&rdev->ibdev, start, length,
4267 fd, mr_access_flags);4275 fd, mr_access_flags);
4268 if (IS_ERR(umem_dmabuf))4276 if (IS_ERR(umem_dmabuf))
@@ -4738,7 +4746,7 @@ static int UVERBS_HANDLER(BNXT_RE_METHOD_GET_TOGGLE_MEM)(struct uverbs_attr_bund · 469 unmodified lines
4738 return err;4746 return err;
4739 4747
4740 err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_OFFSET,4748 err = uverbs_copy_to(attrs, BNXT_RE_TOGGLE_MEM_MMAP_OFFSET,
4741 &offset, sizeof(length));4749 &offset, sizeof(offset));
4742 if (err)4750 if (err)
4743 return err;4751 return err;
4744 4752
drivers/infiniband/hw/bnxt_re/ib_verbs.h+2−0
@@ -258,10 +258,12 @@ struct ib_mw *bnxt_re_alloc_mw(struct ib_pd *ib_pd, enum ib_mw_type type, · 257 unmodified lines
258int bnxt_re_dealloc_mw(struct ib_mw *mw);258int bnxt_re_dealloc_mw(struct ib_mw *mw);
259struct ib_mr *bnxt_re_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,259struct ib_mr *bnxt_re_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
260 u64 virt_addr, int mr_access_flags,260 u64 virt_addr, int mr_access_flags,
261 struct ib_dmah *dmah,
261 struct ib_udata *udata);262 struct ib_udata *udata);
262struct ib_mr *bnxt_re_reg_user_mr_dmabuf(struct ib_pd *ib_pd, u64 start,263struct ib_mr *bnxt_re_reg_user_mr_dmabuf(struct ib_pd *ib_pd, u64 start,
263 u64 length, u64 virt_addr,264 u64 length, u64 virt_addr,
264 int fd, int mr_access_flags,265 int fd, int mr_access_flags,
266 struct ib_dmah *dmah,
265 struct uverbs_attr_bundle *attrs);267 struct uverbs_attr_bundle *attrs);
266int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata);268int bnxt_re_alloc_ucontext(struct ib_ucontext *ctx, struct ib_udata *udata);
267void bnxt_re_dealloc_ucontext(struct ib_ucontext *context);269void bnxt_re_dealloc_ucontext(struct ib_ucontext *context);
drivers/infiniband/hw/bnxt_re/qplib_fp.c+14−14
@@ -1750,9 +1750,9 @@ static void bnxt_qplib_fill_psn_search(struct bnxt_qplib_qp *qp, · 1749 unmodified lines
1750 }1750 }
1751}1751}
1752 1752
1753static int bnxt_qplib_put_inline(struct bnxt_qplib_qp *qp,1753static unsigned int bnxt_qplib_put_inline(struct bnxt_qplib_qp *qp,
1754 struct bnxt_qplib_swqe *wqe,1754 struct bnxt_qplib_swqe *wqe,
1755 u16 *idx)1755 u32 *idx)
1756{1756{
1757 struct bnxt_qplib_hwq *hwq;1757 struct bnxt_qplib_hwq *hwq;
1758 int len, t_len, offt;1758 int len, t_len, offt;
@@ -1769,7 +1769,7 @@ static int bnxt_qplib_put_inline(struct bnxt_qplib_qp *qp, · 10 unmodified lines
1769 il_src = (void *)wqe->sg_list[indx].addr;1769 il_src = (void *)wqe->sg_list[indx].addr;
1770 t_len += len;1770 t_len += len;
1771 if (t_len > qp->max_inline_data)1771 if (t_len > qp->max_inline_data)
1772 return -ENOMEM;1772 return BNXT_RE_INVAL_MSG_SIZE;
1773 while (len) {1773 while (len) {
1774 if (pull_dst) {1774 if (pull_dst) {
1775 pull_dst = false;1775 pull_dst = false;
@@ -1795,9 +1795,9 @@ static int bnxt_qplib_put_inline(struct bnxt_qplib_qp *qp, · 19 unmodified lines
1795 return t_len;1795 return t_len;
1796}1796}
1797 1797
1798static u32 bnxt_qplib_put_sges(struct bnxt_qplib_hwq *hwq,1798static unsigned int bnxt_qplib_put_sges(struct bnxt_qplib_hwq *hwq,
1799 struct bnxt_qplib_sge *ssge,1799 struct bnxt_qplib_sge *ssge,
1800 u16 nsge, u16 *idx)1800 u32 nsge, u32 *idx)
1801{1801{
1802 struct sq_sge *dsge;1802 struct sq_sge *dsge;
1803 int indx, len = 0;1803 int indx, len = 0;
@@ -1878,14 +1878,12 @@ int bnxt_qplib_post_send(struct bnxt_qplib_qp *qp, · 74 unmodified lines
1878 struct bnxt_qplib_hwq *hwq;1878 struct bnxt_qplib_hwq *hwq;
1879 struct bnxt_qplib_swq *swq;1879 struct bnxt_qplib_swq *swq;
1880 bool sch_handler = false;1880 bool sch_handler = false;
1881 u32 wqe_idx, slots, idx;
1881 u16 wqe_sz, qdf = 0;1882 u16 wqe_sz, qdf = 0;
1882 bool msn_update;1883 bool msn_update;
1883 void *base_hdr;1884 void *base_hdr;
1884 void *ext_hdr;1885 void *ext_hdr;
1885 __le32 temp32;1886 __le32 temp32;
1886 u32 wqe_idx;
1887 u32 slots;
1888 u16 idx;
1889 1887
1890 hwq = &sq->hwq;1888 hwq = &sq->hwq;
1891 if (qp->state != CMDQ_MODIFY_QP_NEW_STATE_RTS &&1889 if (qp->state != CMDQ_MODIFY_QP_NEW_STATE_RTS &&
@@ -1937,8 +1935,10 @@ int bnxt_qplib_post_send(struct bnxt_qplib_qp *qp, · 45 unmodified lines
1937 else1935 else
1938 data_len = bnxt_qplib_put_sges(hwq, wqe->sg_list, wqe->num_sge,1936 data_len = bnxt_qplib_put_sges(hwq, wqe->sg_list, wqe->num_sge,
1939 &idx);1937 &idx);
1940 if (data_len < 0)1938 if (data_len > BNXT_RE_MAX_MSG_SIZE) {
1941 goto queue_err;1939 rc = -EINVAL;
1940 goto done;
1941 }
1942 /* Make sure we update MSN table only for wired wqes */1942 /* Make sure we update MSN table only for wired wqes */
1943 msn_update = true;1943 msn_update = true;
1944 /* Specifics */1944 /* Specifics */
@@ -2139,8 +2139,8 @@ int bnxt_qplib_post_recv(struct bnxt_qplib_qp *qp, · 194 unmodified lines
2139 struct bnxt_qplib_hwq *hwq;2139 struct bnxt_qplib_hwq *hwq;
2140 struct bnxt_qplib_swq *swq;2140 struct bnxt_qplib_swq *swq;
2141 bool sch_handler = false;2141 bool sch_handler = false;
2142 u16 wqe_sz, idx;2142 u32 wqe_idx, idx;
2143 u32 wqe_idx;2143 u16 wqe_sz;
2144 int rc = 0;2144 int rc = 0;
2145 2145
2146 hwq = &rq->hwq;2146 hwq = &rq->hwq;
drivers/infiniband/hw/bnxt_re/qplib_fp.h+3−0
@@ -346,6 +346,9 @@ struct bnxt_qplib_qp { · 345 unmodified lines
346 u8 tos_dscp;346 u8 tos_dscp;
347};347};
348 348
349#define BNXT_RE_MAX_MSG_SIZE 0x80000000
350#define BNXT_RE_INVAL_MSG_SIZE 0xFFFFFFFF
351
349#define BNXT_QPLIB_MAX_CQE_ENTRY_SIZE sizeof(struct cq_base)352#define BNXT_QPLIB_MAX_CQE_ENTRY_SIZE sizeof(struct cq_base)
350 353
351#define CQE_CNT_PER_PG (PAGE_SIZE / BNXT_QPLIB_MAX_CQE_ENTRY_SIZE)354#define CQE_CNT_PER_PG (PAGE_SIZE / BNXT_QPLIB_MAX_CQE_ENTRY_SIZE)
drivers/infiniband/hw/bnxt_re/qplib_sp.c+1−1
@@ -674,7 +674,7 @@ int bnxt_qplib_reg_mr(struct bnxt_qplib_res *res, struct bnxt_qplib_mrw *mr, · 673 unmodified lines
674 req.log2_pbl_pg_size = cpu_to_le16(((ilog2(PAGE_SIZE) <<674 req.log2_pbl_pg_size = cpu_to_le16(((ilog2(PAGE_SIZE) <<
675 CMDQ_REGISTER_MR_LOG2_PBL_PG_SIZE_SFT) &675 CMDQ_REGISTER_MR_LOG2_PBL_PG_SIZE_SFT) &
676 CMDQ_REGISTER_MR_LOG2_PBL_PG_SIZE_MASK));676 CMDQ_REGISTER_MR_LOG2_PBL_PG_SIZE_MASK));
677 req.access = (mr->access_flags & 0xFFFF);677 req.access = (mr->access_flags & BNXT_QPLIB_MR_ACCESS_MASK);
678 req.va = cpu_to_le64(mr->va);678 req.va = cpu_to_le64(mr->va);
679 req.key = cpu_to_le32(mr->lkey);679 req.key = cpu_to_le32(mr->lkey);
680 if (_is_alloc_mr_unified(res->dattr->dev_cap_flags))680 if (_is_alloc_mr_unified(res->dattr->dev_cap_flags))
drivers/infiniband/hw/bnxt_re/qplib_sp.h+1−0
@@ -111,6 +111,7 @@ struct bnxt_qplib_mrw { · 110 unmodified lines
111 struct bnxt_qplib_pd *pd;111 struct bnxt_qplib_pd *pd;
112 int type;112 int type;
113 u32 access_flags;113 u32 access_flags;
114#define BNXT_QPLIB_MR_ACCESS_MASK 0xFF
114#define BNXT_QPLIB_FR_PMR 0x80000000115#define BNXT_QPLIB_FR_PMR 0x80000000
115 u32 lkey;116 u32 lkey;
116 u32 rkey;117 u32 rkey;
drivers/infiniband/hw/cxgb4/device.c+1−2
@@ -905,8 +905,7 @@ static int c4iw_rdev_open(struct c4iw_rdev *rdev) · 904 unmodified lines
905 905
906 return 0;906 return 0;
907err_free_status_page_and_wr_log:907err_free_status_page_and_wr_log:
908 if (c4iw_wr_log && rdev->wr_log)908 kfree(rdev->wr_log);
909 kfree(rdev->wr_log);
910 free_page((unsigned long)rdev->status_page);909 free_page((unsigned long)rdev->status_page);
911destroy_ocqp_pool:910destroy_ocqp_pool:
912 c4iw_ocqp_pool_destroy(rdev);911 c4iw_ocqp_pool_destroy(rdev);
drivers/infiniband/hw/cxgb4/iw_cxgb4.h+1−0
@@ -1006,6 +1006,7 @@ int c4iw_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents, · 1005 unmodified lines
1006void c4iw_dealloc(struct uld_ctx *ctx);1006void c4iw_dealloc(struct uld_ctx *ctx);
1007struct ib_mr *c4iw_reg_user_mr(struct ib_pd *pd, u64 start,1007struct ib_mr *c4iw_reg_user_mr(struct ib_pd *pd, u64 start,
1008 u64 length, u64 virt, int acc,1008 u64 length, u64 virt, int acc,
1009 struct ib_dmah *dmah,
1009 struct ib_udata *udata);1010 struct ib_udata *udata);
1010struct ib_mr *c4iw_get_dma_mr(struct ib_pd *pd, int acc);1011struct ib_mr *c4iw_get_dma_mr(struct ib_pd *pd, int acc);
1011int c4iw_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata);1012int c4iw_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata);
drivers/infiniband/hw/cxgb4/mem.c+5−1
@@ -489,7 +489,8 @@ err_free_mhp: · 488 unmodified lines
489}489}
490 490
491struct ib_mr *c4iw_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,491struct ib_mr *c4iw_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
492 u64 virt, int acc, struct ib_udata *udata)492 u64 virt, int acc, struct ib_dmah *dmah,
493 struct ib_udata *udata)
493{494{
494 __be64 *pages;495 __be64 *pages;
495 int shift, n, i;496 int shift, n, i;
@@ -501,6 +502,9 @@ struct ib_mr *c4iw_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, · 5 unmodified lines
501 502
502 pr_debug("ib_pd %p\n", pd);503 pr_debug("ib_pd %p\n", pd);
503 504
505 if (dmah)
506 return ERR_PTR(-EOPNOTSUPP);
507
504 if (length == ~0ULL)508 if (length == ~0ULL)
505 return ERR_PTR(-EINVAL);509 return ERR_PTR(-EINVAL);
506 510
drivers/infiniband/hw/efa/efa.h+5−0
@@ -107,6 +107,7 @@ struct efa_cq { · 106 unmodified lines
107 u16 cq_idx;107 u16 cq_idx;
108 /* NULL when no interrupts requested */108 /* NULL when no interrupts requested */
109 struct efa_eq *eq;109 struct efa_eq *eq;
110 struct ib_umem *umem;
110};111};
111 112
112struct efa_qp {113struct efa_qp {
@@ -162,12 +163,16 @@ int efa_create_qp(struct ib_qp *ibqp, struct ib_qp_init_attr *init_attr, · 49 unmodified lines
162int efa_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata);163int efa_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata);
163int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,164int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
164 struct uverbs_attr_bundle *attrs);165 struct uverbs_attr_bundle *attrs);
166int efa_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
167 struct ib_umem *umem, struct uverbs_attr_bundle *attrs);
165struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,168struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
166 u64 virt_addr, int access_flags,169 u64 virt_addr, int access_flags,
170 struct ib_dmah *dmah,
167 struct ib_udata *udata);171 struct ib_udata *udata);
168struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,172struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
169 u64 length, u64 virt_addr,173 u64 length, u64 virt_addr,
170 int fd, int access_flags,174 int fd, int access_flags,
175 struct ib_dmah *dmah,
171 struct uverbs_attr_bundle *attrs);176 struct uverbs_attr_bundle *attrs);
172int efa_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);177int efa_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
173int efa_get_port_immutable(struct ib_device *ibdev, u32 port_num,178int efa_get_port_immutable(struct ib_device *ibdev, u32 port_num,
drivers/infiniband/hw/efa/efa_admin_cmds_defs.h+16−1
@@ -1,6 +1,6 @@
1/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */1/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
2/*2/*
3 * Copyright 2018-2024 Amazon.com, Inc. or its affiliates. All rights reserved.3 * Copyright 2018-2025 Amazon.com, Inc. or its affiliates. All rights reserved.
4 */4 */
5 5
6#ifndef _EFA_ADMIN_CMDS_H_6#ifndef _EFA_ADMIN_CMDS_H_
@@ -68,6 +68,7 @@ enum efa_admin_get_stats_type { · 61 unmodified lines
68 EFA_ADMIN_GET_STATS_TYPE_MESSAGES = 1,68 EFA_ADMIN_GET_STATS_TYPE_MESSAGES = 1,
69 EFA_ADMIN_GET_STATS_TYPE_RDMA_READ = 2,69 EFA_ADMIN_GET_STATS_TYPE_RDMA_READ = 2,
70 EFA_ADMIN_GET_STATS_TYPE_RDMA_WRITE = 3,70 EFA_ADMIN_GET_STATS_TYPE_RDMA_WRITE = 3,
71 EFA_ADMIN_GET_STATS_TYPE_NETWORK = 4,
71};72};
72 73
73enum efa_admin_get_stats_scope {74enum efa_admin_get_stats_scope {
@@ -651,6 +652,18 @@ struct efa_admin_rdma_write_stats { · 577 unmodified lines
651 u64 write_recv_bytes;652 u64 write_recv_bytes;
652};653};
653 654
655struct efa_admin_network_stats {
656 u64 retrans_bytes;
657
658 u64 retrans_pkts;
659
660 u64 retrans_timeout_events;
661
662 u64 unresponsive_remote_events;
663
664 u64 impaired_remote_conn_events;
665};
666
654struct efa_admin_acq_get_stats_resp {667struct efa_admin_acq_get_stats_resp {
655 struct efa_admin_acq_common_desc acq_common_desc;668 struct efa_admin_acq_common_desc acq_common_desc;
656 669
@@ -662,6 +675,8 @@ struct efa_admin_acq_get_stats_resp { · 5 unmodified lines
662 struct efa_admin_rdma_read_stats rdma_read_stats;675 struct efa_admin_rdma_read_stats rdma_read_stats;
663 676
664 struct efa_admin_rdma_write_stats rdma_write_stats;677 struct efa_admin_rdma_write_stats rdma_write_stats;
678
679 struct efa_admin_network_stats network_stats;
665 } u;680 } u;
666};681};
667 682
drivers/infiniband/hw/efa/efa_com_cmd.c+35−18
@@ -1,6 +1,6 @@
1// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause1// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
2/*2/*
3 * Copyright 2018-2024 Amazon.com, Inc. or its affiliates. All rights reserved.3 * Copyright 2018-2025 Amazon.com, Inc. or its affiliates. All rights reserved.
4 */4 */
5 5
6#include "efa_com.h"6#include "efa_com.h"
@@ -769,6 +769,11 @@ int efa_com_get_stats(struct efa_com_dev *edev, · 762 unmodified lines
769 struct efa_com_admin_queue *aq = &edev->aq;769 struct efa_com_admin_queue *aq = &edev->aq;
770 struct efa_admin_aq_get_stats_cmd cmd = {};770 struct efa_admin_aq_get_stats_cmd cmd = {};
771 struct efa_admin_acq_get_stats_resp resp;771 struct efa_admin_acq_get_stats_resp resp;
772 struct efa_admin_rdma_write_stats *rws;
773 struct efa_admin_rdma_read_stats *rrs;
774 struct efa_admin_messages_stats *ms;
775 struct efa_admin_network_stats *ns;
776 struct efa_admin_basic_stats *bs;
772 int err;777 int err;
773 778
774 cmd.aq_common_descriptor.opcode = EFA_ADMIN_GET_STATS;779 cmd.aq_common_descriptor.opcode = EFA_ADMIN_GET_STATS;
@@ -791,29 +796,41 @@ int efa_com_get_stats(struct efa_com_dev *edev, · 16 unmodified lines
791 796
792 switch (cmd.type) {797 switch (cmd.type) {
793 case EFA_ADMIN_GET_STATS_TYPE_BASIC:798 case EFA_ADMIN_GET_STATS_TYPE_BASIC:
794 result->basic_stats.tx_bytes = resp.u.basic_stats.tx_bytes;799 bs = &resp.u.basic_stats;
795 result->basic_stats.tx_pkts = resp.u.basic_stats.tx_pkts;800 result->basic_stats.tx_bytes = bs->tx_bytes;
796 result->basic_stats.rx_bytes = resp.u.basic_stats.rx_bytes;801 result->basic_stats.tx_pkts = bs->tx_pkts;
797 result->basic_stats.rx_pkts = resp.u.basic_stats.rx_pkts;802 result->basic_stats.rx_bytes = bs->rx_bytes;
798 result->basic_stats.rx_drops = resp.u.basic_stats.rx_drops;803 result->basic_stats.rx_pkts = bs->rx_pkts;
804 result->basic_stats.rx_drops = bs->rx_drops;
799 break;805 break;
800 case EFA_ADMIN_GET_STATS_TYPE_MESSAGES:806 case EFA_ADMIN_GET_STATS_TYPE_MESSAGES:
801 result->messages_stats.send_bytes = resp.u.messages_stats.send_bytes;807 ms = &resp.u.messages_stats;
802 result->messages_stats.send_wrs = resp.u.messages_stats.send_wrs;808 result->messages_stats.send_bytes = ms->send_bytes;
803 result->messages_stats.recv_bytes = resp.u.messages_stats.recv_bytes;809 result->messages_stats.send_wrs = ms->send_wrs;
804 result->messages_stats.recv_wrs = resp.u.messages_stats.recv_wrs;810 result->messages_stats.recv_bytes = ms->recv_bytes;
811 result->messages_stats.recv_wrs = ms->recv_wrs;
805 break;812 break;
806 case EFA_ADMIN_GET_STATS_TYPE_RDMA_READ:813 case EFA_ADMIN_GET_STATS_TYPE_RDMA_READ:
807 result->rdma_read_stats.read_wrs = resp.u.rdma_read_stats.read_wrs;814 rrs = &resp.u.rdma_read_stats;
808 result->rdma_read_stats.read_bytes = resp.u.rdma_read_stats.read_bytes;815 result->rdma_read_stats.read_wrs = rrs->read_wrs;
809 result->rdma_read_stats.read_wr_err = resp.u.rdma_read_stats.read_wr_err;816 result->rdma_read_stats.read_bytes = rrs->read_bytes;
810 result->rdma_read_stats.read_resp_bytes = resp.u.rdma_read_stats.read_resp_bytes;817 result->rdma_read_stats.read_wr_err = rrs->read_wr_err;
818 result->rdma_read_stats.read_resp_bytes = rrs->read_resp_bytes;
811 break;819 break;
812 case EFA_ADMIN_GET_STATS_TYPE_RDMA_WRITE:820 case EFA_ADMIN_GET_STATS_TYPE_RDMA_WRITE:
813 result->rdma_write_stats.write_wrs = resp.u.rdma_write_stats.write_wrs;821 rws = &resp.u.rdma_write_stats;
814 result->rdma_write_stats.write_bytes = resp.u.rdma_write_stats.write_bytes;822 result->rdma_write_stats.write_wrs = rws->write_wrs;
815 result->rdma_write_stats.write_wr_err = resp.u.rdma_write_stats.write_wr_err;823 result->rdma_write_stats.write_bytes = rws->write_bytes;
816 result->rdma_write_stats.write_recv_bytes = resp.u.rdma_write_stats.write_recv_bytes;824 result->rdma_write_stats.write_wr_err = rws->write_wr_err;
825 result->rdma_write_stats.write_recv_bytes = rws->write_recv_bytes;
826 break;
827 case EFA_ADMIN_GET_STATS_TYPE_NETWORK:
828 ns = &resp.u.network_stats;
829 result->network_stats.retrans_bytes = ns->retrans_bytes;
830 result->network_stats.retrans_pkts = ns->retrans_pkts;
831 result->network_stats.retrans_timeout_events = ns->retrans_timeout_events;
832 result->network_stats.unresponsive_remote_events = ns->unresponsive_remote_events;
833 result->network_stats.impaired_remote_conn_events = ns->impaired_remote_conn_events;
817 break;834 break;
818 }835 }
819 836
drivers/infiniband/hw/efa/efa_com_cmd.h+10−1
@@ -1,6 +1,6 @@
1/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */1/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
2/*2/*
3 * Copyright 2018-2024 Amazon.com, Inc. or its affiliates. All rights reserved.3 * Copyright 2018-2025 Amazon.com, Inc. or its affiliates. All rights reserved.
4 */4 */
5 5
6#ifndef _EFA_COM_CMD_H_6#ifndef _EFA_COM_CMD_H_
@@ -283,11 +283,20 @@ struct efa_com_rdma_write_stats { · 276 unmodified lines
283 u64 write_recv_bytes;283 u64 write_recv_bytes;
284};284};
285 285
286struct efa_com_network_stats {
287 u64 retrans_bytes;
288 u64 retrans_pkts;
289 u64 retrans_timeout_events;
290 u64 unresponsive_remote_events;
291 u64 impaired_remote_conn_events;
292};
293
286union efa_com_get_stats_result {294union efa_com_get_stats_result {
287 struct efa_com_basic_stats basic_stats;295 struct efa_com_basic_stats basic_stats;
288 struct efa_com_messages_stats messages_stats;296 struct efa_com_messages_stats messages_stats;
289 struct efa_com_rdma_read_stats rdma_read_stats;297 struct efa_com_rdma_read_stats rdma_read_stats;
290 struct efa_com_rdma_write_stats rdma_write_stats;298 struct efa_com_rdma_write_stats rdma_write_stats;
299 struct efa_com_network_stats network_stats;
291};300};
292 301
293int efa_com_create_qp(struct efa_com_dev *edev,302int efa_com_create_qp(struct efa_com_dev *edev,
drivers/infiniband/hw/efa/efa_main.c+1−0
@@ -372,6 +372,7 @@ static const struct ib_device_ops efa_dev_ops = { · 371 unmodified lines
372 .alloc_pd = efa_alloc_pd,372 .alloc_pd = efa_alloc_pd,
373 .alloc_ucontext = efa_alloc_ucontext,373 .alloc_ucontext = efa_alloc_ucontext,
374 .create_cq = efa_create_cq,374 .create_cq = efa_create_cq,
375 .create_cq_umem = efa_create_cq_umem,
375 .create_qp = efa_create_qp,376 .create_qp = efa_create_qp,
376 .create_user_ah = efa_create_ah,377 .create_user_ah = efa_create_ah,
377 .dealloc_pd = efa_dealloc_pd,378 .dealloc_pd = efa_dealloc_pd,
drivers/infiniband/hw/efa/efa_verbs.c+77−14
@@ -64,6 +64,11 @@ struct efa_user_mmap_entry { · 63 unmodified lines
64 op(EFA_RDMA_WRITE_BYTES, "rdma_write_bytes") \64 op(EFA_RDMA_WRITE_BYTES, "rdma_write_bytes") \
65 op(EFA_RDMA_WRITE_WR_ERR, "rdma_write_wr_err") \65 op(EFA_RDMA_WRITE_WR_ERR, "rdma_write_wr_err") \
66 op(EFA_RDMA_WRITE_RECV_BYTES, "rdma_write_recv_bytes") \66 op(EFA_RDMA_WRITE_RECV_BYTES, "rdma_write_recv_bytes") \
67 op(EFA_RETRANS_BYTES, "retrans_bytes") \
68 op(EFA_RETRANS_PKTS, "retrans_pkts") \
69 op(EFA_RETRANS_TIMEOUT_EVENS, "retrans_timeout_events") \
70 op(EFA_UNRESPONSIVE_REMOTE_EVENTS, "unresponsive_remote_events") \
71 op(EFA_IMPAIRED_REMOTE_CONN_EVENTS, "impaired_remote_conn_events") \
67 72
68#define EFA_STATS_ENUM(ename, name) ename,73#define EFA_STATS_ENUM(ename, name) ename,
69#define EFA_STATS_STR(ename, nam) \74#define EFA_STATS_STR(ename, nam) \
@@ -249,6 +254,7 @@ int efa_query_device(struct ib_device *ibdev, · 179 unmodified lines
249 resp.max_rdma_size = dev_attr->max_rdma_size;254 resp.max_rdma_size = dev_attr->max_rdma_size;
250 255
251 resp.device_caps |= EFA_QUERY_DEVICE_CAPS_CQ_WITH_SGID;256 resp.device_caps |= EFA_QUERY_DEVICE_CAPS_CQ_WITH_SGID;
257 resp.device_caps |= EFA_QUERY_DEVICE_CAPS_CQ_WITH_EXT_MEM;
252 if (EFA_DEV_CAP(dev, RDMA_READ))258 if (EFA_DEV_CAP(dev, RDMA_READ))
253 resp.device_caps |= EFA_QUERY_DEVICE_CAPS_RDMA_READ;259 resp.device_caps |= EFA_QUERY_DEVICE_CAPS_RDMA_READ;
254 260
@@ -1082,8 +1088,11 @@ int efa_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata) · 827 unmodified lines
1082 xa_erase(&dev->cqs_xa, cq->cq_idx);1088 xa_erase(&dev->cqs_xa, cq->cq_idx);
1083 synchronize_irq(cq->eq->irq.irqn);1089 synchronize_irq(cq->eq->irq.irqn);
1084 }1090 }
1085 efa_free_mapped(dev, cq->cpu_addr, cq->dma_addr, cq->size,1091
1086 DMA_FROM_DEVICE);1092 if (cq->umem)
1093 ib_umem_release(cq->umem);
1094 else
1095 efa_free_mapped(dev, cq->cpu_addr, cq->dma_addr, cq->size, DMA_FROM_DEVICE);
1087 return 0;1096 return 0;
1088}1097}
1089 1098
@@ -1122,8 +1131,8 @@ static int cq_mmap_entries_setup(struct efa_dev *dev, struct efa_cq *cq, · 32 unmodified lines
1122 return 0;1131 return 0;
1123}1132}
1124 1133
1125int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,1134int efa_create_cq_umem(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
1126 struct uverbs_attr_bundle *attrs)1135 struct ib_umem *umem, struct uverbs_attr_bundle *attrs)
1127{1136{
1128 struct ib_udata *udata = &attrs->driver_udata;1137 struct ib_udata *udata = &attrs->driver_udata;
1129 struct efa_ucontext *ucontext = rdma_udata_to_drv_context(1138 struct efa_ucontext *ucontext = rdma_udata_to_drv_context(
@@ -1202,11 +1211,30 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, · 72 unmodified lines
1202 1211
1203 cq->ucontext = ucontext;1212 cq->ucontext = ucontext;
1204 cq->size = PAGE_ALIGN(cmd.cq_entry_size * entries * cmd.num_sub_cqs);1213 cq->size = PAGE_ALIGN(cmd.cq_entry_size * entries * cmd.num_sub_cqs);
1205 cq->cpu_addr = efa_zalloc_mapped(dev, &cq->dma_addr, cq->size,1214
1206 DMA_FROM_DEVICE);1215 if (umem) {
1207 if (!cq->cpu_addr) {1216 if (umem->length < cq->size) {
1208 err = -ENOMEM;1217 ibdev_dbg(&dev->ibdev, "External memory too small\n");
1209 goto err_out;1218 err = -EINVAL;
1219 goto err_free_mem;
1220 }
1221
1222 if (!ib_umem_is_contiguous(umem)) {
1223 ibdev_dbg(&dev->ibdev, "Non contiguous CQ unsupported\n");
1224 err = -EINVAL;
1225 goto err_free_mem;
1226 }
1227
1228 cq->cpu_addr = NULL;
1229 cq->dma_addr = ib_umem_start_dma_addr(umem);
1230 cq->umem = umem;
1231 } else {
1232 cq->cpu_addr = efa_zalloc_mapped(dev, &cq->dma_addr, cq->size,
1233 DMA_FROM_DEVICE);
1234 if (!cq->cpu_addr) {
1235 err = -ENOMEM;
1236 goto err_out;
1237 }
1210 }1238 }
1211 1239
1212 params.uarn = cq->ucontext->uarn;1240 params.uarn = cq->ucontext->uarn;
@@ -1223,7 +1251,7 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, · 10 unmodified lines
1223 1251
1224 err = efa_com_create_cq(&dev->edev, &params, &result);1252 err = efa_com_create_cq(&dev->edev, &params, &result);
1225 if (err)1253 if (err)
1226 goto err_free_mapped;1254 goto err_free_mem;
1227 1255
1228 resp.db_off = result.db_off;1256 resp.db_off = result.db_off;
1229 resp.cq_idx = result.cq_idx;1257 resp.cq_idx = result.cq_idx;
@@ -1231,7 +1259,9 @@ int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, · 1 unmodified line
1231 cq->ibcq.cqe = result.actual_depth;1259 cq->ibcq.cqe = result.actual_depth;
1232 WARN_ON_ONCE(entries != result.actual_depth);1260 WARN_ON_ONCE(entries != result.actual_depth);
1233 1261
1234 err = cq_mmap_entries_setup(dev, cq, &resp, result.db_valid);1262 if (!umem)
1263 err = cq_mmap_entries_setup(dev, cq, &resp, result.db_valid);
1264
1235 if (err) {1265 if (err) {
1236 ibdev_dbg(ibdev, "Could not setup cq[%u] mmap entries\n",1266 ibdev_dbg(ibdev, "Could not setup cq[%u] mmap entries\n",
1237 cq->cq_idx);1267 cq->cq_idx);
@@ -1269,15 +1299,23 @@ err_remove_mmap: · 31 unmodified lines
1269 efa_cq_user_mmap_entries_remove(cq);1299 efa_cq_user_mmap_entries_remove(cq);
1270err_destroy_cq:1300err_destroy_cq:
1271 efa_destroy_cq_idx(dev, cq->cq_idx);1301 efa_destroy_cq_idx(dev, cq->cq_idx);
1272err_free_mapped:1302err_free_mem:
1273 efa_free_mapped(dev, cq->cpu_addr, cq->dma_addr, cq->size,1303 if (umem)
1274 DMA_FROM_DEVICE);1304 ib_umem_release(umem);
1305 else
1306 efa_free_mapped(dev, cq->cpu_addr, cq->dma_addr, cq->size, DMA_FROM_DEVICE);
1275 1307
1276err_out:1308err_out:
1277 atomic64_inc(&dev->stats.create_cq_err);1309 atomic64_inc(&dev->stats.create_cq_err);
1278 return err;1310 return err;
1279}1311}
1280 1312
1313int efa_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
1314 struct uverbs_attr_bundle *attrs)
1315{
1316 return efa_create_cq_umem(ibcq, attr, NULL, attrs);
1317}
1318
1281static int umem_to_page_list(struct efa_dev *dev,1319static int umem_to_page_list(struct efa_dev *dev,
1282 struct ib_umem *umem,1320 struct ib_umem *umem,
1283 u64 *page_list,1321 u64 *page_list,
@@ -1727,6 +1765,7 @@ static int efa_register_mr(struct ib_pd *ibpd, struct efa_mr *mr, u64 start, · 443 unmodified lines
1727struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,1765struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
1728 u64 length, u64 virt_addr,1766 u64 length, u64 virt_addr,
1729 int fd, int access_flags,1767 int fd, int access_flags,
1768 struct ib_dmah *dmah,
1730 struct uverbs_attr_bundle *attrs)1769 struct uverbs_attr_bundle *attrs)
1731{1770{
1732 struct efa_dev *dev = to_edev(ibpd->device);1771 struct efa_dev *dev = to_edev(ibpd->device);
@@ -1734,6 +1773,11 @@ struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start, · 1 unmodified line
1734 struct efa_mr *mr;1773 struct efa_mr *mr;
1735 int err;1774 int err;
1736 1775
1776 if (dmah) {
1777 err = -EOPNOTSUPP;
1778 goto err_out;
1779 }
1780
1737 mr = efa_alloc_mr(ibpd, access_flags, &attrs->driver_udata);1781 mr = efa_alloc_mr(ibpd, access_flags, &attrs->driver_udata);
1738 if (IS_ERR(mr)) {1782 if (IS_ERR(mr)) {
1739 err = PTR_ERR(mr);1783 err = PTR_ERR(mr);
@@ -1766,12 +1810,18 @@ err_out: · 26 unmodified lines
1766 1810
1767struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,1811struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
1768 u64 virt_addr, int access_flags,1812 u64 virt_addr, int access_flags,
1813 struct ib_dmah *dmah,
1769 struct ib_udata *udata)1814 struct ib_udata *udata)
1770{1815{
1771 struct efa_dev *dev = to_edev(ibpd->device);1816 struct efa_dev *dev = to_edev(ibpd->device);
1772 struct efa_mr *mr;1817 struct efa_mr *mr;
1773 int err;1818 int err;
1774 1819
1820 if (dmah) {
1821 err = -EOPNOTSUPP;
1822 goto err_out;
1823 }
1824
1775 mr = efa_alloc_mr(ibpd, access_flags, udata);1825 mr = efa_alloc_mr(ibpd, access_flags, udata);
1776 if (IS_ERR(mr)) {1826 if (IS_ERR(mr)) {
1777 err = PTR_ERR(mr);1827 err = PTR_ERR(mr);
@@ -2186,6 +2236,7 @@ static int efa_fill_port_stats(struct efa_dev *dev, struct rdma_hw_stats *stats, · 408 unmodified lines
2186 struct efa_com_rdma_write_stats *rws;2236 struct efa_com_rdma_write_stats *rws;
2187 struct efa_com_rdma_read_stats *rrs;2237 struct efa_com_rdma_read_stats *rrs;
2188 struct efa_com_messages_stats *ms;2238 struct efa_com_messages_stats *ms;
2239 struct efa_com_network_stats *ns;
2189 struct efa_com_basic_stats *bs;2240 struct efa_com_basic_stats *bs;
2190 int err;2241 int err;
2191 2242
@@ -2238,6 +2289,18 @@ static int efa_fill_port_stats(struct efa_dev *dev, struct rdma_hw_stats *stats, · 46 unmodified lines
2238 stats->value[EFA_RDMA_WRITE_RECV_BYTES] = rws->write_recv_bytes;2289 stats->value[EFA_RDMA_WRITE_RECV_BYTES] = rws->write_recv_bytes;
2239 }2290 }
2240 2291
2292 params.type = EFA_ADMIN_GET_STATS_TYPE_NETWORK;
2293 err = efa_com_get_stats(&dev->edev, &params, &result);
2294 if (err)
2295 return err;
2296
2297 ns = &result.network_stats;
2298 stats->value[EFA_RETRANS_BYTES] = ns->retrans_bytes;
2299 stats->value[EFA_RETRANS_PKTS] = ns->retrans_pkts;
2300 stats->value[EFA_RETRANS_TIMEOUT_EVENS] = ns->retrans_timeout_events;
2301 stats->value[EFA_UNRESPONSIVE_REMOTE_EVENTS] = ns->unresponsive_remote_events;
2302 stats->value[EFA_IMPAIRED_REMOTE_CONN_EVENTS] = ns->impaired_remote_conn_events;
2303
2241 return ARRAY_SIZE(efa_port_stats_descs);2304 return ARRAY_SIZE(efa_port_stats_descs);
2242}2305}
2243 2306
drivers/infiniband/hw/erdma/erdma_verbs.c+7−2
@@ -629,7 +629,8 @@ err_free_mtt: · 628 unmodified lines
629static void erdma_destroy_mtt_buf_sg(struct erdma_dev *dev,629static void erdma_destroy_mtt_buf_sg(struct erdma_dev *dev,
630 struct erdma_mtt *mtt)630 struct erdma_mtt *mtt)
631{631{
632 dma_unmap_sg(&dev->pdev->dev, mtt->sglist, mtt->nsg, DMA_TO_DEVICE);632 dma_unmap_sg(&dev->pdev->dev, mtt->sglist,
633 DIV_ROUND_UP(mtt->size, PAGE_SIZE), DMA_TO_DEVICE);
633 vfree(mtt->sglist);634 vfree(mtt->sglist);
634}635}
635 636
@@ -1199,13 +1200,17 @@ int erdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents, · 563 unmodified lines
1199}1200}
1200 1201
1201struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,1202struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,
1202 u64 virt, int access, struct ib_udata *udata)1203 u64 virt, int access, struct ib_dmah *dmah,
1204 struct ib_udata *udata)
1203{1205{
1204 struct erdma_mr *mr = NULL;1206 struct erdma_mr *mr = NULL;
1205 struct erdma_dev *dev = to_edev(ibpd->device);1207 struct erdma_dev *dev = to_edev(ibpd->device);
1206 u32 stag;1208 u32 stag;
1207 int ret;1209 int ret;
1208 1210
1211 if (dmah)
1212 return ERR_PTR(-EOPNOTSUPP);
1213
1209 if (!len || len > dev->attrs.max_mr_size)1214 if (!len || len > dev->attrs.max_mr_size)
1210 return ERR_PTR(-EINVAL);1215 return ERR_PTR(-EINVAL);
1211 1216
drivers/infiniband/hw/erdma/erdma_verbs.h+2−1
@@ -452,7 +452,8 @@ int erdma_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata); · 451 unmodified lines
452void erdma_disassociate_ucontext(struct ib_ucontext *ibcontext);452void erdma_disassociate_ucontext(struct ib_ucontext *ibcontext);
453int erdma_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);453int erdma_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
454struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,454struct ib_mr *erdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,
455 u64 virt, int access, struct ib_udata *udata);455 u64 virt, int access, struct ib_dmah *dmah,
456 struct ib_udata *udata);
456struct ib_mr *erdma_get_dma_mr(struct ib_pd *ibpd, int rights);457struct ib_mr *erdma_get_dma_mr(struct ib_pd *ibpd, int rights);
457int erdma_dereg_mr(struct ib_mr *ibmr, struct ib_udata *data);458int erdma_dereg_mr(struct ib_mr *ibmr, struct ib_udata *data);
458int erdma_mmap(struct ib_ucontext *ctx, struct vm_area_struct *vma);459int erdma_mmap(struct ib_ucontext *ctx, struct vm_area_struct *vma);
drivers/infiniband/hw/hfi1/affinity.c+37−59
@@ -92,9 +92,7 @@ static void cpu_mask_set_put(struct cpu_mask_set *set, int cpu) · 91 unmodified lines
92/* Initialize non-HT cpu cores mask */92/* Initialize non-HT cpu cores mask */
93void init_real_cpu_mask(void)93void init_real_cpu_mask(void)
94{94{
95 int possible, curr_cpu, i, ht;95 int possible, curr_cpu, ht;
96
97 cpumask_clear(&node_affinity.real_cpu_mask);
98 96
99 /* Start with cpu online mask as the real cpu mask */97 /* Start with cpu online mask as the real cpu mask */
100 cpumask_copy(&node_affinity.real_cpu_mask, cpu_online_mask);98 cpumask_copy(&node_affinity.real_cpu_mask, cpu_online_mask);
@@ -110,17 +108,10 @@ void init_real_cpu_mask(void) · 9 unmodified lines
110 * "real" cores. Assumes that HT cores are not enumerated in108 * "real" cores. Assumes that HT cores are not enumerated in
111 * succession (except in the single core case).109 * succession (except in the single core case).
112 */110 */
113 curr_cpu = cpumask_first(&node_affinity.real_cpu_mask);111 curr_cpu = cpumask_nth(possible / ht, &node_affinity.real_cpu_mask) + 1;
114 for (i = 0; i < possible / ht; i++)112
115 curr_cpu = cpumask_next(curr_cpu, &node_affinity.real_cpu_mask);113 /* Step 2. Remove the remaining HT siblings. */
116 /*114 cpumask_clear_cpus(&node_affinity.real_cpu_mask, curr_cpu, nr_cpu_ids - curr_cpu);
117 * Step 2. Remove the remaining HT siblings. Use cpumask_next() to
118 * skip any gaps.
119 */
120 for (; i < possible; i++) {
121 cpumask_clear_cpu(curr_cpu, &node_affinity.real_cpu_mask);
122 curr_cpu = cpumask_next(curr_cpu, &node_affinity.real_cpu_mask);
123 }
124}115}
125 116
126int node_affinity_init(void)117int node_affinity_init(void)
@@ -346,9 +337,10 @@ static int _dev_comp_vect_cpu_get(struct hfi1_devdata *dd, · 219 unmodified lines
346 &entry->def_intr.used);337 &entry->def_intr.used);
347 338
348 /* If there are non-interrupt CPUs available, use them first */339 /* If there are non-interrupt CPUs available, use them first */
349 if (!cpumask_empty(non_intr_cpus))340 cpu = cpumask_first(non_intr_cpus);
350 cpu = cpumask_first(non_intr_cpus);341
351 else /* Otherwise, use interrupt CPUs */342 /* Otherwise, use interrupt CPUs */
343 if (cpu >= nr_cpu_ids)
352 cpu = cpumask_first(available_cpus);344 cpu = cpumask_first(available_cpus);
353 345
354 if (cpu >= nr_cpu_ids) { /* empty */346 if (cpu >= nr_cpu_ids) { /* empty */
@@ -963,32 +955,23 @@ void hfi1_put_irq_affinity(struct hfi1_devdata *dd, · 608 unmodified lines
963static void find_hw_thread_mask(uint hw_thread_no, cpumask_var_t hw_thread_mask,955static void find_hw_thread_mask(uint hw_thread_no, cpumask_var_t hw_thread_mask,
964 struct hfi1_affinity_node_list *affinity)956 struct hfi1_affinity_node_list *affinity)
965{957{
966 int possible, curr_cpu, i;958 int curr_cpu;
967 uint num_cores_per_socket = node_affinity.num_online_cpus /959 uint num_cores;
968 affinity->num_core_siblings /
969 node_affinity.num_online_nodes;
970 960
971 cpumask_copy(hw_thread_mask, &affinity->proc.mask);961 cpumask_copy(hw_thread_mask, &affinity->proc.mask);
972 if (affinity->num_core_siblings > 0) {
973 /* Removing other siblings not needed for now */
974 possible = cpumask_weight(hw_thread_mask);
975 curr_cpu = cpumask_first(hw_thread_mask);
976 for (i = 0;
977 i < num_cores_per_socket * node_affinity.num_online_nodes;
978 i++)
979 curr_cpu = cpumask_next(curr_cpu, hw_thread_mask);
980
981 for (; i < possible; i++) {
982 cpumask_clear_cpu(curr_cpu, hw_thread_mask);
983 curr_cpu = cpumask_next(curr_cpu, hw_thread_mask);
984 }
985 962
986 /* Identifying correct HW threads within physical cores */963 if (affinity->num_core_siblings == 0)
987 cpumask_shift_left(hw_thread_mask, hw_thread_mask,964 return;
988 num_cores_per_socket *965
989 node_affinity.num_online_nodes *966 num_cores = rounddown(node_affinity.num_online_cpus / affinity->num_core_siblings,
990 hw_thread_no);967 node_affinity.num_online_nodes);
991 }968
969 /* Removing other siblings not needed for now */
970 curr_cpu = cpumask_nth(num_cores * node_affinity.num_online_nodes, hw_thread_mask) + 1;
971 cpumask_clear_cpus(hw_thread_mask, curr_cpu, nr_cpu_ids - curr_cpu);
972
973 /* Identifying correct HW threads within physical cores */
974 cpumask_shift_left(hw_thread_mask, hw_thread_mask, num_cores * hw_thread_no);
992}975}
993 976
994int hfi1_get_proc_affinity(int node)977int hfi1_get_proc_affinity(int node)
@@ -1087,22 +1070,19 @@ int hfi1_get_proc_affinity(int node) · 92 unmodified lines
1087 * If HT cores are enabled, identify which HW threads within the1070 * If HT cores are enabled, identify which HW threads within the
1088 * physical cores should be used.1071 * physical cores should be used.
1089 */1072 */
1090 if (affinity->num_core_siblings > 0) {1073 for (i = 0; i < affinity->num_core_siblings; i++) {
1091 for (i = 0; i < affinity->num_core_siblings; i++) {1074 find_hw_thread_mask(i, hw_thread_mask, affinity);
1092 find_hw_thread_mask(i, hw_thread_mask, affinity);
1093 1075
1094 /*1076 /*
1095 * If there's at least one available core for this HW1077 * If there's at least one available core for this HW
1096 * thread number, stop looking for a core.1078 * thread number, stop looking for a core.
1097 *1079 *
1098 * diff will always be not empty at least once in this1080 * diff will always be not empty at least once in this
1099 * loop as the used mask gets reset when1081 * loop as the used mask gets reset when
1100 * (set->mask == set->used) before this loop.1082 * (set->mask == set->used) before this loop.
1101 */1083 */
1102 cpumask_andnot(diff, hw_thread_mask, &set->used);1084 if (cpumask_andnot(diff, hw_thread_mask, &set->used))
1103 if (!cpumask_empty(diff))1085 break;
1104 break;
1105 }
1106 }1086 }
1107 hfi1_cdbg(PROC, "Same available HW thread on all physical CPUs: %*pbl",1087 hfi1_cdbg(PROC, "Same available HW thread on all physical CPUs: %*pbl",
1108 cpumask_pr_args(hw_thread_mask));1088 cpumask_pr_args(hw_thread_mask));
@@ -1133,8 +1113,7 @@ int hfi1_get_proc_affinity(int node) · 24 unmodified lines
1133 * used for process assignments using the same method as1113 * used for process assignments using the same method as
1134 * the preferred NUMA node.1114 * the preferred NUMA node.
1135 */1115 */
1136 cpumask_andnot(diff, available_mask, intrs_mask);1116 if (cpumask_andnot(diff, available_mask, intrs_mask))
1137 if (!cpumask_empty(diff))
1138 cpumask_copy(available_mask, diff);1117 cpumask_copy(available_mask, diff);
1139 1118
1140 /* If we don't have CPUs on the preferred node, use other NUMA nodes */1119 /* If we don't have CPUs on the preferred node, use other NUMA nodes */
@@ -1150,8 +1129,7 @@ int hfi1_get_proc_affinity(int node) · 9 unmodified lines
1150 * At first, we don't want to place processes on the same1129 * At first, we don't want to place processes on the same
1151 * CPUs as interrupt handlers.1130 * CPUs as interrupt handlers.
1152 */1131 */
1153 cpumask_andnot(diff, available_mask, intrs_mask);1132 if (cpumask_andnot(diff, available_mask, intrs_mask))
1154 if (!cpumask_empty(diff))
1155 cpumask_copy(available_mask, diff);1133 cpumask_copy(available_mask, diff);
1156 }1134 }
1157 hfi1_cdbg(PROC, "Possible CPUs for process: %*pbl",1135 hfi1_cdbg(PROC, "Possible CPUs for process: %*pbl",
drivers/infiniband/hw/hns/hns_roce_device.h+2−19
@@ -316,16 +316,6 @@ struct hns_roce_mtr { · 315 unmodified lines
316 struct hns_roce_hem_cfg hem_cfg; /* config for hardware addressing */316 struct hns_roce_hem_cfg hem_cfg; /* config for hardware addressing */
317};317};
318 318
319struct hns_roce_mw {
320 struct ib_mw ibmw;
321 u32 pdn;
322 u32 rkey;
323 int enabled; /* MW's active status */
324 u32 pbl_hop_num;
325 u32 pbl_ba_pg_sz;
326 u32 pbl_buf_pg_sz;
327};
328
329struct hns_roce_mr {319struct hns_roce_mr {
330 struct ib_mr ibmr;320 struct ib_mr ibmr;
331 u64 iova; /* MR's virtual original addr */321 u64 iova; /* MR's virtual original addr */
@@ -856,6 +846,7 @@ struct hns_roce_caps { · 524 unmodified lines
856 u16 default_ceq_arm_st;846 u16 default_ceq_arm_st;
857 u8 cong_cap;847 u8 cong_cap;
858 enum hns_roce_cong_type default_cong_type;848 enum hns_roce_cong_type default_cong_type;
849 u32 max_ack_req_msg_len;
859};850};
860 851
861enum hns_roce_device_state {852enum hns_roce_device_state {
@@ -933,7 +924,6 @@ struct hns_roce_hw { · 71 unmodified lines
933 struct hns_roce_mr *mr, int flags,924 struct hns_roce_mr *mr, int flags,
934 void *mb_buf);925 void *mb_buf);
935 int (*frmr_write_mtpt)(void *mb_buf, struct hns_roce_mr *mr);926 int (*frmr_write_mtpt)(void *mb_buf, struct hns_roce_mr *mr);
936 int (*mw_write_mtpt)(void *mb_buf, struct hns_roce_mw *mw);
937 void (*write_cqc)(struct hns_roce_dev *hr_dev,927 void (*write_cqc)(struct hns_roce_dev *hr_dev,
938 struct hns_roce_cq *hr_cq, void *mb_buf, u64 *mtts,928 struct hns_roce_cq *hr_cq, void *mb_buf, u64 *mtts,
939 dma_addr_t dma_handle);929 dma_addr_t dma_handle);
@@ -1078,11 +1068,6 @@ static inline struct hns_roce_mr *to_hr_mr(struct ib_mr *ibmr) · 138 unmodified lines
1078 return container_of(ibmr, struct hns_roce_mr, ibmr);1068 return container_of(ibmr, struct hns_roce_mr, ibmr);
1079}1069}
1080 1070
1081static inline struct hns_roce_mw *to_hr_mw(struct ib_mw *ibmw)
1082{
1083 return container_of(ibmw, struct hns_roce_mw, ibmw);
1084}
1085
1086static inline struct hns_roce_qp *to_hr_qp(struct ib_qp *ibqp)1071static inline struct hns_roce_qp *to_hr_qp(struct ib_qp *ibqp)
1087{1072{
1088 return container_of(ibqp, struct hns_roce_qp, ibqp);1073 return container_of(ibqp, struct hns_roce_qp, ibqp);
@@ -1234,6 +1219,7 @@ int hns_roce_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata); · 145 unmodified lines
1234struct ib_mr *hns_roce_get_dma_mr(struct ib_pd *pd, int acc);1219struct ib_mr *hns_roce_get_dma_mr(struct ib_pd *pd, int acc);
1235struct ib_mr *hns_roce_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,1220struct ib_mr *hns_roce_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
1236 u64 virt_addr, int access_flags,1221 u64 virt_addr, int access_flags,
1222 struct ib_dmah *dmah,
1237 struct ib_udata *udata);1223 struct ib_udata *udata);
1238struct ib_mr *hns_roce_rereg_user_mr(struct ib_mr *mr, int flags, u64 start,1224struct ib_mr *hns_roce_rereg_user_mr(struct ib_mr *mr, int flags, u64 start,
1239 u64 length, u64 virt_addr,1225 u64 length, u64 virt_addr,
@@ -1246,9 +1232,6 @@ int hns_roce_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents, · 6 unmodified lines
1246int hns_roce_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);1232int hns_roce_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
1247unsigned long key_to_hw_index(u32 key);1233unsigned long key_to_hw_index(u32 key);
1248 1234
1249int hns_roce_alloc_mw(struct ib_mw *mw, struct ib_udata *udata);
1250int hns_roce_dealloc_mw(struct ib_mw *ibmw);
1251
1252void hns_roce_buf_free(struct hns_roce_dev *hr_dev, struct hns_roce_buf *buf);1235void hns_roce_buf_free(struct hns_roce_dev *hr_dev, struct hns_roce_buf *buf);
1253struct hns_roce_buf *hns_roce_buf_alloc(struct hns_roce_dev *hr_dev, u32 size,1236struct hns_roce_buf *hns_roce_buf_alloc(struct hns_roce_dev *hr_dev, u32 size,
1254 u32 page_shift, u32 flags);1237 u32 page_shift, u32 flags);
drivers/infiniband/hw/hns/hns_roce_hem.c+5−13
@@ -249,15 +249,12 @@ int hns_roce_calc_hem_mhop(struct hns_roce_dev *hr_dev, · 248 unmodified lines
249}249}
250 250
251static struct hns_roce_hem *hns_roce_alloc_hem(struct hns_roce_dev *hr_dev,251static struct hns_roce_hem *hns_roce_alloc_hem(struct hns_roce_dev *hr_dev,
252 unsigned long hem_alloc_size,252 unsigned long hem_alloc_size)
253 gfp_t gfp_mask)
254{253{
255 struct hns_roce_hem *hem;254 struct hns_roce_hem *hem;
256 int order;255 int order;
257 void *buf;256 void *buf;
258 257
259 WARN_ON(gfp_mask & __GFP_HIGHMEM);
260
261 order = get_order(hem_alloc_size);258 order = get_order(hem_alloc_size);
262 if (PAGE_SIZE << order != hem_alloc_size) {259 if (PAGE_SIZE << order != hem_alloc_size) {
263 dev_err(hr_dev->dev, "invalid hem_alloc_size: %lu!\n",260 dev_err(hr_dev->dev, "invalid hem_alloc_size: %lu!\n",
@@ -265,13 +262,12 @@ static struct hns_roce_hem *hns_roce_alloc_hem(struct hns_roce_dev *hr_dev, · 1 unmodified line
265 return NULL;262 return NULL;
266 }263 }
267 264
268 hem = kmalloc(sizeof(*hem),265 hem = kmalloc(sizeof(*hem), GFP_KERNEL);
269 gfp_mask & ~(__GFP_HIGHMEM | __GFP_NOWARN));
270 if (!hem)266 if (!hem)
271 return NULL;267 return NULL;
272 268
273 buf = dma_alloc_coherent(hr_dev->dev, hem_alloc_size,269 buf = dma_alloc_coherent(hr_dev->dev, hem_alloc_size,
274 &hem->dma, gfp_mask);270 &hem->dma, GFP_KERNEL);
275 if (!buf)271 if (!buf)
276 goto fail;272 goto fail;
277 273
@@ -378,7 +374,6 @@ static int alloc_mhop_hem(struct hns_roce_dev *hr_dev, · 100 unmodified lines
378{374{
379 u32 bt_size = mhop->bt_chunk_size;375 u32 bt_size = mhop->bt_chunk_size;
380 struct device *dev = hr_dev->dev;376 struct device *dev = hr_dev->dev;
381 gfp_t flag;
382 u64 bt_ba;377 u64 bt_ba;
383 u32 size;378 u32 size;
384 int ret;379 int ret;
@@ -417,8 +412,7 @@ static int alloc_mhop_hem(struct hns_roce_dev *hr_dev, · 32 unmodified lines
417 * alloc bt space chunk for MTT/CQE.412 * alloc bt space chunk for MTT/CQE.
418 */413 */
419 size = table->type < HEM_TYPE_MTT ? mhop->buf_chunk_size : bt_size;414 size = table->type < HEM_TYPE_MTT ? mhop->buf_chunk_size : bt_size;
420 flag = GFP_KERNEL | __GFP_NOWARN;415 table->hem[index->buf] = hns_roce_alloc_hem(hr_dev, size);
421 table->hem[index->buf] = hns_roce_alloc_hem(hr_dev, size, flag);
422 if (!table->hem[index->buf]) {416 if (!table->hem[index->buf]) {
423 ret = -ENOMEM;417 ret = -ENOMEM;
424 goto err_alloc_hem;418 goto err_alloc_hem;
@@ -546,9 +540,7 @@ int hns_roce_table_get(struct hns_roce_dev *hr_dev, · 121 unmodified lines
546 goto out;540 goto out;
547 }541 }
548 542
549 table->hem[i] = hns_roce_alloc_hem(hr_dev,543 table->hem[i] = hns_roce_alloc_hem(hr_dev, table->table_chunk_size);
550 table->table_chunk_size,
551 GFP_KERNEL | __GFP_NOWARN);
552 if (!table->hem[i]) {544 if (!table->hem[i]) {
553 ret = -ENOMEM;545 ret = -ENOMEM;
554 goto out;546 goto out;
drivers/infiniband/hw/hns/hns_roce_hw_v2.c+64−70
@@ -144,7 +144,7 @@ static void set_frmr_seg(struct hns_roce_v2_rc_send_wqe *rc_sq_wqe, · 143 unmodified lines
144 u64 pbl_ba;144 u64 pbl_ba;
145 145
146 /* use ib_access_flags */146 /* use ib_access_flags */
147 hr_reg_write_bool(fseg, FRMR_BIND_EN, wr->access & IB_ACCESS_MW_BIND);147 hr_reg_write_bool(fseg, FRMR_BIND_EN, 0);
148 hr_reg_write_bool(fseg, FRMR_ATOMIC,148 hr_reg_write_bool(fseg, FRMR_ATOMIC,
149 wr->access & IB_ACCESS_REMOTE_ATOMIC);149 wr->access & IB_ACCESS_REMOTE_ATOMIC);
150 hr_reg_write_bool(fseg, FRMR_RR, wr->access & IB_ACCESS_REMOTE_READ);150 hr_reg_write_bool(fseg, FRMR_RR, wr->access & IB_ACCESS_REMOTE_READ);
@@ -2196,31 +2196,36 @@ static void apply_func_caps(struct hns_roce_dev *hr_dev) · 2045 unmodified lines
2196 2196
2197static int hns_roce_query_caps(struct hns_roce_dev *hr_dev)2197static int hns_roce_query_caps(struct hns_roce_dev *hr_dev)
2198{2198{
2199 struct hns_roce_cmq_desc desc[HNS_ROCE_QUERY_PF_CAPS_CMD_NUM];2199 struct hns_roce_cmq_desc desc[HNS_ROCE_QUERY_PF_CAPS_CMD_NUM] = {};
2200 struct hns_roce_caps *caps = &hr_dev->caps;2200 struct hns_roce_caps *caps = &hr_dev->caps;
2201 struct hns_roce_query_pf_caps_a *resp_a;2201 struct hns_roce_query_pf_caps_a *resp_a;
2202 struct hns_roce_query_pf_caps_b *resp_b;2202 struct hns_roce_query_pf_caps_b *resp_b;
2203 struct hns_roce_query_pf_caps_c *resp_c;2203 struct hns_roce_query_pf_caps_c *resp_c;
2204 struct hns_roce_query_pf_caps_d *resp_d;2204 struct hns_roce_query_pf_caps_d *resp_d;
2205 struct hns_roce_query_pf_caps_e *resp_e;2205 struct hns_roce_query_pf_caps_e *resp_e;
2206 struct hns_roce_query_pf_caps_f *resp_f;
2206 enum hns_roce_opcode_type cmd;2207 enum hns_roce_opcode_type cmd;
2207 int ctx_hop_num;2208 int ctx_hop_num;
2208 int pbl_hop_num;2209 int pbl_hop_num;
2210 int cmd_num;
2209 int ret;2211 int ret;
2210 int i;2212 int i;
2211 2213
2212 cmd = hr_dev->is_vf ? HNS_ROCE_OPC_QUERY_VF_CAPS_NUM :2214 cmd = hr_dev->is_vf ? HNS_ROCE_OPC_QUERY_VF_CAPS_NUM :
2213 HNS_ROCE_OPC_QUERY_PF_CAPS_NUM;2215 HNS_ROCE_OPC_QUERY_PF_CAPS_NUM;
2216 cmd_num = hr_dev->pci_dev->revision == PCI_REVISION_ID_HIP08 ?
2217 HNS_ROCE_QUERY_PF_CAPS_CMD_NUM_HIP08 :
2218 HNS_ROCE_QUERY_PF_CAPS_CMD_NUM;
2214 2219
2215 for (i = 0; i < HNS_ROCE_QUERY_PF_CAPS_CMD_NUM; i++) {2220 for (i = 0; i < cmd_num - 1; i++) {
2216 hns_roce_cmq_setup_basic_desc(&desc[i], cmd, true);2221 hns_roce_cmq_setup_basic_desc(&desc[i], cmd, true);
2217 if (i < (HNS_ROCE_QUERY_PF_CAPS_CMD_NUM - 1))2222 desc[i].flag |= cpu_to_le16(HNS_ROCE_CMD_FLAG_NEXT);
2218 desc[i].flag |= cpu_to_le16(HNS_ROCE_CMD_FLAG_NEXT);
2219 else
2220 desc[i].flag &= ~cpu_to_le16(HNS_ROCE_CMD_FLAG_NEXT);
2221 }2223 }
2222 2224
2223 ret = hns_roce_cmq_send(hr_dev, desc, HNS_ROCE_QUERY_PF_CAPS_CMD_NUM);2225 hns_roce_cmq_setup_basic_desc(&desc[cmd_num - 1], cmd, true);
2226 desc[cmd_num - 1].flag &= ~cpu_to_le16(HNS_ROCE_CMD_FLAG_NEXT);
2227
2228 ret = hns_roce_cmq_send(hr_dev, desc, cmd_num);
2224 if (ret)2229 if (ret)
2225 return ret;2230 return ret;
2226 2231
@@ -2229,6 +2234,7 @@ static int hns_roce_query_caps(struct hns_roce_dev *hr_dev) · 2 unmodified lines
2229 resp_c = (struct hns_roce_query_pf_caps_c *)desc[2].data;2234 resp_c = (struct hns_roce_query_pf_caps_c *)desc[2].data;
2230 resp_d = (struct hns_roce_query_pf_caps_d *)desc[3].data;2235 resp_d = (struct hns_roce_query_pf_caps_d *)desc[3].data;
2231 resp_e = (struct hns_roce_query_pf_caps_e *)desc[4].data;2236 resp_e = (struct hns_roce_query_pf_caps_e *)desc[4].data;
2237 resp_f = (struct hns_roce_query_pf_caps_f *)desc[5].data;
2232 2238
2233 caps->local_ca_ack_delay = resp_a->local_ca_ack_delay;2239 caps->local_ca_ack_delay = resp_a->local_ca_ack_delay;
2234 caps->max_sq_sg = le16_to_cpu(resp_a->max_sq_sg);2240 caps->max_sq_sg = le16_to_cpu(resp_a->max_sq_sg);
@@ -2293,6 +2299,8 @@ static int hns_roce_query_caps(struct hns_roce_dev *hr_dev) · 58 unmodified lines
2293 caps->reserved_srqs = hr_reg_read(resp_e, PF_CAPS_E_RSV_SRQS);2299 caps->reserved_srqs = hr_reg_read(resp_e, PF_CAPS_E_RSV_SRQS);
2294 caps->reserved_lkey = hr_reg_read(resp_e, PF_CAPS_E_RSV_LKEYS);2300 caps->reserved_lkey = hr_reg_read(resp_e, PF_CAPS_E_RSV_LKEYS);
2295 2301
2302 caps->max_ack_req_msg_len = le32_to_cpu(resp_f->max_ack_req_msg_len);
2303
2296 caps->qpc_hop_num = ctx_hop_num;2304 caps->qpc_hop_num = ctx_hop_num;
2297 caps->sccc_hop_num = ctx_hop_num;2305 caps->sccc_hop_num = ctx_hop_num;
2298 caps->srqc_hop_num = ctx_hop_num;2306 caps->srqc_hop_num = ctx_hop_num;
@@ -2627,7 +2635,7 @@ static struct ib_pd *free_mr_init_pd(struct hns_roce_dev *hr_dev) · 328 unmodified lines
2627 struct ib_pd *pd;2635 struct ib_pd *pd;
2628 2636
2629 hr_pd = kzalloc(sizeof(*hr_pd), GFP_KERNEL);2637 hr_pd = kzalloc(sizeof(*hr_pd), GFP_KERNEL);
2630 if (ZERO_OR_NULL_PTR(hr_pd))2638 if (!hr_pd)
2631 return NULL;2639 return NULL;
2632 pd = &hr_pd->ibpd;2640 pd = &hr_pd->ibpd;
2633 pd->device = ibdev;2641 pd->device = ibdev;
@@ -2658,7 +2666,7 @@ static struct ib_cq *free_mr_init_cq(struct hns_roce_dev *hr_dev) · 24 unmodified lines
2658 cq_init_attr.cqe = HNS_ROCE_FREE_MR_USED_CQE_NUM;2666 cq_init_attr.cqe = HNS_ROCE_FREE_MR_USED_CQE_NUM;
2659 2667
2660 hr_cq = kzalloc(sizeof(*hr_cq), GFP_KERNEL);2668 hr_cq = kzalloc(sizeof(*hr_cq), GFP_KERNEL);
2661 if (ZERO_OR_NULL_PTR(hr_cq))2669 if (!hr_cq)
2662 return NULL;2670 return NULL;
2663 2671
2664 cq = &hr_cq->ib_cq;2672 cq = &hr_cq->ib_cq;
@@ -2691,7 +2699,7 @@ static int free_mr_init_qp(struct hns_roce_dev *hr_dev, struct ib_cq *cq, · 26 unmodified lines
2691 int ret;2699 int ret;
2692 2700
2693 hr_qp = kzalloc(sizeof(*hr_qp), GFP_KERNEL);2701 hr_qp = kzalloc(sizeof(*hr_qp), GFP_KERNEL);
2694 if (ZERO_OR_NULL_PTR(hr_qp))2702 if (!hr_qp)
2695 return -ENOMEM;2703 return -ENOMEM;
2696 2704
2697 qp = &hr_qp->ibqp;2705 qp = &hr_qp->ibqp;
@@ -2986,14 +2994,22 @@ static int hns_roce_v2_init(struct hns_roce_dev *hr_dev) · 288 unmodified lines
2986{2994{
2987 int ret;2995 int ret;
2988 2996
2997 if (hr_dev->pci_dev->revision == PCI_REVISION_ID_HIP08) {
2998 ret = free_mr_init(hr_dev);
2999 if (ret) {
3000 dev_err(hr_dev->dev, "failed to init free mr!\n");
3001 return ret;
3002 }
3003 }
3004
2989 /* The hns ROCEE requires the extdb info to be cleared before using */3005 /* The hns ROCEE requires the extdb info to be cleared before using */
2990 ret = hns_roce_clear_extdb_list_info(hr_dev);3006 ret = hns_roce_clear_extdb_list_info(hr_dev);
2991 if (ret)3007 if (ret)
2992 return ret;3008 goto err_clear_extdb_failed;
2993 3009
2994 ret = get_hem_table(hr_dev);3010 ret = get_hem_table(hr_dev);
2995 if (ret)3011 if (ret)
2996 return ret;3012 goto err_get_hem_table_failed;
2997 3013
2998 if (hr_dev->is_vf)3014 if (hr_dev->is_vf)
2999 return 0;3015 return 0;
@@ -3008,6 +3024,11 @@ static int hns_roce_v2_init(struct hns_roce_dev *hr_dev) · 8 unmodified lines
3008 3024
3009err_llm_init_failed:3025err_llm_init_failed:
3010 put_hem_table(hr_dev);3026 put_hem_table(hr_dev);
3027err_get_hem_table_failed:
3028 hns_roce_function_clear(hr_dev);
3029err_clear_extdb_failed:
3030 if (hr_dev->pci_dev->revision == PCI_REVISION_ID_HIP08)
3031 free_mr_exit(hr_dev);
3011 3032
3012 return ret;3033 return ret;
3013}3034}
@@ -3313,8 +3334,6 @@ static int hns_roce_v2_write_mtpt(struct hns_roce_dev *hr_dev, · 299 unmodified lines
3313 hr_reg_write(mpt_entry, MPT_ST, V2_MPT_ST_VALID);3334 hr_reg_write(mpt_entry, MPT_ST, V2_MPT_ST_VALID);
3314 hr_reg_write(mpt_entry, MPT_PD, mr->pd);3335 hr_reg_write(mpt_entry, MPT_PD, mr->pd);
3315 3336
3316 hr_reg_write_bool(mpt_entry, MPT_BIND_EN,
3317 mr->access & IB_ACCESS_MW_BIND);
3318 hr_reg_write_bool(mpt_entry, MPT_ATOMIC_EN,3337 hr_reg_write_bool(mpt_entry, MPT_ATOMIC_EN,
3319 mr->access & IB_ACCESS_REMOTE_ATOMIC);3338 mr->access & IB_ACCESS_REMOTE_ATOMIC);
3320 hr_reg_write_bool(mpt_entry, MPT_RR_EN,3339 hr_reg_write_bool(mpt_entry, MPT_RR_EN,
@@ -3358,8 +3377,6 @@ static int hns_roce_v2_rereg_write_mtpt(struct hns_roce_dev *hr_dev, · 37 unmodified lines
3358 hr_reg_write(mpt_entry, MPT_PD, mr->pd);3377 hr_reg_write(mpt_entry, MPT_PD, mr->pd);
3359 3378
3360 if (flags & IB_MR_REREG_ACCESS) {3379 if (flags & IB_MR_REREG_ACCESS) {
3361 hr_reg_write(mpt_entry, MPT_BIND_EN,
3362 (mr_access_flags & IB_ACCESS_MW_BIND ? 1 : 0));
3363 hr_reg_write(mpt_entry, MPT_ATOMIC_EN,3380 hr_reg_write(mpt_entry, MPT_ATOMIC_EN,
3364 mr_access_flags & IB_ACCESS_REMOTE_ATOMIC ? 1 : 0);3381 mr_access_flags & IB_ACCESS_REMOTE_ATOMIC ? 1 : 0);
3365 hr_reg_write(mpt_entry, MPT_RR_EN,3382 hr_reg_write(mpt_entry, MPT_RR_EN,
@@ -3397,7 +3414,6 @@ static int hns_roce_v2_frmr_write_mtpt(void *mb_buf, struct hns_roce_mr *mr) · 31 unmodified lines
3397 hr_reg_enable(mpt_entry, MPT_R_INV_EN);3414 hr_reg_enable(mpt_entry, MPT_R_INV_EN);
3398 3415
3399 hr_reg_enable(mpt_entry, MPT_FRE);3416 hr_reg_enable(mpt_entry, MPT_FRE);
3400 hr_reg_clear(mpt_entry, MPT_MR_MW);
3401 hr_reg_enable(mpt_entry, MPT_BPD);3417 hr_reg_enable(mpt_entry, MPT_BPD);
3402 hr_reg_clear(mpt_entry, MPT_PA);3418 hr_reg_clear(mpt_entry, MPT_PA);
3403 3419
@@ -3417,38 +3433,6 @@ static int hns_roce_v2_frmr_write_mtpt(void *mb_buf, struct hns_roce_mr *mr) · 13 unmodified lines
3417 return 0;3433 return 0;
3418}3434}
3419 3435
3420static int hns_roce_v2_mw_write_mtpt(void *mb_buf, struct hns_roce_mw *mw)
3421{
3422 struct hns_roce_v2_mpt_entry *mpt_entry;
3423
3424 mpt_entry = mb_buf;
3425 memset(mpt_entry, 0, sizeof(*mpt_entry));
3426
3427 hr_reg_write(mpt_entry, MPT_ST, V2_MPT_ST_FREE);
3428 hr_reg_write(mpt_entry, MPT_PD, mw->pdn);
3429
3430 hr_reg_enable(mpt_entry, MPT_R_INV_EN);
3431 hr_reg_enable(mpt_entry, MPT_LW_EN);
3432
3433 hr_reg_enable(mpt_entry, MPT_MR_MW);
3434 hr_reg_enable(mpt_entry, MPT_BPD);
3435 hr_reg_clear(mpt_entry, MPT_PA);
3436 hr_reg_write(mpt_entry, MPT_BQP,
3437 mw->ibmw.type == IB_MW_TYPE_1 ? 0 : 1);
3438
3439 mpt_entry->lkey = cpu_to_le32(mw->rkey);
3440
3441 hr_reg_write(mpt_entry, MPT_PBL_HOP_NUM,
3442 mw->pbl_hop_num == HNS_ROCE_HOP_NUM_0 ? 0 :
3443 mw->pbl_hop_num);
3444 hr_reg_write(mpt_entry, MPT_PBL_BA_PG_SZ,
3445 mw->pbl_ba_pg_sz + PG_SHIFT_OFFSET);
3446 hr_reg_write(mpt_entry, MPT_PBL_BUF_PG_SZ,
3447 mw->pbl_buf_pg_sz + PG_SHIFT_OFFSET);
3448
3449 return 0;
3450}
3451
3452static int free_mr_post_send_lp_wqe(struct hns_roce_qp *hr_qp)3436static int free_mr_post_send_lp_wqe(struct hns_roce_qp *hr_qp)
3453{3437{
3454 struct hns_roce_dev *hr_dev = to_hr_dev(hr_qp->ibqp.device);3438 struct hns_roce_dev *hr_dev = to_hr_dev(hr_qp->ibqp.device);
@@ -3849,7 +3833,6 @@ static const u32 wc_send_op_map[] = { · 394 unmodified lines
3849 HR_WC_OP_MAP(ATOM_MSK_CMP_AND_SWAP, MASKED_COMP_SWAP),3833 HR_WC_OP_MAP(ATOM_MSK_CMP_AND_SWAP, MASKED_COMP_SWAP),
3850 HR_WC_OP_MAP(ATOM_MSK_FETCH_AND_ADD, MASKED_FETCH_ADD),3834 HR_WC_OP_MAP(ATOM_MSK_FETCH_AND_ADD, MASKED_FETCH_ADD),
3851 HR_WC_OP_MAP(FAST_REG_PMR, REG_MR),3835 HR_WC_OP_MAP(FAST_REG_PMR, REG_MR),
3852 HR_WC_OP_MAP(BIND_MW, REG_MR),
3853};3836};
3854 3837
3855static int to_ib_wc_send_op(u32 hr_opcode)3838static int to_ib_wc_send_op(u32 hr_opcode)
@@ -4560,7 +4543,9 @@ static int modify_qp_init_to_rtr(struct ib_qp *ibqp, · 704 unmodified lines
4560 dma_addr_t trrl_ba;4543 dma_addr_t trrl_ba;
4561 dma_addr_t irrl_ba;4544 dma_addr_t irrl_ba;
4562 enum ib_mtu ib_mtu;4545 enum ib_mtu ib_mtu;
4546 u8 ack_req_freq;
4563 const u8 *smac;4547 const u8 *smac;
4548 int lp_msg_len;
4564 u8 lp_pktn_ini;4549 u8 lp_pktn_ini;
4565 u64 *mtts;4550 u64 *mtts;
4566 u8 *dmac;4551 u8 *dmac;
@@ -4643,7 +4628,8 @@ static int modify_qp_init_to_rtr(struct ib_qp *ibqp, · 76 unmodified lines
4643 return -EINVAL;4628 return -EINVAL;
4644#define MIN_LP_MSG_LEN 10244629#define MIN_LP_MSG_LEN 1024
4645 /* mtu * (2 ^ lp_pktn_ini) should be in the range of 1024 to mtu */4630 /* mtu * (2 ^ lp_pktn_ini) should be in the range of 1024 to mtu */
4646 lp_pktn_ini = ilog2(max(mtu, MIN_LP_MSG_LEN) / mtu);4631 lp_msg_len = max(mtu, MIN_LP_MSG_LEN);
4632 lp_pktn_ini = ilog2(lp_msg_len / mtu);
4647 4633
4648 if (attr_mask & IB_QP_PATH_MTU) {4634 if (attr_mask & IB_QP_PATH_MTU) {
4649 hr_reg_write(context, QPC_MTU, ib_mtu);4635 hr_reg_write(context, QPC_MTU, ib_mtu);
@@ -4653,8 +4639,22 @@ static int modify_qp_init_to_rtr(struct ib_qp *ibqp, · 3 unmodified lines
4653 hr_reg_write(context, QPC_LP_PKTN_INI, lp_pktn_ini);4639 hr_reg_write(context, QPC_LP_PKTN_INI, lp_pktn_ini);
4654 hr_reg_clear(qpc_mask, QPC_LP_PKTN_INI);4640 hr_reg_clear(qpc_mask, QPC_LP_PKTN_INI);
4655 4641
4656 /* ACK_REQ_FREQ should be larger than or equal to LP_PKTN_INI */4642 /*
4657 hr_reg_write(context, QPC_ACK_REQ_FREQ, lp_pktn_ini);4643 * There are several constraints for ACK_REQ_FREQ:
4644 * 1. mtu * (2 ^ ACK_REQ_FREQ) should not be too large, otherwise
4645 * it may cause some unexpected retries when sending large
4646 * payload.
4647 * 2. ACK_REQ_FREQ should be larger than or equal to LP_PKTN_INI.
4648 * 3. ACK_REQ_FREQ must be equal to LP_PKTN_INI when using LDCP
4649 * or HC3 congestion control algorithm.
4650 */
4651 if (hr_qp->cong_type == CONG_TYPE_LDCP ||
4652 hr_qp->cong_type == CONG_TYPE_HC3 ||
4653 hr_dev->caps.max_ack_req_msg_len < lp_msg_len)
4654 ack_req_freq = lp_pktn_ini;
4655 else
4656 ack_req_freq = ilog2(hr_dev->caps.max_ack_req_msg_len / mtu);
4657 hr_reg_write(context, QPC_ACK_REQ_FREQ, ack_req_freq);
4658 hr_reg_clear(qpc_mask, QPC_ACK_REQ_FREQ);4658 hr_reg_clear(qpc_mask, QPC_ACK_REQ_FREQ);
4659 4659
4660 hr_reg_clear(qpc_mask, QPC_RX_REQ_PSN_ERR);4660 hr_reg_clear(qpc_mask, QPC_RX_REQ_PSN_ERR);
@@ -5349,11 +5349,10 @@ static int hns_roce_v2_modify_qp(struct ib_qp *ibqp, · 688 unmodified lines
5349{5349{
5350 struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);5350 struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
5351 struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);5351 struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
5352 struct hns_roce_v2_qp_context ctx[2];5352 struct hns_roce_v2_qp_context *context;
5353 struct hns_roce_v2_qp_context *context = ctx;5353 struct hns_roce_v2_qp_context *qpc_mask;
5354 struct hns_roce_v2_qp_context *qpc_mask = ctx + 1;
5355 struct ib_device *ibdev = &hr_dev->ib_dev;5354 struct ib_device *ibdev = &hr_dev->ib_dev;
5356 int ret;5355 int ret = -ENOMEM;
5357 5356
5358 if (attr_mask & ~IB_QP_ATTR_STANDARD_BITS)5357 if (attr_mask & ~IB_QP_ATTR_STANDARD_BITS)
5359 return -EOPNOTSUPP;5358 return -EOPNOTSUPP;
@@ -5364,7 +5363,11 @@ static int hns_roce_v2_modify_qp(struct ib_qp *ibqp, · 4 unmodified lines
5364 * we should set all bits of the relevant fields in context mask to5363 * we should set all bits of the relevant fields in context mask to
5365 * 0 at the same time, else set them to 0x1.5364 * 0 at the same time, else set them to 0x1.
5366 */5365 */
5367 memset(context, 0, hr_dev->caps.qpc_sz);5366 context = kvzalloc(sizeof(*context), GFP_KERNEL);
5367 qpc_mask = kvzalloc(sizeof(*qpc_mask), GFP_KERNEL);
5368 if (!context || !qpc_mask)
5369 goto out;
5370
5368 memset(qpc_mask, 0xff, hr_dev->caps.qpc_sz);5371 memset(qpc_mask, 0xff, hr_dev->caps.qpc_sz);
5369 5372
5370 ret = hns_roce_v2_set_abs_fields(ibqp, attr, attr_mask, cur_state,5373 ret = hns_roce_v2_set_abs_fields(ibqp, attr, attr_mask, cur_state,
@@ -5406,6 +5409,8 @@ static int hns_roce_v2_modify_qp(struct ib_qp *ibqp, · 35 unmodified lines
5406 clear_qp(hr_qp);5409 clear_qp(hr_qp);
5407 5410
5408out:5411out:
5412 kvfree(qpc_mask);
5413 kvfree(context);
5409 return ret;5414 return ret;
5410}5415}
5411 5416
@@ -6948,7 +6953,6 @@ static const struct hns_roce_hw hns_roce_hw_v2 = { · 1536 unmodified lines
6948 .write_mtpt = hns_roce_v2_write_mtpt,6953 .write_mtpt = hns_roce_v2_write_mtpt,
6949 .rereg_write_mtpt = hns_roce_v2_rereg_write_mtpt,6954 .rereg_write_mtpt = hns_roce_v2_rereg_write_mtpt,
6950 .frmr_write_mtpt = hns_roce_v2_frmr_write_mtpt,6955 .frmr_write_mtpt = hns_roce_v2_frmr_write_mtpt,
6951 .mw_write_mtpt = hns_roce_v2_mw_write_mtpt,
6952 .write_cqc = hns_roce_v2_write_cqc,6956 .write_cqc = hns_roce_v2_write_cqc,
6953 .set_hem = hns_roce_v2_set_hem,6957 .set_hem = hns_roce_v2_set_hem,
6954 .clear_hem = hns_roce_v2_clear_hem,6958 .clear_hem = hns_roce_v2_clear_hem,
@@ -7044,21 +7048,11 @@ static int __hns_roce_hw_v2_init_instance(struct hnae3_handle *handle) · 89 unmodified lines
7044 goto error_failed_roce_init;7048 goto error_failed_roce_init;
7045 }7049 }
7046 7050
7047 if (hr_dev->pci_dev->revision == PCI_REVISION_ID_HIP08) {
7048 ret = free_mr_init(hr_dev);
7049 if (ret) {
7050 dev_err(hr_dev->dev, "failed to init free mr!\n");
7051 goto error_failed_free_mr_init;
7052 }
7053 }
7054 7051
7055 handle->priv = hr_dev;7052 handle->priv = hr_dev;
7056 7053
7057 return 0;7054 return 0;
7058 7055
7059error_failed_free_mr_init:
7060 hns_roce_exit(hr_dev);
7061
7062error_failed_roce_init:7056error_failed_roce_init:
7063 kfree(hr_dev->priv);7057 kfree(hr_dev->priv);
7064 7058
drivers/infiniband/hw/hns/hns_roce_hw_v2.h+7−9
@@ -814,24 +814,16 @@ struct hns_roce_v2_mpt_entry { · 813 unmodified lines
814 814
815#define V2_MPT_BYTE_8_LW_EN_S 7815#define V2_MPT_BYTE_8_LW_EN_S 7
816 816
817#define V2_MPT_BYTE_8_MW_CNT_S 8
818#define V2_MPT_BYTE_8_MW_CNT_M GENMASK(31, 8)
819
820#define V2_MPT_BYTE_12_FRE_S 0817#define V2_MPT_BYTE_12_FRE_S 0
821 818
822#define V2_MPT_BYTE_12_PA_S 1819#define V2_MPT_BYTE_12_PA_S 1
823 820
824#define V2_MPT_BYTE_12_MR_MW_S 4
825
826#define V2_MPT_BYTE_12_BPD_S 5821#define V2_MPT_BYTE_12_BPD_S 5
827 822
828#define V2_MPT_BYTE_12_BQP_S 6823#define V2_MPT_BYTE_12_BQP_S 6
829 824
830#define V2_MPT_BYTE_12_INNER_PA_VLD_S 7825#define V2_MPT_BYTE_12_INNER_PA_VLD_S 7
831 826
832#define V2_MPT_BYTE_12_MW_BIND_QPN_S 8
833#define V2_MPT_BYTE_12_MW_BIND_QPN_M GENMASK(31, 8)
834
835#define V2_MPT_BYTE_48_PBL_BA_H_S 0827#define V2_MPT_BYTE_48_PBL_BA_H_S 0
836#define V2_MPT_BYTE_48_PBL_BA_H_M GENMASK(28, 0)828#define V2_MPT_BYTE_48_PBL_BA_H_M GENMASK(28, 0)
837 829
@@ -1168,7 +1160,8 @@ struct hns_roce_cfg_gmv_tb_b { · 330 unmodified lines
1168#define GMV_TB_B_SMAC_H GMV_TB_B_FIELD_LOC(47, 32)1160#define GMV_TB_B_SMAC_H GMV_TB_B_FIELD_LOC(47, 32)
1169#define GMV_TB_B_SGID_IDX GMV_TB_B_FIELD_LOC(71, 64)1161#define GMV_TB_B_SGID_IDX GMV_TB_B_FIELD_LOC(71, 64)
1170 1162
1171#define HNS_ROCE_QUERY_PF_CAPS_CMD_NUM 51163#define HNS_ROCE_QUERY_PF_CAPS_CMD_NUM_HIP08 5
1164#define HNS_ROCE_QUERY_PF_CAPS_CMD_NUM 6
1172struct hns_roce_query_pf_caps_a {1165struct hns_roce_query_pf_caps_a {
1173 u8 number_ports;1166 u8 number_ports;
1174 u8 local_ca_ack_delay;1167 u8 local_ca_ack_delay;
@@ -1280,6 +1273,11 @@ struct hns_roce_query_pf_caps_e { · 105 unmodified lines
1280 __le16 aeq_period;1273 __le16 aeq_period;
1281};1274};
1282 1275
1276struct hns_roce_query_pf_caps_f {
1277 __le32 max_ack_req_msg_len;
1278 __le32 rsv[5];
1279};
1280
1283#define PF_CAPS_E_FIELD_LOC(h, l) \1281#define PF_CAPS_E_FIELD_LOC(h, l) \
1284 FIELD_LOC(struct hns_roce_query_pf_caps_e, h, l)1282 FIELD_LOC(struct hns_roce_query_pf_caps_e, h, l)
1285 1283
drivers/infiniband/hw/hns/hns_roce_main.c+7−25
@@ -672,13 +672,6 @@ static const struct ib_device_ops hns_roce_dev_mr_ops = { · 671 unmodified lines
672 .rereg_user_mr = hns_roce_rereg_user_mr,672 .rereg_user_mr = hns_roce_rereg_user_mr,
673};673};
674 674
675static const struct ib_device_ops hns_roce_dev_mw_ops = {
676 .alloc_mw = hns_roce_alloc_mw,
677 .dealloc_mw = hns_roce_dealloc_mw,
678
679 INIT_RDMA_OBJ_SIZE(ib_mw, hns_roce_mw, ibmw),
680};
681
682static const struct ib_device_ops hns_roce_dev_frmr_ops = {675static const struct ib_device_ops hns_roce_dev_frmr_ops = {
683 .alloc_mr = hns_roce_alloc_mr,676 .alloc_mr = hns_roce_alloc_mr,
684 .map_mr_sg = hns_roce_map_mr_sg,677 .map_mr_sg = hns_roce_map_mr_sg,
@@ -732,9 +725,6 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev) · 47 unmodified lines
732 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_REREG_MR)725 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_REREG_MR)
733 ib_set_device_ops(ib_dev, &hns_roce_dev_mr_ops);726 ib_set_device_ops(ib_dev, &hns_roce_dev_mr_ops);
734 727
735 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_MW)
736 ib_set_device_ops(ib_dev, &hns_roce_dev_mw_ops);
737
738 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_FRMR)728 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_FRMR)
739 ib_set_device_ops(ib_dev, &hns_roce_dev_frmr_ops);729 ib_set_device_ops(ib_dev, &hns_roce_dev_frmr_ops);
740 730
@@ -947,10 +937,7 @@ err_unmap_dmpt: · 206 unmodified lines
947static void hns_roce_teardown_hca(struct hns_roce_dev *hr_dev)937static void hns_roce_teardown_hca(struct hns_roce_dev *hr_dev)
948{938{
949 hns_roce_cleanup_bitmap(hr_dev);939 hns_roce_cleanup_bitmap(hr_dev);
950 940 mutex_destroy(&hr_dev->pgdir_mutex);
951 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_CQ_RECORD_DB ||
952 hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_QP_RECORD_DB)
953 mutex_destroy(&hr_dev->pgdir_mutex);
954}941}
955 942
956/**943/**
@@ -965,11 +952,11 @@ static int hns_roce_setup_hca(struct hns_roce_dev *hr_dev) · 8 unmodified lines
965 952
966 spin_lock_init(&hr_dev->sm_lock);953 spin_lock_init(&hr_dev->sm_lock);
967 954
968 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_CQ_RECORD_DB ||955 INIT_LIST_HEAD(&hr_dev->qp_list);
969 hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_QP_RECORD_DB) {956 spin_lock_init(&hr_dev->qp_list_lock);
970 INIT_LIST_HEAD(&hr_dev->pgdir_list);957
971 mutex_init(&hr_dev->pgdir_mutex);958 INIT_LIST_HEAD(&hr_dev->pgdir_list);
972 }959 mutex_init(&hr_dev->pgdir_mutex);
973 960
974 hns_roce_init_uar_table(hr_dev);961 hns_roce_init_uar_table(hr_dev);
975 962
@@ -1001,9 +988,7 @@ static int hns_roce_setup_hca(struct hns_roce_dev *hr_dev) · 25 unmodified lines
1001 988
1002err_uar_table_free:989err_uar_table_free:
1003 ida_destroy(&hr_dev->uar_ida.ida);990 ida_destroy(&hr_dev->uar_ida.ida);
1004 if (hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_CQ_RECORD_DB ||991 mutex_destroy(&hr_dev->pgdir_mutex);
1005 hr_dev->caps.flags & HNS_ROCE_CAP_FLAG_QP_RECORD_DB)
1006 mutex_destroy(&hr_dev->pgdir_mutex);
1007 992
1008 return ret;993 return ret;
1009}994}
@@ -1132,9 +1117,6 @@ int hns_roce_init(struct hns_roce_dev *hr_dev) · 122 unmodified lines
1132 }1117 }
1133 }1118 }
1134 1119
1135 INIT_LIST_HEAD(&hr_dev->qp_list);
1136 spin_lock_init(&hr_dev->qp_list_lock);
1137
1138 ret = hns_roce_register_device(hr_dev);1120 ret = hns_roce_register_device(hr_dev);
1139 if (ret)1121 if (ret)
1140 goto error_failed_register_device;1122 goto error_failed_register_device;
drivers/infiniband/hw/hns/hns_roce_mr.c+6−114
@@ -231,12 +231,18 @@ err_free: · 230 unmodified lines
231 231
232struct ib_mr *hns_roce_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,232struct ib_mr *hns_roce_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
233 u64 virt_addr, int access_flags,233 u64 virt_addr, int access_flags,
234 struct ib_dmah *dmah,
234 struct ib_udata *udata)235 struct ib_udata *udata)
235{236{
236 struct hns_roce_dev *hr_dev = to_hr_dev(pd->device);237 struct hns_roce_dev *hr_dev = to_hr_dev(pd->device);
237 struct hns_roce_mr *mr;238 struct hns_roce_mr *mr;
238 int ret;239 int ret;
239 240
241 if (dmah) {
242 ret = -EOPNOTSUPP;
243 goto err_out;
244 }
245
240 mr = kzalloc(sizeof(*mr), GFP_KERNEL);246 mr = kzalloc(sizeof(*mr), GFP_KERNEL);
241 if (!mr) {247 if (!mr) {
242 ret = -ENOMEM;248 ret = -ENOMEM;
@@ -483,120 +489,6 @@ err_page_list: · 240 unmodified lines
483 return sg_num;489 return sg_num;
484}490}
485 491
486static void hns_roce_mw_free(struct hns_roce_dev *hr_dev,
487 struct hns_roce_mw *mw)
488{
489 struct device *dev = hr_dev->dev;
490 int ret;
491
492 if (mw->enabled) {
493 ret = hns_roce_destroy_hw_ctx(hr_dev, HNS_ROCE_CMD_DESTROY_MPT,
494 key_to_hw_index(mw->rkey) &
495 (hr_dev->caps.num_mtpts - 1));
496 if (ret)
497 dev_warn(dev, "MW DESTROY_MPT failed (%d)\n", ret);
498
499 hns_roce_table_put(hr_dev, &hr_dev->mr_table.mtpt_table,
500 key_to_hw_index(mw->rkey));
501 }
502
503 ida_free(&hr_dev->mr_table.mtpt_ida.ida,
504 (int)key_to_hw_index(mw->rkey));
505}
506
507static int hns_roce_mw_enable(struct hns_roce_dev *hr_dev,
508 struct hns_roce_mw *mw)
509{
510 struct hns_roce_mr_table *mr_table = &hr_dev->mr_table;
511 struct hns_roce_cmd_mailbox *mailbox;
512 struct device *dev = hr_dev->dev;
513 unsigned long mtpt_idx = key_to_hw_index(mw->rkey);
514 int ret;
515
516 /* prepare HEM entry memory */
517 ret = hns_roce_table_get(hr_dev, &mr_table->mtpt_table, mtpt_idx);
518 if (ret)
519 return ret;
520
521 mailbox = hns_roce_alloc_cmd_mailbox(hr_dev);
522 if (IS_ERR(mailbox)) {
523 ret = PTR_ERR(mailbox);
524 goto err_table;
525 }
526
527 ret = hr_dev->hw->mw_write_mtpt(mailbox->buf, mw);
528 if (ret) {
529 dev_err(dev, "MW write mtpt fail!\n");
530 goto err_page;
531 }
532
533 ret = hns_roce_create_hw_ctx(hr_dev, mailbox, HNS_ROCE_CMD_CREATE_MPT,
534 mtpt_idx & (hr_dev->caps.num_mtpts - 1));
535 if (ret) {
536 dev_err(dev, "MW CREATE_MPT failed (%d)\n", ret);
537 goto err_page;
538 }
539
540 mw->enabled = 1;
541
542 hns_roce_free_cmd_mailbox(hr_dev, mailbox);
543
544 return 0;
545
546err_page:
547 hns_roce_free_cmd_mailbox(hr_dev, mailbox);
548
549err_table:
550 hns_roce_table_put(hr_dev, &mr_table->mtpt_table, mtpt_idx);
551
552 return ret;
553}
554
555int hns_roce_alloc_mw(struct ib_mw *ibmw, struct ib_udata *udata)
556{
557 struct hns_roce_dev *hr_dev = to_hr_dev(ibmw->device);
558 struct hns_roce_ida *mtpt_ida = &hr_dev->mr_table.mtpt_ida;
559 struct ib_device *ibdev = &hr_dev->ib_dev;
560 struct hns_roce_mw *mw = to_hr_mw(ibmw);
561 int ret;
562 int id;
563
564 /* Allocate a key for mw from mr_table */
565 id = ida_alloc_range(&mtpt_ida->ida, mtpt_ida->min, mtpt_ida->max,
566 GFP_KERNEL);
567 if (id < 0) {
568 ibdev_err(ibdev, "failed to alloc id for MW key, id(%d)\n", id);
569 return -ENOMEM;
570 }
571
572 mw->rkey = hw_index_to_key(id);
573
574 ibmw->rkey = mw->rkey;
575 mw->pdn = to_hr_pd(ibmw->pd)->pdn;
576 mw->pbl_hop_num = hr_dev->caps.pbl_hop_num;
577 mw->pbl_ba_pg_sz = hr_dev->caps.pbl_ba_pg_sz;
578 mw->pbl_buf_pg_sz = hr_dev->caps.pbl_buf_pg_sz;
579
580 ret = hns_roce_mw_enable(hr_dev, mw);
581 if (ret)
582 goto err_mw;
583
584 return 0;
585
586err_mw:
587 hns_roce_mw_free(hr_dev, mw);
588 return ret;
589}
590
591int hns_roce_dealloc_mw(struct ib_mw *ibmw)
592{
593 struct hns_roce_dev *hr_dev = to_hr_dev(ibmw->device);
594 struct hns_roce_mw *mw = to_hr_mw(ibmw);
595
596 hns_roce_mw_free(hr_dev, mw);
597 return 0;
598}
599
600static int mtr_map_region(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr,492static int mtr_map_region(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr,
601 struct hns_roce_buf_region *region, dma_addr_t *pages,493 struct hns_roce_buf_region *region, dma_addr_t *pages,
602 int max_count)494 int max_count)
drivers/infiniband/hw/hns/hns_roce_qp.c+2−2
@@ -1003,14 +1003,14 @@ static int alloc_kernel_wrid(struct hns_roce_dev *hr_dev, · 1002 unmodified lines
1003 int ret;1003 int ret;
1004 1004
1005 sq_wrid = kcalloc(hr_qp->sq.wqe_cnt, sizeof(u64), GFP_KERNEL);1005 sq_wrid = kcalloc(hr_qp->sq.wqe_cnt, sizeof(u64), GFP_KERNEL);
1006 if (ZERO_OR_NULL_PTR(sq_wrid)) {1006 if (!sq_wrid) {
1007 ibdev_err(ibdev, "failed to alloc SQ wrid.\n");1007 ibdev_err(ibdev, "failed to alloc SQ wrid.\n");
1008 return -ENOMEM;1008 return -ENOMEM;
1009 }1009 }
1010 1010
1011 if (hr_qp->rq.wqe_cnt) {1011 if (hr_qp->rq.wqe_cnt) {
1012 rq_wrid = kcalloc(hr_qp->rq.wqe_cnt, sizeof(u64), GFP_KERNEL);1012 rq_wrid = kcalloc(hr_qp->rq.wqe_cnt, sizeof(u64), GFP_KERNEL);
1013 if (ZERO_OR_NULL_PTR(rq_wrid)) {1013 if (!rq_wrid) {
1014 ibdev_err(ibdev, "failed to alloc RQ wrid.\n");1014 ibdev_err(ibdev, "failed to alloc RQ wrid.\n");
1015 ret = -ENOMEM;1015 ret = -ENOMEM;
1016 goto err_sq;1016 goto err_sq;
drivers/infiniband/hw/irdma/verbs.c+9−0
@@ -3013,10 +3013,12 @@ static int irdma_reg_user_mr_type_cq(struct irdma_mem_reg_req req, · 3012 unmodified lines
3013 * @len: length of mr3013 * @len: length of mr
3014 * @virt: virtual address3014 * @virt: virtual address
3015 * @access: access of mr3015 * @access: access of mr
3016 * @dmah: dma handle
3016 * @udata: user data3017 * @udata: user data
3017 */3018 */
3018static struct ib_mr *irdma_reg_user_mr(struct ib_pd *pd, u64 start, u64 len,3019static struct ib_mr *irdma_reg_user_mr(struct ib_pd *pd, u64 start, u64 len,
3019 u64 virt, int access,3020 u64 virt, int access,
3021 struct ib_dmah *dmah,
3020 struct ib_udata *udata)3022 struct ib_udata *udata)
3021{3023{
3022#define IRDMA_MEM_REG_MIN_REQ_LEN offsetofend(struct irdma_mem_reg_req, sq_pages)3024#define IRDMA_MEM_REG_MIN_REQ_LEN offsetofend(struct irdma_mem_reg_req, sq_pages)
@@ -3026,6 +3028,9 @@ static struct ib_mr *irdma_reg_user_mr(struct ib_pd *pd, u64 start, u64 len, · 3 unmodified lines
3026 struct irdma_mr *iwmr = NULL;3028 struct irdma_mr *iwmr = NULL;
3027 int err;3029 int err;
3028 3030
3031 if (dmah)
3032 return ERR_PTR(-EOPNOTSUPP);
3033
3029 if (len > iwdev->rf->sc_dev.hw_attrs.max_mr_size)3034 if (len > iwdev->rf->sc_dev.hw_attrs.max_mr_size)
3030 return ERR_PTR(-EINVAL);3035 return ERR_PTR(-EINVAL);
3031 3036
@@ -3085,6 +3090,7 @@ error: · 53 unmodified lines
3085static struct ib_mr *irdma_reg_user_mr_dmabuf(struct ib_pd *pd, u64 start,3090static struct ib_mr *irdma_reg_user_mr_dmabuf(struct ib_pd *pd, u64 start,
3086 u64 len, u64 virt,3091 u64 len, u64 virt,
3087 int fd, int access,3092 int fd, int access,
3093 struct ib_dmah *dmah,
3088 struct uverbs_attr_bundle *attrs)3094 struct uverbs_attr_bundle *attrs)
3089{3095{
3090 struct irdma_device *iwdev = to_iwdev(pd->device);3096 struct irdma_device *iwdev = to_iwdev(pd->device);
@@ -3092,6 +3098,9 @@ static struct ib_mr *irdma_reg_user_mr_dmabuf(struct ib_pd *pd, u64 start, · 1 unmodified line
3092 struct irdma_mr *iwmr;3098 struct irdma_mr *iwmr;
3093 int err;3099 int err;
3094 3100
3101 if (dmah)
3102 return ERR_PTR(-EOPNOTSUPP);
3103
3095 if (len > iwdev->rf->sc_dev.hw_attrs.max_mr_size)3104 if (len > iwdev->rf->sc_dev.hw_attrs.max_mr_size)
3096 return ERR_PTR(-EINVAL);3105 return ERR_PTR(-EINVAL);
3097 3106
drivers/infiniband/hw/mana/counters.c+76−2
@@ -32,8 +32,32 @@ static const struct rdma_stat_desc mana_ib_port_stats_desc[] = { · 31 unmodified lines
32 [MANA_IB_RATE_INC_EVENTS].name = "rate_inc_events",32 [MANA_IB_RATE_INC_EVENTS].name = "rate_inc_events",
33 [MANA_IB_NUM_QPS_RECOVERED].name = "num_qps_recovered",33 [MANA_IB_NUM_QPS_RECOVERED].name = "num_qps_recovered",
34 [MANA_IB_CURRENT_RATE].name = "current_rate",34 [MANA_IB_CURRENT_RATE].name = "current_rate",
35 [MANA_IB_DUP_RX_REQ].name = "dup_rx_requests",
36 [MANA_IB_TX_BYTES].name = "tx_bytes",
37 [MANA_IB_RX_BYTES].name = "rx_bytes",
38 [MANA_IB_RX_SEND_REQ].name = "rx_send_requests",
39 [MANA_IB_RX_WRITE_REQ].name = "rx_write_requests",
40 [MANA_IB_RX_READ_REQ].name = "rx_read_requests",
41 [MANA_IB_TX_PKT].name = "tx_packets",
42 [MANA_IB_RX_PKT].name = "rx_packets",
35};43};
36 44
45static const struct rdma_stat_desc mana_ib_device_stats_desc[] = {
46 [MANA_IB_SENT_CNPS].name = "sent_cnps",
47 [MANA_IB_RECEIVED_ECNS].name = "received_ecns",
48 [MANA_IB_RECEIVED_CNP_COUNT].name = "received_cnp_count",
49 [MANA_IB_QP_CONGESTED_EVENTS].name = "qp_congested_events",
50 [MANA_IB_QP_RECOVERED_EVENTS].name = "qp_recovered_events",
51 [MANA_IB_DEV_RATE_INC_EVENTS].name = "rate_inc_events",
52};
53
54struct rdma_hw_stats *mana_ib_alloc_hw_device_stats(struct ib_device *ibdev)
55{
56 return rdma_alloc_hw_stats_struct(mana_ib_device_stats_desc,
57 ARRAY_SIZE(mana_ib_device_stats_desc),
58 RDMA_HW_STATS_DEFAULT_LIFESPAN);
59}
60
37struct rdma_hw_stats *mana_ib_alloc_hw_port_stats(struct ib_device *ibdev,61struct rdma_hw_stats *mana_ib_alloc_hw_port_stats(struct ib_device *ibdev,
38 u32 port_num)62 u32 port_num)
39{63{
@@ -42,8 +66,39 @@ struct rdma_hw_stats *mana_ib_alloc_hw_port_stats(struct ib_device *ibdev, · 2 unmodified lines
42 RDMA_HW_STATS_DEFAULT_LIFESPAN);66 RDMA_HW_STATS_DEFAULT_LIFESPAN);
43}67}
44 68
45int mana_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,69static int mana_ib_get_hw_device_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats)
46 u32 port_num, int index)70{
71 struct mana_ib_dev *mdev = container_of(ibdev, struct mana_ib_dev,
72 ib_dev);
73 struct mana_rnic_query_device_cntrs_resp resp = {};
74 struct mana_rnic_query_device_cntrs_req req = {};
75 int err;
76
77 mana_gd_init_req_hdr(&req.hdr, MANA_IB_QUERY_DEVICE_COUNTERS,
78 sizeof(req), sizeof(resp));
79 req.hdr.dev_id = mdev->gdma_dev->dev_id;
80 req.adapter = mdev->adapter_handle;
81
82 err = mana_gd_send_request(mdev_to_gc(mdev), sizeof(req), &req,
83 sizeof(resp), &resp);
84 if (err) {
85 ibdev_err(&mdev->ib_dev, "Failed to query device counters err %d",
86 err);
87 return err;
88 }
89
90 stats->value[MANA_IB_SENT_CNPS] = resp.sent_cnps;
91 stats->value[MANA_IB_RECEIVED_ECNS] = resp.received_ecns;
92 stats->value[MANA_IB_RECEIVED_CNP_COUNT] = resp.received_cnp_count;
93 stats->value[MANA_IB_QP_CONGESTED_EVENTS] = resp.qp_congested_events;
94 stats->value[MANA_IB_QP_RECOVERED_EVENTS] = resp.qp_recovered_events;
95 stats->value[MANA_IB_DEV_RATE_INC_EVENTS] = resp.rate_inc_events;
96
97 return ARRAY_SIZE(mana_ib_device_stats_desc);
98}
99
100static int mana_ib_get_hw_port_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,
101 u32 port_num)
47{102{
48 struct mana_ib_dev *mdev = container_of(ibdev, struct mana_ib_dev,103 struct mana_ib_dev *mdev = container_of(ibdev, struct mana_ib_dev,
49 ib_dev);104 ib_dev);
@@ -53,6 +108,7 @@ int mana_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats, · 3 unmodified lines
53 108
54 mana_gd_init_req_hdr(&req.hdr, MANA_IB_QUERY_VF_COUNTERS,109 mana_gd_init_req_hdr(&req.hdr, MANA_IB_QUERY_VF_COUNTERS,
55 sizeof(req), sizeof(resp));110 sizeof(req), sizeof(resp));
111 req.hdr.resp.msg_version = GDMA_MESSAGE_V2;
56 req.hdr.dev_id = mdev->gdma_dev->dev_id;112 req.hdr.dev_id = mdev->gdma_dev->dev_id;
57 req.adapter = mdev->adapter_handle;113 req.adapter = mdev->adapter_handle;
58 114
@@ -101,5 +157,23 @@ int mana_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats, · 42 unmodified lines
101 stats->value[MANA_IB_NUM_QPS_RECOVERED] = resp.num_qps_recovered;157 stats->value[MANA_IB_NUM_QPS_RECOVERED] = resp.num_qps_recovered;
102 stats->value[MANA_IB_CURRENT_RATE] = resp.current_rate;158 stats->value[MANA_IB_CURRENT_RATE] = resp.current_rate;
103 159
160 stats->value[MANA_IB_DUP_RX_REQ] = resp.dup_rx_req;
161 stats->value[MANA_IB_TX_BYTES] = resp.tx_bytes;
162 stats->value[MANA_IB_RX_BYTES] = resp.rx_bytes;
163 stats->value[MANA_IB_RX_SEND_REQ] = resp.rx_send_req;
164 stats->value[MANA_IB_RX_WRITE_REQ] = resp.rx_write_req;
165 stats->value[MANA_IB_RX_READ_REQ] = resp.rx_read_req;
166 stats->value[MANA_IB_TX_PKT] = resp.tx_pkt;
167 stats->value[MANA_IB_RX_PKT] = resp.rx_pkt;
168
104 return ARRAY_SIZE(mana_ib_port_stats_desc);169 return ARRAY_SIZE(mana_ib_port_stats_desc);
105}170}
171
172int mana_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,
173 u32 port_num, int index)
174{
175 if (!port_num)
176 return mana_ib_get_hw_device_stats(ibdev, stats);
177 else
178 return mana_ib_get_hw_port_stats(ibdev, stats, port_num);
179}
drivers/infiniband/hw/mana/counters.h+18−0
@@ -35,10 +35,28 @@ enum mana_ib_port_counters { · 34 unmodified lines
35 MANA_IB_RATE_INC_EVENTS,35 MANA_IB_RATE_INC_EVENTS,
36 MANA_IB_NUM_QPS_RECOVERED,36 MANA_IB_NUM_QPS_RECOVERED,
37 MANA_IB_CURRENT_RATE,37 MANA_IB_CURRENT_RATE,
38 MANA_IB_DUP_RX_REQ,
39 MANA_IB_TX_BYTES,
40 MANA_IB_RX_BYTES,
41 MANA_IB_RX_SEND_REQ,
42 MANA_IB_RX_WRITE_REQ,
43 MANA_IB_RX_READ_REQ,
44 MANA_IB_TX_PKT,
45 MANA_IB_RX_PKT,
46};
47
48enum mana_ib_device_counters {
49 MANA_IB_SENT_CNPS,
50 MANA_IB_RECEIVED_ECNS,
51 MANA_IB_RECEIVED_CNP_COUNT,
52 MANA_IB_QP_CONGESTED_EVENTS,
53 MANA_IB_QP_RECOVERED_EVENTS,
54 MANA_IB_DEV_RATE_INC_EVENTS,
38};55};
39 56
40struct rdma_hw_stats *mana_ib_alloc_hw_port_stats(struct ib_device *ibdev,57struct rdma_hw_stats *mana_ib_alloc_hw_port_stats(struct ib_device *ibdev,
41 u32 port_num);58 u32 port_num);
59struct rdma_hw_stats *mana_ib_alloc_hw_device_stats(struct ib_device *ibdev);
42int mana_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,60int mana_ib_get_hw_stats(struct ib_device *ibdev, struct rdma_hw_stats *stats,
43 u32 port_num, int index);61 u32 port_num, int index);
44#endif /* _COUNTERS_H_ */62#endif /* _COUNTERS_H_ */
drivers/infiniband/hw/mana/device.c+66−54
@@ -65,6 +65,10 @@ static const struct ib_device_ops mana_ib_stats_ops = { · 64 unmodified lines
65 .get_hw_stats = mana_ib_get_hw_stats,65 .get_hw_stats = mana_ib_get_hw_stats,
66};66};
67 67
68static const struct ib_device_ops mana_ib_device_stats_ops = {
69 .alloc_hw_device_stats = mana_ib_alloc_hw_device_stats,
70};
71
68static int mana_ib_netdev_event(struct notifier_block *this,72static int mana_ib_netdev_event(struct notifier_block *this,
69 unsigned long event, void *ptr)73 unsigned long event, void *ptr)
70{74{
@@ -73,28 +77,31 @@ static int mana_ib_netdev_event(struct notifier_block *this, · 2 unmodified lines
73 struct gdma_context *gc = dev->gdma_dev->gdma_context;77 struct gdma_context *gc = dev->gdma_dev->gdma_context;
74 struct mana_context *mc = gc->mana.driver_data;78 struct mana_context *mc = gc->mana.driver_data;
75 struct net_device *ndev;79 struct net_device *ndev;
80 int i;
76 81
77 /* Only process events from our parent device */82 /* Only process events from our parent device */
78 if (event_dev != mc->ports[0])83 for (i = 0; i < dev->ib_dev.phys_port_cnt; i++)
79 return NOTIFY_DONE;84 if (event_dev == mc->ports[i]) {
80 85 switch (event) {
81 switch (event) {86 case NETDEV_CHANGEUPPER:
82 case NETDEV_CHANGEUPPER:87 ndev = mana_get_primary_netdev(mc, i, &dev->dev_tracker);
83 ndev = mana_get_primary_netdev(mc, 0, &dev->dev_tracker);88 /*
84 /*89 * RDMA core will setup GID based on updated netdev.
85 * RDMA core will setup GID based on updated netdev.90 * It's not possible to race with the core as rtnl lock is being
86 * It's not possible to race with the core as rtnl lock is being91 * held.
87 * held.92 */
88 */93 ib_device_set_netdev(&dev->ib_dev, ndev, i + 1);
89 ib_device_set_netdev(&dev->ib_dev, ndev, 1);94
90 95 /* mana_get_primary_netdev() returns ndev with refcount held */
91 /* mana_get_primary_netdev() returns ndev with refcount held */96 if (ndev)
92 netdev_put(ndev, &dev->dev_tracker);97 netdev_put(ndev, &dev->dev_tracker);
93 98
94 return NOTIFY_OK;99 return NOTIFY_OK;
95 default:100 default:
96 return NOTIFY_DONE;101 return NOTIFY_DONE;
97 }102 }
103 }
104 return NOTIFY_DONE;
98}105}
99 106
100static int mana_ib_probe(struct auxiliary_device *adev,107static int mana_ib_probe(struct auxiliary_device *adev,
@@ -107,7 +114,7 @@ static int mana_ib_probe(struct auxiliary_device *adev, · 6 unmodified lines
107 struct net_device *ndev;114 struct net_device *ndev;
108 struct mana_ib_dev *dev;115 struct mana_ib_dev *dev;
109 u8 mac_addr[ETH_ALEN];116 u8 mac_addr[ETH_ALEN];
110 int ret;117 int ret, i;
111 118
112 dev = ib_alloc_device(mana_ib_dev, ib_dev);119 dev = ib_alloc_device(mana_ib_dev, ib_dev);
113 if (!dev)120 if (!dev)
@@ -122,51 +129,56 @@ static int mana_ib_probe(struct auxiliary_device *adev, · 8 unmodified lines
122 129
123 if (mana_ib_is_rnic(dev)) {130 if (mana_ib_is_rnic(dev)) {
124 dev->ib_dev.phys_port_cnt = 1;131 dev->ib_dev.phys_port_cnt = 1;
125 ndev = mana_get_primary_netdev(mc, 0, &dev->dev_tracker);132 addrconf_addr_eui48((u8 *)&dev->ib_dev.node_guid, mc->ports[0]->dev_addr);
126 if (!ndev) {
127 ret = -ENODEV;
128 ibdev_err(&dev->ib_dev, "Failed to get netdev for IB port 1");
129 goto free_ib_device;
130 }
131 ether_addr_copy(mac_addr, ndev->dev_addr);
132 addrconf_addr_eui48((u8 *)&dev->ib_dev.node_guid, ndev->dev_addr);
133 ret = ib_device_set_netdev(&dev->ib_dev, ndev, 1);
134 /* mana_get_primary_netdev() returns ndev with refcount held */
135 netdev_put(ndev, &dev->dev_tracker);
136 if (ret) {
137 ibdev_err(&dev->ib_dev, "Failed to set ib netdev, ret %d", ret);
138 goto free_ib_device;
139 }
140
141 dev->nb.notifier_call = mana_ib_netdev_event;
142 ret = register_netdevice_notifier(&dev->nb);
143 if (ret) {
144 ibdev_err(&dev->ib_dev, "Failed to register net notifier, %d",
145 ret);
146 goto free_ib_device;
147 }
148
149 ret = mana_ib_gd_query_adapter_caps(dev);133 ret = mana_ib_gd_query_adapter_caps(dev);
150 if (ret) {134 if (ret) {
151 ibdev_err(&dev->ib_dev, "Failed to query device caps, ret %d", ret);135 ibdev_err(&dev->ib_dev, "Failed to query device caps, ret %d", ret);
152 goto deregister_net_notifier;136 goto free_ib_device;
153 }137 }
154 138
155 ib_set_device_ops(&dev->ib_dev, &mana_ib_stats_ops);139 ib_set_device_ops(&dev->ib_dev, &mana_ib_stats_ops);
140 if (dev->adapter_caps.feature_flags & MANA_IB_FEATURE_DEV_COUNTERS_SUPPORT)
141 ib_set_device_ops(&dev->ib_dev, &mana_ib_device_stats_ops);
156 142
157 ret = mana_ib_create_eqs(dev);143 ret = mana_ib_create_eqs(dev);
158 if (ret) {144 if (ret) {
159 ibdev_err(&dev->ib_dev, "Failed to create EQs, ret %d", ret);145 ibdev_err(&dev->ib_dev, "Failed to create EQs, ret %d", ret);
160 goto deregister_net_notifier;146 goto free_ib_device;
161 }147 }
162 148
163 ret = mana_ib_gd_create_rnic_adapter(dev);149 ret = mana_ib_gd_create_rnic_adapter(dev);
164 if (ret)150 if (ret)
165 goto destroy_eqs;151 goto destroy_eqs;
166 152
167 ret = mana_ib_gd_config_mac(dev, ADDR_OP_ADD, mac_addr);153 if (dev->adapter_caps.feature_flags & MANA_IB_FEATURE_MULTI_PORTS_SUPPORT)
154 dev->ib_dev.phys_port_cnt = mc->num_ports;
155
156 for (i = 0; i < dev->ib_dev.phys_port_cnt; i++) {
157 ndev = mana_get_primary_netdev(mc, i, &dev->dev_tracker);
158 if (!ndev) {
159 ret = -ENODEV;
160 ibdev_err(&dev->ib_dev,
161 "Failed to get netdev for IB port %d", i + 1);
162 goto destroy_rnic;
163 }
164 ether_addr_copy(mac_addr, ndev->dev_addr);
165 ret = ib_device_set_netdev(&dev->ib_dev, ndev, i + 1);
166 /* mana_get_primary_netdev() returns ndev with refcount held */
167 netdev_put(ndev, &dev->dev_tracker);
168 if (ret) {
169 ibdev_err(&dev->ib_dev, "Failed to set ib netdev, ret %d", ret);
170 goto destroy_rnic;
171 }
172 ret = mana_ib_gd_config_mac(dev, ADDR_OP_ADD, mac_addr);
173 if (ret) {
174 ibdev_err(&dev->ib_dev, "Failed to add Mac address, ret %d", ret);
175 goto destroy_rnic;
176 }
177 }
178 dev->nb.notifier_call = mana_ib_netdev_event;
179 ret = register_netdevice_notifier(&dev->nb);
168 if (ret) {180 if (ret) {
169 ibdev_err(&dev->ib_dev, "Failed to add Mac address, ret %d", ret);181 ibdev_err(&dev->ib_dev, "Failed to register net notifier, %d", ret);
170 goto destroy_rnic;182 goto destroy_rnic;
171 }183 }
172 } else {184 } else {
@@ -182,7 +194,7 @@ static int mana_ib_probe(struct auxiliary_device *adev, · 9 unmodified lines
182 MANA_AV_BUFFER_SIZE, 0);194 MANA_AV_BUFFER_SIZE, 0);
183 if (!dev->av_pool) {195 if (!dev->av_pool) {
184 ret = -ENOMEM;196 ret = -ENOMEM;
185 goto destroy_rnic;197 goto deregister_net_notifier;
186 }198 }
187 199
188 ibdev_dbg(&dev->ib_dev, "mdev=%p id=%d num_ports=%d\n", mdev,200 ibdev_dbg(&dev->ib_dev, "mdev=%p id=%d num_ports=%d\n", mdev,
@@ -199,15 +211,15 @@ static int mana_ib_probe(struct auxiliary_device *adev, · 10 unmodified lines
199 211
200deallocate_pool:212deallocate_pool:
201 dma_pool_destroy(dev->av_pool);213 dma_pool_destroy(dev->av_pool);
214deregister_net_notifier:
215 if (mana_ib_is_rnic(dev))
216 unregister_netdevice_notifier(&dev->nb);
202destroy_rnic:217destroy_rnic:
203 if (mana_ib_is_rnic(dev))218 if (mana_ib_is_rnic(dev))
204 mana_ib_gd_destroy_rnic_adapter(dev);219 mana_ib_gd_destroy_rnic_adapter(dev);
205destroy_eqs:220destroy_eqs:
206 if (mana_ib_is_rnic(dev))221 if (mana_ib_is_rnic(dev))
207 mana_ib_destroy_eqs(dev);222 mana_ib_destroy_eqs(dev);
208deregister_net_notifier:
209 if (mana_ib_is_rnic(dev))
210 unregister_netdevice_notifier(&dev->nb);
211free_ib_device:223free_ib_device:
212 xa_destroy(&dev->qp_table_wq);224 xa_destroy(&dev->qp_table_wq);
213 ib_dealloc_device(&dev->ib_dev);225 ib_dealloc_device(&dev->ib_dev);
@@ -221,9 +233,9 @@ static void mana_ib_remove(struct auxiliary_device *adev) · 7 unmodified lines
221 ib_unregister_device(&dev->ib_dev);233 ib_unregister_device(&dev->ib_dev);
222 dma_pool_destroy(dev->av_pool);234 dma_pool_destroy(dev->av_pool);
223 if (mana_ib_is_rnic(dev)) {235 if (mana_ib_is_rnic(dev)) {
236 unregister_netdevice_notifier(&dev->nb);
224 mana_ib_gd_destroy_rnic_adapter(dev);237 mana_ib_gd_destroy_rnic_adapter(dev);
225 mana_ib_destroy_eqs(dev);238 mana_ib_destroy_eqs(dev);
226 unregister_netdevice_notifier(&dev->nb);
227 }239 }
228 xa_destroy(&dev->qp_table_wq);240 xa_destroy(&dev->qp_table_wq);
229 ib_dealloc_device(&dev->ib_dev);241 ib_dealloc_device(&dev->ib_dev);
drivers/infiniband/hw/mana/main.c+10−3
@@ -563,8 +563,14 @@ int mana_ib_get_port_immutable(struct ib_device *ibdev, u32 port_num, · 562 unmodified lines
563 immutable->gid_tbl_len = attr.gid_tbl_len;563 immutable->gid_tbl_len = attr.gid_tbl_len;
564 564
565 if (mana_ib_is_rnic(dev)) {565 if (mana_ib_is_rnic(dev)) {
566 immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;566 if (port_num == 1) {
567 immutable->max_mad_size = IB_MGMT_MAD_SIZE;567 immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
568 immutable->max_mad_size = IB_MGMT_MAD_SIZE;
569 } else {
570 immutable->core_cap_flags = RDMA_CORE_CAP_PROT_ROCE_UDP_ENCAP
571 | RDMA_CORE_CAP_ETH_AH;
572 immutable->max_mad_size = 0;
573 }
568 } else {574 } else {
569 immutable->core_cap_flags = RDMA_CORE_PORT_RAW_PACKET;575 immutable->core_cap_flags = RDMA_CORE_PORT_RAW_PACKET;
570 }576 }
@@ -633,8 +639,9 @@ int mana_ib_query_port(struct ib_device *ibdev, u32 port, · 62 unmodified lines
633 props->pkey_tbl_len = 1;639 props->pkey_tbl_len = 1;
634 if (mana_ib_is_rnic(dev)) {640 if (mana_ib_is_rnic(dev)) {
635 props->gid_tbl_len = 16;641 props->gid_tbl_len = 16;
636 props->port_cap_flags = IB_PORT_CM_SUP;
637 props->ip_gids = true;642 props->ip_gids = true;
643 if (port == 1)
644 props->port_cap_flags = IB_PORT_CM_SUP;
638 }645 }
639 646
640 return 0;647 return 0;
drivers/infiniband/hw/mana/mana_ib.h+30−0
@@ -210,6 +210,7 @@ enum mana_ib_command_code { · 209 unmodified lines
210 MANA_IB_DESTROY_RC_QP = 0x3000b,210 MANA_IB_DESTROY_RC_QP = 0x3000b,
211 MANA_IB_SET_QP_STATE = 0x3000d,211 MANA_IB_SET_QP_STATE = 0x3000d,
212 MANA_IB_QUERY_VF_COUNTERS = 0x30022,212 MANA_IB_QUERY_VF_COUNTERS = 0x30022,
213 MANA_IB_QUERY_DEVICE_COUNTERS = 0x30023,
213};214};
214 215
215struct mana_ib_query_adapter_caps_req {216struct mana_ib_query_adapter_caps_req {
@@ -218,6 +219,8 @@ struct mana_ib_query_adapter_caps_req { · 2 unmodified lines
218 219
219enum mana_ib_adapter_features {220enum mana_ib_adapter_features {
220 MANA_IB_FEATURE_CLIENT_ERROR_CQE_SUPPORT = BIT(4),221 MANA_IB_FEATURE_CLIENT_ERROR_CQE_SUPPORT = BIT(4),
222 MANA_IB_FEATURE_DEV_COUNTERS_SUPPORT = BIT(5),
223 MANA_IB_FEATURE_MULTI_PORTS_SUPPORT = BIT(6),
221};224};
222 225
223struct mana_ib_query_adapter_caps_resp {226struct mana_ib_query_adapter_caps_resp {
@@ -514,6 +517,31 @@ struct mana_rnic_query_vf_cntrs_resp { · 290 unmodified lines
514 u64 rate_inc_events;517 u64 rate_inc_events;
515 u64 num_qps_recovered;518 u64 num_qps_recovered;
516 u64 current_rate;519 u64 current_rate;
520 u64 dup_rx_req;
521 u64 tx_bytes;
522 u64 rx_bytes;
523 u64 rx_send_req;
524 u64 rx_write_req;
525 u64 rx_read_req;
526 u64 tx_pkt;
527 u64 rx_pkt;
528}; /* HW Data */
529
530struct mana_rnic_query_device_cntrs_req {
531 struct gdma_req_hdr hdr;
532 mana_handle_t adapter;
533}; /* HW Data */
534
535struct mana_rnic_query_device_cntrs_resp {
536 struct gdma_resp_hdr hdr;
537 u32 sent_cnps;
538 u32 received_ecns;
539 u32 reserved1;
540 u32 received_cnp_count;
541 u32 qp_congested_events;
542 u32 qp_recovered_events;
543 u32 rate_inc_events;
544 u32 reserved2;
517}; /* HW Data */545}; /* HW Data */
518 546
519static inline struct gdma_context *mdev_to_gc(struct mana_ib_dev *mdev)547static inline struct gdma_context *mdev_to_gc(struct mana_ib_dev *mdev)
@@ -605,6 +633,7 @@ struct ib_mr *mana_ib_get_dma_mr(struct ib_pd *ibpd, int access_flags); · 85 unmodified lines
605 633
606struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,634struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
607 u64 iova, int access_flags,635 u64 iova, int access_flags,
636 struct ib_dmah *dmah,
608 struct ib_udata *udata);637 struct ib_udata *udata);
609 638
610int mana_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);639int mana_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
@@ -694,5 +723,6 @@ int mana_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags); · 83 unmodified lines
694 723
695struct ib_mr *mana_ib_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start, u64 length,724struct ib_mr *mana_ib_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start, u64 length,
696 u64 iova, int fd, int mr_access_flags,725 u64 iova, int fd, int mr_access_flags,
726 struct ib_dmah *dmah,
697 struct uverbs_attr_bundle *attrs);727 struct uverbs_attr_bundle *attrs);
698#endif728#endif
drivers/infiniband/hw/mana/mr.c+8−0
@@ -106,6 +106,7 @@ static int mana_ib_gd_destroy_mr(struct mana_ib_dev *dev, u64 mr_handle) · 105 unmodified lines
106 106
107struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 length,107struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 length,
108 u64 iova, int access_flags,108 u64 iova, int access_flags,
109 struct ib_dmah *dmah,
109 struct ib_udata *udata)110 struct ib_udata *udata)
110{111{
111 struct mana_ib_pd *pd = container_of(ibpd, struct mana_ib_pd, ibpd);112 struct mana_ib_pd *pd = container_of(ibpd, struct mana_ib_pd, ibpd);
@@ -116,6 +117,9 @@ struct ib_mr *mana_ib_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 length, · 4 unmodified lines
116 u64 dma_region_handle;117 u64 dma_region_handle;
117 int err;118 int err;
118 119
120 if (dmah)
121 return ERR_PTR(-EOPNOTSUPP);
122
119 dev = container_of(ibdev, struct mana_ib_dev, ib_dev);123 dev = container_of(ibdev, struct mana_ib_dev, ib_dev);
120 124
121 ibdev_dbg(ibdev,125 ibdev_dbg(ibdev,
@@ -188,6 +192,7 @@ err_free: · 66 unmodified lines
188 192
189struct ib_mr *mana_ib_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start, u64 length,193struct ib_mr *mana_ib_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start, u64 length,
190 u64 iova, int fd, int access_flags,194 u64 iova, int fd, int access_flags,
195 struct ib_dmah *dmah,
191 struct uverbs_attr_bundle *attrs)196 struct uverbs_attr_bundle *attrs)
192{197{
193 struct mana_ib_pd *pd = container_of(ibpd, struct mana_ib_pd, ibpd);198 struct mana_ib_pd *pd = container_of(ibpd, struct mana_ib_pd, ibpd);
@@ -199,6 +204,9 @@ struct ib_mr *mana_ib_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start, u64 leng · 5 unmodified lines
199 u64 dma_region_handle;204 u64 dma_region_handle;
200 int err;205 int err;
201 206
207 if (dmah)
208 return ERR_PTR(-EOPNOTSUPP);
209
202 dev = container_of(ibdev, struct mana_ib_dev, ib_dev);210 dev = container_of(ibdev, struct mana_ib_dev, ib_dev);
203 211
204 access_flags &= ~IB_ACCESS_OPTIONAL;212 access_flags &= ~IB_ACCESS_OPTIONAL;
drivers/infiniband/hw/mana/qp.c+1−1
@@ -772,7 +772,7 @@ static int mana_ib_gd_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, · 771 unmodified lines
772 req.ah_attr.dest_port = ROCE_V2_UDP_DPORT;772 req.ah_attr.dest_port = ROCE_V2_UDP_DPORT;
773 req.ah_attr.src_port = rdma_get_udp_sport(attr->ah_attr.grh.flow_label,773 req.ah_attr.src_port = rdma_get_udp_sport(attr->ah_attr.grh.flow_label,
774 ibqp->qp_num, attr->dest_qp_num);774 ibqp->qp_num, attr->dest_qp_num);
775 req.ah_attr.traffic_class = attr->ah_attr.grh.traffic_class;775 req.ah_attr.traffic_class = attr->ah_attr.grh.traffic_class >> 2;
776 req.ah_attr.hop_limit = attr->ah_attr.grh.hop_limit;776 req.ah_attr.hop_limit = attr->ah_attr.grh.hop_limit;
777 }777 }
778 778
drivers/infiniband/hw/mlx4/mlx4_ib.h+1−0
@@ -759,6 +759,7 @@ int mlx4_ib_umem_write_mtt(struct mlx4_ib_dev *dev, struct mlx4_mtt *mtt, · 758 unmodified lines
759 struct ib_umem *umem);759 struct ib_umem *umem);
760struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,760struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
761 u64 virt_addr, int access_flags,761 u64 virt_addr, int access_flags,
762 struct ib_dmah *dmah,
762 struct ib_udata *udata);763 struct ib_udata *udata);
763int mlx4_ib_dereg_mr(struct ib_mr *mr, struct ib_udata *udata);764int mlx4_ib_dereg_mr(struct ib_mr *mr, struct ib_udata *udata);
764int mlx4_ib_alloc_mw(struct ib_mw *mw, struct ib_udata *udata);765int mlx4_ib_alloc_mw(struct ib_mw *mw, struct ib_udata *udata);
drivers/infiniband/hw/mlx4/mr.c+4−0
@@ -139,6 +139,7 @@ static struct ib_umem *mlx4_get_umem_mr(struct ib_device *device, u64 start, · 138 unmodified lines
139 139
140struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,140struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
141 u64 virt_addr, int access_flags,141 u64 virt_addr, int access_flags,
142 struct ib_dmah *dmah,
142 struct ib_udata *udata)143 struct ib_udata *udata)
143{144{
144 struct mlx4_ib_dev *dev = to_mdev(pd->device);145 struct mlx4_ib_dev *dev = to_mdev(pd->device);
@@ -147,6 +148,9 @@ struct ib_mr *mlx4_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, · 2 unmodified lines
147 int err;148 int err;
148 int n;149 int n;
149 150
151 if (dmah)
152 return ERR_PTR(-EOPNOTSUPP);
153
150 mr = kzalloc(sizeof(*mr), GFP_KERNEL);154 mr = kzalloc(sizeof(*mr), GFP_KERNEL);
151 if (!mr)155 if (!mr)
152 return ERR_PTR(-ENOMEM);156 return ERR_PTR(-ENOMEM);
drivers/infiniband/hw/mlx5/Makefile+1−0
@@ -8,6 +8,7 @@ mlx5_ib-y := ah.o \ · 7 unmodified lines
8 cq.o \8 cq.o \
9 data_direct.o \9 data_direct.o \
10 dm.o \10 dm.o \
11 dmah.o \
11 doorbell.o \12 doorbell.o \
12 fs.o \13 fs.o \
13 gsi.o \14 gsi.o \
drivers/infiniband/hw/mlx5/counters.c+26−4
@@ -16,6 +16,18 @@ struct mlx5_ib_counter { · 15 unmodified lines
16 u32 type;16 u32 type;
17};17};
18 18
19struct mlx5_rdma_counter {
20 struct rdma_counter rdma_counter;
21
22 struct mlx5_fc *fc[MLX5_IB_OPCOUNTER_MAX];
23 struct xarray qpn_opfc_xa;
24};
25
26static struct mlx5_rdma_counter *to_mcounter(struct rdma_counter *counter)
27{
28 return container_of(counter, struct mlx5_rdma_counter, rdma_counter);
29}
30
19#define INIT_Q_COUNTER(_name) \31#define INIT_Q_COUNTER(_name) \
20 { .name = #_name, .offset = MLX5_BYTE_OFF(query_q_counter_out, _name)}32 { .name = #_name, .offset = MLX5_BYTE_OFF(query_q_counter_out, _name)}
21 33
@@ -602,7 +614,7 @@ static int mlx5_ib_counter_dealloc(struct rdma_counter *counter) · 580 unmodified lines
602 return 0;614 return 0;
603 615
604 WARN_ON(!xa_empty(&mcounter->qpn_opfc_xa));616 WARN_ON(!xa_empty(&mcounter->qpn_opfc_xa));
605 mlx5r_fs_destroy_fcs(dev, counter);617 mlx5r_fs_destroy_fcs(dev, mcounter->fc);
606 MLX5_SET(dealloc_q_counter_in, in, opcode,618 MLX5_SET(dealloc_q_counter_in, in, opcode,
607 MLX5_CMD_OP_DEALLOC_Q_COUNTER);619 MLX5_CMD_OP_DEALLOC_Q_COUNTER);
608 MLX5_SET(dealloc_q_counter_in, in, counter_set_id, counter->id);620 MLX5_SET(dealloc_q_counter_in, in, counter_set_id, counter->id);
@@ -612,6 +624,7 @@ static int mlx5_ib_counter_dealloc(struct rdma_counter *counter) · 3 unmodified lines
612static int mlx5_ib_counter_bind_qp(struct rdma_counter *counter,624static int mlx5_ib_counter_bind_qp(struct rdma_counter *counter,
613 struct ib_qp *qp, u32 port)625 struct ib_qp *qp, u32 port)
614{626{
627 struct mlx5_rdma_counter *mcounter = to_mcounter(counter);
615 struct mlx5_ib_dev *dev = to_mdev(qp->device);628 struct mlx5_ib_dev *dev = to_mdev(qp->device);
616 bool new = false;629 bool new = false;
617 int err;630 int err;
@@ -635,7 +648,11 @@ static int mlx5_ib_counter_bind_qp(struct rdma_counter *counter, · 17 unmodified lines
635 if (err)648 if (err)
636 goto fail_set_counter;649 goto fail_set_counter;
637 650
638 err = mlx5r_fs_bind_op_fc(qp, counter, port);651 if (!counter->mode.bind_opcnt)
652 return 0;
653
654 err = mlx5r_fs_bind_op_fc(qp, mcounter->fc, &mcounter->qpn_opfc_xa,
655 port);
639 if (err)656 if (err)
640 goto fail_bind_op_fc;657 goto fail_bind_op_fc;
641 658
@@ -655,9 +672,12 @@ fail_set_counter: · 13 unmodified lines
655static int mlx5_ib_counter_unbind_qp(struct ib_qp *qp, u32 port)672static int mlx5_ib_counter_unbind_qp(struct ib_qp *qp, u32 port)
656{673{
657 struct rdma_counter *counter = qp->counter;674 struct rdma_counter *counter = qp->counter;
675 struct mlx5_rdma_counter *mcounter;
658 int err;676 int err;
659 677
660 mlx5r_fs_unbind_op_fc(qp, counter);678 mcounter = to_mcounter(counter);
679
680 mlx5r_fs_unbind_op_fc(qp, &mcounter->qpn_opfc_xa);
661 681
662 err = mlx5_ib_qp_set_counter(qp, NULL);682 err = mlx5_ib_qp_set_counter(qp, NULL);
663 if (err)683 if (err)
@@ -666,7 +686,9 @@ static int mlx5_ib_counter_unbind_qp(struct ib_qp *qp, u32 port) · 2 unmodified lines
666 return 0;686 return 0;
667 687
668fail_set_counter:688fail_set_counter:
669 mlx5r_fs_bind_op_fc(qp, counter, port);689 if (counter->mode.bind_opcnt)
690 mlx5r_fs_bind_op_fc(qp, mcounter->fc,
691 &mcounter->qpn_opfc_xa, port);
670 return err;692 return err;
671}693}
672 694
drivers/infiniband/hw/mlx5/counters.h+0−13
@@ -8,19 +8,6 @@ · 7 unmodified lines
8 8
9#include "mlx5_ib.h"9#include "mlx5_ib.h"
10 10
11struct mlx5_rdma_counter {
12 struct rdma_counter rdma_counter;
13
14 struct mlx5_fc *fc[MLX5_IB_OPCOUNTER_MAX];
15 struct xarray qpn_opfc_xa;
16};
17
18static inline struct mlx5_rdma_counter *
19to_mcounter(struct rdma_counter *counter)
20{
21 return container_of(counter, struct mlx5_rdma_counter, rdma_counter);
22}
23
24int mlx5_ib_counters_init(struct mlx5_ib_dev *dev);11int mlx5_ib_counters_init(struct mlx5_ib_dev *dev);
25void mlx5_ib_counters_cleanup(struct mlx5_ib_dev *dev);12void mlx5_ib_counters_cleanup(struct mlx5_ib_dev *dev);
26void mlx5_ib_counters_clear_description(struct ib_counters *counters);13void mlx5_ib_counters_clear_description(struct ib_counters *counters);
drivers/infiniband/hw/mlx5/cq.c+15−4
@@ -1055,20 +1055,31 @@ err_cqb: · 1054 unmodified lines
1055 return err;1055 return err;
1056}1056}
1057 1057
1058int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata)1058int mlx5_ib_pre_destroy_cq(struct ib_cq *cq)
1059{1059{
1060 struct mlx5_ib_dev *dev = to_mdev(cq->device);1060 struct mlx5_ib_dev *dev = to_mdev(cq->device);
1061 struct mlx5_ib_cq *mcq = to_mcq(cq);1061 struct mlx5_ib_cq *mcq = to_mcq(cq);
1062
1063 return mlx5_core_destroy_cq(dev->mdev, &mcq->mcq);
1064}
1065
1066void mlx5_ib_post_destroy_cq(struct ib_cq *cq)
1067{
1068 destroy_cq_kernel(to_mdev(cq->device), to_mcq(cq));
1069}
1070
1071int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata)
1072{
1062 int ret;1073 int ret;
1063 1074
1064 ret = mlx5_core_destroy_cq(dev->mdev, &mcq->mcq);1075 ret = mlx5_ib_pre_destroy_cq(cq);
1065 if (ret)1076 if (ret)
1066 return ret;1077 return ret;
1067 1078
1068 if (udata)1079 if (udata)
1069 destroy_cq_user(mcq, udata);1080 destroy_cq_user(to_mcq(cq), udata);
1070 else1081 else
1071 destroy_cq_kernel(dev, mcq);1082 mlx5_ib_post_destroy_cq(cq);
1072 return 0;1083 return 0;
1073}1084}
1074 1085
drivers/infiniband/hw/mlx5/devx.c+5−1
@@ -159,7 +159,7 @@ int mlx5_ib_devx_create(struct mlx5_ib_dev *dev, bool is_user, u64 req_ucaps) · 158 unmodified lines
159 uctx = MLX5_ADDR_OF(create_uctx_in, in, uctx);159 uctx = MLX5_ADDR_OF(create_uctx_in, in, uctx);
160 if (is_user &&160 if (is_user &&
161 (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RAW_TX) &&161 (MLX5_CAP_GEN(dev->mdev, uctx_cap) & MLX5_UCTX_CAP_RAW_TX) &&
162 capable(CAP_NET_RAW))162 rdma_dev_has_raw_cap(&dev->ib_dev))
163 cap |= MLX5_UCTX_CAP_RAW_TX;163 cap |= MLX5_UCTX_CAP_RAW_TX;
164 if (is_user &&164 if (is_user &&
165 (MLX5_CAP_GEN(dev->mdev, uctx_cap) &165 (MLX5_CAP_GEN(dev->mdev, uctx_cap) &
@@ -1393,6 +1393,10 @@ static int devx_handle_mkey_create(struct mlx5_ib_dev *dev, · 1227 unmodified lines
1393 }1393 }
1394 1394
1395 MLX5_SET(create_mkey_in, in, mkey_umem_valid, 1);1395 MLX5_SET(create_mkey_in, in, mkey_umem_valid, 1);
1396 /* TPH is not allowed to bypass the regular kernel's verbs flow */
1397 MLX5_SET(mkc, mkc, pcie_tph_en, 0);
1398 MLX5_SET(mkc, mkc, pcie_tph_steering_tag_index,
1399 MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX);
1396 return 0;1400 return 0;
1397}1401}
1398 1402
drivers/infiniband/hw/mlx5/dmah.c+54−0
@@ -0,0 +1,54 @@
1// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
2/*
3 * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
4 */
5
6#include <rdma/uverbs_std_types.h>
7#include <linux/pci-tph.h>
8#include "dmah.h"
9
10#define UVERBS_MODULE_NAME mlx5_ib
11#include <rdma/uverbs_named_ioctl.h>
12
13static int mlx5_ib_alloc_dmah(struct ib_dmah *ibdmah,
14 struct uverbs_attr_bundle *attrs)
15{
16 struct mlx5_core_dev *mdev = to_mdev(ibdmah->device)->mdev;
17 struct mlx5_ib_dmah *dmah = to_mdmah(ibdmah);
18 u16 st_bits = BIT(IB_DMAH_CPU_ID_EXISTS) |
19 BIT(IB_DMAH_MEM_TYPE_EXISTS);
20 int err;
21
22 /* PH is a must for TPH following PCIe spec 6.2-1.0 */
23 if (!(ibdmah->valid_fields & BIT(IB_DMAH_PH_EXISTS)))
24 return -EINVAL;
25
26 /* ST is optional; however, partial data for it is not allowed */
27 if (ibdmah->valid_fields & st_bits) {
28 if ((ibdmah->valid_fields & st_bits) != st_bits)
29 return -EINVAL;
30 err = mlx5_st_alloc_index(mdev, ibdmah->mem_type,
31 ibdmah->cpu_id, &dmah->st_index);
32 if (err)
33 return err;
34 }
35
36 return 0;
37}
38
39static int mlx5_ib_dealloc_dmah(struct ib_dmah *ibdmah,
40 struct uverbs_attr_bundle *attrs)
41{
42 struct mlx5_ib_dmah *dmah = to_mdmah(ibdmah);
43 struct mlx5_core_dev *mdev = to_mdev(ibdmah->device)->mdev;
44
45 if (ibdmah->valid_fields & BIT(IB_DMAH_CPU_ID_EXISTS))
46 return mlx5_st_dealloc_index(mdev, dmah->st_index);
47
48 return 0;
49}
50
51const struct ib_device_ops mlx5_ib_dev_dmah_ops = {
52 .alloc_dmah = mlx5_ib_alloc_dmah,
53 .dealloc_dmah = mlx5_ib_dealloc_dmah,
54};
drivers/infiniband/hw/mlx5/dmah.h+23−0
@@ -0,0 +1,23 @@
1/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
2/*
3 * Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved
4 */
5
6#ifndef _MLX5_IB_DMAH_H
7#define _MLX5_IB_DMAH_H
8
9#include "mlx5_ib.h"
10
11extern const struct ib_device_ops mlx5_ib_dev_dmah_ops;
12
13struct mlx5_ib_dmah {
14 struct ib_dmah ibdmah;
15 u16 st_index;
16};
17
18static inline struct mlx5_ib_dmah *to_mdmah(struct ib_dmah *ibdmah)
19{
20 return container_of(ibdmah, struct mlx5_ib_dmah, ibdmah);
21}
22
23#endif /* _MLX5_IB_DMAH_H */
drivers/infiniband/hw/mlx5/fs.c+63−58
@@ -1012,14 +1012,14 @@ static int get_per_qp_prio(struct mlx5_ib_dev *dev, · 1011 unmodified lines
1012 return 0;1012 return 0;
1013}1013}
1014 1014
1015static struct mlx5_per_qp_opfc *1015static struct mlx5_per_qp_opfc *get_per_qp_opfc(struct xarray *qpn_opfc_xa,
1016get_per_qp_opfc(struct mlx5_rdma_counter *mcounter, u32 qp_num, bool *new)1016 u32 qp_num, bool *new)
1017{1017{
1018 struct mlx5_per_qp_opfc *per_qp_opfc;1018 struct mlx5_per_qp_opfc *per_qp_opfc;
1019 1019
1020 *new = false;1020 *new = false;
1021 1021
1022 per_qp_opfc = xa_load(&mcounter->qpn_opfc_xa, qp_num);1022 per_qp_opfc = xa_load(qpn_opfc_xa, qp_num);
1023 if (per_qp_opfc)1023 if (per_qp_opfc)
1024 return per_qp_opfc;1024 return per_qp_opfc;
1025 per_qp_opfc = kzalloc(sizeof(*per_qp_opfc), GFP_KERNEL);1025 per_qp_opfc = kzalloc(sizeof(*per_qp_opfc), GFP_KERNEL);
@@ -1032,7 +1032,8 @@ get_per_qp_opfc(struct mlx5_rdma_counter *mcounter, u32 qp_num, bool *new) · 6 unmodified lines
1032}1032}
1033 1033
1034static int add_op_fc_rules(struct mlx5_ib_dev *dev,1034static int add_op_fc_rules(struct mlx5_ib_dev *dev,
1035 struct mlx5_rdma_counter *mcounter,1035 struct mlx5_fc *fc_arr[MLX5_IB_OPCOUNTER_MAX],
1036 struct xarray *qpn_opfc_xa,
1036 struct mlx5_per_qp_opfc *per_qp_opfc,1037 struct mlx5_per_qp_opfc *per_qp_opfc,
1037 struct mlx5_ib_flow_prio *prio,1038 struct mlx5_ib_flow_prio *prio,
1038 enum mlx5_ib_optional_counter_type type,1039 enum mlx5_ib_optional_counter_type type,
@@ -1055,7 +1056,7 @@ static int add_op_fc_rules(struct mlx5_ib_dev *dev, · 16 unmodified lines
1055 return 0;1056 return 0;
1056 }1057 }
1057 1058
1058 opfc->fc = mcounter->fc[type];1059 opfc->fc = fc_arr[type];
1059 1060
1060 spec = kcalloc(MAX_OPFC_RULES, sizeof(*spec), GFP_KERNEL);1061 spec = kcalloc(MAX_OPFC_RULES, sizeof(*spec), GFP_KERNEL);
1061 if (!spec) {1062 if (!spec) {
@@ -1148,8 +1149,7 @@ static int add_op_fc_rules(struct mlx5_ib_dev *dev, · 86 unmodified lines
1148 }1149 }
1149 prio->refcount += spec_num;1150 prio->refcount += spec_num;
1150 1151
1151 err = xa_err(xa_store(&mcounter->qpn_opfc_xa, qp_num, per_qp_opfc,1152 err = xa_err(xa_store(qpn_opfc_xa, qp_num, per_qp_opfc, GFP_KERNEL));
1152 GFP_KERNEL));
1153 if (err)1153 if (err)
1154 goto del_rules;1154 goto del_rules;
1155 1155
@@ -1168,8 +1168,9 @@ null_fc: · 12 unmodified lines
1168 return err;1168 return err;
1169}1169}
1170 1170
1171static bool is_fc_shared_and_in_use(struct mlx5_rdma_counter *mcounter,1171static bool
1172 u32 type, struct mlx5_fc **fc)1172is_fc_shared_and_in_use(struct mlx5_fc *fc_arr[MLX5_IB_OPCOUNTER_MAX], u32 type,
1173 struct mlx5_fc **fc)
1173{1174{
1174 u32 shared_fc_type;1175 u32 shared_fc_type;
1175 1176
@@ -1190,7 +1191,7 @@ static bool is_fc_shared_and_in_use(struct mlx5_rdma_counter *mcounter, · 14 unmodified lines
1190 return false;1191 return false;
1191 }1192 }
1192 1193
1193 *fc = mcounter->fc[shared_fc_type];1194 *fc = fc_arr[shared_fc_type];
1194 if (!(*fc))1195 if (!(*fc))
1195 return false;1196 return false;
1196 1197
@@ -1198,24 +1199,23 @@ static bool is_fc_shared_and_in_use(struct mlx5_rdma_counter *mcounter, · 1 unmodified line
1198}1199}
1199 1200
1200void mlx5r_fs_destroy_fcs(struct mlx5_ib_dev *dev,1201void mlx5r_fs_destroy_fcs(struct mlx5_ib_dev *dev,
1201 struct rdma_counter *counter)1202 struct mlx5_fc *fc_arr[MLX5_IB_OPCOUNTER_MAX])
1202{1203{
1203 struct mlx5_rdma_counter *mcounter = to_mcounter(counter);
1204 struct mlx5_fc *in_use_fc;1204 struct mlx5_fc *in_use_fc;
1205 int i;1205 int i;
1206 1206
1207 for (i = MLX5_IB_OPCOUNTER_CC_RX_CE_PKTS_PER_QP;1207 for (i = MLX5_IB_OPCOUNTER_CC_RX_CE_PKTS_PER_QP;
1208 i <= MLX5_IB_OPCOUNTER_RDMA_RX_BYTES_PER_QP; i++) {1208 i <= MLX5_IB_OPCOUNTER_RDMA_RX_BYTES_PER_QP; i++) {
1209 if (!mcounter->fc[i])1209 if (!fc_arr[i])
1210 continue;1210 continue;
1211 1211
1212 if (is_fc_shared_and_in_use(mcounter, i, &in_use_fc)) {1212 if (is_fc_shared_and_in_use(fc_arr, i, &in_use_fc)) {
1213 mcounter->fc[i] = NULL;1213 fc_arr[i] = NULL;
1214 continue;1214 continue;
1215 }1215 }
1216 1216
1217 mlx5_fc_destroy(dev->mdev, mcounter->fc[i]);1217 mlx5_fc_destroy(dev->mdev, fc_arr[i]);
1218 mcounter->fc[i] = NULL;1218 fc_arr[i] = NULL;
1219 }1219 }
1220}1220}
1221 1221
@@ -1359,16 +1359,15 @@ void mlx5_ib_fs_remove_op_fc(struct mlx5_ib_dev *dev, · 137 unmodified lines
1359 put_per_qp_prio(dev, type);1359 put_per_qp_prio(dev, type);
1360}1360}
1361 1361
1362void mlx5r_fs_unbind_op_fc(struct ib_qp *qp, struct rdma_counter *counter)1362void mlx5r_fs_unbind_op_fc(struct ib_qp *qp, struct xarray *qpn_opfc_xa)
1363{1363{
1364 struct mlx5_rdma_counter *mcounter = to_mcounter(counter);1364 struct mlx5_ib_dev *dev = to_mdev(qp->device);
1365 struct mlx5_ib_dev *dev = to_mdev(counter->device);
1366 struct mlx5_per_qp_opfc *per_qp_opfc;1365 struct mlx5_per_qp_opfc *per_qp_opfc;
1367 struct mlx5_ib_op_fc *in_use_opfc;1366 struct mlx5_ib_op_fc *in_use_opfc;
1368 struct mlx5_ib_flow_prio *prio;1367 struct mlx5_ib_flow_prio *prio;
1369 int i, j;1368 int i, j;
1370 1369
1371 per_qp_opfc = xa_load(&mcounter->qpn_opfc_xa, qp->qp_num);1370 per_qp_opfc = xa_load(qpn_opfc_xa, qp->qp_num);
1372 if (!per_qp_opfc)1371 if (!per_qp_opfc)
1373 return;1372 return;
1374 1373
@@ -1394,13 +1393,13 @@ void mlx5r_fs_unbind_op_fc(struct ib_qp *qp, struct rdma_counter *counter) · 19 unmodified lines
1394 }1393 }
1395 1394
1396 kfree(per_qp_opfc);1395 kfree(per_qp_opfc);
1397 xa_erase(&mcounter->qpn_opfc_xa, qp->qp_num);1396 xa_erase(qpn_opfc_xa, qp->qp_num);
1398}1397}
1399 1398
1400int mlx5r_fs_bind_op_fc(struct ib_qp *qp, struct rdma_counter *counter,1399int mlx5r_fs_bind_op_fc(struct ib_qp *qp,
1401 u32 port)1400 struct mlx5_fc *fc_arr[MLX5_IB_OPCOUNTER_MAX],
1401 struct xarray *qpn_opfc_xa, u32 port)
1402{1402{
1403 struct mlx5_rdma_counter *mcounter = to_mcounter(counter);
1404 struct mlx5_ib_dev *dev = to_mdev(qp->device);1403 struct mlx5_ib_dev *dev = to_mdev(qp->device);
1405 struct mlx5_per_qp_opfc *per_qp_opfc;1404 struct mlx5_per_qp_opfc *per_qp_opfc;
1406 struct mlx5_ib_flow_prio *prio;1405 struct mlx5_ib_flow_prio *prio;
@@ -1410,9 +1409,6 @@ int mlx5r_fs_bind_op_fc(struct ib_qp *qp, struct rdma_counter *counter, · 3 unmodified lines
1410 int i, err, per_qp_type;1409 int i, err, per_qp_type;
1411 bool new;1410 bool new;
1412 1411
1413 if (!counter->mode.bind_opcnt)
1414 return 0;
1415
1416 cnts = &dev->port[port - 1].cnts;1412 cnts = &dev->port[port - 1].cnts;
1417 1413
1418 for (i = 0; i <= MLX5_IB_OPCOUNTER_RDMA_RX_BYTES; i++) {1414 for (i = 0; i <= MLX5_IB_OPCOUNTER_RDMA_RX_BYTES; i++) {
@@ -1424,23 +1420,22 @@ int mlx5r_fs_bind_op_fc(struct ib_qp *qp, struct rdma_counter *counter, · 5 unmodified lines
1424 prio = get_opfc_prio(dev, per_qp_type);1420 prio = get_opfc_prio(dev, per_qp_type);
1425 WARN_ON(!prio->flow_table);1421 WARN_ON(!prio->flow_table);
1426 1422
1427 if (is_fc_shared_and_in_use(mcounter, per_qp_type, &in_use_fc))1423 if (is_fc_shared_and_in_use(fc_arr, per_qp_type, &in_use_fc))
1428 mcounter->fc[per_qp_type] = in_use_fc;1424 fc_arr[per_qp_type] = in_use_fc;
1429 1425
1430 if (!mcounter->fc[per_qp_type]) {1426 if (!fc_arr[per_qp_type]) {
1431 mcounter->fc[per_qp_type] = mlx5_fc_create(dev->mdev,1427 fc_arr[per_qp_type] = mlx5_fc_create(dev->mdev, false);
1432 false);1428 if (IS_ERR(fc_arr[per_qp_type]))
1433 if (IS_ERR(mcounter->fc[per_qp_type]))1429 return PTR_ERR(fc_arr[per_qp_type]);
1434 return PTR_ERR(mcounter->fc[per_qp_type]);
1435 }1430 }
1436 1431
1437 per_qp_opfc = get_per_qp_opfc(mcounter, qp->qp_num, &new);1432 per_qp_opfc = get_per_qp_opfc(qpn_opfc_xa, qp->qp_num, &new);
1438 if (!per_qp_opfc) {1433 if (!per_qp_opfc) {
1439 err = -ENOMEM;1434 err = -ENOMEM;
1440 goto free_fc;1435 goto free_fc;
1441 }1436 }
1442 err = add_op_fc_rules(dev, mcounter, per_qp_opfc, prio,1437 err = add_op_fc_rules(dev, fc_arr, qpn_opfc_xa, per_qp_opfc,
1443 per_qp_type, qp->qp_num, port);1438 prio, per_qp_type, qp->qp_num, port);
1444 if (err)1439 if (err)
1445 goto del_rules;1440 goto del_rules;
1446 }1441 }
@@ -1448,12 +1443,12 @@ int mlx5r_fs_bind_op_fc(struct ib_qp *qp, struct rdma_counter *counter, · 1 unmodified line
1448 return 0;1443 return 0;
1449 1444
1450del_rules:1445del_rules:
1451 mlx5r_fs_unbind_op_fc(qp, counter);1446 mlx5r_fs_unbind_op_fc(qp, qpn_opfc_xa);
1452 if (new)1447 if (new)
1453 kfree(per_qp_opfc);1448 kfree(per_qp_opfc);
1454free_fc:1449free_fc:
1455 if (xa_empty(&mcounter->qpn_opfc_xa))1450 if (xa_empty(qpn_opfc_xa))
1456 mlx5r_fs_destroy_fcs(dev, counter);1451 mlx5r_fs_destroy_fcs(dev, fc_arr);
1457 return err;1452 return err;
1458}1453}
1459 1454
@@ -1966,7 +1961,8 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority, · 506 unmodified lines
1966 break;1961 break;
1967 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:1962 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
1968 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:1963 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
1969 if (ib_port == 0 || user_priority > MLX5_RDMA_TRANSPORT_BYPASS_PRIO)1964 if (ib_port == 0 ||
1965 user_priority >= MLX5_RDMA_TRANSPORT_BYPASS_PRIO)
1970 return ERR_PTR(-EINVAL);1966 return ERR_PTR(-EINVAL);
1971 ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,1967 ret = mlx5_ib_fill_transport_ns_info(dev, ns_type, &flags,
1972 &vport_idx, &vport,1968 &vport_idx, &vport,
@@ -2016,10 +2012,10 @@ _get_flow_table(struct mlx5_ib_dev *dev, u16 user_priority, · 43 unmodified lines
2016 prio = &dev->flow_db->rdma_tx[priority];2012 prio = &dev->flow_db->rdma_tx[priority];
2017 break;2013 break;
2018 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:2014 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_RX:
2019 prio = &dev->flow_db->rdma_transport_rx[ib_port - 1];2015 prio = &dev->flow_db->rdma_transport_rx[priority][ib_port - 1];
2020 break;2016 break;
2021 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:2017 case MLX5_FLOW_NAMESPACE_RDMA_TRANSPORT_TX:
2022 prio = &dev->flow_db->rdma_transport_tx[ib_port - 1];2018 prio = &dev->flow_db->rdma_transport_tx[priority][ib_port - 1];
2023 break;2019 break;
2024 default: return ERR_PTR(-EINVAL);2020 default: return ERR_PTR(-EINVAL);
2025 }2021 }
@@ -2458,7 +2454,7 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_CREATE_FLOW)( · 432 unmodified lines
2458 struct mlx5_ib_dev *dev;2454 struct mlx5_ib_dev *dev;
2459 u32 flags;2455 u32 flags;
2460 2456
2461 if (!capable(CAP_NET_RAW))2457 if (!rdma_uattrs_has_raw_cap(attrs))
2462 return -EPERM;2458 return -EPERM;
2463 2459
2464 fs_matcher = uverbs_attr_get_obj(attrs,2460 fs_matcher = uverbs_attr_get_obj(attrs,
@@ -2989,7 +2985,7 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_STEERING_ANCHOR_CREATE)( · 524 unmodified lines
2989 u32 ft_id;2985 u32 ft_id;
2990 int err;2986 int err;
2991 2987
2992 if (!capable(CAP_NET_RAW))2988 if (!rdma_dev_has_raw_cap(&dev->ib_dev))
2993 return -EPERM;2989 return -EPERM;
2994 2990
2995 err = uverbs_get_const(&ib_uapi_ft_type, attrs,2991 err = uverbs_get_const(&ib_uapi_ft_type, attrs,
@@ -3466,31 +3462,40 @@ static const struct ib_device_ops flow_ops = { · 470 unmodified lines
3466 3462
3467int mlx5_ib_fs_init(struct mlx5_ib_dev *dev)3463int mlx5_ib_fs_init(struct mlx5_ib_dev *dev)
3468{3464{
3465 int i, j;
3466
3469 dev->flow_db = kzalloc(sizeof(*dev->flow_db), GFP_KERNEL);3467 dev->flow_db = kzalloc(sizeof(*dev->flow_db), GFP_KERNEL);
3470 3468
3471 if (!dev->flow_db)3469 if (!dev->flow_db)
3472 return -ENOMEM;3470 return -ENOMEM;
3473 3471
3474 dev->flow_db->rdma_transport_rx = kcalloc(dev->num_ports,3472 for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++) {
3475 sizeof(struct mlx5_ib_flow_prio),3473 dev->flow_db->rdma_transport_rx[i] =
3476 GFP_KERNEL);3474 kcalloc(dev->num_ports,
3477 if (!dev->flow_db->rdma_transport_rx)3475 sizeof(struct mlx5_ib_flow_prio), GFP_KERNEL);
3478 goto free_flow_db;3476 if (!dev->flow_db->rdma_transport_rx[i])
3477 goto free_rdma_transport_rx;
3478 }
3479 3479
3480 dev->flow_db->rdma_transport_tx = kcalloc(dev->num_ports,3480 for (j = 0; j < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; j++) {
3481 sizeof(struct mlx5_ib_flow_prio),3481 dev->flow_db->rdma_transport_tx[j] =
3482 GFP_KERNEL);3482 kcalloc(dev->num_ports,
3483 if (!dev->flow_db->rdma_transport_tx)3483 sizeof(struct mlx5_ib_flow_prio), GFP_KERNEL);
3484 goto free_rdma_transport_rx;3484 if (!dev->flow_db->rdma_transport_tx[j])
3485 goto free_rdma_transport_tx;
3486 }
3485 3487
3486 mutex_init(&dev->flow_db->lock);3488 mutex_init(&dev->flow_db->lock);
3487 3489
3488 ib_set_device_ops(&dev->ib_dev, &flow_ops);3490 ib_set_device_ops(&dev->ib_dev, &flow_ops);
3489 return 0;3491 return 0;
3490 3492
3493free_rdma_transport_tx:
3494 while (j--)
3495 kfree(dev->flow_db->rdma_transport_tx[j]);
3491free_rdma_transport_rx:3496free_rdma_transport_rx:
3492 kfree(dev->flow_db->rdma_transport_rx);3497 while (i--)
3493free_flow_db:3498 kfree(dev->flow_db->rdma_transport_rx[i]);
3494 kfree(dev->flow_db);3499 kfree(dev->flow_db);
3495 return -ENOMEM;3500 return -ENOMEM;
3496}3501}
drivers/infiniband/hw/mlx5/fs.h+6−2
@@ -13,6 +13,8 @@ void mlx5_ib_fs_cleanup_anchor(struct mlx5_ib_dev *dev); · 12 unmodified lines
13 13
14static inline void mlx5_ib_fs_cleanup(struct mlx5_ib_dev *dev)14static inline void mlx5_ib_fs_cleanup(struct mlx5_ib_dev *dev)
15{15{
16 int i;
17
16 /* When a steering anchor is created, a special flow table is also18 /* When a steering anchor is created, a special flow table is also
17 * created for the user to reference. Since the user can reference it,19 * created for the user to reference. Since the user can reference it,
18 * the kernel cannot trust that when the user destroys the steering20 * the kernel cannot trust that when the user destroys the steering
@@ -25,8 +27,10 @@ static inline void mlx5_ib_fs_cleanup(struct mlx5_ib_dev *dev) · 6 unmodified lines
25 * is a safe assumption that all references are gone.27 * is a safe assumption that all references are gone.
26 */28 */
27 mlx5_ib_fs_cleanup_anchor(dev);29 mlx5_ib_fs_cleanup_anchor(dev);
28 kfree(dev->flow_db->rdma_transport_tx);30 for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++)
29 kfree(dev->flow_db->rdma_transport_rx);31 kfree(dev->flow_db->rdma_transport_tx[i]);
32 for (i = 0; i < MLX5_RDMA_TRANSPORT_BYPASS_PRIO; i++)
33 kfree(dev->flow_db->rdma_transport_rx[i]);
30 kfree(dev->flow_db);34 kfree(dev->flow_db);
31}35}
32#endif /* _MLX5_IB_FS_H */36#endif /* _MLX5_IB_FS_H */
drivers/infiniband/hw/mlx5/ib_rep.c+2−1
@@ -88,7 +88,8 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) · 87 unmodified lines
88 else88 else
89 return mlx5_ib_set_vport_rep(lag_master, rep, vport_index);89 return mlx5_ib_set_vport_rep(lag_master, rep, vport_index);
90 90
91 ibdev = ib_alloc_device(mlx5_ib_dev, ib_dev);91 ibdev = ib_alloc_device_with_net(mlx5_ib_dev, ib_dev,
92 mlx5_core_net(lag_master));
92 if (!ibdev)93 if (!ibdev)
93 return -ENOMEM;94 return -ENOMEM;
94 95
drivers/infiniband/hw/mlx5/main.c+11−2
@@ -50,6 +50,7 @@ · 49 unmodified lines
50#include <rdma/ib_ucaps.h>50#include <rdma/ib_ucaps.h>
51#include "macsec.h"51#include "macsec.h"
52#include "data_direct.h"52#include "data_direct.h"
53#include "dmah.h"
53 54
54#define UVERBS_MODULE_NAME mlx5_ib55#define UVERBS_MODULE_NAME mlx5_ib
55#include <rdma/uverbs_named_ioctl.h>56#include <rdma/uverbs_named_ioctl.h>
@@ -4190,7 +4191,9 @@ static const struct ib_device_ops mlx5_ib_dev_ops = { · 4134 unmodified lines
4190 .modify_port = mlx5_ib_modify_port,4191 .modify_port = mlx5_ib_modify_port,
4191 .modify_qp = mlx5_ib_modify_qp,4192 .modify_qp = mlx5_ib_modify_qp,
4192 .modify_srq = mlx5_ib_modify_srq,4193 .modify_srq = mlx5_ib_modify_srq,
4194 .pre_destroy_cq = mlx5_ib_pre_destroy_cq,
4193 .poll_cq = mlx5_ib_poll_cq,4195 .poll_cq = mlx5_ib_poll_cq,
4196 .post_destroy_cq = mlx5_ib_post_destroy_cq,
4194 .post_recv = mlx5_ib_post_recv_nodrain,4197 .post_recv = mlx5_ib_post_recv_nodrain,
4195 .post_send = mlx5_ib_post_send_nodrain,4198 .post_send = mlx5_ib_post_send_nodrain,
4196 .post_srq_recv = mlx5_ib_post_srq_recv,4199 .post_srq_recv = mlx5_ib_post_srq_recv,
@@ -4212,6 +4215,7 @@ static const struct ib_device_ops mlx5_ib_dev_ops = { · 15 unmodified lines
4212 INIT_RDMA_OBJ_SIZE(ib_ah, mlx5_ib_ah, ibah),4215 INIT_RDMA_OBJ_SIZE(ib_ah, mlx5_ib_ah, ibah),
4213 INIT_RDMA_OBJ_SIZE(ib_counters, mlx5_ib_mcounters, ibcntrs),4216 INIT_RDMA_OBJ_SIZE(ib_counters, mlx5_ib_mcounters, ibcntrs),
4214 INIT_RDMA_OBJ_SIZE(ib_cq, mlx5_ib_cq, ibcq),4217 INIT_RDMA_OBJ_SIZE(ib_cq, mlx5_ib_cq, ibcq),
4218 INIT_RDMA_OBJ_SIZE(ib_dmah, mlx5_ib_dmah, ibdmah),
4215 INIT_RDMA_OBJ_SIZE(ib_pd, mlx5_ib_pd, ibpd),4219 INIT_RDMA_OBJ_SIZE(ib_pd, mlx5_ib_pd, ibpd),
4216 INIT_RDMA_OBJ_SIZE(ib_qp, mlx5_ib_qp, ibqp),4220 INIT_RDMA_OBJ_SIZE(ib_qp, mlx5_ib_qp, ibqp),
4217 INIT_RDMA_OBJ_SIZE(ib_srq, mlx5_ib_srq, ibsrq),4221 INIT_RDMA_OBJ_SIZE(ib_srq, mlx5_ib_srq, ibsrq),
@@ -4339,6 +4343,9 @@ static int mlx5_ib_stage_caps_init(struct mlx5_ib_dev *dev) · 121 unmodified lines
4339 MLX5_GENERAL_OBJ_TYPES_CAP_SW_ICM)4343 MLX5_GENERAL_OBJ_TYPES_CAP_SW_ICM)
4340 ib_set_device_ops(&dev->ib_dev, &mlx5_ib_dev_dm_ops);4344 ib_set_device_ops(&dev->ib_dev, &mlx5_ib_dev_dm_ops);
4341 4345
4346 if (mdev->st)
4347 ib_set_device_ops(&dev->ib_dev, &mlx5_ib_dev_dmah_ops);
4348
4342 ib_set_device_ops(&dev->ib_dev, &mlx5_ib_dev_ops);4349 ib_set_device_ops(&dev->ib_dev, &mlx5_ib_dev_ops);
4343 4350
4344 if (IS_ENABLED(CONFIG_INFINIBAND_USER_ACCESS))4351 if (IS_ENABLED(CONFIG_INFINIBAND_USER_ACCESS))
@@ -4824,7 +4831,8 @@ static struct ib_device *mlx5_ib_add_sub_dev(struct ib_device *parent, · 479 unmodified lines
4824 !MLX5_CAP_GEN_2(mparent->mdev, multiplane_qp_ud))4831 !MLX5_CAP_GEN_2(mparent->mdev, multiplane_qp_ud))
4825 return ERR_PTR(-EOPNOTSUPP);4832 return ERR_PTR(-EOPNOTSUPP);
4826 4833
4827 mplane = ib_alloc_device(mlx5_ib_dev, ib_dev);4834 mplane = ib_alloc_device_with_net(mlx5_ib_dev, ib_dev,
4835 mlx5_core_net(mparent->mdev));
4828 if (!mplane)4836 if (!mplane)
4829 return ERR_PTR(-ENOMEM);4837 return ERR_PTR(-ENOMEM);
4830 4838
@@ -4938,7 +4946,8 @@ static int mlx5r_probe(struct auxiliary_device *adev, · 107 unmodified lines
4938 4946
4939 num_ports = max(MLX5_CAP_GEN(mdev, num_ports),4947 num_ports = max(MLX5_CAP_GEN(mdev, num_ports),
4940 MLX5_CAP_GEN(mdev, num_vhca_ports));4948 MLX5_CAP_GEN(mdev, num_vhca_ports));
4941 dev = ib_alloc_device(mlx5_ib_dev, ib_dev);4949 dev = ib_alloc_device_with_net(mlx5_ib_dev, ib_dev,
4950 mlx5_core_net(mdev));
4942 if (!dev)4951 if (!dev)
4943 return -ENOMEM;4952 return -ENOMEM;
4944 4953
drivers/infiniband/hw/mlx5/mlx5_ib.h+75−24
@@ -104,19 +104,6 @@ unsigned long __mlx5_umem_find_best_quantized_pgoff( · 103 unmodified lines
104 __mlx5_bit_sz(typ, page_offset_fld), 0, scale, \104 __mlx5_bit_sz(typ, page_offset_fld), 0, scale, \
105 page_offset_quantized)105 page_offset_quantized)
106 106
107static inline unsigned long
108mlx5_umem_dmabuf_find_best_pgsz(struct ib_umem_dmabuf *umem_dmabuf)
109{
110 /*
111 * mkeys used for dmabuf are fixed at PAGE_SIZE because we must be able
112 * to hold any sgl after a move operation. Ideally the mkc page size
113 * could be changed at runtime to be optimal, but right now the driver
114 * cannot do that.
115 */
116 return ib_umem_find_best_pgsz(&umem_dmabuf->umem, PAGE_SIZE,
117 umem_dmabuf->umem.iova);
118}
119
120enum {107enum {
121 MLX5_IB_MMAP_OFFSET_START = 9,108 MLX5_IB_MMAP_OFFSET_START = 9,
122 MLX5_IB_MMAP_OFFSET_END = 255,109 MLX5_IB_MMAP_OFFSET_END = 255,
@@ -320,8 +307,8 @@ struct mlx5_ib_flow_db { · 197 unmodified lines
320 struct mlx5_ib_flow_prio rdma_tx[MLX5_IB_NUM_FLOW_FT];307 struct mlx5_ib_flow_prio rdma_tx[MLX5_IB_NUM_FLOW_FT];
321 struct mlx5_ib_flow_prio opfcs[MLX5_IB_OPCOUNTER_MAX];308 struct mlx5_ib_flow_prio opfcs[MLX5_IB_OPCOUNTER_MAX];
322 struct mlx5_flow_table *lag_demux_ft;309 struct mlx5_flow_table *lag_demux_ft;
323 struct mlx5_ib_flow_prio *rdma_transport_rx;310 struct mlx5_ib_flow_prio *rdma_transport_rx[MLX5_RDMA_TRANSPORT_BYPASS_PRIO];
324 struct mlx5_ib_flow_prio *rdma_transport_tx;311 struct mlx5_ib_flow_prio *rdma_transport_tx[MLX5_RDMA_TRANSPORT_BYPASS_PRIO];
325 /* Protect flow steering bypass flow tables312 /* Protect flow steering bypass flow tables
326 * when add/del flow rules.313 * when add/del flow rules.
327 * only single add/removal of flow steering rule could be done314 * only single add/removal of flow steering rule could be done
@@ -352,6 +339,7 @@ struct mlx5_ib_flow_db { · 24 unmodified lines
352#define MLX5_IB_UPD_XLT_ACCESS BIT(5)339#define MLX5_IB_UPD_XLT_ACCESS BIT(5)
353#define MLX5_IB_UPD_XLT_INDIRECT BIT(6)340#define MLX5_IB_UPD_XLT_INDIRECT BIT(6)
354#define MLX5_IB_UPD_XLT_DOWNGRADE BIT(7)341#define MLX5_IB_UPD_XLT_DOWNGRADE BIT(7)
342#define MLX5_IB_UPD_XLT_KEEP_PGSZ BIT(8)
355 343
356/* Private QP creation flags to be passed in ib_qp_init_attr.create_flags.344/* Private QP creation flags to be passed in ib_qp_init_attr.create_flags.
357 *345 *
@@ -650,8 +638,13 @@ enum mlx5_mkey_type { · 292 unmodified lines
650 MLX5_MKEY_IMPLICIT_CHILD,638 MLX5_MKEY_IMPLICIT_CHILD,
651};639};
652 640
641/* Used for non-existent ph value */
642#define MLX5_IB_NO_PH 0xff
643
653struct mlx5r_cache_rb_key {644struct mlx5r_cache_rb_key {
654 u8 ats:1;645 u8 ats:1;
646 u8 ph;
647 u16 st_index;
655 unsigned int access_mode;648 unsigned int access_mode;
656 unsigned int access_flags;649 unsigned int access_flags;
657 unsigned int ndescs;650 unsigned int ndescs;
@@ -739,6 +732,8 @@ struct mlx5_ib_mr { · 81 unmodified lines
739 struct mlx5_ib_mr *dd_crossed_mr;732 struct mlx5_ib_mr *dd_crossed_mr;
740 struct list_head dd_node;733 struct list_head dd_node;
741 u8 revoked :1;734 u8 revoked :1;
735 /* Indicates previous dmabuf page fault occurred */
736 u8 dmabuf_faulted:1;
742 struct mlx5_ib_mkey null_mmkey;737 struct mlx5_ib_mkey null_mmkey;
743 };738 };
744 };739 };
@@ -899,13 +894,14 @@ void mlx5_ib_fs_remove_op_fc(struct mlx5_ib_dev *dev, · 154 unmodified lines
899 struct mlx5_ib_op_fc *opfc,894 struct mlx5_ib_op_fc *opfc,
900 enum mlx5_ib_optional_counter_type type);895 enum mlx5_ib_optional_counter_type type);
901 896
902int mlx5r_fs_bind_op_fc(struct ib_qp *qp, struct rdma_counter *counter,897int mlx5r_fs_bind_op_fc(struct ib_qp *qp,
903 u32 port);898 struct mlx5_fc *fc_arr[MLX5_IB_OPCOUNTER_MAX],
899 struct xarray *qpn_opfc_xa, u32 port);
904 900
905void mlx5r_fs_unbind_op_fc(struct ib_qp *qp, struct rdma_counter *counter);901void mlx5r_fs_unbind_op_fc(struct ib_qp *qp, struct xarray *qpn_opfc_xa);
906 902
907void mlx5r_fs_destroy_fcs(struct mlx5_ib_dev *dev,903void mlx5r_fs_destroy_fcs(struct mlx5_ib_dev *dev,
908 struct rdma_counter *counter);904 struct mlx5_fc *fc_arr[MLX5_IB_OPCOUNTER_MAX]);
909 905
910struct mlx5_ib_multiport_info;906struct mlx5_ib_multiport_info;
911 907
@@ -1372,16 +1368,20 @@ int mlx5_ib_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr, · 460 unmodified lines
1372 struct uverbs_attr_bundle *attrs);1368 struct uverbs_attr_bundle *attrs);
1373int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);1369int mlx5_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);
1374int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);1370int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
1371int mlx5_ib_pre_destroy_cq(struct ib_cq *cq);
1372void mlx5_ib_post_destroy_cq(struct ib_cq *cq);
1375int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);1373int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
1376int mlx5_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period);1374int mlx5_ib_modify_cq(struct ib_cq *cq, u16 cq_count, u16 cq_period);
1377int mlx5_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata);1375int mlx5_ib_resize_cq(struct ib_cq *ibcq, int entries, struct ib_udata *udata);
1378struct ib_mr *mlx5_ib_get_dma_mr(struct ib_pd *pd, int acc);1376struct ib_mr *mlx5_ib_get_dma_mr(struct ib_pd *pd, int acc);
1379struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,1377struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
1380 u64 virt_addr, int access_flags,1378 u64 virt_addr, int access_flags,
1379 struct ib_dmah *dmah,
1381 struct ib_udata *udata);1380 struct ib_udata *udata);
1382struct ib_mr *mlx5_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 start,1381struct ib_mr *mlx5_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 start,
1383 u64 length, u64 virt_addr,1382 u64 length, u64 virt_addr,
1384 int fd, int access_flags,1383 int fd, int access_flags,
1384 struct ib_dmah *dmah,
1385 struct uverbs_attr_bundle *attrs);1385 struct uverbs_attr_bundle *attrs);
1386int mlx5_ib_advise_mr(struct ib_pd *pd,1386int mlx5_ib_advise_mr(struct ib_pd *pd,
1387 enum ib_uverbs_advise_mr_advice advice,1387 enum ib_uverbs_advise_mr_advice advice,
@@ -1748,20 +1748,71 @@ static inline u32 smi_to_native_portnum(struct mlx5_ib_dev *dev, u32 port) · 360 unmodified lines
1748 return (port - 1) / dev->num_ports + 1;1748 return (port - 1) / dev->num_ports + 1;
1749}1749}
1750 1750
1751static inline unsigned int get_max_log_entity_size_cap(struct mlx5_ib_dev *dev,
1752 int access_mode)
1753{
1754 int max_log_size = 0;
1755
1756 if (access_mode == MLX5_MKC_ACCESS_MODE_MTT)
1757 max_log_size =
1758 MLX5_CAP_GEN_2(dev->mdev, max_mkey_log_entity_size_mtt);
1759 else if (access_mode == MLX5_MKC_ACCESS_MODE_KSM)
1760 max_log_size = MLX5_CAP_GEN_2(
1761 dev->mdev, max_mkey_log_entity_size_fixed_buffer);
1762
1763 if (!max_log_size ||
1764 (max_log_size > 31 &&
1765 !MLX5_CAP_GEN_2(dev->mdev, umr_log_entity_size_5)))
1766 max_log_size = 31;
1767
1768 return max_log_size;
1769}
1770
1771static inline unsigned int get_min_log_entity_size_cap(struct mlx5_ib_dev *dev,
1772 int access_mode)
1773{
1774 int min_log_size = 0;
1775
1776 if (access_mode == MLX5_MKC_ACCESS_MODE_KSM &&
1777 MLX5_CAP_GEN_2(dev->mdev,
1778 min_mkey_log_entity_size_fixed_buffer_valid))
1779 min_log_size = MLX5_CAP_GEN_2(
1780 dev->mdev, min_mkey_log_entity_size_fixed_buffer);
1781 else
1782 min_log_size =
1783 MLX5_CAP_GEN_2(dev->mdev, log_min_mkey_entity_size);
1784
1785 min_log_size = max(min_log_size, MLX5_ADAPTER_PAGE_SHIFT);
1786 return min_log_size;
1787}
1788
1751/*1789/*
1752 * For mkc users, instead of a page_offset the command has a start_iova which1790 * For mkc users, instead of a page_offset the command has a start_iova which
1753 * specifies both the page_offset and the on-the-wire IOVA1791 * specifies both the page_offset and the on-the-wire IOVA
1754 */1792 */
1755static __always_inline unsigned long1793static __always_inline unsigned long
1756mlx5_umem_mkc_find_best_pgsz(struct mlx5_ib_dev *dev, struct ib_umem *umem,1794mlx5_umem_mkc_find_best_pgsz(struct mlx5_ib_dev *dev, struct ib_umem *umem,
1757 u64 iova)1795 u64 iova, int access_mode)
1758{1796{
1759 int page_size_bits =1797 unsigned int max_log_entity_size_cap, min_log_entity_size_cap;
1760 MLX5_CAP_GEN_2(dev->mdev, umr_log_entity_size_5) ? 6 : 5;1798 unsigned long bitmap;
1761 unsigned long bitmap =1799
1762 __mlx5_log_page_size_to_bitmap(page_size_bits, 0);1800 max_log_entity_size_cap = get_max_log_entity_size_cap(dev, access_mode);
1801 min_log_entity_size_cap = get_min_log_entity_size_cap(dev, access_mode);
1802
1803 bitmap = GENMASK_ULL(max_log_entity_size_cap, min_log_entity_size_cap);
1763 1804
1764 return ib_umem_find_best_pgsz(umem, bitmap, iova);1805 return ib_umem_find_best_pgsz(umem, bitmap, iova);
1765}1806}
1766 1807
1808static inline unsigned long
1809mlx5_umem_dmabuf_find_best_pgsz(struct ib_umem_dmabuf *umem_dmabuf,
1810 int access_mode)
1811{
1812 return mlx5_umem_mkc_find_best_pgsz(to_mdev(umem_dmabuf->umem.ibdev),
1813 &umem_dmabuf->umem,
1814 umem_dmabuf->umem.iova,
1815 access_mode);
1816}
1817
1767#endif /* MLX5_IB_H */1818#endif /* MLX5_IB_H */
drivers/infiniband/hw/mlx5/mr.c+92−24
@@ -44,6 +44,7 @@ · 43 unmodified lines
44#include "mlx5_ib.h"44#include "mlx5_ib.h"
45#include "umr.h"45#include "umr.h"
46#include "data_direct.h"46#include "data_direct.h"
47#include "dmah.h"
47 48
48enum {49enum {
49 MAX_PENDING_REG_MR = 8,50 MAX_PENDING_REG_MR = 8,
@@ -57,7 +58,7 @@ create_mkey_callback(int status, struct mlx5_async_work *context); · 7 unmodified lines
57static struct mlx5_ib_mr *reg_create(struct ib_pd *pd, struct ib_umem *umem,58static struct mlx5_ib_mr *reg_create(struct ib_pd *pd, struct ib_umem *umem,
58 u64 iova, int access_flags,59 u64 iova, int access_flags,
59 unsigned long page_size, bool populate,60 unsigned long page_size, bool populate,
60 int access_mode);61 int access_mode, u16 st_index, u8 ph);
61static int __mlx5_ib_dereg_mr(struct ib_mr *ibmr);62static int __mlx5_ib_dereg_mr(struct ib_mr *ibmr);
62 63
63static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,64static void set_mkc_access_pd_addr_fields(void *mkc, int acc, u64 start_addr,
@@ -256,6 +257,14 @@ static void set_cache_mkc(struct mlx5_cache_ent *ent, void *mkc) · 192 unmodified lines
256 get_mkc_octo_size(ent->rb_key.access_mode,257 get_mkc_octo_size(ent->rb_key.access_mode,
257 ent->rb_key.ndescs));258 ent->rb_key.ndescs));
258 MLX5_SET(mkc, mkc, log_page_size, PAGE_SHIFT);259 MLX5_SET(mkc, mkc, log_page_size, PAGE_SHIFT);
260
261 if (ent->rb_key.ph != MLX5_IB_NO_PH) {
262 MLX5_SET(mkc, mkc, pcie_tph_en, 1);
263 MLX5_SET(mkc, mkc, pcie_tph_ph, ent->rb_key.ph);
264 if (ent->rb_key.st_index != MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX)
265 MLX5_SET(mkc, mkc, pcie_tph_steering_tag_index,
266 ent->rb_key.st_index);
267 }
259}268}
260 269
261/* Asynchronously schedule new MRs to be populated in the cache. */270/* Asynchronously schedule new MRs to be populated in the cache. */
@@ -641,6 +650,14 @@ static int cache_ent_key_cmp(struct mlx5r_cache_rb_key key1, · 379 unmodified lines
641 if (res)650 if (res)
642 return res;651 return res;
643 652
653 res = key1.st_index - key2.st_index;
654 if (res)
655 return res;
656
657 res = key1.ph - key2.ph;
658 if (res)
659 return res;
660
644 /*661 /*
645 * keep ndescs the last in the compare table since the find function662 * keep ndescs the last in the compare table since the find function
646 * searches for an exact match on all properties and only closest663 * searches for an exact match on all properties and only closest
@@ -712,6 +729,8 @@ mkey_cache_ent_from_rb_key(struct mlx5_ib_dev *dev, · 65 unmodified lines
712 smallest->rb_key.access_mode == rb_key.access_mode &&729 smallest->rb_key.access_mode == rb_key.access_mode &&
713 smallest->rb_key.access_flags == rb_key.access_flags &&730 smallest->rb_key.access_flags == rb_key.access_flags &&
714 smallest->rb_key.ats == rb_key.ats &&731 smallest->rb_key.ats == rb_key.ats &&
732 smallest->rb_key.st_index == rb_key.st_index &&
733 smallest->rb_key.ph == rb_key.ph &&
715 smallest->rb_key.ndescs <= ndescs_limit) ?734 smallest->rb_key.ndescs <= ndescs_limit) ?
716 smallest :735 smallest :
717 NULL;736 NULL;
@@ -786,7 +805,8 @@ struct mlx5_ib_mr *mlx5_mr_cache_alloc(struct mlx5_ib_dev *dev, · 68 unmodified lines
786 struct mlx5r_cache_rb_key rb_key = {805 struct mlx5r_cache_rb_key rb_key = {
787 .ndescs = ndescs,806 .ndescs = ndescs,
788 .access_mode = access_mode,807 .access_mode = access_mode,
789 .access_flags = get_unchangeable_access_flags(dev, access_flags)808 .access_flags = get_unchangeable_access_flags(dev, access_flags),
809 .ph = MLX5_IB_NO_PH,
790 };810 };
791 struct mlx5_cache_ent *ent = mkey_cache_ent_from_rb_key(dev, rb_key);811 struct mlx5_cache_ent *ent = mkey_cache_ent_from_rb_key(dev, rb_key);
792 812
@@ -943,6 +963,7 @@ int mlx5_mkey_cache_init(struct mlx5_ib_dev *dev) · 150 unmodified lines
943 struct rb_root *root = &dev->cache.rb_root;963 struct rb_root *root = &dev->cache.rb_root;
944 struct mlx5r_cache_rb_key rb_key = {964 struct mlx5r_cache_rb_key rb_key = {
945 .access_mode = MLX5_MKC_ACCESS_MODE_MTT,965 .access_mode = MLX5_MKC_ACCESS_MODE_MTT,
966 .ph = MLX5_IB_NO_PH,
946 };967 };
947 struct mlx5_cache_ent *ent;968 struct mlx5_cache_ent *ent;
948 struct rb_node *node;969 struct rb_node *node;
@@ -1119,7 +1140,8 @@ static unsigned int mlx5_umem_dmabuf_default_pgsz(struct ib_umem *umem, · 170 unmodified lines
1119 1140
1120static struct mlx5_ib_mr *alloc_cacheable_mr(struct ib_pd *pd,1141static struct mlx5_ib_mr *alloc_cacheable_mr(struct ib_pd *pd,
1121 struct ib_umem *umem, u64 iova,1142 struct ib_umem *umem, u64 iova,
1122 int access_flags, int access_mode)1143 int access_flags, int access_mode,
1144 u16 st_index, u8 ph)
1123{1145{
1124 struct mlx5_ib_dev *dev = to_mdev(pd->device);1146 struct mlx5_ib_dev *dev = to_mdev(pd->device);
1125 struct mlx5r_cache_rb_key rb_key = {};1147 struct mlx5r_cache_rb_key rb_key = {};
@@ -1130,7 +1152,8 @@ static struct mlx5_ib_mr *alloc_cacheable_mr(struct ib_pd *pd, · 4 unmodified lines
1130 if (umem->is_dmabuf)1152 if (umem->is_dmabuf)
1131 page_size = mlx5_umem_dmabuf_default_pgsz(umem, iova);1153 page_size = mlx5_umem_dmabuf_default_pgsz(umem, iova);
1132 else1154 else
1133 page_size = mlx5_umem_mkc_find_best_pgsz(dev, umem, iova);1155 page_size = mlx5_umem_mkc_find_best_pgsz(dev, umem, iova,
1156 access_mode);
1134 if (WARN_ON(!page_size))1157 if (WARN_ON(!page_size))
1135 return ERR_PTR(-EINVAL);1158 return ERR_PTR(-EINVAL);
1136 1159
@@ -1138,6 +1161,8 @@ static struct mlx5_ib_mr *alloc_cacheable_mr(struct ib_pd *pd, · 1 unmodified line
1138 rb_key.ndescs = ib_umem_num_dma_blocks(umem, page_size);1161 rb_key.ndescs = ib_umem_num_dma_blocks(umem, page_size);
1139 rb_key.ats = mlx5_umem_needs_ats(dev, umem, access_flags);1162 rb_key.ats = mlx5_umem_needs_ats(dev, umem, access_flags);
1140 rb_key.access_flags = get_unchangeable_access_flags(dev, access_flags);1163 rb_key.access_flags = get_unchangeable_access_flags(dev, access_flags);
1164 rb_key.st_index = st_index;
1165 rb_key.ph = ph;
1141 ent = mkey_cache_ent_from_rb_key(dev, rb_key);1166 ent = mkey_cache_ent_from_rb_key(dev, rb_key);
1142 /*1167 /*
1143 * If the MR can't come from the cache then synchronously create an uncached1168 * If the MR can't come from the cache then synchronously create an uncached
@@ -1145,7 +1170,8 @@ static struct mlx5_ib_mr *alloc_cacheable_mr(struct ib_pd *pd, · 1 unmodified line
1145 */1170 */
1146 if (!ent) {1171 if (!ent) {
1147 mutex_lock(&dev->slow_path_mutex);1172 mutex_lock(&dev->slow_path_mutex);
1148 mr = reg_create(pd, umem, iova, access_flags, page_size, false, access_mode);1173 mr = reg_create(pd, umem, iova, access_flags, page_size, false, access_mode,
1174 st_index, ph);
1149 mutex_unlock(&dev->slow_path_mutex);1175 mutex_unlock(&dev->slow_path_mutex);
1150 if (IS_ERR(mr))1176 if (IS_ERR(mr))
1151 return mr;1177 return mr;
@@ -1230,7 +1256,7 @@ err_1: · 78 unmodified lines
1230static struct mlx5_ib_mr *reg_create(struct ib_pd *pd, struct ib_umem *umem,1256static struct mlx5_ib_mr *reg_create(struct ib_pd *pd, struct ib_umem *umem,
1231 u64 iova, int access_flags,1257 u64 iova, int access_flags,
1232 unsigned long page_size, bool populate,1258 unsigned long page_size, bool populate,
1233 int access_mode)1259 int access_mode, u16 st_index, u8 ph)
1234{1260{
1235 struct mlx5_ib_dev *dev = to_mdev(pd->device);1261 struct mlx5_ib_dev *dev = to_mdev(pd->device);
1236 struct mlx5_ib_mr *mr;1262 struct mlx5_ib_mr *mr;
@@ -1240,7 +1266,8 @@ static struct mlx5_ib_mr *reg_create(struct ib_pd *pd, struct ib_umem *umem, · 3 unmodified lines
1240 u32 *in;1266 u32 *in;
1241 int err;1267 int err;
1242 bool pg_cap = !!(MLX5_CAP_GEN(dev->mdev, pg)) &&1268 bool pg_cap = !!(MLX5_CAP_GEN(dev->mdev, pg)) &&
1243 (access_mode == MLX5_MKC_ACCESS_MODE_MTT);1269 (access_mode == MLX5_MKC_ACCESS_MODE_MTT) &&
1270 (ph == MLX5_IB_NO_PH);
1244 bool ksm_mode = (access_mode == MLX5_MKC_ACCESS_MODE_KSM);1271 bool ksm_mode = (access_mode == MLX5_MKC_ACCESS_MODE_KSM);
1245 1272
1246 if (!page_size)1273 if (!page_size)
@@ -1304,6 +1331,13 @@ static struct mlx5_ib_mr *reg_create(struct ib_pd *pd, struct ib_umem *umem, · 57 unmodified lines
1304 get_octo_len(iova, umem->length, mr->page_shift));1331 get_octo_len(iova, umem->length, mr->page_shift));
1305 }1332 }
1306 1333
1334 if (ph != MLX5_IB_NO_PH) {
1335 MLX5_SET(mkc, mkc, pcie_tph_en, 1);
1336 MLX5_SET(mkc, mkc, pcie_tph_ph, ph);
1337 if (st_index != MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX)
1338 MLX5_SET(mkc, mkc, pcie_tph_steering_tag_index, st_index);
1339 }
1340
1307 err = mlx5_ib_create_mkey(dev, &mr->mmkey, in, inlen);1341 err = mlx5_ib_create_mkey(dev, &mr->mmkey, in, inlen);
1308 if (err) {1342 if (err) {
1309 mlx5_ib_warn(dev, "create mkey failed\n");1343 mlx5_ib_warn(dev, "create mkey failed\n");
@@ -1423,24 +1457,37 @@ struct ib_mr *mlx5_ib_reg_dm_mr(struct ib_pd *pd, struct ib_dm *dm, · 113 unmodified lines
1423}1457}
1424 1458
1425static struct ib_mr *create_real_mr(struct ib_pd *pd, struct ib_umem *umem,1459static struct ib_mr *create_real_mr(struct ib_pd *pd, struct ib_umem *umem,
1426 u64 iova, int access_flags)1460 u64 iova, int access_flags,
1461 struct ib_dmah *dmah)
1427{1462{
1428 struct mlx5_ib_dev *dev = to_mdev(pd->device);1463 struct mlx5_ib_dev *dev = to_mdev(pd->device);
1429 struct mlx5_ib_mr *mr = NULL;1464 struct mlx5_ib_mr *mr = NULL;
1430 bool xlt_with_umr;1465 bool xlt_with_umr;
1466 u16 st_index = MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX;
1467 u8 ph = MLX5_IB_NO_PH;
1431 int err;1468 int err;
1432 1469
1470 if (dmah) {
1471 struct mlx5_ib_dmah *mdmah = to_mdmah(dmah);
1472
1473 ph = dmah->ph;
1474 if (dmah->valid_fields & BIT(IB_DMAH_CPU_ID_EXISTS))
1475 st_index = mdmah->st_index;
1476 }
1477
1433 xlt_with_umr = mlx5r_umr_can_load_pas(dev, umem->length);1478 xlt_with_umr = mlx5r_umr_can_load_pas(dev, umem->length);
1434 if (xlt_with_umr) {1479 if (xlt_with_umr) {
1435 mr = alloc_cacheable_mr(pd, umem, iova, access_flags,1480 mr = alloc_cacheable_mr(pd, umem, iova, access_flags,
1436 MLX5_MKC_ACCESS_MODE_MTT);1481 MLX5_MKC_ACCESS_MODE_MTT,
1482 st_index, ph);
1437 } else {1483 } else {
1438 unsigned long page_size =1484 unsigned long page_size = mlx5_umem_mkc_find_best_pgsz(
1439 mlx5_umem_mkc_find_best_pgsz(dev, umem, iova);1485 dev, umem, iova, MLX5_MKC_ACCESS_MODE_MTT);
1440 1486
1441 mutex_lock(&dev->slow_path_mutex);1487 mutex_lock(&dev->slow_path_mutex);
1442 mr = reg_create(pd, umem, iova, access_flags, page_size,1488 mr = reg_create(pd, umem, iova, access_flags, page_size,
1443 true, MLX5_MKC_ACCESS_MODE_MTT);1489 true, MLX5_MKC_ACCESS_MODE_MTT,
1490 st_index, ph);
1444 mutex_unlock(&dev->slow_path_mutex);1491 mutex_unlock(&dev->slow_path_mutex);
1445 }1492 }
1446 if (IS_ERR(mr)) {1493 if (IS_ERR(mr)) {
@@ -1504,7 +1551,9 @@ static struct ib_mr *create_user_odp_mr(struct ib_pd *pd, u64 start, u64 length, · 57 unmodified lines
1504 return ERR_CAST(odp);1551 return ERR_CAST(odp);
1505 1552
1506 mr = alloc_cacheable_mr(pd, &odp->umem, iova, access_flags,1553 mr = alloc_cacheable_mr(pd, &odp->umem, iova, access_flags,
1507 MLX5_MKC_ACCESS_MODE_MTT);1554 MLX5_MKC_ACCESS_MODE_MTT,
1555 MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX,
1556 MLX5_IB_NO_PH);
1508 if (IS_ERR(mr)) {1557 if (IS_ERR(mr)) {
1509 ib_umem_release(&odp->umem);1558 ib_umem_release(&odp->umem);
1510 return ERR_CAST(mr);1559 return ERR_CAST(mr);
@@ -1528,13 +1577,15 @@ err_dereg_mr: · 17 unmodified lines
1528 1577
1529struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,1578struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
1530 u64 iova, int access_flags,1579 u64 iova, int access_flags,
1580 struct ib_dmah *dmah,
1531 struct ib_udata *udata)1581 struct ib_udata *udata)
1532{1582{
1533 struct mlx5_ib_dev *dev = to_mdev(pd->device);1583 struct mlx5_ib_dev *dev = to_mdev(pd->device);
1534 struct ib_umem *umem;1584 struct ib_umem *umem;
1535 int err;1585 int err;
1536 1586
1537 if (!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM))1587 if (!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM) ||
1588 ((access_flags & IB_ACCESS_ON_DEMAND) && dmah))
1538 return ERR_PTR(-EOPNOTSUPP);1589 return ERR_PTR(-EOPNOTSUPP);
1539 1590
1540 mlx5_ib_dbg(dev, "start 0x%llx, iova 0x%llx, length 0x%llx, access_flags 0x%x\n",1591 mlx5_ib_dbg(dev, "start 0x%llx, iova 0x%llx, length 0x%llx, access_flags 0x%x\n",
@@ -1550,7 +1601,7 @@ struct ib_mr *mlx5_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, · 9 unmodified lines
1550 umem = ib_umem_get(&dev->ib_dev, start, length, access_flags);1601 umem = ib_umem_get(&dev->ib_dev, start, length, access_flags);
1551 if (IS_ERR(umem))1602 if (IS_ERR(umem))
1552 return ERR_CAST(umem);1603 return ERR_CAST(umem);
1553 return create_real_mr(pd, umem, iova, access_flags);1604 return create_real_mr(pd, umem, iova, access_flags, dmah);
1554}1605}
1555 1606
1556static void mlx5_ib_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)1607static void mlx5_ib_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)
@@ -1575,12 +1626,15 @@ static struct dma_buf_attach_ops mlx5_ib_dmabuf_attach_ops = { · 18 unmodified lines
1575static struct ib_mr *1626static struct ib_mr *
1576reg_user_mr_dmabuf(struct ib_pd *pd, struct device *dma_device,1627reg_user_mr_dmabuf(struct ib_pd *pd, struct device *dma_device,
1577 u64 offset, u64 length, u64 virt_addr,1628 u64 offset, u64 length, u64 virt_addr,
1578 int fd, int access_flags, int access_mode)1629 int fd, int access_flags, int access_mode,
1630 struct ib_dmah *dmah)
1579{1631{
1580 bool pinned_mode = (access_mode == MLX5_MKC_ACCESS_MODE_KSM);1632 bool pinned_mode = (access_mode == MLX5_MKC_ACCESS_MODE_KSM);
1581 struct mlx5_ib_dev *dev = to_mdev(pd->device);1633 struct mlx5_ib_dev *dev = to_mdev(pd->device);
1582 struct mlx5_ib_mr *mr = NULL;1634 struct mlx5_ib_mr *mr = NULL;
1583 struct ib_umem_dmabuf *umem_dmabuf;1635 struct ib_umem_dmabuf *umem_dmabuf;
1636 u16 st_index = MLX5_MKC_PCIE_TPH_NO_STEERING_TAG_INDEX;
1637 u8 ph = MLX5_IB_NO_PH;
1584 int err;1638 int err;
1585 1639
1586 err = mlx5r_umr_resource_init(dev);1640 err = mlx5r_umr_resource_init(dev);
@@ -1603,8 +1657,17 @@ reg_user_mr_dmabuf(struct ib_pd *pd, struct device *dma_device, · 16 unmodified lines
1603 return ERR_CAST(umem_dmabuf);1657 return ERR_CAST(umem_dmabuf);
1604 }1658 }
1605 1659
1660 if (dmah) {
1661 struct mlx5_ib_dmah *mdmah = to_mdmah(dmah);
1662
1663 ph = dmah->ph;
1664 if (dmah->valid_fields & BIT(IB_DMAH_CPU_ID_EXISTS))
1665 st_index = mdmah->st_index;
1666 }
1667
1606 mr = alloc_cacheable_mr(pd, &umem_dmabuf->umem, virt_addr,1668 mr = alloc_cacheable_mr(pd, &umem_dmabuf->umem, virt_addr,
1607 access_flags, access_mode);1669 access_flags, access_mode,
1670 st_index, ph);
1608 if (IS_ERR(mr)) {1671 if (IS_ERR(mr)) {
1609 ib_umem_release(&umem_dmabuf->umem);1672 ib_umem_release(&umem_dmabuf->umem);
1610 return ERR_CAST(mr);1673 return ERR_CAST(mr);
@@ -1661,7 +1724,8 @@ reg_user_mr_dmabuf_by_data_direct(struct ib_pd *pd, u64 offset, · 50 unmodified lines
1661 access_flags &= ~IB_ACCESS_RELAXED_ORDERING;1724 access_flags &= ~IB_ACCESS_RELAXED_ORDERING;
1662 crossed_mr = reg_user_mr_dmabuf(pd, &data_direct_dev->pdev->dev,1725 crossed_mr = reg_user_mr_dmabuf(pd, &data_direct_dev->pdev->dev,
1663 offset, length, virt_addr, fd,1726 offset, length, virt_addr, fd,
1664 access_flags, MLX5_MKC_ACCESS_MODE_KSM);1727 access_flags, MLX5_MKC_ACCESS_MODE_KSM,
1728 NULL);
1665 if (IS_ERR(crossed_mr)) {1729 if (IS_ERR(crossed_mr)) {
1666 ret = PTR_ERR(crossed_mr);1730 ret = PTR_ERR(crossed_mr);
1667 goto end;1731 goto end;
@@ -1688,6 +1752,7 @@ end: · 20 unmodified lines
1688struct ib_mr *mlx5_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,1752struct ib_mr *mlx5_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
1689 u64 length, u64 virt_addr,1753 u64 length, u64 virt_addr,
1690 int fd, int access_flags,1754 int fd, int access_flags,
1755 struct ib_dmah *dmah,
1691 struct uverbs_attr_bundle *attrs)1756 struct uverbs_attr_bundle *attrs)
1692{1757{
1693 struct mlx5_ib_dev *dev = to_mdev(pd->device);1758 struct mlx5_ib_dev *dev = to_mdev(pd->device);
@@ -1720,7 +1785,8 @@ struct ib_mr *mlx5_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset, · 26 unmodified lines
1720 1785
1721 return reg_user_mr_dmabuf(pd, pd->device->dma_device,1786 return reg_user_mr_dmabuf(pd, pd->device->dma_device,
1722 offset, length, virt_addr,1787 offset, length, virt_addr,
1723 fd, access_flags, MLX5_MKC_ACCESS_MODE_MTT);1788 fd, access_flags, MLX5_MKC_ACCESS_MODE_MTT,
1789 dmah);
1724}1790}
1725 1791
1726/*1792/*
@@ -1754,7 +1820,8 @@ static bool can_use_umr_rereg_pas(struct mlx5_ib_mr *mr, · 27 unmodified lines
1754 if (!mlx5r_umr_can_load_pas(dev, new_umem->length))1820 if (!mlx5r_umr_can_load_pas(dev, new_umem->length))
1755 return false;1821 return false;
1756 1822
1757 *page_size = mlx5_umem_mkc_find_best_pgsz(dev, new_umem, iova);1823 *page_size = mlx5_umem_mkc_find_best_pgsz(
1824 dev, new_umem, iova, mr->mmkey.cache_ent->rb_key.access_mode);
1758 if (WARN_ON(!*page_size))1825 if (WARN_ON(!*page_size))
1759 return false;1826 return false;
1760 return (mr->mmkey.cache_ent->rb_key.ndescs) >=1827 return (mr->mmkey.cache_ent->rb_key.ndescs) >=
@@ -1817,7 +1884,8 @@ struct ib_mr *mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start, · 56 unmodified lines
1817 struct mlx5_ib_mr *mr = to_mmr(ib_mr);1884 struct mlx5_ib_mr *mr = to_mmr(ib_mr);
1818 int err;1885 int err;
1819 1886
1820 if (!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM) || mr->data_direct)1887 if (!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM) || mr->data_direct ||
1888 mr->mmkey.rb_key.ph != MLX5_IB_NO_PH)
1821 return ERR_PTR(-EOPNOTSUPP);1889 return ERR_PTR(-EOPNOTSUPP);
1822 1890
1823 mlx5_ib_dbg(1891 mlx5_ib_dbg(
@@ -1861,7 +1929,7 @@ struct ib_mr *mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start, · 37 unmodified lines
1861 atomic_sub(ib_umem_num_pages(umem), &dev->mdev->priv.reg_pages);1929 atomic_sub(ib_umem_num_pages(umem), &dev->mdev->priv.reg_pages);
1862 1930
1863 return create_real_mr(new_pd, umem, mr->ibmr.iova,1931 return create_real_mr(new_pd, umem, mr->ibmr.iova,
1864 new_access_flags);1932 new_access_flags, NULL);
1865 }1933 }
1866 1934
1867 /*1935 /*
@@ -1892,7 +1960,7 @@ struct ib_mr *mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start, · 24 unmodified lines
1892 }1960 }
1893 return NULL;1961 return NULL;
1894 }1962 }
1895 return create_real_mr(new_pd, new_umem, iova, new_access_flags);1963 return create_real_mr(new_pd, new_umem, iova, new_access_flags, NULL);
1896 }1964 }
1897 1965
1898 /*1966 /*
@@ -1901,7 +1969,7 @@ struct ib_mr *mlx5_ib_rereg_user_mr(struct ib_mr *ib_mr, int flags, u64 start, · 2 unmodified lines
1901 */1969 */
1902recreate:1970recreate:
1903 return mlx5_ib_reg_user_mr(new_pd, start, length, iova,1971 return mlx5_ib_reg_user_mr(new_pd, start, length, iova,
1904 new_access_flags, udata);1972 new_access_flags, NULL, udata);
1905}1973}
1906 1974
1907static int1975static int
drivers/infiniband/hw/mlx5/odp.c+25−7
@@ -836,9 +836,13 @@ static int pagefault_dmabuf_mr(struct mlx5_ib_mr *mr, size_t bcnt, · 835 unmodified lines
836 u32 *bytes_mapped, u32 flags)836 u32 *bytes_mapped, u32 flags)
837{837{
838 struct ib_umem_dmabuf *umem_dmabuf = to_ib_umem_dmabuf(mr->umem);838 struct ib_umem_dmabuf *umem_dmabuf = to_ib_umem_dmabuf(mr->umem);
839 int access_mode = mr->data_direct ? MLX5_MKC_ACCESS_MODE_KSM :
840 MLX5_MKC_ACCESS_MODE_MTT;
841 unsigned int old_page_shift = mr->page_shift;
842 unsigned int page_shift;
843 unsigned long page_size;
839 u32 xlt_flags = 0;844 u32 xlt_flags = 0;
840 int err;845 int err;
841 unsigned long page_size;
842 846
843 if (flags & MLX5_PF_FLAGS_ENABLE)847 if (flags & MLX5_PF_FLAGS_ENABLE)
844 xlt_flags |= MLX5_IB_UPD_XLT_ENABLE;848 xlt_flags |= MLX5_IB_UPD_XLT_ENABLE;
@@ -850,20 +854,33 @@ static int pagefault_dmabuf_mr(struct mlx5_ib_mr *mr, size_t bcnt, · 5 unmodified lines
850 return err;854 return err;
851 }855 }
852 856
853 page_size = mlx5_umem_dmabuf_find_best_pgsz(umem_dmabuf);857 page_size = mlx5_umem_dmabuf_find_best_pgsz(umem_dmabuf, access_mode);
854 if (!page_size) {858 if (!page_size) {
855 ib_umem_dmabuf_unmap_pages(umem_dmabuf);859 ib_umem_dmabuf_unmap_pages(umem_dmabuf);
856 err = -EINVAL;860 err = -EINVAL;
857 } else {861 } else {
858 if (mr->data_direct)862 page_shift = order_base_2(page_size);
859 err = mlx5r_umr_update_data_direct_ksm_pas(mr, xlt_flags);863 if (page_shift != mr->page_shift && mr->dmabuf_faulted) {
860 else864 err = mlx5r_umr_dmabuf_update_pgsz(mr, xlt_flags,
861 err = mlx5r_umr_update_mr_pas(mr, xlt_flags);865 page_shift);
866 } else {
867 mr->page_shift = page_shift;
868 if (mr->data_direct)
869 err = mlx5r_umr_update_data_direct_ksm_pas(
870 mr, xlt_flags);
871 else
872 err = mlx5r_umr_update_mr_pas(mr,
873 xlt_flags);
874 }
862 }875 }
863 dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);876 dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);
864 877
865 if (err)878 if (err) {
879 mr->page_shift = old_page_shift;
866 return err;880 return err;
881 }
882
883 mr->dmabuf_faulted = 1;
867 884
868 if (bytes_mapped)885 if (bytes_mapped)
869 *bytes_mapped += bcnt;886 *bytes_mapped += bcnt;
@@ -1866,6 +1883,7 @@ int mlx5_odp_init_mkey_cache(struct mlx5_ib_dev *dev) · 996 unmodified lines
1866 struct mlx5r_cache_rb_key rb_key = {1883 struct mlx5r_cache_rb_key rb_key = {
1867 .access_mode = MLX5_MKC_ACCESS_MODE_KSM,1884 .access_mode = MLX5_MKC_ACCESS_MODE_KSM,
1868 .ndescs = mlx5_imr_ksm_entries,1885 .ndescs = mlx5_imr_ksm_entries,
1886 .ph = MLX5_IB_NO_PH,
1869 };1887 };
1870 struct mlx5_cache_ent *ent;1888 struct mlx5_cache_ent *ent;
1871 1889
drivers/infiniband/hw/mlx5/umr.c+279−22
@@ -659,6 +659,9 @@ static void mlx5r_umr_final_update_xlt(struct mlx5_ib_dev *dev, · 658 unmodified lines
659 wqe->ctrl_seg.mkey_mask |= get_umr_update_translation_mask(dev);659 wqe->ctrl_seg.mkey_mask |= get_umr_update_translation_mask(dev);
660 if (!mr->ibmr.length)660 if (!mr->ibmr.length)
661 MLX5_SET(mkc, &wqe->mkey_seg, length64, 1);661 MLX5_SET(mkc, &wqe->mkey_seg, length64, 1);
662 if (flags & MLX5_IB_UPD_XLT_KEEP_PGSZ)
663 wqe->ctrl_seg.mkey_mask &=
664 cpu_to_be64(~MLX5_MKEY_MASK_PAGE_SIZE);
662 }665 }
663 666
664 wqe->ctrl_seg.xlt_octowords =667 wqe->ctrl_seg.xlt_octowords =
@@ -666,46 +669,78 @@ static void mlx5r_umr_final_update_xlt(struct mlx5_ib_dev *dev, · 1 unmodified line
666 wqe->data_seg.byte_count = cpu_to_be32(sg->length);669 wqe->data_seg.byte_count = cpu_to_be32(sg->length);
667}670}
668 671
672static void
673_mlx5r_umr_init_wqe(struct mlx5_ib_mr *mr, struct mlx5r_umr_wqe *wqe,
674 struct ib_sge *sg, unsigned int flags,
675 unsigned int page_shift, bool dd)
676{
677 struct mlx5_ib_dev *dev = mr_to_mdev(mr);
678
679 mlx5r_umr_set_update_xlt_ctrl_seg(&wqe->ctrl_seg, flags, sg);
680 mlx5r_umr_set_update_xlt_mkey_seg(dev, &wqe->mkey_seg, mr, page_shift);
681 if (dd) /* Use the data direct internal kernel PD */
682 MLX5_SET(mkc, &wqe->mkey_seg, pd, dev->ddr.pdn);
683 mlx5r_umr_set_update_xlt_data_seg(&wqe->data_seg, sg);
684}
685
669static int686static int
670_mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags, bool dd)687_mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags, bool dd,
688 size_t start_block, size_t nblocks)
671{689{
672 size_t ent_size = dd ? sizeof(struct mlx5_ksm) : sizeof(struct mlx5_mtt);690 size_t ent_size = dd ? sizeof(struct mlx5_ksm) : sizeof(struct mlx5_mtt);
673 struct mlx5_ib_dev *dev = mr_to_mdev(mr);691 struct mlx5_ib_dev *dev = mr_to_mdev(mr);
674 struct device *ddev = &dev->mdev->pdev->dev;692 struct device *ddev = &dev->mdev->pdev->dev;
675 struct mlx5r_umr_wqe wqe = {};693 struct mlx5r_umr_wqe wqe = {};
694 size_t processed_blocks = 0;
676 struct ib_block_iter biter;695 struct ib_block_iter biter;
696 size_t cur_block_idx = 0;
677 struct mlx5_ksm *cur_ksm;697 struct mlx5_ksm *cur_ksm;
678 struct mlx5_mtt *cur_mtt;698 struct mlx5_mtt *cur_mtt;
679 size_t orig_sg_length;699 size_t orig_sg_length;
700 size_t total_blocks;
680 size_t final_size;701 size_t final_size;
681 void *curr_entry;702 void *curr_entry;
682 struct ib_sge sg;703 struct ib_sge sg;
683 void *entry;704 void *entry;
684 u64 offset = 0;705 u64 offset;
685 int err = 0;706 int err = 0;
686 707
687 entry = mlx5r_umr_create_xlt(dev, &sg,708 total_blocks = ib_umem_num_dma_blocks(mr->umem, 1UL << mr->page_shift);
688 ib_umem_num_dma_blocks(mr->umem, 1 << mr->page_shift),709 if (start_block > total_blocks)
689 ent_size, flags);710 return -EINVAL;
711
712 /* nblocks 0 means update all blocks starting from start_block */
713 if (nblocks)
714 total_blocks = nblocks;
715
716 entry = mlx5r_umr_create_xlt(dev, &sg, total_blocks, ent_size, flags);
690 if (!entry)717 if (!entry)
691 return -ENOMEM;718 return -ENOMEM;
692 719
693 orig_sg_length = sg.length;720 orig_sg_length = sg.length;
694 mlx5r_umr_set_update_xlt_ctrl_seg(&wqe.ctrl_seg, flags, &sg);721
695 mlx5r_umr_set_update_xlt_mkey_seg(dev, &wqe.mkey_seg, mr,722 _mlx5r_umr_init_wqe(mr, &wqe, &sg, flags, mr->page_shift, dd);
696 mr->page_shift);723
697 if (dd) {724 /* Set initial translation offset to start_block */
698 /* Use the data direct internal kernel PD */725 offset = (u64)start_block * ent_size;
699 MLX5_SET(mkc, &wqe.mkey_seg, pd, dev->ddr.pdn);726 mlx5r_umr_update_offset(&wqe.ctrl_seg, offset);
727
728 if (dd)
700 cur_ksm = entry;729 cur_ksm = entry;
701 } else {730 else
702 cur_mtt = entry;731 cur_mtt = entry;
703 }
704
705 mlx5r_umr_set_update_xlt_data_seg(&wqe.data_seg, &sg);
706 732
707 curr_entry = entry;733 curr_entry = entry;
734
708 rdma_umem_for_each_dma_block(mr->umem, &biter, BIT(mr->page_shift)) {735 rdma_umem_for_each_dma_block(mr->umem, &biter, BIT(mr->page_shift)) {
736 if (cur_block_idx < start_block) {
737 cur_block_idx++;
738 continue;
739 }
740
741 if (nblocks && processed_blocks >= nblocks)
742 break;
743
709 if (curr_entry == entry + sg.length) {744 if (curr_entry == entry + sg.length) {
710 dma_sync_single_for_device(ddev, sg.addr, sg.length,745 dma_sync_single_for_device(ddev, sg.addr, sg.length,
711 DMA_TO_DEVICE);746 DMA_TO_DEVICE);
@@ -727,6 +762,11 @@ _mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags, bool dd) · 15 unmodified lines
727 if (dd) {762 if (dd) {
728 cur_ksm->va = cpu_to_be64(rdma_block_iter_dma_address(&biter));763 cur_ksm->va = cpu_to_be64(rdma_block_iter_dma_address(&biter));
729 cur_ksm->key = cpu_to_be32(dev->ddr.mkey);764 cur_ksm->key = cpu_to_be32(dev->ddr.mkey);
765 if (mr->umem->is_dmabuf &&
766 (flags & MLX5_IB_UPD_XLT_ZAP)) {
767 cur_ksm->va = 0;
768 cur_ksm->key = 0;
769 }
730 cur_ksm++;770 cur_ksm++;
731 curr_entry = cur_ksm;771 curr_entry = cur_ksm;
732 } else {772 } else {
@@ -738,6 +778,8 @@ _mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags, bool dd) · 5 unmodified lines
738 cur_mtt++;778 cur_mtt++;
739 curr_entry = cur_mtt;779 curr_entry = cur_mtt;
740 }780 }
781
782 processed_blocks++;
741 }783 }
742 784
743 final_size = curr_entry - entry;785 final_size = curr_entry - entry;
@@ -754,13 +796,32 @@ err: · 10 unmodified lines
754 return err;796 return err;
755}797}
756 798
757int mlx5r_umr_update_data_direct_ksm_pas(struct mlx5_ib_mr *mr, unsigned int flags)799int mlx5r_umr_update_data_direct_ksm_pas_range(struct mlx5_ib_mr *mr,
800 unsigned int flags,
801 size_t start_block,
802 size_t nblocks)
758{803{
759 /* No invalidation flow is expected */804 /* No invalidation flow is expected */
760 if (WARN_ON(!mr->umem->is_dmabuf) || (flags & MLX5_IB_UPD_XLT_ZAP))805 if (WARN_ON(!mr->umem->is_dmabuf) || ((flags & MLX5_IB_UPD_XLT_ZAP) &&
806 !(flags & MLX5_IB_UPD_XLT_KEEP_PGSZ)))
761 return -EINVAL;807 return -EINVAL;
762 808
763 return _mlx5r_umr_update_mr_pas(mr, flags, true);809 return _mlx5r_umr_update_mr_pas(mr, flags, true, start_block, nblocks);
810}
811
812int mlx5r_umr_update_data_direct_ksm_pas(struct mlx5_ib_mr *mr,
813 unsigned int flags)
814{
815 return mlx5r_umr_update_data_direct_ksm_pas_range(mr, flags, 0, 0);
816}
817
818int mlx5r_umr_update_mr_pas_range(struct mlx5_ib_mr *mr, unsigned int flags,
819 size_t start_block, size_t nblocks)
820{
821 if (WARN_ON(mr->umem->is_odp))
822 return -EINVAL;
823
824 return _mlx5r_umr_update_mr_pas(mr, flags, false, start_block, nblocks);
764}825}
765 826
766/*827/*
@@ -770,10 +831,7 @@ int mlx5r_umr_update_data_direct_ksm_pas(struct mlx5_ib_mr *mr, unsigned int fla · 3 unmodified lines
770 */831 */
771int mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags)832int mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags)
772{833{
773 if (WARN_ON(mr->umem->is_odp))834 return mlx5r_umr_update_mr_pas_range(mr, flags, 0, 0);
774 return -EINVAL;
775
776 return _mlx5r_umr_update_mr_pas(mr, flags, false);
777}835}
778 836
779static bool umr_can_use_indirect_mkey(struct mlx5_ib_dev *dev)837static bool umr_can_use_indirect_mkey(struct mlx5_ib_dev *dev)
@@ -866,3 +924,202 @@ int mlx5r_umr_update_xlt(struct mlx5_ib_mr *mr, u64 idx, int npages, · 86 unmodified lines
866 mlx5r_umr_unmap_free_xlt(dev, xlt, &sg);924 mlx5r_umr_unmap_free_xlt(dev, xlt, &sg);
867 return err;925 return err;
868}926}
927
928/*
929 * Update only the page-size (log_page_size) field of an existing memory key
930 * using UMR. This is useful when the MR's physical layout stays the same
931 * but the optimal page shift has changed (e.g. dmabuf after pages are
932 * pinned and the HW can switch from 4K to huge-page alignment).
933 */
934int mlx5r_umr_update_mr_page_shift(struct mlx5_ib_mr *mr,
935 unsigned int page_shift,
936 bool dd)
937{
938 struct mlx5_ib_dev *dev = mr_to_mdev(mr);
939 struct mlx5r_umr_wqe wqe = {};
940 int err;
941
942 /* Build UMR wqe: we touch only PAGE_SIZE, so use the dedicated mask */
943 wqe.ctrl_seg.mkey_mask = get_umr_update_translation_mask(dev);
944
945 /* MR must be free while page size is modified */
946 wqe.ctrl_seg.flags = MLX5_UMR_CHECK_FREE | MLX5_UMR_INLINE;
947
948 /* Fill mkey segment with the new page size, keep the rest unchanged */
949 MLX5_SET(mkc, &wqe.mkey_seg, log_page_size, page_shift);
950
951 if (dd)
952 MLX5_SET(mkc, &wqe.mkey_seg, pd, dev->ddr.pdn);
953 else
954 MLX5_SET(mkc, &wqe.mkey_seg, pd, to_mpd(mr->ibmr.pd)->pdn);
955
956 MLX5_SET64(mkc, &wqe.mkey_seg, start_addr, mr->ibmr.iova);
957 MLX5_SET64(mkc, &wqe.mkey_seg, len, mr->ibmr.length);
958 MLX5_SET(mkc, &wqe.mkey_seg, qpn, 0xffffff);
959 MLX5_SET(mkc, &wqe.mkey_seg, mkey_7_0,
960 mlx5_mkey_variant(mr->mmkey.key));
961
962 err = mlx5r_umr_post_send_wait(dev, mr->mmkey.key, &wqe, false);
963 if (!err)
964 mr->page_shift = page_shift;
965
966 return err;
967}
968
969static inline int
970_mlx5r_dmabuf_umr_update_pas(struct mlx5_ib_mr *mr, unsigned int flags,
971 size_t start_block, size_t nblocks, bool dd)
972{
973 if (dd)
974 return mlx5r_umr_update_data_direct_ksm_pas_range(mr, flags,
975 start_block,
976 nblocks);
977 else
978 return mlx5r_umr_update_mr_pas_range(mr, flags, start_block,
979 nblocks);
980}
981
982/**
983 * This function makes an mkey non-present by zapping the translation entries of
984 * the mkey by zapping (zeroing out) the first N entries, where N is determined
985 * by the largest page size supported by the device and the MR length.
986 * It then updates the mkey's page size to the largest possible value, ensuring
987 * the MR is completely non-present and safe for further updates.
988 * It is useful to update the page size of a dmabuf MR on a page fault.
989 *
990 * Return: On success, returns the number of entries that were zapped.
991 * On error, returns a negative error code.
992 */
993static int _mlx5r_umr_zap_mkey(struct mlx5_ib_mr *mr,
994 unsigned int flags,
995 unsigned int page_shift,
996 size_t *nblocks,
997 bool dd)
998{
999 unsigned int old_page_shift = mr->page_shift;
1000 struct mlx5_ib_dev *dev = mr_to_mdev(mr);
1001 unsigned int max_page_shift;
1002 size_t page_shift_nblocks;
1003 unsigned int max_log_size;
1004 int access_mode;
1005 int err;
1006
1007 access_mode = dd ? MLX5_MKC_ACCESS_MODE_KSM : MLX5_MKC_ACCESS_MODE_MTT;
1008 flags |= MLX5_IB_UPD_XLT_KEEP_PGSZ | MLX5_IB_UPD_XLT_ZAP |
1009 MLX5_IB_UPD_XLT_ATOMIC;
1010 max_log_size = get_max_log_entity_size_cap(dev, access_mode);
1011 max_page_shift = order_base_2(mr->ibmr.length);
1012 max_page_shift = min(max(max_page_shift, page_shift), max_log_size);
1013 /* Count blocks in units of max_page_shift, we will zap exactly this
1014 * many to make the whole MR non-present.
1015 * Block size must be aligned to MLX5_UMR_FLEX_ALIGNMENT since it may
1016 * be used as offset into the XLT later on.
1017 */
1018 *nblocks = ib_umem_num_dma_blocks(mr->umem, 1UL << max_page_shift);
1019 if (dd)
1020 *nblocks = ALIGN(*nblocks, MLX5_UMR_KSM_NUM_ENTRIES_ALIGNMENT);
1021 else
1022 *nblocks = ALIGN(*nblocks, MLX5_UMR_MTT_NUM_ENTRIES_ALIGNMENT);
1023 page_shift_nblocks = ib_umem_num_dma_blocks(mr->umem,
1024 1UL << page_shift);
1025 /* If the number of blocks at max possible page shift is greater than
1026 * the number of blocks at the new page size, we should just go over the
1027 * whole mkey entries.
1028 */
1029 if (*nblocks >= page_shift_nblocks)
1030 *nblocks = 0;
1031
1032 /* Make the first nblocks entries non-present without changing
1033 * page size yet.
1034 */
1035 if (*nblocks)
1036 mr->page_shift = max_page_shift;
1037 err = _mlx5r_dmabuf_umr_update_pas(mr, flags, 0, *nblocks, dd);
1038 if (err) {
1039 mr->page_shift = old_page_shift;
1040 return err;
1041 }
1042
1043 /* Change page size to the max page size now that the MR is completely
1044 * non-present.
1045 */
1046 if (*nblocks) {
1047 err = mlx5r_umr_update_mr_page_shift(mr, max_page_shift, dd);
1048 if (err) {
1049 mr->page_shift = old_page_shift;
1050 return err;
1051 }
1052 }
1053
1054 return 0;
1055}
1056
1057/**
1058 * mlx5r_umr_dmabuf_update_pgsz - Safely update DMABUF MR page size and its
1059 * entries accordingly
1060 * @mr: The memory region to update
1061 * @xlt_flags: Translation table update flags
1062 * @page_shift: The new (optimized) page shift to use
1063 *
1064 * This function updates the page size and mkey translation entries for a DMABUF
1065 * MR in a safe, multi-step process to avoid exposing partially updated mappings
1066 * The update is performed in 5 steps:
1067 * 1. Make the first X entries non-present, while X is calculated to be
1068 * minimal according to a large page shift that can be used to cover the
1069 * MR length.
1070 * 2. Update the page size to the large supported page size
1071 * 3. Load the remaining N-X entries according to the (optimized) page_shift
1072 * 4. Update the page size according to the (optimized) page_shift
1073 * 5. Load the first X entries with the correct translations
1074 *
1075 * This ensures that at no point is the MR accessible with a partially updated
1076 * translation table, maintaining correctness and preventing access to stale or
1077 * inconsistent mappings.
1078 *
1079 * Returns 0 on success or a negative error code on failure.
1080 */
1081int mlx5r_umr_dmabuf_update_pgsz(struct mlx5_ib_mr *mr, u32 xlt_flags,
1082 unsigned int page_shift)
1083{
1084 unsigned int old_page_shift = mr->page_shift;
1085 size_t zapped_blocks;
1086 size_t total_blocks;
1087 int err;
1088
1089 err = _mlx5r_umr_zap_mkey(mr, xlt_flags, page_shift, &zapped_blocks,
1090 mr->data_direct);
1091 if (err)
1092 return err;
1093
1094 /* _mlx5r_umr_zap_mkey already enables the mkey */
1095 xlt_flags &= ~MLX5_IB_UPD_XLT_ENABLE;
1096 mr->page_shift = page_shift;
1097 total_blocks = ib_umem_num_dma_blocks(mr->umem, 1UL << mr->page_shift);
1098 if (zapped_blocks && zapped_blocks < total_blocks) {
1099 /* Update PAS according to the new page size but don't update
1100 * the page size in the mkey yet.
1101 */
1102 err = _mlx5r_dmabuf_umr_update_pas(
1103 mr,
1104 xlt_flags | MLX5_IB_UPD_XLT_KEEP_PGSZ,
1105 zapped_blocks,
1106 total_blocks - zapped_blocks,
1107 mr->data_direct);
1108 if (err)
1109 goto err;
1110 }
1111
1112 err = mlx5r_umr_update_mr_page_shift(mr, mr->page_shift,
1113 mr->data_direct);
1114 if (err)
1115 goto err;
1116 err = _mlx5r_dmabuf_umr_update_pas(mr, xlt_flags, 0, zapped_blocks,
1117 mr->data_direct);
1118 if (err)
1119 goto err;
1120
1121 return 0;
1122err:
1123 mr->page_shift = old_page_shift;
1124 return err;
1125}
drivers/infiniband/hw/mlx5/umr.h+12−1
@@ -94,9 +94,20 @@ struct mlx5r_umr_wqe { · 93 unmodified lines
94int mlx5r_umr_revoke_mr(struct mlx5_ib_mr *mr);94int mlx5r_umr_revoke_mr(struct mlx5_ib_mr *mr);
95int mlx5r_umr_rereg_pd_access(struct mlx5_ib_mr *mr, struct ib_pd *pd,95int mlx5r_umr_rereg_pd_access(struct mlx5_ib_mr *mr, struct ib_pd *pd,
96 int access_flags);96 int access_flags);
97int mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags);97int mlx5r_umr_update_data_direct_ksm_pas_range(struct mlx5_ib_mr *mr,
98 unsigned int flags,
99 size_t start_block,
100 size_t nblocks);
98int mlx5r_umr_update_data_direct_ksm_pas(struct mlx5_ib_mr *mr, unsigned int flags);101int mlx5r_umr_update_data_direct_ksm_pas(struct mlx5_ib_mr *mr, unsigned int flags);
102int mlx5r_umr_update_mr_pas_range(struct mlx5_ib_mr *mr, unsigned int flags,
103 size_t start_block, size_t nblocks);
104int mlx5r_umr_update_mr_pas(struct mlx5_ib_mr *mr, unsigned int flags);
99int mlx5r_umr_update_xlt(struct mlx5_ib_mr *mr, u64 idx, int npages,105int mlx5r_umr_update_xlt(struct mlx5_ib_mr *mr, u64 idx, int npages,
100 int page_shift, int flags);106 int page_shift, int flags);
107int mlx5r_umr_update_mr_page_shift(struct mlx5_ib_mr *mr,
108 unsigned int page_shift,
109 bool dd);
110int mlx5r_umr_dmabuf_update_pgsz(struct mlx5_ib_mr *mr, u32 xlt_flags,
111 unsigned int page_shift);
101 112
102#endif /* _MLX5_IB_UMR_H */113#endif /* _MLX5_IB_UMR_H */
drivers/infiniband/hw/mthca/mthca_provider.c+5−1
@@ -825,7 +825,8 @@ static struct ib_mr *mthca_get_dma_mr(struct ib_pd *pd, int acc) · 824 unmodified lines
825}825}
826 826
827static struct ib_mr *mthca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,827static struct ib_mr *mthca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
828 u64 virt, int acc, struct ib_udata *udata)828 u64 virt, int acc, struct ib_dmah *dmah,
829 struct ib_udata *udata)
829{830{
830 struct mthca_dev *dev = to_mdev(pd->device);831 struct mthca_dev *dev = to_mdev(pd->device);
831 struct ib_block_iter biter;832 struct ib_block_iter biter;
@@ -838,6 +839,9 @@ static struct ib_mr *mthca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, · 6 unmodified lines
838 int err = 0;839 int err = 0;
839 int write_mtt_size;840 int write_mtt_size;
840 841
842 if (dmah)
843 return ERR_PTR(-EOPNOTSUPP);
844
841 if (udata->inlen < sizeof ucmd) {845 if (udata->inlen < sizeof ucmd) {
842 if (!context->reg_mr_warned) {846 if (!context->reg_mr_warned) {
843 mthca_warn(dev, "Process '%s' did not pass in MR attrs.\n",847 mthca_warn(dev, "Process '%s' did not pass in MR attrs.\n",
drivers/infiniband/hw/ocrdma/ocrdma_verbs.c+5−1
@@ -847,13 +847,17 @@ static void build_user_pbes(struct ocrdma_dev *dev, struct ocrdma_mr *mr) · 846 unmodified lines
847}847}
848 848
849struct ib_mr *ocrdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,849struct ib_mr *ocrdma_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,
850 u64 usr_addr, int acc, struct ib_udata *udata)850 u64 usr_addr, int acc, struct ib_dmah *dmah,
851 struct ib_udata *udata)
851{852{
852 int status = -ENOMEM;853 int status = -ENOMEM;
853 struct ocrdma_dev *dev = get_ocrdma_dev(ibpd->device);854 struct ocrdma_dev *dev = get_ocrdma_dev(ibpd->device);
854 struct ocrdma_mr *mr;855 struct ocrdma_mr *mr;
855 struct ocrdma_pd *pd;856 struct ocrdma_pd *pd;
856 857
858 if (dmah)
859 return ERR_PTR(-EOPNOTSUPP);
860
857 pd = get_ocrdma_pd(ibpd);861 pd = get_ocrdma_pd(ibpd);
858 862
859 if (acc & IB_ACCESS_REMOTE_WRITE && !(acc & IB_ACCESS_LOCAL_WRITE))863 if (acc & IB_ACCESS_REMOTE_WRITE && !(acc & IB_ACCESS_LOCAL_WRITE))
drivers/infiniband/hw/ocrdma/ocrdma_verbs.h+2−1
@@ -98,7 +98,8 @@ int ocrdma_post_srq_recv(struct ib_srq *, const struct ib_recv_wr *, · 97 unmodified lines
98int ocrdma_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata);98int ocrdma_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata);
99struct ib_mr *ocrdma_get_dma_mr(struct ib_pd *, int acc);99struct ib_mr *ocrdma_get_dma_mr(struct ib_pd *, int acc);
100struct ib_mr *ocrdma_reg_user_mr(struct ib_pd *, u64 start, u64 length,100struct ib_mr *ocrdma_reg_user_mr(struct ib_pd *, u64 start, u64 length,
101 u64 virt, int acc, struct ib_udata *);101 u64 virt, int acc, struct ib_dmah *dmah,
102 struct ib_udata *);
102struct ib_mr *ocrdma_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type,103struct ib_mr *ocrdma_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type,
103 u32 max_num_sg);104 u32 max_num_sg);
104int ocrdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,105int ocrdma_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg, int sg_nents,
drivers/infiniband/hw/qedr/verbs.c+5−1
@@ -2953,13 +2953,17 @@ done: · 2952 unmodified lines
2953}2953}
2954 2954
2955struct ib_mr *qedr_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,2955struct ib_mr *qedr_reg_user_mr(struct ib_pd *ibpd, u64 start, u64 len,
2956 u64 usr_addr, int acc, struct ib_udata *udata)2956 u64 usr_addr, int acc, struct ib_dmah *dmah,
2957 struct ib_udata *udata)
2957{2958{
2958 struct qedr_dev *dev = get_qedr_dev(ibpd->device);2959 struct qedr_dev *dev = get_qedr_dev(ibpd->device);
2959 struct qedr_mr *mr;2960 struct qedr_mr *mr;
2960 struct qedr_pd *pd;2961 struct qedr_pd *pd;
2961 int rc = -ENOMEM;2962 int rc = -ENOMEM;
2962 2963
2964 if (dmah)
2965 return ERR_PTR(-EOPNOTSUPP);
2966
2963 pd = get_qedr_pd(ibpd);2967 pd = get_qedr_pd(ibpd);
2964 DP_DEBUG(dev, QEDR_MSG_MR,2968 DP_DEBUG(dev, QEDR_MSG_MR,
2965 "qedr_register user mr pd = %d start = %lld, len = %lld, usr_addr = %lld, acc = %d\n",2969 "qedr_register user mr pd = %d start = %lld, len = %lld, usr_addr = %lld, acc = %d\n",
drivers/infiniband/hw/qedr/verbs.h+2−1
@@ -79,7 +79,8 @@ int qedr_dereg_mr(struct ib_mr *ib_mr, struct ib_udata *udata); · 78 unmodified lines
79struct ib_mr *qedr_get_dma_mr(struct ib_pd *, int acc);79struct ib_mr *qedr_get_dma_mr(struct ib_pd *, int acc);
80 80
81struct ib_mr *qedr_reg_user_mr(struct ib_pd *, u64 start, u64 length,81struct ib_mr *qedr_reg_user_mr(struct ib_pd *, u64 start, u64 length,
82 u64 virt, int acc, struct ib_udata *);82 u64 virt, int acc, struct ib_dmah *dmah,
83 struct ib_udata *);
83 84
84int qedr_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,85int qedr_map_mr_sg(struct ib_mr *ibmr, struct scatterlist *sg,
85 int sg_nents, unsigned int *sg_offset);86 int sg_nents, unsigned int *sg_offset);
drivers/infiniband/hw/qib/Kconfig+0−17
@@ -1,17 +0,0 @@
1# SPDX-License-Identifier: GPL-2.0-only
2config INFINIBAND_QIB
3 tristate "Intel PCIe HCA support"
4 depends on 64BIT && INFINIBAND_RDMAVT
5 depends on PCI
6 help
7 This is a low-level driver for Intel PCIe QLE InfiniBand host
8 channel adapters. This driver does not support the Intel
9 HyperTransport card (model QHT7140).
10
11config INFINIBAND_QIB_DCA
12 bool "QIB DCA support"
13 depends on INFINIBAND_QIB && DCA && SMP && !(INFINIBAND_QIB=y && DCA=m)
14 default y
15 help
16 Setting this enables DCA support on some Intel chip sets
17 with the iba7322 HCA.
drivers/infiniband/hw/qib/Makefile+0−17
@@ -1,17 +0,0 @@
1# SPDX-License-Identifier: GPL-2.0
2obj-$(CONFIG_INFINIBAND_QIB) += ib_qib.o
3
4ib_qib-y := qib_diag.o qib_driver.o qib_eeprom.o \
5 qib_file_ops.o qib_fs.o qib_init.o qib_intr.o \
6 qib_mad.o qib_pcie.o qib_pio_copy.o \
7 qib_qp.o qib_qsfp.o qib_rc.o qib_ruc.o qib_sdma.o \
8 qib_sysfs.o qib_twsi.o qib_tx.o qib_uc.o qib_ud.o \
9 qib_user_pages.o qib_user_sdma.o qib_iba7220.o \
10 qib_sd7220.o qib_iba7322.o qib_verbs.o
11
12# 6120 has no fallback if no MSI interrupts, others can do INTx
13ib_qib-$(CONFIG_PCI_MSI) += qib_iba6120.o
14
15ib_qib-$(CONFIG_X86_64) += qib_wc_x86_64.o
16ib_qib-$(CONFIG_PPC64) += qib_wc_ppc64.o
17ib_qib-$(CONFIG_DEBUG_FS) += qib_debugfs.o
drivers/infiniband/hw/qib/qib.h+0−1492
@@ -1,1492 +0,0 @@
1#ifndef _QIB_KERNEL_H
2#define _QIB_KERNEL_H
3/*
4 * Copyright (c) 2012 - 2017 Intel Corporation. All rights reserved.
5 * Copyright (c) 2006 - 2012 QLogic Corporation. All rights reserved.
6 * Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved.
7 *
8 * This software is available to you under a choice of one of two
9 * licenses. You may choose to be licensed under the terms of the GNU
10 * General Public License (GPL) Version 2, available from the file
11 * COPYING in the main directory of this source tree, or the
12 * OpenIB.org BSD license below:
13 *
14 * Redistribution and use in source and binary forms, with or
15 * without modification, are permitted provided that the following
16 * conditions are met:
17 *
18 * - Redistributions of source code must retain the above
19 * copyright notice, this list of conditions and the following
20 * disclaimer.
21 *
22 * - Redistributions in binary form must reproduce the above
23 * copyright notice, this list of conditions and the following
24 * disclaimer in the documentation and/or other materials
25 * provided with the distribution.
26 *
27 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
28 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
29 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
30 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
31 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
32 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
33 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
34 * SOFTWARE.
35 */
36
37/*
38 * This header file is the base header file for qlogic_ib kernel code
39 * qib_user.h serves a similar purpose for user code.
40 */
41
42#include <linux/interrupt.h>
43#include <linux/pci.h>
44#include <linux/dma-mapping.h>
45#include <linux/mutex.h>
46#include <linux/list.h>
47#include <linux/scatterlist.h>
48#include <linux/slab.h>
49#include <linux/io.h>
50#include <linux/fs.h>
51#include <linux/completion.h>
52#include <linux/kref.h>
53#include <linux/sched.h>
54#include <linux/kthread.h>
55#include <linux/xarray.h>
56#include <rdma/ib_hdrs.h>
57#include <rdma/rdma_vt.h>
58
59#include "qib_common.h"
60#include "qib_verbs.h"
61
62/* only s/w major version of QLogic_IB we can handle */
63#define QIB_CHIP_VERS_MAJ 2U
64
65/* don't care about this except printing */
66#define QIB_CHIP_VERS_MIN 0U
67
68/* The Organization Unique Identifier (Mfg code), and its position in GUID */
69#define QIB_OUI 0x001175
70#define QIB_OUI_LSB 40
71
72/*
73 * per driver stats, either not device nor port-specific, or
74 * summed over all of the devices and ports.
75 * They are described by name via ipathfs filesystem, so layout
76 * and number of elements can change without breaking compatibility.
77 * If members are added or deleted qib_statnames[] in qib_fs.c must
78 * change to match.
79 */
80struct qlogic_ib_stats {
81 __u64 sps_ints; /* number of interrupts handled */
82 __u64 sps_errints; /* number of error interrupts */
83 __u64 sps_txerrs; /* tx-related packet errors */
84 __u64 sps_rcverrs; /* non-crc rcv packet errors */
85 __u64 sps_hwerrs; /* hardware errors reported (parity, etc.) */
86 __u64 sps_nopiobufs; /* no pio bufs avail from kernel */
87 __u64 sps_ctxts; /* number of contexts currently open */
88 __u64 sps_lenerrs; /* number of kernel packets where RHF != LRH len */
89 __u64 sps_buffull;
90 __u64 sps_hdrfull;
91};
92
93extern struct qlogic_ib_stats qib_stats;
94extern const struct pci_error_handlers qib_pci_err_handler;
95
96#define QIB_CHIP_SWVERSION QIB_CHIP_VERS_MAJ
97/*
98 * First-cut critierion for "device is active" is
99 * two thousand dwords combined Tx, Rx traffic per
100 * 5-second interval. SMA packets are 64 dwords,
101 * and occur "a few per second", presumably each way.
102 */
103#define QIB_TRAFFIC_ACTIVE_THRESHOLD (2000)
104
105/*
106 * Below contains all data related to a single context (formerly called port).
107 */
108
109#ifdef CONFIG_DEBUG_FS
110struct qib_opcode_stats_perctx;
111#endif
112
113struct qib_ctxtdata {
114 void **rcvegrbuf;
115 dma_addr_t *rcvegrbuf_phys;
116 /* rcvhdrq base, needs mmap before useful */
117 void *rcvhdrq;
118 /* kernel virtual address where hdrqtail is updated */
119 void *rcvhdrtail_kvaddr;
120 /*
121 * temp buffer for expected send setup, allocated at open, instead
122 * of each setup call
123 */
124 void *tid_pg_list;
125 /*
126 * Shared page for kernel to signal user processes that send buffers
127 * need disarming. The process should call QIB_CMD_DISARM_BUFS
128 * or QIB_CMD_ACK_EVENT with IPATH_EVENT_DISARM_BUFS set.
129 */
130 unsigned long *user_event_mask;
131 /* when waiting for rcv or pioavail */
132 wait_queue_head_t wait;
133 /*
134 * rcvegr bufs base, physical, must fit
135 * in 44 bits so 32 bit programs mmap64 44 bit works)
136 */
137 dma_addr_t rcvegr_phys;
138 /* mmap of hdrq, must fit in 44 bits */
139 dma_addr_t rcvhdrq_phys;
140 dma_addr_t rcvhdrqtailaddr_phys;
141
142 /*
143 * number of opens (including slave sub-contexts) on this instance
144 * (ignoring forks, dup, etc. for now)
145 */
146 int cnt;
147 /*
148 * how much space to leave at start of eager TID entries for
149 * protocol use, on each TID
150 */
151 /* instead of calculating it */
152 unsigned ctxt;
153 /* local node of context */
154 int node_id;
155 /* non-zero if ctxt is being shared. */
156 u16 subctxt_cnt;
157 /* non-zero if ctxt is being shared. */
158 u16 subctxt_id;
159 /* number of eager TID entries. */
160 u16 rcvegrcnt;
161 /* index of first eager TID entry. */
162 u16 rcvegr_tid_base;
163 /* number of pio bufs for this ctxt (all procs, if shared) */
164 u32 piocnt;
165 /* first pio buffer for this ctxt */
166 u32 pio_base;
167 /* chip offset of PIO buffers for this ctxt */
168 u32 piobufs;
169 /* how many alloc_pages() chunks in rcvegrbuf_pages */
170 u32 rcvegrbuf_chunks;
171 /* how many egrbufs per chunk */
172 u16 rcvegrbufs_perchunk;
173 /* ilog2 of above */
174 u16 rcvegrbufs_perchunk_shift;
175 /* order for rcvegrbuf_pages */
176 size_t rcvegrbuf_size;
177 /* rcvhdrq size (for freeing) */
178 size_t rcvhdrq_size;
179 /* per-context flags for fileops/intr communication */
180 unsigned long flag;
181 /* next expected TID to check when looking for free */
182 u32 tidcursor;
183 /* WAIT_RCV that timed out, no interrupt */
184 u32 rcvwait_to;
185 /* WAIT_PIO that timed out, no interrupt */
186 u32 piowait_to;
187 /* WAIT_RCV already happened, no wait */
188 u32 rcvnowait;
189 /* WAIT_PIO already happened, no wait */
190 u32 pionowait;
191 /* total number of polled urgent packets */
192 u32 urgent;
193 /* saved total number of polled urgent packets for poll edge trigger */
194 u32 urgent_poll;
195 /* pid of process using this ctxt */
196 pid_t pid;
197 pid_t subpid[QLOGIC_IB_MAX_SUBCTXT];
198 /* same size as task_struct .comm[], command that opened context */
199 char comm[TASK_COMM_LEN];
200 /* pkeys set by this use of this ctxt */
201 u16 pkeys[4];
202 /* so file ops can get at unit */
203 struct qib_devdata *dd;
204 /* so funcs that need physical port can get it easily */
205 struct qib_pportdata *ppd;
206 /* A page of memory for rcvhdrhead, rcvegrhead, rcvegrtail * N */
207 void *subctxt_uregbase;
208 /* An array of pages for the eager receive buffers * N */
209 void *subctxt_rcvegrbuf;
210 /* An array of pages for the eager header queue entries * N */
211 void *subctxt_rcvhdr_base;
212 /* The version of the library which opened this ctxt */
213 u32 userversion;
214 /* Bitmask of active slaves */
215 u32 active_slaves;
216 /* Type of packets or conditions we want to poll for */
217 u16 poll_type;
218 /* receive packet sequence counter */
219 u8 seq_cnt;
220 u8 redirect_seq_cnt;
221 /* ctxt rcvhdrq head offset */
222 u32 head;
223 /* QPs waiting for context processing */
224 struct list_head qp_wait_list;
225#ifdef CONFIG_DEBUG_FS
226 /* verbs stats per CTX */
227 struct qib_opcode_stats_perctx *opstats;
228#endif
229};
230
231struct rvt_sge_state;
232
233struct qib_sdma_txreq {
234 int flags;
235 int sg_count;
236 dma_addr_t addr;
237 void (*callback)(struct qib_sdma_txreq *, int);
238 u16 start_idx; /* sdma private */
239 u16 next_descq_idx; /* sdma private */
240 struct list_head list; /* sdma private */
241};
242
243struct qib_sdma_desc {
244 __le64 qw[2];
245};
246
247struct qib_verbs_txreq {
248 struct qib_sdma_txreq txreq;
249 struct rvt_qp *qp;
250 struct rvt_swqe *wqe;
251 u32 dwords;
252 u16 hdr_dwords;
253 u16 hdr_inx;
254 struct qib_pio_header *align_buf;
255 struct rvt_mregion *mr;
256 struct rvt_sge_state *ss;
257};
258
259#define QIB_SDMA_TXREQ_F_USELARGEBUF 0x1
260#define QIB_SDMA_TXREQ_F_HEADTOHOST 0x2
261#define QIB_SDMA_TXREQ_F_INTREQ 0x4
262#define QIB_SDMA_TXREQ_F_FREEBUF 0x8
263#define QIB_SDMA_TXREQ_F_FREEDESC 0x10
264
265#define QIB_SDMA_TXREQ_S_OK 0
266#define QIB_SDMA_TXREQ_S_SENDERROR 1
267#define QIB_SDMA_TXREQ_S_ABORTED 2
268#define QIB_SDMA_TXREQ_S_SHUTDOWN 3
269
270/*
271 * Get/Set IB link-level config parameters for f_get/set_ib_cfg()
272 * Mostly for MADs that set or query link parameters, also ipath
273 * config interfaces
274 */
275#define QIB_IB_CFG_LIDLMC 0 /* LID (LS16b) and Mask (MS16b) */
276#define QIB_IB_CFG_LWID_ENB 2 /* allowed Link-width */
277#define QIB_IB_CFG_LWID 3 /* currently active Link-width */
278#define QIB_IB_CFG_SPD_ENB 4 /* allowed Link speeds */
279#define QIB_IB_CFG_SPD 5 /* current Link spd */
280#define QIB_IB_CFG_RXPOL_ENB 6 /* Auto-RX-polarity enable */
281#define QIB_IB_CFG_LREV_ENB 7 /* Auto-Lane-reversal enable */
282#define QIB_IB_CFG_LINKLATENCY 8 /* Link Latency (IB1.2 only) */
283#define QIB_IB_CFG_HRTBT 9 /* IB heartbeat off/enable/auto; DDR/QDR only */
284#define QIB_IB_CFG_OP_VLS 10 /* operational VLs */
285#define QIB_IB_CFG_VL_HIGH_CAP 11 /* num of VL high priority weights */
286#define QIB_IB_CFG_VL_LOW_CAP 12 /* num of VL low priority weights */
287#define QIB_IB_CFG_OVERRUN_THRESH 13 /* IB overrun threshold */
288#define QIB_IB_CFG_PHYERR_THRESH 14 /* IB PHY error threshold */
289#define QIB_IB_CFG_LINKDEFAULT 15 /* IB link default (sleep/poll) */
290#define QIB_IB_CFG_PKEYS 16 /* update partition keys */
291#define QIB_IB_CFG_MTU 17 /* update MTU in IBC */
292#define QIB_IB_CFG_LSTATE 18 /* update linkcmd and linkinitcmd in IBC */
293#define QIB_IB_CFG_VL_HIGH_LIMIT 19
294#define QIB_IB_CFG_PMA_TICKS 20 /* PMA sample tick resolution */
295#define QIB_IB_CFG_PORT 21 /* switch port we are connected to */
296
297/*
298 * for CFG_LSTATE: LINKCMD in upper 16 bits, LINKINITCMD in lower 16
299 * IB_LINKINITCMD_POLL and SLEEP are also used as set/get values for
300 * QIB_IB_CFG_LINKDEFAULT cmd
301 */
302#define IB_LINKCMD_DOWN (0 << 16)
303#define IB_LINKCMD_ARMED (1 << 16)
304#define IB_LINKCMD_ACTIVE (2 << 16)
305#define IB_LINKINITCMD_NOP 0
306#define IB_LINKINITCMD_POLL 1
307#define IB_LINKINITCMD_SLEEP 2
308#define IB_LINKINITCMD_DISABLE 3
309
310/*
311 * valid states passed to qib_set_linkstate() user call
312 */
313#define QIB_IB_LINKDOWN 0
314#define QIB_IB_LINKARM 1
315#define QIB_IB_LINKACTIVE 2
316#define QIB_IB_LINKDOWN_ONLY 3
317#define QIB_IB_LINKDOWN_SLEEP 4
318#define QIB_IB_LINKDOWN_DISABLE 5
319
320/*
321 * These 7 values (SDR, DDR, and QDR may be ORed for auto-speed
322 * negotiation) are used for the 3rd argument to path_f_set_ib_cfg
323 * with cmd QIB_IB_CFG_SPD_ENB, by direct calls or via sysfs. They
324 * are also the possible values for qib_link_speed_enabled and active
325 * The values were chosen to match values used within the IB spec.
326 */
327#define QIB_IB_SDR 1
328#define QIB_IB_DDR 2
329#define QIB_IB_QDR 4
330
331#define QIB_DEFAULT_MTU 4096
332
333/* max number of IB ports supported per HCA */
334#define QIB_MAX_IB_PORTS 2
335
336/*
337 * Possible IB config parameters for f_get/set_ib_table()
338 */
339#define QIB_IB_TBL_VL_HIGH_ARB 1 /* Get/set VL high priority weights */
340#define QIB_IB_TBL_VL_LOW_ARB 2 /* Get/set VL low priority weights */
341
342/*
343 * Possible "operations" for f_rcvctrl(ppd, op, ctxt)
344 * these are bits so they can be combined, e.g.
345 * QIB_RCVCTRL_INTRAVAIL_ENB | QIB_RCVCTRL_CTXT_ENB
346 */
347#define QIB_RCVCTRL_TAILUPD_ENB 0x01
348#define QIB_RCVCTRL_TAILUPD_DIS 0x02
349#define QIB_RCVCTRL_CTXT_ENB 0x04
350#define QIB_RCVCTRL_CTXT_DIS 0x08
351#define QIB_RCVCTRL_INTRAVAIL_ENB 0x10
352#define QIB_RCVCTRL_INTRAVAIL_DIS 0x20
353#define QIB_RCVCTRL_PKEY_ENB 0x40 /* Note, default is enabled */
354#define QIB_RCVCTRL_PKEY_DIS 0x80
355#define QIB_RCVCTRL_BP_ENB 0x0100
356#define QIB_RCVCTRL_BP_DIS 0x0200
357#define QIB_RCVCTRL_TIDFLOW_ENB 0x0400
358#define QIB_RCVCTRL_TIDFLOW_DIS 0x0800
359
360/*
361 * Possible "operations" for f_sendctrl(ppd, op, var)
362 * these are bits so they can be combined, e.g.
363 * QIB_SENDCTRL_BUFAVAIL_ENB | QIB_SENDCTRL_ENB
364 * Some operations (e.g. DISARM, ABORT) are known to
365 * be "one-shot", so do not modify shadow.
366 */
367#define QIB_SENDCTRL_DISARM (0x1000)
368#define QIB_SENDCTRL_DISARM_BUF(bufn) ((bufn) | QIB_SENDCTRL_DISARM)
369 /* available (0x2000) */
370#define QIB_SENDCTRL_AVAIL_DIS (0x4000)
371#define QIB_SENDCTRL_AVAIL_ENB (0x8000)
372#define QIB_SENDCTRL_AVAIL_BLIP (0x10000)
373#define QIB_SENDCTRL_SEND_DIS (0x20000)
374#define QIB_SENDCTRL_SEND_ENB (0x40000)
375#define QIB_SENDCTRL_FLUSH (0x80000)
376#define QIB_SENDCTRL_CLEAR (0x100000)
377#define QIB_SENDCTRL_DISARM_ALL (0x200000)
378
379/*
380 * These are the generic indices for requesting per-port
381 * counter values via the f_portcntr function. They
382 * are always returned as 64 bit values, although most
383 * are 32 bit counters.
384 */
385/* send-related counters */
386#define QIBPORTCNTR_PKTSEND 0U
387#define QIBPORTCNTR_WORDSEND 1U
388#define QIBPORTCNTR_PSXMITDATA 2U
389#define QIBPORTCNTR_PSXMITPKTS 3U
390#define QIBPORTCNTR_PSXMITWAIT 4U
391#define QIBPORTCNTR_SENDSTALL 5U
392/* receive-related counters */
393#define QIBPORTCNTR_PKTRCV 6U
394#define QIBPORTCNTR_PSRCVDATA 7U
395#define QIBPORTCNTR_PSRCVPKTS 8U
396#define QIBPORTCNTR_RCVEBP 9U
397#define QIBPORTCNTR_RCVOVFL 10U
398#define QIBPORTCNTR_WORDRCV 11U
399/* IB link related error counters */
400#define QIBPORTCNTR_RXLOCALPHYERR 12U
401#define QIBPORTCNTR_RXVLERR 13U
402#define QIBPORTCNTR_ERRICRC 14U
403#define QIBPORTCNTR_ERRVCRC 15U
404#define QIBPORTCNTR_ERRLPCRC 16U
405#define QIBPORTCNTR_BADFORMAT 17U
406#define QIBPORTCNTR_ERR_RLEN 18U
407#define QIBPORTCNTR_IBSYMBOLERR 19U
408#define QIBPORTCNTR_INVALIDRLEN 20U
409#define QIBPORTCNTR_UNSUPVL 21U
410#define QIBPORTCNTR_EXCESSBUFOVFL 22U
411#define QIBPORTCNTR_ERRLINK 23U
412#define QIBPORTCNTR_IBLINKDOWN 24U
413#define QIBPORTCNTR_IBLINKERRRECOV 25U
414#define QIBPORTCNTR_LLI 26U
415/* other error counters */
416#define QIBPORTCNTR_RXDROPPKT 27U
417#define QIBPORTCNTR_VL15PKTDROP 28U
418#define QIBPORTCNTR_ERRPKEY 29U
419#define QIBPORTCNTR_KHDROVFL 30U
420/* sampling counters (these are actually control registers) */
421#define QIBPORTCNTR_PSINTERVAL 31U
422#define QIBPORTCNTR_PSSTART 32U
423#define QIBPORTCNTR_PSSTAT 33U
424
425/* how often we check for packet activity for "power on hours (in seconds) */
426#define ACTIVITY_TIMER 5
427
428#define MAX_NAME_SIZE 64
429
430#ifdef CONFIG_INFINIBAND_QIB_DCA
431struct qib_irq_notify;
432#endif
433
434struct qib_msix_entry {
435 void *arg;
436#ifdef CONFIG_INFINIBAND_QIB_DCA
437 int dca;
438 int rcv;
439 struct qib_irq_notify *notifier;
440#endif
441 cpumask_var_t mask;
442};
443
444/* Below is an opaque struct. Each chip (device) can maintain
445 * private data needed for its operation, but not germane to the
446 * rest of the driver. For convenience, we define another that
447 * is chip-specific, per-port
448 */
449struct qib_chip_specific;
450struct qib_chipport_specific;
451
452enum qib_sdma_states {
453 qib_sdma_state_s00_hw_down,
454 qib_sdma_state_s10_hw_start_up_wait,
455 qib_sdma_state_s20_idle,
456 qib_sdma_state_s30_sw_clean_up_wait,
457 qib_sdma_state_s40_hw_clean_up_wait,
458 qib_sdma_state_s50_hw_halt_wait,
459 qib_sdma_state_s99_running,
460};
461
462enum qib_sdma_events {
463 qib_sdma_event_e00_go_hw_down,
464 qib_sdma_event_e10_go_hw_start,
465 qib_sdma_event_e20_hw_started,
466 qib_sdma_event_e30_go_running,
467 qib_sdma_event_e40_sw_cleaned,
468 qib_sdma_event_e50_hw_cleaned,
469 qib_sdma_event_e60_hw_halted,
470 qib_sdma_event_e70_go_idle,
471 qib_sdma_event_e7220_err_halted,
472 qib_sdma_event_e7322_err_halted,
473 qib_sdma_event_e90_timer_tick,
474};
475
476struct sdma_set_state_action {
477 unsigned op_enable:1;
478 unsigned op_intenable:1;
479 unsigned op_halt:1;
480 unsigned op_drain:1;
481 unsigned go_s99_running_tofalse:1;
482 unsigned go_s99_running_totrue:1;
483};
484
485struct qib_sdma_state {
486 struct kref kref;
487 struct completion comp;
488 enum qib_sdma_states current_state;
489 struct sdma_set_state_action *set_state_action;
490 unsigned current_op;
491 unsigned go_s99_running;
492 unsigned first_sendbuf;
493 unsigned last_sendbuf; /* really last +1 */
494 /* debugging/devel */
495 enum qib_sdma_states previous_state;
496 unsigned previous_op;
497 enum qib_sdma_events last_event;
498};
499
500struct xmit_wait {
501 struct timer_list timer;
502 u64 counter;
503 u8 flags;
504 struct cache {
505 u64 psxmitdata;
506 u64 psrcvdata;
507 u64 psxmitpkts;
508 u64 psrcvpkts;
509 u64 psxmitwait;
510 } counter_cache;
511};
512
513/*
514 * The structure below encapsulates data relevant to a physical IB Port.
515 * Current chips support only one such port, but the separation
516 * clarifies things a bit. Note that to conform to IB conventions,
517 * port-numbers are one-based. The first or only port is port1.
518 */
519struct qib_pportdata {
520 struct qib_ibport ibport_data;
521
522 struct qib_devdata *dd;
523 struct qib_chippport_specific *cpspec; /* chip-specific per-port */
524
525 /* GUID for this interface, in network order */
526 __be64 guid;
527
528 /* QIB_POLL, etc. link-state specific flags, per port */
529 u32 lflags;
530 /* qib_lflags driver is waiting for */
531 u32 state_wanted;
532 spinlock_t lflags_lock;
533
534 /* ref count for each pkey */
535 atomic_t pkeyrefs[4];
536
537 /*
538 * this address is mapped readonly into user processes so they can
539 * get status cheaply, whenever they want. One qword of status per port
540 */
541 u64 *statusp;
542
543 /* SendDMA related entries */
544
545 /* read mostly */
546 struct qib_sdma_desc *sdma_descq;
547 struct workqueue_struct *qib_wq;
548 struct qib_sdma_state sdma_state;
549 dma_addr_t sdma_descq_phys;
550 volatile __le64 *sdma_head_dma; /* DMA'ed by chip */
551 dma_addr_t sdma_head_phys;
552 u16 sdma_descq_cnt;
553
554 /* read/write using lock */
555 spinlock_t sdma_lock ____cacheline_aligned_in_smp;
556 struct list_head sdma_activelist;
557 struct list_head sdma_userpending;
558 u64 sdma_descq_added;
559 u64 sdma_descq_removed;
560 u16 sdma_descq_tail;
561 u16 sdma_descq_head;
562 u8 sdma_generation;
563 u8 sdma_intrequest;
564
565 struct tasklet_struct sdma_sw_clean_up_task
566 ____cacheline_aligned_in_smp;
567
568 wait_queue_head_t state_wait; /* for state_wanted */
569
570 /* HoL blocking for SMP replies */
571 unsigned hol_state;
572 struct timer_list hol_timer;
573
574 /*
575 * Shadow copies of registers; size indicates read access size.
576 * Most of them are readonly, but some are write-only register,
577 * where we manipulate the bits in the shadow copy, and then write
578 * the shadow copy to qlogic_ib.
579 *
580 * We deliberately make most of these 32 bits, since they have
581 * restricted range. For any that we read, we won't to generate 32
582 * bit accesses, since Opteron will generate 2 separate 32 bit HT
583 * transactions for a 64 bit read, and we want to avoid unnecessary
584 * bus transactions.
585 */
586
587 /* This is the 64 bit group */
588 /* last ibcstatus. opaque outside chip-specific code */
589 u64 lastibcstat;
590
591 /* these are the "32 bit" regs */
592
593 /*
594 * the following two are 32-bit bitmasks, but {test,clear,set}_bit
595 * all expect bit fields to be "unsigned long"
596 */
597 unsigned long p_rcvctrl; /* shadow per-port rcvctrl */
598 unsigned long p_sendctrl; /* shadow per-port sendctrl */
599
600 u32 ibmtu; /* The MTU programmed for this unit */
601 /*
602 * Current max size IB packet (in bytes) including IB headers, that
603 * we can send. Changes when ibmtu changes.
604 */
605 u32 ibmaxlen;
606 /*
607 * ibmaxlen at init time, limited by chip and by receive buffer
608 * size. Not changed after init.
609 */
610 u32 init_ibmaxlen;
611 /* LID programmed for this instance */
612 u16 lid;
613 /* list of pkeys programmed; 0 if not set */
614 u16 pkeys[4];
615 /* LID mask control */
616 u8 lmc;
617 u8 link_width_supported;
618 u16 link_speed_supported;
619 u8 link_width_enabled;
620 u16 link_speed_enabled;
621 u8 link_width_active;
622 u16 link_speed_active;
623 u8 vls_supported;
624 u8 vls_operational;
625 /* Rx Polarity inversion (compensate for ~tx on partner) */
626 u8 rx_pol_inv;
627
628 u8 hw_pidx; /* physical port index */
629 u32 port; /* IB port number and index into dd->pports - 1 */
630
631 u8 delay_mult;
632
633 /* used to override LED behavior */
634 u8 led_override; /* Substituted for normal value, if non-zero */
635 u16 led_override_timeoff; /* delta to next timer event */
636 u8 led_override_vals[2]; /* Alternates per blink-frame */
637 u8 led_override_phase; /* Just counts, LSB picks from vals[] */
638 atomic_t led_override_timer_active;
639 /* Used to flash LEDs in override mode */
640 struct timer_list led_override_timer;
641 struct xmit_wait cong_stats;
642 struct timer_list symerr_clear_timer;
643
644 /* Synchronize access between driver writes and sysfs reads */
645 spinlock_t cc_shadow_lock
646 ____cacheline_aligned_in_smp;
647
648 /* Shadow copy of the congestion control table */
649 struct cc_table_shadow *ccti_entries_shadow;
650
651 /* Shadow copy of the congestion control entries */
652 struct ib_cc_congestion_setting_attr_shadow *congestion_entries_shadow;
653
654 /* List of congestion control table entries */
655 struct ib_cc_table_entry_shadow *ccti_entries;
656
657 /* 16 congestion entries with each entry corresponding to a SL */
658 struct ib_cc_congestion_entry_shadow *congestion_entries;
659
660 /* Maximum number of congestion control entries that the agent expects
661 * the manager to send.
662 */
663 u16 cc_supported_table_entries;
664
665 /* Total number of congestion control table entries */
666 u16 total_cct_entry;
667
668 /* Bit map identifying service level */
669 u16 cc_sl_control_map;
670
671 /* maximum congestion control table index */
672 u16 ccti_limit;
673
674 /* CA's max number of 64 entry units in the congestion control table */
675 u8 cc_max_table_entries;
676};
677
678/* Observers. Not to be taken lightly, possibly not to ship. */
679/*
680 * If a diag read or write is to (bottom <= offset <= top),
681 * the "hook" is called, allowing, e.g. shadows to be
682 * updated in sync with the driver. struct diag_observer
683 * is the "visible" part.
684 */
685struct diag_observer;
686
687typedef int (*diag_hook) (struct qib_devdata *dd,
688 const struct diag_observer *op,
689 u32 offs, u64 *data, u64 mask, int only_32);
690
691struct diag_observer {
692 diag_hook hook;
693 u32 bottom;
694 u32 top;
695};
696
697extern int qib_register_observer(struct qib_devdata *dd,
698 const struct diag_observer *op);
699
700/* Only declared here, not defined. Private to diags */
701struct diag_observer_list_elt;
702
703/* device data struct now contains only "general per-device" info.
704 * fields related to a physical IB port are in a qib_pportdata struct,
705 * described above) while fields only used by a particular chip-type are in
706 * a qib_chipdata struct, whose contents are opaque to this file.
707 */
708struct qib_devdata {
709 struct qib_ibdev verbs_dev; /* must be first */
710 struct list_head list;
711 /* pointers to related structs for this device */
712 /* pci access data structure */
713 struct pci_dev *pcidev;
714 struct cdev *user_cdev;
715 struct cdev *diag_cdev;
716 struct device *user_device;
717 struct device *diag_device;
718
719 /* mem-mapped pointer to base of chip regs */
720 u64 __iomem *kregbase;
721 /* end of mem-mapped chip space excluding sendbuf and user regs */
722 u64 __iomem *kregend;
723 /* physical address of chip for io_remap, etc. */
724 resource_size_t physaddr;
725 /* qib_cfgctxts pointers */
726 struct qib_ctxtdata **rcd; /* Receive Context Data */
727
728 /* qib_pportdata, points to array of (physical) port-specific
729 * data structs, indexed by pidx (0..n-1)
730 */
731 struct qib_pportdata *pport;
732 struct qib_chip_specific *cspec; /* chip-specific */
733
734 /* kvirt address of 1st 2k pio buffer */
735 void __iomem *pio2kbase;
736 /* kvirt address of 1st 4k pio buffer */
737 void __iomem *pio4kbase;
738 /* mem-mapped pointer to base of PIO buffers (if using WC PAT) */
739 void __iomem *piobase;
740 /* mem-mapped pointer to base of user chip regs (if using WC PAT) */
741 u64 __iomem *userbase;
742 void __iomem *piovl15base; /* base of VL15 buffers, if not WC */
743 /*
744 * points to area where PIOavail registers will be DMA'ed.
745 * Has to be on a page of it's own, because the page will be
746 * mapped into user program space. This copy is *ONLY* ever
747 * written by DMA, not by the driver! Need a copy per device
748 * when we get to multiple devices
749 */
750 volatile __le64 *pioavailregs_dma; /* DMA'ed by chip */
751 /* physical address where updates occur */
752 dma_addr_t pioavailregs_phys;
753
754 /* device-specific implementations of functions needed by
755 * common code. Contrary to previous consensus, we can't
756 * really just point to a device-specific table, because we
757 * may need to "bend", e.g. *_f_put_tid
758 */
759 /* fallback to alternate interrupt type if possible */
760 int (*f_intr_fallback)(struct qib_devdata *);
761 /* hard reset chip */
762 int (*f_reset)(struct qib_devdata *);
763 void (*f_quiet_serdes)(struct qib_pportdata *);
764 int (*f_bringup_serdes)(struct qib_pportdata *);
765 int (*f_early_init)(struct qib_devdata *);
766 void (*f_clear_tids)(struct qib_devdata *, struct qib_ctxtdata *);
767 void (*f_put_tid)(struct qib_devdata *, u64 __iomem*,
768 u32, unsigned long);
769 void (*f_cleanup)(struct qib_devdata *);
770 void (*f_setextled)(struct qib_pportdata *, u32);
771 /* fill out chip-specific fields */
772 int (*f_get_base_info)(struct qib_ctxtdata *, struct qib_base_info *);
773 /* free irq */
774 void (*f_free_irq)(struct qib_devdata *);
775 struct qib_message_header *(*f_get_msgheader)
776 (struct qib_devdata *, __le32 *);
777 void (*f_config_ctxts)(struct qib_devdata *);
778 int (*f_get_ib_cfg)(struct qib_pportdata *, int);
779 int (*f_set_ib_cfg)(struct qib_pportdata *, int, u32);
780 int (*f_set_ib_loopback)(struct qib_pportdata *, const char *);
781 int (*f_get_ib_table)(struct qib_pportdata *, int, void *);
782 int (*f_set_ib_table)(struct qib_pportdata *, int, void *);
783 u32 (*f_iblink_state)(u64);
784 u8 (*f_ibphys_portstate)(u64);
785 void (*f_xgxs_reset)(struct qib_pportdata *);
786 /* per chip actions needed for IB Link up/down changes */
787 int (*f_ib_updown)(struct qib_pportdata *, int, u64);
788 u32 __iomem *(*f_getsendbuf)(struct qib_pportdata *, u64, u32 *);
789 /* Read/modify/write of GPIO pins (potentially chip-specific */
790 int (*f_gpio_mod)(struct qib_devdata *dd, u32 out, u32 dir,
791 u32 mask);
792 /* Enable writes to config EEPROM (if supported) */
793 int (*f_eeprom_wen)(struct qib_devdata *dd, int wen);
794 /*
795 * modify rcvctrl shadow[s] and write to appropriate chip-regs.
796 * see above QIB_RCVCTRL_xxx_ENB/DIS for operations.
797 * (ctxt == -1) means "all contexts", only meaningful for
798 * clearing. Could remove if chip_spec shutdown properly done.
799 */
800 void (*f_rcvctrl)(struct qib_pportdata *, unsigned int op,
801 int ctxt);
802 /* Read/modify/write sendctrl appropriately for op and port. */
803 void (*f_sendctrl)(struct qib_pportdata *, u32 op);
804 void (*f_set_intr_state)(struct qib_devdata *, u32);
805 void (*f_set_armlaunch)(struct qib_devdata *, u32);
806 void (*f_wantpiobuf_intr)(struct qib_devdata *, u32);
807 int (*f_late_initreg)(struct qib_devdata *);
808 int (*f_init_sdma_regs)(struct qib_pportdata *);
809 u16 (*f_sdma_gethead)(struct qib_pportdata *);
810 int (*f_sdma_busy)(struct qib_pportdata *);
811 void (*f_sdma_update_tail)(struct qib_pportdata *, u16);
812 void (*f_sdma_set_desc_cnt)(struct qib_pportdata *, unsigned);
813 void (*f_sdma_sendctrl)(struct qib_pportdata *, unsigned);
814 void (*f_sdma_hw_clean_up)(struct qib_pportdata *);
815 void (*f_sdma_hw_start_up)(struct qib_pportdata *);
816 void (*f_sdma_init_early)(struct qib_pportdata *);
817 void (*f_set_cntr_sample)(struct qib_pportdata *, u32, u32);
818 void (*f_update_usrhead)(struct qib_ctxtdata *, u64, u32, u32, u32);
819 u32 (*f_hdrqempty)(struct qib_ctxtdata *);
820 u64 (*f_portcntr)(struct qib_pportdata *, u32);
821 u32 (*f_read_cntrs)(struct qib_devdata *, loff_t, char **,
822 u64 **);
823 u32 (*f_read_portcntrs)(struct qib_devdata *, loff_t, u32,
824 char **, u64 **);
825 u32 (*f_setpbc_control)(struct qib_pportdata *, u32, u8, u8);
826 void (*f_initvl15_bufs)(struct qib_devdata *);
827 void (*f_init_ctxt)(struct qib_ctxtdata *);
828 void (*f_txchk_change)(struct qib_devdata *, u32, u32, u32,
829 struct qib_ctxtdata *);
830 void (*f_writescratch)(struct qib_devdata *, u32);
831 int (*f_tempsense_rd)(struct qib_devdata *, int regnum);
832#ifdef CONFIG_INFINIBAND_QIB_DCA
833 int (*f_notify_dca)(struct qib_devdata *, unsigned long event);
834#endif
835
836 char *boardname; /* human readable board info */
837
838 /* template for writing TIDs */
839 u64 tidtemplate;
840 /* value to write to free TIDs */
841 u64 tidinvalid;
842
843 /* number of registers used for pioavail */
844 u32 pioavregs;
845 /* device (not port) flags, basically device capabilities */
846 u32 flags;
847 /* last buffer for user use */
848 u32 lastctxt_piobuf;
849
850 /* reset value */
851 u64 z_int_counter;
852 /* percpu intcounter */
853 u64 __percpu *int_counter;
854
855 /* pio bufs allocated per ctxt */
856 u32 pbufsctxt;
857 /* if remainder on bufs/ctxt, ctxts < extrabuf get 1 extra */
858 u32 ctxts_extrabuf;
859 /*
860 * number of ctxts configured as max; zero is set to number chip
861 * supports, less gives more pio bufs/ctxt, etc.
862 */
863 u32 cfgctxts;
864 /*
865 * number of ctxts available for PSM open
866 */
867 u32 freectxts;
868
869 /*
870 * hint that we should update pioavailshadow before
871 * looking for a PIO buffer
872 */
873 u32 upd_pio_shadow;
874
875 /* internal debugging stats */
876 u32 maxpkts_call;
877 u32 avgpkts_call;
878 u64 nopiobufs;
879
880 /* PCI Vendor ID (here for NodeInfo) */
881 u16 vendorid;
882 /* PCI Device ID (here for NodeInfo) */
883 u16 deviceid;
884 /* for write combining settings */
885 int wc_cookie;
886 unsigned long wc_base;
887 unsigned long wc_len;
888
889 /* shadow copy of struct page *'s for exp tid pages */
890 struct page **pageshadow;
891 /* shadow copy of dma handles for exp tid pages */
892 dma_addr_t *physshadow;
893 u64 __iomem *egrtidbase;
894 spinlock_t sendctrl_lock; /* protect changes to sendctrl shadow */
895 /* around rcd and (user ctxts) ctxt_cnt use (intr vs free) */
896 spinlock_t uctxt_lock; /* rcd and user context changes */
897 /*
898 * per unit status, see also portdata statusp
899 * mapped readonly into user processes so they can get unit and
900 * IB link status cheaply
901 */
902 u64 *devstatusp;
903 char *freezemsg; /* freeze msg if hw error put chip in freeze */
904 u32 freezelen; /* max length of freezemsg */
905 /* timer used to prevent stats overflow, error throttling, etc. */
906 struct timer_list stats_timer;
907
908 /* timer to verify interrupts work, and fallback if possible */
909 struct timer_list intrchk_timer;
910 unsigned long ureg_align; /* user register alignment */
911
912 /*
913 * Protects pioavailshadow, pioavailkernel, pio_need_disarm, and
914 * pio_writing.
915 */
916 spinlock_t pioavail_lock;
917 /*
918 * index of last buffer to optimize search for next
919 */
920 u32 last_pio;
921 /*
922 * min kernel pio buffer to optimize search
923 */
924 u32 min_kernel_pio;
925 /*
926 * Shadow copies of registers; size indicates read access size.
927 * Most of them are readonly, but some are write-only register,
928 * where we manipulate the bits in the shadow copy, and then write
929 * the shadow copy to qlogic_ib.
930 *
931 * We deliberately make most of these 32 bits, since they have
932 * restricted range. For any that we read, we won't to generate 32
933 * bit accesses, since Opteron will generate 2 separate 32 bit HT
934 * transactions for a 64 bit read, and we want to avoid unnecessary
935 * bus transactions.
936 */
937
938 /* This is the 64 bit group */
939
940 unsigned long pioavailshadow[6];
941 /* bitmap of send buffers available for the kernel to use with PIO. */
942 unsigned long pioavailkernel[6];
943 /* bitmap of send buffers which need to be disarmed. */
944 unsigned long pio_need_disarm[3];
945 /* bitmap of send buffers which are being written to. */
946 unsigned long pio_writing[3];
947 /* kr_revision shadow */
948 u64 revision;
949 /* Base GUID for device (from eeprom, network order) */
950 __be64 base_guid;
951
952 /*
953 * kr_sendpiobufbase value (chip offset of pio buffers), and the
954 * base of the 2KB buffer s(user processes only use 2K)
955 */
956 u64 piobufbase;
957 u32 pio2k_bufbase;
958
959 /* these are the "32 bit" regs */
960
961 /* number of GUIDs in the flash for this interface */
962 u32 nguid;
963 /*
964 * the following two are 32-bit bitmasks, but {test,clear,set}_bit
965 * all expect bit fields to be "unsigned long"
966 */
967 unsigned long rcvctrl; /* shadow per device rcvctrl */
968 unsigned long sendctrl; /* shadow per device sendctrl */
969
970 /* value we put in kr_rcvhdrcnt */
971 u32 rcvhdrcnt;
972 /* value we put in kr_rcvhdrsize */
973 u32 rcvhdrsize;
974 /* value we put in kr_rcvhdrentsize */
975 u32 rcvhdrentsize;
976 /* kr_ctxtcnt value */
977 u32 ctxtcnt;
978 /* kr_pagealign value */
979 u32 palign;
980 /* number of "2KB" PIO buffers */
981 u32 piobcnt2k;
982 /* size in bytes of "2KB" PIO buffers */
983 u32 piosize2k;
984 /* max usable size in dwords of a "2KB" PIO buffer before going "4KB" */
985 u32 piosize2kmax_dwords;
986 /* number of "4KB" PIO buffers */
987 u32 piobcnt4k;
988 /* size in bytes of "4KB" PIO buffers */
989 u32 piosize4k;
990 /* kr_rcvegrbase value */
991 u32 rcvegrbase;
992 /* kr_rcvtidbase value */
993 u32 rcvtidbase;
994 /* kr_rcvtidcnt value */
995 u32 rcvtidcnt;
996 /* kr_userregbase */
997 u32 uregbase;
998 /* shadow the control register contents */
999 u32 control;
1000
1001 /* chip address space used by 4k pio buffers */
1002 u32 align4k;
1003 /* size of each rcvegrbuffer */
1004 u16 rcvegrbufsize;
1005 /* log2 of above */
1006 u16 rcvegrbufsize_shift;
1007 /* localbus width (1, 2,4,8,16,32) from config space */
1008 u32 lbus_width;
1009 /* localbus speed in MHz */
1010 u32 lbus_speed;
1011 int unit; /* unit # of this chip */
1012
1013 /* start of CHIP_SPEC move to chipspec, but need code changes */
1014 /* low and high portions of MSI capability/vector */
1015 u32 msi_lo;
1016 /* saved after PCIe init for restore after reset */
1017 u32 msi_hi;
1018 /* MSI data (vector) saved for restore */
1019 u16 msi_data;
1020 /* so we can rewrite it after a chip reset */
1021 u32 pcibar0;
1022 /* so we can rewrite it after a chip reset */
1023 u32 pcibar1;
1024 u64 rhdrhead_intr_off;
1025
1026 /*
1027 * ASCII serial number, from flash, large enough for original
1028 * all digit strings, and longer QLogic serial number format
1029 */
1030 u8 serial[16];
1031 /* human readable board version */
1032 u8 boardversion[96];
1033 u8 lbus_info[32]; /* human readable localbus info */
1034 /* chip major rev, from qib_revision */
1035 u8 majrev;
1036 /* chip minor rev, from qib_revision */
1037 u8 minrev;
1038
1039 /* Misc small ints */
1040 /* Number of physical ports available */
1041 u8 num_pports;
1042 /* Lowest context number which can be used by user processes */
1043 u8 first_user_ctxt;
1044 u8 n_krcv_queues;
1045 u8 qpn_mask;
1046 u8 skip_kctxt_mask;
1047
1048 u16 rhf_offset; /* offset of RHF within receive header entry */
1049
1050 /*
1051 * GPIO pins for twsi-connected devices, and device code for eeprom
1052 */
1053 u8 gpio_sda_num;
1054 u8 gpio_scl_num;
1055 u8 twsi_eeprom_dev;
1056 u8 board_atten;
1057
1058 /* Support (including locks) for EEPROM logging of errors and time */
1059 /* control access to actual counters, timer */
1060 spinlock_t eep_st_lock;
1061 /* control high-level access to EEPROM */
1062 struct mutex eep_lock;
1063 uint64_t traffic_wds;
1064 struct qib_diag_client *diag_client;
1065 spinlock_t qib_diag_trans_lock; /* protect diag observer ops */
1066 struct diag_observer_list_elt *diag_observer_list;
1067
1068 u8 psxmitwait_supported;
1069 /* cycle length of PS* counters in HW (in picoseconds) */
1070 u16 psxmitwait_check_rate;
1071 /* high volume overflow errors defered to tasklet */
1072 struct tasklet_struct error_tasklet;
1073
1074 int assigned_node_id; /* NUMA node closest to HCA */
1075};
1076
1077/* hol_state values */
1078#define QIB_HOL_UP 0
1079#define QIB_HOL_INIT 1
1080
1081#define QIB_SDMA_SENDCTRL_OP_ENABLE (1U << 0)
1082#define QIB_SDMA_SENDCTRL_OP_INTENABLE (1U << 1)
1083#define QIB_SDMA_SENDCTRL_OP_HALT (1U << 2)
1084#define QIB_SDMA_SENDCTRL_OP_CLEANUP (1U << 3)
1085#define QIB_SDMA_SENDCTRL_OP_DRAIN (1U << 4)
1086
1087/* operation types for f_txchk_change() */
1088#define TXCHK_CHG_TYPE_DIS1 3
1089#define TXCHK_CHG_TYPE_ENAB1 2
1090#define TXCHK_CHG_TYPE_KERN 1
1091#define TXCHK_CHG_TYPE_USER 0
1092
1093#define QIB_CHASE_TIME msecs_to_jiffies(145)
1094#define QIB_CHASE_DIS_TIME msecs_to_jiffies(160)
1095
1096/* Private data for file operations */
1097struct qib_filedata {
1098 struct qib_ctxtdata *rcd;
1099 unsigned subctxt;
1100 unsigned tidcursor;
1101 struct qib_user_sdma_queue *pq;
1102 int rec_cpu_num; /* for cpu affinity; -1 if none */
1103};
1104
1105extern struct xarray qib_dev_table;
1106extern struct qib_devdata *qib_lookup(int unit);
1107extern u32 qib_cpulist_count;
1108extern unsigned long *qib_cpulist;
1109extern unsigned qib_cc_table_size;
1110
1111int qib_init(struct qib_devdata *, int);
1112int init_chip_wc_pat(struct qib_devdata *dd, u32);
1113int qib_enable_wc(struct qib_devdata *dd);
1114void qib_disable_wc(struct qib_devdata *dd);
1115int qib_count_units(int *npresentp, int *nupp);
1116int qib_count_active_units(void);
1117
1118int qib_cdev_init(int minor, const char *name,
1119 const struct file_operations *fops,
1120 struct cdev **cdevp, struct device **devp);
1121void qib_cdev_cleanup(struct cdev **cdevp, struct device **devp);
1122int qib_dev_init(void);
1123void qib_dev_cleanup(void);
1124
1125int qib_diag_add(struct qib_devdata *);
1126void qib_diag_remove(struct qib_devdata *);
1127void qib_handle_e_ibstatuschanged(struct qib_pportdata *, u64);
1128void qib_sdma_update_tail(struct qib_pportdata *, u16); /* hold sdma_lock */
1129
1130int qib_decode_err(struct qib_devdata *dd, char *buf, size_t blen, u64 err);
1131void qib_bad_intrstatus(struct qib_devdata *);
1132void qib_handle_urcv(struct qib_devdata *, u64);
1133
1134/* clean up any per-chip chip-specific stuff */
1135void qib_chip_cleanup(struct qib_devdata *);
1136/* clean up any chip type-specific stuff */
1137void qib_chip_done(void);
1138
1139/* check to see if we have to force ordering for write combining */
1140int qib_unordered_wc(void);
1141void qib_pio_copy(void __iomem *to, const void *from, size_t count);
1142
1143void qib_disarm_piobufs(struct qib_devdata *, unsigned, unsigned);
1144int qib_disarm_piobufs_ifneeded(struct qib_ctxtdata *);
1145void qib_disarm_piobufs_set(struct qib_devdata *, unsigned long *, unsigned);
1146void qib_cancel_sends(struct qib_pportdata *);
1147
1148int qib_create_rcvhdrq(struct qib_devdata *, struct qib_ctxtdata *);
1149int qib_setup_eagerbufs(struct qib_ctxtdata *);
1150void qib_set_ctxtcnt(struct qib_devdata *);
1151int qib_create_ctxts(struct qib_devdata *dd);
1152struct qib_ctxtdata *qib_create_ctxtdata(struct qib_pportdata *, u32, int);
1153int qib_init_pportdata(struct qib_pportdata *, struct qib_devdata *, u8, u8);
1154void qib_free_ctxtdata(struct qib_devdata *, struct qib_ctxtdata *);
1155
1156u32 qib_kreceive(struct qib_ctxtdata *, u32 *, u32 *);
1157int qib_reset_device(int);
1158int qib_wait_linkstate(struct qib_pportdata *, u32, int);
1159int qib_set_linkstate(struct qib_pportdata *, u8);
1160int qib_set_mtu(struct qib_pportdata *, u16);
1161int qib_set_lid(struct qib_pportdata *, u32, u8);
1162void qib_hol_down(struct qib_pportdata *);
1163void qib_hol_init(struct qib_pportdata *);
1164void qib_hol_up(struct qib_pportdata *);
1165void qib_hol_event(struct timer_list *);
1166void qib_disable_after_error(struct qib_devdata *);
1167int qib_set_uevent_bits(struct qib_pportdata *, const int);
1168
1169/* for use in system calls, where we want to know device type, etc. */
1170#define ctxt_fp(fp) \
1171 (((struct qib_filedata *)(fp)->private_data)->rcd)
1172#define subctxt_fp(fp) \
1173 (((struct qib_filedata *)(fp)->private_data)->subctxt)
1174#define tidcursor_fp(fp) \
1175 (((struct qib_filedata *)(fp)->private_data)->tidcursor)
1176#define user_sdma_queue_fp(fp) \
1177 (((struct qib_filedata *)(fp)->private_data)->pq)
1178
1179static inline struct qib_devdata *dd_from_ppd(struct qib_pportdata *ppd)
1180{
1181 return ppd->dd;
1182}
1183
1184static inline struct qib_devdata *dd_from_dev(struct qib_ibdev *dev)
1185{
1186 return container_of(dev, struct qib_devdata, verbs_dev);
1187}
1188
1189static inline struct qib_devdata *dd_from_ibdev(struct ib_device *ibdev)
1190{
1191 return dd_from_dev(to_idev(ibdev));
1192}
1193
1194static inline struct qib_pportdata *ppd_from_ibp(struct qib_ibport *ibp)
1195{
1196 return container_of(ibp, struct qib_pportdata, ibport_data);
1197}
1198
1199static inline struct qib_ibport *to_iport(struct ib_device *ibdev, u32 port)
1200{
1201 struct qib_devdata *dd = dd_from_ibdev(ibdev);
1202 u32 pidx = port - 1; /* IB number port from 1, hdw from 0 */
1203
1204 WARN_ON(pidx >= dd->num_pports);
1205 return &dd->pport[pidx].ibport_data;
1206}
1207
1208/*
1209 * values for dd->flags (_device_ related flags) and
1210 */
1211#define QIB_HAS_LINK_LATENCY 0x1 /* supports link latency (IB 1.2) */
1212#define QIB_INITTED 0x2 /* chip and driver up and initted */
1213#define QIB_DOING_RESET 0x4 /* in the middle of doing chip reset */
1214#define QIB_PRESENT 0x8 /* chip accesses can be done */
1215#define QIB_PIO_FLUSH_WC 0x10 /* Needs Write combining flush for PIO */
1216#define QIB_HAS_THRESH_UPDATE 0x40
1217#define QIB_HAS_SDMA_TIMEOUT 0x80
1218#define QIB_USE_SPCL_TRIG 0x100 /* SpecialTrigger launch enabled */
1219#define QIB_NODMA_RTAIL 0x200 /* rcvhdrtail register DMA enabled */
1220#define QIB_HAS_INTX 0x800 /* Supports INTx interrupts */
1221#define QIB_HAS_SEND_DMA 0x1000 /* Supports Send DMA */
1222#define QIB_HAS_VLSUPP 0x2000 /* Supports multiple VLs; PBC different */
1223#define QIB_HAS_HDRSUPP 0x4000 /* Supports header suppression */
1224#define QIB_BADINTR 0x8000 /* severe interrupt problems */
1225#define QIB_DCA_ENABLED 0x10000 /* Direct Cache Access enabled */
1226#define QIB_HAS_QSFP 0x20000 /* device (card instance) has QSFP */
1227#define QIB_SHUTDOWN 0x40000 /* device is shutting down */
1228
1229/*
1230 * values for ppd->lflags (_ib_port_ related flags)
1231 */
1232#define QIBL_LINKV 0x1 /* IB link state valid */
1233#define QIBL_LINKDOWN 0x8 /* IB link is down */
1234#define QIBL_LINKINIT 0x10 /* IB link level is up */
1235#define QIBL_LINKARMED 0x20 /* IB link is ARMED */
1236#define QIBL_LINKACTIVE 0x40 /* IB link is ACTIVE */
1237/* leave a gap for more IB-link state */
1238#define QIBL_IB_AUTONEG_INPROG 0x1000 /* non-IBTA DDR/QDR neg active */
1239#define QIBL_IB_AUTONEG_FAILED 0x2000 /* non-IBTA DDR/QDR neg failed */
1240#define QIBL_IB_LINK_DISABLED 0x4000 /* Linkdown-disable forced,
1241 * Do not try to bring up */
1242#define QIBL_IB_FORCE_NOTIFY 0x8000 /* force notify on next ib change */
1243
1244/* IB dword length mask in PBC (lower 11 bits); same for all chips */
1245#define QIB_PBC_LENGTH_MASK ((1 << 11) - 1)
1246
1247
1248/* ctxt_flag bit offsets */
1249 /* waiting for a packet to arrive */
1250#define QIB_CTXT_WAITING_RCV 2
1251 /* master has not finished initializing */
1252#define QIB_CTXT_MASTER_UNINIT 4
1253 /* waiting for an urgent packet to arrive */
1254#define QIB_CTXT_WAITING_URG 5
1255
1256/* free up any allocated data at closes */
1257void qib_free_data(struct qib_ctxtdata *dd);
1258void qib_chg_pioavailkernel(struct qib_devdata *, unsigned, unsigned,
1259 u32, struct qib_ctxtdata *);
1260struct qib_devdata *qib_init_iba7322_funcs(struct pci_dev *,
1261 const struct pci_device_id *);
1262struct qib_devdata *qib_init_iba7220_funcs(struct pci_dev *,
1263 const struct pci_device_id *);
1264struct qib_devdata *qib_init_iba6120_funcs(struct pci_dev *,
1265 const struct pci_device_id *);
1266void qib_free_devdata(struct qib_devdata *);
1267struct qib_devdata *qib_alloc_devdata(struct pci_dev *pdev, size_t extra);
1268
1269#define QIB_TWSI_NO_DEV 0xFF
1270/* Below qib_twsi_ functions must be called with eep_lock held */
1271int qib_twsi_reset(struct qib_devdata *dd);
1272int qib_twsi_blk_rd(struct qib_devdata *dd, int dev, int addr, void *buffer,
1273 int len);
1274int qib_twsi_blk_wr(struct qib_devdata *dd, int dev, int addr,
1275 const void *buffer, int len);
1276void qib_get_eeprom_info(struct qib_devdata *);
1277void qib_dump_lookup_output_queue(struct qib_devdata *);
1278void qib_force_pio_avail_update(struct qib_devdata *);
1279void qib_clear_symerror_on_linkup(struct timer_list *t);
1280
1281/*
1282 * Set LED override, only the two LSBs have "public" meaning, but
1283 * any non-zero value substitutes them for the Link and LinkTrain
1284 * LED states.
1285 */
1286#define QIB_LED_PHYS 1 /* Physical (linktraining) GREEN LED */
1287#define QIB_LED_LOG 2 /* Logical (link) YELLOW LED */
1288void qib_set_led_override(struct qib_pportdata *ppd, unsigned int val);
1289
1290/* send dma routines */
1291int qib_setup_sdma(struct qib_pportdata *);
1292void qib_teardown_sdma(struct qib_pportdata *);
1293void __qib_sdma_intr(struct qib_pportdata *);
1294void qib_sdma_intr(struct qib_pportdata *);
1295void qib_user_sdma_send_desc(struct qib_pportdata *dd,
1296 struct list_head *pktlist);
1297int qib_sdma_verbs_send(struct qib_pportdata *, struct rvt_sge_state *,
1298 u32, struct qib_verbs_txreq *);
1299/* ppd->sdma_lock should be locked before calling this. */
1300int qib_sdma_make_progress(struct qib_pportdata *dd);
1301
1302/* must be called under qib_sdma_lock */
1303static inline u16 qib_sdma_descq_freecnt(const struct qib_pportdata *ppd)
1304{
1305 return ppd->sdma_descq_cnt -
1306 (ppd->sdma_descq_added - ppd->sdma_descq_removed) - 1;
1307}
1308
1309static inline int __qib_sdma_running(struct qib_pportdata *ppd)
1310{
1311 return ppd->sdma_state.current_state == qib_sdma_state_s99_running;
1312}
1313int qib_sdma_running(struct qib_pportdata *);
1314void dump_sdma_state(struct qib_pportdata *ppd);
1315void __qib_sdma_process_event(struct qib_pportdata *, enum qib_sdma_events);
1316void qib_sdma_process_event(struct qib_pportdata *, enum qib_sdma_events);
1317
1318/*
1319 * number of words used for protocol header if not set by qib_userinit();
1320 */
1321#define QIB_DFLT_RCVHDRSIZE 9
1322
1323/*
1324 * We need to be able to handle an IB header of at least 24 dwords.
1325 * We need the rcvhdrq large enough to handle largest IB header, but
1326 * still have room for a 2KB MTU standard IB packet.
1327 * Additionally, some processor/memory controller combinations
1328 * benefit quite strongly from having the DMA'ed data be cacheline
1329 * aligned and a cacheline multiple, so we set the size to 32 dwords
1330 * (2 64-byte primary cachelines for pretty much all processors of
1331 * interest). The alignment hurts nothing, other than using somewhat
1332 * more memory.
1333 */
1334#define QIB_RCVHDR_ENTSIZE 32
1335
1336int qib_get_user_pages(unsigned long, size_t, struct page **);
1337void qib_release_user_pages(struct page **, size_t);
1338int qib_eeprom_read(struct qib_devdata *, u8, void *, int);
1339int qib_eeprom_write(struct qib_devdata *, u8, const void *, int);
1340u32 __iomem *qib_getsendbuf_range(struct qib_devdata *, u32 *, u32, u32);
1341void qib_sendbuf_done(struct qib_devdata *, unsigned);
1342
1343static inline void qib_clear_rcvhdrtail(const struct qib_ctxtdata *rcd)
1344{
1345 *((u64 *) rcd->rcvhdrtail_kvaddr) = 0ULL;
1346}
1347
1348static inline u32 qib_get_rcvhdrtail(const struct qib_ctxtdata *rcd)
1349{
1350 /*
1351 * volatile because it's a DMA target from the chip, routine is
1352 * inlined, and don't want register caching or reordering.
1353 */
1354 return (u32) le64_to_cpu(
1355 *((volatile __le64 *)rcd->rcvhdrtail_kvaddr)); /* DMA'ed */
1356}
1357
1358/*
1359 * sysfs interface.
1360 */
1361
1362extern const struct attribute_group qib_attr_group;
1363extern const struct attribute_group *qib_attr_port_groups[];
1364
1365int qib_device_create(struct qib_devdata *);
1366void qib_device_remove(struct qib_devdata *);
1367
1368/* Hook for sysfs read of QSFP */
1369extern int qib_qsfp_dump(struct qib_pportdata *ppd, char *buf, int len);
1370
1371int __init qib_init_qibfs(void);
1372int __exit qib_exit_qibfs(void);
1373
1374int qibfs_add(struct qib_devdata *);
1375int qibfs_remove(struct qib_devdata *);
1376
1377int qib_pcie_init(struct pci_dev *, const struct pci_device_id *);
1378int qib_pcie_ddinit(struct qib_devdata *, struct pci_dev *,
1379 const struct pci_device_id *);
1380void qib_pcie_ddcleanup(struct qib_devdata *);
1381int qib_pcie_params(struct qib_devdata *dd, u32 minw, u32 *nent);
1382void qib_free_irq(struct qib_devdata *dd);
1383int qib_reinit_intr(struct qib_devdata *dd);
1384void qib_pcie_getcmd(struct qib_devdata *, u16 *, u8 *, u8 *);
1385void qib_pcie_reenable(struct qib_devdata *, u16, u8, u8);
1386/* interrupts for device */
1387u64 qib_int_counter(struct qib_devdata *);
1388/* interrupt for all devices */
1389u64 qib_sps_ints(void);
1390
1391/*
1392 * dma_addr wrappers - all 0's invalid for hw
1393 */
1394int qib_map_page(struct pci_dev *d, struct page *p, dma_addr_t *daddr);
1395struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi);
1396
1397/*
1398 * Flush write combining store buffers (if present) and perform a write
1399 * barrier.
1400 */
1401static inline void qib_flush_wc(void)
1402{
1403#if defined(CONFIG_X86_64)
1404 asm volatile("sfence" : : : "memory");
1405#else
1406 wmb(); /* no reorder around wc flush */
1407#endif
1408}
1409
1410/* global module parameter variables */
1411extern unsigned qib_ibmtu;
1412extern ushort qib_cfgctxts;
1413extern ushort qib_num_cfg_vls;
1414extern ushort qib_mini_init; /* If set, do few (ideally 0) writes to chip */
1415extern unsigned qib_n_krcv_queues;
1416extern unsigned qib_sdma_fetch_arb;
1417extern unsigned qib_compat_ddr_negotiate;
1418extern int qib_special_trigger;
1419extern unsigned qib_numa_aware;
1420
1421extern struct mutex qib_mutex;
1422
1423/* Number of seconds before our card status check... */
1424#define STATUS_TIMEOUT 60
1425
1426#define QIB_DRV_NAME "ib_qib"
1427#define QIB_USER_MINOR_BASE 0
1428#define QIB_TRACE_MINOR 127
1429#define QIB_DIAGPKT_MINOR 128
1430#define QIB_DIAG_MINOR_BASE 129
1431#define QIB_NMINORS 255
1432
1433#define PCI_VENDOR_ID_PATHSCALE 0x1fc1
1434#define PCI_VENDOR_ID_QLOGIC 0x1077
1435#define PCI_DEVICE_ID_QLOGIC_IB_6120 0x10
1436#define PCI_DEVICE_ID_QLOGIC_IB_7220 0x7220
1437#define PCI_DEVICE_ID_QLOGIC_IB_7322 0x7322
1438
1439/*
1440 * qib_early_err is used (only!) to print early errors before devdata is
1441 * allocated, or when dd->pcidev may not be valid, and at the tail end of
1442 * cleanup when devdata may have been freed, etc. qib_dev_porterr is
1443 * the same as qib_dev_err, but is used when the message really needs
1444 * the IB port# to be definitive as to what's happening..
1445 * All of these go to the trace log, and the trace log entry is done
1446 * first to avoid possible serial port delays from printk.
1447 */
1448#define qib_early_err(dev, fmt, ...) \
1449 dev_err(dev, fmt, ##__VA_ARGS__)
1450
1451#define qib_dev_err(dd, fmt, ...) \
1452 dev_err(&(dd)->pcidev->dev, "%s: " fmt, \
1453 rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)
1454
1455#define qib_dev_warn(dd, fmt, ...) \
1456 dev_warn(&(dd)->pcidev->dev, "%s: " fmt, \
1457 rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), ##__VA_ARGS__)
1458
1459#define qib_dev_porterr(dd, port, fmt, ...) \
1460 dev_err(&(dd)->pcidev->dev, "%s: IB%u:%u " fmt, \
1461 rvt_get_ibdev_name(&(dd)->verbs_dev.rdi), (dd)->unit, (port), \
1462 ##__VA_ARGS__)
1463
1464#define qib_devinfo(pcidev, fmt, ...) \
1465 dev_info(&(pcidev)->dev, fmt, ##__VA_ARGS__)
1466
1467/*
1468 * this is used for formatting hw error messages...
1469 */
1470struct qib_hwerror_msgs {
1471 u64 mask;
1472 const char *msg;
1473 size_t sz;
1474};
1475
1476#define QLOGIC_IB_HWE_MSG(a, b) { .mask = a, .msg = b }
1477
1478/* in qib_intr.c... */
1479void qib_format_hwerrors(u64 hwerrs,
1480 const struct qib_hwerror_msgs *hwerrmsgs,
1481 size_t nhwerrmsgs, char *msg, size_t lmsg);
1482
1483void qib_stop_send_queue(struct rvt_qp *qp);
1484void qib_quiesce_qp(struct rvt_qp *qp);
1485void qib_flush_qp_waiters(struct rvt_qp *qp);
1486int qib_mtu_to_path_mtu(u32 mtu);
1487u32 qib_mtu_from_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp, u32 pmtu);
1488void qib_notify_error_qp(struct rvt_qp *qp);
1489int qib_get_pmtu_from_attr(struct rvt_dev_info *rdi, struct rvt_qp *qp,
1490 struct ib_qp_attr *attr);
1491
1492#endif /* _QIB_KERNEL_H */
drivers/infiniband/hw/qib/qib_6120_regs.h+0−977
@@ -1,977 +0,0 @@
1/*
2 * Copyright (c) 2008, 2009, 2010 QLogic Corporation. All rights reserved.
3 *
4 * This software is available to you under a choice of one of two
5 * licenses. You may choose to be licensed under the terms of the GNU
6 * General Public License (GPL) Version 2, available from the file
7 * COPYING in the main directory of this source tree, or the
8 * OpenIB.org BSD license below:
9 *
10 * Redistribution and use in source and binary forms, with or
11 * without modification, are permitted provided that the following
12 * conditions are met:
13 *
14 * - Redistributions of source code must retain the above
15 * copyright notice, this list of conditions and the following
16 * disclaimer.
17 *
18 * - Redistributions in binary form must reproduce the above
19 * copyright notice, this list of conditions and the following
20 * disclaimer in the documentation and/or other materials
21 * provided with the distribution.
22 *
23 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
26 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
27 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
28 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
29 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
30 * SOFTWARE.
31 */
32
33/* This file is mechanically generated from RTL. Any hand-edits will be lost! */
34
35#define QIB_6120_Revision_OFFS 0x0
36#define QIB_6120_Revision_R_Simulator_LSB 0x3F
37#define QIB_6120_Revision_R_Simulator_RMASK 0x1
38#define QIB_6120_Revision_Reserved_LSB 0x28
39#define QIB_6120_Revision_Reserved_RMASK 0x7FFFFF
40#define QIB_6120_Revision_BoardID_LSB 0x20
41#define QIB_6120_Revision_BoardID_RMASK 0xFF
42#define QIB_6120_Revision_R_SW_LSB 0x18
43#define QIB_6120_Revision_R_SW_RMASK 0xFF
44#define QIB_6120_Revision_R_Arch_LSB 0x10
45#define QIB_6120_Revision_R_Arch_RMASK 0xFF
46#define QIB_6120_Revision_R_ChipRevMajor_LSB 0x8
47#define QIB_6120_Revision_R_ChipRevMajor_RMASK 0xFF
48#define QIB_6120_Revision_R_ChipRevMinor_LSB 0x0
49#define QIB_6120_Revision_R_ChipRevMinor_RMASK 0xFF
50
51#define QIB_6120_Control_OFFS 0x8
52#define QIB_6120_Control_TxLatency_LSB 0x4
53#define QIB_6120_Control_TxLatency_RMASK 0x1
54#define QIB_6120_Control_PCIERetryBufDiagEn_LSB 0x3
55#define QIB_6120_Control_PCIERetryBufDiagEn_RMASK 0x1
56#define QIB_6120_Control_LinkEn_LSB 0x2
57#define QIB_6120_Control_LinkEn_RMASK 0x1
58#define QIB_6120_Control_FreezeMode_LSB 0x1
59#define QIB_6120_Control_FreezeMode_RMASK 0x1
60#define QIB_6120_Control_SyncReset_LSB 0x0
61#define QIB_6120_Control_SyncReset_RMASK 0x1
62
63#define QIB_6120_PageAlign_OFFS 0x10
64
65#define QIB_6120_PortCnt_OFFS 0x18
66
67#define QIB_6120_SendRegBase_OFFS 0x30
68
69#define QIB_6120_UserRegBase_OFFS 0x38
70
71#define QIB_6120_CntrRegBase_OFFS 0x40
72
73#define QIB_6120_Scratch_OFFS 0x48
74#define QIB_6120_Scratch_TopHalf_LSB 0x20
75#define QIB_6120_Scratch_TopHalf_RMASK 0xFFFFFFFF
76#define QIB_6120_Scratch_BottomHalf_LSB 0x0
77#define QIB_6120_Scratch_BottomHalf_RMASK 0xFFFFFFFF
78
79#define QIB_6120_IntBlocked_OFFS 0x60
80#define QIB_6120_IntBlocked_ErrorIntBlocked_LSB 0x1F
81#define QIB_6120_IntBlocked_ErrorIntBlocked_RMASK 0x1
82#define QIB_6120_IntBlocked_PioSetIntBlocked_LSB 0x1E
83#define QIB_6120_IntBlocked_PioSetIntBlocked_RMASK 0x1
84#define QIB_6120_IntBlocked_PioBufAvailIntBlocked_LSB 0x1D
85#define QIB_6120_IntBlocked_PioBufAvailIntBlocked_RMASK 0x1
86#define QIB_6120_IntBlocked_assertGPIOIntBlocked_LSB 0x1C
87#define QIB_6120_IntBlocked_assertGPIOIntBlocked_RMASK 0x1
88#define QIB_6120_IntBlocked_Reserved_LSB 0xF
89#define QIB_6120_IntBlocked_Reserved_RMASK 0x1FFF
90#define QIB_6120_IntBlocked_RcvAvail4IntBlocked_LSB 0x10
91#define QIB_6120_IntBlocked_RcvAvail4IntBlocked_RMASK 0x1
92#define QIB_6120_IntBlocked_RcvAvail3IntBlocked_LSB 0xF
93#define QIB_6120_IntBlocked_RcvAvail3IntBlocked_RMASK 0x1
94#define QIB_6120_IntBlocked_RcvAvail2IntBlocked_LSB 0xE
95#define QIB_6120_IntBlocked_RcvAvail2IntBlocked_RMASK 0x1
96#define QIB_6120_IntBlocked_RcvAvail1IntBlocked_LSB 0xD
97#define QIB_6120_IntBlocked_RcvAvail1IntBlocked_RMASK 0x1
98#define QIB_6120_IntBlocked_RcvAvail0IntBlocked_LSB 0xC
99#define QIB_6120_IntBlocked_RcvAvail0IntBlocked_RMASK 0x1
100#define QIB_6120_IntBlocked_Reserved1_LSB 0x5
101#define QIB_6120_IntBlocked_Reserved1_RMASK 0x7F
102#define QIB_6120_IntBlocked_RcvUrg4IntBlocked_LSB 0x4
103#define QIB_6120_IntBlocked_RcvUrg4IntBlocked_RMASK 0x1
104#define QIB_6120_IntBlocked_RcvUrg3IntBlocked_LSB 0x3
105#define QIB_6120_IntBlocked_RcvUrg3IntBlocked_RMASK 0x1
106#define QIB_6120_IntBlocked_RcvUrg2IntBlocked_LSB 0x2
107#define QIB_6120_IntBlocked_RcvUrg2IntBlocked_RMASK 0x1
108#define QIB_6120_IntBlocked_RcvUrg1IntBlocked_LSB 0x1
109#define QIB_6120_IntBlocked_RcvUrg1IntBlocked_RMASK 0x1
110#define QIB_6120_IntBlocked_RcvUrg0IntBlocked_LSB 0x0
111#define QIB_6120_IntBlocked_RcvUrg0IntBlocked_RMASK 0x1
112
113#define QIB_6120_IntMask_OFFS 0x68
114#define QIB_6120_IntMask_ErrorIntMask_LSB 0x1F
115#define QIB_6120_IntMask_ErrorIntMask_RMASK 0x1
116#define QIB_6120_IntMask_PioSetIntMask_LSB 0x1E
117#define QIB_6120_IntMask_PioSetIntMask_RMASK 0x1
118#define QIB_6120_IntMask_PioBufAvailIntMask_LSB 0x1D
119#define QIB_6120_IntMask_PioBufAvailIntMask_RMASK 0x1
120#define QIB_6120_IntMask_assertGPIOIntMask_LSB 0x1C
121#define QIB_6120_IntMask_assertGPIOIntMask_RMASK 0x1
122#define QIB_6120_IntMask_Reserved_LSB 0x11
123#define QIB_6120_IntMask_Reserved_RMASK 0x7FF
124#define QIB_6120_IntMask_RcvAvail4IntMask_LSB 0x10
125#define QIB_6120_IntMask_RcvAvail4IntMask_RMASK 0x1
126#define QIB_6120_IntMask_RcvAvail3IntMask_LSB 0xF
127#define QIB_6120_IntMask_RcvAvail3IntMask_RMASK 0x1
128#define QIB_6120_IntMask_RcvAvail2IntMask_LSB 0xE
129#define QIB_6120_IntMask_RcvAvail2IntMask_RMASK 0x1
130#define QIB_6120_IntMask_RcvAvail1IntMask_LSB 0xD
131#define QIB_6120_IntMask_RcvAvail1IntMask_RMASK 0x1
132#define QIB_6120_IntMask_RcvAvail0IntMask_LSB 0xC
133#define QIB_6120_IntMask_RcvAvail0IntMask_RMASK 0x1
134#define QIB_6120_IntMask_Reserved1_LSB 0x5
135#define QIB_6120_IntMask_Reserved1_RMASK 0x7F
136#define QIB_6120_IntMask_RcvUrg4IntMask_LSB 0x4
137#define QIB_6120_IntMask_RcvUrg4IntMask_RMASK 0x1
138#define QIB_6120_IntMask_RcvUrg3IntMask_LSB 0x3
139#define QIB_6120_IntMask_RcvUrg3IntMask_RMASK 0x1
140#define QIB_6120_IntMask_RcvUrg2IntMask_LSB 0x2
141#define QIB_6120_IntMask_RcvUrg2IntMask_RMASK 0x1
142#define QIB_6120_IntMask_RcvUrg1IntMask_LSB 0x1
143#define QIB_6120_IntMask_RcvUrg1IntMask_RMASK 0x1
144#define QIB_6120_IntMask_RcvUrg0IntMask_LSB 0x0
145#define QIB_6120_IntMask_RcvUrg0IntMask_RMASK 0x1
146
147#define QIB_6120_IntStatus_OFFS 0x70
148#define QIB_6120_IntStatus_Error_LSB 0x1F
149#define QIB_6120_IntStatus_Error_RMASK 0x1
150#define QIB_6120_IntStatus_PioSent_LSB 0x1E
151#define QIB_6120_IntStatus_PioSent_RMASK 0x1
152#define QIB_6120_IntStatus_PioBufAvail_LSB 0x1D
153#define QIB_6120_IntStatus_PioBufAvail_RMASK 0x1
154#define QIB_6120_IntStatus_assertGPIO_LSB 0x1C
155#define QIB_6120_IntStatus_assertGPIO_RMASK 0x1
156#define QIB_6120_IntStatus_Reserved_LSB 0xF
157#define QIB_6120_IntStatus_Reserved_RMASK 0x1FFF
158#define QIB_6120_IntStatus_RcvAvail4_LSB 0x10
159#define QIB_6120_IntStatus_RcvAvail4_RMASK 0x1
160#define QIB_6120_IntStatus_RcvAvail3_LSB 0xF
161#define QIB_6120_IntStatus_RcvAvail3_RMASK 0x1
162#define QIB_6120_IntStatus_RcvAvail2_LSB 0xE
163#define QIB_6120_IntStatus_RcvAvail2_RMASK 0x1
164#define QIB_6120_IntStatus_RcvAvail1_LSB 0xD
165#define QIB_6120_IntStatus_RcvAvail1_RMASK 0x1
166#define QIB_6120_IntStatus_RcvAvail0_LSB 0xC
167#define QIB_6120_IntStatus_RcvAvail0_RMASK 0x1
168#define QIB_6120_IntStatus_Reserved1_LSB 0x5
169#define QIB_6120_IntStatus_Reserved1_RMASK 0x7F
170#define QIB_6120_IntStatus_RcvUrg4_LSB 0x4
171#define QIB_6120_IntStatus_RcvUrg4_RMASK 0x1
172#define QIB_6120_IntStatus_RcvUrg3_LSB 0x3
173#define QIB_6120_IntStatus_RcvUrg3_RMASK 0x1
174#define QIB_6120_IntStatus_RcvUrg2_LSB 0x2
175#define QIB_6120_IntStatus_RcvUrg2_RMASK 0x1
176#define QIB_6120_IntStatus_RcvUrg1_LSB 0x1
177#define QIB_6120_IntStatus_RcvUrg1_RMASK 0x1
178#define QIB_6120_IntStatus_RcvUrg0_LSB 0x0
179#define QIB_6120_IntStatus_RcvUrg0_RMASK 0x1
180
181#define QIB_6120_IntClear_OFFS 0x78
182#define QIB_6120_IntClear_ErrorIntClear_LSB 0x1F
183#define QIB_6120_IntClear_ErrorIntClear_RMASK 0x1
184#define QIB_6120_IntClear_PioSetIntClear_LSB 0x1E
185#define QIB_6120_IntClear_PioSetIntClear_RMASK 0x1
186#define QIB_6120_IntClear_PioBufAvailIntClear_LSB 0x1D
187#define QIB_6120_IntClear_PioBufAvailIntClear_RMASK 0x1
188#define QIB_6120_IntClear_assertGPIOIntClear_LSB 0x1C
189#define QIB_6120_IntClear_assertGPIOIntClear_RMASK 0x1
190#define QIB_6120_IntClear_Reserved_LSB 0xF
191#define QIB_6120_IntClear_Reserved_RMASK 0x1FFF
192#define QIB_6120_IntClear_RcvAvail4IntClear_LSB 0x10
193#define QIB_6120_IntClear_RcvAvail4IntClear_RMASK 0x1
194#define QIB_6120_IntClear_RcvAvail3IntClear_LSB 0xF
195#define QIB_6120_IntClear_RcvAvail3IntClear_RMASK 0x1
196#define QIB_6120_IntClear_RcvAvail2IntClear_LSB 0xE
197#define QIB_6120_IntClear_RcvAvail2IntClear_RMASK 0x1
198#define QIB_6120_IntClear_RcvAvail1IntClear_LSB 0xD
199#define QIB_6120_IntClear_RcvAvail1IntClear_RMASK 0x1
200#define QIB_6120_IntClear_RcvAvail0IntClear_LSB 0xC
201#define QIB_6120_IntClear_RcvAvail0IntClear_RMASK 0x1
202#define QIB_6120_IntClear_Reserved1_LSB 0x5
203#define QIB_6120_IntClear_Reserved1_RMASK 0x7F
204#define QIB_6120_IntClear_RcvUrg4IntClear_LSB 0x4
205#define QIB_6120_IntClear_RcvUrg4IntClear_RMASK 0x1
206#define QIB_6120_IntClear_RcvUrg3IntClear_LSB 0x3
207#define QIB_6120_IntClear_RcvUrg3IntClear_RMASK 0x1
208#define QIB_6120_IntClear_RcvUrg2IntClear_LSB 0x2
209#define QIB_6120_IntClear_RcvUrg2IntClear_RMASK 0x1
210#define QIB_6120_IntClear_RcvUrg1IntClear_LSB 0x1
211#define QIB_6120_IntClear_RcvUrg1IntClear_RMASK 0x1
212#define QIB_6120_IntClear_RcvUrg0IntClear_LSB 0x0
213#define QIB_6120_IntClear_RcvUrg0IntClear_RMASK 0x1
214
215#define QIB_6120_ErrMask_OFFS 0x80
216#define QIB_6120_ErrMask_Reserved_LSB 0x34
217#define QIB_6120_ErrMask_Reserved_RMASK 0xFFF
218#define QIB_6120_ErrMask_HardwareErrMask_LSB 0x33
219#define QIB_6120_ErrMask_HardwareErrMask_RMASK 0x1
220#define QIB_6120_ErrMask_ResetNegatedMask_LSB 0x32
221#define QIB_6120_ErrMask_ResetNegatedMask_RMASK 0x1
222#define QIB_6120_ErrMask_InvalidAddrErrMask_LSB 0x31
223#define QIB_6120_ErrMask_InvalidAddrErrMask_RMASK 0x1
224#define QIB_6120_ErrMask_IBStatusChangedMask_LSB 0x30
225#define QIB_6120_ErrMask_IBStatusChangedMask_RMASK 0x1
226#define QIB_6120_ErrMask_Reserved1_LSB 0x26
227#define QIB_6120_ErrMask_Reserved1_RMASK 0x3FF
228#define QIB_6120_ErrMask_SendUnsupportedVLErrMask_LSB 0x25
229#define QIB_6120_ErrMask_SendUnsupportedVLErrMask_RMASK 0x1
230#define QIB_6120_ErrMask_SendUnexpectedPktNumErrMask_LSB 0x24
231#define QIB_6120_ErrMask_SendUnexpectedPktNumErrMask_RMASK 0x1
232#define QIB_6120_ErrMask_SendPioArmLaunchErrMask_LSB 0x23
233#define QIB_6120_ErrMask_SendPioArmLaunchErrMask_RMASK 0x1
234#define QIB_6120_ErrMask_SendDroppedDataPktErrMask_LSB 0x22
235#define QIB_6120_ErrMask_SendDroppedDataPktErrMask_RMASK 0x1
236#define QIB_6120_ErrMask_SendDroppedSmpPktErrMask_LSB 0x21
237#define QIB_6120_ErrMask_SendDroppedSmpPktErrMask_RMASK 0x1
238#define QIB_6120_ErrMask_SendPktLenErrMask_LSB 0x20
239#define QIB_6120_ErrMask_SendPktLenErrMask_RMASK 0x1
240#define QIB_6120_ErrMask_SendUnderRunErrMask_LSB 0x1F
241#define QIB_6120_ErrMask_SendUnderRunErrMask_RMASK 0x1
242#define QIB_6120_ErrMask_SendMaxPktLenErrMask_LSB 0x1E
243#define QIB_6120_ErrMask_SendMaxPktLenErrMask_RMASK 0x1
244#define QIB_6120_ErrMask_SendMinPktLenErrMask_LSB 0x1D
245#define QIB_6120_ErrMask_SendMinPktLenErrMask_RMASK 0x1
246#define QIB_6120_ErrMask_Reserved2_LSB 0x12
247#define QIB_6120_ErrMask_Reserved2_RMASK 0x7FF
248#define QIB_6120_ErrMask_RcvIBLostLinkErrMask_LSB 0x11
249#define QIB_6120_ErrMask_RcvIBLostLinkErrMask_RMASK 0x1
250#define QIB_6120_ErrMask_RcvHdrErrMask_LSB 0x10
251#define QIB_6120_ErrMask_RcvHdrErrMask_RMASK 0x1
252#define QIB_6120_ErrMask_RcvHdrLenErrMask_LSB 0xF
253#define QIB_6120_ErrMask_RcvHdrLenErrMask_RMASK 0x1
254#define QIB_6120_ErrMask_RcvBadTidErrMask_LSB 0xE
255#define QIB_6120_ErrMask_RcvBadTidErrMask_RMASK 0x1
256#define QIB_6120_ErrMask_RcvHdrFullErrMask_LSB 0xD
257#define QIB_6120_ErrMask_RcvHdrFullErrMask_RMASK 0x1
258#define QIB_6120_ErrMask_RcvEgrFullErrMask_LSB 0xC
259#define QIB_6120_ErrMask_RcvEgrFullErrMask_RMASK 0x1
260#define QIB_6120_ErrMask_RcvBadVersionErrMask_LSB 0xB
261#define QIB_6120_ErrMask_RcvBadVersionErrMask_RMASK 0x1
262#define QIB_6120_ErrMask_RcvIBFlowErrMask_LSB 0xA
263#define QIB_6120_ErrMask_RcvIBFlowErrMask_RMASK 0x1
264#define QIB_6120_ErrMask_RcvEBPErrMask_LSB 0x9
265#define QIB_6120_ErrMask_RcvEBPErrMask_RMASK 0x1
266#define QIB_6120_ErrMask_RcvUnsupportedVLErrMask_LSB 0x8
267#define QIB_6120_ErrMask_RcvUnsupportedVLErrMask_RMASK 0x1
268#define QIB_6120_ErrMask_RcvUnexpectedCharErrMask_LSB 0x7
269#define QIB_6120_ErrMask_RcvUnexpectedCharErrMask_RMASK 0x1
270#define QIB_6120_ErrMask_RcvShortPktLenErrMask_LSB 0x6
271#define QIB_6120_ErrMask_RcvShortPktLenErrMask_RMASK 0x1
272#define QIB_6120_ErrMask_RcvLongPktLenErrMask_LSB 0x5
273#define QIB_6120_ErrMask_RcvLongPktLenErrMask_RMASK 0x1
274#define QIB_6120_ErrMask_RcvMaxPktLenErrMask_LSB 0x4
275#define QIB_6120_ErrMask_RcvMaxPktLenErrMask_RMASK 0x1
276#define QIB_6120_ErrMask_RcvMinPktLenErrMask_LSB 0x3
277#define QIB_6120_ErrMask_RcvMinPktLenErrMask_RMASK 0x1
278#define QIB_6120_ErrMask_RcvICRCErrMask_LSB 0x2
279#define QIB_6120_ErrMask_RcvICRCErrMask_RMASK 0x1
280#define QIB_6120_ErrMask_RcvVCRCErrMask_LSB 0x1
281#define QIB_6120_ErrMask_RcvVCRCErrMask_RMASK 0x1
282#define QIB_6120_ErrMask_RcvFormatErrMask_LSB 0x0
283#define QIB_6120_ErrMask_RcvFormatErrMask_RMASK 0x1
284
285#define QIB_6120_ErrStatus_OFFS 0x88
286#define QIB_6120_ErrStatus_Reserved_LSB 0x34
287#define QIB_6120_ErrStatus_Reserved_RMASK 0xFFF
288#define QIB_6120_ErrStatus_HardwareErr_LSB 0x33
289#define QIB_6120_ErrStatus_HardwareErr_RMASK 0x1
290#define QIB_6120_ErrStatus_ResetNegated_LSB 0x32
291#define QIB_6120_ErrStatus_ResetNegated_RMASK 0x1
292#define QIB_6120_ErrStatus_InvalidAddrErr_LSB 0x31
293#define QIB_6120_ErrStatus_InvalidAddrErr_RMASK 0x1
294#define QIB_6120_ErrStatus_IBStatusChanged_LSB 0x30
295#define QIB_6120_ErrStatus_IBStatusChanged_RMASK 0x1
296#define QIB_6120_ErrStatus_Reserved1_LSB 0x26
297#define QIB_6120_ErrStatus_Reserved1_RMASK 0x3FF
298#define QIB_6120_ErrStatus_SendUnsupportedVLErr_LSB 0x25
299#define QIB_6120_ErrStatus_SendUnsupportedVLErr_RMASK 0x1
300#define QIB_6120_ErrStatus_SendUnexpectedPktNumErr_LSB 0x24
301#define QIB_6120_ErrStatus_SendUnexpectedPktNumErr_RMASK 0x1
302#define QIB_6120_ErrStatus_SendPioArmLaunchErr_LSB 0x23
303#define QIB_6120_ErrStatus_SendPioArmLaunchErr_RMASK 0x1
304#define QIB_6120_ErrStatus_SendDroppedDataPktErr_LSB 0x22
305#define QIB_6120_ErrStatus_SendDroppedDataPktErr_RMASK 0x1
306#define QIB_6120_ErrStatus_SendDroppedSmpPktErr_LSB 0x21
307#define QIB_6120_ErrStatus_SendDroppedSmpPktErr_RMASK 0x1
308#define QIB_6120_ErrStatus_SendPktLenErr_LSB 0x20
309#define QIB_6120_ErrStatus_SendPktLenErr_RMASK 0x1
310#define QIB_6120_ErrStatus_SendUnderRunErr_LSB 0x1F
311#define QIB_6120_ErrStatus_SendUnderRunErr_RMASK 0x1
312#define QIB_6120_ErrStatus_SendMaxPktLenErr_LSB 0x1E
313#define QIB_6120_ErrStatus_SendMaxPktLenErr_RMASK 0x1
314#define QIB_6120_ErrStatus_SendMinPktLenErr_LSB 0x1D
315#define QIB_6120_ErrStatus_SendMinPktLenErr_RMASK 0x1
316#define QIB_6120_ErrStatus_Reserved2_LSB 0x12
317#define QIB_6120_ErrStatus_Reserved2_RMASK 0x7FF
318#define QIB_6120_ErrStatus_RcvIBLostLinkErr_LSB 0x11
319#define QIB_6120_ErrStatus_RcvIBLostLinkErr_RMASK 0x1
320#define QIB_6120_ErrStatus_RcvHdrErr_LSB 0x10
321#define QIB_6120_ErrStatus_RcvHdrErr_RMASK 0x1
322#define QIB_6120_ErrStatus_RcvHdrLenErr_LSB 0xF
323#define QIB_6120_ErrStatus_RcvHdrLenErr_RMASK 0x1
324#define QIB_6120_ErrStatus_RcvBadTidErr_LSB 0xE
325#define QIB_6120_ErrStatus_RcvBadTidErr_RMASK 0x1
326#define QIB_6120_ErrStatus_RcvHdrFullErr_LSB 0xD
327#define QIB_6120_ErrStatus_RcvHdrFullErr_RMASK 0x1
328#define QIB_6120_ErrStatus_RcvEgrFullErr_LSB 0xC
329#define QIB_6120_ErrStatus_RcvEgrFullErr_RMASK 0x1
330#define QIB_6120_ErrStatus_RcvBadVersionErr_LSB 0xB
331#define QIB_6120_ErrStatus_RcvBadVersionErr_RMASK 0x1
332#define QIB_6120_ErrStatus_RcvIBFlowErr_LSB 0xA
333#define QIB_6120_ErrStatus_RcvIBFlowErr_RMASK 0x1
334#define QIB_6120_ErrStatus_RcvEBPErr_LSB 0x9
335#define QIB_6120_ErrStatus_RcvEBPErr_RMASK 0x1
336#define QIB_6120_ErrStatus_RcvUnsupportedVLErr_LSB 0x8
337#define QIB_6120_ErrStatus_RcvUnsupportedVLErr_RMASK 0x1
338#define QIB_6120_ErrStatus_RcvUnexpectedCharErr_LSB 0x7
339#define QIB_6120_ErrStatus_RcvUnexpectedCharErr_RMASK 0x1
340#define QIB_6120_ErrStatus_RcvShortPktLenErr_LSB 0x6
341#define QIB_6120_ErrStatus_RcvShortPktLenErr_RMASK 0x1
342#define QIB_6120_ErrStatus_RcvLongPktLenErr_LSB 0x5
343#define QIB_6120_ErrStatus_RcvLongPktLenErr_RMASK 0x1
344#define QIB_6120_ErrStatus_RcvMaxPktLenErr_LSB 0x4
345#define QIB_6120_ErrStatus_RcvMaxPktLenErr_RMASK 0x1
346#define QIB_6120_ErrStatus_RcvMinPktLenErr_LSB 0x3
347#define QIB_6120_ErrStatus_RcvMinPktLenErr_RMASK 0x1
348#define QIB_6120_ErrStatus_RcvICRCErr_LSB 0x2
349#define QIB_6120_ErrStatus_RcvICRCErr_RMASK 0x1
350#define QIB_6120_ErrStatus_RcvVCRCErr_LSB 0x1
351#define QIB_6120_ErrStatus_RcvVCRCErr_RMASK 0x1
352#define QIB_6120_ErrStatus_RcvFormatErr_LSB 0x0
353#define QIB_6120_ErrStatus_RcvFormatErr_RMASK 0x1
354
355#define QIB_6120_ErrClear_OFFS 0x90
356#define QIB_6120_ErrClear_Reserved_LSB 0x34
357#define QIB_6120_ErrClear_Reserved_RMASK 0xFFF
358#define QIB_6120_ErrClear_HardwareErrClear_LSB 0x33
359#define QIB_6120_ErrClear_HardwareErrClear_RMASK 0x1
360#define QIB_6120_ErrClear_ResetNegatedClear_LSB 0x32
361#define QIB_6120_ErrClear_ResetNegatedClear_RMASK 0x1
362#define QIB_6120_ErrClear_InvalidAddrErrClear_LSB 0x31
363#define QIB_6120_ErrClear_InvalidAddrErrClear_RMASK 0x1
364#define QIB_6120_ErrClear_IBStatusChangedClear_LSB 0x30
365#define QIB_6120_ErrClear_IBStatusChangedClear_RMASK 0x1
366#define QIB_6120_ErrClear_Reserved1_LSB 0x26
367#define QIB_6120_ErrClear_Reserved1_RMASK 0x3FF
368#define QIB_6120_ErrClear_SendUnsupportedVLErrClear_LSB 0x25
369#define QIB_6120_ErrClear_SendUnsupportedVLErrClear_RMASK 0x1
370#define QIB_6120_ErrClear_SendUnexpectedPktNumErrClear_LSB 0x24
371#define QIB_6120_ErrClear_SendUnexpectedPktNumErrClear_RMASK 0x1
372#define QIB_6120_ErrClear_SendPioArmLaunchErrClear_LSB 0x23
373#define QIB_6120_ErrClear_SendPioArmLaunchErrClear_RMASK 0x1
374#define QIB_6120_ErrClear_SendDroppedDataPktErrClear_LSB 0x22
375#define QIB_6120_ErrClear_SendDroppedDataPktErrClear_RMASK 0x1
376#define QIB_6120_ErrClear_SendDroppedSmpPktErrClear_LSB 0x21
377#define QIB_6120_ErrClear_SendDroppedSmpPktErrClear_RMASK 0x1
378#define QIB_6120_ErrClear_SendPktLenErrClear_LSB 0x20
379#define QIB_6120_ErrClear_SendPktLenErrClear_RMASK 0x1
380#define QIB_6120_ErrClear_SendUnderRunErrClear_LSB 0x1F
381#define QIB_6120_ErrClear_SendUnderRunErrClear_RMASK 0x1
382#define QIB_6120_ErrClear_SendMaxPktLenErrClear_LSB 0x1E
383#define QIB_6120_ErrClear_SendMaxPktLenErrClear_RMASK 0x1
384#define QIB_6120_ErrClear_SendMinPktLenErrClear_LSB 0x1D
385#define QIB_6120_ErrClear_SendMinPktLenErrClear_RMASK 0x1
386#define QIB_6120_ErrClear_Reserved2_LSB 0x12
387#define QIB_6120_ErrClear_Reserved2_RMASK 0x7FF
388#define QIB_6120_ErrClear_RcvIBLostLinkErrClear_LSB 0x11
389#define QIB_6120_ErrClear_RcvIBLostLinkErrClear_RMASK 0x1
390#define QIB_6120_ErrClear_RcvHdrErrClear_LSB 0x10
391#define QIB_6120_ErrClear_RcvHdrErrClear_RMASK 0x1
392#define QIB_6120_ErrClear_RcvHdrLenErrClear_LSB 0xF
393#define QIB_6120_ErrClear_RcvHdrLenErrClear_RMASK 0x1
394#define QIB_6120_ErrClear_RcvBadTidErrClear_LSB 0xE
395#define QIB_6120_ErrClear_RcvBadTidErrClear_RMASK 0x1
396#define QIB_6120_ErrClear_RcvHdrFullErrClear_LSB 0xD
397#define QIB_6120_ErrClear_RcvHdrFullErrClear_RMASK 0x1
398#define QIB_6120_ErrClear_RcvEgrFullErrClear_LSB 0xC
399#define QIB_6120_ErrClear_RcvEgrFullErrClear_RMASK 0x1
400#define QIB_6120_ErrClear_RcvBadVersionErrClear_LSB 0xB
401#define QIB_6120_ErrClear_RcvBadVersionErrClear_RMASK 0x1
402#define QIB_6120_ErrClear_RcvIBFlowErrClear_LSB 0xA
403#define QIB_6120_ErrClear_RcvIBFlowErrClear_RMASK 0x1
404#define QIB_6120_ErrClear_RcvEBPErrClear_LSB 0x9
405#define QIB_6120_ErrClear_RcvEBPErrClear_RMASK 0x1
406#define QIB_6120_ErrClear_RcvUnsupportedVLErrClear_LSB 0x8
407#define QIB_6120_ErrClear_RcvUnsupportedVLErrClear_RMASK 0x1
408#define QIB_6120_ErrClear_RcvUnexpectedCharErrClear_LSB 0x7
409#define QIB_6120_ErrClear_RcvUnexpectedCharErrClear_RMASK 0x1
410#define QIB_6120_ErrClear_RcvShortPktLenErrClear_LSB 0x6
411#define QIB_6120_ErrClear_RcvShortPktLenErrClear_RMASK 0x1
412#define QIB_6120_ErrClear_RcvLongPktLenErrClear_LSB 0x5
413#define QIB_6120_ErrClear_RcvLongPktLenErrClear_RMASK 0x1
414#define QIB_6120_ErrClear_RcvMaxPktLenErrClear_LSB 0x4
415#define QIB_6120_ErrClear_RcvMaxPktLenErrClear_RMASK 0x1
416#define QIB_6120_ErrClear_RcvMinPktLenErrClear_LSB 0x3
417#define QIB_6120_ErrClear_RcvMinPktLenErrClear_RMASK 0x1
418#define QIB_6120_ErrClear_RcvICRCErrClear_LSB 0x2
419#define QIB_6120_ErrClear_RcvICRCErrClear_RMASK 0x1
420#define QIB_6120_ErrClear_RcvVCRCErrClear_LSB 0x1
421#define QIB_6120_ErrClear_RcvVCRCErrClear_RMASK 0x1
422#define QIB_6120_ErrClear_RcvFormatErrClear_LSB 0x0
423#define QIB_6120_ErrClear_RcvFormatErrClear_RMASK 0x1
424
425#define QIB_6120_HwErrMask_OFFS 0x98
426#define QIB_6120_HwErrMask_IBCBusFromSPCParityErrMask_LSB 0x3F
427#define QIB_6120_HwErrMask_IBCBusFromSPCParityErrMask_RMASK 0x1
428#define QIB_6120_HwErrMask_IBCBusToSPCParityErrMask_LSB 0x3E
429#define QIB_6120_HwErrMask_IBCBusToSPCParityErrMask_RMASK 0x1
430#define QIB_6120_HwErrMask_Reserved_LSB 0x3D
431#define QIB_6120_HwErrMask_Reserved_RMASK 0x1
432#define QIB_6120_HwErrMask_IBSerdesPClkNotDetectMask_LSB 0x3C
433#define QIB_6120_HwErrMask_IBSerdesPClkNotDetectMask_RMASK 0x1
434#define QIB_6120_HwErrMask_PCIESerdesQ0PClkNotDetectMask_LSB 0x3B
435#define QIB_6120_HwErrMask_PCIESerdesQ0PClkNotDetectMask_RMASK 0x1
436#define QIB_6120_HwErrMask_PCIESerdesQ1PClkNotDetectMask_LSB 0x3A
437#define QIB_6120_HwErrMask_PCIESerdesQ1PClkNotDetectMask_RMASK 0x1
438#define QIB_6120_HwErrMask_Reserved1_LSB 0x39
439#define QIB_6120_HwErrMask_Reserved1_RMASK 0x1
440#define QIB_6120_HwErrMask_IBPLLrfSlipMask_LSB 0x38
441#define QIB_6120_HwErrMask_IBPLLrfSlipMask_RMASK 0x1
442#define QIB_6120_HwErrMask_IBPLLfbSlipMask_LSB 0x37
443#define QIB_6120_HwErrMask_IBPLLfbSlipMask_RMASK 0x1
444#define QIB_6120_HwErrMask_PowerOnBISTFailedMask_LSB 0x36
445#define QIB_6120_HwErrMask_PowerOnBISTFailedMask_RMASK 0x1
446#define QIB_6120_HwErrMask_Reserved2_LSB 0x33
447#define QIB_6120_HwErrMask_Reserved2_RMASK 0x7
448#define QIB_6120_HwErrMask_RXEMemParityErrMask_LSB 0x2C
449#define QIB_6120_HwErrMask_RXEMemParityErrMask_RMASK 0x7F
450#define QIB_6120_HwErrMask_TXEMemParityErrMask_LSB 0x28
451#define QIB_6120_HwErrMask_TXEMemParityErrMask_RMASK 0xF
452#define QIB_6120_HwErrMask_Reserved3_LSB 0x22
453#define QIB_6120_HwErrMask_Reserved3_RMASK 0x3F
454#define QIB_6120_HwErrMask_PCIeBusParityErrMask_LSB 0x1F
455#define QIB_6120_HwErrMask_PCIeBusParityErrMask_RMASK 0x7
456#define QIB_6120_HwErrMask_PcieCplTimeoutMask_LSB 0x1E
457#define QIB_6120_HwErrMask_PcieCplTimeoutMask_RMASK 0x1
458#define QIB_6120_HwErrMask_PoisonedTLPMask_LSB 0x1D
459#define QIB_6120_HwErrMask_PoisonedTLPMask_RMASK 0x1
460#define QIB_6120_HwErrMask_Reserved4_LSB 0x6
461#define QIB_6120_HwErrMask_Reserved4_RMASK 0x7FFFFF
462#define QIB_6120_HwErrMask_PCIeMemParityErrMask_LSB 0x0
463#define QIB_6120_HwErrMask_PCIeMemParityErrMask_RMASK 0x3F
464
465#define QIB_6120_HwErrStatus_OFFS 0xA0
466#define QIB_6120_HwErrStatus_IBCBusFromSPCParityErr_LSB 0x3F
467#define QIB_6120_HwErrStatus_IBCBusFromSPCParityErr_RMASK 0x1
468#define QIB_6120_HwErrStatus_IBCBusToSPCParityErr_LSB 0x3E
469#define QIB_6120_HwErrStatus_IBCBusToSPCParityErr_RMASK 0x1
470#define QIB_6120_HwErrStatus_Reserved_LSB 0x3D
471#define QIB_6120_HwErrStatus_Reserved_RMASK 0x1
472#define QIB_6120_HwErrStatus_IBSerdesPClkNotDetect_LSB 0x3C
473#define QIB_6120_HwErrStatus_IBSerdesPClkNotDetect_RMASK 0x1
474#define QIB_6120_HwErrStatus_PCIESerdesQ0PClkNotDetect_LSB 0x3B
475#define QIB_6120_HwErrStatus_PCIESerdesQ0PClkNotDetect_RMASK 0x1
476#define QIB_6120_HwErrStatus_PCIESerdesQ1PClkNotDetect_LSB 0x3A
477#define QIB_6120_HwErrStatus_PCIESerdesQ1PClkNotDetect_RMASK 0x1
478#define QIB_6120_HwErrStatus_Reserved1_LSB 0x39
479#define QIB_6120_HwErrStatus_Reserved1_RMASK 0x1
480#define QIB_6120_HwErrStatus_IBPLLrfSlip_LSB 0x38
481#define QIB_6120_HwErrStatus_IBPLLrfSlip_RMASK 0x1
482#define QIB_6120_HwErrStatus_IBPLLfbSlip_LSB 0x37
483#define QIB_6120_HwErrStatus_IBPLLfbSlip_RMASK 0x1
484#define QIB_6120_HwErrStatus_PowerOnBISTFailed_LSB 0x36
485#define QIB_6120_HwErrStatus_PowerOnBISTFailed_RMASK 0x1
486#define QIB_6120_HwErrStatus_Reserved2_LSB 0x33
487#define QIB_6120_HwErrStatus_Reserved2_RMASK 0x7
488#define QIB_6120_HwErrStatus_RXEMemParity_LSB 0x2C
489#define QIB_6120_HwErrStatus_RXEMemParity_RMASK 0x7F
490#define QIB_6120_HwErrStatus_TXEMemParity_LSB 0x28
491#define QIB_6120_HwErrStatus_TXEMemParity_RMASK 0xF
492#define QIB_6120_HwErrStatus_Reserved3_LSB 0x22
493#define QIB_6120_HwErrStatus_Reserved3_RMASK 0x3F
494#define QIB_6120_HwErrStatus_PCIeBusParity_LSB 0x1F
495#define QIB_6120_HwErrStatus_PCIeBusParity_RMASK 0x7
496#define QIB_6120_HwErrStatus_PcieCplTimeout_LSB 0x1E
497#define QIB_6120_HwErrStatus_PcieCplTimeout_RMASK 0x1
498#define QIB_6120_HwErrStatus_PoisenedTLP_LSB 0x1D
499#define QIB_6120_HwErrStatus_PoisenedTLP_RMASK 0x1
500#define QIB_6120_HwErrStatus_Reserved4_LSB 0x6
501#define QIB_6120_HwErrStatus_Reserved4_RMASK 0x7FFFFF
502#define QIB_6120_HwErrStatus_PCIeMemParity_LSB 0x0
503#define QIB_6120_HwErrStatus_PCIeMemParity_RMASK 0x3F
504
505#define QIB_6120_HwErrClear_OFFS 0xA8
506#define QIB_6120_HwErrClear_IBCBusFromSPCParityErrClear_LSB 0x3F
507#define QIB_6120_HwErrClear_IBCBusFromSPCParityErrClear_RMASK 0x1
508#define QIB_6120_HwErrClear_IBCBusToSPCparityErrClear_LSB 0x3E
509#define QIB_6120_HwErrClear_IBCBusToSPCparityErrClear_RMASK 0x1
510#define QIB_6120_HwErrClear_Reserved_LSB 0x3D
511#define QIB_6120_HwErrClear_Reserved_RMASK 0x1
512#define QIB_6120_HwErrClear_IBSerdesPClkNotDetectClear_LSB 0x3C
513#define QIB_6120_HwErrClear_IBSerdesPClkNotDetectClear_RMASK 0x1
514#define QIB_6120_HwErrClear_PCIESerdesQ0PClkNotDetectClear_LSB 0x3B
515#define QIB_6120_HwErrClear_PCIESerdesQ0PClkNotDetectClear_RMASK 0x1
516#define QIB_6120_HwErrClear_PCIESerdesQ1PClkNotDetectClear_LSB 0x3A
517#define QIB_6120_HwErrClear_PCIESerdesQ1PClkNotDetectClear_RMASK 0x1
518#define QIB_6120_HwErrClear_Reserved1_LSB 0x39
519#define QIB_6120_HwErrClear_Reserved1_RMASK 0x1
520#define QIB_6120_HwErrClear_IBPLLrfSlipClear_LSB 0x38
521#define QIB_6120_HwErrClear_IBPLLrfSlipClear_RMASK 0x1
522#define QIB_6120_HwErrClear_IBPLLfbSlipClear_LSB 0x37
523#define QIB_6120_HwErrClear_IBPLLfbSlipClear_RMASK 0x1
524#define QIB_6120_HwErrClear_PowerOnBISTFailedClear_LSB 0x36
525#define QIB_6120_HwErrClear_PowerOnBISTFailedClear_RMASK 0x1
526#define QIB_6120_HwErrClear_Reserved2_LSB 0x33
527#define QIB_6120_HwErrClear_Reserved2_RMASK 0x7
528#define QIB_6120_HwErrClear_RXEMemParityClear_LSB 0x2C
529#define QIB_6120_HwErrClear_RXEMemParityClear_RMASK 0x7F
530#define QIB_6120_HwErrClear_TXEMemParityClear_LSB 0x28
531#define QIB_6120_HwErrClear_TXEMemParityClear_RMASK 0xF
532#define QIB_6120_HwErrClear_Reserved3_LSB 0x22
533#define QIB_6120_HwErrClear_Reserved3_RMASK 0x3F
534#define QIB_6120_HwErrClear_PCIeBusParityClr_LSB 0x1F
535#define QIB_6120_HwErrClear_PCIeBusParityClr_RMASK 0x7
536#define QIB_6120_HwErrClear_PcieCplTimeoutClear_LSB 0x1E
537#define QIB_6120_HwErrClear_PcieCplTimeoutClear_RMASK 0x1
538#define QIB_6120_HwErrClear_PoisonedTLPClear_LSB 0x1D
539#define QIB_6120_HwErrClear_PoisonedTLPClear_RMASK 0x1
540#define QIB_6120_HwErrClear_Reserved4_LSB 0x6
541#define QIB_6120_HwErrClear_Reserved4_RMASK 0x7FFFFF
542#define QIB_6120_HwErrClear_PCIeMemParityClr_LSB 0x0
543#define QIB_6120_HwErrClear_PCIeMemParityClr_RMASK 0x3F
544
545#define QIB_6120_HwDiagCtrl_OFFS 0xB0
546#define QIB_6120_HwDiagCtrl_ForceIBCBusFromSPCParityErr_LSB 0x3F
547#define QIB_6120_HwDiagCtrl_ForceIBCBusFromSPCParityErr_RMASK 0x1
548#define QIB_6120_HwDiagCtrl_ForceIBCBusToSPCParityErr_LSB 0x3E
549#define QIB_6120_HwDiagCtrl_ForceIBCBusToSPCParityErr_RMASK 0x1
550#define QIB_6120_HwDiagCtrl_CounterWrEnable_LSB 0x3D
551#define QIB_6120_HwDiagCtrl_CounterWrEnable_RMASK 0x1
552#define QIB_6120_HwDiagCtrl_CounterDisable_LSB 0x3C
553#define QIB_6120_HwDiagCtrl_CounterDisable_RMASK 0x1
554#define QIB_6120_HwDiagCtrl_Reserved_LSB 0x33
555#define QIB_6120_HwDiagCtrl_Reserved_RMASK 0x1FF
556#define QIB_6120_HwDiagCtrl_ForceRxMemParityErr_LSB 0x2C
557#define QIB_6120_HwDiagCtrl_ForceRxMemParityErr_RMASK 0x7F
558#define QIB_6120_HwDiagCtrl_ForceTxMemparityErr_LSB 0x28
559#define QIB_6120_HwDiagCtrl_ForceTxMemparityErr_RMASK 0xF
560#define QIB_6120_HwDiagCtrl_Reserved1_LSB 0x23
561#define QIB_6120_HwDiagCtrl_Reserved1_RMASK 0x1F
562#define QIB_6120_HwDiagCtrl_forcePCIeBusParity_LSB 0x1F
563#define QIB_6120_HwDiagCtrl_forcePCIeBusParity_RMASK 0xF
564#define QIB_6120_HwDiagCtrl_Reserved2_LSB 0x6
565#define QIB_6120_HwDiagCtrl_Reserved2_RMASK 0x1FFFFFF
566#define QIB_6120_HwDiagCtrl_forcePCIeMemParity_LSB 0x0
567#define QIB_6120_HwDiagCtrl_forcePCIeMemParity_RMASK 0x3F
568
569#define QIB_6120_IBCStatus_OFFS 0xC0
570#define QIB_6120_IBCStatus_TxCreditOk_LSB 0x1F
571#define QIB_6120_IBCStatus_TxCreditOk_RMASK 0x1
572#define QIB_6120_IBCStatus_TxReady_LSB 0x1E
573#define QIB_6120_IBCStatus_TxReady_RMASK 0x1
574#define QIB_6120_IBCStatus_Reserved_LSB 0x7
575#define QIB_6120_IBCStatus_Reserved_RMASK 0x7FFFFF
576#define QIB_6120_IBCStatus_LinkState_LSB 0x4
577#define QIB_6120_IBCStatus_LinkState_RMASK 0x7
578#define QIB_6120_IBCStatus_LinkTrainingState_LSB 0x0
579#define QIB_6120_IBCStatus_LinkTrainingState_RMASK 0xF
580
581#define QIB_6120_IBCCtrl_OFFS 0xC8
582#define QIB_6120_IBCCtrl_Loopback_LSB 0x3F
583#define QIB_6120_IBCCtrl_Loopback_RMASK 0x1
584#define QIB_6120_IBCCtrl_LinkDownDefaultState_LSB 0x3E
585#define QIB_6120_IBCCtrl_LinkDownDefaultState_RMASK 0x1
586#define QIB_6120_IBCCtrl_Reserved_LSB 0x2B
587#define QIB_6120_IBCCtrl_Reserved_RMASK 0x7FFFF
588#define QIB_6120_IBCCtrl_CreditScale_LSB 0x28
589#define QIB_6120_IBCCtrl_CreditScale_RMASK 0x7
590#define QIB_6120_IBCCtrl_OverrunThreshold_LSB 0x24
591#define QIB_6120_IBCCtrl_OverrunThreshold_RMASK 0xF
592#define QIB_6120_IBCCtrl_PhyerrThreshold_LSB 0x20
593#define QIB_6120_IBCCtrl_PhyerrThreshold_RMASK 0xF
594#define QIB_6120_IBCCtrl_Reserved1_LSB 0x1F
595#define QIB_6120_IBCCtrl_Reserved1_RMASK 0x1
596#define QIB_6120_IBCCtrl_MaxPktLen_LSB 0x14
597#define QIB_6120_IBCCtrl_MaxPktLen_RMASK 0x7FF
598#define QIB_6120_IBCCtrl_LinkCmd_LSB 0x12
599#define QIB_6120_IBCCtrl_LinkCmd_RMASK 0x3
600#define QIB_6120_IBCCtrl_LinkInitCmd_LSB 0x10
601#define QIB_6120_IBCCtrl_LinkInitCmd_RMASK 0x3
602#define QIB_6120_IBCCtrl_FlowCtrlWaterMark_LSB 0x8
603#define QIB_6120_IBCCtrl_FlowCtrlWaterMark_RMASK 0xFF
604#define QIB_6120_IBCCtrl_FlowCtrlPeriod_LSB 0x0
605#define QIB_6120_IBCCtrl_FlowCtrlPeriod_RMASK 0xFF
606
607#define QIB_6120_EXTStatus_OFFS 0xD0
608#define QIB_6120_EXTStatus_GPIOIn_LSB 0x30
609#define QIB_6120_EXTStatus_GPIOIn_RMASK 0xFFFF
610#define QIB_6120_EXTStatus_Reserved_LSB 0x20
611#define QIB_6120_EXTStatus_Reserved_RMASK 0xFFFF
612#define QIB_6120_EXTStatus_Reserved1_LSB 0x10
613#define QIB_6120_EXTStatus_Reserved1_RMASK 0xFFFF
614#define QIB_6120_EXTStatus_MemBISTFoundErr_LSB 0xF
615#define QIB_6120_EXTStatus_MemBISTFoundErr_RMASK 0x1
616#define QIB_6120_EXTStatus_MemBISTEndTest_LSB 0xE
617#define QIB_6120_EXTStatus_MemBISTEndTest_RMASK 0x1
618#define QIB_6120_EXTStatus_Reserved2_LSB 0x0
619#define QIB_6120_EXTStatus_Reserved2_RMASK 0x3FFF
620
621#define QIB_6120_EXTCtrl_OFFS 0xD8
622#define QIB_6120_EXTCtrl_GPIOOe_LSB 0x30
623#define QIB_6120_EXTCtrl_GPIOOe_RMASK 0xFFFF
624#define QIB_6120_EXTCtrl_GPIOInvert_LSB 0x20
625#define QIB_6120_EXTCtrl_GPIOInvert_RMASK 0xFFFF
626#define QIB_6120_EXTCtrl_Reserved_LSB 0x4
627#define QIB_6120_EXTCtrl_Reserved_RMASK 0xFFFFFFF
628#define QIB_6120_EXTCtrl_LEDPriPortGreenOn_LSB 0x3
629#define QIB_6120_EXTCtrl_LEDPriPortGreenOn_RMASK 0x1
630#define QIB_6120_EXTCtrl_LEDPriPortYellowOn_LSB 0x2
631#define QIB_6120_EXTCtrl_LEDPriPortYellowOn_RMASK 0x1
632#define QIB_6120_EXTCtrl_LEDGblOkGreenOn_LSB 0x1
633#define QIB_6120_EXTCtrl_LEDGblOkGreenOn_RMASK 0x1
634#define QIB_6120_EXTCtrl_LEDGblErrRedOff_LSB 0x0
635#define QIB_6120_EXTCtrl_LEDGblErrRedOff_RMASK 0x1
636
637#define QIB_6120_GPIOOut_OFFS 0xE0
638
639#define QIB_6120_GPIOMask_OFFS 0xE8
640
641#define QIB_6120_GPIOStatus_OFFS 0xF0
642
643#define QIB_6120_GPIOClear_OFFS 0xF8
644
645#define QIB_6120_RcvCtrl_OFFS 0x100
646#define QIB_6120_RcvCtrl_TailUpd_LSB 0x1F
647#define QIB_6120_RcvCtrl_TailUpd_RMASK 0x1
648#define QIB_6120_RcvCtrl_RcvPartitionKeyDisable_LSB 0x1E
649#define QIB_6120_RcvCtrl_RcvPartitionKeyDisable_RMASK 0x1
650#define QIB_6120_RcvCtrl_Reserved_LSB 0x15
651#define QIB_6120_RcvCtrl_Reserved_RMASK 0x1FF
652#define QIB_6120_RcvCtrl_IntrAvail_LSB 0x10
653#define QIB_6120_RcvCtrl_IntrAvail_RMASK 0x1F
654#define QIB_6120_RcvCtrl_Reserved1_LSB 0x9
655#define QIB_6120_RcvCtrl_Reserved1_RMASK 0x7F
656#define QIB_6120_RcvCtrl_Reserved2_LSB 0x5
657#define QIB_6120_RcvCtrl_Reserved2_RMASK 0xF
658#define QIB_6120_RcvCtrl_PortEnable_LSB 0x0
659#define QIB_6120_RcvCtrl_PortEnable_RMASK 0x1F
660
661#define QIB_6120_RcvBTHQP_OFFS 0x108
662#define QIB_6120_RcvBTHQP_BTHQP_Mask_LSB 0x1E
663#define QIB_6120_RcvBTHQP_BTHQP_Mask_RMASK 0x3
664#define QIB_6120_RcvBTHQP_Reserved_LSB 0x18
665#define QIB_6120_RcvBTHQP_Reserved_RMASK 0x3F
666#define QIB_6120_RcvBTHQP_RcvBTHQP_LSB 0x0
667#define QIB_6120_RcvBTHQP_RcvBTHQP_RMASK 0xFFFFFF
668
669#define QIB_6120_RcvHdrSize_OFFS 0x110
670
671#define QIB_6120_RcvHdrCnt_OFFS 0x118
672
673#define QIB_6120_RcvHdrEntSize_OFFS 0x120
674
675#define QIB_6120_RcvTIDBase_OFFS 0x128
676
677#define QIB_6120_RcvTIDCnt_OFFS 0x130
678
679#define QIB_6120_RcvEgrBase_OFFS 0x138
680
681#define QIB_6120_RcvEgrCnt_OFFS 0x140
682
683#define QIB_6120_RcvBufBase_OFFS 0x148
684
685#define QIB_6120_RcvBufSize_OFFS 0x150
686
687#define QIB_6120_RxIntMemBase_OFFS 0x158
688
689#define QIB_6120_RxIntMemSize_OFFS 0x160
690
691#define QIB_6120_RcvPartitionKey_OFFS 0x168
692
693#define QIB_6120_RcvPktLEDCnt_OFFS 0x178
694#define QIB_6120_RcvPktLEDCnt_ONperiod_LSB 0x20
695#define QIB_6120_RcvPktLEDCnt_ONperiod_RMASK 0xFFFFFFFF
696#define QIB_6120_RcvPktLEDCnt_OFFperiod_LSB 0x0
697#define QIB_6120_RcvPktLEDCnt_OFFperiod_RMASK 0xFFFFFFFF
698
699#define QIB_6120_SendCtrl_OFFS 0x1C0
700#define QIB_6120_SendCtrl_Disarm_LSB 0x1F
701#define QIB_6120_SendCtrl_Disarm_RMASK 0x1
702#define QIB_6120_SendCtrl_Reserved_LSB 0x17
703#define QIB_6120_SendCtrl_Reserved_RMASK 0xFF
704#define QIB_6120_SendCtrl_DisarmPIOBuf_LSB 0x10
705#define QIB_6120_SendCtrl_DisarmPIOBuf_RMASK 0x7F
706#define QIB_6120_SendCtrl_Reserved1_LSB 0x4
707#define QIB_6120_SendCtrl_Reserved1_RMASK 0xFFF
708#define QIB_6120_SendCtrl_PIOEnable_LSB 0x3
709#define QIB_6120_SendCtrl_PIOEnable_RMASK 0x1
710#define QIB_6120_SendCtrl_PIOBufAvailUpd_LSB 0x2
711#define QIB_6120_SendCtrl_PIOBufAvailUpd_RMASK 0x1
712#define QIB_6120_SendCtrl_PIOIntBufAvail_LSB 0x1
713#define QIB_6120_SendCtrl_PIOIntBufAvail_RMASK 0x1
714#define QIB_6120_SendCtrl_Abort_LSB 0x0
715#define QIB_6120_SendCtrl_Abort_RMASK 0x1
716
717#define QIB_6120_SendPIOBufBase_OFFS 0x1C8
718#define QIB_6120_SendPIOBufBase_Reserved_LSB 0x35
719#define QIB_6120_SendPIOBufBase_Reserved_RMASK 0x7FF
720#define QIB_6120_SendPIOBufBase_BaseAddr_LargePIO_LSB 0x20
721#define QIB_6120_SendPIOBufBase_BaseAddr_LargePIO_RMASK 0x1FFFFF
722#define QIB_6120_SendPIOBufBase_Reserved1_LSB 0x15
723#define QIB_6120_SendPIOBufBase_Reserved1_RMASK 0x7FF
724#define QIB_6120_SendPIOBufBase_BaseAddr_SmallPIO_LSB 0x0
725#define QIB_6120_SendPIOBufBase_BaseAddr_SmallPIO_RMASK 0x1FFFFF
726
727#define QIB_6120_SendPIOSize_OFFS 0x1D0
728#define QIB_6120_SendPIOSize_Reserved_LSB 0x2D
729#define QIB_6120_SendPIOSize_Reserved_RMASK 0xFFFFF
730#define QIB_6120_SendPIOSize_Size_LargePIO_LSB 0x20
731#define QIB_6120_SendPIOSize_Size_LargePIO_RMASK 0x1FFF
732#define QIB_6120_SendPIOSize_Reserved1_LSB 0xC
733#define QIB_6120_SendPIOSize_Reserved1_RMASK 0xFFFFF
734#define QIB_6120_SendPIOSize_Size_SmallPIO_LSB 0x0
735#define QIB_6120_SendPIOSize_Size_SmallPIO_RMASK 0xFFF
736
737#define QIB_6120_SendPIOBufCnt_OFFS 0x1D8
738#define QIB_6120_SendPIOBufCnt_Reserved_LSB 0x24
739#define QIB_6120_SendPIOBufCnt_Reserved_RMASK 0xFFFFFFF
740#define QIB_6120_SendPIOBufCnt_Num_LargePIO_LSB 0x20
741#define QIB_6120_SendPIOBufCnt_Num_LargePIO_RMASK 0xF
742#define QIB_6120_SendPIOBufCnt_Reserved1_LSB 0x9
743#define QIB_6120_SendPIOBufCnt_Reserved1_RMASK 0x7FFFFF
744#define QIB_6120_SendPIOBufCnt_Num_SmallPIO_LSB 0x0
745#define QIB_6120_SendPIOBufCnt_Num_SmallPIO_RMASK 0x1FF
746
747#define QIB_6120_SendPIOAvailAddr_OFFS 0x1E0
748#define QIB_6120_SendPIOAvailAddr_SendPIOAvailAddr_LSB 0x6
749#define QIB_6120_SendPIOAvailAddr_SendPIOAvailAddr_RMASK 0x3FFFFFFFF
750#define QIB_6120_SendPIOAvailAddr_Reserved_LSB 0x0
751#define QIB_6120_SendPIOAvailAddr_Reserved_RMASK 0x3F
752
753#define QIB_6120_SendBufErr0_OFFS 0x240
754#define QIB_6120_SendBufErr0_SendBufErrPIO_63_0_LSB 0x0
755#define QIB_6120_SendBufErr0_SendBufErrPIO_63_0_RMASK 0x0
756
757#define QIB_6120_RcvHdrAddr0_OFFS 0x280
758#define QIB_6120_RcvHdrAddr0_RcvHdrAddr0_LSB 0x2
759#define QIB_6120_RcvHdrAddr0_RcvHdrAddr0_RMASK 0x3FFFFFFFFF
760#define QIB_6120_RcvHdrAddr0_Reserved_LSB 0x0
761#define QIB_6120_RcvHdrAddr0_Reserved_RMASK 0x3
762
763#define QIB_6120_RcvHdrTailAddr0_OFFS 0x300
764#define QIB_6120_RcvHdrTailAddr0_RcvHdrTailAddr0_LSB 0x2
765#define QIB_6120_RcvHdrTailAddr0_RcvHdrTailAddr0_RMASK 0x3FFFFFFFFF
766#define QIB_6120_RcvHdrTailAddr0_Reserved_LSB 0x0
767#define QIB_6120_RcvHdrTailAddr0_Reserved_RMASK 0x3
768
769#define QIB_6120_SerdesCfg0_OFFS 0x3C0
770#define QIB_6120_SerdesCfg0_DisableIBTxIdleDetect_LSB 0x3F
771#define QIB_6120_SerdesCfg0_DisableIBTxIdleDetect_RMASK 0x1
772#define QIB_6120_SerdesCfg0_Reserved_LSB 0x38
773#define QIB_6120_SerdesCfg0_Reserved_RMASK 0x7F
774#define QIB_6120_SerdesCfg0_RxEqCtl_LSB 0x36
775#define QIB_6120_SerdesCfg0_RxEqCtl_RMASK 0x3
776#define QIB_6120_SerdesCfg0_TxTermAdj_LSB 0x34
777#define QIB_6120_SerdesCfg0_TxTermAdj_RMASK 0x3
778#define QIB_6120_SerdesCfg0_RxTermAdj_LSB 0x32
779#define QIB_6120_SerdesCfg0_RxTermAdj_RMASK 0x3
780#define QIB_6120_SerdesCfg0_TermAdj1_LSB 0x31
781#define QIB_6120_SerdesCfg0_TermAdj1_RMASK 0x1
782#define QIB_6120_SerdesCfg0_TermAdj0_LSB 0x30
783#define QIB_6120_SerdesCfg0_TermAdj0_RMASK 0x1
784#define QIB_6120_SerdesCfg0_LPBKA_LSB 0x2F
785#define QIB_6120_SerdesCfg0_LPBKA_RMASK 0x1
786#define QIB_6120_SerdesCfg0_LPBKB_LSB 0x2E
787#define QIB_6120_SerdesCfg0_LPBKB_RMASK 0x1
788#define QIB_6120_SerdesCfg0_LPBKC_LSB 0x2D
789#define QIB_6120_SerdesCfg0_LPBKC_RMASK 0x1
790#define QIB_6120_SerdesCfg0_LPBKD_LSB 0x2C
791#define QIB_6120_SerdesCfg0_LPBKD_RMASK 0x1
792#define QIB_6120_SerdesCfg0_PW_LSB 0x2B
793#define QIB_6120_SerdesCfg0_PW_RMASK 0x1
794#define QIB_6120_SerdesCfg0_RefSel_LSB 0x29
795#define QIB_6120_SerdesCfg0_RefSel_RMASK 0x3
796#define QIB_6120_SerdesCfg0_ParReset_LSB 0x28
797#define QIB_6120_SerdesCfg0_ParReset_RMASK 0x1
798#define QIB_6120_SerdesCfg0_ParLPBK_LSB 0x27
799#define QIB_6120_SerdesCfg0_ParLPBK_RMASK 0x1
800#define QIB_6120_SerdesCfg0_OffsetEn_LSB 0x26
801#define QIB_6120_SerdesCfg0_OffsetEn_RMASK 0x1
802#define QIB_6120_SerdesCfg0_Offset_LSB 0x1E
803#define QIB_6120_SerdesCfg0_Offset_RMASK 0xFF
804#define QIB_6120_SerdesCfg0_L2PwrDn_LSB 0x1D
805#define QIB_6120_SerdesCfg0_L2PwrDn_RMASK 0x1
806#define QIB_6120_SerdesCfg0_ResetPLL_LSB 0x1C
807#define QIB_6120_SerdesCfg0_ResetPLL_RMASK 0x1
808#define QIB_6120_SerdesCfg0_RxTermEnX_LSB 0x18
809#define QIB_6120_SerdesCfg0_RxTermEnX_RMASK 0xF
810#define QIB_6120_SerdesCfg0_BeaconTxEnX_LSB 0x14
811#define QIB_6120_SerdesCfg0_BeaconTxEnX_RMASK 0xF
812#define QIB_6120_SerdesCfg0_RxDetEnX_LSB 0x10
813#define QIB_6120_SerdesCfg0_RxDetEnX_RMASK 0xF
814#define QIB_6120_SerdesCfg0_TxIdeEnX_LSB 0xC
815#define QIB_6120_SerdesCfg0_TxIdeEnX_RMASK 0xF
816#define QIB_6120_SerdesCfg0_RxIdleEnX_LSB 0x8
817#define QIB_6120_SerdesCfg0_RxIdleEnX_RMASK 0xF
818#define QIB_6120_SerdesCfg0_L1PwrDnA_LSB 0x7
819#define QIB_6120_SerdesCfg0_L1PwrDnA_RMASK 0x1
820#define QIB_6120_SerdesCfg0_L1PwrDnB_LSB 0x6
821#define QIB_6120_SerdesCfg0_L1PwrDnB_RMASK 0x1
822#define QIB_6120_SerdesCfg0_L1PwrDnC_LSB 0x5
823#define QIB_6120_SerdesCfg0_L1PwrDnC_RMASK 0x1
824#define QIB_6120_SerdesCfg0_L1PwrDnD_LSB 0x4
825#define QIB_6120_SerdesCfg0_L1PwrDnD_RMASK 0x1
826#define QIB_6120_SerdesCfg0_ResetA_LSB 0x3
827#define QIB_6120_SerdesCfg0_ResetA_RMASK 0x1
828#define QIB_6120_SerdesCfg0_ResetB_LSB 0x2
829#define QIB_6120_SerdesCfg0_ResetB_RMASK 0x1
830#define QIB_6120_SerdesCfg0_ResetC_LSB 0x1
831#define QIB_6120_SerdesCfg0_ResetC_RMASK 0x1
832#define QIB_6120_SerdesCfg0_ResetD_LSB 0x0
833#define QIB_6120_SerdesCfg0_ResetD_RMASK 0x1
834
835#define QIB_6120_SerdesStat_OFFS 0x3D0
836#define QIB_6120_SerdesStat_Reserved_LSB 0xC
837#define QIB_6120_SerdesStat_Reserved_RMASK 0xFFFFFFFFFFFFF
838#define QIB_6120_SerdesStat_BeaconDetA_LSB 0xB
839#define QIB_6120_SerdesStat_BeaconDetA_RMASK 0x1
840#define QIB_6120_SerdesStat_BeaconDetB_LSB 0xA
841#define QIB_6120_SerdesStat_BeaconDetB_RMASK 0x1
842#define QIB_6120_SerdesStat_BeaconDetC_LSB 0x9
843#define QIB_6120_SerdesStat_BeaconDetC_RMASK 0x1
844#define QIB_6120_SerdesStat_BeaconDetD_LSB 0x8
845#define QIB_6120_SerdesStat_BeaconDetD_RMASK 0x1
846#define QIB_6120_SerdesStat_RxDetA_LSB 0x7
847#define QIB_6120_SerdesStat_RxDetA_RMASK 0x1
848#define QIB_6120_SerdesStat_RxDetB_LSB 0x6
849#define QIB_6120_SerdesStat_RxDetB_RMASK 0x1
850#define QIB_6120_SerdesStat_RxDetC_LSB 0x5
851#define QIB_6120_SerdesStat_RxDetC_RMASK 0x1
852#define QIB_6120_SerdesStat_RxDetD_LSB 0x4
853#define QIB_6120_SerdesStat_RxDetD_RMASK 0x1
854#define QIB_6120_SerdesStat_TxIdleDetA_LSB 0x3
855#define QIB_6120_SerdesStat_TxIdleDetA_RMASK 0x1
856#define QIB_6120_SerdesStat_TxIdleDetB_LSB 0x2
857#define QIB_6120_SerdesStat_TxIdleDetB_RMASK 0x1
858#define QIB_6120_SerdesStat_TxIdleDetC_LSB 0x1
859#define QIB_6120_SerdesStat_TxIdleDetC_RMASK 0x1
860#define QIB_6120_SerdesStat_TxIdleDetD_LSB 0x0
861#define QIB_6120_SerdesStat_TxIdleDetD_RMASK 0x1
862
863#define QIB_6120_XGXSCfg_OFFS 0x3D8
864#define QIB_6120_XGXSCfg_ArmLaunchErrorDisable_LSB 0x3F
865#define QIB_6120_XGXSCfg_ArmLaunchErrorDisable_RMASK 0x1
866#define QIB_6120_XGXSCfg_Reserved_LSB 0x17
867#define QIB_6120_XGXSCfg_Reserved_RMASK 0xFFFFFFFFFF
868#define QIB_6120_XGXSCfg_polarity_inv_LSB 0x13
869#define QIB_6120_XGXSCfg_polarity_inv_RMASK 0xF
870#define QIB_6120_XGXSCfg_link_sync_mask_LSB 0x9
871#define QIB_6120_XGXSCfg_link_sync_mask_RMASK 0x3FF
872#define QIB_6120_XGXSCfg_port_addr_LSB 0x4
873#define QIB_6120_XGXSCfg_port_addr_RMASK 0x1F
874#define QIB_6120_XGXSCfg_mdd_30_LSB 0x3
875#define QIB_6120_XGXSCfg_mdd_30_RMASK 0x1
876#define QIB_6120_XGXSCfg_xcv_resetn_LSB 0x2
877#define QIB_6120_XGXSCfg_xcv_resetn_RMASK 0x1
878#define QIB_6120_XGXSCfg_Reserved1_LSB 0x1
879#define QIB_6120_XGXSCfg_Reserved1_RMASK 0x1
880#define QIB_6120_XGXSCfg_tx_rx_resetn_LSB 0x0
881#define QIB_6120_XGXSCfg_tx_rx_resetn_RMASK 0x1
882
883#define QIB_6120_LBIntCnt_OFFS 0x12000
884
885#define QIB_6120_LBFlowStallCnt_OFFS 0x12008
886
887#define QIB_6120_TxUnsupVLErrCnt_OFFS 0x12018
888
889#define QIB_6120_TxDataPktCnt_OFFS 0x12020
890
891#define QIB_6120_TxFlowPktCnt_OFFS 0x12028
892
893#define QIB_6120_TxDwordCnt_OFFS 0x12030
894
895#define QIB_6120_TxLenErrCnt_OFFS 0x12038
896
897#define QIB_6120_TxMaxMinLenErrCnt_OFFS 0x12040
898
899#define QIB_6120_TxUnderrunCnt_OFFS 0x12048
900
901#define QIB_6120_TxFlowStallCnt_OFFS 0x12050
902
903#define QIB_6120_TxDroppedPktCnt_OFFS 0x12058
904
905#define QIB_6120_RxDroppedPktCnt_OFFS 0x12060
906
907#define QIB_6120_RxDataPktCnt_OFFS 0x12068
908
909#define QIB_6120_RxFlowPktCnt_OFFS 0x12070
910
911#define QIB_6120_RxDwordCnt_OFFS 0x12078
912
913#define QIB_6120_RxLenErrCnt_OFFS 0x12080
914
915#define QIB_6120_RxMaxMinLenErrCnt_OFFS 0x12088
916
917#define QIB_6120_RxICRCErrCnt_OFFS 0x12090
918
919#define QIB_6120_RxVCRCErrCnt_OFFS 0x12098
920
921#define QIB_6120_RxFlowCtrlErrCnt_OFFS 0x120A0
922
923#define QIB_6120_RxBadFormatCnt_OFFS 0x120A8
924
925#define QIB_6120_RxLinkProblemCnt_OFFS 0x120B0
926
927#define QIB_6120_RxEBPCnt_OFFS 0x120B8
928
929#define QIB_6120_RxLPCRCErrCnt_OFFS 0x120C0
930
931#define QIB_6120_RxBufOvflCnt_OFFS 0x120C8
932
933#define QIB_6120_RxTIDFullErrCnt_OFFS 0x120D0
934
935#define QIB_6120_RxTIDValidErrCnt_OFFS 0x120D8
936
937#define QIB_6120_RxPKeyMismatchCnt_OFFS 0x120E0
938
939#define QIB_6120_RxP0HdrEgrOvflCnt_OFFS 0x120E8
940
941#define QIB_6120_IBStatusChangeCnt_OFFS 0x12140
942
943#define QIB_6120_IBLinkErrRecoveryCnt_OFFS 0x12148
944
945#define QIB_6120_IBLinkDownedCnt_OFFS 0x12150
946
947#define QIB_6120_IBSymbolErrCnt_OFFS 0x12158
948
949#define QIB_6120_PcieRetryBufDiagQwordCnt_OFFS 0x12170
950
951#define QIB_6120_RcvEgrArray0_OFFS 0x14000
952
953#define QIB_6120_RcvTIDArray0_OFFS 0x54000
954
955#define QIB_6120_PIOLaunchFIFO_OFFS 0x64000
956
957#define QIB_6120_SendPIOpbcCache_OFFS 0x64800
958
959#define QIB_6120_RcvBuf1_OFFS 0x72000
960
961#define QIB_6120_RcvBuf2_OFFS 0x75000
962
963#define QIB_6120_RcvFlags_OFFS 0x77000
964
965#define QIB_6120_RcvLookupBuf1_OFFS 0x79000
966
967#define QIB_6120_RcvDMABuf_OFFS 0x7B000
968
969#define QIB_6120_MiscRXEIntMem_OFFS 0x7C000
970
971#define QIB_6120_PCIERcvBuf_OFFS 0x80000
972
973#define QIB_6120_PCIERetryBuf_OFFS 0x82000
974
975#define QIB_6120_PCIERcvBufRdToWrAddr_OFFS 0x84000
976
977#define QIB_6120_PIOBuf0_MA_OFFS 0x100000
drivers/infiniband/hw/qib/qib_7220.h+0−149
@@ -1,149 +0,0 @@
1#ifndef _QIB_7220_H
2#define _QIB_7220_H
3/*
4 * Copyright (c) 2007, 2009, 2010 QLogic Corporation. All rights reserved.
5 *
6 * This software is available to you under a choice of one of two
7 * licenses. You may choose to be licensed under the terms of the GNU
8 * General Public License (GPL) Version 2, available from the file
9 * COPYING in the main directory of this source tree, or the
10 * OpenIB.org BSD license below:
11 *
12 * Redistribution and use in source and binary forms, with or
13 * without modification, are permitted provided that the following
14 * conditions are met:
15 *
16 * - Redistributions of source code must retain the above
17 * copyright notice, this list of conditions and the following
18 * disclaimer.
19 *
20 * - Redistributions in binary form must reproduce the above
21 * copyright notice, this list of conditions and the following
22 * disclaimer in the documentation and/or other materials
23 * provided with the distribution.
24 *
25 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
26 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
27 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
28 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
29 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
30 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
31 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
32 * SOFTWARE.
33 */
34
35/* grab register-defs auto-generated by HW */
36#include "qib_7220_regs.h"
37
38/* The number of eager receive TIDs for context zero. */
39#define IBA7220_KRCVEGRCNT 2048U
40
41#define IB_7220_LT_STATE_CFGRCVFCFG 0x09
42#define IB_7220_LT_STATE_CFGWAITRMT 0x0a
43#define IB_7220_LT_STATE_TXREVLANES 0x0d
44#define IB_7220_LT_STATE_CFGENH 0x10
45
46struct qib_chip_specific {
47 u64 __iomem *cregbase;
48 u64 *cntrs;
49 u64 *portcntrs;
50 spinlock_t sdepb_lock; /* serdes EPB bus */
51 spinlock_t rcvmod_lock; /* protect rcvctrl shadow changes */
52 spinlock_t gpio_lock; /* RMW of shadows/regs for ExtCtrl and GPIO */
53 u64 hwerrmask;
54 u64 errormask;
55 u64 gpio_out; /* shadow of kr_gpio_out, for rmw ops */
56 u64 gpio_mask; /* shadow the gpio mask register */
57 u64 extctrl; /* shadow the gpio output enable, etc... */
58 u32 ncntrs;
59 u32 nportcntrs;
60 u32 cntrnamelen;
61 u32 portcntrnamelen;
62 u32 numctxts;
63 u32 rcvegrcnt;
64 u32 autoneg_tries;
65 u32 serdes_first_init_done;
66 u32 sdmabufcnt;
67 u32 lastbuf_for_pio;
68 u32 updthresh; /* current AvailUpdThld */
69 u32 updthresh_dflt; /* default AvailUpdThld */
70 u8 presets_needed;
71 u8 relock_timer_active;
72 char emsgbuf[128];
73 char sdmamsgbuf[192];
74 char bitsmsgbuf[64];
75 struct timer_list relock_timer;
76 unsigned int relock_interval; /* in jiffies */
77 struct qib_devdata *dd;
78};
79
80struct qib_chippport_specific {
81 struct qib_pportdata pportdata;
82 wait_queue_head_t autoneg_wait;
83 struct delayed_work autoneg_work;
84 struct timer_list chase_timer;
85 /*
86 * these 5 fields are used to establish deltas for IB symbol
87 * errors and linkrecovery errors. They can be reported on
88 * some chips during link negotiation prior to INIT, and with
89 * DDR when faking DDR negotiations with non-IBTA switches.
90 * The chip counters are adjusted at driver unload if there is
91 * a non-zero delta.
92 */
93 u64 ibdeltainprog;
94 u64 ibsymdelta;
95 u64 ibsymsnap;
96 u64 iblnkerrdelta;
97 u64 iblnkerrsnap;
98 u64 ibcctrl; /* kr_ibcctrl shadow */
99 u64 ibcddrctrl; /* kr_ibcddrctrl shadow */
100 unsigned long chase_end;
101 u32 last_delay_mult;
102};
103
104/*
105 * This header file provides the declarations and common definitions
106 * for (mostly) manipulation of the SerDes blocks within the IBA7220.
107 * the functions declared should only be called from within other
108 * 7220-related files such as qib_iba7220.c or qib_sd7220.c.
109 */
110int qib_sd7220_presets(struct qib_devdata *dd);
111int qib_sd7220_init(struct qib_devdata *dd);
112void qib_sd7220_clr_ibpar(struct qib_devdata *);
113/*
114 * Below used for sdnum parameter, selecting one of the two sections
115 * used for PCIe, or the single SerDes used for IB, which is the
116 * only one currently used
117 */
118#define IB_7220_SERDES 2
119
120static inline u32 qib_read_kreg32(const struct qib_devdata *dd,
121 const u16 regno)
122{
123 if (!dd->kregbase || !(dd->flags & QIB_PRESENT))
124 return -1;
125 return readl((u32 __iomem *)&dd->kregbase[regno]);
126}
127
128static inline u64 qib_read_kreg64(const struct qib_devdata *dd,
129 const u16 regno)
130{
131 if (!dd->kregbase || !(dd->flags & QIB_PRESENT))
132 return -1;
133
134 return readq(&dd->kregbase[regno]);
135}
136
137static inline void qib_write_kreg(const struct qib_devdata *dd,
138 const u16 regno, u64 value)
139{
140 if (dd->kregbase)
141 writeq(value, &dd->kregbase[regno]);
142}
143
144void set_7220_relock_poll(struct qib_devdata *, int);
145void shutdown_7220_relock_poll(struct qib_devdata *);
146void toggle_7220_rclkrls(struct qib_devdata *);
147
148
149#endif /* _QIB_7220_H */
drivers/infiniband/hw/qib/qib_7220_regs.h+0−1496
@@ -1,1496 +0,0 @@
1/*
2 * Copyright (c) 2008, 2009, 2010 QLogic Corporation. All rights reserved.
3 *
4 *
5 * This software is available to you under a choice of one of two
6 * licenses. You may choose to be licensed under the terms of the GNU
7 * General Public License (GPL) Version 2, available from the file
8 * COPYING in the main directory of this source tree, or the
9 * OpenIB.org BSD license below:
10 *
11 * Redistribution and use in source and binary forms, with or
12 * without modification, are permitted provided that the following
13 * conditions are met:
14 *
15 * - Redistributions of source code must retain the above
16 * copyright notice, this list of conditions and the following
17 * disclaimer.
18 *
19 * - Redistributions in binary form must reproduce the above
20 * copyright notice, this list of conditions and the following
21 * disclaimer in the documentation and/or other materials
22 * provided with the distribution.
23 *
24 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
25 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
26 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
27 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
28 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
29 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
30 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
31 * SOFTWARE.
32 *
33 */
34
35/* This file is mechanically generated from RTL. Any hand-edits will be lost! */
36
37#define QIB_7220_Revision_OFFS 0x0
38#define QIB_7220_Revision_R_Simulator_LSB 0x3F
39#define QIB_7220_Revision_R_Simulator_RMASK 0x1
40#define QIB_7220_Revision_R_Emulation_LSB 0x3E
41#define QIB_7220_Revision_R_Emulation_RMASK 0x1
42#define QIB_7220_Revision_R_Emulation_Revcode_LSB 0x28
43#define QIB_7220_Revision_R_Emulation_Revcode_RMASK 0x3FFFFF
44#define QIB_7220_Revision_BoardID_LSB 0x20
45#define QIB_7220_Revision_BoardID_RMASK 0xFF
46#define QIB_7220_Revision_R_SW_LSB 0x18
47#define QIB_7220_Revision_R_SW_RMASK 0xFF
48#define QIB_7220_Revision_R_Arch_LSB 0x10
49#define QIB_7220_Revision_R_Arch_RMASK 0xFF
50#define QIB_7220_Revision_R_ChipRevMajor_LSB 0x8
51#define QIB_7220_Revision_R_ChipRevMajor_RMASK 0xFF
52#define QIB_7220_Revision_R_ChipRevMinor_LSB 0x0
53#define QIB_7220_Revision_R_ChipRevMinor_RMASK 0xFF
54
55#define QIB_7220_Control_OFFS 0x8
56#define QIB_7220_Control_SyncResetExceptPcieIRAMRST_LSB 0x7
57#define QIB_7220_Control_SyncResetExceptPcieIRAMRST_RMASK 0x1
58#define QIB_7220_Control_PCIECplQDiagEn_LSB 0x6
59#define QIB_7220_Control_PCIECplQDiagEn_RMASK 0x1
60#define QIB_7220_Control_Reserved_LSB 0x5
61#define QIB_7220_Control_Reserved_RMASK 0x1
62#define QIB_7220_Control_TxLatency_LSB 0x4
63#define QIB_7220_Control_TxLatency_RMASK 0x1
64#define QIB_7220_Control_PCIERetryBufDiagEn_LSB 0x3
65#define QIB_7220_Control_PCIERetryBufDiagEn_RMASK 0x1
66#define QIB_7220_Control_LinkEn_LSB 0x2
67#define QIB_7220_Control_LinkEn_RMASK 0x1
68#define QIB_7220_Control_FreezeMode_LSB 0x1
69#define QIB_7220_Control_FreezeMode_RMASK 0x1
70#define QIB_7220_Control_SyncReset_LSB 0x0
71#define QIB_7220_Control_SyncReset_RMASK 0x1
72
73#define QIB_7220_PageAlign_OFFS 0x10
74
75#define QIB_7220_PortCnt_OFFS 0x18
76
77#define QIB_7220_SendRegBase_OFFS 0x30
78
79#define QIB_7220_UserRegBase_OFFS 0x38
80
81#define QIB_7220_CntrRegBase_OFFS 0x40
82
83#define QIB_7220_Scratch_OFFS 0x48
84
85#define QIB_7220_IntMask_OFFS 0x68
86#define QIB_7220_IntMask_SDmaIntMask_LSB 0x3F
87#define QIB_7220_IntMask_SDmaIntMask_RMASK 0x1
88#define QIB_7220_IntMask_SDmaDisabledMasked_LSB 0x3E
89#define QIB_7220_IntMask_SDmaDisabledMasked_RMASK 0x1
90#define QIB_7220_IntMask_Reserved_LSB 0x31
91#define QIB_7220_IntMask_Reserved_RMASK 0x1FFF
92#define QIB_7220_IntMask_RcvUrg16IntMask_LSB 0x30
93#define QIB_7220_IntMask_RcvUrg16IntMask_RMASK 0x1
94#define QIB_7220_IntMask_RcvUrg15IntMask_LSB 0x2F
95#define QIB_7220_IntMask_RcvUrg15IntMask_RMASK 0x1
96#define QIB_7220_IntMask_RcvUrg14IntMask_LSB 0x2E
97#define QIB_7220_IntMask_RcvUrg14IntMask_RMASK 0x1
98#define QIB_7220_IntMask_RcvUrg13IntMask_LSB 0x2D
99#define QIB_7220_IntMask_RcvUrg13IntMask_RMASK 0x1
100#define QIB_7220_IntMask_RcvUrg12IntMask_LSB 0x2C
101#define QIB_7220_IntMask_RcvUrg12IntMask_RMASK 0x1
102#define QIB_7220_IntMask_RcvUrg11IntMask_LSB 0x2B
103#define QIB_7220_IntMask_RcvUrg11IntMask_RMASK 0x1
104#define QIB_7220_IntMask_RcvUrg10IntMask_LSB 0x2A
105#define QIB_7220_IntMask_RcvUrg10IntMask_RMASK 0x1
106#define QIB_7220_IntMask_RcvUrg9IntMask_LSB 0x29
107#define QIB_7220_IntMask_RcvUrg9IntMask_RMASK 0x1
108#define QIB_7220_IntMask_RcvUrg8IntMask_LSB 0x28
109#define QIB_7220_IntMask_RcvUrg8IntMask_RMASK 0x1
110#define QIB_7220_IntMask_RcvUrg7IntMask_LSB 0x27
111#define QIB_7220_IntMask_RcvUrg7IntMask_RMASK 0x1
112#define QIB_7220_IntMask_RcvUrg6IntMask_LSB 0x26
113#define QIB_7220_IntMask_RcvUrg6IntMask_RMASK 0x1
114#define QIB_7220_IntMask_RcvUrg5IntMask_LSB 0x25
115#define QIB_7220_IntMask_RcvUrg5IntMask_RMASK 0x1
116#define QIB_7220_IntMask_RcvUrg4IntMask_LSB 0x24
117#define QIB_7220_IntMask_RcvUrg4IntMask_RMASK 0x1
118#define QIB_7220_IntMask_RcvUrg3IntMask_LSB 0x23
119#define QIB_7220_IntMask_RcvUrg3IntMask_RMASK 0x1
120#define QIB_7220_IntMask_RcvUrg2IntMask_LSB 0x22
121#define QIB_7220_IntMask_RcvUrg2IntMask_RMASK 0x1
122#define QIB_7220_IntMask_RcvUrg1IntMask_LSB 0x21
123#define QIB_7220_IntMask_RcvUrg1IntMask_RMASK 0x1
124#define QIB_7220_IntMask_RcvUrg0IntMask_LSB 0x20
125#define QIB_7220_IntMask_RcvUrg0IntMask_RMASK 0x1
126#define QIB_7220_IntMask_ErrorIntMask_LSB 0x1F
127#define QIB_7220_IntMask_ErrorIntMask_RMASK 0x1
128#define QIB_7220_IntMask_PioSetIntMask_LSB 0x1E
129#define QIB_7220_IntMask_PioSetIntMask_RMASK 0x1
130#define QIB_7220_IntMask_PioBufAvailIntMask_LSB 0x1D
131#define QIB_7220_IntMask_PioBufAvailIntMask_RMASK 0x1
132#define QIB_7220_IntMask_assertGPIOIntMask_LSB 0x1C
133#define QIB_7220_IntMask_assertGPIOIntMask_RMASK 0x1
134#define QIB_7220_IntMask_IBSerdesTrimDoneIntMask_LSB 0x1B
135#define QIB_7220_IntMask_IBSerdesTrimDoneIntMask_RMASK 0x1
136#define QIB_7220_IntMask_JIntMask_LSB 0x1A
137#define QIB_7220_IntMask_JIntMask_RMASK 0x1
138#define QIB_7220_IntMask_Reserved1_LSB 0x11
139#define QIB_7220_IntMask_Reserved1_RMASK 0x1FF
140#define QIB_7220_IntMask_RcvAvail16IntMask_LSB 0x10
141#define QIB_7220_IntMask_RcvAvail16IntMask_RMASK 0x1
142#define QIB_7220_IntMask_RcvAvail15IntMask_LSB 0xF
143#define QIB_7220_IntMask_RcvAvail15IntMask_RMASK 0x1
144#define QIB_7220_IntMask_RcvAvail14IntMask_LSB 0xE
145#define QIB_7220_IntMask_RcvAvail14IntMask_RMASK 0x1
146#define QIB_7220_IntMask_RcvAvail13IntMask_LSB 0xD
147#define QIB_7220_IntMask_RcvAvail13IntMask_RMASK 0x1
148#define QIB_7220_IntMask_RcvAvail12IntMask_LSB 0xC
149#define QIB_7220_IntMask_RcvAvail12IntMask_RMASK 0x1
150#define QIB_7220_IntMask_RcvAvail11IntMask_LSB 0xB
151#define QIB_7220_IntMask_RcvAvail11IntMask_RMASK 0x1
152#define QIB_7220_IntMask_RcvAvail10IntMask_LSB 0xA
153#define QIB_7220_IntMask_RcvAvail10IntMask_RMASK 0x1
154#define QIB_7220_IntMask_RcvAvail9IntMask_LSB 0x9
155#define QIB_7220_IntMask_RcvAvail9IntMask_RMASK 0x1
156#define QIB_7220_IntMask_RcvAvail8IntMask_LSB 0x8
157#define QIB_7220_IntMask_RcvAvail8IntMask_RMASK 0x1
158#define QIB_7220_IntMask_RcvAvail7IntMask_LSB 0x7
159#define QIB_7220_IntMask_RcvAvail7IntMask_RMASK 0x1
160#define QIB_7220_IntMask_RcvAvail6IntMask_LSB 0x6
161#define QIB_7220_IntMask_RcvAvail6IntMask_RMASK 0x1
162#define QIB_7220_IntMask_RcvAvail5IntMask_LSB 0x5
163#define QIB_7220_IntMask_RcvAvail5IntMask_RMASK 0x1
164#define QIB_7220_IntMask_RcvAvail4IntMask_LSB 0x4
165#define QIB_7220_IntMask_RcvAvail4IntMask_RMASK 0x1
166#define QIB_7220_IntMask_RcvAvail3IntMask_LSB 0x3
167#define QIB_7220_IntMask_RcvAvail3IntMask_RMASK 0x1
168#define QIB_7220_IntMask_RcvAvail2IntMask_LSB 0x2
169#define QIB_7220_IntMask_RcvAvail2IntMask_RMASK 0x1
170#define QIB_7220_IntMask_RcvAvail1IntMask_LSB 0x1
171#define QIB_7220_IntMask_RcvAvail1IntMask_RMASK 0x1
172#define QIB_7220_IntMask_RcvAvail0IntMask_LSB 0x0
173#define QIB_7220_IntMask_RcvAvail0IntMask_RMASK 0x1
174
175#define QIB_7220_IntStatus_OFFS 0x70
176#define QIB_7220_IntStatus_SDmaInt_LSB 0x3F
177#define QIB_7220_IntStatus_SDmaInt_RMASK 0x1
178#define QIB_7220_IntStatus_SDmaDisabled_LSB 0x3E
179#define QIB_7220_IntStatus_SDmaDisabled_RMASK 0x1
180#define QIB_7220_IntStatus_Reserved_LSB 0x31
181#define QIB_7220_IntStatus_Reserved_RMASK 0x1FFF
182#define QIB_7220_IntStatus_RcvUrg16_LSB 0x30
183#define QIB_7220_IntStatus_RcvUrg16_RMASK 0x1
184#define QIB_7220_IntStatus_RcvUrg15_LSB 0x2F
185#define QIB_7220_IntStatus_RcvUrg15_RMASK 0x1
186#define QIB_7220_IntStatus_RcvUrg14_LSB 0x2E
187#define QIB_7220_IntStatus_RcvUrg14_RMASK 0x1
188#define QIB_7220_IntStatus_RcvUrg13_LSB 0x2D
189#define QIB_7220_IntStatus_RcvUrg13_RMASK 0x1
190#define QIB_7220_IntStatus_RcvUrg12_LSB 0x2C
191#define QIB_7220_IntStatus_RcvUrg12_RMASK 0x1
192#define QIB_7220_IntStatus_RcvUrg11_LSB 0x2B
193#define QIB_7220_IntStatus_RcvUrg11_RMASK 0x1
194#define QIB_7220_IntStatus_RcvUrg10_LSB 0x2A
195#define QIB_7220_IntStatus_RcvUrg10_RMASK 0x1
196#define QIB_7220_IntStatus_RcvUrg9_LSB 0x29
197#define QIB_7220_IntStatus_RcvUrg9_RMASK 0x1
198#define QIB_7220_IntStatus_RcvUrg8_LSB 0x28
199#define QIB_7220_IntStatus_RcvUrg8_RMASK 0x1
200#define QIB_7220_IntStatus_RcvUrg7_LSB 0x27
201#define QIB_7220_IntStatus_RcvUrg7_RMASK 0x1
202#define QIB_7220_IntStatus_RcvUrg6_LSB 0x26
203#define QIB_7220_IntStatus_RcvUrg6_RMASK 0x1
204#define QIB_7220_IntStatus_RcvUrg5_LSB 0x25
205#define QIB_7220_IntStatus_RcvUrg5_RMASK 0x1
206#define QIB_7220_IntStatus_RcvUrg4_LSB 0x24
207#define QIB_7220_IntStatus_RcvUrg4_RMASK 0x1
208#define QIB_7220_IntStatus_RcvUrg3_LSB 0x23
209#define QIB_7220_IntStatus_RcvUrg3_RMASK 0x1
210#define QIB_7220_IntStatus_RcvUrg2_LSB 0x22
211#define QIB_7220_IntStatus_RcvUrg2_RMASK 0x1
212#define QIB_7220_IntStatus_RcvUrg1_LSB 0x21
213#define QIB_7220_IntStatus_RcvUrg1_RMASK 0x1
214#define QIB_7220_IntStatus_RcvUrg0_LSB 0x20
215#define QIB_7220_IntStatus_RcvUrg0_RMASK 0x1
216#define QIB_7220_IntStatus_Error_LSB 0x1F
217#define QIB_7220_IntStatus_Error_RMASK 0x1
218#define QIB_7220_IntStatus_PioSent_LSB 0x1E
219#define QIB_7220_IntStatus_PioSent_RMASK 0x1
220#define QIB_7220_IntStatus_PioBufAvail_LSB 0x1D
221#define QIB_7220_IntStatus_PioBufAvail_RMASK 0x1
222#define QIB_7220_IntStatus_assertGPIO_LSB 0x1C
223#define QIB_7220_IntStatus_assertGPIO_RMASK 0x1
224#define QIB_7220_IntStatus_IBSerdesTrimDone_LSB 0x1B
225#define QIB_7220_IntStatus_IBSerdesTrimDone_RMASK 0x1
226#define QIB_7220_IntStatus_JInt_LSB 0x1A
227#define QIB_7220_IntStatus_JInt_RMASK 0x1
228#define QIB_7220_IntStatus_Reserved1_LSB 0x11
229#define QIB_7220_IntStatus_Reserved1_RMASK 0x1FF
230#define QIB_7220_IntStatus_RcvAvail16_LSB 0x10
231#define QIB_7220_IntStatus_RcvAvail16_RMASK 0x1
232#define QIB_7220_IntStatus_RcvAvail15_LSB 0xF
233#define QIB_7220_IntStatus_RcvAvail15_RMASK 0x1
234#define QIB_7220_IntStatus_RcvAvail14_LSB 0xE
235#define QIB_7220_IntStatus_RcvAvail14_RMASK 0x1
236#define QIB_7220_IntStatus_RcvAvail13_LSB 0xD
237#define QIB_7220_IntStatus_RcvAvail13_RMASK 0x1
238#define QIB_7220_IntStatus_RcvAvail12_LSB 0xC
239#define QIB_7220_IntStatus_RcvAvail12_RMASK 0x1
240#define QIB_7220_IntStatus_RcvAvail11_LSB 0xB
241#define QIB_7220_IntStatus_RcvAvail11_RMASK 0x1
242#define QIB_7220_IntStatus_RcvAvail10_LSB 0xA
243#define QIB_7220_IntStatus_RcvAvail10_RMASK 0x1
244#define QIB_7220_IntStatus_RcvAvail9_LSB 0x9
245#define QIB_7220_IntStatus_RcvAvail9_RMASK 0x1
246#define QIB_7220_IntStatus_RcvAvail8_LSB 0x8
247#define QIB_7220_IntStatus_RcvAvail8_RMASK 0x1
248#define QIB_7220_IntStatus_RcvAvail7_LSB 0x7
249#define QIB_7220_IntStatus_RcvAvail7_RMASK 0x1
250#define QIB_7220_IntStatus_RcvAvail6_LSB 0x6
251#define QIB_7220_IntStatus_RcvAvail6_RMASK 0x1
252#define QIB_7220_IntStatus_RcvAvail5_LSB 0x5
253#define QIB_7220_IntStatus_RcvAvail5_RMASK 0x1
254#define QIB_7220_IntStatus_RcvAvail4_LSB 0x4
255#define QIB_7220_IntStatus_RcvAvail4_RMASK 0x1
256#define QIB_7220_IntStatus_RcvAvail3_LSB 0x3
257#define QIB_7220_IntStatus_RcvAvail3_RMASK 0x1
258#define QIB_7220_IntStatus_RcvAvail2_LSB 0x2
259#define QIB_7220_IntStatus_RcvAvail2_RMASK 0x1
260#define QIB_7220_IntStatus_RcvAvail1_LSB 0x1
261#define QIB_7220_IntStatus_RcvAvail1_RMASK 0x1
262#define QIB_7220_IntStatus_RcvAvail0_LSB 0x0
263#define QIB_7220_IntStatus_RcvAvail0_RMASK 0x1
264
265#define QIB_7220_IntClear_OFFS 0x78
266#define QIB_7220_IntClear_SDmaIntClear_LSB 0x3F
267#define QIB_7220_IntClear_SDmaIntClear_RMASK 0x1
268#define QIB_7220_IntClear_SDmaDisabledClear_LSB 0x3E
269#define QIB_7220_IntClear_SDmaDisabledClear_RMASK 0x1
270#define QIB_7220_IntClear_Reserved_LSB 0x31
271#define QIB_7220_IntClear_Reserved_RMASK 0x1FFF
272#define QIB_7220_IntClear_RcvUrg16IntClear_LSB 0x30
273#define QIB_7220_IntClear_RcvUrg16IntClear_RMASK 0x1
274#define QIB_7220_IntClear_RcvUrg15IntClear_LSB 0x2F
275#define QIB_7220_IntClear_RcvUrg15IntClear_RMASK 0x1
276#define QIB_7220_IntClear_RcvUrg14IntClear_LSB 0x2E
277#define QIB_7220_IntClear_RcvUrg14IntClear_RMASK 0x1
278#define QIB_7220_IntClear_RcvUrg13IntClear_LSB 0x2D
279#define QIB_7220_IntClear_RcvUrg13IntClear_RMASK 0x1
280#define QIB_7220_IntClear_RcvUrg12IntClear_LSB 0x2C
281#define QIB_7220_IntClear_RcvUrg12IntClear_RMASK 0x1
282#define QIB_7220_IntClear_RcvUrg11IntClear_LSB 0x2B
283#define QIB_7220_IntClear_RcvUrg11IntClear_RMASK 0x1
284#define QIB_7220_IntClear_RcvUrg10IntClear_LSB 0x2A
285#define QIB_7220_IntClear_RcvUrg10IntClear_RMASK 0x1
286#define QIB_7220_IntClear_RcvUrg9IntClear_LSB 0x29
287#define QIB_7220_IntClear_RcvUrg9IntClear_RMASK 0x1
288#define QIB_7220_IntClear_RcvUrg8IntClear_LSB 0x28
289#define QIB_7220_IntClear_RcvUrg8IntClear_RMASK 0x1
290#define QIB_7220_IntClear_RcvUrg7IntClear_LSB 0x27
291#define QIB_7220_IntClear_RcvUrg7IntClear_RMASK 0x1
292#define QIB_7220_IntClear_RcvUrg6IntClear_LSB 0x26
293#define QIB_7220_IntClear_RcvUrg6IntClear_RMASK 0x1
294#define QIB_7220_IntClear_RcvUrg5IntClear_LSB 0x25
295#define QIB_7220_IntClear_RcvUrg5IntClear_RMASK 0x1
296#define QIB_7220_IntClear_RcvUrg4IntClear_LSB 0x24
297#define QIB_7220_IntClear_RcvUrg4IntClear_RMASK 0x1
298#define QIB_7220_IntClear_RcvUrg3IntClear_LSB 0x23
299#define QIB_7220_IntClear_RcvUrg3IntClear_RMASK 0x1
300#define QIB_7220_IntClear_RcvUrg2IntClear_LSB 0x22
301#define QIB_7220_IntClear_RcvUrg2IntClear_RMASK 0x1
302#define QIB_7220_IntClear_RcvUrg1IntClear_LSB 0x21
303#define QIB_7220_IntClear_RcvUrg1IntClear_RMASK 0x1
304#define QIB_7220_IntClear_RcvUrg0IntClear_LSB 0x20
305#define QIB_7220_IntClear_RcvUrg0IntClear_RMASK 0x1
306#define QIB_7220_IntClear_ErrorIntClear_LSB 0x1F
307#define QIB_7220_IntClear_ErrorIntClear_RMASK 0x1
308#define QIB_7220_IntClear_PioSetIntClear_LSB 0x1E
309#define QIB_7220_IntClear_PioSetIntClear_RMASK 0x1
310#define QIB_7220_IntClear_PioBufAvailIntClear_LSB 0x1D
311#define QIB_7220_IntClear_PioBufAvailIntClear_RMASK 0x1
312#define QIB_7220_IntClear_assertGPIOIntClear_LSB 0x1C
313#define QIB_7220_IntClear_assertGPIOIntClear_RMASK 0x1
314#define QIB_7220_IntClear_IBSerdesTrimDoneClear_LSB 0x1B
315#define QIB_7220_IntClear_IBSerdesTrimDoneClear_RMASK 0x1
316#define QIB_7220_IntClear_JIntClear_LSB 0x1A
317#define QIB_7220_IntClear_JIntClear_RMASK 0x1
318#define QIB_7220_IntClear_Reserved1_LSB 0x11
319#define QIB_7220_IntClear_Reserved1_RMASK 0x1FF
320#define QIB_7220_IntClear_RcvAvail16IntClear_LSB 0x10
321#define QIB_7220_IntClear_RcvAvail16IntClear_RMASK 0x1
322#define QIB_7220_IntClear_RcvAvail15IntClear_LSB 0xF
323#define QIB_7220_IntClear_RcvAvail15IntClear_RMASK 0x1
324#define QIB_7220_IntClear_RcvAvail14IntClear_LSB 0xE
325#define QIB_7220_IntClear_RcvAvail14IntClear_RMASK 0x1
326#define QIB_7220_IntClear_RcvAvail13IntClear_LSB 0xD
327#define QIB_7220_IntClear_RcvAvail13IntClear_RMASK 0x1
328#define QIB_7220_IntClear_RcvAvail12IntClear_LSB 0xC
329#define QIB_7220_IntClear_RcvAvail12IntClear_RMASK 0x1
330#define QIB_7220_IntClear_RcvAvail11IntClear_LSB 0xB
331#define QIB_7220_IntClear_RcvAvail11IntClear_RMASK 0x1
332#define QIB_7220_IntClear_RcvAvail10IntClear_LSB 0xA
333#define QIB_7220_IntClear_RcvAvail10IntClear_RMASK 0x1
334#define QIB_7220_IntClear_RcvAvail9IntClear_LSB 0x9
335#define QIB_7220_IntClear_RcvAvail9IntClear_RMASK 0x1
336#define QIB_7220_IntClear_RcvAvail8IntClear_LSB 0x8
337#define QIB_7220_IntClear_RcvAvail8IntClear_RMASK 0x1
338#define QIB_7220_IntClear_RcvAvail7IntClear_LSB 0x7
339#define QIB_7220_IntClear_RcvAvail7IntClear_RMASK 0x1
340#define QIB_7220_IntClear_RcvAvail6IntClear_LSB 0x6
341#define QIB_7220_IntClear_RcvAvail6IntClear_RMASK 0x1
342#define QIB_7220_IntClear_RcvAvail5IntClear_LSB 0x5
343#define QIB_7220_IntClear_RcvAvail5IntClear_RMASK 0x1
344#define QIB_7220_IntClear_RcvAvail4IntClear_LSB 0x4
345#define QIB_7220_IntClear_RcvAvail4IntClear_RMASK 0x1
346#define QIB_7220_IntClear_RcvAvail3IntClear_LSB 0x3
347#define QIB_7220_IntClear_RcvAvail3IntClear_RMASK 0x1
348#define QIB_7220_IntClear_RcvAvail2IntClear_LSB 0x2
349#define QIB_7220_IntClear_RcvAvail2IntClear_RMASK 0x1
350#define QIB_7220_IntClear_RcvAvail1IntClear_LSB 0x1
351#define QIB_7220_IntClear_RcvAvail1IntClear_RMASK 0x1
352#define QIB_7220_IntClear_RcvAvail0IntClear_LSB 0x0
353#define QIB_7220_IntClear_RcvAvail0IntClear_RMASK 0x1
354
355#define QIB_7220_ErrMask_OFFS 0x80
356#define QIB_7220_ErrMask_Reserved_LSB 0x36
357#define QIB_7220_ErrMask_Reserved_RMASK 0x3FF
358#define QIB_7220_ErrMask_InvalidEEPCmdMask_LSB 0x35
359#define QIB_7220_ErrMask_InvalidEEPCmdMask_RMASK 0x1
360#define QIB_7220_ErrMask_SDmaDescAddrMisalignErrMask_LSB 0x34
361#define QIB_7220_ErrMask_SDmaDescAddrMisalignErrMask_RMASK 0x1
362#define QIB_7220_ErrMask_HardwareErrMask_LSB 0x33
363#define QIB_7220_ErrMask_HardwareErrMask_RMASK 0x1
364#define QIB_7220_ErrMask_ResetNegatedMask_LSB 0x32
365#define QIB_7220_ErrMask_ResetNegatedMask_RMASK 0x1
366#define QIB_7220_ErrMask_InvalidAddrErrMask_LSB 0x31
367#define QIB_7220_ErrMask_InvalidAddrErrMask_RMASK 0x1
368#define QIB_7220_ErrMask_IBStatusChangedMask_LSB 0x30
369#define QIB_7220_ErrMask_IBStatusChangedMask_RMASK 0x1
370#define QIB_7220_ErrMask_SDmaUnexpDataErrMask_LSB 0x2F
371#define QIB_7220_ErrMask_SDmaUnexpDataErrMask_RMASK 0x1
372#define QIB_7220_ErrMask_SDmaMissingDwErrMask_LSB 0x2E
373#define QIB_7220_ErrMask_SDmaMissingDwErrMask_RMASK 0x1
374#define QIB_7220_ErrMask_SDmaDwEnErrMask_LSB 0x2D
375#define QIB_7220_ErrMask_SDmaDwEnErrMask_RMASK 0x1
376#define QIB_7220_ErrMask_SDmaRpyTagErrMask_LSB 0x2C
377#define QIB_7220_ErrMask_SDmaRpyTagErrMask_RMASK 0x1
378#define QIB_7220_ErrMask_SDma1stDescErrMask_LSB 0x2B
379#define QIB_7220_ErrMask_SDma1stDescErrMask_RMASK 0x1
380#define QIB_7220_ErrMask_SDmaBaseErrMask_LSB 0x2A
381#define QIB_7220_ErrMask_SDmaBaseErrMask_RMASK 0x1
382#define QIB_7220_ErrMask_SDmaTailOutOfBoundErrMask_LSB 0x29
383#define QIB_7220_ErrMask_SDmaTailOutOfBoundErrMask_RMASK 0x1
384#define QIB_7220_ErrMask_SDmaOutOfBoundErrMask_LSB 0x28
385#define QIB_7220_ErrMask_SDmaOutOfBoundErrMask_RMASK 0x1
386#define QIB_7220_ErrMask_SDmaGenMismatchErrMask_LSB 0x27
387#define QIB_7220_ErrMask_SDmaGenMismatchErrMask_RMASK 0x1
388#define QIB_7220_ErrMask_SendBufMisuseErrMask_LSB 0x26
389#define QIB_7220_ErrMask_SendBufMisuseErrMask_RMASK 0x1
390#define QIB_7220_ErrMask_SendUnsupportedVLErrMask_LSB 0x25
391#define QIB_7220_ErrMask_SendUnsupportedVLErrMask_RMASK 0x1
392#define QIB_7220_ErrMask_SendUnexpectedPktNumErrMask_LSB 0x24
393#define QIB_7220_ErrMask_SendUnexpectedPktNumErrMask_RMASK 0x1
394#define QIB_7220_ErrMask_SendPioArmLaunchErrMask_LSB 0x23
395#define QIB_7220_ErrMask_SendPioArmLaunchErrMask_RMASK 0x1
396#define QIB_7220_ErrMask_SendDroppedDataPktErrMask_LSB 0x22
397#define QIB_7220_ErrMask_SendDroppedDataPktErrMask_RMASK 0x1
398#define QIB_7220_ErrMask_SendDroppedSmpPktErrMask_LSB 0x21
399#define QIB_7220_ErrMask_SendDroppedSmpPktErrMask_RMASK 0x1
400#define QIB_7220_ErrMask_SendPktLenErrMask_LSB 0x20
401#define QIB_7220_ErrMask_SendPktLenErrMask_RMASK 0x1
402#define QIB_7220_ErrMask_SendUnderRunErrMask_LSB 0x1F
403#define QIB_7220_ErrMask_SendUnderRunErrMask_RMASK 0x1
404#define QIB_7220_ErrMask_SendMaxPktLenErrMask_LSB 0x1E
405#define QIB_7220_ErrMask_SendMaxPktLenErrMask_RMASK 0x1
406#define QIB_7220_ErrMask_SendMinPktLenErrMask_LSB 0x1D
407#define QIB_7220_ErrMask_SendMinPktLenErrMask_RMASK 0x1
408#define QIB_7220_ErrMask_SDmaDisabledErrMask_LSB 0x1C
409#define QIB_7220_ErrMask_SDmaDisabledErrMask_RMASK 0x1
410#define QIB_7220_ErrMask_SendSpecialTriggerErrMask_LSB 0x1B
411#define QIB_7220_ErrMask_SendSpecialTriggerErrMask_RMASK 0x1
412#define QIB_7220_ErrMask_Reserved1_LSB 0x12
413#define QIB_7220_ErrMask_Reserved1_RMASK 0x1FF
414#define QIB_7220_ErrMask_RcvIBLostLinkErrMask_LSB 0x11
415#define QIB_7220_ErrMask_RcvIBLostLinkErrMask_RMASK 0x1
416#define QIB_7220_ErrMask_RcvHdrErrMask_LSB 0x10
417#define QIB_7220_ErrMask_RcvHdrErrMask_RMASK 0x1
418#define QIB_7220_ErrMask_RcvHdrLenErrMask_LSB 0xF
419#define QIB_7220_ErrMask_RcvHdrLenErrMask_RMASK 0x1
420#define QIB_7220_ErrMask_RcvBadTidErrMask_LSB 0xE
421#define QIB_7220_ErrMask_RcvBadTidErrMask_RMASK 0x1
422#define QIB_7220_ErrMask_RcvHdrFullErrMask_LSB 0xD
423#define QIB_7220_ErrMask_RcvHdrFullErrMask_RMASK 0x1
424#define QIB_7220_ErrMask_RcvEgrFullErrMask_LSB 0xC
425#define QIB_7220_ErrMask_RcvEgrFullErrMask_RMASK 0x1
426#define QIB_7220_ErrMask_RcvBadVersionErrMask_LSB 0xB
427#define QIB_7220_ErrMask_RcvBadVersionErrMask_RMASK 0x1
428#define QIB_7220_ErrMask_RcvIBFlowErrMask_LSB 0xA
429#define QIB_7220_ErrMask_RcvIBFlowErrMask_RMASK 0x1
430#define QIB_7220_ErrMask_RcvEBPErrMask_LSB 0x9
431#define QIB_7220_ErrMask_RcvEBPErrMask_RMASK 0x1
432#define QIB_7220_ErrMask_RcvUnsupportedVLErrMask_LSB 0x8
433#define QIB_7220_ErrMask_RcvUnsupportedVLErrMask_RMASK 0x1
434#define QIB_7220_ErrMask_RcvUnexpectedCharErrMask_LSB 0x7
435#define QIB_7220_ErrMask_RcvUnexpectedCharErrMask_RMASK 0x1
436#define QIB_7220_ErrMask_RcvShortPktLenErrMask_LSB 0x6
437#define QIB_7220_ErrMask_RcvShortPktLenErrMask_RMASK 0x1
438#define QIB_7220_ErrMask_RcvLongPktLenErrMask_LSB 0x5
439#define QIB_7220_ErrMask_RcvLongPktLenErrMask_RMASK 0x1
440#define QIB_7220_ErrMask_RcvMaxPktLenErrMask_LSB 0x4
441#define QIB_7220_ErrMask_RcvMaxPktLenErrMask_RMASK 0x1
442#define QIB_7220_ErrMask_RcvMinPktLenErrMask_LSB 0x3
443#define QIB_7220_ErrMask_RcvMinPktLenErrMask_RMASK 0x1
444#define QIB_7220_ErrMask_RcvICRCErrMask_LSB 0x2
445#define QIB_7220_ErrMask_RcvICRCErrMask_RMASK 0x1
446#define QIB_7220_ErrMask_RcvVCRCErrMask_LSB 0x1
447#define QIB_7220_ErrMask_RcvVCRCErrMask_RMASK 0x1
448#define QIB_7220_ErrMask_RcvFormatErrMask_LSB 0x0
449#define QIB_7220_ErrMask_RcvFormatErrMask_RMASK 0x1
450
451#define QIB_7220_ErrStatus_OFFS 0x88
452#define QIB_7220_ErrStatus_Reserved_LSB 0x36
453#define QIB_7220_ErrStatus_Reserved_RMASK 0x3FF
454#define QIB_7220_ErrStatus_InvalidEEPCmdErr_LSB 0x35
455#define QIB_7220_ErrStatus_InvalidEEPCmdErr_RMASK 0x1
456#define QIB_7220_ErrStatus_SDmaDescAddrMisalignErr_LSB 0x34
457#define QIB_7220_ErrStatus_SDmaDescAddrMisalignErr_RMASK 0x1
458#define QIB_7220_ErrStatus_HardwareErr_LSB 0x33
459#define QIB_7220_ErrStatus_HardwareErr_RMASK 0x1
460#define QIB_7220_ErrStatus_ResetNegated_LSB 0x32
461#define QIB_7220_ErrStatus_ResetNegated_RMASK 0x1
462#define QIB_7220_ErrStatus_InvalidAddrErr_LSB 0x31
463#define QIB_7220_ErrStatus_InvalidAddrErr_RMASK 0x1
464#define QIB_7220_ErrStatus_IBStatusChanged_LSB 0x30
465#define QIB_7220_ErrStatus_IBStatusChanged_RMASK 0x1
466#define QIB_7220_ErrStatus_SDmaUnexpDataErr_LSB 0x2F
467#define QIB_7220_ErrStatus_SDmaUnexpDataErr_RMASK 0x1
468#define QIB_7220_ErrStatus_SDmaMissingDwErr_LSB 0x2E
469#define QIB_7220_ErrStatus_SDmaMissingDwErr_RMASK 0x1
470#define QIB_7220_ErrStatus_SDmaDwEnErr_LSB 0x2D
471#define QIB_7220_ErrStatus_SDmaDwEnErr_RMASK 0x1
472#define QIB_7220_ErrStatus_SDmaRpyTagErr_LSB 0x2C
473#define QIB_7220_ErrStatus_SDmaRpyTagErr_RMASK 0x1
474#define QIB_7220_ErrStatus_SDma1stDescErr_LSB 0x2B
475#define QIB_7220_ErrStatus_SDma1stDescErr_RMASK 0x1
476#define QIB_7220_ErrStatus_SDmaBaseErr_LSB 0x2A
477#define QIB_7220_ErrStatus_SDmaBaseErr_RMASK 0x1
478#define QIB_7220_ErrStatus_SDmaTailOutOfBoundErr_LSB 0x29
479#define QIB_7220_ErrStatus_SDmaTailOutOfBoundErr_RMASK 0x1
480#define QIB_7220_ErrStatus_SDmaOutOfBoundErr_LSB 0x28
481#define QIB_7220_ErrStatus_SDmaOutOfBoundErr_RMASK 0x1
482#define QIB_7220_ErrStatus_SDmaGenMismatchErr_LSB 0x27
483#define QIB_7220_ErrStatus_SDmaGenMismatchErr_RMASK 0x1
484#define QIB_7220_ErrStatus_SendBufMisuseErr_LSB 0x26
485#define QIB_7220_ErrStatus_SendBufMisuseErr_RMASK 0x1
486#define QIB_7220_ErrStatus_SendUnsupportedVLErr_LSB 0x25
487#define QIB_7220_ErrStatus_SendUnsupportedVLErr_RMASK 0x1
488#define QIB_7220_ErrStatus_SendUnexpectedPktNumErr_LSB 0x24
489#define QIB_7220_ErrStatus_SendUnexpectedPktNumErr_RMASK 0x1
490#define QIB_7220_ErrStatus_SendPioArmLaunchErr_LSB 0x23
491#define QIB_7220_ErrStatus_SendPioArmLaunchErr_RMASK 0x1
492#define QIB_7220_ErrStatus_SendDroppedDataPktErr_LSB 0x22
493#define QIB_7220_ErrStatus_SendDroppedDataPktErr_RMASK 0x1
494#define QIB_7220_ErrStatus_SendDroppedSmpPktErr_LSB 0x21
495#define QIB_7220_ErrStatus_SendDroppedSmpPktErr_RMASK 0x1
496#define QIB_7220_ErrStatus_SendPktLenErr_LSB 0x20
497#define QIB_7220_ErrStatus_SendPktLenErr_RMASK 0x1
498#define QIB_7220_ErrStatus_SendUnderRunErr_LSB 0x1F
499#define QIB_7220_ErrStatus_SendUnderRunErr_RMASK 0x1
500#define QIB_7220_ErrStatus_SendMaxPktLenErr_LSB 0x1E
501#define QIB_7220_ErrStatus_SendMaxPktLenErr_RMASK 0x1
502#define QIB_7220_ErrStatus_SendMinPktLenErr_LSB 0x1D
503#define QIB_7220_ErrStatus_SendMinPktLenErr_RMASK 0x1
504#define QIB_7220_ErrStatus_SDmaDisabledErr_LSB 0x1C
505#define QIB_7220_ErrStatus_SDmaDisabledErr_RMASK 0x1
506#define QIB_7220_ErrStatus_SendSpecialTriggerErr_LSB 0x1B
507#define QIB_7220_ErrStatus_SendSpecialTriggerErr_RMASK 0x1
508#define QIB_7220_ErrStatus_Reserved1_LSB 0x12
509#define QIB_7220_ErrStatus_Reserved1_RMASK 0x1FF
510#define QIB_7220_ErrStatus_RcvIBLostLinkErr_LSB 0x11
511#define QIB_7220_ErrStatus_RcvIBLostLinkErr_RMASK 0x1
512#define QIB_7220_ErrStatus_RcvHdrErr_LSB 0x10
513#define QIB_7220_ErrStatus_RcvHdrErr_RMASK 0x1
514#define QIB_7220_ErrStatus_RcvHdrLenErr_LSB 0xF
515#define QIB_7220_ErrStatus_RcvHdrLenErr_RMASK 0x1
516#define QIB_7220_ErrStatus_RcvBadTidErr_LSB 0xE
517#define QIB_7220_ErrStatus_RcvBadTidErr_RMASK 0x1
518#define QIB_7220_ErrStatus_RcvHdrFullErr_LSB 0xD
519#define QIB_7220_ErrStatus_RcvHdrFullErr_RMASK 0x1
520#define QIB_7220_ErrStatus_RcvEgrFullErr_LSB 0xC
521#define QIB_7220_ErrStatus_RcvEgrFullErr_RMASK 0x1
522#define QIB_7220_ErrStatus_RcvBadVersionErr_LSB 0xB
523#define QIB_7220_ErrStatus_RcvBadVersionErr_RMASK 0x1
524#define QIB_7220_ErrStatus_RcvIBFlowErr_LSB 0xA
525#define QIB_7220_ErrStatus_RcvIBFlowErr_RMASK 0x1
526#define QIB_7220_ErrStatus_RcvEBPErr_LSB 0x9
527#define QIB_7220_ErrStatus_RcvEBPErr_RMASK 0x1
528#define QIB_7220_ErrStatus_RcvUnsupportedVLErr_LSB 0x8
529#define QIB_7220_ErrStatus_RcvUnsupportedVLErr_RMASK 0x1
530#define QIB_7220_ErrStatus_RcvUnexpectedCharErr_LSB 0x7
531#define QIB_7220_ErrStatus_RcvUnexpectedCharErr_RMASK 0x1
532#define QIB_7220_ErrStatus_RcvShortPktLenErr_LSB 0x6
533#define QIB_7220_ErrStatus_RcvShortPktLenErr_RMASK 0x1
534#define QIB_7220_ErrStatus_RcvLongPktLenErr_LSB 0x5
535#define QIB_7220_ErrStatus_RcvLongPktLenErr_RMASK 0x1
536#define QIB_7220_ErrStatus_RcvMaxPktLenErr_LSB 0x4
537#define QIB_7220_ErrStatus_RcvMaxPktLenErr_RMASK 0x1
538#define QIB_7220_ErrStatus_RcvMinPktLenErr_LSB 0x3
539#define QIB_7220_ErrStatus_RcvMinPktLenErr_RMASK 0x1
540#define QIB_7220_ErrStatus_RcvICRCErr_LSB 0x2
541#define QIB_7220_ErrStatus_RcvICRCErr_RMASK 0x1
542#define QIB_7220_ErrStatus_RcvVCRCErr_LSB 0x1
543#define QIB_7220_ErrStatus_RcvVCRCErr_RMASK 0x1
544#define QIB_7220_ErrStatus_RcvFormatErr_LSB 0x0
545#define QIB_7220_ErrStatus_RcvFormatErr_RMASK 0x1
546
547#define QIB_7220_ErrClear_OFFS 0x90
548#define QIB_7220_ErrClear_Reserved_LSB 0x36
549#define QIB_7220_ErrClear_Reserved_RMASK 0x3FF
550#define QIB_7220_ErrClear_InvalidEEPCmdErrClear_LSB 0x35
551#define QIB_7220_ErrClear_InvalidEEPCmdErrClear_RMASK 0x1
552#define QIB_7220_ErrClear_SDmaDescAddrMisalignErrClear_LSB 0x34
553#define QIB_7220_ErrClear_SDmaDescAddrMisalignErrClear_RMASK 0x1
554#define QIB_7220_ErrClear_HardwareErrClear_LSB 0x33
555#define QIB_7220_ErrClear_HardwareErrClear_RMASK 0x1
556#define QIB_7220_ErrClear_ResetNegatedClear_LSB 0x32
557#define QIB_7220_ErrClear_ResetNegatedClear_RMASK 0x1
558#define QIB_7220_ErrClear_InvalidAddrErrClear_LSB 0x31
559#define QIB_7220_ErrClear_InvalidAddrErrClear_RMASK 0x1
560#define QIB_7220_ErrClear_IBStatusChangedClear_LSB 0x30
561#define QIB_7220_ErrClear_IBStatusChangedClear_RMASK 0x1
562#define QIB_7220_ErrClear_SDmaUnexpDataErrClear_LSB 0x2F
563#define QIB_7220_ErrClear_SDmaUnexpDataErrClear_RMASK 0x1
564#define QIB_7220_ErrClear_SDmaMissingDwErrClear_LSB 0x2E
565#define QIB_7220_ErrClear_SDmaMissingDwErrClear_RMASK 0x1
566#define QIB_7220_ErrClear_SDmaDwEnErrClear_LSB 0x2D
567#define QIB_7220_ErrClear_SDmaDwEnErrClear_RMASK 0x1
568#define QIB_7220_ErrClear_SDmaRpyTagErrClear_LSB 0x2C
569#define QIB_7220_ErrClear_SDmaRpyTagErrClear_RMASK 0x1
570#define QIB_7220_ErrClear_SDma1stDescErrClear_LSB 0x2B
571#define QIB_7220_ErrClear_SDma1stDescErrClear_RMASK 0x1
572#define QIB_7220_ErrClear_SDmaBaseErrClear_LSB 0x2A
573#define QIB_7220_ErrClear_SDmaBaseErrClear_RMASK 0x1
574#define QIB_7220_ErrClear_SDmaTailOutOfBoundErrClear_LSB 0x29
575#define QIB_7220_ErrClear_SDmaTailOutOfBoundErrClear_RMASK 0x1
576#define QIB_7220_ErrClear_SDmaOutOfBoundErrClear_LSB 0x28
577#define QIB_7220_ErrClear_SDmaOutOfBoundErrClear_RMASK 0x1
578#define QIB_7220_ErrClear_SDmaGenMismatchErrClear_LSB 0x27
579#define QIB_7220_ErrClear_SDmaGenMismatchErrClear_RMASK 0x1
580#define QIB_7220_ErrClear_SendBufMisuseErrClear_LSB 0x26
581#define QIB_7220_ErrClear_SendBufMisuseErrClear_RMASK 0x1
582#define QIB_7220_ErrClear_SendUnsupportedVLErrClear_LSB 0x25
583#define QIB_7220_ErrClear_SendUnsupportedVLErrClear_RMASK 0x1
584#define QIB_7220_ErrClear_SendUnexpectedPktNumErrClear_LSB 0x24
585#define QIB_7220_ErrClear_SendUnexpectedPktNumErrClear_RMASK 0x1
586#define QIB_7220_ErrClear_SendPioArmLaunchErrClear_LSB 0x23
587#define QIB_7220_ErrClear_SendPioArmLaunchErrClear_RMASK 0x1
588#define QIB_7220_ErrClear_SendDroppedDataPktErrClear_LSB 0x22
589#define QIB_7220_ErrClear_SendDroppedDataPktErrClear_RMASK 0x1
590#define QIB_7220_ErrClear_SendDroppedSmpPktErrClear_LSB 0x21
591#define QIB_7220_ErrClear_SendDroppedSmpPktErrClear_RMASK 0x1
592#define QIB_7220_ErrClear_SendPktLenErrClear_LSB 0x20
593#define QIB_7220_ErrClear_SendPktLenErrClear_RMASK 0x1
594#define QIB_7220_ErrClear_SendUnderRunErrClear_LSB 0x1F
595#define QIB_7220_ErrClear_SendUnderRunErrClear_RMASK 0x1
596#define QIB_7220_ErrClear_SendMaxPktLenErrClear_LSB 0x1E
597#define QIB_7220_ErrClear_SendMaxPktLenErrClear_RMASK 0x1
598#define QIB_7220_ErrClear_SendMinPktLenErrClear_LSB 0x1D
599#define QIB_7220_ErrClear_SendMinPktLenErrClear_RMASK 0x1
600#define QIB_7220_ErrClear_SDmaDisabledErrClear_LSB 0x1C
601#define QIB_7220_ErrClear_SDmaDisabledErrClear_RMASK 0x1
602#define QIB_7220_ErrClear_SendSpecialTriggerErrClear_LSB 0x1B
603#define QIB_7220_ErrClear_SendSpecialTriggerErrClear_RMASK 0x1
604#define QIB_7220_ErrClear_Reserved1_LSB 0x12
605#define QIB_7220_ErrClear_Reserved1_RMASK 0x1FF
606#define QIB_7220_ErrClear_RcvIBLostLinkErrClear_LSB 0x11
607#define QIB_7220_ErrClear_RcvIBLostLinkErrClear_RMASK 0x1
608#define QIB_7220_ErrClear_RcvHdrErrClear_LSB 0x10
609#define QIB_7220_ErrClear_RcvHdrErrClear_RMASK 0x1
610#define QIB_7220_ErrClear_RcvHdrLenErrClear_LSB 0xF
611#define QIB_7220_ErrClear_RcvHdrLenErrClear_RMASK 0x1
612#define QIB_7220_ErrClear_RcvBadTidErrClear_LSB 0xE
613#define QIB_7220_ErrClear_RcvBadTidErrClear_RMASK 0x1
614#define QIB_7220_ErrClear_RcvHdrFullErrClear_LSB 0xD
615#define QIB_7220_ErrClear_RcvHdrFullErrClear_RMASK 0x1
616#define QIB_7220_ErrClear_RcvEgrFullErrClear_LSB 0xC
617#define QIB_7220_ErrClear_RcvEgrFullErrClear_RMASK 0x1
618#define QIB_7220_ErrClear_RcvBadVersionErrClear_LSB 0xB
619#define QIB_7220_ErrClear_RcvBadVersionErrClear_RMASK 0x1
620#define QIB_7220_ErrClear_RcvIBFlowErrClear_LSB 0xA
621#define QIB_7220_ErrClear_RcvIBFlowErrClear_RMASK 0x1
622#define QIB_7220_ErrClear_RcvEBPErrClear_LSB 0x9
623#define QIB_7220_ErrClear_RcvEBPErrClear_RMASK 0x1
624#define QIB_7220_ErrClear_RcvUnsupportedVLErrClear_LSB 0x8
625#define QIB_7220_ErrClear_RcvUnsupportedVLErrClear_RMASK 0x1
626#define QIB_7220_ErrClear_RcvUnexpectedCharErrClear_LSB 0x7
627#define QIB_7220_ErrClear_RcvUnexpectedCharErrClear_RMASK 0x1
628#define QIB_7220_ErrClear_RcvShortPktLenErrClear_LSB 0x6
629#define QIB_7220_ErrClear_RcvShortPktLenErrClear_RMASK 0x1
630#define QIB_7220_ErrClear_RcvLongPktLenErrClear_LSB 0x5
631#define QIB_7220_ErrClear_RcvLongPktLenErrClear_RMASK 0x1
632#define QIB_7220_ErrClear_RcvMaxPktLenErrClear_LSB 0x4
633#define QIB_7220_ErrClear_RcvMaxPktLenErrClear_RMASK 0x1
634#define QIB_7220_ErrClear_RcvMinPktLenErrClear_LSB 0x3
635#define QIB_7220_ErrClear_RcvMinPktLenErrClear_RMASK 0x1
636#define QIB_7220_ErrClear_RcvICRCErrClear_LSB 0x2
637#define QIB_7220_ErrClear_RcvICRCErrClear_RMASK 0x1
638#define QIB_7220_ErrClear_RcvVCRCErrClear_LSB 0x1
639#define QIB_7220_ErrClear_RcvVCRCErrClear_RMASK 0x1
640#define QIB_7220_ErrClear_RcvFormatErrClear_LSB 0x0
641#define QIB_7220_ErrClear_RcvFormatErrClear_RMASK 0x1
642
643#define QIB_7220_HwErrMask_OFFS 0x98
644#define QIB_7220_HwErrMask_IBCBusFromSPCParityErrMask_LSB 0x3F
645#define QIB_7220_HwErrMask_IBCBusFromSPCParityErrMask_RMASK 0x1
646#define QIB_7220_HwErrMask_IBCBusToSPCParityErrMask_LSB 0x3E
647#define QIB_7220_HwErrMask_IBCBusToSPCParityErrMask_RMASK 0x1
648#define QIB_7220_HwErrMask_Clk_uC_PLLNotLockedMask_LSB 0x3D
649#define QIB_7220_HwErrMask_Clk_uC_PLLNotLockedMask_RMASK 0x1
650#define QIB_7220_HwErrMask_IBSerdesPClkNotDetectMask_LSB 0x3C
651#define QIB_7220_HwErrMask_IBSerdesPClkNotDetectMask_RMASK 0x1
652#define QIB_7220_HwErrMask_PCIESerdesQ3PClkNotDetectMask_LSB 0x3B
653#define QIB_7220_HwErrMask_PCIESerdesQ3PClkNotDetectMask_RMASK 0x1
654#define QIB_7220_HwErrMask_PCIESerdesQ2PClkNotDetectMask_LSB 0x3A
655#define QIB_7220_HwErrMask_PCIESerdesQ2PClkNotDetectMask_RMASK 0x1
656#define QIB_7220_HwErrMask_PCIESerdesQ1PClkNotDetectMask_LSB 0x39
657#define QIB_7220_HwErrMask_PCIESerdesQ1PClkNotDetectMask_RMASK 0x1
658#define QIB_7220_HwErrMask_PCIESerdesQ0PClkNotDetectMask_LSB 0x38
659#define QIB_7220_HwErrMask_PCIESerdesQ0PClkNotDetectMask_RMASK 0x1
660#define QIB_7220_HwErrMask_Reserved_LSB 0x37
661#define QIB_7220_HwErrMask_Reserved_RMASK 0x1
662#define QIB_7220_HwErrMask_PowerOnBISTFailedMask_LSB 0x36
663#define QIB_7220_HwErrMask_PowerOnBISTFailedMask_RMASK 0x1
664#define QIB_7220_HwErrMask_Reserved1_LSB 0x33
665#define QIB_7220_HwErrMask_Reserved1_RMASK 0x7
666#define QIB_7220_HwErrMask_RXEMemParityErrMask_LSB 0x2C
667#define QIB_7220_HwErrMask_RXEMemParityErrMask_RMASK 0x7F
668#define QIB_7220_HwErrMask_TXEMemParityErrMask_LSB 0x28
669#define QIB_7220_HwErrMask_TXEMemParityErrMask_RMASK 0xF
670#define QIB_7220_HwErrMask_DDSRXEQMemoryParityErrMask_LSB 0x27
671#define QIB_7220_HwErrMask_DDSRXEQMemoryParityErrMask_RMASK 0x1
672#define QIB_7220_HwErrMask_IB_uC_MemoryParityErrMask_LSB 0x26
673#define QIB_7220_HwErrMask_IB_uC_MemoryParityErrMask_RMASK 0x1
674#define QIB_7220_HwErrMask_PCIEOct1_uC_MemoryParityErrMask_LSB 0x25
675#define QIB_7220_HwErrMask_PCIEOct1_uC_MemoryParityErrMask_RMASK 0x1
676#define QIB_7220_HwErrMask_PCIEOct0_uC_MemoryParityErrMask_LSB 0x24
677#define QIB_7220_HwErrMask_PCIEOct0_uC_MemoryParityErrMask_RMASK 0x1
678#define QIB_7220_HwErrMask_Reserved2_LSB 0x22
679#define QIB_7220_HwErrMask_Reserved2_RMASK 0x3
680#define QIB_7220_HwErrMask_PCIeBusParityErrMask_LSB 0x1F
681#define QIB_7220_HwErrMask_PCIeBusParityErrMask_RMASK 0x7
682#define QIB_7220_HwErrMask_PcieCplTimeoutMask_LSB 0x1E
683#define QIB_7220_HwErrMask_PcieCplTimeoutMask_RMASK 0x1
684#define QIB_7220_HwErrMask_PoisonedTLPMask_LSB 0x1D
685#define QIB_7220_HwErrMask_PoisonedTLPMask_RMASK 0x1
686#define QIB_7220_HwErrMask_SDmaMemReadErrMask_LSB 0x1C
687#define QIB_7220_HwErrMask_SDmaMemReadErrMask_RMASK 0x1
688#define QIB_7220_HwErrMask_Reserved3_LSB 0x8
689#define QIB_7220_HwErrMask_Reserved3_RMASK 0xFFFFF
690#define QIB_7220_HwErrMask_PCIeMemParityErrMask_LSB 0x0
691#define QIB_7220_HwErrMask_PCIeMemParityErrMask_RMASK 0xFF
692
693#define QIB_7220_HwErrStatus_OFFS 0xA0
694#define QIB_7220_HwErrStatus_IBCBusFromSPCParityErr_LSB 0x3F
695#define QIB_7220_HwErrStatus_IBCBusFromSPCParityErr_RMASK 0x1
696#define QIB_7220_HwErrStatus_IBCBusToSPCParityErr_LSB 0x3E
697#define QIB_7220_HwErrStatus_IBCBusToSPCParityErr_RMASK 0x1
698#define QIB_7220_HwErrStatus_Clk_uC_PLLNotLocked_LSB 0x3D
699#define QIB_7220_HwErrStatus_Clk_uC_PLLNotLocked_RMASK 0x1
700#define QIB_7220_HwErrStatus_IBSerdesPClkNotDetect_LSB 0x3C
701#define QIB_7220_HwErrStatus_IBSerdesPClkNotDetect_RMASK 0x1
702#define QIB_7220_HwErrStatus_PCIESerdesQ3PClkNotDetect_LSB 0x3B
703#define QIB_7220_HwErrStatus_PCIESerdesQ3PClkNotDetect_RMASK 0x1
704#define QIB_7220_HwErrStatus_PCIESerdesQ2PClkNotDetect_LSB 0x3A
705#define QIB_7220_HwErrStatus_PCIESerdesQ2PClkNotDetect_RMASK 0x1
706#define QIB_7220_HwErrStatus_PCIESerdesQ1PClkNotDetect_LSB 0x39
707#define QIB_7220_HwErrStatus_PCIESerdesQ1PClkNotDetect_RMASK 0x1
708#define QIB_7220_HwErrStatus_PCIESerdesQ0PClkNotDetect_LSB 0x38
709#define QIB_7220_HwErrStatus_PCIESerdesQ0PClkNotDetect_RMASK 0x1
710#define QIB_7220_HwErrStatus_Reserved_LSB 0x37
711#define QIB_7220_HwErrStatus_Reserved_RMASK 0x1
712#define QIB_7220_HwErrStatus_PowerOnBISTFailed_LSB 0x36
713#define QIB_7220_HwErrStatus_PowerOnBISTFailed_RMASK 0x1
714#define QIB_7220_HwErrStatus_Reserved1_LSB 0x33
715#define QIB_7220_HwErrStatus_Reserved1_RMASK 0x7
716#define QIB_7220_HwErrStatus_RXEMemParity_LSB 0x2C
717#define QIB_7220_HwErrStatus_RXEMemParity_RMASK 0x7F
718#define QIB_7220_HwErrStatus_TXEMemParity_LSB 0x28
719#define QIB_7220_HwErrStatus_TXEMemParity_RMASK 0xF
720#define QIB_7220_HwErrStatus_DDSRXEQMemoryParityErr_LSB 0x27
721#define QIB_7220_HwErrStatus_DDSRXEQMemoryParityErr_RMASK 0x1
722#define QIB_7220_HwErrStatus_IB_uC_MemoryParityErr_LSB 0x26
723#define QIB_7220_HwErrStatus_IB_uC_MemoryParityErr_RMASK 0x1
724#define QIB_7220_HwErrStatus_PCIE_uC_Oct1MemoryParityErr_LSB 0x25
725#define QIB_7220_HwErrStatus_PCIE_uC_Oct1MemoryParityErr_RMASK 0x1
726#define QIB_7220_HwErrStatus_PCIE_uC_Oct0MemoryParityErr_LSB 0x24
727#define QIB_7220_HwErrStatus_PCIE_uC_Oct0MemoryParityErr_RMASK 0x1
728#define QIB_7220_HwErrStatus_Reserved2_LSB 0x22
729#define QIB_7220_HwErrStatus_Reserved2_RMASK 0x3
730#define QIB_7220_HwErrStatus_PCIeBusParity_LSB 0x1F
731#define QIB_7220_HwErrStatus_PCIeBusParity_RMASK 0x7
732#define QIB_7220_HwErrStatus_PcieCplTimeout_LSB 0x1E
733#define QIB_7220_HwErrStatus_PcieCplTimeout_RMASK 0x1
734#define QIB_7220_HwErrStatus_PoisenedTLP_LSB 0x1D
735#define QIB_7220_HwErrStatus_PoisenedTLP_RMASK 0x1
736#define QIB_7220_HwErrStatus_SDmaMemReadErr_LSB 0x1C
737#define QIB_7220_HwErrStatus_SDmaMemReadErr_RMASK 0x1
738#define QIB_7220_HwErrStatus_Reserved3_LSB 0x8
739#define QIB_7220_HwErrStatus_Reserved3_RMASK 0xFFFFF
740#define QIB_7220_HwErrStatus_PCIeMemParity_LSB 0x0
741#define QIB_7220_HwErrStatus_PCIeMemParity_RMASK 0xFF
742
743#define QIB_7220_HwErrClear_OFFS 0xA8
744#define QIB_7220_HwErrClear_IBCBusFromSPCParityErrClear_LSB 0x3F
745#define QIB_7220_HwErrClear_IBCBusFromSPCParityErrClear_RMASK 0x1
746#define QIB_7220_HwErrClear_IBCBusToSPCparityErrClear_LSB 0x3E
747#define QIB_7220_HwErrClear_IBCBusToSPCparityErrClear_RMASK 0x1
748#define QIB_7220_HwErrClear_Clk_uC_PLLNotLockedClear_LSB 0x3D
749#define QIB_7220_HwErrClear_Clk_uC_PLLNotLockedClear_RMASK 0x1
750#define QIB_7220_HwErrClear_IBSerdesPClkNotDetectClear_LSB 0x3C
751#define QIB_7220_HwErrClear_IBSerdesPClkNotDetectClear_RMASK 0x1
752#define QIB_7220_HwErrClear_PCIESerdesQ3PClkNotDetectClear_LSB 0x3B
753#define QIB_7220_HwErrClear_PCIESerdesQ3PClkNotDetectClear_RMASK 0x1
754#define QIB_7220_HwErrClear_PCIESerdesQ2PClkNotDetectClear_LSB 0x3A
755#define QIB_7220_HwErrClear_PCIESerdesQ2PClkNotDetectClear_RMASK 0x1
756#define QIB_7220_HwErrClear_PCIESerdesQ1PClkNotDetectClear_LSB 0x39
757#define QIB_7220_HwErrClear_PCIESerdesQ1PClkNotDetectClear_RMASK 0x1
758#define QIB_7220_HwErrClear_PCIESerdesQ0PClkNotDetectClear_LSB 0x38
759#define QIB_7220_HwErrClear_PCIESerdesQ0PClkNotDetectClear_RMASK 0x1
760#define QIB_7220_HwErrClear_Reserved_LSB 0x37
761#define QIB_7220_HwErrClear_Reserved_RMASK 0x1
762#define QIB_7220_HwErrClear_PowerOnBISTFailedClear_LSB 0x36
763#define QIB_7220_HwErrClear_PowerOnBISTFailedClear_RMASK 0x1
764#define QIB_7220_HwErrClear_Reserved1_LSB 0x33
765#define QIB_7220_HwErrClear_Reserved1_RMASK 0x7
766#define QIB_7220_HwErrClear_RXEMemParityClear_LSB 0x2C
767#define QIB_7220_HwErrClear_RXEMemParityClear_RMASK 0x7F
768#define QIB_7220_HwErrClear_TXEMemParityClear_LSB 0x28
769#define QIB_7220_HwErrClear_TXEMemParityClear_RMASK 0xF
770#define QIB_7220_HwErrClear_DDSRXEQMemoryParityErrClear_LSB 0x27
771#define QIB_7220_HwErrClear_DDSRXEQMemoryParityErrClear_RMASK 0x1
772#define QIB_7220_HwErrClear_IB_uC_MemoryParityErrClear_LSB 0x26
773#define QIB_7220_HwErrClear_IB_uC_MemoryParityErrClear_RMASK 0x1
774#define QIB_7220_HwErrClear_PCIE_uC_Oct1MemoryParityErrClear_LSB 0x25
775#define QIB_7220_HwErrClear_PCIE_uC_Oct1MemoryParityErrClear_RMASK 0x1
776#define QIB_7220_HwErrClear_PCIE_uC_Oct0MemoryParityErrClear_LSB 0x24
777#define QIB_7220_HwErrClear_PCIE_uC_Oct0MemoryParityErrClear_RMASK 0x1
778#define QIB_7220_HwErrClear_Reserved2_LSB 0x22
779#define QIB_7220_HwErrClear_Reserved2_RMASK 0x3
780#define QIB_7220_HwErrClear_PCIeBusParityClr_LSB 0x1F
781#define QIB_7220_HwErrClear_PCIeBusParityClr_RMASK 0x7
782#define QIB_7220_HwErrClear_PcieCplTimeoutClear_LSB 0x1E
783#define QIB_7220_HwErrClear_PcieCplTimeoutClear_RMASK 0x1
784#define QIB_7220_HwErrClear_PoisonedTLPClear_LSB 0x1D
785#define QIB_7220_HwErrClear_PoisonedTLPClear_RMASK 0x1
786#define QIB_7220_HwErrClear_SDmaMemReadErrClear_LSB 0x1C
787#define QIB_7220_HwErrClear_SDmaMemReadErrClear_RMASK 0x1
788#define QIB_7220_HwErrClear_Reserved3_LSB 0x8
789#define QIB_7220_HwErrClear_Reserved3_RMASK 0xFFFFF
790#define QIB_7220_HwErrClear_PCIeMemParityClr_LSB 0x0
791#define QIB_7220_HwErrClear_PCIeMemParityClr_RMASK 0xFF
792
793#define QIB_7220_HwDiagCtrl_OFFS 0xB0
794#define QIB_7220_HwDiagCtrl_ForceIBCBusFromSPCParityErr_LSB 0x3F
795#define QIB_7220_HwDiagCtrl_ForceIBCBusFromSPCParityErr_RMASK 0x1
796#define QIB_7220_HwDiagCtrl_ForceIBCBusToSPCParityErr_LSB 0x3E
797#define QIB_7220_HwDiagCtrl_ForceIBCBusToSPCParityErr_RMASK 0x1
798#define QIB_7220_HwDiagCtrl_CounterWrEnable_LSB 0x3D
799#define QIB_7220_HwDiagCtrl_CounterWrEnable_RMASK 0x1
800#define QIB_7220_HwDiagCtrl_CounterDisable_LSB 0x3C
801#define QIB_7220_HwDiagCtrl_CounterDisable_RMASK 0x1
802#define QIB_7220_HwDiagCtrl_Reserved_LSB 0x33
803#define QIB_7220_HwDiagCtrl_Reserved_RMASK 0x1FF
804#define QIB_7220_HwDiagCtrl_ForceRxMemParityErr_LSB 0x2C
805#define QIB_7220_HwDiagCtrl_ForceRxMemParityErr_RMASK 0x7F
806#define QIB_7220_HwDiagCtrl_ForceTxMemparityErr_LSB 0x28
807#define QIB_7220_HwDiagCtrl_ForceTxMemparityErr_RMASK 0xF
808#define QIB_7220_HwDiagCtrl_ForceDDSRXEQMemoryParityErr_LSB 0x27
809#define QIB_7220_HwDiagCtrl_ForceDDSRXEQMemoryParityErr_RMASK 0x1
810#define QIB_7220_HwDiagCtrl_ForceIB_uC_MemoryParityErr_LSB 0x26
811#define QIB_7220_HwDiagCtrl_ForceIB_uC_MemoryParityErr_RMASK 0x1
812#define QIB_7220_HwDiagCtrl_ForcePCIE_uC_Oct1MemoryParityErr_LSB 0x25
813#define QIB_7220_HwDiagCtrl_ForcePCIE_uC_Oct1MemoryParityErr_RMASK 0x1
814#define QIB_7220_HwDiagCtrl_ForcePCIE_uC_Oct0MemoryParityErr_LSB 0x24
815#define QIB_7220_HwDiagCtrl_ForcePCIE_uC_Oct0MemoryParityErr_RMASK 0x1
816#define QIB_7220_HwDiagCtrl_Reserved1_LSB 0x23
817#define QIB_7220_HwDiagCtrl_Reserved1_RMASK 0x1
818#define QIB_7220_HwDiagCtrl_forcePCIeBusParity_LSB 0x1F
819#define QIB_7220_HwDiagCtrl_forcePCIeBusParity_RMASK 0xF
820#define QIB_7220_HwDiagCtrl_Reserved2_LSB 0x8
821#define QIB_7220_HwDiagCtrl_Reserved2_RMASK 0x7FFFFF
822#define QIB_7220_HwDiagCtrl_forcePCIeMemParity_LSB 0x0
823#define QIB_7220_HwDiagCtrl_forcePCIeMemParity_RMASK 0xFF
824
825#define QIB_7220_REG_0000B8_OFFS 0xB8
826
827#define QIB_7220_IBCStatus_OFFS 0xC0
828#define QIB_7220_IBCStatus_TxCreditOk_LSB 0x1F
829#define QIB_7220_IBCStatus_TxCreditOk_RMASK 0x1
830#define QIB_7220_IBCStatus_TxReady_LSB 0x1E
831#define QIB_7220_IBCStatus_TxReady_RMASK 0x1
832#define QIB_7220_IBCStatus_Reserved_LSB 0xE
833#define QIB_7220_IBCStatus_Reserved_RMASK 0xFFFF
834#define QIB_7220_IBCStatus_IBTxLaneReversed_LSB 0xD
835#define QIB_7220_IBCStatus_IBTxLaneReversed_RMASK 0x1
836#define QIB_7220_IBCStatus_IBRxLaneReversed_LSB 0xC
837#define QIB_7220_IBCStatus_IBRxLaneReversed_RMASK 0x1
838#define QIB_7220_IBCStatus_IB_SERDES_TRIM_DONE_LSB 0xB
839#define QIB_7220_IBCStatus_IB_SERDES_TRIM_DONE_RMASK 0x1
840#define QIB_7220_IBCStatus_DDS_RXEQ_FAIL_LSB 0xA
841#define QIB_7220_IBCStatus_DDS_RXEQ_FAIL_RMASK 0x1
842#define QIB_7220_IBCStatus_LinkWidthActive_LSB 0x9
843#define QIB_7220_IBCStatus_LinkWidthActive_RMASK 0x1
844#define QIB_7220_IBCStatus_LinkSpeedActive_LSB 0x8
845#define QIB_7220_IBCStatus_LinkSpeedActive_RMASK 0x1
846#define QIB_7220_IBCStatus_LinkState_LSB 0x5
847#define QIB_7220_IBCStatus_LinkState_RMASK 0x7
848#define QIB_7220_IBCStatus_LinkTrainingState_LSB 0x0
849#define QIB_7220_IBCStatus_LinkTrainingState_RMASK 0x1F
850
851#define QIB_7220_IBCCtrl_OFFS 0xC8
852#define QIB_7220_IBCCtrl_Loopback_LSB 0x3F
853#define QIB_7220_IBCCtrl_Loopback_RMASK 0x1
854#define QIB_7220_IBCCtrl_LinkDownDefaultState_LSB 0x3E
855#define QIB_7220_IBCCtrl_LinkDownDefaultState_RMASK 0x1
856#define QIB_7220_IBCCtrl_Reserved_LSB 0x2B
857#define QIB_7220_IBCCtrl_Reserved_RMASK 0x7FFFF
858#define QIB_7220_IBCCtrl_CreditScale_LSB 0x28
859#define QIB_7220_IBCCtrl_CreditScale_RMASK 0x7
860#define QIB_7220_IBCCtrl_OverrunThreshold_LSB 0x24
861#define QIB_7220_IBCCtrl_OverrunThreshold_RMASK 0xF
862#define QIB_7220_IBCCtrl_PhyerrThreshold_LSB 0x20
863#define QIB_7220_IBCCtrl_PhyerrThreshold_RMASK 0xF
864#define QIB_7220_IBCCtrl_MaxPktLen_LSB 0x15
865#define QIB_7220_IBCCtrl_MaxPktLen_RMASK 0x7FF
866#define QIB_7220_IBCCtrl_LinkCmd_LSB 0x13
867#define QIB_7220_IBCCtrl_LinkCmd_RMASK 0x3
868#define QIB_7220_IBCCtrl_LinkInitCmd_LSB 0x10
869#define QIB_7220_IBCCtrl_LinkInitCmd_RMASK 0x7
870#define QIB_7220_IBCCtrl_FlowCtrlWaterMark_LSB 0x8
871#define QIB_7220_IBCCtrl_FlowCtrlWaterMark_RMASK 0xFF
872#define QIB_7220_IBCCtrl_FlowCtrlPeriod_LSB 0x0
873#define QIB_7220_IBCCtrl_FlowCtrlPeriod_RMASK 0xFF
874
875#define QIB_7220_EXTStatus_OFFS 0xD0
876#define QIB_7220_EXTStatus_GPIOIn_LSB 0x30
877#define QIB_7220_EXTStatus_GPIOIn_RMASK 0xFFFF
878#define QIB_7220_EXTStatus_Reserved_LSB 0x20
879#define QIB_7220_EXTStatus_Reserved_RMASK 0xFFFF
880#define QIB_7220_EXTStatus_Reserved1_LSB 0x10
881#define QIB_7220_EXTStatus_Reserved1_RMASK 0xFFFF
882#define QIB_7220_EXTStatus_MemBISTDisabled_LSB 0xF
883#define QIB_7220_EXTStatus_MemBISTDisabled_RMASK 0x1
884#define QIB_7220_EXTStatus_MemBISTEndTest_LSB 0xE
885#define QIB_7220_EXTStatus_MemBISTEndTest_RMASK 0x1
886#define QIB_7220_EXTStatus_Reserved2_LSB 0x0
887#define QIB_7220_EXTStatus_Reserved2_RMASK 0x3FFF
888
889#define QIB_7220_EXTCtrl_OFFS 0xD8
890#define QIB_7220_EXTCtrl_GPIOOe_LSB 0x30
891#define QIB_7220_EXTCtrl_GPIOOe_RMASK 0xFFFF
892#define QIB_7220_EXTCtrl_GPIOInvert_LSB 0x20
893#define QIB_7220_EXTCtrl_GPIOInvert_RMASK 0xFFFF
894#define QIB_7220_EXTCtrl_Reserved_LSB 0x4
895#define QIB_7220_EXTCtrl_Reserved_RMASK 0xFFFFFFF
896#define QIB_7220_EXTCtrl_LEDPriPortGreenOn_LSB 0x3
897#define QIB_7220_EXTCtrl_LEDPriPortGreenOn_RMASK 0x1
898#define QIB_7220_EXTCtrl_LEDPriPortYellowOn_LSB 0x2
899#define QIB_7220_EXTCtrl_LEDPriPortYellowOn_RMASK 0x1
900#define QIB_7220_EXTCtrl_LEDGblOkGreenOn_LSB 0x1
901#define QIB_7220_EXTCtrl_LEDGblOkGreenOn_RMASK 0x1
902#define QIB_7220_EXTCtrl_LEDGblErrRedOff_LSB 0x0
903#define QIB_7220_EXTCtrl_LEDGblErrRedOff_RMASK 0x1
904
905#define QIB_7220_GPIOOut_OFFS 0xE0
906
907#define QIB_7220_GPIOMask_OFFS 0xE8
908
909#define QIB_7220_GPIOStatus_OFFS 0xF0
910
911#define QIB_7220_GPIOClear_OFFS 0xF8
912
913#define QIB_7220_RcvCtrl_OFFS 0x100
914#define QIB_7220_RcvCtrl_Reserved_LSB 0x27
915#define QIB_7220_RcvCtrl_Reserved_RMASK 0x1FFFFFF
916#define QIB_7220_RcvCtrl_RcvQPMapEnable_LSB 0x26
917#define QIB_7220_RcvCtrl_RcvQPMapEnable_RMASK 0x1
918#define QIB_7220_RcvCtrl_PortCfg_LSB 0x24
919#define QIB_7220_RcvCtrl_PortCfg_RMASK 0x3
920#define QIB_7220_RcvCtrl_TailUpd_LSB 0x23
921#define QIB_7220_RcvCtrl_TailUpd_RMASK 0x1
922#define QIB_7220_RcvCtrl_RcvPartitionKeyDisable_LSB 0x22
923#define QIB_7220_RcvCtrl_RcvPartitionKeyDisable_RMASK 0x1
924#define QIB_7220_RcvCtrl_IntrAvail_LSB 0x11
925#define QIB_7220_RcvCtrl_IntrAvail_RMASK 0x1FFFF
926#define QIB_7220_RcvCtrl_PortEnable_LSB 0x0
927#define QIB_7220_RcvCtrl_PortEnable_RMASK 0x1FFFF
928
929#define QIB_7220_RcvBTHQP_OFFS 0x108
930#define QIB_7220_RcvBTHQP_Reserved_LSB 0x18
931#define QIB_7220_RcvBTHQP_Reserved_RMASK 0xFF
932#define QIB_7220_RcvBTHQP_RcvBTHQP_LSB 0x0
933#define QIB_7220_RcvBTHQP_RcvBTHQP_RMASK 0xFFFFFF
934
935#define QIB_7220_RcvHdrSize_OFFS 0x110
936
937#define QIB_7220_RcvHdrCnt_OFFS 0x118
938
939#define QIB_7220_RcvHdrEntSize_OFFS 0x120
940
941#define QIB_7220_RcvTIDBase_OFFS 0x128
942
943#define QIB_7220_RcvTIDCnt_OFFS 0x130
944
945#define QIB_7220_RcvEgrBase_OFFS 0x138
946
947#define QIB_7220_RcvEgrCnt_OFFS 0x140
948
949#define QIB_7220_RcvBufBase_OFFS 0x148
950
951#define QIB_7220_RcvBufSize_OFFS 0x150
952
953#define QIB_7220_RxIntMemBase_OFFS 0x158
954
955#define QIB_7220_RxIntMemSize_OFFS 0x160
956
957#define QIB_7220_RcvPartitionKey_OFFS 0x168
958
959#define QIB_7220_RcvQPMulticastPort_OFFS 0x170
960#define QIB_7220_RcvQPMulticastPort_Reserved_LSB 0x5
961#define QIB_7220_RcvQPMulticastPort_Reserved_RMASK 0x7FFFFFFFFFFFFFF
962#define QIB_7220_RcvQPMulticastPort_RcvQpMcPort_LSB 0x0
963#define QIB_7220_RcvQPMulticastPort_RcvQpMcPort_RMASK 0x1F
964
965#define QIB_7220_RcvPktLEDCnt_OFFS 0x178
966#define QIB_7220_RcvPktLEDCnt_ONperiod_LSB 0x20
967#define QIB_7220_RcvPktLEDCnt_ONperiod_RMASK 0xFFFFFFFF
968#define QIB_7220_RcvPktLEDCnt_OFFperiod_LSB 0x0
969#define QIB_7220_RcvPktLEDCnt_OFFperiod_RMASK 0xFFFFFFFF
970
971#define QIB_7220_IBCDDRCtrl_OFFS 0x180
972#define QIB_7220_IBCDDRCtrl_IB_DLID_MASK_LSB 0x30
973#define QIB_7220_IBCDDRCtrl_IB_DLID_MASK_RMASK 0xFFFF
974#define QIB_7220_IBCDDRCtrl_IB_DLID_LSB 0x20
975#define QIB_7220_IBCDDRCtrl_IB_DLID_RMASK 0xFFFF
976#define QIB_7220_IBCDDRCtrl_Reserved_LSB 0x1B
977#define QIB_7220_IBCDDRCtrl_Reserved_RMASK 0x1F
978#define QIB_7220_IBCDDRCtrl_HRTBT_REQ_LSB 0x1A
979#define QIB_7220_IBCDDRCtrl_HRTBT_REQ_RMASK 0x1
980#define QIB_7220_IBCDDRCtrl_HRTBT_PORT_LSB 0x12
981#define QIB_7220_IBCDDRCtrl_HRTBT_PORT_RMASK 0xFF
982#define QIB_7220_IBCDDRCtrl_HRTBT_AUTO_LSB 0x11
983#define QIB_7220_IBCDDRCtrl_HRTBT_AUTO_RMASK 0x1
984#define QIB_7220_IBCDDRCtrl_HRTBT_ENB_LSB 0x10
985#define QIB_7220_IBCDDRCtrl_HRTBT_ENB_RMASK 0x1
986#define QIB_7220_IBCDDRCtrl_SD_DDS_LSB 0xC
987#define QIB_7220_IBCDDRCtrl_SD_DDS_RMASK 0xF
988#define QIB_7220_IBCDDRCtrl_SD_DDSV_LSB 0xB
989#define QIB_7220_IBCDDRCtrl_SD_DDSV_RMASK 0x1
990#define QIB_7220_IBCDDRCtrl_SD_ADD_ENB_LSB 0xA
991#define QIB_7220_IBCDDRCtrl_SD_ADD_ENB_RMASK 0x1
992#define QIB_7220_IBCDDRCtrl_SD_RX_EQUAL_ENABLE_LSB 0x9
993#define QIB_7220_IBCDDRCtrl_SD_RX_EQUAL_ENABLE_RMASK 0x1
994#define QIB_7220_IBCDDRCtrl_IB_LANE_REV_SUPPORTED_LSB 0x8
995#define QIB_7220_IBCDDRCtrl_IB_LANE_REV_SUPPORTED_RMASK 0x1
996#define QIB_7220_IBCDDRCtrl_IB_POLARITY_REV_SUPP_LSB 0x7
997#define QIB_7220_IBCDDRCtrl_IB_POLARITY_REV_SUPP_RMASK 0x1
998#define QIB_7220_IBCDDRCtrl_IB_NUM_CHANNELS_LSB 0x5
999#define QIB_7220_IBCDDRCtrl_IB_NUM_CHANNELS_RMASK 0x3
1000#define QIB_7220_IBCDDRCtrl_SD_SPEED_QDR_LSB 0x4
1001#define QIB_7220_IBCDDRCtrl_SD_SPEED_QDR_RMASK 0x1
1002#define QIB_7220_IBCDDRCtrl_SD_SPEED_DDR_LSB 0x3
1003#define QIB_7220_IBCDDRCtrl_SD_SPEED_DDR_RMASK 0x1
1004#define QIB_7220_IBCDDRCtrl_SD_SPEED_SDR_LSB 0x2
1005#define QIB_7220_IBCDDRCtrl_SD_SPEED_SDR_RMASK 0x1
1006#define QIB_7220_IBCDDRCtrl_SD_SPEED_LSB 0x1
1007#define QIB_7220_IBCDDRCtrl_SD_SPEED_RMASK 0x1
1008#define QIB_7220_IBCDDRCtrl_IB_ENHANCED_MODE_LSB 0x0
1009#define QIB_7220_IBCDDRCtrl_IB_ENHANCED_MODE_RMASK 0x1
1010
1011#define QIB_7220_HRTBT_GUID_OFFS 0x188
1012
1013#define QIB_7220_IBCDDRCtrl2_OFFS 0x1A0
1014#define QIB_7220_IBCDDRCtrl2_IB_BACK_PORCH_LSB 0x5
1015#define QIB_7220_IBCDDRCtrl2_IB_BACK_PORCH_RMASK 0x1F
1016#define QIB_7220_IBCDDRCtrl2_IB_FRONT_PORCH_LSB 0x0
1017#define QIB_7220_IBCDDRCtrl2_IB_FRONT_PORCH_RMASK 0x1F
1018
1019#define QIB_7220_IBCDDRStatus_OFFS 0x1A8
1020#define QIB_7220_IBCDDRStatus_heartbeat_timed_out_LSB 0x24
1021#define QIB_7220_IBCDDRStatus_heartbeat_timed_out_RMASK 0x1
1022#define QIB_7220_IBCDDRStatus_heartbeat_crosstalk_LSB 0x20
1023#define QIB_7220_IBCDDRStatus_heartbeat_crosstalk_RMASK 0xF
1024#define QIB_7220_IBCDDRStatus_RxEqLocalDevice_LSB 0x1E
1025#define QIB_7220_IBCDDRStatus_RxEqLocalDevice_RMASK 0x3
1026#define QIB_7220_IBCDDRStatus_ReqDDSLocalFromRmt_LSB 0x1A
1027#define QIB_7220_IBCDDRStatus_ReqDDSLocalFromRmt_RMASK 0xF
1028#define QIB_7220_IBCDDRStatus_LinkRoundTripLatency_LSB 0x0
1029#define QIB_7220_IBCDDRStatus_LinkRoundTripLatency_RMASK 0x3FFFFFF
1030
1031#define QIB_7220_JIntReload_OFFS 0x1B0
1032#define QIB_7220_JIntReload_J_limit_reload_LSB 0x10
1033#define QIB_7220_JIntReload_J_limit_reload_RMASK 0xFFFF
1034#define QIB_7220_JIntReload_J_reload_LSB 0x0
1035#define QIB_7220_JIntReload_J_reload_RMASK 0xFFFF
1036
1037#define QIB_7220_IBNCModeCtrl_OFFS 0x1B8
1038#define QIB_7220_IBNCModeCtrl_Reserved_LSB 0x1A
1039#define QIB_7220_IBNCModeCtrl_Reserved_RMASK 0x3FFFFFFFFF
1040#define QIB_7220_IBNCModeCtrl_TSMCode_TS2_LSB 0x11
1041#define QIB_7220_IBNCModeCtrl_TSMCode_TS2_RMASK 0x1FF
1042#define QIB_7220_IBNCModeCtrl_TSMCode_TS1_LSB 0x8
1043#define QIB_7220_IBNCModeCtrl_TSMCode_TS1_RMASK 0x1FF
1044#define QIB_7220_IBNCModeCtrl_Reserved1_LSB 0x3
1045#define QIB_7220_IBNCModeCtrl_Reserved1_RMASK 0x1F
1046#define QIB_7220_IBNCModeCtrl_TSMEnable_ignore_TSM_on_rx_LSB 0x2
1047#define QIB_7220_IBNCModeCtrl_TSMEnable_ignore_TSM_on_rx_RMASK 0x1
1048#define QIB_7220_IBNCModeCtrl_TSMEnable_send_TS2_LSB 0x1
1049#define QIB_7220_IBNCModeCtrl_TSMEnable_send_TS2_RMASK 0x1
1050#define QIB_7220_IBNCModeCtrl_TSMEnable_send_TS1_LSB 0x0
1051#define QIB_7220_IBNCModeCtrl_TSMEnable_send_TS1_RMASK 0x1
1052
1053#define QIB_7220_SendCtrl_OFFS 0x1C0
1054#define QIB_7220_SendCtrl_Disarm_LSB 0x1F
1055#define QIB_7220_SendCtrl_Disarm_RMASK 0x1
1056#define QIB_7220_SendCtrl_Reserved_LSB 0x1D
1057#define QIB_7220_SendCtrl_Reserved_RMASK 0x3
1058#define QIB_7220_SendCtrl_AvailUpdThld_LSB 0x18
1059#define QIB_7220_SendCtrl_AvailUpdThld_RMASK 0x1F
1060#define QIB_7220_SendCtrl_DisarmPIOBuf_LSB 0x10
1061#define QIB_7220_SendCtrl_DisarmPIOBuf_RMASK 0xFF
1062#define QIB_7220_SendCtrl_Reserved1_LSB 0xD
1063#define QIB_7220_SendCtrl_Reserved1_RMASK 0x7
1064#define QIB_7220_SendCtrl_SDmaHalt_LSB 0xC
1065#define QIB_7220_SendCtrl_SDmaHalt_RMASK 0x1
1066#define QIB_7220_SendCtrl_SDmaEnable_LSB 0xB
1067#define QIB_7220_SendCtrl_SDmaEnable_RMASK 0x1
1068#define QIB_7220_SendCtrl_SDmaSingleDescriptor_LSB 0xA
1069#define QIB_7220_SendCtrl_SDmaSingleDescriptor_RMASK 0x1
1070#define QIB_7220_SendCtrl_SDmaIntEnable_LSB 0x9
1071#define QIB_7220_SendCtrl_SDmaIntEnable_RMASK 0x1
1072#define QIB_7220_SendCtrl_Reserved2_LSB 0x5
1073#define QIB_7220_SendCtrl_Reserved2_RMASK 0xF
1074#define QIB_7220_SendCtrl_SSpecialTriggerEn_LSB 0x4
1075#define QIB_7220_SendCtrl_SSpecialTriggerEn_RMASK 0x1
1076#define QIB_7220_SendCtrl_SPioEnable_LSB 0x3
1077#define QIB_7220_SendCtrl_SPioEnable_RMASK 0x1
1078#define QIB_7220_SendCtrl_SendBufAvailUpd_LSB 0x2
1079#define QIB_7220_SendCtrl_SendBufAvailUpd_RMASK 0x1
1080#define QIB_7220_SendCtrl_SendIntBufAvail_LSB 0x1
1081#define QIB_7220_SendCtrl_SendIntBufAvail_RMASK 0x1
1082#define QIB_7220_SendCtrl_Abort_LSB 0x0
1083#define QIB_7220_SendCtrl_Abort_RMASK 0x1
1084
1085#define QIB_7220_SendBufBase_OFFS 0x1C8
1086#define QIB_7220_SendBufBase_Reserved_LSB 0x35
1087#define QIB_7220_SendBufBase_Reserved_RMASK 0x7FF
1088#define QIB_7220_SendBufBase_BaseAddr_LargePIO_LSB 0x20
1089#define QIB_7220_SendBufBase_BaseAddr_LargePIO_RMASK 0x1FFFFF
1090#define QIB_7220_SendBufBase_Reserved1_LSB 0x15
1091#define QIB_7220_SendBufBase_Reserved1_RMASK 0x7FF
1092#define QIB_7220_SendBufBase_BaseAddr_SmallPIO_LSB 0x0
1093#define QIB_7220_SendBufBase_BaseAddr_SmallPIO_RMASK 0x1FFFFF
1094
1095#define QIB_7220_SendBufSize_OFFS 0x1D0
1096#define QIB_7220_SendBufSize_Reserved_LSB 0x2D
1097#define QIB_7220_SendBufSize_Reserved_RMASK 0xFFFFF
1098#define QIB_7220_SendBufSize_Size_LargePIO_LSB 0x20
1099#define QIB_7220_SendBufSize_Size_LargePIO_RMASK 0x1FFF
1100#define QIB_7220_SendBufSize_Reserved1_LSB 0xC
1101#define QIB_7220_SendBufSize_Reserved1_RMASK 0xFFFFF
1102#define QIB_7220_SendBufSize_Size_SmallPIO_LSB 0x0
1103#define QIB_7220_SendBufSize_Size_SmallPIO_RMASK 0xFFF
1104
1105#define QIB_7220_SendBufCnt_OFFS 0x1D8
1106#define QIB_7220_SendBufCnt_Reserved_LSB 0x24
1107#define QIB_7220_SendBufCnt_Reserved_RMASK 0xFFFFFFF
1108#define QIB_7220_SendBufCnt_Num_LargeBuffers_LSB 0x20
1109#define QIB_7220_SendBufCnt_Num_LargeBuffers_RMASK 0xF
1110#define QIB_7220_SendBufCnt_Reserved1_LSB 0x9
1111#define QIB_7220_SendBufCnt_Reserved1_RMASK 0x7FFFFF
1112#define QIB_7220_SendBufCnt_Num_SmallBuffers_LSB 0x0
1113#define QIB_7220_SendBufCnt_Num_SmallBuffers_RMASK 0x1FF
1114
1115#define QIB_7220_SendBufAvailAddr_OFFS 0x1E0
1116#define QIB_7220_SendBufAvailAddr_SendBufAvailAddr_LSB 0x6
1117#define QIB_7220_SendBufAvailAddr_SendBufAvailAddr_RMASK 0x3FFFFFFFF
1118#define QIB_7220_SendBufAvailAddr_Reserved_LSB 0x0
1119#define QIB_7220_SendBufAvailAddr_Reserved_RMASK 0x3F
1120
1121#define QIB_7220_TxIntMemBase_OFFS 0x1E8
1122
1123#define QIB_7220_TxIntMemSize_OFFS 0x1F0
1124
1125#define QIB_7220_SendDmaBase_OFFS 0x1F8
1126#define QIB_7220_SendDmaBase_Reserved_LSB 0x30
1127#define QIB_7220_SendDmaBase_Reserved_RMASK 0xFFFF
1128#define QIB_7220_SendDmaBase_SendDmaBase_LSB 0x0
1129#define QIB_7220_SendDmaBase_SendDmaBase_RMASK 0xFFFFFFFFFFFF
1130
1131#define QIB_7220_SendDmaLenGen_OFFS 0x200
1132#define QIB_7220_SendDmaLenGen_Reserved_LSB 0x13
1133#define QIB_7220_SendDmaLenGen_Reserved_RMASK 0x1FFFFFFFFFFF
1134#define QIB_7220_SendDmaLenGen_Generation_LSB 0x10
1135#define QIB_7220_SendDmaLenGen_Generation_MSB 0x12
1136#define QIB_7220_SendDmaLenGen_Generation_RMASK 0x7
1137#define QIB_7220_SendDmaLenGen_Length_LSB 0x0
1138#define QIB_7220_SendDmaLenGen_Length_RMASK 0xFFFF
1139
1140#define QIB_7220_SendDmaTail_OFFS 0x208
1141#define QIB_7220_SendDmaTail_Reserved_LSB 0x10
1142#define QIB_7220_SendDmaTail_Reserved_RMASK 0xFFFFFFFFFFFF
1143#define QIB_7220_SendDmaTail_SendDmaTail_LSB 0x0
1144#define QIB_7220_SendDmaTail_SendDmaTail_RMASK 0xFFFF
1145
1146#define QIB_7220_SendDmaHead_OFFS 0x210
1147#define QIB_7220_SendDmaHead_Reserved_LSB 0x30
1148#define QIB_7220_SendDmaHead_Reserved_RMASK 0xFFFF
1149#define QIB_7220_SendDmaHead_InternalSendDmaHead_LSB 0x20
1150#define QIB_7220_SendDmaHead_InternalSendDmaHead_RMASK 0xFFFF
1151#define QIB_7220_SendDmaHead_Reserved1_LSB 0x10
1152#define QIB_7220_SendDmaHead_Reserved1_RMASK 0xFFFF
1153#define QIB_7220_SendDmaHead_SendDmaHead_LSB 0x0
1154#define QIB_7220_SendDmaHead_SendDmaHead_RMASK 0xFFFF
1155
1156#define QIB_7220_SendDmaHeadAddr_OFFS 0x218
1157#define QIB_7220_SendDmaHeadAddr_Reserved_LSB 0x30
1158#define QIB_7220_SendDmaHeadAddr_Reserved_RMASK 0xFFFF
1159#define QIB_7220_SendDmaHeadAddr_SendDmaHeadAddr_LSB 0x0
1160#define QIB_7220_SendDmaHeadAddr_SendDmaHeadAddr_RMASK 0xFFFFFFFFFFFF
1161
1162#define QIB_7220_SendDmaBufMask0_OFFS 0x220
1163#define QIB_7220_SendDmaBufMask0_BufMask_63_0_LSB 0x0
1164#define QIB_7220_SendDmaBufMask0_BufMask_63_0_RMASK 0x0
1165
1166#define QIB_7220_SendDmaStatus_OFFS 0x238
1167#define QIB_7220_SendDmaStatus_ScoreBoardDrainInProg_LSB 0x3F
1168#define QIB_7220_SendDmaStatus_ScoreBoardDrainInProg_RMASK 0x1
1169#define QIB_7220_SendDmaStatus_AbortInProg_LSB 0x3E
1170#define QIB_7220_SendDmaStatus_AbortInProg_RMASK 0x1
1171#define QIB_7220_SendDmaStatus_InternalSDmaEnable_LSB 0x3D
1172#define QIB_7220_SendDmaStatus_InternalSDmaEnable_RMASK 0x1
1173#define QIB_7220_SendDmaStatus_ScbDescIndex_13_0_LSB 0x2F
1174#define QIB_7220_SendDmaStatus_ScbDescIndex_13_0_RMASK 0x3FFF
1175#define QIB_7220_SendDmaStatus_RpyLowAddr_6_0_LSB 0x28
1176#define QIB_7220_SendDmaStatus_RpyLowAddr_6_0_RMASK 0x7F
1177#define QIB_7220_SendDmaStatus_RpyTag_7_0_LSB 0x20
1178#define QIB_7220_SendDmaStatus_RpyTag_7_0_RMASK 0xFF
1179#define QIB_7220_SendDmaStatus_ScbFull_LSB 0x1F
1180#define QIB_7220_SendDmaStatus_ScbFull_RMASK 0x1
1181#define QIB_7220_SendDmaStatus_ScbEmpty_LSB 0x1E
1182#define QIB_7220_SendDmaStatus_ScbEmpty_RMASK 0x1
1183#define QIB_7220_SendDmaStatus_ScbEntryValid_LSB 0x1D
1184#define QIB_7220_SendDmaStatus_ScbEntryValid_RMASK 0x1
1185#define QIB_7220_SendDmaStatus_ScbFetchDescFlag_LSB 0x1C
1186#define QIB_7220_SendDmaStatus_ScbFetchDescFlag_RMASK 0x1
1187#define QIB_7220_SendDmaStatus_SplFifoReadyToGo_LSB 0x1B
1188#define QIB_7220_SendDmaStatus_SplFifoReadyToGo_RMASK 0x1
1189#define QIB_7220_SendDmaStatus_SplFifoDisarmed_LSB 0x1A
1190#define QIB_7220_SendDmaStatus_SplFifoDisarmed_RMASK 0x1
1191#define QIB_7220_SendDmaStatus_SplFifoEmpty_LSB 0x19
1192#define QIB_7220_SendDmaStatus_SplFifoEmpty_RMASK 0x1
1193#define QIB_7220_SendDmaStatus_SplFifoFull_LSB 0x18
1194#define QIB_7220_SendDmaStatus_SplFifoFull_RMASK 0x1
1195#define QIB_7220_SendDmaStatus_SplFifoBufNum_LSB 0x10
1196#define QIB_7220_SendDmaStatus_SplFifoBufNum_RMASK 0xFF
1197#define QIB_7220_SendDmaStatus_SplFifoDescIndex_LSB 0x0
1198#define QIB_7220_SendDmaStatus_SplFifoDescIndex_RMASK 0xFFFF
1199
1200#define QIB_7220_SendBufErr0_OFFS 0x240
1201#define QIB_7220_SendBufErr0_SendBufErr_63_0_LSB 0x0
1202#define QIB_7220_SendBufErr0_SendBufErr_63_0_RMASK 0x0
1203
1204#define QIB_7220_RcvHdrAddr0_OFFS 0x270
1205#define QIB_7220_RcvHdrAddr0_RcvHdrAddr0_LSB 0x2
1206#define QIB_7220_RcvHdrAddr0_RcvHdrAddr0_RMASK 0x3FFFFFFFFF
1207#define QIB_7220_RcvHdrAddr0_Reserved_LSB 0x0
1208#define QIB_7220_RcvHdrAddr0_Reserved_RMASK 0x3
1209
1210#define QIB_7220_RcvHdrTailAddr0_OFFS 0x300
1211#define QIB_7220_RcvHdrTailAddr0_RcvHdrTailAddr0_LSB 0x2
1212#define QIB_7220_RcvHdrTailAddr0_RcvHdrTailAddr0_RMASK 0x3FFFFFFFFF
1213#define QIB_7220_RcvHdrTailAddr0_Reserved_LSB 0x0
1214#define QIB_7220_RcvHdrTailAddr0_Reserved_RMASK 0x3
1215
1216#define QIB_7220_ibsd_epb_access_ctrl_OFFS 0x3C0
1217#define QIB_7220_ibsd_epb_access_ctrl_sw_ib_epb_req_granted_LSB 0x8
1218#define QIB_7220_ibsd_epb_access_ctrl_sw_ib_epb_req_granted_RMASK 0x1
1219#define QIB_7220_ibsd_epb_access_ctrl_Reserved_LSB 0x1
1220#define QIB_7220_ibsd_epb_access_ctrl_Reserved_RMASK 0x7F
1221#define QIB_7220_ibsd_epb_access_ctrl_sw_ib_epb_req_LSB 0x0
1222#define QIB_7220_ibsd_epb_access_ctrl_sw_ib_epb_req_RMASK 0x1
1223
1224#define QIB_7220_ibsd_epb_transaction_reg_OFFS 0x3C8
1225#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_rdy_LSB 0x1F
1226#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_rdy_RMASK 0x1
1227#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_req_error_LSB 0x1E
1228#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_req_error_RMASK 0x1
1229#define QIB_7220_ibsd_epb_transaction_reg_Reserved_LSB 0x1D
1230#define QIB_7220_ibsd_epb_transaction_reg_Reserved_RMASK 0x1
1231#define QIB_7220_ibsd_epb_transaction_reg_mem_data_parity_LSB 0x1C
1232#define QIB_7220_ibsd_epb_transaction_reg_mem_data_parity_RMASK 0x1
1233#define QIB_7220_ibsd_epb_transaction_reg_Reserved1_LSB 0x1B
1234#define QIB_7220_ibsd_epb_transaction_reg_Reserved1_RMASK 0x1
1235#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_cs_LSB 0x19
1236#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_cs_RMASK 0x3
1237#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_read_write_LSB 0x18
1238#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_read_write_RMASK 0x1
1239#define QIB_7220_ibsd_epb_transaction_reg_Reserved2_LSB 0x17
1240#define QIB_7220_ibsd_epb_transaction_reg_Reserved2_RMASK 0x1
1241#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_address_LSB 0x8
1242#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_address_RMASK 0x7FFF
1243#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_data_LSB 0x0
1244#define QIB_7220_ibsd_epb_transaction_reg_ib_epb_data_RMASK 0xFF
1245
1246#define QIB_7220_XGXSCfg_OFFS 0x3D8
1247#define QIB_7220_XGXSCfg_sel_link_down_for_fctrl_lane_sync_reset_LSB 0x3F
1248#define QIB_7220_XGXSCfg_sel_link_down_for_fctrl_lane_sync_reset_RMASK 0x1
1249#define QIB_7220_XGXSCfg_Reserved_LSB 0x13
1250#define QIB_7220_XGXSCfg_Reserved_RMASK 0xFFFFFFFFFFF
1251#define QIB_7220_XGXSCfg_link_sync_mask_LSB 0x9
1252#define QIB_7220_XGXSCfg_link_sync_mask_RMASK 0x3FF
1253#define QIB_7220_XGXSCfg_Reserved1_LSB 0x3
1254#define QIB_7220_XGXSCfg_Reserved1_RMASK 0x3F
1255#define QIB_7220_XGXSCfg_xcv_reset_LSB 0x2
1256#define QIB_7220_XGXSCfg_xcv_reset_RMASK 0x1
1257#define QIB_7220_XGXSCfg_Reserved2_LSB 0x1
1258#define QIB_7220_XGXSCfg_Reserved2_RMASK 0x1
1259#define QIB_7220_XGXSCfg_tx_rx_reset_LSB 0x0
1260#define QIB_7220_XGXSCfg_tx_rx_reset_RMASK 0x1
1261
1262#define QIB_7220_IBSerDesCtrl_OFFS 0x3E0
1263#define QIB_7220_IBSerDesCtrl_Reserved_LSB 0x2D
1264#define QIB_7220_IBSerDesCtrl_Reserved_RMASK 0x7FFFF
1265#define QIB_7220_IBSerDesCtrl_INT_uC_LSB 0x2C
1266#define QIB_7220_IBSerDesCtrl_INT_uC_RMASK 0x1
1267#define QIB_7220_IBSerDesCtrl_CKSEL_uC_LSB 0x2A
1268#define QIB_7220_IBSerDesCtrl_CKSEL_uC_RMASK 0x3
1269#define QIB_7220_IBSerDesCtrl_PLLN_LSB 0x28
1270#define QIB_7220_IBSerDesCtrl_PLLN_RMASK 0x3
1271#define QIB_7220_IBSerDesCtrl_PLLM_LSB 0x25
1272#define QIB_7220_IBSerDesCtrl_PLLM_RMASK 0x7
1273#define QIB_7220_IBSerDesCtrl_TXOBPD_LSB 0x24
1274#define QIB_7220_IBSerDesCtrl_TXOBPD_RMASK 0x1
1275#define QIB_7220_IBSerDesCtrl_TWC_LSB 0x23
1276#define QIB_7220_IBSerDesCtrl_TWC_RMASK 0x1
1277#define QIB_7220_IBSerDesCtrl_RXIDLE_LSB 0x22
1278#define QIB_7220_IBSerDesCtrl_RXIDLE_RMASK 0x1
1279#define QIB_7220_IBSerDesCtrl_RXINV_LSB 0x21
1280#define QIB_7220_IBSerDesCtrl_RXINV_RMASK 0x1
1281#define QIB_7220_IBSerDesCtrl_TXINV_LSB 0x20
1282#define QIB_7220_IBSerDesCtrl_TXINV_RMASK 0x1
1283#define QIB_7220_IBSerDesCtrl_Reserved1_LSB 0x12
1284#define QIB_7220_IBSerDesCtrl_Reserved1_RMASK 0x3FFF
1285#define QIB_7220_IBSerDesCtrl_NumSerDesRegsToWrForRXEQ_LSB 0xD
1286#define QIB_7220_IBSerDesCtrl_NumSerDesRegsToWrForRXEQ_RMASK 0x1F
1287#define QIB_7220_IBSerDesCtrl_NumSerDesRegsToWrForDDS_LSB 0x8
1288#define QIB_7220_IBSerDesCtrl_NumSerDesRegsToWrForDDS_RMASK 0x1F
1289#define QIB_7220_IBSerDesCtrl_Reserved2_LSB 0x1
1290#define QIB_7220_IBSerDesCtrl_Reserved2_RMASK 0x7F
1291#define QIB_7220_IBSerDesCtrl_ResetIB_uC_Core_LSB 0x0
1292#define QIB_7220_IBSerDesCtrl_ResetIB_uC_Core_RMASK 0x1
1293
1294#define QIB_7220_pciesd_epb_access_ctrl_OFFS 0x400
1295#define QIB_7220_pciesd_epb_access_ctrl_sw_pcie_epb_req_granted_LSB 0x8
1296#define QIB_7220_pciesd_epb_access_ctrl_sw_pcie_epb_req_granted_RMASK 0x1
1297#define QIB_7220_pciesd_epb_access_ctrl_Reserved_LSB 0x3
1298#define QIB_7220_pciesd_epb_access_ctrl_Reserved_RMASK 0x1F
1299#define QIB_7220_pciesd_epb_access_ctrl_sw_pcieepb_star_en_LSB 0x1
1300#define QIB_7220_pciesd_epb_access_ctrl_sw_pcieepb_star_en_RMASK 0x3
1301#define QIB_7220_pciesd_epb_access_ctrl_sw_pcie_epb_req_LSB 0x0
1302#define QIB_7220_pciesd_epb_access_ctrl_sw_pcie_epb_req_RMASK 0x1
1303
1304#define QIB_7220_pciesd_epb_transaction_reg_OFFS 0x408
1305#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_rdy_LSB 0x1F
1306#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_rdy_RMASK 0x1
1307#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_req_error_LSB 0x1E
1308#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_req_error_RMASK 0x1
1309#define QIB_7220_pciesd_epb_transaction_reg_Reserved_LSB 0x1D
1310#define QIB_7220_pciesd_epb_transaction_reg_Reserved_RMASK 0x1
1311#define QIB_7220_pciesd_epb_transaction_reg_mem_data_parity_LSB 0x1C
1312#define QIB_7220_pciesd_epb_transaction_reg_mem_data_parity_RMASK 0x1
1313#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_cs_LSB 0x19
1314#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_cs_RMASK 0x7
1315#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_read_write_LSB 0x18
1316#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_read_write_RMASK 0x1
1317#define QIB_7220_pciesd_epb_transaction_reg_Reserved1_LSB 0x17
1318#define QIB_7220_pciesd_epb_transaction_reg_Reserved1_RMASK 0x1
1319#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_address_LSB 0x8
1320#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_address_RMASK 0x7FFF
1321#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_data_LSB 0x0
1322#define QIB_7220_pciesd_epb_transaction_reg_pcie_epb_data_RMASK 0xFF
1323
1324#define QIB_7220_SerDes_DDSRXEQ0_OFFS 0x500
1325#define QIB_7220_SerDes_DDSRXEQ0_reg_addr_LSB 0x4
1326#define QIB_7220_SerDes_DDSRXEQ0_reg_addr_RMASK 0x3F
1327#define QIB_7220_SerDes_DDSRXEQ0_element_num_LSB 0x0
1328#define QIB_7220_SerDes_DDSRXEQ0_element_num_RMASK 0xF
1329
1330#define QIB_7220_LBIntCnt_OFFS 0x13000
1331
1332#define QIB_7220_LBFlowStallCnt_OFFS 0x13008
1333
1334#define QIB_7220_TxSDmaDescCnt_OFFS 0x13010
1335
1336#define QIB_7220_TxUnsupVLErrCnt_OFFS 0x13018
1337
1338#define QIB_7220_TxDataPktCnt_OFFS 0x13020
1339
1340#define QIB_7220_TxFlowPktCnt_OFFS 0x13028
1341
1342#define QIB_7220_TxDwordCnt_OFFS 0x13030
1343
1344#define QIB_7220_TxLenErrCnt_OFFS 0x13038
1345
1346#define QIB_7220_TxMaxMinLenErrCnt_OFFS 0x13040
1347
1348#define QIB_7220_TxUnderrunCnt_OFFS 0x13048
1349
1350#define QIB_7220_TxFlowStallCnt_OFFS 0x13050
1351
1352#define QIB_7220_TxDroppedPktCnt_OFFS 0x13058
1353
1354#define QIB_7220_RxDroppedPktCnt_OFFS 0x13060
1355
1356#define QIB_7220_RxDataPktCnt_OFFS 0x13068
1357
1358#define QIB_7220_RxFlowPktCnt_OFFS 0x13070
1359
1360#define QIB_7220_RxDwordCnt_OFFS 0x13078
1361
1362#define QIB_7220_RxLenErrCnt_OFFS 0x13080
1363
1364#define QIB_7220_RxMaxMinLenErrCnt_OFFS 0x13088
1365
1366#define QIB_7220_RxICRCErrCnt_OFFS 0x13090
1367
1368#define QIB_7220_RxVCRCErrCnt_OFFS 0x13098
1369
1370#define QIB_7220_RxFlowCtrlViolCnt_OFFS 0x130A0
1371
1372#define QIB_7220_RxVersionErrCnt_OFFS 0x130A8
1373
1374#define QIB_7220_RxLinkMalformCnt_OFFS 0x130B0
1375
1376#define QIB_7220_RxEBPCnt_OFFS 0x130B8
1377
1378#define QIB_7220_RxLPCRCErrCnt_OFFS 0x130C0
1379
1380#define QIB_7220_RxBufOvflCnt_OFFS 0x130C8
1381
1382#define QIB_7220_RxTIDFullErrCnt_OFFS 0x130D0
1383
1384#define QIB_7220_RxTIDValidErrCnt_OFFS 0x130D8
1385
1386#define QIB_7220_RxPKeyMismatchCnt_OFFS 0x130E0
1387
1388#define QIB_7220_RxP0HdrEgrOvflCnt_OFFS 0x130E8
1389
1390#define QIB_7220_IBStatusChangeCnt_OFFS 0x13170
1391
1392#define QIB_7220_IBLinkErrRecoveryCnt_OFFS 0x13178
1393
1394#define QIB_7220_IBLinkDownedCnt_OFFS 0x13180
1395
1396#define QIB_7220_IBSymbolErrCnt_OFFS 0x13188
1397
1398#define QIB_7220_RxVL15DroppedPktCnt_OFFS 0x13190
1399
1400#define QIB_7220_RxOtherLocalPhyErrCnt_OFFS 0x13198
1401
1402#define QIB_7220_PcieRetryBufDiagQwordCnt_OFFS 0x131A0
1403
1404#define QIB_7220_ExcessBufferOvflCnt_OFFS 0x131A8
1405
1406#define QIB_7220_LocalLinkIntegrityErrCnt_OFFS 0x131B0
1407
1408#define QIB_7220_RxVlErrCnt_OFFS 0x131B8
1409
1410#define QIB_7220_RxDlidFltrCnt_OFFS 0x131C0
1411
1412#define QIB_7220_CNT_0131C8_OFFS 0x131C8
1413
1414#define QIB_7220_PSStat_OFFS 0x13200
1415
1416#define QIB_7220_PSStart_OFFS 0x13208
1417
1418#define QIB_7220_PSInterval_OFFS 0x13210
1419
1420#define QIB_7220_PSRcvDataCount_OFFS 0x13218
1421
1422#define QIB_7220_PSRcvPktsCount_OFFS 0x13220
1423
1424#define QIB_7220_PSXmitDataCount_OFFS 0x13228
1425
1426#define QIB_7220_PSXmitPktsCount_OFFS 0x13230
1427
1428#define QIB_7220_PSXmitWaitCount_OFFS 0x13238
1429
1430#define QIB_7220_CNT_013240_OFFS 0x13240
1431
1432#define QIB_7220_RcvEgrArray_OFFS 0x14000
1433
1434#define QIB_7220_MEM_038000_OFFS 0x38000
1435
1436#define QIB_7220_RcvTIDArray0_OFFS 0x53000
1437
1438#define QIB_7220_PIOLaunchFIFO_OFFS 0x64000
1439
1440#define QIB_7220_MEM_064480_OFFS 0x64480
1441
1442#define QIB_7220_SendPIOpbcCache_OFFS 0x64800
1443
1444#define QIB_7220_MEM_064C80_OFFS 0x64C80
1445
1446#define QIB_7220_PreLaunchFIFO_OFFS 0x65000
1447
1448#define QIB_7220_MEM_065080_OFFS 0x65080
1449
1450#define QIB_7220_ScoreBoard_OFFS 0x65400
1451
1452#define QIB_7220_MEM_065440_OFFS 0x65440
1453
1454#define QIB_7220_DescriptorFIFO_OFFS 0x65800
1455
1456#define QIB_7220_MEM_065880_OFFS 0x65880
1457
1458#define QIB_7220_RcvBuf1_OFFS 0x72000
1459
1460#define QIB_7220_MEM_074800_OFFS 0x74800
1461
1462#define QIB_7220_RcvBuf2_OFFS 0x75000
1463
1464#define QIB_7220_MEM_076400_OFFS 0x76400
1465
1466#define QIB_7220_RcvFlags_OFFS 0x77000
1467
1468#define QIB_7220_MEM_078400_OFFS 0x78400
1469
1470#define QIB_7220_RcvLookupBuf1_OFFS 0x79000
1471
1472#define QIB_7220_MEM_07A400_OFFS 0x7A400
1473
1474#define QIB_7220_RcvDMADatBuf_OFFS 0x7B000
1475
1476#define QIB_7220_RcvDMAHdrBuf_OFFS 0x7B800
1477
1478#define QIB_7220_MiscRXEIntMem_OFFS 0x7C000
1479
1480#define QIB_7220_MEM_07D400_OFFS 0x7D400
1481
1482#define QIB_7220_PCIERcvBuf_OFFS 0x80000
1483
1484#define QIB_7220_PCIERetryBuf_OFFS 0x84000
1485
1486#define QIB_7220_PCIERcvBufRdToWrAddr_OFFS 0x88000
1487
1488#define QIB_7220_PCIECplBuf_OFFS 0x90000
1489
1490#define QIB_7220_IBSerDesMappTable_OFFS 0x94000
1491
1492#define QIB_7220_MEM_095000_OFFS 0x95000
1493
1494#define QIB_7220_SendBuf0_MA_OFFS 0x100000
1495
1496#define QIB_7220_MEM_1A0000_OFFS 0x1A0000
drivers/infiniband/hw/qib/qib_7322_regs.h+0−3163
@@ -1,3163 +0,0 @@
1/*
2 * Copyright (c) 2008, 2009, 2010 QLogic Corporation. All rights reserved.
3 *
4 * This software is available to you under a choice of one of two
5 * licenses. You may choose to be licensed under the terms of the GNU
6 * General Public License (GPL) Version 2, available from the file
7 * COPYING in the main directory of this source tree, or the
8 * OpenIB.org BSD license below:
9 *
10 * Redistribution and use in source and binary forms, with or
11 * without modification, are permitted provided that the following
12 * conditions are met:
13 *
14 * - Redistributions of source code must retain the above
15 * copyright notice, this list of conditions and the following
16 * disclaimer.
17 *
18 * - Redistributions in binary form must reproduce the above
19 * copyright notice, this list of conditions and the following
20 * disclaimer in the documentation and/or other materials
21 * provided with the distribution.
22 *
23 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
26 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
27 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
28 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
29 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
30 * SOFTWARE.
31 */
32
33/* This file is mechanically generated from RTL. Any hand-edits will be lost! */
34
35#define QIB_7322_Revision_OFFS 0x0
36#define QIB_7322_Revision_DEF 0x0000000002010601
37#define QIB_7322_Revision_R_Simulator_LSB 0x3F
38#define QIB_7322_Revision_R_Simulator_MSB 0x3F
39#define QIB_7322_Revision_R_Simulator_RMASK 0x1
40#define QIB_7322_Revision_R_Emulation_LSB 0x3E
41#define QIB_7322_Revision_R_Emulation_MSB 0x3E
42#define QIB_7322_Revision_R_Emulation_RMASK 0x1
43#define QIB_7322_Revision_R_Emulation_Revcode_LSB 0x28
44#define QIB_7322_Revision_R_Emulation_Revcode_MSB 0x3D
45#define QIB_7322_Revision_R_Emulation_Revcode_RMASK 0x3FFFFF
46#define QIB_7322_Revision_BoardID_LSB 0x20
47#define QIB_7322_Revision_BoardID_MSB 0x27
48#define QIB_7322_Revision_BoardID_RMASK 0xFF
49#define QIB_7322_Revision_R_SW_LSB 0x18
50#define QIB_7322_Revision_R_SW_MSB 0x1F
51#define QIB_7322_Revision_R_SW_RMASK 0xFF
52#define QIB_7322_Revision_R_Arch_LSB 0x10
53#define QIB_7322_Revision_R_Arch_MSB 0x17
54#define QIB_7322_Revision_R_Arch_RMASK 0xFF
55#define QIB_7322_Revision_R_ChipRevMajor_LSB 0x8
56#define QIB_7322_Revision_R_ChipRevMajor_MSB 0xF
57#define QIB_7322_Revision_R_ChipRevMajor_RMASK 0xFF
58#define QIB_7322_Revision_R_ChipRevMinor_LSB 0x0
59#define QIB_7322_Revision_R_ChipRevMinor_MSB 0x7
60#define QIB_7322_Revision_R_ChipRevMinor_RMASK 0xFF
61
62#define QIB_7322_Control_OFFS 0x8
63#define QIB_7322_Control_DEF 0x0000000000000000
64#define QIB_7322_Control_PCIECplQDiagEn_LSB 0x6
65#define QIB_7322_Control_PCIECplQDiagEn_MSB 0x6
66#define QIB_7322_Control_PCIECplQDiagEn_RMASK 0x1
67#define QIB_7322_Control_PCIEPostQDiagEn_LSB 0x5
68#define QIB_7322_Control_PCIEPostQDiagEn_MSB 0x5
69#define QIB_7322_Control_PCIEPostQDiagEn_RMASK 0x1
70#define QIB_7322_Control_SDmaDescFetchPriorityEn_LSB 0x4
71#define QIB_7322_Control_SDmaDescFetchPriorityEn_MSB 0x4
72#define QIB_7322_Control_SDmaDescFetchPriorityEn_RMASK 0x1
73#define QIB_7322_Control_PCIERetryBufDiagEn_LSB 0x3
74#define QIB_7322_Control_PCIERetryBufDiagEn_MSB 0x3
75#define QIB_7322_Control_PCIERetryBufDiagEn_RMASK 0x1
76#define QIB_7322_Control_FreezeMode_LSB 0x1
77#define QIB_7322_Control_FreezeMode_MSB 0x1
78#define QIB_7322_Control_FreezeMode_RMASK 0x1
79#define QIB_7322_Control_SyncReset_LSB 0x0
80#define QIB_7322_Control_SyncReset_MSB 0x0
81#define QIB_7322_Control_SyncReset_RMASK 0x1
82
83#define QIB_7322_PageAlign_OFFS 0x10
84#define QIB_7322_PageAlign_DEF 0x0000000000001000
85
86#define QIB_7322_ContextCnt_OFFS 0x18
87#define QIB_7322_ContextCnt_DEF 0x0000000000000012
88
89#define QIB_7322_Scratch_OFFS 0x20
90#define QIB_7322_Scratch_DEF 0x0000000000000000
91
92#define QIB_7322_CntrRegBase_OFFS 0x28
93#define QIB_7322_CntrRegBase_DEF 0x0000000000011000
94
95#define QIB_7322_SendRegBase_OFFS 0x30
96#define QIB_7322_SendRegBase_DEF 0x0000000000003000
97
98#define QIB_7322_UserRegBase_OFFS 0x38
99#define QIB_7322_UserRegBase_DEF 0x0000000000200000
100
101#define QIB_7322_IntMask_OFFS 0x68
102#define QIB_7322_IntMask_DEF 0x0000000000000000
103#define QIB_7322_IntMask_SDmaIntMask_1_LSB 0x3F
104#define QIB_7322_IntMask_SDmaIntMask_1_MSB 0x3F
105#define QIB_7322_IntMask_SDmaIntMask_1_RMASK 0x1
106#define QIB_7322_IntMask_SDmaIntMask_0_LSB 0x3E
107#define QIB_7322_IntMask_SDmaIntMask_0_MSB 0x3E
108#define QIB_7322_IntMask_SDmaIntMask_0_RMASK 0x1
109#define QIB_7322_IntMask_SDmaProgressIntMask_1_LSB 0x3D
110#define QIB_7322_IntMask_SDmaProgressIntMask_1_MSB 0x3D
111#define QIB_7322_IntMask_SDmaProgressIntMask_1_RMASK 0x1
112#define QIB_7322_IntMask_SDmaProgressIntMask_0_LSB 0x3C
113#define QIB_7322_IntMask_SDmaProgressIntMask_0_MSB 0x3C
114#define QIB_7322_IntMask_SDmaProgressIntMask_0_RMASK 0x1
115#define QIB_7322_IntMask_SDmaIdleIntMask_1_LSB 0x3B
116#define QIB_7322_IntMask_SDmaIdleIntMask_1_MSB 0x3B
117#define QIB_7322_IntMask_SDmaIdleIntMask_1_RMASK 0x1
118#define QIB_7322_IntMask_SDmaIdleIntMask_0_LSB 0x3A
119#define QIB_7322_IntMask_SDmaIdleIntMask_0_MSB 0x3A
120#define QIB_7322_IntMask_SDmaIdleIntMask_0_RMASK 0x1
121#define QIB_7322_IntMask_SDmaCleanupDoneMask_1_LSB 0x39
122#define QIB_7322_IntMask_SDmaCleanupDoneMask_1_MSB 0x39
123#define QIB_7322_IntMask_SDmaCleanupDoneMask_1_RMASK 0x1
124#define QIB_7322_IntMask_SDmaCleanupDoneMask_0_LSB 0x38
125#define QIB_7322_IntMask_SDmaCleanupDoneMask_0_MSB 0x38
126#define QIB_7322_IntMask_SDmaCleanupDoneMask_0_RMASK 0x1
127#define QIB_7322_IntMask_RcvUrg17IntMask_LSB 0x31
128#define QIB_7322_IntMask_RcvUrg17IntMask_MSB 0x31
129#define QIB_7322_IntMask_RcvUrg17IntMask_RMASK 0x1
130#define QIB_7322_IntMask_RcvUrg16IntMask_LSB 0x30
131#define QIB_7322_IntMask_RcvUrg16IntMask_MSB 0x30
132#define QIB_7322_IntMask_RcvUrg16IntMask_RMASK 0x1
133#define QIB_7322_IntMask_RcvUrg15IntMask_LSB 0x2F
134#define QIB_7322_IntMask_RcvUrg15IntMask_MSB 0x2F
135#define QIB_7322_IntMask_RcvUrg15IntMask_RMASK 0x1
136#define QIB_7322_IntMask_RcvUrg14IntMask_LSB 0x2E
137#define QIB_7322_IntMask_RcvUrg14IntMask_MSB 0x2E
138#define QIB_7322_IntMask_RcvUrg14IntMask_RMASK 0x1
139#define QIB_7322_IntMask_RcvUrg13IntMask_LSB 0x2D
140#define QIB_7322_IntMask_RcvUrg13IntMask_MSB 0x2D
141#define QIB_7322_IntMask_RcvUrg13IntMask_RMASK 0x1
142#define QIB_7322_IntMask_RcvUrg12IntMask_LSB 0x2C
143#define QIB_7322_IntMask_RcvUrg12IntMask_MSB 0x2C
144#define QIB_7322_IntMask_RcvUrg12IntMask_RMASK 0x1
145#define QIB_7322_IntMask_RcvUrg11IntMask_LSB 0x2B
146#define QIB_7322_IntMask_RcvUrg11IntMask_MSB 0x2B
147#define QIB_7322_IntMask_RcvUrg11IntMask_RMASK 0x1
148#define QIB_7322_IntMask_RcvUrg10IntMask_LSB 0x2A
149#define QIB_7322_IntMask_RcvUrg10IntMask_MSB 0x2A
150#define QIB_7322_IntMask_RcvUrg10IntMask_RMASK 0x1
151#define QIB_7322_IntMask_RcvUrg9IntMask_LSB 0x29
152#define QIB_7322_IntMask_RcvUrg9IntMask_MSB 0x29
153#define QIB_7322_IntMask_RcvUrg9IntMask_RMASK 0x1
154#define QIB_7322_IntMask_RcvUrg8IntMask_LSB 0x28
155#define QIB_7322_IntMask_RcvUrg8IntMask_MSB 0x28
156#define QIB_7322_IntMask_RcvUrg8IntMask_RMASK 0x1
157#define QIB_7322_IntMask_RcvUrg7IntMask_LSB 0x27
158#define QIB_7322_IntMask_RcvUrg7IntMask_MSB 0x27
159#define QIB_7322_IntMask_RcvUrg7IntMask_RMASK 0x1
160#define QIB_7322_IntMask_RcvUrg6IntMask_LSB 0x26
161#define QIB_7322_IntMask_RcvUrg6IntMask_MSB 0x26
162#define QIB_7322_IntMask_RcvUrg6IntMask_RMASK 0x1
163#define QIB_7322_IntMask_RcvUrg5IntMask_LSB 0x25
164#define QIB_7322_IntMask_RcvUrg5IntMask_MSB 0x25
165#define QIB_7322_IntMask_RcvUrg5IntMask_RMASK 0x1
166#define QIB_7322_IntMask_RcvUrg4IntMask_LSB 0x24
167#define QIB_7322_IntMask_RcvUrg4IntMask_MSB 0x24
168#define QIB_7322_IntMask_RcvUrg4IntMask_RMASK 0x1
169#define QIB_7322_IntMask_RcvUrg3IntMask_LSB 0x23
170#define QIB_7322_IntMask_RcvUrg3IntMask_MSB 0x23
171#define QIB_7322_IntMask_RcvUrg3IntMask_RMASK 0x1
172#define QIB_7322_IntMask_RcvUrg2IntMask_LSB 0x22
173#define QIB_7322_IntMask_RcvUrg2IntMask_MSB 0x22
174#define QIB_7322_IntMask_RcvUrg2IntMask_RMASK 0x1
175#define QIB_7322_IntMask_RcvUrg1IntMask_LSB 0x21
176#define QIB_7322_IntMask_RcvUrg1IntMask_MSB 0x21
177#define QIB_7322_IntMask_RcvUrg1IntMask_RMASK 0x1
178#define QIB_7322_IntMask_RcvUrg0IntMask_LSB 0x20
179#define QIB_7322_IntMask_RcvUrg0IntMask_MSB 0x20
180#define QIB_7322_IntMask_RcvUrg0IntMask_RMASK 0x1
181#define QIB_7322_IntMask_ErrIntMask_1_LSB 0x1F
182#define QIB_7322_IntMask_ErrIntMask_1_MSB 0x1F
183#define QIB_7322_IntMask_ErrIntMask_1_RMASK 0x1
184#define QIB_7322_IntMask_ErrIntMask_0_LSB 0x1E
185#define QIB_7322_IntMask_ErrIntMask_0_MSB 0x1E
186#define QIB_7322_IntMask_ErrIntMask_0_RMASK 0x1
187#define QIB_7322_IntMask_ErrIntMask_LSB 0x1D
188#define QIB_7322_IntMask_ErrIntMask_MSB 0x1D
189#define QIB_7322_IntMask_ErrIntMask_RMASK 0x1
190#define QIB_7322_IntMask_AssertGPIOIntMask_LSB 0x1C
191#define QIB_7322_IntMask_AssertGPIOIntMask_MSB 0x1C
192#define QIB_7322_IntMask_AssertGPIOIntMask_RMASK 0x1
193#define QIB_7322_IntMask_SendDoneIntMask_1_LSB 0x19
194#define QIB_7322_IntMask_SendDoneIntMask_1_MSB 0x19
195#define QIB_7322_IntMask_SendDoneIntMask_1_RMASK 0x1
196#define QIB_7322_IntMask_SendDoneIntMask_0_LSB 0x18
197#define QIB_7322_IntMask_SendDoneIntMask_0_MSB 0x18
198#define QIB_7322_IntMask_SendDoneIntMask_0_RMASK 0x1
199#define QIB_7322_IntMask_SendBufAvailIntMask_LSB 0x17
200#define QIB_7322_IntMask_SendBufAvailIntMask_MSB 0x17
201#define QIB_7322_IntMask_SendBufAvailIntMask_RMASK 0x1
202#define QIB_7322_IntMask_RcvAvail17IntMask_LSB 0x11
203#define QIB_7322_IntMask_RcvAvail17IntMask_MSB 0x11
204#define QIB_7322_IntMask_RcvAvail17IntMask_RMASK 0x1
205#define QIB_7322_IntMask_RcvAvail16IntMask_LSB 0x10
206#define QIB_7322_IntMask_RcvAvail16IntMask_MSB 0x10
207#define QIB_7322_IntMask_RcvAvail16IntMask_RMASK 0x1
208#define QIB_7322_IntMask_RcvAvail15IntMask_LSB 0xF
209#define QIB_7322_IntMask_RcvAvail15IntMask_MSB 0xF
210#define QIB_7322_IntMask_RcvAvail15IntMask_RMASK 0x1
211#define QIB_7322_IntMask_RcvAvail14IntMask_LSB 0xE
212#define QIB_7322_IntMask_RcvAvail14IntMask_MSB 0xE
213#define QIB_7322_IntMask_RcvAvail14IntMask_RMASK 0x1
214#define QIB_7322_IntMask_RcvAvail13IntMask_LSB 0xD
215#define QIB_7322_IntMask_RcvAvail13IntMask_MSB 0xD
216#define QIB_7322_IntMask_RcvAvail13IntMask_RMASK 0x1
217#define QIB_7322_IntMask_RcvAvail12IntMask_LSB 0xC
218#define QIB_7322_IntMask_RcvAvail12IntMask_MSB 0xC
219#define QIB_7322_IntMask_RcvAvail12IntMask_RMASK 0x1
220#define QIB_7322_IntMask_RcvAvail11IntMask_LSB 0xB
221#define QIB_7322_IntMask_RcvAvail11IntMask_MSB 0xB
222#define QIB_7322_IntMask_RcvAvail11IntMask_RMASK 0x1
223#define QIB_7322_IntMask_RcvAvail10IntMask_LSB 0xA
224#define QIB_7322_IntMask_RcvAvail10IntMask_MSB 0xA
225#define QIB_7322_IntMask_RcvAvail10IntMask_RMASK 0x1
226#define QIB_7322_IntMask_RcvAvail9IntMask_LSB 0x9
227#define QIB_7322_IntMask_RcvAvail9IntMask_MSB 0x9
228#define QIB_7322_IntMask_RcvAvail9IntMask_RMASK 0x1
229#define QIB_7322_IntMask_RcvAvail8IntMask_LSB 0x8
230#define QIB_7322_IntMask_RcvAvail8IntMask_MSB 0x8
231#define QIB_7322_IntMask_RcvAvail8IntMask_RMASK 0x1
232#define QIB_7322_IntMask_RcvAvail7IntMask_LSB 0x7
233#define QIB_7322_IntMask_RcvAvail7IntMask_MSB 0x7
234#define QIB_7322_IntMask_RcvAvail7IntMask_RMASK 0x1
235#define QIB_7322_IntMask_RcvAvail6IntMask_LSB 0x6
236#define QIB_7322_IntMask_RcvAvail6IntMask_MSB 0x6
237#define QIB_7322_IntMask_RcvAvail6IntMask_RMASK 0x1
238#define QIB_7322_IntMask_RcvAvail5IntMask_LSB 0x5
239#define QIB_7322_IntMask_RcvAvail5IntMask_MSB 0x5
240#define QIB_7322_IntMask_RcvAvail5IntMask_RMASK 0x1
241#define QIB_7322_IntMask_RcvAvail4IntMask_LSB 0x4
242#define QIB_7322_IntMask_RcvAvail4IntMask_MSB 0x4
243#define QIB_7322_IntMask_RcvAvail4IntMask_RMASK 0x1
244#define QIB_7322_IntMask_RcvAvail3IntMask_LSB 0x3
245#define QIB_7322_IntMask_RcvAvail3IntMask_MSB 0x3
246#define QIB_7322_IntMask_RcvAvail3IntMask_RMASK 0x1
247#define QIB_7322_IntMask_RcvAvail2IntMask_LSB 0x2
248#define QIB_7322_IntMask_RcvAvail2IntMask_MSB 0x2
249#define QIB_7322_IntMask_RcvAvail2IntMask_RMASK 0x1
250#define QIB_7322_IntMask_RcvAvail1IntMask_LSB 0x1
251#define QIB_7322_IntMask_RcvAvail1IntMask_MSB 0x1
252#define QIB_7322_IntMask_RcvAvail1IntMask_RMASK 0x1
253#define QIB_7322_IntMask_RcvAvail0IntMask_LSB 0x0
254#define QIB_7322_IntMask_RcvAvail0IntMask_MSB 0x0
255#define QIB_7322_IntMask_RcvAvail0IntMask_RMASK 0x1
256
257#define QIB_7322_IntStatus_OFFS 0x70
258#define QIB_7322_IntStatus_DEF 0x0000000000000000
259#define QIB_7322_IntStatus_SDmaInt_1_LSB 0x3F
260#define QIB_7322_IntStatus_SDmaInt_1_MSB 0x3F
261#define QIB_7322_IntStatus_SDmaInt_1_RMASK 0x1
262#define QIB_7322_IntStatus_SDmaInt_0_LSB 0x3E
263#define QIB_7322_IntStatus_SDmaInt_0_MSB 0x3E
264#define QIB_7322_IntStatus_SDmaInt_0_RMASK 0x1
265#define QIB_7322_IntStatus_SDmaProgressInt_1_LSB 0x3D
266#define QIB_7322_IntStatus_SDmaProgressInt_1_MSB 0x3D
267#define QIB_7322_IntStatus_SDmaProgressInt_1_RMASK 0x1
268#define QIB_7322_IntStatus_SDmaProgressInt_0_LSB 0x3C
269#define QIB_7322_IntStatus_SDmaProgressInt_0_MSB 0x3C
270#define QIB_7322_IntStatus_SDmaProgressInt_0_RMASK 0x1
271#define QIB_7322_IntStatus_SDmaIdleInt_1_LSB 0x3B
272#define QIB_7322_IntStatus_SDmaIdleInt_1_MSB 0x3B
273#define QIB_7322_IntStatus_SDmaIdleInt_1_RMASK 0x1
274#define QIB_7322_IntStatus_SDmaIdleInt_0_LSB 0x3A
275#define QIB_7322_IntStatus_SDmaIdleInt_0_MSB 0x3A
276#define QIB_7322_IntStatus_SDmaIdleInt_0_RMASK 0x1
277#define QIB_7322_IntStatus_SDmaCleanupDone_1_LSB 0x39
278#define QIB_7322_IntStatus_SDmaCleanupDone_1_MSB 0x39
279#define QIB_7322_IntStatus_SDmaCleanupDone_1_RMASK 0x1
280#define QIB_7322_IntStatus_SDmaCleanupDone_0_LSB 0x38
281#define QIB_7322_IntStatus_SDmaCleanupDone_0_MSB 0x38
282#define QIB_7322_IntStatus_SDmaCleanupDone_0_RMASK 0x1
283#define QIB_7322_IntStatus_RcvUrg17_LSB 0x31
284#define QIB_7322_IntStatus_RcvUrg17_MSB 0x31
285#define QIB_7322_IntStatus_RcvUrg17_RMASK 0x1
286#define QIB_7322_IntStatus_RcvUrg16_LSB 0x30
287#define QIB_7322_IntStatus_RcvUrg16_MSB 0x30
288#define QIB_7322_IntStatus_RcvUrg16_RMASK 0x1
289#define QIB_7322_IntStatus_RcvUrg15_LSB 0x2F
290#define QIB_7322_IntStatus_RcvUrg15_MSB 0x2F
291#define QIB_7322_IntStatus_RcvUrg15_RMASK 0x1
292#define QIB_7322_IntStatus_RcvUrg14_LSB 0x2E
293#define QIB_7322_IntStatus_RcvUrg14_MSB 0x2E
294#define QIB_7322_IntStatus_RcvUrg14_RMASK 0x1
295#define QIB_7322_IntStatus_RcvUrg13_LSB 0x2D
296#define QIB_7322_IntStatus_RcvUrg13_MSB 0x2D
297#define QIB_7322_IntStatus_RcvUrg13_RMASK 0x1
298#define QIB_7322_IntStatus_RcvUrg12_LSB 0x2C
299#define QIB_7322_IntStatus_RcvUrg12_MSB 0x2C
300#define QIB_7322_IntStatus_RcvUrg12_RMASK 0x1
301#define QIB_7322_IntStatus_RcvUrg11_LSB 0x2B
302#define QIB_7322_IntStatus_RcvUrg11_MSB 0x2B
303#define QIB_7322_IntStatus_RcvUrg11_RMASK 0x1
304#define QIB_7322_IntStatus_RcvUrg10_LSB 0x2A
305#define QIB_7322_IntStatus_RcvUrg10_MSB 0x2A
306#define QIB_7322_IntStatus_RcvUrg10_RMASK 0x1
307#define QIB_7322_IntStatus_RcvUrg9_LSB 0x29
308#define QIB_7322_IntStatus_RcvUrg9_MSB 0x29
309#define QIB_7322_IntStatus_RcvUrg9_RMASK 0x1
310#define QIB_7322_IntStatus_RcvUrg8_LSB 0x28
311#define QIB_7322_IntStatus_RcvUrg8_MSB 0x28
312#define QIB_7322_IntStatus_RcvUrg8_RMASK 0x1
313#define QIB_7322_IntStatus_RcvUrg7_LSB 0x27
314#define QIB_7322_IntStatus_RcvUrg7_MSB 0x27
315#define QIB_7322_IntStatus_RcvUrg7_RMASK 0x1
316#define QIB_7322_IntStatus_RcvUrg6_LSB 0x26
317#define QIB_7322_IntStatus_RcvUrg6_MSB 0x26
318#define QIB_7322_IntStatus_RcvUrg6_RMASK 0x1
319#define QIB_7322_IntStatus_RcvUrg5_LSB 0x25
320#define QIB_7322_IntStatus_RcvUrg5_MSB 0x25
321#define QIB_7322_IntStatus_RcvUrg5_RMASK 0x1
322#define QIB_7322_IntStatus_RcvUrg4_LSB 0x24
323#define QIB_7322_IntStatus_RcvUrg4_MSB 0x24
324#define QIB_7322_IntStatus_RcvUrg4_RMASK 0x1
325#define QIB_7322_IntStatus_RcvUrg3_LSB 0x23
326#define QIB_7322_IntStatus_RcvUrg3_MSB 0x23
327#define QIB_7322_IntStatus_RcvUrg3_RMASK 0x1
328#define QIB_7322_IntStatus_RcvUrg2_LSB 0x22
329#define QIB_7322_IntStatus_RcvUrg2_MSB 0x22
330#define QIB_7322_IntStatus_RcvUrg2_RMASK 0x1
331#define QIB_7322_IntStatus_RcvUrg1_LSB 0x21
332#define QIB_7322_IntStatus_RcvUrg1_MSB 0x21
333#define QIB_7322_IntStatus_RcvUrg1_RMASK 0x1
334#define QIB_7322_IntStatus_RcvUrg0_LSB 0x20
335#define QIB_7322_IntStatus_RcvUrg0_MSB 0x20
336#define QIB_7322_IntStatus_RcvUrg0_RMASK 0x1
337#define QIB_7322_IntStatus_Err_1_LSB 0x1F
338#define QIB_7322_IntStatus_Err_1_MSB 0x1F
339#define QIB_7322_IntStatus_Err_1_RMASK 0x1
340#define QIB_7322_IntStatus_Err_0_LSB 0x1E
341#define QIB_7322_IntStatus_Err_0_MSB 0x1E
342#define QIB_7322_IntStatus_Err_0_RMASK 0x1
343#define QIB_7322_IntStatus_Err_LSB 0x1D
344#define QIB_7322_IntStatus_Err_MSB 0x1D
345#define QIB_7322_IntStatus_Err_RMASK 0x1
346#define QIB_7322_IntStatus_AssertGPIO_LSB 0x1C
347#define QIB_7322_IntStatus_AssertGPIO_MSB 0x1C
348#define QIB_7322_IntStatus_AssertGPIO_RMASK 0x1
349#define QIB_7322_IntStatus_SendDone_1_LSB 0x19
350#define QIB_7322_IntStatus_SendDone_1_MSB 0x19
351#define QIB_7322_IntStatus_SendDone_1_RMASK 0x1
352#define QIB_7322_IntStatus_SendDone_0_LSB 0x18
353#define QIB_7322_IntStatus_SendDone_0_MSB 0x18
354#define QIB_7322_IntStatus_SendDone_0_RMASK 0x1
355#define QIB_7322_IntStatus_SendBufAvail_LSB 0x17
356#define QIB_7322_IntStatus_SendBufAvail_MSB 0x17
357#define QIB_7322_IntStatus_SendBufAvail_RMASK 0x1
358#define QIB_7322_IntStatus_RcvAvail17_LSB 0x11
359#define QIB_7322_IntStatus_RcvAvail17_MSB 0x11
360#define QIB_7322_IntStatus_RcvAvail17_RMASK 0x1
361#define QIB_7322_IntStatus_RcvAvail16_LSB 0x10
362#define QIB_7322_IntStatus_RcvAvail16_MSB 0x10
363#define QIB_7322_IntStatus_RcvAvail16_RMASK 0x1
364#define QIB_7322_IntStatus_RcvAvail15_LSB 0xF
365#define QIB_7322_IntStatus_RcvAvail15_MSB 0xF
366#define QIB_7322_IntStatus_RcvAvail15_RMASK 0x1
367#define QIB_7322_IntStatus_RcvAvail14_LSB 0xE
368#define QIB_7322_IntStatus_RcvAvail14_MSB 0xE
369#define QIB_7322_IntStatus_RcvAvail14_RMASK 0x1
370#define QIB_7322_IntStatus_RcvAvail13_LSB 0xD
371#define QIB_7322_IntStatus_RcvAvail13_MSB 0xD
372#define QIB_7322_IntStatus_RcvAvail13_RMASK 0x1
373#define QIB_7322_IntStatus_RcvAvail12_LSB 0xC
374#define QIB_7322_IntStatus_RcvAvail12_MSB 0xC
375#define QIB_7322_IntStatus_RcvAvail12_RMASK 0x1
376#define QIB_7322_IntStatus_RcvAvail11_LSB 0xB
377#define QIB_7322_IntStatus_RcvAvail11_MSB 0xB
378#define QIB_7322_IntStatus_RcvAvail11_RMASK 0x1
379#define QIB_7322_IntStatus_RcvAvail10_LSB 0xA
380#define QIB_7322_IntStatus_RcvAvail10_MSB 0xA
381#define QIB_7322_IntStatus_RcvAvail10_RMASK 0x1
382#define QIB_7322_IntStatus_RcvAvail9_LSB 0x9
383#define QIB_7322_IntStatus_RcvAvail9_MSB 0x9
384#define QIB_7322_IntStatus_RcvAvail9_RMASK 0x1
385#define QIB_7322_IntStatus_RcvAvail8_LSB 0x8
386#define QIB_7322_IntStatus_RcvAvail8_MSB 0x8
387#define QIB_7322_IntStatus_RcvAvail8_RMASK 0x1
388#define QIB_7322_IntStatus_RcvAvail7_LSB 0x7
389#define QIB_7322_IntStatus_RcvAvail7_MSB 0x7
390#define QIB_7322_IntStatus_RcvAvail7_RMASK 0x1
391#define QIB_7322_IntStatus_RcvAvail6_LSB 0x6
392#define QIB_7322_IntStatus_RcvAvail6_MSB 0x6
393#define QIB_7322_IntStatus_RcvAvail6_RMASK 0x1
394#define QIB_7322_IntStatus_RcvAvail5_LSB 0x5
395#define QIB_7322_IntStatus_RcvAvail5_MSB 0x5
396#define QIB_7322_IntStatus_RcvAvail5_RMASK 0x1
397#define QIB_7322_IntStatus_RcvAvail4_LSB 0x4
398#define QIB_7322_IntStatus_RcvAvail4_MSB 0x4
399#define QIB_7322_IntStatus_RcvAvail4_RMASK 0x1
400#define QIB_7322_IntStatus_RcvAvail3_LSB 0x3
401#define QIB_7322_IntStatus_RcvAvail3_MSB 0x3
402#define QIB_7322_IntStatus_RcvAvail3_RMASK 0x1
403#define QIB_7322_IntStatus_RcvAvail2_LSB 0x2
404#define QIB_7322_IntStatus_RcvAvail2_MSB 0x2
405#define QIB_7322_IntStatus_RcvAvail2_RMASK 0x1
406#define QIB_7322_IntStatus_RcvAvail1_LSB 0x1
407#define QIB_7322_IntStatus_RcvAvail1_MSB 0x1
408#define QIB_7322_IntStatus_RcvAvail1_RMASK 0x1
409#define QIB_7322_IntStatus_RcvAvail0_LSB 0x0
410#define QIB_7322_IntStatus_RcvAvail0_MSB 0x0
411#define QIB_7322_IntStatus_RcvAvail0_RMASK 0x1
412
413#define QIB_7322_IntClear_OFFS 0x78
414#define QIB_7322_IntClear_DEF 0x0000000000000000
415#define QIB_7322_IntClear_SDmaIntClear_1_LSB 0x3F
416#define QIB_7322_IntClear_SDmaIntClear_1_MSB 0x3F
417#define QIB_7322_IntClear_SDmaIntClear_1_RMASK 0x1
418#define QIB_7322_IntClear_SDmaIntClear_0_LSB 0x3E
419#define QIB_7322_IntClear_SDmaIntClear_0_MSB 0x3E
420#define QIB_7322_IntClear_SDmaIntClear_0_RMASK 0x1
421#define QIB_7322_IntClear_SDmaProgressIntClear_1_LSB 0x3D
422#define QIB_7322_IntClear_SDmaProgressIntClear_1_MSB 0x3D
423#define QIB_7322_IntClear_SDmaProgressIntClear_1_RMASK 0x1
424#define QIB_7322_IntClear_SDmaProgressIntClear_0_LSB 0x3C
425#define QIB_7322_IntClear_SDmaProgressIntClear_0_MSB 0x3C
426#define QIB_7322_IntClear_SDmaProgressIntClear_0_RMASK 0x1
427#define QIB_7322_IntClear_SDmaIdleIntClear_1_LSB 0x3B
428#define QIB_7322_IntClear_SDmaIdleIntClear_1_MSB 0x3B
429#define QIB_7322_IntClear_SDmaIdleIntClear_1_RMASK 0x1
430#define QIB_7322_IntClear_SDmaIdleIntClear_0_LSB 0x3A
431#define QIB_7322_IntClear_SDmaIdleIntClear_0_MSB 0x3A
432#define QIB_7322_IntClear_SDmaIdleIntClear_0_RMASK 0x1
433#define QIB_7322_IntClear_SDmaCleanupDoneClear_1_LSB 0x39
434#define QIB_7322_IntClear_SDmaCleanupDoneClear_1_MSB 0x39
435#define QIB_7322_IntClear_SDmaCleanupDoneClear_1_RMASK 0x1
436#define QIB_7322_IntClear_SDmaCleanupDoneClear_0_LSB 0x38
437#define QIB_7322_IntClear_SDmaCleanupDoneClear_0_MSB 0x38
438#define QIB_7322_IntClear_SDmaCleanupDoneClear_0_RMASK 0x1
439#define QIB_7322_IntClear_RcvUrg17IntClear_LSB 0x31
440#define QIB_7322_IntClear_RcvUrg17IntClear_MSB 0x31
441#define QIB_7322_IntClear_RcvUrg17IntClear_RMASK 0x1
442#define QIB_7322_IntClear_RcvUrg16IntClear_LSB 0x30
443#define QIB_7322_IntClear_RcvUrg16IntClear_MSB 0x30
444#define QIB_7322_IntClear_RcvUrg16IntClear_RMASK 0x1
445#define QIB_7322_IntClear_RcvUrg15IntClear_LSB 0x2F
446#define QIB_7322_IntClear_RcvUrg15IntClear_MSB 0x2F
447#define QIB_7322_IntClear_RcvUrg15IntClear_RMASK 0x1
448#define QIB_7322_IntClear_RcvUrg14IntClear_LSB 0x2E
449#define QIB_7322_IntClear_RcvUrg14IntClear_MSB 0x2E
450#define QIB_7322_IntClear_RcvUrg14IntClear_RMASK 0x1
451#define QIB_7322_IntClear_RcvUrg13IntClear_LSB 0x2D
452#define QIB_7322_IntClear_RcvUrg13IntClear_MSB 0x2D
453#define QIB_7322_IntClear_RcvUrg13IntClear_RMASK 0x1
454#define QIB_7322_IntClear_RcvUrg12IntClear_LSB 0x2C
455#define QIB_7322_IntClear_RcvUrg12IntClear_MSB 0x2C
456#define QIB_7322_IntClear_RcvUrg12IntClear_RMASK 0x1
457#define QIB_7322_IntClear_RcvUrg11IntClear_LSB 0x2B
458#define QIB_7322_IntClear_RcvUrg11IntClear_MSB 0x2B
459#define QIB_7322_IntClear_RcvUrg11IntClear_RMASK 0x1
460#define QIB_7322_IntClear_RcvUrg10IntClear_LSB 0x2A
461#define QIB_7322_IntClear_RcvUrg10IntClear_MSB 0x2A
462#define QIB_7322_IntClear_RcvUrg10IntClear_RMASK 0x1
463#define QIB_7322_IntClear_RcvUrg9IntClear_LSB 0x29
464#define QIB_7322_IntClear_RcvUrg9IntClear_MSB 0x29
465#define QIB_7322_IntClear_RcvUrg9IntClear_RMASK 0x1
466#define QIB_7322_IntClear_RcvUrg8IntClear_LSB 0x28
467#define QIB_7322_IntClear_RcvUrg8IntClear_MSB 0x28
468#define QIB_7322_IntClear_RcvUrg8IntClear_RMASK 0x1
469#define QIB_7322_IntClear_RcvUrg7IntClear_LSB 0x27
470#define QIB_7322_IntClear_RcvUrg7IntClear_MSB 0x27
471#define QIB_7322_IntClear_RcvUrg7IntClear_RMASK 0x1
472#define QIB_7322_IntClear_RcvUrg6IntClear_LSB 0x26
473#define QIB_7322_IntClear_RcvUrg6IntClear_MSB 0x26
474#define QIB_7322_IntClear_RcvUrg6IntClear_RMASK 0x1
475#define QIB_7322_IntClear_RcvUrg5IntClear_LSB 0x25
476#define QIB_7322_IntClear_RcvUrg5IntClear_MSB 0x25
477#define QIB_7322_IntClear_RcvUrg5IntClear_RMASK 0x1
478#define QIB_7322_IntClear_RcvUrg4IntClear_LSB 0x24
479#define QIB_7322_IntClear_RcvUrg4IntClear_MSB 0x24
480#define QIB_7322_IntClear_RcvUrg4IntClear_RMASK 0x1
481#define QIB_7322_IntClear_RcvUrg3IntClear_LSB 0x23
482#define QIB_7322_IntClear_RcvUrg3IntClear_MSB 0x23
483#define QIB_7322_IntClear_RcvUrg3IntClear_RMASK 0x1
484#define QIB_7322_IntClear_RcvUrg2IntClear_LSB 0x22
485#define QIB_7322_IntClear_RcvUrg2IntClear_MSB 0x22
486#define QIB_7322_IntClear_RcvUrg2IntClear_RMASK 0x1
487#define QIB_7322_IntClear_RcvUrg1IntClear_LSB 0x21
488#define QIB_7322_IntClear_RcvUrg1IntClear_MSB 0x21
489#define QIB_7322_IntClear_RcvUrg1IntClear_RMASK 0x1
490#define QIB_7322_IntClear_RcvUrg0IntClear_LSB 0x20
491#define QIB_7322_IntClear_RcvUrg0IntClear_MSB 0x20
492#define QIB_7322_IntClear_RcvUrg0IntClear_RMASK 0x1
493#define QIB_7322_IntClear_ErrIntClear_1_LSB 0x1F
494#define QIB_7322_IntClear_ErrIntClear_1_MSB 0x1F
495#define QIB_7322_IntClear_ErrIntClear_1_RMASK 0x1
496#define QIB_7322_IntClear_ErrIntClear_0_LSB 0x1E
497#define QIB_7322_IntClear_ErrIntClear_0_MSB 0x1E
498#define QIB_7322_IntClear_ErrIntClear_0_RMASK 0x1
499#define QIB_7322_IntClear_ErrIntClear_LSB 0x1D
500#define QIB_7322_IntClear_ErrIntClear_MSB 0x1D
501#define QIB_7322_IntClear_ErrIntClear_RMASK 0x1
502#define QIB_7322_IntClear_AssertGPIOIntClear_LSB 0x1C
503#define QIB_7322_IntClear_AssertGPIOIntClear_MSB 0x1C
504#define QIB_7322_IntClear_AssertGPIOIntClear_RMASK 0x1
505#define QIB_7322_IntClear_SendDoneIntClear_1_LSB 0x19
506#define QIB_7322_IntClear_SendDoneIntClear_1_MSB 0x19
507#define QIB_7322_IntClear_SendDoneIntClear_1_RMASK 0x1
508#define QIB_7322_IntClear_SendDoneIntClear_0_LSB 0x18
509#define QIB_7322_IntClear_SendDoneIntClear_0_MSB 0x18
510#define QIB_7322_IntClear_SendDoneIntClear_0_RMASK 0x1
511#define QIB_7322_IntClear_SendBufAvailIntClear_LSB 0x17
512#define QIB_7322_IntClear_SendBufAvailIntClear_MSB 0x17
513#define QIB_7322_IntClear_SendBufAvailIntClear_RMASK 0x1
514#define QIB_7322_IntClear_RcvAvail17IntClear_LSB 0x11
515#define QIB_7322_IntClear_RcvAvail17IntClear_MSB 0x11
516#define QIB_7322_IntClear_RcvAvail17IntClear_RMASK 0x1
517#define QIB_7322_IntClear_RcvAvail16IntClear_LSB 0x10
518#define QIB_7322_IntClear_RcvAvail16IntClear_MSB 0x10
519#define QIB_7322_IntClear_RcvAvail16IntClear_RMASK 0x1
520#define QIB_7322_IntClear_RcvAvail15IntClear_LSB 0xF
521#define QIB_7322_IntClear_RcvAvail15IntClear_MSB 0xF
522#define QIB_7322_IntClear_RcvAvail15IntClear_RMASK 0x1
523#define QIB_7322_IntClear_RcvAvail14IntClear_LSB 0xE
524#define QIB_7322_IntClear_RcvAvail14IntClear_MSB 0xE
525#define QIB_7322_IntClear_RcvAvail14IntClear_RMASK 0x1
526#define QIB_7322_IntClear_RcvAvail13IntClear_LSB 0xD
527#define QIB_7322_IntClear_RcvAvail13IntClear_MSB 0xD
528#define QIB_7322_IntClear_RcvAvail13IntClear_RMASK 0x1
529#define QIB_7322_IntClear_RcvAvail12IntClear_LSB 0xC
530#define QIB_7322_IntClear_RcvAvail12IntClear_MSB 0xC
531#define QIB_7322_IntClear_RcvAvail12IntClear_RMASK 0x1
532#define QIB_7322_IntClear_RcvAvail11IntClear_LSB 0xB
533#define QIB_7322_IntClear_RcvAvail11IntClear_MSB 0xB
534#define QIB_7322_IntClear_RcvAvail11IntClear_RMASK 0x1
535#define QIB_7322_IntClear_RcvAvail10IntClear_LSB 0xA
536#define QIB_7322_IntClear_RcvAvail10IntClear_MSB 0xA
537#define QIB_7322_IntClear_RcvAvail10IntClear_RMASK 0x1
538#define QIB_7322_IntClear_RcvAvail9IntClear_LSB 0x9
539#define QIB_7322_IntClear_RcvAvail9IntClear_MSB 0x9
540#define QIB_7322_IntClear_RcvAvail9IntClear_RMASK 0x1
541#define QIB_7322_IntClear_RcvAvail8IntClear_LSB 0x8
542#define QIB_7322_IntClear_RcvAvail8IntClear_MSB 0x8
543#define QIB_7322_IntClear_RcvAvail8IntClear_RMASK 0x1
544#define QIB_7322_IntClear_RcvAvail7IntClear_LSB 0x7
545#define QIB_7322_IntClear_RcvAvail7IntClear_MSB 0x7
546#define QIB_7322_IntClear_RcvAvail7IntClear_RMASK 0x1
547#define QIB_7322_IntClear_RcvAvail6IntClear_LSB 0x6
548#define QIB_7322_IntClear_RcvAvail6IntClear_MSB 0x6
549#define QIB_7322_IntClear_RcvAvail6IntClear_RMASK 0x1
550#define QIB_7322_IntClear_RcvAvail5IntClear_LSB 0x5
551#define QIB_7322_IntClear_RcvAvail5IntClear_MSB 0x5
552#define QIB_7322_IntClear_RcvAvail5IntClear_RMASK 0x1
553#define QIB_7322_IntClear_RcvAvail4IntClear_LSB 0x4
554#define QIB_7322_IntClear_RcvAvail4IntClear_MSB 0x4
555#define QIB_7322_IntClear_RcvAvail4IntClear_RMASK 0x1
556#define QIB_7322_IntClear_RcvAvail3IntClear_LSB 0x3
557#define QIB_7322_IntClear_RcvAvail3IntClear_MSB 0x3
558#define QIB_7322_IntClear_RcvAvail3IntClear_RMASK 0x1
559#define QIB_7322_IntClear_RcvAvail2IntClear_LSB 0x2
560#define QIB_7322_IntClear_RcvAvail2IntClear_MSB 0x2
561#define QIB_7322_IntClear_RcvAvail2IntClear_RMASK 0x1
562#define QIB_7322_IntClear_RcvAvail1IntClear_LSB 0x1
563#define QIB_7322_IntClear_RcvAvail1IntClear_MSB 0x1
564#define QIB_7322_IntClear_RcvAvail1IntClear_RMASK 0x1
565#define QIB_7322_IntClear_RcvAvail0IntClear_LSB 0x0
566#define QIB_7322_IntClear_RcvAvail0IntClear_MSB 0x0
567#define QIB_7322_IntClear_RcvAvail0IntClear_RMASK 0x1
568
569#define QIB_7322_ErrMask_OFFS 0x80
570#define QIB_7322_ErrMask_DEF 0x0000000000000000
571#define QIB_7322_ErrMask_ResetNegatedMask_LSB 0x3F
572#define QIB_7322_ErrMask_ResetNegatedMask_MSB 0x3F
573#define QIB_7322_ErrMask_ResetNegatedMask_RMASK 0x1
574#define QIB_7322_ErrMask_HardwareErrMask_LSB 0x3E
575#define QIB_7322_ErrMask_HardwareErrMask_MSB 0x3E
576#define QIB_7322_ErrMask_HardwareErrMask_RMASK 0x1
577#define QIB_7322_ErrMask_InvalidAddrErrMask_LSB 0x3D
578#define QIB_7322_ErrMask_InvalidAddrErrMask_MSB 0x3D
579#define QIB_7322_ErrMask_InvalidAddrErrMask_RMASK 0x1
580#define QIB_7322_ErrMask_SDmaVL15ErrMask_LSB 0x38
581#define QIB_7322_ErrMask_SDmaVL15ErrMask_MSB 0x38
582#define QIB_7322_ErrMask_SDmaVL15ErrMask_RMASK 0x1
583#define QIB_7322_ErrMask_SBufVL15MisUseErrMask_LSB 0x37
584#define QIB_7322_ErrMask_SBufVL15MisUseErrMask_MSB 0x37
585#define QIB_7322_ErrMask_SBufVL15MisUseErrMask_RMASK 0x1
586#define QIB_7322_ErrMask_InvalidEEPCmdMask_LSB 0x35
587#define QIB_7322_ErrMask_InvalidEEPCmdMask_MSB 0x35
588#define QIB_7322_ErrMask_InvalidEEPCmdMask_RMASK 0x1
589#define QIB_7322_ErrMask_RcvContextShareErrMask_LSB 0x34
590#define QIB_7322_ErrMask_RcvContextShareErrMask_MSB 0x34
591#define QIB_7322_ErrMask_RcvContextShareErrMask_RMASK 0x1
592#define QIB_7322_ErrMask_SendVLMismatchErrMask_LSB 0x24
593#define QIB_7322_ErrMask_SendVLMismatchErrMask_MSB 0x24
594#define QIB_7322_ErrMask_SendVLMismatchErrMask_RMASK 0x1
595#define QIB_7322_ErrMask_SendArmLaunchErrMask_LSB 0x23
596#define QIB_7322_ErrMask_SendArmLaunchErrMask_MSB 0x23
597#define QIB_7322_ErrMask_SendArmLaunchErrMask_RMASK 0x1
598#define QIB_7322_ErrMask_SendSpecialTriggerErrMask_LSB 0x1B
599#define QIB_7322_ErrMask_SendSpecialTriggerErrMask_MSB 0x1B
600#define QIB_7322_ErrMask_SendSpecialTriggerErrMask_RMASK 0x1
601#define QIB_7322_ErrMask_SDmaWrongPortErrMask_LSB 0x1A
602#define QIB_7322_ErrMask_SDmaWrongPortErrMask_MSB 0x1A
603#define QIB_7322_ErrMask_SDmaWrongPortErrMask_RMASK 0x1
604#define QIB_7322_ErrMask_SDmaBufMaskDuplicateErrMask_LSB 0x19
605#define QIB_7322_ErrMask_SDmaBufMaskDuplicateErrMask_MSB 0x19
606#define QIB_7322_ErrMask_SDmaBufMaskDuplicateErrMask_RMASK 0x1
607#define QIB_7322_ErrMask_RcvHdrFullErrMask_LSB 0xD
608#define QIB_7322_ErrMask_RcvHdrFullErrMask_MSB 0xD
609#define QIB_7322_ErrMask_RcvHdrFullErrMask_RMASK 0x1
610#define QIB_7322_ErrMask_RcvEgrFullErrMask_LSB 0xC
611#define QIB_7322_ErrMask_RcvEgrFullErrMask_MSB 0xC
612#define QIB_7322_ErrMask_RcvEgrFullErrMask_RMASK 0x1
613
614#define QIB_7322_ErrStatus_OFFS 0x88
615#define QIB_7322_ErrStatus_DEF 0x0000000000000000
616#define QIB_7322_ErrStatus_ResetNegated_LSB 0x3F
617#define QIB_7322_ErrStatus_ResetNegated_MSB 0x3F
618#define QIB_7322_ErrStatus_ResetNegated_RMASK 0x1
619#define QIB_7322_ErrStatus_HardwareErr_LSB 0x3E
620#define QIB_7322_ErrStatus_HardwareErr_MSB 0x3E
621#define QIB_7322_ErrStatus_HardwareErr_RMASK 0x1
622#define QIB_7322_ErrStatus_InvalidAddrErr_LSB 0x3D
623#define QIB_7322_ErrStatus_InvalidAddrErr_MSB 0x3D
624#define QIB_7322_ErrStatus_InvalidAddrErr_RMASK 0x1
625#define QIB_7322_ErrStatus_SDmaVL15Err_LSB 0x38
626#define QIB_7322_ErrStatus_SDmaVL15Err_MSB 0x38
627#define QIB_7322_ErrStatus_SDmaVL15Err_RMASK 0x1
628#define QIB_7322_ErrStatus_SBufVL15MisUseErr_LSB 0x37
629#define QIB_7322_ErrStatus_SBufVL15MisUseErr_MSB 0x37
630#define QIB_7322_ErrStatus_SBufVL15MisUseErr_RMASK 0x1
631#define QIB_7322_ErrStatus_InvalidEEPCmdErr_LSB 0x35
632#define QIB_7322_ErrStatus_InvalidEEPCmdErr_MSB 0x35
633#define QIB_7322_ErrStatus_InvalidEEPCmdErr_RMASK 0x1
634#define QIB_7322_ErrStatus_RcvContextShareErr_LSB 0x34
635#define QIB_7322_ErrStatus_RcvContextShareErr_MSB 0x34
636#define QIB_7322_ErrStatus_RcvContextShareErr_RMASK 0x1
637#define QIB_7322_ErrStatus_SendVLMismatchErr_LSB 0x24
638#define QIB_7322_ErrStatus_SendVLMismatchErr_MSB 0x24
639#define QIB_7322_ErrStatus_SendVLMismatchErr_RMASK 0x1
640#define QIB_7322_ErrStatus_SendArmLaunchErr_LSB 0x23
641#define QIB_7322_ErrStatus_SendArmLaunchErr_MSB 0x23
642#define QIB_7322_ErrStatus_SendArmLaunchErr_RMASK 0x1
643#define QIB_7322_ErrStatus_SendSpecialTriggerErr_LSB 0x1B
644#define QIB_7322_ErrStatus_SendSpecialTriggerErr_MSB 0x1B
645#define QIB_7322_ErrStatus_SendSpecialTriggerErr_RMASK 0x1
646#define QIB_7322_ErrStatus_SDmaWrongPortErr_LSB 0x1A
647#define QIB_7322_ErrStatus_SDmaWrongPortErr_MSB 0x1A
648#define QIB_7322_ErrStatus_SDmaWrongPortErr_RMASK 0x1
649#define QIB_7322_ErrStatus_SDmaBufMaskDuplicateErr_LSB 0x19
650#define QIB_7322_ErrStatus_SDmaBufMaskDuplicateErr_MSB 0x19
651#define QIB_7322_ErrStatus_SDmaBufMaskDuplicateErr_RMASK 0x1
652#define QIB_7322_ErrStatus_RcvHdrFullErr_LSB 0xD
653#define QIB_7322_ErrStatus_RcvHdrFullErr_MSB 0xD
654#define QIB_7322_ErrStatus_RcvHdrFullErr_RMASK 0x1
655#define QIB_7322_ErrStatus_RcvEgrFullErr_LSB 0xC
656#define QIB_7322_ErrStatus_RcvEgrFullErr_MSB 0xC
657#define QIB_7322_ErrStatus_RcvEgrFullErr_RMASK 0x1
658
659#define QIB_7322_ErrClear_OFFS 0x90
660#define QIB_7322_ErrClear_DEF 0x0000000000000000
661#define QIB_7322_ErrClear_ResetNegatedClear_LSB 0x3F
662#define QIB_7322_ErrClear_ResetNegatedClear_MSB 0x3F
663#define QIB_7322_ErrClear_ResetNegatedClear_RMASK 0x1
664#define QIB_7322_ErrClear_HardwareErrClear_LSB 0x3E
665#define QIB_7322_ErrClear_HardwareErrClear_MSB 0x3E
666#define QIB_7322_ErrClear_HardwareErrClear_RMASK 0x1
667#define QIB_7322_ErrClear_InvalidAddrErrClear_LSB 0x3D
668#define QIB_7322_ErrClear_InvalidAddrErrClear_MSB 0x3D
669#define QIB_7322_ErrClear_InvalidAddrErrClear_RMASK 0x1
670#define QIB_7322_ErrClear_SDmaVL15ErrClear_LSB 0x38
671#define QIB_7322_ErrClear_SDmaVL15ErrClear_MSB 0x38
672#define QIB_7322_ErrClear_SDmaVL15ErrClear_RMASK 0x1
673#define QIB_7322_ErrClear_SBufVL15MisUseErrClear_LSB 0x37
674#define QIB_7322_ErrClear_SBufVL15MisUseErrClear_MSB 0x37
675#define QIB_7322_ErrClear_SBufVL15MisUseErrClear_RMASK 0x1
676#define QIB_7322_ErrClear_InvalidEEPCmdErrClear_LSB 0x35
677#define QIB_7322_ErrClear_InvalidEEPCmdErrClear_MSB 0x35
678#define QIB_7322_ErrClear_InvalidEEPCmdErrClear_RMASK 0x1
679#define QIB_7322_ErrClear_RcvContextShareErrClear_LSB 0x34
680#define QIB_7322_ErrClear_RcvContextShareErrClear_MSB 0x34
681#define QIB_7322_ErrClear_RcvContextShareErrClear_RMASK 0x1
682#define QIB_7322_ErrClear_SendVLMismatchErrMask_LSB 0x24
683#define QIB_7322_ErrClear_SendVLMismatchErrMask_MSB 0x24
684#define QIB_7322_ErrClear_SendVLMismatchErrMask_RMASK 0x1
685#define QIB_7322_ErrClear_SendArmLaunchErrClear_LSB 0x23
686#define QIB_7322_ErrClear_SendArmLaunchErrClear_MSB 0x23
687#define QIB_7322_ErrClear_SendArmLaunchErrClear_RMASK 0x1
688#define QIB_7322_ErrClear_SendSpecialTriggerErrClear_LSB 0x1B
689#define QIB_7322_ErrClear_SendSpecialTriggerErrClear_MSB 0x1B
690#define QIB_7322_ErrClear_SendSpecialTriggerErrClear_RMASK 0x1
691#define QIB_7322_ErrClear_SDmaWrongPortErrClear_LSB 0x1A
692#define QIB_7322_ErrClear_SDmaWrongPortErrClear_MSB 0x1A
693#define QIB_7322_ErrClear_SDmaWrongPortErrClear_RMASK 0x1
694#define QIB_7322_ErrClear_SDmaBufMaskDuplicateErrClear_LSB 0x19
695#define QIB_7322_ErrClear_SDmaBufMaskDuplicateErrClear_MSB 0x19
696#define QIB_7322_ErrClear_SDmaBufMaskDuplicateErrClear_RMASK 0x1
697#define QIB_7322_ErrClear_RcvHdrFullErrClear_LSB 0xD
698#define QIB_7322_ErrClear_RcvHdrFullErrClear_MSB 0xD
699#define QIB_7322_ErrClear_RcvHdrFullErrClear_RMASK 0x1
700#define QIB_7322_ErrClear_RcvEgrFullErrClear_LSB 0xC
701#define QIB_7322_ErrClear_RcvEgrFullErrClear_MSB 0xC
702#define QIB_7322_ErrClear_RcvEgrFullErrClear_RMASK 0x1
703
704#define QIB_7322_HwErrMask_OFFS 0x98
705#define QIB_7322_HwErrMask_DEF 0x0000000000000000
706#define QIB_7322_HwErrMask_IBSerdesPClkNotDetectMask_1_LSB 0x3F
707#define QIB_7322_HwErrMask_IBSerdesPClkNotDetectMask_1_MSB 0x3F
708#define QIB_7322_HwErrMask_IBSerdesPClkNotDetectMask_1_RMASK 0x1
709#define QIB_7322_HwErrMask_IBSerdesPClkNotDetectMask_0_LSB 0x3E
710#define QIB_7322_HwErrMask_IBSerdesPClkNotDetectMask_0_MSB 0x3E
711#define QIB_7322_HwErrMask_IBSerdesPClkNotDetectMask_0_RMASK 0x1
712#define QIB_7322_HwErrMask_PCIESerdesPClkNotDetectMask_LSB 0x37
713#define QIB_7322_HwErrMask_PCIESerdesPClkNotDetectMask_MSB 0x37
714#define QIB_7322_HwErrMask_PCIESerdesPClkNotDetectMask_RMASK 0x1
715#define QIB_7322_HwErrMask_PowerOnBISTFailedMask_LSB 0x36
716#define QIB_7322_HwErrMask_PowerOnBISTFailedMask_MSB 0x36
717#define QIB_7322_HwErrMask_PowerOnBISTFailedMask_RMASK 0x1
718#define QIB_7322_HwErrMask_TempsenseTholdReachedMask_LSB 0x35
719#define QIB_7322_HwErrMask_TempsenseTholdReachedMask_MSB 0x35
720#define QIB_7322_HwErrMask_TempsenseTholdReachedMask_RMASK 0x1
721#define QIB_7322_HwErrMask_MemoryErrMask_LSB 0x30
722#define QIB_7322_HwErrMask_MemoryErrMask_MSB 0x30
723#define QIB_7322_HwErrMask_MemoryErrMask_RMASK 0x1
724#define QIB_7322_HwErrMask_pcie_phy_txParityErr_LSB 0x22
725#define QIB_7322_HwErrMask_pcie_phy_txParityErr_MSB 0x22
726#define QIB_7322_HwErrMask_pcie_phy_txParityErr_RMASK 0x1
727#define QIB_7322_HwErrMask_PCIeBusParityErrMask_LSB 0x1F
728#define QIB_7322_HwErrMask_PCIeBusParityErrMask_MSB 0x21
729#define QIB_7322_HwErrMask_PCIeBusParityErrMask_RMASK 0x7
730#define QIB_7322_HwErrMask_PcieCplTimeoutMask_LSB 0x1E
731#define QIB_7322_HwErrMask_PcieCplTimeoutMask_MSB 0x1E
732#define QIB_7322_HwErrMask_PcieCplTimeoutMask_RMASK 0x1
733#define QIB_7322_HwErrMask_PciePoisonedTLPMask_LSB 0x1D
734#define QIB_7322_HwErrMask_PciePoisonedTLPMask_MSB 0x1D
735#define QIB_7322_HwErrMask_PciePoisonedTLPMask_RMASK 0x1
736#define QIB_7322_HwErrMask_SDmaMemReadErrMask_1_LSB 0x1C
737#define QIB_7322_HwErrMask_SDmaMemReadErrMask_1_MSB 0x1C
738#define QIB_7322_HwErrMask_SDmaMemReadErrMask_1_RMASK 0x1
739#define QIB_7322_HwErrMask_SDmaMemReadErrMask_0_LSB 0x1B
740#define QIB_7322_HwErrMask_SDmaMemReadErrMask_0_MSB 0x1B
741#define QIB_7322_HwErrMask_SDmaMemReadErrMask_0_RMASK 0x1
742#define QIB_7322_HwErrMask_IBCBusFromSPCParityErrMask_1_LSB 0xF
743#define QIB_7322_HwErrMask_IBCBusFromSPCParityErrMask_1_MSB 0xF
744#define QIB_7322_HwErrMask_IBCBusFromSPCParityErrMask_1_RMASK 0x1
745#define QIB_7322_HwErrMask_IBCBusToSPCParityErrMask_1_LSB 0xE
746#define QIB_7322_HwErrMask_IBCBusToSPCParityErrMask_1_MSB 0xE
747#define QIB_7322_HwErrMask_IBCBusToSPCParityErrMask_1_RMASK 0x1
748#define QIB_7322_HwErrMask_IBCBusFromSPCParityErrMask_0_LSB 0xD
749#define QIB_7322_HwErrMask_IBCBusFromSPCParityErrMask_0_MSB 0xD
750#define QIB_7322_HwErrMask_IBCBusFromSPCParityErrMask_0_RMASK 0x1
751#define QIB_7322_HwErrMask_statusValidNoEopMask_LSB 0xC
752#define QIB_7322_HwErrMask_statusValidNoEopMask_MSB 0xC
753#define QIB_7322_HwErrMask_statusValidNoEopMask_RMASK 0x1
754#define QIB_7322_HwErrMask_LATriggeredMask_LSB 0xB
755#define QIB_7322_HwErrMask_LATriggeredMask_MSB 0xB
756#define QIB_7322_HwErrMask_LATriggeredMask_RMASK 0x1
757
758#define QIB_7322_HwErrStatus_OFFS 0xA0
759#define QIB_7322_HwErrStatus_DEF 0x0000000000000000
760#define QIB_7322_HwErrStatus_IBSerdesPClkNotDetect_1_LSB 0x3F
761#define QIB_7322_HwErrStatus_IBSerdesPClkNotDetect_1_MSB 0x3F
762#define QIB_7322_HwErrStatus_IBSerdesPClkNotDetect_1_RMASK 0x1
763#define QIB_7322_HwErrStatus_IBSerdesPClkNotDetect_0_LSB 0x3E
764#define QIB_7322_HwErrStatus_IBSerdesPClkNotDetect_0_MSB 0x3E
765#define QIB_7322_HwErrStatus_IBSerdesPClkNotDetect_0_RMASK 0x1
766#define QIB_7322_HwErrStatus_PCIESerdesPClkNotDetect_LSB 0x37
767#define QIB_7322_HwErrStatus_PCIESerdesPClkNotDetect_MSB 0x37
768#define QIB_7322_HwErrStatus_PCIESerdesPClkNotDetect_RMASK 0x1
769#define QIB_7322_HwErrStatus_PowerOnBISTFailed_LSB 0x36
770#define QIB_7322_HwErrStatus_PowerOnBISTFailed_MSB 0x36
771#define QIB_7322_HwErrStatus_PowerOnBISTFailed_RMASK 0x1
772#define QIB_7322_HwErrStatus_TempsenseTholdReached_LSB 0x35
773#define QIB_7322_HwErrStatus_TempsenseTholdReached_MSB 0x35
774#define QIB_7322_HwErrStatus_TempsenseTholdReached_RMASK 0x1
775#define QIB_7322_HwErrStatus_MemoryErr_LSB 0x30
776#define QIB_7322_HwErrStatus_MemoryErr_MSB 0x30
777#define QIB_7322_HwErrStatus_MemoryErr_RMASK 0x1
778#define QIB_7322_HwErrStatus_pcie_phy_txParityErr_LSB 0x22
779#define QIB_7322_HwErrStatus_pcie_phy_txParityErr_MSB 0x22
780#define QIB_7322_HwErrStatus_pcie_phy_txParityErr_RMASK 0x1
781#define QIB_7322_HwErrStatus_PCIeBusParity_LSB 0x1F
782#define QIB_7322_HwErrStatus_PCIeBusParity_MSB 0x21
783#define QIB_7322_HwErrStatus_PCIeBusParity_RMASK 0x7
784#define QIB_7322_HwErrStatus_PcieCplTimeout_LSB 0x1E
785#define QIB_7322_HwErrStatus_PcieCplTimeout_MSB 0x1E
786#define QIB_7322_HwErrStatus_PcieCplTimeout_RMASK 0x1
787#define QIB_7322_HwErrStatus_PciePoisonedTLP_LSB 0x1D
788#define QIB_7322_HwErrStatus_PciePoisonedTLP_MSB 0x1D
789#define QIB_7322_HwErrStatus_PciePoisonedTLP_RMASK 0x1
790#define QIB_7322_HwErrStatus_SDmaMemReadErr_1_LSB 0x1C
791#define QIB_7322_HwErrStatus_SDmaMemReadErr_1_MSB 0x1C
792#define QIB_7322_HwErrStatus_SDmaMemReadErr_1_RMASK 0x1
793#define QIB_7322_HwErrStatus_SDmaMemReadErr_0_LSB 0x1B
794#define QIB_7322_HwErrStatus_SDmaMemReadErr_0_MSB 0x1B
795#define QIB_7322_HwErrStatus_SDmaMemReadErr_0_RMASK 0x1
796#define QIB_7322_HwErrStatus_IBCBusFromSPCParityErr_1_LSB 0xF
797#define QIB_7322_HwErrStatus_IBCBusFromSPCParityErr_1_MSB 0xF
798#define QIB_7322_HwErrStatus_IBCBusFromSPCParityErr_1_RMASK 0x1
799#define QIB_7322_HwErrStatus_IBCBusToSPCParityErr_1_LSB 0xE
800#define QIB_7322_HwErrStatus_IBCBusToSPCParityErr_1_MSB 0xE
801#define QIB_7322_HwErrStatus_IBCBusToSPCParityErr_1_RMASK 0x1
802#define QIB_7322_HwErrStatus_IBCBusFromSPCParityErr_0_LSB 0xD
803#define QIB_7322_HwErrStatus_IBCBusFromSPCParityErr_0_MSB 0xD
804#define QIB_7322_HwErrStatus_IBCBusFromSPCParityErr_0_RMASK 0x1
805#define QIB_7322_HwErrStatus_statusValidNoEop_LSB 0xC
806#define QIB_7322_HwErrStatus_statusValidNoEop_MSB 0xC
807#define QIB_7322_HwErrStatus_statusValidNoEop_RMASK 0x1
808#define QIB_7322_HwErrStatus_LATriggered_LSB 0xB
809#define QIB_7322_HwErrStatus_LATriggered_MSB 0xB
810#define QIB_7322_HwErrStatus_LATriggered_RMASK 0x1
811
812#define QIB_7322_HwErrClear_OFFS 0xA8
813#define QIB_7322_HwErrClear_DEF 0x0000000000000000
814#define QIB_7322_HwErrClear_IBSerdesPClkNotDetectClear_1_LSB 0x3F
815#define QIB_7322_HwErrClear_IBSerdesPClkNotDetectClear_1_MSB 0x3F
816#define QIB_7322_HwErrClear_IBSerdesPClkNotDetectClear_1_RMASK 0x1
817#define QIB_7322_HwErrClear_IBSerdesPClkNotDetectClear_0_LSB 0x3E
818#define QIB_7322_HwErrClear_IBSerdesPClkNotDetectClear_0_MSB 0x3E
819#define QIB_7322_HwErrClear_IBSerdesPClkNotDetectClear_0_RMASK 0x1
820#define QIB_7322_HwErrClear_PCIESerdesPClkNotDetectClear_LSB 0x37
821#define QIB_7322_HwErrClear_PCIESerdesPClkNotDetectClear_MSB 0x37
822#define QIB_7322_HwErrClear_PCIESerdesPClkNotDetectClear_RMASK 0x1
823#define QIB_7322_HwErrClear_PowerOnBISTFailedClear_LSB 0x36
824#define QIB_7322_HwErrClear_PowerOnBISTFailedClear_MSB 0x36
825#define QIB_7322_HwErrClear_PowerOnBISTFailedClear_RMASK 0x1
826#define QIB_7322_HwErrClear_TempsenseTholdReachedClear_LSB 0x35
827#define QIB_7322_HwErrClear_TempsenseTholdReachedClear_MSB 0x35
828#define QIB_7322_HwErrClear_TempsenseTholdReachedClear_RMASK 0x1
829#define QIB_7322_HwErrClear_MemoryErrClear_LSB 0x30
830#define QIB_7322_HwErrClear_MemoryErrClear_MSB 0x30
831#define QIB_7322_HwErrClear_MemoryErrClear_RMASK 0x1
832#define QIB_7322_HwErrClear_pcie_phy_txParityErr_LSB 0x22
833#define QIB_7322_HwErrClear_pcie_phy_txParityErr_MSB 0x22
834#define QIB_7322_HwErrClear_pcie_phy_txParityErr_RMASK 0x1
835#define QIB_7322_HwErrClear_PCIeBusParityClear_LSB 0x1F
836#define QIB_7322_HwErrClear_PCIeBusParityClear_MSB 0x21
837#define QIB_7322_HwErrClear_PCIeBusParityClear_RMASK 0x7
838#define QIB_7322_HwErrClear_PcieCplTimeoutClear_LSB 0x1E
839#define QIB_7322_HwErrClear_PcieCplTimeoutClear_MSB 0x1E
840#define QIB_7322_HwErrClear_PcieCplTimeoutClear_RMASK 0x1
841#define QIB_7322_HwErrClear_PciePoisonedTLPClear_LSB 0x1D
842#define QIB_7322_HwErrClear_PciePoisonedTLPClear_MSB 0x1D
843#define QIB_7322_HwErrClear_PciePoisonedTLPClear_RMASK 0x1
844#define QIB_7322_HwErrClear_SDmaMemReadErrClear_1_LSB 0x1C
845#define QIB_7322_HwErrClear_SDmaMemReadErrClear_1_MSB 0x1C
846#define QIB_7322_HwErrClear_SDmaMemReadErrClear_1_RMASK 0x1
847#define QIB_7322_HwErrClear_SDmaMemReadErrClear_0_LSB 0x1B
848#define QIB_7322_HwErrClear_SDmaMemReadErrClear_0_MSB 0x1B
849#define QIB_7322_HwErrClear_SDmaMemReadErrClear_0_RMASK 0x1
850#define QIB_7322_HwErrClear_IBCBusFromSPCParityErrClear_1_LSB 0xF
851#define QIB_7322_HwErrClear_IBCBusFromSPCParityErrClear_1_MSB 0xF
852#define QIB_7322_HwErrClear_IBCBusFromSPCParityErrClear_1_RMASK 0x1
853#define QIB_7322_HwErrClear_IBCBusToSPCParityErrClear_1_LSB 0xE
854#define QIB_7322_HwErrClear_IBCBusToSPCParityErrClear_1_MSB 0xE
855#define QIB_7322_HwErrClear_IBCBusToSPCParityErrClear_1_RMASK 0x1
856#define QIB_7322_HwErrClear_IBCBusFromSPCParityErrClear_0_LSB 0xD
857#define QIB_7322_HwErrClear_IBCBusFromSPCParityErrClear_0_MSB 0xD
858#define QIB_7322_HwErrClear_IBCBusFromSPCParityErrClear_0_RMASK 0x1
859#define QIB_7322_HwErrClear_statusValidNoEopClear_LSB 0xC
860#define QIB_7322_HwErrClear_statusValidNoEopClear_MSB 0xC
861#define QIB_7322_HwErrClear_statusValidNoEopClear_RMASK 0x1
862#define QIB_7322_HwErrClear_LATriggeredClear_LSB 0xB
863#define QIB_7322_HwErrClear_LATriggeredClear_MSB 0xB
864#define QIB_7322_HwErrClear_LATriggeredClear_RMASK 0x1
865
866#define QIB_7322_HwDiagCtrl_OFFS 0xB0
867#define QIB_7322_HwDiagCtrl_DEF 0x0000000000000000
868#define QIB_7322_HwDiagCtrl_Diagnostic_LSB 0x3F
869#define QIB_7322_HwDiagCtrl_Diagnostic_MSB 0x3F
870#define QIB_7322_HwDiagCtrl_Diagnostic_RMASK 0x1
871#define QIB_7322_HwDiagCtrl_CounterWrEnable_LSB 0x3D
872#define QIB_7322_HwDiagCtrl_CounterWrEnable_MSB 0x3D
873#define QIB_7322_HwDiagCtrl_CounterWrEnable_RMASK 0x1
874#define QIB_7322_HwDiagCtrl_CounterDisable_LSB 0x3C
875#define QIB_7322_HwDiagCtrl_CounterDisable_MSB 0x3C
876#define QIB_7322_HwDiagCtrl_CounterDisable_RMASK 0x1
877#define QIB_7322_HwDiagCtrl_forcePCIeBusParity_LSB 0x1F
878#define QIB_7322_HwDiagCtrl_forcePCIeBusParity_MSB 0x22
879#define QIB_7322_HwDiagCtrl_forcePCIeBusParity_RMASK 0xF
880#define QIB_7322_HwDiagCtrl_ForceIBCBusFromSPCParityErr_1_LSB 0xF
881#define QIB_7322_HwDiagCtrl_ForceIBCBusFromSPCParityErr_1_MSB 0xF
882#define QIB_7322_HwDiagCtrl_ForceIBCBusFromSPCParityErr_1_RMASK 0x1
883#define QIB_7322_HwDiagCtrl_ForceIBCBusToSPCParityErr_1_LSB 0xE
884#define QIB_7322_HwDiagCtrl_ForceIBCBusToSPCParityErr_1_MSB 0xE
885#define QIB_7322_HwDiagCtrl_ForceIBCBusToSPCParityErr_1_RMASK 0x1
886#define QIB_7322_HwDiagCtrl_ForceIBCBusFromSPCParityErr_0_LSB 0xD
887#define QIB_7322_HwDiagCtrl_ForceIBCBusFromSPCParityErr_0_MSB 0xD
888#define QIB_7322_HwDiagCtrl_ForceIBCBusFromSPCParityErr_0_RMASK 0x1
889#define QIB_7322_HwDiagCtrl_ForceIBCBusToSPCParityErr_0_LSB 0xC
890#define QIB_7322_HwDiagCtrl_ForceIBCBusToSPCParityErr_0_MSB 0xC
891#define QIB_7322_HwDiagCtrl_ForceIBCBusToSPCParityErr_0_RMASK 0x1
892
893#define QIB_7322_EXTStatus_OFFS 0xC0
894#define QIB_7322_EXTStatus_DEF 0x000000000000X000
895#define QIB_7322_EXTStatus_GPIOIn_LSB 0x30
896#define QIB_7322_EXTStatus_GPIOIn_MSB 0x3F
897#define QIB_7322_EXTStatus_GPIOIn_RMASK 0xFFFF
898#define QIB_7322_EXTStatus_MemBISTDisabled_LSB 0xF
899#define QIB_7322_EXTStatus_MemBISTDisabled_MSB 0xF
900#define QIB_7322_EXTStatus_MemBISTDisabled_RMASK 0x1
901#define QIB_7322_EXTStatus_MemBISTEndTest_LSB 0xE
902#define QIB_7322_EXTStatus_MemBISTEndTest_MSB 0xE
903#define QIB_7322_EXTStatus_MemBISTEndTest_RMASK 0x1
904
905#define QIB_7322_EXTCtrl_OFFS 0xC8
906#define QIB_7322_EXTCtrl_DEF 0x0000000000000000
907#define QIB_7322_EXTCtrl_GPIOOe_LSB 0x30
908#define QIB_7322_EXTCtrl_GPIOOe_MSB 0x3F
909#define QIB_7322_EXTCtrl_GPIOOe_RMASK 0xFFFF
910#define QIB_7322_EXTCtrl_GPIOInvert_LSB 0x20
911#define QIB_7322_EXTCtrl_GPIOInvert_MSB 0x2F
912#define QIB_7322_EXTCtrl_GPIOInvert_RMASK 0xFFFF
913#define QIB_7322_EXTCtrl_LEDPort1GreenOn_LSB 0x3
914#define QIB_7322_EXTCtrl_LEDPort1GreenOn_MSB 0x3
915#define QIB_7322_EXTCtrl_LEDPort1GreenOn_RMASK 0x1
916#define QIB_7322_EXTCtrl_LEDPort1YellowOn_LSB 0x2
917#define QIB_7322_EXTCtrl_LEDPort1YellowOn_MSB 0x2
918#define QIB_7322_EXTCtrl_LEDPort1YellowOn_RMASK 0x1
919#define QIB_7322_EXTCtrl_LEDPort0GreenOn_LSB 0x1
920#define QIB_7322_EXTCtrl_LEDPort0GreenOn_MSB 0x1
921#define QIB_7322_EXTCtrl_LEDPort0GreenOn_RMASK 0x1
922#define QIB_7322_EXTCtrl_LEDPort0YellowOn_LSB 0x0
923#define QIB_7322_EXTCtrl_LEDPort0YellowOn_MSB 0x0
924#define QIB_7322_EXTCtrl_LEDPort0YellowOn_RMASK 0x1
925
926#define QIB_7322_GPIOOut_OFFS 0xE0
927#define QIB_7322_GPIOOut_DEF 0x0000000000000000
928
929#define QIB_7322_GPIOMask_OFFS 0xE8
930#define QIB_7322_GPIOMask_DEF 0x0000000000000000
931
932#define QIB_7322_GPIOStatus_OFFS 0xF0
933#define QIB_7322_GPIOStatus_DEF 0x0000000000000000
934
935#define QIB_7322_GPIOClear_OFFS 0xF8
936#define QIB_7322_GPIOClear_DEF 0x0000000000000000
937
938#define QIB_7322_RcvCtrl_OFFS 0x100
939#define QIB_7322_RcvCtrl_DEF 0x0000000000000000
940#define QIB_7322_RcvCtrl_TidReDirect_LSB 0x30
941#define QIB_7322_RcvCtrl_TidReDirect_MSB 0x3F
942#define QIB_7322_RcvCtrl_TidReDirect_RMASK 0xFFFF
943#define QIB_7322_RcvCtrl_TailUpd_LSB 0x2F
944#define QIB_7322_RcvCtrl_TailUpd_MSB 0x2F
945#define QIB_7322_RcvCtrl_TailUpd_RMASK 0x1
946#define QIB_7322_RcvCtrl_XrcTypeCode_LSB 0x2C
947#define QIB_7322_RcvCtrl_XrcTypeCode_MSB 0x2E
948#define QIB_7322_RcvCtrl_XrcTypeCode_RMASK 0x7
949#define QIB_7322_RcvCtrl_TidFlowEnable_LSB 0x2B
950#define QIB_7322_RcvCtrl_TidFlowEnable_MSB 0x2B
951#define QIB_7322_RcvCtrl_TidFlowEnable_RMASK 0x1
952#define QIB_7322_RcvCtrl_ContextCfg_LSB 0x29
953#define QIB_7322_RcvCtrl_ContextCfg_MSB 0x2A
954#define QIB_7322_RcvCtrl_ContextCfg_RMASK 0x3
955#define QIB_7322_RcvCtrl_IntrAvail_LSB 0x14
956#define QIB_7322_RcvCtrl_IntrAvail_MSB 0x25
957#define QIB_7322_RcvCtrl_IntrAvail_RMASK 0x3FFFF
958#define QIB_7322_RcvCtrl_dontDropRHQFull_LSB 0x0
959#define QIB_7322_RcvCtrl_dontDropRHQFull_MSB 0x11
960#define QIB_7322_RcvCtrl_dontDropRHQFull_RMASK 0x3FFFF
961
962#define QIB_7322_RcvHdrSize_OFFS 0x110
963#define QIB_7322_RcvHdrSize_DEF 0x0000000000000000
964
965#define QIB_7322_RcvHdrCnt_OFFS 0x118
966#define QIB_7322_RcvHdrCnt_DEF 0x0000000000000000
967
968#define QIB_7322_RcvHdrEntSize_OFFS 0x120
969#define QIB_7322_RcvHdrEntSize_DEF 0x0000000000000000
970
971#define QIB_7322_RcvTIDBase_OFFS 0x128
972#define QIB_7322_RcvTIDBase_DEF 0x0000000000050000
973
974#define QIB_7322_RcvTIDCnt_OFFS 0x130
975#define QIB_7322_RcvTIDCnt_DEF 0x0000000000000200
976
977#define QIB_7322_RcvEgrBase_OFFS 0x138
978#define QIB_7322_RcvEgrBase_DEF 0x0000000000014000
979
980#define QIB_7322_RcvEgrCnt_OFFS 0x140
981#define QIB_7322_RcvEgrCnt_DEF 0x0000000000001000
982
983#define QIB_7322_RcvBufBase_OFFS 0x148
984#define QIB_7322_RcvBufBase_DEF 0x0000000000080000
985
986#define QIB_7322_RcvBufSize_OFFS 0x150
987#define QIB_7322_RcvBufSize_DEF 0x0000000000005000
988
989#define QIB_7322_RxIntMemBase_OFFS 0x158
990#define QIB_7322_RxIntMemBase_DEF 0x0000000000077000
991
992#define QIB_7322_RxIntMemSize_OFFS 0x160
993#define QIB_7322_RxIntMemSize_DEF 0x0000000000007000
994
995#define QIB_7322_feature_mask_OFFS 0x190
996#define QIB_7322_feature_mask_DEF 0x00000000000000XX
997
998#define QIB_7322_active_feature_mask_OFFS 0x198
999#define QIB_7322_active_feature_mask_DEF 0x00000000000000XX
1000#define QIB_7322_active_feature_mask_Port1_QDR_Enabled_LSB 0x5
1001#define QIB_7322_active_feature_mask_Port1_QDR_Enabled_MSB 0x5
1002#define QIB_7322_active_feature_mask_Port1_QDR_Enabled_RMASK 0x1
1003#define QIB_7322_active_feature_mask_Port1_DDR_Enabled_LSB 0x4
1004#define QIB_7322_active_feature_mask_Port1_DDR_Enabled_MSB 0x4
1005#define QIB_7322_active_feature_mask_Port1_DDR_Enabled_RMASK 0x1
1006#define QIB_7322_active_feature_mask_Port1_SDR_Enabled_LSB 0x3
1007#define QIB_7322_active_feature_mask_Port1_SDR_Enabled_MSB 0x3
1008#define QIB_7322_active_feature_mask_Port1_SDR_Enabled_RMASK 0x1
1009#define QIB_7322_active_feature_mask_Port0_QDR_Enabled_LSB 0x2
1010#define QIB_7322_active_feature_mask_Port0_QDR_Enabled_MSB 0x2
1011#define QIB_7322_active_feature_mask_Port0_QDR_Enabled_RMASK 0x1
1012#define QIB_7322_active_feature_mask_Port0_DDR_Enabled_LSB 0x1
1013#define QIB_7322_active_feature_mask_Port0_DDR_Enabled_MSB 0x1
1014#define QIB_7322_active_feature_mask_Port0_DDR_Enabled_RMASK 0x1
1015#define QIB_7322_active_feature_mask_Port0_SDR_Enabled_LSB 0x0
1016#define QIB_7322_active_feature_mask_Port0_SDR_Enabled_MSB 0x0
1017#define QIB_7322_active_feature_mask_Port0_SDR_Enabled_RMASK 0x1
1018
1019#define QIB_7322_SendCtrl_OFFS 0x1C0
1020#define QIB_7322_SendCtrl_DEF 0x0000000000000000
1021#define QIB_7322_SendCtrl_Disarm_LSB 0x1F
1022#define QIB_7322_SendCtrl_Disarm_MSB 0x1F
1023#define QIB_7322_SendCtrl_Disarm_RMASK 0x1
1024#define QIB_7322_SendCtrl_SendBufAvailPad64Byte_LSB 0x1D
1025#define QIB_7322_SendCtrl_SendBufAvailPad64Byte_MSB 0x1D
1026#define QIB_7322_SendCtrl_SendBufAvailPad64Byte_RMASK 0x1
1027#define QIB_7322_SendCtrl_AvailUpdThld_LSB 0x18
1028#define QIB_7322_SendCtrl_AvailUpdThld_MSB 0x1C
1029#define QIB_7322_SendCtrl_AvailUpdThld_RMASK 0x1F
1030#define QIB_7322_SendCtrl_DisarmSendBuf_LSB 0x10
1031#define QIB_7322_SendCtrl_DisarmSendBuf_MSB 0x17
1032#define QIB_7322_SendCtrl_DisarmSendBuf_RMASK 0xFF
1033#define QIB_7322_SendCtrl_SpecialTriggerEn_LSB 0x4
1034#define QIB_7322_SendCtrl_SpecialTriggerEn_MSB 0x4
1035#define QIB_7322_SendCtrl_SpecialTriggerEn_RMASK 0x1
1036#define QIB_7322_SendCtrl_SendBufAvailUpd_LSB 0x2
1037#define QIB_7322_SendCtrl_SendBufAvailUpd_MSB 0x2
1038#define QIB_7322_SendCtrl_SendBufAvailUpd_RMASK 0x1
1039#define QIB_7322_SendCtrl_SendIntBufAvail_LSB 0x1
1040#define QIB_7322_SendCtrl_SendIntBufAvail_MSB 0x1
1041#define QIB_7322_SendCtrl_SendIntBufAvail_RMASK 0x1
1042
1043#define QIB_7322_SendBufBase_OFFS 0x1C8
1044#define QIB_7322_SendBufBase_DEF 0x0018000000100000
1045#define QIB_7322_SendBufBase_BaseAddr_LargePIO_LSB 0x20
1046#define QIB_7322_SendBufBase_BaseAddr_LargePIO_MSB 0x34
1047#define QIB_7322_SendBufBase_BaseAddr_LargePIO_RMASK 0x1FFFFF
1048#define QIB_7322_SendBufBase_BaseAddr_SmallPIO_LSB 0x0
1049#define QIB_7322_SendBufBase_BaseAddr_SmallPIO_MSB 0x14
1050#define QIB_7322_SendBufBase_BaseAddr_SmallPIO_RMASK 0x1FFFFF
1051
1052#define QIB_7322_SendBufSize_OFFS 0x1D0
1053#define QIB_7322_SendBufSize_DEF 0x0000108000000880
1054#define QIB_7322_SendBufSize_Size_LargePIO_LSB 0x20
1055#define QIB_7322_SendBufSize_Size_LargePIO_MSB 0x2C
1056#define QIB_7322_SendBufSize_Size_LargePIO_RMASK 0x1FFF
1057#define QIB_7322_SendBufSize_Size_SmallPIO_LSB 0x0
1058#define QIB_7322_SendBufSize_Size_SmallPIO_MSB 0xB
1059#define QIB_7322_SendBufSize_Size_SmallPIO_RMASK 0xFFF
1060
1061#define QIB_7322_SendBufCnt_OFFS 0x1D8
1062#define QIB_7322_SendBufCnt_DEF 0x0000002000000080
1063#define QIB_7322_SendBufCnt_Num_LargeBuffers_LSB 0x20
1064#define QIB_7322_SendBufCnt_Num_LargeBuffers_MSB 0x25
1065#define QIB_7322_SendBufCnt_Num_LargeBuffers_RMASK 0x3F
1066#define QIB_7322_SendBufCnt_Num_SmallBuffers_LSB 0x0
1067#define QIB_7322_SendBufCnt_Num_SmallBuffers_MSB 0x8
1068#define QIB_7322_SendBufCnt_Num_SmallBuffers_RMASK 0x1FF
1069
1070#define QIB_7322_SendBufAvailAddr_OFFS 0x1E0
1071#define QIB_7322_SendBufAvailAddr_DEF 0x0000000000000000
1072#define QIB_7322_SendBufAvailAddr_SendBufAvailAddr_LSB 0x6
1073#define QIB_7322_SendBufAvailAddr_SendBufAvailAddr_MSB 0x27
1074#define QIB_7322_SendBufAvailAddr_SendBufAvailAddr_RMASK 0x3FFFFFFFF
1075
1076#define QIB_7322_SendBufErr0_OFFS 0x240
1077#define QIB_7322_SendBufErr0_DEF 0x0000000000000000
1078#define QIB_7322_SendBufErr0_SendBufErr_63_0_LSB 0x0
1079#define QIB_7322_SendBufErr0_SendBufErr_63_0_MSB 0x3F
1080#define QIB_7322_SendBufErr0_SendBufErr_63_0_RMASK 0x0
1081
1082#define QIB_7322_AvailUpdCount_OFFS 0x268
1083#define QIB_7322_AvailUpdCount_DEF 0x0000000000000000
1084#define QIB_7322_AvailUpdCount_AvailUpdCount_LSB 0x0
1085#define QIB_7322_AvailUpdCount_AvailUpdCount_MSB 0x4
1086#define QIB_7322_AvailUpdCount_AvailUpdCount_RMASK 0x1F
1087
1088#define QIB_7322_RcvHdrAddr0_OFFS 0x280
1089#define QIB_7322_RcvHdrAddr0_DEF 0x0000000000000000
1090#define QIB_7322_RcvHdrAddr0_RcvHdrAddr_LSB 0x2
1091#define QIB_7322_RcvHdrAddr0_RcvHdrAddr_MSB 0x27
1092#define QIB_7322_RcvHdrAddr0_RcvHdrAddr_RMASK 0x3FFFFFFFFF
1093
1094#define QIB_7322_RcvHdrTailAddr0_OFFS 0x340
1095#define QIB_7322_RcvHdrTailAddr0_DEF 0x0000000000000000
1096#define QIB_7322_RcvHdrTailAddr0_RcvHdrTailAddr_LSB 0x2
1097#define QIB_7322_RcvHdrTailAddr0_RcvHdrTailAddr_MSB 0x27
1098#define QIB_7322_RcvHdrTailAddr0_RcvHdrTailAddr_RMASK 0x3FFFFFFFFF
1099
1100#define QIB_7322_ahb_access_ctrl_OFFS 0x460
1101#define QIB_7322_ahb_access_ctrl_DEF 0x0000000000000000
1102#define QIB_7322_ahb_access_ctrl_sw_sel_ahb_trgt_LSB 0x1
1103#define QIB_7322_ahb_access_ctrl_sw_sel_ahb_trgt_MSB 0x2
1104#define QIB_7322_ahb_access_ctrl_sw_sel_ahb_trgt_RMASK 0x3
1105#define QIB_7322_ahb_access_ctrl_sw_ahb_sel_LSB 0x0
1106#define QIB_7322_ahb_access_ctrl_sw_ahb_sel_MSB 0x0
1107#define QIB_7322_ahb_access_ctrl_sw_ahb_sel_RMASK 0x1
1108
1109#define QIB_7322_ahb_transaction_reg_OFFS 0x468
1110#define QIB_7322_ahb_transaction_reg_DEF 0x0000000080000000
1111#define QIB_7322_ahb_transaction_reg_ahb_data_LSB 0x20
1112#define QIB_7322_ahb_transaction_reg_ahb_data_MSB 0x3F
1113#define QIB_7322_ahb_transaction_reg_ahb_data_RMASK 0xFFFFFFFF
1114#define QIB_7322_ahb_transaction_reg_ahb_rdy_LSB 0x1F
1115#define QIB_7322_ahb_transaction_reg_ahb_rdy_MSB 0x1F
1116#define QIB_7322_ahb_transaction_reg_ahb_rdy_RMASK 0x1
1117#define QIB_7322_ahb_transaction_reg_ahb_req_err_LSB 0x1E
1118#define QIB_7322_ahb_transaction_reg_ahb_req_err_MSB 0x1E
1119#define QIB_7322_ahb_transaction_reg_ahb_req_err_RMASK 0x1
1120#define QIB_7322_ahb_transaction_reg_write_not_read_LSB 0x1B
1121#define QIB_7322_ahb_transaction_reg_write_not_read_MSB 0x1B
1122#define QIB_7322_ahb_transaction_reg_write_not_read_RMASK 0x1
1123#define QIB_7322_ahb_transaction_reg_ahb_address_LSB 0x10
1124#define QIB_7322_ahb_transaction_reg_ahb_address_MSB 0x1A
1125#define QIB_7322_ahb_transaction_reg_ahb_address_RMASK 0x7FF
1126
1127#define QIB_7322_SPC_JTAG_ACCESS_REG_OFFS 0x470
1128#define QIB_7322_SPC_JTAG_ACCESS_REG_DEF 0x0000000000000001
1129#define QIB_7322_SPC_JTAG_ACCESS_REG_SPC_JTAG_ACCESS_EN_LSB 0xA
1130#define QIB_7322_SPC_JTAG_ACCESS_REG_SPC_JTAG_ACCESS_EN_MSB 0xA
1131#define QIB_7322_SPC_JTAG_ACCESS_REG_SPC_JTAG_ACCESS_EN_RMASK 0x1
1132#define QIB_7322_SPC_JTAG_ACCESS_REG_bist_en_LSB 0x5
1133#define QIB_7322_SPC_JTAG_ACCESS_REG_bist_en_MSB 0x9
1134#define QIB_7322_SPC_JTAG_ACCESS_REG_bist_en_RMASK 0x1F
1135#define QIB_7322_SPC_JTAG_ACCESS_REG_opcode_LSB 0x3
1136#define QIB_7322_SPC_JTAG_ACCESS_REG_opcode_MSB 0x4
1137#define QIB_7322_SPC_JTAG_ACCESS_REG_opcode_RMASK 0x3
1138#define QIB_7322_SPC_JTAG_ACCESS_REG_tdi_LSB 0x2
1139#define QIB_7322_SPC_JTAG_ACCESS_REG_tdi_MSB 0x2
1140#define QIB_7322_SPC_JTAG_ACCESS_REG_tdi_RMASK 0x1
1141#define QIB_7322_SPC_JTAG_ACCESS_REG_tdo_LSB 0x1
1142#define QIB_7322_SPC_JTAG_ACCESS_REG_tdo_MSB 0x1
1143#define QIB_7322_SPC_JTAG_ACCESS_REG_tdo_RMASK 0x1
1144#define QIB_7322_SPC_JTAG_ACCESS_REG_rdy_LSB 0x0
1145#define QIB_7322_SPC_JTAG_ACCESS_REG_rdy_MSB 0x0
1146#define QIB_7322_SPC_JTAG_ACCESS_REG_rdy_RMASK 0x1
1147
1148#define QIB_7322_SendCheckMask0_OFFS 0x4C0
1149#define QIB_7322_SendCheckMask0_DEF 0x0000000000000000
1150#define QIB_7322_SendCheckMask0_SendCheckMask_63_32_LSB 0x0
1151#define QIB_7322_SendCheckMask0_SendCheckMask_63_32_MSB 0x3F
1152#define QIB_7322_SendCheckMask0_SendCheckMask_63_32_RMASK 0x0
1153
1154#define QIB_7322_SendGRHCheckMask0_OFFS 0x4E0
1155#define QIB_7322_SendGRHCheckMask0_DEF 0x0000000000000000
1156#define QIB_7322_SendGRHCheckMask0_SendGRHCheckMask_63_32_LSB 0x0
1157#define QIB_7322_SendGRHCheckMask0_SendGRHCheckMask_63_32_MSB 0x3F
1158#define QIB_7322_SendGRHCheckMask0_SendGRHCheckMask_63_32_RMASK 0x0
1159
1160#define QIB_7322_SendIBPacketMask0_OFFS 0x500
1161#define QIB_7322_SendIBPacketMask0_DEF 0x0000000000000000
1162#define QIB_7322_SendIBPacketMask0_SendIBPacketMask_63_32_LSB 0x0
1163#define QIB_7322_SendIBPacketMask0_SendIBPacketMask_63_32_MSB 0x3F
1164#define QIB_7322_SendIBPacketMask0_SendIBPacketMask_63_32_RMASK 0x0
1165
1166#define QIB_7322_IntRedirect0_OFFS 0x540
1167#define QIB_7322_IntRedirect0_DEF 0x0000000000000000
1168#define QIB_7322_IntRedirect0_vec11_LSB 0x37
1169#define QIB_7322_IntRedirect0_vec11_MSB 0x3B
1170#define QIB_7322_IntRedirect0_vec11_RMASK 0x1F
1171#define QIB_7322_IntRedirect0_vec10_LSB 0x32
1172#define QIB_7322_IntRedirect0_vec10_MSB 0x36
1173#define QIB_7322_IntRedirect0_vec10_RMASK 0x1F
1174#define QIB_7322_IntRedirect0_vec9_LSB 0x2D
1175#define QIB_7322_IntRedirect0_vec9_MSB 0x31
1176#define QIB_7322_IntRedirect0_vec9_RMASK 0x1F
1177#define QIB_7322_IntRedirect0_vec8_LSB 0x28
1178#define QIB_7322_IntRedirect0_vec8_MSB 0x2C
1179#define QIB_7322_IntRedirect0_vec8_RMASK 0x1F
1180#define QIB_7322_IntRedirect0_vec7_LSB 0x23
1181#define QIB_7322_IntRedirect0_vec7_MSB 0x27
1182#define QIB_7322_IntRedirect0_vec7_RMASK 0x1F
1183#define QIB_7322_IntRedirect0_vec6_LSB 0x1E
1184#define QIB_7322_IntRedirect0_vec6_MSB 0x22
1185#define QIB_7322_IntRedirect0_vec6_RMASK 0x1F
1186#define QIB_7322_IntRedirect0_vec5_LSB 0x19
1187#define QIB_7322_IntRedirect0_vec5_MSB 0x1D
1188#define QIB_7322_IntRedirect0_vec5_RMASK 0x1F
1189#define QIB_7322_IntRedirect0_vec4_LSB 0x14
1190#define QIB_7322_IntRedirect0_vec4_MSB 0x18
1191#define QIB_7322_IntRedirect0_vec4_RMASK 0x1F
1192#define QIB_7322_IntRedirect0_vec3_LSB 0xF
1193#define QIB_7322_IntRedirect0_vec3_MSB 0x13
1194#define QIB_7322_IntRedirect0_vec3_RMASK 0x1F
1195#define QIB_7322_IntRedirect0_vec2_LSB 0xA
1196#define QIB_7322_IntRedirect0_vec2_MSB 0xE
1197#define QIB_7322_IntRedirect0_vec2_RMASK 0x1F
1198#define QIB_7322_IntRedirect0_vec1_LSB 0x5
1199#define QIB_7322_IntRedirect0_vec1_MSB 0x9
1200#define QIB_7322_IntRedirect0_vec1_RMASK 0x1F
1201#define QIB_7322_IntRedirect0_vec0_LSB 0x0
1202#define QIB_7322_IntRedirect0_vec0_MSB 0x4
1203#define QIB_7322_IntRedirect0_vec0_RMASK 0x1F
1204
1205#define QIB_7322_Int_Granted_OFFS 0x570
1206#define QIB_7322_Int_Granted_DEF 0x0000000000000000
1207
1208#define QIB_7322_vec_clr_without_int_OFFS 0x578
1209#define QIB_7322_vec_clr_without_int_DEF 0x0000000000000000
1210
1211#define QIB_7322_DCACtrlA_OFFS 0x580
1212#define QIB_7322_DCACtrlA_DEF 0x0000000000000000
1213#define QIB_7322_DCACtrlA_SendDMAHead1DCAEnable_LSB 0x4
1214#define QIB_7322_DCACtrlA_SendDMAHead1DCAEnable_MSB 0x4
1215#define QIB_7322_DCACtrlA_SendDMAHead1DCAEnable_RMASK 0x1
1216#define QIB_7322_DCACtrlA_SendDMAHead0DCAEnable_LSB 0x3
1217#define QIB_7322_DCACtrlA_SendDMAHead0DCAEnable_MSB 0x3
1218#define QIB_7322_DCACtrlA_SendDMAHead0DCAEnable_RMASK 0x1
1219#define QIB_7322_DCACtrlA_RcvTailUpdDCAEnable_LSB 0x2
1220#define QIB_7322_DCACtrlA_RcvTailUpdDCAEnable_MSB 0x2
1221#define QIB_7322_DCACtrlA_RcvTailUpdDCAEnable_RMASK 0x1
1222#define QIB_7322_DCACtrlA_EagerDCAEnable_LSB 0x1
1223#define QIB_7322_DCACtrlA_EagerDCAEnable_MSB 0x1
1224#define QIB_7322_DCACtrlA_EagerDCAEnable_RMASK 0x1
1225#define QIB_7322_DCACtrlA_RcvHdrqDCAEnable_LSB 0x0
1226#define QIB_7322_DCACtrlA_RcvHdrqDCAEnable_MSB 0x0
1227#define QIB_7322_DCACtrlA_RcvHdrqDCAEnable_RMASK 0x1
1228
1229#define QIB_7322_DCACtrlB_OFFS 0x588
1230#define QIB_7322_DCACtrlB_DEF 0x0000000000000000
1231#define QIB_7322_DCACtrlB_RcvHdrq3DCAXfrCnt_LSB 0x36
1232#define QIB_7322_DCACtrlB_RcvHdrq3DCAXfrCnt_MSB 0x3B
1233#define QIB_7322_DCACtrlB_RcvHdrq3DCAXfrCnt_RMASK 0x3F
1234#define QIB_7322_DCACtrlB_RcvHdrq3DCAOPH_LSB 0x2E
1235#define QIB_7322_DCACtrlB_RcvHdrq3DCAOPH_MSB 0x35
1236#define QIB_7322_DCACtrlB_RcvHdrq3DCAOPH_RMASK 0xFF
1237#define QIB_7322_DCACtrlB_RcvHdrq2DCAXfrCnt_LSB 0x28
1238#define QIB_7322_DCACtrlB_RcvHdrq2DCAXfrCnt_MSB 0x2D
1239#define QIB_7322_DCACtrlB_RcvHdrq2DCAXfrCnt_RMASK 0x3F
1240#define QIB_7322_DCACtrlB_RcvHdrq2DCAOPH_LSB 0x20
1241#define QIB_7322_DCACtrlB_RcvHdrq2DCAOPH_MSB 0x27
1242#define QIB_7322_DCACtrlB_RcvHdrq2DCAOPH_RMASK 0xFF
1243#define QIB_7322_DCACtrlB_RcvHdrq1DCAXfrCnt_LSB 0x16
1244#define QIB_7322_DCACtrlB_RcvHdrq1DCAXfrCnt_MSB 0x1B
1245#define QIB_7322_DCACtrlB_RcvHdrq1DCAXfrCnt_RMASK 0x3F
1246#define QIB_7322_DCACtrlB_RcvHdrq1DCAOPH_LSB 0xE
1247#define QIB_7322_DCACtrlB_RcvHdrq1DCAOPH_MSB 0x15
1248#define QIB_7322_DCACtrlB_RcvHdrq1DCAOPH_RMASK 0xFF
1249#define QIB_7322_DCACtrlB_RcvHdrq0DCAXfrCnt_LSB 0x8
1250#define QIB_7322_DCACtrlB_RcvHdrq0DCAXfrCnt_MSB 0xD
1251#define QIB_7322_DCACtrlB_RcvHdrq0DCAXfrCnt_RMASK 0x3F
1252#define QIB_7322_DCACtrlB_RcvHdrq0DCAOPH_LSB 0x0
1253#define QIB_7322_DCACtrlB_RcvHdrq0DCAOPH_MSB 0x7
1254#define QIB_7322_DCACtrlB_RcvHdrq0DCAOPH_RMASK 0xFF
1255
1256#define QIB_7322_DCACtrlC_OFFS 0x590
1257#define QIB_7322_DCACtrlC_DEF 0x0000000000000000
1258#define QIB_7322_DCACtrlC_RcvHdrq7DCAXfrCnt_LSB 0x36
1259#define QIB_7322_DCACtrlC_RcvHdrq7DCAXfrCnt_MSB 0x3B
1260#define QIB_7322_DCACtrlC_RcvHdrq7DCAXfrCnt_RMASK 0x3F
1261#define QIB_7322_DCACtrlC_RcvHdrq7DCAOPH_LSB 0x2E
1262#define QIB_7322_DCACtrlC_RcvHdrq7DCAOPH_MSB 0x35
1263#define QIB_7322_DCACtrlC_RcvHdrq7DCAOPH_RMASK 0xFF
1264#define QIB_7322_DCACtrlC_RcvHdrq6DCAXfrCnt_LSB 0x28
1265#define QIB_7322_DCACtrlC_RcvHdrq6DCAXfrCnt_MSB 0x2D
1266#define QIB_7322_DCACtrlC_RcvHdrq6DCAXfrCnt_RMASK 0x3F
1267#define QIB_7322_DCACtrlC_RcvHdrq6DCAOPH_LSB 0x20
1268#define QIB_7322_DCACtrlC_RcvHdrq6DCAOPH_MSB 0x27
1269#define QIB_7322_DCACtrlC_RcvHdrq6DCAOPH_RMASK 0xFF
1270#define QIB_7322_DCACtrlC_RcvHdrq5DCAXfrCnt_LSB 0x16
1271#define QIB_7322_DCACtrlC_RcvHdrq5DCAXfrCnt_MSB 0x1B
1272#define QIB_7322_DCACtrlC_RcvHdrq5DCAXfrCnt_RMASK 0x3F
1273#define QIB_7322_DCACtrlC_RcvHdrq5DCAOPH_LSB 0xE
1274#define QIB_7322_DCACtrlC_RcvHdrq5DCAOPH_MSB 0x15
1275#define QIB_7322_DCACtrlC_RcvHdrq5DCAOPH_RMASK 0xFF
1276#define QIB_7322_DCACtrlC_RcvHdrq4DCAXfrCnt_LSB 0x8
1277#define QIB_7322_DCACtrlC_RcvHdrq4DCAXfrCnt_MSB 0xD
1278#define QIB_7322_DCACtrlC_RcvHdrq4DCAXfrCnt_RMASK 0x3F
1279#define QIB_7322_DCACtrlC_RcvHdrq4DCAOPH_LSB 0x0
1280#define QIB_7322_DCACtrlC_RcvHdrq4DCAOPH_MSB 0x7
1281#define QIB_7322_DCACtrlC_RcvHdrq4DCAOPH_RMASK 0xFF
1282
1283#define QIB_7322_DCACtrlD_OFFS 0x598
1284#define QIB_7322_DCACtrlD_DEF 0x0000000000000000
1285#define QIB_7322_DCACtrlD_RcvHdrq11DCAXfrCnt_LSB 0x36
1286#define QIB_7322_DCACtrlD_RcvHdrq11DCAXfrCnt_MSB 0x3B
1287#define QIB_7322_DCACtrlD_RcvHdrq11DCAXfrCnt_RMASK 0x3F
1288#define QIB_7322_DCACtrlD_RcvHdrq11DCAOPH_LSB 0x2E
1289#define QIB_7322_DCACtrlD_RcvHdrq11DCAOPH_MSB 0x35
1290#define QIB_7322_DCACtrlD_RcvHdrq11DCAOPH_RMASK 0xFF
1291#define QIB_7322_DCACtrlD_RcvHdrq10DCAXfrCnt_LSB 0x28
1292#define QIB_7322_DCACtrlD_RcvHdrq10DCAXfrCnt_MSB 0x2D
1293#define QIB_7322_DCACtrlD_RcvHdrq10DCAXfrCnt_RMASK 0x3F
1294#define QIB_7322_DCACtrlD_RcvHdrq10DCAOPH_LSB 0x20
1295#define QIB_7322_DCACtrlD_RcvHdrq10DCAOPH_MSB 0x27
1296#define QIB_7322_DCACtrlD_RcvHdrq10DCAOPH_RMASK 0xFF
1297#define QIB_7322_DCACtrlD_RcvHdrq9DCAXfrCnt_LSB 0x16
1298#define QIB_7322_DCACtrlD_RcvHdrq9DCAXfrCnt_MSB 0x1B
1299#define QIB_7322_DCACtrlD_RcvHdrq9DCAXfrCnt_RMASK 0x3F
1300#define QIB_7322_DCACtrlD_RcvHdrq9DCAOPH_LSB 0xE
1301#define QIB_7322_DCACtrlD_RcvHdrq9DCAOPH_MSB 0x15
1302#define QIB_7322_DCACtrlD_RcvHdrq9DCAOPH_RMASK 0xFF
1303#define QIB_7322_DCACtrlD_RcvHdrq8DCAXfrCnt_LSB 0x8
1304#define QIB_7322_DCACtrlD_RcvHdrq8DCAXfrCnt_MSB 0xD
1305#define QIB_7322_DCACtrlD_RcvHdrq8DCAXfrCnt_RMASK 0x3F
1306#define QIB_7322_DCACtrlD_RcvHdrq8DCAOPH_LSB 0x0
1307#define QIB_7322_DCACtrlD_RcvHdrq8DCAOPH_MSB 0x7
1308#define QIB_7322_DCACtrlD_RcvHdrq8DCAOPH_RMASK 0xFF
1309
1310#define QIB_7322_DCACtrlE_OFFS 0x5A0
1311#define QIB_7322_DCACtrlE_DEF 0x0000000000000000
1312#define QIB_7322_DCACtrlE_RcvHdrq15DCAXfrCnt_LSB 0x36
1313#define QIB_7322_DCACtrlE_RcvHdrq15DCAXfrCnt_MSB 0x3B
1314#define QIB_7322_DCACtrlE_RcvHdrq15DCAXfrCnt_RMASK 0x3F
1315#define QIB_7322_DCACtrlE_RcvHdrq15DCAOPH_LSB 0x2E
1316#define QIB_7322_DCACtrlE_RcvHdrq15DCAOPH_MSB 0x35
1317#define QIB_7322_DCACtrlE_RcvHdrq15DCAOPH_RMASK 0xFF
1318#define QIB_7322_DCACtrlE_RcvHdrq14DCAXfrCnt_LSB 0x28
1319#define QIB_7322_DCACtrlE_RcvHdrq14DCAXfrCnt_MSB 0x2D
1320#define QIB_7322_DCACtrlE_RcvHdrq14DCAXfrCnt_RMASK 0x3F
1321#define QIB_7322_DCACtrlE_RcvHdrq14DCAOPH_LSB 0x20
1322#define QIB_7322_DCACtrlE_RcvHdrq14DCAOPH_MSB 0x27
1323#define QIB_7322_DCACtrlE_RcvHdrq14DCAOPH_RMASK 0xFF
1324#define QIB_7322_DCACtrlE_RcvHdrq13DCAXfrCnt_LSB 0x16
1325#define QIB_7322_DCACtrlE_RcvHdrq13DCAXfrCnt_MSB 0x1B
1326#define QIB_7322_DCACtrlE_RcvHdrq13DCAXfrCnt_RMASK 0x3F
1327#define QIB_7322_DCACtrlE_RcvHdrq13DCAOPH_LSB 0xE
1328#define QIB_7322_DCACtrlE_RcvHdrq13DCAOPH_MSB 0x15
1329#define QIB_7322_DCACtrlE_RcvHdrq13DCAOPH_RMASK 0xFF
1330#define QIB_7322_DCACtrlE_RcvHdrq12DCAXfrCnt_LSB 0x8
1331#define QIB_7322_DCACtrlE_RcvHdrq12DCAXfrCnt_MSB 0xD
1332#define QIB_7322_DCACtrlE_RcvHdrq12DCAXfrCnt_RMASK 0x3F
1333#define QIB_7322_DCACtrlE_RcvHdrq12DCAOPH_LSB 0x0
1334#define QIB_7322_DCACtrlE_RcvHdrq12DCAOPH_MSB 0x7
1335#define QIB_7322_DCACtrlE_RcvHdrq12DCAOPH_RMASK 0xFF
1336
1337#define QIB_7322_DCACtrlF_OFFS 0x5A8
1338#define QIB_7322_DCACtrlF_DEF 0x0000000000000000
1339#define QIB_7322_DCACtrlF_SendDma1DCAOPH_LSB 0x28
1340#define QIB_7322_DCACtrlF_SendDma1DCAOPH_MSB 0x2F
1341#define QIB_7322_DCACtrlF_SendDma1DCAOPH_RMASK 0xFF
1342#define QIB_7322_DCACtrlF_SendDma0DCAOPH_LSB 0x20
1343#define QIB_7322_DCACtrlF_SendDma0DCAOPH_MSB 0x27
1344#define QIB_7322_DCACtrlF_SendDma0DCAOPH_RMASK 0xFF
1345#define QIB_7322_DCACtrlF_RcvHdrq17DCAXfrCnt_LSB 0x16
1346#define QIB_7322_DCACtrlF_RcvHdrq17DCAXfrCnt_MSB 0x1B
1347#define QIB_7322_DCACtrlF_RcvHdrq17DCAXfrCnt_RMASK 0x3F
1348#define QIB_7322_DCACtrlF_RcvHdrq17DCAOPH_LSB 0xE
1349#define QIB_7322_DCACtrlF_RcvHdrq17DCAOPH_MSB 0x15
1350#define QIB_7322_DCACtrlF_RcvHdrq17DCAOPH_RMASK 0xFF
1351#define QIB_7322_DCACtrlF_RcvHdrq16DCAXfrCnt_LSB 0x8
1352#define QIB_7322_DCACtrlF_RcvHdrq16DCAXfrCnt_MSB 0xD
1353#define QIB_7322_DCACtrlF_RcvHdrq16DCAXfrCnt_RMASK 0x3F
1354#define QIB_7322_DCACtrlF_RcvHdrq16DCAOPH_LSB 0x0
1355#define QIB_7322_DCACtrlF_RcvHdrq16DCAOPH_MSB 0x7
1356#define QIB_7322_DCACtrlF_RcvHdrq16DCAOPH_RMASK 0xFF
1357
1358#define QIB_7322_RcvAvailTimeOut0_OFFS 0xC00
1359#define QIB_7322_RcvAvailTimeOut0_DEF 0x0000000000000000
1360#define QIB_7322_RcvAvailTimeOut0_RcvAvailTOCount_LSB 0x10
1361#define QIB_7322_RcvAvailTimeOut0_RcvAvailTOCount_MSB 0x1F
1362#define QIB_7322_RcvAvailTimeOut0_RcvAvailTOCount_RMASK 0xFFFF
1363#define QIB_7322_RcvAvailTimeOut0_RcvAvailTOReload_LSB 0x0
1364#define QIB_7322_RcvAvailTimeOut0_RcvAvailTOReload_MSB 0xF
1365#define QIB_7322_RcvAvailTimeOut0_RcvAvailTOReload_RMASK 0xFFFF
1366
1367#define QIB_7322_CntrRegBase_0_OFFS 0x1028
1368#define QIB_7322_CntrRegBase_0_DEF 0x0000000000012000
1369
1370#define QIB_7322_ErrMask_0_OFFS 0x1080
1371#define QIB_7322_ErrMask_0_DEF 0x0000000000000000
1372#define QIB_7322_ErrMask_0_IBStatusChangedMask_LSB 0x3A
1373#define QIB_7322_ErrMask_0_IBStatusChangedMask_MSB 0x3A
1374#define QIB_7322_ErrMask_0_IBStatusChangedMask_RMASK 0x1
1375#define QIB_7322_ErrMask_0_SHeadersErrMask_LSB 0x39
1376#define QIB_7322_ErrMask_0_SHeadersErrMask_MSB 0x39
1377#define QIB_7322_ErrMask_0_SHeadersErrMask_RMASK 0x1
1378#define QIB_7322_ErrMask_0_VL15BufMisuseErrMask_LSB 0x36
1379#define QIB_7322_ErrMask_0_VL15BufMisuseErrMask_MSB 0x36
1380#define QIB_7322_ErrMask_0_VL15BufMisuseErrMask_RMASK 0x1
1381#define QIB_7322_ErrMask_0_SDmaHaltErrMask_LSB 0x31
1382#define QIB_7322_ErrMask_0_SDmaHaltErrMask_MSB 0x31
1383#define QIB_7322_ErrMask_0_SDmaHaltErrMask_RMASK 0x1
1384#define QIB_7322_ErrMask_0_SDmaDescAddrMisalignErrMask_LSB 0x30
1385#define QIB_7322_ErrMask_0_SDmaDescAddrMisalignErrMask_MSB 0x30
1386#define QIB_7322_ErrMask_0_SDmaDescAddrMisalignErrMask_RMASK 0x1
1387#define QIB_7322_ErrMask_0_SDmaUnexpDataErrMask_LSB 0x2F
1388#define QIB_7322_ErrMask_0_SDmaUnexpDataErrMask_MSB 0x2F
1389#define QIB_7322_ErrMask_0_SDmaUnexpDataErrMask_RMASK 0x1
1390#define QIB_7322_ErrMask_0_SDmaMissingDwErrMask_LSB 0x2E
1391#define QIB_7322_ErrMask_0_SDmaMissingDwErrMask_MSB 0x2E
1392#define QIB_7322_ErrMask_0_SDmaMissingDwErrMask_RMASK 0x1
1393#define QIB_7322_ErrMask_0_SDmaDwEnErrMask_LSB 0x2D
1394#define QIB_7322_ErrMask_0_SDmaDwEnErrMask_MSB 0x2D
1395#define QIB_7322_ErrMask_0_SDmaDwEnErrMask_RMASK 0x1
1396#define QIB_7322_ErrMask_0_SDmaRpyTagErrMask_LSB 0x2C
1397#define QIB_7322_ErrMask_0_SDmaRpyTagErrMask_MSB 0x2C
1398#define QIB_7322_ErrMask_0_SDmaRpyTagErrMask_RMASK 0x1
1399#define QIB_7322_ErrMask_0_SDma1stDescErrMask_LSB 0x2B
1400#define QIB_7322_ErrMask_0_SDma1stDescErrMask_MSB 0x2B
1401#define QIB_7322_ErrMask_0_SDma1stDescErrMask_RMASK 0x1
1402#define QIB_7322_ErrMask_0_SDmaBaseErrMask_LSB 0x2A
1403#define QIB_7322_ErrMask_0_SDmaBaseErrMask_MSB 0x2A
1404#define QIB_7322_ErrMask_0_SDmaBaseErrMask_RMASK 0x1
1405#define QIB_7322_ErrMask_0_SDmaTailOutOfBoundErrMask_LSB 0x29
1406#define QIB_7322_ErrMask_0_SDmaTailOutOfBoundErrMask_MSB 0x29
1407#define QIB_7322_ErrMask_0_SDmaTailOutOfBoundErrMask_RMASK 0x1
1408#define QIB_7322_ErrMask_0_SDmaOutOfBoundErrMask_LSB 0x28
1409#define QIB_7322_ErrMask_0_SDmaOutOfBoundErrMask_MSB 0x28
1410#define QIB_7322_ErrMask_0_SDmaOutOfBoundErrMask_RMASK 0x1
1411#define QIB_7322_ErrMask_0_SDmaGenMismatchErrMask_LSB 0x27
1412#define QIB_7322_ErrMask_0_SDmaGenMismatchErrMask_MSB 0x27
1413#define QIB_7322_ErrMask_0_SDmaGenMismatchErrMask_RMASK 0x1
1414#define QIB_7322_ErrMask_0_SendBufMisuseErrMask_LSB 0x26
1415#define QIB_7322_ErrMask_0_SendBufMisuseErrMask_MSB 0x26
1416#define QIB_7322_ErrMask_0_SendBufMisuseErrMask_RMASK 0x1
1417#define QIB_7322_ErrMask_0_SendUnsupportedVLErrMask_LSB 0x25
1418#define QIB_7322_ErrMask_0_SendUnsupportedVLErrMask_MSB 0x25
1419#define QIB_7322_ErrMask_0_SendUnsupportedVLErrMask_RMASK 0x1
1420#define QIB_7322_ErrMask_0_SendUnexpectedPktNumErrMask_LSB 0x24
1421#define QIB_7322_ErrMask_0_SendUnexpectedPktNumErrMask_MSB 0x24
1422#define QIB_7322_ErrMask_0_SendUnexpectedPktNumErrMask_RMASK 0x1
1423#define QIB_7322_ErrMask_0_SendDroppedDataPktErrMask_LSB 0x22
1424#define QIB_7322_ErrMask_0_SendDroppedDataPktErrMask_MSB 0x22
1425#define QIB_7322_ErrMask_0_SendDroppedDataPktErrMask_RMASK 0x1
1426#define QIB_7322_ErrMask_0_SendDroppedSmpPktErrMask_LSB 0x21
1427#define QIB_7322_ErrMask_0_SendDroppedSmpPktErrMask_MSB 0x21
1428#define QIB_7322_ErrMask_0_SendDroppedSmpPktErrMask_RMASK 0x1
1429#define QIB_7322_ErrMask_0_SendPktLenErrMask_LSB 0x20
1430#define QIB_7322_ErrMask_0_SendPktLenErrMask_MSB 0x20
1431#define QIB_7322_ErrMask_0_SendPktLenErrMask_RMASK 0x1
1432#define QIB_7322_ErrMask_0_SendUnderRunErrMask_LSB 0x1F
1433#define QIB_7322_ErrMask_0_SendUnderRunErrMask_MSB 0x1F
1434#define QIB_7322_ErrMask_0_SendUnderRunErrMask_RMASK 0x1
1435#define QIB_7322_ErrMask_0_SendMaxPktLenErrMask_LSB 0x1E
1436#define QIB_7322_ErrMask_0_SendMaxPktLenErrMask_MSB 0x1E
1437#define QIB_7322_ErrMask_0_SendMaxPktLenErrMask_RMASK 0x1
1438#define QIB_7322_ErrMask_0_SendMinPktLenErrMask_LSB 0x1D
1439#define QIB_7322_ErrMask_0_SendMinPktLenErrMask_MSB 0x1D
1440#define QIB_7322_ErrMask_0_SendMinPktLenErrMask_RMASK 0x1
1441#define QIB_7322_ErrMask_0_RcvIBLostLinkErrMask_LSB 0x11
1442#define QIB_7322_ErrMask_0_RcvIBLostLinkErrMask_MSB 0x11
1443#define QIB_7322_ErrMask_0_RcvIBLostLinkErrMask_RMASK 0x1
1444#define QIB_7322_ErrMask_0_RcvHdrErrMask_LSB 0x10
1445#define QIB_7322_ErrMask_0_RcvHdrErrMask_MSB 0x10
1446#define QIB_7322_ErrMask_0_RcvHdrErrMask_RMASK 0x1
1447#define QIB_7322_ErrMask_0_RcvHdrLenErrMask_LSB 0xF
1448#define QIB_7322_ErrMask_0_RcvHdrLenErrMask_MSB 0xF
1449#define QIB_7322_ErrMask_0_RcvHdrLenErrMask_RMASK 0x1
1450#define QIB_7322_ErrMask_0_RcvBadTidErrMask_LSB 0xE
1451#define QIB_7322_ErrMask_0_RcvBadTidErrMask_MSB 0xE
1452#define QIB_7322_ErrMask_0_RcvBadTidErrMask_RMASK 0x1
1453#define QIB_7322_ErrMask_0_RcvBadVersionErrMask_LSB 0xB
1454#define QIB_7322_ErrMask_0_RcvBadVersionErrMask_MSB 0xB
1455#define QIB_7322_ErrMask_0_RcvBadVersionErrMask_RMASK 0x1
1456#define QIB_7322_ErrMask_0_RcvIBFlowErrMask_LSB 0xA
1457#define QIB_7322_ErrMask_0_RcvIBFlowErrMask_MSB 0xA
1458#define QIB_7322_ErrMask_0_RcvIBFlowErrMask_RMASK 0x1
1459#define QIB_7322_ErrMask_0_RcvEBPErrMask_LSB 0x9
1460#define QIB_7322_ErrMask_0_RcvEBPErrMask_MSB 0x9
1461#define QIB_7322_ErrMask_0_RcvEBPErrMask_RMASK 0x1
1462#define QIB_7322_ErrMask_0_RcvUnsupportedVLErrMask_LSB 0x8
1463#define QIB_7322_ErrMask_0_RcvUnsupportedVLErrMask_MSB 0x8
1464#define QIB_7322_ErrMask_0_RcvUnsupportedVLErrMask_RMASK 0x1
1465#define QIB_7322_ErrMask_0_RcvUnexpectedCharErrMask_LSB 0x7
1466#define QIB_7322_ErrMask_0_RcvUnexpectedCharErrMask_MSB 0x7
1467#define QIB_7322_ErrMask_0_RcvUnexpectedCharErrMask_RMASK 0x1
1468#define QIB_7322_ErrMask_0_RcvShortPktLenErrMask_LSB 0x6
1469#define QIB_7322_ErrMask_0_RcvShortPktLenErrMask_MSB 0x6
1470#define QIB_7322_ErrMask_0_RcvShortPktLenErrMask_RMASK 0x1
1471#define QIB_7322_ErrMask_0_RcvLongPktLenErrMask_LSB 0x5
1472#define QIB_7322_ErrMask_0_RcvLongPktLenErrMask_MSB 0x5
1473#define QIB_7322_ErrMask_0_RcvLongPktLenErrMask_RMASK 0x1
1474#define QIB_7322_ErrMask_0_RcvMaxPktLenErrMask_LSB 0x4
1475#define QIB_7322_ErrMask_0_RcvMaxPktLenErrMask_MSB 0x4
1476#define QIB_7322_ErrMask_0_RcvMaxPktLenErrMask_RMASK 0x1
1477#define QIB_7322_ErrMask_0_RcvMinPktLenErrMask_LSB 0x3
1478#define QIB_7322_ErrMask_0_RcvMinPktLenErrMask_MSB 0x3
1479#define QIB_7322_ErrMask_0_RcvMinPktLenErrMask_RMASK 0x1
1480#define QIB_7322_ErrMask_0_RcvICRCErrMask_LSB 0x2
1481#define QIB_7322_ErrMask_0_RcvICRCErrMask_MSB 0x2
1482#define QIB_7322_ErrMask_0_RcvICRCErrMask_RMASK 0x1
1483#define QIB_7322_ErrMask_0_RcvVCRCErrMask_LSB 0x1
1484#define QIB_7322_ErrMask_0_RcvVCRCErrMask_MSB 0x1
1485#define QIB_7322_ErrMask_0_RcvVCRCErrMask_RMASK 0x1
1486#define QIB_7322_ErrMask_0_RcvFormatErrMask_LSB 0x0
1487#define QIB_7322_ErrMask_0_RcvFormatErrMask_MSB 0x0
1488#define QIB_7322_ErrMask_0_RcvFormatErrMask_RMASK 0x1
1489
1490#define QIB_7322_ErrStatus_0_OFFS 0x1088
1491#define QIB_7322_ErrStatus_0_DEF 0x0000000000000000
1492#define QIB_7322_ErrStatus_0_IBStatusChanged_LSB 0x3A
1493#define QIB_7322_ErrStatus_0_IBStatusChanged_MSB 0x3A
1494#define QIB_7322_ErrStatus_0_IBStatusChanged_RMASK 0x1
1495#define QIB_7322_ErrStatus_0_SHeadersErr_LSB 0x39
1496#define QIB_7322_ErrStatus_0_SHeadersErr_MSB 0x39
1497#define QIB_7322_ErrStatus_0_SHeadersErr_RMASK 0x1
1498#define QIB_7322_ErrStatus_0_VL15BufMisuseErr_LSB 0x36
1499#define QIB_7322_ErrStatus_0_VL15BufMisuseErr_MSB 0x36
1500#define QIB_7322_ErrStatus_0_VL15BufMisuseErr_RMASK 0x1
1501#define QIB_7322_ErrStatus_0_SDmaHaltErr_LSB 0x31
1502#define QIB_7322_ErrStatus_0_SDmaHaltErr_MSB 0x31
1503#define QIB_7322_ErrStatus_0_SDmaHaltErr_RMASK 0x1
1504#define QIB_7322_ErrStatus_0_SDmaDescAddrMisalignErr_LSB 0x30
1505#define QIB_7322_ErrStatus_0_SDmaDescAddrMisalignErr_MSB 0x30
1506#define QIB_7322_ErrStatus_0_SDmaDescAddrMisalignErr_RMASK 0x1
1507#define QIB_7322_ErrStatus_0_SDmaUnexpDataErr_LSB 0x2F
1508#define QIB_7322_ErrStatus_0_SDmaUnexpDataErr_MSB 0x2F
1509#define QIB_7322_ErrStatus_0_SDmaUnexpDataErr_RMASK 0x1
1510#define QIB_7322_ErrStatus_0_SDmaMissingDwErr_LSB 0x2E
1511#define QIB_7322_ErrStatus_0_SDmaMissingDwErr_MSB 0x2E
1512#define QIB_7322_ErrStatus_0_SDmaMissingDwErr_RMASK 0x1
1513#define QIB_7322_ErrStatus_0_SDmaDwEnErr_LSB 0x2D
1514#define QIB_7322_ErrStatus_0_SDmaDwEnErr_MSB 0x2D
1515#define QIB_7322_ErrStatus_0_SDmaDwEnErr_RMASK 0x1
1516#define QIB_7322_ErrStatus_0_SDmaRpyTagErr_LSB 0x2C
1517#define QIB_7322_ErrStatus_0_SDmaRpyTagErr_MSB 0x2C
1518#define QIB_7322_ErrStatus_0_SDmaRpyTagErr_RMASK 0x1
1519#define QIB_7322_ErrStatus_0_SDma1stDescErr_LSB 0x2B
1520#define QIB_7322_ErrStatus_0_SDma1stDescErr_MSB 0x2B
1521#define QIB_7322_ErrStatus_0_SDma1stDescErr_RMASK 0x1
1522#define QIB_7322_ErrStatus_0_SDmaBaseErr_LSB 0x2A
1523#define QIB_7322_ErrStatus_0_SDmaBaseErr_MSB 0x2A
1524#define QIB_7322_ErrStatus_0_SDmaBaseErr_RMASK 0x1
1525#define QIB_7322_ErrStatus_0_SDmaTailOutOfBoundErr_LSB 0x29
1526#define QIB_7322_ErrStatus_0_SDmaTailOutOfBoundErr_MSB 0x29
1527#define QIB_7322_ErrStatus_0_SDmaTailOutOfBoundErr_RMASK 0x1
1528#define QIB_7322_ErrStatus_0_SDmaOutOfBoundErr_LSB 0x28
1529#define QIB_7322_ErrStatus_0_SDmaOutOfBoundErr_MSB 0x28
1530#define QIB_7322_ErrStatus_0_SDmaOutOfBoundErr_RMASK 0x1
1531#define QIB_7322_ErrStatus_0_SDmaGenMismatchErr_LSB 0x27
1532#define QIB_7322_ErrStatus_0_SDmaGenMismatchErr_MSB 0x27
1533#define QIB_7322_ErrStatus_0_SDmaGenMismatchErr_RMASK 0x1
1534#define QIB_7322_ErrStatus_0_SendBufMisuseErr_LSB 0x26
1535#define QIB_7322_ErrStatus_0_SendBufMisuseErr_MSB 0x26
1536#define QIB_7322_ErrStatus_0_SendBufMisuseErr_RMASK 0x1
1537#define QIB_7322_ErrStatus_0_SendUnsupportedVLErr_LSB 0x25
1538#define QIB_7322_ErrStatus_0_SendUnsupportedVLErr_MSB 0x25
1539#define QIB_7322_ErrStatus_0_SendUnsupportedVLErr_RMASK 0x1
1540#define QIB_7322_ErrStatus_0_SendUnexpectedPktNumErr_LSB 0x24
1541#define QIB_7322_ErrStatus_0_SendUnexpectedPktNumErr_MSB 0x24
1542#define QIB_7322_ErrStatus_0_SendUnexpectedPktNumErr_RMASK 0x1
1543#define QIB_7322_ErrStatus_0_SendDroppedDataPktErr_LSB 0x22
1544#define QIB_7322_ErrStatus_0_SendDroppedDataPktErr_MSB 0x22
1545#define QIB_7322_ErrStatus_0_SendDroppedDataPktErr_RMASK 0x1
1546#define QIB_7322_ErrStatus_0_SendDroppedSmpPktErr_LSB 0x21
1547#define QIB_7322_ErrStatus_0_SendDroppedSmpPktErr_MSB 0x21
1548#define QIB_7322_ErrStatus_0_SendDroppedSmpPktErr_RMASK 0x1
1549#define QIB_7322_ErrStatus_0_SendPktLenErr_LSB 0x20
1550#define QIB_7322_ErrStatus_0_SendPktLenErr_MSB 0x20
1551#define QIB_7322_ErrStatus_0_SendPktLenErr_RMASK 0x1
1552#define QIB_7322_ErrStatus_0_SendUnderRunErr_LSB 0x1F
1553#define QIB_7322_ErrStatus_0_SendUnderRunErr_MSB 0x1F
1554#define QIB_7322_ErrStatus_0_SendUnderRunErr_RMASK 0x1
1555#define QIB_7322_ErrStatus_0_SendMaxPktLenErr_LSB 0x1E
1556#define QIB_7322_ErrStatus_0_SendMaxPktLenErr_MSB 0x1E
1557#define QIB_7322_ErrStatus_0_SendMaxPktLenErr_RMASK 0x1
1558#define QIB_7322_ErrStatus_0_SendMinPktLenErr_LSB 0x1D
1559#define QIB_7322_ErrStatus_0_SendMinPktLenErr_MSB 0x1D
1560#define QIB_7322_ErrStatus_0_SendMinPktLenErr_RMASK 0x1
1561#define QIB_7322_ErrStatus_0_RcvIBLostLinkErr_LSB 0x11
1562#define QIB_7322_ErrStatus_0_RcvIBLostLinkErr_MSB 0x11
1563#define QIB_7322_ErrStatus_0_RcvIBLostLinkErr_RMASK 0x1
1564#define QIB_7322_ErrStatus_0_RcvHdrErr_LSB 0x10
1565#define QIB_7322_ErrStatus_0_RcvHdrErr_MSB 0x10
1566#define QIB_7322_ErrStatus_0_RcvHdrErr_RMASK 0x1
1567#define QIB_7322_ErrStatus_0_RcvHdrLenErr_LSB 0xF
1568#define QIB_7322_ErrStatus_0_RcvHdrLenErr_MSB 0xF
1569#define QIB_7322_ErrStatus_0_RcvHdrLenErr_RMASK 0x1
1570#define QIB_7322_ErrStatus_0_RcvBadTidErr_LSB 0xE
1571#define QIB_7322_ErrStatus_0_RcvBadTidErr_MSB 0xE
1572#define QIB_7322_ErrStatus_0_RcvBadTidErr_RMASK 0x1
1573#define QIB_7322_ErrStatus_0_RcvBadVersionErr_LSB 0xB
1574#define QIB_7322_ErrStatus_0_RcvBadVersionErr_MSB 0xB
1575#define QIB_7322_ErrStatus_0_RcvBadVersionErr_RMASK 0x1
1576#define QIB_7322_ErrStatus_0_RcvIBFlowErr_LSB 0xA
1577#define QIB_7322_ErrStatus_0_RcvIBFlowErr_MSB 0xA
1578#define QIB_7322_ErrStatus_0_RcvIBFlowErr_RMASK 0x1
1579#define QIB_7322_ErrStatus_0_RcvEBPErr_LSB 0x9
1580#define QIB_7322_ErrStatus_0_RcvEBPErr_MSB 0x9
1581#define QIB_7322_ErrStatus_0_RcvEBPErr_RMASK 0x1
1582#define QIB_7322_ErrStatus_0_RcvUnsupportedVLErr_LSB 0x8
1583#define QIB_7322_ErrStatus_0_RcvUnsupportedVLErr_MSB 0x8
1584#define QIB_7322_ErrStatus_0_RcvUnsupportedVLErr_RMASK 0x1
1585#define QIB_7322_ErrStatus_0_RcvUnexpectedCharErr_LSB 0x7
1586#define QIB_7322_ErrStatus_0_RcvUnexpectedCharErr_MSB 0x7
1587#define QIB_7322_ErrStatus_0_RcvUnexpectedCharErr_RMASK 0x1
1588#define QIB_7322_ErrStatus_0_RcvShortPktLenErr_LSB 0x6
1589#define QIB_7322_ErrStatus_0_RcvShortPktLenErr_MSB 0x6
1590#define QIB_7322_ErrStatus_0_RcvShortPktLenErr_RMASK 0x1
1591#define QIB_7322_ErrStatus_0_RcvLongPktLenErr_LSB 0x5
1592#define QIB_7322_ErrStatus_0_RcvLongPktLenErr_MSB 0x5
1593#define QIB_7322_ErrStatus_0_RcvLongPktLenErr_RMASK 0x1
1594#define QIB_7322_ErrStatus_0_RcvMaxPktLenErr_LSB 0x4
1595#define QIB_7322_ErrStatus_0_RcvMaxPktLenErr_MSB 0x4
1596#define QIB_7322_ErrStatus_0_RcvMaxPktLenErr_RMASK 0x1
1597#define QIB_7322_ErrStatus_0_RcvMinPktLenErr_LSB 0x3
1598#define QIB_7322_ErrStatus_0_RcvMinPktLenErr_MSB 0x3
1599#define QIB_7322_ErrStatus_0_RcvMinPktLenErr_RMASK 0x1
1600#define QIB_7322_ErrStatus_0_RcvICRCErr_LSB 0x2
1601#define QIB_7322_ErrStatus_0_RcvICRCErr_MSB 0x2
1602#define QIB_7322_ErrStatus_0_RcvICRCErr_RMASK 0x1
1603#define QIB_7322_ErrStatus_0_RcvVCRCErr_LSB 0x1
1604#define QIB_7322_ErrStatus_0_RcvVCRCErr_MSB 0x1
1605#define QIB_7322_ErrStatus_0_RcvVCRCErr_RMASK 0x1
1606#define QIB_7322_ErrStatus_0_RcvFormatErr_LSB 0x0
1607#define QIB_7322_ErrStatus_0_RcvFormatErr_MSB 0x0
1608#define QIB_7322_ErrStatus_0_RcvFormatErr_RMASK 0x1
1609
1610#define QIB_7322_ErrClear_0_OFFS 0x1090
1611#define QIB_7322_ErrClear_0_DEF 0x0000000000000000
1612#define QIB_7322_ErrClear_0_IBStatusChangedClear_LSB 0x3A
1613#define QIB_7322_ErrClear_0_IBStatusChangedClear_MSB 0x3A
1614#define QIB_7322_ErrClear_0_IBStatusChangedClear_RMASK 0x1
1615#define QIB_7322_ErrClear_0_SHeadersErrClear_LSB 0x39
1616#define QIB_7322_ErrClear_0_SHeadersErrClear_MSB 0x39
1617#define QIB_7322_ErrClear_0_SHeadersErrClear_RMASK 0x1
1618#define QIB_7322_ErrClear_0_VL15BufMisuseErrClear_LSB 0x36
1619#define QIB_7322_ErrClear_0_VL15BufMisuseErrClear_MSB 0x36
1620#define QIB_7322_ErrClear_0_VL15BufMisuseErrClear_RMASK 0x1
1621#define QIB_7322_ErrClear_0_SDmaHaltErrClear_LSB 0x31
1622#define QIB_7322_ErrClear_0_SDmaHaltErrClear_MSB 0x31
1623#define QIB_7322_ErrClear_0_SDmaHaltErrClear_RMASK 0x1
1624#define QIB_7322_ErrClear_0_SDmaDescAddrMisalignErrClear_LSB 0x30
1625#define QIB_7322_ErrClear_0_SDmaDescAddrMisalignErrClear_MSB 0x30
1626#define QIB_7322_ErrClear_0_SDmaDescAddrMisalignErrClear_RMASK 0x1
1627#define QIB_7322_ErrClear_0_SDmaUnexpDataErrClear_LSB 0x2F
1628#define QIB_7322_ErrClear_0_SDmaUnexpDataErrClear_MSB 0x2F
1629#define QIB_7322_ErrClear_0_SDmaUnexpDataErrClear_RMASK 0x1
1630#define QIB_7322_ErrClear_0_SDmaMissingDwErrClear_LSB 0x2E
1631#define QIB_7322_ErrClear_0_SDmaMissingDwErrClear_MSB 0x2E
1632#define QIB_7322_ErrClear_0_SDmaMissingDwErrClear_RMASK 0x1
1633#define QIB_7322_ErrClear_0_SDmaDwEnErrClear_LSB 0x2D
1634#define QIB_7322_ErrClear_0_SDmaDwEnErrClear_MSB 0x2D
1635#define QIB_7322_ErrClear_0_SDmaDwEnErrClear_RMASK 0x1
1636#define QIB_7322_ErrClear_0_SDmaRpyTagErrClear_LSB 0x2C
1637#define QIB_7322_ErrClear_0_SDmaRpyTagErrClear_MSB 0x2C
1638#define QIB_7322_ErrClear_0_SDmaRpyTagErrClear_RMASK 0x1
1639#define QIB_7322_ErrClear_0_SDma1stDescErrClear_LSB 0x2B
1640#define QIB_7322_ErrClear_0_SDma1stDescErrClear_MSB 0x2B
1641#define QIB_7322_ErrClear_0_SDma1stDescErrClear_RMASK 0x1
1642#define QIB_7322_ErrClear_0_SDmaBaseErrClear_LSB 0x2A
1643#define QIB_7322_ErrClear_0_SDmaBaseErrClear_MSB 0x2A
1644#define QIB_7322_ErrClear_0_SDmaBaseErrClear_RMASK 0x1
1645#define QIB_7322_ErrClear_0_SDmaTailOutOfBoundErrClear_LSB 0x29
1646#define QIB_7322_ErrClear_0_SDmaTailOutOfBoundErrClear_MSB 0x29
1647#define QIB_7322_ErrClear_0_SDmaTailOutOfBoundErrClear_RMASK 0x1
1648#define QIB_7322_ErrClear_0_SDmaOutOfBoundErrClear_LSB 0x28
1649#define QIB_7322_ErrClear_0_SDmaOutOfBoundErrClear_MSB 0x28
1650#define QIB_7322_ErrClear_0_SDmaOutOfBoundErrClear_RMASK 0x1
1651#define QIB_7322_ErrClear_0_SDmaGenMismatchErrClear_LSB 0x27
1652#define QIB_7322_ErrClear_0_SDmaGenMismatchErrClear_MSB 0x27
1653#define QIB_7322_ErrClear_0_SDmaGenMismatchErrClear_RMASK 0x1
1654#define QIB_7322_ErrClear_0_SendBufMisuseErrClear_LSB 0x26
1655#define QIB_7322_ErrClear_0_SendBufMisuseErrClear_MSB 0x26
1656#define QIB_7322_ErrClear_0_SendBufMisuseErrClear_RMASK 0x1
1657#define QIB_7322_ErrClear_0_SendUnsupportedVLErrClear_LSB 0x25
1658#define QIB_7322_ErrClear_0_SendUnsupportedVLErrClear_MSB 0x25
1659#define QIB_7322_ErrClear_0_SendUnsupportedVLErrClear_RMASK 0x1
1660#define QIB_7322_ErrClear_0_SendUnexpectedPktNumErrClear_LSB 0x24
1661#define QIB_7322_ErrClear_0_SendUnexpectedPktNumErrClear_MSB 0x24
1662#define QIB_7322_ErrClear_0_SendUnexpectedPktNumErrClear_RMASK 0x1
1663#define QIB_7322_ErrClear_0_SendDroppedDataPktErrClear_LSB 0x22
1664#define QIB_7322_ErrClear_0_SendDroppedDataPktErrClear_MSB 0x22
1665#define QIB_7322_ErrClear_0_SendDroppedDataPktErrClear_RMASK 0x1
1666#define QIB_7322_ErrClear_0_SendDroppedSmpPktErrClear_LSB 0x21
1667#define QIB_7322_ErrClear_0_SendDroppedSmpPktErrClear_MSB 0x21
1668#define QIB_7322_ErrClear_0_SendDroppedSmpPktErrClear_RMASK 0x1
1669#define QIB_7322_ErrClear_0_SendPktLenErrClear_LSB 0x20
1670#define QIB_7322_ErrClear_0_SendPktLenErrClear_MSB 0x20
1671#define QIB_7322_ErrClear_0_SendPktLenErrClear_RMASK 0x1
1672#define QIB_7322_ErrClear_0_SendUnderRunErrClear_LSB 0x1F
1673#define QIB_7322_ErrClear_0_SendUnderRunErrClear_MSB 0x1F
1674#define QIB_7322_ErrClear_0_SendUnderRunErrClear_RMASK 0x1
1675#define QIB_7322_ErrClear_0_SendMaxPktLenErrClear_LSB 0x1E
1676#define QIB_7322_ErrClear_0_SendMaxPktLenErrClear_MSB 0x1E
1677#define QIB_7322_ErrClear_0_SendMaxPktLenErrClear_RMASK 0x1
1678#define QIB_7322_ErrClear_0_SendMinPktLenErrClear_LSB 0x1D
1679#define QIB_7322_ErrClear_0_SendMinPktLenErrClear_MSB 0x1D
1680#define QIB_7322_ErrClear_0_SendMinPktLenErrClear_RMASK 0x1
1681#define QIB_7322_ErrClear_0_RcvIBLostLinkErrClear_LSB 0x11
1682#define QIB_7322_ErrClear_0_RcvIBLostLinkErrClear_MSB 0x11
1683#define QIB_7322_ErrClear_0_RcvIBLostLinkErrClear_RMASK 0x1
1684#define QIB_7322_ErrClear_0_RcvHdrErrClear_LSB 0x10
1685#define QIB_7322_ErrClear_0_RcvHdrErrClear_MSB 0x10
1686#define QIB_7322_ErrClear_0_RcvHdrErrClear_RMASK 0x1
1687#define QIB_7322_ErrClear_0_RcvHdrLenErrClear_LSB 0xF
1688#define QIB_7322_ErrClear_0_RcvHdrLenErrClear_MSB 0xF
1689#define QIB_7322_ErrClear_0_RcvHdrLenErrClear_RMASK 0x1
1690#define QIB_7322_ErrClear_0_RcvBadTidErrClear_LSB 0xE
1691#define QIB_7322_ErrClear_0_RcvBadTidErrClear_MSB 0xE
1692#define QIB_7322_ErrClear_0_RcvBadTidErrClear_RMASK 0x1
1693#define QIB_7322_ErrClear_0_RcvBadVersionErrClear_LSB 0xB
1694#define QIB_7322_ErrClear_0_RcvBadVersionErrClear_MSB 0xB
1695#define QIB_7322_ErrClear_0_RcvBadVersionErrClear_RMASK 0x1
1696#define QIB_7322_ErrClear_0_RcvIBFlowErrClear_LSB 0xA
1697#define QIB_7322_ErrClear_0_RcvIBFlowErrClear_MSB 0xA
1698#define QIB_7322_ErrClear_0_RcvIBFlowErrClear_RMASK 0x1
1699#define QIB_7322_ErrClear_0_RcvEBPErrClear_LSB 0x9
1700#define QIB_7322_ErrClear_0_RcvEBPErrClear_MSB 0x9
1701#define QIB_7322_ErrClear_0_RcvEBPErrClear_RMASK 0x1
1702#define QIB_7322_ErrClear_0_RcvUnsupportedVLErrClear_LSB 0x8
1703#define QIB_7322_ErrClear_0_RcvUnsupportedVLErrClear_MSB 0x8
1704#define QIB_7322_ErrClear_0_RcvUnsupportedVLErrClear_RMASK 0x1
1705#define QIB_7322_ErrClear_0_RcvUnexpectedCharErrClear_LSB 0x7
1706#define QIB_7322_ErrClear_0_RcvUnexpectedCharErrClear_MSB 0x7
1707#define QIB_7322_ErrClear_0_RcvUnexpectedCharErrClear_RMASK 0x1
1708#define QIB_7322_ErrClear_0_RcvShortPktLenErrClear_LSB 0x6
1709#define QIB_7322_ErrClear_0_RcvShortPktLenErrClear_MSB 0x6
1710#define QIB_7322_ErrClear_0_RcvShortPktLenErrClear_RMASK 0x1
1711#define QIB_7322_ErrClear_0_RcvLongPktLenErrClear_LSB 0x5
1712#define QIB_7322_ErrClear_0_RcvLongPktLenErrClear_MSB 0x5
1713#define QIB_7322_ErrClear_0_RcvLongPktLenErrClear_RMASK 0x1
1714#define QIB_7322_ErrClear_0_RcvMaxPktLenErrClear_LSB 0x4
1715#define QIB_7322_ErrClear_0_RcvMaxPktLenErrClear_MSB 0x4
1716#define QIB_7322_ErrClear_0_RcvMaxPktLenErrClear_RMASK 0x1
1717#define QIB_7322_ErrClear_0_RcvMinPktLenErrClear_LSB 0x3
1718#define QIB_7322_ErrClear_0_RcvMinPktLenErrClear_MSB 0x3
1719#define QIB_7322_ErrClear_0_RcvMinPktLenErrClear_RMASK 0x1
1720#define QIB_7322_ErrClear_0_RcvICRCErrClear_LSB 0x2
1721#define QIB_7322_ErrClear_0_RcvICRCErrClear_MSB 0x2
1722#define QIB_7322_ErrClear_0_RcvICRCErrClear_RMASK 0x1
1723#define QIB_7322_ErrClear_0_RcvVCRCErrClear_LSB 0x1
1724#define QIB_7322_ErrClear_0_RcvVCRCErrClear_MSB 0x1
1725#define QIB_7322_ErrClear_0_RcvVCRCErrClear_RMASK 0x1
1726#define QIB_7322_ErrClear_0_RcvFormatErrClear_LSB 0x0
1727#define QIB_7322_ErrClear_0_RcvFormatErrClear_MSB 0x0
1728#define QIB_7322_ErrClear_0_RcvFormatErrClear_RMASK 0x1
1729
1730#define QIB_7322_TXEStatus_0_OFFS 0x10B8
1731#define QIB_7322_TXEStatus_0_DEF 0x0000000XC00080FF
1732#define QIB_7322_TXEStatus_0_TXE_IBC_Idle_LSB 0x1F
1733#define QIB_7322_TXEStatus_0_TXE_IBC_Idle_MSB 0x1F
1734#define QIB_7322_TXEStatus_0_TXE_IBC_Idle_RMASK 0x1
1735#define QIB_7322_TXEStatus_0_RmFifoEmpty_LSB 0x1E
1736#define QIB_7322_TXEStatus_0_RmFifoEmpty_MSB 0x1E
1737#define QIB_7322_TXEStatus_0_RmFifoEmpty_RMASK 0x1
1738#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL15_LSB 0xF
1739#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL15_MSB 0xF
1740#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL15_RMASK 0x1
1741#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL7_LSB 0x7
1742#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL7_MSB 0x7
1743#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL7_RMASK 0x1
1744#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL6_LSB 0x6
1745#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL6_MSB 0x6
1746#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL6_RMASK 0x1
1747#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL5_LSB 0x5
1748#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL5_MSB 0x5
1749#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL5_RMASK 0x1
1750#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL4_LSB 0x4
1751#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL4_MSB 0x4
1752#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL4_RMASK 0x1
1753#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL3_LSB 0x3
1754#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL3_MSB 0x3
1755#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL3_RMASK 0x1
1756#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL2_LSB 0x2
1757#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL2_MSB 0x2
1758#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL2_RMASK 0x1
1759#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL1_LSB 0x1
1760#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL1_MSB 0x1
1761#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL1_RMASK 0x1
1762#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL0_LSB 0x0
1763#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL0_MSB 0x0
1764#define QIB_7322_TXEStatus_0_LaFifoEmpty_VL0_RMASK 0x1
1765
1766#define QIB_7322_RcvCtrl_0_OFFS 0x1100
1767#define QIB_7322_RcvCtrl_0_DEF 0x0000000000000000
1768#define QIB_7322_RcvCtrl_0_RcvResetCredit_LSB 0x2A
1769#define QIB_7322_RcvCtrl_0_RcvResetCredit_MSB 0x2A
1770#define QIB_7322_RcvCtrl_0_RcvResetCredit_RMASK 0x1
1771#define QIB_7322_RcvCtrl_0_RcvPartitionKeyDisable_LSB 0x29
1772#define QIB_7322_RcvCtrl_0_RcvPartitionKeyDisable_MSB 0x29
1773#define QIB_7322_RcvCtrl_0_RcvPartitionKeyDisable_RMASK 0x1
1774#define QIB_7322_RcvCtrl_0_RcvQPMapEnable_LSB 0x28
1775#define QIB_7322_RcvCtrl_0_RcvQPMapEnable_MSB 0x28
1776#define QIB_7322_RcvCtrl_0_RcvQPMapEnable_RMASK 0x1
1777#define QIB_7322_RcvCtrl_0_RcvIBPortEnable_LSB 0x27
1778#define QIB_7322_RcvCtrl_0_RcvIBPortEnable_MSB 0x27
1779#define QIB_7322_RcvCtrl_0_RcvIBPortEnable_RMASK 0x1
1780#define QIB_7322_RcvCtrl_0_ContextEnableUser_LSB 0x2
1781#define QIB_7322_RcvCtrl_0_ContextEnableUser_MSB 0x11
1782#define QIB_7322_RcvCtrl_0_ContextEnableUser_RMASK 0xFFFF
1783#define QIB_7322_RcvCtrl_0_ContextEnableKernel_LSB 0x0
1784#define QIB_7322_RcvCtrl_0_ContextEnableKernel_MSB 0x0
1785#define QIB_7322_RcvCtrl_0_ContextEnableKernel_RMASK 0x1
1786
1787#define QIB_7322_RcvBTHQP_0_OFFS 0x1108
1788#define QIB_7322_RcvBTHQP_0_DEF 0x0000000000000000
1789#define QIB_7322_RcvBTHQP_0_RcvBTHQP_LSB 0x0
1790#define QIB_7322_RcvBTHQP_0_RcvBTHQP_MSB 0x17
1791#define QIB_7322_RcvBTHQP_0_RcvBTHQP_RMASK 0xFFFFFF
1792
1793#define QIB_7322_RcvQPMapTableA_0_OFFS 0x1110
1794#define QIB_7322_RcvQPMapTableA_0_DEF 0x0000000000000000
1795#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext5_LSB 0x19
1796#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext5_MSB 0x1D
1797#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext5_RMASK 0x1F
1798#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext4_LSB 0x14
1799#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext4_MSB 0x18
1800#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext4_RMASK 0x1F
1801#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext3_LSB 0xF
1802#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext3_MSB 0x13
1803#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext3_RMASK 0x1F
1804#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext2_LSB 0xA
1805#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext2_MSB 0xE
1806#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext2_RMASK 0x1F
1807#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext1_LSB 0x5
1808#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext1_MSB 0x9
1809#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext1_RMASK 0x1F
1810#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext0_LSB 0x0
1811#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext0_MSB 0x4
1812#define QIB_7322_RcvQPMapTableA_0_RcvQPMapContext0_RMASK 0x1F
1813
1814#define QIB_7322_RcvQPMapTableB_0_OFFS 0x1118
1815#define QIB_7322_RcvQPMapTableB_0_DEF 0x0000000000000000
1816#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext11_LSB 0x19
1817#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext11_MSB 0x1D
1818#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext11_RMASK 0x1F
1819#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext10_LSB 0x14
1820#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext10_MSB 0x18
1821#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext10_RMASK 0x1F
1822#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext9_LSB 0xF
1823#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext9_MSB 0x13
1824#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext9_RMASK 0x1F
1825#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext8_LSB 0xA
1826#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext8_MSB 0xE
1827#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext8_RMASK 0x1F
1828#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext7_LSB 0x5
1829#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext7_MSB 0x9
1830#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext7_RMASK 0x1F
1831#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext6_LSB 0x0
1832#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext6_MSB 0x4
1833#define QIB_7322_RcvQPMapTableB_0_RcvQPMapContext6_RMASK 0x1F
1834
1835#define QIB_7322_RcvQPMapTableC_0_OFFS 0x1120
1836#define QIB_7322_RcvQPMapTableC_0_DEF 0x0000000000000000
1837#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext17_LSB 0x19
1838#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext17_MSB 0x1D
1839#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext17_RMASK 0x1F
1840#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext16_LSB 0x14
1841#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext16_MSB 0x18
1842#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext16_RMASK 0x1F
1843#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext15_LSB 0xF
1844#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext15_MSB 0x13
1845#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext15_RMASK 0x1F
1846#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext14_LSB 0xA
1847#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext14_MSB 0xE
1848#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext14_RMASK 0x1F
1849#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext13_LSB 0x5
1850#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext13_MSB 0x9
1851#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext13_RMASK 0x1F
1852#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext12_LSB 0x0
1853#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext12_MSB 0x4
1854#define QIB_7322_RcvQPMapTableC_0_RcvQPMapContext12_RMASK 0x1F
1855
1856#define QIB_7322_RcvQPMapTableD_0_OFFS 0x1128
1857#define QIB_7322_RcvQPMapTableD_0_DEF 0x0000000000000000
1858#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext23_LSB 0x19
1859#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext23_MSB 0x1D
1860#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext23_RMASK 0x1F
1861#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext22_LSB 0x14
1862#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext22_MSB 0x18
1863#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext22_RMASK 0x1F
1864#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext21_LSB 0xF
1865#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext21_MSB 0x13
1866#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext21_RMASK 0x1F
1867#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext20_LSB 0xA
1868#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext20_MSB 0xE
1869#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext20_RMASK 0x1F
1870#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext19_LSB 0x5
1871#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext19_MSB 0x9
1872#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext19_RMASK 0x1F
1873#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext18_LSB 0x0
1874#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext18_MSB 0x4
1875#define QIB_7322_RcvQPMapTableD_0_RcvQPMapContext18_RMASK 0x1F
1876
1877#define QIB_7322_RcvQPMapTableE_0_OFFS 0x1130
1878#define QIB_7322_RcvQPMapTableE_0_DEF 0x0000000000000000
1879#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext29_LSB 0x19
1880#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext29_MSB 0x1D
1881#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext29_RMASK 0x1F
1882#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext28_LSB 0x14
1883#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext28_MSB 0x18
1884#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext28_RMASK 0x1F
1885#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext27_LSB 0xF
1886#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext27_MSB 0x13
1887#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext27_RMASK 0x1F
1888#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext26_LSB 0xA
1889#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext26_MSB 0xE
1890#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext26_RMASK 0x1F
1891#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext25_LSB 0x5
1892#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext25_MSB 0x9
1893#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext25_RMASK 0x1F
1894#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext24_LSB 0x0
1895#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext24_MSB 0x4
1896#define QIB_7322_RcvQPMapTableE_0_RcvQPMapContext24_RMASK 0x1F
1897
1898#define QIB_7322_RcvQPMapTableF_0_OFFS 0x1138
1899#define QIB_7322_RcvQPMapTableF_0_DEF 0x0000000000000000
1900#define QIB_7322_RcvQPMapTableF_0_RcvQPMapContext31_LSB 0x5
1901#define QIB_7322_RcvQPMapTableF_0_RcvQPMapContext31_MSB 0x9
1902#define QIB_7322_RcvQPMapTableF_0_RcvQPMapContext31_RMASK 0x1F
1903#define QIB_7322_RcvQPMapTableF_0_RcvQPMapContext30_LSB 0x0
1904#define QIB_7322_RcvQPMapTableF_0_RcvQPMapContext30_MSB 0x4
1905#define QIB_7322_RcvQPMapTableF_0_RcvQPMapContext30_RMASK 0x1F
1906
1907#define QIB_7322_PSStat_0_OFFS 0x1140
1908#define QIB_7322_PSStat_0_DEF 0x0000000000000000
1909
1910#define QIB_7322_PSStart_0_OFFS 0x1148
1911#define QIB_7322_PSStart_0_DEF 0x0000000000000000
1912
1913#define QIB_7322_PSInterval_0_OFFS 0x1150
1914#define QIB_7322_PSInterval_0_DEF 0x0000000000000000
1915
1916#define QIB_7322_RcvStatus_0_OFFS 0x1160
1917#define QIB_7322_RcvStatus_0_DEF 0x0000000000000000
1918#define QIB_7322_RcvStatus_0_DmaeqBlockingContext_LSB 0x1
1919#define QIB_7322_RcvStatus_0_DmaeqBlockingContext_MSB 0x5
1920#define QIB_7322_RcvStatus_0_DmaeqBlockingContext_RMASK 0x1F
1921#define QIB_7322_RcvStatus_0_RxPktInProgress_LSB 0x0
1922#define QIB_7322_RcvStatus_0_RxPktInProgress_MSB 0x0
1923#define QIB_7322_RcvStatus_0_RxPktInProgress_RMASK 0x1
1924
1925#define QIB_7322_RcvPartitionKey_0_OFFS 0x1168
1926#define QIB_7322_RcvPartitionKey_0_DEF 0x0000000000000000
1927
1928#define QIB_7322_RcvQPMulticastContext_0_OFFS 0x1170
1929#define QIB_7322_RcvQPMulticastContext_0_DEF 0x0000000000000000
1930#define QIB_7322_RcvQPMulticastContext_0_RcvQpMcContext_LSB 0x0
1931#define QIB_7322_RcvQPMulticastContext_0_RcvQpMcContext_MSB 0x4
1932#define QIB_7322_RcvQPMulticastContext_0_RcvQpMcContext_RMASK 0x1F
1933
1934#define QIB_7322_RcvPktLEDCnt_0_OFFS 0x1178
1935#define QIB_7322_RcvPktLEDCnt_0_DEF 0x0000000000000000
1936#define QIB_7322_RcvPktLEDCnt_0_ONperiod_LSB 0x20
1937#define QIB_7322_RcvPktLEDCnt_0_ONperiod_MSB 0x3F
1938#define QIB_7322_RcvPktLEDCnt_0_ONperiod_RMASK 0xFFFFFFFF
1939#define QIB_7322_RcvPktLEDCnt_0_OFFperiod_LSB 0x0
1940#define QIB_7322_RcvPktLEDCnt_0_OFFperiod_MSB 0x1F
1941#define QIB_7322_RcvPktLEDCnt_0_OFFperiod_RMASK 0xFFFFFFFF
1942
1943#define QIB_7322_SendDmaIdleCnt_0_OFFS 0x1180
1944#define QIB_7322_SendDmaIdleCnt_0_DEF 0x0000000000000000
1945#define QIB_7322_SendDmaIdleCnt_0_SendDmaIdleCnt_LSB 0x0
1946#define QIB_7322_SendDmaIdleCnt_0_SendDmaIdleCnt_MSB 0xF
1947#define QIB_7322_SendDmaIdleCnt_0_SendDmaIdleCnt_RMASK 0xFFFF
1948
1949#define QIB_7322_SendDmaReloadCnt_0_OFFS 0x1188
1950#define QIB_7322_SendDmaReloadCnt_0_DEF 0x0000000000000000
1951#define QIB_7322_SendDmaReloadCnt_0_SendDmaReloadCnt_LSB 0x0
1952#define QIB_7322_SendDmaReloadCnt_0_SendDmaReloadCnt_MSB 0xF
1953#define QIB_7322_SendDmaReloadCnt_0_SendDmaReloadCnt_RMASK 0xFFFF
1954
1955#define QIB_7322_SendDmaDescCnt_0_OFFS 0x1190
1956#define QIB_7322_SendDmaDescCnt_0_DEF 0x0000000000000000
1957#define QIB_7322_SendDmaDescCnt_0_SendDmaDescCnt_LSB 0x0
1958#define QIB_7322_SendDmaDescCnt_0_SendDmaDescCnt_MSB 0xF
1959#define QIB_7322_SendDmaDescCnt_0_SendDmaDescCnt_RMASK 0xFFFF
1960
1961#define QIB_7322_SendCtrl_0_OFFS 0x11C0
1962#define QIB_7322_SendCtrl_0_DEF 0x0000000000000000
1963#define QIB_7322_SendCtrl_0_IBVLArbiterEn_LSB 0xF
1964#define QIB_7322_SendCtrl_0_IBVLArbiterEn_MSB 0xF
1965#define QIB_7322_SendCtrl_0_IBVLArbiterEn_RMASK 0x1
1966#define QIB_7322_SendCtrl_0_TxeDrainRmFifo_LSB 0xE
1967#define QIB_7322_SendCtrl_0_TxeDrainRmFifo_MSB 0xE
1968#define QIB_7322_SendCtrl_0_TxeDrainRmFifo_RMASK 0x1
1969#define QIB_7322_SendCtrl_0_TxeDrainLaFifo_LSB 0xD
1970#define QIB_7322_SendCtrl_0_TxeDrainLaFifo_MSB 0xD
1971#define QIB_7322_SendCtrl_0_TxeDrainLaFifo_RMASK 0x1
1972#define QIB_7322_SendCtrl_0_SDmaHalt_LSB 0xC
1973#define QIB_7322_SendCtrl_0_SDmaHalt_MSB 0xC
1974#define QIB_7322_SendCtrl_0_SDmaHalt_RMASK 0x1
1975#define QIB_7322_SendCtrl_0_SDmaEnable_LSB 0xB
1976#define QIB_7322_SendCtrl_0_SDmaEnable_MSB 0xB
1977#define QIB_7322_SendCtrl_0_SDmaEnable_RMASK 0x1
1978#define QIB_7322_SendCtrl_0_SDmaSingleDescriptor_LSB 0xA
1979#define QIB_7322_SendCtrl_0_SDmaSingleDescriptor_MSB 0xA
1980#define QIB_7322_SendCtrl_0_SDmaSingleDescriptor_RMASK 0x1
1981#define QIB_7322_SendCtrl_0_SDmaIntEnable_LSB 0x9
1982#define QIB_7322_SendCtrl_0_SDmaIntEnable_MSB 0x9
1983#define QIB_7322_SendCtrl_0_SDmaIntEnable_RMASK 0x1
1984#define QIB_7322_SendCtrl_0_SDmaCleanup_LSB 0x8
1985#define QIB_7322_SendCtrl_0_SDmaCleanup_MSB 0x8
1986#define QIB_7322_SendCtrl_0_SDmaCleanup_RMASK 0x1
1987#define QIB_7322_SendCtrl_0_ForceCreditUpToDate_LSB 0x7
1988#define QIB_7322_SendCtrl_0_ForceCreditUpToDate_MSB 0x7
1989#define QIB_7322_SendCtrl_0_ForceCreditUpToDate_RMASK 0x1
1990#define QIB_7322_SendCtrl_0_SendEnable_LSB 0x3
1991#define QIB_7322_SendCtrl_0_SendEnable_MSB 0x3
1992#define QIB_7322_SendCtrl_0_SendEnable_RMASK 0x1
1993#define QIB_7322_SendCtrl_0_TxeBypassIbc_LSB 0x1
1994#define QIB_7322_SendCtrl_0_TxeBypassIbc_MSB 0x1
1995#define QIB_7322_SendCtrl_0_TxeBypassIbc_RMASK 0x1
1996#define QIB_7322_SendCtrl_0_TxeAbortIbc_LSB 0x0
1997#define QIB_7322_SendCtrl_0_TxeAbortIbc_MSB 0x0
1998#define QIB_7322_SendCtrl_0_TxeAbortIbc_RMASK 0x1
1999
2000#define QIB_7322_SendDmaBase_0_OFFS 0x11F8
2001#define QIB_7322_SendDmaBase_0_DEF 0x0000000000000000
2002#define QIB_7322_SendDmaBase_0_SendDmaBase_LSB 0x0
2003#define QIB_7322_SendDmaBase_0_SendDmaBase_MSB 0x2F
2004#define QIB_7322_SendDmaBase_0_SendDmaBase_RMASK 0xFFFFFFFFFFFF
2005
2006#define QIB_7322_SendDmaLenGen_0_OFFS 0x1200
2007#define QIB_7322_SendDmaLenGen_0_DEF 0x0000000000000000
2008#define QIB_7322_SendDmaLenGen_0_Generation_LSB 0x10
2009#define QIB_7322_SendDmaLenGen_0_Generation_MSB 0x12
2010#define QIB_7322_SendDmaLenGen_0_Generation_RMASK 0x7
2011#define QIB_7322_SendDmaLenGen_0_Length_LSB 0x0
2012#define QIB_7322_SendDmaLenGen_0_Length_MSB 0xF
2013#define QIB_7322_SendDmaLenGen_0_Length_RMASK 0xFFFF
2014
2015#define QIB_7322_SendDmaTail_0_OFFS 0x1208
2016#define QIB_7322_SendDmaTail_0_DEF 0x0000000000000000
2017#define QIB_7322_SendDmaTail_0_SendDmaTail_LSB 0x0
2018#define QIB_7322_SendDmaTail_0_SendDmaTail_MSB 0xF
2019#define QIB_7322_SendDmaTail_0_SendDmaTail_RMASK 0xFFFF
2020
2021#define QIB_7322_SendDmaHead_0_OFFS 0x1210
2022#define QIB_7322_SendDmaHead_0_DEF 0x0000000000000000
2023#define QIB_7322_SendDmaHead_0_InternalSendDmaHead_LSB 0x20
2024#define QIB_7322_SendDmaHead_0_InternalSendDmaHead_MSB 0x2F
2025#define QIB_7322_SendDmaHead_0_InternalSendDmaHead_RMASK 0xFFFF
2026#define QIB_7322_SendDmaHead_0_SendDmaHead_LSB 0x0
2027#define QIB_7322_SendDmaHead_0_SendDmaHead_MSB 0xF
2028#define QIB_7322_SendDmaHead_0_SendDmaHead_RMASK 0xFFFF
2029
2030#define QIB_7322_SendDmaHeadAddr_0_OFFS 0x1218
2031#define QIB_7322_SendDmaHeadAddr_0_DEF 0x0000000000000000
2032#define QIB_7322_SendDmaHeadAddr_0_SendDmaHeadAddr_LSB 0x0
2033#define QIB_7322_SendDmaHeadAddr_0_SendDmaHeadAddr_MSB 0x2F
2034#define QIB_7322_SendDmaHeadAddr_0_SendDmaHeadAddr_RMASK 0xFFFFFFFFFFFF
2035
2036#define QIB_7322_SendDmaBufMask0_0_OFFS 0x1220
2037#define QIB_7322_SendDmaBufMask0_0_DEF 0x0000000000000000
2038#define QIB_7322_SendDmaBufMask0_0_BufMask_63_0_LSB 0x0
2039#define QIB_7322_SendDmaBufMask0_0_BufMask_63_0_MSB 0x3F
2040#define QIB_7322_SendDmaBufMask0_0_BufMask_63_0_RMASK 0x0
2041
2042#define QIB_7322_SendDmaStatus_0_OFFS 0x1238
2043#define QIB_7322_SendDmaStatus_0_DEF 0x0000000042000000
2044#define QIB_7322_SendDmaStatus_0_ScoreBoardDrainInProg_LSB 0x3F
2045#define QIB_7322_SendDmaStatus_0_ScoreBoardDrainInProg_MSB 0x3F
2046#define QIB_7322_SendDmaStatus_0_ScoreBoardDrainInProg_RMASK 0x1
2047#define QIB_7322_SendDmaStatus_0_HaltInProg_LSB 0x3E
2048#define QIB_7322_SendDmaStatus_0_HaltInProg_MSB 0x3E
2049#define QIB_7322_SendDmaStatus_0_HaltInProg_RMASK 0x1
2050#define QIB_7322_SendDmaStatus_0_InternalSDmaHalt_LSB 0x3D
2051#define QIB_7322_SendDmaStatus_0_InternalSDmaHalt_MSB 0x3D
2052#define QIB_7322_SendDmaStatus_0_InternalSDmaHalt_RMASK 0x1
2053#define QIB_7322_SendDmaStatus_0_ScbDescIndex_13_0_LSB 0x2F
2054#define QIB_7322_SendDmaStatus_0_ScbDescIndex_13_0_MSB 0x3C
2055#define QIB_7322_SendDmaStatus_0_ScbDescIndex_13_0_RMASK 0x3FFF
2056#define QIB_7322_SendDmaStatus_0_RpyLowAddr_6_0_LSB 0x28
2057#define QIB_7322_SendDmaStatus_0_RpyLowAddr_6_0_MSB 0x2E
2058#define QIB_7322_SendDmaStatus_0_RpyLowAddr_6_0_RMASK 0x7F
2059#define QIB_7322_SendDmaStatus_0_RpyTag_7_0_LSB 0x20
2060#define QIB_7322_SendDmaStatus_0_RpyTag_7_0_MSB 0x27
2061#define QIB_7322_SendDmaStatus_0_RpyTag_7_0_RMASK 0xFF
2062#define QIB_7322_SendDmaStatus_0_ScbFull_LSB 0x1F
2063#define QIB_7322_SendDmaStatus_0_ScbFull_MSB 0x1F
2064#define QIB_7322_SendDmaStatus_0_ScbFull_RMASK 0x1
2065#define QIB_7322_SendDmaStatus_0_ScbEmpty_LSB 0x1E
2066#define QIB_7322_SendDmaStatus_0_ScbEmpty_MSB 0x1E
2067#define QIB_7322_SendDmaStatus_0_ScbEmpty_RMASK 0x1
2068#define QIB_7322_SendDmaStatus_0_ScbEntryValid_LSB 0x1D
2069#define QIB_7322_SendDmaStatus_0_ScbEntryValid_MSB 0x1D
2070#define QIB_7322_SendDmaStatus_0_ScbEntryValid_RMASK 0x1
2071#define QIB_7322_SendDmaStatus_0_ScbFetchDescFlag_LSB 0x1C
2072#define QIB_7322_SendDmaStatus_0_ScbFetchDescFlag_MSB 0x1C
2073#define QIB_7322_SendDmaStatus_0_ScbFetchDescFlag_RMASK 0x1
2074#define QIB_7322_SendDmaStatus_0_SplFifoReadyToGo_LSB 0x1B
2075#define QIB_7322_SendDmaStatus_0_SplFifoReadyToGo_MSB 0x1B
2076#define QIB_7322_SendDmaStatus_0_SplFifoReadyToGo_RMASK 0x1
2077#define QIB_7322_SendDmaStatus_0_SplFifoDisarmed_LSB 0x1A
2078#define QIB_7322_SendDmaStatus_0_SplFifoDisarmed_MSB 0x1A
2079#define QIB_7322_SendDmaStatus_0_SplFifoDisarmed_RMASK 0x1
2080#define QIB_7322_SendDmaStatus_0_SplFifoEmpty_LSB 0x19
2081#define QIB_7322_SendDmaStatus_0_SplFifoEmpty_MSB 0x19
2082#define QIB_7322_SendDmaStatus_0_SplFifoEmpty_RMASK 0x1
2083#define QIB_7322_SendDmaStatus_0_SplFifoFull_LSB 0x18
2084#define QIB_7322_SendDmaStatus_0_SplFifoFull_MSB 0x18
2085#define QIB_7322_SendDmaStatus_0_SplFifoFull_RMASK 0x1
2086#define QIB_7322_SendDmaStatus_0_SplFifoBufNum_LSB 0x10
2087#define QIB_7322_SendDmaStatus_0_SplFifoBufNum_MSB 0x17
2088#define QIB_7322_SendDmaStatus_0_SplFifoBufNum_RMASK 0xFF
2089#define QIB_7322_SendDmaStatus_0_SplFifoDescIndex_LSB 0x0
2090#define QIB_7322_SendDmaStatus_0_SplFifoDescIndex_MSB 0xF
2091#define QIB_7322_SendDmaStatus_0_SplFifoDescIndex_RMASK 0xFFFF
2092
2093#define QIB_7322_SendDmaPriorityThld_0_OFFS 0x1258
2094#define QIB_7322_SendDmaPriorityThld_0_DEF 0x0000000000000000
2095#define QIB_7322_SendDmaPriorityThld_0_PriorityThreshold_LSB 0x0
2096#define QIB_7322_SendDmaPriorityThld_0_PriorityThreshold_MSB 0x3
2097#define QIB_7322_SendDmaPriorityThld_0_PriorityThreshold_RMASK 0xF
2098
2099#define QIB_7322_SendHdrErrSymptom_0_OFFS 0x1260
2100#define QIB_7322_SendHdrErrSymptom_0_DEF 0x0000000000000000
2101#define QIB_7322_SendHdrErrSymptom_0_NonKeyPacket_LSB 0x6
2102#define QIB_7322_SendHdrErrSymptom_0_NonKeyPacket_MSB 0x6
2103#define QIB_7322_SendHdrErrSymptom_0_NonKeyPacket_RMASK 0x1
2104#define QIB_7322_SendHdrErrSymptom_0_GRHFail_LSB 0x5
2105#define QIB_7322_SendHdrErrSymptom_0_GRHFail_MSB 0x5
2106#define QIB_7322_SendHdrErrSymptom_0_GRHFail_RMASK 0x1
2107#define QIB_7322_SendHdrErrSymptom_0_PkeyFail_LSB 0x4
2108#define QIB_7322_SendHdrErrSymptom_0_PkeyFail_MSB 0x4
2109#define QIB_7322_SendHdrErrSymptom_0_PkeyFail_RMASK 0x1
2110#define QIB_7322_SendHdrErrSymptom_0_QPFail_LSB 0x3
2111#define QIB_7322_SendHdrErrSymptom_0_QPFail_MSB 0x3
2112#define QIB_7322_SendHdrErrSymptom_0_QPFail_RMASK 0x1
2113#define QIB_7322_SendHdrErrSymptom_0_SLIDFail_LSB 0x2
2114#define QIB_7322_SendHdrErrSymptom_0_SLIDFail_MSB 0x2
2115#define QIB_7322_SendHdrErrSymptom_0_SLIDFail_RMASK 0x1
2116#define QIB_7322_SendHdrErrSymptom_0_RawIPV6_LSB 0x1
2117#define QIB_7322_SendHdrErrSymptom_0_RawIPV6_MSB 0x1
2118#define QIB_7322_SendHdrErrSymptom_0_RawIPV6_RMASK 0x1
2119#define QIB_7322_SendHdrErrSymptom_0_PacketTooSmall_LSB 0x0
2120#define QIB_7322_SendHdrErrSymptom_0_PacketTooSmall_MSB 0x0
2121#define QIB_7322_SendHdrErrSymptom_0_PacketTooSmall_RMASK 0x1
2122
2123#define QIB_7322_RxCreditVL0_0_OFFS 0x1280
2124#define QIB_7322_RxCreditVL0_0_DEF 0x0000000000000000
2125#define QIB_7322_RxCreditVL0_0_RxBufrConsumedVL_LSB 0x10
2126#define QIB_7322_RxCreditVL0_0_RxBufrConsumedVL_MSB 0x1B
2127#define QIB_7322_RxCreditVL0_0_RxBufrConsumedVL_RMASK 0xFFF
2128#define QIB_7322_RxCreditVL0_0_RxMaxCreditVL_LSB 0x0
2129#define QIB_7322_RxCreditVL0_0_RxMaxCreditVL_MSB 0xB
2130#define QIB_7322_RxCreditVL0_0_RxMaxCreditVL_RMASK 0xFFF
2131
2132#define QIB_7322_SendDmaBufUsed0_0_OFFS 0x1480
2133#define QIB_7322_SendDmaBufUsed0_0_DEF 0x0000000000000000
2134#define QIB_7322_SendDmaBufUsed0_0_BufUsed_63_0_LSB 0x0
2135#define QIB_7322_SendDmaBufUsed0_0_BufUsed_63_0_MSB 0x3F
2136#define QIB_7322_SendDmaBufUsed0_0_BufUsed_63_0_RMASK 0x0
2137
2138#define QIB_7322_SendCheckControl_0_OFFS 0x14A8
2139#define QIB_7322_SendCheckControl_0_DEF 0x0000000000000000
2140#define QIB_7322_SendCheckControl_0_PKey_En_LSB 0x4
2141#define QIB_7322_SendCheckControl_0_PKey_En_MSB 0x4
2142#define QIB_7322_SendCheckControl_0_PKey_En_RMASK 0x1
2143#define QIB_7322_SendCheckControl_0_BTHQP_En_LSB 0x3
2144#define QIB_7322_SendCheckControl_0_BTHQP_En_MSB 0x3
2145#define QIB_7322_SendCheckControl_0_BTHQP_En_RMASK 0x1
2146#define QIB_7322_SendCheckControl_0_SLID_En_LSB 0x2
2147#define QIB_7322_SendCheckControl_0_SLID_En_MSB 0x2
2148#define QIB_7322_SendCheckControl_0_SLID_En_RMASK 0x1
2149#define QIB_7322_SendCheckControl_0_RawIPV6_En_LSB 0x1
2150#define QIB_7322_SendCheckControl_0_RawIPV6_En_MSB 0x1
2151#define QIB_7322_SendCheckControl_0_RawIPV6_En_RMASK 0x1
2152#define QIB_7322_SendCheckControl_0_PacketTooSmall_En_LSB 0x0
2153#define QIB_7322_SendCheckControl_0_PacketTooSmall_En_MSB 0x0
2154#define QIB_7322_SendCheckControl_0_PacketTooSmall_En_RMASK 0x1
2155
2156#define QIB_7322_SendIBSLIDMask_0_OFFS 0x14B0
2157#define QIB_7322_SendIBSLIDMask_0_DEF 0x0000000000000000
2158#define QIB_7322_SendIBSLIDMask_0_SendIBSLIDMask_15_0_LSB 0x0
2159#define QIB_7322_SendIBSLIDMask_0_SendIBSLIDMask_15_0_MSB 0xF
2160#define QIB_7322_SendIBSLIDMask_0_SendIBSLIDMask_15_0_RMASK 0xFFFF
2161
2162#define QIB_7322_SendIBSLIDAssign_0_OFFS 0x14B8
2163#define QIB_7322_SendIBSLIDAssign_0_DEF 0x0000000000000000
2164#define QIB_7322_SendIBSLIDAssign_0_SendIBSLIDAssign_15_0_LSB 0x0
2165#define QIB_7322_SendIBSLIDAssign_0_SendIBSLIDAssign_15_0_MSB 0xF
2166#define QIB_7322_SendIBSLIDAssign_0_SendIBSLIDAssign_15_0_RMASK 0xFFFF
2167
2168#define QIB_7322_IBCStatusA_0_OFFS 0x1540
2169#define QIB_7322_IBCStatusA_0_DEF 0x0000000000000X02
2170#define QIB_7322_IBCStatusA_0_TxCreditOk_VL7_LSB 0x27
2171#define QIB_7322_IBCStatusA_0_TxCreditOk_VL7_MSB 0x27
2172#define QIB_7322_IBCStatusA_0_TxCreditOk_VL7_RMASK 0x1
2173#define QIB_7322_IBCStatusA_0_TxCreditOk_VL6_LSB 0x26
2174#define QIB_7322_IBCStatusA_0_TxCreditOk_VL6_MSB 0x26
2175#define QIB_7322_IBCStatusA_0_TxCreditOk_VL6_RMASK 0x1
2176#define QIB_7322_IBCStatusA_0_TxCreditOk_VL5_LSB 0x25
2177#define QIB_7322_IBCStatusA_0_TxCreditOk_VL5_MSB 0x25
2178#define QIB_7322_IBCStatusA_0_TxCreditOk_VL5_RMASK 0x1
2179#define QIB_7322_IBCStatusA_0_TxCreditOk_VL4_LSB 0x24
2180#define QIB_7322_IBCStatusA_0_TxCreditOk_VL4_MSB 0x24
2181#define QIB_7322_IBCStatusA_0_TxCreditOk_VL4_RMASK 0x1
2182#define QIB_7322_IBCStatusA_0_TxCreditOk_VL3_LSB 0x23
2183#define QIB_7322_IBCStatusA_0_TxCreditOk_VL3_MSB 0x23
2184#define QIB_7322_IBCStatusA_0_TxCreditOk_VL3_RMASK 0x1
2185#define QIB_7322_IBCStatusA_0_TxCreditOk_VL2_LSB 0x22
2186#define QIB_7322_IBCStatusA_0_TxCreditOk_VL2_MSB 0x22
2187#define QIB_7322_IBCStatusA_0_TxCreditOk_VL2_RMASK 0x1
2188#define QIB_7322_IBCStatusA_0_TxCreditOk_VL1_LSB 0x21
2189#define QIB_7322_IBCStatusA_0_TxCreditOk_VL1_MSB 0x21
2190#define QIB_7322_IBCStatusA_0_TxCreditOk_VL1_RMASK 0x1
2191#define QIB_7322_IBCStatusA_0_TxCreditOk_VL0_LSB 0x20
2192#define QIB_7322_IBCStatusA_0_TxCreditOk_VL0_MSB 0x20
2193#define QIB_7322_IBCStatusA_0_TxCreditOk_VL0_RMASK 0x1
2194#define QIB_7322_IBCStatusA_0_TxReady_LSB 0x1E
2195#define QIB_7322_IBCStatusA_0_TxReady_MSB 0x1E
2196#define QIB_7322_IBCStatusA_0_TxReady_RMASK 0x1
2197#define QIB_7322_IBCStatusA_0_LinkSpeedQDR_LSB 0x1D
2198#define QIB_7322_IBCStatusA_0_LinkSpeedQDR_MSB 0x1D
2199#define QIB_7322_IBCStatusA_0_LinkSpeedQDR_RMASK 0x1
2200#define QIB_7322_IBCStatusA_0_ScrambleCapRemote_LSB 0xF
2201#define QIB_7322_IBCStatusA_0_ScrambleCapRemote_MSB 0xF
2202#define QIB_7322_IBCStatusA_0_ScrambleCapRemote_RMASK 0x1
2203#define QIB_7322_IBCStatusA_0_ScrambleEn_LSB 0xE
2204#define QIB_7322_IBCStatusA_0_ScrambleEn_MSB 0xE
2205#define QIB_7322_IBCStatusA_0_ScrambleEn_RMASK 0x1
2206#define QIB_7322_IBCStatusA_0_IBTxLaneReversed_LSB 0xD
2207#define QIB_7322_IBCStatusA_0_IBTxLaneReversed_MSB 0xD
2208#define QIB_7322_IBCStatusA_0_IBTxLaneReversed_RMASK 0x1
2209#define QIB_7322_IBCStatusA_0_IBRxLaneReversed_LSB 0xC
2210#define QIB_7322_IBCStatusA_0_IBRxLaneReversed_MSB 0xC
2211#define QIB_7322_IBCStatusA_0_IBRxLaneReversed_RMASK 0x1
2212#define QIB_7322_IBCStatusA_0_DDS_RXEQ_FAIL_LSB 0xA
2213#define QIB_7322_IBCStatusA_0_DDS_RXEQ_FAIL_MSB 0xA
2214#define QIB_7322_IBCStatusA_0_DDS_RXEQ_FAIL_RMASK 0x1
2215#define QIB_7322_IBCStatusA_0_LinkWidthActive_LSB 0x9
2216#define QIB_7322_IBCStatusA_0_LinkWidthActive_MSB 0x9
2217#define QIB_7322_IBCStatusA_0_LinkWidthActive_RMASK 0x1
2218#define QIB_7322_IBCStatusA_0_LinkSpeedActive_LSB 0x8
2219#define QIB_7322_IBCStatusA_0_LinkSpeedActive_MSB 0x8
2220#define QIB_7322_IBCStatusA_0_LinkSpeedActive_RMASK 0x1
2221#define QIB_7322_IBCStatusA_0_LinkState_LSB 0x5
2222#define QIB_7322_IBCStatusA_0_LinkState_MSB 0x7
2223#define QIB_7322_IBCStatusA_0_LinkState_RMASK 0x7
2224#define QIB_7322_IBCStatusA_0_LinkTrainingState_LSB 0x0
2225#define QIB_7322_IBCStatusA_0_LinkTrainingState_MSB 0x4
2226#define QIB_7322_IBCStatusA_0_LinkTrainingState_RMASK 0x1F
2227
2228#define QIB_7322_IBCStatusB_0_OFFS 0x1548
2229#define QIB_7322_IBCStatusB_0_DEF 0x00000000XXXXXXXX
2230#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_debug_LSB 0x27
2231#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_debug_MSB 0x27
2232#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_debug_RMASK 0x1
2233#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_reached_threshold_LSB 0x26
2234#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_reached_threshold_MSB 0x26
2235#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_reached_threshold_RMASK 0x1
2236#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_started_LSB 0x25
2237#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_started_MSB 0x25
2238#define QIB_7322_IBCStatusB_0_ibsd_adaptation_timer_started_RMASK 0x1
2239#define QIB_7322_IBCStatusB_0_heartbeat_timed_out_LSB 0x24
2240#define QIB_7322_IBCStatusB_0_heartbeat_timed_out_MSB 0x24
2241#define QIB_7322_IBCStatusB_0_heartbeat_timed_out_RMASK 0x1
2242#define QIB_7322_IBCStatusB_0_heartbeat_crosstalk_LSB 0x20
2243#define QIB_7322_IBCStatusB_0_heartbeat_crosstalk_MSB 0x23
2244#define QIB_7322_IBCStatusB_0_heartbeat_crosstalk_RMASK 0xF
2245#define QIB_7322_IBCStatusB_0_RxEqLocalDevice_LSB 0x1E
2246#define QIB_7322_IBCStatusB_0_RxEqLocalDevice_MSB 0x1F
2247#define QIB_7322_IBCStatusB_0_RxEqLocalDevice_RMASK 0x3
2248#define QIB_7322_IBCStatusB_0_ReqDDSLocalFromRmt_LSB 0x1A
2249#define QIB_7322_IBCStatusB_0_ReqDDSLocalFromRmt_MSB 0x1D
2250#define QIB_7322_IBCStatusB_0_ReqDDSLocalFromRmt_RMASK 0xF
2251#define QIB_7322_IBCStatusB_0_LinkRoundTripLatency_LSB 0x0
2252#define QIB_7322_IBCStatusB_0_LinkRoundTripLatency_MSB 0x19
2253#define QIB_7322_IBCStatusB_0_LinkRoundTripLatency_RMASK 0x3FFFFFF
2254
2255#define QIB_7322_IBCCtrlA_0_OFFS 0x1560
2256#define QIB_7322_IBCCtrlA_0_DEF 0x0000000000000000
2257#define QIB_7322_IBCCtrlA_0_Loopback_LSB 0x3F
2258#define QIB_7322_IBCCtrlA_0_Loopback_MSB 0x3F
2259#define QIB_7322_IBCCtrlA_0_Loopback_RMASK 0x1
2260#define QIB_7322_IBCCtrlA_0_LinkDownDefaultState_LSB 0x3E
2261#define QIB_7322_IBCCtrlA_0_LinkDownDefaultState_MSB 0x3E
2262#define QIB_7322_IBCCtrlA_0_LinkDownDefaultState_RMASK 0x1
2263#define QIB_7322_IBCCtrlA_0_IBLinkEn_LSB 0x3D
2264#define QIB_7322_IBCCtrlA_0_IBLinkEn_MSB 0x3D
2265#define QIB_7322_IBCCtrlA_0_IBLinkEn_RMASK 0x1
2266#define QIB_7322_IBCCtrlA_0_IBStatIntReductionEn_LSB 0x3C
2267#define QIB_7322_IBCCtrlA_0_IBStatIntReductionEn_MSB 0x3C
2268#define QIB_7322_IBCCtrlA_0_IBStatIntReductionEn_RMASK 0x1
2269#define QIB_7322_IBCCtrlA_0_NumVLane_LSB 0x30
2270#define QIB_7322_IBCCtrlA_0_NumVLane_MSB 0x32
2271#define QIB_7322_IBCCtrlA_0_NumVLane_RMASK 0x7
2272#define QIB_7322_IBCCtrlA_0_OverrunThreshold_LSB 0x24
2273#define QIB_7322_IBCCtrlA_0_OverrunThreshold_MSB 0x27
2274#define QIB_7322_IBCCtrlA_0_OverrunThreshold_RMASK 0xF
2275#define QIB_7322_IBCCtrlA_0_PhyerrThreshold_LSB 0x20
2276#define QIB_7322_IBCCtrlA_0_PhyerrThreshold_MSB 0x23
2277#define QIB_7322_IBCCtrlA_0_PhyerrThreshold_RMASK 0xF
2278#define QIB_7322_IBCCtrlA_0_MaxPktLen_LSB 0x15
2279#define QIB_7322_IBCCtrlA_0_MaxPktLen_MSB 0x1F
2280#define QIB_7322_IBCCtrlA_0_MaxPktLen_RMASK 0x7FF
2281#define QIB_7322_IBCCtrlA_0_LinkCmd_LSB 0x13
2282#define QIB_7322_IBCCtrlA_0_LinkCmd_MSB 0x14
2283#define QIB_7322_IBCCtrlA_0_LinkCmd_RMASK 0x3
2284#define QIB_7322_IBCCtrlA_0_LinkInitCmd_LSB 0x10
2285#define QIB_7322_IBCCtrlA_0_LinkInitCmd_MSB 0x12
2286#define QIB_7322_IBCCtrlA_0_LinkInitCmd_RMASK 0x7
2287#define QIB_7322_IBCCtrlA_0_FlowCtrlWaterMark_LSB 0x8
2288#define QIB_7322_IBCCtrlA_0_FlowCtrlWaterMark_MSB 0xF
2289#define QIB_7322_IBCCtrlA_0_FlowCtrlWaterMark_RMASK 0xFF
2290#define QIB_7322_IBCCtrlA_0_FlowCtrlPeriod_LSB 0x0
2291#define QIB_7322_IBCCtrlA_0_FlowCtrlPeriod_MSB 0x7
2292#define QIB_7322_IBCCtrlA_0_FlowCtrlPeriod_RMASK 0xFF
2293
2294#define QIB_7322_IBCCtrlB_0_OFFS 0x1568
2295#define QIB_7322_IBCCtrlB_0_DEF 0x00000000000305FF
2296#define QIB_7322_IBCCtrlB_0_IB_DLID_MASK_LSB 0x30
2297#define QIB_7322_IBCCtrlB_0_IB_DLID_MASK_MSB 0x3F
2298#define QIB_7322_IBCCtrlB_0_IB_DLID_MASK_RMASK 0xFFFF
2299#define QIB_7322_IBCCtrlB_0_IB_DLID_LSB 0x20
2300#define QIB_7322_IBCCtrlB_0_IB_DLID_MSB 0x2F
2301#define QIB_7322_IBCCtrlB_0_IB_DLID_RMASK 0xFFFF
2302#define QIB_7322_IBCCtrlB_0_IB_ENABLE_FILT_DPKT_LSB 0x1B
2303#define QIB_7322_IBCCtrlB_0_IB_ENABLE_FILT_DPKT_MSB 0x1B
2304#define QIB_7322_IBCCtrlB_0_IB_ENABLE_FILT_DPKT_RMASK 0x1
2305#define QIB_7322_IBCCtrlB_0_HRTBT_REQ_LSB 0x1A
2306#define QIB_7322_IBCCtrlB_0_HRTBT_REQ_MSB 0x1A
2307#define QIB_7322_IBCCtrlB_0_HRTBT_REQ_RMASK 0x1
2308#define QIB_7322_IBCCtrlB_0_HRTBT_PORT_LSB 0x12
2309#define QIB_7322_IBCCtrlB_0_HRTBT_PORT_MSB 0x19
2310#define QIB_7322_IBCCtrlB_0_HRTBT_PORT_RMASK 0xFF
2311#define QIB_7322_IBCCtrlB_0_HRTBT_AUTO_LSB 0x11
2312#define QIB_7322_IBCCtrlB_0_HRTBT_AUTO_MSB 0x11
2313#define QIB_7322_IBCCtrlB_0_HRTBT_AUTO_RMASK 0x1
2314#define QIB_7322_IBCCtrlB_0_HRTBT_ENB_LSB 0x10
2315#define QIB_7322_IBCCtrlB_0_HRTBT_ENB_MSB 0x10
2316#define QIB_7322_IBCCtrlB_0_HRTBT_ENB_RMASK 0x1
2317#define QIB_7322_IBCCtrlB_0_SD_DDS_LSB 0xC
2318#define QIB_7322_IBCCtrlB_0_SD_DDS_MSB 0xF
2319#define QIB_7322_IBCCtrlB_0_SD_DDS_RMASK 0xF
2320#define QIB_7322_IBCCtrlB_0_SD_DDSV_LSB 0xB
2321#define QIB_7322_IBCCtrlB_0_SD_DDSV_MSB 0xB
2322#define QIB_7322_IBCCtrlB_0_SD_DDSV_RMASK 0x1
2323#define QIB_7322_IBCCtrlB_0_SD_ADD_ENB_LSB 0xA
2324#define QIB_7322_IBCCtrlB_0_SD_ADD_ENB_MSB 0xA
2325#define QIB_7322_IBCCtrlB_0_SD_ADD_ENB_RMASK 0x1
2326#define QIB_7322_IBCCtrlB_0_SD_RX_EQUAL_ENABLE_LSB 0x9
2327#define QIB_7322_IBCCtrlB_0_SD_RX_EQUAL_ENABLE_MSB 0x9
2328#define QIB_7322_IBCCtrlB_0_SD_RX_EQUAL_ENABLE_RMASK 0x1
2329#define QIB_7322_IBCCtrlB_0_IB_LANE_REV_SUPPORTED_LSB 0x8
2330#define QIB_7322_IBCCtrlB_0_IB_LANE_REV_SUPPORTED_MSB 0x8
2331#define QIB_7322_IBCCtrlB_0_IB_LANE_REV_SUPPORTED_RMASK 0x1
2332#define QIB_7322_IBCCtrlB_0_IB_POLARITY_REV_SUPP_LSB 0x7
2333#define QIB_7322_IBCCtrlB_0_IB_POLARITY_REV_SUPP_MSB 0x7
2334#define QIB_7322_IBCCtrlB_0_IB_POLARITY_REV_SUPP_RMASK 0x1
2335#define QIB_7322_IBCCtrlB_0_IB_NUM_CHANNELS_LSB 0x5
2336#define QIB_7322_IBCCtrlB_0_IB_NUM_CHANNELS_MSB 0x6
2337#define QIB_7322_IBCCtrlB_0_IB_NUM_CHANNELS_RMASK 0x3
2338#define QIB_7322_IBCCtrlB_0_SD_SPEED_QDR_LSB 0x4
2339#define QIB_7322_IBCCtrlB_0_SD_SPEED_QDR_MSB 0x4
2340#define QIB_7322_IBCCtrlB_0_SD_SPEED_QDR_RMASK 0x1
2341#define QIB_7322_IBCCtrlB_0_SD_SPEED_DDR_LSB 0x3
2342#define QIB_7322_IBCCtrlB_0_SD_SPEED_DDR_MSB 0x3
2343#define QIB_7322_IBCCtrlB_0_SD_SPEED_DDR_RMASK 0x1
2344#define QIB_7322_IBCCtrlB_0_SD_SPEED_SDR_LSB 0x2
2345#define QIB_7322_IBCCtrlB_0_SD_SPEED_SDR_MSB 0x2
2346#define QIB_7322_IBCCtrlB_0_SD_SPEED_SDR_RMASK 0x1
2347#define QIB_7322_IBCCtrlB_0_SD_SPEED_LSB 0x1
2348#define QIB_7322_IBCCtrlB_0_SD_SPEED_MSB 0x1
2349#define QIB_7322_IBCCtrlB_0_SD_SPEED_RMASK 0x1
2350#define QIB_7322_IBCCtrlB_0_IB_ENHANCED_MODE_LSB 0x0
2351#define QIB_7322_IBCCtrlB_0_IB_ENHANCED_MODE_MSB 0x0
2352#define QIB_7322_IBCCtrlB_0_IB_ENHANCED_MODE_RMASK 0x1
2353
2354#define QIB_7322_IBCCtrlC_0_OFFS 0x1570
2355#define QIB_7322_IBCCtrlC_0_DEF 0x0000000000000301
2356#define QIB_7322_IBCCtrlC_0_IB_BACK_PORCH_LSB 0x5
2357#define QIB_7322_IBCCtrlC_0_IB_BACK_PORCH_MSB 0x9
2358#define QIB_7322_IBCCtrlC_0_IB_BACK_PORCH_RMASK 0x1F
2359#define QIB_7322_IBCCtrlC_0_IB_FRONT_PORCH_LSB 0x0
2360#define QIB_7322_IBCCtrlC_0_IB_FRONT_PORCH_MSB 0x4
2361#define QIB_7322_IBCCtrlC_0_IB_FRONT_PORCH_RMASK 0x1F
2362
2363#define QIB_7322_HRTBT_GUID_0_OFFS 0x1588
2364#define QIB_7322_HRTBT_GUID_0_DEF 0x0000000000000000
2365
2366#define QIB_7322_IB_SDTEST_IF_TX_0_OFFS 0x1590
2367#define QIB_7322_IB_SDTEST_IF_TX_0_DEF 0x0000000000000000
2368#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_RX_CFG_LSB 0x30
2369#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_RX_CFG_MSB 0x3F
2370#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_RX_CFG_RMASK 0xFFFF
2371#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_TX_CFG_LSB 0x20
2372#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_TX_CFG_MSB 0x2F
2373#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_TX_CFG_RMASK 0xFFFF
2374#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_SPEED_LSB 0xD
2375#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_SPEED_MSB 0xF
2376#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_SPEED_RMASK 0x7
2377#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_OPCODE_LSB 0xB
2378#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_OPCODE_MSB 0xC
2379#define QIB_7322_IB_SDTEST_IF_TX_0_TS_TX_OPCODE_RMASK 0x3
2380#define QIB_7322_IB_SDTEST_IF_TX_0_CREDIT_CHANGE_LSB 0x4
2381#define QIB_7322_IB_SDTEST_IF_TX_0_CREDIT_CHANGE_MSB 0x4
2382#define QIB_7322_IB_SDTEST_IF_TX_0_CREDIT_CHANGE_RMASK 0x1
2383#define QIB_7322_IB_SDTEST_IF_TX_0_VL_CAP_LSB 0x2
2384#define QIB_7322_IB_SDTEST_IF_TX_0_VL_CAP_MSB 0x3
2385#define QIB_7322_IB_SDTEST_IF_TX_0_VL_CAP_RMASK 0x3
2386#define QIB_7322_IB_SDTEST_IF_TX_0_TS_3_TX_VALID_LSB 0x1
2387#define QIB_7322_IB_SDTEST_IF_TX_0_TS_3_TX_VALID_MSB 0x1
2388#define QIB_7322_IB_SDTEST_IF_TX_0_TS_3_TX_VALID_RMASK 0x1
2389#define QIB_7322_IB_SDTEST_IF_TX_0_TS_T_TX_VALID_LSB 0x0
2390#define QIB_7322_IB_SDTEST_IF_TX_0_TS_T_TX_VALID_MSB 0x0
2391#define QIB_7322_IB_SDTEST_IF_TX_0_TS_T_TX_VALID_RMASK 0x1
2392
2393#define QIB_7322_IB_SDTEST_IF_RX_0_OFFS 0x1598
2394#define QIB_7322_IB_SDTEST_IF_RX_0_DEF 0x0000000000000000
2395#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_RX_CFG_LSB 0x30
2396#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_RX_CFG_MSB 0x3F
2397#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_RX_CFG_RMASK 0xFFFF
2398#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_TX_CFG_LSB 0x20
2399#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_TX_CFG_MSB 0x2F
2400#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_TX_CFG_RMASK 0xFFFF
2401#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_B_LSB 0x18
2402#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_B_MSB 0x1F
2403#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_B_RMASK 0xFF
2404#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_A_LSB 0x10
2405#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_A_MSB 0x17
2406#define QIB_7322_IB_SDTEST_IF_RX_0_TS_RX_A_RMASK 0xFF
2407#define QIB_7322_IB_SDTEST_IF_RX_0_TS_3_RX_VALID_LSB 0x1
2408#define QIB_7322_IB_SDTEST_IF_RX_0_TS_3_RX_VALID_MSB 0x1
2409#define QIB_7322_IB_SDTEST_IF_RX_0_TS_3_RX_VALID_RMASK 0x1
2410#define QIB_7322_IB_SDTEST_IF_RX_0_TS_T_RX_VALID_LSB 0x0
2411#define QIB_7322_IB_SDTEST_IF_RX_0_TS_T_RX_VALID_MSB 0x0
2412#define QIB_7322_IB_SDTEST_IF_RX_0_TS_T_RX_VALID_RMASK 0x1
2413
2414#define QIB_7322_IBNCModeCtrl_0_OFFS 0x15B8
2415#define QIB_7322_IBNCModeCtrl_0_DEF 0x0000000000000000
2416#define QIB_7322_IBNCModeCtrl_0_ScrambleCapRemoteForce_LSB 0x22
2417#define QIB_7322_IBNCModeCtrl_0_ScrambleCapRemoteForce_MSB 0x22
2418#define QIB_7322_IBNCModeCtrl_0_ScrambleCapRemoteForce_RMASK 0x1
2419#define QIB_7322_IBNCModeCtrl_0_ScrambleCapRemoteMask_LSB 0x21
2420#define QIB_7322_IBNCModeCtrl_0_ScrambleCapRemoteMask_MSB 0x21
2421#define QIB_7322_IBNCModeCtrl_0_ScrambleCapRemoteMask_RMASK 0x1
2422#define QIB_7322_IBNCModeCtrl_0_ScrambleCapLocal_LSB 0x20
2423#define QIB_7322_IBNCModeCtrl_0_ScrambleCapLocal_MSB 0x20
2424#define QIB_7322_IBNCModeCtrl_0_ScrambleCapLocal_RMASK 0x1
2425#define QIB_7322_IBNCModeCtrl_0_TSMCode_TS2_LSB 0x11
2426#define QIB_7322_IBNCModeCtrl_0_TSMCode_TS2_MSB 0x19
2427#define QIB_7322_IBNCModeCtrl_0_TSMCode_TS2_RMASK 0x1FF
2428#define QIB_7322_IBNCModeCtrl_0_TSMCode_TS1_LSB 0x8
2429#define QIB_7322_IBNCModeCtrl_0_TSMCode_TS1_MSB 0x10
2430#define QIB_7322_IBNCModeCtrl_0_TSMCode_TS1_RMASK 0x1FF
2431#define QIB_7322_IBNCModeCtrl_0_TSMEnable_ignore_TSM_on_rx_LSB 0x2
2432#define QIB_7322_IBNCModeCtrl_0_TSMEnable_ignore_TSM_on_rx_MSB 0x2
2433#define QIB_7322_IBNCModeCtrl_0_TSMEnable_ignore_TSM_on_rx_RMASK 0x1
2434#define QIB_7322_IBNCModeCtrl_0_TSMEnable_send_TS2_LSB 0x1
2435#define QIB_7322_IBNCModeCtrl_0_TSMEnable_send_TS2_MSB 0x1
2436#define QIB_7322_IBNCModeCtrl_0_TSMEnable_send_TS2_RMASK 0x1
2437#define QIB_7322_IBNCModeCtrl_0_TSMEnable_send_TS1_LSB 0x0
2438#define QIB_7322_IBNCModeCtrl_0_TSMEnable_send_TS1_MSB 0x0
2439#define QIB_7322_IBNCModeCtrl_0_TSMEnable_send_TS1_RMASK 0x1
2440
2441#define QIB_7322_IBSerdesStatus_0_OFFS 0x15D0
2442#define QIB_7322_IBSerdesStatus_0_DEF 0x0000000000000000
2443
2444#define QIB_7322_IBPCSConfig_0_OFFS 0x15D8
2445#define QIB_7322_IBPCSConfig_0_DEF 0x0000000000000007
2446#define QIB_7322_IBPCSConfig_0_link_sync_mask_LSB 0x9
2447#define QIB_7322_IBPCSConfig_0_link_sync_mask_MSB 0x12
2448#define QIB_7322_IBPCSConfig_0_link_sync_mask_RMASK 0x3FF
2449#define QIB_7322_IBPCSConfig_0_xcv_rreset_LSB 0x2
2450#define QIB_7322_IBPCSConfig_0_xcv_rreset_MSB 0x2
2451#define QIB_7322_IBPCSConfig_0_xcv_rreset_RMASK 0x1
2452#define QIB_7322_IBPCSConfig_0_xcv_treset_LSB 0x1
2453#define QIB_7322_IBPCSConfig_0_xcv_treset_MSB 0x1
2454#define QIB_7322_IBPCSConfig_0_xcv_treset_RMASK 0x1
2455#define QIB_7322_IBPCSConfig_0_tx_rx_reset_LSB 0x0
2456#define QIB_7322_IBPCSConfig_0_tx_rx_reset_MSB 0x0
2457#define QIB_7322_IBPCSConfig_0_tx_rx_reset_RMASK 0x1
2458
2459#define QIB_7322_IBSerdesCtrl_0_OFFS 0x15E0
2460#define QIB_7322_IBSerdesCtrl_0_DEF 0x0000000000FFA00F
2461#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_QDR_LSB 0x1A
2462#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_QDR_MSB 0x1A
2463#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_QDR_RMASK 0x1
2464#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_DDR_LSB 0x19
2465#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_DDR_MSB 0x19
2466#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_DDR_RMASK 0x1
2467#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_SDR_LSB 0x18
2468#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_SDR_MSB 0x18
2469#define QIB_7322_IBSerdesCtrl_0_DISABLE_RXLATOFF_SDR_RMASK 0x1
2470#define QIB_7322_IBSerdesCtrl_0_CHANNEL_RESET_N_LSB 0x14
2471#define QIB_7322_IBSerdesCtrl_0_CHANNEL_RESET_N_MSB 0x17
2472#define QIB_7322_IBSerdesCtrl_0_CHANNEL_RESET_N_RMASK 0xF
2473#define QIB_7322_IBSerdesCtrl_0_CGMODE_LSB 0x10
2474#define QIB_7322_IBSerdesCtrl_0_CGMODE_MSB 0x13
2475#define QIB_7322_IBSerdesCtrl_0_CGMODE_RMASK 0xF
2476#define QIB_7322_IBSerdesCtrl_0_IB_LAT_MODE_LSB 0xF
2477#define QIB_7322_IBSerdesCtrl_0_IB_LAT_MODE_MSB 0xF
2478#define QIB_7322_IBSerdesCtrl_0_IB_LAT_MODE_RMASK 0x1
2479#define QIB_7322_IBSerdesCtrl_0_RXLOSEN_LSB 0xD
2480#define QIB_7322_IBSerdesCtrl_0_RXLOSEN_MSB 0xD
2481#define QIB_7322_IBSerdesCtrl_0_RXLOSEN_RMASK 0x1
2482#define QIB_7322_IBSerdesCtrl_0_LPEN_LSB 0xC
2483#define QIB_7322_IBSerdesCtrl_0_LPEN_MSB 0xC
2484#define QIB_7322_IBSerdesCtrl_0_LPEN_RMASK 0x1
2485#define QIB_7322_IBSerdesCtrl_0_PLLPD_LSB 0xB
2486#define QIB_7322_IBSerdesCtrl_0_PLLPD_MSB 0xB
2487#define QIB_7322_IBSerdesCtrl_0_PLLPD_RMASK 0x1
2488#define QIB_7322_IBSerdesCtrl_0_TXPD_LSB 0xA
2489#define QIB_7322_IBSerdesCtrl_0_TXPD_MSB 0xA
2490#define QIB_7322_IBSerdesCtrl_0_TXPD_RMASK 0x1
2491#define QIB_7322_IBSerdesCtrl_0_RXPD_LSB 0x9
2492#define QIB_7322_IBSerdesCtrl_0_RXPD_MSB 0x9
2493#define QIB_7322_IBSerdesCtrl_0_RXPD_RMASK 0x1
2494#define QIB_7322_IBSerdesCtrl_0_TXIDLE_LSB 0x8
2495#define QIB_7322_IBSerdesCtrl_0_TXIDLE_MSB 0x8
2496#define QIB_7322_IBSerdesCtrl_0_TXIDLE_RMASK 0x1
2497#define QIB_7322_IBSerdesCtrl_0_CMODE_LSB 0x0
2498#define QIB_7322_IBSerdesCtrl_0_CMODE_MSB 0x6
2499#define QIB_7322_IBSerdesCtrl_0_CMODE_RMASK 0x7F
2500
2501#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_OFFS 0x1600
2502#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_DEF 0x0000000000000000
2503#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_tx_override_deemphasis_select_LSB 0x1F
2504#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_tx_override_deemphasis_select_MSB 0x1F
2505#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_tx_override_deemphasis_select_RMASK 0x1
2506#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_reset_tx_deemphasis_override_LSB 0x1E
2507#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_reset_tx_deemphasis_override_MSB 0x1E
2508#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_reset_tx_deemphasis_override_RMASK 0x1
2509#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txampcntl_d2a_LSB 0xE
2510#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txampcntl_d2a_MSB 0x11
2511#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txampcntl_d2a_RMASK 0xF
2512#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txc0_ena_LSB 0x9
2513#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txc0_ena_MSB 0xD
2514#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txc0_ena_RMASK 0x1F
2515#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcp1_ena_LSB 0x5
2516#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcp1_ena_MSB 0x8
2517#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcp1_ena_RMASK 0xF
2518#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcn1_xtra_emph0_LSB 0x3
2519#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcn1_xtra_emph0_MSB 0x4
2520#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcn1_xtra_emph0_RMASK 0x3
2521#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcn1_ena_LSB 0x0
2522#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcn1_ena_MSB 0x2
2523#define QIB_7322_IBSD_TX_DEEMPHASIS_OVERRIDE_0_txcn1_ena_RMASK 0x7
2524
2525#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_OFFS 0x1640
2526#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_DEF 0x0000000000000000
2527#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch3_LSB 0x27
2528#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch3_MSB 0x27
2529#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch3_RMASK 0x1
2530#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch2_LSB 0x26
2531#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch2_MSB 0x26
2532#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch2_RMASK 0x1
2533#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch1_LSB 0x25
2534#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch1_MSB 0x25
2535#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch1_RMASK 0x1
2536#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch0_LSB 0x24
2537#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch0_MSB 0x24
2538#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenagain_sdr_ch0_RMASK 0x1
2539#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch3_LSB 0x23
2540#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch3_MSB 0x23
2541#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch3_RMASK 0x1
2542#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch2_LSB 0x22
2543#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch2_MSB 0x22
2544#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch2_RMASK 0x1
2545#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch1_LSB 0x21
2546#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch1_MSB 0x21
2547#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch1_RMASK 0x1
2548#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch0_LSB 0x20
2549#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch0_MSB 0x20
2550#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenale_sdr_ch0_RMASK 0x1
2551#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch3_LSB 0x18
2552#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch3_MSB 0x1F
2553#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch3_RMASK 0xFF
2554#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch2_LSB 0x10
2555#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch2_MSB 0x17
2556#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch2_RMASK 0xFF
2557#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch1_LSB 0x8
2558#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch1_MSB 0xF
2559#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch1_RMASK 0xFF
2560#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch0_LSB 0x0
2561#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch0_MSB 0x7
2562#define QIB_7322_ADAPT_DISABLE_STATIC_SDR_0_static_disable_rxenadfe_sdr_ch0_RMASK 0xFF
2563
2564#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_OFFS 0x1648
2565#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_DEF 0x0000000000000000
2566#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch3_LSB 0x27
2567#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch3_MSB 0x27
2568#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch3_RMASK 0x1
2569#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch2_LSB 0x26
2570#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch2_MSB 0x26
2571#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch2_RMASK 0x1
2572#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch1_LSB 0x25
2573#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch1_MSB 0x25
2574#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch1_RMASK 0x1
2575#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch0_LSB 0x24
2576#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch0_MSB 0x24
2577#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenagain_sdr_ch0_RMASK 0x1
2578#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch3_LSB 0x23
2579#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch3_MSB 0x23
2580#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch3_RMASK 0x1
2581#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch2_LSB 0x22
2582#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch2_MSB 0x22
2583#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch2_RMASK 0x1
2584#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch1_LSB 0x21
2585#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch1_MSB 0x21
2586#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch1_RMASK 0x1
2587#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch0_LSB 0x20
2588#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch0_MSB 0x20
2589#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenale_sdr_ch0_RMASK 0x1
2590#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch3_LSB 0x18
2591#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch3_MSB 0x1F
2592#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch3_RMASK 0xFF
2593#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch2_LSB 0x10
2594#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch2_MSB 0x17
2595#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch2_RMASK 0xFF
2596#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch1_LSB 0x8
2597#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch1_MSB 0xF
2598#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch1_RMASK 0xFF
2599#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch0_LSB 0x0
2600#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch0_MSB 0x7
2601#define QIB_7322_ADAPT_DISABLE_DYNAMIC_SDR_0_dyn_disable_rxenadfe_sdr_ch0_RMASK 0xFF
2602
2603#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_OFFS 0x1650
2604#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_DEF 0x0000000000000000
2605#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch3_LSB 0x27
2606#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch3_MSB 0x27
2607#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch3_RMASK 0x1
2608#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch2_LSB 0x26
2609#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch2_MSB 0x26
2610#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch2_RMASK 0x1
2611#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch1_LSB 0x25
2612#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch1_MSB 0x25
2613#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch1_RMASK 0x1
2614#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch0_LSB 0x24
2615#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch0_MSB 0x24
2616#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenagain_ddr_ch0_RMASK 0x1
2617#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch3_LSB 0x23
2618#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch3_MSB 0x23
2619#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch3_RMASK 0x1
2620#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch2_LSB 0x22
2621#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch2_MSB 0x22
2622#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch2_RMASK 0x1
2623#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch1_LSB 0x21
2624#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch1_MSB 0x21
2625#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch1_RMASK 0x1
2626#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch0_LSB 0x20
2627#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch0_MSB 0x20
2628#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenale_ddr_ch0_RMASK 0x1
2629#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch3_LSB 0x18
2630#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch3_MSB 0x1F
2631#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch3_RMASK 0xFF
2632#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch2_LSB 0x10
2633#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch2_MSB 0x17
2634#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch2_RMASK 0xFF
2635#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch1_LSB 0x8
2636#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch1_MSB 0xF
2637#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch1_RMASK 0xFF
2638#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch0_LSB 0x0
2639#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch0_MSB 0x7
2640#define QIB_7322_ADAPT_DISABLE_STATIC_DDR_0_static_disable_rxenadfe_ddr_ch0_RMASK 0xFF
2641
2642#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_OFFS 0x1658
2643#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_DEF 0x0000000000000000
2644#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch3_LSB 0x27
2645#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch3_MSB 0x27
2646#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch3_RMASK 0x1
2647#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch2_LSB 0x26
2648#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch2_MSB 0x26
2649#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch2_RMASK 0x1
2650#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch1_LSB 0x25
2651#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch1_MSB 0x25
2652#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch1_RMASK 0x1
2653#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch0_LSB 0x24
2654#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch0_MSB 0x24
2655#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenagain_ddr_ch0_RMASK 0x1
2656#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch3_LSB 0x23
2657#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch3_MSB 0x23
2658#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch3_RMASK 0x1
2659#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch2_LSB 0x22
2660#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch2_MSB 0x22
2661#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch2_RMASK 0x1
2662#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch1_LSB 0x21
2663#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch1_MSB 0x21
2664#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch1_RMASK 0x1
2665#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch0_LSB 0x20
2666#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch0_MSB 0x20
2667#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenale_ddr_ch0_RMASK 0x1
2668#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch3_LSB 0x18
2669#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch3_MSB 0x1F
2670#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch3_RMASK 0xFF
2671#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch2_LSB 0x10
2672#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch2_MSB 0x17
2673#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch2_RMASK 0xFF
2674#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch1_LSB 0x8
2675#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch1_MSB 0xF
2676#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch1_RMASK 0xFF
2677#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch0_LSB 0x0
2678#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch0_MSB 0x7
2679#define QIB_7322_ADAPT_DISABLE_DYNAMIC_DDR_0_dyn_disable_rxenadfe_ddr_ch0_RMASK 0xFF
2680
2681#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_OFFS 0x1660
2682#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_DEF 0x0000000000000000
2683#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch3_LSB 0x27
2684#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch3_MSB 0x27
2685#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch3_RMASK 0x1
2686#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch2_LSB 0x26
2687#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch2_MSB 0x26
2688#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch2_RMASK 0x1
2689#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch1_LSB 0x25
2690#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch1_MSB 0x25
2691#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch1_RMASK 0x1
2692#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch0_LSB 0x24
2693#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch0_MSB 0x24
2694#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenagain_qdr_ch0_RMASK 0x1
2695#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch3_LSB 0x23
2696#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch3_MSB 0x23
2697#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch3_RMASK 0x1
2698#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch2_LSB 0x22
2699#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch2_MSB 0x22
2700#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch2_RMASK 0x1
2701#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch1_LSB 0x21
2702#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch1_MSB 0x21
2703#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch1_RMASK 0x1
2704#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch0_LSB 0x20
2705#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch0_MSB 0x20
2706#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenale_qdr_ch0_RMASK 0x1
2707#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch3_LSB 0x18
2708#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch3_MSB 0x1F
2709#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch3_RMASK 0xFF
2710#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch2_LSB 0x10
2711#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch2_MSB 0x17
2712#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch2_RMASK 0xFF
2713#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch1_LSB 0x8
2714#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch1_MSB 0xF
2715#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch1_RMASK 0xFF
2716#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch0_LSB 0x0
2717#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch0_MSB 0x7
2718#define QIB_7322_ADAPT_DISABLE_STATIC_QDR_0_static_disable_rxenadfe_qdr_ch0_RMASK 0xFF
2719
2720#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_OFFS 0x1668
2721#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_DEF 0x0000000000000000
2722#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch3_LSB 0x27
2723#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch3_MSB 0x27
2724#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch3_RMASK 0x1
2725#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch2_LSB 0x26
2726#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch2_MSB 0x26
2727#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch2_RMASK 0x1
2728#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch1_LSB 0x25
2729#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch1_MSB 0x25
2730#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch1_RMASK 0x1
2731#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch0_LSB 0x24
2732#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch0_MSB 0x24
2733#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenagain_qdr_ch0_RMASK 0x1
2734#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch3_LSB 0x23
2735#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch3_MSB 0x23
2736#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch3_RMASK 0x1
2737#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch2_LSB 0x22
2738#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch2_MSB 0x22
2739#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch2_RMASK 0x1
2740#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch1_LSB 0x21
2741#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch1_MSB 0x21
2742#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch1_RMASK 0x1
2743#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch0_LSB 0x20
2744#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch0_MSB 0x20
2745#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenale_qdr_ch0_RMASK 0x1
2746#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch3_LSB 0x18
2747#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch3_MSB 0x1F
2748#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch3_RMASK 0xFF
2749#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch2_LSB 0x10
2750#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch2_MSB 0x17
2751#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch2_RMASK 0xFF
2752#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch1_LSB 0x8
2753#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch1_MSB 0xF
2754#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch1_RMASK 0xFF
2755#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch0_LSB 0x0
2756#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch0_MSB 0x7
2757#define QIB_7322_ADAPT_DISABLE_DYNAMIC_QDR_0_dyn_disable_rxenadfe_qdr_ch0_RMASK 0xFF
2758
2759#define QIB_7322_ADAPT_DISABLE_TIMER_THRESHOLD_0_OFFS 0x1670
2760#define QIB_7322_ADAPT_DISABLE_TIMER_THRESHOLD_0_DEF 0x0000000000000000
2761
2762#define QIB_7322_HighPriorityLimit_0_OFFS 0x1BC0
2763#define QIB_7322_HighPriorityLimit_0_DEF 0x0000000000000000
2764#define QIB_7322_HighPriorityLimit_0_Limit_LSB 0x0
2765#define QIB_7322_HighPriorityLimit_0_Limit_MSB 0x7
2766#define QIB_7322_HighPriorityLimit_0_Limit_RMASK 0xFF
2767
2768#define QIB_7322_LowPriority0_0_OFFS 0x1C00
2769#define QIB_7322_LowPriority0_0_DEF 0x0000000000000000
2770#define QIB_7322_LowPriority0_0_VirtualLane_LSB 0x10
2771#define QIB_7322_LowPriority0_0_VirtualLane_MSB 0x12
2772#define QIB_7322_LowPriority0_0_VirtualLane_RMASK 0x7
2773#define QIB_7322_LowPriority0_0_Weight_LSB 0x0
2774#define QIB_7322_LowPriority0_0_Weight_MSB 0x7
2775#define QIB_7322_LowPriority0_0_Weight_RMASK 0xFF
2776
2777#define QIB_7322_HighPriority0_0_OFFS 0x1E00
2778#define QIB_7322_HighPriority0_0_DEF 0x0000000000000000
2779#define QIB_7322_HighPriority0_0_VirtualLane_LSB 0x10
2780#define QIB_7322_HighPriority0_0_VirtualLane_MSB 0x12
2781#define QIB_7322_HighPriority0_0_VirtualLane_RMASK 0x7
2782#define QIB_7322_HighPriority0_0_Weight_LSB 0x0
2783#define QIB_7322_HighPriority0_0_Weight_MSB 0x7
2784#define QIB_7322_HighPriority0_0_Weight_RMASK 0xFF
2785
2786#define QIB_7322_CntrRegBase_1_OFFS 0x2028
2787#define QIB_7322_CntrRegBase_1_DEF 0x0000000000013000
2788
2789#define QIB_7322_RcvQPMulticastContext_1_OFFS 0x2170
2790
2791#define QIB_7322_SendCtrl_1_OFFS 0x21C0
2792
2793#define QIB_7322_SendBufAvail0_OFFS 0x3000
2794#define QIB_7322_SendBufAvail0_DEF 0x0000000000000000
2795#define QIB_7322_SendBufAvail0_SendBuf_31_0_LSB 0x0
2796#define QIB_7322_SendBufAvail0_SendBuf_31_0_MSB 0x3F
2797#define QIB_7322_SendBufAvail0_SendBuf_31_0_RMASK 0x0
2798
2799#define QIB_7322_MsixTable_OFFS 0x8000
2800#define QIB_7322_MsixTable_DEF 0x0000000000000000
2801
2802#define QIB_7322_MsixPba_OFFS 0x9000
2803#define QIB_7322_MsixPba_DEF 0x0000000000000000
2804
2805#define QIB_7322_LAMemory_OFFS 0xA000
2806#define QIB_7322_LAMemory_DEF 0x0000000000000000
2807
2808#define QIB_7322_LBIntCnt_OFFS 0x11000
2809#define QIB_7322_LBIntCnt_DEF 0x0000000000000000
2810
2811#define QIB_7322_LBFlowStallCnt_OFFS 0x11008
2812#define QIB_7322_LBFlowStallCnt_DEF 0x0000000000000000
2813
2814#define QIB_7322_RxTIDFullErrCnt_OFFS 0x110D0
2815#define QIB_7322_RxTIDFullErrCnt_DEF 0x0000000000000000
2816
2817#define QIB_7322_RxTIDValidErrCnt_OFFS 0x110D8
2818#define QIB_7322_RxTIDValidErrCnt_DEF 0x0000000000000000
2819
2820#define QIB_7322_RxP0HdrEgrOvflCnt_OFFS 0x110E8
2821#define QIB_7322_RxP0HdrEgrOvflCnt_DEF 0x0000000000000000
2822
2823#define QIB_7322_PcieRetryBufDiagQwordCnt_OFFS 0x111A0
2824#define QIB_7322_PcieRetryBufDiagQwordCnt_DEF 0x0000000000000000
2825
2826#define QIB_7322_RxTidFlowDropCnt_OFFS 0x111E0
2827#define QIB_7322_RxTidFlowDropCnt_DEF 0x0000000000000000
2828
2829#define QIB_7322_LBIntCnt_0_OFFS 0x12000
2830#define QIB_7322_LBIntCnt_0_DEF 0x0000000000000000
2831
2832#define QIB_7322_TxCreditUpToDateTimeOut_0_OFFS 0x12008
2833#define QIB_7322_TxCreditUpToDateTimeOut_0_DEF 0x0000000000000000
2834
2835#define QIB_7322_TxSDmaDescCnt_0_OFFS 0x12010
2836#define QIB_7322_TxSDmaDescCnt_0_DEF 0x0000000000000000
2837
2838#define QIB_7322_TxUnsupVLErrCnt_0_OFFS 0x12018
2839#define QIB_7322_TxUnsupVLErrCnt_0_DEF 0x0000000000000000
2840
2841#define QIB_7322_TxDataPktCnt_0_OFFS 0x12020
2842#define QIB_7322_TxDataPktCnt_0_DEF 0x0000000000000000
2843
2844#define QIB_7322_TxFlowPktCnt_0_OFFS 0x12028
2845#define QIB_7322_TxFlowPktCnt_0_DEF 0x0000000000000000
2846
2847#define QIB_7322_TxDwordCnt_0_OFFS 0x12030
2848#define QIB_7322_TxDwordCnt_0_DEF 0x0000000000000000
2849
2850#define QIB_7322_TxLenErrCnt_0_OFFS 0x12038
2851#define QIB_7322_TxLenErrCnt_0_DEF 0x0000000000000000
2852
2853#define QIB_7322_TxMaxMinLenErrCnt_0_OFFS 0x12040
2854#define QIB_7322_TxMaxMinLenErrCnt_0_DEF 0x0000000000000000
2855
2856#define QIB_7322_TxUnderrunCnt_0_OFFS 0x12048
2857#define QIB_7322_TxUnderrunCnt_0_DEF 0x0000000000000000
2858
2859#define QIB_7322_TxFlowStallCnt_0_OFFS 0x12050
2860#define QIB_7322_TxFlowStallCnt_0_DEF 0x0000000000000000
2861
2862#define QIB_7322_TxDroppedPktCnt_0_OFFS 0x12058
2863#define QIB_7322_TxDroppedPktCnt_0_DEF 0x0000000000000000
2864
2865#define QIB_7322_RxDroppedPktCnt_0_OFFS 0x12060
2866#define QIB_7322_RxDroppedPktCnt_0_DEF 0x0000000000000000
2867
2868#define QIB_7322_RxDataPktCnt_0_OFFS 0x12068
2869#define QIB_7322_RxDataPktCnt_0_DEF 0x0000000000000000
2870
2871#define QIB_7322_RxFlowPktCnt_0_OFFS 0x12070
2872#define QIB_7322_RxFlowPktCnt_0_DEF 0x0000000000000000
2873
2874#define QIB_7322_RxDwordCnt_0_OFFS 0x12078
2875#define QIB_7322_RxDwordCnt_0_DEF 0x0000000000000000
2876
2877#define QIB_7322_RxLenErrCnt_0_OFFS 0x12080
2878#define QIB_7322_RxLenErrCnt_0_DEF 0x0000000000000000
2879
2880#define QIB_7322_RxMaxMinLenErrCnt_0_OFFS 0x12088
2881#define QIB_7322_RxMaxMinLenErrCnt_0_DEF 0x0000000000000000
2882
2883#define QIB_7322_RxICRCErrCnt_0_OFFS 0x12090
2884#define QIB_7322_RxICRCErrCnt_0_DEF 0x0000000000000000
2885
2886#define QIB_7322_RxVCRCErrCnt_0_OFFS 0x12098
2887#define QIB_7322_RxVCRCErrCnt_0_DEF 0x0000000000000000
2888
2889#define QIB_7322_RxFlowCtrlViolCnt_0_OFFS 0x120A0
2890#define QIB_7322_RxFlowCtrlViolCnt_0_DEF 0x0000000000000000
2891
2892#define QIB_7322_RxVersionErrCnt_0_OFFS 0x120A8
2893#define QIB_7322_RxVersionErrCnt_0_DEF 0x0000000000000000
2894
2895#define QIB_7322_RxLinkMalformCnt_0_OFFS 0x120B0
2896#define QIB_7322_RxLinkMalformCnt_0_DEF 0x0000000000000000
2897
2898#define QIB_7322_RxEBPCnt_0_OFFS 0x120B8
2899#define QIB_7322_RxEBPCnt_0_DEF 0x0000000000000000
2900
2901#define QIB_7322_RxLPCRCErrCnt_0_OFFS 0x120C0
2902#define QIB_7322_RxLPCRCErrCnt_0_DEF 0x0000000000000000
2903
2904#define QIB_7322_RxBufOvflCnt_0_OFFS 0x120C8
2905#define QIB_7322_RxBufOvflCnt_0_DEF 0x0000000000000000
2906
2907#define QIB_7322_RxLenTruncateCnt_0_OFFS 0x120D0
2908#define QIB_7322_RxLenTruncateCnt_0_DEF 0x0000000000000000
2909
2910#define QIB_7322_RxPKeyMismatchCnt_0_OFFS 0x120E0
2911#define QIB_7322_RxPKeyMismatchCnt_0_DEF 0x0000000000000000
2912
2913#define QIB_7322_IBLinkDownedCnt_0_OFFS 0x12180
2914#define QIB_7322_IBLinkDownedCnt_0_DEF 0x0000000000000000
2915
2916#define QIB_7322_IBSymbolErrCnt_0_OFFS 0x12188
2917#define QIB_7322_IBSymbolErrCnt_0_DEF 0x0000000000000000
2918
2919#define QIB_7322_IBStatusChangeCnt_0_OFFS 0x12190
2920#define QIB_7322_IBStatusChangeCnt_0_DEF 0x0000000000000000
2921
2922#define QIB_7322_IBLinkErrRecoveryCnt_0_OFFS 0x12198
2923#define QIB_7322_IBLinkErrRecoveryCnt_0_DEF 0x0000000000000000
2924
2925#define QIB_7322_ExcessBufferOvflCnt_0_OFFS 0x121A8
2926#define QIB_7322_ExcessBufferOvflCnt_0_DEF 0x0000000000000000
2927
2928#define QIB_7322_LocalLinkIntegrityErrCnt_0_OFFS 0x121B0
2929#define QIB_7322_LocalLinkIntegrityErrCnt_0_DEF 0x0000000000000000
2930
2931#define QIB_7322_RxVlErrCnt_0_OFFS 0x121B8
2932#define QIB_7322_RxVlErrCnt_0_DEF 0x0000000000000000
2933
2934#define QIB_7322_RxDlidFltrCnt_0_OFFS 0x121C0
2935#define QIB_7322_RxDlidFltrCnt_0_DEF 0x0000000000000000
2936
2937#define QIB_7322_RxVL15DroppedPktCnt_0_OFFS 0x121C8
2938#define QIB_7322_RxVL15DroppedPktCnt_0_DEF 0x0000000000000000
2939
2940#define QIB_7322_RxOtherLocalPhyErrCnt_0_OFFS 0x121D0
2941#define QIB_7322_RxOtherLocalPhyErrCnt_0_DEF 0x0000000000000000
2942
2943#define QIB_7322_RxQPInvalidContextCnt_0_OFFS 0x121D8
2944#define QIB_7322_RxQPInvalidContextCnt_0_DEF 0x0000000000000000
2945
2946#define QIB_7322_TxHeadersErrCnt_0_OFFS 0x121F8
2947#define QIB_7322_TxHeadersErrCnt_0_DEF 0x0000000000000000
2948
2949#define QIB_7322_PSRcvDataCount_0_OFFS 0x12218
2950#define QIB_7322_PSRcvDataCount_0_DEF 0x0000000000000000
2951
2952#define QIB_7322_PSRcvPktsCount_0_OFFS 0x12220
2953#define QIB_7322_PSRcvPktsCount_0_DEF 0x0000000000000000
2954
2955#define QIB_7322_PSXmitDataCount_0_OFFS 0x12228
2956#define QIB_7322_PSXmitDataCount_0_DEF 0x0000000000000000
2957
2958#define QIB_7322_PSXmitPktsCount_0_OFFS 0x12230
2959#define QIB_7322_PSXmitPktsCount_0_DEF 0x0000000000000000
2960
2961#define QIB_7322_PSXmitWaitCount_0_OFFS 0x12238
2962#define QIB_7322_PSXmitWaitCount_0_DEF 0x0000000000000000
2963
2964#define QIB_7322_LBIntCnt_1_OFFS 0x13000
2965#define QIB_7322_LBIntCnt_1_DEF 0x0000000000000000
2966
2967#define QIB_7322_TxCreditUpToDateTimeOut_1_OFFS 0x13008
2968#define QIB_7322_TxCreditUpToDateTimeOut_1_DEF 0x0000000000000000
2969
2970#define QIB_7322_TxSDmaDescCnt_1_OFFS 0x13010
2971#define QIB_7322_TxSDmaDescCnt_1_DEF 0x0000000000000000
2972
2973#define QIB_7322_TxUnsupVLErrCnt_1_OFFS 0x13018
2974#define QIB_7322_TxUnsupVLErrCnt_1_DEF 0x0000000000000000
2975
2976#define QIB_7322_TxDataPktCnt_1_OFFS 0x13020
2977#define QIB_7322_TxDataPktCnt_1_DEF 0x0000000000000000
2978
2979#define QIB_7322_TxFlowPktCnt_1_OFFS 0x13028
2980#define QIB_7322_TxFlowPktCnt_1_DEF 0x0000000000000000
2981
2982#define QIB_7322_TxDwordCnt_1_OFFS 0x13030
2983#define QIB_7322_TxDwordCnt_1_DEF 0x0000000000000000
2984
2985#define QIB_7322_TxLenErrCnt_1_OFFS 0x13038
2986#define QIB_7322_TxLenErrCnt_1_DEF 0x0000000000000000
2987
2988#define QIB_7322_TxMaxMinLenErrCnt_1_OFFS 0x13040
2989#define QIB_7322_TxMaxMinLenErrCnt_1_DEF 0x0000000000000000
2990
2991#define QIB_7322_TxUnderrunCnt_1_OFFS 0x13048
2992#define QIB_7322_TxUnderrunCnt_1_DEF 0x0000000000000000
2993
2994#define QIB_7322_TxFlowStallCnt_1_OFFS 0x13050
2995#define QIB_7322_TxFlowStallCnt_1_DEF 0x0000000000000000
2996
2997#define QIB_7322_TxDroppedPktCnt_1_OFFS 0x13058
2998#define QIB_7322_TxDroppedPktCnt_1_DEF 0x0000000000000000
2999
3000#define QIB_7322_RxDroppedPktCnt_1_OFFS 0x13060
3001#define QIB_7322_RxDroppedPktCnt_1_DEF 0x0000000000000000
3002
3003#define QIB_7322_RxDataPktCnt_1_OFFS 0x13068
3004#define QIB_7322_RxDataPktCnt_1_DEF 0x0000000000000000
3005
3006#define QIB_7322_RxFlowPktCnt_1_OFFS 0x13070
3007#define QIB_7322_RxFlowPktCnt_1_DEF 0x0000000000000000
3008
3009#define QIB_7322_RxDwordCnt_1_OFFS 0x13078
3010#define QIB_7322_RxDwordCnt_1_DEF 0x0000000000000000
3011
3012#define QIB_7322_RxLenErrCnt_1_OFFS 0x13080
3013#define QIB_7322_RxLenErrCnt_1_DEF 0x0000000000000000
3014
3015#define QIB_7322_RxMaxMinLenErrCnt_1_OFFS 0x13088
3016#define QIB_7322_RxMaxMinLenErrCnt_1_DEF 0x0000000000000000
3017
3018#define QIB_7322_RxICRCErrCnt_1_OFFS 0x13090
3019#define QIB_7322_RxICRCErrCnt_1_DEF 0x0000000000000000
3020
3021#define QIB_7322_RxVCRCErrCnt_1_OFFS 0x13098
3022#define QIB_7322_RxVCRCErrCnt_1_DEF 0x0000000000000000
3023
3024#define QIB_7322_RxFlowCtrlViolCnt_1_OFFS 0x130A0
3025#define QIB_7322_RxFlowCtrlViolCnt_1_DEF 0x0000000000000000
3026
3027#define QIB_7322_RxVersionErrCnt_1_OFFS 0x130A8
3028#define QIB_7322_RxVersionErrCnt_1_DEF 0x0000000000000000
3029
3030#define QIB_7322_RxLinkMalformCnt_1_OFFS 0x130B0
3031#define QIB_7322_RxLinkMalformCnt_1_DEF 0x0000000000000000
3032
3033#define QIB_7322_RxEBPCnt_1_OFFS 0x130B8
3034#define QIB_7322_RxEBPCnt_1_DEF 0x0000000000000000
3035
3036#define QIB_7322_RxLPCRCErrCnt_1_OFFS 0x130C0
3037#define QIB_7322_RxLPCRCErrCnt_1_DEF 0x0000000000000000
3038
3039#define QIB_7322_RxBufOvflCnt_1_OFFS 0x130C8
3040#define QIB_7322_RxBufOvflCnt_1_DEF 0x0000000000000000
3041
3042#define QIB_7322_RxLenTruncateCnt_1_OFFS 0x130D0
3043#define QIB_7322_RxLenTruncateCnt_1_DEF 0x0000000000000000
3044
3045#define QIB_7322_RxPKeyMismatchCnt_1_OFFS 0x130E0
3046#define QIB_7322_RxPKeyMismatchCnt_1_DEF 0x0000000000000000
3047
3048#define QIB_7322_IBLinkDownedCnt_1_OFFS 0x13180
3049#define QIB_7322_IBLinkDownedCnt_1_DEF 0x0000000000000000
3050
3051#define QIB_7322_IBSymbolErrCnt_1_OFFS 0x13188
3052#define QIB_7322_IBSymbolErrCnt_1_DEF 0x0000000000000000
3053
3054#define QIB_7322_IBStatusChangeCnt_1_OFFS 0x13190
3055#define QIB_7322_IBStatusChangeCnt_1_DEF 0x0000000000000000
3056
3057#define QIB_7322_IBLinkErrRecoveryCnt_1_OFFS 0x13198
3058#define QIB_7322_IBLinkErrRecoveryCnt_1_DEF 0x0000000000000000
3059
3060#define QIB_7322_ExcessBufferOvflCnt_1_OFFS 0x131A8
3061#define QIB_7322_ExcessBufferOvflCnt_1_DEF 0x0000000000000000
3062
3063#define QIB_7322_LocalLinkIntegrityErrCnt_1_OFFS 0x131B0
3064#define QIB_7322_LocalLinkIntegrityErrCnt_1_DEF 0x0000000000000000
3065
3066#define QIB_7322_RxVlErrCnt_1_OFFS 0x131B8
3067#define QIB_7322_RxVlErrCnt_1_DEF 0x0000000000000000
3068
3069#define QIB_7322_RxDlidFltrCnt_1_OFFS 0x131C0
3070#define QIB_7322_RxDlidFltrCnt_1_DEF 0x0000000000000000
3071
3072#define QIB_7322_RxVL15DroppedPktCnt_1_OFFS 0x131C8
3073#define QIB_7322_RxVL15DroppedPktCnt_1_DEF 0x0000000000000000
3074
3075#define QIB_7322_RxOtherLocalPhyErrCnt_1_OFFS 0x131D0
3076#define QIB_7322_RxOtherLocalPhyErrCnt_1_DEF 0x0000000000000000
3077
3078#define QIB_7322_RxQPInvalidContextCnt_1_OFFS 0x131D8
3079#define QIB_7322_RxQPInvalidContextCnt_1_DEF 0x0000000000000000
3080
3081#define QIB_7322_TxHeadersErrCnt_1_OFFS 0x131F8
3082#define QIB_7322_TxHeadersErrCnt_1_DEF 0x0000000000000000
3083
3084#define QIB_7322_PSRcvDataCount_1_OFFS 0x13218
3085#define QIB_7322_PSRcvDataCount_1_DEF 0x0000000000000000
3086
3087#define QIB_7322_PSRcvPktsCount_1_OFFS 0x13220
3088#define QIB_7322_PSRcvPktsCount_1_DEF 0x0000000000000000
3089
3090#define QIB_7322_PSXmitDataCount_1_OFFS 0x13228
3091#define QIB_7322_PSXmitDataCount_1_DEF 0x0000000000000000
3092
3093#define QIB_7322_PSXmitPktsCount_1_OFFS 0x13230
3094#define QIB_7322_PSXmitPktsCount_1_DEF 0x0000000000000000
3095
3096#define QIB_7322_PSXmitWaitCount_1_OFFS 0x13238
3097#define QIB_7322_PSXmitWaitCount_1_DEF 0x0000000000000000
3098
3099#define QIB_7322_RcvEgrArray_OFFS 0x14000
3100#define QIB_7322_RcvEgrArray_DEF 0x0000000000000000
3101#define QIB_7322_RcvEgrArray_RT_BufSize_LSB 0x25
3102#define QIB_7322_RcvEgrArray_RT_BufSize_MSB 0x27
3103#define QIB_7322_RcvEgrArray_RT_BufSize_RMASK 0x7
3104#define QIB_7322_RcvEgrArray_RT_Addr_LSB 0x0
3105#define QIB_7322_RcvEgrArray_RT_Addr_MSB 0x24
3106#define QIB_7322_RcvEgrArray_RT_Addr_RMASK 0x1FFFFFFFFF
3107
3108#define QIB_7322_RcvTIDArray0_OFFS 0x50000
3109#define QIB_7322_RcvTIDArray0_DEF 0x0000000000000000
3110#define QIB_7322_RcvTIDArray0_RT_BufSize_LSB 0x25
3111#define QIB_7322_RcvTIDArray0_RT_BufSize_MSB 0x27
3112#define QIB_7322_RcvTIDArray0_RT_BufSize_RMASK 0x7
3113#define QIB_7322_RcvTIDArray0_RT_Addr_LSB 0x0
3114#define QIB_7322_RcvTIDArray0_RT_Addr_MSB 0x24
3115#define QIB_7322_RcvTIDArray0_RT_Addr_RMASK 0x1FFFFFFFFF
3116
3117#define QIB_7322_IBSD_DDS_MAP_TABLE_0_OFFS 0xD0000
3118#define QIB_7322_IBSD_DDS_MAP_TABLE_0_DEF 0x0000000000000000
3119
3120#define QIB_7322_RcvHdrTail0_OFFS 0x200000
3121#define QIB_7322_RcvHdrTail0_DEF 0x0000000000000000
3122
3123#define QIB_7322_RcvHdrHead0_OFFS 0x200008
3124#define QIB_7322_RcvHdrHead0_DEF 0x0000000000000000
3125#define QIB_7322_RcvHdrHead0_counter_LSB 0x20
3126#define QIB_7322_RcvHdrHead0_counter_MSB 0x2F
3127#define QIB_7322_RcvHdrHead0_counter_RMASK 0xFFFF
3128#define QIB_7322_RcvHdrHead0_RcvHeadPointer_LSB 0x0
3129#define QIB_7322_RcvHdrHead0_RcvHeadPointer_MSB 0x1F
3130#define QIB_7322_RcvHdrHead0_RcvHeadPointer_RMASK 0xFFFFFFFF
3131
3132#define QIB_7322_RcvEgrIndexTail0_OFFS 0x200010
3133#define QIB_7322_RcvEgrIndexTail0_DEF 0x0000000000000000
3134
3135#define QIB_7322_RcvEgrIndexHead0_OFFS 0x200018
3136#define QIB_7322_RcvEgrIndexHead0_DEF 0x0000000000000000
3137
3138#define QIB_7322_RcvTIDFlowTable0_OFFS 0x201000
3139#define QIB_7322_RcvTIDFlowTable0_DEF 0x0000000000000000
3140#define QIB_7322_RcvTIDFlowTable0_GenMismatch_LSB 0x1C
3141#define QIB_7322_RcvTIDFlowTable0_GenMismatch_MSB 0x1C
3142#define QIB_7322_RcvTIDFlowTable0_GenMismatch_RMASK 0x1
3143#define QIB_7322_RcvTIDFlowTable0_SeqMismatch_LSB 0x1B
3144#define QIB_7322_RcvTIDFlowTable0_SeqMismatch_MSB 0x1B
3145#define QIB_7322_RcvTIDFlowTable0_SeqMismatch_RMASK 0x1
3146#define QIB_7322_RcvTIDFlowTable0_KeepOnGenErr_LSB 0x16
3147#define QIB_7322_RcvTIDFlowTable0_KeepOnGenErr_MSB 0x16
3148#define QIB_7322_RcvTIDFlowTable0_KeepOnGenErr_RMASK 0x1
3149#define QIB_7322_RcvTIDFlowTable0_KeepAfterSeqErr_LSB 0x15
3150#define QIB_7322_RcvTIDFlowTable0_KeepAfterSeqErr_MSB 0x15
3151#define QIB_7322_RcvTIDFlowTable0_KeepAfterSeqErr_RMASK 0x1
3152#define QIB_7322_RcvTIDFlowTable0_HdrSuppEnabled_LSB 0x14
3153#define QIB_7322_RcvTIDFlowTable0_HdrSuppEnabled_MSB 0x14
3154#define QIB_7322_RcvTIDFlowTable0_HdrSuppEnabled_RMASK 0x1
3155#define QIB_7322_RcvTIDFlowTable0_FlowValid_LSB 0x13
3156#define QIB_7322_RcvTIDFlowTable0_FlowValid_MSB 0x13
3157#define QIB_7322_RcvTIDFlowTable0_FlowValid_RMASK 0x1
3158#define QIB_7322_RcvTIDFlowTable0_GenVal_LSB 0xB
3159#define QIB_7322_RcvTIDFlowTable0_GenVal_MSB 0x12
3160#define QIB_7322_RcvTIDFlowTable0_GenVal_RMASK 0xFF
3161#define QIB_7322_RcvTIDFlowTable0_SeqNum_LSB 0x0
3162#define QIB_7322_RcvTIDFlowTable0_SeqNum_MSB 0xA
3163#define QIB_7322_RcvTIDFlowTable0_SeqNum_RMASK 0x7FF
drivers/infiniband/hw/qib/qib_common.h+0−798
@@ -1,798 +0,0 @@
1/*
2 * Copyright (c) 2006, 2007, 2008, 2009, 2010 QLogic Corporation.
3 * All rights reserved.
4 * Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved.
5 *
6 * This software is available to you under a choice of one of two
7 * licenses. You may choose to be licensed under the terms of the GNU
8 * General Public License (GPL) Version 2, available from the file
9 * COPYING in the main directory of this source tree, or the
10 * OpenIB.org BSD license below:
11 *
12 * Redistribution and use in source and binary forms, with or
13 * without modification, are permitted provided that the following
14 * conditions are met:
15 *
16 * - Redistributions of source code must retain the above
17 * copyright notice, this list of conditions and the following
18 * disclaimer.
19 *
20 * - Redistributions in binary form must reproduce the above
21 * copyright notice, this list of conditions and the following
22 * disclaimer in the documentation and/or other materials
23 * provided with the distribution.
24 *
25 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
26 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
27 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
28 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
29 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
30 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
31 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
32 * SOFTWARE.
33 */
34
35#ifndef _QIB_COMMON_H
36#define _QIB_COMMON_H
37
38/*
39 * This file contains defines, structures, etc. that are used
40 * to communicate between kernel and user code.
41 */
42
43/* This is the IEEE-assigned OUI for QLogic Inc. QLogic_IB */
44#define QIB_SRC_OUI_1 0x00
45#define QIB_SRC_OUI_2 0x11
46#define QIB_SRC_OUI_3 0x75
47
48/* version of protocol header (known to chip also). In the long run,
49 * we should be able to generate and accept a range of version numbers;
50 * for now we only accept one, and it's compiled in.
51 */
52#define IPS_PROTO_VERSION 2
53
54/*
55 * These are compile time constants that you may want to enable or disable
56 * if you are trying to debug problems with code or performance.
57 * QIB_VERBOSE_TRACING define as 1 if you want additional tracing in
58 * fastpath code
59 * QIB_TRACE_REGWRITES define as 1 if you want register writes to be
60 * traced in fastpath code
61 * _QIB_TRACING define as 0 if you want to remove all tracing in a
62 * compilation unit
63 */
64
65/*
66 * The value in the BTH QP field that QLogic_IB uses to differentiate
67 * an qlogic_ib protocol IB packet vs standard IB transport
68 * This it needs to be even (0x656b78), because the LSB is sometimes
69 * used for the MSB of context. The change may cause a problem
70 * interoperating with older software.
71 */
72#define QIB_KD_QP 0x656b78
73
74/*
75 * These are the status bits readable (in ascii form, 64bit value)
76 * from the "status" sysfs file. For binary compatibility, values
77 * must remain as is; removed states can be reused for different
78 * purposes.
79 */
80#define QIB_STATUS_INITTED 0x1 /* basic initialization done */
81/* Chip has been found and initted */
82#define QIB_STATUS_CHIP_PRESENT 0x20
83/* IB link is at ACTIVE, usable for data traffic */
84#define QIB_STATUS_IB_READY 0x40
85/* link is configured, LID, MTU, etc. have been set */
86#define QIB_STATUS_IB_CONF 0x80
87/* A Fatal hardware error has occurred. */
88#define QIB_STATUS_HWERROR 0x200
89
90/*
91 * The list of usermode accessible registers. Also see Reg_* later in file.
92 */
93enum qib_ureg {
94 /* (RO) DMA RcvHdr to be used next. */
95 ur_rcvhdrtail = 0,
96 /* (RW) RcvHdr entry to be processed next by host. */
97 ur_rcvhdrhead = 1,
98 /* (RO) Index of next Eager index to use. */
99 ur_rcvegrindextail = 2,
100 /* (RW) Eager TID to be processed next */
101 ur_rcvegrindexhead = 3,
102 /* For internal use only; max register number. */
103 _QIB_UregMax
104};
105
106/* bit values for spi_runtime_flags */
107#define QIB_RUNTIME_PCIE 0x0002
108#define QIB_RUNTIME_FORCE_WC_ORDER 0x0004
109#define QIB_RUNTIME_RCVHDR_COPY 0x0008
110#define QIB_RUNTIME_MASTER 0x0010
111#define QIB_RUNTIME_RCHK 0x0020
112#define QIB_RUNTIME_NODMA_RTAIL 0x0080
113#define QIB_RUNTIME_SPECIAL_TRIGGER 0x0100
114#define QIB_RUNTIME_SDMA 0x0200
115#define QIB_RUNTIME_FORCE_PIOAVAIL 0x0400
116#define QIB_RUNTIME_PIO_REGSWAPPED 0x0800
117#define QIB_RUNTIME_CTXT_MSB_IN_QP 0x1000
118#define QIB_RUNTIME_CTXT_REDIRECT 0x2000
119#define QIB_RUNTIME_HDRSUPP 0x4000
120
121/*
122 * This structure is returned by qib_userinit() immediately after
123 * open to get implementation-specific info, and info specific to this
124 * instance.
125 *
126 * This struct must have explict pad fields where type sizes
127 * may result in different alignments between 32 and 64 bit
128 * programs, since the 64 bit * bit kernel requires the user code
129 * to have matching offsets
130 */
131struct qib_base_info {
132 /* version of hardware, for feature checking. */
133 __u32 spi_hw_version;
134 /* version of software, for feature checking. */
135 __u32 spi_sw_version;
136 /* QLogic_IB context assigned, goes into sent packets */
137 __u16 spi_ctxt;
138 __u16 spi_subctxt;
139 /*
140 * IB MTU, packets IB data must be less than this.
141 * The MTU is in bytes, and will be a multiple of 4 bytes.
142 */
143 __u32 spi_mtu;
144 /*
145 * Size of a PIO buffer. Any given packet's total size must be less
146 * than this (in words). Included is the starting control word, so
147 * if 513 is returned, then total pkt size is 512 words or less.
148 */
149 __u32 spi_piosize;
150 /* size of the TID cache in qlogic_ib, in entries */
151 __u32 spi_tidcnt;
152 /* size of the TID Eager list in qlogic_ib, in entries */
153 __u32 spi_tidegrcnt;
154 /* size of a single receive header queue entry in words. */
155 __u32 spi_rcvhdrent_size;
156 /*
157 * Count of receive header queue entries allocated.
158 * This may be less than the spu_rcvhdrcnt passed in!.
159 */
160 __u32 spi_rcvhdr_cnt;
161
162 /* per-chip and other runtime features bitmap (QIB_RUNTIME_*) */
163 __u32 spi_runtime_flags;
164
165 /* address where hardware receive header queue is mapped */
166 __u64 spi_rcvhdr_base;
167
168 /* user program. */
169
170 /* base address of eager TID receive buffers used by hardware. */
171 __u64 spi_rcv_egrbufs;
172
173 /* Allocated by initialization code, not by protocol. */
174
175 /*
176 * Size of each TID buffer in host memory, starting at
177 * spi_rcv_egrbufs. The buffers are virtually contiguous.
178 */
179 __u32 spi_rcv_egrbufsize;
180 /*
181 * The special QP (queue pair) value that identifies an qlogic_ib
182 * protocol packet from standard IB packets. More, probably much
183 * more, to be added.
184 */
185 __u32 spi_qpair;
186
187 /*
188 * User register base for init code, not to be used directly by
189 * protocol or applications. Always points to chip registers,
190 * for normal or shared context.
191 */
192 __u64 spi_uregbase;
193 /*
194 * Maximum buffer size in bytes that can be used in a single TID
195 * entry (assuming the buffer is aligned to this boundary). This is
196 * the minimum of what the hardware and software support Guaranteed
197 * to be a power of 2.
198 */
199 __u32 spi_tid_maxsize;
200 /*
201 * alignment of each pio send buffer (byte count
202 * to add to spi_piobufbase to get to second buffer)
203 */
204 __u32 spi_pioalign;
205 /*
206 * The index of the first pio buffer available to this process;
207 * needed to do lookup in spi_pioavailaddr; not added to
208 * spi_piobufbase.
209 */
210 __u32 spi_pioindex;
211 /* number of buffers mapped for this process */
212 __u32 spi_piocnt;
213
214 /*
215 * Base address of writeonly pio buffers for this process.
216 * Each buffer has spi_piosize words, and is aligned on spi_pioalign
217 * boundaries. spi_piocnt buffers are mapped from this address
218 */
219 __u64 spi_piobufbase;
220
221 /*
222 * Base address of readonly memory copy of the pioavail registers.
223 * There are 2 bits for each buffer.
224 */
225 __u64 spi_pioavailaddr;
226
227 /*
228 * Address where driver updates a copy of the interface and driver
229 * status (QIB_STATUS_*) as a 64 bit value. It's followed by a
230 * link status qword (formerly combined with driver status), then a
231 * string indicating hardware error, if there was one.
232 */
233 __u64 spi_status;
234
235 /* number of chip ctxts available to user processes */
236 __u32 spi_nctxts;
237 __u16 spi_unit; /* unit number of chip we are using */
238 __u16 spi_port; /* IB port number we are using */
239 /* num bufs in each contiguous set */
240 __u32 spi_rcv_egrperchunk;
241 /* size in bytes of each contiguous set */
242 __u32 spi_rcv_egrchunksize;
243 /* total size of mmap to cover full rcvegrbuffers */
244 __u32 spi_rcv_egrbuftotlen;
245 __u32 spi_rhf_offset; /* dword offset in hdrqent for rcvhdr flags */
246 /* address of readonly memory copy of the rcvhdrq tail register. */
247 __u64 spi_rcvhdr_tailaddr;
248
249 /*
250 * shared memory pages for subctxts if ctxt is shared; these cover
251 * all the processes in the group sharing a single context.
252 * all have enough space for the num_subcontexts value on this job.
253 */
254 __u64 spi_subctxt_uregbase;
255 __u64 spi_subctxt_rcvegrbuf;
256 __u64 spi_subctxt_rcvhdr_base;
257
258 /* shared memory page for send buffer disarm status */
259 __u64 spi_sendbuf_status;
260} __aligned(8);
261
262/*
263 * This version number is given to the driver by the user code during
264 * initialization in the spu_userversion field of qib_user_info, so
265 * the driver can check for compatibility with user code.
266 *
267 * The major version changes when data structures
268 * change in an incompatible way. The driver must be the same or higher
269 * for initialization to succeed. In some cases, a higher version
270 * driver will not interoperate with older software, and initialization
271 * will return an error.
272 */
273#define QIB_USER_SWMAJOR 1
274
275/*
276 * Minor version differences are always compatible
277 * a within a major version, however if user software is larger
278 * than driver software, some new features and/or structure fields
279 * may not be implemented; the user code must deal with this if it
280 * cares, or it must abort after initialization reports the difference.
281 */
282#define QIB_USER_SWMINOR 13
283
284#define QIB_USER_SWVERSION ((QIB_USER_SWMAJOR << 16) | QIB_USER_SWMINOR)
285
286#ifndef QIB_KERN_TYPE
287#define QIB_KERN_TYPE 0
288#endif
289
290/*
291 * Similarly, this is the kernel version going back to the user. It's
292 * slightly different, in that we want to tell if the driver was built as
293 * part of a QLogic release, or from the driver from openfabrics.org,
294 * kernel.org, or a standard distribution, for support reasons.
295 * The high bit is 0 for non-QLogic and 1 for QLogic-built/supplied.
296 *
297 * It's returned by the driver to the user code during initialization in the
298 * spi_sw_version field of qib_base_info, so the user code can in turn
299 * check for compatibility with the kernel.
300*/
301#define QIB_KERN_SWVERSION ((QIB_KERN_TYPE << 31) | QIB_USER_SWVERSION)
302
303/*
304 * Define the driver version number. This is something that refers only
305 * to the driver itself, not the software interfaces it supports.
306 */
307#define QIB_DRIVER_VERSION_BASE "1.11"
308
309/* create the final driver version string */
310#ifdef QIB_IDSTR
311#define QIB_DRIVER_VERSION QIB_DRIVER_VERSION_BASE " " QIB_IDSTR
312#else
313#define QIB_DRIVER_VERSION QIB_DRIVER_VERSION_BASE
314#endif
315
316/*
317 * If the unit is specified via open, HCA choice is fixed. If port is
318 * specified, it's also fixed. Otherwise we try to spread contexts
319 * across ports and HCAs, using different algorithims. WITHIN is
320 * the old default, prior to this mechanism.
321 */
322#define QIB_PORT_ALG_ACROSS 0 /* round robin contexts across HCAs, then
323 * ports; this is the default */
324#define QIB_PORT_ALG_WITHIN 1 /* use all contexts on an HCA (round robin
325 * active ports within), then next HCA */
326#define QIB_PORT_ALG_COUNT 2 /* number of algorithm choices */
327
328/*
329 * This structure is passed to qib_userinit() to tell the driver where
330 * user code buffers are, sizes, etc. The offsets and sizes of the
331 * fields must remain unchanged, for binary compatibility. It can
332 * be extended, if userversion is changed so user code can tell, if needed
333 */
334struct qib_user_info {
335 /*
336 * version of user software, to detect compatibility issues.
337 * Should be set to QIB_USER_SWVERSION.
338 */
339 __u32 spu_userversion;
340
341 __u32 _spu_unused2;
342
343 /* size of struct base_info to write to */
344 __u32 spu_base_info_size;
345
346 __u32 spu_port_alg; /* which QIB_PORT_ALG_*; unused user minor < 11 */
347
348 /*
349 * If two or more processes wish to share a context, each process
350 * must set the spu_subctxt_cnt and spu_subctxt_id to the same
351 * values. The only restriction on the spu_subctxt_id is that
352 * it be unique for a given node.
353 */
354 __u16 spu_subctxt_cnt;
355 __u16 spu_subctxt_id;
356
357 __u32 spu_port; /* IB port requested by user if > 0 */
358
359 /*
360 * address of struct base_info to write to
361 */
362 __u64 spu_base_info;
363
364} __aligned(8);
365
366/* User commands. */
367
368/* 16 available, was: old set up userspace (for old user code) */
369#define QIB_CMD_CTXT_INFO 17 /* find out what resources we got */
370#define QIB_CMD_RECV_CTRL 18 /* control receipt of packets */
371#define QIB_CMD_TID_UPDATE 19 /* update expected TID entries */
372#define QIB_CMD_TID_FREE 20 /* free expected TID entries */
373#define QIB_CMD_SET_PART_KEY 21 /* add partition key */
374/* 22 available, was: return info on slave processes (for old user code) */
375#define QIB_CMD_ASSIGN_CTXT 23 /* allocate HCA and ctxt */
376#define QIB_CMD_USER_INIT 24 /* set up userspace */
377#define QIB_CMD_UNUSED_1 25
378#define QIB_CMD_UNUSED_2 26
379#define QIB_CMD_PIOAVAILUPD 27 /* force an update of PIOAvail reg */
380#define QIB_CMD_POLL_TYPE 28 /* set the kind of polling we want */
381#define QIB_CMD_ARMLAUNCH_CTRL 29 /* armlaunch detection control */
382/* 30 is unused */
383#define QIB_CMD_SDMA_INFLIGHT 31 /* sdma inflight counter request */
384#define QIB_CMD_SDMA_COMPLETE 32 /* sdma completion counter request */
385/* 33 available, was a testing feature */
386#define QIB_CMD_DISARM_BUFS 34 /* disarm send buffers w/ errors */
387#define QIB_CMD_ACK_EVENT 35 /* ack & clear bits */
388#define QIB_CMD_CPUS_LIST 36 /* list of cpus allocated, for pinned
389 * processes: qib_cpus_list */
390
391/*
392 * QIB_CMD_ACK_EVENT obsoletes QIB_CMD_DISARM_BUFS, but we keep it for
393 * compatibility with libraries from previous release. The ACK_EVENT
394 * will take appropriate driver action (if any, just DISARM for now),
395 * then clear the bits passed in as part of the mask. These bits are
396 * in the first 64bit word at spi_sendbuf_status, and are passed to
397 * the driver in the event_mask union as well.
398 */
399#define _QIB_EVENT_DISARM_BUFS_BIT 0
400#define _QIB_EVENT_LINKDOWN_BIT 1
401#define _QIB_EVENT_LID_CHANGE_BIT 2
402#define _QIB_EVENT_LMC_CHANGE_BIT 3
403#define _QIB_EVENT_SL2VL_CHANGE_BIT 4
404#define _QIB_MAX_EVENT_BIT _QIB_EVENT_SL2VL_CHANGE_BIT
405
406#define QIB_EVENT_DISARM_BUFS_BIT (1UL << _QIB_EVENT_DISARM_BUFS_BIT)
407#define QIB_EVENT_LINKDOWN_BIT (1UL << _QIB_EVENT_LINKDOWN_BIT)
408#define QIB_EVENT_LID_CHANGE_BIT (1UL << _QIB_EVENT_LID_CHANGE_BIT)
409#define QIB_EVENT_LMC_CHANGE_BIT (1UL << _QIB_EVENT_LMC_CHANGE_BIT)
410#define QIB_EVENT_SL2VL_CHANGE_BIT (1UL << _QIB_EVENT_SL2VL_CHANGE_BIT)
411
412
413/*
414 * Poll types
415 */
416#define QIB_POLL_TYPE_ANYRCV 0x0
417#define QIB_POLL_TYPE_URGENT 0x1
418
419struct qib_ctxt_info {
420 __u16 num_active; /* number of active units */
421 __u16 unit; /* unit (chip) assigned to caller */
422 __u16 port; /* IB port assigned to caller (1-based) */
423 __u16 ctxt; /* ctxt on unit assigned to caller */
424 __u16 subctxt; /* subctxt on unit assigned to caller */
425 __u16 num_ctxts; /* number of ctxts available on unit */
426 __u16 num_subctxts; /* number of subctxts opened on ctxt */
427 __u16 rec_cpu; /* cpu # for affinity (ffff if none) */
428};
429
430struct qib_tid_info {
431 __u32 tidcnt;
432 /* make structure same size in 32 and 64 bit */
433 __u32 tid__unused;
434 /* virtual address of first page in transfer */
435 __u64 tidvaddr;
436 /* pointer (same size 32/64 bit) to __u16 tid array */
437 __u64 tidlist;
438
439 /*
440 * pointer (same size 32/64 bit) to bitmap of TIDs used
441 * for this call; checked for being large enough at open
442 */
443 __u64 tidmap;
444};
445
446struct qib_cmd {
447 __u32 type; /* command type */
448 union {
449 struct qib_tid_info tid_info;
450 struct qib_user_info user_info;
451
452 /*
453 * address in userspace where we should put the sdma
454 * inflight counter
455 */
456 __u64 sdma_inflight;
457 /*
458 * address in userspace where we should put the sdma
459 * completion counter
460 */
461 __u64 sdma_complete;
462 /* address in userspace of struct qib_ctxt_info to
463 write result to */
464 __u64 ctxt_info;
465 /* enable/disable receipt of packets */
466 __u32 recv_ctrl;
467 /* enable/disable armlaunch errors (non-zero to enable) */
468 __u32 armlaunch_ctrl;
469 /* partition key to set */
470 __u16 part_key;
471 /* user address of __u32 bitmask of active slaves */
472 __u64 slave_mask_addr;
473 /* type of polling we want */
474 __u16 poll_type;
475 /* back pressure enable bit for one particular context */
476 __u8 ctxt_bp;
477 /* qib_user_event_ack(), IPATH_EVENT_* bits */
478 __u64 event_mask;
479 } cmd;
480};
481
482struct qib_iovec {
483 /* Pointer to data, but same size 32 and 64 bit */
484 __u64 iov_base;
485
486 /*
487 * Length of data; don't need 64 bits, but want
488 * qib_sendpkt to remain same size as before 32 bit changes, so...
489 */
490 __u64 iov_len;
491};
492
493/*
494 * Describes a single packet for send. Each packet can have one or more
495 * buffers, but the total length (exclusive of IB headers) must be less
496 * than the MTU, and if using the PIO method, entire packet length,
497 * including IB headers, must be less than the qib_piosize value (words).
498 * Use of this necessitates including sys/uio.h
499 */
500struct __qib_sendpkt {
501 __u32 sps_flags; /* flags for packet (TBD) */
502 __u32 sps_cnt; /* number of entries to use in sps_iov */
503 /* array of iov's describing packet. TEMPORARY */
504 struct qib_iovec sps_iov[4];
505};
506
507/*
508 * Diagnostics can send a packet by "writing" the following
509 * structs to the diag data special file.
510 * This allows a custom
511 * pbc (+ static rate) qword, so that special modes and deliberate
512 * changes to CRCs can be used. The elements were also re-ordered
513 * for better alignment and to avoid padding issues.
514 */
515#define _DIAG_XPKT_VERS 3
516struct qib_diag_xpkt {
517 __u16 version;
518 __u16 unit;
519 __u16 port;
520 __u16 len;
521 __u64 data;
522 __u64 pbc_wd;
523};
524
525/*
526 * Data layout in I2C flash (for GUID, etc.)
527 * All fields are little-endian binary unless otherwise stated
528 */
529#define QIB_FLASH_VERSION 2
530struct qib_flash {
531 /* flash layout version (QIB_FLASH_VERSION) */
532 __u8 if_fversion;
533 /* checksum protecting if_length bytes */
534 __u8 if_csum;
535 /*
536 * valid length (in use, protected by if_csum), including
537 * if_fversion and if_csum themselves)
538 */
539 __u8 if_length;
540 /* the GUID, in network order */
541 __u8 if_guid[8];
542 /* number of GUIDs to use, starting from if_guid */
543 __u8 if_numguid;
544 /* the (last 10 characters of) board serial number, in ASCII */
545 char if_serial[12];
546 /* board mfg date (YYYYMMDD ASCII) */
547 char if_mfgdate[8];
548 /* last board rework/test date (YYYYMMDD ASCII) */
549 char if_testdate[8];
550 /* logging of error counts, TBD */
551 __u8 if_errcntp[4];
552 /* powered on hours, updated at driver unload */
553 __u8 if_powerhour[2];
554 /* ASCII free-form comment field */
555 char if_comment[32];
556 /* Backwards compatible prefix for longer QLogic Serial Numbers */
557 char if_sprefix[4];
558 /* 82 bytes used, min flash size is 128 bytes */
559 __u8 if_future[46];
560};
561
562/*
563 * These are the counters implemented in the chip, and are listed in order.
564 * The InterCaps naming is taken straight from the chip spec.
565 */
566struct qlogic_ib_counters {
567 __u64 LBIntCnt;
568 __u64 LBFlowStallCnt;
569 __u64 TxSDmaDescCnt; /* was Reserved1 */
570 __u64 TxUnsupVLErrCnt;
571 __u64 TxDataPktCnt;
572 __u64 TxFlowPktCnt;
573 __u64 TxDwordCnt;
574 __u64 TxLenErrCnt;
575 __u64 TxMaxMinLenErrCnt;
576 __u64 TxUnderrunCnt;
577 __u64 TxFlowStallCnt;
578 __u64 TxDroppedPktCnt;
579 __u64 RxDroppedPktCnt;
580 __u64 RxDataPktCnt;
581 __u64 RxFlowPktCnt;
582 __u64 RxDwordCnt;
583 __u64 RxLenErrCnt;
584 __u64 RxMaxMinLenErrCnt;
585 __u64 RxICRCErrCnt;
586 __u64 RxVCRCErrCnt;
587 __u64 RxFlowCtrlErrCnt;
588 __u64 RxBadFormatCnt;
589 __u64 RxLinkProblemCnt;
590 __u64 RxEBPCnt;
591 __u64 RxLPCRCErrCnt;
592 __u64 RxBufOvflCnt;
593 __u64 RxTIDFullErrCnt;
594 __u64 RxTIDValidErrCnt;
595 __u64 RxPKeyMismatchCnt;
596 __u64 RxP0HdrEgrOvflCnt;
597 __u64 RxP1HdrEgrOvflCnt;
598 __u64 RxP2HdrEgrOvflCnt;
599 __u64 RxP3HdrEgrOvflCnt;
600 __u64 RxP4HdrEgrOvflCnt;
601 __u64 RxP5HdrEgrOvflCnt;
602 __u64 RxP6HdrEgrOvflCnt;
603 __u64 RxP7HdrEgrOvflCnt;
604 __u64 RxP8HdrEgrOvflCnt;
605 __u64 RxP9HdrEgrOvflCnt;
606 __u64 RxP10HdrEgrOvflCnt;
607 __u64 RxP11HdrEgrOvflCnt;
608 __u64 RxP12HdrEgrOvflCnt;
609 __u64 RxP13HdrEgrOvflCnt;
610 __u64 RxP14HdrEgrOvflCnt;
611 __u64 RxP15HdrEgrOvflCnt;
612 __u64 RxP16HdrEgrOvflCnt;
613 __u64 IBStatusChangeCnt;
614 __u64 IBLinkErrRecoveryCnt;
615 __u64 IBLinkDownedCnt;
616 __u64 IBSymbolErrCnt;
617 __u64 RxVL15DroppedPktCnt;
618 __u64 RxOtherLocalPhyErrCnt;
619 __u64 PcieRetryBufDiagQwordCnt;
620 __u64 ExcessBufferOvflCnt;
621 __u64 LocalLinkIntegrityErrCnt;
622 __u64 RxVlErrCnt;
623 __u64 RxDlidFltrCnt;
624};
625
626/*
627 * The next set of defines are for packet headers, and chip register
628 * and memory bits that are visible to and/or used by user-mode software.
629 */
630
631/* RcvHdrFlags bits */
632#define QLOGIC_IB_RHF_LENGTH_MASK 0x7FF
633#define QLOGIC_IB_RHF_LENGTH_SHIFT 0
634#define QLOGIC_IB_RHF_RCVTYPE_MASK 0x7
635#define QLOGIC_IB_RHF_RCVTYPE_SHIFT 11
636#define QLOGIC_IB_RHF_EGRINDEX_MASK 0xFFF
637#define QLOGIC_IB_RHF_EGRINDEX_SHIFT 16
638#define QLOGIC_IB_RHF_SEQ_MASK 0xF
639#define QLOGIC_IB_RHF_SEQ_SHIFT 0
640#define QLOGIC_IB_RHF_HDRQ_OFFSET_MASK 0x7FF
641#define QLOGIC_IB_RHF_HDRQ_OFFSET_SHIFT 4
642#define QLOGIC_IB_RHF_H_ICRCERR 0x80000000
643#define QLOGIC_IB_RHF_H_VCRCERR 0x40000000
644#define QLOGIC_IB_RHF_H_PARITYERR 0x20000000
645#define QLOGIC_IB_RHF_H_LENERR 0x10000000
646#define QLOGIC_IB_RHF_H_MTUERR 0x08000000
647#define QLOGIC_IB_RHF_H_IHDRERR 0x04000000
648#define QLOGIC_IB_RHF_H_TIDERR 0x02000000
649#define QLOGIC_IB_RHF_H_MKERR 0x01000000
650#define QLOGIC_IB_RHF_H_IBERR 0x00800000
651#define QLOGIC_IB_RHF_H_ERR_MASK 0xFF800000
652#define QLOGIC_IB_RHF_L_USE_EGR 0x80000000
653#define QLOGIC_IB_RHF_L_SWA 0x00008000
654#define QLOGIC_IB_RHF_L_SWB 0x00004000
655
656/* qlogic_ib header fields */
657#define QLOGIC_IB_I_VERS_MASK 0xF
658#define QLOGIC_IB_I_VERS_SHIFT 28
659#define QLOGIC_IB_I_CTXT_MASK 0xF
660#define QLOGIC_IB_I_CTXT_SHIFT 24
661#define QLOGIC_IB_I_TID_MASK 0x7FF
662#define QLOGIC_IB_I_TID_SHIFT 13
663#define QLOGIC_IB_I_OFFSET_MASK 0x1FFF
664#define QLOGIC_IB_I_OFFSET_SHIFT 0
665
666/* K_PktFlags bits */
667#define QLOGIC_IB_KPF_INTR 0x1
668#define QLOGIC_IB_KPF_SUBCTXT_MASK 0x3
669#define QLOGIC_IB_KPF_SUBCTXT_SHIFT 1
670
671#define QLOGIC_IB_MAX_SUBCTXT 4
672
673/* SendPIO per-buffer control */
674#define QLOGIC_IB_SP_TEST 0x40
675#define QLOGIC_IB_SP_TESTEBP 0x20
676#define QLOGIC_IB_SP_TRIGGER_SHIFT 15
677
678/* SendPIOAvail bits */
679#define QLOGIC_IB_SENDPIOAVAIL_BUSY_SHIFT 1
680#define QLOGIC_IB_SENDPIOAVAIL_CHECK_SHIFT 0
681
682/* qlogic_ib header format */
683struct qib_header {
684 /*
685 * Version - 4 bits, Context - 4 bits, TID - 10 bits and Offset -
686 * 14 bits before ECO change ~28 Dec 03. After that, Vers 4,
687 * Context 4, TID 11, offset 13.
688 */
689 __le32 ver_ctxt_tid_offset;
690 __le16 chksum;
691 __le16 pkt_flags;
692};
693
694/*
695 * qlogic_ib user message header format.
696 * This structure contains the first 4 fields common to all protocols
697 * that employ qlogic_ib.
698 */
699struct qib_message_header {
700 __be16 lrh[4];
701 __be32 bth[3];
702 /* fields below this point are in host byte order */
703 struct qib_header iph;
704 /* fields below are simplified, but should match PSM */
705 /* some are accessed by driver when packet spliting is needed */
706 __u8 sub_opcode;
707 __u8 flags;
708 __u16 commidx;
709 __u32 ack_seq_num;
710 __u8 flowid;
711 __u8 hdr_dlen;
712 __u16 mqhdr;
713 __u32 uwords[4];
714};
715
716/* sequence number bits for message */
717union qib_seqnum {
718 struct {
719 __u32 seq:11;
720 __u32 gen:8;
721 __u32 flow:5;
722 };
723 struct {
724 __u32 pkt:16;
725 __u32 msg:8;
726 };
727 __u32 val;
728};
729
730/* qib receiving-dma tid-session-member */
731struct qib_tid_session_member {
732 __u16 tid;
733 __u16 offset;
734 __u16 length;
735};
736
737/* IB - LRH header consts */
738#define QIB_LRH_GRH 0x0003 /* 1. word of IB LRH - next header: GRH */
739#define QIB_LRH_BTH 0x0002 /* 1. word of IB LRH - next header: BTH */
740
741/* misc. */
742#define SIZE_OF_CRC 1
743
744#define QIB_DEFAULT_P_KEY 0xFFFF
745#define QIB_PSN_MASK 0xFFFFFF
746#define QIB_EAGER_TID_ID QLOGIC_IB_I_TID_MASK
747#define QIB_MULTICAST_QPN 0xFFFFFF
748
749/* Receive Header Queue: receive type (from qlogic_ib) */
750#define RCVHQ_RCV_TYPE_EXPECTED 0
751#define RCVHQ_RCV_TYPE_EAGER 1
752#define RCVHQ_RCV_TYPE_NON_KD 2
753#define RCVHQ_RCV_TYPE_ERROR 3
754
755#define QIB_HEADER_QUEUE_WORDS 9
756
757/* functions for extracting fields from rcvhdrq entries for the driver.
758 */
759static inline __u32 qib_hdrget_err_flags(const __le32 *rbuf)
760{
761 return __le32_to_cpu(rbuf[1]) & QLOGIC_IB_RHF_H_ERR_MASK;
762}
763
764static inline __u32 qib_hdrget_rcv_type(const __le32 *rbuf)
765{
766 return (__le32_to_cpu(rbuf[0]) >> QLOGIC_IB_RHF_RCVTYPE_SHIFT) &
767 QLOGIC_IB_RHF_RCVTYPE_MASK;
768}
769
770static inline __u32 qib_hdrget_length_in_bytes(const __le32 *rbuf)
771{
772 return ((__le32_to_cpu(rbuf[0]) >> QLOGIC_IB_RHF_LENGTH_SHIFT) &
773 QLOGIC_IB_RHF_LENGTH_MASK) << 2;
774}
775
776static inline __u32 qib_hdrget_index(const __le32 *rbuf)
777{
778 return (__le32_to_cpu(rbuf[0]) >> QLOGIC_IB_RHF_EGRINDEX_SHIFT) &
779 QLOGIC_IB_RHF_EGRINDEX_MASK;
780}
781
782static inline __u32 qib_hdrget_seq(const __le32 *rbuf)
783{
784 return (__le32_to_cpu(rbuf[1]) >> QLOGIC_IB_RHF_SEQ_SHIFT) &
785 QLOGIC_IB_RHF_SEQ_MASK;
786}
787
788static inline __u32 qib_hdrget_offset(const __le32 *rbuf)
789{
790 return (__le32_to_cpu(rbuf[1]) >> QLOGIC_IB_RHF_HDRQ_OFFSET_SHIFT) &
791 QLOGIC_IB_RHF_HDRQ_OFFSET_MASK;
792}
793
794static inline __u32 qib_hdrget_use_egr_buf(const __le32 *rbuf)
795{
796 return __le32_to_cpu(rbuf[0]) & QLOGIC_IB_RHF_L_USE_EGR;
797}
798#endif /* _QIB_COMMON_H */
drivers/infiniband/hw/qib/qib_debugfs.c+0−274
@@ -1,274 +0,0 @@
1/*
2 * Copyright (c) 2013 - 2017 Intel Corporation. All rights reserved.
3 *
4 * This software is available to you under a choice of one of two
5 * licenses. You may choose to be licensed under the terms of the GNU
6 * General Public License (GPL) Version 2, available from the file
7 * COPYING in the main directory of this source tree, or the
8 * OpenIB.org BSD license below:
9 *
10 * Redistribution and use in source and binary forms, with or
11 * without modification, are permitted provided that the following
12 * conditions are met:
13 *
14 * - Redistributions of source code must retain the above
15 * copyright notice, this list of conditions and the following
16 * disclaimer.
17 *
18 * - Redistributions in binary form must reproduce the above
19 * copyright notice, this list of conditions and the following
20 * disclaimer in the documentation and/or other materials
21 * provided with the distribution.
22 *
23 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
26 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
27 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
28 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
29 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
30 * SOFTWARE.
31 */
32#include <linux/debugfs.h>
33#include <linux/seq_file.h>
34#include <linux/kernel.h>
35#include <linux/export.h>
36
37#include "qib.h"
38#include "qib_verbs.h"
39#include "qib_debugfs.h"
40
41static struct dentry *qib_dbg_root;
42
43#define DEBUGFS_FILE(name) \
44static const struct seq_operations _##name##_seq_ops = { \
45 .start = _##name##_seq_start, \
46 .next = _##name##_seq_next, \
47 .stop = _##name##_seq_stop, \
48 .show = _##name##_seq_show \
49}; \
50static int _##name##_open(struct inode *inode, struct file *s) \
51{ \
52 struct seq_file *seq; \
53 int ret; \
54 ret = seq_open(s, &_##name##_seq_ops); \
55 if (ret) \
56 return ret; \
57 seq = s->private_data; \
58 seq->private = inode->i_private; \
59 return 0; \
60} \
61static const struct file_operations _##name##_file_ops = { \
62 .owner = THIS_MODULE, \
63 .open = _##name##_open, \
64 .read = seq_read, \
65 .llseek = seq_lseek, \
66 .release = seq_release \
67};
68
69static void *_opcode_stats_seq_start(struct seq_file *s, loff_t *pos)
70{
71 struct qib_opcode_stats_perctx *opstats;
72
73 if (*pos >= ARRAY_SIZE(opstats->stats))
74 return NULL;
75 return pos;
76}
77
78static void *_opcode_stats_seq_next(struct seq_file *s, void *v, loff_t *pos)
79{
80 struct qib_opcode_stats_perctx *opstats;
81
82 ++*pos;
83 if (*pos >= ARRAY_SIZE(opstats->stats))
84 return NULL;
85 return pos;
86}
87
88
89static void _opcode_stats_seq_stop(struct seq_file *s, void *v)
90{
91 /* nothing allocated */
92}
93
94static int _opcode_stats_seq_show(struct seq_file *s, void *v)
95{
96 loff_t *spos = v;
97 loff_t i = *spos, j;
98 u64 n_packets = 0, n_bytes = 0;
99 struct qib_ibdev *ibd = (struct qib_ibdev *)s->private;
100 struct qib_devdata *dd = dd_from_dev(ibd);
101
102 for (j = 0; j < dd->first_user_ctxt; j++) {
103 if (!dd->rcd[j])
104 continue;
105 n_packets += dd->rcd[j]->opstats->stats[i].n_packets;
106 n_bytes += dd->rcd[j]->opstats->stats[i].n_bytes;
107 }
108 if (!n_packets && !n_bytes)
109 return SEQ_SKIP;
110 seq_printf(s, "%02llx %llu/%llu\n", i,
111 (unsigned long long) n_packets,
112 (unsigned long long) n_bytes);
113
114 return 0;
115}
116
117DEBUGFS_FILE(opcode_stats)
118
119static void *_ctx_stats_seq_start(struct seq_file *s, loff_t *pos)
120{
121 struct qib_ibdev *ibd = (struct qib_ibdev *)s->private;
122 struct qib_devdata *dd = dd_from_dev(ibd);
123
124 if (!*pos)
125 return SEQ_START_TOKEN;
126 if (*pos >= dd->first_user_ctxt)
127 return NULL;
128 return pos;
129}
130
131static void *_ctx_stats_seq_next(struct seq_file *s, void *v, loff_t *pos)
132{
133 struct qib_ibdev *ibd = (struct qib_ibdev *)s->private;
134 struct qib_devdata *dd = dd_from_dev(ibd);
135
136 if (v == SEQ_START_TOKEN)
137 return pos;
138
139 ++*pos;
140 if (*pos >= dd->first_user_ctxt)
141 return NULL;
142 return pos;
143}
144
145static void _ctx_stats_seq_stop(struct seq_file *s, void *v)
146{
147 /* nothing allocated */
148}
149
150static int _ctx_stats_seq_show(struct seq_file *s, void *v)
151{
152 loff_t *spos;
153 loff_t i, j;
154 u64 n_packets = 0;
155 struct qib_ibdev *ibd = (struct qib_ibdev *)s->private;
156 struct qib_devdata *dd = dd_from_dev(ibd);
157
158 if (v == SEQ_START_TOKEN) {
159 seq_puts(s, "Ctx:npkts\n");
160 return 0;
161 }
162
163 spos = v;
164 i = *spos;
165
166 if (!dd->rcd[i])
167 return SEQ_SKIP;
168
169 for (j = 0; j < ARRAY_SIZE(dd->rcd[i]->opstats->stats); j++)
170 n_packets += dd->rcd[i]->opstats->stats[j].n_packets;
171
172 if (!n_packets)
173 return SEQ_SKIP;
174
175 seq_printf(s, " %llu:%llu\n", i, n_packets);
176 return 0;
177}
178
179DEBUGFS_FILE(ctx_stats)
180
181static void *_qp_stats_seq_start(struct seq_file *s, loff_t *pos)
182 __acquires(RCU)
183{
184 struct rvt_qp_iter *iter;
185 loff_t n = *pos;
186
187 iter = rvt_qp_iter_init(s->private, 0, NULL);
188
189 /* stop calls rcu_read_unlock */
190 rcu_read_lock();
191
192 if (!iter)
193 return NULL;
194
195 do {
196 if (rvt_qp_iter_next(iter)) {
197 kfree(iter);
198 return NULL;
199 }
200 } while (n--);
201
202 return iter;
203}
204
205static void *_qp_stats_seq_next(struct seq_file *s, void *iter_ptr,
206 loff_t *pos)
207 __must_hold(RCU)
208{
209 struct rvt_qp_iter *iter = iter_ptr;
210
211 (*pos)++;
212
213 if (rvt_qp_iter_next(iter)) {
214 kfree(iter);
215 return NULL;
216 }
217
218 return iter;
219}
220
221static void _qp_stats_seq_stop(struct seq_file *s, void *iter_ptr)
222 __releases(RCU)
223{
224 rcu_read_unlock();
225}
226
227static int _qp_stats_seq_show(struct seq_file *s, void *iter_ptr)
228{
229 struct rvt_qp_iter *iter = iter_ptr;
230
231 if (!iter)
232 return 0;
233
234 qib_qp_iter_print(s, iter);
235
236 return 0;
237}
238
239DEBUGFS_FILE(qp_stats)
240
241void qib_dbg_ibdev_init(struct qib_ibdev *ibd)
242{
243 struct dentry *root;
244 char name[10];
245
246 snprintf(name, sizeof(name), "qib%d", dd_from_dev(ibd)->unit);
247 root = debugfs_create_dir(name, qib_dbg_root);
248 ibd->qib_ibdev_dbg = root;
249
250 debugfs_create_file("opcode_stats", 0400, root, ibd,
251 &_opcode_stats_file_ops);
252 debugfs_create_file("ctx_stats", 0400, root, ibd, &_ctx_stats_file_ops);
253 debugfs_create_file("qp_stats", 0400, root, ibd, &_qp_stats_file_ops);
254}
255
256void qib_dbg_ibdev_exit(struct qib_ibdev *ibd)
257{
258 if (!qib_dbg_root)
259 goto out;
260 debugfs_remove_recursive(ibd->qib_ibdev_dbg);
261out:
262 ibd->qib_ibdev_dbg = NULL;
263}
264
265void qib_dbg_init(void)
266{
267 qib_dbg_root = debugfs_create_dir(QIB_DRV_NAME, NULL);
268}
269
270void qib_dbg_exit(void)
271{
272 debugfs_remove_recursive(qib_dbg_root);
273 qib_dbg_root = NULL;
274}
drivers/infiniband/hw/qib/qib_debugfs.h+0−45
@@ -1,45 +0,0 @@
1#ifndef _QIB_DEBUGFS_H
2#define _QIB_DEBUGFS_H
3
4#ifdef CONFIG_DEBUG_FS
5/*
6 * Copyright (c) 2013 Intel Corporation. All rights reserved.
7 *
8 * This software is available to you under a choice of one of two
9 * licenses. You may choose to be licensed under the terms of the GNU
10 * General Public License (GPL) Version 2, available from the file
11 * COPYING in the main directory of this source tree, or the
12 * OpenIB.org BSD license below:
13 *
14 * Redistribution and use in source and binary forms, with or
15 * without modification, are permitted provided that the following
16 * conditions are met:
17 *
18 * - Redistributions of source code must retain the above
19 * copyright notice, this list of conditions and the following
20 * disclaimer.
21 *
22 * - Redistributions in binary form must reproduce the above
23 * copyright notice, this list of conditions and the following
24 * disclaimer in the documentation and/or other materials
25 * provided with the distribution.
26 *
27 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
28 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
29 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
30 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
31 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
32 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
33 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
34 * SOFTWARE.
35 */
36
37struct qib_ibdev;
38void qib_dbg_ibdev_init(struct qib_ibdev *ibd);
39void qib_dbg_ibdev_exit(struct qib_ibdev *ibd);
40void qib_dbg_init(void);
41void qib_dbg_exit(void);
42
43#endif
44
45#endif /* _QIB_DEBUGFS_H */
drivers/infiniband/hw/qib/qib_diag.c+0−906
@@ -1,906 +0,0 @@
1/*
2 * Copyright (c) 2012 Intel Corporation. All rights reserved.
3 * Copyright (c) 2006 - 2012 QLogic Corporation. All rights reserved.
4 * Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved.
5 *
6 * This software is available to you under a choice of one of two
7 * licenses. You may choose to be licensed under the terms of the GNU
8 * General Public License (GPL) Version 2, available from the file
9 * COPYING in the main directory of this source tree, or the
10 * OpenIB.org BSD license below:
11 *
12 * Redistribution and use in source and binary forms, with or
13 * without modification, are permitted provided that the following
14 * conditions are met:
15 *
16 * - Redistributions of source code must retain the above
17 * copyright notice, this list of conditions and the following
18 * disclaimer.
19 *
20 * - Redistributions in binary form must reproduce the above
21 * copyright notice, this list of conditions and the following
22 * disclaimer in the documentation and/or other materials
23 * provided with the distribution.
24 *
25 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
26 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
27 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
28 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
29 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
30 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
31 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
32 * SOFTWARE.
33 */
34
35/*
36 * This file contains support for diagnostic functions. It is accessed by
37 * opening the qib_diag device, normally minor number 129. Diagnostic use
38 * of the QLogic_IB chip may render the chip or board unusable until the
39 * driver is unloaded, or in some cases, until the system is rebooted.
40 *
41 * Accesses to the chip through this interface are not similar to going
42 * through the /sys/bus/pci resource mmap interface.
43 */
44
45#include <linux/io.h>
46#include <linux/pci.h>
47#include <linux/poll.h>
48#include <linux/vmalloc.h>
49#include <linux/export.h>
50#include <linux/fs.h>
51#include <linux/uaccess.h>
52
53#include "qib.h"
54#include "qib_common.h"
55
56#undef pr_fmt
57#define pr_fmt(fmt) QIB_DRV_NAME ": " fmt
58
59/*
60 * Each client that opens the diag device must read then write
61 * offset 0, to prevent lossage from random cat or od. diag_state
62 * sequences this "handshake".
63 */
64enum diag_state { UNUSED = 0, OPENED, INIT, READY };
65
66/* State for an individual client. PID so children cannot abuse handshake */
67static struct qib_diag_client {
68 struct qib_diag_client *next;
69 struct qib_devdata *dd;
70 pid_t pid;
71 enum diag_state state;
72} *client_pool;
73
74/*
75 * Get a client struct. Recycled if possible, else kmalloc.
76 * Must be called with qib_mutex held
77 */
78static struct qib_diag_client *get_client(struct qib_devdata *dd)
79{
80 struct qib_diag_client *dc;
81
82 dc = client_pool;
83 if (dc)
84 /* got from pool remove it and use */
85 client_pool = dc->next;
86 else
87 /* None in pool, alloc and init */
88 dc = kmalloc(sizeof(*dc), GFP_KERNEL);
89
90 if (dc) {
91 dc->next = NULL;
92 dc->dd = dd;
93 dc->pid = current->pid;
94 dc->state = OPENED;
95 }
96 return dc;
97}
98
99/*
100 * Return to pool. Must be called with qib_mutex held
101 */
102static void return_client(struct qib_diag_client *dc)
103{
104 struct qib_devdata *dd = dc->dd;
105 struct qib_diag_client *tdc, *rdc;
106
107 rdc = NULL;
108 if (dc == dd->diag_client) {
109 dd->diag_client = dc->next;
110 rdc = dc;
111 } else {
112 tdc = dc->dd->diag_client;
113 while (tdc) {
114 if (dc == tdc->next) {
115 tdc->next = dc->next;
116 rdc = dc;
117 break;
118 }
119 tdc = tdc->next;
120 }
121 }
122 if (rdc) {
123 rdc->state = UNUSED;
124 rdc->dd = NULL;
125 rdc->pid = 0;
126 rdc->next = client_pool;
127 client_pool = rdc;
128 }
129}
130
131static int qib_diag_open(struct inode *in, struct file *fp);
132static int qib_diag_release(struct inode *in, struct file *fp);
133static ssize_t qib_diag_read(struct file *fp, char __user *data,
134 size_t count, loff_t *off);
135static ssize_t qib_diag_write(struct file *fp, const char __user *data,
136 size_t count, loff_t *off);
137
138static const struct file_operations diag_file_ops = {
139 .owner = THIS_MODULE,
140 .write = qib_diag_write,
141 .read = qib_diag_read,
142 .open = qib_diag_open,
143 .release = qib_diag_release,
144 .llseek = default_llseek,
145};
146
147static atomic_t diagpkt_count = ATOMIC_INIT(0);
148static struct cdev *diagpkt_cdev;
149static struct device *diagpkt_device;
150
151static ssize_t qib_diagpkt_write(struct file *fp, const char __user *data,
152 size_t count, loff_t *off);
153
154static const struct file_operations diagpkt_file_ops = {
155 .owner = THIS_MODULE,
156 .write = qib_diagpkt_write,
157 .llseek = noop_llseek,
158};
159
160int qib_diag_add(struct qib_devdata *dd)
161{
162 char name[16];
163 int ret = 0;
164
165 if (atomic_inc_return(&diagpkt_count) == 1) {
166 ret = qib_cdev_init(QIB_DIAGPKT_MINOR, "ipath_diagpkt",
167 &diagpkt_file_ops, &diagpkt_cdev,
168 &diagpkt_device);
169 if (ret)
170 goto done;
171 }
172
173 snprintf(name, sizeof(name), "ipath_diag%d", dd->unit);
174 ret = qib_cdev_init(QIB_DIAG_MINOR_BASE + dd->unit, name,
175 &diag_file_ops, &dd->diag_cdev,
176 &dd->diag_device);
177done:
178 return ret;
179}
180
181static void qib_unregister_observers(struct qib_devdata *dd);
182
183void qib_diag_remove(struct qib_devdata *dd)
184{
185 struct qib_diag_client *dc;
186
187 if (atomic_dec_and_test(&diagpkt_count))
188 qib_cdev_cleanup(&diagpkt_cdev, &diagpkt_device);
189
190 qib_cdev_cleanup(&dd->diag_cdev, &dd->diag_device);
191
192 /*
193 * Return all diag_clients of this device. There should be none,
194 * as we are "guaranteed" that no clients are still open
195 */
196 while (dd->diag_client)
197 return_client(dd->diag_client);
198
199 /* Now clean up all unused client structs */
200 while (client_pool) {
201 dc = client_pool;
202 client_pool = dc->next;
203 kfree(dc);
204 }
205 /* Clean up observer list */
206 qib_unregister_observers(dd);
207}
208
209/* qib_remap_ioaddr32 - remap an offset into chip address space to __iomem *
210 *
211 * @dd: the qlogic_ib device
212 * @offs: the offset in chip-space
213 * @cntp: Pointer to max (byte) count for transfer starting at offset
214 * This returns a u32 __iomem * so it can be used for both 64 and 32-bit
215 * mapping. It is needed because with the use of PAT for control of
216 * write-combining, the logically contiguous address-space of the chip
217 * may be split into virtually non-contiguous spaces, with different
218 * attributes, which are them mapped to contiguous physical space
219 * based from the first BAR.
220 *
221 * The code below makes the same assumptions as were made in
222 * init_chip_wc_pat() (qib_init.c), copied here:
223 * Assumes chip address space looks like:
224 * - kregs + sregs + cregs + uregs (in any order)
225 * - piobufs (2K and 4K bufs in either order)
226 * or:
227 * - kregs + sregs + cregs (in any order)
228 * - piobufs (2K and 4K bufs in either order)
229 * - uregs
230 *
231 * If cntp is non-NULL, returns how many bytes from offset can be accessed
232 * Returns 0 if the offset is not mapped.
233 */
234static u32 __iomem *qib_remap_ioaddr32(struct qib_devdata *dd, u32 offset,
235 u32 *cntp)
236{
237 u32 kreglen;
238 u32 snd_bottom, snd_lim = 0;
239 u32 __iomem *krb32 = (u32 __iomem *)dd->kregbase;
240 u32 __iomem *map = NULL;
241 u32 cnt = 0;
242 u32 tot4k, offs4k;
243
244 /* First, simplest case, offset is within the first map. */
245 kreglen = (dd->kregend - dd->kregbase) * sizeof(u64);
246 if (offset < kreglen) {
247 map = krb32 + (offset / sizeof(u32));
248 cnt = kreglen - offset;
249 goto mapped;
250 }
251
252 /*
253 * Next check for user regs, the next most common case,
254 * and a cheap check because if they are not in the first map
255 * they are last in chip.
256 */
257 if (dd->userbase) {
258 /* If user regs mapped, they are after send, so set limit. */
259 u32 ulim = (dd->cfgctxts * dd->ureg_align) + dd->uregbase;
260
261 if (!dd->piovl15base)
262 snd_lim = dd->uregbase;
263 krb32 = (u32 __iomem *)dd->userbase;
264 if (offset >= dd->uregbase && offset < ulim) {
265 map = krb32 + (offset - dd->uregbase) / sizeof(u32);
266 cnt = ulim - offset;
267 goto mapped;
268 }
269 }
270
271 /*
272 * Lastly, check for offset within Send Buffers.
273 * This is gnarly because struct devdata is deliberately vague
274 * about things like 7322 VL15 buffers, and we are not in
275 * chip-specific code here, so should not make many assumptions.
276 * The one we _do_ make is that the only chip that has more sndbufs
277 * than we admit is the 7322, and it has userregs above that, so
278 * we know the snd_lim.
279 */
280 /* Assume 2K buffers are first. */
281 snd_bottom = dd->pio2k_bufbase;
282 if (snd_lim == 0) {
283 u32 tot2k = dd->piobcnt2k * ALIGN(dd->piosize2k, dd->palign);
284
285 snd_lim = snd_bottom + tot2k;
286 }
287 /* If 4k buffers exist, account for them by bumping
288 * appropriate limit.
289 */
290 tot4k = dd->piobcnt4k * dd->align4k;
291 offs4k = dd->piobufbase >> 32;
292 if (dd->piobcnt4k) {
293 if (snd_bottom > offs4k)
294 snd_bottom = offs4k;
295 else {
296 /* 4k above 2k. Bump snd_lim, if needed*/
297 if (!dd->userbase || dd->piovl15base)
298 snd_lim = offs4k + tot4k;
299 }
300 }
301 /*
302 * Judgement call: can we ignore the space between SendBuffs and
303 * UserRegs, where we would like to see vl15 buffs, but not more?
304 */
305 if (offset >= snd_bottom && offset < snd_lim) {
306 offset -= snd_bottom;
307 map = (u32 __iomem *)dd->piobase + (offset / sizeof(u32));
308 cnt = snd_lim - offset;
309 }
310
311 if (!map && offs4k && dd->piovl15base) {
312 snd_lim = offs4k + tot4k + 2 * dd->align4k;
313 if (offset >= (offs4k + tot4k) && offset < snd_lim) {
314 map = (u32 __iomem *)dd->piovl15base +
315 ((offset - (offs4k + tot4k)) / sizeof(u32));
316 cnt = snd_lim - offset;
317 }
318 }
319
320mapped:
321 if (cntp)
322 *cntp = cnt;
323 return map;
324}
325
326/*
327 * qib_read_umem64 - read a 64-bit quantity from the chip into user space
328 * @dd: the qlogic_ib device
329 * @uaddr: the location to store the data in user memory
330 * @regoffs: the offset from BAR0 (_NOT_ full pointer, anymore)
331 * @count: number of bytes to copy (multiple of 32 bits)
332 *
333 * This function also localizes all chip memory accesses.
334 * The copy should be written such that we read full cacheline packets
335 * from the chip. This is usually used for a single qword
336 *
337 * NOTE: This assumes the chip address is 64-bit aligned.
338 */
339static int qib_read_umem64(struct qib_devdata *dd, void __user *uaddr,
340 u32 regoffs, size_t count)
341{
342 const u64 __iomem *reg_addr;
343 const u64 __iomem *reg_end;
344 u32 limit;
345 int ret;
346
347 reg_addr = (const u64 __iomem *)qib_remap_ioaddr32(dd, regoffs, &limit);
348 if (reg_addr == NULL || limit == 0 || !(dd->flags & QIB_PRESENT)) {
349 ret = -EINVAL;
350 goto bail;
351 }
352 if (count >= limit)
353 count = limit;
354 reg_end = reg_addr + (count / sizeof(u64));
355
356 /* not very efficient, but it works for now */
357 while (reg_addr < reg_end) {
358 u64 data = readq(reg_addr);
359
360 if (copy_to_user(uaddr, &data, sizeof(u64))) {
361 ret = -EFAULT;
362 goto bail;
363 }
364 reg_addr++;
365 uaddr += sizeof(u64);
366 }
367 ret = 0;
368bail:
369 return ret;
370}
371
372/*
373 * qib_write_umem64 - write a 64-bit quantity to the chip from user space
374 * @dd: the qlogic_ib device
375 * @regoffs: the offset from BAR0 (_NOT_ full pointer, anymore)
376 * @uaddr: the source of the data in user memory
377 * @count: the number of bytes to copy (multiple of 32 bits)
378 *
379 * This is usually used for a single qword
380 * NOTE: This assumes the chip address is 64-bit aligned.
381 */
382
383static int qib_write_umem64(struct qib_devdata *dd, u32 regoffs,
384 const void __user *uaddr, size_t count)
385{
386 u64 __iomem *reg_addr;
387 const u64 __iomem *reg_end;
388 u32 limit;
389 int ret;
390
391 reg_addr = (u64 __iomem *)qib_remap_ioaddr32(dd, regoffs, &limit);
392 if (reg_addr == NULL || limit == 0 || !(dd->flags & QIB_PRESENT)) {
393 ret = -EINVAL;
394 goto bail;
395 }
396 if (count >= limit)
397 count = limit;
398 reg_end = reg_addr + (count / sizeof(u64));
399
400 /* not very efficient, but it works for now */
401 while (reg_addr < reg_end) {
402 u64 data;
403
404 if (copy_from_user(&data, uaddr, sizeof(data))) {
405 ret = -EFAULT;
406 goto bail;
407 }
408 writeq(data, reg_addr);
409
410 reg_addr++;
411 uaddr += sizeof(u64);
412 }
413 ret = 0;
414bail:
415 return ret;
416}
417
418/*
419 * qib_read_umem32 - read a 32-bit quantity from the chip into user space
420 * @dd: the qlogic_ib device
421 * @uaddr: the location to store the data in user memory
422 * @regoffs: the offset from BAR0 (_NOT_ full pointer, anymore)
423 * @count: number of bytes to copy
424 *
425 * read 32 bit values, not 64 bit; for memories that only
426 * support 32 bit reads; usually a single dword.
427 */
428static int qib_read_umem32(struct qib_devdata *dd, void __user *uaddr,
429 u32 regoffs, size_t count)
430{
431 const u32 __iomem *reg_addr;
432 const u32 __iomem *reg_end;
433 u32 limit;
434 int ret;
435
436 reg_addr = qib_remap_ioaddr32(dd, regoffs, &limit);
437 if (reg_addr == NULL || limit == 0 || !(dd->flags & QIB_PRESENT)) {
438 ret = -EINVAL;
439 goto bail;
440 }
441 if (count >= limit)
442 count = limit;
443 reg_end = reg_addr + (count / sizeof(u32));
444
445 /* not very efficient, but it works for now */
446 while (reg_addr < reg_end) {
447 u32 data = readl(reg_addr);
448
449 if (copy_to_user(uaddr, &data, sizeof(data))) {
450 ret = -EFAULT;
451 goto bail;
452 }
453
454 reg_addr++;
455 uaddr += sizeof(u32);
456
457 }
458 ret = 0;
459bail:
460 return ret;
461}
462
463/*
464 * qib_write_umem32 - write a 32-bit quantity to the chip from user space
465 * @dd: the qlogic_ib device
466 * @regoffs: the offset from BAR0 (_NOT_ full pointer, anymore)
467 * @uaddr: the source of the data in user memory
468 * @count: number of bytes to copy
469 *
470 * write 32 bit values, not 64 bit; for memories that only
471 * support 32 bit write; usually a single dword.
472 */
473
474static int qib_write_umem32(struct qib_devdata *dd, u32 regoffs,
475 const void __user *uaddr, size_t count)
476{
477 u32 __iomem *reg_addr;
478 const u32 __iomem *reg_end;
479 u32 limit;
480 int ret;
481
482 reg_addr = qib_remap_ioaddr32(dd, regoffs, &limit);
483 if (reg_addr == NULL || limit == 0 || !(dd->flags & QIB_PRESENT)) {
484 ret = -EINVAL;
485 goto bail;
486 }
487 if (count >= limit)
488 count = limit;
489 reg_end = reg_addr + (count / sizeof(u32));
490
491 while (reg_addr < reg_end) {
492 u32 data;
493
494 if (copy_from_user(&data, uaddr, sizeof(data))) {
495 ret = -EFAULT;
496 goto bail;
497 }
498 writel(data, reg_addr);
499
500 reg_addr++;
501 uaddr += sizeof(u32);
502 }
503 ret = 0;
504bail:
505 return ret;
506}
507
508static int qib_diag_open(struct inode *in, struct file *fp)
509{
510 int unit = iminor(in) - QIB_DIAG_MINOR_BASE;
511 struct qib_devdata *dd;
512 struct qib_diag_client *dc;
513 int ret;
514
515 mutex_lock(&qib_mutex);
516
517 dd = qib_lookup(unit);
518
519 if (dd == NULL || !(dd->flags & QIB_PRESENT) ||
520 !dd->kregbase) {
521 ret = -ENODEV;
522 goto bail;
523 }
524
525 dc = get_client(dd);
526 if (!dc) {
527 ret = -ENOMEM;
528 goto bail;
529 }
530 dc->next = dd->diag_client;
531 dd->diag_client = dc;
532 fp->private_data = dc;
533 ret = 0;
534bail:
535 mutex_unlock(&qib_mutex);
536
537 return ret;
538}
539
540/**
541 * qib_diagpkt_write - write an IB packet
542 * @fp: the diag data device file pointer
543 * @data: qib_diag_pkt structure saying where to get the packet
544 * @count: size of data to write
545 * @off: unused by this code
546 */
547static ssize_t qib_diagpkt_write(struct file *fp,
548 const char __user *data,
549 size_t count, loff_t *off)
550{
551 u32 __iomem *piobuf;
552 u32 plen, pbufn, maxlen_reserve;
553 struct qib_diag_xpkt dp;
554 u32 *tmpbuf = NULL;
555 struct qib_devdata *dd;
556 struct qib_pportdata *ppd;
557 ssize_t ret = 0;
558
559 if (count != sizeof(dp)) {
560 ret = -EINVAL;
561 goto bail;
562 }
563 if (copy_from_user(&dp, data, sizeof(dp))) {
564 ret = -EFAULT;
565 goto bail;
566 }
567
568 dd = qib_lookup(dp.unit);
569 if (!dd || !(dd->flags & QIB_PRESENT) || !dd->kregbase) {
570 ret = -ENODEV;
571 goto bail;
572 }
573 if (!(dd->flags & QIB_INITTED)) {
574 /* no hardware, freeze, etc. */
575 ret = -ENODEV;
576 goto bail;
577 }
578
579 if (dp.version != _DIAG_XPKT_VERS) {
580 qib_dev_err(dd, "Invalid version %u for diagpkt_write\n",
581 dp.version);
582 ret = -EINVAL;
583 goto bail;
584 }
585 /* send count must be an exact number of dwords */
586 if (dp.len & 3) {
587 ret = -EINVAL;
588 goto bail;
589 }
590 if (!dp.port || dp.port > dd->num_pports) {
591 ret = -EINVAL;
592 goto bail;
593 }
594 ppd = &dd->pport[dp.port - 1];
595
596 /*
597 * need total length before first word written, plus 2 Dwords. One Dword
598 * is for padding so we get the full user data when not aligned on
599 * a word boundary. The other Dword is to make sure we have room for the
600 * ICRC which gets tacked on later.
601 */
602 maxlen_reserve = 2 * sizeof(u32);
603 if (dp.len > ppd->ibmaxlen - maxlen_reserve) {
604 ret = -EINVAL;
605 goto bail;
606 }
607
608 plen = sizeof(u32) + dp.len;
609
610 tmpbuf = vmalloc(plen);
611 if (!tmpbuf) {
612 ret = -ENOMEM;
613 goto bail;
614 }
615
616 if (copy_from_user(tmpbuf,
617 u64_to_user_ptr(dp.data),
618 dp.len)) {
619 ret = -EFAULT;
620 goto bail;
621 }
622
623 plen >>= 2; /* in dwords */
624
625 if (dp.pbc_wd == 0)
626 dp.pbc_wd = plen;
627
628 piobuf = dd->f_getsendbuf(ppd, dp.pbc_wd, &pbufn);
629 if (!piobuf) {
630 ret = -EBUSY;
631 goto bail;
632 }
633 /* disarm it just to be extra sure */
634 dd->f_sendctrl(dd->pport, QIB_SENDCTRL_DISARM_BUF(pbufn));
635
636 /* disable header check on pbufn for this packet */
637 dd->f_txchk_change(dd, pbufn, 1, TXCHK_CHG_TYPE_DIS1, NULL);
638
639 writeq(dp.pbc_wd, piobuf);
640 /*
641 * Copy all but the trigger word, then flush, so it's written
642 * to chip before trigger word, then write trigger word, then
643 * flush again, so packet is sent.
644 */
645 if (dd->flags & QIB_PIO_FLUSH_WC) {
646 qib_flush_wc();
647 qib_pio_copy(piobuf + 2, tmpbuf, plen - 1);
648 qib_flush_wc();
649 __raw_writel(tmpbuf[plen - 1], piobuf + plen + 1);
650 } else
651 qib_pio_copy(piobuf + 2, tmpbuf, plen);
652
653 if (dd->flags & QIB_USE_SPCL_TRIG) {
654 u32 spcl_off = (pbufn >= dd->piobcnt2k) ? 2047 : 1023;
655
656 qib_flush_wc();
657 __raw_writel(0xaebecede, piobuf + spcl_off);
658 }
659
660 /*
661 * Ensure buffer is written to the chip, then re-enable
662 * header checks (if supported by chip). The txchk
663 * code will ensure seen by chip before returning.
664 */
665 qib_flush_wc();
666 qib_sendbuf_done(dd, pbufn);
667 dd->f_txchk_change(dd, pbufn, 1, TXCHK_CHG_TYPE_ENAB1, NULL);
668
669 ret = sizeof(dp);
670
671bail:
672 vfree(tmpbuf);
673 return ret;
674}
675
676static int qib_diag_release(struct inode *in, struct file *fp)
677{
678 mutex_lock(&qib_mutex);
679 return_client(fp->private_data);
680 fp->private_data = NULL;
681 mutex_unlock(&qib_mutex);
682 return 0;
683}
684
685/*
686 * Chip-specific code calls to register its interest in
687 * a specific range.
688 */
689struct diag_observer_list_elt {
690 struct diag_observer_list_elt *next;
691 const struct diag_observer *op;
692};
693
694int qib_register_observer(struct qib_devdata *dd,
695 const struct diag_observer *op)
696{
697 struct diag_observer_list_elt *olp;
698 unsigned long flags;
699
700 if (!dd || !op)
701 return -EINVAL;
702 olp = vmalloc(sizeof(*olp));
703 if (!olp)
704 return -ENOMEM;
705
706 spin_lock_irqsave(&dd->qib_diag_trans_lock, flags);
707 olp->op = op;
708 olp->next = dd->diag_observer_list;
709 dd->diag_observer_list = olp;
710 spin_unlock_irqrestore(&dd->qib_diag_trans_lock, flags);
711
712 return 0;
713}
714
715/* Remove all registered observers when device is closed */
716static void qib_unregister_observers(struct qib_devdata *dd)
717{
718 struct diag_observer_list_elt *olp;
719 unsigned long flags;
720
721 spin_lock_irqsave(&dd->qib_diag_trans_lock, flags);
722 olp = dd->diag_observer_list;
723 while (olp) {
724 /* Pop one observer, let go of lock */
725 dd->diag_observer_list = olp->next;
726 spin_unlock_irqrestore(&dd->qib_diag_trans_lock, flags);
727 vfree(olp);
728 /* try again. */
729 spin_lock_irqsave(&dd->qib_diag_trans_lock, flags);
730 olp = dd->diag_observer_list;
731 }
732 spin_unlock_irqrestore(&dd->qib_diag_trans_lock, flags);
733}
734
735/*
736 * Find the observer, if any, for the specified address. Initial implementation
737 * is simple stack of observers. This must be called with diag transaction
738 * lock held.
739 */
740static const struct diag_observer *diag_get_observer(struct qib_devdata *dd,
741 u32 addr)
742{
743 struct diag_observer_list_elt *olp;
744 const struct diag_observer *op = NULL;
745
746 olp = dd->diag_observer_list;
747 while (olp) {
748 op = olp->op;
749 if (addr >= op->bottom && addr <= op->top)
750 break;
751 olp = olp->next;
752 }
753 if (!olp)
754 op = NULL;
755
756 return op;
757}
758
759static ssize_t qib_diag_read(struct file *fp, char __user *data,
760 size_t count, loff_t *off)
761{
762 struct qib_diag_client *dc = fp->private_data;
763 struct qib_devdata *dd = dc->dd;
764 ssize_t ret;
765
766 if (dc->pid != current->pid) {
767 ret = -EPERM;
768 goto bail;
769 }
770
771 if (count == 0)
772 ret = 0;
773 else if ((count % 4) || (*off % 4))
774 /* address or length is not 32-bit aligned, hence invalid */
775 ret = -EINVAL;
776 else if (dc->state < READY && (*off || count != 8))
777 ret = -EINVAL; /* prevent cat /dev/qib_diag* */
778 else {
779 unsigned long flags;
780 u64 data64 = 0;
781 int use_32;
782 const struct diag_observer *op;
783
784 use_32 = (count % 8) || (*off % 8);
785 ret = -1;
786 spin_lock_irqsave(&dd->qib_diag_trans_lock, flags);
787 /*
788 * Check for observer on this address range.
789 * we only support a single 32 or 64-bit read
790 * via observer, currently.
791 */
792 op = diag_get_observer(dd, *off);
793 if (op) {
794 u32 offset = *off;
795
796 ret = op->hook(dd, op, offset, &data64, 0, use_32);
797 }
798 /*
799 * We need to release lock before any copy_to_user(),
800 * whether implicit in qib_read_umem* or explicit below.
801 */
802 spin_unlock_irqrestore(&dd->qib_diag_trans_lock, flags);
803 if (!op) {
804 if (use_32)
805 /*
806 * Address or length is not 64-bit aligned;
807 * do 32-bit rd
808 */
809 ret = qib_read_umem32(dd, data, (u32) *off,
810 count);
811 else
812 ret = qib_read_umem64(dd, data, (u32) *off,
813 count);
814 } else if (ret == count) {
815 /* Below finishes case where observer existed */
816 ret = copy_to_user(data, &data64, use_32 ?
817 sizeof(u32) : sizeof(u64));
818 if (ret)
819 ret = -EFAULT;
820 }
821 }
822
823 if (ret >= 0) {
824 *off += count;
825 ret = count;
826 if (dc->state == OPENED)
827 dc->state = INIT;
828 }
829bail:
830 return ret;
831}
832
833static ssize_t qib_diag_write(struct file *fp, const char __user *data,
834 size_t count, loff_t *off)
835{
836 struct qib_diag_client *dc = fp->private_data;
837 struct qib_devdata *dd = dc->dd;
838 ssize_t ret;
839
840 if (dc->pid != current->pid) {
841 ret = -EPERM;
842 goto bail;
843 }
844
845 if (count == 0)
846 ret = 0;
847 else if ((count % 4) || (*off % 4))
848 /* address or length is not 32-bit aligned, hence invalid */
849 ret = -EINVAL;
850 else if (dc->state < READY &&
851 ((*off || count != 8) || dc->state != INIT))
852 /* No writes except second-step of init seq */
853 ret = -EINVAL; /* before any other write allowed */
854 else {
855 unsigned long flags;
856 const struct diag_observer *op = NULL;
857 int use_32 = (count % 8) || (*off % 8);
858
859 /*
860 * Check for observer on this address range.
861 * We only support a single 32 or 64-bit write
862 * via observer, currently. This helps, because
863 * we would otherwise have to jump through hoops
864 * to make "diag transaction" meaningful when we
865 * cannot do a copy_from_user while holding the lock.
866 */
867 if (count == 4 || count == 8) {
868 u64 data64;
869 u32 offset = *off;
870
871 ret = copy_from_user(&data64, data, count);
872 if (ret) {
873 ret = -EFAULT;
874 goto bail;
875 }
876 spin_lock_irqsave(&dd->qib_diag_trans_lock, flags);
877 op = diag_get_observer(dd, *off);
878 if (op)
879 ret = op->hook(dd, op, offset, &data64, ~0Ull,
880 use_32);
881 spin_unlock_irqrestore(&dd->qib_diag_trans_lock, flags);
882 }
883
884 if (!op) {
885 if (use_32)
886 /*
887 * Address or length is not 64-bit aligned;
888 * do 32-bit write
889 */
890 ret = qib_write_umem32(dd, (u32) *off, data,
891 count);
892 else
893 ret = qib_write_umem64(dd, (u32) *off, data,
894 count);
895 }
896 }
897
898 if (ret >= 0) {
899 *off += count;
900 ret = count;
901 if (dc->state == INIT)
902 dc->state = READY; /* all read/write OK now */
903 }
904bail:
905 return ret;
906}
drivers/infiniband/hw/qib/qib_driver.c+0−798
@@ -1,798 +0,0 @@
1/*
2 * Copyright (c) 2021 Cornelis Networks. All rights reserved.
3 * Copyright (c) 2013 Intel Corporation. All rights reserved.
4 * Copyright (c) 2006, 2007, 2008, 2009 QLogic Corporation. All rights reserved.
5 * Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved.
6 *
7 * This software is available to you under a choice of one of two
8 * licenses. You may choose to be licensed under the terms of the GNU
9 * General Public License (GPL) Version 2, available from the file
10 * COPYING in the main directory of this source tree, or the
11 * OpenIB.org BSD license below:
12 *
13 * Redistribution and use in source and binary forms, with or
14 * without modification, are permitted provided that the following
15 * conditions are met:
16 *
17 * - Redistributions of source code must retain the above
18 * copyright notice, this list of conditions and the following
19 * disclaimer.
20 *
21 * - Redistributions in binary form must reproduce the above
22 * copyright notice, this list of conditions and the following
23 * disclaimer in the documentation and/or other materials
24 * provided with the distribution.
25 *
26 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
27 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
28 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
29 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
30 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
31 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
32 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
33 * SOFTWARE.
34 */
35
36#include <linux/spinlock.h>
37#include <linux/pci.h>
38#include <linux/io.h>
39#include <linux/delay.h>
40#include <linux/netdevice.h>
41#include <linux/vmalloc.h>
42#include <linux/module.h>
43#include <linux/prefetch.h>
44
45#include "qib.h"
46
47DEFINE_MUTEX(qib_mutex); /* general driver use */
48
49unsigned qib_ibmtu;
50module_param_named(ibmtu, qib_ibmtu, uint, S_IRUGO);
51MODULE_PARM_DESC(ibmtu, "Set max IB MTU (0=2KB, 1=256, 2=512, ... 5=4096");
52
53unsigned qib_compat_ddr_negotiate = 1;
54module_param_named(compat_ddr_negotiate, qib_compat_ddr_negotiate, uint,
55 S_IWUSR | S_IRUGO);
56MODULE_PARM_DESC(compat_ddr_negotiate,
57 "Attempt pre-IBTA 1.2 DDR speed negotiation");
58
59MODULE_LICENSE("Dual BSD/GPL");
60MODULE_AUTHOR("Cornelis <support@cornelisnetworks.com>");
61MODULE_DESCRIPTION("Cornelis IB driver");
62
63/*
64 * QIB_PIO_MAXIBHDR is the max IB header size allowed for in our
65 * PIO send buffers. This is well beyond anything currently
66 * defined in the InfiniBand spec.
67 */
68#define QIB_PIO_MAXIBHDR 128
69
70/*
71 * QIB_MAX_PKT_RCV is the max # if packets processed per receive interrupt.
72 */
73#define QIB_MAX_PKT_RECV 64
74
75struct qlogic_ib_stats qib_stats;
76
77struct pci_dev *qib_get_pci_dev(struct rvt_dev_info *rdi)
78{
79 struct qib_ibdev *ibdev = container_of(rdi, struct qib_ibdev, rdi);
80 struct qib_devdata *dd = container_of(ibdev,
81 struct qib_devdata, verbs_dev);
82 return dd->pcidev;
83}
84
85/*
86 * Return count of units with at least one port ACTIVE.
87 */
88int qib_count_active_units(void)
89{
90 struct qib_devdata *dd;
91 struct qib_pportdata *ppd;
92 unsigned long index, flags;
93 int pidx, nunits_active = 0;
94
95 xa_lock_irqsave(&qib_dev_table, flags);
96 xa_for_each(&qib_dev_table, index, dd) {
97 if (!(dd->flags & QIB_PRESENT) || !dd->kregbase)
98 continue;
99 for (pidx = 0; pidx < dd->num_pports; ++pidx) {
100 ppd = dd->pport + pidx;
101 if (ppd->lid && (ppd->lflags & (QIBL_LINKINIT |
102 QIBL_LINKARMED | QIBL_LINKACTIVE))) {
103 nunits_active++;
104 break;
105 }
106 }
107 }
108 xa_unlock_irqrestore(&qib_dev_table, flags);
109 return nunits_active;
110}
111
112/*
113 * Return count of all units, optionally return in arguments
114 * the number of usable (present) units, and the number of
115 * ports that are up.
116 */
117int qib_count_units(int *npresentp, int *nupp)
118{
119 int nunits = 0, npresent = 0, nup = 0;
120 struct qib_devdata *dd;
121 unsigned long index, flags;
122 int pidx;
123 struct qib_pportdata *ppd;
124
125 xa_lock_irqsave(&qib_dev_table, flags);
126 xa_for_each(&qib_dev_table, index, dd) {
127 nunits++;
128 if ((dd->flags & QIB_PRESENT) && dd->kregbase)
129 npresent++;
130 for (pidx = 0; pidx < dd->num_pports; ++pidx) {
131 ppd = dd->pport + pidx;
132 if (ppd->lid && (ppd->lflags & (QIBL_LINKINIT |
133 QIBL_LINKARMED | QIBL_LINKACTIVE)))
134 nup++;
135 }
136 }
137 xa_unlock_irqrestore(&qib_dev_table, flags);
138
139 if (npresentp)
140 *npresentp = npresent;
141 if (nupp)
142 *nupp = nup;
143
144 return nunits;
145}
146
147/**
148 * qib_wait_linkstate - wait for an IB link state change to occur
149 * @ppd: the qlogic_ib device
150 * @state: the state to wait for
151 * @msecs: the number of milliseconds to wait
152 *
153 * wait up to msecs milliseconds for IB link state change to occur for
154 * now, take the easy polling route. Currently used only by
155 * qib_set_linkstate. Returns 0 if state reached, otherwise
156 * -ETIMEDOUT state can have multiple states set, for any of several
157 * transitions.
158 */
159int qib_wait_linkstate(struct qib_pportdata *ppd, u32 state, int msecs)
160{
161 int ret;
162 unsigned long flags;
163
164 spin_lock_irqsave(&ppd->lflags_lock, flags);
165 if (ppd->state_wanted) {
166 spin_unlock_irqrestore(&ppd->lflags_lock, flags);
167 ret = -EBUSY;
168 goto bail;
169 }
170 ppd->state_wanted = state;
171 spin_unlock_irqrestore(&ppd->lflags_lock, flags);
172 wait_event_interruptible_timeout(ppd->state_wait,
173 (ppd->lflags & state),
174 msecs_to_jiffies(msecs));
175 spin_lock_irqsave(&ppd->lflags_lock, flags);
176 ppd->state_wanted = 0;
177 spin_unlock_irqrestore(&ppd->lflags_lock, flags);
178
179 if (!(ppd->lflags & state))
180 ret = -ETIMEDOUT;
181 else
182 ret = 0;
183bail:
184 return ret;
185}
186
187int qib_set_linkstate(struct qib_pportdata *ppd, u8 newstate)
188{
189 u32 lstate;
190 int ret;
191 struct qib_devdata *dd = ppd->dd;
192 unsigned long flags;
193
194 switch (newstate) {
195 case QIB_IB_LINKDOWN_ONLY:
196 dd->f_set_ib_cfg(ppd, QIB_IB_CFG_LSTATE,
197 IB_LINKCMD_DOWN | IB_LINKINITCMD_NOP);
198 /* don't wait */
199 ret = 0;
200 goto bail;
201
202 case QIB_IB_LINKDOWN:
203 dd->f_set_ib_cfg(ppd, QIB_IB_CFG_LSTATE,
204 IB_LINKCMD_DOWN | IB_LINKINITCMD_POLL);
205 /* don't wait */
206 ret = 0;
207 goto bail;
208
209 case QIB_IB_LINKDOWN_SLEEP:
210 dd->f_set_ib_cfg(ppd, QIB_IB_CFG_LSTATE,
211 IB_LINKCMD_DOWN | IB_LINKINITCMD_SLEEP);
212 /* don't wait */
213 ret = 0;
214 goto bail;
215
216 case QIB_IB_LINKDOWN_DISABLE:
217 dd->f_set_ib_cfg(ppd, QIB_IB_CFG_LSTATE,
218 IB_LINKCMD_DOWN | IB_LINKINITCMD_DISABLE);
219 /* don't wait */
220 ret = 0;
221 goto bail;
222
223 case QIB_IB_LINKARM:
224 if (ppd->lflags & QIBL_LINKARMED) {
225 ret = 0;
226 goto bail;
227 }
228 if (!(ppd->lflags & (QIBL_LINKINIT | QIBL_LINKACTIVE))) {
229 ret = -EINVAL;
230 goto bail;
231 }
232 /*
233 * Since the port can be ACTIVE when we ask for ARMED,
234 * clear QIBL_LINKV so we can wait for a transition.
235 * If the link isn't ARMED, then something else happened
236 * and there is no point waiting for ARMED.
237 */
238 spin_lock_irqsave(&ppd->lflags_lock, flags);
239 ppd->lflags &= ~QIBL_LINKV;
240 spin_unlock_irqrestore(&ppd->lflags_lock, flags);
241 dd->f_set_ib_cfg(ppd, QIB_IB_CFG_LSTATE,
242 IB_LINKCMD_ARMED | IB_LINKINITCMD_NOP);
243 lstate = QIBL_LINKV;
244 break;
245
246 case QIB_IB_LINKACTIVE:
247 if (ppd->lflags & QIBL_LINKACTIVE) {
248 ret = 0;
249 goto bail;
250 }
251 if (!(ppd->lflags & QIBL_LINKARMED)) {
252 ret = -EINVAL;
253 goto bail;
254 }
255 dd->f_set_ib_cfg(ppd, QIB_IB_CFG_LSTATE,
256 IB_LINKCMD_ACTIVE | IB_LINKINITCMD_NOP);
257 lstate = QIBL_LINKACTIVE;
258 break;
259
260 default:
261 ret = -EINVAL;
262 goto bail;
263 }
264 ret = qib_wait_linkstate(ppd, lstate, 10);
265
266bail:
267 return ret;
268}
269
270/*
271 * Get address of eager buffer from it's index (allocated in chunks, not
272 * contiguous).
273 */
274static inline void *qib_get_egrbuf(const struct qib_ctxtdata *rcd, u32 etail)
275{
276 const u32 chunk = etail >> rcd->rcvegrbufs_perchunk_shift;
277 const u32 idx = etail & ((u32)rcd->rcvegrbufs_perchunk - 1);
278
279 return rcd->rcvegrbuf[chunk] + (idx << rcd->dd->rcvegrbufsize_shift);
280}
281
282/*
283 * Returns 1 if error was a CRC, else 0.
284 * Needed for some chip's synthesized error counters.
285 */
286static u32 qib_rcv_hdrerr(struct qib_ctxtdata *rcd, struct qib_pportdata *ppd,
287 u32 ctxt, u32 eflags, u32 l, u32 etail,
288 __le32 *rhf_addr, struct qib_message_header *rhdr)
289{
290 u32 ret = 0;
291
292 if (eflags & (QLOGIC_IB_RHF_H_ICRCERR | QLOGIC_IB_RHF_H_VCRCERR))
293 ret = 1;
294 else if (eflags == QLOGIC_IB_RHF_H_TIDERR) {
295 /* For TIDERR and RC QPs premptively schedule a NAK */
296 struct ib_header *hdr = (struct ib_header *)rhdr;
297 struct ib_other_headers *ohdr = NULL;
298 struct qib_ibport *ibp = &ppd->ibport_data;
299 struct qib_devdata *dd = ppd->dd;
300 struct rvt_dev_info *rdi = &dd->verbs_dev.rdi;
301 struct rvt_qp *qp = NULL;
302 u32 tlen = qib_hdrget_length_in_bytes(rhf_addr);
303 u16 lid = be16_to_cpu(hdr->lrh[1]);
304 int lnh = be16_to_cpu(hdr->lrh[0]) & 3;
305 u32 qp_num;
306 u32 opcode;
307 u32 psn;
308 int diff;
309
310 /* Sanity check packet */
311 if (tlen < 24)
312 goto drop;
313
314 if (lid < be16_to_cpu(IB_MULTICAST_LID_BASE)) {
315 lid &= ~((1 << ppd->lmc) - 1);
316 if (unlikely(lid != ppd->lid))
317 goto drop;
318 }
319
320 /* Check for GRH */
321 if (lnh == QIB_LRH_BTH)
322 ohdr = &hdr->u.oth;
323 else if (lnh == QIB_LRH_GRH) {
324 u32 vtf;
325
326 ohdr = &hdr->u.l.oth;
327 if (hdr->u.l.grh.next_hdr != IB_GRH_NEXT_HDR)
328 goto drop;
329 vtf = be32_to_cpu(hdr->u.l.grh.version_tclass_flow);
330 if ((vtf >> IB_GRH_VERSION_SHIFT) != IB_GRH_VERSION)
331 goto drop;
332 } else
333 goto drop;
334
335 /* Get opcode and PSN from packet */
336 opcode = be32_to_cpu(ohdr->bth[0]);
337 opcode >>= 24;
338 psn = be32_to_cpu(ohdr->bth[2]);
339
340 /* Get the destination QP number. */
341 qp_num = be32_to_cpu(ohdr->bth[1]) & RVT_QPN_MASK;
342 if (qp_num != QIB_MULTICAST_QPN) {
343 int ruc_res;
344
345 rcu_read_lock();
346 qp = rvt_lookup_qpn(rdi, &ibp->rvp, qp_num);
347 if (!qp) {
348 rcu_read_unlock();
349 goto drop;
350 }
351
352 /*
353 * Handle only RC QPs - for other QP types drop error
354 * packet.
355 */
356 spin_lock(&qp->r_lock);
357
358 /* Check for valid receive state. */
359 if (!(ib_rvt_state_ops[qp->state] &
360 RVT_PROCESS_RECV_OK)) {
361 ibp->rvp.n_pkt_drops++;
362 goto unlock;
363 }
364
365 switch (qp->ibqp.qp_type) {
366 case IB_QPT_RC:
367 ruc_res =
368 qib_ruc_check_hdr(
369 ibp, hdr,
370 lnh == QIB_LRH_GRH,
371 qp,
372 be32_to_cpu(ohdr->bth[0]));
373 if (ruc_res)
374 goto unlock;
375
376 /* Only deal with RDMA Writes for now */
377 if (opcode <
378 IB_OPCODE_RC_RDMA_READ_RESPONSE_FIRST) {
379 diff = qib_cmp24(psn, qp->r_psn);
380 if (!qp->r_nak_state && diff >= 0) {
381 ibp->rvp.n_rc_seqnak++;
382 qp->r_nak_state =
383 IB_NAK_PSN_ERROR;
384 /* Use the expected PSN. */
385 qp->r_ack_psn = qp->r_psn;
386 /*
387 * Wait to send the sequence
388 * NAK until all packets
389 * in the receive queue have
390 * been processed.
391 * Otherwise, we end up
392 * propagating congestion.
393 */
394 if (list_empty(&qp->rspwait)) {
395 qp->r_flags |=
396 RVT_R_RSP_NAK;
397 rvt_get_qp(qp);
398 list_add_tail(
399 &qp->rspwait,
400 &rcd->qp_wait_list);
401 }
402 } /* Out of sequence NAK */
403 } /* QP Request NAKs */
404 break;
405 case IB_QPT_SMI:
406 case IB_QPT_GSI:
407 case IB_QPT_UD:
408 case IB_QPT_UC:
409 default:
410 /* For now don't handle any other QP types */
411 break;
412 }
413
414unlock:
415 spin_unlock(&qp->r_lock);
416 rcu_read_unlock();
417 } /* Unicast QP */
418 } /* Valid packet with TIDErr */
419
420drop:
421 return ret;
422}
423
424/*
425 * qib_kreceive - receive a packet
426 * @rcd: the qlogic_ib context
427 * @llic: gets count of good packets needed to clear lli,
428 * (used with chips that need need to track crcs for lli)
429 *
430 * called from interrupt handler for errors or receive interrupt
431 * Returns number of CRC error packets, needed by some chips for
432 * local link integrity tracking. crcs are adjusted down by following
433 * good packets, if any, and count of good packets is also tracked.
434 */
435u32 qib_kreceive(struct qib_ctxtdata *rcd, u32 *llic, u32 *npkts)
436{
437 struct qib_devdata *dd = rcd->dd;
438 struct qib_pportdata *ppd = rcd->ppd;
439 __le32 *rhf_addr;
440 void *ebuf;
441 const u32 rsize = dd->rcvhdrentsize; /* words */
442 const u32 maxcnt = dd->rcvhdrcnt * rsize; /* words */
443 u32 etail = -1, l, hdrqtail;
444 struct qib_message_header *hdr;
445 u32 eflags, etype, tlen, i = 0, updegr = 0, crcs = 0;
446 int last;
447 u64 lval;
448 struct rvt_qp *qp, *nqp;
449
450 l = rcd->head;
451 rhf_addr = (__le32 *) rcd->rcvhdrq + l + dd->rhf_offset;
452 if (dd->flags & QIB_NODMA_RTAIL) {
453 u32 seq = qib_hdrget_seq(rhf_addr);
454
455 if (seq != rcd->seq_cnt)
456 goto bail;
457 hdrqtail = 0;
458 } else {
459 hdrqtail = qib_get_rcvhdrtail(rcd);
460 if (l == hdrqtail)
461 goto bail;
462 smp_rmb(); /* prevent speculative reads of dma'ed hdrq */
463 }
464
465 for (last = 0, i = 1; !last; i += !last) {
466 hdr = dd->f_get_msgheader(dd, rhf_addr);
467 eflags = qib_hdrget_err_flags(rhf_addr);
468 etype = qib_hdrget_rcv_type(rhf_addr);
469 /* total length */
470 tlen = qib_hdrget_length_in_bytes(rhf_addr);
471 ebuf = NULL;
472 if ((dd->flags & QIB_NODMA_RTAIL) ?
473 qib_hdrget_use_egr_buf(rhf_addr) :
474 (etype != RCVHQ_RCV_TYPE_EXPECTED)) {
475 etail = qib_hdrget_index(rhf_addr);
476 updegr = 1;
477 if (tlen > sizeof(*hdr) ||
478 etype >= RCVHQ_RCV_TYPE_NON_KD) {
479 ebuf = qib_get_egrbuf(rcd, etail);
480 prefetch_range(ebuf, tlen - sizeof(*hdr));
481 }
482 }
483 if (!eflags) {
484 u16 lrh_len = be16_to_cpu(hdr->lrh[2]) << 2;
485
486 if (lrh_len != tlen) {
487 qib_stats.sps_lenerrs++;
488 goto move_along;
489 }
490 }
491 if (etype == RCVHQ_RCV_TYPE_NON_KD && !eflags &&
492 ebuf == NULL &&
493 tlen > (dd->rcvhdrentsize - 2 + 1 -
494 qib_hdrget_offset(rhf_addr)) << 2) {
495 goto move_along;
496 }
497
498 /*
499 * Both tiderr and qibhdrerr are set for all plain IB
500 * packets; only qibhdrerr should be set.
501 */
502 if (unlikely(eflags))
503 crcs += qib_rcv_hdrerr(rcd, ppd, rcd->ctxt, eflags, l,
504 etail, rhf_addr, hdr);
505 else if (etype == RCVHQ_RCV_TYPE_NON_KD) {
506 qib_ib_rcv(rcd, hdr, ebuf, tlen);
507 if (crcs)
508 crcs--;
509 else if (llic && *llic)
510 --*llic;
511 }
512move_along:
513 l += rsize;
514 if (l >= maxcnt)
515 l = 0;
516 if (i == QIB_MAX_PKT_RECV)
517 last = 1;
518
519 rhf_addr = (__le32 *) rcd->rcvhdrq + l + dd->rhf_offset;
520 if (dd->flags & QIB_NODMA_RTAIL) {
521 u32 seq = qib_hdrget_seq(rhf_addr);
522
523 if (++rcd->seq_cnt > 13)
524 rcd->seq_cnt = 1;
525 if (seq != rcd->seq_cnt)
526 last = 1;
527 } else if (l == hdrqtail)
528 last = 1;
529 /*
530 * Update head regs etc., every 16 packets, if not last pkt,
531 * to help prevent rcvhdrq overflows, when many packets
532 * are processed and queue is nearly full.
533 * Don't request an interrupt for intermediate updates.
534 */
535 lval = l;
536 if (!last && !(i & 0xf)) {
537 dd->f_update_usrhead(rcd, lval, updegr, etail, i);
538 updegr = 0;
539 }
540 }
541
542 rcd->head = l;
543
544 /*
545 * Iterate over all QPs waiting to respond.
546 * The list won't change since the IRQ is only run on one CPU.
547 */
548 list_for_each_entry_safe(qp, nqp, &rcd->qp_wait_list, rspwait) {
549 list_del_init(&qp->rspwait);
550 if (qp->r_flags & RVT_R_RSP_NAK) {
551 qp->r_flags &= ~RVT_R_RSP_NAK;
552 qib_send_rc_ack(qp);
553 }
554 if (qp->r_flags & RVT_R_RSP_SEND) {
555 unsigned long flags;
556
557 qp->r_flags &= ~RVT_R_RSP_SEND;
558 spin_lock_irqsave(&qp->s_lock, flags);
559 if (ib_rvt_state_ops[qp->state] &
560 RVT_PROCESS_OR_FLUSH_SEND)
561 qib_schedule_send(qp);
562 spin_unlock_irqrestore(&qp->s_lock, flags);
563 }
564 rvt_put_qp(qp);
565 }
566
567bail:
568 /* Report number of packets consumed */
569 if (npkts)
570 *npkts = i;
571
572 /*
573 * Always write head at end, and setup rcv interrupt, even
574 * if no packets were processed.
575 */
576 lval = (u64)rcd->head | dd->rhdrhead_intr_off;
577 dd->f_update_usrhead(rcd, lval, updegr, etail, i);
578 return crcs;
579}
580
581/**
582 * qib_set_mtu - set the MTU
583 * @ppd: the perport data
584 * @arg: the new MTU
585 *
586 * We can handle "any" incoming size, the issue here is whether we
587 * need to restrict our outgoing size. For now, we don't do any
588 * sanity checking on this, and we don't deal with what happens to
589 * programs that are already running when the size changes.
590 * NOTE: changing the MTU will usually cause the IBC to go back to
591 * link INIT state...
592 */
593int qib_set_mtu(struct qib_pportdata *ppd, u16 arg)
594{
595 u32 piosize;
596 int ret, chk;
597
598 if (arg != 256 && arg != 512 && arg != 1024 && arg != 2048 &&
599 arg != 4096) {
600 ret = -EINVAL;
601 goto bail;
602 }
603 chk = ib_mtu_enum_to_int(qib_ibmtu);
604 if (chk > 0 && arg > chk) {
605 ret = -EINVAL;
606 goto bail;
607 }
608
609 piosize = ppd->ibmaxlen;
610 ppd->ibmtu = arg;
611
612 if (arg >= (piosize - QIB_PIO_MAXIBHDR)) {
613 /* Only if it's not the initial value (or reset to it) */
614 if (piosize != ppd->init_ibmaxlen) {
615 if (arg > piosize && arg <= ppd->init_ibmaxlen)
616 piosize = ppd->init_ibmaxlen - 2 * sizeof(u32);
617 ppd->ibmaxlen = piosize;
618 }
619 } else if ((arg + QIB_PIO_MAXIBHDR) != ppd->ibmaxlen) {
620 piosize = arg + QIB_PIO_MAXIBHDR - 2 * sizeof(u32);
621 ppd->ibmaxlen = piosize;
622 }
623
624 ppd->dd->f_set_ib_cfg(ppd, QIB_IB_CFG_MTU, 0);
625
626 ret = 0;
627
628bail:
629 return ret;
630}
631
632int qib_set_lid(struct qib_pportdata *ppd, u32 lid, u8 lmc)
633{
634 struct qib_devdata *dd = ppd->dd;
635
636 ppd->lid = lid;
637 ppd->lmc = lmc;
638
639 dd->f_set_ib_cfg(ppd, QIB_IB_CFG_LIDLMC,
640 lid | (~((1U << lmc) - 1)) << 16);
641
642 qib_devinfo(dd->pcidev, "IB%u:%u got a lid: 0x%x\n",
643 dd->unit, ppd->port, lid);
644
645 return 0;
646}
647
648/*
649 * Following deal with the "obviously simple" task of overriding the state
650 * of the LEDS, which normally indicate link physical and logical status.
651 * The complications arise in dealing with different hardware mappings
652 * and the board-dependent routine being called from interrupts.
653 * and then there's the requirement to _flash_ them.
654 */
655#define LED_OVER_FREQ_SHIFT 8
656#define LED_OVER_FREQ_MASK (0xFF<<LED_OVER_FREQ_SHIFT)
657/* Below is "non-zero" to force override, but both actual LEDs are off */
658#define LED_OVER_BOTH_OFF (8)
659
660static void qib_run_led_override(struct timer_list *t)
661{
662 struct qib_pportdata *ppd = timer_container_of(ppd, t,
663 led_override_timer);
664 struct qib_devdata *dd = ppd->dd;
665 int timeoff;
666 int ph_idx;
667
668 if (!(dd->flags & QIB_INITTED))
669 return;
670
671 ph_idx = ppd->led_override_phase++ & 1;
672 ppd->led_override = ppd->led_override_vals[ph_idx];
673 timeoff = ppd->led_override_timeoff;
674
675 dd->f_setextled(ppd, 1);
676 /*
677 * don't re-fire the timer if user asked for it to be off; we let
678 * it fire one more time after they turn it off to simplify
679 */
680 if (ppd->led_override_vals[0] || ppd->led_override_vals[1])
681 mod_timer(&ppd->led_override_timer, jiffies + timeoff);
682}
683
684void qib_set_led_override(struct qib_pportdata *ppd, unsigned int val)
685{
686 struct qib_devdata *dd = ppd->dd;
687 int timeoff, freq;
688
689 if (!(dd->flags & QIB_INITTED))
690 return;
691
692 /* First check if we are blinking. If not, use 1HZ polling */
693 timeoff = HZ;
694 freq = (val & LED_OVER_FREQ_MASK) >> LED_OVER_FREQ_SHIFT;
695
696 if (freq) {
697 /* For blink, set each phase from one nybble of val */
698 ppd->led_override_vals[0] = val & 0xF;
699 ppd->led_override_vals[1] = (val >> 4) & 0xF;
700 timeoff = (HZ << 4)/freq;
701 } else {
702 /* Non-blink set both phases the same. */
703 ppd->led_override_vals[0] = val & 0xF;
704 ppd->led_override_vals[1] = val & 0xF;
705 }
706 ppd->led_override_timeoff = timeoff;
707
708 /*
709 * If the timer has not already been started, do so. Use a "quick"
710 * timeout so the function will be called soon, to look at our request.
711 */
712 if (atomic_inc_return(&ppd->led_override_timer_active) == 1) {
713 /* Need to start timer */
714 timer_setup(&ppd->led_override_timer, qib_run_led_override, 0);
715 ppd->led_override_timer.expires = jiffies + 1;
716 add_timer(&ppd->led_override_timer);
717 } else {
718 if (ppd->led_override_vals[0] || ppd->led_override_vals[1])
719 mod_timer(&ppd->led_override_timer, jiffies + 1);
720 atomic_dec(&ppd->led_override_timer_active);
721 }
722}
723
724/**
725 * qib_reset_device - reset the chip if possible
726 * @unit: the device to reset
727 *
728 * Whether or not reset is successful, we attempt to re-initialize the chip
729 * (that is, much like a driver unload/reload). We clear the INITTED flag
730 * so that the various entry points will fail until we reinitialize. For
731 * now, we only allow this if no user contexts are open that use chip resources
732 */
733int qib_reset_device(int unit)
734{
735 int ret, i;
736 struct qib_devdata *dd = qib_lookup(unit);
737 struct qib_pportdata *ppd;
738 unsigned long flags;
739 int pidx;
740
741 if (!dd) {
742 ret = -ENODEV;
743 goto bail;
744 }
745
746 qib_devinfo(dd->pcidev, "Reset on unit %u requested\n", unit);
747
748 if (!dd->kregbase || !(dd->flags & QIB_PRESENT)) {
749 qib_devinfo(dd->pcidev,
750 "Invalid unit number %u or not initialized or not present\n",
751 unit);
752 ret = -ENXIO;
753 goto bail;
754 }
755
756 spin_lock_irqsave(&dd->uctxt_lock, flags);
757 if (dd->rcd)
758 for (i = dd->first_user_ctxt; i < dd->cfgctxts; i++) {
759 if (!dd->rcd[i] || !dd->rcd[i]->cnt)
760 continue;
761 spin_unlock_irqrestore(&dd->uctxt_lock, flags);
762 ret = -EBUSY;
763 goto bail;
764 }
765 spin_unlock_irqrestore(&dd->uctxt_lock, flags);
766
767 for (pidx = 0; pidx < dd->num_pports; ++pidx) {
768 ppd = dd->pport + pidx;
769 if (atomic_read(&ppd->led_override_timer_active)) {
770 /* Need to stop LED timer, _then_ shut off LEDs */
771 timer_delete_sync(&ppd->led_override_timer);
772 atomic_set(&ppd->led_override_timer_active, 0);
773 }
774
775 /* Shut off LEDs after we are sure timer is not running */
776 ppd->led_override = LED_OVER_BOTH_OFF;
777 dd->f_setextled(ppd, 0);
778 if (dd->flags & QIB_HAS_SEND_DMA)
779 qib_teardown_sdma(ppd);
780 }
781
782 ret = dd->f_reset(dd);
783 if (ret == 1)
784 ret = qib_init(dd, 1);
785 else
786 ret = -EAGAIN;
787 if (ret)
788 qib_dev_err(dd,
789 "Reinitialize unit %u after reset failed with %d\n",
790 unit, ret);
791 else
792 qib_devinfo(dd->pcidev,
793 "Reinitialized unit %u after resetting\n",
794 unit);
795
796bail:
797 return ret;
798}
drivers/infiniband/hw/qib/qib_eeprom.c+0−271
@@ -1,271 +0,0 @@
1/*
2 * Copyright (c) 2012 Intel Corporation. All rights reserved.
3 * Copyright (c) 2006 - 2012 QLogic Corporation. All rights reserved.
4 * Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved.
5 *
6 * This software is available to you under a choice of one of two
7 * licenses. You may choose to be licensed under the terms of the GNU
8 * General Public License (GPL) Version 2, available from the file
9 * COPYING in the main directory of this source tree, or the
10 * OpenIB.org BSD license below:
11 *
12 * Redistribution and use in source and binary forms, with or
13 * without modification, are permitted provided that the following
14 * conditions are met:
15 *
16 * - Redistributions of source code must retain the above
17 * copyright notice, this list of conditions and the following
18 * disclaimer.
19 *
20 * - Redistributions in binary form must reproduce the above
21 * copyright notice, this list of conditions and the following
22 * disclaimer in the documentation and/or other materials
23 * provided with the distribution.
24 *
25 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
26 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
27 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
28 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
29 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
30 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
31 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
32 * SOFTWARE.
33 */
34
35#include <linux/delay.h>
36#include <linux/pci.h>
37#include <linux/vmalloc.h>
38
39#include "qib.h"
40
41/*
42 * Functions specific to the serial EEPROM on cards handled by ib_qib.
43 * The actual serail interface code is in qib_twsi.c. This file is a client
44 */
45
46/**
47 * qib_eeprom_read - receives bytes from the eeprom via I2C
48 * @dd: the qlogic_ib device
49 * @eeprom_offset: address to read from
50 * @buff: where to store result
51 * @len: number of bytes to receive
52 */
53int qib_eeprom_read(struct qib_devdata *dd, u8 eeprom_offset,
54 void *buff, int len)
55{
56 int ret;
57
58 ret = mutex_lock_interruptible(&dd->eep_lock);
59 if (!ret) {
60 ret = qib_twsi_reset(dd);
61 if (ret)
62 qib_dev_err(dd, "EEPROM Reset for read failed\n");
63 else
64 ret = qib_twsi_blk_rd(dd, dd->twsi_eeprom_dev,
65 eeprom_offset, buff, len);
66 mutex_unlock(&dd->eep_lock);
67 }
68
69 return ret;
70}
71
72/*
73 * Actually update the eeprom, first doing write enable if
74 * needed, then restoring write enable state.
75 * Must be called with eep_lock held
76 */
77static int eeprom_write_with_enable(struct qib_devdata *dd, u8 offset,
78 const void *buf, int len)
79{
80 int ret, pwen;
81
82 pwen = dd->f_eeprom_wen(dd, 1);
83 ret = qib_twsi_reset(dd);
84 if (ret)
85 qib_dev_err(dd, "EEPROM Reset for write failed\n");
86 else
87 ret = qib_twsi_blk_wr(dd, dd->twsi_eeprom_dev,
88 offset, buf, len);
89 dd->f_eeprom_wen(dd, pwen);
90 return ret;
91}
92
93/**
94 * qib_eeprom_write - writes data to the eeprom via I2C
95 * @dd: the qlogic_ib device
96 * @eeprom_offset: where to place data
97 * @buff: data to write
98 * @len: number of bytes to write
99 */
100int qib_eeprom_write(struct qib_devdata *dd, u8 eeprom_offset,
101 const void *buff, int len)
102{
103 int ret;
104
105 ret = mutex_lock_interruptible(&dd->eep_lock);
106 if (!ret) {
107 ret = eeprom_write_with_enable(dd, eeprom_offset, buff, len);
108 mutex_unlock(&dd->eep_lock);
109 }
110
111 return ret;
112}
113
114static u8 flash_csum(struct qib_flash *ifp, int adjust)
115{
116 u8 *ip = (u8 *) ifp;
117 u8 csum = 0, len;
118
119 /*
120 * Limit length checksummed to max length of actual data.
121 * Checksum of erased eeprom will still be bad, but we avoid
122 * reading past the end of the buffer we were passed.
123 */
124 len = ifp->if_length;
125 if (len > sizeof(struct qib_flash))
126 len = sizeof(struct qib_flash);
127 while (len--)
128 csum += *ip++;
129 csum -= ifp->if_csum;
130 csum = ~csum;
131 if (adjust)
132 ifp->if_csum = csum;
133
134 return csum;
135}
136
137/**
138 * qib_get_eeprom_info- get the GUID et al. from the TSWI EEPROM device
139 * @dd: the qlogic_ib device
140 *
141 * We have the capability to use the nguid field, and get
142 * the guid from the first chip's flash, to use for all of them.
143 */
144void qib_get_eeprom_info(struct qib_devdata *dd)
145{
146 void *buf;
147 struct qib_flash *ifp;
148 __be64 guid;
149 int len, eep_stat;
150 u8 csum, *bguid;
151 int t = dd->unit;
152 struct qib_devdata *dd0 = qib_lookup(0);
153
154 if (t && dd0->nguid > 1 && t <= dd0->nguid) {
155 u8 oguid;
156
157 dd->base_guid = dd0->base_guid;
158 bguid = (u8 *) &dd->base_guid;
159
160 oguid = bguid[7];
161 bguid[7] += t;
162 if (oguid > bguid[7]) {
163 if (bguid[6] == 0xff) {
164 if (bguid[5] == 0xff) {
165 qib_dev_err(dd,
166 "Can't set GUID from base, wraps to OUI!\n");
167 dd->base_guid = 0;
168 goto bail;
169 }
170 bguid[5]++;
171 }
172 bguid[6]++;
173 }
174 dd->nguid = 1;
175 goto bail;
176 }
177
178 /*
179 * Read full flash, not just currently used part, since it may have
180 * been written with a newer definition.
181 * */
182 len = sizeof(struct qib_flash);
183 buf = vmalloc(len);
184 if (!buf)
185 goto bail;
186
187 /*
188 * Use "public" eeprom read function, which does locking and
189 * figures out device. This will migrate to chip-specific.
190 */
191 eep_stat = qib_eeprom_read(dd, 0, buf, len);
192
193 if (eep_stat) {
194 qib_dev_err(dd, "Failed reading GUID from eeprom\n");
195 goto done;
196 }
197 ifp = (struct qib_flash *)buf;
198
199 csum = flash_csum(ifp, 0);
200 if (csum != ifp->if_csum) {
201 qib_devinfo(dd->pcidev,
202 "Bad I2C flash checksum: 0x%x, not 0x%x\n",
203 csum, ifp->if_csum);
204 goto done;
205 }
206 if (*(__be64 *) ifp->if_guid == cpu_to_be64(0) ||
207 *(__be64 *) ifp->if_guid == ~cpu_to_be64(0)) {
208 qib_dev_err(dd,
209 "Invalid GUID %llx from flash; ignoring\n",
210 *(unsigned long long *) ifp->if_guid);
211 /* don't allow GUID if all 0 or all 1's */
212 goto done;
213 }
214
215 /* complain, but allow it */
216 if (*(u64 *) ifp->if_guid == 0x100007511000000ULL)
217 qib_devinfo(dd->pcidev,
218 "Warning, GUID %llx is default, probably not correct!\n",
219 *(unsigned long long *) ifp->if_guid);
220
221 bguid = ifp->if_guid;
222 if (!bguid[0] && !bguid[1] && !bguid[2]) {
223 /*
224 * Original incorrect GUID format in flash; fix in
225 * core copy, by shifting up 2 octets; don't need to
226 * change top octet, since both it and shifted are 0.
227 */
228 bguid[1] = bguid[3];
229 bguid[2] = bguid[4];
230 bguid[3] = 0;
231 bguid[4] = 0;
232 guid = *(__be64 *) ifp->if_guid;
233 } else
234 guid = *(__be64 *) ifp->if_guid;
235 dd->base_guid = guid;
236 dd->nguid = ifp->if_numguid;
237 /*
238 * Things are slightly complicated by the desire to transparently
239 * support both the Pathscale 10-digit serial number and the QLogic
240 * 13-character version.
241 */
242 if ((ifp->if_fversion > 1) && ifp->if_sprefix[0] &&
243 ((u8 *) ifp->if_sprefix)[0] != 0xFF) {
244 char *snp = dd->serial;
245
246 /*
247 * This board has a Serial-prefix, which is stored
248 * elsewhere for backward-compatibility.
249 */
250 memcpy(snp, ifp->if_sprefix, sizeof(ifp->if_sprefix));
251 snp[sizeof(ifp->if_sprefix)] = '\0';
252 len = strlen(snp);
253 snp += len;
254 len = sizeof(dd->serial) - len;
255 if (len > sizeof(ifp->if_serial))
256 len = sizeof(ifp->if_serial);
257 memcpy(snp, ifp->if_serial, len);
258 } else {
259 memcpy(dd->serial, ifp->if_serial, sizeof(ifp->if_serial));
260 }
261 if (!strstr(ifp->if_comment, "Tested successfully"))
262 qib_dev_err(dd,
263 "Board SN %s did not pass functional test: %s\n",
264 dd->serial, ifp->if_comment);
265
266done:
267 vfree(buf);
268
269bail:;
270}
271
drivers/infiniband/hw/qib/qib_file_ops.c+0−2401
@@ -1,2401 +0,0 @@
1/*
2 * Copyright (c) 2012, 2013 Intel Corporation. All rights reserved.
3 * Copyright (c) 2006 - 2012 QLogic Corporation. All rights reserved.
4 * Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved.
5 *
6 * This software is available to you under a choice of one of two
7 * licenses. You may choose to be licensed under the terms of the GNU
8 * General Public License (GPL) Version 2, available from the file
9 * COPYING in the main directory of this source tree, or the
10 * OpenIB.org BSD license below:
11 *
12 * Redistribution and use in source and binary forms, with or
13 * without modification, are permitted provided that the following
14 * conditions are met:
15 *
16 * - Redistributions of source code must retain the above
17 * copyright notice, this list of conditions and the following
18 * disclaimer.
19 *
20 * - Redistributions in binary form must reproduce the above
21 * copyright notice, this list of conditions and the following
22 * disclaimer in the documentation and/or other materials
23 * provided with the distribution.
24 *
25 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
26 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
27 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
28 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
29 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
30 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
31 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
32 * SOFTWARE.
33 */
34
35#include <linux/pci.h>
36#include <linux/poll.h>
37#include <linux/cdev.h>
38#include <linux/swap.h>
39#include <linux/vmalloc.h>
40#include <linux/highmem.h>
41#include <linux/io.h>
42#include <linux/jiffies.h>
43#include <linux/delay.h>
44#include <linux/export.h>
45#include <linux/uio.h>
46#include <linux/pgtable.h>
47
48#include <rdma/ib.h>
49
50#include "qib.h"
51#include "qib_common.h"
52#include "qib_user_sdma.h"
53
54#undef pr_fmt
55#define pr_fmt(fmt) QIB_DRV_NAME ": " fmt
56
57static int qib_open(struct inode *, struct file *);
58static int qib_close(struct inode *, struct file *);
59static ssize_t qib_write(struct file *, const char __user *, size_t, loff_t *);
60static ssize_t qib_write_iter(struct kiocb *, struct iov_iter *);
61static __poll_t qib_poll(struct file *, struct poll_table_struct *);
62static int qib_mmapf(struct file *, struct vm_area_struct *);
63
64/*
65 * This is really, really weird shit - write() and writev() here
66 * have completely unrelated semantics. Sucky userland ABI,
67 * film at 11.
68 */
69static const struct file_operations qib_file_ops = {
70 .owner = THIS_MODULE,
71 .write = qib_write,
72 .write_iter = qib_write_iter,
73 .open = qib_open,
74 .release = qib_close,
75 .poll = qib_poll,
76 .mmap = qib_mmapf,
77 .llseek = noop_llseek,
78};
79
80/*
81 * Convert kernel virtual addresses to physical addresses so they don't
82 * potentially conflict with the chip addresses used as mmap offsets.
83 * It doesn't really matter what mmap offset we use as long as we can
84 * interpret it correctly.
85 */
86static u64 cvt_kvaddr(void *p)
87{
88 struct page *page;
89 u64 paddr = 0;
90
91 page = vmalloc_to_page(p);
92 if (page)
93 paddr = page_to_pfn(page) << PAGE_SHIFT;
94
95 return paddr;
96}
97
98static int qib_get_base_info(struct file *fp, void __user *ubase,
99 size_t ubase_size)
100{
101 struct qib_ctxtdata *rcd = ctxt_fp(fp);
102 int ret = 0;
103 struct qib_base_info *kinfo = NULL;
104 struct qib_devdata *dd = rcd->dd;
105 struct qib_pportdata *ppd = rcd->ppd;
106 unsigned subctxt_cnt;
107 int shared, master;
108 size_t sz;
109
110 subctxt_cnt = rcd->subctxt_cnt;
111 if (!subctxt_cnt) {
112 shared = 0;
113 master = 0;
114 subctxt_cnt = 1;
115 } else {
116 shared = 1;
117 master = !subctxt_fp(fp);
118 }
119
120 sz = sizeof(*kinfo);
121 /* If context sharing is not requested, allow the old size structure */
122 if (!shared)
123 sz -= 7 * sizeof(u64);
124 if (ubase_size < sz) {
125 ret = -EINVAL;
126 goto bail;
127 }
128
129 kinfo = kzalloc(sizeof(*kinfo), GFP_KERNEL);
130 if (kinfo == NULL) {
131 ret = -ENOMEM;
132 goto bail;
133 }
134
135 ret = dd->f_get_base_info(rcd, kinfo);
136 if (ret < 0)
137 goto bail;
138
139 kinfo->spi_rcvhdr_cnt = dd->rcvhdrcnt;
140 kinfo->spi_rcvhdrent_size = dd->rcvhdrentsize;
141 kinfo->spi_tidegrcnt = rcd->rcvegrcnt;
142 kinfo->spi_rcv_egrbufsize = dd->rcvegrbufsize;
143 /*
144 * have to mmap whole thing
145 */
146 kinfo->spi_rcv_egrbuftotlen =
147 rcd->rcvegrbuf_chunks * rcd->rcvegrbuf_size;
148 kinfo->spi_rcv_egrperchunk = rcd->rcvegrbufs_perchunk;
149 kinfo->spi_rcv_egrchunksize = kinfo->spi_rcv_egrbuftotlen /
150 rcd->rcvegrbuf_chunks;
151 kinfo->spi_tidcnt = dd->rcvtidcnt / subctxt_cnt;
152 if (master)
153 kinfo->spi_tidcnt += dd->rcvtidcnt % subctxt_cnt;
154 /*
155 * for this use, may be cfgctxts summed over all chips that
156 * are configured and present
157 */
158 kinfo->spi_nctxts = dd->cfgctxts;
159 /* unit (chip/board) our context is on */
160 kinfo->spi_unit = dd->unit;
161 kinfo->spi_port = ppd->port;
162 /* for now, only a single page */
163 kinfo->spi_tid_maxsize = PAGE_SIZE;
164
165 /*
166 * Doing this per context, and based on the skip value, etc. This has
167 * to be the actual buffer size, since the protocol code treats it
168 * as an array.
169 *
170 * These have to be set to user addresses in the user code via mmap.
171 * These values are used on return to user code for the mmap target
172 * addresses only. For 32 bit, same 44 bit address problem, so use
173 * the physical address, not virtual. Before 2.6.11, using the
174 * page_address() macro worked, but in 2.6.11, even that returns the
175 * full 64 bit address (upper bits all 1's). So far, using the
176 * physical addresses (or chip offsets, for chip mapping) works, but
177 * no doubt some future kernel release will change that, and we'll be
178 * on to yet another method of dealing with this.
179 * Normally only one of rcvhdr_tailaddr or rhf_offset is useful
180 * since the chips with non-zero rhf_offset don't normally
181 * enable tail register updates to host memory, but for testing,
182 * both can be enabled and used.
183 */
184 kinfo->spi_rcvhdr_base = (u64) rcd->rcvhdrq_phys;
185 kinfo->spi_rcvhdr_tailaddr = (u64) rcd->rcvhdrqtailaddr_phys;
186 kinfo->spi_rhf_offset = dd->rhf_offset;
187 kinfo->spi_rcv_egrbufs = (u64) rcd->rcvegr_phys;
188 kinfo->spi_pioavailaddr = (u64) dd->pioavailregs_phys;
189 /* setup per-unit (not port) status area for user programs */
190 kinfo->spi_status = (u64) kinfo->spi_pioavailaddr +
191 (char *) ppd->statusp -
192 (char *) dd->pioavailregs_dma;
193 kinfo->spi_uregbase = (u64) dd->uregbase + dd->ureg_align * rcd->ctxt;
194 if (!shared) {
195 kinfo->spi_piocnt = rcd->piocnt;
196 kinfo->spi_piobufbase = (u64) rcd->piobufs;
197 kinfo->spi_sendbuf_status = cvt_kvaddr(rcd->user_event_mask);
198 } else if (master) {
199 kinfo->spi_piocnt = (rcd->piocnt / subctxt_cnt) +
200 (rcd->piocnt % subctxt_cnt);
201 /* Master's PIO buffers are after all the slave's */
202 kinfo->spi_piobufbase = (u64) rcd->piobufs +
203 dd->palign *
204 (rcd->piocnt - kinfo->spi_piocnt);
205 } else {
206 unsigned slave = subctxt_fp(fp) - 1;
207
208 kinfo->spi_piocnt = rcd->piocnt / subctxt_cnt;
209 kinfo->spi_piobufbase = (u64) rcd->piobufs +
210 dd->palign * kinfo->spi_piocnt * slave;
211 }
212
213 if (shared) {
214 kinfo->spi_sendbuf_status =
215 cvt_kvaddr(&rcd->user_event_mask[subctxt_fp(fp)]);
216 /* only spi_subctxt_* fields should be set in this block! */
217 kinfo->spi_subctxt_uregbase = cvt_kvaddr(rcd->subctxt_uregbase);
218
219 kinfo->spi_subctxt_rcvegrbuf =
220 cvt_kvaddr(rcd->subctxt_rcvegrbuf);
221 kinfo->spi_subctxt_rcvhdr_base =
222 cvt_kvaddr(rcd->subctxt_rcvhdr_base);
223 }
224
225 /*
226 * All user buffers are 2KB buffers. If we ever support
227 * giving 4KB buffers to user processes, this will need some
228 * work. Can't use piobufbase directly, because it has
229 * both 2K and 4K buffer base values.
230 */
231 kinfo->spi_pioindex = (kinfo->spi_piobufbase - dd->pio2k_bufbase) /
232 dd->palign;
233 kinfo->spi_pioalign = dd->palign;
234 kinfo->spi_qpair = QIB_KD_QP;
235 /*
236 * user mode PIO buffers are always 2KB, even when 4KB can
237 * be received, and sent via the kernel; this is ibmaxlen
238 * for 2K MTU.
239 */
240 kinfo->spi_piosize = dd->piosize2k - 2 * sizeof(u32);
241 kinfo->spi_mtu = ppd->ibmaxlen; /* maxlen, not ibmtu */
242 kinfo->spi_ctxt = rcd->ctxt;
243 kinfo->spi_subctxt = subctxt_fp(fp);
244 kinfo->spi_sw_version = QIB_KERN_SWVERSION;
245 kinfo->spi_sw_version |= 1U << 31; /* QLogic-built, not kernel.org */
246 kinfo->spi_hw_version = dd->revision;
247
248 if (master)
249 kinfo->spi_runtime_flags |= QIB_RUNTIME_MASTER;
250
251 sz = (ubase_size < sizeof(*kinfo)) ? ubase_size : sizeof(*kinfo);
252 if (copy_to_user(ubase, kinfo, sz))
253 ret = -EFAULT;
254bail:
255 kfree(kinfo);
256 return ret;
257}
258
259/**
260 * qib_tid_update - update a context TID
261 * @rcd: the context
262 * @fp: the qib device file
263 * @ti: the TID information
264 *
265 * The new implementation as of Oct 2004 is that the driver assigns
266 * the tid and returns it to the caller. To reduce search time, we
267 * keep a cursor for each context, walking the shadow tid array to find
268 * one that's not in use.
269 *
270 * For now, if we can't allocate the full list, we fail, although
271 * in the long run, we'll allocate as many as we can, and the
272 * caller will deal with that by trying the remaining pages later.
273 * That means that when we fail, we have to mark the tids as not in
274 * use again, in our shadow copy.
275 *
276 * It's up to the caller to free the tids when they are done.
277 * We'll unlock the pages as they free them.
278 *
279 * Also, right now we are locking one page at a time, but since
280 * the intended use of this routine is for a single group of
281 * virtually contiguous pages, that should change to improve
282 * performance.
283 */
284static int qib_tid_update(struct qib_ctxtdata *rcd, struct file *fp,
285 const struct qib_tid_info *ti)
286{
287 int ret = 0, ntids;
288 u32 tid, ctxttid, cnt, i, tidcnt, tidoff;
289 u16 *tidlist;
290 struct qib_devdata *dd = rcd->dd;
291 u64 physaddr;
292 unsigned long vaddr;
293 u64 __iomem *tidbase;
294 unsigned long tidmap[8];
295 struct page **pagep = NULL;
296 unsigned subctxt = subctxt_fp(fp);
297
298 if (!dd->pageshadow) {
299 ret = -ENOMEM;
300 goto done;
301 }
302
303 cnt = ti->tidcnt;
304 if (!cnt) {
305 ret = -EFAULT;
306 goto done;
307 }
308 ctxttid = rcd->ctxt * dd->rcvtidcnt;
309 if (!rcd->subctxt_cnt) {
310 tidcnt = dd->rcvtidcnt;
311 tid = rcd->tidcursor;
312 tidoff = 0;
313 } else if (!subctxt) {
314 tidcnt = (dd->rcvtidcnt / rcd->subctxt_cnt) +
315 (dd->rcvtidcnt % rcd->subctxt_cnt);
316 tidoff = dd->rcvtidcnt - tidcnt;
317 ctxttid += tidoff;
318 tid = tidcursor_fp(fp);
319 } else {
320 tidcnt = dd->rcvtidcnt / rcd->subctxt_cnt;
321 tidoff = tidcnt * (subctxt - 1);
322 ctxttid += tidoff;
323 tid = tidcursor_fp(fp);
324 }
325 if (cnt > tidcnt) {
326 /* make sure it all fits in tid_pg_list */
327 qib_devinfo(dd->pcidev,
328 "Process tried to allocate %u TIDs, only trying max (%u)\n",
329 cnt, tidcnt);
330 cnt = tidcnt;
331 }
332 pagep = (struct page **) rcd->tid_pg_list;
333 tidlist = (u16 *) &pagep[dd->rcvtidcnt];
334 pagep += tidoff;
335 tidlist += tidoff;
336
337 memset(tidmap, 0, sizeof(tidmap));
338 /* before decrement; chip actual # */
339 ntids = tidcnt;
340 tidbase = (u64 __iomem *) (((char __iomem *) dd->kregbase) +
341 dd->rcvtidbase +
342 ctxttid * sizeof(*tidbase));
343
344 /* virtual address of first page in transfer */
345 vaddr = ti->tidvaddr;
346 if (!access_ok((void __user *) vaddr,
347 cnt * PAGE_SIZE)) {
348 ret = -EFAULT;
349 goto done;
350 }
351 ret = qib_get_user_pages(vaddr, cnt, pagep);
352 if (ret) {
353 /*
354 * if (ret == -EBUSY)
355 * We can't continue because the pagep array won't be
356 * initialized. This should never happen,
357 * unless perhaps the user has mpin'ed the pages
358 * themselves.
359 */
360 qib_devinfo(
361 dd->pcidev,
362 "Failed to lock addr %p, %u pages: errno %d\n",
363 (void *) vaddr, cnt, -ret);
364 goto done;
365 }
366 for (i = 0; i < cnt; i++, vaddr += PAGE_SIZE) {
367 dma_addr_t daddr;
368
369 for (; ntids--; tid++) {
370 if (tid == tidcnt)
371 tid = 0;
372 if (!dd->pageshadow[ctxttid + tid])
373 break;
374 }
375 if (ntids < 0) {
376 /*
377 * Oops, wrapped all the way through their TIDs,
378 * and didn't have enough free; see comments at
379 * start of routine
380 */
381 i--; /* last tidlist[i] not filled in */
382 ret = -ENOMEM;
383 break;
384 }
385 ret = qib_map_page(dd->pcidev, pagep[i], &daddr);
386 if (ret)
387 break;
388
389 tidlist[i] = tid + tidoff;
390 /* we "know" system pages and TID pages are same size */
391 dd->pageshadow[ctxttid + tid] = pagep[i];
392 dd->physshadow[ctxttid + tid] = daddr;
393 /*
394 * don't need atomic or it's overhead
395 */
396 __set_bit(tid, tidmap);
397 physaddr = dd->physshadow[ctxttid + tid];
398 /* PERFORMANCE: below should almost certainly be cached */
399 dd->f_put_tid(dd, &tidbase[tid],
400 RCVHQ_RCV_TYPE_EXPECTED, physaddr);
401 /*
402 * don't check this tid in qib_ctxtshadow, since we
403 * just filled it in; start with the next one.
404 */
405 tid++;
406 }
407
408 if (ret) {
409 u32 limit;
410cleanup:
411 /* jump here if copy out of updated info failed... */
412 /* same code that's in qib_free_tid() */
413 limit = sizeof(tidmap) * BITS_PER_BYTE;
414 if (limit > tidcnt)
415 /* just in case size changes in future */
416 limit = tidcnt;
417 tid = find_first_bit((const unsigned long *)tidmap, limit);
418 for (; tid < limit; tid++) {
419 if (!test_bit(tid, tidmap))
420 continue;
421 if (dd->pageshadow[ctxttid + tid]) {
422 dma_addr_t phys;
423
424 phys = dd->physshadow[ctxttid + tid];
425 dd->physshadow[ctxttid + tid] = dd->tidinvalid;
426 /* PERFORMANCE: below should almost certainly
427 * be cached
428 */
429 dd->f_put_tid(dd, &tidbase[tid],
430 RCVHQ_RCV_TYPE_EXPECTED,
431 dd->tidinvalid);
432 dma_unmap_page(&dd->pcidev->dev, phys,
433 PAGE_SIZE, DMA_FROM_DEVICE);
434 dd->pageshadow[ctxttid + tid] = NULL;
435 }
436 }
437 qib_release_user_pages(pagep, cnt);
438 } else {
439 /*
440 * Copy the updated array, with qib_tid's filled in, back
441 * to user. Since we did the copy in already, this "should
442 * never fail" If it does, we have to clean up...
443 */
444 if (copy_to_user((void __user *)
445 (unsigned long) ti->tidlist,
446 tidlist, cnt * sizeof(*tidlist))) {
447 ret = -EFAULT;
448 goto cleanup;
449 }
450 if (copy_to_user(u64_to_user_ptr(ti->tidmap),
451 tidmap, sizeof(tidmap))) {
452 ret = -EFAULT;
453 goto cleanup;
454 }
455 if (tid == tidcnt)
456 tid = 0;
457 if (!rcd->subctxt_cnt)
458 rcd->tidcursor = tid;
459 else
460 tidcursor_fp(fp) = tid;
461 }
462
463done:
464 return ret;
465}
466
467/**
468 * qib_tid_free - free a context TID
469 * @rcd: the context
470 * @subctxt: the subcontext
471 * @ti: the TID info
472 *
473 * right now we are unlocking one page at a time, but since
474 * the intended use of this routine is for a single group of
475 * virtually contiguous pages, that should change to improve
476 * performance. We check that the TID is in range for this context
477 * but otherwise don't check validity; if user has an error and
478 * frees the wrong tid, it's only their own data that can thereby
479 * be corrupted. We do check that the TID was in use, for sanity
480 * We always use our idea of the saved address, not the address that
481 * they pass in to us.
482 */
483static int qib_tid_free(struct qib_ctxtdata *rcd, unsigned subctxt,
484 const struct qib_tid_info *ti)
485{
486 int ret = 0;
487 u32 tid, ctxttid, limit, tidcnt;
488 struct qib_devdata *dd = rcd->dd;
489 u64 __iomem *tidbase;
490 unsigned long tidmap[8];
491
492 if (!dd->pageshadow) {
493 ret = -ENOMEM;
494 goto done;
495 }
496
497 if (copy_from_user(tidmap, u64_to_user_ptr(ti->tidmap),
498 sizeof(tidmap))) {
499 ret = -EFAULT;
500 goto done;
501 }
502
503 ctxttid = rcd->ctxt * dd->rcvtidcnt;
504 if (!rcd->subctxt_cnt)
505 tidcnt = dd->rcvtidcnt;
506 else if (!subctxt) {
507 tidcnt = (dd->rcvtidcnt / rcd->subctxt_cnt) +
508 (dd->rcvtidcnt % rcd->subctxt_cnt);
509 ctxttid += dd->rcvtidcnt - tidcnt;
510 } else {
511 tidcnt = dd->rcvtidcnt / rcd->subctxt_cnt;
512 ctxttid += tidcnt * (subctxt - 1);
513 }
514 tidbase = (u64 __iomem *) ((char __iomem *)(dd->kregbase) +
515 dd->rcvtidbase +
516 ctxttid * sizeof(*tidbase));
517
518 limit = sizeof(tidmap) * BITS_PER_BYTE;
519 if (limit > tidcnt)
520 /* just in case size changes in future */
521 limit = tidcnt;
522 tid = find_first_bit(tidmap, limit);
523 for (; tid < limit; tid++) {
524 /*
525 * small optimization; if we detect a run of 3 or so without
526 * any set, use find_first_bit again. That's mainly to
527 * accelerate the case where we wrapped, so we have some at
528 * the beginning, and some at the end, and a big gap
529 * in the middle.
530 */
531 if (!test_bit(tid, tidmap))
532 continue;
533
534 if (dd->pageshadow[ctxttid + tid]) {
535 struct page *p;
536 dma_addr_t phys;
537
538 p = dd->pageshadow[ctxttid + tid];
539 dd->pageshadow[ctxttid + tid] = NULL;
540 phys = dd->physshadow[ctxttid + tid];
541 dd->physshadow[ctxttid + tid] = dd->tidinvalid;
542 /* PERFORMANCE: below should almost certainly be
543 * cached
544 */
545 dd->f_put_tid(dd, &tidbase[tid],
546 RCVHQ_RCV_TYPE_EXPECTED, dd->tidinvalid);
547 dma_unmap_page(&dd->pcidev->dev, phys, PAGE_SIZE,
548 DMA_FROM_DEVICE);
549 qib_release_user_pages(&p, 1);
550 }
551 }
552done:
553 return ret;
554}
555
556/**
557 * qib_set_part_key - set a partition key
558 * @rcd: the context
559 * @key: the key
560 *
561 * We can have up to 4 active at a time (other than the default, which is
562 * always allowed). This is somewhat tricky, since multiple contexts may set
563 * the same key, so we reference count them, and clean up at exit. All 4
564 * partition keys are packed into a single qlogic_ib register. It's an
565 * error for a process to set the same pkey multiple times. We provide no
566 * mechanism to de-allocate a pkey at this time, we may eventually need to
567 * do that. I've used the atomic operations, and no locking, and only make
568 * a single pass through what's available. This should be more than
569 * adequate for some time. I'll think about spinlocks or the like if and as
570 * it's necessary.
571 */
572static int qib_set_part_key(struct qib_ctxtdata *rcd, u16 key)
573{
574 struct qib_pportdata *ppd = rcd->ppd;
575 int i, pidx = -1;
576 bool any = false;
577 u16 lkey = key & 0x7FFF;
578
579 if (lkey == (QIB_DEFAULT_P_KEY & 0x7FFF))
580 /* nothing to do; this key always valid */
581 return 0;
582
583 if (!lkey)
584 return -EINVAL;
585
586 /*
587 * Set the full membership bit, because it has to be
588 * set in the register or the packet, and it seems
589 * cleaner to set in the register than to force all
590 * callers to set it.
591 */
592 key |= 0x8000;
593
594 for (i = 0; i < ARRAY_SIZE(rcd->pkeys); i++) {
595 if (!rcd->pkeys[i] && pidx == -1)
596 pidx = i;
597 if (rcd->pkeys[i] == key)
598 return -EEXIST;
599 }
600 if (pidx == -1)
601 return -EBUSY;
602 for (i = 0; i < ARRAY_SIZE(ppd->pkeys); i++) {
603 if (!ppd->pkeys[i]) {
604 any = true;
605 continue;
606 }
607 if (ppd->pkeys[i] == key) {
608 atomic_t *pkrefs = &ppd->pkeyrefs[i];
609
610 if (atomic_inc_return(pkrefs) > 1) {
611 rcd->pkeys[pidx] = key;
612 return 0;
613 }
614 /*
615 * lost race, decrement count, catch below
616 */
617 atomic_dec(pkrefs);
618 any = true;
619 }
620 if ((ppd->pkeys[i] & 0x7FFF) == lkey)
621 /*
622 * It makes no sense to have both the limited and
623 * full membership PKEY set at the same time since
624 * the unlimited one will disable the limited one.
625 */
626 return -EEXIST;
627 }
628 if (!any)
629 return -EBUSY;
630 for (i = 0; i < ARRAY_SIZE(ppd->pkeys); i++) {
631 if (!ppd->pkeys[i] &&
632 atomic_inc_return(&ppd->pkeyrefs[i]) == 1) {
633 rcd->pkeys[pidx] = key;
634 ppd->pkeys[i] = key;
635 (void) ppd->dd->f_set_ib_cfg(ppd, QIB_IB_CFG_PKEYS, 0);
636 return 0;
637 }
638 }
639 return -EBUSY;
640}
641
642/**
643 * qib_manage_rcvq - manage a context's receive queue
644 * @rcd: the context
645 * @subctxt: the subcontext
646 * @start_stop: action to carry out
647 *
648 * start_stop == 0 disables receive on the context, for use in queue
649 * overflow conditions. start_stop==1 re-enables, to be used to
650 * re-init the software copy of the head register
651 */
652static int qib_manage_rcvq(struct qib_ctxtdata *rcd, unsigned subctxt,
653 int start_stop)
654{
655 struct qib_devdata *dd = rcd->dd;
656 unsigned int rcvctrl_op;
657
658 if (subctxt)
659 goto bail;
660 /* atomically clear receive enable ctxt. */
661 if (start_stop) {
662 /*
663 * On enable, force in-memory copy of the tail register to
664 * 0, so that protocol code doesn't have to worry about
665 * whether or not the chip has yet updated the in-memory
666 * copy or not on return from the system call. The chip
667 * always resets it's tail register back to 0 on a
668 * transition from disabled to enabled.
669 */
670 if (rcd->rcvhdrtail_kvaddr)
671 qib_clear_rcvhdrtail(rcd);
672 rcvctrl_op = QIB_RCVCTRL_CTXT_ENB;
673 } else
674 rcvctrl_op = QIB_RCVCTRL_CTXT_DIS;
675 dd->f_rcvctrl(rcd->ppd, rcvctrl_op, rcd->ctxt);
676 /* always; new head should be equal to new tail; see above */
677bail:
678 return 0;
679}
680
681static void qib_clean_part_key(struct qib_ctxtdata *rcd,
682 struct qib_devdata *dd)
683{
684 int i, j, pchanged = 0;
685 struct qib_pportdata *ppd = rcd->ppd;
686
687 for (i = 0; i < ARRAY_SIZE(rcd->pkeys); i++) {
688 if (!rcd->pkeys[i])
689 continue;
690 for (j = 0; j < ARRAY_SIZE(ppd->pkeys); j++) {
691 /* check for match independent of the global bit */
692 if ((ppd->pkeys[j] & 0x7fff) !=
693 (rcd->pkeys[i] & 0x7fff))
694 continue;
695 if (atomic_dec_and_test(&ppd->pkeyrefs[j])) {
696 ppd->pkeys[j] = 0;
697 pchanged++;
698 }
699 break;
700 }
701 rcd->pkeys[i] = 0;
702 }
703 if (pchanged)
704 (void) ppd->dd->f_set_ib_cfg(ppd, QIB_IB_CFG_PKEYS, 0);
705}
706
707/* common code for the mappings on dma_alloc_coherent mem */
708static int qib_mmap_mem(struct vm_area_struct *vma, struct qib_ctxtdata *rcd,
709 unsigned len, void *kvaddr, u32 write_ok, char *what)
710{
711 struct qib_devdata *dd = rcd->dd;
712 unsigned long pfn;
713 int ret;
714
715 if ((vma->vm_end - vma->vm_start) > len) {
716 qib_devinfo(dd->pcidev,
717 "FAIL on %s: len %lx > %x\n", what,
718 vma->vm_end - vma->vm_start, len);
719 ret = -EFAULT;
720 goto bail;
721 }
722
723 /*
724 * shared context user code requires rcvhdrq mapped r/w, others
725 * only allowed readonly mapping.
726 */
727 if (!write_ok) {
728 if (vma->vm_flags & VM_WRITE) {
729 qib_devinfo(dd->pcidev,
730 "%s must be mapped readonly\n", what);
731 ret = -EPERM;
732 goto bail;
733 }
734
735 /* don't allow them to later change with mprotect */
736 vm_flags_clear(vma, VM_MAYWRITE);
737 }
738
739 pfn = virt_to_phys(kvaddr) >> PAGE_SHIFT;
740 ret = remap_pfn_range(vma, vma->vm_start, pfn,
741 len, vma->vm_page_prot);
742 if (ret)
743 qib_devinfo(dd->pcidev,
744 "%s ctxt%u mmap of %lx, %x bytes failed: %d\n",
745 what, rcd->ctxt, pfn, len, ret);
746bail:
747 return ret;
748}
749
750static int mmap_ureg(struct vm_area_struct *vma, struct qib_devdata *dd,
751 u64 ureg)
752{
753 unsigned long phys;
754 unsigned long sz;
755 int ret;
756
757 /*
758 * This is real hardware, so use io_remap. This is the mechanism
759 * for the user process to update the head registers for their ctxt
760 * in the chip.
761 */
762 sz = dd->flags & QIB_HAS_HDRSUPP ? 2 * PAGE_SIZE : PAGE_SIZE;
763 if ((vma->vm_end - vma->vm_start) > sz) {
764 qib_devinfo(dd->pcidev,
765 "FAIL mmap userreg: reqlen %lx > PAGE\n",
766 vma->vm_end - vma->vm_start);
767 ret = -EFAULT;
768 } else {
769 phys = dd->physaddr + ureg;
770 vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
771
772 vm_flags_set(vma, VM_DONTCOPY | VM_DONTEXPAND);
773 ret = io_remap_pfn_range(vma, vma->vm_start,
774 phys >> PAGE_SHIFT,
775 vma->vm_end - vma->vm_start,
776 vma->vm_page_prot);
777 }
778 return ret;
779}
780
781static int mmap_piobufs(struct vm_area_struct *vma,
782 struct qib_devdata *dd,
783 struct qib_ctxtdata *rcd,
784 unsigned piobufs, unsigned piocnt)
785{
786 unsigned long phys;
787 int ret;
788
789 /*
790 * When we map the PIO buffers in the chip, we want to map them as
791 * writeonly, no read possible; unfortunately, x86 doesn't allow
792 * for this in hardware, but we still prevent users from asking
793 * for it.
794 */
795 if ((vma->vm_end - vma->vm_start) > (piocnt * dd->palign)) {
796 qib_devinfo(dd->pcidev,
797 "FAIL mmap piobufs: reqlen %lx > PAGE\n",
798 vma->vm_end - vma->vm_start);
799 ret = -EINVAL;
800 goto bail;
801 }
802
803 phys = dd->physaddr + piobufs;
804
805#if defined(__powerpc__)
806 vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
807#endif
808
809 /*
810 * don't allow them to later change to readable with mprotect (for when
811 * not initially mapped readable, as is normally the case)
812 */
813 vm_flags_mod(vma, VM_DONTCOPY | VM_DONTEXPAND, VM_MAYREAD);
814
815 /* We used PAT if wc_cookie == 0 */
816 if (!dd->wc_cookie)
817 vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
818
819 ret = io_remap_pfn_range(vma, vma->vm_start, phys >> PAGE_SHIFT,
820 vma->vm_end - vma->vm_start,
821 vma->vm_page_prot);
822bail:
823 return ret;
824}
825
826static int mmap_rcvegrbufs(struct vm_area_struct *vma,
827 struct qib_ctxtdata *rcd)
828{
829 struct qib_devdata *dd = rcd->dd;
830 unsigned long start, size;
831 size_t total_size, i;
832 unsigned long pfn;
833 int ret;
834
835 size = rcd->rcvegrbuf_size;
836 total_size = rcd->rcvegrbuf_chunks * size;
837 if ((vma->vm_end - vma->vm_start) > total_size) {
838 qib_devinfo(dd->pcidev,
839 "FAIL on egr bufs: reqlen %lx > actual %lx\n",
840 vma->vm_end - vma->vm_start,
841 (unsigned long) total_size);
842 ret = -EINVAL;
843 goto bail;
844 }
845
846 if (vma->vm_flags & VM_WRITE) {
847 qib_devinfo(dd->pcidev,
848 "Can't map eager buffers as writable (flags=%lx)\n",
849 vma->vm_flags);
850 ret = -EPERM;
851 goto bail;
852 }
853 /* don't allow them to later change to writable with mprotect */
854 vm_flags_clear(vma, VM_MAYWRITE);
855
856 start = vma->vm_start;
857
858 for (i = 0; i < rcd->rcvegrbuf_chunks; i++, start += size) {
859 pfn = virt_to_phys(rcd->rcvegrbuf[i]) >> PAGE_SHIFT;
860 ret = remap_pfn_range(vma, start, pfn, size,
861 vma->vm_page_prot);
862 if (ret < 0)
863 goto bail;
864 }
865 ret = 0;
866
867bail:
868 return ret;
869}
870
871/*
872 * qib_file_vma_fault - handle a VMA page fault.
873 */
874static vm_fault_t qib_file_vma_fault(struct vm_fault *vmf)
875{
876 struct page *page;
877
878 page = vmalloc_to_page((void *)(vmf->pgoff << PAGE_SHIFT));
879 if (!page)
880 return VM_FAULT_SIGBUS;
881
882 get_page(page);
883 vmf->page = page;
884
885 return 0;
886}
887
888static const struct vm_operations_struct qib_file_vm_ops = {
889 .fault = qib_file_vma_fault,
890};
891
892static int mmap_kvaddr(struct vm_area_struct *vma, u64 pgaddr,
893 struct qib_ctxtdata *rcd, unsigned subctxt)
894{
895 struct qib_devdata *dd = rcd->dd;
896 unsigned subctxt_cnt;
897 unsigned long len;
898 void *addr;
899 size_t size;
900 int ret = 0;
901
902 subctxt_cnt = rcd->subctxt_cnt;
903 size = rcd->rcvegrbuf_chunks * rcd->rcvegrbuf_size;
904
905 /*
906 * Each process has all the subctxt uregbase, rcvhdrq, and
907 * rcvegrbufs mmapped - as an array for all the processes,
908 * and also separately for this process.
909 */
910 if (pgaddr == cvt_kvaddr(rcd->subctxt_uregbase)) {
911 addr = rcd->subctxt_uregbase;
912 size = PAGE_SIZE * subctxt_cnt;
913 } else if (pgaddr == cvt_kvaddr(rcd->subctxt_rcvhdr_base)) {
914 addr = rcd->subctxt_rcvhdr_base;
915 size = rcd->rcvhdrq_size * subctxt_cnt;
916 } else if (pgaddr == cvt_kvaddr(rcd->subctxt_rcvegrbuf)) {
917 addr = rcd->subctxt_rcvegrbuf;
918 size *= subctxt_cnt;
919 } else if (pgaddr == cvt_kvaddr(rcd->subctxt_uregbase +
920 PAGE_SIZE * subctxt)) {
921 addr = rcd->subctxt_uregbase + PAGE_SIZE * subctxt;
922 size = PAGE_SIZE;
923 } else if (pgaddr == cvt_kvaddr(rcd->subctxt_rcvhdr_base +
924 rcd->rcvhdrq_size * subctxt)) {
925 addr = rcd->subctxt_rcvhdr_base +
926 rcd->rcvhdrq_size * subctxt;
927 size = rcd->rcvhdrq_size;
928 } else if (pgaddr == cvt_kvaddr(&rcd->user_event_mask[subctxt])) {
929 addr = rcd->user_event_mask;
930 size = PAGE_SIZE;
931 } else if (pgaddr == cvt_kvaddr(rcd->subctxt_rcvegrbuf +
932 size * subctxt)) {
933 addr = rcd->subctxt_rcvegrbuf + size * subctxt;
934 /* rcvegrbufs are read-only on the slave */
935 if (vma->vm_flags & VM_WRITE) {
936 qib_devinfo(dd->pcidev,
937 "Can't map eager buffers as writable (flags=%lx)\n",
938 vma->vm_flags);
939 ret = -EPERM;
940 goto bail;
941 }
942 /*
943 * Don't allow permission to later change to writable
944 * with mprotect.
945 */
946 vm_flags_clear(vma, VM_MAYWRITE);
947 } else
948 goto bail;
949 len = vma->vm_end - vma->vm_start;
950 if (len > size) {
951 ret = -EINVAL;
952 goto bail;
953 }
954
955 vma->vm_pgoff = (unsigned long) addr >> PAGE_SHIFT;
956 vma->vm_ops = &qib_file_vm_ops;
957 vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
958 ret = 1;
959
960bail:
961 return ret;
962}
963
964/**
965 * qib_mmapf - mmap various structures into user space
966 * @fp: the file pointer
967 * @vma: the VM area
968 *
969 * We use this to have a shared buffer between the kernel and the user code
970 * for the rcvhdr queue, egr buffers, and the per-context user regs and pio
971 * buffers in the chip. We have the open and close entries so we can bump
972 * the ref count and keep the driver from being unloaded while still mapped.
973 */
974static int qib_mmapf(struct file *fp, struct vm_area_struct *vma)
975{
976 struct qib_ctxtdata *rcd;
977 struct qib_devdata *dd;
978 u64 pgaddr, ureg;
979 unsigned piobufs, piocnt;
980 int ret, match = 1;
981
982 rcd = ctxt_fp(fp);
983 if (!rcd || !(vma->vm_flags & VM_SHARED)) {
984 ret = -EINVAL;
985 goto bail;
986 }
987 dd = rcd->dd;
988
989 /*
990 * This is the qib_do_user_init() code, mapping the shared buffers
991 * and per-context user registers into the user process. The address
992 * referred to by vm_pgoff is the file offset passed via mmap().
993 * For shared contexts, this is the kernel vmalloc() address of the
994 * pages to share with the master.
995 * For non-shared or master ctxts, this is a physical address.
996 * We only do one mmap for each space mapped.
997 */
998 pgaddr = vma->vm_pgoff << PAGE_SHIFT;
999
1000 /*
1001 * Check for 0 in case one of the allocations failed, but user
1002 * called mmap anyway.
1003 */
1004 if (!pgaddr) {
1005 ret = -EINVAL;
1006 goto bail;
1007 }
1008
1009 /*
1010 * Physical addresses must fit in 40 bits for our hardware.
1011 * Check for kernel virtual addresses first, anything else must
1012 * match a HW or memory address.
1013 */
1014 ret = mmap_kvaddr(vma, pgaddr, rcd, subctxt_fp(fp));
1015 if (ret) {
1016 if (ret > 0)
1017 ret = 0;
1018 goto bail;
1019 }
1020
1021 ureg = dd->uregbase + dd->ureg_align * rcd->ctxt;
1022 if (!rcd->subctxt_cnt) {
1023 /* ctxt is not shared */
1024 piocnt = rcd->piocnt;
1025 piobufs = rcd->piobufs;
1026 } else if (!subctxt_fp(fp)) {
1027 /* caller is the master */
1028 piocnt = (rcd->piocnt / rcd->subctxt_cnt) +
1029 (rcd->piocnt % rcd->subctxt_cnt);
1030 piobufs = rcd->piobufs +
1031 dd->palign * (rcd->piocnt - piocnt);
1032 } else {
1033 unsigned slave = subctxt_fp(fp) - 1;
1034
1035 /* caller is a slave */
1036 piocnt = rcd->piocnt / rcd->subctxt_cnt;
1037 piobufs = rcd->piobufs + dd->palign * piocnt * slave;
1038 }
1039
1040 if (pgaddr == ureg)
1041 ret = mmap_ureg(vma, dd, ureg);
1042 else if (pgaddr == piobufs)
1043 ret = mmap_piobufs(vma, dd, rcd, piobufs, piocnt);
1044 else if (pgaddr == dd->pioavailregs_phys)
1045 /* in-memory copy of pioavail registers */
1046 ret = qib_mmap_mem(vma, rcd, PAGE_SIZE,
1047 (void *) dd->pioavailregs_dma, 0,
1048 "pioavail registers");
1049 else if (pgaddr == rcd->rcvegr_phys)
1050 ret = mmap_rcvegrbufs(vma, rcd);
1051 else if (pgaddr == (u64) rcd->rcvhdrq_phys)
1052 /*
1053 * The rcvhdrq itself; multiple pages, contiguous
1054 * from an i/o perspective. Shared contexts need
1055 * to map r/w, so we allow writing.
1056 */
1057 ret = qib_mmap_mem(vma, rcd, rcd->rcvhdrq_size,
1058 rcd->rcvhdrq, 1, "rcvhdrq");
1059 else if (pgaddr == (u64) rcd->rcvhdrqtailaddr_phys)
1060 /* in-memory copy of rcvhdrq tail register */
1061 ret = qib_mmap_mem(vma, rcd, PAGE_SIZE,
1062 rcd->rcvhdrtail_kvaddr, 0,
1063 "rcvhdrq tail");
1064 else
1065 match = 0;
1066 if (!match)
1067 ret = -EINVAL;
1068
1069 vma->vm_private_data = NULL;
1070
1071 if (ret < 0)
1072 qib_devinfo(dd->pcidev,
1073 "mmap Failure %d: off %llx len %lx\n",
1074 -ret, (unsigned long long)pgaddr,
1075 vma->vm_end - vma->vm_start);
1076bail:
1077 return ret;
1078}
1079
1080static __poll_t qib_poll_urgent(struct qib_ctxtdata *rcd,
1081 struct file *fp,
1082 struct poll_table_struct *pt)
1083{
1084 struct qib_devdata *dd = rcd->dd;
1085 __poll_t pollflag;
1086
1087 poll_wait(fp, &rcd->wait, pt);
1088
1089 spin_lock_irq(&dd->uctxt_lock);
1090 if (rcd->urgent != rcd->urgent_poll) {
1091 pollflag = EPOLLIN | EPOLLRDNORM;
1092 rcd->urgent_poll = rcd->urgent;
1093 } else {
1094 pollflag = 0;
1095 set_bit(QIB_CTXT_WAITING_URG, &rcd->flag);
1096 }
1097 spin_unlock_irq(&dd->uctxt_lock);
1098
1099 return pollflag;
1100}
1101
1102static __poll_t qib_poll_next(struct qib_ctxtdata *rcd,
1103 struct file *fp,
1104 struct poll_table_struct *pt)
1105{
1106 struct qib_devdata *dd = rcd->dd;
1107 __poll_t pollflag;
1108
1109 poll_wait(fp, &rcd->wait, pt);
1110
1111 spin_lock_irq(&dd->uctxt_lock);
1112 if (dd->f_hdrqempty(rcd)) {
1113 set_bit(QIB_CTXT_WAITING_RCV, &rcd->flag);
1114 dd->f_rcvctrl(rcd->ppd, QIB_RCVCTRL_INTRAVAIL_ENB, rcd->ctxt);
1115 pollflag = 0;
1116 } else
1117 pollflag = EPOLLIN | EPOLLRDNORM;
1118 spin_unlock_irq(&dd->uctxt_lock);
1119
1120 return pollflag;
1121}
1122
1123static __poll_t qib_poll(struct file *fp, struct poll_table_struct *pt)
1124{
1125 struct qib_ctxtdata *rcd;
1126 __poll_t pollflag;
1127
1128 rcd = ctxt_fp(fp);
1129 if (!rcd)
1130 pollflag = EPOLLERR;
1131 else if (rcd->poll_type == QIB_POLL_TYPE_URGENT)
1132 pollflag = qib_poll_urgent(rcd, fp, pt);
1133 else if (rcd->poll_type == QIB_POLL_TYPE_ANYRCV)
1134 pollflag = qib_poll_next(rcd, fp, pt);
1135 else /* invalid */
1136 pollflag = EPOLLERR;
1137
1138 return pollflag;
1139}
1140
1141static void assign_ctxt_affinity(struct file *fp, struct qib_devdata *dd)
1142{
1143 struct qib_filedata *fd = fp->private_data;
1144 const unsigned int weight = current->nr_cpus_allowed;
1145 const struct cpumask *local_mask = cpumask_of_pcibus(dd->pcidev->bus);
1146 int local_cpu;
1147
1148 /*
1149 * If process has NOT already set it's affinity, select and
1150 * reserve a processor for it on the local NUMA node.
1151 */
1152 if ((weight >= qib_cpulist_count) &&
1153 (cpumask_weight(local_mask) <= qib_cpulist_count)) {
1154 for_each_cpu(local_cpu, local_mask)
1155 if (!test_and_set_bit(local_cpu, qib_cpulist)) {
1156 fd->rec_cpu_num = local_cpu;
1157 return;
1158 }
1159 }
1160
1161 /*
1162 * If process has NOT already set it's affinity, select and
1163 * reserve a processor for it, as a rendevous for all
1164 * users of the driver. If they don't actually later
1165 * set affinity to this cpu, or set it to some other cpu,
1166 * it just means that sooner or later we don't recommend
1167 * a cpu, and let the scheduler do it's best.
1168 */
1169 if (weight >= qib_cpulist_count) {
1170 int cpu;
1171
1172 cpu = find_first_zero_bit(qib_cpulist,
1173 qib_cpulist_count);
1174 if (cpu == qib_cpulist_count)
1175 qib_dev_err(dd,
1176 "no cpus avail for affinity PID %u\n",
1177 current->pid);
1178 else {
1179 __set_bit(cpu, qib_cpulist);
1180 fd->rec_cpu_num = cpu;
1181 }
1182 }
1183}
1184
1185/*
1186 * Check that userland and driver are compatible for subcontexts.
1187 */
1188static int qib_compatible_subctxts(int user_swmajor, int user_swminor)
1189{
1190 /* this code is written long-hand for clarity */
1191 if (QIB_USER_SWMAJOR != user_swmajor) {
1192 /* no promise of compatibility if major mismatch */
1193 return 0;
1194 }
1195 if (QIB_USER_SWMAJOR == 1) {
1196 switch (QIB_USER_SWMINOR) {
1197 case 0:
1198 case 1:
1199 case 2:
1200 /* no subctxt implementation so cannot be compatible */
1201 return 0;
1202 case 3:
1203 /* 3 is only compatible with itself */
1204 return user_swminor == 3;
1205 default:
1206 /* >= 4 are compatible (or are expected to be) */
1207 return user_swminor <= QIB_USER_SWMINOR;
1208 }
1209 }
1210 /* make no promises yet for future major versions */
1211 return 0;
1212}
1213
1214static int init_subctxts(struct qib_devdata *dd,
1215 struct qib_ctxtdata *rcd,
1216 const struct qib_user_info *uinfo)
1217{
1218 int ret = 0;
1219 unsigned num_subctxts;
1220 size_t size;
1221
1222 /*
1223 * If the user is requesting zero subctxts,
1224 * skip the subctxt allocation.
1225 */
1226 if (uinfo->spu_subctxt_cnt <= 0)
1227 goto bail;
1228 num_subctxts = uinfo->spu_subctxt_cnt;
1229
1230 /* Check for subctxt compatibility */
1231 if (!qib_compatible_subctxts(uinfo->spu_userversion >> 16,
1232 uinfo->spu_userversion & 0xffff)) {
1233 qib_devinfo(dd->pcidev,
1234 "Mismatched user version (%d.%d) and driver version (%d.%d) while context sharing. Ensure that driver and library are from the same release.\n",
1235 (int) (uinfo->spu_userversion >> 16),
1236 (int) (uinfo->spu_userversion & 0xffff),
1237 QIB_USER_SWMAJOR, QIB_USER_SWMINOR);
1238 goto bail;
1239 }
1240 if (num_subctxts > QLOGIC_IB_MAX_SUBCTXT) {
1241 ret = -EINVAL;
1242 goto bail;
1243 }
1244
1245 rcd->subctxt_uregbase = vmalloc_user(PAGE_SIZE * num_subctxts);
1246 if (!rcd->subctxt_uregbase) {
1247 ret = -ENOMEM;
1248 goto bail;
1249 }
1250 /* Note: rcd->rcvhdrq_size isn't initialized yet. */
1251 size = ALIGN(dd->rcvhdrcnt * dd->rcvhdrentsize *
1252 sizeof(u32), PAGE_SIZE) * num_subctxts;
1253 rcd->subctxt_rcvhdr_base = vmalloc_user(size);
1254 if (!rcd->subctxt_rcvhdr_base) {
1255 ret = -ENOMEM;
1256 goto bail_ureg;
1257 }
1258
1259 rcd->subctxt_rcvegrbuf = vmalloc_user(rcd->rcvegrbuf_chunks *
1260 rcd->rcvegrbuf_size *
1261 num_subctxts);
1262 if (!rcd->subctxt_rcvegrbuf) {
1263 ret = -ENOMEM;
1264 goto bail_rhdr;
1265 }
1266
1267 rcd->subctxt_cnt = uinfo->spu_subctxt_cnt;
1268 rcd->subctxt_id = uinfo->spu_subctxt_id;
1269 rcd->active_slaves = 1;
1270 rcd->redirect_seq_cnt = 1;
1271 set_bit(QIB_CTXT_MASTER_UNINIT, &rcd->flag);
1272 goto bail;
1273
1274bail_rhdr:
1275 vfree(rcd->subctxt_rcvhdr_base);
1276bail_ureg:
1277 vfree(rcd->subctxt_uregbase);
1278 rcd->subctxt_uregbase = NULL;
1279bail:
1280 return ret;
1281}
1282
1283static int setup_ctxt(struct qib_pportdata *ppd, int ctxt,
1284 struct file *fp, const struct qib_user_info *uinfo)
1285{
1286 struct qib_filedata *fd = fp->private_data;
1287 struct qib_devdata *dd = ppd->dd;
1288 struct qib_ctxtdata *rcd;
1289 void *ptmp = NULL;
1290 int ret;
1291 int numa_id;
1292
1293 assign_ctxt_affinity(fp, dd);
1294
1295 numa_id = qib_numa_aware ? ((fd->rec_cpu_num != -1) ?
1296 cpu_to_node(fd->rec_cpu_num) :
1297 numa_node_id()) : dd->assigned_node_id;
1298
1299 rcd = qib_create_ctxtdata(ppd, ctxt, numa_id);
1300
1301 /*
1302 * Allocate memory for use in qib_tid_update() at open to
1303 * reduce cost of expected send setup per message segment
1304 */
1305 if (rcd)
1306 ptmp = kmalloc(dd->rcvtidcnt * sizeof(u16) +
1307 dd->rcvtidcnt * sizeof(struct page **),
1308 GFP_KERNEL);
1309
1310 if (!rcd || !ptmp) {
1311 qib_dev_err(dd,
1312 "Unable to allocate ctxtdata memory, failing open\n");
1313 ret = -ENOMEM;
1314 goto bailerr;
1315 }
1316 rcd->userversion = uinfo->spu_userversion;
1317 ret = init_subctxts(dd, rcd, uinfo);
1318 if (ret)
1319 goto bailerr;
1320 rcd->tid_pg_list = ptmp;
1321 rcd->pid = current->pid;
1322 init_waitqueue_head(&dd->rcd[ctxt]->wait);
1323 get_task_comm(rcd->comm, current);
1324 ctxt_fp(fp) = rcd;
1325 qib_stats.sps_ctxts++;
1326 dd->freectxts--;
1327 ret = 0;
1328 goto bail;
1329
1330bailerr:
1331 if (fd->rec_cpu_num != -1)
1332 __clear_bit(fd->rec_cpu_num, qib_cpulist);
1333
1334 dd->rcd[ctxt] = NULL;
1335 kfree(rcd);
1336 kfree(ptmp);
1337bail:
1338 return ret;
1339}
1340
1341static inline int usable(struct qib_pportdata *ppd)
1342{
1343 struct qib_devdata *dd = ppd->dd;
1344
1345 return dd && (dd->flags & QIB_PRESENT) && dd->kregbase && ppd->lid &&
1346 (ppd->lflags & QIBL_LINKACTIVE);
1347}
1348
1349/*
1350 * Select a context on the given device, either using a requested port
1351 * or the port based on the context number.
1352 */
1353static int choose_port_ctxt(struct file *fp, struct qib_devdata *dd, u32 port,
1354 const struct qib_user_info *uinfo)
1355{
1356 struct qib_pportdata *ppd = NULL;
1357 int ret, ctxt;
1358
1359 if (port) {
1360 if (!usable(dd->pport + port - 1)) {
1361 ret = -ENETDOWN;
1362 goto done;
1363 } else
1364 ppd = dd->pport + port - 1;
1365 }
1366 for (ctxt = dd->first_user_ctxt; ctxt < dd->cfgctxts && dd->rcd[ctxt];
1367 ctxt++)
1368 ;
1369 if (ctxt == dd->cfgctxts) {
1370 ret = -EBUSY;
1371 goto done;
1372 }
1373 if (!ppd) {
1374 u32 pidx = ctxt % dd->num_pports;
1375
1376 if (usable(dd->pport + pidx))
1377 ppd = dd->pport + pidx;
1378 else {
1379 for (pidx = 0; pidx < dd->num_pports && !ppd;
1380 pidx++)
1381 if (usable(dd->pport + pidx))
1382 ppd = dd->pport + pidx;
1383 }
1384 }
1385 ret = ppd ? setup_ctxt(ppd, ctxt, fp, uinfo) : -ENETDOWN;
1386done:
1387 return ret;
1388}
1389
1390static int find_free_ctxt(int unit, struct file *fp,
1391 const struct qib_user_info *uinfo)
1392{
1393 struct qib_devdata *dd = qib_lookup(unit);
1394 int ret;
1395
1396 if (!dd || (uinfo->spu_port && uinfo->spu_port > dd->num_pports))
1397 ret = -ENODEV;
1398 else
1399 ret = choose_port_ctxt(fp, dd, uinfo->spu_port, uinfo);
1400
1401 return ret;
1402}
1403
1404static int get_a_ctxt(struct file *fp, const struct qib_user_info *uinfo,
1405 unsigned alg)
1406{
1407 struct qib_devdata *udd = NULL;
1408 int ret = 0, devmax, npresent, nup, ndev, dusable = 0, i;
1409 u32 port = uinfo->spu_port, ctxt;
1410
1411 devmax = qib_count_units(&npresent, &nup);
1412 if (!npresent) {
1413 ret = -ENXIO;
1414 goto done;
1415 }
1416 if (nup == 0) {
1417 ret = -ENETDOWN;
1418 goto done;
1419 }
1420
1421 if (alg == QIB_PORT_ALG_ACROSS) {
1422 unsigned inuse = ~0U;
1423
1424 /* find device (with ACTIVE ports) with fewest ctxts in use */
1425 for (ndev = 0; ndev < devmax; ndev++) {
1426 struct qib_devdata *dd = qib_lookup(ndev);
1427 unsigned cused = 0, cfree = 0, pusable = 0;
1428
1429 if (!dd)
1430 continue;
1431 if (port && port <= dd->num_pports &&
1432 usable(dd->pport + port - 1))
1433 pusable = 1;
1434 else
1435 for (i = 0; i < dd->num_pports; i++)
1436 if (usable(dd->pport + i))
1437 pusable++;
1438 if (!pusable)
1439 continue;
1440 for (ctxt = dd->first_user_ctxt; ctxt < dd->cfgctxts;
1441 ctxt++)
1442 if (dd->rcd[ctxt])
1443 cused++;
1444 else
1445 cfree++;
1446 if (cfree && cused < inuse) {
1447 udd = dd;
1448 inuse = cused;
1449 }
1450 }
1451 if (udd) {
1452 ret = choose_port_ctxt(fp, udd, port, uinfo);
1453 goto done;
1454 }
1455 } else {
1456 for (ndev = 0; ndev < devmax; ndev++) {
1457 struct qib_devdata *dd = qib_lookup(ndev);
1458
1459 if (dd) {
1460 ret = choose_port_ctxt(fp, dd, port, uinfo);
1461 if (!ret)
1462 goto done;
1463 if (ret == -EBUSY)
1464 dusable++;
1465 }
1466 }
1467 }
1468 ret = dusable ? -EBUSY : -ENETDOWN;
1469
1470done:
1471 return ret;
1472}
1473
1474static int find_shared_ctxt(struct file *fp,
1475 const struct qib_user_info *uinfo)
1476{
1477 int devmax, ndev, i;
1478 int ret = 0;
1479
1480 devmax = qib_count_units(NULL, NULL);
1481
1482 for (ndev = 0; ndev < devmax; ndev++) {
1483 struct qib_devdata *dd = qib_lookup(ndev);
1484
1485 /* device portion of usable() */
1486 if (!(dd && (dd->flags & QIB_PRESENT) && dd->kregbase))
1487 continue;
1488 for (i = dd->first_user_ctxt; i < dd->cfgctxts; i++) {
1489 struct qib_ctxtdata *rcd = dd->rcd[i];
1490
1491 /* Skip ctxts which are not yet open */
1492 if (!rcd || !rcd->cnt)
1493 continue;
1494 /* Skip ctxt if it doesn't match the requested one */
1495 if (rcd->subctxt_id != uinfo->spu_subctxt_id)
1496 continue;
1497 /* Verify the sharing process matches the master */
1498 if (rcd->subctxt_cnt != uinfo->spu_subctxt_cnt ||
1499 rcd->userversion != uinfo->spu_userversion ||
1500 rcd->cnt >= rcd->subctxt_cnt) {
1501 ret = -EINVAL;
1502 goto done;
1503 }
1504 ctxt_fp(fp) = rcd;
1505 subctxt_fp(fp) = rcd->cnt++;
1506 rcd->subpid[subctxt_fp(fp)] = current->pid;
1507 tidcursor_fp(fp) = 0;
1508 rcd->active_slaves |= 1 << subctxt_fp(fp);
1509 ret = 1;
1510 goto done;
1511 }
1512 }
1513
1514done:
1515 return ret;
1516}
1517
1518static int qib_open(struct inode *in, struct file *fp)
1519{
1520 /* The real work is performed later in qib_assign_ctxt() */
1521 fp->private_data = kzalloc(sizeof(struct qib_filedata), GFP_KERNEL);
1522 if (fp->private_data) /* no cpu affinity by default */
1523 ((struct qib_filedata *)fp->private_data)->rec_cpu_num = -1;
1524 return fp->private_data ? 0 : -ENOMEM;
1525}
1526
1527static int find_hca(unsigned int cpu, int *unit)
1528{
1529 int ret = 0, devmax, npresent, nup, ndev;
1530
1531 *unit = -1;
1532
1533 devmax = qib_count_units(&npresent, &nup);
1534 if (!npresent) {
1535 ret = -ENXIO;
1536 goto done;
1537 }
1538 if (!nup) {
1539 ret = -ENETDOWN;
1540 goto done;
1541 }
1542 for (ndev = 0; ndev < devmax; ndev++) {
1543 struct qib_devdata *dd = qib_lookup(ndev);
1544
1545 if (dd) {
1546 if (pcibus_to_node(dd->pcidev->bus) < 0) {
1547 ret = -EINVAL;
1548 goto done;
1549 }
1550 if (cpu_to_node(cpu) ==
1551 pcibus_to_node(dd->pcidev->bus)) {
1552 *unit = ndev;
1553 goto done;
1554 }
1555 }
1556 }
1557done:
1558 return ret;
1559}
1560
1561static int do_qib_user_sdma_queue_create(struct file *fp)
1562{
1563 struct qib_filedata *fd = fp->private_data;
1564 struct qib_ctxtdata *rcd = fd->rcd;
1565 struct qib_devdata *dd = rcd->dd;
1566
1567 if (dd->flags & QIB_HAS_SEND_DMA) {
1568
1569 fd->pq = qib_user_sdma_queue_create(&dd->pcidev->dev,
1570 dd->unit,
1571 rcd->ctxt,
1572 fd->subctxt);
1573 if (!fd->pq)
1574 return -ENOMEM;
1575 }
1576
1577 return 0;
1578}
1579
1580/*
1581 * Get ctxt early, so can set affinity prior to memory allocation.
1582 */
1583static int qib_assign_ctxt(struct file *fp, const struct qib_user_info *uinfo)
1584{
1585 int ret;
1586 int i_minor;
1587 unsigned swmajor, swminor, alg = QIB_PORT_ALG_ACROSS;
1588
1589 /* Check to be sure we haven't already initialized this file */
1590 if (ctxt_fp(fp)) {
1591 ret = -EINVAL;
1592 goto done;
1593 }
1594
1595 /* for now, if major version is different, bail */
1596 swmajor = uinfo->spu_userversion >> 16;
1597 if (swmajor != QIB_USER_SWMAJOR) {
1598 ret = -ENODEV;
1599 goto done;
1600 }
1601
1602 swminor = uinfo->spu_userversion & 0xffff;
1603
1604 if (swminor >= 11 && uinfo->spu_port_alg < QIB_PORT_ALG_COUNT)
1605 alg = uinfo->spu_port_alg;
1606
1607 mutex_lock(&qib_mutex);
1608
1609 if (qib_compatible_subctxts(swmajor, swminor) &&
1610 uinfo->spu_subctxt_cnt) {
1611 ret = find_shared_ctxt(fp, uinfo);
1612 if (ret > 0) {
1613 ret = do_qib_user_sdma_queue_create(fp);
1614 if (!ret)
1615 assign_ctxt_affinity(fp, (ctxt_fp(fp))->dd);
1616 goto done_ok;
1617 }
1618 }
1619
1620 i_minor = iminor(file_inode(fp)) - QIB_USER_MINOR_BASE;
1621 if (i_minor)
1622 ret = find_free_ctxt(i_minor - 1, fp, uinfo);
1623 else {
1624 int unit;
1625 const unsigned int cpu = cpumask_first(current->cpus_ptr);
1626 const unsigned int weight = current->nr_cpus_allowed;
1627
1628 if (weight == 1 && !test_bit(cpu, qib_cpulist))
1629 if (!find_hca(cpu, &unit) && unit >= 0)
1630 if (!find_free_ctxt(unit, fp, uinfo)) {
1631 ret = 0;
1632 goto done_chk_sdma;
1633 }
1634 ret = get_a_ctxt(fp, uinfo, alg);
1635 }
1636
1637done_chk_sdma:
1638 if (!ret)
1639 ret = do_qib_user_sdma_queue_create(fp);
1640done_ok:
1641 mutex_unlock(&qib_mutex);
1642
1643done:
1644 return ret;
1645}
1646
1647
1648static int qib_do_user_init(struct file *fp,
1649 const struct qib_user_info *uinfo)
1650{
1651 int ret;
1652 struct qib_ctxtdata *rcd = ctxt_fp(fp);
1653 struct qib_devdata *dd;
1654 unsigned uctxt;
1655
1656 /* Subctxts don't need to initialize anything since master did it. */
1657 if (subctxt_fp(fp)) {
1658 ret = wait_event_interruptible(rcd->wait,
1659 !test_bit(QIB_CTXT_MASTER_UNINIT, &rcd->flag));
1660 goto bail;
1661 }
1662
1663 dd = rcd->dd;
1664
1665 /* some ctxts may get extra buffers, calculate that here */
1666 uctxt = rcd->ctxt - dd->first_user_ctxt;
1667 if (uctxt < dd->ctxts_extrabuf) {
1668 rcd->piocnt = dd->pbufsctxt + 1;
1669 rcd->pio_base = rcd->piocnt * uctxt;
1670 } else {
1671 rcd->piocnt = dd->pbufsctxt;
1672 rcd->pio_base = rcd->piocnt * uctxt +
1673 dd->ctxts_extrabuf;
1674 }
1675
1676 /*
1677 * All user buffers are 2KB buffers. If we ever support
1678 * giving 4KB buffers to user processes, this will need some
1679 * work. Can't use piobufbase directly, because it has
1680 * both 2K and 4K buffer base values. So check and handle.
1681 */
1682 if ((rcd->pio_base + rcd->piocnt) > dd->piobcnt2k) {
1683 if (rcd->pio_base >= dd->piobcnt2k) {
1684 qib_dev_err(dd,
1685 "%u:ctxt%u: no 2KB buffers available\n",
1686 dd->unit, rcd->ctxt);
1687 ret = -ENOBUFS;
1688 goto bail;
1689 }
1690 rcd->piocnt = dd->piobcnt2k - rcd->pio_base;
1691 qib_dev_err(dd, "Ctxt%u: would use 4KB bufs, using %u\n",
1692 rcd->ctxt, rcd->piocnt);
1693 }
1694
1695 rcd->piobufs = dd->pio2k_bufbase + rcd->pio_base * dd->palign;
1696 qib_chg_pioavailkernel(dd, rcd->pio_base, rcd->piocnt,
1697 TXCHK_CHG_TYPE_USER, rcd);
1698 /*
1699 * try to ensure that processes start up with consistent avail update
1700 * for their own range, at least. If system very quiet, it might
1701 * have the in-memory copy out of date at startup for this range of
1702 * buffers, when a context gets re-used. Do after the chg_pioavail
1703 * and before the rest of setup, so it's "almost certain" the dma
1704 * will have occurred (can't 100% guarantee, but should be many
1705 * decimals of 9s, with this ordering), given how much else happens
1706 * after this.
1707 */
1708 dd->f_sendctrl(dd->pport, QIB_SENDCTRL_AVAIL_BLIP);
1709
1710 /*
1711 * Now allocate the rcvhdr Q and eager TIDs; skip the TID
1712 * array for time being. If rcd->ctxt > chip-supported,
1713 * we need to do extra stuff here to handle by handling overflow
1714 * through ctxt 0, someday
1715 */
1716 ret = qib_create_rcvhdrq(dd, rcd);
1717 if (!ret)
1718 ret = qib_setup_eagerbufs(rcd);
1719 if (ret)
1720 goto bail_pio;
1721
1722 rcd->tidcursor = 0; /* start at beginning after open */
1723
1724 /* initialize poll variables... */
1725 rcd->urgent = 0;
1726 rcd->urgent_poll = 0;
1727
1728 /*
1729 * Now enable the ctxt for receive.
1730 * For chips that are set to DMA the tail register to memory
1731 * when they change (and when the update bit transitions from
1732 * 0 to 1. So for those chips, we turn it off and then back on.
1733 * This will (very briefly) affect any other open ctxts, but the
1734 * duration is very short, and therefore isn't an issue. We
1735 * explicitly set the in-memory tail copy to 0 beforehand, so we
1736 * don't have to wait to be sure the DMA update has happened
1737 * (chip resets head/tail to 0 on transition to enable).
1738 */
1739 if (rcd->rcvhdrtail_kvaddr)
1740 qib_clear_rcvhdrtail(rcd);
1741
1742 dd->f_rcvctrl(rcd->ppd, QIB_RCVCTRL_CTXT_ENB | QIB_RCVCTRL_TIDFLOW_ENB,
1743 rcd->ctxt);
1744
1745 /* Notify any waiting slaves */
1746 if (rcd->subctxt_cnt) {
1747 clear_bit(QIB_CTXT_MASTER_UNINIT, &rcd->flag);
1748 wake_up(&rcd->wait);
1749 }
1750 return 0;
1751
1752bail_pio:
1753 qib_chg_pioavailkernel(dd, rcd->pio_base, rcd->piocnt,
1754 TXCHK_CHG_TYPE_KERN, rcd);
1755bail:
1756 return ret;
1757}
1758
1759/**
1760 * unlock_expected_tids - unlock any expected TID entries context still had
1761 * in use
1762 * @rcd: ctxt
1763 *
1764 * We don't actually update the chip here, because we do a bulk update
1765 * below, using f_clear_tids.
1766 */
1767static void unlock_expected_tids(struct qib_ctxtdata *rcd)
1768{
1769 struct qib_devdata *dd = rcd->dd;
1770 int ctxt_tidbase = rcd->ctxt * dd->rcvtidcnt;
1771 int i, maxtid = ctxt_tidbase + dd->rcvtidcnt;
1772
1773 for (i = ctxt_tidbase; i < maxtid; i++) {
1774 struct page *p = dd->pageshadow[i];
1775 dma_addr_t phys;
1776
1777 if (!p)
1778 continue;
1779
1780 phys = dd->physshadow[i];
1781 dd->physshadow[i] = dd->tidinvalid;
1782 dd->pageshadow[i] = NULL;
1783 dma_unmap_page(&dd->pcidev->dev, phys, PAGE_SIZE,
1784 DMA_FROM_DEVICE);
1785 qib_release_user_pages(&p, 1);
1786 }
1787}
1788
1789static int qib_close(struct inode *in, struct file *fp)
1790{
1791 struct qib_filedata *fd;
1792 struct qib_ctxtdata *rcd;
1793 struct qib_devdata *dd;
1794 unsigned long flags;
1795 unsigned ctxt;
1796
1797 mutex_lock(&qib_mutex);
1798
1799 fd = fp->private_data;
1800 fp->private_data = NULL;
1801 rcd = fd->rcd;
1802 if (!rcd) {
1803 mutex_unlock(&qib_mutex);
1804 goto bail;
1805 }
1806
1807 dd = rcd->dd;
1808
1809 /* ensure all pio buffer writes in progress are flushed */
1810 qib_flush_wc();
1811
1812 /* drain user sdma queue */
1813 if (fd->pq) {
1814 qib_user_sdma_queue_drain(rcd->ppd, fd->pq);
1815 qib_user_sdma_queue_destroy(fd->pq);
1816 }
1817
1818 if (fd->rec_cpu_num != -1)
1819 __clear_bit(fd->rec_cpu_num, qib_cpulist);
1820
1821 if (--rcd->cnt) {
1822 /*
1823 * XXX If the master closes the context before the slave(s),
1824 * revoke the mmap for the eager receive queue so
1825 * the slave(s) don't wait for receive data forever.
1826 */
1827 rcd->active_slaves &= ~(1 << fd->subctxt);
1828 rcd->subpid[fd->subctxt] = 0;
1829 mutex_unlock(&qib_mutex);
1830 goto bail;
1831 }
1832
1833 /* early; no interrupt users after this */
1834 spin_lock_irqsave(&dd->uctxt_lock, flags);
1835 ctxt = rcd->ctxt;
1836 dd->rcd[ctxt] = NULL;
1837 rcd->pid = 0;
1838 spin_unlock_irqrestore(&dd->uctxt_lock, flags);
1839
1840 if (rcd->rcvwait_to || rcd->piowait_to ||
1841 rcd->rcvnowait || rcd->pionowait) {
1842 rcd->rcvwait_to = 0;
1843 rcd->piowait_to = 0;
1844 rcd->rcvnowait = 0;
1845 rcd->pionowait = 0;
1846 }
1847 if (rcd->flag)
1848 rcd->flag = 0;
1849
1850 if (dd->kregbase) {
1851 /* atomically clear receive enable ctxt and intr avail. */
1852 dd->f_rcvctrl(rcd->ppd, QIB_RCVCTRL_CTXT_DIS |
1853 QIB_RCVCTRL_INTRAVAIL_DIS, ctxt);
1854
1855 /* clean up the pkeys for this ctxt user */
1856 qib_clean_part_key(rcd, dd);
1857 qib_disarm_piobufs(dd, rcd->pio_base, rcd->piocnt);
1858 qib_chg_pioavailkernel(dd, rcd->pio_base,
1859 rcd->piocnt, TXCHK_CHG_TYPE_KERN, NULL);
1860
1861 dd->f_clear_tids(dd, rcd);
1862
1863 if (dd->pageshadow)
1864 unlock_expected_tids(rcd);
1865 qib_stats.sps_ctxts--;
1866 dd->freectxts++;
1867 }
1868
1869 mutex_unlock(&qib_mutex);
1870 qib_free_ctxtdata(dd, rcd); /* after releasing the mutex */
1871
1872bail:
1873 kfree(fd);
1874 return 0;
1875}
1876
1877static int qib_ctxt_info(struct file *fp, struct qib_ctxt_info __user *uinfo)
1878{
1879 struct qib_ctxt_info info;
1880 int ret;
1881 size_t sz;
1882 struct qib_ctxtdata *rcd = ctxt_fp(fp);
1883 struct qib_filedata *fd;
1884
1885 fd = fp->private_data;
1886
1887 info.num_active = qib_count_active_units();
1888 info.unit = rcd->dd->unit;
1889 info.port = rcd->ppd->port;
1890 info.ctxt = rcd->ctxt;
1891 info.subctxt = subctxt_fp(fp);
1892 /* Number of user ctxts available for this device. */
1893 info.num_ctxts = rcd->dd->cfgctxts - rcd->dd->first_user_ctxt;
1894 info.num_subctxts = rcd->subctxt_cnt;
1895 info.rec_cpu = fd->rec_cpu_num;
1896 sz = sizeof(info);
1897
1898 if (copy_to_user(uinfo, &info, sz)) {
1899 ret = -EFAULT;
1900 goto bail;
1901 }
1902 ret = 0;
1903
1904bail:
1905 return ret;
1906}
1907
1908static int qib_sdma_get_inflight(struct qib_user_sdma_queue *pq,
1909 u32 __user *inflightp)
1910{
1911 const u32 val = qib_user_sdma_inflight_counter(pq);
1912
1913 if (put_user(val, inflightp))
1914 return -EFAULT;
1915
1916 return 0;
1917}
1918
1919static int qib_sdma_get_complete(struct qib_pportdata *ppd,
1920 struct qib_user_sdma_queue *pq,
1921 u32 __user *completep)
1922{
1923 u32 val;
1924 int err;
1925
1926 if (!pq)
1927 return -EINVAL;
1928
1929 err = qib_user_sdma_make_progress(ppd, pq);
1930 if (err < 0)
1931 return err;
1932
1933 val = qib_user_sdma_complete_counter(pq);
1934 if (put_user(val, completep))
1935 return -EFAULT;
1936
1937 return 0;
1938}
1939
1940static int disarm_req_delay(struct qib_ctxtdata *rcd)
1941{
1942 int ret = 0;
1943
1944 if (!usable(rcd->ppd)) {
1945 int i;
1946 /*
1947 * if link is down, or otherwise not usable, delay
1948 * the caller up to 30 seconds, so we don't thrash
1949 * in trying to get the chip back to ACTIVE, and
1950 * set flag so they make the call again.
1951 */
1952 if (rcd->user_event_mask) {
1953 /*
1954 * subctxt_cnt is 0 if not shared, so do base
1955 * separately, first, then remaining subctxt, if any
1956 */
1957 set_bit(_QIB_EVENT_DISARM_BUFS_BIT,
1958 &rcd->user_event_mask[0]);
1959 for (i = 1; i < rcd->subctxt_cnt; i++)
1960 set_bit(_QIB_EVENT_DISARM_BUFS_BIT,
1961 &rcd->user_event_mask[i]);
1962 }
1963 for (i = 0; !usable(rcd->ppd) && i < 300; i++)
1964 msleep(100);
1965 ret = -ENETDOWN;
1966 }
1967 return ret;
1968}
1969
1970/*
1971 * Find all user contexts in use, and set the specified bit in their
1972 * event mask.
1973 * See also find_ctxt() for a similar use, that is specific to send buffers.
1974 */
1975int qib_set_uevent_bits(struct qib_pportdata *ppd, const int evtbit)
1976{
1977 struct qib_ctxtdata *rcd;
1978 unsigned ctxt;
1979 int ret = 0;
1980 unsigned long flags;
1981
1982 spin_lock_irqsave(&ppd->dd->uctxt_lock, flags);
1983 for (ctxt = ppd->dd->first_user_ctxt; ctxt < ppd->dd->cfgctxts;
1984 ctxt++) {
1985 rcd = ppd->dd->rcd[ctxt];
1986 if (!rcd)
1987 continue;
1988 if (rcd->user_event_mask) {
1989 int i;
1990 /*
1991 * subctxt_cnt is 0 if not shared, so do base
1992 * separately, first, then remaining subctxt, if any
1993 */
1994 set_bit(evtbit, &rcd->user_event_mask[0]);
1995 for (i = 1; i < rcd->subctxt_cnt; i++)
1996 set_bit(evtbit, &rcd->user_event_mask[i]);
1997 }
1998 ret = 1;
1999 break;
2000 }
2001 spin_unlock_irqrestore(&ppd->dd->uctxt_lock, flags);
2002
2003 return ret;
2004}
2005
2006/*
2007 * clear the event notifier events for this context.
2008 * For the DISARM_BUFS case, we also take action (this obsoletes
2009 * the older QIB_CMD_DISARM_BUFS, but we keep it for backwards
2010 * compatibility.
2011 * Other bits don't currently require actions, just atomically clear.
2012 * User process then performs actions appropriate to bit having been
2013 * set, if desired, and checks again in future.
2014 */
2015static int qib_user_event_ack(struct qib_ctxtdata *rcd, int subctxt,
2016 unsigned long events)
2017{
2018 int ret = 0, i;
2019
2020 for (i = 0; i <= _QIB_MAX_EVENT_BIT; i++) {
2021 if (!test_bit(i, &events))
2022 continue;
2023 if (i == _QIB_EVENT_DISARM_BUFS_BIT) {
2024 (void)qib_disarm_piobufs_ifneeded(rcd);
2025 ret = disarm_req_delay(rcd);
2026 } else
2027 clear_bit(i, &rcd->user_event_mask[subctxt]);
2028 }
2029 return ret;
2030}
2031
2032static ssize_t qib_write(struct file *fp, const char __user *data,
2033 size_t count, loff_t *off)
2034{
2035 const struct qib_cmd __user *ucmd;
2036 struct qib_ctxtdata *rcd;
2037 const void __user *src;
2038 size_t consumed, copy = 0;
2039 struct qib_cmd cmd;
2040 ssize_t ret = 0;
2041 void *dest;
2042
2043 if (!ib_safe_file_access(fp)) {
2044 pr_err_once("qib_write: process %d (%s) changed security contexts after opening file descriptor, this is not allowed.\n",
2045 task_tgid_vnr(current), current->comm);
2046 return -EACCES;
2047 }
2048
2049 if (count < sizeof(cmd.type)) {
2050 ret = -EINVAL;
2051 goto bail;
2052 }
2053
2054 ucmd = (const struct qib_cmd __user *) data;
2055
2056 if (copy_from_user(&cmd.type, &ucmd->type, sizeof(cmd.type))) {
2057 ret = -EFAULT;
2058 goto bail;
2059 }
2060
2061 consumed = sizeof(cmd.type);
2062
2063 switch (cmd.type) {
2064 case QIB_CMD_ASSIGN_CTXT:
2065 case QIB_CMD_USER_INIT:
2066 copy = sizeof(cmd.cmd.user_info);
2067 dest = &cmd.cmd.user_info;
2068 src = &ucmd->cmd.user_info;
2069 break;
2070
2071 case QIB_CMD_RECV_CTRL:
2072 copy = sizeof(cmd.cmd.recv_ctrl);
2073 dest = &cmd.cmd.recv_ctrl;
2074 src = &ucmd->cmd.recv_ctrl;
2075 break;
2076
2077 case QIB_CMD_CTXT_INFO:
2078 copy = sizeof(cmd.cmd.ctxt_info);
2079 dest = &cmd.cmd.ctxt_info;
2080 src = &ucmd->cmd.ctxt_info;
2081 break;
2082
2083 case QIB_CMD_TID_UPDATE:
2084 case QIB_CMD_TID_FREE:
2085 copy = sizeof(cmd.cmd.tid_info);
2086 dest = &cmd.cmd.tid_info;
2087 src = &ucmd->cmd.tid_info;
2088 break;
2089
2090 case QIB_CMD_SET_PART_KEY:
2091 copy = sizeof(cmd.cmd.part_key);
2092 dest = &cmd.cmd.part_key;
2093 src = &ucmd->cmd.part_key;
2094 break;
2095
2096 case QIB_CMD_DISARM_BUFS:
2097 case QIB_CMD_PIOAVAILUPD: /* force an update of PIOAvail reg */
2098 copy = 0;
2099 src = NULL;
2100 dest = NULL;
2101 break;
2102
2103 case QIB_CMD_POLL_TYPE:
2104 copy = sizeof(cmd.cmd.poll_type);
2105 dest = &cmd.cmd.poll_type;
2106 src = &ucmd->cmd.poll_type;
2107 break;
2108
2109 case QIB_CMD_ARMLAUNCH_CTRL:
2110 copy = sizeof(cmd.cmd.armlaunch_ctrl);
2111 dest = &cmd.cmd.armlaunch_ctrl;
2112 src = &ucmd->cmd.armlaunch_ctrl;
2113 break;
2114
2115 case QIB_CMD_SDMA_INFLIGHT:
2116 copy = sizeof(cmd.cmd.sdma_inflight);
2117 dest = &cmd.cmd.sdma_inflight;
2118 src = &ucmd->cmd.sdma_inflight;
2119 break;
2120
2121 case QIB_CMD_SDMA_COMPLETE:
2122 copy = sizeof(cmd.cmd.sdma_complete);
2123 dest = &cmd.cmd.sdma_complete;
2124 src = &ucmd->cmd.sdma_complete;
2125 break;
2126
2127 case QIB_CMD_ACK_EVENT:
2128 copy = sizeof(cmd.cmd.event_mask);
2129 dest = &cmd.cmd.event_mask;
2130 src = &ucmd->cmd.event_mask;
2131 break;
2132
2133 default:
2134 ret = -EINVAL;
2135 goto bail;
2136 }
2137
2138 if (copy) {
2139 if ((count - consumed) < copy) {
2140 ret = -EINVAL;
2141 goto bail;
2142 }
2143 if (copy_from_user(dest, src, copy)) {
2144 ret = -EFAULT;
2145 goto bail;
2146 }
2147 consumed += copy;
2148 }
2149
2150 rcd = ctxt_fp(fp);
2151 if (!rcd && cmd.type != QIB_CMD_ASSIGN_CTXT) {
2152 ret = -EINVAL;
2153 goto bail;
2154 }
2155
2156 switch (cmd.type) {
2157 case QIB_CMD_ASSIGN_CTXT:
2158 if (rcd) {
2159 ret = -EINVAL;
2160 goto bail;
2161 }
2162
2163 ret = qib_assign_ctxt(fp, &cmd.cmd.user_info);
2164 if (ret)
2165 goto bail;
2166 break;
2167
2168 case QIB_CMD_USER_INIT:
2169 ret = qib_do_user_init(fp, &cmd.cmd.user_info);
2170 if (ret)
2171 goto bail;
2172 ret = qib_get_base_info(fp, u64_to_user_ptr(
2173 cmd.cmd.user_info.spu_base_info),
2174 cmd.cmd.user_info.spu_base_info_size);
2175 break;
2176
2177 case QIB_CMD_RECV_CTRL:
2178 ret = qib_manage_rcvq(rcd, subctxt_fp(fp), cmd.cmd.recv_ctrl);
2179 break;
2180
2181 case QIB_CMD_CTXT_INFO:
2182 ret = qib_ctxt_info(fp, (struct qib_ctxt_info __user *)
2183 (unsigned long) cmd.cmd.ctxt_info);
2184 break;
2185
2186 case QIB_CMD_TID_UPDATE:
2187 ret = qib_tid_update(rcd, fp, &cmd.cmd.tid_info);
2188 break;
2189
2190 case QIB_CMD_TID_FREE:
2191 ret = qib_tid_free(rcd, subctxt_fp(fp), &cmd.cmd.tid_info);
2192 break;
2193
2194 case QIB_CMD_SET_PART_KEY:
2195 ret = qib_set_part_key(rcd, cmd.cmd.part_key);
2196 break;
2197
2198 case QIB_CMD_DISARM_BUFS:
2199 (void)qib_disarm_piobufs_ifneeded(rcd);
2200 ret = disarm_req_delay(rcd);
2201 break;
2202
2203 case QIB_CMD_PIOAVAILUPD:
2204 qib_force_pio_avail_update(rcd->dd);
2205 break;
2206
2207 case QIB_CMD_POLL_TYPE:
2208 rcd->poll_type = cmd.cmd.poll_type;
2209 break;
2210
2211 case QIB_CMD_ARMLAUNCH_CTRL:
2212 rcd->dd->f_set_armlaunch(rcd->dd, cmd.cmd.armlaunch_ctrl);
2213 break;
2214
2215 case QIB_CMD_SDMA_INFLIGHT:
2216 ret = qib_sdma_get_inflight(user_sdma_queue_fp(fp),
2217 (u32 __user *) (unsigned long)
2218 cmd.cmd.sdma_inflight);
2219 break;
2220
2221 case QIB_CMD_SDMA_COMPLETE:
2222 ret = qib_sdma_get_complete(rcd->ppd,
2223 user_sdma_queue_fp(fp),
2224 (u32 __user *) (unsigned long)
2225 cmd.cmd.sdma_complete);
2226 break;
2227
2228 case QIB_CMD_ACK_EVENT:
2229 ret = qib_user_event_ack(rcd, subctxt_fp(fp),
2230 cmd.cmd.event_mask);
2231 break;
2232 }
2233
2234 if (ret >= 0)
2235 ret = consumed;
2236
2237bail:
2238 return ret;
2239}
2240
2241static ssize_t qib_write_iter(struct kiocb *iocb, struct iov_iter *from)
2242{
2243 struct qib_filedata *fp = iocb->ki_filp->private_data;
2244 struct qib_ctxtdata *rcd = ctxt_fp(iocb->ki_filp);
2245 struct qib_user_sdma_queue *pq = fp->pq;
2246
2247 if (!user_backed_iter(from) || !from->nr_segs || !pq)
2248 return -EINVAL;
2249
2250 return qib_user_sdma_writev(rcd, pq, iter_iov(from), from->nr_segs);
2251}
2252
2253static const struct class qib_class = {
2254 .name = "ipath",
2255};
2256static dev_t qib_dev;
2257
2258int qib_cdev_init(int minor, const char *name,
2259 const struct file_operations *fops,
2260 struct cdev **cdevp, struct device **devp)
2261{
2262 const dev_t dev = MKDEV(MAJOR(qib_dev), minor);
2263 struct cdev *cdev;
2264 struct device *device = NULL;
2265 int ret;
2266
2267 cdev = cdev_alloc();
2268 if (!cdev) {
2269 pr_err("Could not allocate cdev for minor %d, %s\n",
2270 minor, name);
2271 ret = -ENOMEM;
2272 goto done;
2273 }
2274
2275 cdev->owner = THIS_MODULE;
2276 cdev->ops = fops;
2277 kobject_set_name(&cdev->kobj, name);
2278
2279 ret = cdev_add(cdev, dev, 1);
2280 if (ret < 0) {
2281 pr_err("Could not add cdev for minor %d, %s (err %d)\n",
2282 minor, name, -ret);
2283 goto err_cdev;
2284 }
2285
2286 device = device_create(&qib_class, NULL, dev, NULL, "%s", name);
2287 if (!IS_ERR(device))
2288 goto done;
2289 ret = PTR_ERR(device);
2290 device = NULL;
2291 pr_err("Could not create device for minor %d, %s (err %d)\n",
2292 minor, name, -ret);
2293err_cdev:
2294 cdev_del(cdev);
2295 cdev = NULL;
2296done:
2297 *cdevp = cdev;
2298 *devp = device;
2299 return ret;
2300}
2301
2302void qib_cdev_cleanup(struct cdev **cdevp, struct device **devp)
2303{
2304 struct device *device = *devp;
2305
2306 if (device) {
2307 device_unregister(device);
2308 *devp = NULL;
2309 }
2310
2311 if (*cdevp) {
2312 cdev_del(*cdevp);
2313 *cdevp = NULL;
2314 }
2315}
2316
2317static struct cdev *wildcard_cdev;
2318static struct device *wildcard_device;
2319
2320int __init qib_dev_init(void)
2321{
2322 int ret;
2323
2324 ret = alloc_chrdev_region(&qib_dev, 0, QIB_NMINORS, QIB_DRV_NAME);
2325 if (ret < 0) {
2326 pr_err("Could not allocate chrdev region (err %d)\n", -ret);
2327 goto done;
2328 }
2329
2330 ret = class_register(&qib_class);
2331 if (ret) {
2332 pr_err("Could not create device class (err %d)\n", -ret);
2333 unregister_chrdev_region(qib_dev, QIB_NMINORS);
2334 }
2335
2336done:
2337 return ret;
2338}
2339
2340void qib_dev_cleanup(void)
2341{
2342 if (class_is_registered(&qib_class))
2343 class_unregister(&qib_class);
2344
2345 unregister_chrdev_region(qib_dev, QIB_NMINORS);
2346}
2347
2348static atomic_t user_count = ATOMIC_INIT(0);
2349
2350static void qib_user_remove(struct qib_devdata *dd)
2351{
2352 if (atomic_dec_return(&user_count) == 0)
2353 qib_cdev_cleanup(&wildcard_cdev, &wildcard_device);
2354
2355 qib_cdev_cleanup(&dd->user_cdev, &dd->user_device);
2356}
2357
2358static int qib_user_add(struct qib_devdata *dd)
2359{
2360 char name[10];
2361 int ret;
2362
2363 if (atomic_inc_return(&user_count) == 1) {
2364 ret = qib_cdev_init(0, "ipath", &qib_file_ops,
2365 &wildcard_cdev, &wildcard_device);
2366 if (ret)
2367 goto done;
2368 }
2369
2370 snprintf(name, sizeof(name), "ipath%d", dd->unit);
2371 ret = qib_cdev_init(dd->unit + 1, name, &qib_file_ops,
2372 &dd->user_cdev, &dd->user_device);
2373 if (ret)
2374 qib_user_remove(dd);
2375done:
2376 return ret;
2377}
2378
2379/*
2380 * Create per-unit files in /dev
2381 */
2382int qib_device_create(struct qib_devdata *dd)
2383{
2384 int r, ret;
2385
2386 r = qib_user_add(dd);
2387 ret = qib_diag_add(dd);
2388 if (r && !ret)
2389 ret = r;
2390 return ret;
2391}
2392
2393/*
2394 * Remove per-unit files in /dev
2395 * void, core kernel returns no errors for this stuff
2396 */
2397void qib_device_remove(struct qib_devdata *dd)
2398{
2399 qib_user_remove(dd);
2400 qib_diag_remove(dd);
2401}
drivers/infiniband/hw/qib/qib_fs.c+0−549
@@ -1,549 +0,0 @@
1/*
2 * Copyright (c) 2012 Intel Corporation. All rights reserved.
3 * Copyright (c) 2006 - 2012 QLogic Corporation. All rights reserved.
4 * Copyright (c) 2006 PathScale, Inc. All rights reserved.
5 *
6 * This software is available to you under a choice of one of two
7 * licenses. You may choose to be licensed under the terms of the GNU
8 * General Public License (GPL) Version 2, available from the file
9 * COPYING in the main directory of this source tree, or the
10 * OpenIB.org BSD license below:
11 *
12 * Redistribution and use in source and binary forms, with or
13 * without modification, are permitted provided that the following
14 * conditions are met:
15 *
16 * - Redistributions of source code must retain the above
17 * copyright notice, this list of conditions and the following
18 * disclaimer.
19 *
20 * - Redistributions in binary form must reproduce the above
21 * copyright notice, this list of conditions and the following
22 * disclaimer in the documentation and/or other materials
23 * provided with the distribution.
24 *
25 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
26 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
27 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
28 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
29 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
30 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
31 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
32 * SOFTWARE.
33 */
34
35#include <linux/fs.h>
36#include <linux/fs_context.h>
37#include <linux/mount.h>
38#include <linux/pagemap.h>
39#include <linux/init.h>
40#include <linux/namei.h>
41
42#include "qib.h"
43
44#define QIBFS_MAGIC 0x726a77
45
46static struct super_block *qib_super;
47
48#define private2dd(file) (file_inode(file)->i_private)
49
50static int qibfs_mknod(struct inode *dir, struct dentry *dentry,
51 umode_t mode, const struct file_operations *fops,
52 void *data)
53{
54 int error;
55 struct inode *inode = new_inode(dir->i_sb);
56
57 if (!inode) {
58 dput(dentry);
59 error = -EPERM;
60 goto bail;
61 }
62
63 inode->i_ino = get_next_ino();
64 inode->i_mode = mode;
65 inode->i_uid = GLOBAL_ROOT_UID;
66 inode->i_gid = GLOBAL_ROOT_GID;
67 inode->i_blocks = 0;
68 simple_inode_init_ts(inode);
69
70 inode->i_private = data;
71 if (S_ISDIR(mode)) {
72 inode->i_op = &simple_dir_inode_operations;
73 inc_nlink(inode);
74 inc_nlink(dir);
75 }
76
77 inode->i_fop = fops;
78
79 d_instantiate(dentry, inode);
80 error = 0;
81
82bail:
83 return error;
84}
85
86static int create_file(const char *name, umode_t mode,
87 struct dentry *parent, struct dentry **dentry,
88 const struct file_operations *fops, void *data)
89{
90 int error;
91
92 inode_lock(d_inode(parent));
93 *dentry = lookup_noperm(&QSTR(name), parent);
94 if (!IS_ERR(*dentry))
95 error = qibfs_mknod(d_inode(parent), *dentry,
96 mode, fops, data);
97 else
98 error = PTR_ERR(*dentry);
99 inode_unlock(d_inode(parent));
100
101 return error;
102}
103
104static ssize_t driver_stats_read(struct file *file, char __user *buf,
105 size_t count, loff_t *ppos)
106{
107 qib_stats.sps_ints = qib_sps_ints();
108 return simple_read_from_buffer(buf, count, ppos, &qib_stats,
109 sizeof(qib_stats));
110}
111
112/*
113 * driver stats field names, one line per stat, single string. Used by
114 * programs like ipathstats to print the stats in a way which works for
115 * different versions of drivers, without changing program source.
116 * if qlogic_ib_stats changes, this needs to change. Names need to be
117 * 12 chars or less (w/o newline), for proper display by ipathstats utility.
118 */
119static const char qib_statnames[] =
120 "KernIntr\n"
121 "ErrorIntr\n"
122 "Tx_Errs\n"
123 "Rcv_Errs\n"
124 "H/W_Errs\n"
125 "NoPIOBufs\n"
126 "CtxtsOpen\n"
127 "RcvLen_Errs\n"
128 "EgrBufFull\n"
129 "EgrHdrFull\n"
130 ;
131
132static ssize_t driver_names_read(struct file *file, char __user *buf,
133 size_t count, loff_t *ppos)
134{
135 return simple_read_from_buffer(buf, count, ppos, qib_statnames,
136 sizeof(qib_statnames) - 1); /* no null */
137}
138
139static const struct file_operations driver_ops[] = {
140 { .read = driver_stats_read, .llseek = generic_file_llseek, },
141 { .read = driver_names_read, .llseek = generic_file_llseek, },
142};
143
144/* read the per-device counters */
145static ssize_t dev_counters_read(struct file *file, char __user *buf,
146 size_t count, loff_t *ppos)
147{
148 u64 *counters;
149 size_t avail;
150 struct qib_devdata *dd = private2dd(file);
151
152 avail = dd->f_read_cntrs(dd, *ppos, NULL, &counters);
153 return simple_read_from_buffer(buf, count, ppos, counters, avail);
154}
155
156/* read the per-device counters */
157static ssize_t dev_names_read(struct file *file, char __user *buf,
158 size_t count, loff_t *ppos)
159{
160 char *names;
161 size_t avail;
162 struct qib_devdata *dd = private2dd(file);
163
164 avail = dd->f_read_cntrs(dd, *ppos, &names, NULL);
165 return simple_read_from_buffer(buf, count, ppos, names, avail);
166}
167
168static const struct file_operations cntr_ops[] = {
169 { .read = dev_counters_read, .llseek = generic_file_llseek, },
170 { .read = dev_names_read, .llseek = generic_file_llseek, },
171};
172
173/*
174 * Could use file_inode(file)->i_ino to figure out which file,
175 * instead of separate routine for each, but for now, this works...
176 */
177
178/* read the per-port names (same for each port) */
179static ssize_t portnames_read(struct file *file, char __user *buf,
180 size_t count, loff_t *ppos)
181{
182 char *names;
183 size_t avail;
184 struct qib_devdata *dd = private2dd(file);
185
186 avail = dd->f_read_portcntrs(dd, *ppos, 0, &names, NULL);
187 return simple_read_from_buffer(buf, count, ppos, names, avail);
188}
189
190/* read the per-port counters for port 1 (pidx 0) */
191static ssize_t portcntrs_1_read(struct file *file, char __user *buf,
192 size_t count, loff_t *ppos)
193{
194 u64 *counters;
195 size_t avail;
196 struct qib_devdata *dd = private2dd(file);
197
198 avail = dd->f_read_portcntrs(dd, *ppos, 0, NULL, &counters);
199 return simple_read_from_buffer(buf, count, ppos, counters, avail);
200}
201
202/* read the per-port counters for port 2 (pidx 1) */
203static ssize_t portcntrs_2_read(struct file *file, char __user *buf,
204 size_t count, loff_t *ppos)
205{
206 u64 *counters;
207 size_t avail;
208 struct qib_devdata *dd = private2dd(file);
209
210 avail = dd->f_read_portcntrs(dd, *ppos, 1, NULL, &counters);
211 return simple_read_from_buffer(buf, count, ppos, counters, avail);
212}
213
214static const struct file_operations portcntr_ops[] = {
215 { .read = portnames_read, .llseek = generic_file_llseek, },
216 { .read = portcntrs_1_read, .llseek = generic_file_llseek, },
217 { .read = portcntrs_2_read, .llseek = generic_file_llseek, },
218};
219
220/*
221 * read the per-port QSFP data for port 1 (pidx 0)
222 */
223static ssize_t qsfp_1_read(struct file *file, char __user *buf,
224 size_t count, loff_t *ppos)
225{
226 struct qib_devdata *dd = private2dd(file);
227 char *tmp;
228 int ret;
229
230 tmp = kmalloc(PAGE_SIZE, GFP_KERNEL);
231 if (!tmp)
232 return -ENOMEM;
233
234 ret = qib_qsfp_dump(dd->pport, tmp, PAGE_SIZE);
235 if (ret > 0)
236 ret = simple_read_from_buffer(buf, count, ppos, tmp, ret);
237 kfree(tmp);
238 return ret;
239}
240
241/*
242 * read the per-port QSFP data for port 2 (pidx 1)
243 */
244static ssize_t qsfp_2_read(struct file *file, char __user *buf,
245 size_t count, loff_t *ppos)
246{
247 struct qib_devdata *dd = private2dd(file);
248 char *tmp;
249 int ret;
250
251 if (dd->num_pports < 2)
252 return -ENODEV;
253
254 tmp = kmalloc(PAGE_SIZE, GFP_KERNEL);
255 if (!tmp)
256 return -ENOMEM;
257
258 ret = qib_qsfp_dump(dd->pport + 1, tmp, PAGE_SIZE);
259 if (ret > 0)
260 ret = simple_read_from_buffer(buf, count, ppos, tmp, ret);
261 kfree(tmp);
262 return ret;
263}
264
265static const struct file_operations qsfp_ops[] = {
266 { .read = qsfp_1_read, .llseek = generic_file_llseek, },
267 { .read = qsfp_2_read, .llseek = generic_file_llseek, },
268};
269
270static ssize_t flash_read(struct file *file, char __user *buf,
271 size_t count, loff_t *ppos)
272{
273 struct qib_devdata *dd;
274 ssize_t ret;
275 loff_t pos;
276 char *tmp;
277
278 pos = *ppos;
279
280 if (pos < 0) {
281 ret = -EINVAL;
282 goto bail;
283 }
284
285 if (pos >= sizeof(struct qib_flash)) {
286 ret = 0;
287 goto bail;
288 }
289
290 if (count > sizeof(struct qib_flash) - pos)
291 count = sizeof(struct qib_flash) - pos;
292
293 tmp = kmalloc(count, GFP_KERNEL);
294 if (!tmp) {
295 ret = -ENOMEM;
296 goto bail;
297 }
298
299 dd = private2dd(file);
300 if (qib_eeprom_read(dd, pos, tmp, count)) {
301 qib_dev_err(dd, "failed to read from flash\n");
302 ret = -ENXIO;
303 goto bail_tmp;
304 }
305
306 if (copy_to_user(buf, tmp, count)) {
307 ret = -EFAULT;
308 goto bail_tmp;
309 }
310
311 *ppos = pos + count;
312 ret = count;
313
314bail_tmp:
315 kfree(tmp);
316
317bail:
318 return ret;
319}
320
321static ssize_t flash_write(struct file *file, const char __user *buf,
322 size_t count, loff_t *ppos)
323{
324 struct qib_devdata *dd;
325 ssize_t ret;
326 loff_t pos;
327 char *tmp;
328
329 pos = *ppos;
330
331 if (pos != 0 || count != sizeof(struct qib_flash))
332 return -EINVAL;
333
334 tmp = memdup_user(buf, count);
335 if (IS_ERR(tmp))
336 return PTR_ERR(tmp);
337
338 dd = private2dd(file);
339 if (qib_eeprom_write(dd, pos, tmp, count)) {
340 ret = -ENXIO;
341 qib_dev_err(dd, "failed to write to flash\n");
342 goto bail_tmp;
343 }
344
345 *ppos = pos + count;
346 ret = count;
347
348bail_tmp:
349 kfree(tmp);
350 return ret;
351}
352
353static const struct file_operations flash_ops = {
354 .read = flash_read,
355 .write = flash_write,
356 .llseek = default_llseek,
357};
358
359static int add_cntr_files(struct super_block *sb, struct qib_devdata *dd)
360{
361 struct dentry *dir, *tmp;
362 char unit[10];
363 int ret, i;
364
365 /* create the per-unit directory */
366 snprintf(unit, sizeof(unit), "%u", dd->unit);
367 ret = create_file(unit, S_IFDIR|S_IRUGO|S_IXUGO, sb->s_root, &dir,
368 &simple_dir_operations, dd);
369 if (ret) {
370 pr_err("create_file(%s) failed: %d\n", unit, ret);
371 goto bail;
372 }
373
374 /* create the files in the new directory */
375 ret = create_file("counters", S_IFREG|S_IRUGO, dir, &tmp,
376 &cntr_ops[0], dd);
377 if (ret) {
378 pr_err("create_file(%s/counters) failed: %d\n",
379 unit, ret);
380 goto bail;
381 }
382 ret = create_file("counter_names", S_IFREG|S_IRUGO, dir, &tmp,
383 &cntr_ops[1], dd);
384 if (ret) {
385 pr_err("create_file(%s/counter_names) failed: %d\n",
386 unit, ret);
387 goto bail;
388 }
389 ret = create_file("portcounter_names", S_IFREG|S_IRUGO, dir, &tmp,
390 &portcntr_ops[0], dd);
391 if (ret) {
392 pr_err("create_file(%s/%s) failed: %d\n",
393 unit, "portcounter_names", ret);
394 goto bail;
395 }
396 for (i = 1; i <= dd->num_pports; i++) {
397 char fname[24];
398
399 sprintf(fname, "port%dcounters", i);
400 /* create the files in the new directory */
401 ret = create_file(fname, S_IFREG|S_IRUGO, dir, &tmp,
402 &portcntr_ops[i], dd);
403 if (ret) {
404 pr_err("create_file(%s/%s) failed: %d\n",
405 unit, fname, ret);
406 goto bail;
407 }
408 if (!(dd->flags & QIB_HAS_QSFP))
409 continue;
410 sprintf(fname, "qsfp%d", i);
411 ret = create_file(fname, S_IFREG|S_IRUGO, dir, &tmp,
412 &qsfp_ops[i - 1], dd);
413 if (ret) {
414 pr_err("create_file(%s/%s) failed: %d\n",
415 unit, fname, ret);
416 goto bail;
417 }
418 }
419
420 ret = create_file("flash", S_IFREG|S_IWUSR|S_IRUGO, dir, &tmp,
421 &flash_ops, dd);
422 if (ret)
423 pr_err("create_file(%s/flash) failed: %d\n",
424 unit, ret);
425bail:
426 return ret;
427}
428
429static int remove_device_files(struct super_block *sb,
430 struct qib_devdata *dd)
431{
432 struct dentry *dir;
433 char unit[10];
434
435 snprintf(unit, sizeof(unit), "%u", dd->unit);
436 dir = lookup_noperm_unlocked(&QSTR(unit), sb->s_root);
437
438 if (IS_ERR(dir)) {
439 pr_err("Lookup of %s failed\n", unit);
440 return PTR_ERR(dir);
441 }
442 simple_recursive_removal(dir, NULL);
443 dput(dir);
444 return 0;
445}
446
447/*
448 * This fills everything in when the fs is mounted, to handle umount/mount
449 * after device init. The direct add_cntr_files() call handles adding
450 * them from the init code, when the fs is already mounted.
451 */
452static int qibfs_fill_super(struct super_block *sb, struct fs_context *fc)
453{
454 struct qib_devdata *dd;
455 unsigned long index;
456 int ret;
457
458 static const struct tree_descr files[] = {
459 [2] = {"driver_stats", &driver_ops[0], S_IRUGO},
460 [3] = {"driver_stats_names", &driver_ops[1], S_IRUGO},
461 {""},
462 };
463
464 ret = simple_fill_super(sb, QIBFS_MAGIC, files);
465 if (ret) {
466 pr_err("simple_fill_super failed: %d\n", ret);
467 goto bail;
468 }
469
470 xa_for_each(&qib_dev_table, index, dd) {
471 ret = add_cntr_files(sb, dd);
472 if (ret)
473 goto bail;
474 }
475
476bail:
477 return ret;
478}
479
480static int qibfs_get_tree(struct fs_context *fc)
481{
482 int ret = get_tree_single(fc, qibfs_fill_super);
483 if (ret == 0)
484 qib_super = fc->root->d_sb;
485 return ret;
486}
487
488static const struct fs_context_operations qibfs_context_ops = {
489 .get_tree = qibfs_get_tree,
490};
491
492static int qibfs_init_fs_context(struct fs_context *fc)
493{
494 fc->ops = &qibfs_context_ops;
495 return 0;
496}
497
498static void qibfs_kill_super(struct super_block *s)
499{
500 kill_litter_super(s);
501 qib_super = NULL;
502}
503
504int qibfs_add(struct qib_devdata *dd)
505{
506 int ret;
507
508 /*
509 * On first unit initialized, qib_super will not yet exist
510 * because nobody has yet tried to mount the filesystem, so
511 * we can't consider that to be an error; if an error occurs
512 * during the mount, that will get a complaint, so this is OK.
513 * add_cntr_files() for all units is done at mount from
514 * qibfs_fill_super(), so one way or another, everything works.
515 */
516 if (qib_super == NULL)
517 ret = 0;
518 else
519 ret = add_cntr_files(qib_super, dd);
520 return ret;
521}
522
523int qibfs_remove(struct qib_devdata *dd)
524{
525 int ret = 0;
526
527 if (qib_super)
528 ret = remove_device_files(qib_super, dd);
529
530 return ret;
531}
532
533static struct file_system_type qibfs_fs_type = {
534 .owner = THIS_MODULE,
535 .name = "ipathfs",
536 .init_fs_context = qibfs_init_fs_context,
537 .kill_sb = qibfs_kill_super,
538};
539MODULE_ALIAS_FS("ipathfs");
540
541int __init qib_init_qibfs(void)
542{
543 return register_filesystem(&qibfs_fs_type);
544}
545
546int __exit qib_exit_qibfs(void)
547{
548 return unregister_filesystem(&qibfs_fs_type);
549}
drivers/infiniband/hw/qib/qib_iba6120.c+0−3533
@@ -1,3533 +0,0 @@
1/*
2 * Copyright (c) 2013 - 2017 Intel Corporation. All rights reserved.
3 * Copyright (c) 2006, 2007, 2008, 2009, 2010 QLogic Corporation.
4 * All rights reserved.
5 * Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved.
6 *
7 * This software is available to you under a choice of one of two
8 * licenses. You may choose to be licensed under the terms of the GNU
9 * General Public License (GPL) Version 2, available from the file
10 * COPYING in the main directory of this source tree, or the
11 * OpenIB.org BSD license below:
12 *
13 * Redistribution and use in source and binary forms, with or
14 * without modification, are permitted provided that the following
15 * conditions are met:
16 *
17 * - Redistributions of source code must retain the above
18 * copyright notice, this list of conditions and the following
19 * disclaimer.
20 *
21 * - Redistributions in binary form must reproduce the above
22 * copyright notice, this list of conditions and the following
23 * disclaimer in the documentation and/or other materials
24 * provided with the distribution.
25 *
26 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
27 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
28 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
29 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
30 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
31 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
32 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
33 * SOFTWARE.
34 */
35/*
36 * This file contains all of the code that is specific to the
37 * QLogic_IB 6120 PCIe chip.
38 */
39
40#include <linux/interrupt.h>
41#include <linux/pci.h>
42#include <linux/delay.h>
43#include <rdma/ib_verbs.h>
44
45#include "qib.h"
46#include "qib_6120_regs.h"
47
48static void qib_6120_setup_setextled(struct qib_pportdata *, u32);
49static void sendctrl_6120_mod(struct qib_pportdata *ppd, u32 op);
50static u8 qib_6120_phys_portstate(u64);
51static u32 qib_6120_iblink_state(u64);
52
53/*
54 * This file contains all the chip-specific register information and
55 * access functions for the Intel Intel_IB PCI-Express chip.
56 *
57 */
58
59/* KREG_IDX uses machine-generated #defines */
60#define KREG_IDX(regname) (QIB_6120_##regname##_OFFS / sizeof(u64))
61
62/* Use defines to tie machine-generated names to lower-case names */
63#define kr_extctrl KREG_IDX(EXTCtrl)
64#define kr_extstatus KREG_IDX(EXTStatus)
65#define kr_gpio_clear KREG_IDX(GPIOClear)
66#define kr_gpio_mask KREG_IDX(GPIOMask)
67#define kr_gpio_out KREG_IDX(GPIOOut)
68#define kr_gpio_status KREG_IDX(GPIOStatus)
69#define kr_rcvctrl KREG_IDX(RcvCtrl)
70#define kr_sendctrl KREG_IDX(SendCtrl)
71#define kr_partitionkey KREG_IDX(RcvPartitionKey)
72#define kr_hwdiagctrl KREG_IDX(HwDiagCtrl)
73#define kr_ibcstatus KREG_IDX(IBCStatus)
74#define kr_ibcctrl KREG_IDX(IBCCtrl)
75#define kr_sendbuffererror KREG_IDX(SendBufErr0)
76#define kr_rcvbthqp KREG_IDX(RcvBTHQP)
77#define kr_counterregbase KREG_IDX(CntrRegBase)
78#define kr_palign KREG_IDX(PageAlign)
79#define kr_rcvegrbase KREG_IDX(RcvEgrBase)
80#define kr_rcvegrcnt KREG_IDX(RcvEgrCnt)
81#define kr_rcvhdrcnt KREG_IDX(RcvHdrCnt)
82#define kr_rcvhdrentsize KREG_IDX(RcvHdrEntSize)
83#define kr_rcvhdrsize KREG_IDX(RcvHdrSize)
84#define kr_rcvtidbase KREG_IDX(RcvTIDBase)
85#define kr_rcvtidcnt KREG_IDX(RcvTIDCnt)
86#define kr_scratch KREG_IDX(Scratch)
87#define kr_sendctrl KREG_IDX(SendCtrl)
88#define kr_sendpioavailaddr KREG_IDX(SendPIOAvailAddr)
89#define kr_sendpiobufbase KREG_IDX(SendPIOBufBase)
90#define kr_sendpiobufcnt KREG_IDX(SendPIOBufCnt)
91#define kr_sendpiosize KREG_IDX(SendPIOSize)
92#define kr_sendregbase KREG_IDX(SendRegBase)
93#define kr_userregbase KREG_IDX(UserRegBase)
94#define kr_control KREG_IDX(Control)
95#define kr_intclear KREG_IDX(IntClear)
96#define kr_intmask KREG_IDX(IntMask)
97#define kr_intstatus KREG_IDX(IntStatus)
98#define kr_errclear KREG_IDX(ErrClear)
99#define kr_errmask KREG_IDX(ErrMask)
100#define kr_errstatus KREG_IDX(ErrStatus)
101#define kr_hwerrclear KREG_IDX(HwErrClear)
102#define kr_hwerrmask KREG_IDX(HwErrMask)
103#define kr_hwerrstatus KREG_IDX(HwErrStatus)
104#define kr_revision KREG_IDX(Revision)
105#define kr_portcnt KREG_IDX(PortCnt)
106#define kr_serdes_cfg0 KREG_IDX(SerdesCfg0)
107#define kr_serdes_cfg1 (kr_serdes_cfg0 + 1)
108#define kr_serdes_stat KREG_IDX(SerdesStat)
109#define kr_xgxs_cfg KREG_IDX(XGXSCfg)
110
111/* These must only be written via qib_write_kreg_ctxt() */
112#define kr_rcvhdraddr KREG_IDX(RcvHdrAddr0)
113#define kr_rcvhdrtailaddr KREG_IDX(RcvHdrTailAddr0)
114
115#define CREG_IDX(regname) ((QIB_6120_##regname##_OFFS - \
116 QIB_6120_LBIntCnt_OFFS) / sizeof(u64))
117
118#define cr_badformat CREG_IDX(RxBadFormatCnt)
119#define cr_erricrc CREG_IDX(RxICRCErrCnt)
120#define cr_errlink CREG_IDX(RxLinkProblemCnt)
121#define cr_errlpcrc CREG_IDX(RxLPCRCErrCnt)
122#define cr_errpkey CREG_IDX(RxPKeyMismatchCnt)
123#define cr_rcvflowctrl_err CREG_IDX(RxFlowCtrlErrCnt)
124#define cr_err_rlen CREG_IDX(RxLenErrCnt)
125#define cr_errslen CREG_IDX(TxLenErrCnt)
126#define cr_errtidfull CREG_IDX(RxTIDFullErrCnt)
127#define cr_errtidvalid CREG_IDX(RxTIDValidErrCnt)
128#define cr_errvcrc CREG_IDX(RxVCRCErrCnt)
129#define cr_ibstatuschange CREG_IDX(IBStatusChangeCnt)
130#define cr_lbint CREG_IDX(LBIntCnt)
131#define cr_invalidrlen CREG_IDX(RxMaxMinLenErrCnt)
132#define cr_invalidslen CREG_IDX(TxMaxMinLenErrCnt)
133#define cr_lbflowstall CREG_IDX(LBFlowStallCnt)
134#define cr_pktrcv CREG_IDX(RxDataPktCnt)
135#define cr_pktrcvflowctrl CREG_IDX(RxFlowPktCnt)
136#define cr_pktsend CREG_IDX(TxDataPktCnt)
137#define cr_pktsendflow CREG_IDX(TxFlowPktCnt)
138#define cr_portovfl CREG_IDX(RxP0HdrEgrOvflCnt)
139#define cr_rcvebp CREG_IDX(RxEBPCnt)
140#define cr_rcvovfl CREG_IDX(RxBufOvflCnt)
141#define cr_senddropped CREG_IDX(TxDroppedPktCnt)
142#define cr_sendstall CREG_IDX(TxFlowStallCnt)
143#define cr_sendunderrun CREG_IDX(TxUnderrunCnt)
144#define cr_wordrcv CREG_IDX(RxDwordCnt)
145#define cr_wordsend CREG_IDX(TxDwordCnt)
146#define cr_txunsupvl CREG_IDX(TxUnsupVLErrCnt)
147#define cr_rxdroppkt CREG_IDX(RxDroppedPktCnt)
148#define cr_iblinkerrrecov CREG_IDX(IBLinkErrRecoveryCnt)
149#define cr_iblinkdown CREG_IDX(IBLinkDownedCnt)
150#define cr_ibsymbolerr CREG_IDX(IBSymbolErrCnt)
151
152#define SYM_RMASK(regname, fldname) ((u64) \
153 QIB_6120_##regname##_##fldname##_RMASK)
154#define SYM_MASK(regname, fldname) ((u64) \
155 QIB_6120_##regname##_##fldname##_RMASK << \
156 QIB_6120_##regname##_##fldname##_LSB)
157#define SYM_LSB(regname, fldname) (QIB_6120_##regname##_##fldname##_LSB)
158
159#define SYM_FIELD(value, regname, fldname) ((u64) \
160 (((value) >> SYM_LSB(regname, fldname)) & \
161 SYM_RMASK(regname, fldname)))
162#define ERR_MASK(fldname) SYM_MASK(ErrMask, fldname##Mask)
163#define HWE_MASK(fldname) SYM_MASK(HwErrMask, fldname##Mask)
164
165/* link training states, from IBC */
166#define IB_6120_LT_STATE_DISABLED 0x00
167#define IB_6120_LT_STATE_LINKUP 0x01
168#define IB_6120_LT_STATE_POLLACTIVE 0x02
169#define IB_6120_LT_STATE_POLLQUIET 0x03
170#define IB_6120_LT_STATE_SLEEPDELAY 0x04
171#define IB_6120_LT_STATE_SLEEPQUIET 0x05
172#define IB_6120_LT_STATE_CFGDEBOUNCE 0x08
173#define IB_6120_LT_STATE_CFGRCVFCFG 0x09
174#define IB_6120_LT_STATE_CFGWAITRMT 0x0a
175#define IB_6120_LT_STATE_CFGIDLE 0x0b
176#define IB_6120_LT_STATE_RECOVERRETRAIN 0x0c
177#define IB_6120_LT_STATE_RECOVERWAITRMT 0x0e
178#define IB_6120_LT_STATE_RECOVERIDLE 0x0f
179
180/* link state machine states from IBC */
181#define IB_6120_L_STATE_DOWN 0x0
182#define IB_6120_L_STATE_INIT 0x1
183#define IB_6120_L_STATE_ARM 0x2
184#define IB_6120_L_STATE_ACTIVE 0x3
185#define IB_6120_L_STATE_ACT_DEFER 0x4
186
187static const u8 qib_6120_physportstate[0x20] = {
188 [IB_6120_LT_STATE_DISABLED] = IB_PHYSPORTSTATE_DISABLED,
189 [IB_6120_LT_STATE_LINKUP] = IB_PHYSPORTSTATE_LINKUP,
190 [IB_6120_LT_STATE_POLLACTIVE] = IB_PHYSPORTSTATE_POLL,
191 [IB_6120_LT_STATE_POLLQUIET] = IB_PHYSPORTSTATE_POLL,
192 [IB_6120_LT_STATE_SLEEPDELAY] = IB_PHYSPORTSTATE_SLEEP,
193 [IB_6120_LT_STATE_SLEEPQUIET] = IB_PHYSPORTSTATE_SLEEP,
194 [IB_6120_LT_STATE_CFGDEBOUNCE] =
195 IB_PHYSPORTSTATE_CFG_TRAIN,
196 [IB_6120_LT_STATE_CFGRCVFCFG] =
197 IB_PHYSPORTSTATE_CFG_TRAIN,
198 [IB_6120_LT_STATE_CFGWAITRMT] =
199 IB_PHYSPORTSTATE_CFG_TRAIN,
200 [IB_6120_LT_STATE_CFGIDLE] = IB_PHYSPORTSTATE_CFG_TRAIN,
201 [IB_6120_LT_STATE_RECOVERRETRAIN] =
202 IB_PHYSPORTSTATE_LINK_ERR_RECOVER,
203 [IB_6120_LT_STATE_RECOVERWAITRMT] =
204 IB_PHYSPORTSTATE_LINK_ERR_RECOVER,
205 [IB_6120_LT_STATE_RECOVERIDLE] =
206 IB_PHYSPORTSTATE_LINK_ERR_RECOVER,
207 [0x10] = IB_PHYSPORTSTATE_CFG_TRAIN,
208 [0x11] = IB_PHYSPORTSTATE_CFG_TRAIN,
209 [0x12] = IB_PHYSPORTSTATE_CFG_TRAIN,
210 [0x13] = IB_PHYSPORTSTATE_CFG_TRAIN,
211 [0x14] = IB_PHYSPORTSTATE_CFG_TRAIN,
212 [0x15] = IB_PHYSPORTSTATE_CFG_TRAIN,
213 [0x16] = IB_PHYSPORTSTATE_CFG_TRAIN,
214 [0x17] = IB_PHYSPORTSTATE_CFG_TRAIN
215};
216
217
218struct qib_chip_specific {
219 u64 __iomem *cregbase;
220 u64 *cntrs;
221 u64 *portcntrs;
222 void *dummy_hdrq; /* used after ctxt close */
223 dma_addr_t dummy_hdrq_phys;
224 spinlock_t kernel_tid_lock; /* no back to back kernel TID writes */
225 spinlock_t user_tid_lock; /* no back to back user TID writes */
226 spinlock_t rcvmod_lock; /* protect rcvctrl shadow changes */
227 spinlock_t gpio_lock; /* RMW of shadows/regs for ExtCtrl and GPIO */
228 u64 hwerrmask;
229 u64 errormask;
230 u64 gpio_out; /* shadow of kr_gpio_out, for rmw ops */
231 u64 gpio_mask; /* shadow the gpio mask register */
232 u64 extctrl; /* shadow the gpio output enable, etc... */
233 /*
234 * these 5 fields are used to establish deltas for IB symbol
235 * errors and linkrecovery errors. They can be reported on
236 * some chips during link negotiation prior to INIT, and with
237 * DDR when faking DDR negotiations with non-IBTA switches.
238 * The chip counters are adjusted at driver unload if there is
239 * a non-zero delta.
240 */
241 u64 ibdeltainprog;
242 u64 ibsymdelta;
243 u64 ibsymsnap;
244 u64 iblnkerrdelta;
245 u64 iblnkerrsnap;
246 u64 ibcctrl; /* shadow for kr_ibcctrl */
247 u32 lastlinkrecov; /* link recovery issue */
248 u32 cntrnamelen;
249 u32 portcntrnamelen;
250 u32 ncntrs;
251 u32 nportcntrs;
252 /* used with gpio interrupts to implement IB counters */
253 u32 rxfc_unsupvl_errs;
254 u32 overrun_thresh_errs;
255 /*
256 * these count only cases where _successive_ LocalLinkIntegrity
257 * errors were seen in the receive headers of IB standard packets
258 */
259 u32 lli_errs;
260 u32 lli_counter;
261 u64 lli_thresh;
262 u64 sword; /* total dwords sent (sample result) */
263 u64 rword; /* total dwords received (sample result) */
264 u64 spkts; /* total packets sent (sample result) */
265 u64 rpkts; /* total packets received (sample result) */
266 u64 xmit_wait; /* # of ticks no data sent (sample result) */
267 struct timer_list pma_timer;
268 struct qib_pportdata *ppd;
269 char emsgbuf[128];
270 char bitsmsgbuf[64];
271 u8 pma_sample_status;
272};
273
274/* ibcctrl bits */
275#define QLOGIC_IB_IBCC_LINKINITCMD_DISABLE 1
276/* cycle through TS1/TS2 till OK */
277#define QLOGIC_IB_IBCC_LINKINITCMD_POLL 2
278/* wait for TS1, then go on */
279#define QLOGIC_IB_IBCC_LINKINITCMD_SLEEP 3
280#define QLOGIC_IB_IBCC_LINKINITCMD_SHIFT 16
281
282#define QLOGIC_IB_IBCC_LINKCMD_DOWN 1 /* move to 0x11 */
283#define QLOGIC_IB_IBCC_LINKCMD_ARMED 2 /* move to 0x21 */
284#define QLOGIC_IB_IBCC_LINKCMD_ACTIVE 3 /* move to 0x31 */
285#define QLOGIC_IB_IBCC_LINKCMD_SHIFT 18
286
287/*
288 * We could have a single register get/put routine, that takes a group type,
289 * but this is somewhat clearer and cleaner. It also gives us some error
290 * checking. 64 bit register reads should always work, but are inefficient
291 * on opteron (the northbridge always generates 2 separate HT 32 bit reads),
292 * so we use kreg32 wherever possible. User register and counter register
293 * reads are always 32 bit reads, so only one form of those routines.
294 */
295
296/**
297 * qib_read_ureg32 - read 32-bit virtualized per-context register
298 * @dd: device
299 * @regno: register number
300 * @ctxt: context number
301 *
302 * Return the contents of a register that is virtualized to be per context.
303 * Returns -1 on errors (not distinguishable from valid contents at
304 * runtime; we may add a separate error variable at some point).
305 */
306static inline u32 qib_read_ureg32(const struct qib_devdata *dd,
307 enum qib_ureg regno, int ctxt)
308{
309 if (!dd->kregbase || !(dd->flags & QIB_PRESENT))
310 return 0;
311
312 if (dd->userbase)
313 return readl(regno + (u64 __iomem *)
314 ((char __iomem *)dd->userbase +
315 dd->ureg_align * ctxt));
316 else
317 return readl(regno + (u64 __iomem *)
318 (dd->uregbase +
319 (char __iomem *)dd->kregbase +
320 dd->ureg_align * ctxt));
321}
322
323/**
324 * qib_write_ureg - write 32-bit virtualized per-context register
325 * @dd: device
326 * @regno: register number
327 * @value: value
328 * @ctxt: context
329 *
330 * Write the contents of a register that is virtualized to be per context.
331 */
332static inline void qib_write_ureg(const struct qib_devdata *dd,
333 enum qib_ureg regno, u64 value, int ctxt)
334{
335 u64 __iomem *ubase;
336
337 if (dd->userbase)
338 ubase = (u64 __iomem *)
339 ((char __iomem *) dd->userbase +
340 dd->ureg_align * ctxt);
341 else
342 ubase = (u64 __iomem *)
343 (dd->uregbase +
344 (char __iomem *) dd->kregbase +
345 dd->ureg_align * ctxt);
346
347 if (dd->kregbase && (dd->flags & QIB_PRESENT))
348 writeq(value, &ubase[regno]);
349}
350
351static inline u32 qib_read_kreg32(const struct qib_devdata *dd,
352 const u16 regno)
353{
354 if (!dd->kregbase || !(dd->flags & QIB_PRESENT))
355 return -1;
356 return readl((u32 __iomem *)&dd->kregbase[regno]);
357}
358
359static inline u64 qib_read_kreg64(const struct qib_devdata *dd,
360 const u16 regno)
361{
362 if (!dd->kregbase || !(dd->flags & QIB_PRESENT))
363 return -1;
364
365 return readq(&dd->kregbase[regno]);
366}
367
368static inline void qib_write_kreg(const struct qib_devdata *dd,
369 const u16 regno, u64 value)
370{
371 if (dd->kregbase && (dd->flags & QIB_PRESENT))
372 writeq(value, &dd->kregbase[regno]);
373}
374
375/**
376 * qib_write_kreg_ctxt - write a device's per-ctxt 64-bit kernel register
377 * @dd: the qlogic_ib device
378 * @regno: the register number to write
379 * @ctxt: the context containing the register
380 * @value: the value to write
381 */
382static inline void qib_write_kreg_ctxt(const struct qib_devdata *dd,
383 const u16 regno, unsigned ctxt,
384 u64 value)
385{
386 qib_write_kreg(dd, regno + ctxt, value);
387}
388
389static inline void write_6120_creg(const struct qib_devdata *dd,
390 u16 regno, u64 value)
391{
392 if (dd->cspec->cregbase && (dd->flags & QIB_PRESENT))
393 writeq(value, &dd->cspec->cregbase[regno]);
394}
395
396static inline u64 read_6120_creg(const struct qib_devdata *dd, u16 regno)
397{
398 if (!dd->cspec->cregbase || !(dd->flags & QIB_PRESENT))
399 return 0;
400 return readq(&dd->cspec->cregbase[regno]);
401}
402
403static inline u32 read_6120_creg32(const struct qib_devdata *dd, u16 regno)
404{
405 if (!dd->cspec->cregbase || !(dd->flags & QIB_PRESENT))
406 return 0;
407 return readl(&dd->cspec->cregbase[regno]);
408}
409
410/* kr_control bits */
411#define QLOGIC_IB_C_RESET 1U
412
413/* kr_intstatus, kr_intclear, kr_intmask bits */
414#define QLOGIC_IB_I_RCVURG_MASK ((1U << 5) - 1)
415#define QLOGIC_IB_I_RCVURG_SHIFT 0
416#define QLOGIC_IB_I_RCVAVAIL_MASK ((1U << 5) - 1)
417#define QLOGIC_IB_I_RCVAVAIL_SHIFT 12
418
419#define QLOGIC_IB_C_FREEZEMODE 0x00000002
420#define QLOGIC_IB_C_LINKENABLE 0x00000004
421#define QLOGIC_IB_I_ERROR 0x0000000080000000ULL
422#define QLOGIC_IB_I_SPIOSENT 0x0000000040000000ULL
423#define QLOGIC_IB_I_SPIOBUFAVAIL 0x0000000020000000ULL
424#define QLOGIC_IB_I_GPIO 0x0000000010000000ULL
425#define QLOGIC_IB_I_BITSEXTANT \
426 ((QLOGIC_IB_I_RCVURG_MASK << QLOGIC_IB_I_RCVURG_SHIFT) | \
427 (QLOGIC_IB_I_RCVAVAIL_MASK << \
428 QLOGIC_IB_I_RCVAVAIL_SHIFT) | \
429 QLOGIC_IB_I_ERROR | QLOGIC_IB_I_SPIOSENT | \
430 QLOGIC_IB_I_SPIOBUFAVAIL | QLOGIC_IB_I_GPIO)
431
432/* kr_hwerrclear, kr_hwerrmask, kr_hwerrstatus, bits */
433#define QLOGIC_IB_HWE_PCIEMEMPARITYERR_MASK 0x000000000000003fULL
434#define QLOGIC_IB_HWE_PCIEMEMPARITYERR_SHIFT 0
435#define QLOGIC_IB_HWE_PCIEPOISONEDTLP 0x0000000010000000ULL
436#define QLOGIC_IB_HWE_PCIECPLTIMEOUT 0x0000000020000000ULL
437#define QLOGIC_IB_HWE_PCIEBUSPARITYXTLH 0x0000000040000000ULL
438#define QLOGIC_IB_HWE_PCIEBUSPARITYXADM 0x0000000080000000ULL
439#define QLOGIC_IB_HWE_PCIEBUSPARITYRADM 0x0000000100000000ULL
440#define QLOGIC_IB_HWE_COREPLL_FBSLIP 0x0080000000000000ULL
441#define QLOGIC_IB_HWE_COREPLL_RFSLIP 0x0100000000000000ULL
442#define QLOGIC_IB_HWE_PCIE1PLLFAILED 0x0400000000000000ULL
443#define QLOGIC_IB_HWE_PCIE0PLLFAILED 0x0800000000000000ULL
444#define QLOGIC_IB_HWE_SERDESPLLFAILED 0x1000000000000000ULL
445
446
447/* kr_extstatus bits */
448#define QLOGIC_IB_EXTS_FREQSEL 0x2
449#define QLOGIC_IB_EXTS_SERDESSEL 0x4
450#define QLOGIC_IB_EXTS_MEMBIST_ENDTEST 0x0000000000004000
451#define QLOGIC_IB_EXTS_MEMBIST_FOUND 0x0000000000008000
452
453/* kr_xgxsconfig bits */
454#define QLOGIC_IB_XGXS_RESET 0x5ULL
455
456#define _QIB_GPIO_SDA_NUM 1
457#define _QIB_GPIO_SCL_NUM 0
458
459/* Bits in GPIO for the added IB link interrupts */
460#define GPIO_RXUVL_BIT 3
461#define GPIO_OVRUN_BIT 4
462#define GPIO_LLI_BIT 5
463#define GPIO_ERRINTR_MASK 0x38
464
465
466#define QLOGIC_IB_RT_BUFSIZE_MASK 0xe0000000ULL
467#define QLOGIC_IB_RT_BUFSIZE_SHIFTVAL(tid) \
468 ((((tid) & QLOGIC_IB_RT_BUFSIZE_MASK) >> 29) + 11 - 1)
469#define QLOGIC_IB_RT_BUFSIZE(tid) (1 << QLOGIC_IB_RT_BUFSIZE_SHIFTVAL(tid))
470#define QLOGIC_IB_RT_IS_VALID(tid) \
471 (((tid) & QLOGIC_IB_RT_BUFSIZE_MASK) && \
472 ((((tid) & QLOGIC_IB_RT_BUFSIZE_MASK) != QLOGIC_IB_RT_BUFSIZE_MASK)))
473#define QLOGIC_IB_RT_ADDR_MASK 0x1FFFFFFFULL /* 29 bits valid */
474#define QLOGIC_IB_RT_ADDR_SHIFT 10
475
476#define QLOGIC_IB_R_INTRAVAIL_SHIFT 16
477#define QLOGIC_IB_R_TAILUPD_SHIFT 31
478#define IBA6120_R_PKEY_DIS_SHIFT 30
479
480#define PBC_6120_VL15_SEND_CTRL (1ULL << 31) /* pbc; VL15; link_buf only */
481
482#define IBCBUSFRSPCPARITYERR HWE_MASK(IBCBusFromSPCParityErr)
483#define IBCBUSTOSPCPARITYERR HWE_MASK(IBCBusToSPCParityErr)
484
485#define SYM_MASK_BIT(regname, fldname, bit) ((u64) \
486 ((1ULL << (SYM_LSB(regname, fldname) + (bit)))))
487
488#define TXEMEMPARITYERR_PIOBUF \
489 SYM_MASK_BIT(HwErrMask, TXEMemParityErrMask, 0)
490#define TXEMEMPARITYERR_PIOPBC \
491 SYM_MASK_BIT(HwErrMask, TXEMemParityErrMask, 1)
492#define TXEMEMPARITYERR_PIOLAUNCHFIFO \
493 SYM_MASK_BIT(HwErrMask, TXEMemParityErrMask, 2)
494
495#define RXEMEMPARITYERR_RCVBUF \
496 SYM_MASK_BIT(HwErrMask, RXEMemParityErrMask, 0)
497#define RXEMEMPARITYERR_LOOKUPQ \
498 SYM_MASK_BIT(HwErrMask, RXEMemParityErrMask, 1)
499#define RXEMEMPARITYERR_EXPTID \
500 SYM_MASK_BIT(HwErrMask, RXEMemParityErrMask, 2)
501#define RXEMEMPARITYERR_EAGERTID \
502 SYM_MASK_BIT(HwErrMask, RXEMemParityErrMask, 3)
503#define RXEMEMPARITYERR_FLAGBUF \
504 SYM_MASK_BIT(HwErrMask, RXEMemParityErrMask, 4)
505#define RXEMEMPARITYERR_DATAINFO \
506 SYM_MASK_BIT(HwErrMask, RXEMemParityErrMask, 5)
507#define RXEMEMPARITYERR_HDRINFO \
508 SYM_MASK_BIT(HwErrMask, RXEMemParityErrMask, 6)
509
510/* 6120 specific hardware errors... */
511static const struct qib_hwerror_msgs qib_6120_hwerror_msgs[] = {
512 /* generic hardware errors */
513 QLOGIC_IB_HWE_MSG(IBCBUSFRSPCPARITYERR, "QIB2IB Parity"),
514 QLOGIC_IB_HWE_MSG(IBCBUSTOSPCPARITYERR, "IB2QIB Parity"),
515
516 QLOGIC_IB_HWE_MSG(TXEMEMPARITYERR_PIOBUF,
517 "TXE PIOBUF Memory Parity"),
518 QLOGIC_IB_HWE_MSG(TXEMEMPARITYERR_PIOPBC,
519 "TXE PIOPBC Memory Parity"),
520 QLOGIC_IB_HWE_MSG(TXEMEMPARITYERR_PIOLAUNCHFIFO,
521 "TXE PIOLAUNCHFIFO Memory Parity"),
522
523 QLOGIC_IB_HWE_MSG(RXEMEMPARITYERR_RCVBUF,
524 "RXE RCVBUF Memory Parity"),
525 QLOGIC_IB_HWE_MSG(RXEMEMPARITYERR_LOOKUPQ,
526 "RXE LOOKUPQ Memory Parity"),
527 QLOGIC_IB_HWE_MSG(RXEMEMPARITYERR_EAGERTID,
528 "RXE EAGERTID Memory Parity"),
529 QLOGIC_IB_HWE_MSG(RXEMEMPARITYERR_EXPTID,
530 "RXE EXPTID Memory Parity"),
531 QLOGIC_IB_HWE_MSG(RXEMEMPARITYERR_FLAGBUF,
532 "RXE FLAGBUF Memory Parity"),
533 QLOGIC_IB_HWE_MSG(RXEMEMPARITYERR_DATAINFO,
534 "RXE DATAINFO Memory Parity"),
535 QLOGIC_IB_HWE_MSG(RXEMEMPARITYERR_HDRINFO,
536 "RXE HDRINFO Memory Parity"),
537
538 /* chip-specific hardware errors */
539 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_PCIEPOISONEDTLP,
540 "PCIe Poisoned TLP"),
541 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_PCIECPLTIMEOUT,
542 "PCIe completion timeout"),
543 /*
544 * In practice, it's unlikely wthat we'll see PCIe PLL, or bus
545 * parity or memory parity error failures, because most likely we
546 * won't be able to talk to the core of the chip. Nonetheless, we
547 * might see them, if they are in parts of the PCIe core that aren't
548 * essential.
549 */
550 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_PCIE1PLLFAILED,
551 "PCIePLL1"),
552 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_PCIE0PLLFAILED,
553 "PCIePLL0"),
554 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_PCIEBUSPARITYXTLH,
555 "PCIe XTLH core parity"),
556 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_PCIEBUSPARITYXADM,
557 "PCIe ADM TX core parity"),
558 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_PCIEBUSPARITYRADM,
559 "PCIe ADM RX core parity"),
560 QLOGIC_IB_HWE_MSG(QLOGIC_IB_HWE_SERDESPLLFAILED,
561 "SerDes PLL"),
562};
563
564#define TXE_PIO_PARITY (TXEMEMPARITYERR_PIOBUF | TXEMEMPARITYERR_PIOPBC)
565#define _QIB_PLL_FAIL (QLOGIC_IB_HWE_COREPLL_FBSLIP | \
566 QLOGIC_IB_HWE_COREPLL_RFSLIP)
567
568 /* variables for sanity checking interrupt and errors */
569#define IB_HWE_BITSEXTANT \
570 (HWE_MASK(RXEMemParityErr) | \
571 HWE_MASK(TXEMemParityErr) | \
572 (QLOGIC_IB_HWE_PCIEMEMPARITYERR_MASK << \
573 QLOGIC_IB_HWE_PCIEMEMPARITYERR_SHIFT) | \
574 QLOGIC_IB_HWE_PCIE1PLLFAILED | \
575 QLOGIC_IB_HWE_PCIE0PLLFAILED | \
576 QLOGIC_IB_HWE_PCIEPOISONEDTLP | \
577 QLOGIC_IB_HWE_PCIECPLTIMEOUT | \
578 QLOGIC_IB_HWE_PCIEBUSPARITYXTLH | \
579 QLOGIC_IB_HWE_PCIEBUSPARITYXADM | \
580 QLOGIC_IB_HWE_PCIEBUSPARITYRADM | \
581 HWE_MASK(PowerOnBISTFailed) | \
582 QLOGIC_IB_HWE_COREPLL_FBSLIP | \
583 QLOGIC_IB_HWE_COREPLL_RFSLIP | \
584 QLOGIC_IB_HWE_SERDESPLLFAILED | \
585 HWE_MASK(IBCBusToSPCParityErr) | \
586 HWE_MASK(IBCBusFromSPCParityErr))
587
588#define IB_E_BITSEXTANT \
589 (ERR_MASK(RcvFormatErr) | ERR_MASK(RcvVCRCErr) | \
590 ERR_MASK(RcvICRCErr) | ERR_MASK(RcvMinPktLenErr) | \
591 ERR_MASK(RcvMaxPktLenErr) | ERR_MASK(RcvLongPktLenErr) | \
592 ERR_MASK(RcvShortPktLenErr) | ERR_MASK(RcvUnexpectedCharErr) | \
593 ERR_MASK(RcvUnsupportedVLErr) | ERR_MASK(RcvEBPErr) | \
594 ERR_MASK(RcvIBFlowErr) | ERR_MASK(RcvBadVersionErr) | \
595 ERR_MASK(RcvEgrFullErr) | ERR_MASK(RcvHdrFullErr) | \
596 ERR_MASK(RcvBadTidErr) | ERR_MASK(RcvHdrLenErr) | \
597 ERR_MASK(RcvHdrErr) | ERR_MASK(RcvIBLostLinkErr) | \
598 ERR_MASK(SendMinPktLenErr) | ERR_MASK(SendMaxPktLenErr) | \
599 ERR_MASK(SendUnderRunErr) | ERR_MASK(SendPktLenErr) | \
600 ERR_MASK(SendDroppedSmpPktErr) | \
601 ERR_MASK(SendDroppedDataPktErr) | \
602 ERR_MASK(SendPioArmLaunchErr) | \
603 ERR_MASK(SendUnexpectedPktNumErr) | \
604 ERR_MASK(SendUnsupportedVLErr) | ERR_MASK(IBStatusChanged) | \
605 ERR_MASK(InvalidAddrErr) | ERR_MASK(ResetNegated) | \
606 ERR_MASK(HardwareErr))
607
608#define QLOGIC_IB_E_PKTERRS ( \
609 ERR_MASK(SendPktLenErr) | \
610 ERR_MASK(SendDroppedDataPktErr) | \
611 ERR_MASK(RcvVCRCErr) | \
612 ERR_MASK(RcvICRCErr) | \
613 ERR_MASK(RcvShortPktLenErr) | \
614 ERR_MASK(RcvEBPErr))
615
616/* These are all rcv-related errors which we want to count for stats */
617#define E_SUM_PKTERRS \
618 (ERR_MASK(RcvHdrLenErr) | ERR_MASK(RcvBadTidErr) | \
619 ERR_MASK(RcvBadVersionErr) | ERR_MASK(RcvHdrErr) | \
620 ERR_MASK(RcvLongPktLenErr) | ERR_MASK(RcvShortPktLenErr) | \
621 ERR_MASK(RcvMaxPktLenErr) | ERR_MASK(RcvMinPktLenErr) | \
622 ERR_MASK(RcvFormatErr) | ERR_MASK(RcvUnsupportedVLErr) | \
623 ERR_MASK(RcvUnexpectedCharErr) | ERR_MASK(RcvEBPErr))
624
625/* These are all send-related errors which we want to count for stats */
626#define E_SUM_ERRS \
627 (ERR_MASK(SendPioArmLaunchErr) | \
628 ERR_MASK(SendUnexpectedPktNumErr) | \
629 ERR_MASK(SendDroppedDataPktErr) | \
630 ERR_MASK(SendDroppedSmpPktErr) | \
631 ERR_MASK(SendMaxPktLenErr) | ERR_MASK(SendUnsupportedVLErr) | \
632 ERR_MASK(SendMinPktLenErr) | ERR_MASK(SendPktLenErr) | \
633 ERR_MASK(InvalidAddrErr))
634
635/*
636 * this is similar to E_SUM_ERRS, but can't ignore armlaunch, don't ignore
637 * errors not related to freeze and cancelling buffers. Can't ignore
638 * armlaunch because could get more while still cleaning up, and need
639 * to cancel those as they happen.
640 */
641#define E_SPKT_ERRS_IGNORE \
642 (ERR_MASK(SendDroppedDataPktErr) | \
643 ERR_MASK(SendDroppedSmpPktErr) | \
644 ERR_MASK(SendMaxPktLenErr) | ERR_MASK(SendMinPktLenErr) | \
645 ERR_MASK(SendPktLenErr))
646
647/*
648 * these are errors that can occur when the link changes state while
649 * a packet is being sent or received. This doesn't cover things
650 * like EBP or VCRC that can be the result of a sending having the
651 * link change state, so we receive a "known bad" packet.
652 */
653#define E_SUM_LINK_PKTERRS \
654 (ERR_MASK(SendDroppedDataPktErr) | \
655 ERR_MASK(SendDroppedSmpPktErr) | \
656 ERR_MASK(SendMinPktLenErr) | ERR_MASK(SendPktLenErr) | \
657 ERR_MASK(RcvShortPktLenErr) | ERR_MASK(RcvMinPktLenErr) | \
658 ERR_MASK(RcvUnexpectedCharErr))
659
660static void qib_6120_put_tid_2(struct qib_devdata *, u64 __iomem *,
661 u32, unsigned long);
662
663/*
664 * On platforms using this chip, and not having ordered WC stores, we
665 * can get TXE parity errors due to speculative reads to the PIO buffers,
666 * and this, due to a chip issue can result in (many) false parity error
667 * reports. So it's a debug print on those, and an info print on systems
668 * where the speculative reads don't occur.
669 */
670static void qib_6120_txe_recover(struct qib_devdata *dd)
671{
672 if (!qib_unordered_wc())
673 qib_devinfo(dd->pcidev,
674 "Recovering from TXE PIO parity error\n");
675}
676
677/* enable/disable chip from delivering interrupts */
678static void qib_6120_set_intr_state(struct qib_devdata *dd, u32 enable)
679{
680 if (enable) {
681 if (dd->flags & QIB_BADINTR)
682 return;
683 qib_write_kreg(dd, kr_intmask, ~0ULL);
684 /* force re-interrupt of any pending interrupts. */
685 qib_write_kreg(dd, kr_intclear, 0ULL);
686 } else
687 qib_write_kreg(dd, kr_intmask, 0ULL);
688}
689
690/*
691 * Try to cleanup as much as possible for anything that might have gone
692 * wrong while in freeze mode, such as pio buffers being written by user
693 * processes (causing armlaunch), send errors due to going into freeze mode,
694 * etc., and try to avoid causing extra interrupts while doing so.
695 * Forcibly update the in-memory pioavail register copies after cleanup
696 * because the chip won't do it while in freeze mode (the register values
697 * themselves are kept correct).
698 * Make sure that we don't lose any important interrupts by using the chip
699 * feature that says that writing 0 to a bit in *clear that is set in
700 * *status will cause an interrupt to be generated again (if allowed by
701 * the *mask value).
702 * This is in chip-specific code because of all of the register accesses,
703 * even though the details are similar on most chips
704 */
705static void qib_6120_clear_freeze(struct qib_devdata *dd)
706{
707 /* disable error interrupts, to avoid confusion */
708 qib_write_kreg(dd, kr_errmask, 0ULL);
709
710 /* also disable interrupts; errormask is sometimes overwritten */
711 qib_6120_set_intr_state(dd, 0);
712
713 qib_cancel_sends(dd->pport);
714
715 /* clear the freeze, and be sure chip saw it */
716 qib_write_kreg(dd, kr_control, dd->control);
717 qib_read_kreg32(dd, kr_scratch);
718
719 /* force in-memory update now we are out of freeze */
720 qib_force_pio_avail_update(dd);
721
722 /*
723 * force new interrupt if any hwerr, error or interrupt bits are
724 * still set, and clear "safe" send packet errors related to freeze
725 * and cancelling sends. Re-enable error interrupts before possible
726 * force of re-interrupt on pending interrupts.
727 */
728 qib_write_kreg(dd, kr_hwerrclear, 0ULL);
729 qib_write_kreg(dd, kr_errclear, E_SPKT_ERRS_IGNORE);
730 qib_write_kreg(dd, kr_errmask, dd->cspec->errormask);
731 qib_6120_set_intr_state(dd, 1);
732}
733
734/**
735 * qib_handle_6120_hwerrors - display hardware errors.
736 * @dd: the qlogic_ib device
737 * @msg: the output buffer
738 * @msgl: the size of the output buffer
739 *
740 * Use same msg buffer as regular errors to avoid excessive stack
741 * use. Most hardware errors are catastrophic, but for right now,
742 * we'll print them and continue. Reuse the same message buffer as
743 * handle_6120_errors() to avoid excessive stack usage.
744 */
745static void qib_handle_6120_hwerrors(struct qib_devdata *dd, char *msg,
746 size_t msgl)
747{
748 u64 hwerrs;
749 u32 bits, ctrl;
750 int isfatal = 0;
751 char *bitsmsg;
752
753 hwerrs = qib_read_kreg64(dd, kr_hwerrstatus);
754 if (!hwerrs)
755 return;
756 if (hwerrs == ~0ULL) {
757 qib_dev_err(dd,
758 "Read of hardware error status failed (all bits set); ignoring\n");
759 return;
760 }
761 qib_stats.sps_hwerrs++;
762
763 /* Always clear the error status register, except MEMBISTFAIL,
764 * regardless of whether we continue or stop using the chip.
765 * We want that set so we know it failed, even across driver reload.
766 * We'll still ignore it in the hwerrmask. We do this partly for
767 * diagnostics, but also for support */
768 qib_write_kreg(dd, kr_hwerrclear,
769 hwerrs & ~HWE_MASK(PowerOnBISTFailed));
770
771 hwerrs &= dd->cspec->hwerrmask;
772
773 /*
774 * Make sure we get this much out, unless told to be quiet,
775 * or it's occurred within the last 5 seconds.
776 */
777 if (hwerrs & ~(TXE_PIO_PARITY | RXEMEMPARITYERR_EAGERTID))
778 qib_devinfo(dd->pcidev,
779 "Hardware error: hwerr=0x%llx (cleared)\n",
780 (unsigned long long) hwerrs);
781
782 if (hwerrs & ~IB_HWE_BITSEXTANT)
783 qib_dev_err(dd,
784 "hwerror interrupt with unknown errors %llx set\n",
785 (unsigned long long)(hwerrs & ~IB_HWE_BITSEXTANT));
786
787 ctrl = qib_read_kreg32(dd, kr_control);
788 if ((ctrl & QLOGIC_IB_C_FREEZEMODE) && !dd->diag_client) {
789 /*
790 * Parity errors in send memory are recoverable,
791 * just cancel the send (if indicated in * sendbuffererror),
792 * count the occurrence, unfreeze (if no other handled
793 * hardware error bits are set), and continue. They can
794 * occur if a processor speculative read is done to the PIO
795 * buffer while we are sending a packet, for example.
796 */
797 if (hwerrs & TXE_PIO_PARITY) {
798 qib_6120_txe_recover(dd);
799 hwerrs &= ~TXE_PIO_PARITY;
800 }
801
802 if (!hwerrs)
803 qib_6120_clear_freeze(dd);
804 else
805 isfatal = 1;
806 }
807
808 *msg = '\0';
809
810 if (hwerrs & HWE_MASK(PowerOnBISTFailed)) {
811 isfatal = 1;
812 strlcat(msg,
813 "[Memory BIST test failed, InfiniPath hardware unusable]",
814 msgl);
815 /* ignore from now on, so disable until driver reloaded */
816 dd->cspec->hwerrmask &= ~HWE_MASK(PowerOnBISTFailed);
817 qib_write_kreg(dd, kr_hwerrmask, dd->cspec->hwerrmask);
818 }
819
820 qib_format_hwerrors(hwerrs, qib_6120_hwerror_msgs,
821 ARRAY_SIZE(qib_6120_hwerror_msgs), msg, msgl);
822
823 bitsmsg = dd->cspec->bitsmsgbuf;
824 if (hwerrs & (QLOGIC_IB_HWE_PCIEMEMPARITYERR_MASK <<
825 QLOGIC_IB_HWE_PCIEMEMPARITYERR_SHIFT)) {
826 bits = (u32) ((hwerrs >>
827 QLOGIC_IB_HWE_PCIEMEMPARITYERR_SHIFT) &
828 QLOGIC_IB_HWE_PCIEMEMPARITYERR_MASK);
829 snprintf(bitsmsg, sizeof(dd->cspec->bitsmsgbuf),
830 "[PCIe Mem Parity Errs %x] ", bits);
831 strlcat(msg, bitsmsg, msgl);
832 }
833
834 if (hwerrs & _QIB_PLL_FAIL) {
835 isfatal = 1;
836 snprintf(bitsmsg, sizeof(dd->cspec->bitsmsgbuf),
837 "[PLL failed (%llx), InfiniPath hardware unusable]",
838 (unsigned long long) hwerrs & _QIB_PLL_FAIL);
839 strlcat(msg, bitsmsg, msgl);
840 /* ignore from now on, so disable until driver reloaded */
841 dd->cspec->hwerrmask &= ~(hwerrs & _QIB_PLL_FAIL);
842 qib_write_kreg(dd, kr_hwerrmask, dd->cspec->hwerrmask);
843 }
844
845 if (hwerrs & QLOGIC_IB_HWE_SERDESPLLFAILED) {
846 /*
847 * If it occurs, it is left masked since the external
848 * interface is unused
849 */
850 dd->cspec->hwerrmask &= ~QLOGIC_IB_HWE_SERDESPLLFAILED;
851 qib_write_kreg(dd, kr_hwerrmask, dd->cspec->hwerrmask);
852 }
853
854 if (hwerrs)
855 /*
856 * if any set that we aren't ignoring; only
857 * make the complaint once, in case it's stuck
858 * or recurring, and we get here multiple
859 * times.
860 */
861 qib_dev_err(dd, "%s hardware error\n", msg);
862 else
863 *msg = 0; /* recovered from all of them */
864
865 if (isfatal && !dd->diag_client) {
866 qib_dev_err(dd,
867 "Fatal Hardware Error, no longer usable, SN %.16s\n",
868 dd->serial);
869 /*
870 * for /sys status file and user programs to print; if no
871 * trailing brace is copied, we'll know it was truncated.
872 */
873 if (dd->freezemsg)
874 snprintf(dd->freezemsg, dd->freezelen,
875 "{%s}", msg);
876 qib_disable_after_error(dd);
877 }
878}
879
880/*
881 * Decode the error status into strings, deciding whether to always
882 * print * it or not depending on "normal packet errors" vs everything
883 * else. Return 1 if "real" errors, otherwise 0 if only packet
884 * errors, so caller can decide what to print with the string.
885 */
886static int qib_decode_6120_err(struct qib_devdata *dd, char *buf, size_t blen,
887 u64 err)
888{
889 int iserr = 1;
890
891 *buf = '\0';
892 if (err & QLOGIC_IB_E_PKTERRS) {
893 if (!(err & ~QLOGIC_IB_E_PKTERRS))
894 iserr = 0;
895 if ((err & ERR_MASK(RcvICRCErr)) &&
896 !(err&(ERR_MASK(RcvVCRCErr)|ERR_MASK(RcvEBPErr))))
897 strlcat(buf, "CRC ", blen);
898 if (!iserr)
899 goto done;
900 }
901 if (err & ERR_MASK(RcvHdrLenErr))
902 strlcat(buf, "rhdrlen ", blen);
903 if (err & ERR_MASK(RcvBadTidErr))
904 strlcat(buf, "rbadtid ", blen);
905 if (err & ERR_MASK(RcvBadVersionErr))
906 strlcat(buf, "rbadversion ", blen);
907 if (err & ERR_MASK(RcvHdrErr))
908 strlcat(buf, "rhdr ", blen);
909 if (err & ERR_MASK(RcvLongPktLenErr))
910 strlcat(buf, "rlongpktlen ", blen);
911 if (err & ERR_MASK(RcvMaxPktLenErr))
912 strlcat(buf, "rmaxpktlen ", blen);
913 if (err & ERR_MASK(RcvMinPktLenErr))
914 strlcat(buf, "rminpktlen ", blen);
915 if (err & ERR_MASK(SendMinPktLenErr))
916 strlcat(buf, "sminpktlen ", blen);
917 if (err & ERR_MASK(RcvFormatErr))
918 strlcat(buf, "rformaterr ", blen);
919 if (err & ERR_MASK(RcvUnsupportedVLErr))
920 strlcat(buf, "runsupvl ", blen);
921 if (err & ERR_MASK(RcvUnexpectedCharErr))
922 strlcat(buf, "runexpchar ", blen);
923 if (err & ERR_MASK(RcvIBFlowErr))
924 strlcat(buf, "ribflow ", blen);
925 if (err & ERR_MASK(SendUnderRunErr))
926 strlcat(buf, "sunderrun ", blen);
927 if (err & ERR_MASK(SendPioArmLaunchErr))
928 strlcat(buf, "spioarmlaunch ", blen);
929 if (err & ERR_MASK(SendUnexpectedPktNumErr))
930 strlcat(buf, "sunexperrpktnum ", blen);
931 if (err & ERR_MASK(SendDroppedSmpPktErr))
932 strlcat(buf, "sdroppedsmppkt ", blen);
933 if (err & ERR_MASK(SendMaxPktLenErr))
934 strlcat(buf, "smaxpktlen ", blen);
935 if (err & ERR_MASK(SendUnsupportedVLErr))
936 strlcat(buf, "sunsupVL ", blen);
937 if (err & ERR_MASK(InvalidAddrErr))
938 strlcat(buf, "invalidaddr ", blen);
939 if (err & ERR_MASK(RcvEgrFullErr))
940 strlcat(buf, "rcvegrfull ", blen);
941 if (err & ERR_MASK(RcvHdrFullErr))
942 strlcat(buf, "rcvhdrfull ", blen);
943 if (err & ERR_MASK(IBStatusChanged))
944 strlcat(buf, "ibcstatuschg ", blen);
945 if (err & ERR_MASK(RcvIBLostLinkErr))
946 strlcat(buf, "riblostlink ", blen);
947 if (err & ERR_MASK(HardwareErr))
948 strlcat(buf, "hardware ", blen);
949 if (err & ERR_MASK(ResetNegated))
950 strlcat(buf, "reset ", blen);
951done:
952 return iserr;
953}
954
955/*
956 * Called when we might have an error that is specific to a particular
957 * PIO buffer, and may need to cancel that buffer, so it can be re-used.
958 */
959static void qib_disarm_6120_senderrbufs(struct qib_pportdata *ppd)
960{
961 unsigned long sbuf[2];
962 struct qib_devdata *dd = ppd->dd;
963
964 /*
965 * It's possible that sendbuffererror could have bits set; might
966 * have already done this as a result of hardware error handling.
967 */
968 sbuf[0] = qib_read_kreg64(dd, kr_sendbuffererror);
969 sbuf[1] = qib_read_kreg64(dd, kr_sendbuffererror + 1);
970
971 if (sbuf[0] || sbuf[1])
972 qib_disarm_piobufs_set(dd, sbuf,
973 dd->piobcnt2k + dd->piobcnt4k);
974}
975
976static int chk_6120_linkrecovery(struct qib_devdata *dd, u64 ibcs)
977{
978 int ret = 1;
979 u32 ibstate = qib_6120_iblink_state(ibcs);
980 u32 linkrecov = read_6120_creg32(dd, cr_iblinkerrrecov);
981
982 if (linkrecov != dd->cspec->lastlinkrecov) {
983 /* and no more until active again */
984 dd->cspec->lastlinkrecov = 0;
985 qib_set_linkstate(dd->pport, QIB_IB_LINKDOWN);
986 ret = 0;
987 }
988 if (ibstate == IB_PORT_ACTIVE)
989 dd->cspec->lastlinkrecov =
990 read_6120_creg32(dd, cr_iblinkerrrecov);
991 return ret;
992}
993
994static void handle_6120_errors(struct qib_devdata *dd, u64 errs)
995{
996 char *msg;
997 u64 ignore_this_time = 0;
998 u64 iserr = 0;
999 struct qib_pportdata *ppd = dd->pport;
1000 u64 mask;
1001
1002 /* don't report errors that are masked */
1003 errs &= dd->cspec->errormask;
1004 msg = dd->cspec->emsgbuf;
1005
1006 /* do these first, they are most important */
1007 if (errs & ERR_MASK(HardwareErr))
1008 qib_handle_6120_hwerrors(dd, msg, sizeof(dd->cspec->emsgbuf));
1009
1010 if (errs & ~IB_E_BITSEXTANT)
1011 qib_dev_err(dd,
1012 "error interrupt with unknown errors %llx set\n",
1013 (unsigned long long) (errs & ~IB_E_BITSEXTANT));
1014
1015 if (errs & E_SUM_ERRS) {
1016 qib_disarm_6120_senderrbufs(ppd);
1017 if ((errs & E_SUM_LINK_PKTERRS) &&
1018 !(ppd->lflags & QIBL_LINKACTIVE)) {
1019 /*
1020 * This can happen when trying to bring the link
1021 * up, but the IB link changes state at the "wrong"
1022 * time. The IB logic then complains that the packet
1023 * isn't valid. We don't want to confuse people, so
1024 * we just don't print them, except at debug
1025 */
1026 ignore_this_time = errs & E_SUM_LINK_PKTERRS;
1027 }
1028 } else if ((errs & E_SUM_LINK_PKTERRS) &&
1029 !(ppd->lflags & QIBL_LINKACTIVE)) {
1030 /*
1031 * This can happen when SMA is trying to bring the link
1032 * up, but the IB link changes state at the "wrong" time.
1033 * The IB logic then complains that the packet isn't
1034 * valid. We don't want to confuse people, so we just
1035 * don't print them, except at debug
1036 */
1037 ignore_this_time = errs & E_SUM_LINK_PKTERRS;
1038 }
1039
1040 qib_write_kreg(dd, kr_errclear, errs);
1041
1042 errs &= ~ignore_this_time;
1043 if (!errs)
1044 goto done;
1045
1046 /*
1047 * The ones we mask off are handled specially below
1048 * or above.

2485 more lines aren’t shown here.

drivers/infiniband/hw/qib/qib_iba7220.c+0−4596

This diff is large: its 4597 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_iba7322.c+0−8475

This diff is large: its 8476 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_init.c+0−1782

This diff is large: its 1783 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_intr.c+0−241

This diff is large: its 242 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_mad.c+0−2450

This diff is large: its 2451 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_mad.h+0−300

This diff is large: its 301 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_pcie.c+0−598

This diff is large: its 599 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_pio_copy.c+0−64

This diff is large: its 65 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_qp.c+0−454

This diff is large: its 455 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_qsfp.c+0−549

This diff is large: its 550 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_qsfp.h+0−188

This diff is large: its 189 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_rc.c+0−2131

This diff is large: its 2132 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_ruc.c+0−314

This diff is large: its 315 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_sd7220.c+0−1445

This diff is large: its 1446 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_sdma.c+0−999

This diff is large: its 1000 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_sysfs.c+0−731

This diff is large: its 732 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_twsi.c+0−502

This diff is large: its 503 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_tx.c+0−566

This diff is large: its 567 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_uc.c+0−521

This diff is large: its 522 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_ud.c+0−583

This diff is large: its 584 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_user_pages.c+0−137

This diff is large: its 138 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_user_sdma.c+0−1470

This diff is large: its 1471 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_user_sdma.h+0−52

This diff is large: its 53 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_verbs.c+0−1705

This diff is large: its 1706 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_verbs.h+0−398

This diff is large: its 399 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_wc_ppc64.c+0−62

This diff is large: its 63 lines aren’t shown here.

drivers/infiniband/hw/qib/qib_wc_x86_64.c+0−150

This diff is large: its 151 lines aren’t shown here.

drivers/infiniband/hw/usnic/usnic_ib_verbs.c+4−0

This diff is large: its 18 lines aren’t shown here.

drivers/infiniband/hw/usnic/usnic_ib_verbs.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/infiniband/hw/vmw_pvrdma/pvrdma_mr.c+5−0

This diff is large: its 25 lines aren’t shown here.

drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/infiniband/sw/rdmavt/mr.c+5−0

This diff is large: its 25 lines aren’t shown here.

drivers/infiniband/sw/rdmavt/mr.h+1−0

This diff is large: its 8 lines aren’t shown here.

drivers/infiniband/sw/rdmavt/vt.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/infiniband/sw/rxe/rxe.c+7−0

This diff is large: its 21 lines aren’t shown here.

drivers/infiniband/sw/rxe/rxe_loc.h+12−0

This diff is large: its 26 lines aren’t shown here.

drivers/infiniband/sw/rxe/rxe_odp.c+175−17

This diff is large: its 237 lines aren’t shown here.

drivers/infiniband/sw/rxe/rxe_verbs.c+4−0

This diff is large: its 18 lines aren’t shown here.

drivers/infiniband/sw/siw/siw_qp_tx.c+16−6

This diff is large: its 65 lines aren’t shown here.

drivers/infiniband/sw/siw/siw_verbs.c+6−1

This diff is large: its 24 lines aren’t shown here.

drivers/infiniband/sw/siw/siw_verbs.h+2−1

This diff is large: its 10 lines aren’t shown here.

drivers/infiniband/ulp/ipoib/ipoib_main.c+2−0

This diff is large: its 9 lines aren’t shown here.

drivers/infiniband/ulp/iser/iscsi_iser.c+1−5

This diff is large: its 21 lines aren’t shown here.

drivers/net/ethernet/mellanox/mlx5/core/Makefile+5−0

This diff is large: its 11 lines aren’t shown here.

drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h+0−5

This diff is large: its 12 lines aren’t shown here.

drivers/net/ethernet/mellanox/mlx5/core/lib/st.c+164−0

This diff is large: its 165 lines aren’t shown here.

drivers/net/ethernet/mellanox/mlx5/core/main.c+2−0

This diff is large: its 16 lines aren’t shown here.

drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h+9−0

This diff is large: its 16 lines aren’t shown here.

drivers/pci/tph.c+6−5

This diff is large: its 53 lines aren’t shown here.

include/linux/cpumask.h+12−0

This diff is large: its 19 lines aren’t shown here.

include/linux/mlx5/driver.h+25−0

This diff is large: its 58 lines aren’t shown here.

include/linux/mlx5/mlx5_ifc.h+12−2

This diff is large: its 35 lines aren’t shown here.

include/linux/pci-tph.h+1−0

This diff is large: its 8 lines aren’t shown here.

include/rdma/ib_umem.h+21−4

This diff is large: its 48 lines aren’t shown here.

include/rdma/ib_verbs.h+63−2

This diff is large: its 146 lines aren’t shown here.

include/rdma/restrack.h+4−0

This diff is large: its 11 lines aren’t shown here.

include/uapi/rdma/efa-abi.h+2−1

This diff is large: its 16 lines aren’t shown here.

include/uapi/rdma/ib_user_ioctl_cmds.h+36−0

This diff is large: its 71 lines aren’t shown here.