mana_rdma_remove() sets gd->rdma_teardown to stop mana_rdma_service_handle() from acting on servicing events, but nothing ever clears it. A hardware service reset (GDMA_EQE_HWC_RESET_REQUEST) goes through mana_gd_suspend() -> mana_rdma_remove() and mana_gd_resume() -> mana_rdma_probe(), so from the first reset onwards every GDMA_EQE_HWC_SOC_SERVICE event returns early and RDMA suspend/resume servicing is silently dropped for the life of the device. gd->is_suspended has the same problem: it is set when servicing removes the adev and is cleared only by a matching resume. A reset while RDMA is suspended re-adds the adev but leaves is_suspended set, so a later resume event calls add_adev() on top of a live gd->adev and leaks it. This is currently masked by the rdma_teardown bug. Clear both in mana_rdma_probe(). On the reset path mana_rdma_remove() has closed the gate and drained the service workqueue, so clear is_suspended first and re-open the gate with smp_store_release(), paired with smp_load_acquire() in the handler, so the handler cannot observe an open gate with a stale is_suspended. On the initial probe path the gate was never closed and both flags are already clear. This does not order gd->adev, which add_adev() publishes afterwards. A servicing event arriving in that window is still dropped, as it is in mainline today on the initial probe path; closing it needs probe and the handler to be serialized and is left to a separate change.
Unified · Split
| @@ -3987,7 +3987,8 @@ static void mana_rdma_service_handle(struct work_struct *work) · 3986 unmodified lines | ||
| 3987 | 3987 | struct device *dev = gd->gdma_context->dev; |
| 3988 | 3988 | int ret; |
| 3989 | 3989 | |
| 3990 | − if (READ_ONCE(gd->rdma_teardown)) | |
| 3990 | + /* Pairs with the smp_store_release() in mana_rdma_probe(). */ | |
| 3991 | + if (smp_load_acquire(&gd->rdma_teardown)) | |
| 3991 | 3992 | goto out; |
| 3992 | 3993 | |
| 3993 | 3994 | switch (serv_work->event) { |
| @@ -4283,6 +4284,21 @@ int mana_rdma_probe(struct gdma_dev *gd) · 289 unmodified lines | ||
| 4283 | 4284 | if (err) |
| 4284 | 4285 | return err; |
| 4285 | 4286 | |
| 4287 | + /* Clear the state left by a previous mana_rdma_remove() so servicing | |
| 4288 | + * events are handled again after a reset cycle. | |
| 4289 | + */ | |
| 4290 | + gd->is_suspended = false; | |
| 4291 | + | |
| 4292 | + /* Publish is_suspended before re-opening the gate, so the handler | |
| 4293 | + * cannot observe an open gate with a stale is_suspended. Pairs | |
| 4294 | + * with the smp_load_acquire() in mana_rdma_service_handle(). This | |
| 4295 | + * matters on the reset path, where mana_rdma_remove() closed the | |
| 4296 | + * gate and drained the workqueue; on the initial probe path the | |
| 4297 | + * gate was never closed and both flags are already clear. It does | |
| 4298 | + * not order gd->adev, which add_adev() publishes below. | |
| 4299 | + */ | |
| 4300 | + smp_store_release(&gd->rdma_teardown, false); | |
| 4301 | + | |
| 4286 | 4302 | err = add_adev(gd, "rdma"); |
| 4287 | 4303 | if (err) |
| 4288 | 4304 | mana_gd_deregister_device(gd); |