mana_rdma_remove() sets gd->rdma_teardown to stop mana_rdma_service_handle() from acting on servicing events, but nothing ever clears it. A hardware service reset (GDMA_EQE_HWC_RESET_REQUEST) goes through mana_gd_suspend() -> mana_rdma_remove() and mana_gd_resume() -> mana_rdma_probe(), so from the first reset onwards every GDMA_EQE_HWC_SOC_SERVICE event returns early and RDMA suspend/resume servicing is silently dropped for the life of the device. gd->is_suspended has the same problem: it is set when servicing removes the adev and is cleared only by a matching resume. A reset while RDMA is suspended re-adds the adev but leaves is_suspended set, so a later resume event calls add_adev() on top of a live gd->adev and leaks it. This is currently masked by the rdma_teardown bug. Clear both in mana_rdma_probe(). On the reset path mana_rdma_remove() has closed the gate and drained the service workqueue, so clear is_suspended first and re-open the gate with smp_store_release(), paired with smp_load_acquire() in the handler, so the handler cannot observe an open gate with a stale is_suspended. On the initial probe path the gate was never closed and both flags are already clear. This does not order gd->adev, which add_adev() publishes afterwards. A servicing event arriving in that window is still dropped, as it is in mainline today on the initial probe path; closing it needs probe and the handler to be serialized and is left to a separate change.
Unified · Split
| @@ -3987,7 +3987,8 @@ static void mana_rdma_service_handle(struct work_struct *work) · 3986 unmodified lines | |||
| 3987 | struct device *dev = gd->gdma_context->dev; | 3987 | struct device *dev = gd->gdma_context->dev; |
| 3988 | int ret; | 3988 | int ret; |
| 3989 | 3989 | ||
| 3990 | if (READ_ONCE(gd->rdma_teardown)) | 3990 | /* Pairs with the smp_store_release() in mana_rdma_probe(). */ |
| 3991 | if (smp_load_acquire(&gd->rdma_teardown)) | ||
| 3991 | goto out; | 3992 | goto out; |
| 3992 | 3993 | ||
| 3993 | switch (serv_work->event) { | 3994 | switch (serv_work->event) { |
| @@ -4283,6 +4284,21 @@ int mana_rdma_probe(struct gdma_dev *gd) · 289 unmodified lines | |||
| 4283 | if (err) | 4284 | if (err) |
| 4284 | return err; | 4285 | return err; |
| 4285 | 4286 | ||
| 4287 | /* Clear the state left by a previous mana_rdma_remove() so servicing | ||
| 4288 | * events are handled again after a reset cycle. | ||
| 4289 | */ | ||
| 4290 | gd->is_suspended = false; | ||
| 4291 | |||
| 4292 | /* Publish is_suspended before re-opening the gate, so the handler | ||
| 4293 | * cannot observe an open gate with a stale is_suspended. Pairs | ||
| 4294 | * with the smp_load_acquire() in mana_rdma_service_handle(). This | ||
| 4295 | * matters on the reset path, where mana_rdma_remove() closed the | ||
| 4296 | * gate and drained the workqueue; on the initial probe path the | ||
| 4297 | * gate was never closed and both flags are already clear. It does | ||
| 4298 | * not order gd->adev, which add_adev() publishes below. | ||
| 4299 | */ | ||
| 4300 | smp_store_release(&gd->rdma_teardown, false); | ||
| 4301 | |||
| 4286 | err = add_adev(gd, "rdma"); | 4302 | err = add_adev(gd, "rdma"); |
| 4287 | if (err) | 4303 | if (err) |
| 4288 | mana_gd_deregister_device(gd); | 4304 | mana_gd_deregister_device(gd); |