scsi: ufs: core: Decouple CQ sweep from request iterator in MCQ
In MCQ mode, ufshcd_mcq_compl_pending_transfer() uses
blk_mq_tagset_busy_iter() to iterate over busy requests during error
recovery and host reset. However, both iterator callbacks perform
whole-queue operations redundantly for each visited request:
- force_compl == true: ufshcd_mcq_force_compl_one() calls
ufshcd_mcq_compl_all_cqes_lock() on every busy request, sweeping the
entire completion ring (hwq->max_entries slots) once per active request
under spin_lock_irqsave even though the first sweep already cleared all
completion entries.
- force_compl == false: ufshcd_mcq_compl_one() acquires cq_lock and polls
CQTPy over MMIO via ufshcd_mcq_poll_cqe_lock() for every busy request
without doing any per-request work.
Sweep or poll each hardware queue (hba->uhq[i]) once at the start of
ufshcd_mcq_compl_pending_transfer(). When force_compl is true, run
blk_mq_tagset_busy_iter() afterward to complete residual in-flight
requests with DID_REQUEUE, and remove the now-unused
ufshcd_mcq_compl_one() callback.
Fixes: ab248643d3d6 ("scsi: ufs: core: Add error handling for MCQ mode")
Cc: stable@vger.kernel.org
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Stanley Jhu <stanleyjhu@google.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260928035816.1294326-3-stanleyjhu@google.com
Signed-off-by: Martin K. Petersen (Oracle) <mkp@kernel.org>
1 file changed