| From 9515743bfb39c61aaf3d4f3219a645c8d1fe9a0e Mon Sep 17 00:00:00 2001 |
| From: Bijan Mottahedeh <bijan.mottahedeh@oracle.com> |
| Date: Wed, 26 Feb 2020 18:53:43 -0800 |
| Subject: nvme-pci: Hold cq_poll_lock while completing CQEs |
| |
| From: Bijan Mottahedeh <bijan.mottahedeh@oracle.com> |
| |
| commit 9515743bfb39c61aaf3d4f3219a645c8d1fe9a0e upstream. |
| |
| Completions need to consumed in the same order the controller submitted |
| them, otherwise future completion entries may overwrite ones we haven't |
| handled yet. Hold the nvme queue's poll lock while completing new CQEs to |
| prevent another thread from freeing command tags for reuse out-of-order. |
| |
| Fixes: dabcefab45d3 ("nvme: provide optimized poll function for separate poll queues") |
| Signed-off-by: Bijan Mottahedeh <bijan.mottahedeh@oracle.com> |
| Reviewed-by: Sagi Grimberg <sagi@grimberg.me> |
| Reviewed-by: Jens Axboe <axboe@kernel.dk> |
| Signed-off-by: Keith Busch <kbusch@kernel.org> |
| Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
| |
| --- |
| drivers/nvme/host/pci.c | 2 +- |
| 1 file changed, 1 insertion(+), 1 deletion(-) |
| |
| --- a/drivers/nvme/host/pci.c |
| +++ b/drivers/nvme/host/pci.c |
| @@ -1078,9 +1078,9 @@ static int nvme_poll(struct blk_mq_hw_ct |
| |
| spin_lock(&nvmeq->cq_poll_lock); |
| found = nvme_process_cq(nvmeq, &start, &end, -1); |
| + nvme_complete_cqes(nvmeq, start, end); |
| spin_unlock(&nvmeq->cq_poll_lock); |
| |
| - nvme_complete_cqes(nvmeq, start, end); |
| return found; |
| } |
| |