| From ffee02adca499d06c6f3368d0c8d726d61bfbb07 Mon Sep 17 00:00:00 2001 |
| From: Sasha Levin <sashal@kernel.org> |
| Date: Thu, 6 Sep 2018 17:59:50 +0100 |
| Subject: iommu/io-pgtable-arm: Fix race handling in split_blk_unmap() |
| |
| From: Robin Murphy <robin.murphy@arm.com> |
| |
| [ Upstream commit 85c7a0f1ef624ef58173ef52ea77780257bdfe04 ] |
| |
| In removing the pagetable-wide lock, we gained the possibility of the |
| vanishingly unlikely case where we have a race between two concurrent |
| unmappers splitting the same block entry. The logic to handle this is |
| fairly straightforward - whoever loses the race frees their partial |
| next-level table and instead dereferences the winner's newly-installed |
| entry in order to fall back to a regular unmap, which intentionally |
| echoes the pre-existing case of recursively splitting a 1GB block down |
| to 4KB pages by installing a full table of 2MB blocks first. |
| |
| Unfortunately, the chump who implemented that logic failed to update the |
| condition check for that fallback, meaning that if said race occurs at |
| the last level (where the loser's unmap_idx is valid) then the unmap |
| won't actually happen. Fix that to properly account for both the race |
| and recursive cases. |
| |
| Fixes: 2c3d273eabe8 ("iommu/io-pgtable-arm: Support lockless operation") |
| Signed-off-by: Robin Murphy <robin.murphy@arm.com> |
| [will: re-jig control flow to avoid duplicate cmpxchg test] |
| Signed-off-by: Will Deacon <will.deacon@arm.com> |
| Signed-off-by: Sasha Levin <sashal@kernel.org> |
| --- |
| drivers/iommu/io-pgtable-arm.c | 9 ++++----- |
| 1 file changed, 4 insertions(+), 5 deletions(-) |
| |
| diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c |
| index e8018a308868e..17a9225283dd1 100644 |
| --- a/drivers/iommu/io-pgtable-arm.c |
| +++ b/drivers/iommu/io-pgtable-arm.c |
| @@ -551,13 +551,12 @@ static int arm_lpae_split_blk_unmap(struct arm_lpae_io_pgtable *data, |
| return 0; |
| |
| tablep = iopte_deref(pte, data); |
| + } else if (unmap_idx >= 0) { |
| + io_pgtable_tlb_add_flush(&data->iop, iova, size, size, true); |
| + return size; |
| } |
| |
| - if (unmap_idx < 0) |
| - return __arm_lpae_unmap(data, iova, size, lvl, tablep); |
| - |
| - io_pgtable_tlb_add_flush(&data->iop, iova, size, size, true); |
| - return size; |
| + return __arm_lpae_unmap(data, iova, size, lvl, tablep); |
| } |
| |
| static int __arm_lpae_unmap(struct arm_lpae_io_pgtable *data, |
| -- |
| 2.20.1 |
| |