blob: df6e7afa397415c8f8f3b2e0f03bc013a422471a [file] [log] [blame]
From: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Subject: fix folio conversion in __dma_page_dev_to_cpu()
Date: Wed, 23 Aug 2023 20:18:52 +0100
Russell and Marek pointed out some assumptions I was making about how sg
lists work; eg that they are limited to 2GB and that the initial offset
lies within the first page (or at least within the first folio that a page
belongs to). While I think those assumptions are true, it's not too hard
to write a version which does not have those assumptions and also
calculates folio_size() only once per loop iteration.
Link: https://lkml.kernel.org/r/20230823191852.1556561-1-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Mike Rapoport (IBM) <rppt@kernel.org>
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
arch/arm/mm/dma-mapping.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
--- a/arch/arm/mm/dma-mapping.c~arm-implement-the-new-page-table-range-api-fix-2
+++ a/arch/arm/mm/dma-mapping.c
@@ -695,7 +695,6 @@ static void __dma_page_cpu_to_dev(struct
static void __dma_page_dev_to_cpu(struct page *page, unsigned long off,
size_t size, enum dma_data_direction dir)
{
- struct folio *folio = page_folio(page);
phys_addr_t paddr = page_to_phys(page) + off;
/* FIXME: non-speculating: not required */
@@ -710,18 +709,19 @@ static void __dma_page_dev_to_cpu(struct
* Mark the D-cache clean for these pages to avoid extra flushing.
*/
if (dir != DMA_TO_DEVICE && size >= PAGE_SIZE) {
- ssize_t left = size;
+ struct folio *folio = pfn_folio(paddr / PAGE_SIZE);
size_t offset = offset_in_folio(folio, paddr);
- if (offset) {
- left -= folio_size(folio) - offset;
- folio = folio_next(folio);
- }
+ for (;;) {
+ size_t sz = folio_size(folio) - offset;
- while (left >= (ssize_t)folio_size(folio)) {
- left -= folio_size(folio);
- set_bit(PG_dcache_clean, &folio->flags);
- if (!left)
+ if (size < sz)
+ break;
+ if (!offset)
+ set_bit(PG_dcache_clean, &folio->flags);
+ offset = 0;
+ size -= sz;
+ if (!size)
break;
folio = folio_next(folio);
}
_