| From 2771739a7162782c0aa6424b2e3dd874e884a15d Mon Sep 17 00:00:00 2001 |
| From: Muchun Song <songmuchun@bytedance.com> |
| Date: Tue, 22 Mar 2022 14:41:56 -0700 |
| Subject: mm: fix missing cache flush for all tail pages of compound page |
| |
| From: Muchun Song <songmuchun@bytedance.com> |
| |
| commit 2771739a7162782c0aa6424b2e3dd874e884a15d upstream. |
| |
| The D-cache maintenance inside move_to_new_page() only consider one |
| page, there is still D-cache maintenance issue for tail pages of |
| compound page (e.g. THP or HugeTLB). |
| |
| THP migration is only enabled on x86_64, ARM64 and powerpc, while |
| powerpc and arm64 need to maintain the consistency between I-Cache and |
| D-Cache, which depends on flush_dcache_page() to maintain the |
| consistency between I-Cache and D-Cache. |
| |
| But there is no issues on arm64 and powerpc since they already considers |
| the compound page cache flushing in their icache flush function. |
| HugeTLB migration is enabled on arm, arm64, mips, parisc, powerpc, |
| riscv, s390 and sh, while arm has handled the compound page cache flush |
| in flush_dcache_page(), but most others do not. |
| |
| In theory, the issue exists on many architectures. Fix this by not |
| using flush_dcache_folio() since it is not backportable. |
| |
| Link: https://lkml.kernel.org/r/20220210123058.79206-3-songmuchun@bytedance.com |
| Fixes: 290408d4a250 ("hugetlb: hugepage migration core") |
| Signed-off-by: Muchun Song <songmuchun@bytedance.com> |
| Reviewed-by: Zi Yan <ziy@nvidia.com> |
| Cc: Axel Rasmussen <axelrasmussen@google.com> |
| Cc: David Rientjes <rientjes@google.com> |
| Cc: Fam Zheng <fam.zheng@bytedance.com> |
| Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> |
| Cc: Lars Persson <lars.persson@axis.com> |
| Cc: Mike Kravetz <mike.kravetz@oracle.com> |
| Cc: Peter Xu <peterx@redhat.com> |
| Cc: Xiongchun Duan <duanxiongchun@bytedance.com> |
| Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
| Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
| Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
| --- |
| mm/migrate.c | 7 +++++-- |
| 1 file changed, 5 insertions(+), 2 deletions(-) |
| |
| --- a/mm/migrate.c |
| +++ b/mm/migrate.c |
| @@ -994,9 +994,12 @@ static int move_to_new_page(struct page |
| if (!PageMappingFlags(page)) |
| page->mapping = NULL; |
| |
| - if (likely(!is_zone_device_page(newpage))) |
| - flush_dcache_page(newpage); |
| + if (likely(!is_zone_device_page(newpage))) { |
| + int i, nr = compound_nr(newpage); |
| |
| + for (i = 0; i < nr; i++) |
| + flush_dcache_page(newpage + i); |
| + } |
| } |
| out: |
| return rc; |