| From: Gavin Shan <gshan@redhat.com> |
| Subject: mm: migrate: fix THP's mapcount on isolation |
| Date: Thu, 24 Nov 2022 17:55:23 +0800 |
| |
| The issue is reported when removing memory through virtio_mem device. The |
| transparent huge page, experienced copy-on-write fault, is wrongly |
| regarded as pinned. The transparent huge page is escaped from being |
| isolated in isolate_migratepages_block(). The transparent huge page can't |
| be migrated and the corresponding memory block can't be put into offline |
| state. |
| |
| Fix it by replacing page_mapcount() with total_mapcount(). With this, the |
| transparent huge page can be isolated and migrated, and the memory block |
| can be put into offline state. Besides, The page's refcount is increased |
| a bit earlier to avoid the page is released when the check is executed. |
| |
| Link: https://lkml.kernel.org/r/20221124095523.31061-1-gshan@redhat.com |
| Fixes: 1da2f328fa64 ("mm,thp,compaction,cma: allow THP migration for CMA allocations") |
| Signed-off-by: Gavin Shan <gshan@redhat.com> |
| Reported-by: Zhenyu Zhang <zhenyzha@redhat.com> |
| Tested-by: Zhenyu Zhang <zhenyzha@redhat.com> |
| Suggested-by: David Hildenbrand <david@redhat.com> |
| Acked-by: David Hildenbrand <david@redhat.com> |
| Cc: Alistair Popple <apopple@nvidia.com> |
| Cc: Hugh Dickins <hughd@google.com> |
| Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> |
| Cc: Matthew Wilcox <willy@infradead.org> |
| Cc: William Kucharski <william.kucharski@oracle.com> |
| Cc: Zi Yan <ziy@nvidia.com> |
| Cc: <stable@vger.kernel.org> [5.7+] |
| Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
| --- |
| |
| mm/compaction.c | 22 +++++++++++----------- |
| 1 file changed, 11 insertions(+), 11 deletions(-) |
| |
| --- a/mm/compaction.c~mm-migrate-fix-thps-mapcount-on-isolation |
| +++ a/mm/compaction.c |
| @@ -985,28 +985,28 @@ isolate_migratepages_block(struct compac |
| } |
| |
| /* |
| + * Be careful not to clear PageLRU until after we're |
| + * sure the page is not being freed elsewhere -- the |
| + * page release code relies on it. |
| + */ |
| + if (unlikely(!get_page_unless_zero(page))) |
| + goto isolate_fail; |
| + |
| + /* |
| * Migration will fail if an anonymous page is pinned in memory, |
| * so avoid taking lru_lock and isolating it unnecessarily in an |
| * admittedly racy check. |
| */ |
| mapping = page_mapping(page); |
| - if (!mapping && page_count(page) > page_mapcount(page)) |
| - goto isolate_fail; |
| + if (!mapping && (page_count(page) - 1) > total_mapcount(page)) |
| + goto isolate_fail_put; |
| |
| /* |
| * Only allow to migrate anonymous pages in GFP_NOFS context |
| * because those do not depend on fs locks. |
| */ |
| if (!(cc->gfp_mask & __GFP_FS) && mapping) |
| - goto isolate_fail; |
| - |
| - /* |
| - * Be careful not to clear PageLRU until after we're |
| - * sure the page is not being freed elsewhere -- the |
| - * page release code relies on it. |
| - */ |
| - if (unlikely(!get_page_unless_zero(page))) |
| - goto isolate_fail; |
| + goto isolate_fail_put; |
| |
| /* Only take pages on LRU: a check now makes later tests safe */ |
| if (!PageLRU(page)) |
| _ |