| From f8303c2582b889351e261ff18c4d8eb197a77db2 Mon Sep 17 00:00:00 2001 |
| From: Waiman Long <Waiman.Long@hp.com> |
| Date: Wed, 6 Aug 2014 16:05:36 -0700 |
| Subject: mm, thp: move invariant bug check out of loop in __split_huge_page_map |
| |
| From: Waiman Long <Waiman.Long@hp.com> |
| |
| commit f8303c2582b889351e261ff18c4d8eb197a77db2 upstream. |
| |
| In __split_huge_page_map(), the check for page_mapcount(page) is |
| invariant within the for loop. Because of the fact that the macro is |
| implemented using atomic_read(), the redundant check cannot be optimized |
| away by the compiler leading to unnecessary read to the page structure. |
| |
| This patch moves the invariant bug check out of the loop so that it will |
| be done only once. On a 3.16-rc1 based kernel, the execution time of a |
| microbenchmark that broke up 1000 transparent huge pages using munmap() |
| had an execution time of 38,245us and 38,548us with and without the |
| patch respectively. The performance gain is about 1%. |
| |
| Signed-off-by: Waiman Long <Waiman.Long@hp.com> |
| Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> |
| Cc: Andrea Arcangeli <aarcange@redhat.com> |
| Cc: Mel Gorman <mgorman@suse.de> |
| Cc: Rik van Riel <riel@redhat.com> |
| Cc: Scott J Norton <scott.norton@hp.com> |
| Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
| Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
| Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
| |
| --- |
| mm/huge_memory.c | 4 ++-- |
| 1 file changed, 2 insertions(+), 2 deletions(-) |
| |
| --- a/mm/huge_memory.c |
| +++ b/mm/huge_memory.c |
| @@ -1775,6 +1775,8 @@ static int __split_huge_page_map(struct |
| if (pmd) { |
| pgtable = pgtable_trans_huge_withdraw(mm, pmd); |
| pmd_populate(mm, &_pmd, pgtable); |
| + if (pmd_write(*pmd)) |
| + BUG_ON(page_mapcount(page) != 1); |
| |
| haddr = address; |
| for (i = 0; i < HPAGE_PMD_NR; i++, haddr += PAGE_SIZE) { |
| @@ -1784,8 +1786,6 @@ static int __split_huge_page_map(struct |
| entry = maybe_mkwrite(pte_mkdirty(entry), vma); |
| if (!pmd_write(*pmd)) |
| entry = pte_wrprotect(entry); |
| - else |
| - BUG_ON(page_mapcount(page) != 1); |
| if (!pmd_young(*pmd)) |
| entry = pte_mkold(entry); |
| if (pmd_numa(*pmd)) |