| From foo@baz Tue Aug 14 16:14:56 CEST 2018 |
| From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> |
| Date: Fri, 8 Sep 2017 16:10:46 -0700 |
| Subject: mm: x86: move _PAGE_SWP_SOFT_DIRTY from bit 7 to bit 1 |
| |
| From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> |
| |
| commit eee4818baac0f2b37848fdf90e4b16430dc536ac upstream |
| |
| _PAGE_PSE is used to distinguish between a truly non-present |
| (_PAGE_PRESENT=0) PMD, and a PMD which is undergoing a THP split and |
| should be treated as present. |
| |
| But _PAGE_SWP_SOFT_DIRTY currently uses the _PAGE_PSE bit, which would |
| cause confusion between one of those PMDs undergoing a THP split, and a |
| soft-dirty PMD. Dropping _PAGE_PSE check in pmd_present() does not work |
| well, because it can hurt optimization of tlb handling in thp split. |
| |
| Thus, we need to move the bit. |
| |
| In the current kernel, bits 1-4 are not used in non-present format since |
| commit 00839ee3b299 ("x86/mm: Move swap offset/type up in PTE to work |
| around erratum"). So let's move _PAGE_SWP_SOFT_DIRTY to bit 1. Bit 7 |
| is used as reserved (always clear), so please don't use it for other |
| purpose. |
| |
| [dwmw2: Pulled in to 4.9 backport to support L1TF changes] |
| |
| Link: http://lkml.kernel.org/r/20170717193955.20207-3-zi.yan@sent.com |
| Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> |
| Signed-off-by: Zi Yan <zi.yan@cs.rutgers.edu> |
| Acked-by: Dave Hansen <dave.hansen@intel.com> |
| Cc: "H. Peter Anvin" <hpa@zytor.com> |
| Cc: Anshuman Khandual <khandual@linux.vnet.ibm.com> |
| Cc: David Nellans <dnellans@nvidia.com> |
| Cc: Ingo Molnar <mingo@elte.hu> |
| Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> |
| Cc: Mel Gorman <mgorman@techsingularity.net> |
| Cc: Minchan Kim <minchan@kernel.org> |
| Cc: Thomas Gleixner <tglx@linutronix.de> |
| Cc: Vlastimil Babka <vbabka@suse.cz> |
| Cc: Andrea Arcangeli <aarcange@redhat.com> |
| Cc: Michal Hocko <mhocko@kernel.org> |
| Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
| Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> |
| Signed-off-by: David Woodhouse <dwmw@amazon.co.uk> |
| Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
| --- |
| arch/x86/include/asm/pgtable_64.h | 12 +++++++++--- |
| arch/x86/include/asm/pgtable_types.h | 10 +++++----- |
| 2 files changed, 14 insertions(+), 8 deletions(-) |
| |
| --- a/arch/x86/include/asm/pgtable_64.h |
| +++ b/arch/x86/include/asm/pgtable_64.h |
| @@ -166,15 +166,21 @@ static inline int pgd_large(pgd_t pgd) { |
| /* |
| * Encode and de-code a swap entry |
| * |
| - * | ... | 11| 10| 9|8|7|6|5| 4| 3|2|1|0| <- bit number |
| - * | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U|W|P| <- bit names |
| - * | OFFSET (14->63) | TYPE (9-13) |0|X|X|X| X| X|X|X|0| <- swp entry |
| + * | ... | 11| 10| 9|8|7|6|5| 4| 3|2| 1|0| <- bit number |
| + * | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U| W|P| <- bit names |
| + * | OFFSET (14->63) | TYPE (9-13) |0|0|X|X| X| X|X|SD|0| <- swp entry |
| * |
| * G (8) is aliased and used as a PROT_NONE indicator for |
| * !present ptes. We need to start storing swap entries above |
| * there. We also need to avoid using A and D because of an |
| * erratum where they can be incorrectly set by hardware on |
| * non-present PTEs. |
| + * |
| + * SD (1) in swp entry is used to store soft dirty bit, which helps us |
| + * remember soft dirty over page migration |
| + * |
| + * Bit 7 in swp entry should be 0 because pmd_present checks not only P, |
| + * but also L and G. |
| */ |
| #define SWP_TYPE_FIRST_BIT (_PAGE_BIT_PROTNONE + 1) |
| #define SWP_TYPE_BITS 5 |
| --- a/arch/x86/include/asm/pgtable_types.h |
| +++ b/arch/x86/include/asm/pgtable_types.h |
| @@ -97,15 +97,15 @@ |
| /* |
| * Tracking soft dirty bit when a page goes to a swap is tricky. |
| * We need a bit which can be stored in pte _and_ not conflict |
| - * with swap entry format. On x86 bits 6 and 7 are *not* involved |
| - * into swap entry computation, but bit 6 is used for nonlinear |
| - * file mapping, so we borrow bit 7 for soft dirty tracking. |
| + * with swap entry format. On x86 bits 1-4 are *not* involved |
| + * into swap entry computation, but bit 7 is used for thp migration, |
| + * so we borrow bit 1 for soft dirty tracking. |
| * |
| * Please note that this bit must be treated as swap dirty page |
| - * mark if and only if the PTE has present bit clear! |
| + * mark if and only if the PTE/PMD has present bit clear! |
| */ |
| #ifdef CONFIG_MEM_SOFT_DIRTY |
| -#define _PAGE_SWP_SOFT_DIRTY _PAGE_PSE |
| +#define _PAGE_SWP_SOFT_DIRTY _PAGE_RW |
| #else |
| #define _PAGE_SWP_SOFT_DIRTY (_AT(pteval_t, 0)) |
| #endif |