mm: COW: skip the page lock in the COW copy path

The only guarantee required to be allowed to copy is that mapcount is
not equal 1. That is enough not to risk overestimating mapcount.

This should retain the performance improvement reported below for the
COW copy path, but without introducing any regression to the page
reuse accuracy.

https://lkml.kernel.org/r/20200914024321.GG26874@shao2-debian

In addition this optimizes the THP COW with the same logic.

lkp@lists.01.org, lkp@intel.com, https://github.com/intel/lkp-tests.git reports:

---------
FYI, we noticed a 55.2% improvement of vm-scalability.throughput due to commit:

commit: 599aa62474f51a470408b28fd4365320a5357aca ("mm: COW: skip the page lock in the COW copy path")
https://git.kernel.org/cgit/linux/kernel/git/andrea/aa.git master

in testcase: vm-scalability
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:

        runtime: 300s
        size: 8T
        test: anon-cow-seq
        cpufreq_governor: performance
        ucode: 0x5003006

                              vm-scalability.throughput

  6e+07 +-------------------------------------------------------------------+
        | OO O    O      O O    O   O    O    O     O    O  O               |
  5e+07 |-+      O      O         O   OO   O O    O        O                |
        |      +. +.         O          .+.   +.O    +.   .+  O             |
        |.++   : +  +.+.++. .+.++.+.+.++   +.+  +.+.+  +.+  +.+.+.+         |
  4e+07 |-+:   :           +                                       :        |
        |  :  :                                                    +.+.+.++.|
  3e+07 |-+ : :                                                             |
        |   : :                                                             |
  2e+07 |-+ : :                                                             |
        |   : :                                                             |
        |   : :                                                             |
  1e+07 |-+  :                                                              |
        |    :                                                              |
      0 +-------------------------------------------------------------------+
---------

Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
2 files changed