mm: COW: skip the page lock in the COW copy path
The only guarantee required to be allowed to copy is that mapcount is
not equal 1. That is enough not to risk overestimating mapcount.
This should retain the performance improvement reported below for the
COW copy path, but without introducing any regression to the page
reuse accuracy.
https://lkml.kernel.org/r/20200914024321.GG26874@shao2-debian
In addition this optimizes the THP COW with the same logic.
lkp@lists.01.org, lkp@intel.com, https://github.com/intel/lkp-tests.git reports:
---------
FYI, we noticed a 55.2% improvement of vm-scalability.throughput due to commit:
commit: 599aa62474f51a470408b28fd4365320a5357aca ("mm: COW: skip the page lock in the COW copy path")
https://git.kernel.org/cgit/linux/kernel/git/andrea/aa.git master
in testcase: vm-scalability
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
runtime: 300s
size: 8T
test: anon-cow-seq
cpufreq_governor: performance
ucode: 0x5003006
vm-scalability.throughput
6e+07 +-------------------------------------------------------------------+
| OO O O O O O O O O O O O |
5e+07 |-+ O O O OO O O O O |
| +. +. O .+. +.O +. .+ O |
|.++ : + +.+.++. .+.++.+.+.++ +.+ +.+.+ +.+ +.+.+.+ |
4e+07 |-+: : + : |
| : : +.+.+.++.|
3e+07 |-+ : : |
| : : |
2e+07 |-+ : : |
| : : |
| : : |
1e+07 |-+ : |
| : |
0 +-------------------------------------------------------------------+
---------
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
2 files changed