| From a8d1179e0149f29467a483a0cf5be2b8d655edc2 Mon Sep 17 00:00:00 2001 |
| From: Sasha Levin <sashal@kernel.org> |
| Date: Fri, 9 Oct 2020 16:42:25 +0200 |
| Subject: x86/alternative: Don't call text_poke() in lazy TLB mode |
| |
| From: Juergen Gross <jgross@suse.com> |
| |
| [ Upstream commit abee7c494d8c41bb388839bccc47e06247f0d7de ] |
| |
| When running in lazy TLB mode the currently active page tables might |
| be the ones of a previous process, e.g. when running a kernel thread. |
| |
| This can be problematic in case kernel code is being modified via |
| text_poke() in a kernel thread, and on another processor exit_mmap() |
| is active for the process which was running on the first cpu before |
| the kernel thread. |
| |
| As text_poke() is using a temporary address space and the former |
| address space (obtained via cpu_tlbstate.loaded_mm) is restored |
| afterwards, there is a race possible in case the cpu on which |
| exit_mmap() is running wants to make sure there are no stale |
| references to that address space on any cpu active (this e.g. is |
| required when running as a Xen PV guest, where this problem has been |
| observed and analyzed). |
| |
| In order to avoid that, drop off TLB lazy mode before switching to the |
| temporary address space. |
| |
| Fixes: cefa929c034eb5d ("x86/mm: Introduce temporary mm structs") |
| Signed-off-by: Juergen Gross <jgross@suse.com> |
| Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> |
| Link: https://lkml.kernel.org/r/20201009144225.12019-1-jgross@suse.com |
| Signed-off-by: Sasha Levin <sashal@kernel.org> |
| --- |
| arch/x86/kernel/alternative.c | 9 +++++++++ |
| 1 file changed, 9 insertions(+) |
| |
| diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c |
| index cdaab30880b91..cd6be6f143e85 100644 |
| --- a/arch/x86/kernel/alternative.c |
| +++ b/arch/x86/kernel/alternative.c |
| @@ -807,6 +807,15 @@ static inline temp_mm_state_t use_temporary_mm(struct mm_struct *mm) |
| temp_mm_state_t temp_state; |
| |
| lockdep_assert_irqs_disabled(); |
| + |
| + /* |
| + * Make sure not to be in TLB lazy mode, as otherwise we'll end up |
| + * with a stale address space WITHOUT being in lazy mode after |
| + * restoring the previous mm. |
| + */ |
| + if (this_cpu_read(cpu_tlbstate.is_lazy)) |
| + leave_mm(smp_processor_id()); |
| + |
| temp_state.mm = this_cpu_read(cpu_tlbstate.loaded_mm); |
| switch_mm_irqs_off(NULL, mm, current); |
| |
| -- |
| 2.27.0 |
| |