| From 5d7c631d926b59aa16f3c56eaeb83f1036c81dc7 Mon Sep 17 00:00:00 2001 |
| From: Shaohua Li <shli@fb.com> |
| Date: Thu, 30 Jul 2015 16:24:43 -0700 |
| Subject: x86/apic: Serialize LVTT and TSC_DEADLINE writes |
| |
| From: Shaohua Li <shli@fb.com> |
| |
| commit 5d7c631d926b59aa16f3c56eaeb83f1036c81dc7 upstream. |
| |
| The APIC LVTT register is MMIO mapped but the TSC_DEADLINE register is an |
| MSR. The write to the TSC_DEADLINE MSR is not serializing, so it's not |
| guaranteed that the write to LVTT has reached the APIC before the |
| TSC_DEADLINE MSR is written. In such a case the write to the MSR is |
| ignored and as a consequence the local timer interrupt never fires. |
| |
| The SDM decribes this issue for xAPIC and x2APIC modes. The |
| serialization methods recommended by the SDM differ. |
| |
| xAPIC: |
| "1. Memory-mapped write to LVT Timer Register, setting bits 18:17 to 10b. |
| 2. WRMSR to the IA32_TSC_DEADLINE MSR a value much larger than current time-stamp counter. |
| 3. If RDMSR of the IA32_TSC_DEADLINE MSR returns zero, go to step 2. |
| 4. WRMSR to the IA32_TSC_DEADLINE MSR the desired deadline." |
| |
| x2APIC: |
| "To allow for efficient access to the APIC registers in x2APIC mode, |
| the serializing semantics of WRMSR are relaxed when writing to the |
| APIC registers. Thus, system software should not use 'WRMSR to APIC |
| registers in x2APIC mode' as a serializing instruction. Read and write |
| accesses to the APIC registers will occur in program order. A WRMSR to |
| an APIC register may complete before all preceding stores are globally |
| visible; software can prevent this by inserting a serializing |
| instruction, an SFENCE, or an MFENCE before the WRMSR." |
| |
| The xAPIC method is to just wait for the memory mapped write to hit |
| the LVTT by checking whether the MSR write has reached the hardware. |
| There is no reason why a proper MFENCE after the memory mapped write would |
| not do the same. Andi Kleen confirmed that MFENCE is sufficient for the |
| xAPIC case as well. |
| |
| Issue MFENCE before writing to the TSC_DEADLINE MSR. This can be done |
| unconditionally as all CPUs which have TSC_DEADLINE also have MFENCE |
| support. |
| |
| [ tglx: Massaged the changelog ] |
| |
| Signed-off-by: Shaohua Li <shli@fb.com> |
| Reviewed-by: Ingo Molnar <mingo@kernel.org> |
| Cc: <Kernel-team@fb.com> |
| Cc: <lenb@kernel.org> |
| Cc: <fenghua.yu@intel.com> |
| Cc: Andi Kleen <ak@linux.intel.com> |
| Cc: H. Peter Anvin <hpa@zytor.com> |
| Link: http://lkml.kernel.org/r/20150909041352.GA2059853@devbig257.prn2.facebook.com |
| Signed-off-by: Thomas Gleixner <tglx@linutronix.de> |
| Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
| |
| --- |
| arch/x86/kernel/apic/apic.c | 7 +++++++ |
| 1 file changed, 7 insertions(+) |
| |
| --- a/arch/x86/kernel/apic/apic.c |
| +++ b/arch/x86/kernel/apic/apic.c |
| @@ -359,6 +359,13 @@ static void __setup_APIC_LVTT(unsigned i |
| apic_write(APIC_LVTT, lvtt_value); |
| |
| if (lvtt_value & APIC_LVT_TIMER_TSCDEADLINE) { |
| + /* |
| + * See Intel SDM: TSC-Deadline Mode chapter. In xAPIC mode, |
| + * writing to the APIC LVTT and TSC_DEADLINE MSR isn't serialized. |
| + * According to Intel, MFENCE can do the serialization here. |
| + */ |
| + asm volatile("mfence" : : : "memory"); |
| + |
| printk_once(KERN_DEBUG "TSC deadline timer enabled\n"); |
| return; |
| } |