| From b63f20a778c88b6a04458ed6ffc69da953d3a109 Mon Sep 17 00:00:00 2001 |
| From: Sean Christopherson <sean.j.christopherson@intel.com> |
| Date: Thu, 22 Aug 2019 14:11:22 -0700 |
| Subject: x86/retpoline: Don't clobber RFLAGS during CALL_NOSPEC on i386 |
| |
| From: Sean Christopherson <sean.j.christopherson@intel.com> |
| |
| commit b63f20a778c88b6a04458ed6ffc69da953d3a109 upstream. |
| |
| Use 'lea' instead of 'add' when adjusting %rsp in CALL_NOSPEC so as to |
| avoid clobbering flags. |
| |
| KVM's emulator makes indirect calls into a jump table of sorts, where |
| the destination of the CALL_NOSPEC is a small blob of code that performs |
| fast emulation by executing the target instruction with fixed operands. |
| |
| adcb_al_dl: |
| 0x000339f8 <+0>: adc %dl,%al |
| 0x000339fa <+2>: ret |
| |
| A major motiviation for doing fast emulation is to leverage the CPU to |
| handle consumption and manipulation of arithmetic flags, i.e. RFLAGS is |
| both an input and output to the target of CALL_NOSPEC. Clobbering flags |
| results in all sorts of incorrect emulation, e.g. Jcc instructions often |
| take the wrong path. Sans the nops... |
| |
| asm("push %[flags]; popf; " CALL_NOSPEC " ; pushf; pop %[flags]\n" |
| 0x0003595a <+58>: mov 0xc0(%ebx),%eax |
| 0x00035960 <+64>: mov 0x60(%ebx),%edx |
| 0x00035963 <+67>: mov 0x90(%ebx),%ecx |
| 0x00035969 <+73>: push %edi |
| 0x0003596a <+74>: popf |
| 0x0003596b <+75>: call *%esi |
| 0x000359a0 <+128>: pushf |
| 0x000359a1 <+129>: pop %edi |
| 0x000359a2 <+130>: mov %eax,0xc0(%ebx) |
| 0x000359b1 <+145>: mov %edx,0x60(%ebx) |
| |
| ctxt->eflags = (ctxt->eflags & ~EFLAGS_MASK) | (flags & EFLAGS_MASK); |
| 0x000359a8 <+136>: mov -0x10(%ebp),%eax |
| 0x000359ab <+139>: and $0x8d5,%edi |
| 0x000359b4 <+148>: and $0xfffff72a,%eax |
| 0x000359b9 <+153>: or %eax,%edi |
| 0x000359bd <+157>: mov %edi,0x4(%ebx) |
| |
| For the most part this has gone unnoticed as emulation of guest code |
| that can trigger fast emulation is effectively limited to MMIO when |
| running on modern hardware, and MMIO is rarely, if ever, accessed by |
| instructions that affect or consume flags. |
| |
| Breakage is almost instantaneous when running with unrestricted guest |
| disabled, in which case KVM must emulate all instructions when the guest |
| has invalid state, e.g. when the guest is in Big Real Mode during early |
| BIOS. |
| |
| Fixes: 776b043848fd2 ("x86/retpoline: Add initial retpoline support") |
| Fixes: 1a29b5b7f347a ("KVM: x86: Make indirect calls in emulator speculation safe") |
| Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com> |
| Signed-off-by: Thomas Gleixner <tglx@linutronix.de> |
| Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> |
| Cc: stable@vger.kernel.org |
| Link: https://lkml.kernel.org/r/20190822211122.27579-1-sean.j.christopherson@intel.com |
| Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> |
| |
| --- |
| arch/x86/include/asm/nospec-branch.h | 2 +- |
| 1 file changed, 1 insertion(+), 1 deletion(-) |
| |
| --- a/arch/x86/include/asm/nospec-branch.h |
| +++ b/arch/x86/include/asm/nospec-branch.h |
| @@ -202,7 +202,7 @@ |
| " lfence;\n" \ |
| " jmp 902b;\n" \ |
| " .align 16\n" \ |
| - "903: addl $4, %%esp;\n" \ |
| + "903: lea 4(%%esp), %%esp;\n" \ |
| " pushl %[thunk_target];\n" \ |
| " ret;\n" \ |
| " .align 16\n" \ |