| From: Mark Rutland <mark.rutland@arm.com> |
| Date: Wed, 3 May 2017 16:09:34 +0100 |
| Subject: arm64: ensure extension of smp_store_release value |
| |
| commit 994870bead4ab19087a79492400a5478e2906196 upstream. |
| |
| When an inline assembly operand's type is narrower than the register it |
| is allocated to, the least significant bits of the register (up to the |
| operand type's width) are valid, and any other bits are permitted to |
| contain any arbitrary value. This aligns with the AAPCS64 parameter |
| passing rules. |
| |
| Our __smp_store_release() implementation does not account for this, and |
| implicitly assumes that operands have been zero-extended to the width of |
| the type being stored to. Thus, we may store unknown values to memory |
| when the value type is narrower than the pointer type (e.g. when storing |
| a char to a long). |
| |
| This patch fixes the issue by casting the value operand to the same |
| width as the pointer operand in all cases, which ensures that the value |
| is zero-extended as we expect. We use the same union trickery as |
| __smp_load_acquire and {READ,WRITE}_ONCE() to avoid GCC complaining that |
| pointers are potentially cast to narrower width integers in unreachable |
| paths. |
| |
| A whitespace issue at the top of __smp_store_release() is also |
| corrected. |
| |
| No changes are necessary for __smp_load_acquire(). Load instructions |
| implicitly clear any upper bits of the register, and the compiler will |
| only consider the least significant bits of the register as valid |
| regardless. |
| |
| Fixes: 47933ad41a86 ("arch: Introduce smp_load_acquire(), smp_store_release()") |
| Fixes: 878a84d5a8a1 ("arm64: add missing data types in smp_load_acquire/smp_store_release") |
| Acked-by: Will Deacon <will.deacon@arm.com> |
| Signed-off-by: Mark Rutland <mark.rutland@arm.com> |
| Cc: Matthias Kaehlcke <mka@chromium.org> |
| Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> |
| [bwh: The same upstream commit was already backported to 3.16, but we |
| didn't have the 1-byte and 2-byte cases then so I dropped that part. |
| Now that we do, pick up that part again.] |
| Signed-off-by: Ben Hutchings <ben@decadent.org.uk> |
| --- |
| arch/arm64/include/asm/barrier.h | 20 +++++++++++++++----- |
| 1 file changed, 15 insertions(+), 5 deletions(-) |
| |
| --- a/arch/arm64/include/asm/barrier.h |
| +++ b/arch/arm64/include/asm/barrier.h |
| @@ -66,11 +66,15 @@ do { \ |
| switch (sizeof(*p)) { \ |
| case 1: \ |
| asm volatile ("stlrb %w1, %0" \ |
| - : "=Q" (*p) : "r" (v) : "memory"); \ |
| + : "=Q" (*p) \ |
| + : "r" (*(__u8 *)__u.__c) \ |
| + : "memory"); \ |
| break; \ |
| case 2: \ |
| asm volatile ("stlrh %w1, %0" \ |
| - : "=Q" (*p) : "r" (v) : "memory"); \ |
| + : "=Q" (*p) \ |
| + : "r" (*(__u16 *)__u.__c) \ |
| + : "memory"); \ |
| break; \ |
| case 4: \ |
| asm volatile ("stlr %w1, %0" \ |