virtio-pci: Deconfigure interrupts on device shutdown
Using kexec in a guest that uses virtio-net results in the
following splat:
root@canarsie:~# kexec -l zImage-4.10
root@canarsie:~# kexec -e
[ 56.686857] kexec_core: Starting new kernel
[ 56.692674] Disabling non-boot CPUs ...
Message from syslogd@canarsie at Mar 2 11:59:33 ...
kernel:[ 56.686857] kexec_core: Starting new kernel
[ 56.747522] ------------[ cut here ]------------
[ 56.751475] WARNING: CPU: 1 PID: 13 at drivers/pci/msi.c:1261 pci_irq_vector+0x118/0x11c
[ 56.757689] Modules linked in:
[ 56.759252] CPU: 1 PID: 13 Comm: cpuhp/1 Not tainted 4.10.0+ #6933
[ 56.762937] Hardware name: Generic DT based system
[ 56.766791] [<c022fb78>] (unwind_backtrace) from [<c0229d8c>] (show_stack+0x18/0x1c)
[ 56.770767] [<c0229d8c>] (show_stack) from [<c05192f4>] (dump_stack+0x84/0x98)
[ 56.774186] [<c05192f4>] (dump_stack) from [<c023d940>] (__warn+0xf4/0x10c)
[ 56.778590] [<c023d940>] (__warn) from [<c023da20>] (warn_slowpath_null+0x28/0x30)
[ 56.785325] [<c023da20>] (warn_slowpath_null) from [<c0578edc>] (pci_irq_vector+0x118/0x11c)
[ 56.792024] [<c0578edc>] (pci_irq_vector) from [<c059da24>] (vp_set_vq_affinity+0x54/0xa8)
[ 56.796682] [<c059da24>] (vp_set_vq_affinity) from [<c067dbc8>] (virtnet_clean_affinity.part.1+0x54/0x94)
[ 56.804829] [<c067dbc8>] (virtnet_clean_affinity.part.1) from [<c067dc30>] (virtnet_cpu_down_prep+0x28/0x30)
[ 56.813232] [<c067dc30>] (virtnet_cpu_down_prep) from [<c023e434>] (cpuhp_invoke_callback+0x434/0x63c)
[ 56.821302] [<c023e434>] (cpuhp_invoke_callback) from [<c023e720>] (cpuhp_down_callbacks+0x50/0x7c)
[ 56.828660] [<c023e720>] (cpuhp_down_callbacks) from [<c023e918>] (cpuhp_thread_fun+0xa4/0x10c)
[ 56.835121] [<c023e918>] (cpuhp_thread_fun) from [<c0260da8>] (smpboot_thread_fn+0x158/0x270)
[ 56.842591] [<c0260da8>] (smpboot_thread_fn) from [<c025ce94>] (kthread+0x114/0x144)
[ 56.849116] [<c025ce94>] (kthread) from [<c0225948>] (ret_from_fork+0x14/0x2c)
[ 56.855490] ---[ end trace b08a8d277c87da3d ]---
The cause of the splat is a bit odd:
- vp_dev->msix_enabled is set to 1
- pci_dev->msix_enabled is set to 0...
That's because kexec tries to shutdown all devices, as shown in
this stack trace:
[ 32.790404] [<c057abd4>] (pci_msix_shutdown) from [<c056ca5c>] (pci_device_shutdown+0x4c/0x78)
[ 32.796780] [<c056ca5c>] (pci_device_shutdown) from [<c05ee2e4>] (device_shutdown+0x13c/0x1c0)
[ 32.803373] [<c05ee2e4>] (device_shutdown) from [<c02c4594>] (kernel_kexec+0x60/0x88)
[ 32.807913] [<c02c4594>] (kernel_kexec) from [<c02601c4>] (SyS_reboot+0x1a0/0x1d8)
[ 32.812292] [<c02601c4>] (SyS_reboot) from [<c02258a0>] (ret_fast_syscall+0x0/0x34)
and virtio-pci is completely oblivious that MSIs are now disabled.
Calling pci_irq_vector from vp_set_vq_affinity ends up generating
the above warning.
In order to solve things in a nicer way, let's implement a shutdown
callback that deals with this, calling vp_del_vqs to deconfigure
all interrupts. This ensures that there is no additional surprise
when the CPU hotplug starts tearing things down.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
1 file changed
