ice-vfio-pci: implement PCI .reset_done handling
Add an implementation callback for the PCI .reset_done handler to enable
cleanup after a PCI reset. This function has one rather nasty locking
complexity due to the way the various locks interact.
The VFIO layer holds the mm_lock across a reset, and a naive implementation
which just takes the state mutex would trigger a simple ABBA deadlock
between the state_mutex and the mm_lock.
To avoid this, allow deferring handling cleanup after a PCI reset until the
current thread holding the state_mutex exits.
This is done through adding a reset_lock spinlock and a needs_reset
boolean. All flows which previously simply released the state_mutex now
call a specialized ice_vfio_pci_state_mutex_unlock() handler.
This handler acquires the reset_lock, and checks if a reset was deferred.
If so, the reset_lock is released, cleanup is handled, then the reset_lock
is reacquired and the thread loops to check for another deferred reset.
Eventually the needs_reset is false, and the function exits by releasing
the state_mutex and then the deferred reset_lock.
The actual reset_done implementation acquires the reset lock, sets
needs_reset to true, then uses try_lock to acquire the state mutex. If
it fails to acquire the state mutex, this means another thread is handling
business and will perform the deferred reset cleanup as part of unlocking
the state mutex. Finally, if the reset_done does acquire the state mutex,
it simply unlocks using the ice_vfio_pci_state_mutex_unlock helper which
will immediately handle the "deferred" reset.
This is complicated, but is similar to code in other VFIO migration drivers
including the mlx5 driver and logic in the virtiovf migration code.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
1 file changed
