ice-vfio-pci: add ice VFIO PCI live migration driver
Add the ice-vfio-pci driver module which enables live migration support via
the vfio_migration_ops for the ice E800 series hardware.
To use this module, you can create VFs in the usual way and then unbind
them from iavf, and bind them to ice-vfio-pci:
echo 2 >/sys/class/net/enp175s0f0np0/device/sriov_numvfs
echo "0000:af:01.0" >/sys/bus/pci/drivers/iavf/unbind
echo "0000:af:01.1" >/sys/bus/pci/drivers/iavf/unbind
modprobe ice_vfio_pci
echo "8086 1889" >/sys/bus/pci/drivers/ice-vfio-pci/new_id
I've tested with QEMU using the "enable-migration=on" and
"x-pre-copy-dirty-page-tracking=off" settings, as we do not currently
support dirty page tracking.
The initial host QEMU instance is launched as usual, while the target QEMU
instance is launched with the -incoming tcp:localhost:4444 option.
To initiate migration you can issue the migration command from the QEMU
console:
migrate tcp:localhost:4444
The ice-vfio-pci driver connects to the ice driver using the interface
defined in <linux/net/intel/ice_migration.h>. The migration driver
initializes by calling ice_migration_init_dev(). To save device state, the
VF is paused using ice_migration_suspend_dev(), and then state is captured
by ice_migration_save_devstate().
Some information about the VF must be saved during device suspend, as
otherwise the data could be lost when stopping the device.
For this reason, the ice_migration_suspend_dev() function takes a boolean
indicating whether state should be saved. The VFIO migration state machine
must suspend the initial device when stopping, but also suspends the target
device when resuming. In the resume case, we do not need to save the state,
so this can be elided when the VFIO state machine is transitioning to the
resuming state.
Note that full support is not functional until the PCI .reset_done handler
is implemented in a following change. This was split out in order to better
callout and explain the locking mechanism due to the complexity required to
avoid ABBA locking violations.
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
6 files changed
