History

Merge tag 'vfio-v5.18-rc1' of https://github.com/awilliam/linux-vfio

Pull VFIO updates from Alex Williamson: - Introduce new device migration uAPI and implement device specific mlx5 vfio-pci variant driver supporting new protocol (Jason Gunthorpe, Yishai Hadas, Leon Romanovsky) - New HiSilicon acc vfio-pci variant driver, also supporting migration interface (Shameer Kolothum, Longfang Liu) - D3hot fixes for vfio-pci-core (Abhishek Sahu) - Document new vfio-pci variant driver acceptance criteria (Alex Williamson) - Fix UML build unresolved ioport_{un}map() functions (Alex Williamson) - Fix MAINTAINERS due to header movement (Lukas Bulwahn) * tag 'vfio-v5.18-rc1' of https://github.com/awilliam/linux-vfio: (31 commits) vfio-pci: Provide reviewers and acceptance criteria for variant drivers MAINTAINERS: adjust entry for header movement in hisilicon qm driver hisi_acc_vfio_pci: Use its own PCI reset_done error handler hisi_acc_vfio_pci: Add support for VFIO live migration crypto: hisilicon/qm: Set the VF QM state register hisi_acc_vfio_pci: Add helper to retrieve the struct pci_driver hisi_acc_vfio_pci: Restrict access to VF dev BAR2 migration region hisi_acc_vfio_pci: add new vfio_pci driver for HiSilicon ACC devices hisi_acc_qm: Move VF PCI device IDs to common header crypto: hisilicon/qm: Move few definitions to common header crypto: hisilicon/qm: Move the QM header to include/linux vfio/mlx5: Fix to not use 0 as NULL pointer PCI/IOV: Fix wrong kernel-doc identifier vfio/mlx5: Use its own PCI reset_done error handler vfio/pci: Expose vfio_pci_core_aer_err_detected() vfio/mlx5: Implement vfio_pci driver for mlx5 devices vfio/mlx5: Expose migration commands over mlx5 device vfio: Remove migration protocol v1 documentation vfio: Extend the device migration protocol with RUNNING_P2P vfio: Define device migration protocol v2 ...

Linus Torvalds committed 4 years ago

Commit 7403e6d2 parents 66711cf f621eb140 files · +3558 −358Browse files

Showing the changes to include/uapi/linux (1 of 40 files). Show all

Unified · Split

include/uapi/linux/vfio.h+195−211
@@ -323,7 +323,7 @@ struct vfio_region_info_cap_type { · 322 unmodified lines
323323 #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0xffff)
324324 #define VFIO_REGION_TYPE_GFX (1)
325325 #define VFIO_REGION_TYPE_CCW (2)
326−#define VFIO_REGION_TYPE_MIGRATION (3)
326+#define VFIO_REGION_TYPE_MIGRATION_DEPRECATED (3)
327327
328328 /* sub-types for VFIO_REGION_TYPE_PCI_* */
329329
@@ -405,225 +405,29 @@ struct vfio_region_gfx_edid { · 75 unmodified lines
405405 #define VFIO_REGION_SUBTYPE_CCW_CRW (3)
406406
407407 /* sub-types for VFIO_REGION_TYPE_MIGRATION */
408−#define VFIO_REGION_SUBTYPE_MIGRATION (1)
409−
410−/*
411− * The structure vfio_device_migration_info is placed at the 0th offset of
412− * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device related
413− * migration information. Field accesses from this structure are only supported
414− * at their native width and alignment. Otherwise, the result is undefined and
415− * vendor drivers should return an error.
416− *
417− * device_state: (read/write)
418− * - The user application writes to this field to inform the vendor driver
419− * about the device state to be transitioned to.
420− * - The vendor driver should take the necessary actions to change the
421− * device state. After successful transition to a given state, the
422− * vendor driver should return success on write(device_state, state)
423− * system call. If the device state transition fails, the vendor driver
424− * should return an appropriate -errno for the fault condition.
425− * - On the user application side, if the device state transition fails,
426− * that is, if write(device_state, state) returns an error, read
427− * device_state again to determine the current state of the device from
428− * the vendor driver.
429− * - The vendor driver should return previous state of the device unless
430− * the vendor driver has encountered an internal error, in which case
431− * the vendor driver may report the device_state VFIO_DEVICE_STATE_ERROR.
432− * - The user application must use the device reset ioctl to recover the
433− * device from VFIO_DEVICE_STATE_ERROR state. If the device is
434− * indicated to be in a valid device state by reading device_state, the
435− * user application may attempt to transition the device to any valid
436− * state reachable from the current state or terminate itself.
437− *
438− * device_state consists of 3 bits:
439− * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is clear,
440− * it indicates the _STOP state. When the device state is changed to
441− * _STOP, driver should stop the device before write() returns.
442− * - If bit 1 is set, it indicates the _SAVING state, which means that the
443− * driver should start gathering device state information that will be
444− * provided to the VFIO user application to save the device's state.
445− * - If bit 2 is set, it indicates the _RESUMING state, which means that
446− * the driver should prepare to resume the device. Data provided through
447− * the migration region should be used to resume the device.
448− * Bits 3 - 31 are reserved for future use. To preserve them, the user
449− * application should perform a read-modify-write operation on this
450− * field when modifying the specified bits.
451− *
452− * +------- _RESUMING
453− * |+------ _SAVING
454− * ||+----- _RUNNING
455− * |||
456− * 000b => Device Stopped, not saving or resuming
457− * 001b => Device running, which is the default state
458− * 010b => Stop the device & save the device state, stop-and-copy state
459− * 011b => Device running and save the device state, pre-copy state
460− * 100b => Device stopped and the device state is resuming
461− * 101b => Invalid state
462− * 110b => Error state
463− * 111b => Invalid state
464− *
465− * State transitions:
466− *
467− * _RESUMING _RUNNING Pre-copy Stop-and-copy _STOP
468− * (100b) (001b) (011b) (010b) (000b)
469− * 0. Running or default state
470− * |
471− *
472− * 1. Normal Shutdown (optional)
473− * |------------------------------------->|
474− *
475− * 2. Save the state or suspend
476− * |------------------------->|---------->|
477− *
478− * 3. Save the state during live migration
479− * |----------->|------------>|---------->|
480− *
481− * 4. Resuming
482− * |<---------|
483− *
484− * 5. Resumed
485− * |--------->|
486− *
487− * 0. Default state of VFIO device is _RUNNING when the user application starts.
488− * 1. During normal shutdown of the user application, the user application may
489− * optionally change the VFIO device state from _RUNNING to _STOP. This
490− * transition is optional. The vendor driver must support this transition but
491− * must not require it.
492− * 2. When the user application saves state or suspends the application, the
493− * device state transitions from _RUNNING to stop-and-copy and then to _STOP.
494− * On state transition from _RUNNING to stop-and-copy, driver must stop the
495− * device, save the device state and send it to the application through the
496− * migration region. The sequence to be followed for such transition is given
497− * below.
498− * 3. In live migration of user application, the state transitions from _RUNNING
499− * to pre-copy, to stop-and-copy, and to _STOP.
500− * On state transition from _RUNNING to pre-copy, the driver should start
501− * gathering the device state while the application is still running and send
502− * the device state data to application through the migration region.
503− * On state transition from pre-copy to stop-and-copy, the driver must stop
504− * the device, save the device state and send it to the user application
505− * through the migration region.
506− * Vendor drivers must support the pre-copy state even for implementations
507− * where no data is provided to the user before the stop-and-copy state. The
508− * user must not be required to consume all migration data before the device
509− * transitions to a new state, including the stop-and-copy state.
510− * The sequence to be followed for above two transitions is given below.
511− * 4. To start the resuming phase, the device state should be transitioned from
512− * the _RUNNING to the _RESUMING state.
513− * In the _RESUMING state, the driver should use the device state data
514− * received through the migration region to resume the device.
515− * 5. After providing saved device data to the driver, the application should
516− * change the state from _RESUMING to _RUNNING.
517− *
518− * reserved:
519− * Reads on this field return zero and writes are ignored.
520− *
521− * pending_bytes: (read only)
522− * The number of pending bytes still to be migrated from the vendor driver.
523− *
524− * data_offset: (read only)
525− * The user application should read data_offset field from the migration
526− * region. The user application should read the device data from this
527− * offset within the migration region during the _SAVING state or write
528− * the device data during the _RESUMING state. See below for details of
529− * sequence to be followed.
530− *
531− * data_size: (read/write)
532− * The user application should read data_size to get the size in bytes of
533− * the data copied in the migration region during the _SAVING state and
534− * write the size in bytes of the data copied in the migration region
535− * during the _RESUMING state.
536− *
537− * The format of the migration region is as follows:
538− * ------------------------------------------------------------------
539− * |vfio_device_migration_info| data section |
540− * | | /////////////////////////////// |
541− * ------------------------------------------------------------------
542− * ^ ^
543− * offset 0-trapped part data_offset
544− *
545− * The structure vfio_device_migration_info is always followed by the data
546− * section in the region, so data_offset will always be nonzero. The offset
547− * from where the data is copied is decided by the kernel driver. The data
548− * section can be trapped, mmapped, or partitioned, depending on how the kernel
549− * driver defines the data section. The data section partition can be defined
550− * as mapped by the sparse mmap capability. If mmapped, data_offset must be
551− * page aligned, whereas initial section which contains the
552− * vfio_device_migration_info structure, might not end at the offset, which is
553− * page aligned. The user is not required to access through mmap regardless
554− * of the capabilities of the region mmap.
555− * The vendor driver should determine whether and how to partition the data
556− * section. The vendor driver should return data_offset accordingly.
557− *
558− * The sequence to be followed while in pre-copy state and stop-and-copy state
559− * is as follows:
560− * a. Read pending_bytes, indicating the start of a new iteration to get device
561− * data. Repeated read on pending_bytes at this stage should have no side
562− * effects.
563− * If pending_bytes == 0, the user application should not iterate to get data
564− * for that device.
565− * If pending_bytes > 0, perform the following steps.
566− * b. Read data_offset, indicating that the vendor driver should make data
567− * available through the data section. The vendor driver should return this
568− * read operation only after data is available from (region + data_offset)
569− * to (region + data_offset + data_size).
570− * c. Read data_size, which is the amount of data in bytes available through
571− * the migration region.
572− * Read on data_offset and data_size should return the offset and size of
573− * the current buffer if the user application reads data_offset and
574− * data_size more than once here.
575− * d. Read data_size bytes of data from (region + data_offset) from the
576− * migration region.
577− * e. Process the data.
578− * f. Read pending_bytes, which indicates that the data from the previous
579− * iteration has been read. If pending_bytes > 0, go to step b.
580− *
581− * The user application can transition from the _SAVING|_RUNNING
582− * (pre-copy state) to the _SAVING (stop-and-copy) state regardless of the
583− * number of pending bytes. The user application should iterate in _SAVING
584− * (stop-and-copy) until pending_bytes is 0.
585− *
586− * The sequence to be followed while _RESUMING device state is as follows:
587− * While data for this device is available, repeat the following steps:
588− * a. Read data_offset from where the user application should write data.
589− * b. Write migration data starting at the migration region + data_offset for
590− * the length determined by data_size from the migration source.
591− * c. Write data_size, which indicates to the vendor driver that data is
592− * written in the migration region. Vendor driver must return this write
593− * operations on consuming data. Vendor driver should apply the
594− * user-provided migration region data to the device resume state.
595− *
596− * If an error occurs during the above sequences, the vendor driver can return
597− * an error code for next read() or write() operation, which will terminate the
598− * loop. The user application should then take the next necessary action, for
599− * example, failing migration or terminating the user application.
600− *
601− * For the user application, data is opaque. The user application should write
602− * data in the same order as the data is received and the data should be of
603− * same transaction size at the source.
604− */
408+#define VFIO_REGION_SUBTYPE_MIGRATION_DEPRECATED (1)
605409
606410 struct vfio_device_migration_info {
607411 __u32 device_state; /* VFIO device state */
608−#define VFIO_DEVICE_STATE_STOP (0)
609−#define VFIO_DEVICE_STATE_RUNNING (1 << 0)
610−#define VFIO_DEVICE_STATE_SAVING (1 << 1)
611−#define VFIO_DEVICE_STATE_RESUMING (1 << 2)
612−#define VFIO_DEVICE_STATE_MASK (VFIO_DEVICE_STATE_RUNNING | \
613− VFIO_DEVICE_STATE_SAVING | \
614− VFIO_DEVICE_STATE_RESUMING)
412+#define VFIO_DEVICE_STATE_V1_STOP (0)
413+#define VFIO_DEVICE_STATE_V1_RUNNING (1 << 0)
414+#define VFIO_DEVICE_STATE_V1_SAVING (1 << 1)
415+#define VFIO_DEVICE_STATE_V1_RESUMING (1 << 2)
416+#define VFIO_DEVICE_STATE_MASK (VFIO_DEVICE_STATE_V1_RUNNING | \
417+ VFIO_DEVICE_STATE_V1_SAVING | \
418+ VFIO_DEVICE_STATE_V1_RESUMING)
615419
616420 #define VFIO_DEVICE_STATE_VALID(state) \
617− (state & VFIO_DEVICE_STATE_RESUMING ? \
618− (state & VFIO_DEVICE_STATE_MASK) == VFIO_DEVICE_STATE_RESUMING : 1)
421+ (state & VFIO_DEVICE_STATE_V1_RESUMING ? \
422+ (state & VFIO_DEVICE_STATE_MASK) == VFIO_DEVICE_STATE_V1_RESUMING : 1)
619423
620424 #define VFIO_DEVICE_STATE_IS_ERROR(state) \
621− ((state & VFIO_DEVICE_STATE_MASK) == (VFIO_DEVICE_STATE_SAVING | \
622− VFIO_DEVICE_STATE_RESUMING))
425+ ((state & VFIO_DEVICE_STATE_MASK) == (VFIO_DEVICE_STATE_V1_SAVING | \
426+ VFIO_DEVICE_STATE_V1_RESUMING))
623427
624428 #define VFIO_DEVICE_STATE_SET_ERROR(state) \
625− ((state & ~VFIO_DEVICE_STATE_MASK) | VFIO_DEVICE_SATE_SAVING | \
626− VFIO_DEVICE_STATE_RESUMING)
429+ ((state & ~VFIO_DEVICE_STATE_MASK) | VFIO_DEVICE_STATE_V1_SAVING | \
430+ VFIO_DEVICE_STATE_V1_RESUMING)
627431
628432 __u32 reserved;
629433 __u64 pending_bytes;
@@ -1002,6 +806,186 @@ struct vfio_device_feature { · 372 unmodified lines
1002806 */
1003807 #define VFIO_DEVICE_FEATURE_PCI_VF_TOKEN (0)
1004808
809+/*
810+ * Indicates the device can support the migration API through
811+ * VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE. If this GET succeeds, the RUNNING and
812+ * ERROR states are always supported. Support for additional states is
813+ * indicated via the flags field; at least VFIO_MIGRATION_STOP_COPY must be
814+ * set.
815+ *
816+ * VFIO_MIGRATION_STOP_COPY means that STOP, STOP_COPY and
817+ * RESUMING are supported.
818+ *
819+ * VFIO_MIGRATION_STOP_COPY | VFIO_MIGRATION_P2P means that RUNNING_P2P
820+ * is supported in addition to the STOP_COPY states.
821+ *
822+ * Other combinations of flags have behavior to be defined in the future.
823+ */
824+struct vfio_device_feature_migration {
825+ __aligned_u64 flags;
826+#define VFIO_MIGRATION_STOP_COPY (1 << 0)
827+#define VFIO_MIGRATION_P2P (1 << 1)
828+};
829+#define VFIO_DEVICE_FEATURE_MIGRATION 1
830+
831+/*
832+ * Upon VFIO_DEVICE_FEATURE_SET, execute a migration state change on the VFIO
833+ * device. The new state is supplied in device_state, see enum
834+ * vfio_device_mig_state for details
835+ *
836+ * The kernel migration driver must fully transition the device to the new state
837+ * value before the operation returns to the user.
838+ *
839+ * The kernel migration driver must not generate asynchronous device state
840+ * transitions outside of manipulation by the user or the VFIO_DEVICE_RESET
841+ * ioctl as described above.
842+ *
843+ * If this function fails then current device_state may be the original
844+ * operating state or some other state along the combination transition path.
845+ * The user can then decide if it should execute a VFIO_DEVICE_RESET, attempt
846+ * to return to the original state, or attempt to return to some other state
847+ * such as RUNNING or STOP.
848+ *
849+ * If the new_state starts a new data transfer session then the FD associated
850+ * with that session is returned in data_fd. The user is responsible to close
851+ * this FD when it is finished. The user must consider the migration data stream
852+ * carried over the FD to be opaque and must preserve the byte order of the
853+ * stream. The user is not required to preserve buffer segmentation when writing
854+ * the data stream during the RESUMING operation.
855+ *
856+ * Upon VFIO_DEVICE_FEATURE_GET, get the current migration state of the VFIO
857+ * device, data_fd will be -1.
858+ */
859+struct vfio_device_feature_mig_state {
860+ __u32 device_state; /* From enum vfio_device_mig_state */
861+ __s32 data_fd;
862+};
863+#define VFIO_DEVICE_FEATURE_MIG_DEVICE_STATE 2
864+
865+/*
866+ * The device migration Finite State Machine is described by the enum
867+ * vfio_device_mig_state. Some of the FSM arcs will create a migration data
868+ * transfer session by returning a FD, in this case the migration data will
869+ * flow over the FD using read() and write() as discussed below.
870+ *
871+ * There are 5 states to support VFIO_MIGRATION_STOP_COPY:
872+ * RUNNING - The device is running normally
873+ * STOP - The device does not change the internal or external state
874+ * STOP_COPY - The device internal state can be read out
875+ * RESUMING - The device is stopped and is loading a new internal state
876+ * ERROR - The device has failed and must be reset
877+ *
878+ * And 1 optional state to support VFIO_MIGRATION_P2P:
879+ * RUNNING_P2P - RUNNING, except the device cannot do peer to peer DMA
880+ *
881+ * The FSM takes actions on the arcs between FSM states. The driver implements
882+ * the following behavior for the FSM arcs:
883+ *
884+ * RUNNING_P2P -> STOP
885+ * STOP_COPY -> STOP
886+ * While in STOP the device must stop the operation of the device. The device
887+ * must not generate interrupts, DMA, or any other change to external state.
888+ * It must not change its internal state. When stopped the device and kernel
889+ * migration driver must accept and respond to interaction to support external
890+ * subsystems in the STOP state, for example PCI MSI-X and PCI config space.
891+ * Failure by the user to restrict device access while in STOP must not result
892+ * in error conditions outside the user context (ex. host system faults).
893+ *
894+ * The STOP_COPY arc will terminate a data transfer session.
895+ *
896+ * RESUMING -> STOP
897+ * Leaving RESUMING terminates a data transfer session and indicates the
898+ * device should complete processing of the data delivered by write(). The
899+ * kernel migration driver should complete the incorporation of data written
900+ * to the data transfer FD into the device internal state and perform
901+ * final validity and consistency checking of the new device state. If the
902+ * user provided data is found to be incomplete, inconsistent, or otherwise
903+ * invalid, the migration driver must fail the SET_STATE ioctl and
904+ * optionally go to the ERROR state as described below.
905+ *
906+ * While in STOP the device has the same behavior as other STOP states
907+ * described above.
908+ *
909+ * To abort a RESUMING session the device must be reset.
910+ *
911+ * RUNNING_P2P -> RUNNING
912+ * While in RUNNING the device is fully operational, the device may generate
913+ * interrupts, DMA, respond to MMIO, all vfio device regions are functional,
914+ * and the device may advance its internal state.
915+ *
916+ * RUNNING -> RUNNING_P2P
917+ * STOP -> RUNNING_P2P
918+ * While in RUNNING_P2P the device is partially running in the P2P quiescent
919+ * state defined below.
920+ *
921+ * STOP -> STOP_COPY
922+ * This arc begin the process of saving the device state and will return a
923+ * new data_fd.
924+ *
925+ * While in the STOP_COPY state the device has the same behavior as STOP
926+ * with the addition that the data transfers session continues to stream the
927+ * migration state. End of stream on the FD indicates the entire device
928+ * state has been transferred.
929+ *
930+ * The user should take steps to restrict access to vfio device regions while
931+ * the device is in STOP_COPY or risk corruption of the device migration data
932+ * stream.
933+ *
934+ * STOP -> RESUMING
935+ * Entering the RESUMING state starts a process of restoring the device state
936+ * and will return a new data_fd. The data stream fed into the data_fd should
937+ * be taken from the data transfer output of a single FD during saving from
938+ * a compatible device. The migration driver may alter/reset the internal
939+ * device state for this arc if required to prepare the device to receive the
940+ * migration data.
941+ *
942+ * any -> ERROR
943+ * ERROR cannot be specified as a device state, however any transition request
944+ * can be failed with an errno return and may then move the device_state into
945+ * ERROR. In this case the device was unable to execute the requested arc and
946+ * was also unable to restore the device to any valid device_state.
947+ * To recover from ERROR VFIO_DEVICE_RESET must be used to return the
948+ * device_state back to RUNNING.
949+ *
950+ * The optional peer to peer (P2P) quiescent state is intended to be a quiescent
951+ * state for the device for the purposes of managing multiple devices within a
952+ * user context where peer-to-peer DMA between devices may be active. The
953+ * RUNNING_P2P states must prevent the device from initiating
954+ * any new P2P DMA transactions. If the device can identify P2P transactions
955+ * then it can stop only P2P DMA, otherwise it must stop all DMA. The migration
956+ * driver must complete any such outstanding operations prior to completing the
957+ * FSM arc into a P2P state. For the purpose of specification the states
958+ * behave as though the device was fully running if not supported. Like while in
959+ * STOP or STOP_COPY the user must not touch the device, otherwise the state
960+ * can be exited.
961+ *
962+ * The remaining possible transitions are interpreted as combinations of the
963+ * above FSM arcs. As there are multiple paths through the FSM arcs the path
964+ * should be selected based on the following rules:
965+ * - Select the shortest path.
966+ * Refer to vfio_mig_get_next_state() for the result of the algorithm.
967+ *
968+ * The automatic transit through the FSM arcs that make up the combination
969+ * transition is invisible to the user. When working with combination arcs the
970+ * user may see any step along the path in the device_state if SET_STATE
971+ * fails. When handling these types of errors users should anticipate future
972+ * revisions of this protocol using new states and those states becoming
973+ * visible in this case.
974+ *
975+ * The optional states cannot be used with SET_STATE if the device does not
976+ * support them. The user can discover if these states are supported by using
977+ * VFIO_DEVICE_FEATURE_MIGRATION. By using combination transitions the user can
978+ * avoid knowing about these optional states if the kernel driver supports them.
979+ */
980+enum vfio_device_mig_state {
981+ VFIO_DEVICE_STATE_ERROR = 0,
982+ VFIO_DEVICE_STATE_STOP = 1,
983+ VFIO_DEVICE_STATE_RUNNING = 2,
984+ VFIO_DEVICE_STATE_STOP_COPY = 3,
985+ VFIO_DEVICE_STATE_RESUMING = 4,
986+ VFIO_DEVICE_STATE_RUNNING_P2P = 5,
987+};
988+
1005989 /* -------- API for Type1 VFIO IOMMU -------- */
1006990
1007991 /**