// SPDX-License-Identifier: GPL-2.0-only OR MIT/* * Copyright (c) 2022 Red Hat. * * Permission is hereby granted, free of charge, to any person obtaining a * copy of this software and associated documentation files (the "Software"), * to deal in the Software without restriction, including without limitation * the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. * * Authors: * Danilo Krummrich <dakr@redhat.com> * */ #include <drm/drm_drv.h>#include <drm/drm_gpuvm.h>#include <drm/drm_print.h> #include <linux/export.h>#include <linux/interval_tree_generic.h>#include <linux/mm.h> /** * DOC: Overview * * The DRM GPU VA Manager, represented by struct drm_gpuvm keeps track of a * GPU's virtual address (VA) space and manages the corresponding virtual * mappings represented by &drm_gpuva objects. It also keeps track of the * mapping's backing &drm_gem_object buffers. * * &drm_gem_object buffers maintain a list of &drm_gpuva objects representing * all existing GPU VA mappings using this &drm_gem_object as backing buffer. * * GPU VAs can be flagged as sparse, such that drivers may use GPU VAs to also * keep track of sparse PTEs in order to support Vulkan 'Sparse Resources'. * * The GPU VA manager internally uses a rb-tree to manage the * &drm_gpuva mappings within a GPU's virtual address space. * * The &drm_gpuvm structure contains a special &drm_gpuva representing the * portion of VA space reserved by the kernel. This node is initialized together * with the GPU VA manager instance and removed when the GPU VA manager is * destroyed. * * In a typical application drivers would embed struct drm_gpuvm and * struct drm_gpuva within their own driver specific structures, there won't be * any memory allocations of its own nor memory allocations of &drm_gpuva * entries. * * The data structures needed to store &drm_gpuvas within the &drm_gpuvm are * contained within struct drm_gpuva already. Hence, for inserting &drm_gpuva * entries from within dma-fence signalling critical sections it is enough to * pre-allocate the &drm_gpuva structures. * * &drm_gem_objects which are private to a single VM can share a common * &dma_resv in order to improve locking efficiency (e.g. with &drm_exec). * For this purpose drivers must pass a &drm_gem_object to drm_gpuvm_init(), in * the following called 'resv object', which serves as the container of the * GPUVM's shared &dma_resv. This resv object can be a driver specific * &drm_gem_object, such as the &drm_gem_object containing the root page table, * but it can also be a 'dummy' object, which can be allocated with * drm_gpuvm_resv_object_alloc(). * * In order to connect a struct drm_gpuva to its backing &drm_gem_object each * &drm_gem_object maintains a list of &drm_gpuvm_bo structures, and each * &drm_gpuvm_bo contains a list of &drm_gpuva structures. * * A &drm_gpuvm_bo is an abstraction that represents a combination of a * &drm_gpuvm and a &drm_gem_object. Every such combination should be unique. * This is ensured by the API through drm_gpuvm_bo_obtain() and * drm_gpuvm_bo_obtain_prealloc() which first look into the corresponding * &drm_gem_object list of &drm_gpuvm_bos for an existing instance of this * particular combination. If not present, a new instance is created and linked * to the &drm_gem_object. * * &drm_gpuvm_bo structures, since unique for a given &drm_gpuvm, are also used * as entry for the &drm_gpuvm's lists of external and evicted objects. Those * lists are maintained in order to accelerate locking of dma-resv locks and * validation of evicted objects bound in a &drm_gpuvm. For instance, all * &drm_gem_object's &dma_resv of a given &drm_gpuvm can be locked by calling * drm_gpuvm_exec_lock(). Once locked drivers can call drm_gpuvm_validate() in * order to validate all evicted &drm_gem_objects. It is also possible to lock * additional &drm_gem_objects by providing the corresponding parameters to * drm_gpuvm_exec_lock() as well as open code the &drm_exec loop while making * use of helper functions such as drm_gpuvm_prepare_range() or * drm_gpuvm_prepare_objects(). * * Every bound &drm_gem_object is treated as external object when its &dma_resv * structure is different than the &drm_gpuvm's common &dma_resv structure. */ /** * DOC: Split and Merge * * Besides its capability to manage and represent a GPU VA space, the * GPU VA manager also provides functions to let the &drm_gpuvm calculate a * sequence of operations to satisfy a given map or unmap request. * * Therefore the DRM GPU VA manager provides an algorithm implementing splitting * and merging of existing GPU VA mappings with the ones that are requested to * be mapped or unmapped. This feature is required by the Vulkan API to * implement Vulkan 'Sparse Memory Bindings' - drivers UAPIs often refer to this * as VM BIND. * * Drivers can call drm_gpuvm_sm_map() to receive a sequence of callbacks * containing map, unmap and remap operations for a given newly requested * mapping. The sequence of callbacks represents the set of operations to * execute in order to integrate the new mapping cleanly into the current state * of the GPU VA space. * * Depending on how the new GPU VA mapping intersects with the existing mappings * of the GPU VA space the &drm_gpuvm_ops callbacks contain an arbitrary amount * of unmap operations, a maximum of two remap operations and a single map * operation. The caller might receive no callback at all if no operation is * required, e.g. if the requested mapping already exists in the exact same way. * * The single map operation represents the original map operation requested by * the caller. * * &drm_gpuva_op_unmap contains a 'keep' field, which indicates whether the * &drm_gpuva to unmap is physically contiguous with the original mapping * request. Optionally, if 'keep' is set, drivers may keep the actual page table * entries for this &drm_gpuva, adding the missing page table entries only and * update the &drm_gpuvm's view of things accordingly. * * Drivers may do the same optimization, namely delta page table updates, also * for remap operations. This is possible since &drm_gpuva_op_remap consists of * one unmap operation and one or two map operations, such that drivers can * derive the page table update delta accordingly. * * Note that there can't be more than two existing mappings to split up, one at * the beginning and one at the end of the new mapping, hence there is a * maximum of two remap operations. * * Analogous to drm_gpuvm_sm_map() drm_gpuvm_sm_unmap() uses &drm_gpuvm_ops to * call back into the driver in order to unmap a range of GPU VA space. The * logic behind this function is way simpler though: For all existing mappings * enclosed by the given range unmap operations are created. For mappings which * are only partially located within the given range, remap operations are * created such that those mappings are split up and re-mapped partially. * * As an alternative to drm_gpuvm_sm_map() and drm_gpuvm_sm_unmap(), * drm_gpuvm_sm_map_ops_create() and drm_gpuvm_sm_unmap_ops_create() can be used * to directly obtain an instance of struct drm_gpuva_ops containing a list of * &drm_gpuva_op, which can be iterated with drm_gpuva_for_each_op(). This list * contains the &drm_gpuva_ops analogous to the callbacks one would receive when * calling drm_gpuvm_sm_map() or drm_gpuvm_sm_unmap(). While this way requires * more memory (to allocate the &drm_gpuva_ops), it provides drivers a way to * iterate the &drm_gpuva_op multiple times, e.g. once in a context where memory * allocations are possible (e.g. to allocate GPU page tables) and once in the * dma-fence signalling critical path. * * To update the &drm_gpuvm's view of the GPU VA space drm_gpuva_insert() and * drm_gpuva_remove() may be used. These functions can safely be used from * &drm_gpuvm_ops callbacks originating from drm_gpuvm_sm_map() or * drm_gpuvm_sm_unmap(). However, it might be more convenient to use the * provided helper functions drm_gpuva_map(), drm_gpuva_remap() and * drm_gpuva_unmap() instead. * * The following diagram depicts the basic relationships of existing GPU VA * mappings, a newly requested mapping and the resulting mappings as implemented * by drm_gpuvm_sm_map() - it doesn't cover any arbitrary combinations of these. * * 1) Requested mapping is identical. Replace it, but indicate the backing PTEs * could be kept. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=n) * * 0 a 1 * new: |-----------| (bo_offset=n) * * * 2) Requested mapping is identical, except for the BO offset, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=m) * * 0 a 1 * new: |-----------| (bo_offset=m) * * * 3) Requested mapping is identical, except for the backing BO, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----------| (bo_offset=n) * * 0 b 1 * new: |-----------| (bo_offset=n) * * * 4) Existent mapping is a left aligned subset of the requested one, hence * replace the existing one. * * :: * * 0 a 1 * old: |-----| (bo_offset=n) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 5) Requested mapping's range is a left aligned subset of the existing one, * but backed by a different BO. Hence, map the requested mapping and split * the existing one adjusting its BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----| (bo_offset=n) * * 0 b 1 a' 2 * new: |-----|-----| (b.bo_offset=n, a.bo_offset=n+1) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 6) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----| (bo_offset=n) * * 0 a 1 a' 2 * new: |-----|-----| (a.bo_offset=n, a'.bo_offset=n+1) * * * 7) Requested mapping's range is a right aligned subset of the existing one, * but backed by a different BO. Hence, map the requested mapping and split * the existing one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 * new: |-----|-----| (a.bo_offset=n,b.bo_offset=m) * * * 8) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 * new: |-----|-----| (a'.bo_offset=n, a.bo_offset=n+1) * * * 9) Existent mapping is overlapped at the end by the requested mapping backed * by a different BO. Hence, map the requested mapping and split up the * existing one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 3 * req: |-----------| (bo_offset=m) * * 0 a 1 b 3 * new: |-----|-----------| (a.bo_offset=n,b.bo_offset=m) * * * 10) Existent mapping is overlapped by the requested mapping, both having the * same backing BO with a contiguous offset. Indicate the backing PTEs of * the old mapping could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 3 * req: |-----------| (bo_offset=n+1) * * 0 a' 1 a 3 * new: |-----|-----------| (a'.bo_offset=n, a.bo_offset=n+1) * * * 11) Requested mapping's range is a centered subset of the existing one * having a different backing BO. Hence, map the requested mapping and split * up the existing one in two mappings, adjusting the BO offset of the right * one accordingly. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 a' 3 * new: |-----|-----|-----| (a.bo_offset=n,b.bo_offset=m,a'.bo_offset=n+2) * * * 12) Requested mapping is a contiguous subset of the existing one. Split it * up, but indicate that the backing PTEs could be kept. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 a'' 3 * old: |-----|-----|-----| (a'.bo_offset=n, a.bo_offset=n+1, a''.bo_offset=n+2) * * * 13) Existent mapping is a right aligned subset of the requested one, hence * replace the existing one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 14) Existent mapping is a centered subset of the requested one, hence * replace the existing one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 3 * req: |----------------| (bo_offset=n) * * 0 a 3 * new: |----------------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 15) Existent mappings is overlapped at the beginning by the requested mapping * backed by a different BO. Hence, map the requested mapping and split up * the existing one, adjusting its BO offset accordingly. * * :: * * 1 a 3 * old: |-----------| (bo_offset=n) * * 0 b 2 * req: |-----------| (bo_offset=m) * * 0 b 2 a' 3 * new: |-----------|-----| (b.bo_offset=m,a.bo_offset=n+2) */ /** * DOC: Madvise Logic - Splitting and Traversal * * This logic handles GPU VA range updates by generating remap and map operations * without performing unmaps or merging existing mappings. * * 1) The requested range lies entirely within a single drm_gpuva. The logic splits * the existing mapping at the start and end boundaries and inserts a new map. * * :: * a start end b * pre: |-----------------------| * drm_gpuva1 * * a start end b * new: |-----|=========|-------| * remap map remap * * one REMAP and one MAP : Same behaviour as SPLIT and MERGE * * 2) The requested range spans multiple drm_gpuva regions. The logic traverses * across boundaries, remapping the start and end segments, and inserting two * map operations to cover the full range. * * :: a start b c end d * pre: |------------------|--------------|------------------| * drm_gpuva1 drm_gpuva2 drm_gpuva3 * * a start b c end d * new: |-------|==========|--------------|========|---------| * remap1 map1 drm_gpuva2 map2 remap2 * * two REMAPS and two MAPS * * 3) Either start or end lies within a drm_gpuva. A single remap and map operation * are generated to update the affected portion. * * * :: a/start b c end d * pre: |------------------|--------------|------------------| * drm_gpuva1 drm_gpuva2 drm_gpuva3 * * a/start b c end d * new: |------------------|--------------|========|---------| * drm_gpuva1 drm_gpuva2 map1 remap1 * * :: a start b c/end d * pre: |------------------|--------------|------------------| * drm_gpuva1 drm_gpuva2 drm_gpuva3 * * a start b c/end d * new: |-------|==========|--------------|------------------| * remap1 map1 drm_gpuva2 drm_gpuva3 * * one REMAP and one MAP * * 4) Both start and end align with existing drm_gpuva boundaries. No operations * are needed as the range is already covered. * * 5) No existing drm_gpuvas. No operations. * * Unlike drm_gpuvm_sm_map_ops_create, this logic avoids unmaps and merging, * focusing solely on remap and map operations for efficient traversal and update. */ /** * DOC: Locking * * In terms of managing &drm_gpuva entries DRM GPUVM does not take care of * locking itself, it is the drivers responsibility to take care about locking. * Drivers might want to protect the following operations: inserting, removing * and iterating &drm_gpuva objects as well as generating all kinds of * operations, such as split / merge or prefetch. * * DRM GPUVM also does not take care of the locking of the backing * &drm_gem_object buffers GPU VA lists and &drm_gpuvm_bo abstractions byLines 1–500 of 3206. Load more
// SPDX-License-Identifier: GPL-2.0-only OR MIT /* * Copyright (c) 2022 Red Hat. * * Permission is hereby granted, free of charge, to any person obtaining a * copy of this software and associated documentation files (the "Software"), * to deal in the Software without restriction, including without limitation * the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. * * Authors: * Danilo Krummrich <dakr@redhat.com> * */ #include <drm/drm_drv.h> #include <drm/drm_gpuvm.h> #include <drm/drm_print.h> #include <linux/export.h> #include <linux/interval_tree_generic.h> #include <linux/mm.h> /** * DOC: Overview * * The DRM GPU VA Manager, represented by struct drm_gpuvm keeps track of a * GPU's virtual address (VA) space and manages the corresponding virtual * mappings represented by &drm_gpuva objects. It also keeps track of the * mapping's backing &drm_gem_object buffers. * * &drm_gem_object buffers maintain a list of &drm_gpuva objects representing * all existing GPU VA mappings using this &drm_gem_object as backing buffer. * * GPU VAs can be flagged as sparse, such that drivers may use GPU VAs to also * keep track of sparse PTEs in order to support Vulkan 'Sparse Resources'. * * The GPU VA manager internally uses a rb-tree to manage the * &drm_gpuva mappings within a GPU's virtual address space. * * The &drm_gpuvm structure contains a special &drm_gpuva representing the * portion of VA space reserved by the kernel. This node is initialized together * with the GPU VA manager instance and removed when the GPU VA manager is * destroyed. * * In a typical application drivers would embed struct drm_gpuvm and * struct drm_gpuva within their own driver specific structures, there won't be * any memory allocations of its own nor memory allocations of &drm_gpuva * entries. * * The data structures needed to store &drm_gpuvas within the &drm_gpuvm are * contained within struct drm_gpuva already. Hence, for inserting &drm_gpuva * entries from within dma-fence signalling critical sections it is enough to * pre-allocate the &drm_gpuva structures. * * &drm_gem_objects which are private to a single VM can share a common * &dma_resv in order to improve locking efficiency (e.g. with &drm_exec). * For this purpose drivers must pass a &drm_gem_object to drm_gpuvm_init(), in * the following called 'resv object', which serves as the container of the * GPUVM's shared &dma_resv. This resv object can be a driver specific * &drm_gem_object, such as the &drm_gem_object containing the root page table, * but it can also be a 'dummy' object, which can be allocated with * drm_gpuvm_resv_object_alloc(). * * In order to connect a struct drm_gpuva to its backing &drm_gem_object each * &drm_gem_object maintains a list of &drm_gpuvm_bo structures, and each * &drm_gpuvm_bo contains a list of &drm_gpuva structures. * * A &drm_gpuvm_bo is an abstraction that represents a combination of a * &drm_gpuvm and a &drm_gem_object. Every such combination should be unique. * This is ensured by the API through drm_gpuvm_bo_obtain() and * drm_gpuvm_bo_obtain_prealloc() which first look into the corresponding * &drm_gem_object list of &drm_gpuvm_bos for an existing instance of this * particular combination. If not present, a new instance is created and linked * to the &drm_gem_object. * * &drm_gpuvm_bo structures, since unique for a given &drm_gpuvm, are also used * as entry for the &drm_gpuvm's lists of external and evicted objects. Those * lists are maintained in order to accelerate locking of dma-resv locks and * validation of evicted objects bound in a &drm_gpuvm. For instance, all * &drm_gem_object's &dma_resv of a given &drm_gpuvm can be locked by calling * drm_gpuvm_exec_lock(). Once locked drivers can call drm_gpuvm_validate() in * order to validate all evicted &drm_gem_objects. It is also possible to lock * additional &drm_gem_objects by providing the corresponding parameters to * drm_gpuvm_exec_lock() as well as open code the &drm_exec loop while making * use of helper functions such as drm_gpuvm_prepare_range() or * drm_gpuvm_prepare_objects(). * * Every bound &drm_gem_object is treated as external object when its &dma_resv * structure is different than the &drm_gpuvm's common &dma_resv structure. */ /** * DOC: Split and Merge * * Besides its capability to manage and represent a GPU VA space, the * GPU VA manager also provides functions to let the &drm_gpuvm calculate a * sequence of operations to satisfy a given map or unmap request. * * Therefore the DRM GPU VA manager provides an algorithm implementing splitting * and merging of existing GPU VA mappings with the ones that are requested to * be mapped or unmapped. This feature is required by the Vulkan API to * implement Vulkan 'Sparse Memory Bindings' - drivers UAPIs often refer to this * as VM BIND. * * Drivers can call drm_gpuvm_sm_map() to receive a sequence of callbacks * containing map, unmap and remap operations for a given newly requested * mapping. The sequence of callbacks represents the set of operations to * execute in order to integrate the new mapping cleanly into the current state * of the GPU VA space. * * Depending on how the new GPU VA mapping intersects with the existing mappings * of the GPU VA space the &drm_gpuvm_ops callbacks contain an arbitrary amount * of unmap operations, a maximum of two remap operations and a single map * operation. The caller might receive no callback at all if no operation is * required, e.g. if the requested mapping already exists in the exact same way. * * The single map operation represents the original map operation requested by * the caller. * * &drm_gpuva_op_unmap contains a 'keep' field, which indicates whether the * &drm_gpuva to unmap is physically contiguous with the original mapping * request. Optionally, if 'keep' is set, drivers may keep the actual page table * entries for this &drm_gpuva, adding the missing page table entries only and * update the &drm_gpuvm's view of things accordingly. * * Drivers may do the same optimization, namely delta page table updates, also * for remap operations. This is possible since &drm_gpuva_op_remap consists of * one unmap operation and one or two map operations, such that drivers can * derive the page table update delta accordingly. * * Note that there can't be more than two existing mappings to split up, one at * the beginning and one at the end of the new mapping, hence there is a * maximum of two remap operations. * * Analogous to drm_gpuvm_sm_map() drm_gpuvm_sm_unmap() uses &drm_gpuvm_ops to * call back into the driver in order to unmap a range of GPU VA space. The * logic behind this function is way simpler though: For all existing mappings * enclosed by the given range unmap operations are created. For mappings which * are only partially located within the given range, remap operations are * created such that those mappings are split up and re-mapped partially. * * As an alternative to drm_gpuvm_sm_map() and drm_gpuvm_sm_unmap(), * drm_gpuvm_sm_map_ops_create() and drm_gpuvm_sm_unmap_ops_create() can be used * to directly obtain an instance of struct drm_gpuva_ops containing a list of * &drm_gpuva_op, which can be iterated with drm_gpuva_for_each_op(). This list * contains the &drm_gpuva_ops analogous to the callbacks one would receive when * calling drm_gpuvm_sm_map() or drm_gpuvm_sm_unmap(). While this way requires * more memory (to allocate the &drm_gpuva_ops), it provides drivers a way to * iterate the &drm_gpuva_op multiple times, e.g. once in a context where memory * allocations are possible (e.g. to allocate GPU page tables) and once in the * dma-fence signalling critical path. * * To update the &drm_gpuvm's view of the GPU VA space drm_gpuva_insert() and * drm_gpuva_remove() may be used. These functions can safely be used from * &drm_gpuvm_ops callbacks originating from drm_gpuvm_sm_map() or * drm_gpuvm_sm_unmap(). However, it might be more convenient to use the * provided helper functions drm_gpuva_map(), drm_gpuva_remap() and * drm_gpuva_unmap() instead. * * The following diagram depicts the basic relationships of existing GPU VA * mappings, a newly requested mapping and the resulting mappings as implemented * by drm_gpuvm_sm_map() - it doesn't cover any arbitrary combinations of these. * * 1) Requested mapping is identical. Replace it, but indicate the backing PTEs * could be kept. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=n) * * 0 a 1 * new: |-----------| (bo_offset=n) * * * 2) Requested mapping is identical, except for the BO offset, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=m) * * 0 a 1 * new: |-----------| (bo_offset=m) * * * 3) Requested mapping is identical, except for the backing BO, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----------| (bo_offset=n) * * 0 b 1 * new: |-----------| (bo_offset=n) * * * 4) Existent mapping is a left aligned subset of the requested one, hence * replace the existing one. * * :: * * 0 a 1 * old: |-----| (bo_offset=n) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 5) Requested mapping's range is a left aligned subset of the existing one, * but backed by a different BO. Hence, map the requested mapping and split * the existing one adjusting its BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----| (bo_offset=n) * * 0 b 1 a' 2 * new: |-----|-----| (b.bo_offset=n, a.bo_offset=n+1) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 6) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----| (bo_offset=n) * * 0 a 1 a' 2 * new: |-----|-----| (a.bo_offset=n, a'.bo_offset=n+1) * * * 7) Requested mapping's range is a right aligned subset of the existing one, * but backed by a different BO. Hence, map the requested mapping and split * the existing one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 * new: |-----|-----| (a.bo_offset=n,b.bo_offset=m) * * * 8) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 * new: |-----|-----| (a'.bo_offset=n, a.bo_offset=n+1) * * * 9) Existent mapping is overlapped at the end by the requested mapping backed * by a different BO. Hence, map the requested mapping and split up the * existing one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 3 * req: |-----------| (bo_offset=m) * * 0 a 1 b 3 * new: |-----|-----------| (a.bo_offset=n,b.bo_offset=m) * * * 10) Existent mapping is overlapped by the requested mapping, both having the * same backing BO with a contiguous offset. Indicate the backing PTEs of * the old mapping could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 3 * req: |-----------| (bo_offset=n+1) * * 0 a' 1 a 3 * new: |-----|-----------| (a'.bo_offset=n, a.bo_offset=n+1) * * * 11) Requested mapping's range is a centered subset of the existing one * having a different backing BO. Hence, map the requested mapping and split * up the existing one in two mappings, adjusting the BO offset of the right * one accordingly. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 a' 3 * new: |-----|-----|-----| (a.bo_offset=n,b.bo_offset=m,a'.bo_offset=n+2) * * * 12) Requested mapping is a contiguous subset of the existing one. Split it * up, but indicate that the backing PTEs could be kept. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 a'' 3 * old: |-----|-----|-----| (a'.bo_offset=n, a.bo_offset=n+1, a''.bo_offset=n+2) * * * 13) Existent mapping is a right aligned subset of the requested one, hence * replace the existing one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 14) Existent mapping is a centered subset of the requested one, hence * replace the existing one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 3 * req: |----------------| (bo_offset=n) * * 0 a 3 * new: |----------------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 15) Existent mappings is overlapped at the beginning by the requested mapping * backed by a different BO. Hence, map the requested mapping and split up * the existing one, adjusting its BO offset accordingly. * * :: * * 1 a 3 * old: |-----------| (bo_offset=n) * * 0 b 2 * req: |-----------| (bo_offset=m) * * 0 b 2 a' 3 * new: |-----------|-----| (b.bo_offset=m,a.bo_offset=n+2) */ /** * DOC: Madvise Logic - Splitting and Traversal * * This logic handles GPU VA range updates by generating remap and map operations * without performing unmaps or merging existing mappings. * * 1) The requested range lies entirely within a single drm_gpuva. The logic splits * the existing mapping at the start and end boundaries and inserts a new map. * * :: * a start end b * pre: |-----------------------| * drm_gpuva1 * * a start end b * new: |-----|=========|-------| * remap map remap * * one REMAP and one MAP : Same behaviour as SPLIT and MERGE * * 2) The requested range spans multiple drm_gpuva regions. The logic traverses * across boundaries, remapping the start and end segments, and inserting two * map operations to cover the full range. * * :: a start b c end d * pre: |------------------|--------------|------------------| * drm_gpuva1 drm_gpuva2 drm_gpuva3 * * a start b c end d * new: |-------|==========|--------------|========|---------| * remap1 map1 drm_gpuva2 map2 remap2 * * two REMAPS and two MAPS * * 3) Either start or end lies within a drm_gpuva. A single remap and map operation * are generated to update the affected portion. * * * :: a/start b c end d * pre: |------------------|--------------|------------------| * drm_gpuva1 drm_gpuva2 drm_gpuva3 * * a/start b c end d * new: |------------------|--------------|========|---------| * drm_gpuva1 drm_gpuva2 map1 remap1 * * :: a start b c/end d * pre: |------------------|--------------|------------------| * drm_gpuva1 drm_gpuva2 drm_gpuva3 * * a start b c/end d * new: |-------|==========|--------------|------------------| * remap1 map1 drm_gpuva2 drm_gpuva3 * * one REMAP and one MAP * * 4) Both start and end align with existing drm_gpuva boundaries. No operations * are needed as the range is already covered. * * 5) No existing drm_gpuvas. No operations. * * Unlike drm_gpuvm_sm_map_ops_create, this logic avoids unmaps and merging, * focusing solely on remap and map operations for efficient traversal and update. */ /** * DOC: Locking * * In terms of managing &drm_gpuva entries DRM GPUVM does not take care of * locking itself, it is the drivers responsibility to take care about locking. * Drivers might want to protect the following operations: inserting, removing * and iterating &drm_gpuva objects as well as generating all kinds of * operations, such as split / merge or prefetch. * * DRM GPUVM also does not take care of the locking of the backing * &drm_gem_object buffers GPU VA lists and &drm_gpuvm_bo abstractions by