// SPDX-License-Identifier: GPL-2.0-only OR MIT/* * Copyright (c) 2022 Red Hat. * * Permission is hereby granted, free of charge, to any person obtaining a * copy of this software and associated documentation files (the "Software"), * to deal in the Software without restriction, including without limitation * the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. * * Authors: * Danilo Krummrich <dakr@redhat.com> * */ #include <drm/drm_gpuvm.h> #include <linux/interval_tree_generic.h>#include <linux/mm.h> /** * DOC: Overview * * The DRM GPU VA Manager, represented by struct drm_gpuvm keeps track of a * GPU's virtual address (VA) space and manages the corresponding virtual * mappings represented by &drm_gpuva objects. It also keeps track of the * mapping's backing &drm_gem_object buffers. * * &drm_gem_object buffers maintain a list of &drm_gpuva objects representing * all existent GPU VA mappings using this &drm_gem_object as backing buffer. * * GPU VAs can be flagged as sparse, such that drivers may use GPU VAs to also * keep track of sparse PTEs in order to support Vulkan 'Sparse Resources'. * * The GPU VA manager internally uses a rb-tree to manage the * &drm_gpuva mappings within a GPU's virtual address space. * * The &drm_gpuvm structure contains a special &drm_gpuva representing the * portion of VA space reserved by the kernel. This node is initialized together * with the GPU VA manager instance and removed when the GPU VA manager is * destroyed. * * In a typical application drivers would embed struct drm_gpuvm and * struct drm_gpuva within their own driver specific structures, there won't be * any memory allocations of its own nor memory allocations of &drm_gpuva * entries. * * The data structures needed to store &drm_gpuvas within the &drm_gpuvm are * contained within struct drm_gpuva already. Hence, for inserting &drm_gpuva * entries from within dma-fence signalling critical sections it is enough to * pre-allocate the &drm_gpuva structures. * * &drm_gem_objects which are private to a single VM can share a common * &dma_resv in order to improve locking efficiency (e.g. with &drm_exec). * For this purpose drivers must pass a &drm_gem_object to drm_gpuvm_init(), in * the following called 'resv object', which serves as the container of the * GPUVM's shared &dma_resv. This resv object can be a driver specific * &drm_gem_object, such as the &drm_gem_object containing the root page table, * but it can also be a 'dummy' object, which can be allocated with * drm_gpuvm_resv_object_alloc(). * * In order to connect a struct drm_gpuva its backing &drm_gem_object each * &drm_gem_object maintains a list of &drm_gpuvm_bo structures, and each * &drm_gpuvm_bo contains a list of &drm_gpuva structures. * * A &drm_gpuvm_bo is an abstraction that represents a combination of a * &drm_gpuvm and a &drm_gem_object. Every such combination should be unique. * This is ensured by the API through drm_gpuvm_bo_obtain() and * drm_gpuvm_bo_obtain_prealloc() which first look into the corresponding * &drm_gem_object list of &drm_gpuvm_bos for an existing instance of this * particular combination. If not existent a new instance is created and linked * to the &drm_gem_object. * * &drm_gpuvm_bo structures, since unique for a given &drm_gpuvm, are also used * as entry for the &drm_gpuvm's lists of external and evicted objects. Those * lists are maintained in order to accelerate locking of dma-resv locks and * validation of evicted objects bound in a &drm_gpuvm. For instance, all * &drm_gem_object's &dma_resv of a given &drm_gpuvm can be locked by calling * drm_gpuvm_exec_lock(). Once locked drivers can call drm_gpuvm_validate() in * order to validate all evicted &drm_gem_objects. It is also possible to lock * additional &drm_gem_objects by providing the corresponding parameters to * drm_gpuvm_exec_lock() as well as open code the &drm_exec loop while making * use of helper functions such as drm_gpuvm_prepare_range() or * drm_gpuvm_prepare_objects(). * * Every bound &drm_gem_object is treated as external object when its &dma_resv * structure is different than the &drm_gpuvm's common &dma_resv structure. */ /** * DOC: Split and Merge * * Besides its capability to manage and represent a GPU VA space, the * GPU VA manager also provides functions to let the &drm_gpuvm calculate a * sequence of operations to satisfy a given map or unmap request. * * Therefore the DRM GPU VA manager provides an algorithm implementing splitting * and merging of existent GPU VA mappings with the ones that are requested to * be mapped or unmapped. This feature is required by the Vulkan API to * implement Vulkan 'Sparse Memory Bindings' - drivers UAPIs often refer to this * as VM BIND. * * Drivers can call drm_gpuvm_sm_map() to receive a sequence of callbacks * containing map, unmap and remap operations for a given newly requested * mapping. The sequence of callbacks represents the set of operations to * execute in order to integrate the new mapping cleanly into the current state * of the GPU VA space. * * Depending on how the new GPU VA mapping intersects with the existent mappings * of the GPU VA space the &drm_gpuvm_ops callbacks contain an arbitrary amount * of unmap operations, a maximum of two remap operations and a single map * operation. The caller might receive no callback at all if no operation is * required, e.g. if the requested mapping already exists in the exact same way. * * The single map operation represents the original map operation requested by * the caller. * * &drm_gpuva_op_unmap contains a 'keep' field, which indicates whether the * &drm_gpuva to unmap is physically contiguous with the original mapping * request. Optionally, if 'keep' is set, drivers may keep the actual page table * entries for this &drm_gpuva, adding the missing page table entries only and * update the &drm_gpuvm's view of things accordingly. * * Drivers may do the same optimization, namely delta page table updates, also * for remap operations. This is possible since &drm_gpuva_op_remap consists of * one unmap operation and one or two map operations, such that drivers can * derive the page table update delta accordingly. * * Note that there can't be more than two existent mappings to split up, one at * the beginning and one at the end of the new mapping, hence there is a * maximum of two remap operations. * * Analogous to drm_gpuvm_sm_map() drm_gpuvm_sm_unmap() uses &drm_gpuvm_ops to * call back into the driver in order to unmap a range of GPU VA space. The * logic behind this function is way simpler though: For all existent mappings * enclosed by the given range unmap operations are created. For mappings which * are only partically located within the given range, remap operations are * created such that those mappings are split up and re-mapped partically. * * As an alternative to drm_gpuvm_sm_map() and drm_gpuvm_sm_unmap(), * drm_gpuvm_sm_map_ops_create() and drm_gpuvm_sm_unmap_ops_create() can be used * to directly obtain an instance of struct drm_gpuva_ops containing a list of * &drm_gpuva_op, which can be iterated with drm_gpuva_for_each_op(). This list * contains the &drm_gpuva_ops analogous to the callbacks one would receive when * calling drm_gpuvm_sm_map() or drm_gpuvm_sm_unmap(). While this way requires * more memory (to allocate the &drm_gpuva_ops), it provides drivers a way to * iterate the &drm_gpuva_op multiple times, e.g. once in a context where memory * allocations are possible (e.g. to allocate GPU page tables) and once in the * dma-fence signalling critical path. * * To update the &drm_gpuvm's view of the GPU VA space drm_gpuva_insert() and * drm_gpuva_remove() may be used. These functions can safely be used from * &drm_gpuvm_ops callbacks originating from drm_gpuvm_sm_map() or * drm_gpuvm_sm_unmap(). However, it might be more convenient to use the * provided helper functions drm_gpuva_map(), drm_gpuva_remap() and * drm_gpuva_unmap() instead. * * The following diagram depicts the basic relationships of existent GPU VA * mappings, a newly requested mapping and the resulting mappings as implemented * by drm_gpuvm_sm_map() - it doesn't cover any arbitrary combinations of these. * * 1) Requested mapping is identical. Replace it, but indicate the backing PTEs * could be kept. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=n) * * 0 a 1 * new: |-----------| (bo_offset=n) * * * 2) Requested mapping is identical, except for the BO offset, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=m) * * 0 a 1 * new: |-----------| (bo_offset=m) * * * 3) Requested mapping is identical, except for the backing BO, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----------| (bo_offset=n) * * 0 b 1 * new: |-----------| (bo_offset=n) * * * 4) Existent mapping is a left aligned subset of the requested one, hence * replace the existent one. * * :: * * 0 a 1 * old: |-----| (bo_offset=n) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 5) Requested mapping's range is a left aligned subset of the existent one, * but backed by a different BO. Hence, map the requested mapping and split * the existent one adjusting its BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----| (bo_offset=n) * * 0 b 1 a' 2 * new: |-----|-----| (b.bo_offset=n, a.bo_offset=n+1) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 6) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----| (bo_offset=n) * * 0 a 1 a' 2 * new: |-----|-----| (a.bo_offset=n, a'.bo_offset=n+1) * * * 7) Requested mapping's range is a right aligned subset of the existent one, * but backed by a different BO. Hence, map the requested mapping and split * the existent one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 * new: |-----|-----| (a.bo_offset=n,b.bo_offset=m) * * * 8) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 * new: |-----|-----| (a'.bo_offset=n, a.bo_offset=n+1) * * * 9) Existent mapping is overlapped at the end by the requested mapping backed * by a different BO. Hence, map the requested mapping and split up the * existent one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 3 * req: |-----------| (bo_offset=m) * * 0 a 1 b 3 * new: |-----|-----------| (a.bo_offset=n,b.bo_offset=m) * * * 10) Existent mapping is overlapped by the requested mapping, both having the * same backing BO with a contiguous offset. Indicate the backing PTEs of * the old mapping could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 3 * req: |-----------| (bo_offset=n+1) * * 0 a' 1 a 3 * new: |-----|-----------| (a'.bo_offset=n, a.bo_offset=n+1) * * * 11) Requested mapping's range is a centered subset of the existent one * having a different backing BO. Hence, map the requested mapping and split * up the existent one in two mappings, adjusting the BO offset of the right * one accordingly. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 a' 3 * new: |-----|-----|-----| (a.bo_offset=n,b.bo_offset=m,a'.bo_offset=n+2) * * * 12) Requested mapping is a contiguous subset of the existent one. Split it * up, but indicate that the backing PTEs could be kept. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 a'' 3 * old: |-----|-----|-----| (a'.bo_offset=n, a.bo_offset=n+1, a''.bo_offset=n+2) * * * 13) Existent mapping is a right aligned subset of the requested one, hence * replace the existent one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 14) Existent mapping is a centered subset of the requested one, hence * replace the existent one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 3 * req: |----------------| (bo_offset=n) * * 0 a 3 * new: |----------------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 15) Existent mappings is overlapped at the beginning by the requested mapping * backed by a different BO. Hence, map the requested mapping and split up * the existent one, adjusting its BO offset accordingly. * * :: * * 1 a 3 * old: |-----------| (bo_offset=n) * * 0 b 2 * req: |-----------| (bo_offset=m) * * 0 b 2 a' 3 * new: |-----------|-----| (b.bo_offset=m,a.bo_offset=n+2) */ /** * DOC: Locking * * In terms of managing &drm_gpuva entries DRM GPUVM does not take care of * locking itself, it is the drivers responsibility to take care about locking. * Drivers might want to protect the following operations: inserting, removing * and iterating &drm_gpuva objects as well as generating all kinds of * operations, such as split / merge or prefetch. * * DRM GPUVM also does not take care of the locking of the backing * &drm_gem_object buffers GPU VA lists and &drm_gpuvm_bo abstractions by * itself; drivers are responsible to enforce mutual exclusion using either the * GEMs dma_resv lock or alternatively a driver specific external lock. For the * latter see also drm_gem_gpuva_set_lock(). * * However, DRM GPUVM contains lockdep checks to ensure callers of its API hold * the corresponding lock whenever the &drm_gem_objects GPU VA list is accessed * by functions such as drm_gpuva_link() or drm_gpuva_unlink(), but also * drm_gpuvm_bo_obtain() and drm_gpuvm_bo_put(). * * The latter is required since on creation and destruction of a &drm_gpuvm_bo * the &drm_gpuvm_bo is attached / removed from the &drm_gem_objects gpuva list. * Subsequent calls to drm_gpuvm_bo_obtain() for the same &drm_gpuvm and * &drm_gem_object must be able to observe previous creations and destructions * of &drm_gpuvm_bos in order to keep instances unique. * * The &drm_gpuvm's lists for keeping track of external and evicted objects are * protected against concurrent insertion / removal and iteration internally. * * However, drivers still need ensure to protect concurrent calls to functions * iterating those lists, namely drm_gpuvm_prepare_objects() and * drm_gpuvm_validate(). * * Alternatively, drivers can set the &DRM_GPUVM_RESV_PROTECTED flag to indicate * that the corresponding &dma_resv locks are held in order to protect the * lists. If &DRM_GPUVM_RESV_PROTECTED is set, internal locking is disabled and * the corresponding lockdep checks are enabled. This is an optimization for * drivers which are capable of taking the corresponding &dma_resv locks and * hence do not require internal locking. */ /** * DOC: Examples * * This section gives two examples on how to let the DRM GPUVA Manager generate * &drm_gpuva_op in order to satisfy a given map or unmap request and how to * make use of them. * * The below code is strictly limited to illustrate the generic usage pattern. * To maintain simplicitly, it doesn't make use of any abstractions for common * code, different (asyncronous) stages with fence signalling critical paths, * any other helpers or error handling in terms of freeing memory and dropping * previously taken locks. * * 1) Obtain a list of &drm_gpuva_op to create a new mapping:: * * // Allocates a new &drm_gpuva. * struct drm_gpuva * driver_gpuva_alloc(void); * * // Typically drivers would embedd the &drm_gpuvm and &drm_gpuva * // structure in individual driver structures and lock the dma-resv with * // drm_exec or similar helpers. * int driver_mapping_create(struct drm_gpuvm *gpuvm, * u64 addr, u64 range, * struct drm_gem_object *obj, u64 offset) * { * struct drm_gpuva_ops *ops; * struct drm_gpuva_op *op * struct drm_gpuvm_bo *vm_bo; * * driver_lock_va_space(); * ops = drm_gpuvm_sm_map_ops_create(gpuvm, addr, range, * obj, offset); * if (IS_ERR(ops)) * return PTR_ERR(ops); * * vm_bo = drm_gpuvm_bo_obtain(gpuvm, obj); * if (IS_ERR(vm_bo)) * return PTR_ERR(vm_bo);Lines 1–500 of 2752. Load more
// SPDX-License-Identifier: GPL-2.0-only OR MIT /* * Copyright (c) 2022 Red Hat. * * Permission is hereby granted, free of charge, to any person obtaining a * copy of this software and associated documentation files (the "Software"), * to deal in the Software without restriction, including without limitation * the rights to use, copy, modify, merge, publish, distribute, sublicense, * and/or sell copies of the Software, and to permit persons to whom the * Software is furnished to do so, subject to the following conditions: * * The above copyright notice and this permission notice shall be included in * all copies or substantial portions of the Software. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. * * Authors: * Danilo Krummrich <dakr@redhat.com> * */ #include <drm/drm_gpuvm.h> #include <linux/interval_tree_generic.h> #include <linux/mm.h> /** * DOC: Overview * * The DRM GPU VA Manager, represented by struct drm_gpuvm keeps track of a * GPU's virtual address (VA) space and manages the corresponding virtual * mappings represented by &drm_gpuva objects. It also keeps track of the * mapping's backing &drm_gem_object buffers. * * &drm_gem_object buffers maintain a list of &drm_gpuva objects representing * all existent GPU VA mappings using this &drm_gem_object as backing buffer. * * GPU VAs can be flagged as sparse, such that drivers may use GPU VAs to also * keep track of sparse PTEs in order to support Vulkan 'Sparse Resources'. * * The GPU VA manager internally uses a rb-tree to manage the * &drm_gpuva mappings within a GPU's virtual address space. * * The &drm_gpuvm structure contains a special &drm_gpuva representing the * portion of VA space reserved by the kernel. This node is initialized together * with the GPU VA manager instance and removed when the GPU VA manager is * destroyed. * * In a typical application drivers would embed struct drm_gpuvm and * struct drm_gpuva within their own driver specific structures, there won't be * any memory allocations of its own nor memory allocations of &drm_gpuva * entries. * * The data structures needed to store &drm_gpuvas within the &drm_gpuvm are * contained within struct drm_gpuva already. Hence, for inserting &drm_gpuva * entries from within dma-fence signalling critical sections it is enough to * pre-allocate the &drm_gpuva structures. * * &drm_gem_objects which are private to a single VM can share a common * &dma_resv in order to improve locking efficiency (e.g. with &drm_exec). * For this purpose drivers must pass a &drm_gem_object to drm_gpuvm_init(), in * the following called 'resv object', which serves as the container of the * GPUVM's shared &dma_resv. This resv object can be a driver specific * &drm_gem_object, such as the &drm_gem_object containing the root page table, * but it can also be a 'dummy' object, which can be allocated with * drm_gpuvm_resv_object_alloc(). * * In order to connect a struct drm_gpuva its backing &drm_gem_object each * &drm_gem_object maintains a list of &drm_gpuvm_bo structures, and each * &drm_gpuvm_bo contains a list of &drm_gpuva structures. * * A &drm_gpuvm_bo is an abstraction that represents a combination of a * &drm_gpuvm and a &drm_gem_object. Every such combination should be unique. * This is ensured by the API through drm_gpuvm_bo_obtain() and * drm_gpuvm_bo_obtain_prealloc() which first look into the corresponding * &drm_gem_object list of &drm_gpuvm_bos for an existing instance of this * particular combination. If not existent a new instance is created and linked * to the &drm_gem_object. * * &drm_gpuvm_bo structures, since unique for a given &drm_gpuvm, are also used * as entry for the &drm_gpuvm's lists of external and evicted objects. Those * lists are maintained in order to accelerate locking of dma-resv locks and * validation of evicted objects bound in a &drm_gpuvm. For instance, all * &drm_gem_object's &dma_resv of a given &drm_gpuvm can be locked by calling * drm_gpuvm_exec_lock(). Once locked drivers can call drm_gpuvm_validate() in * order to validate all evicted &drm_gem_objects. It is also possible to lock * additional &drm_gem_objects by providing the corresponding parameters to * drm_gpuvm_exec_lock() as well as open code the &drm_exec loop while making * use of helper functions such as drm_gpuvm_prepare_range() or * drm_gpuvm_prepare_objects(). * * Every bound &drm_gem_object is treated as external object when its &dma_resv * structure is different than the &drm_gpuvm's common &dma_resv structure. */ /** * DOC: Split and Merge * * Besides its capability to manage and represent a GPU VA space, the * GPU VA manager also provides functions to let the &drm_gpuvm calculate a * sequence of operations to satisfy a given map or unmap request. * * Therefore the DRM GPU VA manager provides an algorithm implementing splitting * and merging of existent GPU VA mappings with the ones that are requested to * be mapped or unmapped. This feature is required by the Vulkan API to * implement Vulkan 'Sparse Memory Bindings' - drivers UAPIs often refer to this * as VM BIND. * * Drivers can call drm_gpuvm_sm_map() to receive a sequence of callbacks * containing map, unmap and remap operations for a given newly requested * mapping. The sequence of callbacks represents the set of operations to * execute in order to integrate the new mapping cleanly into the current state * of the GPU VA space. * * Depending on how the new GPU VA mapping intersects with the existent mappings * of the GPU VA space the &drm_gpuvm_ops callbacks contain an arbitrary amount * of unmap operations, a maximum of two remap operations and a single map * operation. The caller might receive no callback at all if no operation is * required, e.g. if the requested mapping already exists in the exact same way. * * The single map operation represents the original map operation requested by * the caller. * * &drm_gpuva_op_unmap contains a 'keep' field, which indicates whether the * &drm_gpuva to unmap is physically contiguous with the original mapping * request. Optionally, if 'keep' is set, drivers may keep the actual page table * entries for this &drm_gpuva, adding the missing page table entries only and * update the &drm_gpuvm's view of things accordingly. * * Drivers may do the same optimization, namely delta page table updates, also * for remap operations. This is possible since &drm_gpuva_op_remap consists of * one unmap operation and one or two map operations, such that drivers can * derive the page table update delta accordingly. * * Note that there can't be more than two existent mappings to split up, one at * the beginning and one at the end of the new mapping, hence there is a * maximum of two remap operations. * * Analogous to drm_gpuvm_sm_map() drm_gpuvm_sm_unmap() uses &drm_gpuvm_ops to * call back into the driver in order to unmap a range of GPU VA space. The * logic behind this function is way simpler though: For all existent mappings * enclosed by the given range unmap operations are created. For mappings which * are only partically located within the given range, remap operations are * created such that those mappings are split up and re-mapped partically. * * As an alternative to drm_gpuvm_sm_map() and drm_gpuvm_sm_unmap(), * drm_gpuvm_sm_map_ops_create() and drm_gpuvm_sm_unmap_ops_create() can be used * to directly obtain an instance of struct drm_gpuva_ops containing a list of * &drm_gpuva_op, which can be iterated with drm_gpuva_for_each_op(). This list * contains the &drm_gpuva_ops analogous to the callbacks one would receive when * calling drm_gpuvm_sm_map() or drm_gpuvm_sm_unmap(). While this way requires * more memory (to allocate the &drm_gpuva_ops), it provides drivers a way to * iterate the &drm_gpuva_op multiple times, e.g. once in a context where memory * allocations are possible (e.g. to allocate GPU page tables) and once in the * dma-fence signalling critical path. * * To update the &drm_gpuvm's view of the GPU VA space drm_gpuva_insert() and * drm_gpuva_remove() may be used. These functions can safely be used from * &drm_gpuvm_ops callbacks originating from drm_gpuvm_sm_map() or * drm_gpuvm_sm_unmap(). However, it might be more convenient to use the * provided helper functions drm_gpuva_map(), drm_gpuva_remap() and * drm_gpuva_unmap() instead. * * The following diagram depicts the basic relationships of existent GPU VA * mappings, a newly requested mapping and the resulting mappings as implemented * by drm_gpuvm_sm_map() - it doesn't cover any arbitrary combinations of these. * * 1) Requested mapping is identical. Replace it, but indicate the backing PTEs * could be kept. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=n) * * 0 a 1 * new: |-----------| (bo_offset=n) * * * 2) Requested mapping is identical, except for the BO offset, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----------| (bo_offset=m) * * 0 a 1 * new: |-----------| (bo_offset=m) * * * 3) Requested mapping is identical, except for the backing BO, hence replace * the mapping. * * :: * * 0 a 1 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----------| (bo_offset=n) * * 0 b 1 * new: |-----------| (bo_offset=n) * * * 4) Existent mapping is a left aligned subset of the requested one, hence * replace the existent one. * * :: * * 0 a 1 * old: |-----| (bo_offset=n) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 5) Requested mapping's range is a left aligned subset of the existent one, * but backed by a different BO. Hence, map the requested mapping and split * the existent one adjusting its BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 b 1 * req: |-----| (bo_offset=n) * * 0 b 1 a' 2 * new: |-----|-----| (b.bo_offset=n, a.bo_offset=n+1) * * .. note:: * We expect to see the same result for a request with a different BO * and/or non-contiguous BO offset. * * * 6) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 0 a 1 * req: |-----| (bo_offset=n) * * 0 a 1 a' 2 * new: |-----|-----| (a.bo_offset=n, a'.bo_offset=n+1) * * * 7) Requested mapping's range is a right aligned subset of the existent one, * but backed by a different BO. Hence, map the requested mapping and split * the existent one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 * new: |-----|-----| (a.bo_offset=n,b.bo_offset=m) * * * 8) Existent mapping is a superset of the requested mapping. Split it up, but * indicate that the backing PTEs could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 * new: |-----|-----| (a'.bo_offset=n, a.bo_offset=n+1) * * * 9) Existent mapping is overlapped at the end by the requested mapping backed * by a different BO. Hence, map the requested mapping and split up the * existent one, without adjusting the BO offset. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 b 3 * req: |-----------| (bo_offset=m) * * 0 a 1 b 3 * new: |-----|-----------| (a.bo_offset=n,b.bo_offset=m) * * * 10) Existent mapping is overlapped by the requested mapping, both having the * same backing BO with a contiguous offset. Indicate the backing PTEs of * the old mapping could be kept. * * :: * * 0 a 2 * old: |-----------| (bo_offset=n) * * 1 a 3 * req: |-----------| (bo_offset=n+1) * * 0 a' 1 a 3 * new: |-----|-----------| (a'.bo_offset=n, a.bo_offset=n+1) * * * 11) Requested mapping's range is a centered subset of the existent one * having a different backing BO. Hence, map the requested mapping and split * up the existent one in two mappings, adjusting the BO offset of the right * one accordingly. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 b 2 * req: |-----| (bo_offset=m) * * 0 a 1 b 2 a' 3 * new: |-----|-----|-----| (a.bo_offset=n,b.bo_offset=m,a'.bo_offset=n+2) * * * 12) Requested mapping is a contiguous subset of the existent one. Split it * up, but indicate that the backing PTEs could be kept. * * :: * * 0 a 3 * old: |-----------------| (bo_offset=n) * * 1 a 2 * req: |-----| (bo_offset=n+1) * * 0 a' 1 a 2 a'' 3 * old: |-----|-----|-----| (a'.bo_offset=n, a.bo_offset=n+1, a''.bo_offset=n+2) * * * 13) Existent mapping is a right aligned subset of the requested one, hence * replace the existent one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 2 * req: |-----------| (bo_offset=n) * * 0 a 2 * new: |-----------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 14) Existent mapping is a centered subset of the requested one, hence * replace the existent one. * * :: * * 1 a 2 * old: |-----| (bo_offset=n+1) * * 0 a 3 * req: |----------------| (bo_offset=n) * * 0 a 3 * new: |----------------| (bo_offset=n) * * .. note:: * We expect to see the same result for a request with a different bo * and/or non-contiguous bo_offset. * * * 15) Existent mappings is overlapped at the beginning by the requested mapping * backed by a different BO. Hence, map the requested mapping and split up * the existent one, adjusting its BO offset accordingly. * * :: * * 1 a 3 * old: |-----------| (bo_offset=n) * * 0 b 2 * req: |-----------| (bo_offset=m) * * 0 b 2 a' 3 * new: |-----------|-----| (b.bo_offset=m,a.bo_offset=n+2) */ /** * DOC: Locking * * In terms of managing &drm_gpuva entries DRM GPUVM does not take care of * locking itself, it is the drivers responsibility to take care about locking. * Drivers might want to protect the following operations: inserting, removing * and iterating &drm_gpuva objects as well as generating all kinds of * operations, such as split / merge or prefetch. * * DRM GPUVM also does not take care of the locking of the backing * &drm_gem_object buffers GPU VA lists and &drm_gpuvm_bo abstractions by * itself; drivers are responsible to enforce mutual exclusion using either the * GEMs dma_resv lock or alternatively a driver specific external lock. For the * latter see also drm_gem_gpuva_set_lock(). * * However, DRM GPUVM contains lockdep checks to ensure callers of its API hold * the corresponding lock whenever the &drm_gem_objects GPU VA list is accessed * by functions such as drm_gpuva_link() or drm_gpuva_unlink(), but also * drm_gpuvm_bo_obtain() and drm_gpuvm_bo_put(). * * The latter is required since on creation and destruction of a &drm_gpuvm_bo * the &drm_gpuvm_bo is attached / removed from the &drm_gem_objects gpuva list. * Subsequent calls to drm_gpuvm_bo_obtain() for the same &drm_gpuvm and * &drm_gem_object must be able to observe previous creations and destructions * of &drm_gpuvm_bos in order to keep instances unique. * * The &drm_gpuvm's lists for keeping track of external and evicted objects are * protected against concurrent insertion / removal and iteration internally. * * However, drivers still need ensure to protect concurrent calls to functions * iterating those lists, namely drm_gpuvm_prepare_objects() and * drm_gpuvm_validate(). * * Alternatively, drivers can set the &DRM_GPUVM_RESV_PROTECTED flag to indicate * that the corresponding &dma_resv locks are held in order to protect the * lists. If &DRM_GPUVM_RESV_PROTECTED is set, internal locking is disabled and * the corresponding lockdep checks are enabled. This is an optimization for * drivers which are capable of taking the corresponding &dma_resv locks and * hence do not require internal locking. */ /** * DOC: Examples * * This section gives two examples on how to let the DRM GPUVA Manager generate * &drm_gpuva_op in order to satisfy a given map or unmap request and how to * make use of them. * * The below code is strictly limited to illustrate the generic usage pattern. * To maintain simplicitly, it doesn't make use of any abstractions for common * code, different (asyncronous) stages with fence signalling critical paths, * any other helpers or error handling in terms of freeing memory and dropping * previously taken locks. * * 1) Obtain a list of &drm_gpuva_op to create a new mapping:: * * // Allocates a new &drm_gpuva. * struct drm_gpuva * driver_gpuva_alloc(void); * * // Typically drivers would embedd the &drm_gpuvm and &drm_gpuva * // structure in individual driver structures and lock the dma-resv with * // drm_exec or similar helpers. * int driver_mapping_create(struct drm_gpuvm *gpuvm, * u64 addr, u64 range, * struct drm_gem_object *obj, u64 offset) * { * struct drm_gpuva_ops *ops; * struct drm_gpuva_op *op * struct drm_gpuvm_bo *vm_bo; * * driver_lock_va_space(); * ops = drm_gpuvm_sm_map_ops_create(gpuvm, addr, range, * obj, offset); * if (IS_ERR(ops)) * return PTR_ERR(ops); * * vm_bo = drm_gpuvm_bo_obtain(gpuvm, obj); * if (IS_ERR(vm_bo)) * return PTR_ERR(vm_bo);