Modern Linux systems require large amount of graphics memory to store frame buffers, textures, vertices and other graphics-related data. Given the very dynamic nature of many of that data, managing graphics memory efficiently is thus crucial for the graphics stack and plays a central role in the DRM infrastructure.
The DRM core includes two memory managers, namely Translation Table Maps (TTM) and Graphics Execution Manager (GEM). TTM was the first DRM memory manager to be developed and tried to be a one-size-fits-them all solution. It provides a single userspace API to accommodate the need of all hardware, supporting both Unified Memory Architecture (UMA) devices and devices with dedicated video RAM (i.e. most discrete video cards). This resulted in a large, complex piece of code that turned out to be hard to use for driver development.
GEM started as an Intel-sponsored project in reaction to TTM’s complexity. Its design philosophy is completely different: instead of providing a solution to every graphics memory-related problems, GEM identified common code between drivers and created a support library to share it. GEM has simpler initialization and execution requirements than TTM, but has no video RAM management capabilities and is thus limited to UMA devices.
TTM design background and information belongs here.
Warning
This section is outdated.
Drivers wishing to support TTM must fill out a drm_bo_driver structure. The structure contains several fields with function pointers for initializing the TTM, allocating and freeing memory, waiting for command completion and fence synchronization, and memory migration. See the radeon_ttm.c file for an example of usage.
The ttm_global_reference structure is made up of several fields:
struct ttm_global_reference {
enum ttm_global_types global_type;
size_t size;
void *object;
int (*init) (struct ttm_global_reference *);
void (*release) (struct ttm_global_reference *);
};
There should be one global reference structure for your memory manager as a whole, and there will be others for each object created by the memory manager at runtime. Your global TTM should have a type of TTM_GLOBAL_TTM_MEM. The size field for the global object should be sizeof(struct ttm_mem_global), and the init and release hooks should point at your driver-specific init and release routines, which probably eventually call ttm_mem_global_init and ttm_mem_global_release, respectively.
Once your global TTM accounting structure is set up and initialized by calling ttm_global_item_ref() on it, you need to create a buffer object TTM to provide a pool for buffer object allocation by clients and the kernel itself. The type of this object should be TTM_GLOBAL_TTM_BO, and its size should be sizeof(struct ttm_bo_global). Again, driver-specific init and release functions may be provided, likely eventually calling ttm_bo_global_init() and ttm_bo_global_release(), respectively. Also, like the previous object, ttm_global_item_ref() is used to create an initial reference count for the TTM, which will call your initialization function.
The GEM design approach has resulted in a memory manager that doesn’t provide full coverage of all (or even all common) use cases in its userspace or kernel API. GEM exposes a set of standard memory-related operations to userspace and a set of helper functions to drivers, and let drivers implement hardware-specific operations with their own private API.
The GEM userspace API is described in the GEM - the Graphics Execution Manager article on LWN. While slightly outdated, the document provides a good overview of the GEM API principles. Buffer allocation and read and write operations, described as part of the common GEM API, are currently implemented using driver-specific ioctls.
GEM is data-agnostic. It manages abstract buffer objects without knowing what individual buffers contain. APIs that require knowledge of buffer contents or purpose, such as buffer allocation or synchronization primitives, are thus outside of the scope of GEM and must be implemented using driver-specific ioctls.
On a fundamental level, GEM involves several operations:
Buffer object allocation is relatively straightforward and largely provided by Linux’s shmem layer, which provides memory to back each object.
Device-specific operations, such as command execution, pinning, buffer read & write, mapping, and domain ownership transfers are left to driver-specific ioctls.
Drivers that use GEM must set the DRIVER_GEM bit in the struct struct drm_driver driver_features field. The DRM core will then automatically initialize the GEM core before calling the load operation. Behind the scene, this will create a DRM Memory Manager object which provides an address space pool for object allocation.
In a KMS configuration, drivers need to allocate and initialize a command ring buffer following core GEM initialization if required by the hardware. UMA devices usually have what is called a “stolen” memory region, which provides space for the initial framebuffer and large, contiguous memory regions required by the device. This space is typically not managed by GEM, and must be initialized separately into its own DRM MM object.
GEM splits creation of GEM objects and allocation of the memory that backs them in two distinct operations.
GEM objects are represented by an instance of struct struct drm_gem_object. Drivers usually need to extend GEM objects with private information and thus create a driver-specific GEM object structure type that embeds an instance of struct struct drm_gem_object.
To create a GEM object, a driver allocates memory for an instance of its specific GEM object type and initializes the embedded struct struct drm_gem_object with a call to drm_gem_object_init(). The function takes a pointer to the DRM device, a pointer to the GEM object and the buffer object size in bytes.
GEM uses shmem to allocate anonymous pageable memory. drm_gem_object_init() will create an shmfs file of the requested size and store it into the struct struct drm_gem_object filp field. The memory is used as either main storage for the object when the graphics hardware uses system memory directly or as a backing store otherwise.
Drivers are responsible for the actual physical pages allocation by calling shmem_read_mapping_page_gfp() for each page. Note that they can decide to allocate pages when initializing the GEM object, or to delay allocation until the memory is needed (for instance when a page fault occurs as a result of a userspace memory access or when the driver needs to start a DMA transfer involving the memory).
Anonymous pageable memory allocation is not always desired, for instance when the hardware requires physically contiguous system memory as is often the case in embedded devices. Drivers can create GEM objects with no shmfs backing (called private GEM objects) by initializing them with a call to drm_gem_private_object_init() instead of drm_gem_object_init(). Storage for private GEM objects must be managed by drivers.
All GEM objects are reference-counted by the GEM core. References can be acquired and release by calling drm_gem_object_reference() and drm_gem_object_unreference() respectively. The caller must hold the struct drm_device struct_mutex lock when calling drm_gem_object_reference(). As a convenience, GEM provides drm_gem_object_unreference_unlocked() functions that can be called without holding the lock.
When the last reference to a GEM object is released the GEM core calls the struct drm_driver gem_free_object operation. That operation is mandatory for GEM-enabled drivers and must free the GEM object and all associated resources.
void (*gem_free_object) (struct drm_gem_object *obj); Drivers are responsible for freeing all GEM object resources. This includes the resources created by the GEM core, which need to be released with drm_gem_object_release().
Communication between userspace and the kernel refers to GEM objects using local handles, global names or, more recently, file descriptors. All of those are 32-bit integer values; the usual Linux kernel limits apply to the file descriptors.
GEM handles are local to a DRM file. Applications get a handle to a GEM object through a driver-specific ioctl, and can use that handle to refer to the GEM object in other standard or driver-specific ioctls. Closing a DRM file handle frees all its GEM handles and dereferences the associated GEM objects.
To create a handle for a GEM object drivers call drm_gem_handle_create(). The function takes a pointer to the DRM file and the GEM object and returns a locally unique handle. When the handle is no longer needed drivers delete it with a call to drm_gem_handle_delete(). Finally the GEM object associated with a handle can be retrieved by a call to drm_gem_object_lookup().
Handles don’t take ownership of GEM objects, they only take a reference to the object that will be dropped when the handle is destroyed. To avoid leaking GEM objects, drivers must make sure they drop the reference(s) they own (such as the initial reference taken at object creation time) as appropriate, without any special consideration for the handle. For example, in the particular case of combined GEM object and handle creation in the implementation of the dumb_create operation, drivers must drop the initial reference to the GEM object before returning the handle.
GEM names are similar in purpose to handles but are not local to DRM files. They can be passed between processes to reference a GEM object globally. Names can’t be used directly to refer to objects in the DRM API, applications must convert handles to names and names to handles using the DRM_IOCTL_GEM_FLINK and DRM_IOCTL_GEM_OPEN ioctls respectively. The conversion is handled by the DRM core without any driver-specific support.
GEM also supports buffer sharing with dma-buf file descriptors through PRIME. GEM-based drivers must use the provided helpers functions to implement the exporting and importing correctly. See ?. Since sharing file descriptors is inherently more secure than the easily guessable and global GEM names it is the preferred buffer sharing mechanism. Sharing buffers through GEM names is only supported for legacy userspace. Furthermore PRIME also allows cross-device buffer sharing since it is based on dma-bufs.
Because mapping operations are fairly heavyweight GEM favours read/write-like access to buffers, implemented through driver-specific ioctls, over mapping buffers to userspace. However, when random access to the buffer is needed (to perform software rendering for instance), direct access to the object can be more efficient.
The mmap system call can’t be used directly to map GEM objects, as they don’t have their own file handle. Two alternative methods currently co-exist to map GEM objects to userspace. The first method uses a driver-specific ioctl to perform the mapping operation, calling do_mmap() under the hood. This is often considered dubious, seems to be discouraged for new GEM-enabled drivers, and will thus not be described here.
The second method uses the mmap system call on the DRM file handle. void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset); DRM identifies the GEM object to be mapped by a fake offset passed through the mmap offset argument. Prior to being mapped, a GEM object must thus be associated with a fake offset. To do so, drivers must call drm_gem_create_mmap_offset() on the object.
Once allocated, the fake offset value must be passed to the application in a driver-specific way and can then be used as the mmap offset argument.
The GEM core provides a helper method drm_gem_mmap() to handle object mapping. The method can be set directly as the mmap file operation handler. It will look up the GEM object based on the offset value and set the VMA operations to the struct drm_driver gem_vm_ops field. Note that drm_gem_mmap() doesn’t map memory to userspace, but relies on the driver-provided fault handler to map pages individually.
To use drm_gem_mmap(), drivers must fill the struct struct drm_driver gem_vm_ops field with a pointer to VM operations.
struct vm_operations_struct *gem_vm_ops struct vm_operations_struct { void (*open)(struct vm_area_struct * area); void (*close)(struct vm_area_struct * area); int (*fault)(struct vm_area_struct *vma, struct vm_fault *vmf); };
The open and close operations must update the GEM object reference count. Drivers can use the drm_gem_vm_open() and drm_gem_vm_close() helper functions directly as open and close handlers.
The fault operation handler is responsible for mapping individual pages to userspace when a page fault occurs. Depending on the memory allocation scheme, drivers can allocate pages at fault time, or can decide to allocate memory for the GEM object at the time the object is created.
Drivers that want to map the GEM object upfront instead of handling page faults can implement their own mmap file operation handler.
When mapped to the device or used in a command buffer, backing pages for an object are flushed to memory and marked write combined so as to be coherent with the GPU. Likewise, if the CPU accesses an object after the GPU has finished rendering to the object, then the object must be made coherent with the CPU’s view of memory, usually involving GPU cache flushing of various kinds. This core CPU<->GPU coherency management is provided by a device-specific ioctl, which evaluates an object’s current domain and performs any necessary flushing or synchronization to put the object into the desired coherency domain (note that the object may be busy, i.e. an active render target; in that case, setting the domain blocks the client and waits for rendering to complete before performing any necessary flushing operations).
Perhaps the most important GEM function for GPU devices is providing a command execution interface to clients. Client programs construct command buffers containing references to previously allocated memory objects, and then submit them to GEM. At that point, GEM takes care to bind all the objects into the GTT, execute the buffer, and provide necessary synchronization between clients accessing the same buffers. This often involves evicting some objects from the GTT and re-binding others (a fairly expensive operation), and providing relocation support which hides fixed GTT offsets from clients. Clients must take care not to submit command buffers that reference more objects than can fit in the GTT; otherwise, GEM will reject them and no rendering will occur. Similarly, if several objects in the buffer require fence registers to be allocated for correct rendering (e.g. 2D blits on pre-965 chips), care must be taken not to require more fence registers than are available to the client. Such resource management should be abstracted from the client in libdrm.
initialize an allocated shmem-backed GEM object
Parameters
Description
Initialize an already allocated GEM object of the specified size with shmfs backing store.
initialize an allocated private GEM object
Parameters
Description
Initialize an already allocated GEM object of the specified size with no GEM provided backing store. Instead the caller is responsible for backing the object and handling it.
deletes the given file-private handle
Parameters
Description
Removes the GEM handle from the filp lookup table which has been added with drm_gem_handle_create(). If this is the last handle also cleans up linked resources like GEM names.
dumb fb callback helper for gem based drivers
Parameters
Description
This implements the ->dumb_destroy kms driver callback for drivers which use gem to manage their backing storage.
create a gem handle for an object
Parameters
Description
Create a handle for this object. This adds a handle reference to the object, which includes a regular reference count. Callers will likely want to dereference the object afterwards.
release a fake mmap offset for an object
Parameters
Description
This routine frees fake offsets allocated by drm_gem_create_mmap_offset().
Note that drm_gem_object_release() already calls this function, so drivers don’t have to take care of releasing the mmap offset themselves when freeing the GEM object.
create a fake mmap offset for an object
Parameters
Description
GEM memory mapping works by handing back to userspace a fake mmap offset it can use in a subsequent mmap(2) call. The DRM core code then looks up the object based on the offset and sets up the various memory mapping structures.
This routine allocates and attaches a fake offset for obj, in cases where the virtual size differs from the physical size (ie. obj->size). Otherwise just use drm_gem_create_mmap_offset().
This function is idempotent and handles an already allocated mmap offset transparently. Drivers do not need to check for this case.
create a fake mmap offset for an object
Parameters
Description
GEM memory mapping works by handing back to userspace a fake mmap offset it can use in a subsequent mmap(2) call. The DRM core code then looks up the object based on the offset and sets up the various memory mapping structures.
This routine allocates and attaches a fake offset for obj.
Drivers can call drm_gem_free_mmap_offset() before freeing obj to release the fake offset again.
helper to allocate backing pages for a GEM object from shmem
Parameters
Description
This reads the page-array of the shmem-backing storage of the given gem object. An array of pages is returned. If a page is not allocated or swapped-out, this will allocate/swap-in the required pages. Note that the whole object is covered by the page-array and pinned in memory.
Use drm_gem_put_pages() to release the array and unpin all pages.
This uses the GFP-mask set on the shmem-mapping (see mapping_set_gfp_mask()). If you require other GFP-masks, you have to do those allocations yourself.
Note that you are not allowed to change gfp-zones during runtime. That is, shmem_read_mapping_page_gfp() must be called with the same gfp_zone(gfp) as set during initialization. If you have special zone constraints, set them after drm_gem_init_object() via mapping_set_gfp_mask(). shmem-core takes care to keep pages in the required zone during swap-in.
helper to free backing pages for a GEM object
Parameters
look up a GEM object from it’s handle
Parameters
Return
A reference to the object named by the handle if such exists on filp, NULL otherwise.
release GEM buffer object resources
Parameters
Description
This releases any structures and resources used by obj and is the invers of drm_gem_object_init().
free a GEM object
Parameters
Description
Called after the last reference to the object has been lost. Must be called holding drm_device->struct_mutex.
Frees the object
release a GEM BO reference
Parameters
Description
This releases a reference to obj. Callers must not hold the dev->struct_mutex lock when calling this function.
See also __drm_gem_object_unreference().
release a GEM BO reference
Parameters
Description
This releases a reference to obj. Callers must hold the dev->struct_mutex lock when calling this function, even when the driver doesn’t use dev->struct_mutex for anything.
For drivers not encumbered with legacy locking use drm_gem_object_unreference_unlocked() instead.
vma->ops->open implementation for GEM
Parameters
Description
This function implements the #vm_operations_struct open() callback for GEM drivers. This must be used together with drm_gem_vm_close().
vma->ops->close implementation for GEM
Parameters
Description
This function implements the #vm_operations_struct close() callback for GEM drivers. This must be used together with drm_gem_vm_open().
memory map a GEM object
Parameters
Description
Set up the VMA to prepare mapping of the GEM object using the gem_vm_ops provided by the driver. Depending on their requirements, drivers can either provide a fault handler in their gem_vm_ops (in which case any accesses to the object will be trapped, to perform migration, GTT binding, surface register allocation, or performance monitoring), or mmap the buffer memory synchronously after calling drm_gem_mmap_obj.
This function is mainly intended to implement the DMABUF mmap operation, when the GEM object is not looked up based on its fake offset. To implement the DRM mmap operation, drivers should use the drm_gem_mmap() function.
drm_gem_mmap_obj() assumes the user is granted access to the buffer while drm_gem_mmap() prevents unprivileged users from mapping random objects. So callers must verify access restrictions before calling this helper.
Return 0 or success or -EINVAL if the object size is smaller than the VMA size, or if no gem_vm_ops are provided.
memory map routine for GEM objects
Parameters
Description
If a driver supports GEM object mapping, mmap calls on the DRM file descriptor will end up here.
Look up the GEM object based on the offset passed in (vma->vm_pgoff will contain the fake offset we created when the GTT map ioctl was called on the object) and map it with a call to drm_gem_mmap_obj().
If the caller is not granted access to the buffer object, the mmap will fail with EACCES. Please see the vma manager for more information.
GEM buffer object
Definition
struct drm_gem_object {
struct kref refcount;
unsigned handle_count;
struct drm_device * dev;
struct file * filp;
struct drm_vma_offset_node vma_node;
size_t size;
int name;
uint32_t read_domains;
uint32_t write_domain;
uint32_t pending_read_domains;
uint32_t pending_write_domain;
struct dma_buf * dma_buf;
struct dma_buf_attachment * import_attach;
};
Members
Reference count of this object
Please use drm_gem_object_reference() to acquire and drm_gem_object_unreference() or drm_gem_object_unreference_unlocked() to release a reference to a GEM buffer object.
This is the GEM file_priv handle count of this object.
Each handle also holds a reference. Note that when the handle_count drops to 0 any global names (e.g. the id in the flink namespace) will be cleared.
Protected by dev->object_name_lock.
Mapping info for this object to support mmap. Drivers are supposed to allocate the mmap offset using drm_gem_create_mmap_offset(). The offset itself can be retrieved using drm_vma_node_offset_addr().
Memory mapping itself is handled by drm_gem_mmap(), which also checks that userspace is allowed to access the object.
dma-buf associated with this GEM object.
Pointer to the dma-buf associated with this gem object (either through importing or exporting). We break the resulting reference loop when the last gem handle for this object is released.
Protected by obj->object_name_lock.
dma-buf attachment backing this object.
Any foreign dma_buf imported as a gem object has this set to the attachment point for the device. This is invariant over the lifetime of a gem object.
The driver’s ->gem_free_object callback is responsible for cleaning up the dma_buf attachment and references acquired at import time.
Note that the drm gem/prime core does not depend upon drivers setting this field any more. So for drivers where this doesn’t make sense (e.g. virtual devices or a displaylink behind an usb bus) they can simply leave it as NULL.
Description
This structure defines the generic parts for GEM buffer objects, which are mostly around handling mmap and userspace handles.
Buffer objects are often abbreviated to BO.
acquire a GEM BO reference
Parameters
Description
This acquires additional reference to obj. It is illegal to call this without already holding a reference. No locks required.
raw function to release a GEM BO reference
Parameters
Description
This function is meant to be used by drivers which are not encumbered with dev->struct_mutex legacy locking and which are using the gem_free_object_unlocked callback. It avoids all the locking checks and locking overhead of drm_gem_object_unreference() and drm_gem_object_unreference_unlocked().
Drivers should never call this directly in their code. Instead they should wrap it up into a driver_gem_object_unreference(struct driver_gem_object *obj) wrapper function, and use that. Shared code should never call this, to avoid breaking drivers by accident which still depend upon dev->struct_mutex locking.
The vma-manager is responsible to map arbitrary driver-dependent memory regions into the linear user address-space. It provides offsets to the caller which can then be used on the address_space of the drm-device. It takes care to not overlap regions, size them appropriately and to not confuse mm-core by inconsistent fake vm_pgoff fields. Drivers shouldn’t use this for object placement in VMEM. This manager should only be used to manage mappings into linear user-space VMs.
We use drm_mm as backend to manage object allocations. But it is highly optimized for alloc/free calls, not lookups. Hence, we use an rb-tree to speed up offset lookups.
You must not use multiple offset managers on a single address_space. Otherwise, mm-core will be unable to tear down memory mappings as the VM will no longer be linear.
This offset manager works on page-based addresses. That is, every argument and return code (with the exception of drm_vma_node_offset_addr()) is given in number of pages, not number of bytes. That means, object sizes and offsets must always be page-aligned (as usual). If you want to get a valid byte-based user-space address for a given offset, please see drm_vma_node_offset_addr().
Additionally to offset management, the vma offset manager also handles access management. For every open-file context that is allowed to access a given node, you must call drm_vma_node_allow(). Otherwise, an mmap() call on this open-file with the offset of the node will fail with -EACCES. To revoke access again, use drm_vma_node_revoke(). However, the caller is responsible for destroying already existing mappings, if required.
Initialize new offset-manager
Parameters
Description
Initialize a new offset-manager. The offset and area size available for the manager are given as page_offset and size. Both are interpreted as page-numbers, not bytes.
Adding/removing nodes from the manager is locked internally and protected against concurrent access. However, node allocation and destruction is left for the caller. While calling into the vma-manager, a given node must always be guaranteed to be referenced.
Destroy offset manager
Parameters
Description
Destroy an object manager which was previously created via drm_vma_offset_manager_init(). The caller must remove all allocated nodes before destroying the manager. Otherwise, drm_mm will refuse to free the requested resources.
The manager must not be accessed after this function is called.
Find node in offset space
Parameters
Description
Find a node given a start address and object size. This returns the _best_ match for the given node. That is, start may point somewhere into a valid region and the given node will be returned, as long as the node spans the whole requested area (given the size in number of pages as pages).
Note that before lookup the vma offset manager lookup lock must be acquired with drm_vma_offset_lock_lookup(). See there for an example. This can then be used to implement weakly referenced lookups using kref_get_unless_zero().
Example
drm_vma_offset_lock_lookup(mgr);
node = drm_vma_offset_lookup_locked(mgr);
if (node)
kref_get_unless_zero(container_of(node, sth, entr));
drm_vma_offset_unlock_lookup(mgr);
Return
Returns NULL if no suitable node can be found. Otherwise, the best match is returned. It’s the caller’s responsibility to make sure the node doesn’t get destroyed before the caller can access it.
Add offset node to manager
Parameters
Description
Add a node to the offset-manager. If the node was already added, this does nothing and return 0. pages is the size of the object given in number of pages. After this call succeeds, you can access the offset of the node until it is removed again.
If this call fails, it is safe to retry the operation or call drm_vma_offset_remove(), anyway. However, no cleanup is required in that case.
pages is not required to be the same size as the underlying memory object that you want to map. It only limits the size that user-space can map into their address space.
Return
0 on success, negative error code on failure.
Remove offset node from manager
Parameters
Description
Remove a node from the offset manager. If the node wasn’t added before, this does nothing. After this call returns, the offset and size will be 0 until a new offset is allocated via drm_vma_offset_add() again. Helper functions like drm_vma_node_start() and drm_vma_node_offset_addr() will return 0 if no offset is allocated.
Add open-file to list of allowed users
Parameters
Description
Add filp to the list of allowed open-files for this node. If filp is already on this list, the ref-count is incremented.
The list of allowed-users is preserved across drm_vma_offset_add() and drm_vma_offset_remove() calls. You may even call it if the node is currently not added to any offset-manager.
You must remove all open-files the same number of times as you added them before destroying the node. Otherwise, you will leak memory.
This is locked against concurrent access internally.
Return
0 on success, negative error code on internal failure (out-of-mem)
Remove open-file from list of allowed users
Parameters
Description
Decrement the ref-count of filp in the list of allowed open-files on node. If the ref-count drops to zero, remove filp from the list. You must call this once for every drm_vma_node_allow() on filp.
This is locked against concurrent access internally.
If filp is not on the list, nothing is done.
Check whether an open-file is granted access
Parameters
Description
Search the list in node whether filp is currently on the list of allowed open-files (see drm_vma_node_allow()).
This is locked against concurrent access internally.
Return
true iff filp is on the list
Look up node by exact address
Parameters
Description
Same as drm_vma_offset_lookup_locked() but does not allow any offset into the node. It only returns the exact object with the given start address.
Return
Node at exact start address start.
Lock lookup for extended private use
Parameters
Description
Lock VMA manager for extended lookups. Only locked VMA function calls are allowed while holding this lock. All other contexts are blocked from VMA until the lock is released via drm_vma_offset_unlock_lookup().
Use this if you need to take a reference to the objects returned by drm_vma_offset_lookup_locked() before releasing this lock again.
This lock must not be used for anything else than extended lookups. You must not call any other VMA helpers while holding this lock.
Note
You’re in atomic-context while holding this lock!
Unlock lookup for extended private use
Parameters
Description
Release lookup-lock. See drm_vma_offset_lock_lookup() for more information.
Initialize or reset node object
Parameters
Description
Reset a node to its initial state. This must be called before using it with any VMA offset manager.
This must not be called on an already allocated node, or you will leak memory.
Return start address for page-based addressing
Parameters
Description
Return the start address of the given node. This can be used as offset into the linear VM space that is provided by the VMA offset manager. Note that this can only be used for page-based addressing. If you need a proper offset for user-space mappings, you must apply “<< PAGE_SHIFT” or use the drm_vma_node_offset_addr() helper instead.
Return
Start address of node for page-based addressing. 0 if the node does not have an offset allocated.
Return size (page-based)
Parameters
Description
Return the size as number of pages for the given node. This is the same size that was passed to drm_vma_offset_add(). If no offset is allocated for the node, this is 0.
Return
Size of node as number of pages. 0 if the node does not have an offset allocated.
Return sanitized offset for user-space mmaps
Parameters
Description
Same as drm_vma_node_start() but returns the address as a valid offset that can be used for user-space mappings during mmap(). This must not be called on unlinked nodes.
Return
Offset of node for byte-based addressing. 0 if the node does not have an object allocated.
Unmap offset node
Parameters
Description
Unmap all userspace mappings for a given offset node. The mappings must be associated with the file_mapping address-space. If no offset exists nothing is done.
This call is unlocked. The caller must guarantee that drm_vma_offset_remove() is not called on this node concurrently.
Access verification helper for TTM
Parameters
Description
This checks whether filp is granted access to node. It is the same as drm_vma_node_is_allowed() but suitable as drop-in helper for TTM verify_access() callbacks.
Return
0 if access is granted, -EACCES otherwise.
PRIME is the cross device buffer sharing framework in drm, originally created for the OPTIMUS range of multi-gpu platforms. To userspace PRIME buffers are dma-buf based file descriptors.
Similar to GEM global names, PRIME file descriptors are also used to share buffer objects across processes. They offer additional security: as file descriptors must be explicitly sent over UNIX domain sockets to be shared between applications, they can’t be guessed like the globally unique GEM names.
Drivers that support the PRIME API must set the DRIVER_PRIME bit in the struct struct drm_driver driver_features field, and implement the prime_handle_to_fd and prime_fd_to_handle operations.
int (*prime_handle_to_fd)(struct drm_device *dev, struct drm_file *file_priv, uint32_t handle, uint32_t flags, int *prime_fd); int (*prime_fd_to_handle)(struct drm_device *dev, struct drm_file *file_priv, int prime_fd, uint32_t *handle); Those two operations convert a handle to a PRIME file descriptor and vice versa. Drivers must use the kernel dma-buf buffer sharing framework to manage the PRIME file descriptors. Similar to the mode setting API PRIME is agnostic to the underlying buffer object manager, as long as handles are 32bit unsigned integers.
While non-GEM drivers must implement the operations themselves, GEM drivers must use the drm_gem_prime_handle_to_fd() and drm_gem_prime_fd_to_handle() helper functions. Those helpers rely on the driver gem_prime_export and gem_prime_import operations to create a dma-buf instance from a GEM object (dma-buf exporter role) and to create a GEM object from a dma-buf instance (dma-buf importer role).
struct dma_buf * (*gem_prime_export)(struct drm_device *dev, struct drm_gem_object *obj, int flags); struct drm_gem_object * (*gem_prime_import)(struct drm_device *dev, struct dma_buf *dma_buf); These two operations are mandatory for GEM drivers that support PRIME.
Drivers can implement gem_prime_export and gem_prime_import in terms of simpler APIs by using the helper functions drm_gem_prime_export and drm_gem_prime_import. These functions implement dma-buf support in terms of six lower-level driver callbacks:
Export callbacks:
- gem_prime_pin (optional): prepare a GEM object for exporting
- gem_prime_get_sg_table: provide a scatter/gather table of pinned pages
- gem_prime_vmap: vmap a buffer exported by your driver
- gem_prime_vunmap: vunmap a buffer exported by your driver
- gem_prime_mmap (optional): mmap a buffer exported by your driver
Import callback:
- gem_prime_import_sg_table (import): produce a GEM object from another driver’s scatter/gather table
dma_buf release implementation for GEM
Parameters
Description
Generic release function for dma_bufs exported as PRIME buffers. GEM drivers must use this in their dma_buf ops structure as the release callback.
helper library implementation of the export callback
Parameters
Description
This is the implementation of the gem_prime_export functions for GEM drivers using the PRIME helpers.
PRIME export function for GEM drivers
Parameters
Description
This is the PRIME export function which must be used mandatorily by GEM drivers to ensure correct lifetime management of the underlying GEM object. The actual exporting from GEM object to a dma-buf is done through the gem_prime_export driver callback.
helper library implementation of the import callback
Parameters
Description
This is the implementation of the gem_prime_import functions for GEM drivers using the PRIME helpers.
PRIME import function for GEM drivers
Parameters
Description
This is the PRIME import function which must be used mandatorily by GEM drivers to ensure correct lifetime management of the underlying GEM object. The actual importing of GEM object from the dma-buf is done through the gem_import_export driver callback.
converts a page array into an sg list
Parameters
Description
This helper creates an sg table object from a set of pages the driver is responsible for mapping the pages into the importers address space for use with dma_buf itself.
convert an sg table into a page array
Parameters
Description
Exports an sg table into an array of pages and addresses. This is currently required by the TTM driver in order to do correct fault handling.
helper to clean up a PRIME-imported GEM object
Parameters
Description
This is the cleanup functions which GEM drivers need to call when they use drm_gem_prime_import to import dma-bufs.
drm_mm provides a simple range allocator. The drivers are free to use the resource allocator from the linux core if it suits them, the upside of drm_mm is that it’s in the DRM core. Which means that it’s easier to extend for some of the crazier special purpose needs of gpus.
The main data struct is drm_mm, allocations are tracked in drm_mm_node. Drivers are free to embed either of them into their own suitable datastructures. drm_mm itself will not do any allocations of its own, so if drivers choose not to embed nodes they need to still allocate them themselves.
The range allocator also supports reservation of preallocated blocks. This is useful for taking over initial mode setting configurations from the firmware, where an object needs to be created which exactly matches the firmware’s scanout target. As long as the range is still free it can be inserted anytime after the allocator is initialized, which helps with avoiding looped depencies in the driver load sequence.
drm_mm maintains a stack of most recently freed holes, which of all simplistic datastructures seems to be a fairly decent approach to clustering allocations and avoiding too much fragmentation. This means free space searches are O(num_holes). Given that all the fancy features drm_mm supports something better would be fairly complex and since gfx thrashing is a fairly steep cliff not a real concern. Removing a node again is O(1).
drm_mm supports a few features: Alignment and range restrictions can be supplied. Further more every drm_mm_node has a color value (which is just an opaqua unsigned long) which in conjunction with a driver callback can be used to implement sophisticated placement restrictions. The i915 DRM driver uses this to implement guard pages between incompatible caching domains in the graphics TT.
Two behaviors are supported for searching and allocating: bottom-up and top-down. The default is bottom-up. Top-down allocation can be used if the memory area has different restrictions, or just to reduce fragmentation.
Finally iteration helpers to walk all nodes and all holes are provided as are some basic allocator dumpers for debugging.
Very often GPUs need to have continuous allocations for a given object. When evicting objects to make space for a new one it is therefore not most efficient when we simply start to select all objects from the tail of an LRU until there’s a suitable hole: Especially for big objects or nodes that otherwise have special allocation constraints there’s a good chance we evict lots of (smaller) objects unecessarily.
The DRM range allocator supports this use-case through the scanning interfaces. First a scan operation needs to be initialized with drm_mm_init_scan() or drm_mm_init_scan_with_range(). The the driver adds objects to the roaster (probably by walking an LRU list, but this can be freely implemented) until a suitable hole is found or there’s no further evitable object.
The the driver must walk through all objects again in exactly the reverse order to restore the allocator state. Note that while the allocator is used in the scan mode no other operation is allowed.
Finally the driver evicts all objects selected in the scan. Adding and removing an object is O(1), and since freeing a node is also O(1) the overall complexity is O(scanned_objects). So like the free stack which needs to be walked before a scan operation even begins this is linear in the number of objects. It doesn’t seem to hurt badly.
insert an pre-initialized node
Parameters
Description
This functions inserts an already set-up drm_mm_node into the allocator, meaning that start, size and color must be set by the caller. This is useful to initialize the allocator with preallocated objects which must be set-up before the range allocator can be set-up, e.g. when taking over a firmware framebuffer.
Return
0 on success, -ENOSPC if there’s no hole where node is.
search for space and insert node
Parameters
Description
The preallocated node must be cleared to 0.
Return
0 on success, -ENOSPC if there’s no suitable hole.
ranged search for space and insert node
Parameters
Description
The preallocated node must be cleared to 0.
Return
0 on success, -ENOSPC if there’s no suitable hole.
Remove a memory node from the allocator.
Parameters
Description
This just removes a node from its drm_mm allocator. The node does not need to be cleared again before it can be re-inserted into this or any other drm_mm allocator. It is a bug to call this function on a un-allocated node.
move an allocation from old to new
Parameters
Description
This is useful for when drivers embed the drm_mm_node structure and hence can’t move allocations by reassigning pointers. It’s a combination of remove and insert with the guarantee that the allocation start will match.
initialize lru scanning
Parameters
Description
This simply sets up the scanning routines with the parameters for the desired hole. Note that there’s no need to specify allocation flags, since they only change the place a node is allocated from within a suitable hole.
Warning: As long as the scan list is non-empty, no other operations than adding/removing nodes to/from the scan list are allowed.
initialize range-restricted lru scanning
Parameters
Description
This simply sets up the scanning routines with the parameters for the desired hole. Note that there’s no need to specify allocation flags, since they only change the place a node is allocated from within a suitable hole.
Warning: As long as the scan list is non-empty, no other operations than adding/removing nodes to/from the scan list are allowed.
add a node to the scan list
Parameters
Description
Add a node to the scan list that might be freed to make space for the desired hole.
Return
True if a hole has been found, false otherwise.
remove a node from the scan list
Parameters
Description
Nodes _must_ be removed in the exact same order from the scan list as they have been added, otherwise the internal state of the memory manager will be corrupted.
When the scan list is empty, the selected memory nodes can be freed. An immediately following drm_mm_search_free with !DRM_MM_SEARCH_BEST will then return the just freed block (because its at the top of the free_stack list).
Return
True if this block should be evicted, false otherwise. Will always return false when no hole has been found.
checks whether an allocator is clean
Parameters
Return
True if the allocator is completely free, false if there’s still a node allocated in it.
initialize a drm-mm allocator
Parameters
Description
Note that mm must be cleared to 0 before calling this function.
clean up a drm_mm allocator
Parameters
Description
Note that it is a bug to call this function on an allocator which is not clean.
dump allocator state to dmesg
Parameters
dump allocator state to a seq_file
Parameters
checks whether a node is allocated
Parameters
Description
Drivers should use this helpers for proper encapusulation of drm_mm internals.
Return
True if the node is allocated.
checks whether an allocator is initialized
Parameters
Description
Drivers should use this helpers for proper encapusulation of drm_mm internals.
Return
True if the mm is initialized.
computes the start of the hole following node
Parameters
Description
This is useful for driver-sepific debug dumpers. Otherwise drivers should not inspect holes themselves. Drivers must check first whether a hole indeed follows by looking at node->hole_follows.
Return
Start of the subsequent hole.
computes the end of the hole following node
Parameters
Description
This is useful for driver-sepific debug dumpers. Otherwise drivers should not inspect holes themselves. Drivers must check first whether a hole indeed follows by looking at node->hole_follows.
Return
End of the subsequent hole.
iterator to walk over all allocated nodes
Parameters
Description
This iterator walks over all nodes in the range allocator. It is implemented with list_for_each, so not save against removal of elements.
iterator to walk over all holes
Parameters
Description
This iterator walks over all holes in the range allocator. It is implemented with list_for_each, so not save against removal of elements. entry is used internally and will not reflect a real drm_mm_node for the very first hole. Hence users of this iterator may not access it.
Implementation Note: We need to inline list_for_each_entry in order to be able to set hole_start and hole_end on each iteration while keeping the macro sane.
The __drm_mm_for_each_hole version is similar, but with added support for going backwards.
search for space and insert node
Parameters
Description
This is a simplified version of drm_mm_insert_node_generic() with color set to 0.
The preallocated node must be cleared to 0.
Return
0 on success, -ENOSPC if there’s no suitable hole.
ranged search for space and insert node
Parameters
Description
This is a simplified version of drm_mm_insert_node_in_range_generic() with color set to 0.
The preallocated node must be cleared to 0.
Return
0 on success, -ENOSPC if there’s no suitable hole.
The Contiguous Memory Allocator reserves a pool of memory at early boot that is used to service requests for large blocks of contiguous memory.
The DRM GEM/CMA helpers use this allocator as a means to provide buffer objects that are physically contiguous in memory. This is useful for display drivers that are unable to map scattered buffers via an IOMMU.
allocate an object with the given size
Parameters
Description
This function creates a CMA GEM object and allocates a contiguous chunk of memory as backing store. The backing memory has the writecombine attribute set.
Return
A struct drm_gem_cma_object * on success or an ERR_PTR()-encoded negative error code on failure.
free resources associated with a CMA GEM object
Parameters
Description
This function frees the backing memory of the CMA GEM object, cleans up the GEM object state and frees the memory used to store the object itself. Drivers using the CMA helpers should set this as their DRM driver’s ->:c:func:gem_free_object() callback.
create a dumb buffer object
Parameters
Description
This aligns the pitch and size arguments to the minimum required. This is an internal helper that can be wrapped by a driver to account for hardware with more specific alignment requirements. It should not be used directly as the ->:c:func:dumb_create() callback in a DRM driver.
Return
0 on success or a negative error code on failure.
create a dumb buffer object
Parameters
Description
This function computes the pitch of the dumb buffer and rounds it up to an integer number of bytes per pixel. Drivers for hardware that doesn’t have any additional restrictions on the pitch can directly use this function as their ->:c:func:dumb_create() callback.
For hardware with additional restrictions, drivers can adjust the fields set up by userspace and pass the IOCTL data along to the drm_gem_cma_dumb_create_internal() function.
Return
0 on success or a negative error code on failure.
return the fake mmap offset for a CMA GEM object
Parameters
Description
This function look up an object by its handle and returns the fake mmap offset associated with it. Drivers using the CMA helpers should set this as their DRM driver’s ->:c:func:dumb_map_offset() callback.
Return
0 on success or a negative error code on failure.
memory-map a CMA GEM object
Parameters
Description
This function implements an augmented version of the GEM DRM file mmap operation for CMA objects: In addition to the usual GEM VMA setup it immediately faults in the entire object instead of using on-demaind faulting. Drivers which employ the CMA helpers should use this function as their ->:c:func:mmap() handler in the DRM device file’s file_operations structure.
Return
0 on success or a negative error code on failure.
describe a CMA GEM object for debugfs
Parameters
Description
This function can be used to dump a human-readable representation of the CMA GEM object into a synthetic file.
provide a scatter/gather table of pinned pages for a CMA GEM object
Parameters
Description
This function exports a scatter/gather table suitable for PRIME usage by calling the standard DMA mapping API. Drivers using the CMA helpers should set this as their DRM driver’s ->:c:func:gem_prime_get_sg_table() callback.
Return
A pointer to the scatter/gather table of pinned pages or NULL on failure.
produce a CMA GEM object from another driver’s scatter/gather table of pinned pages
Parameters
Description
This function imports a scatter/gather table exported via DMA-BUF by another driver. Imported buffers must be physically contiguous in memory (i.e. the scatter/gather table must contain a single entry). Drivers that use the CMA helpers should set this as their DRM driver’s ->:c:func:gem_prime_import_sg_table() callback.
Return
A pointer to a newly created GEM object or an ERR_PTR-encoded negative error code on failure.
memory-map an exported CMA GEM object
Parameters
Description
This function maps a buffer imported via DRM PRIME into a userspace process’s address space. Drivers that use the CMA helpers should set this as their DRM driver’s ->:c:func:gem_prime_mmap() callback.
Return
0 on success or a negative error code on failure.
map a CMA GEM object into the kernel’s virtual address space
Parameters
Description
This function maps a buffer exported via DRM PRIME into the kernel’s virtual address space. Since the CMA buffers are already mapped into the kernel virtual address space this simply returns the cached virtual address. Drivers using the CMA helpers should set this as their DRM driver’s ->:c:func:gem_prime_vmap() callback.
Return
The kernel virtual address of the CMA GEM object’s backing store.
unmap a CMA GEM object from the kernel’s virtual address space
Parameters
Description
This function removes a buffer exported via DRM PRIME from the kernel’s virtual address space. This is a no-op because CMA buffers cannot be unmapped from kernel space. Drivers using the CMA helpers should set this as their DRM driver’s ->:c:func:gem_prime_vunmap() callback.
GEM object backed by CMA memory allocations
Definition
struct drm_gem_cma_object {
struct drm_gem_object base;
dma_addr_t paddr;
struct sg_table * sgt;
void * vaddr;
};
Members