CUDA Driver API. API Reference Manual (January 2024) - page 5

 

  Index      Manuals     CUDA Driver API. API Reference Manual (January 2024)

 

Search            copyright infringement  

 

   

 

    

 

Content      ..     3      4      5      6     ..

 

 

 

CUDA Driver API. API Reference Manual (January 2024) - page 5

 

 

Modules
‣ This function uses standard default stream semantics.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D32, cuMemsetD2D32Async, cuMemsetD8, cuMemsetD8Async,
cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cuMemsetD32Async, cudaMemset2DAsync
CUresult cuMemsetD2D32 (CUdeviceptr dstDevice, size_t
dstPitch, unsigned int ui, size_t Width, size_t Height)
Initializes device memory.
Parameters
dstDevice
- Destination device pointer
dstPitch
- Pitch of destination device pointer(Unused if Height is 1)
ui
- Value to set
Width
- Width of row
Height
- Number of rows
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
Description
Sets the 2D memory range of Width 32-bit values to the specified value ui. Height specifies
the number of rows to set, and dstPitch specifies the number of bytes between each row. The
dstDevice pointer and dstPitch offset must be four byte aligned. This function performs fastest
when the pitch is one that has been passed back by cuMemAllocPitch().
|
262
Modules
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D16Async, cuMemsetD2D32Async, cuMemsetD8,
cuMemsetD8Async, cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cuMemsetD32Async,
cudaMemset2D
CUresult cuMemsetD2D32Async (CUdeviceptr dstDevice,
size_t dstPitch, unsigned int ui, size_t Width, size_t Height,
CUstream hStream)
Sets device memory.
Parameters
dstDevice
- Destination device pointer
dstPitch
- Pitch of destination device pointer(Unused if Height is 1)
ui
- Value to set
Width
- Width of row
Height
- Number of rows
hStream
- Stream identifier
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
|
263
Modules
Description
Sets the 2D memory range of Width 32-bit values to the specified value ui. Height specifies
the number of rows to set, and dstPitch specifies the number of bytes between each row. The
dstDevice pointer and dstPitch offset must be four byte aligned. This function performs fastest
when the pitch is one that has been passed back by cuMemAllocPitch().
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
‣ This function uses standard default stream semantics.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D16Async, cuMemsetD2D32, cuMemsetD8, cuMemsetD8Async,
cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cuMemsetD32Async, cudaMemset2DAsync
CUresult cuMemsetD2D8 (CUdeviceptr dstDevice, size_t
dstPitch, unsigned char uc, size_t Width, size_t Height)
Initializes device memory.
Parameters
dstDevice
- Destination device pointer
dstPitch
- Pitch of destination device pointer(Unused if Height is 1)
uc
- Value to set
Width
- Width of row
Height
- Number of rows
|
264
Modules
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
Description
Sets the 2D memory range of Width 8-bit values to the specified value uc. Height specifies the
number of rows to set, and dstPitch specifies the number of bytes between each row. This function
performs fastest when the pitch is one that has been passed back by cuMemAllocPitch().
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync,
cuMemcpyHtoD, cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange,
cuMemGetInfo, cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D16Async, cuMemsetD2D32, cuMemsetD2D32Async, cuMemsetD8,
cuMemsetD8Async, cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cuMemsetD32Async,
cudaMemset2D
CUresult cuMemsetD2D8Async (CUdeviceptr dstDevice,
size_t dstPitch, unsigned char uc, size_t Width, size_t
Height, CUstream hStream)
Sets device memory.
Parameters
dstDevice
- Destination device pointer
dstPitch
- Pitch of destination device pointer(Unused if Height is 1)
uc
- Value to set
|
265
Modules
Width
- Width of row
Height
- Number of rows
hStream
- Stream identifier
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
Description
Sets the 2D memory range of Width 8-bit values to the specified value uc. Height specifies the
number of rows to set, and dstPitch specifies the number of bytes between each row. This function
performs fastest when the pitch is one that has been passed back by cuMemAllocPitch().
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
‣ This function uses standard default stream semantics.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D16,
cuMemsetD2D16Async, cuMemsetD2D32, cuMemsetD2D32Async, cuMemsetD8,
cuMemsetD8Async, cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cuMemsetD32Async,
cudaMemset2DAsync
|
266
Modules
CUresult cuMemsetD32 (CUdeviceptr dstDevice, unsigned
int ui, size_t N)
Initializes device memory.
Parameters
dstDevice
- Destination device pointer
ui
- Value to set
N
- Number of elements
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
Description
Sets the memory range of N 32-bit values to the specified value ui. The dstDevice pointer must be
four byte aligned.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D16Async, cuMemsetD2D32, cuMemsetD2D32Async, cuMemsetD8,
cuMemsetD8Async, cuMemsetD16, cuMemsetD16Async, cuMemsetD32Async, cudaMemset
|
267
Modules
CUresult cuMemsetD32Async (CUdeviceptr dstDevice,
unsigned int ui, size_t N, CUstream hStream)
Sets device memory.
Parameters
dstDevice
- Destination device pointer
ui
- Value to set
N
- Number of elements
hStream
- Stream identifier
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
Description
Sets the memory range of N 32-bit values to the specified value ui. The dstDevice pointer must be
four byte aligned.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
‣ This function uses standard default stream semantics.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D16Async, cuMemsetD2D32, cuMemsetD2D32Async, cuMemsetD8,
cuMemsetD8Async, cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cudaMemsetAsync
|
268
Modules
CUresult cuMemsetD8 (CUdeviceptr dstDevice, unsigned
char uc, size_t N)
Initializes device memory.
Parameters
dstDevice
- Destination device pointer
uc
- Value to set
N
- Number of elements
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
Description
Sets the memory range of N 8-bit values to the specified value uc.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D16Async, cuMemsetD2D32, cuMemsetD2D32Async,
cuMemsetD8Async, cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cuMemsetD32Async,
cudaMemset
|
269
Modules
CUresult cuMemsetD8Async (CUdeviceptr dstDevice,
unsigned char uc, size_t N, CUstream hStream)
Sets device memory.
Parameters
dstDevice
- Destination device pointer
uc
- Value to set
N
- Number of elements
hStream
- Stream identifier
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE
Description
Sets the memory range of N 8-bit values to the specified value uc.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ See also memset synchronization details.
‣ This function uses standard default stream semantics.
See also:
cuArray3DCreate, cuArray3DGetDescriptor, cuArrayCreate, cuArrayDestroy, cuArrayGetDescriptor,
cuMemAlloc, cuMemAllocHost, cuMemAllocPitch, cuMemcpy2D, cuMemcpy2DAsync,
cuMemcpy2DUnaligned, cuMemcpy3D, cuMemcpy3DAsync, cuMemcpyAtoA, cuMemcpyAtoD,
cuMemcpyAtoH, cuMemcpyAtoHAsync, cuMemcpyDtoA, cuMemcpyDtoD, cuMemcpyDtoDAsync,
cuMemcpyDtoH, cuMemcpyDtoHAsync, cuMemcpyHtoA, cuMemcpyHtoAAsync, cuMemcpyHtoD,
cuMemcpyHtoDAsync, cuMemFree, cuMemFreeHost, cuMemGetAddressRange, cuMemGetInfo,
cuMemHostAlloc, cuMemHostGetDevicePointer, cuMemsetD2D8, cuMemsetD2D8Async,
cuMemsetD2D16, cuMemsetD2D16Async, cuMemsetD2D32, cuMemsetD2D32Async, cuMemsetD8,
cuMemsetD16, cuMemsetD16Async, cuMemsetD32, cuMemsetD32Async, cudaMemsetAsync
|
270
Modules
CUresult cuMipmappedArrayCreate (CUmipmappedArray
*pHandle, const CUDA_ARRAY3D_DESCRIPTOR
*pMipmappedArrayDesc, unsigned int numMipmapLevels)
Creates a CUDA mipmapped array.
Parameters
pHandle
- Returned mipmapped array
pMipmappedArrayDesc
- mipmapped array descriptor
numMipmapLevels
- Number of mipmap levels
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE,
CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_UNKNOWN
Description
Creates a CUDA mipmapped array according to the CUDA_ARRAY3D_DESCRIPTOR structure
pMipmappedArrayDesc and returns a handle to the new CUDA mipmapped array in *pHandle.
numMipmapLevels specifies the number of mipmap levels to be allocated. This value is clamped to
the range [1, 1 + floor(log2(max(width, height, depth)))].
The CUDA_ARRAY3D_DESCRIPTOR is defined as:
‎
typedef struct {
unsigned int Width;
unsigned int Height;
unsigned int Depth;
CUarray_format Format;
unsigned int NumChannels;
unsigned int Flags;
} CUDA_ARRAY3D_DESCRIPTOR;
where:
‣ Width, Height, and Depth are the width, height, and depth of the CUDA array (in elements);
the following types of CUDA arrays can be allocated:
‣ A 1D mipmapped array is allocated if Height and Depth extents are both zero.
‣ A 2D mipmapped array is allocated if only Depth extent is zero.
‣ A 3D mipmapped array is allocated if all three extents are non-zero.
‣ A 1D layered CUDA mipmapped array is allocated if only Height is zero and the
CUDA_ARRAY3D_LAYERED flag is set. Each layer is a 1D array. The number of layers is
determined by the depth extent.
|
271
Modules
‣ A 2D layered CUDA mipmapped array is allocated if all three extents are non-zero and the
CUDA_ARRAY3D_LAYERED flag is set. Each layer is a 2D array. The number of layers is
determined by the depth extent.
‣ A cubemap CUDA mipmapped array is allocated if all three extents are non-zero and the
CUDA_ARRAY3D_CUBEMAP flag is set. Width must be equal to Height, and Depth
must be six. A cubemap is a special type of 2D layered CUDA array, where the six layers
represent the six faces of a cube. The order of the six layers in memory is the same as that
listed in CUarray_cubemap_face.
‣ A cubemap layered CUDA mipmapped array is allocated if all three extents are non-zero,
and both, CUDA_ARRAY3D_CUBEMAP and CUDA_ARRAY3D_LAYERED flags are
set. Width must be equal to Height, and Depth must be a multiple of six. A cubemap
layered CUDA array is a special type of 2D layered CUDA array that consists of a collection
of cubemaps. The first six layers represent the first cubemap, the next six layers form the
second cubemap, and so on.
‣ Format specifies the format of the elements; CUarray_format is defined as:
‎
typedef enum CUarray_format_enum {
CU_AD_FORMAT_UNSIGNED_INT8 = 0x01,
CU_AD_FORMAT_UNSIGNED_INT16 = 0x02,
CU_AD_FORMAT_UNSIGNED_INT32 = 0x03,
CU_AD_FORMAT_SIGNED_INT8 = 0x08,
CU_AD_FORMAT_SIGNED_INT16 = 0x09,
CU_AD_FORMAT_SIGNED_INT32 = 0x0a,
CU_AD_FORMAT_HALF = 0x10,
CU_AD_FORMAT_FLOAT = 0x20,
CU_AD_FORMAT_NV12 = 0xb0,
CU_AD_FORMAT_UNORM_INT8X1 = 0xc0,
CU_AD_FORMAT_UNORM_INT8X2 = 0xc1,
CU_AD_FORMAT_UNORM_INT8X4 = 0xc2,
CU_AD_FORMAT_UNORM_INT16X1 = 0xc3,
CU_AD_FORMAT_UNORM_INT16X2 = 0xc4,
CU_AD_FORMAT_UNORM_INT16X4 = 0xc5,
CU_AD_FORMAT_SNORM_INT8X1 = 0xc6,
CU_AD_FORMAT_SNORM_INT8X2 = 0xc7,
CU_AD_FORMAT_SNORM_INT8X4 = 0xc8,
CU_AD_FORMAT_SNORM_INT16X1 = 0xc9,
CU_AD_FORMAT_SNORM_INT16X2 = 0xca,
CU_AD_FORMAT_SNORM_INT16X4 = 0xcb,
CU_AD_FORMAT_BC1_UNORM = 0x91,
CU_AD_FORMAT_BC1_UNORM_SRGB = 0x92,
CU_AD_FORMAT_BC2_UNORM = 0x93,
CU_AD_FORMAT_BC2_UNORM_SRGB = 0x94,
CU_AD_FORMAT_BC3_UNORM = 0x95,
CU_AD_FORMAT_BC3_UNORM_SRGB = 0x96,
CU_AD_FORMAT_BC4_UNORM = 0x97,
CU_AD_FORMAT_BC4_SNORM = 0x98,
CU_AD_FORMAT_BC5_UNORM = 0x99,
CU_AD_FORMAT_BC5_SNORM = 0x9a,
CU_AD_FORMAT_BC6H_UF16 = 0x9b,
CU_AD_FORMAT_BC6H_SF16 = 0x9c,
CU_AD_FORMAT_BC7_UNORM = 0x9d,
CU_AD_FORMAT_BC7_UNORM_SRGB = 0x9e,
CU_AD_FORMAT_P010 = 0x9f,
CU_AD_FORMAT_P016 = 0xa1,
CU_AD_FORMAT_NV16 = 0xa2,
CU_AD_FORMAT_P210 = 0xa3,
CU_AD_FORMAT_P216 = 0xa4,
CU_AD_FORMAT_YUY2 = 0xa5,
|
272
Modules
CU_AD_FORMAT_Y210 = 0xa6,
CU_AD_FORMAT_Y216 = 0xa7,
CU_AD_FORMAT_AYUV = 0xa8,
CU_AD_FORMAT_Y410 = 0xa9,
CU_AD_FORMAT_Y416 = 0xb1,
CU_AD_FORMAT_Y444_PLANAR8 = 0xb2,
CU_AD_FORMAT_Y444_PLANAR10 = 0xb3,
CU_AD_FORMAT_YUV444_8bit_SemiPlanar = 0xb4,
CU_AD_FORMAT_YUV444_16bit_SemiPlanar = 0xb5,
CU_AD_FORMAT_UNORM_INT_101010_2 = 0x50,
} CUarray_format;
‣ NumChannels specifies the number of packed components per CUDA array element; it may be
1, 2, or 4;
‣ Flags may be set to
‣ CUDA_ARRAY3D_LAYERED to enable creation of layered CUDA mipmapped arrays. If
this flag is set, Depth specifies the number of layers, not the depth of a 3D array.
‣ CUDA_ARRAY3D_SURFACE_LDST to enable surface references to be bound to individual
mipmap levels of the CUDA mipmapped array. If this flag is not set, cuSurfRefSetArray will
fail when attempting to bind a mipmap level of the CUDA mipmapped array to a surface
reference.
‣ CUDA_ARRAY3D_CUBEMAP to enable creation of mipmapped cubemaps. If
this flag is set, Width must be equal to Height, and Depth must be six. If the
CUDA_ARRAY3D_LAYERED flag is also set, then Depth must be a multiple of six.
‣ CUDA_ARRAY3D_TEXTURE_GATHER to indicate that the CUDA mipmapped array will
be used for texture gather. Texture gather can only be performed on 2D CUDA mipmapped
arrays.
Width, Height and Depth must meet certain size requirements as listed in the following table.
All values are specified in elements. Note that for brevity's sake, the full name of the device attribute
is not specified. For ex., TEXTURE1D_MIPMAPPED_WIDTH refers to the device attribute
CU_DEVICE_ATTRIBUTE_MAXIMUM_TEXTURE1D_MIPMAPPED_WIDTH.
CUDA array type
Valid extents that must always be
Valid extents with
met {(width range in elements),
CUDA_ARRAY3D_SURFACE_LD
ST
(height range), (depth range)}
set {(width range in elements),
(height range), (depth range)}
1D
{ (1,TEXTURE1D_MIPMAPPED_
{ (1,SURFACE1D_WIDTH), 0, 0 }
0, 0 }
2D
{ (1,TEXTURE2D_MIPMAPPED_W{ (1,SURFACE2D_WIDTH),
(1,TEXTURE2D_MIPMAPPED_HEI(1,SU,
RFACE2D_HEIGHT), 0 }
0 }
3D
{ (1,TEXTURE3D_WIDTH),
{ (1,SURFACE3D_WIDTH),
(1,TEXTURE3D_HEIGHT),
(1,SURFACE3D_HEIGHT),
(1,TEXTURE3D_DEPTH) } OR
(1,SURFACE3D_DEPTH) }
|
273
Modules
{ (1,TEXTURE3D_WIDTH_ALTERNATE),
(1,TEXTURE3D_HEIGHT_ALTERNATE),
(1,TEXTURE3D_DEPTH_ALTERNATE) }
1D Layered
{ (1,TEXTURE1D_LAYERED_WID{ (1,SURFACE1D_LAYERED_WIDTH),
0,
0,
(1,TEXTURE1D_LAYERED_LAYE
1,SURFACE1D_LAYERED_LAYE
RS) }
2D Layered
{ (1,TEXTURE2D_LAYERED_WID{ (1,SURFACE2D_LAYERED_WIDTH),
(1,TEXTURE2D_LAYERED_HEIG(1,SURFACE2D_LAYERED_HEIG
HT),
(1,TEXTURE2D_LAYERED_LAYE
1,SURFACE2D_LAYERED_LAYE
RS) }
Cubemap
{ (1,TEXTURECUBEMAP_WIDTH),
{ (1,SURFACECUBEMAP_WIDTH
),
(1,TEXTURECUBEMAP_WIDTH),
(1,SURFACECUBEMAP_WIDTH),
6 }
6 }
Cubemap Layered
{ (1,TEXTURECUBEMAP_LAYER{ (1,SURFACECUBEMAP_LAYER
ED_WIDTH),
(1,TEXTURECUBEMAP_LAYERE(1,SURFACECUBEMAP_LAYERE
D_WIDTH),
(1,TEXTURECUBEMAP_LAYERE
(1,SURFACECUBEMAP_LAYERE
D_LAYERS) }
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuMipmappedArrayDestroy, cuMipmappedArrayGetLevel, cuArrayCreate,
cudaMallocMipmappedArray
CUresult cuMipmappedArrayDestroy
(CUmipmappedArray hMipmappedArray)
Destroys a CUDA mipmapped array.
Parameters
hMipmappedArray
- Mipmapped array to destroy
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_HANDLE,
CUDA_ERROR_ARRAY_IS_MAPPED, CUDA_ERROR_CONTEXT_IS_DESTROYED
|
274
Modules
Description
Destroys the CUDA mipmapped array hMipmappedArray.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuMipmappedArrayCreate, cuMipmappedArrayGetLevel, cuArrayCreate, cudaFreeMipmappedArray
CUresult cuMipmappedArrayGetLevel (CUarray
*pLevelArray, CUmipmappedArray hMipmappedArray,
unsigned int level)
Gets a mipmap level of a CUDA mipmapped array.
Parameters
pLevelArray
- Returned mipmap level CUDA array
hMipmappedArray
- CUDA mipmapped array
level
- Mipmap level
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE,
CUDA_ERROR_INVALID_HANDLE
Description
Returns in *pLevelArray a CUDA array that represents a single mipmap level of the CUDA
mipmapped array hMipmappedArray.
If level is greater than the maximum number of levels in this mipmapped array,
CUDA_ERROR_INVALID_VALUE is returned.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
|
275
Modules
See also:
cuMipmappedArrayCreate, cuMipmappedArrayDestroy, cuArrayCreate,
cudaGetMipmappedArrayLevel
CUresult cuMipmappedArrayGetMemoryRequirements
(CUDA_ARRAY_MEMORY_REQUIREMENTS
*memoryRequirements, CUmipmappedArray mipmap,
CUdevice device)
Returns the memory requirements of a CUDA mipmapped array.
Parameters
memoryRequirements
- Pointer to CUDA_ARRAY_MEMORY_REQUIREMENTS
mipmap
- CUDA mipmapped array to get the memory requirements of
device
- Device to get the memory requirements for
Returns
CUDA_SUCCESS CUDA_ERROR_INVALID_VALUE
Description
Returns the memory requirements of a CUDA mipmapped array in memoryRequirements If the
CUDA mipmapped array is not allocated with flag CUDA_ARRAY3D_DEFERRED_MAPPING
CUDA_ERROR_INVALID_VALUE will be returned.
The returned value in CUDA_ARRAY_MEMORY_REQUIREMENTS::size
represents the total size of the CUDA mipmapped array. The returned value in
CUDA_ARRAY_MEMORY_REQUIREMENTS::alignment represents the alignment necessary for
mapping the CUDA mipmapped array.
See also:
cuArrayGetMemoryRequirements, cuMemMapArrayAsync
|
276
Modules
CUresult cuMipmappedArrayGetSparseProperties
(CUDA_ARRAY_SPARSE_PROPERTIES
*sparseProperties, CUmipmappedArray mipmap)
Returns the layout properties of a sparse CUDA mipmapped array.
Parameters
sparseProperties
- Pointer to CUDA_ARRAY_SPARSE_PROPERTIES
mipmap
- CUDA mipmapped array to get the sparse properties of
Returns
CUDA_SUCCESS CUDA_ERROR_INVALID_VALUE
Description
Returns the sparse array layout properties in sparseProperties If the CUDA mipmapped array
is not allocated with flag CUDA_ARRAY3D_SPARSE CUDA_ERROR_INVALID_VALUE will be
returned.
For non-layered CUDA mipmapped arrays, CUDA_ARRAY_SPARSE_PROPERTIES::miptailSize
returns the size of the mip tail region. The mip tail region includes all mip
levels whose width, height or depth is less than that of the tile. For layered
CUDA mipmapped arrays, if CUDA_ARRAY_SPARSE_PROPERTIES::flags
contains CU_ARRAY_SPARSE_PROPERTIES_SINGLE_MIPTAIL, then
CUDA_ARRAY_SPARSE_PROPERTIES::miptailSize specifies the size of the mip tail of all layers
combined. Otherwise, CUDA_ARRAY_SPARSE_PROPERTIES::miptailSize specifies mip tail size
per layer. The returned value of CUDA_ARRAY_SPARSE_PROPERTIES::miptailFirstLevel is valid
only if CUDA_ARRAY_SPARSE_PROPERTIES::miptailSize is non-zero.
See also:
cuArrayGetSparseProperties, cuMemMapArrayAsync
6.14. Virtual Memory Management
This section describes the virtual memory management functions of the low-level CUDA driver
application programming interface.
|
277
Modules
CUresult cuMemAddressFree (CUdeviceptr ptr, size_t size)
Free an address range reservation.
Parameters
ptr
- Starting address of the virtual address range to free
size
- Size of the virtual address region to free
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
Frees a virtual address range reserved by cuMemAddressReserve. The size must match what was given
to memAddressReserve and the ptr given must match what was returned from memAddressReserve.
See also:
cuMemAddressReserve
CUresult cuMemAddressReserve (CUdeviceptr *ptr, size_t
size, size_t alignment, CUdeviceptr addr, unsigned long
long flags)
Allocate an address range reservation.
Parameters
ptr
- Resulting pointer to start of virtual address range allocated
size
- Size of the reserved virtual address range requested
alignment
- Alignment of the reserved virtual address range requested
addr
- Hint address for the start of the address range
flags
- Currently unused, must be zero
|
278
Modules
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY,
CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED,
CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED
Description
Reserves a virtual address range based on the given parameters, giving the starting address of the range
in ptr. This API requires a system that supports UVA. The size and address parameters must be a
multiple of the host page size and the alignment must be a power of two or zero for default alignment.
If addr is 0, then the driver chooses the address at which to place the start of the reservation whereas
when it is non-zero then the driver treats it as a hint about where to place the reservation.
See also:
cuMemAddressFree
CUresult cuMemCreate (CUmemGenericAllocationHandle
*handle, size_t size, const CUmemAllocationProp *prop,
unsigned long long flags)
Create a CUDA memory handle representing a memory allocation of a given size described by the
given properties.
Parameters
handle
- Value of handle returned. All operations on this allocation are to be performed using this handle.
size
- Size of the allocation requested
prop
- Properties of the allocation to create.
flags
- flags for future use, must be zero now.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY,
CUDA_ERROR_INVALID_DEVICE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
|
279
Modules
Description
This creates a memory allocation on the target device specified through the prop
structure. The created allocation will not have any device or host mappings. The generic
memory handle for the allocation can be mapped to the address space of calling
process via cuMemMap. This handle cannot be transmitted directly to other processes
(see cuMemExportToShareableHandle). On Windows, the caller must also pass an
LPSECURITYATTRIBUTE in prop to be associated with this handle which limits or allows
access to this handle for a recipient process (see CUmemAllocationProp::win32HandleMetaData
for more). The size of this allocation must be a multiple of the the value given via
cuMemGetAllocationGranularity with the CU_MEM_ALLOC_GRANULARITY_MINIMUM
flag. To create a CPU allocation targeting a specific host NUMA node, applications must set
CUmemAllocationProp::CUmemLocation::type to CU_MEM_LOCATION_TYPE_HOST_NUMA
and CUmemAllocationProp::CUmemLocation::id must specify the NUMA ID of the CPU.
On systems where NUMA is not available CUmemAllocationProp::CUmemLocation::id
must be set to 0. Specifying CU_MEM_LOCATION_TYPE_HOST_NUMA_CURRENT
or CU_MEM_LOCATION_TYPE_HOST as the CUmemLocation::type will result in
CUDA_ERROR_INVALID_VALUE.
Applications that intend to use CU_MEM_HANDLE_TYPE_FABRIC based memory sharing must
ensure: (1) `nvidia-caps-imex-channels` character device is created by the driver and is listed under /
proc/devices (2) have at least one IMEX channel file accessible by the user launching the application.
When exporter and importer CUDA processes have been granted access to the same IMEX channel,
they can securely share memory.
The IMEX channel security model works on a per user basis. Which means all processes under a user
can share memory if the user has access to a valid IMEX channel. When multi-user isolation is desired,
a separate IMEX channel is required for each user.
These channel files exist in /dev/nvidia-caps-imex-channels/channel* and can be created using standard
OS native calls like mknod on Linux. For example: To create channel0 with the major number from /
proc/devices users can execute the following command: `mknod /dev/nvidia-caps-imex-channels/
channel0 c <major number>=""> 0`
If CUmemAllocationProp::allocFlags::usage contains CU_MEM_CREATE_USAGE_TILE_POOL
flag then the memory allocation is intended only to be used as backing tile pool for sparse CUDA
arrays and sparse CUDA mipmapped arrays. (see cuMemMapArrayAsync).
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuMemRelease, cuMemExportToShareableHandle, cuMemImportFromShareableHandle
|
280
Modules
CUresult cuMemExportToShareableHandle (void
*shareableHandle, CUmemGenericAllocationHandle
handle, CUmemAllocationHandleType handleType,
unsigned long long flags)
Exports an allocation to a requested shareable handle type.
Parameters
shareableHandle
- Pointer to the location in which to store the requested handle type
handle
- CUDA handle for the memory allocation
handleType
- Type of shareable handle requested (defines type and size of the shareableHandle output
parameter)
flags
- Reserved, must be zero
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
Given a CUDA memory handle, create a shareable memory allocation handle that can be used to share
the memory with other processes. The recipient process can convert the shareable handle back into a
CUDA memory handle using cuMemImportFromShareableHandle and map it with cuMemMap. The
implementation of what this handle is and how it can be transferred is defined by the requested handle
type in handleType
Once all shareable handles are closed and the allocation is released, the allocated memory referenced
will be released back to the OS and uses of the CUDA handle afterward will lead to undefined
behavior.
This API can also be used in conjunction with other APIs (e.g. Vulkan, OpenGL) that support
importing memory from the shareable type
See also:
cuMemImportFromShareableHandle
|
281
Modules
CUresult cuMemGetAccess (unsigned long long *flags,
const CUmemLocation *location, CUdeviceptr ptr)
Get the access flags set for the given location and ptr.
Parameters
flags
- Flags set for this location
location
- Location in which to check the flags for
ptr
- Address in which to check the access flags for
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE,
CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED,
CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED
Description
See also:
cuMemSetAccess
CUresult cuMemGetAllocationGranularity (size_t
*granularity, const CUmemAllocationProp *prop,
CUmemAllocationGranularity_flags option)
Calculates either the minimal or recommended granularity.
Parameters
granularity
Returned granularity.
prop
Property for which to determine the granularity for
option
Determines which granularity to return
|
282
Modules
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
Calculates either the minimal or recommended granularity for a given allocation specification and
returns it in granularity. This granularity can be used as a multiple for alignment, size, or address
mapping.
See also:
cuMemCreate, cuMemMap
CUresult cuMemGetAllocationPropertiesFromHandle
(CUmemAllocationProp *prop,
CUmemGenericAllocationHandle handle)
Retrieve the contents of the property structure defining properties for this handle.
Parameters
prop
- Pointer to a properties structure which will hold the information about this handle
handle
- Handle which to perform the query on
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
See also:
cuMemCreate, cuMemImportFromShareableHandle
|
283
Modules
CUresult cuMemImportFromShareableHandle
(CUmemGenericAllocationHandle *handle, void
*osHandle, CUmemAllocationHandleType shHandleType)
Imports an allocation from a requested shareable handle type.
Parameters
handle
- CUDA Memory handle for the memory allocation.
osHandle
- Shareable Handle representing the memory allocation that is to be imported.
shHandleType
- handle type of the exported handle CUmemAllocationHandleType.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
If the current process cannot support the memory described by this shareable handle, this API will error
as CUDA_ERROR_NOT_SUPPORTED.
If shHandleType is CU_MEM_HANDLE_TYPE_FABRIC and the importer process has not
been granted access to the same IMEX channel as the exporter process, this API will error as
CUDA_ERROR_NOT_PERMITTED.
Note:
Importing shareable handles exported from some graphics APIs(VUlkan, OpenGL, etc)
created on devices under an SLI group may not be supported, and thus this API will return
CUDA_ERROR_NOT_SUPPORTED. There is no guarantee that the contents of handle will be the
same CUDA memory handle for the same given OS shareable handle, or the same underlying allocation.
See also:
cuMemExportToShareableHandle, cuMemMap, cuMemRelease
|
284
Modules
CUresult cuMemMap (CUdeviceptr ptr, size_t size, size_t
offset, CUmemGenericAllocationHandle handle, unsigned
long long flags)
Maps an allocation handle to a reserved virtual address range.
Parameters
ptr
- Address where memory will be mapped.
size
- Size of the memory mapping.
offset
handle from which to start mapping Note: currently must be zero.
- Offset into the memory represented by
handle
- Handle to a shareable memory
flags
- flags for future use, must be zero now.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE,
CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_ILLEGAL_STATE
‣ handle from which to start mapping
‣ Note: currently must be zero.
Description
Maps bytes of memory represented by handle starting from byte offset to size to
address range [addr, addr + size]. This range must be an address reservation previously
reserved with cuMemAddressReserve, and offset + size must be less than the size of the
memory allocation. Both ptr, size, and offset must be a multiple of the value given via
cuMemGetAllocationGranularity with the CU_MEM_ALLOC_GRANULARITY_MINIMUM flag. If
handle represents a multicast object, ptr, size and offset must be aligned to the value returned
by cuMulticastGetGranularity with the flag CU_MULTICAST_MINIMUM_GRANULARITY.
For best performance however, it is recommended that ptr, size and offset
be aligned to the value returned by cuMulticastGetGranularity with the flag
CU_MULTICAST_RECOMMENDED_GRANULARITY.
|
285
Modules
When handle represents a multicast object, this call may return CUDA_ERROR_ILLEGAL_STATE
if the system configuration is in an illegal state. In such cases, to continue using multicast, verify that
the system configuration is in a valid state and all required driver daemons are running properly.
Please note calling cuMemMap does not make the address accessible, the caller needs to update
accessibility of a contiguous mapped VA range by calling cuMemSetAccess.
Once a recipient process obtains a shareable memory handle from
cuMemImportFromShareableHandle, the process must use cuMemMap to map the memory into its
address ranges before setting accessibility with cuMemSetAccess.
cuMemMap can only create mappings on VA range reservations that are not currently mapped.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuMemUnmap, cuMemSetAccess, cuMemCreate, cuMemAddressReserve,
cuMemImportFromShareableHandle
CUresult cuMemMapArrayAsync (CUarrayMapInfo
*mapInfoList, unsigned int count, CUstream hStream)
Maps or unmaps subregions of sparse CUDA arrays and sparse CUDA mipmapped arrays.
Parameters
mapInfoList
- List of CUarrayMapInfo
count
- Count of CUarrayMapInfo in mapInfoList
hStream
- Stream identifier for the stream to use for map or unmap operations
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_HANDLE
Description
Performs map or unmap operations on subregions of sparse CUDA arrays and sparse CUDA
mipmapped arrays. Each operation is specified by a CUarrayMapInfo entry in the mapInfoList
array of size count. The structure CUarrayMapInfo is defined as follow:
‎
typedef struct CUarrayMapInfo_st {
CUresourcetype resourceType;
union {
|
286
Modules
CUmipmappedArray mipmap;
CUarray array;
} resource;
CUarraySparseSubresourceType subresourceType;
union {
struct {
unsigned int level;
unsigned int layer;
unsigned int offsetX;
unsigned int offsetY;
unsigned int offsetZ;
unsigned int extentWidth;
unsigned int extentHeight;
unsigned int extentDepth;
} sparseLevel;
struct {
unsigned int layer;
unsigned long long offset;
unsigned long long size;
} miptail;
} subresource;
CUmemOperationType memOperationType;
CUmemHandleType memHandleType;
union {
CUmemGenericAllocationHandle memHandle;
} memHandle;
unsigned long long offset;
unsigned int deviceBitMask;
unsigned int flags;
unsigned int reserved[2];
} CUarrayMapInfo;
where CUarrayMapInfo::resourceType specifies the type of resource to be operated on. If
CUarrayMapInfo::resourceType is set to CUresourcetype::CU_RESOURCE_TYPE_ARRAY
then CUarrayMapInfo::resource::array must be set to a valid sparse CUDA array
handle. The CUDA array must be either a 2D, 2D layered or 3D CUDA array and
must have been allocated using cuArrayCreate or cuArray3DCreate with the flag
CUDA_ARRAY3D_SPARSE or CUDA_ARRAY3D_DEFERRED_MAPPING.
For CUDA arrays obtained using cuMipmappedArrayGetLevel,
CUDA_ERROR_INVALID_VALUE will be returned. If CUarrayMapInfo::resourceType
is set to CUresourcetype::CU_RESOURCE_TYPE_MIPMAPPED_ARRAY then
CUarrayMapInfo::resource::mipmap must be set to a valid sparse CUDA mipmapped array handle. The
CUDA mipmapped array must be either a 2D, 2D layered or 3D CUDA mipmapped array and must
have been allocated using cuMipmappedArrayCreate with the flag CUDA_ARRAY3D_SPARSE or
CUDA_ARRAY3D_DEFERRED_MAPPING.
CUarrayMapInfo::subresourceType specifies the type of subresource within the resource.
CUarraySparseSubresourceType_enum is defined as:
‎
typedef enum CUarraySparseSubresourceType_enum {
CU_ARRAY_SPARSE_SUBRESOURCE_TYPE_SPARSE_LEVEL = 0,
CU_ARRAY_SPARSE_SUBRESOURCE_TYPE_MIPTAIL = 1
} CUarraySparseSubresourceType;
where
CUarraySparseSubresourceType::CU_ARRAY_SPARSE_SUBRESOURCE_TYPE_SPARSE_LEVEL
|
287
Modules
indicates a sparse-miplevel which spans at least one tile in every dimension. The remaining miplevels
which are too small to span at least one tile in any dimension constitute the mip tail region as indicated
by CUarraySparseSubresourceType::CU_ARRAY_SPARSE_SUBRESOURCE_TYPE_MIPTAIL
subresource type.
If CUarrayMapInfo::subresourceType is set to
CUarraySparseSubresourceType::CU_ARRAY_SPARSE_SUBRESOURCE_TYPE_SPARSE_LEVEL
then CUarrayMapInfo::subresource::sparseLevel struct must contain valid array
subregion offsets and extents. The CUarrayMapInfo::subresource::sparseLevel::offsetX,
CUarrayMapInfo::subresource::sparseLevel::offsetY and
CUarrayMapInfo::subresource::sparseLevel::offsetZ must specify valid X, Y and Z
offsets respectively. The CUarrayMapInfo::subresource::sparseLevel::extentWidth,
CUarrayMapInfo::subresource::sparseLevel::extentHeight and
CUarrayMapInfo::subresource::sparseLevel::extentDepth must specify valid width, height and
depth extents respectively. These offsets and extents must be aligned to the corresponding tile
dimension. For CUDA mipmapped arrays CUarrayMapInfo::subresource::sparseLevel::level must
specify a valid mip level index. Otherwise, must be zero. For layered CUDA arrays and layered
CUDA mipmapped arrays CUarrayMapInfo::subresource::sparseLevel::layer must specify a valid
layer index. Otherwise, must be zero. CUarrayMapInfo::subresource::sparseLevel::offsetZ must
be zero and CUarrayMapInfo::subresource::sparseLevel::extentDepth must be set to 1 for 2D and
2D layered CUDA arrays and CUDA mipmapped arrays. Tile extents can be obtained by calling
cuArrayGetSparseProperties and cuMipmappedArrayGetSparseProperties
If CUarrayMapInfo::subresourceType is set to
CUarraySparseSubresourceType::CU_ARRAY_SPARSE_SUBRESOURCE_TYPE_MIPTAIL
then CUarrayMapInfo::subresource::miptail struct must contain valid mip tail offset in
CUarrayMapInfo::subresource::miptail::offset and size in CUarrayMapInfo::subresource::miptail::size.
Both, mip tail offset and mip tail size must be aligned to the tile size. For layered CUDA mipmapped
arrays which don't have the flag CU_ARRAY_SPARSE_PROPERTIES_SINGLE_MIPTAIL
set in CUDA_ARRAY_SPARSE_PROPERTIES::flags as returned by
cuMipmappedArrayGetSparseProperties, CUarrayMapInfo::subresource::miptail::layer must specify a
valid layer index. Otherwise, must be zero.
If CUarrayMapInfo::resource::array or CUarrayMapInfo::resource::mipmap was created with
CUDA_ARRAY3D_DEFERRED_MAPPING flag set the CUarrayMapInfo::subresourceType and the
contents of CUarrayMapInfo::subresource will be ignored.
CUarrayMapInfo::memOperationType specifies the type of operation. CUmemOperationType is
defined as:
‎
typedef enum CUmemOperationType_enum {
CU_MEM_OPERATION_TYPE_MAP = 1,
CU_MEM_OPERATION_TYPE_UNMAP = 2
} CUmemOperationType;
If CUarrayMapInfo::memOperationType is set to
CUmemOperationType::CU_MEM_OPERATION_TYPE_MAP then the subresource
will be mapped onto the tile pool memory specified by CUarrayMapInfo::memHandle
at offset CUarrayMapInfo::offset. The tile pool allocation has to be created
|
288
Modules
by specifying the CU_MEM_CREATE_USAGE_TILE_POOL flag when
calling cuMemCreate. Also, CUarrayMapInfo::memHandleType must be set to
CUmemHandleType::CU_MEM_HANDLE_TYPE_GENERIC.
If CUarrayMapInfo::memOperationType is set to
CUmemOperationType::CU_MEM_OPERATION_TYPE_UNMAP then an unmapping operation is
performed. CUarrayMapInfo::memHandle must be NULL.
CUarrayMapInfo::deviceBitMask specifies the list of devices that must map or unmap physical
memory. Currently, this mask must have exactly one bit set, and the corresponding device must
match the device associated with the stream. If CUarrayMapInfo::memOperationType is set to
CUmemOperationType::CU_MEM_OPERATION_TYPE_MAP, the device must also match the
device associated with the tile pool memory allocation as specified by CUarrayMapInfo::memHandle.
CUarrayMapInfo::flags and CUarrayMapInfo::reserved[] are unused and must be set to zero.
See also:
cuMipmappedArrayCreate, cuArrayCreate, cuArray3DCreate, cuMemCreate,
cuArrayGetSparseProperties, cuMipmappedArrayGetSparseProperties
CUresult cuMemRelease
(CUmemGenericAllocationHandle handle)
Release a memory handle representing a memory allocation which was previously allocated through
cuMemCreate.
Parameters
handle
Value of handle which was returned previously by cuMemCreate.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
Frees the memory that was allocated on a device through cuMemCreate.
The memory allocation will be freed when all outstanding mappings to the memory are unmapped and
when all outstanding references to the handle (including it's shareable counterparts) are also released.
The generic memory handle can be freed when there are still outstanding mappings made with this
handle. Each time a recipient process imports a shareable handle, it needs to pair it with cuMemRelease
for the handle to be freed. If handle is not a valid handle the behavior is undefined.
|
289
Modules
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuMemCreate
CUresult cuMemRetainAllocationHandle
(CUmemGenericAllocationHandle *handle, void *addr)
Given an address addr, returns the allocation handle of the backing memory allocation.
Parameters
handle
CUDA Memory handle for the backing memory allocation.
addr
Memory address to query, that has been mapped previously.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
The handle is guaranteed to be the same handle value used to map the memory. If the address requested
is not mapped, the function will fail. The returned handle must be released with corresponding number
of calls to cuMemRelease.
Note:
The address addr, can be any address in a range previously mapped by cuMemMap, and not
necessarily the start address.
See also:
cuMemCreate, cuMemRelease, cuMemMap
|
290
Modules
CUresult cuMemSetAccess (CUdeviceptr ptr, size_t size,
const CUmemAccessDesc *desc, size_t count)
Set the access flags for each location specified in desc for the given virtual address range.
Parameters
ptr
- Starting address for the virtual address range
size
- Length of the virtual address range
desc
mapping for each location specified
- Array of CUmemAccessDesc that describe how to change the
count
- Number of CUmemAccessDesc in desc
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE,
CUDA_ERROR_NOT_SUPPORTED
‣ mapping for each location specified
Description
Given the virtual address range via ptr and size, and the locations in the array given by desc
and count, set the access flags for the target locations. The range must be a fully mapped
address range containing all allocations created by cuMemMap / cuMemCreate. Users cannot
specify CU_MEM_LOCATION_TYPE_HOST_NUMA accessibility for allocations created
on with other location types. Note: When CUmemAccessDesc::CUmemLocation::type is
CU_MEM_LOCATION_TYPE_HOST_NUMA, CUmemAccessDesc::CUmemLocation::id is
ignored. When setting the access flags for a virtual address range mapping a multicast object,
ptr and size must be aligned to the value returned by cuMulticastGetGranularity with the flag
CU_MULTICAST_MINIMUM_GRANULARITY. For best performance however, it is recommended
that ptr and size be aligned to the value returned by cuMulticastGetGranularity with the flag
CU_MULTICAST_RECOMMENDED_GRANULARITY.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ This function exhibits synchronous behavior for most use cases.
See also:
|
291
Modules
cuMemSetAccess, cuMemCreate, :cuMemMap
CUresult cuMemUnmap (CUdeviceptr ptr, size_t size)
Unmap the backing memory of a given address range.
Parameters
ptr
- Starting address for the virtual address range to unmap
size
- Size of the virtual address range to unmap
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
The range must be the entire contiguous address range that was mapped to. In other words,
cuMemUnmap cannot unmap a sub-range of an address range mapped by cuMemCreate / cuMemMap.
Any backing memory allocations will be freed if there are no existing mappings and there are no
unreleased memory handles.
When cuMemUnmap returns successfully the address range is converted to an address reservation and
can be used for a future calls to cuMemMap. Any new mapping to this virtual address will need to have
access granted through cuMemSetAccess, as all mappings start with no accessibility setup.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ This function exhibits synchronous behavior for most use cases.
See also:
cuMemCreate, cuMemAddressReserve
6.15. Stream Ordered Memory Allocator
This section describes the stream ordered memory allocator exposed by the low-level CUDA driver
application programming interface.
overview
|
292
Modules
The asynchronous allocator allows the user to allocate and free in stream order. All asynchronous
accesses of the allocation must happen between the stream executions of the allocation and the free.
If the memory is accessed outside of the promised stream order, a use before allocation / use after free
error will cause undefined behavior.
The allocator is free to reallocate the memory as long as it can guarantee that compliant memory
accesses will not overlap temporally. The allocator may refer to internal stream ordering as well as
inter-stream dependencies (such as CUDA events and null stream dependencies) when establishing the
temporal guarantee. The allocator may also insert inter-stream dependencies to establish the temporal
guarantee.
Supported Platforms
Whether or not a device supports the integrated stream ordered memory allocator
may be queried by calling cuDeviceGetAttribute() with the device attribute
CU_DEVICE_ATTRIBUTE_MEMORY_POOLS_SUPPORTED
CUresult cuMemAllocAsync (CUdeviceptr *dptr, size_t
bytesize, CUstream hStream)
Allocates memory with stream ordered semantics.
Parameters
dptr
- Returned device pointer
bytesize
- Number of bytes to allocate
hStream
- The stream establishing the stream ordering contract and the memory pool to allocate from
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT (default stream specified with no current context),
CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_OUT_OF_MEMORY
Description
Inserts an allocation operation into hStream. A pointer to the allocated memory is returned
immediately in *dptr. The allocation must not be accessed until the the allocation operation completes.
The allocation comes from the memory pool current to the stream's device.
Note:
‣ The default memory pool of a device contains device memory from that device.
|
293
Modules
‣ Basic stream ordering allows future work submitted into the same stream to use the allocation.
Stream query, stream synchronize, and CUDA events can be used to guarantee that the allocation
operation completes before work submitted in a separate stream runs.
‣ During stream capture, this function results in the creation of an allocation node. In this case, the
allocation is owned by the graph instead of the memory pool. The memory pool's properties are used
to set the node's creation parameters.
See also:
cuMemAllocFromPoolAsync, cuMemFreeAsync, cuDeviceSetMemPool,
cuDeviceGetDefaultMemPool, cuDeviceGetMemPool, cuMemPoolCreate, cuMemPoolSetAccess,
cuMemPoolSetAttribute
CUresult cuMemAllocFromPoolAsync (CUdeviceptr *dptr,
size_t bytesize, CUmemoryPool pool, CUstream hStream)
Allocates memory from a specified pool with stream ordered semantics.
Parameters
dptr
- Returned device pointer
bytesize
- Number of bytes to allocate
pool
- The pool to allocate from
hStream
- The stream establishing the stream ordering semantic
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT (default stream specified with no current context),
CUDA_ERROR_NOT_SUPPORTED, CUDA_ERROR_OUT_OF_MEMORY
Description
Inserts an allocation operation into hStream. A pointer to the allocated memory is returned
immediately in *dptr. The allocation must not be accessed until the the allocation operation completes.
The allocation comes from the specified memory pool.
Note:
‣ The specified memory pool may be from a device different than that of the specified hStream.
|
294
Modules
‣ Basic stream ordering allows future work submitted into the same stream to use the allocation.
Stream query, stream synchronize, and CUDA events can be used to guarantee that the allocation
operation completes before work submitted in a separate stream runs.
Note:
During stream capture, this function results in the creation of an allocation node. In this case, the
allocation is owned by the graph instead of the memory pool. The memory pool's properties are used to
set the node's creation parameters.
See also:
cuMemAllocAsync, cuMemFreeAsync, cuDeviceGetDefaultMemPool, cuDeviceGetMemPool,
cuMemPoolCreate, cuMemPoolSetAccess, cuMemPoolSetAttribute
CUresult cuMemFreeAsync (CUdeviceptr dptr, CUstream
hStream)
Frees memory with stream ordered semantics.
Parameters
dptr
- memory to free
hStream
- The stream establishing the stream ordering contract.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT (default stream specified with no current context),
CUDA_ERROR_NOT_SUPPORTED
Description
Inserts a free operation into hStream. The allocation must not be accessed after stream execution
reaches the free. After this API returns, accessing the memory from any subsequent work launched on
the GPU or querying its pointer attributes results in undefined behavior.
Note:
During stream capture, this function results in the creation of a free node and must therefore be passed
the address of a graph allocation.
|
295
Modules
CUresult cuMemPoolCreate (CUmemoryPool *pool, const
CUmemPoolProps *poolProps)
Creates a memory pool.
Returns
CUDA_SUCCESS, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE,
CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
Creates a CUDA memory pool and returns the handle in pool. The poolProps determines the
properties of the pool such as the backing device and IPC capabilities.
To create a memory pool targeting a specific host NUMA node, applications must set
CUmemPoolProps::CUmemLocation::type to CU_MEM_LOCATION_TYPE_HOST_NUMA
and CUmemPoolProps::CUmemLocation::id must specify the NUMA ID of the host
memory node. Specifying CU_MEM_LOCATION_TYPE_HOST_NUMA_CURRENT or
CU_MEM_LOCATION_TYPE_HOST as the CUmemPoolProps::CUmemLocation::type
will result in CUDA_ERROR_INVALID_VALUE. By default, the pool's memory
will be accessible from the device it is allocated on. In the case of pools created with
CU_MEM_LOCATION_TYPE_HOST_NUMA, their default accessibility will be from the host
CPU. Applications can control the maximum size of the pool by specifying a non-zero value for
CUmemPoolProps::maxSize. If set to 0, the maximum size of the pool will default to a system
dependent value.
Applications that intend to use CU_MEM_HANDLE_TYPE_FABRIC based memory sharing must
ensure: (1) `nvidia-caps-imex-channels` character device is created by the driver and is listed under /
proc/devices (2) have at least one IMEX channel file accessible by the user launching the application.
When exporter and importer CUDA processes have been granted access to the same IMEX channel,
they can securely share memory.
The IMEX channel security model works on a per user basis. Which means all processes under a user
can share memory if the user has access to a valid IMEX channel. When multi-user isolation is desired,
a separate IMEX channel is required for each user.
These channel files exist in /dev/nvidia-caps-imex-channels/channel* and can be created using standard
OS native calls like mknod on Linux. For example: To create channel0 with the major number from /
proc/devices users can execute the following command: `mknod /dev/nvidia-caps-imex-channels/
channel0 c <major number>=""> 0`
Note:
Specifying CU_MEM_HANDLE_TYPE_NONE creates a memory pool that will not support IPC.
|
296
Modules
See also:
cuDeviceSetMemPool, cuDeviceGetMemPool, cuDeviceGetDefaultMemPool,
cuMemAllocFromPoolAsync, cuMemPoolExportToShareableHandle
CUresult cuMemPoolDestroy (CUmemoryPool pool)
Destroys the specified memory pool.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE
Description
If any pointers obtained from this pool haven't been freed or the pool has free operations that haven't
completed when cuMemPoolDestroy is invoked, the function will return immediately and the resources
associated with the pool will be released automatically once there are no more outstanding allocations.
Destroying the current mempool of a device sets the default mempool of that device as the current
mempool for that device.
Note:
A device's default memory pool cannot be destroyed.
See also:
cuMemFreeAsync, cuDeviceSetMemPool, cuDeviceGetMemPool, cuDeviceGetDefaultMemPool,
cuMemPoolCreate
CUresult cuMemPoolExportPointer
(CUmemPoolPtrExportData *shareData_out, CUdeviceptr
ptr)
Export data to share a memory pool allocation between processes.
Parameters
shareData_out
- Returned export data
ptr
- pointer to memory being exported
|
297
Modules
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_OUT_OF_MEMORY
Description
Constructs shareData_out for sharing a specific allocation from an already shared memory pool.
The recipient process can import the allocation with the cuMemPoolImportPointer api. The data is not
a handle and may be shared through any IPC mechanism.
See also:
cuMemPoolExportToShareableHandle, cuMemPoolImportFromShareableHandle,
cuMemPoolImportPointer
CUresult cuMemPoolExportToShareableHandle
(void *handle_out, CUmemoryPool pool,
CUmemAllocationHandleType handleType, unsigned long
long flags)
Exports a memory pool to the requested handle type.
Parameters
handle_out
- Returned OS handle
pool
- pool to export
handleType
- the type of handle to create
flags
- must be 0
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_OUT_OF_MEMORY
Description
Given an IPC capable mempool, create an OS handle to share the pool with another
process. A recipient process can convert the shareable handle into a mempool with
cuMemPoolImportFromShareableHandle. Individual pointers can then be shared with the
|
298
Modules
cuMemPoolExportPointer and cuMemPoolImportPointer APIs. The implementation of what the
shareable handle is and how it can be transferred is defined by the requested handle type.
Note:
: To create an IPC capable mempool, create a mempool with a CUmemAllocationHandleType other than
CU_MEM_HANDLE_TYPE_NONE.
See also:
cuMemPoolImportFromShareableHandle, cuMemPoolExportPointer, cuMemPoolImportPointer,
cuMemAllocAsync, cuMemFreeAsync, cuDeviceGetDefaultMemPool, cuDeviceGetMemPool,
cuMemPoolCreate, cuMemPoolSetAccess, cuMemPoolSetAttribute
CUresult cuMemPoolGetAccess (CUmemAccess_flags
*flags, CUmemoryPool memPool, CUmemLocation
*location)
Returns the accessibility of a pool from a device.
Parameters
flags
- the accessibility of the pool from the specified location
memPool
- the pool being queried
location
- the location accessing the pool
Description
Returns the accessibility of the pool's memory from the specified location.
See also:
cuMemAllocAsync, cuMemFreeAsync, cuDeviceGetDefaultMemPool, cuDeviceGetMemPool,
cuMemPoolCreate
|
299
Modules
CUresult cuMemPoolGetAttribute (CUmemoryPool pool,
CUmemPool_attribute attr, void *value)
Gets attributes of a memory pool.
Parameters
pool
- The memory pool to get attributes of
attr
- The attribute to get
value
- Retrieved value
Returns
CUDA_SUCCESS, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE
Description
Supported attributes are:
‣ CU_MEMPOOL_ATTR_RELEASE_THRESHOLD: (value type = cuuint64_t) Amount of
reserved memory in bytes to hold onto before trying to release memory back to the OS. When more
than the release threshold bytes of memory are held by the memory pool, the allocator will try to
release memory back to the OS on the next call to stream, event or context synchronize. (default 0)
‣ CU_MEMPOOL_ATTR_REUSE_FOLLOW_EVENT_DEPENDENCIES: (value type = int)
Allow cuMemAllocAsync to use memory asynchronously freed in another stream as long as a
stream ordering dependency of the allocating stream on the free action exists. Cuda events and null
stream interactions can create the required stream ordered dependencies. (default enabled)
‣ CU_MEMPOOL_ATTR_REUSE_ALLOW_OPPORTUNISTIC: (value type = int) Allow reuse
of already completed frees when there is no dependency between the free and allocation. (default
enabled)
‣ CU_MEMPOOL_ATTR_REUSE_ALLOW_INTERNAL_DEPENDENCIES: (value type = int)
Allow cuMemAllocAsync to insert new stream dependencies in order to establish the stream
ordering required to reuse a piece of memory released by cuMemFreeAsync (default enabled).
‣ CU_MEMPOOL_ATTR_RESERVED_MEM_CURRENT: (value type = cuuint64_t) Amount of
backing memory currently allocated for the mempool
‣ CU_MEMPOOL_ATTR_RESERVED_MEM_HIGH: (value type = cuuint64_t) High watermark
of backing memory allocated for the mempool since the last time it was reset.
‣ CU_MEMPOOL_ATTR_USED_MEM_CURRENT: (value type = cuuint64_t) Amount of
memory from the pool that is currently in use by the application.
‣ CU_MEMPOOL_ATTR_USED_MEM_HIGH: (value type = cuuint64_t) High watermark of the
amount of memory from the pool that was in use by the application.
|
300
Modules
See also:
cuMemAllocAsync, cuMemFreeAsync, cuDeviceGetDefaultMemPool, cuDeviceGetMemPool,
cuMemPoolCreate
CUresult cuMemPoolImportFromShareableHandle
(CUmemoryPool *pool_out, void *handle,
CUmemAllocationHandleType handleType, unsigned long
long flags)
imports a memory pool from a shared handle.
Parameters
pool_out
- Returned memory pool
handle
- OS handle of the pool to open
handleType
- The type of handle being imported
flags
- must be 0
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_OUT_OF_MEMORY
Description
Specific allocations can be imported from the imported pool with cuMemPoolImportPointer.
If handleType is CU_MEM_HANDLE_TYPE_FABRIC and the importer process has not
been granted access to the same IMEX channel as the exporter process, this API will error as
CUDA_ERROR_NOT_PERMITTED.
Note:
Imported memory pools do not support creating new allocations. As such imported memory pools may
not be used in cuDeviceSetMemPool or cuMemAllocFromPoolAsync calls.
See also:
cuMemPoolExportToShareableHandle, cuMemPoolExportPointer, cuMemPoolImportPointer
|
301
Modules
CUresult cuMemPoolImportPointer (CUdeviceptr *ptr_out,
CUmemoryPool pool, CUmemPoolPtrExportData
*shareData)
Import a memory pool allocation from another process.
Parameters
ptr_out
- pointer to imported memory
pool
- pool from which to import
shareData
- data specifying the memory to import
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_OUT_OF_MEMORY
Description
Returns in ptr_out a pointer to the imported memory. The imported memory must not be accessed
before the allocation operation completes in the exporting process. The imported memory must be
freed from all importing processes before being freed in the exporting process. The pointer may be
freed with cuMemFree or cuMemFreeAsync. If cuMemFreeAsync is used, the free must be completed
on the importing process before the free operation on the exporting process.
Note:
The cuMemFreeAsync api may be used in the exporting process before the cuMemFreeAsync operation
completes in its stream as long as the cuMemFreeAsync in the exporting process specifies a stream with
a stream dependency on the importing process's cuMemFreeAsync.
See also:
cuMemPoolExportToShareableHandle, cuMemPoolImportFromShareableHandle,
cuMemPoolExportPointer
|
302
Modules
CUresult cuMemPoolSetAccess (CUmemoryPool pool,
const CUmemAccessDesc *map, size_t count)
Controls visibility of pools between devices.
Parameters
pool
- The pool being modified
map
- Array of access descriptors. Each descriptor instructs the access to enable for a single gpu.
count
- Number of descriptors in the map array.
Returns
CUDA_SUCCESS, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE
Description
See also:
cuMemAllocAsync, cuMemFreeAsync, cuDeviceGetDefaultMemPool, cuDeviceGetMemPool,
cuMemPoolCreate
CUresult cuMemPoolSetAttribute (CUmemoryPool pool,
CUmemPool_attribute attr, void *value)
Sets attributes of a memory pool.
Parameters
pool
- The memory pool to modify
attr
- The attribute to modify
value
- Pointer to the value to assign
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE
Description
Supported attributes are:
|
303
Modules
‣ CU_MEMPOOL_ATTR_RELEASE_THRESHOLD: (value type = cuuint64_t) Amount of
reserved memory in bytes to hold onto before trying to release memory back to the OS. When more
than the release threshold bytes of memory are held by the memory pool, the allocator will try to
release memory back to the OS on the next call to stream, event or context synchronize. (default 0)
‣ CU_MEMPOOL_ATTR_REUSE_FOLLOW_EVENT_DEPENDENCIES: (value type = int)
Allow cuMemAllocAsync to use memory asynchronously freed in another stream as long as a
stream ordering dependency of the allocating stream on the free action exists. Cuda events and null
stream interactions can create the required stream ordered dependencies. (default enabled)
‣ CU_MEMPOOL_ATTR_REUSE_ALLOW_OPPORTUNISTIC: (value type = int) Allow reuse
of already completed frees when there is no dependency between the free and allocation. (default
enabled)
‣ CU_MEMPOOL_ATTR_REUSE_ALLOW_INTERNAL_DEPENDENCIES: (value type = int)
Allow cuMemAllocAsync to insert new stream dependencies in order to establish the stream
ordering required to reuse a piece of memory released by cuMemFreeAsync (default enabled).
‣ CU_MEMPOOL_ATTR_RESERVED_MEM_HIGH: (value type = cuuint64_t) Reset the high
watermark that tracks the amount of backing memory that was allocated for the memory pool. It is
illegal to set this attribute to a non-zero value.
‣ CU_MEMPOOL_ATTR_USED_MEM_HIGH: (value type = cuuint64_t) Reset the high
watermark that tracks the amount of used memory that was allocated for the memory pool.
See also:
cuMemAllocAsync, cuMemFreeAsync, cuDeviceGetDefaultMemPool, cuDeviceGetMemPool,
cuMemPoolCreate
CUresult cuMemPoolTrimTo (CUmemoryPool pool, size_t
minBytesToKeep)
Tries to release memory back to the OS.
Parameters
pool
- The memory pool to trim
minBytesToKeep
- If the pool has less than minBytesToKeep reserved, the TrimTo operation is a no-op. Otherwise
the pool will be guaranteed to have at least minBytesToKeep bytes reserved after the operation.
Returns
CUDA_SUCCESS, CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_INVALID_VALUE
|
304
Modules
Description
Releases memory back to the OS until the pool contains fewer than minBytesToKeep reserved bytes,
or there is no more memory that the allocator can safely release. The allocator cannot release OS
allocations that back outstanding asynchronous allocations. The OS allocations may happen at different
granularity from the user allocations.
Note:
‣
: Allocations that have not been freed count as outstanding.
‣
: Allocations that have been asynchronously freed but whose completion has not been observed on
the host (eg. by a synchronize) can count as outstanding.
See also:
cuMemAllocAsync, cuMemFreeAsync, cuDeviceGetDefaultMemPool, cuDeviceGetMemPool,
cuMemPoolCreate
6.16. Multicast Object Management
This section describes the CUDA multicast object operations exposed by the low-level CUDA driver
application programming interface.
overview
A multicast object created via cuMulticastCreate enables certain memory operations to be broadcast
to a team of devices. Devices can be added to a multicast object via cuMulticastAddDevice. Memory
can be bound on each participating device via either cuMulticastBindMem or cuMulticastBindAddr.
Multicast objects can be mapped into a device's virtual address space using the virtual memmory
management APIs (see cuMemMap and cuMemSetAccess).
Supported Platforms
Support for multicast on a specific device can be queried using the device attribute
CU_DEVICE_ATTRIBUTE_MULTICAST_SUPPORTED
|
305
Modules
CUresult cuMulticastAddDevice
(CUmemGenericAllocationHandle mcHandle, CUdevice
dev)
Associate a device to a multicast object.
Parameters
mcHandle
Handle representing a multicast object.
dev
Device that will be associated to the multicast object.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY,
CUDA_ERROR_INVALID_DEVICE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
Associates a device to a multicast object. The added device will be a part of the multicast team of size
specified by CUmulticastObjectProp::numDevices during cuMulticastCreate. The association of the
device to the multicast object is permanent during the life time of the multicast object. All devices must
be added to the multicast team before any memory can be bound to any device in the team. Any calls to
cuMulticastBindMem or cuMulticastBindAddr will block until all devices have been added. Similarly
all devices must be added to the multicast team before a virtual address range can be mapped to the
multicast object. A call to cuMemMap will block until all devices have been added.
See also:
cuMulticastCreate, cuMulticastBindMem, cuMulticastBindAddr
|
306
Modules
CUresult cuMulticastBindAddr
(CUmemGenericAllocationHandle mcHandle, size_t
mcOffset, CUdeviceptr memptr, size_t size, unsigned long
long flags)
Bind a memory allocation represented by a virtual address to a multicast object.
Parameters
mcHandle
Handle representing a multicast object.
mcOffset
Offset into multicast va range for attachment.
memptr
Virtual address of the memory allocation.
size
Size of memory that will be bound to the multicast object.
flags
Flags for future use, must be zero now.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE,
CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED,
CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED,
CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_SYSTEM_NOT_READY,
CUDA_ERROR_ILLEGAL_STATE
Description
Binds a memory allocation specified by its mapped address memptr to a multicast object
represented by mcHandle. The memory must have been allocated via cuMemCreate or
cudaMallocAsync. The intended size of the bind, the offset in the multicast range mcOffset
and memptr must be a multiple of the value returned by cuMulticastGetGranularity with the flag
CU_MULTICAST_GRANULARITY_MINIMUM. For best performance however, size, mcOffset
and memptr should be aligned to the value returned by cuMulticastGetGranularity with the flag
CU_MULTICAST_GRANULARITY_RECOMMENDED.
The size cannot be larger than the size of the allocated memory. Similarly the size + mcOffset
cannot be larger than the total size of the multicast object. The memory allocation must have beeen
created on one of the devices that was added to the multicast team via cuMulticastAddDevice.
Externally shareable as well as imported multicast objects can be bound only to externally
shareable memory. Note that this call will return CUDA_ERROR_OUT_OF_MEMORY
if there are insufficient resources required to perform the bind. This call may also return
|
307
Modules
CUDA_ERROR_SYSTEM_NOT_READY if the necessary system software is not initialized or
running.
This call may return CUDA_ERROR_ILLEGAL_STATE if the system configuration is in an illegal
state. In such cases, to continue using multicast, verify that the system configuration is in a valid state
and all required driver daemons are running properly.
See also:
cuMulticastCreate, cuMulticastAddDevice, cuMemCreate
CUresult cuMulticastBindMem
(CUmemGenericAllocationHandle mcHandle, size_t
mcOffset, CUmemGenericAllocationHandle memHandle,
size_t memOffset, size_t size, unsigned long long flags)
Bind a memory allocation represented by a handle to a multicast object.
Parameters
mcHandle
Handle representing a multicast object.
mcOffset
Offset into the multicast object for attachment.
memHandle
Handle representing a memory allocation.
memOffset
Offset into the memory for attachment.
size
Size of the memory that will be bound to the multicast object.
flags
Flags for future use, must be zero for now.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE,
CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED,
CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED,
CUDA_ERROR_OUT_OF_MEMORY, CUDA_ERROR_SYSTEM_NOT_READY,
CUDA_ERROR_ILLEGAL_STATE
Description
Binds a memory allocation specified by memHandle and created via cuMemCreate to a
multicast object represented by mcHandle and created via cuMulticastCreate. The intended
|
308
Modules
size of the bind, the offset in the multicast range mcOffset as well as the offset in the
memory memOffset must be a multiple of the value returned by cuMulticastGetGranularity
with the flag CU_MULTICAST_GRANULARITY_MINIMUM. For best performance
however, size, mcOffset and memOffset should be aligned to the granularity of
the memory allocation(see ::cuMemGetAllocationGranularity) or to the value returned by
cuMulticastGetGranularity with the flag CU_MULTICAST_GRANULARITY_RECOMMENDED.
The size + memOffset cannot be larger than the size of the allocated memory. Similarly
the size + mcOffset cannot be larger than the size of the multicast object. The memory
allocation must have beeen created on one of the devices that was added to the multicast team via
cuMulticastAddDevice. Externally shareable as well as imported multicast objects can be bound only
to externally shareable memory. Note that this call will return CUDA_ERROR_OUT_OF_MEMORY
if there are insufficient resources required to perform the bind. This call may also return
CUDA_ERROR_SYSTEM_NOT_READY if the necessary system software is not initialized or
running.
This call may return CUDA_ERROR_ILLEGAL_STATE if the system configuration is in an illegal
state. In such cases, to continue using multicast, verify that the system configuration is in a valid state
and all required driver daemons are running properly.
See also:
cuMulticastCreate, cuMulticastAddDevice, cuMemCreate
CUresult cuMulticastCreate
(CUmemGenericAllocationHandle *mcHandle, const
CUmulticastObjectProp *prop)
Create a generic allocation handle representing a multicast object described by the given properties.
Parameters
mcHandle
Value of handle returned.
prop
Properties of the multicast object to create.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_OUT_OF_MEMORY,
CUDA_ERROR_INVALID_DEVICE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
|
309
Modules
Description
This creates a multicast object as described by prop. The number of participating devices
is specified by CUmulticastObjectProp::numDevices. Devices can be added to the multicast
object via cuMulticastAddDevice. All participating devices must be added to the multicast
object before memory can be bound to it. Memory is bound to the multicast object via either
cuMulticastBindMem or cuMulticastBindAddr, and can be unbound via cuMulticastUnbind. The
total amount of memory that can be bound per device is specified by :CUmulticastObjectProp::size.
This size must be a multiple of the value returned by cuMulticastGetGranularity with the
flag CU_MULTICAST_GRANULARITY_MINIMUM. For best performance however,
the size should be aligned to the value returned by cuMulticastGetGranularity with the flag
CU_MULTICAST_GRANULARITY_RECOMMENDED.
After all participating devices have been added, multicast objects can also be mapped to a
device's virtual address space using the virtual memory management APIs (see cuMemMap and
cuMemSetAccess). Multicast objects can also be shared with other processes by requesting a shareable
handle via cuMemExportToShareableHandle. Note that the desired types of shareable handles must be
specified in the bitmask CUmulticastObjectProp::handleTypes. Multicast objects can be released using
the virtual memory management API cuMemRelease.
See also:
cuMulticastAddDevice, cuMulticastBindMem, cuMulticastBindAddr, cuMulticastUnbind
cuMemCreate, cuMemRelease, cuMemExportToShareableHandle,
cuMemImportFromShareableHandle
CUresult cuMulticastGetGranularity (size_t
*granularity, const CUmulticastObjectProp *prop,
CUmulticastGranularity_flags option)
Calculates either the minimal or recommended granularity for multicast object.
Parameters
granularity
Returned granularity.
prop
Properties of the multicast object.
option
Determines which granularity to return.
|
310
Modules
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_PERMITTED,
CUDA_ERROR_NOT_SUPPORTED
Description
Calculates either the minimal or recommended granularity for a given set of multicast object properties
and returns it in granularity. This granularity can be used as a multiple for size, bind offsets and address
mappings of the multicast object.
See also:
cuMulticastCreate, cuMulticastBindMem, cuMulticastBindAddr, cuMulticastUnbind
CUresult cuMulticastUnbind
(CUmemGenericAllocationHandle mcHandle, CUdevice
dev, size_t mcOffset, size_t size)
Unbind any memory allocations bound to a multicast object at a given offset and upto a given size.
Parameters
mcHandle
Handle representing a multicast object.
dev
Device that hosts the memory allocation.
mcOffset
Offset into the multicast object.
size
Desired size to unbind.
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE,
CUDA_ERROR_NOT_INITIALIZED, CUDA_ERROR_DEINITIALIZED,
CUDA_ERROR_NOT_PERMITTED, CUDA_ERROR_NOT_SUPPORTED
Description
Unbinds any memory allocations hosted on dev and bound to a multicast object at mcOffset
and upto a given size. The intended size of the unbind and the offset in the multicast range
( mcOffset ) must be a multiple of the value returned by cuMulticastGetGranularity flag
|
311
Modules
CU_MULTICAST_GRANULARITY_MINIMUM. The size + mcOffset cannot be larger than the
total size of the multicast object.
Note:
Warning: The mcOffset and the size must match the corresponding values specified during the
bind call. Any other values may result in undefined behavior.
See also:
cuMulticastBindMem, cuMulticastBindAddr
6.17. Unified Addressing
This section describes the unified addressing functions of the low-level CUDA driver application
programming interface.
Overview
CUDA devices can share a unified address space with the host. For these devices there is no distinction
between a device pointer and a host pointer -- the same pointer value may be used to access memory
from the host program and from a kernel running on the device (with exceptions enumerated below).
Supported Platforms
Whether or not a device supports unified addressing may be queried by calling cuDeviceGetAttribute()
with the device attribute CU_DEVICE_ATTRIBUTE_UNIFIED_ADDRESSING.
Unified addressing is automatically enabled in 64-bit processes
Looking Up Information from Pointer Values
It is possible to look up information about the memory which backs a pointer value. For instance, one
may want to know if a pointer points to host or device memory. As another example, in the case of
device memory, one may want to know on which CUDA device the memory resides. These properties
may be queried using the function cuPointerGetAttribute()
Since pointers are unique, it is not necessary to specify information about the pointers specified to
the various copy functions in the CUDA API. The function cuMemcpy() may be used to perform
a copy between two pointers, ignoring whether they point to host or device memory (making
cuMemcpyHtoD(), cuMemcpyDtoD(), and cuMemcpyDtoH() unnecessary for devices supporting
unified addressing). For multidimensional copies, the memory type CU_MEMORYTYPE_UNIFIED
may be used to specify that the CUDA driver should infer the location of the pointer from its value.
Automatic Mapping of Host Allocated Host Memory
All host memory allocated in all contexts using cuMemAllocHost() and cuMemHostAlloc() is
always directly accessible from all contexts on all devices that support unified addressing. This
|
312
Modules
is the case regardless of whether or not the flags CU_MEMHOSTALLOC_PORTABLE and
CU_MEMHOSTALLOC_DEVICEMAP are specified.
The pointer value through which allocated host memory may be accessed in kernels on all devices that
support unified addressing is the same as the pointer value through which that memory is accessed on
the host, so it is not necessary to call cuMemHostGetDevicePointer() to get the device pointer for these
allocations.
Note that this is not the case for memory allocated using the flag
CU_MEMHOSTALLOC_WRITECOMBINED, as discussed below.
Automatic Registration of Peer Memory
Upon enabling direct access from a context that supports unified addressing to another peer context that
supports unified addressing using cuCtxEnablePeerAccess() all memory allocated in the peer context
using cuMemAlloc() and cuMemAllocPitch() will immediately be accessible by the current context.
The device pointer value through which any peer memory may be accessed in the current context is the
same pointer value through which that memory may be accessed in the peer context.
Exceptions, Disjoint Addressing
Not all memory may be accessed on devices through the same pointer value through which they are
accessed on the host. These exceptions are host memory registered using cuMemHostRegister() and
host memory allocated using the flag CU_MEMHOSTALLOC_WRITECOMBINED. For these
exceptions, there exists a distinct host and device address for the memory. The device address is
guaranteed to not overlap any valid host pointer range and is guaranteed to have the same value across
all contexts that support unified addressing.
This device address may be queried using cuMemHostGetDevicePointer() when a context using
unified addressing is current. Either the host or the unified device pointer value may be used to refer
to this memory through cuMemcpy() and similar functions using the CU_MEMORYTYPE_UNIFIED
memory type.
CUresult cuMemAdvise (CUdeviceptr devPtr, size_t count,
CUmem_advise advice, CUdevice device)
Advise about the usage of a given memory range.
Parameters
devPtr
- Pointer to memory to set the advice for
count
- Size in bytes of the memory range
advice
- Advice to be applied for the specified memory range
device
- Device to apply the advice for
|
313
Modules
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE
Description
Note there is a later version of this API, cuMemAdvise_v2. It will supplant this version in 13.0, which
is retained for minor version compatibility.
Advise the Unified Memory subsystem about the usage pattern for the memory range starting at
devPtr with a size of count bytes. The start address and end address of the memory range will be
rounded down and rounded up respectively to be aligned to CPU page size before the advice is applied.
The memory range must refer to managed memory allocated via cuMemAllocManaged or declared
via __managed__ variables. The memory range could also refer to system-allocated pageable memory
provided it represents a valid, host-accessible region of memory and all additional constraints imposed
by advice as outlined below are also satisfied. Specifying an invalid system-allocated pageable
memory range results in an error being returned.
The advice parameter can take the following values:
‣ CU_MEM_ADVISE_SET_READ_MOSTLY: This implies that the data is mostly going to
be read from and only occasionally written to. Any read accesses from any processor to this
region will create a read-only copy of at least the accessed pages in that processor's memory.
Additionally, if cuMemPrefetchAsync is called on this region, it will create a read-only copy
of the data on the destination processor. If any processor writes to this region, all copies of
the corresponding page will be invalidated except for the one where the write occurred. The
device argument is ignored for this advice. Note that for a page to be read-duplicated,
the accessing processor must either be the CPU or a GPU that has a non-zero value for the
device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS.
Also, if a context is created on a device that does not have the device attribute
CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS set, then read-duplication
will not occur until all such contexts are destroyed. If the memory region refers to valid
system-allocated pageable memory, then the accessing device must have a non-zero value
for the device attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS
for a read-only copy to be created on that device. Note however that if
the accessing device also has a non-zero value for the device attribute
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS_USES_HOST_PAGE_TABLES,
then setting this advice will not create a read-only copy when that device accesses this memory
region.
‣ CU_MEM_ADVISE_UNSET_READ_MOSTLY: Undoes the effect of
CU_MEM_ADVISE_SET_READ_MOSTLY and also prevents the Unified Memory driver from
attempting heuristic read-duplication on the memory range. Any read-duplicated copies of the
data will be collapsed into a single copy. The location for the collapsed copy will be the preferred
location if the page has a preferred location and one of the read-duplicated copies was resident at
that location. Otherwise, the location chosen is arbitrary.
|
314
Modules
‣ CU_MEM_ADVISE_SET_PREFERRED_LOCATION: This advice sets the preferred location for
the data to be the memory belonging to device. Passing in CU_DEVICE_CPU for device sets
the preferred location as host memory. If device is a GPU, then it must have a non-zero value
for the device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS.
Setting the preferred location does not cause data to migrate to that location immediately. Instead,
it guides the migration policy when a fault occurs on that memory region. If the data is already
in its preferred location and the faulting processor can establish a mapping without requiring
the data to be migrated, then data migration will be avoided. On the other hand, if the data is
not in its preferred location or if a direct mapping cannot be established, then it will be migrated
to the processor accessing it. It is important to note that setting the preferred location does not
prevent data prefetching done using cuMemPrefetchAsync. Having a preferred location can
override the page thrash detection and resolution logic in the Unified Memory driver. Normally,
if a page is detected to be constantly thrashing between for example host and device memory,
the page may eventually be pinned to host memory by the Unified Memory driver. But if the
preferred location is set as device memory, then the page will continue to thrash indefinitely. If
CU_MEM_ADVISE_SET_READ_MOSTLY is also set on this memory region or any subset of
it, then the policies associated with that advice will override the policies of this advice, unless read
accesses from device will not result in a read-only copy being created on that device as outlined
in description for the advice CU_MEM_ADVISE_SET_READ_MOSTLY. If the memory region
refers to valid system-allocated pageable memory, then device must have a non-zero value for
the device attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
‣ CU_MEM_ADVISE_UNSET_PREFERRED_LOCATION: Undoes the effect of
CU_MEM_ADVISE_SET_PREFERRED_LOCATION and changes the preferred location to
none.
‣ CU_MEM_ADVISE_SET_ACCESSED_BY: This advice implies that the
data will be accessed by device. Passing in CU_DEVICE_CPU for device
will set the advice for the CPU. If device is a GPU, then the device attribute
CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS must be non-zero.
This advice does not cause data migration and has no impact on the location of the data per
se. Instead, it causes the data to always be mapped in the specified processor's page tables, as
long as the location of the data permits a mapping to be established. If the data gets migrated
for any reason, the mappings are updated accordingly. This advice is recommended in
scenarios where data locality is not important, but avoiding faults is. Consider for example a
system containing multiple GPUs with peer-to-peer access enabled, where the data located
on one GPU is occasionally accessed by peer GPUs. In such scenarios, migrating data over
to the other GPUs is not as important because the accesses are infrequent and the overhead
of migration may be too high. But preventing faults can still help improve performance,
and so having a mapping set up in advance is useful. Note that on CPU access of this data,
the data may be migrated to host memory because the CPU typically cannot access device
memory directly. Any GPU that had the CU_MEM_ADVISE_SET_ACCESSED_BY flag
set for this data will now have its mapping updated to point to the page in host memory. If
CU_MEM_ADVISE_SET_READ_MOSTLY is also set on this memory region or any subset
|
315
Modules
of it, then the policies associated with that advice will override the policies of this advice.
Additionally, if the preferred location of this memory region or any subset of it is also device,
then the policies associated with CU_MEM_ADVISE_SET_PREFERRED_LOCATION
will override the policies of this advice. If the memory region refers to valid system-
allocated pageable memory, then device must have a non-zero value for the
device attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
Additionally, if device has a non-zero value for the device attribute
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS_USES_HOST_PAGE_TABLES,
then this call has no effect.
‣ CU_MEM_ADVISE_UNSET_ACCESSED_BY: Undoes the effect of
CU_MEM_ADVISE_SET_ACCESSED_BY. Any mappings to the data from device may be
removed at any time causing accesses to result in non-fatal page faults. If the memory region
refers to valid system-allocated pageable memory, then device must have a non-zero value
for the device attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
Additionally, if device has a non-zero value for the device attribute
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS_USES_HOST_PAGE_TABLES,
then this call has no effect.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ This function exhibits asynchronous behavior for most use cases.
‣ This function uses standard default stream semantics.
See also:
cuMemcpy, cuMemcpyPeer, cuMemcpyAsync, cuMemcpy3DPeerAsync, cuMemPrefetchAsync,
cuMemAdvise_v2, cudaMemAdvise
CUresult cuMemAdvise_v2 (CUdeviceptr devPtr, size_t
count, CUmem_advise advice, CUmemLocation location)
Advise about the usage of a given memory range.
Parameters
devPtr
- Pointer to memory to set the advice for
count
- Size in bytes of the memory range
advice
- Advice to be applied for the specified memory range
|
316
Modules
location
- location to apply the advice for
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE
Description
Advise the Unified Memory subsystem about the usage pattern for the memory range starting at
devPtr with a size of count bytes. The start address and end address of the memory range will be
rounded down and rounded up respectively to be aligned to CPU page size before the advice is applied.
The memory range must refer to managed memory allocated via cuMemAllocManaged or declared
via __managed__ variables. The memory range could also refer to system-allocated pageable memory
provided it represents a valid, host-accessible region of memory and all additional constraints imposed
by advice as outlined below are also satisfied. Specifying an invalid system-allocated pageable
memory range results in an error being returned.
The advice parameter can take the following values:
‣ CU_MEM_ADVISE_SET_READ_MOSTLY: This implies that the data is mostly going to
be read from and only occasionally written to. Any read accesses from any processor to this
region will create a read-only copy of at least the accessed pages in that processor's memory.
Additionally, if cuMemPrefetchAsync or cuMemPrefetchAsync_v2 is called on this region, it
will create a read-only copy of the data on the destination processor. If the target location for
cuMemPrefetchAsync_v2 is a host NUMA node and a read-only copy already exists on another
host NUMA node, that copy will be migrated to the targeted host NUMA node. If any processor
writes to this region, all copies of the corresponding page will be invalidated except for the
one where the write occurred. If the writing processor is the CPU and the preferred location
of the page is a host NUMA node, then the page will also be migrated to that host NUMA
node. The location argument is ignored for this advice. Note that for a page to be read-
duplicated, the accessing processor must either be the CPU or a GPU that has a non-zero value
for the device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS.
Also, if a context is created on a device that does not have the device attribute
CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS set, then read-duplication
will not occur until all such contexts are destroyed. If the memory region refers to valid
system-allocated pageable memory, then the accessing device must have a non-zero value
for the device attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS
for a read-only copy to be created on that device. Note however that if
the accessing device also has a non-zero value for the device attribute
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS_USES_HOST_PAGE_TABLES,
then setting this advice will not create a read-only copy when that device accesses this memory
region.
‣ CU_MEM_ADVISE_UNSET_READ_MOSTLY: Undoes the effect of
CU_MEM_ADVISE_SET_READ_MOSTLY and also prevents the Unified Memory driver from
|
317
Modules
attempting heuristic read-duplication on the memory range. Any read-duplicated copies of the
data will be collapsed into a single copy. The location for the collapsed copy will be the preferred
location if the page has a preferred location and one of the read-duplicated copies was resident
at that location. Otherwise, the location chosen is arbitrary. Note: The location argument is
ignored for this advice.
‣ CU_MEM_ADVISE_SET_PREFERRED_LOCATION: This advice sets the preferred location
for the data to be the memory belonging to location. When CUmemLocation::type is
CU_MEM_LOCATION_TYPE_HOST, CUmemLocation::id is ignored and the preferred
location is set to be host memory. To set the preferred location to a specific host NUMA node,
applications must set CUmemLocation::type to CU_MEM_LOCATION_TYPE_HOST_NUMA
and CUmemLocation::id must specify the NUMA ID of the host NUMA node. If
CUmemLocation::type is set to CU_MEM_LOCATION_TYPE_HOST_NUMA_CURRENT,
CUmemLocation::id will be ignored and the the host NUMA node closest to the
calling thread's CPU will be used as the preferred location. If CUmemLocation::type
is a CU_MEM_LOCATION_TYPE_DEVICE, then CUmemLocation::id must be a
valid device ordinal and the device must have a non-zero value for the device attribute
CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS. Setting the preferred
location does not cause data to migrate to that location immediately. Instead, it guides the
migration policy when a fault occurs on that memory region. If the data is already in its preferred
location and the faulting processor can establish a mapping without requiring the data to
be migrated, then data migration will be avoided. On the other hand, if the data is not in its
preferred location or if a direct mapping cannot be established, then it will be migrated to the
processor accessing it. It is important to note that setting the preferred location does not prevent
data prefetching done using cuMemPrefetchAsync. Having a preferred location can override
the page thrash detection and resolution logic in the Unified Memory driver. Normally, if a
page is detected to be constantly thrashing between for example host and device memory,
the page may eventually be pinned to host memory by the Unified Memory driver. But if the
preferred location is set as device memory, then the page will continue to thrash indefinitely. If
CU_MEM_ADVISE_SET_READ_MOSTLY is also set on this memory region or any subset
of it, then the policies associated with that advice will override the policies of this advice,
unless read accesses from location will not result in a read-only copy being created on that
procesor as outlined in description for the advice CU_MEM_ADVISE_SET_READ_MOSTLY.
If the memory region refers to valid system-allocated pageable memory, and
CUmemLocation::type is CU_MEM_LOCATION_TYPE_DEVICE then
CUmemLocation::id must be a valid device that has a non-zero alue for the device attribute
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
‣ CU_MEM_ADVISE_UNSET_PREFERRED_LOCATION: Undoes the effect of
CU_MEM_ADVISE_SET_PREFERRED_LOCATION and changes the preferred location to
none. The location argument is ignored for this advice.
‣ CU_MEM_ADVISE_SET_ACCESSED_BY: This advice implies that the data will
be accessed by processor location. The CUmemLocation::type must be either
CU_MEM_LOCATION_TYPE_DEVICE with CUmemLocation::id representing a valid
|
318
Modules
device ordinal or CU_MEM_LOCATION_TYPE_HOST and CUmemLocation::id will be
ignored. All other location types are invalid. If CUmemLocation::id is a GPU, then the device
attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS must be
non-zero. This advice does not cause data migration and has no impact on the location of the
data per se. Instead, it causes the data to always be mapped in the specified processor's page
tables, as long as the location of the data permits a mapping to be established. If the data gets
migrated for any reason, the mappings are updated accordingly. This advice is recommended
in scenarios where data locality is not important, but avoiding faults is. Consider for example
a system containing multiple GPUs with peer-to-peer access enabled, where the data located
on one GPU is occasionally accessed by peer GPUs. In such scenarios, migrating data over
to the other GPUs is not as important because the accesses are infrequent and the overhead
of migration may be too high. But preventing faults can still help improve performance,
and so having a mapping set up in advance is useful. Note that on CPU access of this data,
the data may be migrated to host memory because the CPU typically cannot access device
memory directly. Any GPU that had the CU_MEM_ADVISE_SET_ACCESSED_BY flag
set for this data will now have its mapping updated to point to the page in host memory. If
CU_MEM_ADVISE_SET_READ_MOSTLY is also set on this memory region or any subset
of it, then the policies associated with that advice will override the policies of this advice.
Additionally, if the preferred location of this memory region or any subset of it is also location,
then the policies associated with CU_MEM_ADVISE_SET_PREFERRED_LOCATION
will override the policies of this advice. If the memory region refers to valid system-allocated
pageable memory, and CUmemLocation::type is CU_MEM_LOCATION_TYPE_DEVICE
then device in CUmemLocation::id must have a non-zero value for the device
attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
Additionally, if CUmemLocation::id has a non-zero value for the device attribute
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS_USES_HOST_PAGE_TABLES,
then this call has no effect.
‣ CU_MEM_ADVISE_UNSET_ACCESSED_BY: Undoes the effect of
CU_MEM_ADVISE_SET_ACCESSED_BY. Any mappings to the data from
location may be removed at any time causing accesses to result in non-
fatal page faults. If the memory region refers to valid system-allocated pageable
memory, and CUmemLocation::type is CU_MEM_LOCATION_TYPE_DEVICE
then device in CUmemLocation::id must have a non-zero value for the device
attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
Additionally, if CUmemLocation::id has a non-zero value for the device attribute
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS_USES_HOST_PAGE_TABLES,
then this call has no effect.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ This function exhibits asynchronous behavior for most use cases.
|
319
Modules
‣ This function uses standard default stream semantics.
See also:
cuMemcpy, cuMemcpyPeer, cuMemcpyAsync, cuMemcpy3DPeerAsync, cuMemPrefetchAsync,
cuMemAdvise, cudaMemAdvise
CUresult cuMemPrefetchAsync (CUdeviceptr devPtr,
size_t count, CUdevice dstDevice, CUstream hStream)
Prefetches memory to the specified destination device.
Parameters
devPtr
- Pointer to be prefetched
count
- Size in bytes
dstDevice
- Destination device to prefetch to
hStream
- Stream to enqueue prefetch operation
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE
Description
Note there is a later version of this API, cuMemPrefetchAsync_v2. It will supplant this version in 13.0,
which is retained for minor version compatibility.
Prefetches memory to the specified destination device. devPtr is the base device pointer
of the memory to be prefetched and dstDevice is the destination device. count specifies
the number of bytes to copy. hStream is the stream in which the operation is enqueued. The
memory range must refer to managed memory allocated via cuMemAllocManaged or declared via
__managed__ variables or it may also refer to system-allocated memory on systems with non-zero
CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
Passing in CU_DEVICE_CPU for dstDevice will prefetch the data
to host memory. If dstDevice is a GPU, then the device attribute
CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS must be non-zero.
Additionally, hStream must be associated with a device that has a non-zero value for the device
attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS.
The start address and end address of the memory range will be rounded down and rounded up
respectively to be aligned to CPU page size before the prefetch operation is enqueued in the stream.
|
320
Modules
If no physical memory has been allocated for this region, then this memory region will be populated
and mapped on the destination device. If there's insufficient memory to prefetch the desired region, the
Unified Memory driver may evict pages from other cuMemAllocManaged allocations to host memory
in order to make room. Device memory allocated using cuMemAlloc or cuArrayCreate will not be
evicted.
By default, any mappings to the previous location of the migrated pages are removed and mappings
for the new location are only setup on dstDevice. The exact behavior however also depends on the
settings applied to this memory range via cuMemAdvise as described below:
If CU_MEM_ADVISE_SET_READ_MOSTLY was set on any subset of this memory range, then that
subset will create a read-only copy of the pages on dstDevice.
If CU_MEM_ADVISE_SET_PREFERRED_LOCATION was called on any subset of this memory
range, then the pages will be migrated to dstDevice even if dstDevice is not the preferred
location of any pages in the memory range.
If CU_MEM_ADVISE_SET_ACCESSED_BY was called on any subset of this memory range, then
mappings to those pages from all the appropriate processors are updated to refer to the new location if
establishing such a mapping is possible. Otherwise, those mappings are cleared.
Note that this API is not required for functionality and only serves to improve performance by allowing
the application to migrate data to a suitable location before it is accessed. Memory accesses to this
range are always coherent and are allowed even when the data is actively being migrated.
Note that this function is asynchronous with respect to the host and all work on other devices.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ This function exhibits asynchronous behavior for most use cases.
‣ This function uses standard default stream semantics.
See also:
cuMemcpy, cuMemcpyPeer, cuMemcpyAsync, cuMemcpy3DPeerAsync, cuMemAdvise,
cuMemPrefetchAsync cudaMemPrefetchAsync_v2
|
321
Modules
CUresult cuMemPrefetchAsync_v2 (CUdeviceptr devPtr,
size_t count, CUmemLocation location, unsigned int flags,
CUstream hStream)
Prefetches memory to the specified destination location.
Parameters
devPtr
- Pointer to be prefetched
count
- Size in bytes
location
- Location to prefetch to
flags
- flags for future use, must be zero now.
hStream
- Stream to enqueue prefetch operation
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE
Description
Prefetches memory to the specified destination location. devPtr is the base device pointer of
the memory to be prefetched and location specifies the destination location. count specifies
the number of bytes to copy. hStream is the stream in which the operation is enqueued. The
memory range must refer to managed memory allocated via cuMemAllocManaged or declared via
__managed__ variables.
Specifying CU_MEM_LOCATION_TYPE_DEVICE for CUmemLocation::type will prefetch
memory to GPU specified by device ordinal CUmemLocation::id which must have non-zero value
for the device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS.
Additionally, hStream must be associated with a device that has a non-zero value for the
device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS.
Specifying CU_MEM_LOCATION_TYPE_HOST as CUmemLocation::type will prefetch
data to host memory. Applications can request prefetching memory to a specific host NUMA
node by specifying CU_MEM_LOCATION_TYPE_HOST_NUMA for CUmemLocation::type
and a valid host NUMA node id in CUmemLocation::id Users can also request prefetching
memory to the host NUMA node closest to the current thread's CPU by specifying
CU_MEM_LOCATION_TYPE_HOST_NUMA_CURRENT for CUmemLocation::type.
Note when CUmemLocation::type is etiher CU_MEM_LOCATION_TYPE_HOST OR
CU_MEM_LOCATION_TYPE_HOST_NUMA_CURRENT, CUmemLocation::id will be ignored.
|
322
Modules
The start address and end address of the memory range will be rounded down and rounded up
respectively to be aligned to CPU page size before the prefetch operation is enqueued in the stream.
If no physical memory has been allocated for this region, then this memory region will be populated
and mapped on the destination device. If there's insufficient memory to prefetch the desired region, the
Unified Memory driver may evict pages from other cuMemAllocManaged allocations to host memory
in order to make room. Device memory allocated using cuMemAlloc or cuArrayCreate will not be
evicted.
By default, any mappings to the previous location of the migrated pages are removed and mappings for
the new location are only setup on the destination location. The exact behavior however also depends
on the settings applied to this memory range via cuMemAdvise as described below:
If CU_MEM_ADVISE_SET_READ_MOSTLY was set on any subset of this memory range, then
that subset will create a read-only copy of the pages on destination location. If however the destination
location is a host NUMA node, then any pages of that subset that are already in another host NUMA
node will be transferred to the destination.
If CU_MEM_ADVISE_SET_PREFERRED_LOCATION was called on any subset of this memory
range, then the pages will be migrated to location even if location is not the preferred location
of any pages in the memory range.
If CU_MEM_ADVISE_SET_ACCESSED_BY was called on any subset of this memory range, then
mappings to those pages from all the appropriate processors are updated to refer to the new location if
establishing such a mapping is possible. Otherwise, those mappings are cleared.
Note that this API is not required for functionality and only serves to improve performance by allowing
the application to migrate data to a suitable location before it is accessed. Memory accesses to this
range are always coherent and are allowed even when the data is actively being migrated.
Note that this function is asynchronous with respect to the host and all work on other devices.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ This function exhibits asynchronous behavior for most use cases.
‣ This function uses standard default stream semantics.
See also:
cuMemcpy, cuMemcpyPeer, cuMemcpyAsync, cuMemcpy3DPeerAsync, cuMemAdvise,
cuMemPrefetchAsync, cudaMemPrefetchAsync_v2
|
323
Modules
CUresult cuMemRangeGetAttribute (void *data, size_t
dataSize, CUmem_range_attribute attribute, CUdeviceptr
devPtr, size_t count)
Query an attribute of a given memory range.
Parameters
data
- A pointers to a memory location where the result of each attribute query will be written to.
dataSize
- Array containing the size of data
attribute
- The attribute to query
devPtr
- Start of the range to query
count
- Size of the range to query
Returns
CUDA_SUCCESS, CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE
Description
Query an attribute about the memory range starting at devPtr with a size of count bytes. The
memory range must refer to managed memory allocated via cuMemAllocManaged or declared via
__managed__ variables.
The attribute parameter can take the following values:
‣ CU_MEM_RANGE_ATTRIBUTE_READ_MOSTLY: If this attribute is specified, data will be
interpreted as a 32-bit integer, and dataSize must be 4. The result returned will be 1 if all pages
in the given memory range have read-duplication enabled, or 0 otherwise.
‣ CU_MEM_RANGE_ATTRIBUTE_PREFERRED_LOCATION: If this attribute is specified,
data will be interpreted as a 32-bit integer, and dataSize must be 4. The result returned will
be a GPU device id if all pages in the memory range have that GPU as their preferred location,
or it will be CU_DEVICE_CPU if all pages in the memory range have the CPU as their preferred
location, or it will be CU_DEVICE_INVALID if either all the pages don't have the same preferred
location or some of the pages don't have a preferred location at all. Note that the actual location
of the pages in the memory range at the time of the query may be different from the preferred
location.
‣ CU_MEM_RANGE_ATTRIBUTE_ACCESSED_BY: If this attribute is specified, data will be
interpreted as an array of 32-bit integers, and dataSize must be a non-zero multiple of 4. The
result returned will be a list of device ids that had CU_MEM_ADVISE_SET_ACCESSED_BY
|
324
Modules
set for that entire memory range. If any device does not have that advice set for the entire memory
range, that device will not be included. If data is larger than the number of devices that have that
advice set for that memory range, CU_DEVICE_INVALID will be returned in all the extra space
provided. For ex., if dataSize is 12 (i.e. data has 3 elements) and only device 0 has the advice
set, then the result returned will be { 0, CU_DEVICE_INVALID, CU_DEVICE_INVALID }. If
data is smaller than the number of devices that have that advice set, then only as many devices
will be returned as can fit in the array. There is no guarantee on which specific devices will be
returned, however.
‣ CU_MEM_RANGE_ATTRIBUTE_LAST_PREFETCH_LOCATION: If this attribute is specified,
data will be interpreted as a 32-bit integer, and dataSize must be 4. The result returned
will be the last location to which all pages in the memory range were prefetched explicitly via
cuMemPrefetchAsync. This will either be a GPU id or CU_DEVICE_CPU depending on whether
the last location for prefetch was a GPU or the CPU respectively. If any page in the memory
range was never explicitly prefetched or if all pages were not prefetched to the same location,
CU_DEVICE_INVALID will be returned. Note that this simply returns the last location that the
application requested to prefetch the memory range to. It gives no indication as to whether the
prefetch operation to that location has completed or even begun.
‣ CU_MEM_RANGE_ATTRIBUTE_PREFERRED_LOCATION_TYPE: If this attribute
is specified, data will be interpreted as a CUmemLocationType, and dataSize
must be sizeof(CUmemLocationType). The CUmemLocationType returned will be
CU_MEM_LOCATION_TYPE_DEVICE if all pages in the memory range have the same GPU as
their preferred location, or CUmemLocationType will be CU_MEM_LOCATION_TYPE_HOST
if all pages in the memory range have the CPU as their preferred location, or it will
be CU_MEM_LOCATION_TYPE_HOST_NUMA if all the pages in the memory
range have the same host NUMA node ID as their preferred location or it will be
CU_MEM_LOCATION_TYPE_INVALID if either all the pages don't have the same preferred
location or some of the pages don't have a preferred location at all. Note that the actual location
type of the pages in the memory range at the time of the query may be different from the preferred
location type.
‣ CU_MEM_RANGE_ATTRIBUTE_PREFERRED_LOCATION_ID: If this attribute is
specified, data will be interpreted as a 32-bit integer, and dataSize must be 4. If the
CU_MEM_RANGE_ATTRIBUTE_PREFERRED_LOCATION_TYPE query for the same
address range returns CU_MEM_LOCATION_TYPE_DEVICE, it will be a valid device
ordinal or if it returns CU_MEM_LOCATION_TYPE_HOST_NUMA, it will be a valid host
NUMA node ID or if it returns any other location type, the id should be ignored.
‣ CU_MEM_RANGE_ATTRIBUTE_LAST_PREFETCH_LOCATION_TYPE: If this attribute
is specified, data will be interpreted as a CUmemLocationType, and dataSize must
be sizeof(CUmemLocationType). The result returned will be the last location to which
all pages in the memory range were prefetched explicitly via cuMemPrefetchAsync. The
CUmemLocationType returned will be CU_MEM_LOCATION_TYPE_DEVICE if the
last prefetch location was a GPU or CU_MEM_LOCATION_TYPE_HOST if it was the
CPU or CU_MEM_LOCATION_TYPE_HOST_NUMA if the last prefetch location was a
|
325
Modules
specific host NUMA node. If any page in the memory range was never explicitly prefetched
or if all pages were not prefetched to the same location, CUmemLocationType will be
CU_MEM_LOCATION_TYPE_INVALID. Note that this simply returns the last location type that
the application requested to prefetch the memory range to. It gives no indication as to whether the
prefetch operation to that location has completed or even begun.
‣ CU_MEM_RANGE_ATTRIBUTE_LAST_PREFETCH_LOCATION_ID: If this attribute
is specified, data will be interpreted as a 32-bit integer, and dataSize must be 4. If the
CU_MEM_RANGE_ATTRIBUTE_LAST_PREFETCH_LOCATION_TYPE query for the
same address range returns CU_MEM_LOCATION_TYPE_DEVICE, it will be a valid device
ordinal or if it returns CU_MEM_LOCATION_TYPE_HOST_NUMA, it will be a valid host
NUMA node ID or if it returns any other location type, the id should be ignored.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ This function exhibits asynchronous behavior for most use cases.
‣ This function uses standard default stream semantics.
See also:
cuMemRangeGetAttributes, cuMemPrefetchAsync, cuMemAdvise, cudaMemRangeGetAttribute
CUresult cuMemRangeGetAttributes (void **data, size_t
*dataSizes, CUmem_range_attribute *attributes, size_t
numAttributes, CUdeviceptr devPtr, size_t count)
Query attributes of a given memory range.
Parameters
data
- A two-dimensional array containing pointers to memory locations where the result of each
attribute query will be written to.
dataSizes
- Array containing the sizes of each result
attributes
- An array of attributes to query (numAttributes and the number of attributes in this array should
match)
numAttributes
- Number of attributes to query
devPtr
- Start of the range to query
|
326
Modules
count
- Size of the range to query
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_INVALID_CONTEXT,
CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE
Description
Query attributes of the memory range starting at devPtr with a size of count bytes. The
memory range must refer to managed memory allocated via cuMemAllocManaged or declared via
__managed__ variables. The attributes array will be interpreted to have numAttributes
entries. The dataSizes array will also be interpreted to have numAttributes entries. The results
of the query will be stored in data.
The list of supported attributes are given below. Please refer to cuMemRangeGetAttribute for attribute
descriptions and restrictions.
‣ CU_MEM_RANGE_ATTRIBUTE_READ_MOSTLY
‣ CU_MEM_RANGE_ATTRIBUTE_PREFERRED_LOCATION
‣ CU_MEM_RANGE_ATTRIBUTE_ACCESSED_BY
‣ CU_MEM_RANGE_ATTRIBUTE_LAST_PREFETCH_LOCATION
‣ CU_MEM_RANGE_ATTRIBUTE_PREFERRED_LOCATION_TYPE
‣ CU_MEM_RANGE_ATTRIBUTE_PREFERRED_LOCATION_ID
‣ CU_MEM_RANGE_ATTRIBUTE_LAST_PREFETCH_LOCATION_TYPE
‣ CU_MEM_RANGE_ATTRIBUTE_LAST_PREFETCH_LOCATION_ID
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuMemRangeGetAttribute, cuMemAdvise, cuMemPrefetchAsync, cudaMemRangeGetAttributes
CUresult cuPointerGetAttribute (void *data,
CUpointer_attribute attribute, CUdeviceptr ptr)
Returns information about a pointer.
Parameters
data
- Returned pointer attribute value
|
327
Modules
attribute
- Pointer attribute to query
ptr
- Pointer
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE,
CUDA_ERROR_INVALID_DEVICE
Description
The supported attributes are:
‣ CU_POINTER_ATTRIBUTE_CONTEXT:
Returns in *data the CUcontext in which ptr was allocated or registered. The type of data must be
CUcontext *.
If ptr was not allocated by, mapped by, or registered with a CUcontext which uses unified virtual
addressing then CUDA_ERROR_INVALID_VALUE is returned.
‣ CU_POINTER_ATTRIBUTE_MEMORY_TYPE:
Returns in *data the physical memory type of the memory that ptr addresses as a CUmemorytype
enumerated value. The type of data must be unsigned int.
If ptr addresses device memory then *data is set to CU_MEMORYTYPE_DEVICE. The
particular CUdevice on which the memory resides is the CUdevice of the CUcontext returned by the
CU_POINTER_ATTRIBUTE_CONTEXT attribute of ptr.
If ptr addresses host memory then *data is set to CU_MEMORYTYPE_HOST.
If ptr was not allocated by, mapped by, or registered with a CUcontext which uses unified virtual
addressing then CUDA_ERROR_INVALID_VALUE is returned.
If the current CUcontext does not support unified virtual addressing then
CUDA_ERROR_INVALID_CONTEXT is returned.
‣ CU_POINTER_ATTRIBUTE_DEVICE_POINTER:
Returns in *data the device pointer value through which ptr may be accessed by kernels running in
the current CUcontext. The type of data must be CUdeviceptr *.
If there exists no device pointer value through which kernels running in the current CUcontext may
access ptr then CUDA_ERROR_INVALID_VALUE is returned.
If there is no current CUcontext then CUDA_ERROR_INVALID_CONTEXT is returned.
Except in the exceptional disjoint addressing cases discussed below, the value returned in *data will
equal the input value ptr.
|
328
Modules
‣ CU_POINTER_ATTRIBUTE_HOST_POINTER:
Returns in *data the host pointer value through which ptr may be accessed by by the host program.
The type of data must be void **. If there exists no host pointer value through which the host
program may directly access ptr then CUDA_ERROR_INVALID_VALUE is returned.
Except in the exceptional disjoint addressing cases discussed below, the value returned in *data will
equal the input value ptr.
‣ CU_POINTER_ATTRIBUTE_P2P_TOKENS:
Returns in *data two tokens for use with the nv-p2p.h Linux kernel interface. data must be a struct
of type CUDA_POINTER_ATTRIBUTE_P2P_TOKENS.
ptr must be a pointer to memory obtained from :cuMemAlloc(). Note that p2pToken and
vaSpaceToken are only valid for the lifetime of the source allocation. A subsequent allocation at the
same address may return completely different tokens. Querying this attribute has a side effect of setting
the attribute CU_POINTER_ATTRIBUTE_SYNC_MEMOPS for the region of memory that ptr
points to.
‣ CU_POINTER_ATTRIBUTE_SYNC_MEMOPS:
A boolean attribute which when set, ensures that synchronous memory operations initiated on the
region of memory that ptr points to will always synchronize. See further documentation in the section
titled "API synchronization behavior" to learn more about cases when synchronous memory operations
can exhibit asynchronous behavior.
‣ CU_POINTER_ATTRIBUTE_BUFFER_ID:
Returns in *data a buffer ID which is guaranteed to be unique within the process. data must point to
an unsigned long long.
ptr must be a pointer to memory obtained from a CUDA memory allocation API. Every memory
allocation from any of the CUDA memory allocation APIs will have a unique ID over a process
lifetime. Subsequent allocations do not reuse IDs from previous freed allocations. IDs are only unique
within a single process.
‣ CU_POINTER_ATTRIBUTE_IS_MANAGED:
Returns in *data a boolean that indicates whether the pointer points to managed memory or not.
If ptr is not a valid CUDA pointer then CUDA_ERROR_INVALID_VALUE is returned.
‣ CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL:
Returns in *data an integer representing a device ordinal of a device against which the memory was
allocated or registered.
‣ CU_POINTER_ATTRIBUTE_IS_LEGACY_CUDA_IPC_CAPABLE:
Returns in *data a boolean that indicates if this pointer maps to an allocation that is suitable for
cudaIpcGetMemHandle.
|
329
Modules
‣ CU_POINTER_ATTRIBUTE_RANGE_START_ADDR:
Returns in *data the starting address for the allocation referenced by the device pointer ptr. Note
that this is not necessarily the address of the mapped region, but the address of the mappable address
range ptr references (e.g. from cuMemAddressReserve).
‣ CU_POINTER_ATTRIBUTE_RANGE_SIZE:
Returns in *data the size for the allocation referenced by the device pointer ptr. Note that this
is not necessarily the size of the mapped region, but the size of the mappable address range ptr
references (e.g. from cuMemAddressReserve). To retrieve the size of the mapped region, see
cuMemGetAddressRange
‣ CU_POINTER_ATTRIBUTE_MAPPED:
Returns in *data a boolean that indicates if this pointer is in a valid address range that is mapped to a
backing allocation.
‣ CU_POINTER_ATTRIBUTE_ALLOWED_HANDLE_TYPES:
Returns a bitmask of the allowed handle types for an allocation that may be passed to
cuMemExportToShareableHandle.
‣ CU_POINTER_ATTRIBUTE_MEMPOOL_HANDLE:
Returns in *data the handle to the mempool that the allocation was obtained from.
‣ CU_POINTER_ATTRIBUTE_IS_HW_DECOMPRESS_CAPABLE:
Returns in *data a boolean that indicates whether the pointer points to memory that is capable to be
used for hardware accelerated decompression.
Note that for most allocations in the unified virtual address space the host and device pointer for
accessing the allocation will be the same. The exceptions to this are
‣ user memory registered using cuMemHostRegister
‣ host memory allocated using cuMemHostAlloc with the
CU_MEMHOSTALLOC_WRITECOMBINED flag For these types of allocation there will exist
separate, disjoint host and device addresses for accessing the allocation. In particular
‣ The host address will correspond to an invalid unmapped device address (which will result in an
exception if accessed from the device)
‣ The device address will correspond to an invalid unmapped host address
(which will result in an exception if accessed from the host). For these types
of allocations, querying CU_POINTER_ATTRIBUTE_HOST_POINTER and
CU_POINTER_ATTRIBUTE_DEVICE_POINTER may be used to retrieve the host and device
addresses from either address.
Note:
|
330
Modules
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuPointerSetAttribute, cuMemAlloc, cuMemFree, cuMemAllocHost, cuMemFreeHost,
cuMemHostAlloc, cuMemHostRegister, cuMemHostUnregister, cudaPointerGetAttributes
CUresult cuPointerGetAttributes (unsigned int
numAttributes, CUpointer_attribute *attributes, void
**data, CUdeviceptr ptr)
Returns information about a pointer.
Parameters
numAttributes
- Number of attributes to query
attributes
- An array of attributes to query (numAttributes and the number of attributes in this array should
match)
data
- A two-dimensional array containing pointers to memory locations where the result of each
attribute query will be written to.
ptr
- Pointer to query
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_INVALID_CONTEXT,
CUDA_ERROR_INVALID_VALUE, CUDA_ERROR_INVALID_DEVICE
Description
The supported attributes are (refer to cuPointerGetAttribute for attribute descriptions and restrictions):
‣ CU_POINTER_ATTRIBUTE_CONTEXT
‣ CU_POINTER_ATTRIBUTE_MEMORY_TYPE
‣ CU_POINTER_ATTRIBUTE_DEVICE_POINTER
‣ CU_POINTER_ATTRIBUTE_HOST_POINTER
‣ CU_POINTER_ATTRIBUTE_SYNC_MEMOPS
‣ CU_POINTER_ATTRIBUTE_BUFFER_ID
‣ CU_POINTER_ATTRIBUTE_IS_MANAGED
‣ CU_POINTER_ATTRIBUTE_DEVICE_ORDINAL
‣ CU_POINTER_ATTRIBUTE_RANGE_START_ADDR
|
331
Modules
‣ CU_POINTER_ATTRIBUTE_RANGE_SIZE
‣ CU_POINTER_ATTRIBUTE_MAPPED
‣ CU_POINTER_ATTRIBUTE_IS_LEGACY_CUDA_IPC_CAPABLE
‣ CU_POINTER_ATTRIBUTE_ALLOWED_HANDLE_TYPES
‣ CU_POINTER_ATTRIBUTE_MEMPOOL_HANDLE
‣ CU_POINTER_ATTRIBUTE_IS_HW_DECOMPRESS_CAPABLE
Unlike cuPointerGetAttribute, this function will not return an error when the ptr encountered is not a
valid CUDA pointer. Instead, the attributes are assigned default NULL values and CUDA_SUCCESS
is returned.
If ptr was not allocated by, mapped by, or registered with a CUcontext which uses UVA (Unified
Virtual Addressing), CUDA_ERROR_INVALID_CONTEXT is returned.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuPointerGetAttribute, cuPointerSetAttribute, cudaPointerGetAttributes
CUresult cuPointerSetAttribute (const void *value,
CUpointer_attribute attribute, CUdeviceptr ptr)
Set attributes on a previously allocated memory region.
Parameters
value
- Pointer to memory containing the value to be set
attribute
- Pointer attribute to set
ptr
- Pointer to a memory region allocated using CUDA memory allocation APIs
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_VALUE,
CUDA_ERROR_INVALID_DEVICE
Description
The supported attributes are:
|
332
Modules
‣ CU_POINTER_ATTRIBUTE_SYNC_MEMOPS:
A boolean attribute that can either be set (1) or unset (0). When set, the region of memory that ptr
points to is guaranteed to always synchronize memory operations that are synchronous. If there are
some previously initiated synchronous memory operations that are pending when this attribute is set,
the function does not return until those memory operations are complete. See further documentation in
the section titled "API synchronization behavior" to learn more about cases when synchronous memory
operations can exhibit asynchronous behavior. value will be considered as a pointer to an unsigned
integer to which this attribute is to be set.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuPointerGetAttribute, cuPointerGetAttributes, cuMemAlloc, cuMemFree, cuMemAllocHost,
cuMemFreeHost, cuMemHostAlloc, cuMemHostRegister, cuMemHostUnregister
6.18. Stream Management
This section describes the stream management functions of the low-level CUDA driver application
programming interface.
CUresult cuStreamAddCallback (CUstream hStream,
CUstreamCallback callback, void *userData, unsigned int
flags)
Add a callback to a compute stream.
Parameters
hStream
- Stream to add callback to
callback
- The function to call once preceding stream operations are complete
userData
- User specified data to be passed to the callback function
flags
- Reserved for future use, must be 0
|
333
Modules
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_HANDLE,
CUDA_ERROR_NOT_SUPPORTED
Description
Note:
This function is slated for eventual deprecation and removal. If you do not require the callback to execute
in case of a device error, consider using cuLaunchHostFunc. Additionally, this function is not supported
with cuStreamBeginCapture and cuStreamEndCapture, unlike cuLaunchHostFunc.
Adds a callback to be called on the host after all currently enqueued items in the stream have
completed. For each cuStreamAddCallback call, the callback will be executed exactly once. The
callback will block later work in the stream until it is finished.
The callback may be passed CUDA_SUCCESS or an error code. In the event of a device error, all
subsequently executed callbacks will receive an appropriate CUresult.
Callbacks must not make any CUDA API calls. Attempting to use a CUDA API will result in
CUDA_ERROR_NOT_PERMITTED. Callbacks must not perform any synchronization that may
depend on outstanding device work or other callbacks that are not mandated to run earlier. Callbacks
without a mandated order (in independent streams) execute in undefined order and may be serialized.
For the purposes of Unified Memory, callback execution makes a number of guarantees:
‣ The callback stream is considered idle for the duration of the callback. Thus, for example, a
callback may always use memory attached to the callback stream.
‣ The start of execution of a callback has the same effect as synchronizing an event recorded in
the same stream immediately prior to the callback. It thus synchronizes streams which have been
"joined" prior to the callback.
‣ Adding device work to any stream does not have the effect of making the stream active until all
preceding host functions and stream callbacks have executed. Thus, for example, a callback might
use global attached memory even if work has been added to another stream, if the work has been
ordered behind the callback with an event.
‣ Completion of a callback does not cause a stream to become active except as described above. The
callback stream will remain idle if no device work follows the callback, and will remain idle across
consecutive callbacks without device work in between. Thus, for example, stream synchronization
can be done by signaling from a callback at the end of the stream.
Note:
‣ This function uses standard default stream semantics.
|
334
Modules
‣ Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuStreamCreate, cuStreamQuery, cuStreamSynchronize, cuStreamWaitEvent, cuStreamDestroy,
cuMemAllocManaged, cuStreamAttachMemAsync, cuLaunchHostFunc, cudaStreamAddCallback
CUresult cuStreamAttachMemAsync (CUstream hStream,
CUdeviceptr dptr, size_t length, unsigned int flags)
Attach memory to a stream asynchronously.
Parameters
hStream
- Stream in which to enqueue the attach operation
dptr
- Pointer to memory (must be a pointer to managed memory or to a valid host-accessible region of
system-allocated pageable memory)
length
- Length of memory
flags
- Must be one of CUmemAttach_flags
Returns
CUDA_SUCCESS, CUDA_ERROR_DEINITIALIZED, CUDA_ERROR_NOT_INITIALIZED,
CUDA_ERROR_INVALID_CONTEXT, CUDA_ERROR_INVALID_HANDLE,
CUDA_ERROR_NOT_SUPPORTED
Description
Enqueues an operation in hStream to specify stream association of length bytes of memory
starting from dptr. This function is a stream-ordered operation, meaning that it is dependent on,
and will only take effect when, previous work in stream has completed. Any previous association is
automatically replaced.
dptr must point to one of the following types of memories:
‣ managed memory declared using the __managed__ keyword or allocated with
cuMemAllocManaged.
‣ a valid host-accessible region of system-allocated pageable memory. This type of memory may
only be specified if the device associated with the stream reports a non-zero value for the device
attribute CU_DEVICE_ATTRIBUTE_PAGEABLE_MEMORY_ACCESS.
|
335
Modules
For managed allocations, length must be either zero or the entire allocation's size. Both indicate that
the entire allocation's stream association is being changed. Currently, it is not possible to change stream
association for a portion of a managed allocation.
For pageable host allocations, length must be non-zero.
The stream association is specified using flags which must be one of CUmemAttach_flags. If the
CU_MEM_ATTACH_GLOBAL flag is specified, the memory can be accessed by any stream on
any device. If the CU_MEM_ATTACH_HOST flag is specified, the program makes a guarantee
that it won't access the memory on the device from any stream on a device that has a zero value
for the device attribute CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS.
If the CU_MEM_ATTACH_SINGLE flag is specified and hStream
is associated with a device that has a zero value for the device attribute
CU_DEVICE_ATTRIBUTE_CONCURRENT_MANAGED_ACCESS, the program makes a
guarantee that it will only access the memory on the device from hStream. It is illegal to attach singly
to the NULL stream, because the NULL stream is a virtual global stream and not a specific stream. An
error will be returned in this case.
When memory is associated with a single stream, the Unified Memory system will allow CPU access
to this memory region so long as all operations in hStream have completed, regardless of whether
other streams are active. In effect, this constrains exclusive ownership of the managed memory region
by an active GPU to per-stream activity instead of whole-GPU activity.
Accessing memory on the device from streams that are not associated with it will produce undefined
results. No error checking is performed by the Unified Memory system to ensure that kernels launched
into other streams do not access this region.
It is a program's responsibility to order calls to cuStreamAttachMemAsync via events, synchronization
or other means to ensure legal access to memory at all times. Data visibility and coherency will be
changed appropriately for all kernels which follow a stream-association change.
If hStream is destroyed while data is associated with it, the association is removed and the
association reverts to the default visibility of the allocation as specified at cuMemAllocManaged. For
__managed__ variables, the default association is always CU_MEM_ATTACH_GLOBAL. Note that
destroying a stream is an asynchronous operation, and as a result, the change to default association
won't happen until all work in the stream has completed.
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
See also:
cuStreamCreate, cuStreamQuery, cuStreamSynchronize, cuStreamWaitEvent, cuStreamDestroy,
cuMemAllocManaged, cudaStreamAttachMemAsync
|
336

 

 

 

 

 

 

 

Content      ..     3      4      5      6     ..