|
|
API Reference Manual
|
January
2024
Table of Contents
Chapter 1. Difference between the driver and runtime APIs
1
Chapter 2. API synchronization behavior
3
Chapter 3. Stream synchronization behavior
5
Chapter 4. Graph object thread safety
7
Chapter 5. Rules for version mixing
8
Chapter 6. Modules
9
6.2. Device Management [DEPRECATED]
10
cudaDeviceGetSharedMemConfig
10
cudaDeviceSetSharedMemConfig
11
6.3. Thread Management [DEPRECATED]
12
cudaThreadExit
12
cudaThreadGetCacheConfig
13
cudaThreadGetLimit
14
cudaThreadSetCacheConfig
15
cudaThreadSetLimit
16
cudaThreadSynchronize
17
6.4. Error Handling
18
cudaGetErrorName
18
cudaGetErrorString
18
cudaGetLastError
19
cudaPeekAtLastError
20
6.5. Stream Management
21
cudaStreamCallback_t
21
cudaCtxResetPersistingL2Cache
21
cudaStreamAddCallback
21
cudaStreamAttachMemAsync
23
cudaStreamBeginCapture
25
cudaStreamBeginCaptureToGraph
26
cudaStreamCopyAttributes
27
cudaStreamCreate
28
cudaStreamCreateWithFlags
29
cudaStreamCreateWithPriority
30
cudaStreamDestroy
31
cudaStreamEndCapture
32
|
ii
cudaStreamGetAttribute
32
cudaStreamGetCaptureInfo
33
cudaStreamGetCaptureInfo_v3
35
cudaStreamGetDevice
36
cudaStreamGetFlags
37
cudaStreamGetId
38
cudaStreamGetPriority
39
cudaStreamIsCapturing
40
cudaStreamQuery
41
cudaStreamSetAttribute
41
cudaStreamSynchronize
42
cudaStreamUpdateCaptureDependencies
43
cudaStreamUpdateCaptureDependencies_v2
44
cudaStreamWaitEvent
45
cudaThreadExchangeStreamCaptureMode
46
6.6. Event Management
47
cudaEventCreate
47
cudaEventCreateWithFlags
48
cudaEventDestroy
49
cudaEventElapsedTime
50
cudaEventElapsedTime_v2
51
cudaEventQuery
52
cudaEventRecord
53
cudaEventRecordWithFlags
54
cudaEventSynchronize
55
6.7. External Resource Interoperability
56
cudaDestroyExternalMemory
56
cudaDestroyExternalSemaphore
57
cudaExternalMemoryGetMappedBuffer
58
cudaExternalMemoryGetMappedMipmappedArray
59
cudaImportExternalMemory
60
cudaImportExternalSemaphore
63
cudaSignalExternalSemaphoresAsync
66
cudaWaitExternalSemaphoresAsync
68
6.8. Execution Control
70
cudaFuncGetAttributes
70
cudaFuncGetName
71
cudaFuncGetParamInfo
72
|
iii
cudaFuncSetAttribute
73
cudaFuncSetCacheConfig
75
cudaGetParameterBuffer
76
cudaGridDependencySynchronize
77
cudaLaunchCooperativeKernel
77
cudaLaunchCooperativeKernelMultiDevice
79
cudaLaunchDevice
81
cudaLaunchHostFunc
82
cudaLaunchKernel
84
cudaLaunchKernelExC
85
cudaSetDoubleForDevice
87
cudaSetDoubleForHost
87
cudaTriggerProgrammaticLaunchCompletion
88
6.9. Execution Control [DEPRECATED]
88
cudaFuncSetSharedMemConfig
89
6.10. Occupancy
90
cudaOccupancyAvailableDynamicSMemPerBlock
90
cudaOccupancyMaxActiveBlocksPerMultiprocessor
92
cudaOccupancyMaxActiveBlocksPerMultiprocessorWithFlags
93
cudaOccupancyMaxActiveClusters
94
cudaOccupancyMaxPotentialClusterSize
96
6.11. Memory Management
97
cudaArrayGetInfo
97
cudaArrayGetMemoryRequirements
98
cudaArrayGetPlane
99
cudaArrayGetSparseProperties
100
cudaFree
100
cudaFreeArray
101
cudaFreeHost
102
cudaFreeMipmappedArray
103
cudaGetMipmappedArrayLevel
104
cudaGetSymbolAddress
105
cudaGetSymbolSize
106
cudaHostAlloc
107
cudaHostGetDevicePointer
108
cudaHostGetFlags
109
cudaHostRegister
110
cudaHostUnregister
112
|
iv
cudaMalloc
113
cudaMalloc3D
114
cudaMalloc3DArray
115
cudaMallocArray
117
cudaMallocHost
119
cudaMallocManaged
120
cudaMallocMipmappedArray
122
cudaMallocPitch
125
cudaMemAdvise
126
cudaMemAdvise_v2
129
cudaMemcpy
133
cudaMemcpy2D
134
cudaMemcpy2DArrayToArray
135
cudaMemcpy2DAsync
137
cudaMemcpy2DFromArray
139
cudaMemcpy2DFromArrayAsync
140
cudaMemcpy2DToArray
142
cudaMemcpy2DToArrayAsync
143
cudaMemcpy3D
145
cudaMemcpy3DAsync
147
cudaMemcpy3DBatchAsync
149
cudaMemcpy3DPeer
151
cudaMemcpy3DPeerAsync
152
cudaMemcpyAsync
153
cudaMemcpyBatchAsync
155
cudaMemcpyFromSymbol
157
cudaMemcpyFromSymbolAsync
158
cudaMemcpyPeer
159
cudaMemcpyPeerAsync
160
cudaMemcpyToSymbol
162
cudaMemcpyToSymbolAsync
163
cudaMemGetInfo
164
cudaMemPrefetchAsync
165
cudaMemPrefetchAsync_v2
167
cudaMemRangeGetAttribute
169
cudaMemRangeGetAttributes
172
cudaMemset
173
cudaMemset2D
174
|
v
cudaMemset2DAsync
175
cudaMemset3D
176
cudaMemset3DAsync
177
cudaMemsetAsync
179
cudaMipmappedArrayGetMemoryRequirements
180
cudaMipmappedArrayGetSparseProperties
181
make_cudaExtent
181
make_cudaPitchedPtr
182
make_cudaPos
183
6.12. Memory Management [DEPRECATED]
183
cudaMemcpyArrayToArray
183
cudaMemcpyFromArray
185
cudaMemcpyFromArrayAsync
186
cudaMemcpyToArray
187
cudaMemcpyToArrayAsync
189
6.13. Stream Ordered Memory Allocator
190
cudaFreeAsync
191
cudaMallocAsync
192
cudaMallocFromPoolAsync
193
cudaMemPoolCreate
194
cudaMemPoolDestroy
195
cudaMemPoolExportPointer
196
cudaMemPoolExportToShareableHandle
196
cudaMemPoolGetAccess
197
cudaMemPoolGetAttribute
198
cudaMemPoolImportFromShareableHandle
199
cudaMemPoolImportPointer
200
cudaMemPoolSetAccess
200
cudaMemPoolSetAttribute
201
cudaMemPoolTrimTo
202
6.14. Unified Addressing
203
cudaPointerGetAttributes
204
6.15. Peer Device Memory Access
206
cudaDeviceCanAccessPeer
206
cudaDeviceDisablePeerAccess
207
cudaDeviceEnablePeerAccess
207
6.16. OpenGL Interoperability
208
cudaGLDeviceList
208
|
vi
cudaGLGetDevices
209
cudaGraphicsGLRegisterBuffer
210
cudaGraphicsGLRegisterImage
211
cudaWGLGetDevice
212
6.17. OpenGL Interoperability [DEPRECATED]
213
cudaGLMapFlags
213
cudaGLMapBufferObject
213
cudaGLMapBufferObjectAsync
214
cudaGLRegisterBufferObject
215
cudaGLSetBufferObjectMapFlags
216
cudaGLSetGLDevice
217
cudaGLUnmapBufferObject
217
cudaGLUnmapBufferObjectAsync
218
cudaGLUnregisterBufferObject
219
6.18. Direct3D 9 Interoperability
219
cudaD3D9DeviceList
219
cudaD3D9GetDevice
220
cudaD3D9GetDevices
221
cudaD3D9GetDirect3DDevice
222
cudaD3D9SetDirect3DDevice
222
cudaGraphicsD3D9RegisterResource
223
6.19. Direct3D 9 Interoperability [DEPRECATED]
225
cudaD3D9MapFlags
225
cudaD3D9RegisterFlags
226
cudaD3D9MapResources
226
cudaD3D9RegisterResource
227
cudaD3D9ResourceGetMappedArray
229
cudaD3D9ResourceGetMappedPitch
230
cudaD3D9ResourceGetMappedPointer
231
cudaD3D9ResourceGetMappedSize
232
cudaD3D9ResourceGetSurfaceDimensions
233
cudaD3D9ResourceSetMapFlags
234
cudaD3D9UnmapResources
235
cudaD3D9UnregisterResource
236
6.20. Direct3D 10 Interoperability
236
cudaD3D10DeviceList
237
cudaD3D10GetDevice
237
cudaD3D10GetDevices
238
|
vii
cudaGraphicsD3D10RegisterResource
239
6.21. Direct3D 10 Interoperability [DEPRECATED]
241
cudaD3D10MapFlags
241
cudaD3D10RegisterFlags
241
cudaD3D10GetDirect3DDevice
241
cudaD3D10MapResources
242
cudaD3D10RegisterResource
243
cudaD3D10ResourceGetMappedArray
245
cudaD3D10ResourceGetMappedPitch
246
cudaD3D10ResourceGetMappedPointer
247
cudaD3D10ResourceGetMappedSize
248
cudaD3D10ResourceGetSurfaceDimensions
249
cudaD3D10ResourceSetMapFlags
250
cudaD3D10SetDirect3DDevice
251
cudaD3D10UnmapResources
252
cudaD3D10UnregisterResource
252
6.22. Direct3D 11 Interoperability
253
cudaD3D11DeviceList
253
cudaD3D11GetDevice
254
cudaD3D11GetDevices
254
cudaGraphicsD3D11RegisterResource
256
6.23. Direct3D 11 Interoperability [DEPRECATED]
258
cudaD3D11GetDirect3DDevice
258
cudaD3D11SetDirect3DDevice
258
6.24. VDPAU Interoperability
259
cudaGraphicsVDPAURegisterOutputSurface
259
cudaGraphicsVDPAURegisterVideoSurface
260
cudaVDPAUGetDevice
261
cudaVDPAUSetVDPAUDevice
262
6.25. EGL Interoperability
263
cudaEGLStreamConsumerAcquireFrame
263
cudaEGLStreamConsumerConnect
264
cudaEGLStreamConsumerConnectWithFlags
265
cudaEGLStreamConsumerDisconnect
265
cudaEGLStreamConsumerReleaseFrame
266
cudaEGLStreamProducerConnect
267
cudaEGLStreamProducerDisconnect
267
cudaEGLStreamProducerPresentFrame
268
|
viii
cudaEGLStreamProducerReturnFrame
269
cudaEventCreateFromEGLSync
269
cudaGraphicsEGLRegisterImage
270
cudaGraphicsResourceGetMappedEglFrame
271
6.26. Graphics Interoperability
272
cudaGraphicsMapResources
273
cudaGraphicsResourceGetMappedMipmappedArray
274
cudaGraphicsResourceGetMappedPointer
275
cudaGraphicsResourceSetMapFlags
276
cudaGraphicsSubResourceGetMappedArray
277
cudaGraphicsUnmapResources
278
cudaGraphicsUnregisterResource
279
6.27. Texture Object Management
279
cudaCreateChannelDesc
280
cudaCreateTextureObject
281
cudaDestroyTextureObject
286
cudaGetChannelDesc
286
cudaGetTextureObjectResourceDesc
287
cudaGetTextureObjectResourceViewDesc
288
cudaGetTextureObjectTextureDesc
289
6.28. Surface Object Management
289
cudaCreateSurfaceObject
290
cudaDestroySurfaceObject
291
cudaGetSurfaceObjectResourceDesc
291
6.29. Version Management
292
cudaDriverGetVersion
292
cudaRuntimeGetVersion
293
6.30. Graph Management
294
cudaDeviceGetGraphMemAttribute
294
cudaDeviceGraphMemTrim
295
cudaDeviceSetGraphMemAttribute
296
cudaGetCurrentGraphExec
297
cudaGraphAddChildGraphNode
297
cudaGraphAddDependencies
298
cudaGraphAddDependencies_v2
299
cudaGraphAddEmptyNode
300
cudaGraphAddEventRecordNode
302
cudaGraphAddEventWaitNode
303
|
ix
cudaGraphAddExternalSemaphoresSignalNode
304
cudaGraphAddExternalSemaphoresWaitNode
305
cudaGraphAddHostNode
307
cudaGraphAddKernelNode
308
cudaGraphAddMemAllocNode
310
cudaGraphAddMemcpyNode
312
cudaGraphAddMemcpyNode1D
313
cudaGraphAddMemcpyNodeFromSymbol
315
cudaGraphAddMemcpyNodeToSymbol
317
cudaGraphAddMemFreeNode
318
cudaGraphAddMemsetNode
320
cudaGraphAddNode
321
cudaGraphAddNode_v2
322
cudaGraphChildGraphNodeGetGraph
323
cudaGraphClone
324
cudaGraphConditionalHandleCreate
325
cudaGraphCreate
326
cudaGraphDebugDotPrint
327
cudaGraphDestroy
327
cudaGraphDestroyNode
328
cudaGraphEventRecordNodeGetEvent
329
cudaGraphEventRecordNodeSetEvent
330
cudaGraphEventWaitNodeGetEvent
330
cudaGraphEventWaitNodeSetEvent
331
cudaGraphExecChildGraphNodeSetParams
332
cudaGraphExecDestroy
333
cudaGraphExecEventRecordNodeSetEvent
334
cudaGraphExecEventWaitNodeSetEvent
335
cudaGraphExecExternalSemaphoresSignalNodeSetParams
336
cudaGraphExecExternalSemaphoresWaitNodeSetParams
338
cudaGraphExecGetFlags
339
cudaGraphExecHostNodeSetParams
340
cudaGraphExecKernelNodeSetParams
341
cudaGraphExecMemcpyNodeSetParams
342
cudaGraphExecMemcpyNodeSetParams1D
344
cudaGraphExecMemcpyNodeSetParamsFromSymbol
345
cudaGraphExecMemcpyNodeSetParamsToSymbol
347
cudaGraphExecMemsetNodeSetParams
348
|
x
cudaGraphExecNodeSetParams
350
cudaGraphExecUpdate
351
cudaGraphExternalSemaphoresSignalNodeGetParams
354
cudaGraphExternalSemaphoresSignalNodeSetParams
355
cudaGraphExternalSemaphoresWaitNodeGetParams
356
cudaGraphExternalSemaphoresWaitNodeSetParams
357
cudaGraphGetEdges
358
cudaGraphGetEdges_v2
359
cudaGraphGetNodes
360
cudaGraphGetRootNodes
361
cudaGraphHostNodeGetParams
362
cudaGraphHostNodeSetParams
363
cudaGraphInstantiate
364
cudaGraphInstantiateWithFlags
366
cudaGraphInstantiateWithParams
368
cudaGraphKernelNodeCopyAttributes
370
cudaGraphKernelNodeGetAttribute
371
cudaGraphKernelNodeGetParams
371
cudaGraphKernelNodeSetAttribute
372
cudaGraphKernelNodeSetEnabled
373
cudaGraphKernelNodeSetGridDim
374
cudaGraphKernelNodeSetParam
374
cudaGraphKernelNodeSetParam
375
cudaGraphKernelNodeSetParams
376
cudaGraphKernelNodeUpdatesApply
377
cudaGraphLaunch
378
cudaGraphMemAllocNodeGetParams
379
cudaGraphMemcpyNodeGetParams
380
cudaGraphMemcpyNodeSetParams
381
cudaGraphMemcpyNodeSetParams1D
382
cudaGraphMemcpyNodeSetParamsFromSymbol
383
cudaGraphMemcpyNodeSetParamsToSymbol
384
cudaGraphMemFreeNodeGetParams
385
cudaGraphMemsetNodeGetParams
386
cudaGraphMemsetNodeSetParams
387
cudaGraphNodeFindInClone
388
cudaGraphNodeGetDependencies
389
cudaGraphNodeGetDependencies_v2
390
|
xi
cudaGraphNodeGetDependentNodes
391
cudaGraphNodeGetDependentNodes_v2
392
cudaGraphNodeGetEnabled
393
cudaGraphNodeGetType
394
cudaGraphNodeSetEnabled
395
cudaGraphNodeSetParams
396
cudaGraphReleaseUserObject
397
cudaGraphRemoveDependencies
397
cudaGraphRemoveDependencies_v2
398
cudaGraphRetainUserObject
399
cudaGraphSetConditional
400
cudaGraphUpload
400
cudaUserObjectCreate
401
cudaUserObjectRelease
402
cudaUserObjectRetain
403
6.31. Driver Entry Point Access
403
cudaGetDriverEntryPoint
403
cudaGetDriverEntryPointByVersion
405
6.32. Library Management
407
cudaKernelSetAttributeForDevice
407
cudaLibraryEnumerateKernels
409
cudaLibraryGetGlobal
410
cudaLibraryGetKernel
410
cudaLibraryGetKernelCount
411
cudaLibraryGetManaged
412
cudaLibraryGetUnifiedFunction
413
cudaLibraryLoadData
413
cudaLibraryLoadFromFile
415
cudaLibraryUnload
416
6.33. C++ API Routines
417
__cudaOccupancyB2DHelper
417
cudaCreateChannelDesc
417
cudaEventCreate
418
cudaFuncGetAttributes
419
cudaFuncGetName
420
cudaFuncSetAttribute
421
cudaFuncSetCacheConfig
423
cudaGetKernel
424
|
xii
cudaGetSymbolAddress
424
cudaGetSymbolSize
425
cudaGraphAddMemcpyNodeFromSymbol
426
cudaGraphAddMemcpyNodeToSymbol
428
cudaGraphExecMemcpyNodeSetParamsFromSymbol
429
cudaGraphExecMemcpyNodeSetParamsToSymbol
431
cudaGraphInstantiate
432
cudaGraphMemcpyNodeSetParamsFromSymbol
433
cudaGraphMemcpyNodeSetParamsToSymbol
435
cudaLaunchCooperativeKernel
436
cudaLaunchKernel
438
cudaLaunchKernelEx
439
cudaLaunchKernelEx
440
cudaLibraryGetGlobal
442
cudaLibraryGetManaged
443
cudaLibraryGetUnifiedFunction
443
cudaMallocAsync
444
cudaMallocHost
444
cudaMallocManaged
446
cudaMemAdvise
448
cudaMemcpyBatchAsync
449
cudaMemcpyBatchAsync
449
cudaMemcpyFromSymbol
450
cudaMemcpyFromSymbolAsync
451
cudaMemcpyToSymbol
452
cudaMemcpyToSymbolAsync
453
cudaOccupancyAvailableDynamicSMemPerBlock
455
cudaOccupancyMaxActiveBlocksPerMultiprocessor
456
cudaOccupancyMaxActiveBlocksPerMultiprocessorWithFlags
457
cudaOccupancyMaxActiveClusters
459
cudaOccupancyMaxPotentialBlockSize
460
cudaOccupancyMaxPotentialBlockSizeVariableSMem
461
cudaOccupancyMaxPotentialBlockSizeVariableSMemWithFlags
463
cudaOccupancyMaxPotentialBlockSizeWithFlags
464
cudaOccupancyMaxPotentialClusterSize
466
cudaStreamAttachMemAsync
467
6.34. Interactions with the CUDA Driver API
469
cudaGetFuncBySymbol
471
|
xiii
cudaGetKernel
472
6.35. Profiler Control
472
cudaProfilerStart
473
cudaProfilerStop
473
6.36. Data types used by CUDA Runtime
474
cudaAccessPolicyWindow
475
cudaArrayMemoryRequirements
475
cudaArraySparseProperties
475
cudaAsyncNotificationInfo_t
475
cudaChannelFormatDesc
475
cudaChildGraphNodeParams
475
cudaConditionalNodeParams
475
cudaDeviceProp
475
cudaEglFrame
475
cudaEglPlaneDesc
475
cudaEventRecordNodeParams
475
cudaEventWaitNodeParams
475
cudaExtent
475
cudaExternalMemoryBufferDesc
475
cudaExternalMemoryHandleDesc
475
cudaExternalMemoryMipmappedArrayDesc
475
cudaExternalSemaphoreHandleDesc
475
cudaExternalSemaphoreSignalNodeParams
476
cudaExternalSemaphoreSignalNodeParamsV2
476
cudaExternalSemaphoreSignalParams
476
cudaExternalSemaphoreSignalParams_v1
476
cudaExternalSemaphoreWaitNodeParams
476
cudaExternalSemaphoreWaitNodeParamsV2
476
cudaExternalSemaphoreWaitParams
476
cudaExternalSemaphoreWaitParams_v1
476
cudaFuncAttributes
476
cudaGraphEdgeData
476
cudaGraphExecUpdateResultInfo
476
cudaGraphInstantiateParams
476
cudaGraphKernelNodeUpdate
476
cudaGraphNodeParams
476
cudaHostNodeParams
476
cudaHostNodeParamsV2
476
|
xiv
cudaIpcEventHandle_t
476
cudaIpcMemHandle_t
477
cudaKernelNodeParams
477
cudaKernelNodeParamsV2
477
cudaLaunchAttribute
477
cudaLaunchAttributeValue
477
cudaLaunchConfig_t
477
cudaLaunchMemSyncDomainMap
477
cudaLaunchParams
477
cudaMemAccessDesc
477
cudaMemAllocNodeParams
477
cudaMemAllocNodeParamsV2
477
cudaMemcpy3DOperand
477
cudaMemcpy3DParms
477
cudaMemcpy3DPeerParms
477
cudaMemcpyAttributes
477
cudaMemcpyNodeParams
477
cudaMemFreeNodeParams
477
cudaMemLocation
478
cudaMemPoolProps
478
cudaMemPoolPtrExportData
478
cudaMemsetParams
478
cudaMemsetParamsV2
478
cudaOffset3D
478
cudaPitchedPtr
478
cudaPointerAttributes
478
cudaPos
478
cudaResourceDesc
478
cudaResourceViewDesc
478
cudaTextureDesc
478
CUuuid_st
478
cudaAccessProperty
478
cudaAsyncNotificationType
479
cudaCGScope
479
cudaChannelFormatKind
479
cudaClusterSchedulingPolicy
481
cudaComputeMode
481
cudaDeviceAttr
481
|
xv
cudaDeviceNumaConfig
488
cudaDeviceP2PAttr
488
cudaDriverEntryPointQueryResult
488
cudaEglColorFormat
489
cudaEglFrameType
495
cudaEglResourceLocationFlags
496
cudaError
496
cudaExternalMemoryHandleType
506
cudaExternalSemaphoreHandleType
507
cudaFlushGPUDirectRDMAWritesOptions
508
cudaFlushGPUDirectRDMAWritesScope
508
cudaFlushGPUDirectRDMAWritesTarget
508
cudaFuncAttribute
508
cudaFuncCache
509
cudaGetDriverEntryPointFlags
509
cudaGPUDirectRDMAWritesOrdering
510
cudaGraphChildGraphNodeOwnership
510
cudaGraphConditionalNodeType
510
cudaGraphDebugDotFlags
511
cudaGraphDependencyType
511
cudaGraphExecUpdateResult
512
cudaGraphicsCubeFace
512
cudaGraphicsMapFlags
513
cudaGraphicsRegisterFlags
513
cudaGraphInstantiateFlags
513
cudaGraphInstantiateResult
514
cudaGraphKernelNodeField
514
cudaGraphMemAttributeType
514
cudaGraphNodeType
515
cudaJit_CacheMode
516
cudaJit_Fallback
516
cudaJitOption
516
cudaLaunchAttributeID
518
cudaLaunchMemSyncDomain
520
cudaLibraryOption
521
cudaLimit
521
cudaMemAccessFlags
522
cudaMemAllocationHandleType
522
|
xvi
cudaMemAllocationType
522
cudaMemcpy3DOperandType
523
cudaMemcpyFlags
523
cudaMemcpyKind
523
cudaMemLocationType
524
cudaMemoryAdvise
524
cudaMemoryType
524
cudaMemPoolAttr
525
cudaMemRangeAttribute
525
cudaResourceType
526
cudaResourceViewFormat
526
cudaSharedCarveout
528
cudaSharedMemConfig
528
cudaStreamCaptureMode
529
cudaStreamCaptureStatus
529
cudaStreamUpdateCaptureDependenciesFlags
529
cudaSurfaceBoundaryMode
529
cudaSurfaceFormatMode
530
cudaTextureAddressMode
530
cudaTextureFilterMode
530
cudaTextureReadMode
530
cudaUserObjectFlags
531
cudaUserObjectRetainFlags
531
cudaArray_const_t
531
cudaArray_t
531
cudaAsyncCallbackHandle_t
531
cudaEglStreamConnection
531
cudaError_t
531
cudaEvent_t
532
cudaExternalMemory_t
532
cudaExternalSemaphore_t
532
cudaFunction_t
532
cudaGraph_t
532
cudaGraphConditionalHandle
532
cudaGraphDeviceNode_t
532
cudaGraphExec_t
532
cudaGraphicsResource_t
532
cudaGraphNode_t
532
|
xvii
cudaHostFn_t
533
cudaKernel_t
533
cudaLibrary_t
533
cudaMemPool_t
533
cudaMipmappedArray_const_t
533
cudaMipmappedArray_t
533
cudaStream_t
533
cudaSurfaceObject_t
533
cudaTextureObject_t
533
cudaUserObject_t
533
CUDA_EGL_MAX_PLANES
534
CUDA_IPC_HANDLE_SIZE
534
cudaArrayColorAttachment
534
cudaArrayCubemap
534
cudaArrayDefault
534
cudaArrayDeferredMapping
534
cudaArrayLayered
534
cudaArraySparse
534
cudaArraySparsePropertiesSingleMipTail
534
cudaArraySurfaceLoadStore
534
cudaArrayTextureGather
535
cudaCooperativeLaunchMultiDeviceNoPostSync
535
cudaCooperativeLaunchMultiDeviceNoPreSync
535
cudaCpuDeviceId
535
cudaDeviceBlockingSync
535
cudaDeviceLmemResizeToMax
535
cudaDeviceMapHost
535
cudaDeviceMask
535
cudaDeviceScheduleAuto
536
cudaDeviceScheduleBlockingSync
536
cudaDeviceScheduleMask
536
cudaDeviceScheduleSpin
536
cudaDeviceScheduleYield
536
cudaDeviceSyncMemops
536
cudaEventBlockingSync
536
cudaEventDefault
536
cudaEventDisableTiming
536
cudaEventInterprocess
536
|
xviii
cudaEventRecordDefault
536
cudaEventRecordExternal
537
cudaEventWaitDefault
537
cudaEventWaitExternal
537
cudaExternalMemoryDedicated
537
cudaExternalSemaphoreSignalSkipNvSciBufMemSync
537
cudaExternalSemaphoreWaitSkipNvSciBufMemSync
537
cudaGraphKernelNodePortDefault
537
cudaGraphKernelNodePortLaunchCompletion
538
cudaGraphKernelNodePortProgrammatic
538
cudaHostAllocDefault
538
cudaHostAllocMapped
538
cudaHostAllocPortable
538
cudaHostAllocWriteCombined
538
cudaHostRegisterDefault
538
cudaHostRegisterIoMemory
538
cudaHostRegisterMapped
538
cudaHostRegisterPortable
538
cudaHostRegisterReadOnly
539
cudaInitDeviceFlagsAreValid
539
cudaInvalidDeviceId
539
cudaIpcMemLazyEnablePeerAccess
539
cudaMemAttachGlobal
539
cudaMemAttachHost
539
cudaMemAttachSingle
539
cudaMemPoolCreateUsageHwDecompress
539
cudaNvSciSyncAttrSignal
539
cudaNvSciSyncAttrWait
539
cudaOccupancyDefault
540
cudaOccupancyDisableCachingOverride
540
cudaPeerAccessDefault
540
cudaStreamDefault
540
cudaStreamLegacy
540
cudaStreamNonBlocking
540
cudaStreamPerThread
540
6.1. Difference between the driver and runtime APIs
540
Chapter 7. Data Structures
542
__cudaOccupancyB2DHelper
543
|
xix
cudaAccessPolicyWindow
544
base_ptr
544
hitProp
544
hitRatio
544
missProp
544
num_bytes
544
cudaArrayMemoryRequirements
544
alignment
545
size
545
cudaArraySparseProperties
545
depth
545
flags
545
height
545
miptailFirstLevel
545
miptailSize
545
width
545
cudaAsyncNotificationInfo_t
546
bytesOverBudget
546
info
546
overBudget
546
type
546
cudaChannelFormatDesc
546
f
546
w
546
x
547
y
547
z
547
cudaChildGraphNodeParams
547
graph
547
ownership
547
cudaConditionalNodeParams
547
handle
547
phGraph_out
548
size
548
type
548
cudaDeviceProp
548
accessPolicyMaxWindowSize
548
asyncEngineCount
548
|
xx
canMapHostMemory
549
canUseHostPointerForRegisteredMem
549
clockRate
549
clusterLaunch
549
computeMode
549
computePreemptionSupported
549
concurrentKernels
549
concurrentManagedAccess
549
cooperativeLaunch
549
cooperativeMultiDeviceLaunch
549
deferredMappingCudaArraySupported
549
deviceOverlap
550
directManagedMemAccessFromHost
550
ECCEnabled
550
globalL1CacheSupported
550
gpuDirectRDMAFlushWritesOptions
550
gpuDirectRDMASupported
550
gpuDirectRDMAWritesOrdering
550
hostNativeAtomicSupported
550
hostRegisterReadOnlySupported
550
hostRegisterSupported
550
integrated
551
ipcEventSupported
551
isMultiGpuBoard
551
kernelExecTimeoutEnabled
551
l2CacheSize
551
localL1CacheSupported
551
luid
551
luidDeviceNodeMask
551
major
551
managedMemory
551
maxBlocksPerMultiProcessor
551
maxGridSize
552
maxSurface1D
552
maxSurface1DLayered
552
maxSurface2D
552
maxSurface2DLayered
552
maxSurface3D
552
|
xxi
maxSurfaceCubemap
552
maxSurfaceCubemapLayered
552
maxTexture1D
552
maxTexture1DLayered
552
maxTexture1DLinear
553
maxTexture1DMipmap
553
maxTexture2D
553
maxTexture2DGather
553
maxTexture2DLayered
553
maxTexture2DLinear
553
maxTexture2DMipmap
553
maxTexture3D
553
maxTexture3DAlt
553
maxTextureCubemap
553
maxTextureCubemapLayered
554
maxThreadsDim
554
maxThreadsPerBlock
554
maxThreadsPerMultiProcessor
554
memoryBusWidth
554
memoryClockRate
554
memoryPoolsSupported
554
memoryPoolSupportedHandleTypes
554
memPitch
554
minor
554
multiGpuBoardGroupID
555
multiProcessorCount
555
name
555
pageableMemoryAccess
555
pageableMemoryAccessUsesHostPageTables
555
pciBusID
555
pciDeviceID
555
pciDomainID
555
persistingL2CacheMaxSize
555
regsPerBlock
555
regsPerMultiprocessor
556
reserved
556
reservedSharedMemPerBlock
556
sharedMemPerBlock
556
|
xxii
sharedMemPerBlockOptin
556
sharedMemPerMultiprocessor
556
singleToDoublePrecisionPerfRatio
556
sparseCudaArraySupported
556
streamPrioritiesSupported
556
surfaceAlignment
556
tccDriver
557
textureAlignment
557
texturePitchAlignment
557
timelineSemaphoreInteropSupported
557
totalConstMem
557
totalGlobalMem
557
unifiedAddressing
557
unifiedFunctionPointers
557
uuid
557
warpSize
557
cudaEglFrame
558
eglColorFormat
558
frameType
558
pArray
558
planeCount
558
planeDesc
558
pPitch
558
cudaEglPlaneDesc
558
channelDesc
559
depth
559
height
559
numChannels
559
pitch
559
reserved
559
width
559
cudaEventRecordNodeParams
559
event
559
cudaEventWaitNodeParams
560
event
560
cudaExtent
560
depth
560
height
560
|
xxiii
width
560
cudaExternalMemoryBufferDesc
560
flags
560
offset
561
size
561
cudaExternalMemoryHandleDesc
561
fd
561
flags
561
handle
561
name
561
nvSciBufObject
561
size
561
type
562
win32
562
cudaExternalMemoryMipmappedArrayDesc
562
extent
562
flags
562
formatDesc
562
numLevels
563
offset
563
cudaExternalSemaphoreHandleDesc
563
fd
563
flags
563
handle
563
name
563
nvSciSyncObj
563
type
564
win32
564
cudaExternalSemaphoreSignalNodeParams
564
extSemArray
564
numExtSems
564
paramsArray
565
cudaExternalSemaphoreSignalNodeParamsV2
565
extSemArray
565
numExtSems
565
paramsArray
565
cudaExternalSemaphoreSignalParams
565
fence
565
|
xxiv
fence
566
flags
566
keyedMutex
566
value
566
cudaExternalSemaphoreSignalParams_v1
566
fence
566
fence
566
flags
567
keyedMutex
567
value
567
cudaExternalSemaphoreWaitNodeParams
567
extSemArray
567
numExtSems
567
paramsArray
568
cudaExternalSemaphoreWaitNodeParamsV2
568
extSemArray
568
numExtSems
568
paramsArray
568
cudaExternalSemaphoreWaitParams
568
fence
568
fence
569
flags
569
key
569
keyedMutex
569
timeoutMs
569
value
569
cudaExternalSemaphoreWaitParams_v1
569
fence
570
fence
570
flags
570
key
570
keyedMutex
570
timeoutMs
570
value
570
cudaFuncAttributes
571
binaryVersion
571
cacheModeCA
571
clusterDimMustBeSet
571
|
xxv
clusterSchedulingPolicyPreference
571
constSizeBytes
571
localSizeBytes
571
maxDynamicSharedSizeBytes
571
maxThreadsPerBlock
571
nonPortableClusterSizeAllowed
572
numRegs
572
preferredShmemCarveout
572
ptxVersion
572
requiredClusterWidth
572
reserved
572
sharedSizeBytes
573
cudaGraphEdgeData
573
from_port
573
reserved
573
to_port
573
type
573
cudaGraphExecUpdateResultInfo
574
errorFromNode
574
errorNode
574
result
574
cudaGraphInstantiateParams
574
errNode_out
574
flags
574
result_out
575
uploadStream
575
cudaGraphKernelNodeUpdate
575
field
575
gridDim
575
isEnabled
575
node
575
offset
575
param
576
pValue
576
size
576
updateData
576
cudaGraphNodeParams
576
alloc
576
|
xxvi
conditional
576
eventRecord
576
eventWait
577
extSemSignal
577
extSemWait
577
free
577
graph
577
host
577
kernel
577
memcpy
577
memset
578
reserved0
578
reserved1
578
reserved2
578
type
578
cudaHostNodeParams
578
fn
578
userData
578
cudaHostNodeParamsV2
578
fn
579
userData
579
cudaIpcEventHandle_t
579
cudaIpcMemHandle_t
579
cudaKernelNodeParams
579
blockDim
579
extra
579
func
579
gridDim
579
kernelParams
579
sharedMemBytes
580
cudaKernelNodeParamsV2
580
blockDim
580
extra
580
func
580
gridDim
580
kernelParams
580
sharedMemBytes
580
cudaLaunchAttribute
580
|
xxvii
id
580
val
581
cudaLaunchAttributeValue
581
accessPolicyWindow
581
clusterDim
581
clusterSchedulingPolicyPreference
581
cooperative
581
deviceUpdatableKernelNode
581
launchCompletionEvent
582
memSyncDomain
582
memSyncDomainMap
582
preferredClusterDim
582
priority
582
programmaticEvent
583
programmaticStreamSerializationAllowed
583
sharedMemCarveout
583
syncPolicy
583
cudaLaunchConfig_t
583
attrs
583
blockDim
583
dynamicSmemBytes
584
gridDim
584
numAttrs
584
stream
584
cudaLaunchMemSyncDomainMap
584
default_
584
remote
584
cudaLaunchParams
584
args
585
blockDim
585
func
585
gridDim
585
sharedMem
585
stream
585
cudaMemAccessDesc
585
flags
585
location
585
cudaMemAllocNodeParams
585
|
xxviii
accessDescCount
586
accessDescs
586
bytesize
586
dptr
586
poolProps
586
cudaMemAllocNodeParamsV2
586
accessDescCount
586
accessDescs
586
bytesize
587
dptr
587
poolProps
587
cudaMemcpy3DOperand
587
array
587
layerHeight
587
locHint
587
ptr
588
rowLength
588
cudaMemcpy3DParms
588
dstArray
588
dstPos
588
dstPtr
588
extent
588
kind
588
srcArray
588
srcPos
588
srcPtr
589
cudaMemcpy3DPeerParms
589
dstArray
589
dstDevice
589
dstPos
589
dstPtr
589
extent
589
srcArray
589
srcDevice
589
srcPos
589
srcPtr
590
cudaMemcpyAttributes
590
dstLocHint
590
|
xxix
flags
590
srcAccessOrder
590
srcLocHint
590
cudaMemcpyNodeParams
590
copyParams
590
flags
591
reserved
591
cudaMemFreeNodeParams
591
dptr
591
cudaMemLocation
591
id
591
type
591
cudaMemPoolProps
591
allocType
592
handleTypes
592
location
592
maxSize
592
reserved
592
usage
592
win32SecurityAttributes
592
cudaMemPoolPtrExportData
592
cudaMemsetParams
593
dst
593
elementSize
593
height
593
pitch
593
value
593
width
593
cudaMemsetParamsV2
593
dst
593
elementSize
593
height
594
pitch
594
value
594
width
594
cudaOffset3D
594
cudaPitchedPtr
594
pitch
594
|
xxx
ptr
594
xsize
594
ysize
595
cudaPointerAttributes
595
device
595
devicePointer
595
hostPointer
595
type
595
cudaPos
595
x
596
y
596
z
596
cudaResourceDesc
596
array
596
desc
596
devPtr
596
height
596
mipmap
596
pitchInBytes
596
resType
597
sizeInBytes
597
width
597
cudaResourceViewDesc
597
depth
597
firstLayer
597
firstMipmapLevel
597
format
597
height
597
lastLayer
597
lastMipmapLevel
598
width
598
cudaTextureDesc
598
addressMode
598
borderColor
598
disableTrilinearOptimization
598
filterMode
598
maxAnisotropy
598
maxMipmapLevelClamp
598
|
xxxi
minMipmapLevelClamp
598
mipmapFilterMode
599
mipmapLevelBias
599
normalizedCoords
599
readMode
599
seamlessCubemap
599
sRGB
599
CUuuid_st
599
bytes
599
Chapter 8. Data Fields
600
Chapter 9. Deprecated List
618
|
xxxii
Chapter 1.
Difference between the driver
and runtime APIs
The driver and runtime APIs are very similar and can for the most part be used interchangeably.
However, there are some key differences worth noting between the two.
Complexity vs. control
The runtime API eases device code management by providing implicit initialization, context
management, and module management. This leads to simpler code, but it also lacks the level of control
that the driver API has.
In comparison, the driver API offers more fine-grained control, especially over contexts and module
loading. Kernel launches are much more complex to implement, as the execution configuration and
kernel parameters must be specified with explicit function calls. However, unlike the runtime, where
all the kernels are automatically loaded during initialization and stay loaded for as long as the program
runs, with the driver API it is possible to only keep the modules that are currently needed loaded, or
even dynamically reload modules. The driver API is also language-independent as it only deals with
cubin objects.
Context management
Context management can be done through the driver API, but is not exposed in the runtime API.
Instead, the runtime API decides itself which context to use for a thread: if a context has been made
current to the calling thread through the driver API, the runtime will use that, but if there is no such
context, it uses a "primary context." Primary contexts are created as needed, one per device per
process, are reference-counted, and are then destroyed when there are no more references to them.
Within one process, all users of the runtime API will share the primary context, unless a context has
been made current to each thread. The context that the runtime uses, i.e, either the current context
or primary context, can be synchronized with cudaDeviceSynchronize(), and destroyed with
cudaDeviceReset().
Using the runtime API with primary contexts has its tradeoffs, however. It can cause trouble for users
writing plug-ins for larger software packages, for example, because if all plug-ins run in the same
process, they will all share a context but will likely have no way to communicate with each other. So,
if one of them calls cudaDeviceReset() after finishing all its CUDA work, the other plug-ins will
fail because the context they were using was destroyed without their knowledge. To avoid this issue,
|
1
Difference between the driver and runtime APIs
CUDA clients can use the driver API to create and set the current context, and then use the runtime API
to work with it. However, contexts may consume significant resources, such as device memory, extra
host threads, and performance costs of context switching on the device. This runtime-driver context
sharing is important when using the driver API in conjunction with libraries built on the runtime API,
such as cuBLAS or cuFFT.
|
2
Chapter 2.
API synchronization behavior
The API provides memcpy/memset functions in both synchronous and asynchronous forms, the
latter having an "Async" suffix. This is a misnomer as each function may exhibit synchronous or
asynchronous behavior depending on the arguments passed to the function. The synchronous forms of
these APIs issue these copies through the default stream.
Any CUDA API call may block or synchronize for various reasons such as contention for or
unavailability of internal resources. Such behavior is subject to change and undocumented behavior
should not be relied upon.
Memcpy
In the reference documentation, each memcpy function is categorized as synchronous or asynchronous,
corresponding to the definitions below.
Synchronous
1. For transfers from pageable host memory to device memory, a stream sync is performed before the
copy is initiated. The function will return once the pageable buffer has been copied to the staging
memory for DMA transfer to device memory, but the DMA to final destination may not have
completed.
2. For transfers from pinned host memory to device memory, the function is synchronous with respect
to the host.
3. For transfers from device to either pageable or pinned host memory, the function returns only once
the copy has completed.
4. For transfers from device memory to device memory, no host-side synchronization is performed.
5. For transfers from any host memory to any host memory, the function is fully synchronous with
respect to the host.
Asynchronous
1. For transfers between device memory and pageable host memory, the function might be
synchronous with respect to host.
2. For transfers from any host memory to any host memory, the function is fully synchronous with
respect to the host.
|
3
API synchronization behavior
3. If pageable memory must first be staged to pinned memory, the driver may synchronize with the
stream and stage the copy into pinned memory.
4. For all other transfers, the function should be fully asynchronous.
Memset
The cudaMemset functions are asynchronous with respect to the host except when the target memory is
pinned host memory. The Async versions are always asynchronous with respect to the host.
Kernel Launches
Kernel launches are asynchronous with respect to the host. Details of concurrent kernel execution and
data transfers can be found in the CUDA Programmers Guide.
|
4
Chapter 3.
Stream synchronization
behavior
Default stream
The default stream, used when 0 is passed as a cudaStream_t or by APIs that operate on a stream
implicitly, can be configured to have either legacy or per-thread synchronization behavior as described
below.
The behavior can be controlled per compilation unit with the --default-stream
nvcc option. Alternatively, per-thread behavior can be enabled by defining the
CUDA_API_PER_THREAD_DEFAULT_STREAM macro before including any CUDA headers. Either way,
the CUDA_API_PER_THREAD_DEFAULT_STREAM macro will be defined in compilation units using per-
thread synchronization behavior.
Legacy default stream
The legacy default stream is an implicit stream which synchronizes with all other streams in the same
CUcontext except for non-blocking streams, described below. (For applications using the runtime
APIs only, there will be one context per device.) When an action is taken in the legacy stream such as a
kernel launch or cudaStreamWaitEvent(), the legacy stream first waits on all blocking streams, the
action is queued in the legacy stream, and then all blocking streams wait on the legacy stream.
For example, the following code launches a kernel k_1 in stream s, then k_2 in the legacy stream, then
k_3 in stream s:
k_1<<<1, 1, 0, s>>>();
k_2<<<1, 1>>>();
k_3<<<1, 1, 0, s>>>();
The resulting behavior is that k_2 will block on k_1 and k_3 will block on k_2.
Non-blocking streams which do not synchronize with the legacy stream can be created using the
cudaStreamNonBlocking flag with the stream creation APIs.
The legacy default stream can be used explicitly with the CUstream (cudaStream_t) handle
CU_STREAM_LEGACY (cudaStreamLegacy).
|
5
Stream synchronization behavior
Per-thread default stream
The per-thread default stream is an implicit stream local to both the thread and the CUcontext, and
which does not synchronize with other streams (just like explicitly created streams). The per-thread
default stream is not a non-blocking stream and will synchronize with the legacy default stream if both
are used in a program.
The per-thread default stream can be used explicitly with the CUstream (cudaStream_t) handle
CU_STREAM_PER_THREAD (cudaStreamPerThread).
|
6
Chapter 4.
Graph object thread safety
Graph objects (cudaGraph_t, CUgraph) are not internally synchronized and must not be accessed
concurrently from multiple threads. API calls accessing the same graph object must be serialized
externally.
Note that this includes APIs which may appear to be read-only, such as cudaGraphClone()
(cuGraphClone()) and cudaGraphInstantiate() (cuGraphInstantiate()). No API or pair
of APIs is guaranteed to be safe to call on the same graph object from two different threads without
serialization.
|
7
Chapter 5.
Rules for version mixing
1. Starting with CUDA 11.0, the ABI version for the CUDA runtime is bumped every major release.
CUDA-defined types, whether opaque handles or structures like cudaDeviceProp, have their
ABI tied to the major release of the CUDA runtime. It is unsafe to pass them from function A to
function B if those functions have been compiled with different major versions of the toolkit and
linked together into the same device executable.
2. The CUDA Driver API has a per-function ABI denoted with a _v* extension. CUDA-defined types
(e.g structs) should not be passed across different ABI versions. For example, an application calling
cuMemcpy2D_v2(const CUDA_MEMCPY2D_v2 *pCopy) and using the older version of the
struct CUDA_MEMCPY2D_v1 instead of CUDA_MEMCPY2D_v2.
3. Users should not arbitrarily mix different API versions during the lifetime of a resource. These
resources include IPC handles, memory, streams, contexts, events, etc. For example, a user
who wants to allocate CUDA memory using cuMemAlloc_v2 should free the memory using
cuMemFree_v2 and not cuMemFree.
|
8
Chapter 6.
Modules
Here is a list of all modules:
‣ Device Management
‣ Device Management [DEPRECATED]
‣ Thread Management [DEPRECATED]
‣ Error Handling
‣ Stream Management
‣ Event Management
‣ External Resource Interoperability
‣ Execution Control
‣ Execution Control [DEPRECATED]
‣ Occupancy
‣ Memory Management
‣ Memory Management [DEPRECATED]
‣ Stream Ordered Memory Allocator
‣ Unified Addressing
‣ Peer Device Memory Access
‣ OpenGL Interoperability
‣ OpenGL Interoperability [DEPRECATED]
‣ Direct3D 9 Interoperability
‣ Direct3D 9 Interoperability [DEPRECATED]
‣ Direct3D 10 Interoperability
‣ Direct3D 10 Interoperability [DEPRECATED]
‣ Direct3D 11 Interoperability
‣ Direct3D 11 Interoperability [DEPRECATED]
‣ VDPAU Interoperability
‣ EGL Interoperability
‣ Graphics Interoperability
|
9
Modules
‣ Texture Object Management
‣ Surface Object Management
‣ Version Management
‣ Graph Management
‣ Driver Entry Point Access
‣ Library Management
‣ C++ API Routines
‣ Interactions with the CUDA Driver API
‣ Profiler Control
‣ Data types used by CUDA Runtime
6.2.
Device Management [DEPRECATED]
This section describes the deprecated device management functions of the CUDA runtime application
programming interface.
__host____
device__cudaError_t
cudaDeviceGetSharedMemConfig (cudaSharedMemConfig
*pConfig)
Returns the shared memory configuration for the current device.
Parameters
pConfig
- Returned cache configuration
Returns
cudaSuccess, cudaErrorInvalidValue
Description
Deprecated
This function will return in pConfig the current size of shared memory banks on the current device.
On devices with configurable shared memory banks, cudaDeviceSetSharedMemConfig can be used to
change this setting, so that all subsequent kernel launches will by default use the new bank size. When
cudaDeviceGetSharedMemConfig is called on devices without configurable shared memory, it will
return the fixed bank size of the hardware.
The returned bank configurations can be either:
‣ cudaSharedMemBankSizeFourByte - shared memory bank width is four bytes.
|
10
Modules
‣ cudaSharedMemBankSizeEightByte - shared memory bank width is eight bytes.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaDeviceSetCacheConfig, cudaDeviceGetCacheConfig, cudaDeviceSetSharedMemConfig,
cudaFuncSetCacheConfig, cuCtxGetSharedMemConfig
__host__cudaError_t cudaDeviceSetSharedMemConfig
(cudaSharedMemConfig config)
Sets the shared memory configuration for the current device.
Parameters
config
- Requested cache configuration
Returns
cudaSuccess, cudaErrorInvalidValue
Description
Deprecated
On devices with configurable shared memory banks, this function will set the shared memory bank size
which is used for all subsequent kernel launches. Any per-function setting of shared memory set via
cudaFuncSetSharedMemConfig will override the device wide setting.
Changing the shared memory configuration between launches may introduce a device side
synchronization point.
Changing the shared memory bank size will not increase shared memory usage or affect occupancy of
kernels, but may have major effects on performance. Larger bank sizes will allow for greater potential
bandwidth to shared memory, but will change what kinds of accesses to shared memory will result in
bank conflicts.
This function will do nothing on devices with fixed shared memory bank size.
The supported bank configurations are:
|
11
Modules
‣ cudaSharedMemBankSizeDefault: set bank width the device default (currently, four bytes)
‣ cudaSharedMemBankSizeFourByte: set shared memory bank width to be four bytes natively.
‣ cudaSharedMemBankSizeEightByte: set shared memory bank width to be eight bytes natively.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaDeviceSetCacheConfig, cudaDeviceGetCacheConfig, cudaDeviceGetSharedMemConfig,
cudaFuncSetCacheConfig, cuCtxSetSharedMemConfig
6.3.
Thread Management [DEPRECATED]
This section describes deprecated thread management functions of the CUDA runtime application
programming interface.
__host__cudaError_t cudaThreadExit (void)
Exit and clean up from CUDA launches.
Returns
cudaSuccess
Description
Deprecated
Note that this function is deprecated because its name does not reflect its behavior. Its functionality is
identical to the non-deprecated function cudaDeviceReset(), which should be used instead.
Explicitly destroys all cleans up all resources associated with the current device in the current process.
Any subsequent API call to this device will reinitialize the device.
Note that this function will reset the device immediately. It is the caller's responsibility to ensure that
the device is not being accessed by any other host threads from the process when this function is called.
Note:
|
12
Modules
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaDeviceReset
__host__cudaError_t cudaThreadGetCacheConfig
(cudaFuncCache *pCacheConfig)
Returns the preferred cache configuration for the current device.
Parameters
pCacheConfig
- Returned cache configuration
Returns
cudaSuccess
Description
Deprecated
Note that this function is deprecated because its name does not reflect its behavior. Its functionality is
identical to the non-deprecated function cudaDeviceGetCacheConfig(), which should be used instead.
On devices where the L1 cache and shared memory use the same hardware resources, this returns
through pCacheConfig the preferred cache configuration for the current device. This is only a
preference. The runtime will use the requested configuration if possible, but it is free to choose a
different configuration if required to execute functions.
This will return a pCacheConfig of cudaFuncCachePreferNone on devices where the size of the L1
cache and shared memory are fixed.
The supported cache configurations are:
‣ cudaFuncCachePreferNone: no preference for shared memory or L1 (default)
‣ cudaFuncCachePreferShared: prefer larger shared memory and smaller L1 cache
‣ cudaFuncCachePreferL1: prefer larger L1 cache and smaller shared memory
Note:
|
13
Modules
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaDeviceGetCacheConfig
__host__cudaError_t cudaThreadGetLimit (size_t *pValue,
cudaLimit limit)
Returns resource limits.
Parameters
pValue
- Returned size in bytes of limit
limit
- Limit to query
Returns
cudaSuccess, cudaErrorUnsupportedLimit, cudaErrorInvalidValue
Description
Deprecated
Note that this function is deprecated because its name does not reflect its behavior. Its functionality is
identical to the non-deprecated function cudaDeviceGetLimit(), which should be used instead.
Returns in *pValue the current size of limit. The supported cudaLimit values are:
‣ cudaLimitStackSize: stack size of each GPU thread;
‣ cudaLimitPrintfFifoSize: size of the shared FIFO used by the printf() device system call.
‣ cudaLimitMallocHeapSize: size of the heap used by the malloc() and free() device system calls;
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
|
14
Modules
See also:
cudaDeviceGetLimit
__host__cudaError_t cudaThreadSetCacheConfig
(cudaFuncCache cacheConfig)
Sets the preferred cache configuration for the current device.
Parameters
cacheConfig
- Requested cache configuration
Returns
cudaSuccess
Description
Deprecated
Note that this function is deprecated because its name does not reflect its behavior. Its functionality is
identical to the non-deprecated function cudaDeviceSetCacheConfig(), which should be used instead.
On devices where the L1 cache and shared memory use the same hardware resources, this sets through
cacheConfig the preferred cache configuration for the current device. This is only a preference. The
runtime will use the requested configuration if possible, but it is free to choose a different configuration
if required to execute the function. Any function preference set via cudaFuncSetCacheConfig ( C API)
or cudaFuncSetCacheConfig ( C++ API) will be preferred over this device-wide setting. Setting the
device-wide cache configuration to cudaFuncCachePreferNone will cause subsequent kernel launches
to prefer to not change the cache configuration unless required to launch the kernel.
This setting does nothing on devices where the size of the L1 cache and shared memory are fixed.
Launching a kernel with a different preference than the most recent preference setting may insert a
device-side synchronization point.
The supported cache configurations are:
‣ cudaFuncCachePreferNone: no preference for shared memory or L1 (default)
‣ cudaFuncCachePreferShared: prefer larger shared memory and smaller L1 cache
‣ cudaFuncCachePreferL1: prefer larger L1 cache and smaller shared memory
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
|
15
Modules
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaDeviceSetCacheConfig
__host__cudaError_t cudaThreadSetLimit (cudaLimit limit,
size_t value)
Set resource limits.
Parameters
limit
- Limit to set
value
- Size in bytes of limit
Returns
cudaSuccess, cudaErrorUnsupportedLimit, cudaErrorInvalidValue
Description
Deprecated
Note that this function is deprecated because its name does not reflect its behavior. Its functionality is
identical to the non-deprecated function cudaDeviceSetLimit(), which should be used instead.
Setting limit to value is a request by the application to update the current limit maintained by
the device. The driver is free to modify the requested value to meet h/w requirements (this could be
clamping to minimum or maximum values, rounding up to nearest element size, etc). The application
can use cudaThreadGetLimit() to find out exactly what the limit has been set to.
Setting each cudaLimit has its own specific restrictions, so each is discussed here.
‣ cudaLimitStackSize controls the stack size of each GPU thread.
‣ cudaLimitPrintfFifoSize controls the size of the shared FIFO used by the printf() device system
call. Setting cudaLimitPrintfFifoSize must be performed before launching any kernel that uses the
printf() device system call, otherwise cudaErrorInvalidValue will be returned.
‣ cudaLimitMallocHeapSize controls the size of the heap used by the malloc() and free() device
system calls. Setting cudaLimitMallocHeapSize must be performed before launching any kernel
that uses the malloc() or free() device system calls, otherwise cudaErrorInvalidValue will be
returned.
|
16
Modules
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaDeviceSetLimit
__host__cudaError_t cudaThreadSynchronize (void)
Wait for compute device to finish.
Returns
cudaSuccess
Description
Deprecated
Note that this function is deprecated because its name does not reflect its behavior. Its functionality is
similar to the non-deprecated function cudaDeviceSynchronize(), which should be used instead.
Blocks until the device has completed all preceding requested tasks. cudaThreadSynchronize() returns
an error if one of the preceding tasks has failed. If the cudaDeviceScheduleBlockingSync flag was set
for this device, the host thread will block until the device has finished its work.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaDeviceSynchronize
|
17
Modules
6.4.
Error Handling
This section describes the error handling functions of the CUDA runtime application programming
interface.
__host____
device__const char *cudaGetErrorName
(cudaError_t error)
Returns the string representation of an error code enum name.
Parameters
error
- Error code to convert to string
Returns
char* pointer to a NULL-terminated string
Description
Returns a string containing the name of an error code in the enum. If the error code is not recognized,
"unrecognized error code" is returned.
See also:
cudaGetErrorString, cudaGetLastError, cudaPeekAtLastError, cudaError, cuGetErrorName
__host____
device__const char *cudaGetErrorString
(cudaError_t error)
Returns the description string for an error code.
Parameters
error
- Error code to convert to string
Returns
char* pointer to a NULL-terminated string
Description
Returns the description string for an error code. If the error code is not recognized, "unrecognized error
code" is returned.
|
18
Modules
See also:
cudaGetErrorName, cudaGetLastError, cudaPeekAtLastError, cudaError, cuGetErrorString
__host____
device__cudaError_t cudaGetLastError (void)
Returns the last error from a runtime call.
Returns
cudaSuccess, cudaErrorMissingConfiguration, cudaErrorMemoryAllocation,
cudaErrorInitializationError, cudaErrorLaunchFailure, cudaErrorLaunchTimeout,
cudaErrorLaunchOutOfResources, cudaErrorInvalidDeviceFunction, cudaErrorInvalidConfiguration,
cudaErrorInvalidDevice, cudaErrorInvalidValue, cudaErrorInvalidPitchValue,
cudaErrorInvalidSymbol, cudaErrorUnmapBufferObjectFailed, cudaErrorInvalidDevicePointer,
cudaErrorInvalidTexture, cudaErrorInvalidTextureBinding, cudaErrorInvalidChannelDescriptor,
cudaErrorInvalidMemcpyDirection, cudaErrorInvalidFilterSetting, cudaErrorInvalidNormSetting,
cudaErrorUnknown, cudaErrorInvalidResourceHandle, cudaErrorInsufficientDriver,
cudaErrorNoDevice, cudaErrorSetOnActiveProcess, cudaErrorStartupFailure,
cudaErrorInvalidPtx, cudaErrorUnsupportedPtxVersion, cudaErrorNoKernelImageForDevice,
cudaErrorJitCompilerNotFound, cudaErrorJitCompilationDisabled
Description
Returns the last error that has been produced by any of the runtime calls in the same instance of the
CUDA Runtime library in the host thread and resets it to cudaSuccess.
Note: Multiple instances of the CUDA Runtime library can be present in an application when using a
library that statically links the CUDA Runtime.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaPeekAtLastError, cudaGetErrorName, cudaGetErrorString, cudaError
|
19
Modules
__host____
device__cudaError_t cudaPeekAtLastError
(void)
Returns the last error from a runtime call.
Returns
cudaSuccess, cudaErrorMissingConfiguration, cudaErrorMemoryAllocation,
cudaErrorInitializationError, cudaErrorLaunchFailure, cudaErrorLaunchTimeout,
cudaErrorLaunchOutOfResources, cudaErrorInvalidDeviceFunction, cudaErrorInvalidConfiguration,
cudaErrorInvalidDevice, cudaErrorInvalidValue, cudaErrorInvalidPitchValue,
cudaErrorInvalidSymbol, cudaErrorUnmapBufferObjectFailed, cudaErrorInvalidDevicePointer,
cudaErrorInvalidTexture, cudaErrorInvalidTextureBinding, cudaErrorInvalidChannelDescriptor,
cudaErrorInvalidMemcpyDirection, cudaErrorInvalidFilterSetting, cudaErrorInvalidNormSetting,
cudaErrorUnknown, cudaErrorInvalidResourceHandle, cudaErrorInsufficientDriver,
cudaErrorNoDevice, cudaErrorSetOnActiveProcess, cudaErrorStartupFailure,
cudaErrorInvalidPtx, cudaErrorUnsupportedPtxVersion, cudaErrorNoKernelImageForDevice,
cudaErrorJitCompilerNotFound, cudaErrorJitCompilationDisabled
Description
Returns the last error that has been produced by any of the runtime calls in the same instance of
the CUDA Runtime library in the host thread. This call does not reset the error to cudaSuccess like
cudaGetLastError().
Note: Multiple instances of the CUDA Runtime library can be present in an application when using a
library that statically links the CUDA Runtime.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaGetLastError, cudaGetErrorName, cudaGetErrorString, cudaError
|
20
Modules
6.5.
Stream Management
This section describes the stream management functions of the CUDA runtime application
programming interface.
typedef void (CUDART_CB *cudaStreamCallback_t)
(cudaStream_t stream, cudaError_t status, void* userData)
Type of stream callback functions.
__host__cudaError_t cudaCtxResetPersistingL2Cache
(void)
Resets all persisting lines in cache to normal status.
Returns
cudaSuccess,
Description
Resets all persisting lines in cache to normal status. Takes effect on function return.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaAccessPolicyWindow
__host__cudaError_t cudaStreamAddCallback
(cudaStream_t stream, cudaStreamCallback_t callback,
void *userData, unsigned int flags)
Add a callback to a compute stream.
Parameters
stream
- Stream to add callback to
callback
- The function to call once preceding stream operations are complete
|
21
Modules
userData
- User specified data to be passed to the callback function
flags
- Reserved for future use, must be 0
Returns
cudaSuccess, cudaErrorInvalidResourceHandle, cudaErrorInvalidValue, cudaErrorNotSupported
Description
Note:
This function is slated for eventual deprecation and removal. If you do not require the callback to
execute in case of a device error, consider using cudaLaunchHostFunc. Additionally, this function is not
supported with cudaStreamBeginCapture and cudaStreamEndCapture, unlike cudaLaunchHostFunc.
Adds a callback to be called on the host after all currently enqueued items in the stream have
completed. For each cudaStreamAddCallback call, a callback will be executed exactly once. The
callback will block later work in the stream until it is finished.
The callback may be passed cudaSuccess or an error code. In the event of a device error, all
subsequently executed callbacks will receive an appropriate cudaError_t.
Callbacks must not make any CUDA API calls. Attempting to use CUDA APIs may result in
cudaErrorNotPermitted. Callbacks must not perform any synchronization that may depend on
outstanding device work or other callbacks that are not mandated to run earlier. Callbacks without a
mandated order (in independent streams) execute in undefined order and may be serialized.
For the purposes of Unified Memory, callback execution makes a number of guarantees:
‣ The callback stream is considered idle for the duration of the callback. Thus, for example, a
callback may always use memory attached to the callback stream.
‣ The start of execution of a callback has the same effect as synchronizing an event recorded in
the same stream immediately prior to the callback. It thus synchronizes streams which have been
"joined" prior to the callback.
‣ Adding device work to any stream does not have the effect of making the stream active until
all preceding callbacks have executed. Thus, for example, a callback might use global attached
memory even if work has been added to another stream, if it has been properly ordered with an
event.
‣ Completion of a callback does not cause a stream to become active except as described above. The
callback stream will remain idle if no device work follows the callback, and will remain idle across
consecutive callbacks without device work in between. Thus, for example, stream synchronization
can be done by signaling from a callback at the end of the stream.
|
22
Modules
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreate, cudaStreamCreateWithFlags, cudaStreamQuery, cudaStreamSynchronize,
cudaStreamWaitEvent, cudaStreamDestroy, cudaMallocManaged, cudaStreamAttachMemAsync,
cudaLaunchHostFunc, cuStreamAddCallback
__host__cudaError_t cudaStreamAttachMemAsync
(cudaStream_t stream, void *devPtr, size_t length,
unsigned int flags)
Attach memory to a stream asynchronously.
Parameters
stream
- Stream in which to enqueue the attach operation
devPtr
- Pointer to memory (must be a pointer to managed memory or to a valid host-accessible region of
system-allocated memory)
length
- Length of memory (defaults to zero)
flags
- Must be one of cudaMemAttachGlobal, cudaMemAttachHost or cudaMemAttachSingle (defaults
to cudaMemAttachSingle)
Returns
cudaSuccess, cudaErrorNotReady, cudaErrorInvalidValue, cudaErrorInvalidResourceHandle
Description
Enqueues an operation in stream to specify stream association of length bytes of memory starting
from devPtr. This function is a stream-ordered operation, meaning that it is dependent on, and
will only take effect when, previous work in stream has completed. Any previous association is
automatically replaced.
|
23
Modules
devPtr must point to an one of the following types of memories:
‣ managed memory declared using the __managed__ keyword or allocated with
cudaMallocManaged.
‣ a valid host-accessible region of system-allocated pageable memory. This type of memory may
only be specified if the device associated with the stream reports a non-zero value for the device
attribute cudaDevAttrPageableMemoryAccess.
For managed allocations, length must be either zero or the entire allocation's size. Both indicate that
the entire allocation's stream association is being changed. Currently, it is not possible to change stream
association for a portion of a managed allocation.
For pageable allocations, length must be non-zero.
The stream association is specified using flags which must be one of cudaMemAttachGlobal,
cudaMemAttachHost or cudaMemAttachSingle. The default value for flags is
cudaMemAttachSingle If the cudaMemAttachGlobal flag is specified, the memory can be accessed
by any stream on any device. If the cudaMemAttachHost flag is specified, the program makes a
guarantee that it won't access the memory on the device from any stream on a device that has a zero
value for the device attribute cudaDevAttrConcurrentManagedAccess. If the cudaMemAttachSingle
flag is specified and stream is associated with a device that has a zero value for the device attribute
cudaDevAttrConcurrentManagedAccess, the program makes a guarantee that it will only access the
memory on the device from stream. It is illegal to attach singly to the NULL stream, because the
NULL stream is a virtual global stream and not a specific stream. An error will be returned in this case.
When memory is associated with a single stream, the Unified Memory system will allow CPU access
to this memory region so long as all operations in stream have completed, regardless of whether
other streams are active. In effect, this constrains exclusive ownership of the managed memory region
by an active GPU to per-stream activity instead of whole-GPU activity.
Accessing memory on the device from streams that are not associated with it will produce undefined
results. No error checking is performed by the Unified Memory system to ensure that kernels launched
into other streams do not access this region.
It is a program's responsibility to order calls to cudaStreamAttachMemAsync via events,
synchronization or other means to ensure legal access to memory at all times. Data visibility and
coherency will be changed appropriately for all kernels which follow a stream-association change.
If stream is destroyed while data is associated with it, the association is removed and the association
reverts to the default visibility of the allocation as specified at cudaMallocManaged. For __managed__
variables, the default association is always cudaMemAttachGlobal. Note that destroying a stream is an
asynchronous operation, and as a result, the change to default association won't happen until all work in
the stream has completed.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
|
24
Modules
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreate, cudaStreamCreateWithFlags, cudaStreamWaitEvent, cudaStreamSynchronize,
cudaStreamAddCallback, cudaStreamDestroy, cudaMallocManaged, cuStreamAttachMemAsync
__host__cudaError_t cudaStreamBeginCapture
(cudaStream_t stream, cudaStreamCaptureMode mode)
Begins graph capture on a stream.
Parameters
stream
- Stream in which to initiate capture
mode
- Controls the interaction of this capture sequence with other API calls that are potentially unsafe.
For more details see cudaThreadExchangeStreamCaptureMode.
Returns
cudaSuccess, cudaErrorInvalidValue
Description
Begin graph capture on stream. When a stream is in capture mode, all operations pushed into the
stream will not be executed, but will instead be captured into a graph, which will be returned via
cudaStreamEndCapture. Capture may not be initiated if stream is cudaStreamLegacy. Capture must
be ended on the same stream in which it was initiated, and it may only be initiated if the stream is not
already in capture mode. The capture mode may be queried via cudaStreamIsCapturing. A unique id
representing the capture sequence may be queried via cudaStreamGetCaptureInfo.
If mode is not cudaStreamCaptureModeRelaxed, cudaStreamEndCapture must be called on this stream
from the same thread.
Note:
Kernels captured using this API must not use texture and surface references. Reading or writing through
any texture or surface reference is undefined behavior. This restriction does not apply to texture and
surface objects.
|
25
Modules
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaStreamCreate, cudaStreamIsCapturing, cudaStreamEndCapture,
cudaThreadExchangeStreamCaptureMode
__host__cudaError_t cudaStreamBeginCaptureToGraph
(cudaStream_t stream, cudaGraph_t graph,
const cudaGraphNode_t *dependencies, const
cudaGraphEdgeData *dependencyData, size_t
numDependencies, cudaStreamCaptureMode mode)
Begins graph capture on a stream to an existing graph.
Parameters
stream
- Stream in which to initiate capture.
graph
- Graph to capture into.
dependencies
- Dependencies of the first node captured in the stream. Can be NULL if numDependencies is 0.
dependencyData
- Optional array of data associated with each dependency.
numDependencies
- Number of dependencies.
mode
- Controls the interaction of this capture sequence with other API calls that are potentially unsafe.
For more details see cudaThreadExchangeStreamCaptureMode.
Returns
cudaSuccess, cudaErrorInvalidValue
Description
Begin graph capture on stream. When a stream is in capture mode, all operations pushed into the
stream will not be executed, but will instead be captured into graph, which will be returned via
cudaStreamEndCapture.
|
26
Modules
Capture may not be initiated if stream is cudaStreamLegacy. Capture must be ended on the same
stream in which it was initiated, and it may only be initiated if the stream is not already in capture
mode. The capture mode may be queried via cudaStreamIsCapturing. A unique id representing the
capture sequence may be queried via cudaStreamGetCaptureInfo.
If mode is not cudaStreamCaptureModeRelaxed, cudaStreamEndCapture must be called on this stream
from the same thread.
Note:
Kernels captured using this API must not use texture and surface references. Reading or writing through
any texture or surface reference is undefined behavior. This restriction does not apply to texture and
surface objects.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaStreamCreate, cudaStreamIsCapturing, cudaStreamEndCapture,
cudaThreadExchangeStreamCaptureMode
__host__cudaError_t cudaStreamCopyAttributes
(cudaStream_t dst, cudaStream_t src)
Copies attributes from source stream to destination stream.
Parameters
dst
Destination stream
src
Source stream For attributes see cudaStreamAttrID
Returns
cudaSuccess, cudaErrorNotSupported
Description
Copies attributes from source stream src to destination stream dst. Both streams must have the same
context.
|
27
Modules
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaAccessPolicyWindow
__host__cudaError_t cudaStreamCreate (cudaStream_t
*pStream)
Create an asynchronous stream.
Parameters
pStream
- Pointer to new stream identifier
Returns
cudaSuccess, cudaErrorInvalidValue
Description
Creates a new asynchronous stream on the context that is current to the calling host thread. If no
context is current to the calling host thread, then the primary context for a device is selected, made
current to the calling thread, and initialized before creating a stream on it.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreateWithPriority, cudaStreamCreateWithFlags, cudaStreamGetPriority,
cudaStreamGetFlags, cudaStreamGetDevice, cudaStreamQuery, cudaStreamSynchronize,
cudaStreamWaitEvent, cudaStreamAddCallback, cudaSetDevice, cudaStreamDestroy, cuStreamCreate
|
28
Modules
__host____
device__cudaError_t
cudaStreamCreateWithFlags (cudaStream_t *pStream,
unsigned int flags)
Create an asynchronous stream.
Parameters
pStream
- Pointer to new stream identifier
flags
- Parameters for stream creation
Returns
cudaSuccess, cudaErrorInvalidValue
Description
Creates a new asynchronous stream on the context that is current to the calling host thread. If no
context is current to the calling host thread, then the primary context for a device is selected, made
current to the calling thread, and initialized before creating a stream on it. The flags argument
determines the behaviors of the stream. Valid values for flags are
‣ cudaStreamDefault: Default stream creation flag.
‣ cudaStreamNonBlocking: Specifies that work running in the created stream may run concurrently
with work in stream 0 (the NULL stream), and that the created stream should perform no implicit
synchronization with stream 0.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreate, cudaStreamCreateWithPriority, cudaStreamGetFlags, cudaStreamGetDevice,
cudaStreamQuery, cudaStreamSynchronize, cudaStreamWaitEvent, cudaStreamAddCallback,
cudaSetDevice, cudaStreamDestroy, cuStreamCreate
|
29
Modules
__host__cudaError_t cudaStreamCreateWithPriority
(cudaStream_t *pStream, unsigned int flags, int priority)
Create an asynchronous stream with the specified priority.
Parameters
pStream
- Pointer to new stream identifier
flags
- Flags for stream creation. See cudaStreamCreateWithFlags for a list of valid flags that can be
passed
priority
- Priority of the stream. Lower numbers represent higher priorities. See
cudaDeviceGetStreamPriorityRange for more information about the meaningful stream priorities
that can be passed.
Returns
cudaSuccess, cudaErrorInvalidValue
Description
Creates a stream with the specified priority and returns a handle in pStream. The stream is created on
the context that is current to the calling host thread. If no context is current to the calling host thread,
then the primary context for a device is selected, made current to the calling thread, and initialized
before creating a stream on it. This affects the scheduling priority of work in the stream. Priorities
provide a hint to preferentially run work with higher priority when possible, but do not preempt
already-running work or provide any other functional guarantee on execution order.
priority follows a convention where lower numbers represent higher priorities. '0'
represents default priority. The range of meaningful numerical priorities can be queried using
cudaDeviceGetStreamPriorityRange. If the specified priority is outside the numerical range returned
by cudaDeviceGetStreamPriorityRange, it will automatically be clamped to the lowest or the highest
number in the range.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
‣ Stream priorities are supported only on GPUs with compute capability 3.5 or higher.
|
30
Modules
‣ In the current implementation, only compute kernels launched in priority streams are affected by
the stream's priority. Stream priorities have no effect on host-to-device and device-to-host memory
operations.
See also:
cudaStreamCreate, cudaStreamCreateWithFlags, cudaDeviceGetStreamPriorityRange,
cudaStreamGetPriority, cudaStreamQuery, cudaStreamWaitEvent, cudaStreamAddCallback,
cudaStreamSynchronize, cudaSetDevice, cudaStreamDestroy, cuStreamCreateWithPriority
__host____
device__cudaError_t cudaStreamDestroy
(cudaStream_t stream)
Destroys and cleans up an asynchronous stream.
Parameters
stream
- Stream identifier
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorInvalidResourceHandle
Description
Destroys and cleans up the asynchronous stream specified by stream.
In case the device is still doing work in the stream stream when cudaStreamDestroy() is called,
the function will return immediately and the resources associated with stream will be released
automatically once the device has completed all work in stream.
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
‣ Use of the handle after this call is undefined behavior.
See also:
|
31
Modules
cudaStreamCreate, cudaStreamCreateWithFlags, cudaStreamQuery, cudaStreamWaitEvent,
cudaStreamSynchronize, cudaStreamAddCallback, cuStreamDestroy
__host__cudaError_t cudaStreamEndCapture
(cudaStream_t stream, cudaGraph_t *pGraph)
Ends capture on a stream, returning the captured graph.
Parameters
stream
- Stream to query
pGraph
- The captured graph
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorStreamCaptureWrongThread
Description
End capture on stream, returning the captured graph via pGraph. Capture must have been initiated
on stream via a call to cudaStreamBeginCapture. If capture was invalidated, due to a violation of the
rules of stream capture, then a NULL graph will be returned.
If the mode argument to cudaStreamBeginCapture was not cudaStreamCaptureModeRelaxed, this call
must be from the same thread as cudaStreamBeginCapture.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaStreamCreate, cudaStreamBeginCapture, cudaStreamIsCapturing, cudaGraphDestroy
__host__cudaError_t cudaStreamGetAttribute
(cudaStream_t hStream, cudaStreamAttrID attr,
cudaStreamAttrValue *value_out)
Queries stream attribute.
Parameters
hStream
attr
|
32
Modules
value_out
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorInvalidResourceHandle
Description
Queries attribute attr from hStream and stores it in corresponding member of value_out.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaAccessPolicyWindow
__host__cudaError_t cudaStreamGetCaptureInfo
(cudaStream_t stream, cudaStreamCaptureStatus
*captureStatus_out, unsigned long long *id_out,
cudaGraph_t *graph_out, const cudaGraphNode_t
**dependencies_out, size_t *numDependencies_out)
Query a stream's capture state.
Parameters
stream
- The stream to query
captureStatus_out
- Location to return the capture status of the stream; required
id_out
- Optional location to return an id for the capture sequence, which is unique over the lifetime of the
process
graph_out
- Optional location to return the graph being captured into. All operations other than destroy and
node removal are permitted on the graph while the capture sequence is in progress. This API does
not transfer ownership of the graph, which is transferred or destroyed at cudaStreamEndCapture.
Note that the graph handle may be invalidated before end of capture for certain errors. Nodes that
are or become unreachable from the original stream at cudaStreamEndCapture due to direct actions
on the graph do not trigger cudaErrorStreamCaptureUnjoined.
|
33
Modules
dependencies_out
- Optional location to store a pointer to an array of nodes. The next node to be captured in the
stream will depend on this set of nodes, absent operations such as event wait which modify this set.
The array pointer is valid until the next API call which operates on the stream or until the capture is
terminated. The node handles may be copied out and are valid until they or the graph is destroyed.
The driver-owned array may also be passed directly to APIs that operate on the graph (not the
stream) without copying.
numDependencies_out
- Optional location to store the size of the array returned in dependencies_out.
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorStreamCaptureImplicit
Description
Query stream state related to stream capture.
If called on cudaStreamLegacy (the "null stream") while a stream not created with
cudaStreamNonBlocking is capturing, returns cudaErrorStreamCaptureImplicit.
Valid data (other than capture status) is returned only if both of the following are true:
‣ the call returns cudaSuccess
‣ the returned capture status is cudaStreamCaptureStatusActive
Note:
‣ Graph objects are not threadsafe. More here.
‣ Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaStreamGetCaptureInfo_v3, cudaStreamBeginCapture, cudaStreamIsCapturing,
cudaStreamUpdateCaptureDependencies
|
34
Modules
__host__cudaError_t cudaStreamGetCaptureInfo_v3
(cudaStream_t stream, cudaStreamCaptureStatus
*captureStatus_out, unsigned long long *id_out,
cudaGraph_t *graph_out, const cudaGraphNode_t
**dependencies_out, const cudaGraphEdgeData
**edgeData_out, size_t *numDependencies_out)
Query a stream's capture state (12.3+).
Parameters
stream
- The stream to query
captureStatus_out
- Location to return the capture status of the stream; required
id_out
- Optional location to return an id for the capture sequence, which is unique over the lifetime of the
process
graph_out
- Optional location to return the graph being captured into. All operations other than destroy and
node removal are permitted on the graph while the capture sequence is in progress. This API does
not transfer ownership of the graph, which is transferred or destroyed at cudaStreamEndCapture.
Note that the graph handle may be invalidated before end of capture for certain errors. Nodes that
are or become unreachable from the original stream at cudaStreamEndCapture due to direct actions
on the graph do not trigger cudaErrorStreamCaptureUnjoined.
dependencies_out
- Optional location to store a pointer to an array of nodes. The next node to be captured in the
stream will depend on this set of nodes, absent operations such as event wait which modify this set.
The array pointer is valid until the next API call which operates on the stream or until the capture is
terminated. The node handles may be copied out and are valid until they or the graph is destroyed.
The driver-owned array may also be passed directly to APIs that operate on the graph (not the
stream) without copying.
edgeData_out
- Optional location to store a pointer to an array of graph edge data. This array parallels
dependencies_out; the next node to be added has an edge to dependencies_out[i] with
annotation edgeData_out[i] for each i. The array pointer is valid until the next API call which
operates on the stream or until the capture is terminated.
numDependencies_out
- Optional location to store the size of the array returned in dependencies_out.
|
35
Modules
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorStreamCaptureImplicit, cudaErrorLossyQuery
Description
Query stream state related to stream capture.
If called on cudaStreamLegacy (the "null stream") while a stream not created with
cudaStreamNonBlocking is capturing, returns cudaErrorStreamCaptureImplicit.
Valid data (other than capture status) is returned only if both of the following are true:
‣ the call returns cudaSuccess
‣ the returned capture status is cudaStreamCaptureStatusActive
If edgeData_out is non-NULL then dependencies_out must be as well. If
dependencies_out is non-NULL and edgeData_out is NULL, but there is non-zero edge data
for one or more of the current stream dependencies, the call will return cudaErrorLossyQuery.
Note:
‣ Graph objects are not threadsafe. More here.
‣ Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaStreamBeginCapture, cudaStreamIsCapturing, cudaStreamUpdateCaptureDependencies
__host__cudaError_t cudaStreamGetDevice (cudaStream_t
hStream, int *device)
Query the device of a stream.
Parameters
hStream
- Handle to the stream to be queried
device
- Returns the device to which the stream belongs
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorDeviceUnavailable,
|
36
Modules
Description
Returns in *device the device of the stream.
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaSetDevice, cudaGetDevice, cudaStreamCreate, cudaStreamGetPriority, cudaStreamGetFlags,
cuStreamGetId
__host__cudaError_t cudaStreamGetFlags (cudaStream_t
hStream, unsigned int *flags)
Query the flags of a stream.
Parameters
hStream
- Handle to the stream to be queried
flags
- Pointer to an unsigned integer in which the stream's flags are returned
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorInvalidResourceHandle
Description
Query the flags of a stream. The flags are returned in flags. See cudaStreamCreateWithFlags for a
list of valid flags.
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
|
37
Modules
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreateWithPriority, cudaStreamCreateWithFlags, cudaStreamGetPriority,
cudaStreamGetDevice, cuStreamGetFlags
__host__cudaError_t cudaStreamGetId (cudaStream_t
hStream, unsigned long long *streamId)
Query the Id of a stream.
Parameters
hStream
- Handle to the stream to be queried
streamId
- Pointer to an unsigned long long in which the stream Id is returned
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorInvalidResourceHandle
Description
Query the Id of a stream. The Id is returned in streamId. The Id is unique for the life of the program.
The stream handle hStream can refer to any of the following:
‣ a stream created via any of the CUDA runtime APIs such as cudaStreamCreate,
cudaStreamCreateWithFlags and cudaStreamCreateWithPriority, or their driver API equivalents
such as cuStreamCreate or cuStreamCreateWithPriority. Passing an invalid handle will result in
undefined behavior.
‣ any of the special streams such as the NULL stream, cudaStreamLegacy and cudaStreamPerThread
respectively. The driver API equivalents of these are also accepted which are NULL,
CU_STREAM_LEGACY and CU_STREAM_PER_THREAD.
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
|
38
Modules
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreateWithPriority, cudaStreamCreateWithFlags, cudaStreamGetPriority,
cudaStreamGetFlags, cuStreamGetId
__host__cudaError_t cudaStreamGetPriority (cudaStream_t
hStream, int *priority)
Query the priority of a stream.
Parameters
hStream
- Handle to the stream to be queried
priority
- Pointer to a signed integer in which the stream's priority is returned
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorInvalidResourceHandle
Description
Query the priority of a stream. The priority is returned in in priority. Note that if
the stream was created with a priority outside the meaningful numerical range returned
by cudaDeviceGetStreamPriorityRange, this function returns the clamped priority. See
cudaStreamCreateWithPriority for details about priority clamping.
Note:
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreateWithPriority, cudaDeviceGetStreamPriorityRange, cudaStreamGetFlags,
cudaStreamGetDevice, cuStreamGetPriority
|
39
Modules
__host__cudaError_t cudaStreamIsCapturing
(cudaStream_t stream, cudaStreamCaptureStatus
*pCaptureStatus)
Returns a stream's capture status.
Parameters
stream
- Stream to query
pCaptureStatus
- Returns the stream's capture status
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorStreamCaptureImplicit
Description
Return the capture status of stream via pCaptureStatus. After a successful call,
*pCaptureStatus will contain one of the following:
‣ cudaStreamCaptureStatusNone: The stream is not capturing.
‣ cudaStreamCaptureStatusActive: The stream is capturing.
‣ cudaStreamCaptureStatusInvalidated: The stream was capturing but an error has invalidated the
capture sequence. The capture sequence must be terminated with cudaStreamEndCapture on the
stream where it was initiated in order to continue using stream.
Note that, if this is called on cudaStreamLegacy (the "null stream") while a blocking stream on the
same device is capturing, it will return cudaErrorStreamCaptureImplicit and *pCaptureStatus is
unspecified after the call. The blocking stream capture is not invalidated.
When a blocking stream is capturing, the legacy stream is in an unusable state until the blocking stream
capture is terminated. The legacy stream is not supported for stream capture, but attempted use would
have an implicit dependency on the capturing stream(s).
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaStreamCreate, cudaStreamBeginCapture, cudaStreamEndCapture
|
40
Modules
__host__cudaError_t cudaStreamQuery (cudaStream_t
stream)
Queries an asynchronous stream for completion status.
Parameters
stream
- Stream identifier
Returns
cudaSuccess, cudaErrorNotReady, cudaErrorInvalidResourceHandle
Description
Returns cudaSuccess if all operations in stream have completed, or cudaErrorNotReady if not.
For the purposes of Unified Memory, a return value of cudaSuccess is equivalent to having called
cudaStreamSynchronize().
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
‣ Note that as specified by cudaStreamAddCallback no CUDA function may be called from callback.
cudaErrorNotPermitted may, but is not guaranteed to, be returned as a diagnostic in such case.
See also:
cudaStreamCreate, cudaStreamCreateWithFlags, cudaStreamWaitEvent, cudaStreamSynchronize,
cudaStreamAddCallback, cudaStreamDestroy, cuStreamQuery
__host__cudaError_t cudaStreamSetAttribute
(cudaStream_t hStream, cudaStreamAttrID attr, const
cudaStreamAttrValue *value)
Sets stream attribute.
Parameters
hStream
|
41
Modules
attr
value
Returns
cudaSuccess, cudaErrorInvalidValue, cudaErrorInvalidResourceHandle
Description
Sets attribute attr on hStream from corresponding attribute of value. The updated attribute will
be applied to subsequent work submitted to the stream. It will not affect previously submitted work.
Note:
Note that this function may also return error codes from previous, asynchronous launches.
See also:
cudaAccessPolicyWindow
__host__cudaError_t cudaStreamSynchronize
(cudaStream_t stream)
Waits for stream tasks to complete.
Parameters
stream
- Stream identifier
Returns
cudaSuccess, cudaErrorInvalidResourceHandle
Description
Blocks until stream has completed all operations. If the cudaDeviceScheduleBlockingSync flag was
set for this device, the host thread will block until the stream is finished with all of its tasks.
Note:
‣ This function uses standard default stream semantics.
‣ Note that this function may also return error codes from previous, asynchronous launches.
‣ Note that this function may also return cudaErrorInitializationError, cudaErrorInsufficientDriver or
cudaErrorNoDevice if this call tries to initialize internal CUDA RT state.
|
42
////////////////////////////////////////// |
||
|
|
|