NVIDIA TensorRT 8.6.11 API Reference for DRIVE OS (May 2023)

632

Class Documentation

9.127.3.9 setZeroIsPlaceholder()

void nvinfer1::IShuffleLayer::setZeroIsPlaceholder (

bool zeroIsPlaceholder ) [inline], [noexcept]

Set meaning of 0 in reshape dimensions.

If true, then a 0 in the reshape dimensions denotes copying the corresponding dimension from the first input tensor. If

false, then a 0 in the reshape dimensions denotes a zero-length dimension.

Default: true

See also

getZeroIsPlaceholder();

9.127.4

Member Data Documentation

9.127.4.1 mImpl

apiv::VShuffleLayer∗ nvinfer1::IShuffleLayer::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

9.128

nvinfer1::ISliceLayer Class Reference

Slices an input tensor into an output tensor based on the offset and strides.

#include <NvInfer.h>

Inheritance diagram for nvinfer1::ISliceLayer:

nvinfer1::INoCopy

nvinfer1::ILayer

nvinfer1::ISliceLayer

9.128 nvinfer1::ISliceLayer Class Reference

633

Public Member Functions

• void setStart (Dims start) noexcept

Set the start offset that the slice layer uses to create the output slice.

• Dims getStart () const noexcept

Get the start offset for the slice layer.

• void setSize (Dims size) noexcept

Set the dimensions of the output slice.

• Dims getSize () const noexcept

Get dimensions of the output slice.

• void setStride (Dims stride) noexcept

Set the stride for computing the output slice data.

• Dims getStride () const noexcept

Get the stride for the output slice.

• void setMode (SliceMode mode) noexcept

Set the slice mode.

• SliceMode getMode () const noexcept

Get the slice mode.

• void setInput (int32 t index, ITensor &tensor) noexcept

Append or replace an input of this layer with a specific tensor.

Protected Member Functions

• virtual ∼ISliceLayer () noexcept=default

Protected Attributes

• apiv::VSliceLayer ∗ mImpl

9.128.1

Detailed Description

Slices an input tensor into an output tensor based on the offset and strides.

The slice layer has two variants, static and dynamic. Static slice specifies the start, size, and stride dimensions at layer

creation time via Dims and can use the get/set accessor functions of the ISliceLayer. Dynamic slice specifies one or

more of start, size or stride as ITensors, by using ILayer::setInput to add a second, third, or fourth input respectively.

The corresponding Dims are used if an input is missing or null.

An application can determine if the ISliceLayer has a dynamic output shape based on whether the size input (third

input) is present and non-null.

The slice layer selects for each dimension a start location from within the input tensor, and copies elements to the

output tensor using the specified stride across the input tensor. Start, size, and stride tensors must be 1D Int32 shape

tensors if not specified via Dims.

An example of using slice on a tensor: input = {{0, 2, 4}, {1, 3, 5}} start = {1, 0} size = {1, 2} stride = {1, 2} output

= {{1, 5}}

When the sliceMode is kCLAMP or kREFLECT, for each input dimension, if its size is 0 then the corresponding output

dimension must be 0 too.

A slice layer can produce a shape tensor if the following conditions are met:

634

Class Documentation

• start, size, and stride are build time constants, either as static Dims or as constant input tensors.

• The number of elements in the output tensor does not exceed 2 ∗ Dims::MAX DIMS.

The input tensor is a shape tensor if the output is a shape tensor.

The following constraints must be satisfied to execute this layer on DLA:

• start, size, and stride are build time constants, either as static Dims or as constant input tensors.

• sliceMode is kDEFAULT.

• Strides are 1 for all dimensions.

• Slicing is not performed on the first dimension

• The input tensor has four dimensions

Warning

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

9.128.2

Constructor & Destructor Documentation

9.128.2.1 ∼ISliceLayer()

virtual nvinfer1::ISliceLayer::∼ISliceLayer ( ) [protected], [virtual], [default], [noexcept]

9.128.3

Member Function Documentation

9.128.3.1 getMode()

SliceMode nvinfer1::ISliceLayer::getMode ( ) const [inline], [noexcept]

Get the slice mode.

See also

setMode()

9.128 nvinfer1::ISliceLayer Class Reference

635

9.128.3.2 getSize()

Dims nvinfer1::ISliceLayer::getSize ( ) const [inline], [noexcept]

Get dimensions of the output slice.

Returns

The output dimension, or an invalid Dims structure.

If the third input is present and non-null, this function returns a Dims with nbDims = -1.

See also

setSize

9.128.3.3 getStart()

Dims nvinfer1::ISliceLayer::getStart ( ) const [inline], [noexcept]

Get the start offset for the slice layer.

Returns

The start offset, or an invalid Dims structure.

If the second input is present and non-null, this function returns a Dims with nbDims = -1.

See also

setStart

9.128.3.4 getStride()

Dims nvinfer1::ISliceLayer::getStride ( ) const [inline], [noexcept]

Get the stride for the output slice.

Returns

The slicing stride, or an invalid Dims structure.

If the fourth input is present and non-null, this function returns a Dims with nbDims = -1.

See also

setStride

9.128.3.5 setInput()

void nvinfer1::ILayer::setInput (

int32 t index,

ITensor & tensor ) [inline], [noexcept]

Append or replace an input of this layer with a specific tensor.

636

Class Documentation

Parameters

index

the index of the input to modify.

tensor

the new input tensor

For a slice layer, the values 0-4 are valid. The indices are as follows:

• 0: Tensor to be sliced.

• 1: The start tensor to begin slicing, as a 1D Int32 shape tensor.

• 2: The size tensor of the resulting slice, as a 1D Int32 shape tensor.

• 3: The stride of the slicing operation, as a 1D Int32 shape tensor.

•

4: Value for the kFILL slice mode. The fill value data type should either be the same or be implicitly convertible

to the input data type. Implicit data type conversion is supported among kFLOAT, kHALF, kINT8, and kFP8

data types. This input is disallowed for other modes.

Using the corresponding setter resets the input to null.

If this function is called with a value greater than 0, then the function getNbInputs() changes from returning 1 to index

+ 1.

9.128.3.6 setMode()

void nvinfer1::ISliceLayer::setMode (

SliceMode mode ) [inline], [noexcept]

Set the slice mode.

See also

getMode()

9.128.3.7 setSize()

void nvinfer1::ISliceLayer::setSize (

Dims size ) [inline], [noexcept]

Set the dimensions of the output slice.

Parameters

size

The dimensions of the output slice.

9.128 nvinfer1::ISliceLayer Class Reference

637

If a third input had been used to create this layer, that input is reset to null by this method.

See also

getSize

9.128.3.8 setStart()

void nvinfer1::ISliceLayer::setStart (

Dims start ) [inline], [noexcept]

Set the start offset that the slice layer uses to create the output slice.

Parameters

start

The start offset to read data from the input tensor.

If a second input had been used to create this layer, that input is reset to null by this method.

See also

getStart

9.128.3.9 setStride()

void nvinfer1::ISliceLayer::setStride (

Dims stride ) [inline], [noexcept]

Set the stride for computing the output slice data.

Parameters

stride

The dimensions of the stride to compute the values to store in the output slice.

If a fourth input had been used to create this layer, that input is reset to null by this method.

See also

getStride

638

Class Documentation

9.128.4

Member Data Documentation

9.128.4.1 mImpl

apiv::VSliceLayer∗ nvinfer1::ISliceLayer::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

9.129

nvinfer1::ISoftMaxLayer Class Reference

A Softmax layer in a network definition.

#include <NvInfer.h>

Inheritance diagram for nvinfer1::ISoftMaxLayer:

nvinfer1::INoCopy

nvinfer1::ILayer

nvinfer1::ISoftMaxLayer

Public Member Functions

• void setAxes (uint32 t axes) noexcept

Set the axis along which softmax is computed. Currently, only one axis can be set.

• uint32 t getAxes () const noexcept

Get the axis along which softmax occurs.

Protected Member Functions

• virtual ∼ISoftMaxLayer () noexcept=default

Protected Attributes

• apiv::VSoftMaxLayer ∗ mImpl

9.129 nvinfer1::ISoftMaxLayer Class Reference

639

9.129.1

Detailed Description

A Softmax layer in a network definition.

This layer applies a per-channel softmax to its input.

The output size is the same as the input size.

On Xavier, this layer is not supported on DLA. Otherwise, the following constraints must be satisfied to execute this

layer on DLA:

• Axis must be one of the channel or spatial dimensions.

• There are two classes of supported input sizes:

1. Non-axis, non-batch dimensions are all 1 and the axis dimension is at most 8192. This is the recommended

case for using softmax since it is the most accurate.

2. At least one non-axis, non-batch dimension greater than 1 and the axis dimension is at most 1024. Note

that in this case, there may be some approximation error as the axis dimension size approaches the upper

bound. See the TensorRT Developer Guide for more details on the approximation error.

Warning

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

9.129.2

Constructor & Destructor Documentation

9.129.2.1 ∼ISoftMaxLayer()

virtual nvinfer1::ISoftMaxLayer::∼ISoftMaxLayer ( ) [protected], [virtual], [default], [noexcept]

9.129.3

Member Function Documentation

9.129.3.1 getAxes()

uint32 t nvinfer1::ISoftMaxLayer::getAxes ( ) const [inline], [noexcept]

Get the axis along which softmax occurs.

See also

setAxes()

640

Class Documentation

9.129.3.2 setAxes()

void nvinfer1::ISoftMaxLayer::setAxes (

uint32 t axes ) [inline], [noexcept]

Set the axis along which softmax is computed. Currently, only one axis can be set.

The axis is specified by setting the bit corresponding to the axis to 1. For example, consider an NCHW tensor as input

(three non-batch dimensions).

In implicit mode : Bit 0 corresponds to the C dimension boolean. Bit 1 corresponds to the H dimension boolean. Bit

2 corresponds to the W dimension boolean. By default, softmax is performed on the axis which is the number of axes

minus three. It is 0 if there are fewer than 3 non-batch axes. For example, if the input is NCHW, the default axis is C.

If the input is NHW, then the default axis is H.

In explicit mode : Bit 0 corresponds to the N dimension boolean. Bit 1 corresponds to the C dimension boolean. Bit

2 corresponds to the H dimension boolean. Bit 3 corresponds to the W dimension boolean. By default, softmax is

performed on the axis which is the number of axes minus three. It is 0 if there are fewer than 3 axes. For example, if

the input is NCHW, the default axis is C. If the input is NHW, then the default axis is N.

For example, to perform softmax on axis R of a NPQRCHW input, set bit 2 with implicit batch mode, set bit 3 with

explicit batch mode.

Parameters

axes

The axis along which softmax is computed. Here axes is a bitmap. For example, when doing softmax along

axis 0, bit 0 is set to 1, axes = 1 << axis = 1.

9.129.4

Member Data Documentation

9.129.4.1 mImpl

apiv::VSoftMaxLayer∗ nvinfer1::ISoftMaxLayer::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

9.130

nvinfer1::ITensor Class Reference

A tensor in a network definition.

#include <NvInfer.h>

9.130 nvinfer1::ITensor Class Reference

641

Inheritance diagram for nvinfer1::ITensor:

nvinfer1::INoCopy

nvinfer1::ITensor

Public Member Functions

•

void setName (char const ∗name) noexcept

Set the tensor name.

•

char const ∗ getName () const noexcept

Get the tensor name.

•

void setDimensions (Dims dimensions) noexcept

Set the dimensions of a tensor.

•

Dims getDimensions () const noexcept

Get the dimensions of a tensor.

•

void setType (DataType type) noexcept

Set the data type of a tensor.

•

DataType getType () const noexcept

Get the data type of a tensor.

•

bool setDynamicRange (float min, float max) noexcept

Set dynamic range for the tensor.

•

bool isNetworkInput () const noexcept

Whether the tensor is a network input.

•

bool isNetworkOutput () const noexcept

Whether the tensor is a network output.

•

void setBroadcastAcrossBatch (bool broadcastAcrossBatch) noexcept

Set whether to enable broadcast of tensor across the batch.

•

bool getBroadcastAcrossBatch () const noexcept

Check if tensor is broadcast across the batch.

•

TensorLocation getLocation () const noexcept

Get the storage location of a tensor.

•

void setLocation (TensorLocation location) noexcept

Set the storage location of a tensor.

•

bool dynamicRangeIsSet () const noexcept

Query whether dynamic range is set.

•

void resetDynamicRange () noexcept

Undo effect of setDynamicRange.

•

float getDynamicRangeMin () const noexcept

Get minimum of dynamic range.

•

float getDynamicRangeMax () const noexcept

Get maximum of dynamic range.

•

void setAllowedFormats (TensorFormats formats) noexcept

Set allowed formats for this tensor. By default all formats are allowed. Shape tensors (for which isShapeTensor() returns

true) may only have row major linear format.

642

Class Documentation

• TensorFormats getAllowedFormats () const noexcept

Get a bitmask of TensorFormat values that the tensor supports. For a shape tensor, only row major linear format is

allowed.

• bool isShapeTensor () const noexcept

Whether the tensor is a shape tensor.

• bool isExecutionTensor () const noexcept

Whether the tensor is an execution tensor.

• void setDimensionName (int32 t index, char const ∗name) noexcept

Name a dimension of an input tensor.

• char const ∗ getDimensionName (int32 t index) const noexcept

Get the name of an input dimension.

Protected Member Functions

• virtual ∼ITensor () noexcept=default

Protected Attributes

• apiv::VTensor ∗ mImpl

9.130.1

Detailed Description

A tensor in a network definition.

To remove a tensor from a network definition, use INetworkDefinition::removeTensor().

When using the DLA, the cumulative size of all Tensors that are not marked as Network Input or Output tensors, must

be less than 1GB in size to fit into a single subgraph. If the build option kGPU FALLBACK is specified, then multiple

subgraphs can be created, with each subgraph limited to less than 1GB of internal tensors data.

Warning

The volume of the tensor must be less than 2^∧31 elements. If the tensor is a shape tensor, its volume must

not exceed 64.

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

9.130.2

Constructor & Destructor Documentation

9.130.2.1 ∼ITensor()

virtual nvinfer1::ITensor::∼ITensor ( ) [protected], [virtual], [default], [noexcept]

9.130 nvinfer1::ITensor Class Reference

643

9.130.3

Member Function Documentation

9.130.3.1 dynamicRangeIsSet()

bool nvinfer1::ITensor::dynamicRangeIsSet ( ) const [inline], [noexcept]

Query whether dynamic range is set.

Returns

True if dynamic range is set, false otherwise.

9.130.3.2 getAllowedFormats()

TensorFormats nvinfer1::ITensor::getAllowedFormats ( ) const [inline], [noexcept]

Get a bitmask of TensorFormat values that the tensor supports. For a shape tensor, only row major linear format is

allowed.

Returns

The value specified by setAllowedFormats or all possible formats.

See also

ITensor::setAllowedFormats()

9.130.3.3 getBroadcastAcrossBatch()

bool nvinfer1::ITensor::getBroadcastAcrossBatch ( ) const [inline], [noexcept]

Check if tensor is broadcast across the batch.

When a tensor is broadcast across a batch, it has the same value for every member in the batch. Memory is only

allocated once for the single member. If the network is in explicit batch mode, this function returns true if the leading

dimension is 1.

Returns

True if tensor is broadcast across the batch, false otherwise.

See also

setBroadcastAcrossBatch()

9.130.3.4 getDimensionName()

char const ∗ nvinfer1::ITensor::getDimensionName (

int32 t index ) const [inline], [noexcept]

Get the name of an input dimension.

644

Class Documentation

Parameters

index

index of the dimension

Returns

The name of the input dimension, or nullptr if the dimension has no name. The name is a pointer to a null-

terminated character sequence.

See also

setDimensionName()

9.130.3.5 getDimensions()

Dims nvinfer1::ITensor::getDimensions ( ) const [inline], [noexcept]

Get the dimensions of a tensor.

Returns

The dimensions of the tensor.

Warning

getDimensions() returns a -1 for dimensions that are derived from a wildcard dimension.

See also

setDimensions()

9.130.3.6 getDynamicRangeMax()

float nvinfer1::ITensor::getDynamicRangeMax ( ) const [inline], [noexcept]

Get maximum of dynamic range.

Returns

Maximum of dynamic range, or quiet NaN if range was not set.

646

Class Documentation

9.130.3.10 getType()

DataType nvinfer1::ITensor::getType ( ) const [inline], [noexcept]

Get the data type of a tensor.

Returns

The data type of the tensor.

See also

setType()

9.130.3.11 isExecutionTensor()

bool nvinfer1::ITensor::isExecutionTensor ( ) const [inline], [noexcept]

Whether the tensor is an execution tensor.

Tensors are usually execution tensors. The exceptions are tensors used solely for shape calculations or whose contents

not needed to compute the outputs.

The result of isExecutionTensor() is reliable only when network construction is complete. For example, if a partially

built network has no path from a tensor to a network output, isExecutionTensor() returns false. Completing the path

would cause it to become true.

If a tensor is an execution tensor and becomes an engine input or output, then ICudaEngine::isExecutionBinding will

be true for that tensor.

A tensor with isShapeTensor() == false and isExecutionTensor() == false can still show up as an input to the engine

if its dimensions are required. In that case, only its dimensions need to be set at runtime and a nullptr can be passed

instead of a pointer to its contents.

9.130.3.12 isNetworkInput()

bool nvinfer1::ITensor::isNetworkInput ( ) const [inline], [noexcept]

Whether the tensor is a network input.

9.130 nvinfer1::ITensor Class Reference

647

9.130.3.13 isNetworkOutput()

bool nvinfer1::ITensor::isNetworkOutput ( ) const [inline], [noexcept]

Whether the tensor is a network output.

9.130.3.14 isShapeTensor()

bool nvinfer1::ITensor::isShapeTensor ( ) const [inline], [noexcept]

Whether the tensor is a shape tensor.

A shape tensor is a tensor that is related to shape calculations. It must have type Int32, Bool, or Float, and its shape

must be determinable at build time. Furthermore, it must be needed as a shape tensor, either marked as a network shape

output via markOutputForShapes(), or as a layer input that is required to be a shape tensor, such as the second input to

IShuffleLayer. Some layers are ”polymorphic” in this respect. For example, the inputs to IElementWiseLayer must be

shape tensors if the output is a shape tensor.

The TensorRT Developer Guide give the formal rules for what tensors are shape tensors.

The result of isShapeTensor() is reliable only when network construction is complete. For example, if a partially built

network sums two tensors T1 and T2 to create tensor T3, and none are yet needed as shape tensors, isShapeTensor()

returns false for all three tensors. Setting the second input of IShuffleLayer to be T3 would cause all three tensors to be

shape tensors, because IShuffleLayer requires that its second optional input be a shape tensor, and IElementWiseLayer

is ”polymorphic”.

If a tensor is a shape tensor and becomes an engine input or output, then ICudaEngine::isShapeBinding will be true for

that tensor. Such a shape tensor must have type Int32.

It is possible for a tensor to be both a shape tensor and an execution tensor.

Returns

True if tensor is a shape tensor, false otherwise.

See also

INetworkDefinition::markOutputForShapes(), ICudaEngine::isShapeBinding()

9.130.3.15 resetDynamicRange()

void nvinfer1::ITensor::resetDynamicRange ( ) [inline], [noexcept]

Undo effect of setDynamicRange.

648

Class Documentation

9.130.3.16 setAllowedFormats()

void nvinfer1::ITensor::setAllowedFormats (

TensorFormats formats ) [inline], [noexcept]

Set allowed formats for this tensor. By default all formats are allowed. Shape tensors (for which isShapeTensor()

returns true) may only have row major linear format.

When running network on DLA and the build option kGPU FALLBACK is not specified, if DLA format(kCHW4

with Int8, kCHW4 with FP16, kCHW16 with FP16, kCHW32 with Int8) is set, the input format is treated as native

DLA format with line stride requirement. Input/output binding with these format should have correct layout during

inference.

Parameters

formats

A bitmask of TensorFormat values that are supported for this tensor.

See also

ITensor::getAllowedFormats()

TensorFormats

9.130.3.17 setBroadcastAcrossBatch()

void nvinfer1::ITensor::setBroadcastAcrossBatch (

bool broadcastAcrossBatch ) [inline], [noexcept]

Set whether to enable broadcast of tensor across the batch.

When a tensor is broadcast across a batch, it has the same value for every member in the batch. Memory is only

allocated once for the single member.

This method is only valid for network input tensors, since the flags of layer output tensors are inferred based on layer

inputs and parameters. If this state is modified for a tensor in the network, the states of all dependent tensors will be

recomputed. If the tensor is for an explicit batch network, then this function does nothing.

Warning

The broadcast flag is ignored when using explicit batch network mode.

Parameters

broadcastAcrossBatch

Whether to enable broadcast of tensor across the batch.

9.130 nvinfer1::ITensor Class Reference

649

See also

getBroadcastAcrossBatch()

9.130.3.18 setDimensionName()

void nvinfer1::ITensor::setDimensionName (

int32 t index,

char const ∗ name ) [inline], [noexcept]

Name a dimension of an input tensor.

Associate a runtime dimension of an input tensor with a symbolic name. Dimensions with the same non-empty name

must be equal at runtime. Knowing this equality for runtime dimensions may help the TensorRT optimizer. Both

runtime and build-time dimensions can be named.

For example, setDimensionName(0, ”n”) associates the symbolic name ”n” with the leading dimension.

This method copies the name string. If the function is called again, with the same index, it will overwrite the previous

name. If nullptr is passed as name, it will clear the name of the dimension.

Parameters

index

index of the dimension

name

of the dimension, as a pointer to a null-terminated character sequence.

Warning

The string name must be null-terminated, and be at most 4096 bytes including the terminator.

See also

getDimensionName()

9.130.3.19 setDimensions()

void nvinfer1::ITensor::setDimensions (

Dims dimensions ) [inline], [noexcept]

Set the dimensions of a tensor.

For a network input, the dimensions are assigned by the application. For a network output, the dimensions are computed

based on the layer parameters and the inputs to the layer. If a tensor size or a parameter is modified in the network, the

dimensions of all dependent tensors will be recomputed.

This call is only legal for network input tensors, since the dimensions of layer output tensors are inferred based on

layer inputs and parameters. The volume must be less than 2^∧31 elements.

650

Class Documentation

Parameters

dimensions

The dimensions of the tensor.

See also

getDimensions()

9.130.3.20 setDynamicRange()

bool nvinfer1::ITensor::setDynamicRange (

float min,

float max ) [inline], [noexcept]

Set dynamic range for the tensor.

Currently, only symmetric ranges are supported. Therefore, the larger of the absolute values of the provided bounds is

used.

Returns

Whether the dynamic range was set successfully.

Requires that min and max be finite, and min <= max.

9.130.3.21 setLocation()

void nvinfer1::ITensor::setLocation (

TensorLocation location ) [inline], [noexcept]

Set the storage location of a tensor.

Parameters

location

the location of tensor data

Only network input tensors for storing sequence lengths for RNNv2 are supported. Using host storage for layers that

do not support it will generate errors at build time.

See also

getLocation()

9.130 nvinfer1::ITensor Class Reference

651

9.130.3.22 setName()

void nvinfer1::ITensor::setName (

char const ∗ name ) [inline], [noexcept]

Set the tensor name.

For a network input, the name is assigned by the application. For tensors which are layer outputs, a default name is

assigned consisting of the layer name followed by the index of the output in brackets.

This method copies the name string.

Parameters

name

The name.

Warning

The string name must be null-terminated, and be at most 4096 bytes including the terminator.

See also

getName()

9.130.3.23 setType()

void nvinfer1::ITensor::setType (

DataType type ) [inline], [noexcept]

Set the data type of a tensor.

Parameters

type

The data type of the tensor.

The type is unchanged if the tensor is not a network input tensor, or marked as an output tensor or shape output tensor.

See also

getType()

9.130.4

Member Data Documentation

652

Class Documentation

9.130.4.1 mImpl

apiv::VTensor∗ nvinfer1::ITensor::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

9.131

nvinfer1::ITimingCache Class Reference

Class to handle tactic timing info collected from builder.

#include <NvInfer.h>

Inheritance diagram for nvinfer1::ITimingCache:

nvinfer1::INoCopy

nvinfer1::ITimingCache

Public Member Functions

• virtual ∼ITimingCache () noexcept=default

• nvinfer1::IHostMemory ∗ serialize () const noexcept

Serialize a timing cache to IHostMemory object.

• bool combine (ITimingCache const &inputCache, bool ignoreMismatch) noexcept

Combine input timing cache into local instance.

• bool reset () noexcept

Empty the timing cache.

Protected Attributes

• apiv::VTimingCache ∗ mImpl

Additional Inherited Members

9.131.1

Detailed Description

Class to handle tactic timing info collected from builder.

The timing cache is created or initialized by IBuilderConfig. It can be shared across builder instances to accelerate the

builder wallclock time.

See also

IBuilderConfig

9.131 nvinfer1::ITimingCache Class Reference

653

9.131.2

Constructor & Destructor Documentation

9.131.2.1 ∼ITimingCache()

virtual nvinfer1::ITimingCache::∼ITimingCache ( ) [virtual], [default], [noexcept]

9.131.3

Member Function Documentation

9.131.3.1 combine()

bool nvinfer1::ITimingCache::combine (

ITimingCache const & inputCache,

bool ignoreMismatch ) [inline], [noexcept]

Combine input timing cache into local instance.

This function allows combining entries in the input timing cache to local cache object.

Parameters

inputCache

The input timing cache.

ignoreMismatch

Whether or not to allow cache verification header mismatch.

Returns

True if combined successfully, false otherwise.

Append entries in input cache to local cache. Conflicting entries will be skipped The input cache must be generated by

a TensorRT build of exact same version, otherwise combine will be skipped and return false. ignoreMismatch must be

set to true if combining a timing cache created from a different device.

Warning

Combining caches generated from devices with different device properties may lead to func-

tional/performance bugs!

654

Class Documentation

9.131.3.2 reset()

bool nvinfer1::ITimingCache::reset ( ) [inline], [noexcept]

Empty the timing cache.

Returns

True if reset successfully, false otherwise.

9.131.3.3 serialize()

nvinfer1::IHostMemory ∗ nvinfer1::ITimingCache::serialize ( ) const [inline], [noexcept]

Serialize a timing cache to IHostMemory object.

This function allows serialization of current timing cache.

Returns

A pointer to a IHostMemory object that contains a serialized timing cache.

See also

IHostMemory

9.131.4

Member Data Documentation

9.131.4.1 mImpl

apiv::VTimingCache∗ nvinfer1::ITimingCache::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

9.132 nvinfer1::ITopKLayer Class Reference

655

9.132

nvinfer1::ITopKLayer Class Reference

Layer that represents a TopK reduction.

#include <NvInfer.h>

Inheritance diagram for nvinfer1::ITopKLayer:

nvinfer1::INoCopy

nvinfer1::ILayer

nvinfer1::ITopKLayer

Public Member Functions

• void setOperation (TopKOperation op) noexcept

Set the operation for the layer.

• TopKOperation getOperation () const noexcept

Get the operation for the layer.

• void setK (int32 t k) noexcept

Set the static k value for the layer.

• int32 t getK () const noexcept

Get the k value for the layer.

• void setReduceAxes (uint32 t reduceAxes) noexcept

Set which axes to reduce for the layer.

• uint32 t getReduceAxes () const noexcept

Get the axes to reduce for the layer.

• void setInput (int32 t index, ITensor &tensor) noexcept

Append or replace an input of this layer with a specific tensor.

Protected Member Functions

• virtual ∼ITopKLayer () noexcept=default

Protected Attributes

• apiv::VTopKLayer ∗ mImpl

9.132.1

Detailed Description

Layer that represents a TopK reduction.

This layer can accept both static and dynamic k. Static k can be set through the addTopK() API function, or accessed

using the getK() and setK() functions after layer creation. For dynamic k, use the setInput() method to pass in k as a

tensor with index 1, which overrides the static k value in calculations.

656

Class Documentation

Warning

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

9.132.2

Constructor & Destructor Documentation

9.132.2.1 ∼ITopKLayer()

virtual nvinfer1::ITopKLayer::∼ITopKLayer ( ) [protected], [virtual], [default], [noexcept]

9.132.3

Member Function Documentation

9.132.3.1 getK()

int32 t nvinfer1::ITopKLayer::getK ( ) const [inline], [noexcept]

Get the k value for the layer.

This function will return the static k value passed into addTopK(), or the value passed into setK().

If a second layer input is present and non-null, this function returns -1.

See also

setK()

9.132.3.2 getOperation()

TopKOperation nvinfer1::ITopKLayer::getOperation ( ) const [inline], [noexcept]

Get the operation for the layer.

See also

setOperation(), TopKOperation

9.132 nvinfer1::ITopKLayer Class Reference

657

9.132.3.3 getReduceAxes()

uint32 t nvinfer1::ITopKLayer::getReduceAxes ( ) const [inline], [noexcept]

Get the axes to reduce for the layer.

See also

setReduceAxes()

9.132.3.4 setInput()

void nvinfer1::ILayer::setInput (

int32 t index,

ITensor & tensor ) [inline], [noexcept]

Append or replace an input of this layer with a specific tensor.

Parameters

index

The index of the input to modify.

tensor

The new input tensor.

For a TopK layer, the values 0-1 are valid. The indices are as follows:

• 0: Input data tensor.

•

1: A scalar Int32 tensor containing a positive value corresponding to the number of top elements to retrieve. Val-

ues larger than 3840 will result in a runtime error. If provided, this will override the static k value in calculations.

9.132.3.5 setK()

void nvinfer1::ITopKLayer::setK (

int32 t k ) [inline], [noexcept]

Set the static k value for the layer.

Currently only values up to 3840 are supported.

If a second input to this layer has been set, it will be reset to null by this method.

See also

getK()

658

Class Documentation

9.132.3.6 setOperation()

void nvinfer1::ITopKLayer::setOperation (

TopKOperation op ) [inline], [noexcept]

Set the operation for the layer.

See also

getOperation(), TopKOperation

9.132.3.7 setReduceAxes()

void nvinfer1::ITopKLayer::setReduceAxes (

uint32 t reduceAxes ) [inline], [noexcept]

Set which axes to reduce for the layer.

See also

getReduceAxes()

9.132.4

Member Data Documentation

9.132.4.1 mImpl

apiv::VTopKLayer∗ nvinfer1::ITopKLayer::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

9.133

nvinfer1::ITripLimitLayer Class Reference

#include <NvInfer.h>

Inheritance diagram for nvinfer1::ITripLimitLayer:

nvinfer1::INoCopy

nvinfer1::ILayer

nvinfer1::ILoopBoundaryLayer

nvinfer1::ITripLimitLayer

9.133 nvinfer1::ITripLimitLayer Class Reference

659

Public Member Functions

• TripLimit getTripLimit () const noexcept

Protected Member Functions

• virtual ∼ITripLimitLayer () noexcept=default

Protected Attributes

• apiv::VTripLimitLayer ∗ mImpl

9.133.1

Constructor & Destructor Documentation

9.133.1.1 ∼ITripLimitLayer()

virtual nvinfer1::ITripLimitLayer::∼ITripLimitLayer ( ) [protected], [virtual], [default], [noexcept]

9.133.2

Member Function Documentation

9.133.2.1 getTripLimit()

TripLimit nvinfer1::ITripLimitLayer::getTripLimit ( ) const [inline], [noexcept]

9.133.3

Member Data Documentation

9.133.3.1 mImpl

apiv::VTripLimitLayer∗ nvinfer1::ITripLimitLayer::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

660

Class Documentation

9.134

nvuffparser::IUffParser Class Reference

Class used for parsing models described using the UFF format.

#include <NvUffParser.h>

Public Member Functions

• virtual bool registerInput (char const ∗inputName, nvinfer1::Dims inputDims, UffInputOrder inputOrder) noex-

cept=0

• virtual bool registerOutput (char const ∗outputName) noexcept=0

• virtual bool parse (char const ∗file, nvinfer1::INetworkDefinition &network, nvinfer1::DataType weights←↩

Type=nvinfer1::DataType::kFLOAT) noexcept=0

Parse a UFF file.

• virtual bool parseBuffer (char const ∗buffer, std::size t size, nvinfer1::INetworkDefinition &network,

nvinfer1::DataType weightsType=nvinfer1::DataType::kFLOAT) noexcept=0

Parse a UFF buffer, useful if the file already live in memory.

• virtual TRT DEPRECATED void destroy () noexcept=0

• virtual int32 t getUffRequiredVersionMajor () noexcept=0

Return Version Major of the UFF.

• virtual int32 t getUffRequiredVersionMinor () noexcept=0

Return Version Minor of the UFF.

• virtual int32 t getUffRequiredVersionPatch () noexcept=0

Return Patch Version of the UFF.

• virtual void setPluginNamespace (char const ∗libNamespace) noexcept=0

Set the namespace used to lookup and create plugins in the network.

• virtual ∼IUffParser () noexcept=default

• virtual void setErrorRecorder (nvinfer1::IErrorRecorder ∗recorder) noexcept=0

Set the ErrorRecorder for this interface.

• virtual nvinfer1::IErrorRecorder ∗ getErrorRecorder () const noexcept=0

get the ErrorRecorder assigned to this interface.

9.134.1

Detailed Description

Class used for parsing models described using the UFF format.

Warning

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

9.134.2

Constructor & Destructor Documentation

9.134 nvuffparser::IUffParser Class Reference

661

9.134.2.1 ∼IUffParser()

virtual nvuffparser::IUffParser::∼IUffParser ( ) [virtual], [default], [noexcept]

9.134.3

Member Function Documentation

9.134.3.1 destroy()

virtual TRT DEPRECATED void nvuffparser::IUffParser::destroy ( ) [pure virtual], [noexcept]

Deprecated Use delete instead. Deprecated in TRT 8.0.

9.134.3.2 getErrorRecorder()

virtual nvinfer1::IErrorRecorder ∗ nvuffparser::IUffParser::getErrorRecorder ( ) const [pure

virtual], [noexcept]

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A nullptr will be returned if setErrorRecorder has not

been called.

Returns

A pointer to the IErrorRecorder object that has been registered.

See also

setErrorRecorder()

9.134.3.3 getUffRequiredVersionMajor()

virtual int32 t nvuffparser::IUffParser::getUffRequiredVersionMajor ( ) [pure virtual], [noexcept]

Return Version Major of the UFF.

662

Class Documentation

9.134.3.4 getUffRequiredVersionMinor()

virtual int32 t nvuffparser::IUffParser::getUffRequiredVersionMinor ( ) [pure virtual], [noexcept]

Return Version Minor of the UFF.

9.134.3.5 getUffRequiredVersionPatch()

virtual int32 t nvuffparser::IUffParser::getUffRequiredVersionPatch ( ) [pure virtual], [noexcept]

Return Patch Version of the UFF.

9.134.3.6 parse()

virtual bool nvuffparser::IUffParser::parse (

char const ∗ file,

nvinfer1::INetworkDefinition & network,

nvinfer1::DataType weightsType = nvinfer1::DataType::kFLOAT ) [pure virtual], [noexcept]

Parse a UFF file.

Parameters

file

File name of the UFF file.

network

Network in which the UFFParser will fill the layers.

weightsType

The type on which the weights will transformed in.

9.134.3.7 parseBuffer()

virtual bool nvuffparser::IUffParser::parseBuffer (

char const ∗ buffer,

std::size t size,

nvinfer1::INetworkDefinition & network,

nvinfer1::DataType weightsType = nvinfer1::DataType::kFLOAT ) [pure virtual], [noexcept]

Parse a UFF buffer, useful if the file already live in memory.

Parameters

buffer

Buffer of the UFF file.

9.134 nvuffparser::IUffParser Class Reference

663

Parameters

size

Size of buffer of the UFF file.

network

Network in which the UFFParser will fill the layers.

weightsType

The type on which the weights will transformed in.

9.134.3.8 registerInput()

virtual bool nvuffparser::IUffParser::registerInput (

char const ∗ inputName,

nvinfer1::Dims inputDims,

UffInputOrder inputOrder ) [pure virtual], [noexcept]

Parameters

inputName

Input name.

inputDims

Input dimensions.

inputOrder

Input order on which the framework input was originally.

9.134.3.9 registerOutput()

virtual bool nvuffparser::IUffParser::registerOutput (

char const ∗ outputName ) [pure virtual], [noexcept]

Parameters

outputName

Output name.

9.134.3.10 setErrorRecorder()

virtual void nvuffparser::IUffParser::setErrorRecorder (

nvinfer1::IErrorRecorder ∗ recorder ) [pure virtual], [noexcept]

Set the ErrorRecorder for this interface.

664

Class Documentation

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function

will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder

with the interface, resulting in a call to decRefCount if a recorder has been registered.

If an error recorder is not set, messages will be sent to the global log stream.

Parameters

recorder

The error recorder to register with this interface.

See also

getErrorRecorder()

9.134.3.11 setPluginNamespace()

virtual void nvuffparser::IUffParser::setPluginNamespace (

char const ∗ libNamespace ) [pure virtual], [noexcept]

Set the namespace used to lookup and create plugins in the network.

The documentation for this class was generated from the following file:

• NvUffParser.h

9.135

nvinfer1::IUnaryLayer Class Reference

Layer that represents an unary operation.

#include <NvInfer.h>

Inheritance diagram for nvinfer1::IUnaryLayer:

nvinfer1::INoCopy

nvinfer1::ILayer

nvinfer1::IUnaryLayer

9.135 nvinfer1::IUnaryLayer Class Reference

665

Public Member Functions

• void setOperation (UnaryOperation op) noexcept

Set the unary operation for the layer.

• UnaryOperation getOperation () const noexcept

Get the unary operation for the layer.

Protected Member Functions

• virtual ∼IUnaryLayer () noexcept=default

Protected Attributes

• apiv::VUnaryLayer ∗ mImpl

9.135.1

Detailed Description

Layer that represents an unary operation.

Warning

Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

9.135.2

Constructor & Destructor Documentation

9.135.2.1 ∼IUnaryLayer()

virtual nvinfer1::IUnaryLayer::∼IUnaryLayer ( ) [protected], [virtual], [default], [noexcept]

9.135.3

Member Function Documentation

9.135.3.1 getOperation()

UnaryOperation nvinfer1::IUnaryLayer::getOperation ( ) const [inline], [noexcept]

Get the unary operation for the layer.

See also

setOperation(), UnaryOperation

666

Class Documentation

9.135.3.2 setOperation()

void nvinfer1::IUnaryLayer::setOperation (

UnaryOperation op ) [inline], [noexcept]

Set the unary operation for the layer.

When running this layer on DLA, only UnaryOperation::kABS is supported.

See also

getOperation(), UnaryOperation

9.135.4

Member Data Documentation

9.135.4.1 mImpl

apiv::VUnaryLayer∗ nvinfer1::IUnaryLayer::mImpl [protected]

The documentation for this class was generated from the following file:

• NvInfer.h

9.136

nvinfer1::plugin::NMSParameters Struct Reference

The NMSParameters are used by the BatchedNMSPlugin for performing the non max suppression operation over

boxes for object detection networks.

#include <NvInferPluginUtils.h>

Public Attributes

• bool shareLocation

• int32 t backgroundLabelId

• int32 t numClasses

• int32 t topK

• int32 t keepTopK

• float scoreThreshold

• float iouThreshold

• bool isNormalized

9.136.1

Detailed Description

The NMSParameters are used by the BatchedNMSPlugin for performing the non max suppression operation over

boxes for object detection networks.

9.136 nvinfer1::plugin::NMSParameters Struct Reference

667

Parameters

shareLocation

If set to true, the boxes inputs are shared across all classes. If set to false, the boxes input

should account for per class box data.

background←↩

Label ID for the background class. If there is no background class, set it as -1

LabelId

numClasses

Number of classes in the network.

topK

Number of bounding boxes to be fed into the NMS step.

keepTopK

Number of total bounding boxes to be kept per image after NMS step. Should be less than

or equal to the topK value.

scoreThreshold

Scalar threshold for score (low scoring boxes are removed).

iouThreshold

scalar threshold for IOU (new boxes that have high IOU overlap with previously selected

boxes are removed).

isNormalized

Set to false, if the box coordinates are not normalized, i.e. not in the range [0,1]. Defaults to

false.

9.136.2

Member Data Documentation

9.136.2.1 backgroundLabelId

int32 t nvinfer1::plugin::NMSParameters::backgroundLabelId

9.136.2.2 iouThreshold

float nvinfer1::plugin::NMSParameters::iouThreshold

9.136.2.3 isNormalized

bool nvinfer1::plugin::NMSParameters::isNormalized

9.136.2.4 keepTopK

int32 t nvinfer1::plugin::NMSParameters::keepTopK

9.138 nvinfer1::PluginField Class Reference

669

9.137.1.1 order

int32 t nvinfer1::Permutation::order[Dims::MAX DIMS]

The elements of the permutation. The permutation is applied as outputDimensionIndex = permutation.order[input←↩

DimensionIndex], so to permute from CHW order to HWC order, the required permutation is [1, 2, 0], and to permute

from HWC to CHW, the required permutation is [2, 0, 1].

The documentation for this struct was generated from the following file:

• NvInfer.h

9.138

nvinfer1::PluginField Class Reference

Structure containing plugin attribute field names and associated data This information can be parsed to decode neces-

sary plugin metadata.

#include <NvInferRuntimePlugin.h>

Public Member Functions

• PluginField (AsciiChar const ∗const name =nullptr, void const ∗const data =nullptr, PluginFieldType const

type =PluginFieldType::kUNKNOWN, int32 t const length =0) noexcept

Public Attributes

• AsciiChar const ∗ name

Plugin field attribute name.

• void const ∗ data

Plugin field attribute data.

• PluginFieldType type

Plugin field attribute type.

• int32 t length

Number of data entries in the Plugin attribute.

9.138.1

Detailed Description

Structure containing plugin attribute field names and associated data This information can be parsed to decode neces-

sary plugin metadata.

9.138.2

Constructor & Destructor Documentation

670

Class Documentation

9.138.2.1 PluginField()

nvinfer1::PluginField::PluginField (

AsciiChar const ∗const name = nullptr,

void const ∗const data = nullptr,

PluginFieldType const type = PluginFieldType::kUNKNOWN,

int32 t const length

= 0 ) [inline], [noexcept]

9.138.3

Member Data Documentation

9.138.3.1 data

void const∗ nvinfer1::PluginField::data

Plugin field attribute data.

9.138.3.2 length

int32 t nvinfer1::PluginField::length

Number of data entries in the Plugin attribute.

9.138.3.3 name

AsciiChar const∗ nvinfer1::PluginField::name

Plugin field attribute name.

9.138.3.4 type

PluginFieldType nvinfer1::PluginField::type

Plugin field attribute type.

See also

PluginFieldType

The documentation for this class was generated from the following file:

• NvInferRuntimePlugin.h

9.139 nvinfer1::PluginFieldCollection Struct Reference

671

9.139

nvinfer1::PluginFieldCollection Struct Reference

Plugin field collection struct.

#include <NvInferRuntimePlugin.h>

Public Attributes

• int32 t nbFields

Number of PluginField entries.

• PluginField const ∗ fields

Pointer to PluginField entries.

9.139.1

Detailed Description

Plugin field collection struct.

9.139.2

Member Data Documentation

9.139.2.1 fields

PluginField const∗ nvinfer1::PluginFieldCollection::fields

Pointer to PluginField entries.

9.139.2.2 nbFields

int32 t nvinfer1::PluginFieldCollection::nbFields

Number of PluginField entries.

The documentation for this struct was generated from the following file:

• NvInferRuntimePlugin.h

672

Class Documentation

9.140

nvinfer1::PluginRegistrar< T > Class Template Reference

Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded.

This static object will register all creators available in the library to the registry.

#include <NvInferRuntime.h>

Public Member Functions

• PluginRegistrar ()

9.140.1

Detailed Description

template<typename T>

class nvinfer1::PluginRegistrar< T >

Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded.

This static object will register all creators available in the library to the registry.

Warning

Statically registering plugins should be avoided in the automotive safety context as the application developer

should first register an error recorder with the plugin registry via IPluginRegistry::setErrorRecorder() before

using IPluginRegistry::registerCreator() or other methods.

9.140.2

Constructor & Destructor Documentation

9.140.2.1 PluginRegistrar()

template<typename T >

nvinfer1::PluginRegistrar< T >::PluginRegistrar ( ) [inline]

The documentation for this class was generated from the following file:

• NvInferRuntime.h

9.141

nvinfer1::safe::PluginRegistrar< T > Class Template Reference

Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded.

This static object will register all creators available in the library to the registry.

#include <NvInferSafeRuntime.h>

9.142 nvinfer1::PluginTensorDesc Struct Reference

673

Public Member Functions

• PluginRegistrar ()

9.141.1

Detailed Description

template<typename T>

class nvinfer1::safe::PluginRegistrar< T >

Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded.

This static object will register all creators available in the library to the registry.

Warning

Statically registering plugins should be avoided in the automotive safety context as the application developer

should first register an error recorder with the plugin registry via IPluginRegistry::setErrorRecorder() before

using IPluginRegistry::registerCreator() or other methods.

9.141.2

Constructor & Destructor Documentation

9.141.2.1 PluginRegistrar()

template<typename T >

nvinfer1::safe::PluginRegistrar< T >::PluginRegistrar ( ) [inline]

The documentation for this class was generated from the following file:

• NvInferSafeRuntime.h

9.142

nvinfer1::PluginTensorDesc Struct Reference

Fields that a plugin might see for an input or output.

#include <NvInferRuntimePlugin.h>

Public Attributes

• Dims dims

Dimensions.

• DataType type

• TensorFormat format

Tensor format.

• float scale

Scale for INT8 data type.

9.143 PluginVersion Struct Reference

675

Warning

DataType:kBOOL and DataType::kUINT8 are not supported.

The documentation for this struct was generated from the following file:

• NvInferRuntimePlugin.h

9.143

PluginVersion Struct Reference

Definition of plugin versions.

#include <NvInferRuntimePlugin.h>

9.143.1

Detailed Description

Definition of plugin versions.

Tag for plug-in versions. Used in upper byte of getTensorRTVersion().

The documentation for this struct was generated from the following file:

• NvInferRuntimePlugin.h

9.144

nvinfer1::plugin::PriorBoxParameters Struct Reference

The PriorBox plugin layer generates the prior boxes of designated sizes and aspect ratios across all dimensions (H x

W). PriorBoxParameters defines a set of parameters for creating the PriorBox plugin layer. It contains:

#include <NvInferPluginUtils.h>

Public Attributes

• float ∗ minSize

• float ∗ maxSize

• float ∗ aspectRatios

• int32 t numMinSize

• int32 t numMaxSize

• int32 t numAspectRatios

• bool flip

• bool clip

• float variance [4]

• int32 t imgH

• int32 t imgW

• float stepH

• float stepW

• float offset

9.144.1

Detailed Description

The PriorBox plugin layer generates the prior boxes of designated sizes and aspect ratios across all dimensions (H x

W). PriorBoxParameters defines a set of parameters for creating the PriorBox plugin layer. It contains:

676

Class Documentation

Parameters

minSize

Minimum box size in pixels. Can not be nullptr.

maxSize

Maximum box size in pixels. Can be nullptr.

aspectRatios

Aspect ratios of the boxes. Can be nullptr.

numMinSize

Number of elements in minSize. Must be larger than 0.

numMaxSize

Number of elements in maxSize. Can be 0 or same as numMinSize.

numAspectRatios

Number of elements in aspectRatios. Can be 0.

flip

If true, will flip each aspect ratio. For example, if there is an aspect ratio ”r”, the aspect ratio

”1.0/r” will be generated as well.

clip

If true, will clip the prior so that it is within [0,1].

variance

Variance for adjusting the prior boxes.

imgH

Image height. If 0, then the H dimension of the data tensor will be used.

imgW

Image width. If 0, then the W dimension of the data tensor will be used.

stepH

Step in H. If 0, then (float)imgH/h will be used where h is the H dimension of the 1st input

tensor.

stepW

Step in W. If 0, then (float)imgW/w will be used where w is the W dimension of the 1st input

tensor.

offset

Offset to the top left corner of each cell.

9.144.2

Member Data Documentation

9.144.2.1 aspectRatios

float ∗ nvinfer1::plugin::PriorBoxParameters::aspectRatios

9.144.2.2 clip

bool nvinfer1::plugin::PriorBoxParameters::clip

9.144.2.3 flip

bool nvinfer1::plugin::PriorBoxParameters::flip

678

Class Documentation

9.144.2.11 offset

float nvinfer1::plugin::PriorBoxParameters::offset

9.144.2.12 stepH

float nvinfer1::plugin::PriorBoxParameters::stepH

9.144.2.13 stepW

float nvinfer1::plugin::PriorBoxParameters::stepW

9.144.2.14 variance

float nvinfer1::plugin::PriorBoxParameters::variance[4]

The documentation for this struct was generated from the following file:

• NvInferPluginUtils.h

9.145

nvinfer1::plugin::Quadruple Struct Reference

The Permute plugin layer permutes the input tensor by changing the memory order of the data. Quadruple defines a

structure that contains an array of 4 integers. They can represent the permute orders or the strides in each dimension.

#include <NvInferPluginUtils.h>

Public Attributes

• int32 t data [4]

9.145.1

Detailed Description

The Permute plugin layer permutes the input tensor by changing the memory order of the data. Quadruple defines a

structure that contains an array of 4 integers. They can represent the permute orders or the strides in each dimension.

9.146 nvinfer1::plugin::RegionParameters Struct Reference

679

9.145.2

Member Data Documentation

9.145.2.1 data

int32 t nvinfer1::plugin::Quadruple::data[4]

The documentation for this struct was generated from the following file:

• NvInferPluginUtils.h

9.146

nvinfer1::plugin::RegionParameters Struct Reference

The Region plugin layer performs region proposal calculation: generate 5 bounding boxes per cell (for yolo9000,

generate 3 bounding boxes per cell). For each box, calculating its probablities of objects detections from 80 pre-

defined classifications (yolo9000 has 9418 pre-defined classifications, and these 9418 items are organized as work-tree

structure). RegionParameters defines a set of parameters for creating the Region plugin layer.

#include <NvInferPluginUtils.h>

Public Attributes

• int32 t num

• int32 t coords

• int32 t classes

• softmaxTree ∗ smTree

9.146.1

Detailed Description

The Region plugin layer performs region proposal calculation: generate 5 bounding boxes per cell (for yolo9000,

generate 3 bounding boxes per cell). For each box, calculating its probablities of objects detections from 80 pre-

defined classifications (yolo9000 has 9418 pre-defined classifications, and these 9418 items are organized as work-tree

structure). RegionParameters defines a set of parameters for creating the Region plugin layer.

Parameters

num

Number of predicted bounding box for each grid cell.

coords

Number of coordinates for a bounding box.

classes

Number of classifications to be predicted.

smTree

Helping structure to do softmax on confidence scores.

680

Class Documentation

9.146.2

Member Data Documentation

9.146.2.1 classes

int32 t nvinfer1::plugin::RegionParameters::classes

9.146.2.2 coords

int32 t nvinfer1::plugin::RegionParameters::coords

9.146.2.3 num

int32 t nvinfer1::plugin::RegionParameters::num

9.146.2.4 smTree

softmaxTree∗ nvinfer1::plugin::RegionParameters::smTree

The documentation for this struct was generated from the following file:

• NvInferPluginUtils.h

9.147

nvinfer1::plugin::RPROIParams Struct Reference

RPROIParams is used to create the RPROIPlugin instance. It contains:

#include <NvInferPluginUtils.h>

Public Attributes

• int32 t poolingH

• int32 t poolingW

• int32 t featureStride

• int32 t preNmsTop

• int32 t nmsMaxOut

• int32 t anchorsRatioCount

• int32 t anchorsScaleCount

• float iouThreshold

• float minBoxSize

• float spatialScale

9.147.1

Detailed Description

RPROIParams is used to create the RPROIPlugin instance. It contains:

9.147 nvinfer1::plugin::RPROIParams Struct Reference

681

Parameters

poolingH

Height of the output in pixels after ROI pooling on feature map.

poolingW

Width of the output in pixels after ROI pooling on feature map.

featureStride

Feature stride; ratio of input image size to feature map size. Assuming that max pooling

layers in the neural network use square filters.

preNmsTop

Number of proposals to keep before applying NMS.

nmsMaxOut

Number of remaining proposals after applying NMS.

anchorsRatioCount

Number of anchor box ratios.

anchorsScaleCount

Number of anchor box scales.

iouThreshold

IoU (Intersection over Union) threshold used for the NMS step.

minBoxSize

Minimum allowed bounding box size before scaling, used for anchor box calculation.

spatialScale

Spatial scale between the input image and the last feature map.

9.147.2

Member Data Documentation

9.147.2.1 anchorsRatioCount

int32 t nvinfer1::plugin::RPROIParams::anchorsRatioCount

9.147.2.2 anchorsScaleCount

int32 t nvinfer1::plugin::RPROIParams::anchorsScaleCount

9.147.2.3 featureStride

int32 t nvinfer1::plugin::RPROIParams::featureStride

9.147.2.4 iouThreshold

float nvinfer1::plugin::RPROIParams::iouThreshold

682

Class Documentation

9.147.2.5 minBoxSize

float nvinfer1::plugin::RPROIParams::minBoxSize

9.147.2.6 nmsMaxOut

int32 t nvinfer1::plugin::RPROIParams::nmsMaxOut

9.147.2.7 poolingH

int32 t nvinfer1::plugin::RPROIParams::poolingH

9.147.2.8 poolingW

int32 t nvinfer1::plugin::RPROIParams::poolingW

9.147.2.9 preNmsTop

int32 t nvinfer1::plugin::RPROIParams::preNmsTop

9.147.2.10 spatialScale

float nvinfer1::plugin::RPROIParams::spatialScale

The documentation for this struct was generated from the following file:

• NvInferPluginUtils.h

9.148

nvinfer1::safe::RuntimeErrorInformation Struct Reference

Space to record information about runtime errors.

#include <NvInferSafeRuntime.h>

9.149 nvinfer1::plugin::softmaxTree Struct Reference

683

Public Attributes

• uint64 t bitMask

Each bit represent a RuntimeErrorType has occured during kernel execution.

9.148.1

Detailed Description

Space to record information about runtime errors.

kNAN CONSUMED errors occur when NAN values are stored in an INT8 quantized datatype. kINF CONSUMED

errors occur when +-INF values are stored in an INT8 quantized datatype. kGATHER OOB errors occur when gather

index tensor contains value is out side of data tensor kSCATTER OOB and kSCATTER RACE are reserved for future

use

Mark RuntimeErrorType that occur during asynchronous kernel execution

9.148.2

Member Data Documentation

9.148.2.1 bitMask

uint64 t nvinfer1::safe::RuntimeErrorInformation::bitMask

Each bit represent a RuntimeErrorType has occured during kernel execution.

The documentation for this struct was generated from the following file:

• NvInferSafeRuntime.h

9.149

nvinfer1::plugin::softmaxTree Struct Reference

When performing yolo9000, softmaxTree is helping to do softmax on confidence scores, for element to get the precise

classification through word-tree structured classification definition.

#include <NvInferPluginUtils.h>

Public Attributes

• int32 t ∗ leaf

• int32 t n

• int32 t ∗ parent

• int32 t ∗ child

• int32 t ∗ group

• char ∗∗ name

• int32 t groups

• int32 t ∗ groupSize

• int32 t ∗ groupOffset

9.150 nvinfer1::Weights Class Reference

685

9.149.2.6 leaf

int32 t∗ nvinfer1::plugin::softmaxTree::leaf

9.149.2.7 n

int32 t nvinfer1::plugin::softmaxTree::n

9.149.2.8 name

char∗∗ nvinfer1::plugin::softmaxTree::name

9.149.2.9 parent

int32 t∗ nvinfer1::plugin::softmaxTree::parent

The documentation for this struct was generated from the following file:

• NvInferPluginUtils.h

9.150

nvinfer1::Weights Class Reference

An array of weights used as a layer parameter.

#include <NvInferRuntime.h>

Public Attributes

• DataType type

The type of the weights.

• void const ∗ values

The weight values, in a contiguous array.

• int64 t count

The number of weights in the array.

686

Class Documentation

9.150.1

Detailed Description

An array of weights used as a layer parameter.

When using the DLA, the cumulative size of all Weights used in a network must be less than 512MB in size. If the build

option kGPU FALLBACK is specified, then multiple DLA sub-networks may be generated from the single original

network.

The weights are held by reference until the engine has been built. Therefore the data referenced by values field

should be preserved until the build is complete.

The term ”empty weights” refers to Weights with weight coefficients ( count == 0 and values == nullptr).

9.150.2

Member Data Documentation

9.150.2.1 count

int64 t nvinfer1::Weights::count

The number of weights in the array.

9.150.2.2 type

DataType nvinfer1::Weights::type

The type of the weights.

9.150.2.3 values

void const∗ nvinfer1::Weights::values

The weight values, in a contiguous array.

The documentation for this class was generated from the following file:

• NvInferRuntime.h

Chapter 10

File Documentation

10.1

NvCaffeParser.h File Reference

#include "NvInfer.h"

Classes

• class nvcaffeparser1::IBlobNameToTensor

Object used to store and query Tensors after they have been extracted from a Caffe model using the ICaffeParser.

• class nvcaffeparser1::IBinaryProtoBlob

Object used to store and query data extracted from a binaryproto file using the ICaffeParser.

• class nvcaffeparser1::IPluginFactoryV2

Plugin factory used to configure plugins.

• class nvcaffeparser1::ICaffeParser

Class used for parsing Caffe models.

Namespaces

• namespace nvcaffeparser1

The TensorRT Caffe parser API namespace.

Functions

• ICaffeParser ∗ nvcaffeparser1::createCaffeParser () noexcept

Creates a ICaffeParser object.

• void nvcaffeparser1::shutdownProtobufLibrary () noexcept

Shuts down protocol buffers library.

688

File Documentation

10.1.1

Detailed Description

This is the API for the Caffe Parser

10.2

NvCaffeParser.h

Go to the documentation of this file.

1 /*

* SPDX-License-Identifier: LicenseRef-NvidiaProprietary

* NVIDIA CORPORATION, its affiliates and licensors retain all intellectual

* property and proprietary rights in and to this material, related

* documentation and any modifications thereto. Any use, reproduction,

* disclosure or distribution of this material and related documentation

* without an express license agreement from NVIDIA CORPORATION or

* its affiliates is strictly prohibited.

13 #ifndef NV CAFFE PARSER H

14 #define NV CAFFE PARSER H

16 #include "NvInfer.h"

29 namespace nvcaffeparser1

30 {

43 class IBlobNameToTensor

44 {

45 public:

virtual nvinfer1::ITensor_* find(char const* name) const noexcept = 0;

54 protected:

virtual ˜IBlobNameToTensor() {}

56 };

67 class IBinaryProtoBlob

68 {

69 public:

virtual void const_* getData() noexcept = 0;

virtual nvinfer1::Dims4 getDimensions() noexcept = 0;

virtual nvinfer1::DataType getDataType() noexcept = 0;

TRT DEPRECATED virtual void destroy() noexcept = 0;

virtual ˜IBinaryProtoBlob() noexcept = default;

80 };

87 class IPluginFactoryV2

88 {

89 public:

virtual bool isPluginV2(char const* layerName) noexcept = 0;

105

virtual nvinfer1::IPluginV2_* createPlugin(char const* layerName, nvinfer1::Weights const* weights,

106

int32 t nbWeights, char const* libNamespace = "") noexcept = 0;

107

108

virtual ˜IPluginFactoryV2() noexcept = default;

109 };

119 class ICaffeParser

120 {

121 public:

135

virtual IBlobNameToTensor const_* parse(char const* deploy, char const* model,

nvinfer1::INetworkDefinition& network,

136

nvinfer1::DataType weightType) noexcept = 0;

137

153

virtual IBlobNameToTensor const_* parseBuffers(uint8 t const* deployBuffer, std::size t deployLength,

154

uint8 t const* modelBuffer, std::size t modelLength, nvinfer1::INetworkDefinition& network,

155

nvinfer1::DataType weightType) noexcept = 0;

156

169

virtual IBinaryProtoBlob_* parseBinaryProto(char const* fileName) noexcept = 0;

170

178

virtual void setProtobufBufferSize(size t size) noexcept = 0;

179

187

TRT DEPRECATED virtual void destroy() noexcept = 0;

10.3 NvInfer.h File Reference

689

188

194

virtual void setPluginFactoryV2(IPluginFactoryV2* factory) noexcept = 0;

195

199

virtual void setPluginNamespace(char const* libNamespace) noexcept = 0;

200

201

virtual ˜ICaffeParser() noexcept = default;

202

203 public:

218

virtual void setErrorRecorder(nvinfer1::IErrorRecorder* recorder) noexcept = 0;

219

230

virtual nvinfer1::IErrorRecorder* getErrorRecorder() const noexcept = 0;

231 };

232

243 TENSORRTAPI ICaffeParser_* createCaffeParser() noexcept;

244

250 TENSORRTAPI void shutdownProtobufLibrary() noexcept;

251 } // namespace nvcaffeparser1

252

257 extern "C" TENSORRTAPI void* createNvCaffeParser INTERNAL() noexcept;

258 #endif

10.3

NvInfer.h File Reference

#include "NvInferLegacyDims.h"

#include "NvInferRuntime.h"

Classes

• struct nvinfer1::impl::EnumMaxImpl< ActivationType >

• class nvinfer1::ITensor

A tensor in a network definition.

• class nvinfer1::ILayer

Base class for all layer classes in a network definition.

• struct nvinfer1::impl::EnumMaxImpl< PaddingMode >

• class nvinfer1::IConvolutionLayer

A convolution layer in a network definition.

• class nvinfer1::IFullyConnectedLayer

A fully connected layer in a network definition. This layer expects an input tensor of three or more non-batch dimensions.

The input is automatically reshaped into an MxV tensor X, where V is a product of the last three dimensions and M is a

product of the remaining dimensions (where the product over 0 dimensions is defined as 1). For example:

• class nvinfer1::IActivationLayer

An Activation layer in a network definition.

• struct nvinfer1::impl::EnumMaxImpl< PoolingType >

• class nvinfer1::IPoolingLayer

A Pooling layer in a network definition.

• class nvinfer1::ILRNLayer

A LRN layer in a network definition.

• class nvinfer1::IScaleLayer

A Scale layer in a network definition.

• class nvinfer1::ISoftMaxLayer

A Softmax layer in a network definition.

• class nvinfer1::IConcatenationLayer

A concatenation layer in a network definition.

690

File Documentation

•

class nvinfer1::IDeconvolutionLayer

A deconvolution layer in a network definition.

•

struct nvinfer1::impl::EnumMaxImpl< ElementWiseOperation >

•

class nvinfer1::IElementWiseLayer

A elementwise layer in a network definition.

•

class nvinfer1::IGatherLayer

A Gather layer in a network definition. Supports several kinds of gathering.

•

class nvinfer1::IRNNv2Layer

An RNN layer in a network definition, version 2.

•

class nvinfer1::IPluginV2Layer

Layer type for pluginV2.

•

class nvinfer1::IUnaryLayer

Layer that represents an unary operation.

•

class nvinfer1::IReduceLayer

Layer that represents a reduction across a non-bool tensor.

•

class nvinfer1::IPaddingLayer

Layer that represents a padding operation.

•

struct nvinfer1::Permutation

•

class nvinfer1::IShuffleLayer

Layer type for shuffling data.

•

class nvinfer1::ISliceLayer

Slices an input tensor into an output tensor based on the offset and strides.

•

class nvinfer1::IShapeLayer

Layer type for getting shape of a tensor.

•

class nvinfer1::ITopKLayer

Layer that represents a TopK reduction.

•

class nvinfer1::IMatrixMultiplyLayer

Layer that represents a Matrix Multiplication.

•

class nvinfer1::INonZeroLayer

•

class nvinfer1::IRaggedSoftMaxLayer

A RaggedSoftmax layer in a network definition.

•

class nvinfer1::IIdentityLayer

A layer that represents the identity function.

•

class nvinfer1::ICastLayer

A cast layer in a network.

•

class nvinfer1::IConstantLayer

Layer that represents a constant value.

•

class nvinfer1::IParametricReLULayer

Layer that represents a parametric ReLU operation.

•

struct nvinfer1::impl::EnumMaxImpl< InterpolationMode >

•

struct nvinfer1::impl::EnumMaxImpl< ResizeCoordinateTransformation >

•

struct nvinfer1::impl::EnumMaxImpl< ResizeSelector >

•

struct nvinfer1::impl::EnumMaxImpl< ResizeRoundMode >

•

class nvinfer1::IResizeLayer

A resize layer in a network definition.

•

class nvinfer1::ILoopBoundaryLayer

•

class nvinfer1::IIfConditionalBoundaryLayer

•

class nvinfer1::IConditionLayer

10.3 NvInfer.h File Reference

691

•

class nvinfer1::IIfConditionalOutputLayer

•

class nvinfer1::IIfConditionalInputLayer

•

class nvinfer1::IIfConditional

•

class nvinfer1::IRecurrenceLayer

•

class nvinfer1::ILoopOutputLayer

•

class nvinfer1::ITripLimitLayer

•

class nvinfer1::IIteratorLayer

•

class nvinfer1::ILoop

•

class nvinfer1::ISelectLayer

•

class nvinfer1::IAssertionLayer

An assertion layer in a network.

•

class nvinfer1::IFillLayer

Generate an output tensor with specified mode.

•

class nvinfer1::IQuantizeLayer

A Quantize layer in a network definition.

•

class nvinfer1::IDequantizeLayer

A Dequantize layer in a network definition.

•

class nvinfer1::IEinsumLayer

An Einsum layer in a network.

•

class nvinfer1::IScatterLayer

A scatter layer in a network definition. Supports several kinds of scattering.

•

class nvinfer1::IOneHotLayer

A OneHot layer in a network definition.

•

class nvinfer1::IGridSampleLayer

A GridSample layer in a network definition.

•

class nvinfer1::INMSLayer

A non-maximum suppression layer in a network definition.

•

class nvinfer1::IReverseSequenceLayer

A ReverseSequence layer in a network definition.

•

class nvinfer1::INormalizationLayer

A normalization layer in a network definition.

•

class nvinfer1::INetworkDefinition

A network definition for input to the builder.

•

class nvinfer1::IInt8Calibrator

Application-implemented interface for calibration.

•

class nvinfer1::IInt8EntropyCalibrator

•

class nvinfer1::IInt8EntropyCalibrator2

•

class nvinfer1::IInt8MinMaxCalibrator

•

class nvinfer1::IInt8LegacyCalibrator

•

class nvinfer1::IAlgorithmIOInfo

Carries information about input or output of the algorithm. IAlgorithmIOInfo for all the input and output along

with IAlgorithmVariant denotes the variation of algorithm and can be used to select or reproduce an algorithm using

IAlgorithmSelector::selectAlgorithms().

•

class nvinfer1::IAlgorithmVariant

provides a unique 128-bit identifier, which along with the input and output information denotes the variation of algorithm

and can be used to select or reproduce an algorithm, using IAlgorithmSelector::selectAlgorithms()

•

class nvinfer1::IAlgorithmContext

Describes the context and requirements, that could be fulfilled by one or more instances of IAlgorithm.

•

class nvinfer1::IAlgorithm

692

File Documentation

Describes a variation of execution of a layer. An algorithm is represented by IAlgorithmVariant and the IAlgorithmIOInfo

for each of its inputs and outputs. An algorithm can be selected or reproduced using AlgorithmSelector::select←↩

Algorithms().”.

• class nvinfer1::IAlgorithmSelector

Interface implemented by application for selecting and reporting algorithms of a layer provided by the builder.

• class nvinfer1::ITimingCache

Class to handle tactic timing info collected from builder.

• struct nvinfer1::impl::EnumMaxImpl< PreviewFeature >

• struct nvinfer1::impl::EnumMaxImpl< HardwareCompatibilityLevel >

• class nvinfer1::IBuilderConfig

Holds properties for configuring a builder to produce an engine.

• class nvinfer1::IBuilder

Builds an engine from a network definition.

Namespaces

• namespace nvinfer1

The TensorRT API version 1 namespace.

• namespace nvinfer1::impl

• namespace nvinfer1::safe

The safety subset of TensorRT's API version 1 namespace.

Typedefs

• using nvinfer1::TensorFormats = uint32 t

It is capable of representing one or more TensorFormat by binary OR operations, e.g., 1U << TensorFormat::kCHW4 |

1U << TensorFormat::kCHW32.

• using nvinfer1::SliceMode = SampleMode

• using nvinfer1::ResizeMode = InterpolationMode

• using nvinfer1::QuantizationFlags = uint32 t

Represents one or more QuantizationFlag values using binary OR operations.

• using nvinfer1::BuilderFlags = uint32 t

Represents one or more BuilderFlag values using binary OR operations, e.g., 1U << BuilderFlag::kFP16 | 1U <<

BuilderFlag::kDEBUG.

• using nvinfer1::NetworkDefinitionCreationFlags = uint32 t

Represents one or more NetworkDefinitionCreationFlag flags using binary OR operations.

e.g., 1U

NetworkDefinitionCreationFlag::kEXPLICIT BATCH.

Enumerations

• enum class nvinfer1::LayerType : int32 t {

nvinfer1::kCONVOLUTION = 0 , nvinfer1::kFULLY CONNECTED = 1 , nvinfer1::kACTIVATION = 2 ,

nvinfer1::kPOOLING = 3 ,

nvinfer1::kLRN = 4 , nvinfer1::kSCALE = 5 , nvinfer1::kSOFTMAX = 6 , nvinfer1::kDECONVOLUTION = 7

nvinfer1::kCONCATENATION = 8 , nvinfer1::kELEMENTWISE = 9 , nvinfer1::kPLUGIN = 10 ,

nvinfer1::kUNARY = 11 ,

10.3 NvInfer.h File Reference

693

nvinfer1::kPADDING = 12 , nvinfer1::kSHUFFLE = 13 , nvinfer1::kREDUCE = 14 , nvinfer1::kTOPK = 15 ,

nvinfer1::kGATHER = 16 , nvinfer1::kMATRIX MULTIPLY = 17 , nvinfer1::kRAGGED SOFTMAX = 18 ,

nvinfer1::kCONSTANT = 19 ,

nvinfer1::kRNN V2 = 20 , nvinfer1::kIDENTITY = 21 , nvinfer1::kPLUGIN V2 = 22 , nvinfer1::kSLICE = 23 ,

nvinfer1::kSHAPE = 24 , nvinfer1::kPARAMETRIC RELU = 25 , nvinfer1::kRESIZE = 26 , nvinfer1::kTRIP LIMIT

= 27 ,

nvinfer1::kRECURRENCE = 28 , nvinfer1::kITERATOR = 29 , nvinfer1::kLOOP OUTPUT = 30 ,

nvinfer1::kSELECT = 31 ,

nvinfer1::kFILL = 32 , nvinfer1::kQUANTIZE = 33 , nvinfer1::kDEQUANTIZE = 34 , nvinfer1::kCONDITION

= 35 ,

nvinfer1::kCONDITIONAL INPUT = 36 , nvinfer1::kCONDITIONAL OUTPUT = 37 , nvinfer1::kSCATTER

= 38 , nvinfer1::kEINSUM = 39 ,

nvinfer1::kASSERTION = 40 , nvinfer1::kONE HOT = 41 , nvinfer1::kNON ZERO = 42 , nvinfer1::kGRID SAMPLE

= 43 ,

nvinfer1::kNMS = 44 , nvinfer1::kREVERSE SEQUENCE = 45 , nvinfer1::kNORMALIZATION = 46 ,

nvinfer1::kCAST = 47 }

The type values of layer classes.

•

enum class nvinfer1::ActivationType : int32 t {

nvinfer1::kRELU = 0 , nvinfer1::kSIGMOID = 1 , nvinfer1::kTANH = 2 , nvinfer1::kLEAKY RELU = 3 ,

nvinfer1::kELU = 4 , nvinfer1::kSELU = 5 , nvinfer1::kSOFTSIGN = 6 , nvinfer1::kSOFTPLUS = 7 ,

nvinfer1::kCLIP = 8 , nvinfer1::kHARD SIGMOID = 9 , nvinfer1::kSCALED TANH = 10 , nvinfer1::kTHRESHOLDED RELU

= 11 }

Enumerates the types of activation to perform in an activation layer.

•

enum class nvinfer1::PaddingMode : int32 t {

nvinfer1::kEXPLICIT ROUND DOWN = 0 , nvinfer1::kEXPLICIT ROUND UP = 1 , nvinfer1::kSAME UPPER

= 2 , nvinfer1::kSAME LOWER = 3 ,

nvinfer1::kCAFFE ROUND DOWN = 4 , nvinfer1::kCAFFE ROUND UP = 5 }

Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode takes

precedence if setPaddingMode() and setPrePadding() are also used.

•

enum class nvinfer1::PoolingType :

int32 t { nvinfer1::kMAX = 0 , nvinfer1::kAVERAGE = 1 ,

nvinfer1::kMAX AVERAGE BLEND = 2 }

The type of pooling to perform in a pooling layer.

•

enum class nvinfer1::ScaleMode : int32 t { nvinfer1::kUNIFORM = 0 , nvinfer1::kCHANNEL = 1 ,

nvinfer1::kELEMENTWISE = 2 }

Controls how shift, scale and power are applied in a Scale layer.

•

enum class nvinfer1::ElementWiseOperation : int32 t {

nvinfer1::kSUM = 0 , nvinfer1::kPROD = 1 , nvinfer1::kMAX = 2 , nvinfer1::kMIN = 3 ,

nvinfer1::kSUB = 4 , nvinfer1::kDIV = 5 , nvinfer1::kPOW = 6 , nvinfer1::kFLOOR DIV = 7 ,

nvinfer1::kAND = 8 , nvinfer1::kOR = 9 , nvinfer1::kXOR = 10 , nvinfer1::kEQUAL = 11 ,

nvinfer1::kGREATER = 12 , nvinfer1::kLESS = 13 }

Enumerates the binary operations that may be performed by an ElementWise layer.

•

enum class nvinfer1::GatherMode : int32 t { nvinfer1::kDEFAULT = 0 , nvinfer1::kELEMENT = 1 ,

nvinfer1::kND = 2 }

Control form of IGatherLayer.

•

enum class nvinfer1::RNNOperation : int32 t { nvinfer1::kRELU = 0 , nvinfer1::kTANH = 1 , nvinfer1::kLSTM

= 2 , nvinfer1::kGRU = 3 }

Enumerates the RNN operations that may be performed by an RNN layer.

•

enum class nvinfer1::RNNDirection : int32 t { nvinfer1::kUNIDIRECTION = 0 , nvinfer1::kBIDIRECTION =

Enumerates the RNN direction that may be performed by an RNN layer.

•

enum class nvinfer1::RNNInputMode : int32 t { nvinfer1::kLINEAR = 0 , nvinfer1::kSKIP = 1 }

694

File Documentation

Enumerates the RNN input modes that may occur with an RNN layer.

•

enum class nvinfer1::RNNGateType : int32 t {

nvinfer1::kINPUT = 0 , nvinfer1::kOUTPUT = 1 , nvinfer1::kFORGET = 2 , nvinfer1::kUPDATE = 3 ,

nvinfer1::kRESET = 4 , nvinfer1::kCELL = 5 , nvinfer1::kHIDDEN = 6 }

Identifies an individual gate within an RNN cell.

•

enum class nvinfer1::UnaryOperation : int32 t {

nvinfer1::kEXP = 0 , nvinfer1::kLOG = 1 , nvinfer1::kSQRT = 2 , nvinfer1::kRECIP = 3 ,

nvinfer1::kABS = 4 , nvinfer1::kNEG = 5 , nvinfer1::kSIN = 6 , nvinfer1::kCOS = 7 ,

nvinfer1::kTAN = 8 , nvinfer1::kSINH = 9 , nvinfer1::kCOSH = 10 , nvinfer1::kASIN = 11 ,

nvinfer1::kACOS = 12 , nvinfer1::kATAN = 13 , nvinfer1::kASINH = 14 , nvinfer1::kACOSH = 15 ,

nvinfer1::kATANH = 16 , nvinfer1::kCEIL = 17 , nvinfer1::kFLOOR = 18 , nvinfer1::kERF = 19 ,

nvinfer1::kNOT = 20 , nvinfer1::kSIGN = 21 , nvinfer1::kROUND = 22 , nvinfer1::kISINF = 23 }

Enumerates the unary operations that may be performed by a Unary layer.

•

enum class nvinfer1::ReduceOperation : int32 t {

nvinfer1::kSUM = 0 , nvinfer1::kPROD = 1 , nvinfer1::kMAX = 2 , nvinfer1::kMIN = 3 ,

nvinfer1::kAVG = 4 }

Enumerates the reduce operations that may be performed by a Reduce layer.

•

enum class nvinfer1::SampleMode : int32 t {

nvinfer1::kSTRICT BOUNDS = 0 , nvinfer1::kDEFAULT = kSTRICT BOUNDS , nvinfer1::kWRAP = 1 ,

nvinfer1::kCLAMP = 2 ,

nvinfer1::kFILL = 3 , nvinfer1::kREFLECT = 4 }

Controls how ISliceLayer and IGridSample handle out-of-bounds coordinates.

•

enum class nvinfer1::TopKOperation : int32 t { nvinfer1::kMAX = 0 , nvinfer1::kMIN = 1 }

Enumerates the operations that may be performed by a TopK layer.

•

enum class nvinfer1::MatrixOperation : int32 t { nvinfer1::kNONE , nvinfer1::kTRANSPOSE , nvinfer1::kVECTOR

}

Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication.

•

enum class nvinfer1::InterpolationMode : int32 t { nvinfer1::kNEAREST = 0 , nvinfer1::kLINEAR = 1 ,

nvinfer1::kCUBIC = 2 }

Enumerates various modes of interpolation.

•

enum class nvinfer1::ResizeCoordinateTransformation :

int32 t { nvinfer1::kALIGN CORNERS = 0 ,

nvinfer1::kASYMMETRIC = 1 , nvinfer1::kHALF PIXEL = 2 }

The resize coordinate transformation function.

•

enum class nvinfer1::ResizeSelector : int32 t { nvinfer1::kFORMULA = 0 , nvinfer1::kUPPER = 1 }

The coordinate selector when resize to single pixel output.

•

enum class nvinfer1::ResizeRoundMode : int32 t { nvinfer1::kHALF UP = 0 , nvinfer1::kHALF DOWN = 1 ,

nvinfer1::kFLOOR = 2 , nvinfer1::kCEIL = 3 }

The rounding mode for nearest neighbor resize.

•

enum class nvinfer1::LoopOutput : int32 t { nvinfer1::kLAST VALUE = 0 , nvinfer1::kCONCATENATE = 1 ,

nvinfer1::kREVERSE = 2 }

Enum that describes kinds of loop outputs.

•

enum class nvinfer1::TripLimit : int32 t { nvinfer1::kCOUNT = 0 , nvinfer1::kWHILE = 1 }

Enum that describes kinds of trip limits.

•

enum class nvinfer1::FillOperation : int32 t { nvinfer1::kLINSPACE = 0 , nvinfer1::kRANDOM UNIFORM =

1 , nvinfer1::kRANDOM NORMAL = 2 }

Enumerates the tensor fill operations that may performed by a fill layer.

•

enum class nvinfer1::ScatterMode : int32 t { nvinfer1::kELEMENT = 0 , nvinfer1::kND = 1 }

Control form of IScatterLayer.

•

enum class nvinfer1::BoundingBoxFormat : int32 t { nvinfer1::kCORNER PAIRS = 0 , nvinfer1::kCENTER SIZES

=1}

10.3 NvInfer.h File Reference

695

Representation of bounding box data used for the Boxes input tensor in INMSLayer.

•

enum class nvinfer1::CalibrationAlgoType

int32 t { nvinfer1::kLEGACY CALIBRATION

nvinfer1::kENTROPY CALIBRATION = 1 , nvinfer1::kENTROPY CALIBRATION 2 = 2 , nvinfer1::kMINMAX CALIBRATIO

=3}

Version of calibration algorithm to use.

•

enum class nvinfer1::QuantizationFlag : int32 t { nvinfer1::kCALIBRATE BEFORE FUSION = 0 }

List of valid flags for quantizing the network to int8.

•

enum class nvinfer1::BuilderFlag : int32 t {

nvinfer1::kFP16 = 0 , nvinfer1::kINT8 = 1 , nvinfer1::kDEBUG = 2 , nvinfer1::kGPU FALLBACK = 3 ,

nvinfer1::kSTRICT TYPES = 4 , nvinfer1::kREFIT = 5 , nvinfer1::kDISABLE TIMING CACHE = 6 ,

nvinfer1::kTF32 = 7 ,

nvinfer1::kSPARSE WEIGHTS = 8 , nvinfer1::kSAFETY SCOPE = 9 , nvinfer1::kOBEY PRECISION CONSTRAINTS

= 10 , nvinfer1::kPREFER PRECISION CONSTRAINTS = 11 ,

nvinfer1::kDIRECT IO = 12 , nvinfer1::kREJECT EMPTY ALGORITHMS = 13 , nvinfer1::kENABLE TACTIC HEURISTIC

= 14 , nvinfer1::kVERSION COMPATIBLE = 15 ,

nvinfer1::kEXCLUDE LEAN RUNTIME = 16 , nvinfer1::kFP8 = 17 }

List of valid modes that the builder can enable when creating an engine from a network definition.

•

enum class nvinfer1::MemoryPoolType : int32 t {

nvinfer1::kWORKSPACE = 0 , nvinfer1::kDLA MANAGED SRAM = 1 , nvinfer1::kDLA LOCAL DRAM =

2 , nvinfer1::kDLA GLOBAL DRAM = 3 ,

nvinfer1::kTACTIC DRAM = 4 }

The type for memory pools used by TensorRT.

•

enum class nvinfer1::PreviewFeature :

int32 t { nvinfer1::kFASTER DYNAMIC SHAPES 0805 = 0 ,

nvinfer1::kDISABLE EXTERNAL TACTIC SOURCES FOR CORE 0805 = 1 , nvinfer1::kPROFILE SHARING 0806

=2}

Define preview features.

•

enum class nvinfer1::HardwareCompatibilityLevel : int32 t { nvinfer1::kNONE = 0 , nvinfer1::kAMPERE PLUS

=1}

•

enum class nvinfer1::NetworkDefinitionCreationFlag :

int32 t { nvinfer1::kEXPLICIT BATCH = 0 ,

nvinfer1::kEXPLICIT PRECISION = 1 }

List of immutable network properties expressed at network creation time. NetworkDefinitionCreationFlag is used with

createNetworkV2() to specify immutable properties of the network. Creating a network without NetworkDefinition←↩

CreationFlag::kEXPLICIT BATCH flag has been deprecated.

Functions

• template<> constexpr int32 t nvinfer1::EnumMax< LayerType > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< ScaleMode > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< GatherMode > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< RNNOperation > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< RNNDirection > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< RNNInputMode > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< RNNGateType > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< UnaryOperation > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< ReduceOperation > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< SampleMode > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< TopKOperation > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< MatrixOperation > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< LoopOutput > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< TripLimit > () noexcept

696

File Documentation

• template<> constexpr int32 t nvinfer1::EnumMax< FillOperation > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< ScatterMode > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< BoundingBoxFormat > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< CalibrationAlgoType > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< QuantizationFlag > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< BuilderFlag > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< MemoryPoolType > () noexcept

• template<> constexpr int32 t nvinfer1::EnumMax< NetworkDefinitionCreationFlag > () noexcept

• nvinfer1::IPluginRegistry ∗ nvinfer1::getBuilderPluginRegistry (nvinfer1::EngineCapability capability) noex-

cept

Return the plugin registry for building a Standard engine, or nullptr if no registry exists.

• nvinfer1::safe::IPluginRegistry ∗ nvinfer1::getBuilderSafePluginRegistry (nvinfer1::EngineCapability capabil-

ity) noexcept

Return the plugin registry for building a Safety engine, or nullptr if no registry exists.

10.3.1

Detailed Description

TensorRT Versioning follows Semantic Versioning Guidelines specified here:

https://semver.org/

This is the top-level API file for TensorRT.

10.4

NvInfer.h

Go to the documentation of this file.

1 /*

* SPDX-License-Identifier: LicenseRef-NvidiaProprietary

* NVIDIA CORPORATION, its affiliates and licensors retain all intellectual

* property and proprietary rights in and to this material, related

* documentation and any modifications thereto. Any use, reproduction,

* disclosure or distribution of this material and related documentation

* without an express license agreement from NVIDIA CORPORATION or

* its affiliates is strictly prohibited.

13 #ifndef NV INFER H

14 #define NV INFER H

16 #include "NvInferLegacyDims.h"

17 #include "NvInferRuntime.h"

27 //

42 namespace nvinfer1

43 {

52 enum class LayerType : int32 t

53 {

kCONVOLUTION = 0,

kFULLY CONNECTED = 1,

kACTIVATION = 2,

kPOOLING = 3,

kLRN = 4,

kSCALE = 5,

kSOFTMAX = 6,

kDECONVOLUTION = 7,

kCONCATENATION = 8,

kELEMENTWISE = 9,

kPLUGIN = 10,

10.4 NvInfer.h

697

kUNARY = 11,

kPADDING = 12,

kSHUFFLE = 13,

kREDUCE = 14,

kTOPK = 15,

kGATHER = 16,

kMATRIX MULTIPLY = 17,

kRAGGED SOFTMAX = 18,

kCONSTANT = 19,

kRNN V2 = 20,

kIDENTITY = 21,

kPLUGIN V2 = 22,

kSLICE = 23,

kSHAPE = 24,

kPARAMETRIC RELU = 25,

kRESIZE = 26,

kTRIP LIMIT = 27,

kRECURRENCE = 28,

kITERATOR = 29,

kLOOP OUTPUT = 30,

kSELECT = 31,

kFILL = 32,

kQUANTIZE = 33,

kDEQUANTIZE = 34,

kCONDITION = 35,

kCONDITIONAL INPUT = 36,

kCONDITIONAL OUTPUT = 37,

kSCATTER = 38,

kEINSUM = 39,

kASSERTION = 40,

kONE HOT = 41,

kNON ZERO = 42,

kGRID SAMPLE = 43,

kNMS = 44,

kREVERSE SEQUENCE = 45,

100

kNORMALIZATION = 46,

101

kCAST = 47,

102

};

103

109

template <>

110

constexpr inline int32 t EnumMax<LayerType>() noexcept

111

{

112

return 48;

113

}

114

121

using TensorFormats = uint32 t;

122

128

enum class ActivationType : int32 t

129

{

130

kRELU = 0,

131

kSIGMOID = 1,

132

kTANH = 2,

133

kLEAKY RELU = 3,

134

kELU = 4,

135

kSELU = 5,

136

kSOFTSIGN = 6,

137

kSOFTPLUS = 7,

138

kCLIP = 8,

139

kHARD SIGMOID = 9,

140

kSCALED TANH = 10,

141

kTHRESHOLDED RELU = 11

142

};

143

144

namespace impl

145

{

151

template <>

152

struct EnumMaxImpl<ActivationType>

153

{

154

static constexpr int32 t kVALUE = 12;

155

};

156

} // namespace impl

157

174

class ITensor : public INoCopy

175

{

176

public:

191

void setName(char const* name) noexcept

192

{

193

mImpl->setName(name);

194

}

195

203

char const_* getName() const noexcept

698

File Documentation

204

{

205

return mImpl->getName();

206

}

207

222

void setDimensions(Dims dimensions) noexcept

223

{

224

mImpl->setDimensions(dimensions);

225

}

226

235

Dims getDimensions() const noexcept

236

{

237

return mImpl->getDimensions();

238

}

239

250

void setType(DataType type) noexcept

251

{

252

mImpl->setType(type);

253

}

254

262

DataType getType() const noexcept

263

{

264

return mImpl->getType();

265

}

266

277

bool setDynamicRange(float min, float max) noexcept

278

{

279

return mImpl->setDynamicRange(min, max);

280

}

281

285

bool isNetworkInput() const noexcept

286

{

287

return mImpl->isNetworkInput();

288

}

289

293

bool isNetworkOutput() const noexcept

294

{

295

return mImpl->isNetworkOutput();

296

}

297

315

void setBroadcastAcrossBatch(bool broadcastAcrossBatch) noexcept

316

{

317

mImpl->setBroadcastAcrossBatch(broadcastAcrossBatch);

318

}

319

331

bool getBroadcastAcrossBatch() const noexcept

332

{

333

return mImpl->getBroadcastAcrossBatch();

334

}

335

341

TensorLocation getLocation() const noexcept

342

{

343

return mImpl->getLocation();

344

}

345

356

void setLocation(TensorLocation location) noexcept

357

{

358

mImpl->setLocation(location);

359

}

360

366

bool dynamicRangeIsSet() const noexcept

367

{

368

return mImpl->dynamicRangeIsSet();

369

}

370

374

void resetDynamicRange() noexcept

375

{

376

mImpl->resetDynamicRange();

377

}

378

384

float getDynamicRangeMin() const noexcept

385

{

386

return mImpl->getDynamicRangeMin();

387

}

388

394

float getDynamicRangeMax() const noexcept

395

{

396

return mImpl->getDynamicRangeMax();

397

}

398

413

void setAllowedFormats(TensorFormats formats) noexcept

414

{

10.4 NvInfer.h

699

415

mImpl->setAllowedFormats(formats);

416

}

417

426

TensorFormats getAllowedFormats() const noexcept

427

{

428

return mImpl->getAllowedFormats();

429

}

430

461

bool isShapeTensor() const noexcept

462

{

463

return mImpl->isShapeTensor();

464

}

465

484

bool isExecutionTensor() const noexcept

485

{

486

return mImpl->isExecutionTensor();

487

}

488

510

void setDimensionName(int32 t index, char const* name) noexcept

511

{

512

mImpl->setDimensionName(index, name);

513

}

514

525

char const_* getDimensionName(int32 t index) const noexcept

526

{

527

return mImpl->getDimensionName(index);

528

}

529

530

protected:

531

apiv::VTensor* mImpl;

532

virtual ˜ITensor() noexcept = default;

533

};

534

542

class ILayer : public INoCopy

543

{

544

public:

550

LayerType getType() const noexcept

551

{

552

return mLayer->getType();

553

}

554

564

void setName(char const* name) noexcept

565

{

566

mLayer->setName(name);

567

}

568

574

char const_* getName() const noexcept

575

{

576

return mLayer->getName();

577

}

578

582

int32 t getNbInputs() const noexcept

583

{

584

return mLayer->getNbInputs();

585

}

586

595

ITensor_* getInput(int32 t index) const noexcept

596

{

597

return mLayer->getInput(index);

598

}

599

603

int32 t getNbOutputs() const noexcept

604

{

605

return mLayer->getNbOutputs();

606

}

607

614

ITensor_* getOutput(int32 t index) const noexcept

615

{

616

return mLayer->getOutput(index);

617

}

618

631

void setInput(int32 t index, ITensor& tensor) noexcept

632

{

633

return mLayer->setInput(index, tensor);

634

}

635

659

void setPrecision(DataType dataType) noexcept

660

{

661

mLayer->setPrecision(dataType);

662

}

663

700

File Documentation

671

DataType getPrecision() const noexcept

672

{

673

return mLayer->getPrecision();

674

}

675

683

bool precisionIsSet() const noexcept

684

{

685

return mLayer->precisionIsSet();

686

}

687

693

void resetPrecision() noexcept

694

{

695

mLayer->resetPrecision();

696

}

697

731

void setOutputType(int32 t index, DataType dataType) noexcept

732

{

733

mLayer->setOutputType(index, dataType);

734

}

735

745

DataType getOutputType(int32 t index) const noexcept

746

{

747

return mLayer->getOutputType(index);

748

}

749

758

bool outputTypeIsSet(int32 t index) const noexcept

759

{

760

return mLayer->outputTypeIsSet(index);

761

}

762

770

void resetOutputType(int32 t index) noexcept

771

{

772

return mLayer->resetOutputType(index);

773

}

774

788

void setMetadata(char const* metadata) noexcept

789

{

790

mLayer->setMetadata(metadata);

791

}

792

801

char const_* getMetadata() const noexcept

802

{

803

return mLayer->getMetadata();

804

}

805

806

protected:

807

virtual ˜ILayer() noexcept = default;

808

apiv::VLayer* mLayer;

809

};

810

1033 enum class PaddingMode : int32 t

1034 {

1035

kEXPLICIT ROUND DOWN = 0,

1036

kEXPLICIT ROUND UP = 1,

1037

kSAME UPPER = 2,

1038

kSAME LOWER = 3,

1039

kCAFFE ROUND DOWN = 4,

1040

kCAFFE ROUND UP = 5

1041 };

1042

1043 namespace impl

1044 {

1050 template <>

1051 struct EnumMaxImpl<PaddingMode>

1052 {

1053

static constexpr int32 t kVALUE = 6;

1054 };

1055 } // namespace impl

1056

1069 class IConvolutionLayer : public ILayer

1070 {

1071 public:

1081

TRT DEPRECATED void setKernelSize(DimsHW kernelSize) noexcept

1082

{

1083

mImpl->setKernelSize(kernelSize);

1084

}

1085

1093

TRT DEPRECATED DimsHW getKernelSize() const noexcept

1094

{

1095

return mImpl->getKernelSize();

1096

}

10.4 NvInfer.h

701

1097

1105

void setNbOutputMaps(int32 t nbOutputMaps) noexcept

1106

{

1107

mImpl->setNbOutputMaps(nbOutputMaps);

1108

}

1109

1115

int32 t getNbOutputMaps() const noexcept

1116

{

1117

return mImpl->getNbOutputMaps();

1118

}

1119

1131

TRT DEPRECATED void setStride(DimsHW stride) noexcept

1132

{

1133

mImpl->setStride(stride);

1134

}

1135

1141

TRT DEPRECATED DimsHW getStride() const noexcept

1142

{

1143

return mImpl->getStride();

1144

}

1145

1161

TRT DEPRECATED void setPadding(DimsHW padding) noexcept

1162

{

1163

return mImpl->setPadding(padding);

1164

}

1165

1173

TRT DEPRECATED DimsHW getPadding() const noexcept

1174

{

1175

return mImpl->getPadding();

1176

}

1177

1193

void setNbGroups(int32 t nbGroups) noexcept

1194

{

1195

mImpl->setNbGroups(nbGroups);

1196

}

1197

1203

int32 t getNbGroups() const noexcept

1204

{

1205

return mImpl->getNbGroups();

1206

}

1207

1217

void setKernelWeights(Weights weights) noexcept

1218

{

1219

mImpl->setKernelWeights(weights);

1220

}

1221

1227

Weights getKernelWeights() const noexcept

1228

{

1229

return mImpl->getKernelWeights();

1230

}

1231

1242

void setBiasWeights(Weights weights) noexcept

1243

{

1244

mImpl->setBiasWeights(weights);

1245

}

1246

1252

Weights getBiasWeights() const noexcept

1253

{

1254

return mImpl->getBiasWeights();

1255

}

1256

1268

TRT DEPRECATED void setDilation(DimsHW dilation) noexcept

1269

{

1270

return mImpl->setDilation(dilation);

1271

}

1272

1280

TRT DEPRECATED DimsHW getDilation() const noexcept

1281

{

1282

return mImpl->getDilation();

1283

}

1284

1297

void setPrePadding(Dims padding) noexcept

1298

{

1299

mImpl->setPrePadding(padding);

1300

}

1301

1307

Dims getPrePadding() const noexcept

1308

{

1309

return mImpl->getPrePadding();

1310

}

1311

702

File Documentation

1324

void setPostPadding(Dims padding) noexcept

1325

{

1326

mImpl->setPostPadding(padding);

1327

}

1328

1334

Dims getPostPadding() const noexcept

1335

{

1336

return mImpl->getPostPadding();

1337

}

1338

1348

void setPaddingMode(PaddingMode paddingMode) noexcept

1349

{

1350

mImpl->setPaddingMode(paddingMode);

1351

}

1352

1360

PaddingMode getPaddingMode() const noexcept

1361

{

1362

return mImpl->getPaddingMode();

1363

}

1364

1373

void setKernelSizeNd(Dims kernelSize) noexcept

1374

{

1375

mImpl->setKernelSizeNd(kernelSize);

1376

}

1377

1383

Dims getKernelSizeNd() const noexcept

1384

{

1385

return mImpl->getKernelSizeNd();

1386

}

1387

1398

void setStrideNd(Dims stride) noexcept

1399

{

1400

mImpl->setStrideNd(stride);

1401

}

1402

1408

Dims getStrideNd() const noexcept

1409

{

1410

return mImpl->getStrideNd();

1411

}

1412

1426

void setPaddingNd(Dims padding) noexcept

1427

{

1428

mImpl->setPaddingNd(padding);

1429

}

1430

1438

Dims getPaddingNd() const noexcept

1439

{

1440

return mImpl->getPaddingNd();

1441

}

1442

1452

void setDilationNd(Dims dilation) noexcept

1453

{

1454

mImpl->setDilationNd(dilation);

1455

}

1456

1462

Dims getDilationNd() const noexcept

1463

{

1464

return mImpl->getDilationNd();

1465

}

1466

1480

using ILayer::setInput;

1481

1482

protected:

1483

virtual ˜IConvolutionLayer() noexcept = default;

1484

apiv::VConvolutionLayer* mImpl;

1485

};

1486

1518

class TRT DEPRECATED IFullyConnectedLayer : public ILayer

1519

{

1520

public:

1528

void setNbOutputChannels(int32 t nbOutputs) noexcept

1529

{

1530

mImpl->setNbOutputChannels(nbOutputs);

1531

}

1532

1538

int32 t getNbOutputChannels() const noexcept

1539

{

1540

return mImpl->getNbOutputChannels();

1541

}

1542

1548

void setKernelWeights(Weights weights) noexcept

10.4 NvInfer.h

703

1549

{

1550

mImpl->setKernelWeights(weights);

1551

}

1552

1558

Weights getKernelWeights() const noexcept

1559

{

1560

return mImpl->getKernelWeights();

1561

}

1562

1570

void setBiasWeights(Weights weights) noexcept

1571

{

1572

mImpl->setBiasWeights(weights);

1573

}

1574

1580

Weights getBiasWeights() const noexcept

1581

{

1582

return mImpl->getBiasWeights();

1583

}

1584

1606

using ILayer::setInput;

1607

1608

protected:

1609

virtual ˜IFullyConnectedLayer() noexcept = default;

1610

apiv::VFullyConnectedLayer* mImpl;

1611

};

1612

1626

class IActivationLayer : public ILayer

1627

{

1628

public:

1636

void setActivationType(ActivationType type) noexcept

1637

{

1638

mImpl->setActivationType(type);

1639

}

1640

1646

ActivationType getActivationType() const noexcept

1647

{

1648

return mImpl->getActivationType();

1649

}

1650

1661

void setAlpha(float alpha) noexcept

1662

{

1663

mImpl->setAlpha(alpha);

1664

}

1665

1675

void setBeta(float beta) noexcept

1676

{

1677

mImpl->setBeta(beta);

1678

}

1679

1684

float getAlpha() const noexcept

1685

{

1686

return mImpl->getAlpha();

1687

}

1688

1693

float getBeta() const noexcept

1694

{

1695

return mImpl->getBeta();

1696

}

1697

1698

protected:

1699

virtual ˜IActivationLayer() noexcept = default;

1700

apiv::VActivationLayer* mImpl;

1701

};

1702

1708

enum class PoolingType : int32 t

1709

{

1710

kMAX = 0,

// Maximum over elements

1711

kAVERAGE = 1,

// Average over elements. If the tensor is padded, the count includes the

padding

1712

kMAX AVERAGE BLEND = 2 // Blending between max and average pooling: (1-blendFactor)*maxPool +

blendFactor*avgPool

1713

};

1714

1715

namespace impl

1716

{

1722

template <>

1723

struct EnumMaxImpl<PoolingType>

1724

{

1725

static constexpr int32 t kVALUE = 3;

1726

};

1727

} // namespace impl

704

File Documentation

1728

1740

class IPoolingLayer : public ILayer

1741

{

1742

public:

1750

void setPoolingType(PoolingType type) noexcept

1751

{

1752

mImpl->setPoolingType(type);

1753

}

1754

1760

PoolingType getPoolingType() const noexcept

1761

{

1762

return mImpl->getPoolingType();

1763

}

1764

1774

TRT DEPRECATED void setWindowSize(DimsHW windowSize) noexcept

1775

{

1776

mImpl->setWindowSize(windowSize);

1777

}

1778

1786

TRT DEPRECATED DimsHW getWindowSize() const noexcept

1787

{

1788

return mImpl->getWindowSize();

1789

}

1790

1802

TRT DEPRECATED void setStride(DimsHW stride) noexcept

1803

{

1804

mImpl->setStride(stride);

1805

}

1806

1814

TRT DEPRECATED DimsHW getStride() const noexcept

1815

{

1816

return mImpl->getStride();

1817

}

1818

1830

TRT DEPRECATED void setPadding(DimsHW padding) noexcept

1831

{

1832

mImpl->setPadding(padding);

1833

}

1834

1844

TRT DEPRECATED DimsHW getPadding() const noexcept

1845

{

1846

return mImpl->getPadding();

1847

}

1848

1859

void setBlendFactor(float blendFactor) noexcept

1860

{

1861

mImpl->setBlendFactor(blendFactor);

1862

}

1863

1872

float getBlendFactor() const noexcept

1873

{

1874

return mImpl->getBlendFactor();

1875

}

1876

1889

void setAverageCountExcludesPadding(bool exclusive) noexcept

1890

{

1891

mImpl->setAverageCountExcludesPadding(exclusive);

1892

}

1893

1900

bool getAverageCountExcludesPadding() const noexcept

1901

{

1902

return mImpl->getAverageCountExcludesPadding();

1903

}

1904

1918

void setPrePadding(Dims padding) noexcept

1919

{

1920

mImpl->setPrePadding(padding);

1921

}

1922

1928

Dims getPrePadding() const noexcept

1929

{

1930

return mImpl->getPrePadding();

1931

}

1932

1946

void setPostPadding(Dims padding) noexcept

1947

{

1948

mImpl->setPostPadding(padding);

1949

}

1950

1956

Dims getPostPadding() const noexcept

1957

{

10.4 NvInfer.h

705

1958

return mImpl->getPostPadding();

1959

}

1960

1969

void setPaddingMode(PaddingMode paddingMode) noexcept

1970

{

1971

mImpl->setPaddingMode(paddingMode);

1972

}

1973

1980

PaddingMode getPaddingMode() const noexcept

1981

{

1982

return mImpl->getPaddingMode();

1983

}

1984

1993

void setWindowSizeNd(Dims windowSize) noexcept

1994

{

1995

mImpl->setWindowSizeNd(windowSize);

1996

}

1997

2003

Dims getWindowSizeNd() const noexcept

2004

{

2005

return mImpl->getWindowSizeNd();

2006

}

2007

2018

void setStrideNd(Dims stride) noexcept

2019

{

2020

mImpl->setStrideNd(stride);

2021

}

2022

2028

Dims getStrideNd() const noexcept

2029

{

2030

return mImpl->getStrideNd();

2031

}

2032

2047

void setPaddingNd(Dims padding) noexcept

2048

{

2049

mImpl->setPaddingNd(padding);

2050

}

2051

2059

Dims getPaddingNd() const noexcept

2060

{

2061

return mImpl->getPaddingNd();

2062

}

2063

2064

protected:

2065

virtual ˜IPoolingLayer() noexcept = default;

2066

apiv::VPoolingLayer* mImpl;

2067

};

2068

2078

class ILRNLayer : public ILayer

2079

{

2080

public:

2090

void setWindowSize(int32 t windowSize) noexcept

2091

{

2092

mImpl->setWindowSize(windowSize);

2093

}

2094

2100

int32 t getWindowSize() const noexcept

2101

{

2102

return mImpl->getWindowSize();

2103

}

2104

2111

void setAlpha(float alpha) noexcept

2112

{

2113

mImpl->setAlpha(alpha);

2114

}

2115

2121

float getAlpha() const noexcept

2122

{

2123

return mImpl->getAlpha();

2124

}

2125

2132

void setBeta(float beta) noexcept

2133

{

2134

mImpl->setBeta(beta);

2135

}

2136

2142

float getBeta() const noexcept

2143

{

2144

return mImpl->getBeta();

2145

}

2146

706

File Documentation

2153

void setK(float k) noexcept

2154

{

2155

mImpl->setK(k);

2156

}

2157

2163

float getK() const noexcept

2164

{

2165

return mImpl->getK();

2166

}

2167

2168

protected:

2169

virtual ˜ILRNLayer() noexcept = default;

2170

apiv::VLRNLayer* mImpl;

2171

};

2172

2178

enum class ScaleMode : int32 t

2179

{

2180

kUNIFORM = 0,

2181

kCHANNEL = 1,

2182

kELEMENTWISE = 2

2183

};

2184

2190

template <>

2191

constexpr inline int32 t EnumMax<ScaleMode>() noexcept

2192

{

2193

return 3;

2194

}

2195

2222

class IScaleLayer : public ILayer

2223

{

2224

public:

2230

void setMode(ScaleMode mode) noexcept

2231

{

2232

mImpl->setMode(mode);

2233

}

2234

2240

ScaleMode getMode() const noexcept

2241

{

2242

return mImpl->getMode();

2243

}

2244

2250

void setShift(Weights shift) noexcept

2251

{

2252

mImpl->setShift(shift);

2253

}

2254

2260

Weights getShift() const noexcept

2261

{

2262

return mImpl->getShift();

2263

}

2264

2270

void setScale(Weights scale) noexcept

2271

{

2272

mImpl->setScale(scale);

2273

}

2274

2280

Weights getScale() const noexcept

2281

{

2282

return mImpl->getScale();

2283

}

2284

2290

void setPower(Weights power) noexcept

2291

{

2292

mImpl->setPower(power);

2293

}

2294

2300

Weights getPower() const noexcept

2301

{

2302

return mImpl->getPower();

2303

}

2304

2315

int32 t getChannelAxis() const noexcept

2316

{

2317

return mImpl->getChannelAxis();

2318

}

2319

2336

void setChannelAxis(int32 t channelAxis) noexcept

2337

{

2338

mImpl->setChannelAxis(channelAxis);

2339

}

2340

NVIDIA TensorRT 8.6.11 API Reference for DRIVE OS (May 2023) - page 10