You are here: Start » PyAVL » Deep Learning (PyAVL-DL)

Deep Learning (PyAVL-DL)

PyAVL-DL is the AVLDL module. It runs trained Aurora Deep Learning models on images from Python. The module requires PyAVL, and it uses the types of PyAVL, such as Image, Region and Rectangle2D. Objects created with AVL can therefore be passed directly to the functions of AVLDL and the other way round.

The functions of PyAVL-DL are described in this page. The functions of PyAVL, such as AVL.EnumerateImages or the drawing functions, are used together with them.

Workflow

Working with a model has three steps:

Deploy. A DL_..._Deploy function loads the model from its directory and prepares it for execution on the selected device. It fills a model identifier object, which is passed to the next functions. The only exception is DL_LocateBarcodes_Deploy, which needs no model directory and returns the identifier.
Run. The DL_... function with the same model name processes one image. It can be called any number of times.
Unload. A DL_..._Unload function releases the memory used by the model.

import AVL
import AVLDL

# Deploy the model from the "model" directory on the CPU
model = AVLDL.ClassifyObjectModelId()
AVLDL.DL_ClassifyObject_Deploy("model", model, inDeviceType=AVLDL.DeviceKind.CPU)

# Run the model on every image of a directory
for image, _, filename in AVL.EnumerateImages("images/test"):
    heatmap = AVL.Heatmap()
    result = AVLDL.DL_ClassifyObject(image, model, heatmap)
    print(f"{filename}: {result.outClassName} ({result.outScore:.2f})")

# Release the model
AVLDL.DL_ClassifyObject_Unload(model, inLeaveForRedeploy=False)

The device is selected with inDeviceType, using the values AVLDL.DeviceKind.CPU and AVLDL.DeviceKind.CUDA, and inDeviceIndex for systems with more than one device. The functions that return several values return a named result object, as in the other functions of PyAVL, for example result.outClassName.

Complete scripts for most of the model types are included in the examples, see Python Examples.

Model Types

Every model type has its own identifier type and its own set of functions. The names of the functions follow one pattern. The function that runs the model has the name given in the table, and the functions that deploy and unload the model have the suffixes _Deploy and _Unload. For example, the Classify Object model uses DL_ClassifyObject_Deploy, DL_ClassifyObject and DL_ClassifyObject_Unload.

Model	Function	Identifier	Additional functions
Classify Object	`DL_ClassifyObject`	`ClassifyObjectModelId`	`_AccessModel`
Detect Anomalies 1	`DL_DetectAnomalies1`	`DetectAnomalies1ModelId`	—
Detect Anomalies 2	`DL_DetectAnomalies2`	`DetectAnomalies2ModelId`	`_WithThresholds`, `_AccessModel`
Detect Features	`DL_DetectFeatures`	`DetectFeaturesModelId`	`_AccessModel`
Locate Points	`DL_LocatePoints`	`LocatePointsModelId`	`_AccessModel`
Locate Objects	`DL_LocateObjects`	`LocateObjectsModelId`	—
Segment Instances	`DL_SegmentInstances`	`SegmentInstancesModelId`	—
Read Characters	`DL_ReadCharacters`	`ReadCharactersModelId`	`_Multiple`
Locate Text	`DL_LocateText`	`LocateTextModelId`	—
Locate Barcodes	`DL_LocateBarcodes`	`LocateBarcodesModelId`	—

The additional functions have the name of the function from the table followed by the suffix:

_AccessModel reads the information stored in a trained model, such as the names of its classes or its thresholds, from the model directory, without deploying the model.
_WithThresholds runs the model with thresholds given as arguments.
_Multiple reads the characters in several regions of interest in one call.

The deploy functions of Read Characters and Locate Text can load a pretrained model instead of a model directory. The pretrained model is selected with inPretrainedModelType, for example AVLDL.OcrPretrainedModel.Extended.

Custom Models

Trained models that do not belong to any of the model types above are executed on tensors with the following functions. A model is identified by WeaverModelId.

Function	Description
`WEAVER_DeployModel`	Loads a model and prepares its execution on a specific target device.
`WEAVER_InferSingleTensor`	Executes a model on one input tensor.
`WEAVER_InferMultipleTensors`	Executes a model on several input tensors.
`WEAVER_UnloadModel`	Unloads a model and frees the allocated memory.

Tensor Conversion

The input and the output of the custom models are Tensor objects. The following functions convert images and arrays to tensors and back:

Function	Description
`ConvertImageToTensor`	Converts an image to an `NHWC` tensor, with `N` equal to 1.
`ConvertTensorToImage`	Converts an `NHWC` tensor to an image. Only the data for `N` equal to 0 is converted.
`ConvertImageArrayToTensor`	Converts an array of images with the same format to an `NHWC` tensor.
`ConvertTensorToImageArray`	Converts an `NHWC` tensor to an array of images.
`ConvertRealArrayToTensor`	Converts an array of real numbers to a tensor with the given dimensions and ordering.
`ConvertTensorToRealArray`	Converts a tensor to an array of real numbers.
`ConvertRealArrayArrayToTensor`	Converts a two-dimensional array of real numbers to a tensor with the `NL` ordering.
`ConvertTensorToRealArrayArray`	Converts a tensor with the `NL` ordering to a two-dimensional array of real numbers.

Devices and Service