You are here: Start » User Interface » Creating Deep Learning Model

Creating Deep Learning Model

Introduction

Deep Learning Editor allows the user to create DeepModel objects for Deep Learning tasks.

Currently three major deep learning tools are available:

Anomalies detection - technique which enables the user to find defects based on the provided samples. The user only has to mark which images are good and which are bad.
Features detection - technique in which the user has to select which kind of objects should be found on the provided samples.
Object classification - technique which enables the user to identify images. The user only has to label provided samples with respect to desired number of classes. Images are identified with respect to those classes.

Requirements

Deep Learning license is required to use Deep Learning Editor and filters.
Deep Learning Service must be up and running to perform model training.

Workflow

Please note: to open the Deep Learning Editor, place a relevant Deep Learning filter (Detect features, Detect anomalies or Classify object) in the Program Editor, go to its Properties and click on the icon next to inDeepModel parameter.

Deep Learning model preparation process is split into the following steps:

Loading images - first load the input images,
Marking good and bad samples - indicate features or attach labels for each training image,
Reducing region of interest - select area of image on which training should be done,
Setting training parameters - select parameters with which training should be done,
Training and analyzing results.

Editor overview.

Detecting Anomalies

In this algorithm variant, the user only has to mark which images contain correct cases (good) or incorrect ones (bad).

1. Marking good and bad samples

Use Label as 'Good' or Label as 'Bad' buttons to label each image in the training set. Green and red icons on the right side of the training images indicate to which set an image was assigned.

Labelled images in Deep Learning Editor

2. Configuring additional samples augmentation

Additional sample augmentation can be turned on in case when only few samples are available. For example the user can add additional changes in color intensity to prepare the model for different lighting conditions. Refer to augmentation section for detailed description of parameters. Deep Learning - Augmentation

3. Reducing region of interest

Reduce region of interest to focus only on the important part of the image. Reducing region of interest will speed up the training and model usage.

To get the best results, use the same region of interest for training and evaluation of samples.

By default region of interest contains the whole image.

4. Setting training parameters

Network type - selecting different network architecture.
Density - defines area of neighboring tiles overlap. In general, the higher option is selected, the better results are achieved at the expense of longer computation time.
Network depth - predefined network architecture parameter. For more complex problems higher depth might be necessary.
Iteration count - defines maximal number of times that all samples will be used to train network.
Feature size - defines size of feature. Generally defect should cover about 30% of feature area.
Batch size - number of samples used for single network training. Increasing this value will speed up computation time for bigger GPU with bigger RAM memory.

For more details read Deep Learning - Setting parameters.

5. Performing training

During training, two series are visible: training error and validation error. Both charts should have a similar pattern.

More detailed information is displayed below the chart:

current training statistics (training and validation),
number of processed samples (depends on the number of images and feature size),
estimated training time.

Training process can take a longer time. During this time, training can be cancelled. If no model is present (first training attempt) model with best validation accuracy will be saved. Consecutive training attempts will prompt user about old model replacement.

6. Analyzing results

The window shows a histogram of sample scores and a heatmap of found defects. Left column contains histogram of scores computed for each image in the training set. Additional statistics are displayed below the histogram.

Classify and Classify All buttons can be used to evaluate trained model. It can be useful after adding new images to the data set or after changing the area of interest.

After training, two border values are computed:

Maximum good sample score (T1) - all values from 0 to T1 are marked as good.
Minimum bad sample score (T2) - all values greater than T2 are marked as bad.

All scores between T1 and T2 are marked as "low-quality" scores. Results in this range are uncertain and may not be correct. Filters contain additional output outIsConfident which determines the values which are not in the T1-T2 range.

In the top left corner of the editor, a small rectangle visualizes the selected feature size.

After evaluation, additional training set images filter can be applied.

Detecting features (segmentation)

In this algorithm variant, the user has to define each feature class and then mark features on each image in the training set. This technique is used to find object defects like scratches or color changes, and for detecting image parts trained on a selected pattern.

This technique is a very good solution for problems which are too complex for classical vision algorithms.

1. Defining feature classes (Marking class)

Firstly the user has to prepare classes of defects. Generally it should be features which user would like to check on images. Multiple different classes can be defined for one model.

Class editor is available under sprocket wheel icon on the bottom bar.

To manage classes, Add, Remove or Rename buttons can be used. To customize appearance, color of each class can be changed using Change Color button.

Current class for editing is displayed on the left, the user can select different class after click.

Use drawing tool to mark features on the input images. Tools such as Brush or Rectangle can be used for selecting features.

In addition, class masks can be imported from external files. There are buttons for Import and Export created classes so that the user can create a image of mask automatically prior to Deep Learning model.

The image mask should have the same size as the selected image in the input set. When importing an image mask, all non-black pixels will be included in the current mask.

The user can also load multiple images and masks at the same time, using Add images and masks button.

Directory containing input images should be selected first. Then, directories for each feature class can be selected below. Images and masks are matched automatically using their file names. For example, let's assume that "images" directory contains images 001.png, 002.png, 003.png; "mask_class1" directory contains 001.png, 002.png, 003.png; and "mask_class2" directory contains 001.png, 002.png, 003.png. Then "images\001.png" image will be loaded together with "mask_class1\001.png" and "mask_class2\001.png" masks.

2. Reducing region of interest

The user can reduce input image size to speed up the training process. In many cases, number of features on an image is very large and most of them are the same. In such case region of interest also can be reduced.

On the bottom bar there are tools for applying current ROI to all images as well as resetting the ROI.

3. Setting training parameters

Network depth - predefined network architecture parameter. For more complex problems higher depth might be necessary.
Iteration count - defines maximal number of times that all samples will be used to train network.
Feature size - defines size of feature. Generally defect should cover about 30% of feature area.
Batch size - number of samples used for single network training. Increasing this value will speed up computation time for bigger GPU with bigger RAM memory.
Density - defines percentage overlapping of two consecutive parts of image.

For more details read Deep Learning - Setting parameters.

Details regarding augmentation parameters Deep Learning - Augmentation.

4. Model training

The chart contains two series: training and validation score. Higher score value leads to better results.

5. Result analysis

Image scores (heatmaps) are presented in blue-yellow-red colors palette after using model to evaluation of image. The color represents the probability of the element belonging to the currently selected feature class.

Classify and Classify All buttons can be used to evaluate images. It can be useful after adding new images to the data set or after changing the area of interest.

Image after classification performed using reduced region of interest.

In the top left corner of the editor, a small rectangle visualizes the selected feature size.

Object classification

In this algorithm variant, the user only has to label images with respect to desired number of classes. Labelled images will allow to train model and determine features, which will be used to evaluate new samples and assign them to proper class.

1. Editing number of classes

By default two classes are defined. If the problem is more complex than that, user can edit classes, and define more if needed. Once user is ready with definition of classes, images can be labelled.

Using Classes Editor.

2. Labelling samples

Labelling of samples is possible after adding training images. Each image has corresponding drop down list which allows to assign specific class. It is possible to assign single class to multiple images by selecting desired images in Deep Learning Editor.

Labelling images with classes.