Standard HMI Controls
There are several groups of UI components in the HMI Controls catalog:
- Components – with non-visual elements which extend the HMI window functionality.
- Containers – for organizing the layout with panels, groups, splitters and tab controls.
- Controls – with standard controls for setting parameters and controlling the application state.
- File System – for choosing files and directories.
- Indicators – for displaying inspection results or status.
- Multiple Pages – for creating multi-screen applications.
- Password Protection – for limiting access to some parts of an HMI to authorized personnel.
- Shape Editors – with controls that allow the end user to define object models or measurement primitives.
- Shape Array Editors – with controls that allow the end user to define an array of object models or measurement primitives.
- State Management – for loading and saving state of the controls to a file.
- Video Box ‐ with several variants of the VideoBox control for high-performance image display.
In a basic machine vision application you will need one VideoBox control for displaying an image, possibly with some graphical overlays, and a couple of TrackBars, CheckBoxes, TextBoxes etc. for setting parameters. You will also often use Panels, GroupBoxes, Labels and ImageBoxes to organize and decorate the window space.
Many properties are the same for different standard controls. Here is a summary of the most important ones:
- AutoSize – specifies whether a control will automatically size itself to fit its contents.
- BackColor – the background color of the component.
- BackgroundImage – the background image used for the control.
- BorderStyle – controls how the border of the control looks like.
- Enabled – can be used to make the control not editable by the end user.
- Font – defines the size and style of the font used to display text in the control.
- ForeColor – foreground color of the control, which is used to display text.
- Text – a string being displayed in the control.
- Visible – can be used to make the control invisible (hide it).
With the available properties it is possible to create an application with virtually any look and feel. For example, to use an arbitrary design for a button, we can use a bitmap, while setting FlatAppearance.BorderSize = 0 and FlatStyle = Flat:
Note: Most properties of HMI controls can be also set dynamically during program execution, although only some of them are visible as ports when the control is selected. To expose additional ports, use the "Edit Port Visibility..." command from the control's context menu.
HMICanvas control represents the entire window of the created application. Other controls will be placed on it. Properties of HMICanvas define some global settings of the application, which are effective when it is run with Adaptive Vision Executor (runtime environment):
- ConsoleEnabled – specifies whether diagnostic console can be shown in the runtime application.
- PreventWindowClose – enables preventing the user from closing the Adaptive Vision Executor window when the program is running.
- ProgramControllerVisible – specifies whether Start / Pause / Stop buttons will be visible on the toolbar of Adaptive Vision Executor.
- ToolbarVisible – specifies whether the toolbar of Adaptive Vision Executor will be visible.
- WindowIcon – Icon of the Adaptive Vision Executor window.
- WindowMode – specifies whether the application should be run in a fixed-size window, in a variable-size window or full-screen.
- WindowTitle – specifies the text displayed as the title of the runtime application.
- If you are having trouble selecting the HMI Canvas because it is completely filled with other controls, right click on any control and choose the "Select: HMICanvas" command.
- Some parameters of HMI Canvas are automatically inherited by the contained controls. For example, if one changes the Font property, other controls will automatically have it set to the same value.
- Full-screen mode is especially useful, when it is required that the end user does not have access to elements of the underlying operating system. On Windows systems it is also possible to set Adaptive Vision Executor as the "system shell", thus removing Desktop, Menu Start etc. completely.
Controls for Setting Layout of the Window
Although it is possible to put all individual controls directly onto an HMI Canvas, it is recommended that related items are grouped together for both more convenient editing as well as for better end user's experience. Available controls in the "Containers" section are:
- GroupBox – can be used to group several controls within an enclosing frame with label.
- Panel – can be used to group several controls without additional graphical elements.
- SplitContainer – can be used to create two panels with movable boundary between them.
- TabControl – can be used to create multiple tabs in the user interface.
To use any container drag and drop container and then drag and drop HMI controls inside container. Other method in order to do that is clicking on desired container in "Containers" section and then selecting the area on HMI panel.
Other controls commonly used for making HMI look nice are: ImageBox (for displaying static images such as company logos) and Label (for any textual items).
However, when you would like to create more sophisticated HMI Panel with multiple screens go to The Multi Panel Control
See also: The TabControl Control.
Controls for Displaying Images
The "Video Box" section of the HMI Controls window contains several controls for high-performance display of image streams. These are:
- VideoBox – the basic variant of the image display control.
- SelectingVideoBox – also allows for selecting a box region by the end user.
- ZoomingVideoBox – also allows for zooming and panning the image by the end user.
- FloatingVideoWindow – a component that allows for opening an additional window for displaying images.
To display some additional information over images in a VideoBox it is necessary to modify the images using drawing filters before passing them to the VideoBox. This gives the user full control over the overlay design, but may expand the program. To keep the program simple, while still being able to display some overlay information, it is possible use View2DBox filters, which can be found in he "Indicators" section. They accept images and 2D primitives produced by various filters as input, however the provided control over the overlay style is limited:
- View2DBox – allows for overlaying 2D primitives (Point 2D, Circle 2D, Rectangle 2D etc.) over an image.
- View2DBox_PassFail – the color of the overlaying 2D primitives may change in accordance with a Bool type input.
See also: a similar example with drawing filters.
ImageBox from "Controls" section can be used to improve look of HMI for displaying static images such as company logos
Note: Do NOT use ImageBox control for displaying inspection results. It should only be used for static images.
- DisplayMode – switches between GDI and DirectX implementations. It depends on the underlying hardware, which one is faster.
- SizeMode – specifies how the image should be fitted to the available window space.
Controls for Setting Parameters
When some parameters of the application are to be set by the end user, the following controls can be used:
- TrackBar, Knob, NumericUpDown – for setting numerical values. You set Minimum and Maximum parameters and then the value actually selected by the end user will be available on the outValue output.
- CheckBox, ToggleButton, OnOffButton, RadioButton – for turning something on or off. The outValue or outChecked output will provide the state of the setting.
- ComboBox, EnumBox – for choosing one of several options.
- TextBox – for entering textual data. The entered text will be available on the outText output.
Example: Using TrackBar
TrackBar by default has two connectable inputs and one connectable output. The inValue input and the outValue output represent the same property – the value indicated by the current state of the control. The second input, inEnabled, is a boolean value and controls whether the track-bar is enabled or disabled in the user interface.
Binding with a Label
Very often it is required that the current value of a TrackBar or a similar control is also visible in the HMI. This can be done with an additional Label control placed near the TrackBar, with the AutoValueSource property referring to the selected TrackBar.
It is also possible to specify the format of the displayed text. The picture below shows a connected Label with the AutoValueFormat property set to "Tolerance: % [px]".
Example: Using ComboBox
ComboBox by default has one connectable inputs and two connectable output. To enable more ports, for example inList, right click on HMI control and edit port visibility. StringArray data type value can be connected To inList input. Other way to enter list is adding more items in ComboBox properties in Data overlap.
Selecting index in Selection parameter, user defines based selection which is chosen when program is executed. Without this selection outputs will be Nil.
The outSelection output has integer data type and shows index of selected item. The outText output has String data type and shows text associated with the selection.
EnumBox is similar to ComboBox. Main differences between them is predefined lists in EnumBox. User can choose one of the type present in the AdaptiveVisionStudio changing Type in EnumBox's properties. List allows to chose one option associated with the variable. For example X or Y value can be chosen for Axis type value.
Controls that output some value adjustable by the user often also provide outputs informing about the moments when this value has been changed. For example, the ComboBox control has an output named outSelection (Integer) for indicating the index of the currently selected list item, but it also has outSelectionChanged (Bool) which is set to True for exactly one iteration when the selection has been changed.
Please note, that most of the "outSomethingChanged" outputs are hidden by default and can be shown with the "Edit Port Visibility..." command in the context menu of the control.
See also: Handling HMI Events.
Controls for Displaying Inspection Results
Non-graphical inspection results are most often presented with the following controls:
- Label – for displaying simple text results.
- ListView, DetailedListView – for displaying arrays or tables.
- PassFailIndicator – for displaying text on a colorful background depending on the inspection status.
- BoolIndicatorBoard – for displaying inspection status where multiple objects or features are checked.
- AnalogIndicator, AnalogIndicatorWithScales – for displaying numerical results in a nice-looking graphical way.
See also: example with DetailedListView.
AnalogIndicator and its variant AnalogIndicatorWithScales are used to display numerical results in a retro style – as analog gauges. The latter variant can have green-orange-red ranges defined with properties like GreenColorMinimum, GreenColorMaximum etc. The ranges are usually disjoint, but there can also be nested ranges: e.g. the red color can span throughout the scale, and the green color can have a narrower range around the middle of the scale. The green sector will be displayed on top, producing an effect of a single green sector, with two red sectors outside.
Event Triggering Controls
Most of the standard controls in Adaptive Vision Studio fall into one of two categories: (1) data presenters and (2) parameter setters. There are, however, also some controls that can trigger events – most notably the ImpulseButton control.
The ImpulseButton control is an event triggering control, which has one output, outValue. The value returned is of the Bool type. As long the button is not clicked, it will return False. Once the button is clicked, the value becomes True for exactly one iteration of the program.
For more information about events see: Handling HMI Events.
The TabControl Control
TabControl is the one of the methods to organize HMI panels within a limited space of a form. It is a container with several "tabs", where each page contains a different set of controls.
The MultiPanelControl Control
MultiPanelControl is an easy and convenient way to organize bigger HMI panels. It employs a multi-screen approach to provide more space and enables adding special purpose windows like e.g. a "Configuration" page.
To create a multi-screen HMI, first place a MultiPanelControl control in your window and fit it to the window size. In the upper-right corner of this control you will see a four element Navigation Bar (). Using the left and right arrows you will be able to switch between consecutive screens. The frame button allows for selecting the parent control in the same way as the "Select: MultiPanelControl" command in the control's context menu. And the triple dot button opens a configuration windows as on the picture below:Also when you double-click on the area of a MultiPanelControl, the same configuration window appears. Using this window you are able to organize the pages. During work with pages there are at least two necessary things you have to remember. First – all pages names have to be unique globally. Second – one MultiPanelControl contains many MultiPanelPages. An important thing is the home icon – use it to set the Initial Page of your application. It will be the first page to appear when the program starts.
A MultiPanelControl control is usually accompanied by some MultiPanelSwitchButton controls that navigate between the pages. Usage of a button of this kind is very simple. Just place it on your window and set the TargetPage property (also available through double-clicking on the button). Page switching buttons are usually placed within individual pages, but this is not strictly required – they can also be placed outside of the MultiPanelControl control.
Every HMI Control has built-in input and output ports. In MultiPanelControl, by default, two ports are visible: "inActivePageName" and "outActivePageName", but there are more. (To make them visible click the right mouse button on the control and select "Edit Ports Visibility..."). The "inActivePageName" input enables setting the active page directly from your program. It is another way to handle page changing. One is using the dedicated buttons by the operator and here comes another method. It might be useful e.g. when "something happens" and you would like to change the screen automatically, not by a button click. This can be e.g. information that a connection with remote host was lost and immediate reconfiguration is needed. The "outActivePageName" by definition is an output which allows you to check the current active page.
Note: More information is available as "HMI Multipanel Control" in the official examples of Adaptive Vision Studio.
Program Control Buttons
Controlling the program execution process, i.e. starting, iterating and pausing, by the end user is possible with a set of ProgramControlButton controls, or with the complete panel named ProgramControlBox. These controls are available in the "Controls" section of the HMI Controls window.
- The Stop button is not available for the HMI, because the end user should not be able to stop the entire application at a random point of execution. If you have a good reason to make it possible, use a separate ImpulseButton control connected to an Exit filter.
- ProgramControlButton and ProgramControlBox controls are useful for the purpose of fast prototyping. In many applications, however, it might be advisable to design an application-specific state machine.
File and Directory Picking
There are two controls for selecting paths in the file system, which are FilePicker and DirectoryPicker. They look similar to a TextBox, but they have a button next to them, which brings up a standard system file dialog for browsing a file or directory.
What is worth mentioning about these two controls is the option NilOnEmpty, which can be useful for conditional execution. As the name suggests, it causes the control to return the special NIL value instead of an empty string, when no file or directory is chosen.
Geometrical primitives, regions and fitting fields, that are used as parameters in image analysis algorithms, are usually created by the user of Adaptive Vision Studio. In some cases, however, it is also required that the end user is able to adjust these elements in the runtime environment. For that purpose, in the "Shape Editors" section available are appropriate controls that bring graphical editors known from Adaptive Vision Studio to the HMI Designer environment.
Note: Shape Array Editors controls allows to create primitives array instead of single one.
Each such editor has a form of a button with four ports visible by default:
- inAlignment – for specifying an appropriate coordinate system for the edited shape.
- inReferenceImage – for specifying a background images for the editor.
- inValue – for setting initial value of the edited shape.
- outValue – for getting the value of the shape after it is edited by the end user.
- outStoredReferenceImage – a reference image that was displayed in the editor during last editing approved with the OK button. Active when inStoreReferenceImage is set to True.
When the end user clicks the button, an appropriate graphical editor is opened:
The following properties specific these controls allow for customizing appearance of an opened editor:
- HideSelectors – allows to simplify the editor by removing image and coordinate system selectors.
- Message – an additional text that can be displayed in an opened editor as a hint for the user.
- WindowTitle – text displayed on the title bar of the opened editor's window.
The reference image is copied to HMI control in each macrofilter iteration. To achieve best performance separate vision algorithm from preparation of algorithm data.
Shape editors can be used to create input values for filter or modify existing values. Every shape editor has assigned proper data type to inValue and outValue. The following list shows data types assigned to HMI controls and example filters which uses this type of data.
- Arc2DEditor - Arc2d - CreateArc.
- ArcFittingFieldEditor - ArcFittingField - FitArcToEdges.
- BoxEditor - Box - CreateBox.
- Circle2DEditor - Circle2D - CreateCircle
- CircleFittingFieldEditor - CircleFittingField - FitCircleToEdges.
- Line2DEditor - Line2D - DrawLines_SingleColor.
- LocationEditor - Location - LocationsToRegion.
- PathEditor - Path - OpenPath.
- PathFittingFieldEditor - PathFittingField - FitPathToEdges.
- Point2DEditor - Point2D - AlignPoint.
- Rectangle2DEditor - Rectangle - CreateRectangle.
- Segment2DEditor - Segment2D - CreateSegment.
- SegmentFittingFieldEditor - SegmentFittingField - FitSegmentToEdges.
- ShapeRegionEditor - ShapeRegion - DrawShapeRegions_SingleColor.
ProfileBox is an HMI control that displays a profile of evenly sampled measurements. ProfileBox is available in the "Indicators" section of the HMI Controls window.
ProfileBox may be useful for displaying array of samples of a known length, for example a profile of an image column or a laser line.
Also it is used for displaying dynamic data from sensors outputs in the function of time.
There are several specific properties for this control:
- DomainDefaultLength, DomainDefaultSizeMode, DomainScale and DomainStart – set of parameters responsible for appearance and behavior of the X axis.
- EnableManualZoomChange – for specifying whether the end user is able to zoom the chart area.
- ChartColor, GridColor, BackColor, AxisColor – for specifying the colors of a ProfileBox content.
- HighQualityMode – for specifying whether the chart is drawn with high quality or with low quality, but faster.
- HorizontalAxisName, VerticalAxisName – for specifying the names of axes.
- HorizontalAxisMin, HorizontalAxisMax, VerticalAxisMin, VerticalAxisMax – for setting the limits of the chart axes.
View3DBox is an HMI control that displays 3D primitives. It is available in the "Indicators" section of the HMI Controls window.
Default settings of this HMI control allow to connect four inData inputs. However, it is possible to use up to eight inputs. To make the all inputs visible, right-click on the View3DBox, select Edit Ports Visibility... and check the inputs you want to use.
The color of the displayed primitives can be changed with DataColor input.
There are also other properties that can be used to customize a data preview:
- EnableAntiAliasing – when set to true increases the render quality at the expense of performance.
- GridMode – specifies visibility and orientation of the measurement grid displayed in the scene.
- PointSize – specifies the display size of the cloud points relative to the size of the scene.
- ProjectionMode – selects the view/projection mode of 3D visualization.
- ScaleColoringMode – allows to enable automatic coloring of the cloud points according to the position along specified axis.
- ShowBoundingBoxes – enables displaying range of bounding boxes around point cloud primitives in the preview.
- WorldOrientation – type and orientation of the coordinate system used to display 3D primitives.
Interaction between user and program can be adjusted with parameters in Behaviour section. To customize this interaction the following parameters should be changed:
- Enabled – indicates whether the control is enabled.
- EnableMouseNavigation – when set to true enables the user to change observer position using mouse.
- InitialViewState – specifies the initial position of observer in the scene.
- Visible – determines whether the control is visible or hidden.
Commonly used features of View3DBox control
To make surface differences along Z axis more visible, change the ScaleColoringMode form Solid to ZAxis.
Point3DArray containing up to 300 points is displayed as an array of 3D marks (crosses). If there are more than 300 points, they are displayed as a standard point cloud.
ActivityIndicator is used to visualize working or waiting state. Default settings of this control allow to connect inActive input with bool data type.
Switching IndicatorMode from Busy to Recording changes the controls visualization.
TextSegmentationEditor and OcrModelEditor
TextSegmentationEditor and OcrModelEditor HMI controls allow user to edit text segmentation and text recognition models in runtime environment.
These functionalities can be used when adding new samples is required.
Details about using corresponding OCR filters can be found in Machine Vision Guide: Optical Character Recognition.
More informations about using OCR tools can be found in the following articles:
textSegmentationEditor has 6 default ports which are visible:
- inReferenceImage – is an image which will be used for model training.
- inRoi – region of interest for reference image.
- inRoiAlignment – alignment for inROI.
- outStoredReferenceImage – is the image set as the editor background.
- outValue – is textSegmentation data type. This value can be connected to inSegmentationModel input in ExtractText filter.
- outValueChanged – is the bool data type value which informs when outValue is changed.
After clicking on textSegmentationEditor button in HMI panel, editor will be opened. In order to do this the program must be executed.
When proper parameters are set, text segmentation is ready. Now segmented text should be recognized. In order to do this outCharacters output from ExtractText filter can be connected to inCharacters input in ReadText filter.
Missing ocrModel can be created with OcrModelEditor HMI control.
OcrModelEditor has 5 default ports which are visible:
- inReferenceCharacters – is RegionArray data type value. Characters from ExtractText filter can be connected to this input. This input will be used for model training.
- inReferenceImage – is an image which will be used as background in model training.
- outStoredReferenceImage – is the image set as the editor background.
- outValue – is ocrModel data type value. This value can be connected to inOcrModel input in ReadText filter.
- outValueChanged – is the bool data type value which informs when outValue is changed.
After clicking on ocrModelEditor button on HMI panel, ocr editor will be opened. In order to do that the program must be executed.
KeyboardListener is used for getting information about pressed keys on keyboard. Information from this control is given in IntegerArray which includes key numbers saved in ASCI code.
To use KeyboardListener and any other tolls form components overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.
All default output values allow to check which key is pressed, but there is a few differences between them:
- outKeysDown – emits array of keys that have been recently pressed.
- outKeysPressed – emits array of currently pressed keys. Using this output is recommended is most cases.
- outKeysUP – emits array of keys that have been recently released.
To use those values in other place in program the best solution is copying them with CopyObject filter with IntegerArray data type.
VirtualKeyboard is HMI control used to enter data onto HMI panel.
This functionality can be used in runtime environment when physical keyboard is not available at the workstation.
To use VirtualKeyboard and any other tools form components overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.
User can change displaying virtual keyboard size setting proper InitialKeyboardSize value in properties.
It is possible to display only Numeric Pad in case of entering numeric data types. In order to do this EnableNumPadOnlyKeyboard must be set to True in properties.
ToolTip HMI control is used to display messages created by the user. Those messages are displayed when user moves the pointer over an associated control.
This functionality can be used when developer wants to give helpful tips to final user.
To use ToolTip and any other tools form components overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI. After previous step it is possible to find and edit specific information for each used HMI control. This message can be edited in each HMI control properties in Misc overlap.
ToolTip properties can be adjusted to user's requirements by changing the appropriate values:
- Active – determines if the ToolTip is active. A tip will only appear if the ToolTip has been activated.
- AutomaticDelay – sets the values of AutoPopDelay, InitialDelay and ReshowDelay to the appropriate values.
- AutoPopDelay – determines the length of time the ToolTip window remains visible if the pointer is stationary inside a ToolTip region.
- BackColor – the background color of the ToolTip control.
- ForeColor – the foreground of the ToolTip control.
- InitialDelay – determines the length of time the pointer must remain stationary within a ToolTip region before the ToolTip window appears.
- IsBalloon – indicates whether the ToolTip will take on a balloon form.
- ReshowDelay – determines the length of time it takes for subsegment ToolTip windows to appear as the pointer moves from one ToolTip region to another.
- ToolTipTitle – determines the title of the ToolTip.
- UseAnimation – when set to true, animation are used when the ToolTip is shown or hidden.
- UseFading – when set to true, a fade effect is used when ToolTip are shown or hidden.
ColorPicker is tool which allows user to pick color from HMI panel.
This function can be used when color changing is required in runtime environment.
The default inValue and outValue ports allow user to get and set color which is saved with pixel data type. outValueChanged informs user when the pixel value is changed.
Color can be set in properties or in executed program clicking on ColorPicker icon on HMI panel. In color picking window user can choose color by clicking on proper color in palette of colors or enter manually values describing the desired color.
Logic and Automation tools
BoolAggregator allows to connect multiple booleans signal sources using logic functions for purpose of controls such as EnabledManager.
This function is useful when one HMI bool state must be controlled by multiple other bool states from HMI.
To use BoolAggregator and any other tolls form Logic and Automation tools overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.
Input boolean values can be assigned by inValue ports. Other way to do that is assign proper bool values to sources in BoolAggregator's Properties. As input value in properties user can choose any bool type value from other used control from HMI panel, for example OnOffButton state.
Using BoolAggregator's properties it is also possible to negate source state choosing True state next to the expected value.
With Mode property it is possible to choose aggregation function which is used to connect multiple data source. It is possible to set one of following aggregation functions:
Output value from BoolAggregator can be used with EnabledManager HMI tool or used in other way. OutValue in BoolAggregator is not visible with default setting. To enable this output right click on BoolAggregator icon into HMI panel and edit ports visibility.
EnabledManager allows to automatically manage controls enable state using other HMI controls as boolean value source.
This function is commonly used with BoolAggregator in order to manage controls enable states in more complex cases.
To use EnabledManager and any other tolls form Logic and Automation tools overlap, HMI control must be dragged on HMI screen and dropped in any place on HMI.
After placing EnabledManager tool it is possible to edit enable state of each control placed onto HMI panel. Automation overlap will be present in controls properties. It is possible to choose source data used as an enable signal from list, setting AutoEnabledSource property. It is also possible to Negate source data value changing AutoEnabledNegate from false to true.
In example below TextBox's enable state is controlled with two OnOffButtons through one BoolAggregator and EnabledManager.
EdgeModelEditor is used for creating edge models from the HMI panel.
This function is useful when edge model edition is required in runtime environment
Details about creating edge models can be found in this article: CreateEdgeModel.
Default visible input port should be connected to ReferenceImage which is used for model training. outValue output has EdgeModel data type. This output can be connected to inEdgeModel input in LocateObjects_Edges filter.
EdgeModel can be created in EdgeModelEditor window. One way to get to this editor window is clicking field next to the Value property in control's properties. Another way is clicking EdgeModelEditor icon onto HMI panel when program is executed.
GrayModelEditor is used for creating gray models from the HMI panel.
This function is useful when gray model edition is required in runtime environment
Details about creating gray models can be found in this article: CreateGrayModel.
Default visible input port should be connected to ReferenceImage which is used for model training. outValue output has GrayModel data type. This output can be connected to inGrayModel input in LocateObjects_NCC filter.
GrayModel can be created in GrayModelEditor window. One way to get to this editor window is clicking field next to the Value property in control's properties. Another way is clicking GrayModelEditor icon onto HMI panel when program is executed.
GenICamAddressPicker is used to choose GenICam GenTL device address in runtime environment. This control can be used with GenICam filter.
HMI image recorder example also could be helpful.
GigEVisionAddressPicker is used to choose GigE Vision device address in runtime environment. This control can be used with GigE Vision filter.
MatrixEditor is used to modify or create Matrix data in runtime environment.
Default port inValue allows to connect input Matrix. OutValue returns matrix which was modify which MatrixEditor.
|Previous: Designing HMI||Next: Handling HMI Events|