You are here: Start » HMI » Creating User Controls

Creating User Controls


The HMI system of Aurora Vision Studio makes it possible to design user interfaces by composing it from ready-made components, which are called controls. It is possible to extend this functionality by adding new, external controls by using a mechanism of user controls.

HMI is based on the Windows Forms library from the .NET Framework 4.0. The included controls are compatible with Windows Forms controls, appropriately extended for use in Aurora Vision Studio.

First of all designing new HMI controls mainly consists of creating correct controls for the Windows Forms system, according with the requirements specified in the document Developing Custom Windows Forms Controls with the .NET Framework. There are three possible approaches to creating user controls:

  • composite controls (composition of several controls into a new one),
  • extending controls (extending an existing control with new functionality),
  • custom control (a control created from scratch).

Details of implementation of those methods are presented in the document: Varieties of Custom Controls.

A single HMI control is represented by a single, complete and public class derived from System.Windows.Forms.Control or from one of its derived classes. Despite the created components are called user controls, it is not necessary to derive it from System.Windows.Forms.UserControl. This class is used only when implementing composite controls.

Serialization of a Control to a File

HMI layout together with controls settings are saved to a .avhmi file. A control is saved to a file in a way consistent with the Windows Forms principles. Saving of a control is realized by saving values from its public properties. Only properties designed for that purpose are saved and only when their values differ from default values.

To control if property is supposed to be subject of serialization System.ComponentModel.DesignerSerializationVisibilityAttribute attribute should be used.

A Control can signalize if properties has been changed against their default values in one of the following two ways:

Only one from the above mechanisms should be used. Not using any of them will cause a control property to be always serialized and its value may not be reset in editor.

During loading of HMI from a file, control properties are initialized in unspecified order. If a control can enter improper state when properties are set with erroneous values or controls property values depends against other properties (e.g. it limits the value to a given range between minimum and maximum), then it should implement System.ComponentModel.ISupportInitialize interface with help of which HMI subsystem will initialize properties in a single transaction.

Property values are serialized to a file in a XML format and every value must be converted to a text, which is feasible to be saved in XML node. To ensure it is feasible, types of data used in public properties must support conversion from and to a text independent from a localization. Therefore, they need to have assigned a type converter which is capable of conversion in both direction with System.String type. Most of simple built-in types and types provided by .NET Framework (as System.Integer or System.Drawing.Size) already supports such conversion and does not require additional attention.

Descriptions Added to a Control

Control class as well as every public property of a control can be marked with System.ComponentModel.DescriptionAttribute attribute with description for an end user. Description will be shown in the editor while designing HMI.

Control class can be marked with System.Drawing.ToolboxBitmapAttribute attribute which specifies an icon image to be shown in the editor catalog.

Editing Properties in the HMI Designer

In addition to control layout (placement and size) the editor allows editing public properties of a control. Values of public properties specify configuration and initial state of controls in HMI application.

Not every property can be modified in the HMI editor. To enable property edition, the property must be public and its data type must allow conversion to AMR data. Property value is edited inside the Aurora Vision Studio environment through tools common for HMI and program editor. This data is edited in AMR representation and by controlling the way of conversion to AMR data one can affect the way of edition (e.g. by setting permitted range). More detailed description regarding conversion to AMR can be find further on in this article.

HMI editor uses its own rules for determining property visibility in the editor. To control if property should be visible for edition or hidden it should be marked with HMI.HMIBrowsableAttribute attribute with bool type value specifying its visibility in the editor. If public property of a control is not marked with this attribute then the editor automatically determines its visibility.

Creating Modules of User Controls


Following prerequisites must be met for creating modules of user controls:

  • Installed Aurora Vision Studio environment (in version 4.3 or higher).
  • Installed Microsoft Visual Studio environment (in version 2015 or higher) with installed tools for chosen language of a .NET platform (or other environment enabling creating applications for Microsoft .NET platform based on .NET Framework 4.0).

Creating Modules of Controls

To create a module of controls one should create DLL library (Class Library) for .NET environment, based on .NET Framework 4. Library must make control classes available as public types. Moreover, library must make available one public class of arbitrary name which implements HMI.IUserHMIControlsProvider interface. This class has to implement GetCustomControlTypes method which is returning an array with a list of types (class instance System.Type) indicating class types of controls available through the library. Aurora Vision Studio environment will create an instance of this class to obtain a list of controls available through the module.

Required data types from HMI namespace are available through HMI.dll library included in an Aurora Vision Studio installation directory. User HMI controls project should add a reference to this file.

DLL file prepared in such way should be placed in HMIControls folder in installation folder of adequate Aurora Vision product or in HMIControls folder in sub-folder of Aurora Vision product folder in user documents. Loading of the module requires restarting of the Aurora Vision Studio/Executor environment.

It is recommended to compile the user controls project using "any CPU" platform thanks to which it will be feasible to use in Aurora Vision Studio editions for various processor platforms. In case of necessity to use specifically defined processor architecture it should be ensured to use modules compiled for 32-bit processors with Aurora Vision 32-bit environment and modules compiled for 64-bit processors with 64-bit environment.

Creating Projects of User Controls in Microsoft Visual Studio

To create a project of a user control module:

  • In Microsoft Visual Studio create new project of Windows Forms Class Library type for chosen .NET platform programming language (e.g. C#), based on .NET Framework 4.
  • While creating new project, Microsoft Visual Studio environment will add to it first control named "UserControl1" created as a composite control. If controls will be created utilizing other method this class should be removed and replaced with classes of adequate realization type.
  • Add to the project reference to HMI.dll module placed in Aurora Vision Studio installation directory.
  • In the project create control class definition according to your needs. Each control class must be of a public access, derived from System.Windows.Forms.Control or its derivatives.
  • Add to the project one public class implementing HMI.IUserHMIControlsProvider interface and implement in it GetCustomControlTypes method.
  • Compile project to a DLL library. Resulting DLL file should be placed in HMIControls folder of adequate Aurora Vision product.

If a project using additional HMI controls is created in the Aurora Vision Studio environment it should be remembered that, in order to execute such project in the Aurora Vision Executor environment, all required user control modules must also be added to adequate folders of the environments on the destination computers.

Defining Control Ports

Control ports allow creating connections and perform data synchronization between HMI and vision program. Control ports in HMI are based on control's public properties. One control property can create one input and/or output port. For instance, MyProperty property in a control can create input port inMyProperty and output port outMyProperty. Data synchronization between vision program is realized by writing a program value to a control property (using a set accessor) by the HMI subsystem. Data synchronization between HMI and vision program is realized by reading a value out of control property (using a get accessor) and passing it to the program.

In order to create an output port of a control property, it must allow reading (it must implement a public get accessor). In order to create an input port, it must allow both reading as well as writing (it must implement both get and set public accessors).

In order to create a control port based on a given property, a HMI.HMIPortPropertyAttribute attribute should be specified on the property, passing port visibility type as attribute argument. The attribute argument is a value of HMI.HMIPortDirection enumeration of bit flags, which accepts the following values:

  • HMI.HMIPortDirection.Input - property is visible in the editor as a control's input port ready to connect with a program. By default, port is visible and can be hidden by a user through the port visibility editor.
  • HMI.HMIPortDirection.Output - property is visible in the editor as a control's output port ready to connect with a program. By default, port is visible and can be hidden by a user through the port visibility editor.
  • HMI.HMIPortDirection.HiddenInput - property is available in the editor as a control's input port, but the port is hidden by default. Port visibility can be set by a user through the port visibility editor.
  • HMI.HMIPortDirection.HiddenOutput - property is available in the editor as a control's output port, but the port is hidden by default. Port visibility can be set by a user through the port visibility editor.
  • HMIPortDirection.None - property is not a port (using property as a port is forbidden). Property is neither available in the port visibility editor.

It is also possible to use logic sums of above values to create bidirectional ports.

If public property is not marked with HMI.HMIPortPropertyAttribute attribute then in some cases, when the property type is a basic type, the property can be automatically shown in the port visibility editor. This means that there is a possibility for the end user to promote some properties to ports capable to be connected with the vision application. If a public property of a control should never be used as a HMI port, such property should be marked with a HMI.HMIPortPropertyAttribute attribute set to HMIPortDirection.None value.

If a value of a property used as a port can change as a result of control's internal action (e.g. as a result of editing control's content by a user) then the control should implement property value change notification mechanism. For each such property a public event of a EventHandler type (or a derived type) and with the name consistent with a property name extended with a "Changed" suffix should be created. For instance, for a property named MyProperty, MyPropertyChanged event should be created, which will be invoked by a user control implementation after each change of the property value. This mechanism is particularly necessary when given property realizes both input and output ports, and can be changed by a control itself as well by a write operation from the vision application.

In order for a control's property to be able to create a port, capable to be connected with a vision application, its data type must support conversion to an AMR representation. A port can be connected only to the data types to which conversion is possible (when it is supported by an AMR converter assigned to the property data type). You will find more on AMR conversion below.

Conditional Data Types of Ports

System of vision program types includes concept of conditional and optional types. Such types can take additional special symbol Nil meaning no value (empty value). Inside HMI control, based on .NET data types, Nil symbol is represented as null value. In order for property to be able to explicitly accept connection with conditional and optional types, it must be of data types that enables assigning null value (reference type or constructed on System.Nullable<T>).

By default, properties of basic value types and reference types cannot accept or generate conditional data (by default types are not conditional). In order for a property to be able to accept conditional types, and to be able to generate conditional types, causing conditional execution of connected filters (its data type, from vision program's point of view, was conditional) it must be explicitly marked as conditional. In order to mark property as a conditional one:

  • In case of using value types (simple types or structures), it is necessary to define property of a type that can hold null value (System.Nullable<T>). For instance, to attain Integer? port type, a property of int? (System.Nullable<System.Int32>) type should be declared.
  • In case of using reference types (e.g. System.String), which already can hold null value, property should be marked with a HMI.AmrTypeAttribute attribute, with basic (non-conditional) name of required AMR type given as an argument and with IsNillable argument set to true.

Data Ports out of Control Events

It is also possible to create a HMI output data port based on a .NET control event. This allows for easier interoperability with a standard control event notification schema known from Windows Forms. In order to create an event port, a public event must be implemented by the control class, with delegate type EventHandler (or a type derived from it). Similarly to property ports, a HMI.HMIPortPropertyAttribute attribute should be specified on the event (but only output port direction is allowed). For some kinds of events it is also possible that the event will be automatically show in the port visibility editor.

The following code will create a outMyEvent port in the HMI control:

 public event EventHandler MyEvent;

Event ports are of Bool type in the AVS environment. During normal operation False value will be returned from it. After the underlying event has been invoked by the control, the port will return True value for a single read operation, effectively creating an impulse of True value for a time of a single vision program iteration. This can be used for example to control the flow in a Variant Step macrofilter.

Conversion to AMR

Data in vision application in Aurora Vision environment is represented in internal AMR format. Data in HMI controls based on .NET execution environment is represented in the form of statically typed data of .NET types. To exchange data with vision program, as well as to edit control's property data in edition tools, data must be converted between .NET types and AMR representation. This task is performed by a system of AMR converters. For a single .NET type it assigns a single converter (similarly to converters from Components system), which allows to translate data to a single or more AMR types.

In order to be able to connect control's property to a vision program, as well as to be able to edit it during development, data types of the property must posses AMR converter. HMI System posses set of built-in converters which handle the following standard types:

.NET typeDefault AMR typeAMR types feasible for conversion
int (System.Int32) Integer Integer, Real
float (System.Single) Real Real, Integer
decimal (System.Decimal) Double Real, Double, Integer, Long
bool (System.Boolean) Bool Bool
string (System.String) String String, Integer, Long, Real, Double
System.Drawing.Size Size Size
System.Drawing.Point Point2D Point2D, Location
System.Drawing.Rectangle Box Box
System.Drawing.Color Pixel Pixel
List<int> (System.Collections.Generic.List<System.Int32>) IntegerArray IntegerArray
List<bool> (System.Collections.Generic.List<System.Boolean>) BoolArray BoolArray
List<string> (System.Collections.Generic.List<System.String>) StringArray StringArray, IntegerArray, LongArray, RealArray, DoubleArray, BoolArray, String?Array, Integer?Array, Long?Array, Real?Array, Double?Array, Bool?Array

Conversion of all value types in their counterparts allowing null (System.Nullable<T>) value is also possible.

Additionally, when editing in the Aurora Vision Studio environment, the editor enables edition in property grid of arbitrary enumeration type as well as types required to define appearance and behavior of controls such as System.Drawing.Image, System.Windows.Forms.AnchorStyles or System.Drawing.Font.

In case of establishing a connection between vision program and HMI control's port, AMR data type of control's port does not have to be strictly specified. Instead of that AMR converter assigned to control's property is choosing best possible conversion between port in vision program and control's property. In this way it is possible to, for instance, more precisely convert data between property of System.Decimal type and Integer and Real types, where Decimal type enables universal representation of ranges and precisions required for editing both of those types.

During edition in property grid a default AMR data type of control's property is suggested by an AMR converter. It is possible to change default AMR data type of control's property. In order to do that, HMI.AmrTypeAttribute attribute should be specified on the property and a name of new data type in AMR convention should be given as an argument. Only types supported by an AMR converter or name aliases of those types are supported (e.g. for property of System.String type assigning "File" or "Directory" AMR data types, where both are aliases of String type). HMI.AmrTypeAttribute attribute also allows to:

  • assign permitted range (min...max) for numerical types (using Min and Max parameters),
  • define that specified type should be represented in its conditional (T?) form or can accept Nil special value (property data type must allow for assigning null value).

Saving Controls State

HMI subsystem provides functionality to save an application state (settings chosen by application's end user). Saving of the state is accomplished by saving settings of each of the controls. Saving the state of a single control is realized by the same serialization mechanism as in serialization of control's properties in saving to an avhmi file. Control's state is defined by values visible on chosen control properties. In order for a control to be able to save elements of its state (its settings), the state must be fully represented in public bidirectional properties, which data types have possibility to be converted into a text (type converter handles System.String type conversion). For control's property value to be saved together with the application state, such property must be marked with the HMI.HMIStatePortAttribute attribute.

Properties are serialized in unspecified order and in context of various states of other properties. Controls must then implement properties representing state in such a way that no dependencies will appear between them which can cause errors during execution or lead a control into incorrect state. It is recommended that a control be able to correct input value of such properties without causing errors (e.g. in case of errors in state storage file or HMI application changes between saving and reading of the application state). If given control's state is visible on more than one property (e.g. numeric state can be read/written also in text form on Text property) then only one property should be subject to state saving. State saving properties can, but do not have to, be control's ports enabling connection with vision application. Similarly, properties realizing control's ports can, but do not have to, save its state.

Interoperability with Extended HMI Services

Control might need to cooperate with HMI system elements which extends beyond capability to connect its ports with a vision program, or interpreting type attributes. Such cooperation comes down to communicating with particular extended services of the HMI system. Examples of such services include program execution control or application state saving systems.

Each extended service is available through its interface instance, which is provided by the HMI system, and which is passed to controls. In order for a control to be able to obtain a reference to a service interface, it must implement HMI.IServiceConsumer interface. This interface requires control to implement one method: void SetHMIServiceProvider( System.IServiceProvider provider ). HMI subsystem will call this method once during control initialization (outside design mode) and will pass extended HMI service provider instance in form of the System.IServiceProvider interface. Control can then invoke GetService method of the provider, giving as an argument an interface type of a service to which it wants to gain access. Provider will return a reference of requested interface or null if requested service is not available.

The following example shows how to obtain access to a program execution control service interface:

public class MyControl : Control, HMI.IServiceConsumer
    private HMI.IProgramControlService programControlProvider;

    public void SetHMIServiceProvider( IServiceProvider provider )
        programControlProvider = (IProgramControlService)provider.GetService(typeof(IProgramControlService));

Controlling Program Execution and Reactions to Changes in Program Execution

Program execution controlling consists of, among others, program starting, pausing, starting/resuming, stopping and iterating, as well as events of program transition to particular states after execution of the operations.

Any HMI control can influence program control, as well as follow changes of the program execution state. In order to do that, control must obtain access to a program control service, by acquiring HMI.IProgramControlService interface. This interface provides following elements:

  • ProgramStateChanged event, with a help of which control will be notified about changes of the program execution state. As a part of a call, a program execution flag IsExecuting is passed (i.e. if the program is in state of program execution, regardless of whether filters are currently processed, e.g. program is in executing state in active work or pause state, but is not executing after it was stopped or when it's loading was not completed). Remaining event parameters inform about capability of switching to appropriate execution states.
  • ProgramEvent event, informing about interesting, from control's point of view, application life cycle events.
  • StartProgram, IterateProgram, PauseProgram, StopProgram methods, with a help of which a control can force adequate change in program execution state.

Controls Managing Saving of the HMI State

It is possible to create a control which, through a mechanism of saving HMI application state, will save or load state of whole application. In order to do so, control must acquire HMI.IStateManager interface. This interface provides methods for saving whole application to an indicated file, loading whole application from an indicated file and to find project localization in local file system.

Controlling On-Screen Virtual Keyboard

HMI subsystem allows to display a virtual on-screen keyboard, useful on systems with touch panels. The keyboard is, to a limited extend, shown and hidden automatically by the HMI environment. If required, a HMI control can explicitly control showing and hiding of the virtual keyboard. To do so, control must obtain HMI.IVirtualKeyboardService interface. This interface provides methods for displaying keyboards of a specified types, avoiding the obstruction of the visibility of specified control or specified part of the screen, as well as methods for hiding keyboard.

Explicit control of on-screen keyboard will work only if the HMI application designing user activated the keyboard for given HMI application. When the keyboard is not active in an application, all requests of displaying and hiding the keyboard are ignored without errors.

Previous: Protecting HMI with Password Next: List of all HMI Controls