Camera Calibration and World Coordinates
Camera calibration, also known as camera resectioning, is a process of estimating parameters of a camera model: a set of parameters that describe the internal geometry of image capture process. Accurate camera calibration is is essential for various applications, such as multi-camera setups where images relate to each other, removing geometric distortions due to lens imperfections, or precise measurement of real-world geometric properties (positions, distances, areas, straightness, etc.).
The model to be used is chosen depending on the camera type (e.g. projective camera, telecentric camera) and accuracy requirements. In a case of a standard projective camera, the model (known as pinhole camera model) consists of focal length, principal point location and distortion parameters.
The camera model contains only intrinsic camera parameters, and so it does not change with camera repositioning, rotations, etc. Thanks to that, there is no need for camera calibration in the production environment, the camera can be calibrated beforehand. As soon as the camera has been assembled with the lens and lens adjustments (zoom/focus/f-stop rings) have been tightly locked, the calibration images can be taken and camera calibration performed. Of course any modifications to the camera-lens setup void the calibration parameters, even apparently minor ones such as removing the lens and putting it back on the camera in seemingly the same position.
Camera model can be directly used to obtain an undistorted image (an image, which would have been taken by a camera with the same basic parameters, but without lens distortion present), however for most use cases the camera calibration is just a prerequisite to some other operation. For example, when camera is used for inspection of planar surfaces (or objects lying on such surface), the camera model is needed to perform a World Plane calibration (see World Plane - measurements and rectification section below).
In Adaptive Vision Studio user will be prompted by a GUI when a camera calibration is needed to be performed. Alternatively, filters responsible for camera calibration may be used directly: CalibrateCamera_Pinhole, CalibrateCamera_Telecentric.
World Plane - measurements and rectification
Vision systems which are concerned with observation and inspection of planar (flat) surfaces, or objects lying on such surfaces (e.g. conveyor belts) can take advantage of the image to world plane transform mechanism of Adaptive Vision Studio, which allows for:
- Calculation of real world coordinates from locations on original image. This is crucial, for example, for interoperability with external devices, such as industrial robots. Suppose a object is detected on the image, and its location needs to be transmitted to the robot. The detected object location is given in image coordinates, however the robot is operating in real world with different coordinate system. A common coordinate system is needed, defined by a world plane.
- Image rectification onto the world plane. This is needed when performing image analysis using original image is not feasible (due to high degree of lens and/or perspective distortion). The results of analysis performed on a rectified image can also be transformed to real-world coordinates defined by a world plane coordinate system. Another use case is a multi-camera system – rectification of images from all the cameras onto common world plane gives a simple and well defined relation between those rectified images, which allows for easy superimposing or mosaic stitching.
The image below shows the image coordinate system. Image coordinates are denoted in pixels, with the origin point (0, 0) corresponding to the top-left corner of the image. The X axis starts at the left edge of an image and goes towards the right edge. The Y axis starts at the top of the image towards image bottom. All image pixels have nonnegative coordinates.
The world plane is a distinguished flat surface, defined in the real 3D world. It may be arbitrarily placed with respect to the camera. It has a defined origin position and XY axes.
Images below present the concept of a world plane. First image presents an original image, as captured by a camera that has not been mounted quite straight above the object of interest. The second image presents a world plane, which has been aligned with the surface on which the object is present. This allows for either calculation of world coordinates from pixel locations on original image, or image rectification, as shown on the next images.
In order to use the image to world plane transform mechanism of Adaptive Vision Studio, appropriate UI wizards are supplied:
- For calculation of real world coordinates from locations on original image – use a wizard associated with the inTransform input of ImagePointsToWorldPlane filter
- For image rectification onto the world plane – use a wizard associated with the inMap input of RectifyImage filter
The most complicated use cases may need a direct use of filters, in such a case following steps are to be performed:
- Camera calibration – this step is highly recommended to achieve accurate results, although not strictly necessary (e.g. when lens distortion errors are insignificant).
- World plane calibration – the CalibrateWorldPlane filter computes an RectificationTransform, which represents image to world plane relation
- The image to world plane relation then can be used to:
After performing analysis on an rectified image (i.e. image remapped by RectifyImage), the results can still be transformed to a common coordinate system given by the world plane. This relation (i.e. rectified image to world plane) is given by auxiliary output outRectifiedToWorldHomography of RectifyImage filter. Notice that the rectified image to world plane relation is different than original image to world plane relation. Refer to Application guide below for examples.
- Image to world plane transform is still a valid mechanism for telecentric cameras. Is such a case, the image would be related to world plane by an affine transform.
- Camera distortion is automatically accounted for in both world coordinate calculations and image rectification.
- The spatial map generated by CreateRectificationMap can be thought of as a map performing image undistortion followed by a perspective removal.
Extraction of calibration grids
Both camera calibration and image to world plane transform calculation use extracted calibration grids in the form of image point array and corresponding real-world coordinate array.
Note that the real-world coordinates of the grids are 2D, because the relative \(z\) coordinate of any point on the flat grid is \(0\).
Using custom grids requires a custom solution for extracting the image point array. If the custom grid is a rectangular grid, the AnnotateGridPoints filter may be used to compute corresponding real-world coordinate array.
Note that the most important factor in achieving high accuracy results is the precision and accuracy of extracted calibration points. The calibration grids should be as flat and stiff as possible (cardboard is not a proper backing material, thick glass is perfect). Take care of proper conditions when taking the calibration images: minimize motion blur by proper camera and grid mounts, prevent reflections from the calibration surface (ideally use diffusion lighting). When using custom calibration grid, make sure that the points extractor can achieve subpixel precision. Verify that measurements of the real-world grid coordinates are accurate.
Example 1: Real-world coordinate calculationPlease refer to the calibgui_world_coordinates_from_grid example inside the Adaptive Vision Studio Examples/Camera Calibration and World Coordinates folder.
Example 2: Direct use of filters
The most complicated use cases may need a direct use of filters, without calibration wizard. Please refer to the calibration_world_coordinates_on_original_images and calibration_world_coordinates_with_remapped_images examples inside the Adaptive Vision Studio Examples/Camera Calibration and World Coordinates folder.
|Previous: Optical Character Recognition||Next: Golden Template|