Matrox Imaging Library (MIL) Tools

3D vision

Situations sometime arise where classical 2D vision techniques are unable to perform the required localization, recognition, inspection, or measurement tasks. These circumstances range from an inability to obtain the necessary consistent contrast from conventional illumination to needing the pose of an object with six degrees of freedom. This is where 3D vision tools step in—whether alone or in combination with 2D vision tools—to carry out the job.
 
MIL X has a rich set of tools for performing 3D processing and analysis. These tools work on the 3D data produced by profile and snapshot sensors as well as stereo and time-of-flight (ToF) cameras. Consult the Camera Interfacing section for a list of qualified makes. The 3D data supported by MIL X can also come from a Stanford Polygon Format (PLY) or stereolithography (STL) file.
Point cloud filtering
Point-cloud (meshed) filtering

3D processing tools

The 3D processing tools in MIL X operate on—and in between—point clouds, depth maps, and/or elementary objects. The latter can be a box, cylinder, line, plane, or sphere. Operations on a point cloud include filtering (denoising)4, rotation, scaling, translation, cropping/masking, re-sampling, and meshing into surfaces; computing normal vectors; projecting to a depth map; and extracting a cross-section. Operations on a depth map include addition, subtraction/distance, and minimum/maximum; filling gaps (i.e., caused by invalid or missing data); and extracting a profile. Additional operations on both a point cloud and depth map include establishing a bounding box, computing the centroid, counting the number of points, and calculating the distances to the nearest neighboring point.
A depth map can subsequently be analyzed using MIL X's 2D vision tools like Pattern Recognition—without being affected by illumination variations or surface texture—and Character Recognition, when the alphanumeric code to read protrudes from, but has the same color as, the background. A profile or cross section can be analyzed using Metrology.

Pattern Recognition tool

 
Pattern Recognition on depth map

Character Recognition on depth map

 
Character Recognition on depth map

Metrology (3D cross section)

 
Metrology on cross section

3D shape finding tool

MIL X provides a tool for locating one or more cylindrical or spherical shapes in a point cloud. The shape to find is specified either numerically or from a previously defined elementary object. The search process is defined by the number of occurrences to find, radius range, length span (for cylinders), maximum model coverage, acceptance (minimum score), and certainty (sure score), as well as the fit and reserve point distances. A find operation yields the number of occurrences found and the maximum fit distance; for each occurrence, the tool produces the score, error, number of points, center, and radius. For cylindrical shapes, it also returns the length, central axis, and bases.

3D cylinder finder

 
Cylinder Finder

3d sphere finder

 
Sphere Finder
3D blob analysis
3D Blob Analysis

3D blob analysis tool

The 3D Blob Analysis tool in MIL X is available to locate and inspect objects in a point cloud. It makes it possible to segment a point cloud into blobs, calculate numerous blob features, filter and sort blobs by features, as well as select and combine blobs. Multiple segmentation methods can be selected: point distance, color distance, or similarity of normal vectors.

3D Metrology toolset
3D Metrology

3D metrology tools

MIL X includes a toolset for 3D Metrology. Within this toolset, one tool fits a point cloud or depth map to a cylinder, line, plane, or sphere. Additional tools compute various distances and statistics between point clouds, depth maps, and fitted or user-defined elementary objects. Another tool is available to determine volume in a variety of ways.
3D Registration tool
3D alignment for comparative analysis: Model, target, and difference

3D registration tool

An additional 3D Registration tool in MIL X establishes the fine alignment of two or more point clouds and merges them together if required. This tool provides the means to perform high-accuracy comparative analysis between a 3D model and target, do full object reconstruction from multiple neighboring 3D scans, and align the data from multiple 3D camera or sensors.
3D Profiling tool
3D Profilometry

3D profilometry tools

MIL X also contains tools for 3D Profilometry using a discrete sheet-of-light source (i.e., laser) and a conventional 2D camera. A calculator is included to establish the camera, lens, and alignment needed to achieve the desired measurement resolution and range. MIL X provides straightforward calibration methods and associated tools to produce a point cloud or depth map. The calibration carried out in MIL X can combine multiple sheet-of-light sources and 2D camera pairs to work as one, thus avoiding the need for post alignment and merger. Such configurations are useful to limit occlusion, increase scan density, and image the whole volume of an object. Moreover, MIL X makes use of a unique derivative-based algorithm for beam extraction or peak detection, which is both more accurate and robust than traditional ones based on the center of gravity.
3d (eye-in-hand) calibration
3D (eye-in-hand) Calibration

3D calibration tools

In addition, MIL X provides the necessary calibration services to position and orient a camera or sensor in 3D space and determine the coordinate transformation matrices between it and the rest of a robotic cell. The camera or sensor can either be fixed overhead and/or to the side of the robotic cell (i.e., eye-to-hand calibration) or mounted at the end of the robot arm (i.e., eye-in-hand calibration). The calibration enables to relate what a camera or sensor sees—the object pose established using other analysis MIL X tools—to a robot controller for guidance.