Robert McConnell 0000-00-00 00:00:00
This fi gure shows a simple example of modern color-based recognition (CBR) applied to solder joint inspection. Figures A and B show two classes of joints, unsoldered and soldered. The presence of the solder signifi cantly alters the colors and their proportions in a small region around the crimp connector opening. By simply showing the CBR system examples of each class of joints in that region it learns to distinguish these classes based on the proportions of each color present. Figures C and D show the region where both training and inspection take place and where the results are displayed; red indicates regions interpreted by the system as unsoldered, green indicates soldered. In addition to the on-screen display the system provides digital output for process control. Source: WAY-2C Color Machine Vision Color machine vision has two principal uses. The fi rst is to verify that the correct color product is being produced; the second is to use color properties for object recognition. In the latter case, it is not the colors themselves that are of interest, but identifi cation of the object that generates them. Quite a few recent articles and webinars have covered the basics of color machine vision. These usually discuss methods well suited for color measurements to verify that the correct color product is being produced. These methods also can be used for identifi cation of single colored objects; however, they are usually impractical for multicolored objects and those where the region being inspected may include complex boundaries. Common methods of color-based recognition (CBR), including ones that make it easy to recognize multicolored objects or those with complex boundaries, are the focus here. TYPES OF RECOGNITION APPLICATIONS There are two basic types of recognition applications: classifi cation and anomaly detection. For classifi cation, the system must decide between a known set of possible reference classes, for example a red apple, a green apple, a rotten apple or no apple at all. Once trained on each class, the system must reliably assign test items it has not seen before to the appropriate reference class. For anomaly detection the system is trained on examples of only one reference class; it must decide whether or not each test item belongs to that class. For simplicity this will focus on classifi cation, however, the methods Involved generally cover both types of recognition. TRADITIONAL SINGLE-COLOR APPROACHES TO RECOGNITION Even when items of interest are single colored, the same is seldom true of their images. Texture, topography, positioning and lighting gradients all contribute to broadening of the range of colors that fall on the image sensor. To cope with the broadening, it is traditional to transform color coordinates from RGB, native to cameras and image monitors, into HSI (hue, saturation, intensity), LAB color space (color-opponent space with dimension L for lightness and A and B for the color-opponent dimensions) or other coordinates that conform more closely to human’s perceptions of color. The transformation usually makes it easier to set color boundary thresholds because hue tends to remain relatively constant as lighting details vary. Systems using this approach are often demonstrated Sorting such items as brightly colored M&M candies or other highcontrast single colored items. Extensive discussions of the coordinate transformation and threshold approach appear in textbooks and system suppliers’ literature. It is both intuitively appealing and appropriate for differentiating simple single-colored items. However, system engineering and maintenance costs can quickly become overwhelming when one tries to use it for recognizing multicolored objects or those with complex shapes or unpredictable orientation. Another approach well suited to high speed sorting of items with limited acceptable color range is to assign each possible color to a specifi c class of interest. Although this approach can be time consuming to train and is best implemented in specialized hardware, it can be extremely fast and effi cient for detection of imperfections in high throughput sorting. This article will focus on methods specifi cally designed for generalized color-based recognition. By abandoning the assumption of single colored items and the need to report actual color values, these systems gain the ability to provide robust and cost-effective recognition. While not widely known, the methods involved have a long record of accomplishment in industrial applications where they have demonstrated recognition accuracy rivaling that of humans but much faster and more reliable over extended periods. THE MULTICOLOR WORLD Traditional recognition methods are based on the assumption that items of interest are distinguishable in their images by a unique set of pixel colors, usually colors that are closely clustered in hue space. If the colors are not unique, so that the color distributions of the classes to be distinguished overlap or are hard to separate with well-defi ned thresholds, the traditional approaches can experience signifi cant problems. Other factors, such as effects of an imager’s Bayer pattern, effects of pixels falling on item boundaries or uncertainties about the boundaries themselves can increase the likelihood of overlap among the color distributions of the different classes, whether single colored or multicolored. The bottom line is that for all but a very few color-based recognition problems involving simple boundaries, contrasting colors and carefully designed lighting, the assumptions on which the traditional CBR methods are based rarely hold. ABANDONING THE SINGLECOLOR APPROXIMATION Rather than relying on the simple, single-color model, modern CBR systems assume from the start that items of interest may be characterized by a virtually unlimited combination of colors. They also assume that some, if not all, colors may appear in more than one class; only the proportions of the colors are diagnostic. These systems use powerful concepts from information theory, involving hundreds of parameters, to characterize the color distributions of each reference class. The concepts include well-defi ned methods for classifi cation and anomaly detection. By comparing the color distribution statistics of an unknown object with the statistics for each reference class, the systems can fi nd the reference class most likely to have been the source of the color pattern captured by the camera. In the case of anomaly detection, the theory provides a comprehensive measure of the anomalousness related to the degree to which a test distribution falls inside or outside the reference distribution. Implementation in a commercial software package requires substantial expertise and investment by the original developers. However, in well-designed packages all the Underlying complexity is hidden. Training by example, much as one would train a human inspector, means little special user knowledge is required. It becomes no more diffi cult to train on, and recognize, items with the most complex color distributions than to do the same for the simplest single-colored item. MODERN CBR CHARACTERISTICS Modern CBR systems are available both for Pcs and smart cameras. Although they may offer the ability to return mean numerical color coordinates for a pixel or region, this ability is rarely useful. Instead they are distinguished by intuitive training by example and identifi cations that tend to closely match those of human inspectors. The combination can result in advantages in reduced engineering, equipment and operating costs together with improved performance. Engineering cost reduction comes primarily from system simplifi cation. The general nature of the method used virtually eliminates the need for color space conversions, special algorithm development, color picking and threshold setting. Requirements of lighting uniformity are often signifi cantly reduced, and glints and shadows may be an aid instead of a hindrance in some applications. Modern information theory based CBR is equally at home sorting based on large color contrasts or distinguishing subtle color shades near the limits of human perception. The powerful “looks most like” matching can greatly reduce or eliminate the need for equipment to accurately position items to be identifi ed. Reduced setup and system training time can significantly reduce operator costs. Even manual, train-by-show training is fast, and automated training or retraining for a single class of object may take only a small fraction of a second; inspection is even faster. Even for such complex inspections As fabric sample cards containing 30 or more different swatches, one-time setup may require less than an hour of off-line assistant time, pre-inspection on-line training less than three minutes of operator time. Actual inspection rates can be as high as several cards per second, limited usually by the time necessary to move the sample into position. WHEN TO CONSIDER CBR CBR should be considered for: • Any identification application in which differences in color or form of items create statistically signifi cant color differences in their images. • Any routine sorting, packing or inspection task where valid decisions can be made using color information alone by a minimally skilled worker. • Whenever simple class information at one or more locations in an image is all that is required, particularly when the item may appear in more than one orientation. COLOR IMAGE REQUIREMENTS Up to this point the discussion has focused on methods available to identify objects from their color images. However, whether one is concerned with color verifi cation or color-based recognition, it is important to note that both require consistent stable imaging systems. The more stable the image, the less often retraining is required. The subtler the differences between objects to be distinguished, the more stringent the need for image acquisition system stability. Experienced integrators of color vision systems are familiar with the need for: • Stable, broad spectrum illumination and dealing with the effects of degradation of phosphors and other lighting components. • Proper shielding from the effects of ambient lighting variations. • Strict environmental controls to minimize temperature dependent Drift of lighting, camera and other electronics. Newcomers to machine vision would do well to consult with such an integrator, at least for advice on lighting and imaging. By heeding that advice and devoting the necessary resources to creating and maintaining the appropriate inspection environment, one can expect a high level of success. SOME WORDS OF CAUTION The history of color machine vision is fi lled with unkept promises, and one should take marketing claims with a grain of salt. That being said, do not dismiss any vision system supplier’s claims out of hand. Most have some unique technologies to offer; some may be better suited to the application than others. Ask each potential supplier to demonstrate how their approach will meet requirements using a representative set of samples or sample images that have been chosen. If possible, pick these samples so they can serve as the basis of system acceptance tests. For simple applications there may be little or no charge for demonstrations, particularly if they can be done at the supplier’s site. For applications that are more complex or if the demonstration must be performed at the customer’s facilities, a modest charge may be involved. Color-based recognition makes it easy to recognize multicolored objects and those with complex boundaries. For those who have not worked with it before, it is an option worth looking into.
Published by QualityMagazine. View All Articles.
This page can be found at http://digital.bnpmedia.com/article/Machine+Vision+Meets+Color/298235/29595/article.html.