COMMON VISION BLOX

As a 49% subsidiary of Airbus, Matrium GmbH is largely concerned with logistic tasks in the aviation industry. However, the company founded in 2003 also offers specialised solutions for logistics in other industries. For instance, Matrium has developed a system for customers in the electronics industry where trays for electronic components can be distinguished with the help of a machine vision system and then placed in the allotted compartments by a robot.

These trays serve as transport containers for electronic components, whose sizes can range from 3 x 3 mm to around 40 x 40 mm. The only constant here is the area of the rectangular trays, which is 12.7 x 5.35 inches (322.6 x 136 mm). In order to transport the components securely, the trays feature corresponding troughs into which the processors and other electronic components are placed following their production at various electronic component manufacturers.

"Once the electronic components on the trays have been delivered, we receive the trays back from our customers so that they can be used again after checking and sorting", says Matrium's IT manager Martin Eikel, describing the material circulation.

"The manufacturers wish to get these trays back sorted according to type – and this task is not easy to solve when one considers that there are about 2500 existing variants, some of which differ only very slightly in terms of shape, material and thickness." Moreover, an electrical measurement of the surface resistance is carried out for each tray in accordance with EN 61340, and this can only be done with an automated process.

Complete system replaces manual sorting

For some years now Matrium has been doing the tray sorting as a service provider to one of the world's largest distributors of electronic components and embedded solutions. "For a long time this sorting was done manually by our employees, but we wanted to automate this process and thus reduce the workload and to be able to meet the higher requirements for the resistance measurement", Eikel explains. "Apart from that, it was becoming more and more complicated for our colleagues to distinguish between the different tray types. It presented them with real challenges."

That gave rise to the idea of developing a system that could provide a more cost-effective and reliable sorting solution with the help of machine vision for the sensor system and a portal robot for the handling.

Together with his employee Martin Gericke, the IT manager developed a 3D simulation of the system in the autumn of 2015. That became reality in the spring of 2016 in the form of the MTS-1. MTS stands for Matrium Tray Sorting Machine – and it is precisely this task that this first system of its kind performs. According to Eikel it now does so with 95 percent reliability depending on the tray manufacturer.

Today, one small part of the task is still done manually by Matrium employees: they fill the transport trolleys of the sorting machine with trays stacks up to a maximum height of 180 trays and ensure in this work step that the component carriers are correctly orientated and free from defects. "Before this point, trays with broken-off corners, cracks, sticky labels or other defects are removed from the process by hand during pre-sorting by manufacturer", explains Eikel.

The MTS-1 has five stack shafts that can be refilled while the machine is running. The system can sort fully automatically for three hours without refilling. The MTS-1 places up to 13 processed trays in one of the 330 compartments, which are constantly emptied. Afterwards a sorted stack is packaged, labelled and put into storage until it is called off by the customer.

Machine vision provides for reliable distinction

First of all, a vertically movable axis with a gripper specially designed by Gericke takes the top tray from a stack and positions it in front of the three cameras of the integrated machine vision system. Two of these cameras capture images from above and the other from the side, and they send these image data to the system's Industrial PC.

One of the special features of the image capture in this case was that different reflections arise depending on the material composition, colour and degree of gloss of the trays. This problem was solved by evaluating the first images of each tray as a histogram, whose results were then used to regulate the brightness for the following captures. "The goal was to achieve images with a medium grey-scale value for each object in order to create the optimum conditions for the subsequent image evaluation. Therefore, up to six captures are made per camera with different exposure times until the required grey-scale value is reached."

The approach is somewhat more time consuming and takes up to 2 seconds, but according to Gericke the time taken in this step is not the decisive factor: "During the image acquisition and detection two resistance measurements are made, which also take time. The self-developed portal robot that takes up the trays from the vertical axis after the image capture requires 12 seconds per tray on average until it has placed it in the respective compartment and can pick up the next one."

Despite the time span available for the image evaluation, which is extremely long by machine vision standards on account of this situation, the huge variety of types with only minute differences in some cases made it difficult to implement a reliable distinction between the variants.

"The trays have a raised marking at the edge of the top side and optionally on one side on which the exact type is encoded. This is a bit like the markings on car tyres, but the markings on our trays can be as small as 1.8 mm. Since the component carriers have the most diverse markings and different fonts are in circulation, a large number of patterns and characters are programmed. From the combination of CVB Polimago OCR recognition and the patterns found or not found respectively, the right tray is ultimately determined within a maximum of 2 seconds in a proprietary evaluation process"*, explains Eikel.

Clever programming with CVB Polimago

Neither Martin Eikel nor Martin Gericke had any experience in dealing with machine vision systems until the start of this project. In view of the challenges involved in this sorting application, the IT manager sought professional support and found it with STEMMER IMAGING, whose machine vision experts were able to make use of the entire bandwidth of technical possibilities in order to find the ideal solution for this project.

"We carried out feasibility studies in the laboratory at STEMMER IMAGING and tested various types of lighting, algorithms and pre-processing options amongst other things", Eikel recalls. "We also examined "shape-from-shading" technologies and 3D laser scans for their suitability for this task, but in the end the optimisation of the image creation produced the best results for our application", says Eikel.

On account of the large variety of trays, the programming of patterns for the Matrium project was just as decisive a question as that of the optimum image capture. The CVB Polimago imaging tool from the Common Vision Blox (CVB) software library from STEMMER IMAGING proved here to be the perfect tool for fast and reliable pattern recognition and for reading the markings. CVB Polimago has one decisive advantage: Through the automatic creation of training images the programming work for the user is very much reduced.

"In order for pattern recognition to work reliably under real conditions, the training model must exhibit sufficient variance", explains Florian Mayr from STEMMER IMAGING, who supported Matrium as sales contact during the project. "If geometrical transformations such as rotations, size changes, tilting, occlusions or changes in the lighting situation are already programmed during the training phase, the detection rate in the subsequent process is increased."

Fast way to stable results

Martin Eikel mentions a further aspect that spoke for the choice of CVB Polimago: "With around 2500 tray variants it was important to us to keep the programming work to a minimum. CVB Polimago saves us a lot of time in the training phase, because the algorithm generates artificial views of the model during programming in order to simulate various positions of a component in reality. We can already achieve very stable results with between 20 and 50 training images and are very satisfied with the system detection rate that we have achieved so far. Moreover, the well documented software library enables us to seamlessly integrate the image functions into both the Windows service for the machine and our own database-based management software."

Eikel found the numerous application and program examples made available by STEMMER IMAGING for CVB Polimago to be particularly helpful for a quick start with the CVB tool. The IT manager has very positive words to say about the choice of his machine vision partner. Eikel himself says he was particularly impressed by the excellent advice and support in the selection of the components used as well as the fast reaction to questions: "With little effort and the support of machine vision experts we were able to test all the options in order to optimise the detection quality and to achieve the best possible system design with regard to machine vision. All in all, the support from STEMMER IMAGING was an enormous help to us in implementing the system quickly and achieving reliable sorting results."