Robotics tasks using Machine Vision in Retail
Robotic automation can be found in shipment and distribution centers where they perform picking, sorting and packaging activities. These tasks are a natural evolution of automated warehouses – to which robots move in order to store and retrieve items.
Identification of objects (aka object recognition) is performed by a machine learning classifier along with OCR recognition of the attached text. Since the objects are not necessarily located and oriented in the optimal way for the robot’s cameras – a CNN classifier is used to identify clues of textual and visual features. Mobile robots navigate in a space where other robots are operating, thus requiring real-time update of their environment perception. The navigation system, using a highly capable real-time CNN classifier, updates the 3D environment model of the robot. This process enables autonomously navigating robots to respond to the increasing popularity of internet commerce.
Customer service
Customer service is a relatively new domain among robots functions. They take a form similar to human in order to ease their acceptance by customers. Assistance in item finding, promotions update and even advice customers are just few services the robots provide. Future robots, servicing customers in shops will be able to communicate in natural language using the latest machine learning technologies.
Machine vision functions for customer service are principally object recognition and face recognition. The object recognition is used when a customer is looking for a spare part or any similar object. The object is held in the front of the robot and captured by the robot cameras. A deep-learning based classifier can usually recognize the objects with a very high performance. In the same way, face recognition of the customer (as performed by the robot) can help the robot itself to handle the customer in a more specifically tailored manner (learned from previous visits and other information sources).
Robot navigation is the main challenge: in addition to map-driven navigation (directing customers to items on the shelves) the robot should avoid people and obstacles. At the same time, it should be in-touch with the served customers. Machine vision which performs 3D environment classification and data fusion of cameras stream are supported by CNN classifier. Deep learning training may be more effective with department- specific training: for example, the effect of learning in a fashion department is quite different from that of learning in a hardware department.
Store inventory management
Another application of robots is store inventory management: mobile robots in retail stores continuously scan shelves, count available product items, notify of any problem in the order or price and update the store back-end for action. Product availability and the right price are most important for the customer and robots can provide it.
Machine vision algorithms used for Store inventories management include an OCR function to recognize item codes and prices. Since items may be of any form factor and not necessarily facing directly to the robot’s cameras, a machine vision pre-process is required. Text areas should be undistorted and with good enough definition (contrast, size, intensity, color and so on). Algorithms solving image deformation, as well as image processing methods are employed (see our upcoming article on robots using OCR). Of course, navigation and obstacles overcoming capabilities are also essential. The navigation is more straightforward – scanning the shelves along the isles using map navigation. However, shoppers and their carts must still be avoided. For this purpose, a deep learning classifier assists in building the 3D robot’s environment.
RSIP Vision is already engaged with robotic applications. We are experienced with CNN classifiers, object detection using deep learning and OCR applications.
