Computer Vision News - March 2022

5 MediaPipe 6. Holistic  it combines all the previous tasks in real-time into a semantically consistent end-to-end solution requiring simultaneous inference of multiple, dependent neural networks. 7. Hair Segmentation 8. Object Detection 9. Box Tracking 10. Instant Motion Tracking 11. Objectron  real-time 3D object detection solution for everyday objects. It detects objects in 2D images and estimates their poses through a machine learning model, trained on the Objectron dataset 12. KNIFT  template-based feature matching tool Most of these are offered for Android, iOS and C++ and some also for Python, JS and Coral. How does it work? Basically, MediaPipe is a framework for Computer Vision and Deep Learning that builds perception pipelines . MediaPipe works through graphs which help creating machine learning-based end-to-end tools. To get an idea of how such a pipeline functions, the best way is to load one of the graphs on the so-called visualiser (hosted at viz.mediapipe. dev) which lets you inspect graphs, modify them through the editor and experiment the result live. On the top right, you can choose which ones of the graphs to load. The graph on the right, for example, represents the Hand Tracking pipeline. The purple boxes in this representation are the subgraphs which perform the main tasks, while all the other white boxes are calculators (node of the graph) performing necessary operations, such as managing input and output streams of images, image transformation and loops. A calculator may receive zero or more input streams and/ or side packets and produces zero or more output streams and/or side packets (data containers). These are the main nodes defined for Hand Detection, Hand Landmark and Annotation Rendering.