Computer Vision News - December 2019

Exclusive Interview 22 that everybody is wearing out in public today, but this is of course the ambition and what we are working towards. The current form factor can already do a lot of very useful things. In particular, in industrial scenarios such as in a factory where a worker is repairing a machine, or a surgeon working with their hands to do surgical procedures on someone. So, where people have to do complicated tasks, there exists relevant digital information that can help them do their task better, more efficiently or more safely. If that information can be brought in front of their eyes in an easy way so that it’s natural for them to absorb it while they do the task, that can make a huge difference. That’s what we’re working towards and what I’m really excited about. To do that, there’s a ton of computer vision problems that need to be solved. You want the device bringing information naturally to the person aligned to the real-world problem in front of them. That requires a hologram that visually looks like it’s rigid and attached to the world. It requires you to continually know very precisely, and even actually predict slightly ahead of time, where the device is. Just to show a static hologram in the world it needs SLAM (simultaneous localization and mapping) to run all the time, and really high-quality SLAM, which is a combination of visual-inertial odometry and then larger mapping capabilities. All of this has to run in real time all the time while you use the device. In addition to that, just to place holograms in a way that they interact with the world – for example, placing a hologram that can actually sit on a table instead of somehow being half pushed through it when I’m placing it – requires a 3D model of the world at the same time. We use our depth camera for that, which is the same camera that is now in the Azure Kinect. There’s also a need to be able to track what the user is doing. In HoloLens, there’s computer vision that will continuously track the hands in real time. That’s something that’s done "ETH is indeed a fantastic place. " Best of ICCV 2019