CVPR Daily - Wednesday

Can you tell me something about Microsoft that we, the community, don’t know? Something that I’m really proud of at Microsoft is the way in which we bring cutting-edge computer vision to real- world products. We did that with Kinect. No one knew how to do body tracking before Kinect came out and it just got done. With HoloLens, I’m super proud of something that I didn’t work on. The head tracking in HoloLens v1 done by my colleagues is just an amazing piece of engineering. It’s kind of like you had a moonshot and then you ended up on Mars. What is particularly revolutionary about it? If you ask George Klein, who was responsible for large proportions of the middle of it, he’ll say nothing. Just good engineering. Just we put it together. Sometimes in the research world, and I think young people are maybe better at this now, people say, “It’s just engineering,” as if just engineering is not something to be proud of. I think the distinction between research and engineering is much closer than we think. If we think about how deep networks suddenly became popular, to a large extent, it’s about the engineering that Krizhevsky did in order to make the networks run on the GPU. In some sense, an engineering advance has given us this amazing new tool and capability. I have a title of researcher, but I really think of myself as an engineer. My group at Microsoft, and across Microsoft, more and more we’re trying to remove the boundaries between roles that are called researcher and roles that are called engineer. W hat is the most dramatic boundary that you’ve been able to break? In head tracking on HoloLens, we all knew how to do SLAM, we all knew how to do real-time SLAM, and we all knew how to do structure from motion, but no one knew how to do it for hours and hours and hours. All the demos that we would do as academics were on reasonably short sequences, and no one knew how to do it in incredibly messy environments. The very messy environments that we deal with most often are offices. Why do you think it’s Microsoft that came out with this and not another company? What is special about Microsoft that made it the company who will bring it out? That is a good question. Thank you. That is the third time. If I get to 10, I’ll win a prize. There are a few things. When we were working on Kinect, we have this visionary guy, Alex Kipman, and he phoned up the lab in Cambridge and said, “ I’ve got this brilliant idea. We’re going to do a machine learning system which does body tracking and we’re going to stand in front of the computer and we’re going to play .” I said, “ It’s lucky you came to me because I am a world expert in body tracking, and I can tell you that what you’re saying is impossible. It’s not going to happen. You need to do it in 10 per cent of an Xbox, which is a 2003 era piece of hardware, and you need to do it real- time, and it needs to run for hours, and it’s absolutely impossible. ” Kipman said, “ Oh, it’s funny you say that, because look, I’ve done it .” He produced us a demo which was doing a fantastically good job of body tracking if you started with your arms out in a T-shape. If you started in the right pose, then it would do a fantastic job for about 20 seconds or about a minute. 11 DAILY CVPR Wednesday Andrew Fitzgibbon