CVPR Daily - Wednesday

Jamie Watson is a Researcher at Niantic and a part-time PhD student at UCL. His paper proposes a new method to improve how virtual assets appear alongside real-world objects in augmented reality. He speaks to us ahead of his poster this morning. Virtual Occlusions Through Implicit Depth 8 DAILY CVPR Wednesday Poster Presentation Augmented reality (AR) has revolutionized how we interact with digital content by overlaying virtual objects in the real world. In creating genuinely immersive AR experiences , realistic occlusions play a vital role. Occlusions refer to the ability of virtual objects to be hidden or partially obscured by real- world elements, such as hands, tables, or buildings, thereby enhancing the believability of the augmented content. With an AR character or object in a scene, the traditional approach performs realistic occlusions by predicting depth. When the AR character is closer than the predicted depth, it can be seen, and when it is further away, any object in front of it will occlude it. This method can sometimes be unstable, with networks having difficulty predicting edges for depth, meaning the AR occlusion jumps in and out. In this work, Jamie proposes a solution by training a neural network to predict occlusions directly . “ Occlusions are important for the immersion of AR experiences, ” he tells us. “ If you’re playing one of your favorite Niantic games , such as Pokémon GO , you’ve got your character on the ground in front of you, and you want to pet it or feed it, it ruins the effect if there are no occlusions . Suppose you put your hand over the camera, and the character stays there. In that case, it totally breaks it. It’s no longer real! If it nicely occludes behind your hand, or as you move behind a table, you can only see part of the creature, it’s a much more believable effect! ”