CVPR Daily - Wednesday

9 DAILY CVPR Wednesday Virtual Occlusions Through Implicit Depth Furthermore, predicting occlusions can benefit areas like AR directions , where real-world objects could appropriately occlude visual cues for effective navigation. Also, in surgical video analysis , if highlighted objects or anatomical structures remain visible even when hands or instruments occlude them, it could confuse their positions. By accurately predicting occlusions, surgeons can have a clearer view of the surgical site , enabling more precise and informed interventions. “ The stability of predictions is one challenge we faced , ” Jamie recalls. “ Traditional depth estimation methods often overlook this. Generally, for CVPR, you’re evaluated on how good your depth is on a given frame. It doesn’t care about how stable it is over time, which for real use cases is very important. You could have a good per-frame prediction, but if it constantly changes its mind, your occlusions will look really bad and unbelievable because the character will flicker in and out of view. Tackling that was one of the biggest challenges. ” To solve this, Jamie says he took inspiration from previous works, moving away from depth regression and redefining the problem as binary segmentation , allowing him to incorporate ideas from segmentation methods known for their temporal stability, which significantly enhanced performance. Evaluating the method posed another challenge due to the novelty of the task. Temporal evaluation of occlusions had not been previously explored . He devised a new benchmarking method and planned to introduce it to the community, enabling other researchers to test and refine the approach. A general overview of the method - it takes as input an RGB image as well as the rendering of an augmented reality asset, and directly predicts an occlusion mask as output.