CVPR Daily - Tuesday

Armen Avetisyan is a PhD student at TU Munich working with Matthias Niessner. He speaks to us ahead of his oral and poster today. Armen tells us that in 3D scanning, there are two major problems. One problem is noise, but with really good sensors and cameras you can solve that. One thing that cannot be solved is incomplete capture. For example, when you scan a bedroom, if you don’t scan what’s beneath the bed you will never see it. Even the best sensor won’t tell you. Scan2CAD detects objects from a 3D scan. It takes the bed and replaces it with a complete geometric bed. When you replace the noisy scan with a compact CAD model representation, scan quality is improved, and you can use it for VR environments or save all the information in much smaller memory. You could take a scan of your house or your room and transform it into a game so that you could play, not with a random environment, but in your house! Armen says the challenge of this is that a raw 3D scan is really just a bunch of triangles. There is no segmentation of objects. You don’t have objects; you just have a triangle soup. It’s a challenge to understand which triangles make an object and which don’t make an object. It’s just wall or just floor. Deep learning can help solve this. He explains: “ Deep learning gives you this really strong semantic power. That means this data-driven approach where you can’t really describe mathematically what a chair is, but Scan2CAD: Learning CAD Model Alignment in RGB-D Scans 6 DAILY CVPR Tuesday Presentation “ Deep learning gives you this really strong semantic power. That means this data-driven approach where you can’t really describe mathematically what a chair is, but with lots of examples of a chair you can infer what a chair is!”