CVPR Daily - Wednesday

Another challenge is that there could be more than two people in the scene. When there are many combinations, the model has to decide out of all the possible combinations, which are the ones that are looking at each other. Also, Manuel points out they are dealing with a kind of video material that is uncontrolled. They are using movies and TV shows where it’s not been recorded in a lab with perfect lighting and a perfect background. There are people crossing, there are different backgrounds, there are different movie styles. So, how do they solve this? Manuel tells us: “ We are using deep learning. We are using convolutional neural networks and these convolutional neural networks include information about the head positions in the scene. Also, they’re able to extract automatically what is the important information about the head orientation and the geometry of the persons .” In order to demonstrate how their method works, they have developed a fun application based on the TV show, Friends , which automatically finds out what the relationships are between the main characters. It works! 15 DAILY CVPR Wednesday Manuel Marín