CVPR Daily - Wednesday

Manuel Marin is Associate Professor and member of the Artificial Vision Applications (AVA) group of the University of Cordoba in Spain. He speaks to us following his poster yesterday about automatic analysis of interaction of people in video. From the big goal of understanding videos, the work focuses on one particular problem: how do you detect people looking at each other. Manuel explains: “ Before being able to decide that two people are interacting, previous to the interaction, people are looking at each other. For example, now we are looking at each other, but there is no physical contact. So, if there is no physical contact or there is no motion, typical models for understanding human actions are not valid because the motion is not present in the video. ” Why is it important that people are looking at each other? Because if we look at each other, on top of that you can build applications like analysing if we like or hate each other, or if we are ignoring each other. Generally, if you have a scene with multiple people, in order to be able to understand social relations, the gaze is very important. This work provides a tool that can be used in different applications as a building block to be able to get a higher-level understanding of the scene. Manuel says this is technically difficult because you have to be able to detect the person in the scene. Depending on the viewpoint, some people are seen from the back, so face detectors do not work. You can’t apply a detector to the back of a head. You can’t analyse the eyes, because if the person is seen from the back or they are wearing glasses, you will not see their eyes. In this work, the model uses head tracks as input, so temporal information about the head from which you can infer if the relative position of the head is in a compatible state. Basically, the model will automatically tell you if people are looking at each other or not. LAEO-Net: Revisiting People Looking at Each Other in Videos 14 DAILY CVPR Wednesday Poster Presentation “… if there is no physical contact or there is no motion, typical models for understanding human actions are not valid because the motion is not present in the video” Manuel (left) at his poster, yesterday. With him, co-author Vicky Kalogeiton