CVPR Daily - Thursday

between the embedded clip and those thread banks. Importantly, this must be learned end to end to understand the different types of ongoing activities . Dima says they realized they needed a new method to learn from the vast amount of untrimmed and unlabeled footage, so they proposed a self- supervised approach to build synthetic stories . These are fake stories from videos, which help train the model and finetune on a small subset of labeled data. Did the team encounter any challenges along the way? “ The biggest challenge we had was looking for inspiration for a problem we thought existed, but we didn’t know how to formulate clearly or where to bring the ideas from, ” Dima recalls. “ You always have doubts about whether you’ve done everything right. There was back and forth in the thinking process and setbacks along the way. I remember we were focusing on people recovering from errors at the very beginning. We thought that should be part of our model. We didn’t have many examples to learn from or to evaluate what forms an error. Ultimately, we changed direction and ignored that aspect, focusing purely on the switching between clear goals. ” 10 DAILY CVPR Thursday Poster Presentation