CVPR Daily - Thursday

that the network implicitly learns the association between a grayscale patch and event sequences. “ Another big challenge was the overfitting to training data,” Nico tells us. “We needed to come up with different augmentation strategies to augment the synthetic data. We proposed a novel supervision method that can be used directly on real data and relies only on the camera poses . With that, we can fine-tune our network on the real data. ” Carter adds: “ Our model predicts the incremental flow at every step . Many feature trackers currently in the wild are based on optical flow. To make the model more robust to the different sorts of geometric changes that a patch might undergo at each step, we take the patch and apply similarity transform – rotating it, scaling it, and then forcing the network to predict the augmented flow. We found this strategy to be effective in closing the sim-to-real gap. ” 5 DAILY CVPR Thursday Data-driven Feature Tracking for Event Cameras