Computer Vision News - July 2021

There are various strategies to scale a network , like scaling a ResNet-50 into a ResNet-101 by doubling the depth, but what if you want very high-capacity networks and you want them to be reasonably fast? This work looks at what makes these models both accurate and fast when scaled and provides a paradigm for scaling networks in an efficient way. Until now, people primarily looked at how scaling strategies affect flops – floating point operations – and the accuracy of a model. The problem with this is that flops are not very predictive of runtime on modern accelerators (GPU, TPU) . Modern accelerators are very memory-bound , so often what really matters is the memory these networks use. This work uses a notion called activations , which is the size of the output tensors of convolutional layers in CNNs. Activations – which are roughly correlated to memory – are an analytical measure which you calculate offline about a network and are very predictive of runtime . Piotr and his team designed activations-aware scaling strategies so that when you scale a network, say to double its flops, you are also cognizant of how the activations change. Therefore, these networks are much faster on modern accelerators. “ What’s exciting about this work is that one would think there’s a trade-off between speed and accuracy as you get to these bigger models, ” Piotr tells us. “ In previous work, the very big models tended to be quite slow, or they tended to be inaccurate. But you can get the best of both worlds – models that are very big and both reasonably fast and accurate. It’s a win-win! ” The team has been working on model design and understanding for some time Piotr Dollár is a research director at Facebook AI Research (FAIR). His innovative paper analyzes strategies for scaling convolutional neural networks to larger scales, such that they are both fast and accurate. He speaks to us ahead of his presentation today. Fast and Accurate Model Scaling Presentation 18 Best of CVPR 2021