Computer Vision News - February 2018

Deep Learning using Linear Support Vector Machines SVM or Softmax? Every month, Computer Vision News reviews a research paper from our field. This month we have chosen to review Deep Learning using Linear Support Vector Machines . We are indebted to the author Yichuan Tang from the Department of Computer Science, University of Toronto , for allowing us to use images from the paper to illustrate this review. His work is here . Introduction: Convolutional deep learning neural networks have greatly evolved in recent years, achieving state of the art results in a wide range of computer vision tasks. These achievements include (among others) classification, object detection, localization and segmentation. Most CNNs using fully connected layers and convolutional layers rely on the softmax function for estimating network loss. The author of this paper evaluates an SVM loss function as an alternative to softmax. His results show that replacing softmax with L2-SVM produces a gain when run on the popular MNIST, CIFAR-10 and Facial Expression datasets, indicating that softmax’s status as the go-to solution among developers should be reconsidered. To be able to make a comparison in the same setting between softmax and an SVM function, he needed a differentiable loss function - and chose L2-SVM (SVM squared). The author reproduced the model proposed in Zhong & Ghosh, 2000, but used the L2-SVM in place of the standard SVM (hinge loss). 4 Computer Vision News Research Research by Assaf Spanier “Evaluate an SVM loss function as an alternative to softmax” Softmax SVM The function = − log exp ( exp( )) − Score of the k-th category = min , 1 2 ⋅ + =1 . . ≥ 1 − ∨ ( > 0) = 1 2 ⋅ + =1 max(1 − , 0)

RkJQdWJsaXNoZXIy NTc3NzU=