Computer Vision News - June 2018

The training of deep learning neural networks is a non-trivial process, requiring the update of dozens of parameters. In addition, the behavior of these networks is unpredictable. When writing “normal” code, like a program for sorting an array of numbers, we know the expected behavior of all the algorithms it is made up of and its expected output. In the case of deep learning, however, the behavior of the algorithm is determined by the data input into the network, not by deterministic code. Beyond the non-deterministic nature of deep learning models , we must take into account that these models include tens of millions of parameters, each of which can affect network performance. Networks also have several crucial hyperparameters, which highly impact behavior and performance, such as number of iterations, number of layers, number of hidden units etc., making things quite challenging. When you train a CNN network and it achieves an error rate of 5%, it is nearly impossible to tell whether this performance is optimal without in-depth analysis. Yet, software tools and packages for thorough debugging and analysis of deep learning networks are in their infancy. So what can you do to debug your deep learning set-up? Let’s get to know some of the better known existing tools and get some practical tips on how to use them. This article is divided into three parts. In the first , we’ll give you some practical tips for deep learning network training. In the second , we’ll demonstrate how to implement some of these tips in practice in TensorFlow and Keras. In the third and final part , after you’ve trained your network, we’ll show interpretive approaches to understanding and analyzing what’s been learned, and what is happening ‘under the hood’ of the deep learning network. First thing first, a number of practical tips when training neural networks and a recommended sequence to follow: 10 Focus on: Debug and Analysis Mechanisms Tool by Assaf Spanier “This reduces the overwhelming number of neurons to a small set of groups, distilling the process and power of the neural network’s deep learning.” Computer Vision News Debug and analysis mechanisms for deep learning in TensorFlow and Keras