Computer Vision News - March 2018

RNN networks, especially long short term memory (LSTM) networks, have had significant success in modeling sequences, including for translation and image captioning. However, the reasons for their success, how they manage to represent features and long- distance interdependencies, have proven very difficult to interpret and analyze. RNNs typically use repeated transformations of the hidden state representation consisting of contain millions of parameters. Researchers seeking a better understanding of the inner workings of these models usually turn to analyzing the changes in this hidden state of RNN or LSTM cells. Past research has shown that analysis of hidden state changes over time can indeed uncover patterns, but at the same time these patterns also include a lot of ‘noise’. LSTMVis is a useful visual analysis tool from the Visual Computing Group at Harvard SEAS, led by Hanspeter Pfister for analyzing LSTM and/or RNN networks. Its goal is to enable quick and efficient analysis of the complex network interactions over time and relating them to human-understandable inputs. LSTMVis enables researchers to select an input range to test a hypothesis focused on a certain pattern in the hidden state dynamic. LSTMVis can also search big data sets and find matching patterns to verify a hypothesis of a certain pattern being related to a certain input, or disprove - that is imply that a false pattern or in fact ‘noise’ was identified, or at least put the hypothesis into question, requiring refinement. The LSTMVis users can be divided into three types: ● Architects are users whose purpose is developing truly innovative deep learning methodologies or revolutionizing existing architectures by adapting them to previously unexplored domains. Their goal is comparing the performance of different models to gain as in-depth an understanding of a system’s properties as possible. 10 We Tried for You: RNNVis Tool by Assaf Spanier “The system was developed based on a series of interviews with deep learning experts” Computer Vision News A visualization tool for understanding and debugging RNNs