ICCV Daily 2019 - Tuesday

Xin Chen is a PhD candidate from Tongji University and Lingxi Xie is a senior researcher at Huawei Technologies. They speak to us ahead of their oral and poster presentation today. Xin and Lingxi propose an efficient and progressive method for differentiable neural architecture search . This method has achieved state-of-the-art performance on a benchmark dataset. Lingxi tells us that there are two main challenges in neural architecture search. The first one is how to make it fast. Conventional approaches involve using reinforcement learning or genetic algorithms which need to train a basic network again and again. This process is quite slow. Now, there is a new technique named differentiable architecture search that can make this process faster, but it raises another challenge which is stability. Currently, when you search the network several times, you can get a different architecture every time and these architectures can produce quite different results and levels of accuracy. He explains how their work helps to solve this: “Our paper involves two techniques. The first one is that we propose a gap between the search stage and the evaluation stage. When conventional approaches search on the network, the search stage often involves a shallow network, but the evaluation stage involves a deep network, so there is a gap between them. We use a progressive way of searching architectures that allows us to gradually increase the depth of the search network so that we can close this gap and achieve better results. The second technique involves improving the stability of search. We add some regularization after the skip-connect operation so that the operation can gradually gain the weight rather than they grow up very quickly to surpass other operators. We also make it so that the network is able to preserve a fixed number of skip-connections after the entire search so that the final network will become much more stable.” Lingxi is sure that neural architecture Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation 8 Oral Presentation DA I L Y Xin Lingxi