Computer Vision News - June 2022

17 ICLR 2022 “ I like to look at very simplified settings and try to understand the issue we’re seeing in a toy environment and gain insights from that, ” he explains. “ Onceyoucomeupwithageneralalgorithm, you can go back to the big systems to see if it scales. ” Can he tell us something about DeepMind that we don’t know? He laughs before adding, finally: “ Well, if it’s something you don’t know, you’re probably not supposed to know! ” The key insight is how you deal with optimization and, in this case, meta-optimization. If you solve the dynamics of the optimization problem, you can get a long way. 80% of the problem is probably the data, but the rest is just solving the optimization issues . ” In terms of the next steps for this work, even though intuitively it makes sense, Sebastian says there is no theoretical reason for why and when this type of algorithm works, so a future task would be to gain more understanding of that. He would also like to use this work more directly for few- shot learning. “ We wrote this paper from a meta-learning point of view, but one fascinating experiment we ran shows that the algorithm can be used without meta-learning, ” he reveals. “ You can use the algorithm for zeroth-order optimization and even for optimizing non- differentiable parameters. I think that’s a cool direction to go in as well. ” Outside of this paper, Sebastian’s work is focused on meta-learning research. Together with his peers at DeepMind, he is looking at ways to take extensive systems that train for a long time and give them more autonomy over their learning dynamics to make them less dependent on careful human engineering.