Computer Vision News - October 2021

3 Summary 2 RePAIR Pompeii One of the team’s partners, Ohad Ben- Shahar from the Ben-Gurion University of the Negev , has been working on the problem of solving jigsaw puzzles for more than 10 years and is an expert in the field. He and Marcello actually conceived the idea for this project together a while ago. He has proposed state-of-the-start algorithms and in the project they will use deep learning and related methods to try to learn the compatibility of two pieces and then use that to guess the position of all the pieces. However, in the field of computer vision there is not usually the same level of difficulty that is associated with real frescoes, because tiles are often assumed to be square and there are not always missing pieces. “Given the difficulty of the problem, we don’t think that just using a purely data- driven or deep learning-based approach will allow us to completely solve the puzzle,” Marcello reveals. “We’re imagining a system in which we incorporate expert knowledge and combine it with machine learning to improve that compatibility function . We’re talking about frescoes that were built 2,000 years ago. We have art historians and archaeologists, for example, who will give us precious information concerning the style of painting in those times.” The team will build a system that harnesses the power of deep learning methodology while incorporating expertise in a kind of loop. It will try to propose a solution to the expert who will check it and say, ‘I don’t think these pieces go together, you need to propose another guess,’ and so on.