Computer Vision News - December 2022

41 Emma Robinson defined in this way are not spatially localised, and these operations tend to be highly computationally demanding. This limits their practical deployment in complex, deep networks. For these reasons a range of different flavours of approximation have been implemented from polynomial graph convolutions, to rotationally equivariant, localised, mixture of Gaussian filters (MoNet, Monti 2017). Cortical imaging also has its own variant: Spherical UNet (Zhao MICCAI 2019), which fits localised hexagonal filters to a sphere by leveraging the regular tessellation of an icosahedral grid. Each of these methods trade off computational complexity, rotational equivariance and filter expressivity. Research from my lab (Fawaz A 2022) robustly benchmarked the relative importance of these properties for phenotype regression and cortical segmentation and determined that feature expressivity was key, but that registration independence cannot be obtained without some degree of transformation equivariance. Since then we built from this understanding to develop a range of novel deep learning applications for cortical surface modelling from propagating the HCP multimodal cortical segmentation (Glasser Nature 2016) to UK Biobank data (Williams MLCN 2021), to robust and generalisable frameworks for surface registration (Suliman MA MICCAI & GeoMedIA 2022 - see previous pages), translation of vision transforms to the surface (Dahan MIDL 2022) and novel generative models that can simulate healthy ageing of individual preterm cortices and use deviation from true scans to predict cognitive outcomes (Fawaz A, MIUA 2022, GeoMedIA 2022). These show geometric deep learning offers enormous potential to tackle the biggest challenges in cortical imaging. At the same time, we continue look to ongoing development of group equivariant and efficient geometric networks to further improve the precision of these models and power of what they can represent. To my mind, cortical imaging represents a most fertile ground for the development of geometric deep learning technologies . For many years now, we have known that features of cortical function, microstructure and organisation are best studied on the surface. At the same time, it is becoming abundantly clear that traditional pipelines, based on image registration are insufficient to detect subtle features of pathology and cognition from individual scans. The degree of variability across individuals simply breaks the assumptions of classical registration approaches. Geometric deep learning offers incredible opportunity for registration- free, diagnosis, prognosis, and generative/casual modelling from cortical imaging data . However, deep learning on non-Euclidean domains represents unique challenges, which need to be addressed before this nascent technology can reach its full potential. One feature that makes Euclidean CNNs so powerful is their equivariance to translations, meaning that as the view across an image translates the output of any convolution will translate in the same way. This allows networks to detect, or segment, objects wherever they are in a scene. The equivalent operation for surfaces would be equivariance to rotation but this is highly difficult to achieve since manifolds have no global coordinate system. This means that it is not possible to transform filters over a surface in a transform invariant way , since the orientation of a filter will depend on the path it has taken to that point. For these reasons, early graph networks adopted an approach of fitting convolutional filters in the generalized Fourier domain , where it can be shown that the Fourier transform of the convolution of a filter and an image is equal to the product of their Fourier transforms. An equivalent operation can be defined for any domain: for spheres we use the Spherical Harmonics, for 3D rotations we use the Wigner D matrices, and for general graphs we use the eigenspectrum of the graph Laplacian.  Theproblemwith these implementations is that: filters