MICCAI 2021 Daily – Thursday

To encode the anatomy and the modality into different representations, the team adapted a model they had already developed called SDNet , which is able to disentangle the anatomy from the appearance. “ At inference, we use this model to encode and extract the anatomical and the modality factors separately, ” Spiros explains. “ Let’s say we have MRI images from Patient A and Patient B and pass them through SDNet to extract a semantic group of anatomical factors and a vector group of modality factors. A is healthy and B has a problem with his myocardium. We want to create a non-healthy version of A. To do this, we need to swap the myocardium factors and generate a new group of anatomical factors . This new group of anatomical factors will be used as input to our generator along with the MRI modality factors to be re-entangled and generate the new non-healthy variation of Patient A. ” However, simply swapping two factors does not create a plausible anatomy . The swapped-out factor leaves a hole that the new factor should start filling without overlapping the surrounding anatomy. If a thinner myocardium is removed and replaced with a thicker one, then the surrounding anatomy should be pushed out in order to generate a plausible group of new factors. To tackle this swapping part, the team developed a method called disentangled anatomy arithmetic and a new model called DAA-GAN . To ensure the plausibility of this new group of factors, the team introduce a noise injection mechanism . Noise injection has been used in many models. The most well-known is the StyleGAN , which samples from a Gaussian distribution and adds noise at multiple stages of the convolutional activations, introducing stochastic variation at all spatial locations of the features. However, these local variations can cause deformations. In the case of faces, for example, it creates variations in hair and facial features. 5 DAILY MICCAI Thursday Spiros Thermos