DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring f

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs

Image-to-image translation has recently achieved remarkable results. But despite current success, it suffers from inferior performance when translations between classes require large shape changes.We attribute this to the high-resolution bottlenecks which are used by current state-of-the-art image-to-image methods. Therefore, in this work, we propose a novel deep hierarchical Image-to-Image Translation method, called DeepI2I. We learn a model by leveraging hierarchical features: (a) structural information contained in the shallow layers and (b) semantic information extracted from the deep layers. To enable the training of deep I2I models on small datasets, we propose a novel transfer learning method, that transfers knowledge from pre-trained GANs. Specifically, we leverage the discriminator of a pre-trained GANs (i.e. BigGAN or StyleGAN) to initialize both the encoder and the discriminator and the pre-trained generator to initialize the generator of our model. Applying knowledge transfer leads to an alignment problem between the encoder and generator. We introduce an adaptor network to address this. On many-class image-to-image translation on three datasets (Animal faces, Birds, and Foods) we decrease mFID by at least 35% when compared to the state-of-the-art. Furthermore, we qualitatively and quantitatively demonstrate that transfer learning significantly improves the performance of I2I systems, especially for small datasets. Finally, we are the first to perform I2I translations for domains with over 100 classes.

DeepI2I：通过从GAN传输来实现深度分层的图像到图像转换

图像到图像的翻译最近取得了显著成果。但是，尽管目前取得了成功，但当类之间的翻译需要较大的形状变化时，它的性能却很差。.. 我们将此归因于当前最先进的图像到图像方法所使用的高分辨率瓶颈。因此，在这项工作中，我们提出了一种新颖的深度分层图像到图像转换方法，称为DeepI2I。我们通过利用层次特征来学习模型：（a）浅层中包含的结构信息，以及（b）从深层中提取的语义信息。为了能够在小型数据集上训练深度I2I模型，我们提出了一种新颖的转移学习方法，该方法可以从预先训练的GAN转移知识。具体来说，我们利用预先训练的GAN（即BigGAN或StyleGAN）的鉴别器来初始化编码器和鉴别器，并利用预先训练的生成器来初始化模型的生成器。应用知识转移会导致编码器和生成器之间的对齐问题。我们引入了一个适配器网络来解决这个问题。与最先进的技术相比，在三个数据集（动物面部，鸟类和食物）上进行多类图像到图像的翻译时，我们将mFID降低了至少35％。此外，我们定性和定量地证明了转移学习显着提高了I2I系统的性能，特别是对于小型数据集。最后，我们是第一个对具有100多个类的域执行I2I转换的公司。我们定性和定量地证明了转移学习显着提高了I2I系统的性能，尤其是对于小型数据集。最后，我们是第一个对具有100多个类的域执行I2I转换的公司。我们定性和定量地证明了转移学习显着提高了I2I系统的性能，尤其是对于小型数据集。最后，我们是第一个对具有100多个类的域执行I2I转换的公司。 （阅读更多）