not-so-BigGAN: Generating High-Fidelity Images on Small Compute with Wavelet-bas

not-so-BigGAN: Generating High-Fidelity Images on Small Compute with Wavelet-based Super-Resolution

State-of-the-art models for high-resolution image generation, such as BigGAN and VQVAE-2, require an incredible amount of compute resources and/or time (512 TPU-v3 cores) to train, putting them out of reach for the larger research community. On the other hand, GAN-based image super-resolution models, such as ESRGAN, can not only upscale images to high dimensions, but also are efficient to train.In this paper, we present not-so-big-GAN (nsb-GAN), a simple yet cost-effective two-step training framework for deep generative models (DGMs) of high-dimensional natural images. First, we generate images in low-frequency bands by training a sampler in the wavelet domain. Then, we super-resolve these images from the wavelet domain back to the pixel-space with our novel wavelet super-resolution decoder network. Wavelet-based down-sampling method preserves more structural information than pixel-based methods, leading to significantly better generative quality of the low-resolution sampler (e.g., 64x64). Since the sampler and decoder can be trained in parallel and operate on much lower dimensional spaces than end-to-end models, the training cost is substantially reduced. On ImageNet 512x512, our model achieves a Fr\'echet Inception Distance (FID) of 10.59 -- beating the baseline BigGAN model -- at half the compute (256 TPU-v3 cores).

not-so-BigGAN：使用基于小波的超高分辨率在小型计算上生成高保真图像

诸如BigGAN和VQVAE-2等用于高分辨率图像生成的最新模型需要大量的计算资源和/或时间（512个TPU-v3内核）来进行训练，这使它们无法承受更大的研究社区。另一方面，基于GAN的图像超分辨率模型（例如ESRGAN）不仅可以将图像放大到高尺寸，而且训练效率也很高。.. 在本文中，我们介绍了不太大的GAN（nsb-GAN），这是一种简单但具有成本效益的两步训练框架，适用于高维自然图像的深度生成模型（DGM）。首先，我们通过训练小波域中的采样器来生成低频频带中的图像。然后，我们使用新颖的小波超分辨率解码器网络将这些图像从小波域超级解析回像素空间。与基于像素的方法相比，基于小波的下采样方法保留了更多的结构信息，从而大大降低了低分辨率采样器（例如64x64）的生成质量。由于可以并行训练采样器和解码器，并且可以在比端到端模型低得多的维度空间上操作，因此可以大大降低训练成本。在ImageNet 512x512上，我们的模型实现了Fr \' （阅读更多）