the footprint of the model (size, latency, etc). And as we have described earlier, some of these improved quality metrics can be traded off for a smaller footprint as desired. Continuing with the theme likely wasteful. Regarding the first limitation, we know that model quality can usually be naively improved by acquiring more labels (though the rate of improvement eventually plateaus). However, acquiring model. Even though ResNet-50 was introduced back in 2015, updating it with newer learning techniques improved the accuracy significantly without having to change anything in the architecture. Similarly the

cuDNN persistent RNN’s providing improved speed for smaller RNN’s. ‣ Improved multi-GPU performance in both PyTorch c10d and Apex’s DDP. ‣ Faster weight norm with improved mixed-precision accuracy used used through torch.nn.utils.weight_norm. ‣ Improved functionality of the torch.jit.script and torch.jit.tracepreview features including better support for pointwise operations in fusion. ‣ Added support pytorch/pytorch/releases for significant changes from PyTorch 0.4. ‣ Apex is now entirely Python for improved compatibility. Previous versions of Apex will not work with PyTorch 0.4.1 or newer versions. PyTorch

的参数更新 k 次再对 G的参数更新 1 次. 2. GAN的理论与实现模型 17 GAN的衍生模型 GAN的理论与实现模型 CGAN EBGAN Info GAN DCGAN Improved GAN WGAN ...... 2. GAN的理论与实现模型 18 GAN的衍生模型 GAN的理论与实现模型 （1）CGAN--条件生成对抗网络，为了防止训练崩塌将前置条件加入输入数据。 GAN的理论与实现模型 生成模型 z ~x X 自然输入 编码 判别模型 解码 均方误差 能量 生成输入 随机噪声 23 GAN的衍生模型 GAN的理论与实现模型 （6） Improved GAN--改进生成式对抗网络，提出了使模型训练稳定的五条 经验。 a.特征匹配（feature matching） b.最小批量判断（minibatch

earlier because of a lower accuracy, precision / recall, etc. Effectively, we are exchanging the improved quality for a better footprint. Table 3-2 illustrates this concept. So far, we have introduced model with data augmentation achieves the baseline accuracy in fewer epochs, thus it demonstrates improved sample efficiency. It is also possible to show that data augmentation improves label efficiency

quality metrics such as accuracy, precision, recall etc. without impacting the model footprint. Improved accuracy can then be exchanged for a smaller footprint / a more efficient model by trimming the

sequences and temporal data. These breakthroughs contributed to bigger and bigger models. Although they improved the quality of the solutions, the bigger models posed deployment challenges. What good is a model

Sydney, Australia, 2017. [6] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin 和 A. C. Courville, “Improved Training of Wasserstein GANs,” 出处 Advances in Neural Information Processing Systems 30, I. Guyon

2017] Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196. [Kim, 2014] Kim, Y. (2014)