FloatTensor *Assume 't' is a tensor Autograd • Autograd • Automatic Differentiation Package • Don’t need to worry about partial differentiation, chain rule etc.. • backward() does that • loss.backward()

Python. ● Two main features: ○ N-dimensional Tensor computation (like NumPy) on GPUs ○ Automatic differentiation for training deep neural networks Training Neural Networks Training Define Neural Network

Functionality can be easily extended with common Python libraries such as NumPy, SciPy, and Cython. Automatic differentiation is done with a tape-based system at both a functional and neural network layer level. This includes a collection of highly optimized modules for popular Transformer architectures and an automatic mixed precision-like API that can be used seamlessly with your PyTorch code. ‣ A preview of Torch-TensorRT commit 9130ab38 from July 31, 2019 as well as a cherry- picked TensorRT 5.1.5 Automatic Mixed Precision (AMP) Automatic Mixed Precision (AMP) for PyTorch is available in this container through the native

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746 16.9.1 Automatic exclusion of “nuisance” columns . . . . . . . . . . . . . . . . . . . . . . . . . . 746 16.9.2 Suppressing Tick Resolution Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . 962 22.5.5 Automatic Date Tick Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965 22.5.6 mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects • Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the user can

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744 16.9.1 Automatic exclusion of “nuisance” columns . . . . . . . . . . . . . . . . . . . . . . . . . . 744 16.9.2 Suppressing Tick Resolution Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . 957 22.5.5 Automatic Date Tick Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 960 22.5.6 mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects • Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the user can

Consistency of Range Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1.2.2.13 No Automatic Matplotlib Converters . . . . . . . . . . . . . . . . . . . . . . . . . 24 1.2.2.14 Other API Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 776 16.9.1 Automatic exclusion of “nuisance” columns . . . . . . . . . . . . . . . . . . . . . . . . . . 776 16.9.2 Suppressing Tick Resolution Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . 995 22.5.7 Automatic Date Tick Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 998 22.5.8

键步骤。虽然求导的计算很简单，只需要一些基 本的微积分。但对于复杂的模型，手工进行更新是一件很痛苦的事情（而且经常容易出错）。 深度学习框架通过自动计算导数，即自动微分（automatic differentiation）来加快求导。实际中，根据设计 好的模型，系统会构建一个计算图（computational graph），来跟踪计算是哪些数据通过哪些操作组合起来 产生输出。自动微分使系统 [Papineni et al., 2002] Papineni, K., Roukos, S., Ward, T., & Zhu, W.‐J. (2002). Bleu: a method for automatic evaluation of machine translation. Proceedings of the 40th annual meeting of the Association for

mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects • Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the user can Accessing the array can be useful when you need to do some operation without the index (to disable automatic alignment, for example). Series.array will always be an ExtensionArray. Briefly, an ExtensionArray it begins with 'timestamp' • it is 'modified' • it is 'date' Warning: When reading JSON data, automatic coercing into dtypes has some quirks: • an index can be reconstructed in a different order from

mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects • Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the user can Accessing the array can be useful when you need to do some operation without the index (to disable automatic alignment, for example). Series.array will always be an ExtensionArray. Briefly, an ExtensionArray it begins with 'timestamp' • it is 'modified' • it is 'date' Warning: When reading JSON data, automatic coercing into dtypes has some quirks: • an index can be reconstructed in a different order from

mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects • Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the user can Accessing the array can be useful when you need to do some operation without the index (to disable automatic alignment, for example). Series.array will always be an ExtensionArray. Briefly, an ExtensionArray it begins with 'timestamp' • it is 'modified' • it is 'date' Warning: When reading JSON data, automatic coercing into dtypes has some quirks: • an index can be reconstructed in a different order from