Transformer原本是用来做NLP的工作的，所以ViT的首要任务是将图转换成词的结构，这里采取的方法是如上图左下角所示，将图片分割成小块，每个小块就相当于句子里的一个词。这里把每个小块称作Patch，而Patch Embedding就是把每个Patch再经过一个全连接网络压缩成一定维度的向量。  先将图片分成N×N的patch块(原始论文是16×16) patch块可以重叠(上图没有重叠，是9×9的patch块) ### 2. 模型介绍 将patch打平，对每个 patch 进行线性映射，提取特征 ![Image](/uploads/documents/3/b/2/1/3b21a8bfa

## Transformer 总结 •Transformer 与 RNN 不同，可以比较好地并行训练。 •Transformer 本身是不能利用单词的顺序信息的，因此需要在输入中添加位置 Embedding，否则 Transformer 就是一个词袋模型了。 •Transformer 的重点是 Self-Attention 结构，其中用到的 Q, K, V 矩阵通过输出进行线性变换得到。 •Transformer

such that similar inputs have similar representations. We will call this representation an Embedding. An embedding is a vector of features that represent aspects of an input numerically. It must fulfill the tuple of these two features an embedding, where the two features are its dimensions. We will shortly explain how we can use these embeddings. |Animal|Embedding (cute, dangerous)| |---|---| |dog|(0 very dangerous, but you might want to stay away from it too. Now that we have a two-dimensional embedding for each animal, where each feature represents one dimension, we can represent the animals on a

notebook’s page is not sufficient. 🔍 / examples 🔍 .. 🔍 Builtin Extensions 🔍 Customization 🔍 Embedding 🔍 IPython Kernel 🔍 Interactive Widgets 🔍 Notebook 🔍 Parallel Computing 🔍 images 🔍 utils .reg = [/^%%cython/] text/x-cython is the name of CodeMirror mode name, magic_ prefix will just patch the mode so that the first line that contains a magic does not screw up the highlighting. regis a release. ##### 5.17 4.3.2 4.3.2 is a patch release with a bug fix for CodeMirror and improved handling of the “ editable ” cell metadata field. • Monkey-patch for CodeMirror that resolves #2037 without

notebook’s page is not sufficient. 🔍 / examples 🔍 .. 🔍 Builtin Extensions 🔍 Customization 🔍 Embedding 🔍 IPython Kernel 🔍 Interactive Widgets 🔍 Notebook 🔍 Parallel Computing 🔍 images 🔍 utils .reg = [/^%%cython/] text/x-cython is the name of CodeMirror mode name, magic_ prefix will just patch the mode so that the first line that contains a magic does not screw up the highlighting. regis a release. ##### 5.16 4.3.2 4.3.2 is a patch release with a bug fix for CodeMirror and improved handling of the “ editable ” cell metadata field. • Monkey-patch for CodeMirror that resolves #2037 without

notebook’s page is not sufficient. 🔍 / examples 🔍 .. 🔍 Builtin Extensions 🔍 Customization 🔍 Embedding 🔍 IPython Kernel 🔍 Interactive Widgets 🔍 Notebook 🔍 Parallel Computing 🔍 images 🔍 utils .reg = [/^%%cython/] text/x-cython is the name of CodeMirror mode name, magic_ prefix will just patch the mode so that the first line that contains a magic does not screw up the highlighting. regis a release. ##### 5.19 4.3.2 4.3.2 is a patch release with a bug fix for CodeMirror and improved handling of the “ editable ” cell metadata field. • Monkey-patch for CodeMirror that resolves #2037 without

notebook’s page is not sufficient. 🔍 / examples 🔍 .. 🔍 Builtin Extensions 🔍 Customization 🔍 Embedding 🔍 IPython Kernel 🔍 Interactive Widgets 🔍 Notebook 🔍 Parallel Computing 🔍 images 🔍 utils .reg = [/^%%cython/] text/x-cython is the name of CodeMirror mode name, magic_ prefix will just patch the mode so that the first line that contains a magic does not screw up the highlighting. regis a release. ##### 5.18 4.3.2 4.3.2 is a patch release with a bug fix for CodeMirror and improved handling of the “ editable ” cell metadata field. • Monkey-patch for CodeMirror that resolves #2037 without

notebook server Running a public notebook server Running the notebook with a customized URL prefix Embedding the notebook in another website ○ Known issues • Security in the Jupyter notebook server Alternatives like the following (see above for instructions about modifying jupyter_notebook_config.py): ## Embedding the notebook in another website Sometimes you may want to embed the notebook somewhere on your website, e.g. in an IFrame. To do this, you may need to override the Content-Security-Policy to allow embedding. Assuming your website is at https://mywebsite.example.com, you can embed the notebook on your website

notebook server Running a public notebook server Running the notebook with a customized URL prefix Embedding the notebook in another website ○ Known issues • Security in the Jupyter notebook server Alternatives like the following (see above for instructions about modifying jupyter_notebook_config.py): ## Embedding the notebook in another website Sometimes you may want to embed the notebook somewhere on your website, e.g. in an IFrame. To do this, you may need to override the Content-Security-Policy to allow embedding. Assuming your website is at https://mywebsite.example.com, you can embed the notebook on your website

notebook server Running a public notebook server Running the notebook with a customized URL prefix Embedding the notebook in another website ○ Known issues • Security in the Jupyter notebook server Alternatives like the following (see above for instructions about modifying jupyter_notebook_config.py): ## Embedding the notebook in another website Sometimes you may want to embed the notebook somewhere on your website, e.g. in an IFrame. To do this, you may need to override the Content-Security-Policy to allow embedding. Assuming your website is at https://mywebsite.example.com, you can embed the notebook on your website