## PyTorch ## 高阶OP 主讲人：龙良曲 ## Tensor advanced operation ## Where ## Gather ## where torch.where(condition, x, y) → Tensor Return a tensor of elements selected from either x or y, depending on condition ☐ ☐ ☐ 1 In [198]: cond 2 tensor([[0.6769, 0.7271], 3 [0.8884, 0.4163]]) 4 5 In [199]: a 6 tensor([[0., 0.], 7 [0., 0.]]) 8 9 In [200]: b 10 tensor([[1., 1.], 11 14 Out[203]: 15 tensor([[0., 0.], 16 [0., 1.]]) ## gather torch.gather(input, dim, index, out=None) → Tensor Gathers values along an axis specified by dim. For a 3-D tensor the output is

[105]: b=a.view(2,4) In [106]: c=a.view(2,2,2) In [107]: b tensor([[1., 1., 1., 1.], [1., 1., 1., 1.]]) In [108]: c tensor([[1., 1.], [1., 1.]], [[1 c.norm(1) Out[109]: (tensor(8.), tensor(8.)) In [110]: a.norm(2), b.norm(2), c.norm(2)) Out[110]: (tensor(2.8284), tensor(2.8284)) In [111]: b.norm(1, dim=1) Out[111]: tensor([4., 4.]) In [112]: b b.norm(2, dim=1) Out[112]: tensor([2., 2.]) In [113]: c.norm(1, dim=0) tensor([[2., 2.], [2., 2.]]) In [114]: c.norm(2, dim=0) tensor([[1.4142, 1.4142], [1.4142,

## PyTorch ## 创建Tensor 主讲人：龙良曲 ## I mport from numpy ## ● ● ● 1 In [62]: a=np.array([2,3.3]) 3 In [63]: torch.from_numpy(a) 4 Out[63]: tensor([2.0000, 3.3000], dtype=torch.float64) 6 In [65]: a=np from_numpy(a) 8 Out[66]: 9 tensor([[1., 1., 1.], 10 [1., 1., 1.]], dtype=torch.float64) ## I mport from List ## ● ● ● 1 In [67]: torch.tensor([2., 3.2]) 2 Out[67]: tensor([2.0000, 3.2000]) 4 In [68]: [68]: torch.FloatTensor([2., 3.2]) 5 Out[68]: tensor([2.0000, 3.2000]) 7 In [69]: torch.tensor([[2., 3.2], [1., 8.0], 8.0], 8.0]) 9 tensor([[ 2.0000, 3.2000], 10 [ 1.0000, 22.3000] ) ## uninitialized

## PyTorch ## 基本数据类型 主讲人：龙良曲 ## All is about Tensor |python|PyTorch| |---|---| |Int|IntTensor of size()| |float|FloatTensor of size()| |Int array|IntTensor of size \[d1, d2, ...]| |Float array|FloatTensor One - hot [0, 1, 0, 0, ...] Embedding Word2vec glove ## Data type |Data type|dtype|CPU tensor|GPU tensor| |---|---|---|---| |32-bit floating point|torch.float32 or torch.float|torch.FloatTensor|torch 3) 3 In [6]: a.type() 4 Out[6]: 'torch.FloatTensor' 6 In [7]: type(a) 7 Out[7]: torch.Tensor 9 In [8]: isinstance(a, torch.FloatTensor) 10 Out[8]: True ## ☐ ☐ ☐ 1 In [21]: isinstance(data

In [138]: a[0,0].shape 7 Out[138]: torch.Size([28, 28]) 8 9 In [139]: a[0,0,2,4] 10 Out[139]: tensor(0.8082) ## select first/last N ## ☀️ ☁️ ☁️ 1 In [140]: a.shape 2 Out[140]: torch.Size([4, 3, 28 4) 2 tensor([[ -1.3911, -0.7871, -1.6558, -0.2542], 3 [-0.9011, 0.5404, -0.6612, 0.3917], 4 [-0.3854, 0.2968, 0.6040, 1.5771]]) 6 In [172]: mask = x.ge(0.5) 7 tensor([[0 Out[174]: tensor([0.5404, 0.6040, 1.5771]) 13 14 In [175]: torch.masked_select(x, mask).shape 15 Out[175]: torch.Size([3]) ## select by flatten index ## ● 5 In [177]: torch.take(src, torch.tensor([0, 2

SDK accelerates widely-used deep learning frameworks such as PyTorch. PyTorch is a GPU-accelerated tensor computational framework with a Python front end. Functionality can be easily extended with common memory consumption of your model). Additionally, GEMMs and convolutions with FP16 inputs can run on Tensor Cores, which provide an 8X increase in computational throughput over FP32 arithmetic. APEX AMP is Precision Guide. ## Tensor Core Examples The tensor core examples provided in GitHub and NGC focus on achieving the best performance and convergence from NVIDIA Volta $ ^{TM} $ tensor cores by using the

Iterate over weights examples ## Tensor ## • Tensor? • PyTorch Tensors are just like numpy arrays, but they can run on GPU. ## • Examples: import numpy # create a tensor new_numpy = numpy.array([[1, 2] 2], [3, 4]]) # create a 2 x 3 tensor with random values empty_numpy = numpy.random.rand(2, 3) # create a 2 x 3 tensor with random values between -1 and 1 uniform_numpy = numpy.reshape(np.random.uniform(-1 6), [2, 3]) # create a 2 x 3 tensor with random values from a uniform distribution on the interval [0, 1) rand_numpy = numpy.random.rand(2, 3) # create a 2 x 3 tensor of zeros zero_numpy = numpy.zeros([2

训练一个分类器 ### 1. Tensors张量 ## 01 Tensors张量 02 Autograd自动求导 03 神经网络 04 训练一个分类器 ### 1. Tensors张量的概念 Tensor实际上就是一个多维数组（multidimensional array）标量（0阶张量） 向量（1阶张量） 1.2 矩阵（2阶张量）  ### 1. Tensors张量的概念 ## • 创建张量的几种方法 - 用现有数据创建张量，使用 torch.tensor() • 如torch.tensor([[1., -1.], [1., -1.]]) - 要创建具有特定大小的张量，请使用torch.* • 如torch.randn() # 满足标准正态分布的一组随机数据 创建与其他张量具有相似类型但大小不同的张量，请使用tensor.new_*创建操作。 ### 1. Tensors张量的概念 ## • 查看张量的属性 • 查看Tensor类型 - tensor1 = torch.randn(2,3) #形状为(2,3)一组从标准正态分布中随机抽取的数据 • tensor1.dtype # torch.float32 • 查看Tensor维度和形状 • tensor1.shape #查看形状或尺寸

High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more - Tensor parallelism and pipeline parallelism support for distributed inference - Streaming outputs - OpenAI-compatible supported. LLaVa and encoder-decoder models are not currently enabled in vLLM OpenVINO integration. - Tensor and pipeline parallelism are not currently enabled in vLLM integration. - Speculative sampling is OpenMP threads bound on 0-31 CPU cores. VLLM_CPU_OMP_THREADS_BIND=0-31|32-63 means there will be 2 tensor parallel processes, 32 OpenMP threads of rank0 are bound on 0-31 CPU cores, and the OpenMP threads

High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more - Tensor parallelism and pipeline parallelism support for distributed inference - Streaming outputs - OpenAI-compatible supported. LLaVa and encoder-decoder models are not currently enabled in vLLM OpenVINO integration. - Tensor and pipeline parallelism are not currently enabled in vLLM integration. ## 1.4 Installation with OpenMP threads bound on 0-31 CPU cores. VLLM_CPU_OMP_THREADS_BIND=0-31|32-63 means there will be 2 tensor parallel processes, 32 OpenMP threads of rank0 are bound on 0-31 CPU cores, and the OpenMP threads