Batch Norm 主讲人：龙良曲 Intuitive explanation Intuitive explanation Feature scaling ▪ Image Normalization ▪ Batch Normalization Batch Norm https://medium.com/syncedreview/facebook-ai-proposes-group-normalization- p-normalization- alternative-to-batch-normalization-fb0699bffae7 Pipeline nn.BatchNorm2d Class variables Test Visualization Advantages ▪ Converge faster ▪ Better performance ▪ Robust ▪ stable

3.3.4.2 设备并行 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.3.5 “sample”, “batch”, “epoch” 分别是什么？ . . . . . . . . . . . . . . . . . . . 28 3.3.6 如何保存 Keras 模型？ . . . . . . . . . 5 train_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.2.3.6 test_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2.3.7 predict_on_batch . . . . . 5 train_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.3.3.6 test_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.3.3.7 predict_on_batch . . . . .

io 1 / 79 Where Are We? 2 / 79 Stream Processing Systems Design Issues ▶ Continuous vs. micro-batch processing ▶ Record-at-a-Time vs. declarative APIs 3 / 79 Outline ▶ Spark streaming ▶ Flink Continuous vs. micro-batch processing • Record-at-a-Time vs. declarative APIs 6 / 79 Spark Streaming ▶ Run a streaming computation as a series of very small, deterministic batch jobs. • Chops up the the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations. • Discretized Stream Processing (DStream) 7 / 79 Spark Streaming ▶ Run a streaming computation

dtype=float32) batch_dot It is used to perform the product of two data in batches. Input dimension must be 2 or higher. It is shown below: >>> a_batch = k.ones(shape=(2,3)) >>> b_batch = k.ones(shape=(3 ones(shape=(3,2)) >>> c_batch = k.batch_dot(a_batch,b_batch) >>> c_batch variable It is used to initializes a variable. Let us perform simple transpose layer will have batch size as the first dimension and so, input shape will be represented by (None, 8) and the output shape as (None, 16). Currently, batch size is None as it is not set. Batch size is usually

10) 公式 (3.1.10)中的w和x都是向量。在这里，更优雅的向量表示法比系数表示法（如w1, w2, . . . , wd）更具可读 性。|B|表示每个小批量中的样本数，这也称为批量大小（batch size）。η表示学习率（learning rate）。批量 大小和学习率的值通常是手动预先指定，而不是通过模型训练得到的。这些可以调整但不在训练过程中更新 的参数称为超参数（hyperpa 本并以小批量 方式获取数据。 在下面的代码中，我们定义一个data_iter函数，该函数接收批量大小、特征矩阵和标签向量作为输入，生成 大小为batch_size的小批量。每个小批量包含一组特征和标签。 def data_iter(batch_size, features, labels): num_examples = len(features) indices = list(range(num_examples)) for i in range(0, num_examples, batch_size): batch_indices = torch.tensor( indices[i: min(i + batch_size, num_examples)]) yield features[batch_indices], labels[batch_indices] 通常，我们利用GPU并行运算的优势，处理合

as parameters. It also has two hyperparameters: batch_size and epochs. We use a small batch size because our dataset has just 1020 samples. A large batch size, say 256, will result in a small number (5) train(model, tds, vds, batch_size=24, epochs=100): tds = tds.shuffle(1000, reshuffle_each_iteration=True) batch_tds = tds.batch(batch_size).prefetch(tf.data.AUTOTUNE) batch_vds = vds.batch(256).prefetch(tf ModelCheckpoint(tmpl, save_best_only=True, monitor="val_accuracy") history = model.fit( batch_tds, validation_data=batch_vds, epochs=epochs, callbacks=[checkpoints] ) return history Let’s run a baseline

8.3. Execute batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 8.4. Release batch . . . . . . . 29 8.5. Cancel batch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 8.6. Sync batch . . . . . . . Int32 Minimum type (e.g. ptype_rpc = 2) Chapter 3. Databases 6 Int32 Maximum type (e.g. ptype_batch_send = 3) Int32 Preference weight (e.g. 2) Server Int32 Operation code If operation equals op_accept:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5. Batch Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7.2. Differences Between Batch And Interactive Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7.3. Batch Mode Exit Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 7.4. Errors In Batch Mode Swap To Interactive Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

max_batch_interval/max_batch_rows/max_batch_size max_batch_interval/max_batch_rows/max_batch_size "max_batch_interval" = "20", "max_batch_interval" = "20", "max_batch_rows" = "300000", "max_batch_rows" "max_batch_rows" = "300000", "max_batch_size" = "209715200" "max_batch_size" = "209715200" max_error_number max_error_number max_batch_rows * 10 max_batch_rows * 10 max_error_number max_error_number strict_mode "3",, "max_batch_interval" "max_batch_interval" == "20" "20",, "max_batch_rows" "max_batch_rows" == "300000" "300000",, "max_batch_size" "max_batch_size" == "209715200"

server int 1.0.0 Batch Key Default Meaning Type Since kyuubi.batch.applic ation.check.interval PT5S The interval to check batch job application information. durat ion 1.6.0 kyuubi.batch.conf.i gnore.list comma separated list of ignored keys for batch conf. If the batch conf contains any of them, the key and the corresponding value will be removed silently during batch job submission. Note that this rule is You can also pre- define some config for batch job submission with prefix: kyuubi.batchConf.[batchType]. For example, you can pre-define spark.master for spark batch job with key kyuubi.batchConf.spark.spark