least r 0s among k elements is (1 − 2−r)k We know that (1 − ϵ)1/ϵ = 1/e For ϵ = 2−r → (1 − 2−r)k = e−k2−r 8 ??? Vasiliki Kalavri | Boston University 2020 The probability of not seeing a tail with at elements is (1 − 2−r)k We know that (1 − ϵ)1/ϵ = 1/e For ϵ = 2−r → (1 − 2−r)k = e−k2−r • If k ≫ 2r : k 2r → 0 and e−k2−r → 1 8 ??? Vasiliki Kalavri | Boston University 2020 The probability of not seeing We know that (1 − ϵ)1/ϵ = 1/e For ϵ = 2−r → (1 − 2−r)k = e−k2−r • If k ≫ 2r : k 2r → 0 and e−k2−r → 1 • If k ≪ 2r : k 2r → ∞ and e−k2−r → 0 8 ??? Vasiliki Kalavri | Boston University 2020 The

DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0', 'B1', 'B2', 'B3']}) ....: In [40]: right = pd.DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'C': ['C0' intersection), since how='inner' by default. In [42]: left = pd.DataFrame({'key1': ['K0', 'K0', 'K1', 'K2'], ....: 'key2': ['K0', 'K1', 'K0', 'K1'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0' 0.25.3 (continued from previous page) In [43]: right = pd.DataFrame({'key1': ['K0', 'K1', 'K1', 'K2'], ....: 'key2': ['K0', 'K0', 'K0', 'K0'], ....: 'C': ['C0', 'C1', 'C2', 'C3'], ....: 'D': ['D0'

Let K1 and K2 be two kernel functions then the followings are as well: Direct sum: K(x, z) = K1(x, z) + K2(x, z) Scalar product: K(x, z) = αK1(x, z) Direct product: K(x, z) = K1(x, z)K2(x, z) Kernels

Let K1 and K2 be two kernel functions, then the following rules hold: • Direct sum: K(x, z) = K1(x, z) + K2(x, z) • Scalar product: K(x, z) = αK1(x, z) • Direct product: K(x, z) = K1(x, z)K2(x, z) Kernels

gradient has its magnitude less than or equal to a predefined threshold (say "), i.e. krf(x)k2  " where k · k2 is `2 norm, such that the values of the objective function di↵er very slightly in successive

Emit(key, AsString(result)); MapReduce combiners example: URL access frequency (k2, list(v2)) → list(v2) (k1, v1) → list(k2, v2) map() reduce() 25 ??? Vasiliki Kalavri | Boston University 2020 MapReduce

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 12 计算性能 503 12.1 编译器和解释器 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 12.1.1 多输入多输出通道可以用来扩展卷积层的模型。 • 当以每像素为基础应用时，1 × 1卷积层相当于全连接层。 • 1 × 1卷积层通常用于调整网络层的通道数量和控制模型复杂性。 练习 1. 假设我们有两个卷积核，大小分别为k1和k2（中间没有非线性激活函数）。 1. 证明运算可以用单次卷积来表示。 2. 这个等效的单个卷积核的维数是多少呢？ 3. 反之亦然吗？ 2. 假设输入为ci × h × w，卷积核大小为co × 讨论影响计算性能的主要因素：命令式编程、符号编程、异步计算、自动并行和多GPU计算。通过学习本章， 对于前几章中实现的那些模型，可以进一步提高它们的计算性能。例如，我们可以在不影响准确性的前提下， 大大减少训练时间。 12.1 编译器和解释器 目前为止，本书主要关注的是命令式编程（imperative programming）。命令式编程使用诸如print、“+” 和if之类的语句来更改程序的状态。考虑下面这段简单的命令式程序：

DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0', 'B1', 'B2', 'B3']}) ....: In [40]: right = pd.DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'C': ['C0' intersection), since how='inner' by default. In [42]: left = pd.DataFrame({'key1': ['K0', 'K0', 'K1', 'K2'], ....: 'key2': ['K0', 'K1', 'K0', 'K1'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0' Release 1.0.0 (continued from previous page) In [43]: right = pd.DataFrame({'key1': ['K0', 'K1', 'K1', 'K2'], ....: 'key2': ['K0', 'K0', 'K0', 'K0'], ....: 'C': ['C0', 'C1', 'C2', 'C3'], ....: 'D': ['D0'

DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0', 'B1', 'B2', 'B3']}) ....: In [40]: right = pd.DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'C': ['C0' intersection), since how='inner' by default. In [42]: left = pd.DataFrame({'key1': ['K0', 'K0', 'K1', 'K2'], ....: 'key2': ['K0', 'K1', 'K0', 'K1'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0' 0.25.0 (continued from previous page) In [43]: right = pd.DataFrame({'key1': ['K0', 'K1', 'K1', 'K2'], ....: 'key2': ['K0', 'K0', 'K0', 'K0'], ....: 'C': ['C0', 'C1', 'C2', 'C3'], ....: 'D': ['D0'

DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0', 'B1', 'B2', 'B3']}) ....: In [40]: right = pd.DataFrame({'key': ['K0', 'K1', 'K2', 'K3'], ....: 'C': ['C0' intersection), since how='inner' by default. In [42]: left = pd.DataFrame({'key1': ['K0', 'K0', 'K1', 'K2'], ....: 'key2': ['K0', 'K1', 'K0', 'K1'], ....: 'A': ['A0', 'A1', 'A2', 'A3'], ....: 'B': ['B0' 0.25.1 (continued from previous page) In [43]: right = pd.DataFrame({'key1': ['K0', 'K1', 'K1', 'K2'], ....: 'key2': ['K0', 'K0', 'K0', 'K0'], ....: 'C': ['C0', 'C1', 'C2', 'C3'], ....: 'D': ['D0'