Logistic Regression  MLE and MAP ## Overfitting Problem ![Image](/uploads/documents/4/6/8/4/46846a4b8de1585ec00c8265e0b08eb1/p3_1 data Several ways to define the "best" ## Maximum Likelihood Estimation (MLE) • Maximum Likelihood Estimation (MLE): Choose the parameter $ \theta $ that maximizes the probability of the data of $ \theta $ and defined as: $$ L(\theta)=p(D;\theta)=\prod_{i=1}^{m}p(d^{(i)};\theta) $$ • MLE typically maximizes the log-likelihood instead of the likelihood $$ \ell(\theta)=\log L(\theta)=

og p(y^{(i)})+\sum_{i=1}^{m}\sum_{j=1}^{n}\log p_{j}(x_{j}^{(i)}\mid y^{(i)})\end{align*} $$ ### MLE for Naive Bayes (Contd.) $$ \begin{aligned} \max & \sum_{i=1}^{m} \log p(y^{(i)}) + \sum_{i=1}^{m} & p(y) \geq 0, \forall y \\ & p_j(x \mid y) \geq 0, \forall j, x, y \end{aligned} $$ ### MLE for Naive Bayes (Contd.) ## Theorem 1 The maximum-likelihood estimates for Naive Bayes model are thbf{1}(y^{(i)}=y\land x_{j}^{(i)}=x)}{\sum_{i=1}^{m}\mathbf{1}(y^{(i)}=y)},\forall x,y,j $$ ### MLE for Naive Bayes (Contd.) ## Notation: • The number of training data whose label is y $$ count(

## Toy example  MLE is kind of minimize KLD ![Image](/uploads/documents/9/d/3/3/9d33b9b905124d41f983658e9c910f89/p28_2

LSTM Bi LSTM RNN RNN 输入序列 输入序列 输入序列。。。 解码器内部注意力机制 ## 生成式摘要 强化学习和深度学习相结合的学习方式 • 最优化词的联合概率分布：MLE（最大似然），有监督学习。在这里生成候选的摘要集。 - ROUGE指标评价：不可导，无法采用梯度下降的方式训练，考虑强化学习，鼓励reward高的模型，通过给与反馈来更新模型。最终训练得到表现最好的模型。

Regression Results Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Son, 02 Okt 2016 Pseudo R-squ.: 0.6878 Time: 17:15:45 Log-Likelihood: -143.91

{i} \mid w _ {i - 2}, w _ {i - 1}\right) $$ 这些概率可以通过在大型语料库中进行最大似然估计（Maximum Likelihood Estimation，MLE）来计算。这个术语听起来很复杂，但其思想非常直观：最可能出现的，就是我们在数据中看到次数最多的。例如，对于Bigram模型，我们想计算在词 $ w_{i-1} $出现后，下一个词是 $ w_{i} $的概率

================== Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Fri, 09 Oct 2015 Pseudo R-squ.: 0.6878 Time: 20:59:49 Log-Likelihood: -143.91 converged: pandas: powerful Python data analysis toolkit, Release 0.17.0 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Fri, 09 Oct 2015 Pseudo R-squ.: 0.6878 Time: 20:16:35 Log-Likelihood: -143.91 converged:

================== Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Fri, 07 Jul 2017 Pseudo R-squ.: 0.6878 Time: 12:29:29 Log-Likelihood: -143.91 converged: ================== Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Fri, 07 Jul 2017 Pseudo R-squ.: 0.6878 Time: 12:24:55 Log-Likelihood: -143.91 converged:

Regression Results Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Don, 03 Nov 2016 Pseudo R-squ.: 0.6878 Time: 17:08:14 Log-Likelihood: -143.91 converged: Regression Results Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Don, 03 Nov 2016 Pseudo R-squ.: 0.6878 Time: 16:46:53 Log-Likelihood: -143.91 converged:

================== Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Tue, 12 Dec 2017 Pseudo R-squ.: 0.6878 Time: 06:18:58 Log-Likelihood: -143.91 converged: ================== Dep. Variable: hr No. Observations: 68 Model: Poisson Df Residuals: 63 Method: MLE Df Model: 4 Date: Tue, 12 Dec 2017 Pseudo R-squ.: 0.6878 Time: 06:15:36 Log-Likelihood: -143.91 converged: