朴素贝叶斯原理推导与常见模型

作者: Andrew_jidw | 来源:发表于2020-03-17 11:35 被阅读0次

1.朴素贝叶斯原理推导

2.常见模型

不同的朴素贝叶斯分类器的区别主要在于它们对 $P(x_{i} |y_{k} )$ 分布的假设不同。尽管它们的假设显然过于简化，但naive Bayes分类器在许多实际情况下都能很好地工作，比如著名的文档分类和垃圾邮件过滤。它们需要少量的训练数据来估计必要的参数。(由于理论上的原因，naive Bayes工作得很好，以及它工作的数据类型，请参阅下面的参考资料。)

Naive Bayes learners and classifiers can be extremely fast compared to more sophisticated methods. The decoupling of the class conditional feature distributions means that each distribution can be independently estimated as a one dimensional distribution. This in turn helps to alleviate problems stemming from the curse of dimensionality.

On the flip side, although naive Bayes is known as a decent classifier, it is known to be a bad estimator, so the probability outputs from predict_proba are not to be taken too seriously.

（1）Gaussian Naive Bayes

GaussianNB implements the Gaussian Naive Bayes algorithm for classification. The likelihood of the features is assumed to be Gaussian:

The parameters $σ_{y}$ and $μ_{y}$ are estimated using maximum likelihood.

（2）Multinomial Naive Bayes

MultinomialNB implements the naive Bayes algorithm for multinomially distributed data, and is one of the two classic naive Bayes variants used in text classification (where the data are typically represented as word vector counts, although tf-idf vectors are also known to work well in practice). The distribution is parametrized by vectors $θy=(θ_{y1},…,θ_{yn})$ for each class $y$ , where $n$ is the number of features (in text classification, the size of the vocabulary) and $θ_{yi}$ is the probability $P(x_{i} |y )$ of feature $i$ appearing in a sample belonging to class $y$ .

The parameters $θ_{y}$ is estimated by a smoothed version of maximum likelihood, i.e. relative frequency counting:

where $N_{yi}=∑_{x∈T}x_{i}$ is the number of times feature $i$ appears in a sample of class $y$ in the training set $T$ , and $N_{y}=∑_{i=1}^n nN_{yi}$ is the total count of all features for class $y$ .

The smoothing priors $α≥0$ accounts for features not present in the learning samples and prevents zero probabilities in further computations. Setting $α=1$ is called Laplace smoothing, while $α<1$ is called Lidstone smoothing.

（3） Bernoulli Naive Bayes

BernoulliNB implements the naive Bayes training and classification algorithms for data that is distributed according to multivariate Bernoulli distributions; i.e., there may be multiple features but each one is assumed to be a binary-valued (Bernoulli, boolean) variable. Therefore, this class requires samples to be represented as binary-valued feature vectors; if handed any other kind of data, a BernoulliNB instance may binarize its input (depending on the binarize parameter).

The decision rule for Bernoulli naive Bayes is based on

which differs from multinomial NB’s rule in that it explicitly penalizes the non-occurrence of a feature i that is an indicator for class y, where the multinomial variant would simply ignore a non-occurring feature.

In the case of text classification, word occurrence vectors (rather than word count vectors) may be used to train and use this classifier. BernoulliNB might perform better on some datasets, especially those with shorter documents. It is advisable to evaluate both models, if time permits.

注：原理推导源自：https://blog.csdn.net/u012162613/article/details/48323777

常见模型源自： https://scikit-learn.org/stable/modules/naive_bayes.html

网友评论

本文标题：朴素贝叶斯原理推导与常见模型

本文链接：https://www.haomeiwen.com/subject/wruzehtx.html

延伸阅读

深度阅读

您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

朴素贝叶斯原理推导与常见模型

1.朴素贝叶斯原理推导

2.常见模型

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读