How it works...

A Bayesian classifier is a classifier based on the application of Bayes' theorem. This classifier requires the knowledge of a priori and conditional probabilities related to the problem; quantities that, in general, are not known but are typically estimable. If reliable estimates of the probabilities involved in the theorem can be obtained, the Bayesian classifier is generally reliable and potentially compact.

The probability that a given event (E) occurs, is the ratio between the number (s) of favorable cases of the event itself and the total number (n) of the possible cases, provided all the considered cases are equally probable. This can be better represented using the following formula:

Given two events, A and B, if the two events are independent (the occurrence of one does not affect the probability of the other), the joint probability of the event is equal to the product of the probabilities of A and B:

If the two events are dependent (that is, the occurrence of one affects the probability of the other), then the same rule may apply, provided P(B | A) is the probability of event A given that event B has occurred. This condition introduces conditional probability, which we are going to dive into now:

The probability that event A occurs, calculated on the condition that event B occurred, is called conditional probability, and is indicated by P(A | B). It is calculated using the following formula:

Let A and B be two dependent events, as we stated that the joint probability between them is calculated using the following formula:

Or, similarly, we can use the following formula:

By looking at the two formulas, we see that they have the first equal member. This shows that even the second members are equal, so the following equation can be written:

By solving these equations for conditional probability, we get the following:

The proposed formulas represent the mathematical statement of Bayes' theorem. The use of one or the other depends on what we are looking for.