Bayesian uses the manipulation of conditional probabilities approach to interpret data. In this section, we build a decision system using the Bayesian method. Consider D, called the decision space, which denotes the space of all possible decisions d that could be chosen by the decision maker (DM). Θ is the space of all possible outcomes or state of nature ω, ω ∈ Θ.

Decision system-based Bayesian is built by Bayesian theory. For illustration, I show a simple spam filter using Bayesian. Imagine the sample space X is the set of all possible datasets of words, from which a single dataset word x will result. For each ω ∈ Θ and x ∈ X, the sampling model P(ω) describes a belief that x would be the outcome of spam probability. P(x|ω), prior to distribution, is the true population characteristics and supposes a spam probability for x.P(ω|x), posterior distribution, describes a belief that ω is the true value of spam, having observed dataset x.

The posterior distribution is obtained using Bayes' rule as follows:

This result will return a spam probability value.

Now we can build a decision system. Consider λ (ω,d) as a lost function that states exactly how costly each action d is. Lost function λ(di|ωi) is the loss incurred for taking action di, where the class is ωi. The expected loss or conditional risk is defined as follows:

A decision function d(x) is a mapping from observations to actions. The total risk of a decision function can be calculated as given in the following equation:

A decision function is optimal if it minimizes the total risk. A decision is made based on a minimum risk value for each action. This is a simple explanation. To get further information about Bayesian theory, I suggest you read a textbook about Bayesian.

Decision system-based fuzzy logic

Consider you want to make a decision based on the current temperature, for instance, if the room's temperature is 30°C, then you turn on a cooler machine. Otherwise, if the room's temperature is 18°C, you turn on a heater machine.

This decision happens because we already defined exact values for turning on the machines. What's happening is that we say that we want to turn on the cooler machine if the room's temperature is hot. Furthermore, we also want to turn on the heater machine if the room's temperature is cold.

Cold and hot are two terms related to human linguistics. We should determine how what cold and hot criteria are. A human differentiates the criteria for cold and hot, but how can a computer and machine know?

This problem can be solved using fuzzy logic. The idea of fuzzy logic was first introduced by Dr. Lotfi Zadeh from the University of California at Berkeley in the 1960s. The theory of fuzzy logic is developed with fuzzy sets and memberships.

In general, decision system-based fuzzy logic is described in the following figure:

We can build a decision system with the following steps:

• Define independent variables that represent your problem. This step is a part of the extraction process. These variables usually have numeric values.
• Build fuzzy sets that consist of linguistic variables, for instance, cold, warm, and hot.
• Execute the fuzzification process, which transforms independent variables (numerical values) to dependent variables (linguistic values).
• Build fuzzy inference rules to map between a given input and an output. We can use the if-then approach.
• After aggregating all outputs, we do defuzzification to obtain a single number.

From the output of a single number, we can make a decision. We will do an experiment on how to build a decision system using fuzzy logic in the next section.