Statistical Reasoning-Probability and Baye's Theorem and Causal Networks, Reasoning in Belief Network

Probability and Bayes' Theorem:

Let us consider,

P(H i | E) = The probability that the hypothesis 'Hi' is true for a given evidence E.

P(E | Hi) = The probability that we will observe the evidence E given that hypothesis Hi is true.

P(Hi) = The 'a' prior probability that the hypothesis Hi is true in the absence of any specific evidence. These probabilities are called prior probabilities or prior.

k = The number of possible hypotheses.

Bayes's theorem then states that,

In practical life, we have several pieces of evidence that are not independent.

Example:

S: a patient has spots.

M: a patient has measles.

F: a patient has a high fever.

Here Spots and Fever are not independent events and hence we cannot just sum their effects. There is a need to represent explicitly the conditional probability that arises from their conjunction. In general, a prior body of evidence e and some new observation E is given for which we need to compute,

The size of the set of joint probabilities required to compute this function grows as 2n times if there are n different propositions that are being considered.

Intractability of Bayes's theorem:

There are several reasons for which the Baye's theorem is intractable and they are given below:

• The knowledge acquisition problem is insurmountable.
• The space that may be required to store all the probabilities is too large.
• The time which is required to compute the probabilities is quite large.

Causal Network:

The notion of conditional independence can be used to give a concise representation of many domains. The idea is that for a given random variable Xa a small set of variables may exist that directly affect the variable's value in the sense that X is conditionally independent of other variables for the directly affecting variables. The set of locally affecting variables is called the Markov Blanket.This is the locality that is exploited in a causal network. A Causal network is a directed model of conditional dependence among a set of random variables. The precise statement of conditional independence in a belief network takes the directionality into account.

To define a belief network we shall start with a set of random variables that represent all of the features of the model. Suppose these variables are {X₁,...,X_n}. Then the next thing is to select a total ordering of the variables X₁,...,X_n.

The Causal networks have been claimed to be a good representation of causality. Suppose you have a causal model of a domain in your mind where the domain is specified in terms of a set of random variables. For each pair of random variables X₁ and X₂ if a direct causal connection exists from X₁ to X₂that is intervening to change X₁ in some context of other variables that affects X₂ and this cannot be modeled by having some intervening variable. Add an arc from X₁ to X₂. We would expect that the causal model would obey the independence assumption of the belief network. Thus all the conclusions of the belief network would be valid.

Causal belief network can be seen as a way of axiomatizing in a causal direction. The reasoning in belief networks that corresponds to causes for the abduction and then predicting from these causes. There is an existence of a direct mapping between the logic-based abduction view and belief networks. Belief networks can be modeled as the logic programs with probabilities over possible hypotheses.

References:

1. Elaine Rich, Kevin Knight 1991, "Artificial Intelligence".
2. Nilsson, Nils J. Principles of Artificial Intelligence, Narosa Publishing House New Delhi, 1998.
3. Norvig, Peter & Russel, Stuart Artificial Intelligence: A modern Approach, Prentice Hall, NJ, 1995
4. Patterson, Dan W. Introduction to Artificial Intelligence and Expert Systems, Prentice Hall of India Private Limited New Delhi, 1998.