Probabilistic Language Processing
Using logic to represent and reason we can represent
knowledge about the world with
facts and rules, like the following ones:
bird(tweety).
fly(X) :- bird(X).
We can also use a theorem-prover to reason about the world and
deduct new facts about
the world, for e.g.,
?- fly(tweety).
Yes
However, this often does not work outside of toy domains -
non-tautologous certain
rules are hard to find.
A way to handle knowledge representation in real problems is to
extend logic by using
certainty factors.
In other words, replace
IF condition THEN fact
with
IF condition with certainty x THEN fact with certainty f(x)
Unfortunately cannot really adapt logical inference to
probabilistic inference, since the
latter is not context-free.
Replacing rules with conditional probabilities makes inferencing
simpler.
Replace
smoking -> lung cancer
or
lotsofconditions, smoking -> lung cancer
with
P(lung cancer | smoking) = 0.6
Uncertainty is represented explicitly and
quantitatively within probability theory, a
formalism that has been developed over centuries.
A probabilistic model describes the world in
terms of a set S of possible states - the
sample space. We don’t know the true state of the world, so we
(somehow) come up with
a probability distribution over S which gives the probability of
any state being the true
one. The world usually described by a set of variables or
attributes.
Consider the probabilistic model of a fictitious
medical expert system. The ‘world’ is
described by 8 binary valued variables:
Review
of Probability Theory
The primitives in probabilistic reasoning are random variables. Just like primitives in
Propositional Logic are propositions. A random variable is not in
fact a variable, but a
function from a sample space S to another space, often the real
numbers.
For example, let the random variable Sum (representing
outcome of two die throws) be
defined thus: Sum(die1, die2) = die1 +die2
Each random variable has an associated
probability distribution determined by the
underlying distribution on the sample space
Continuing our example : P(Sum = 2) = 1/36,
P(Sum = 3) = 2/36, . . . , P(Sum = 12) = 1/36
Conditional probability is defined as:
It
means for any value x of A and any value y
of B
