next up previous contents
Next: The Probabilistic Eigenspace Approach Up: Classification Previous: The Problem

An Optimal Decision Rule

In a probabilistic approach, where all available information is contained in the observation tex2html_wrap_inline2816 , decision rules are in general based on the conditional probability tex2html_wrap_inline2898 . A common procedure is to define a cost function that is minimised by the optimal decision rule. In that sense we define tex2html_wrap_inline2900 to be the cost for taking a decision in favour of class i when the correct decision is class j. The overall cost for a decision rule tex2html_wrap_inline2906 is then


We can simplify this equation with the assumption that the cost is zero for a correct decision and unity for a false decision. We obtain


where tex2html_wrap_inline2908 is the usual Kronecker Delta Function. Furthermore, since we are assuming a deterministic rule, the probability that tex2html_wrap_inline2910 will chose the correct class for x can only be 0 or 1, i.e. tex2html_wrap_inline2918 simplifies to tex2html_wrap_inline2920 .

It is now easy to see that the cost is minimised by the following rule:


By applying Bayes' Rule


and the assumption that the tex2html_wrap_inline2922 are equal for all classes, we can obtain the equivalent rule:


which leads to the conclusion that we can construct an optimal classifier that simply maps a pattern tex2html_wrap_inline2816 to the class tex2html_wrap_inline2926 for which the class conditional probability density tex2html_wrap_inline2928 is maximal.

Markus Weber
Tue Jan 7 15:44:13 PST 1997