Add Evaluation section
This commit is contained in:
parent
65c0907685
commit
77e39fd0a2
33
sim.tex
33
sim.tex
@ -291,7 +291,38 @@ multiplied by the weights from the hidden layer. This results in the
|
||||
aforementioned dual process model where negative convolution and positive
|
||||
convolution are employed to form the output.
|
||||
|
||||
\section{Evaluation 200 words}
|
||||
\section{Evaluation}
|
||||
|
||||
An important, if not the most important, part of similarity modeling is
|
||||
evaluating the performance of classifiers. A straightforward way to do so is
|
||||
analyzing the \emph{confusion matrix}. A confusion matrix contains the output of
|
||||
the classifier on one axis and the ground truth on the other axis. If the
|
||||
classifier says something is relevant and the ground truth says that as well, we
|
||||
have a true positive. The same applies to negatives where both agree and these
|
||||
are called true negatives. However, if the ground truth says something is
|
||||
irrelevant, but the classifier says it is relevant, we have a false positive.
|
||||
Conversely, false negatives require the classifier to say something is
|
||||
irrelevant when it is in fact actually relevant.
|
||||
|
||||
From the confusion matrix we can derive \emph{recall} or \emph{true positive
|
||||
rate}. It is calculated by dividing the true positives by the sum of the true
|
||||
positives and false negatives. If the ratio is close to one, the classifier
|
||||
recognizes almost everything correctly. Recall on its own is not always helpful
|
||||
because there is the possibility that the classifier recognizes everything
|
||||
correctly but has a high \emph{false positive rate}. It is defined by the false
|
||||
positives divided by the sum of the false positives and the true negatives. A
|
||||
low value of the false positive rate combined with a high value of recall is
|
||||
desirable. Third, \emph{precision} is another measure for pollution, similarly
|
||||
to the false positive rate. It is defined as the true positives divided by the
|
||||
sum of the true positives and the false positives. An advantage of precision is
|
||||
that it can be calculated just from the output of the classifier. Precision and
|
||||
recall are inversely correlated, in that a recall of one can always be achieved
|
||||
by classifying everything as relevant, but then the precision is zero and
|
||||
vice-versa.
|
||||
|
||||
All three measures can be visualized by the \emph{recall-precision-graph} or the
|
||||
\emph{receiver operating characteristics curve} (ROC curve). The latter plots
|
||||
the false positive rate on the x-axis against the true positive rate.
|
||||
|
||||
\section{Perception and Psychophysics 600 words}
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user