Add Evaluation section

This commit is contained in:
Tobias Eidelpes 2021-10-21 16:23:43 +02:00
parent 65c0907685
commit 77e39fd0a2

33
sim.tex
View File

@ -291,7 +291,38 @@ multiplied by the weights from the hidden layer. This results in the
aforementioned dual process model where negative convolution and positive aforementioned dual process model where negative convolution and positive
convolution are employed to form the output. convolution are employed to form the output.
\section{Evaluation 200 words} \section{Evaluation}
An important, if not the most important, part of similarity modeling is
evaluating the performance of classifiers. A straightforward way to do so is
analyzing the \emph{confusion matrix}. A confusion matrix contains the output of
the classifier on one axis and the ground truth on the other axis. If the
classifier says something is relevant and the ground truth says that as well, we
have a true positive. The same applies to negatives where both agree and these
are called true negatives. However, if the ground truth says something is
irrelevant, but the classifier says it is relevant, we have a false positive.
Conversely, false negatives require the classifier to say something is
irrelevant when it is in fact actually relevant.
From the confusion matrix we can derive \emph{recall} or \emph{true positive
rate}. It is calculated by dividing the true positives by the sum of the true
positives and false negatives. If the ratio is close to one, the classifier
recognizes almost everything correctly. Recall on its own is not always helpful
because there is the possibility that the classifier recognizes everything
correctly but has a high \emph{false positive rate}. It is defined by the false
positives divided by the sum of the false positives and the true negatives. A
low value of the false positive rate combined with a high value of recall is
desirable. Third, \emph{precision} is another measure for pollution, similarly
to the false positive rate. It is defined as the true positives divided by the
sum of the true positives and the false positives. An advantage of precision is
that it can be calculated just from the output of the classifier. Precision and
recall are inversely correlated, in that a recall of one can always be achieved
by classifying everything as relevant, but then the precision is zero and
vice-versa.
All three measures can be visualized by the \emph{recall-precision-graph} or the
\emph{receiver operating characteristics curve} (ROC curve). The latter plots
the false positive rate on the x-axis against the true positive rate.
\section{Perception and Psychophysics 600 words} \section{Perception and Psychophysics 600 words}