From 77e39fd0a269e12e869ba04c5329c503a76d4e98 Mon Sep 17 00:00:00 2001
From: Tobias Eidelpes <tobias@eidelpes.info>
Date: Thu, 21 Oct 2021 16:23:43 +0200
Subject: [PATCH] Add Evaluation section

---
 sim.tex | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/sim.tex b/sim.tex
index 4d9d045..da0382b 100644
--- a/sim.tex
+++ b/sim.tex
@@ -291,7 +291,38 @@ multiplied by the weights from the hidden layer. This results in the
 aforementioned dual process model where negative convolution and positive
 convolution are employed to form the output.
 
-\section{Evaluation 200 words}
+\section{Evaluation}
+
+An important, if not the most important, part of similarity modeling is
+evaluating the performance of classifiers. A straightforward way to do so is
+analyzing the \emph{confusion matrix}. A confusion matrix contains the output of
+the classifier on one axis and the ground truth on the other axis. If the
+classifier says something is relevant and the ground truth says that as well, we
+have a true positive. The same applies to negatives where both agree and these
+are called true negatives. However, if the ground truth says something is
+irrelevant, but the classifier says it is relevant, we have a false positive.
+Conversely, false negatives require the classifier to say something is
+irrelevant when it is in fact actually relevant.
+
+From the confusion matrix we can derive \emph{recall} or \emph{true positive
+rate}. It is calculated by dividing the true positives by the sum of the true
+positives and false negatives. If the ratio is close to one, the classifier
+recognizes almost everything correctly. Recall on its own is not always helpful
+because there is the possibility that the classifier recognizes everything
+correctly but has a high \emph{false positive rate}. It is defined by the false
+positives divided by the sum of the false positives and the true negatives. A
+low value of the false positive rate combined with a high value of recall is
+desirable. Third, \emph{precision} is another measure for pollution, similarly
+to the false positive rate. It is defined as the true positives divided by the
+sum of the true positives and the false positives. An advantage of precision is
+that it can be calculated just from the output of the classifier. Precision and
+recall are inversely correlated, in that a recall of one can always be achieved
+by classifying everything as relevant, but then the precision is zero and
+vice-versa.
+
+All three measures can be visualized by the \emph{recall-precision-graph} or the
+\emph{receiver operating characteristics curve} (ROC curve). The latter plots
+the false positive rate on the x-axis against the true positive rate.
 
 \section{Perception and Psychophysics 600 words}