similarity-modeling-abstracts/sim.tex

\documentclass[conference]{IEEEtran}
\usepackage{cite}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{algorithmic}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{xcolor}
\def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
    T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}
\begin{document}

\title{Similarity Modeling 1/2 Abstracts}

\author{\IEEEauthorblockN{Tobias Eidelpes}
\IEEEauthorblockA{\textit{TU Wien}\\
Vienna, Austria \\
e1527193@student.tuwien.ac.at}
}

\maketitle

% \begin{abstract}
% \end{abstract}

% \begin{IEEEkeywords}
% component, formatting, style, styling, insert
% \end{IEEEkeywords}

\section{Setting}

To understand the term \emph{Similarity Modeling} and what it encompasses, it is
first important to know how we as humans perceive and understand the things we
pick up. An illustrative example for this process is the process of seeing
(\emph{detecting}) a face, \emph{recognizing} it and deriving the emotion
attached to it. These three steps are placed on a figurative \emph{semantic
ladder}, where detecting a face sits on the bottom and recognizing emotion on
the top. Face detection thus carries a relatively low semantic meaning, whereas
recognizing emotion is a much more sophisticated process. All three steps are
only possible to be carried out by humans, because they have internal models for
the faces they see, whether they have seen them before and if there is a known
emotion attached to how the face looks. These models are acquired from a young
age through the process of learning. Visual stimuli and models alone are not
enough to be able to conclude that a certain face appears similar or not. The
process connecting stimuli and models is comparing the two, also called
\emph{looking for similarities}. Together, modeling and looking for
similarities, they can be summarized under the term \emph{Similarity Modeling}.

The goal of Similarity Modeling is usually to find a \emph{class} for the object
of interest. The flow of information thus starts with the stimulus, continues on
to the modeling part, where we derive a model of the stimulus and—after finding
similarities to existing knowledge—ends in a class or label. As mentioned
previously, the existing knowledge is fed during the modeling process which
describes the feedback loop we call learning. The difficult part lies in
properly modeling the input stimulus. It is impossible to store every stimulus
verbatim into our existing knowledge base, because it would be too much data if
every variety of a stimulus would have to be saved. Therefore, classification
systems need the modeling step to \emph{break down} the stimulus into small
components which generalize well. The similarity part is generally the same for
various domains. Once a proper model of a stimulus exists, checking for
similarities in the preexisting knowledge base follows the same patterns,
regardless of the type of stimulus. 

Common problems that arise when engineers try to model and classify stimuli come
from the fact that there is a wide variety of input signals. This variety is
represented by signals which can be local and have large and sudden increases or
drops. Others are smooth and the defining characteristic is the absence of
sudden variations. Still different signals can have recurring patterns (e.g.
EEG) or none at all (e.g. stocks). After detection the most crucial problem
remains, which is understanding semantics (also known as the \emph{semantic
gap}). The next problem is getting away from the individual samples to be able
to construct a model. This is known as the \emph{gravity of the sample}. Another
problem is commonly referred to as the \emph{curse of dimensionality}, where we
end up with a huge parameter space and have to optimize those parameters to find
good models. The last problem is bad data. This can be missing data, misleading
data or noisy data.

\section{Similarity Measurement}

The artificial process of measuring similarity in computers is shaped by the
same rules and fundamentals which are governing similarity measurements in
humans. Understanding how similarity measurements work in humans is thus
invaluable for any kind of measurement done using computers. A concept which
appears in both domains is the \emph{feature space}. An example for a feature
space is one where we have two characteristics of humans, gender and age, which
we want to explore with regards to their relation to each other. Gender exists
on a continuum which goes from male to female. Age, on the other hand, goes from
young to old. Because we are only concerned with two characteristics, we have a
\mbox{two-dimensional} feature space. Theoretically, a feature space can be
$n$-dimensional, where increasing values for $n$ result in increasing
complexity. In our brains processing of inputs happens in neurons which receive
weighted signals from synapses. The neuron contains a summarization operation
and a comparison to a threshold. If the threshold is exceeded, the neuron fires
and sends the information to an axon. The weights constitute the dimensions of
the feature space. In computers we can populate the feature space with samples
and then do either a distance (negative convolution) or a cosine similarity
measurement (positive convolution). Since the cosine similarity measurement uses
the product of two vectors, it is at its maximum when the two factors are the
same. It is much more discriminatory than the distance measurement. Distance
measurements are also called \emph{thematic} or \emph{integral}, whereas cosine
similarity measurements are called \emph{taxonomic} or \emph{separable}. Due to
the latter exhibiting highly taxonomic traits, questions of high semantics such
as ``is this person old?'', which require a \emph{true} (1) or \emph{false} (0)
answer, fit the discriminatory properties of cosine similarity.

The relationship between distance and similarity measurements is described by
the \emph{Generalization} function. Whenever the distance is zero, the similarity
measurement is one. Conversely, similarity is at its lowest when the distance is
at its highest. The relationship in-between the extremes is nonlinear and
described by the function $g(d)=s=e^{-d}$, which means that only small increases
in distance disproportionately affect similarity. Generalization allows us to
convert distance measurements to similarity measurements and vice-versa.

\begin{equation}
\label{eq:dpm}
\mathrm{dpm} = \alpha\cdot\vec{s} + (1-\alpha)\cdot
g(\vec{d})\quad\mathrm{with}\quad\alpha\in[0,1]
\end{equation}

Both, cosine similarity and distance measurements, can be combined to form
\emph{Dual Process Models of Similarity} (DPMs). One such example is given in
\eqref{eq:dpm} where both measurements are weighted and the distance
measurement is expressed as a similarity measure using the generalization
function. DPMs model humans' perception particularly well, but are not widely
used in the computer science domain.

\section{Feature Engineering 500 words}

Contrary to popular opinion, the rise of deep learning methods in areas such as
object recognition has not superseded the classical approach of feature
engineering in other areas. Particularly in the audio domain and for motion
detection in videos for example, feature engineering is still the dominant
method. This is highlighted by the fact that classical methods require much less
processing which can be beneficial or even crucial for certain applications
(e.g. edge computing). 

Feature engineering is part of the pipeline which transforms input data into
classes and labels for that data. After modeling comes feature extraction so
that these features can be mapped in the feature space. After the classification
step, we end up with labels corresponding to the input data and the features we
want. In practice, feature engineering deals with analyzing input signals.
Common features one might be interested in during analysis is the loudness
(amplitude), rhythm or motion of a signal.

There are four main features of interest when analyzing visual data: color,
texture, shape and foreground versus background. Starting with color, the first
thing that springs to mind is to use the RGB color space to detect specific
colors. Depending on the application, this might not be the best choice due to
the three colors being represented by their \emph{pure} versions and different
hues of a color requiring a change of all three parameters (red, green and
blue). Other color spaces such as hue, saturation and value (HSV) are better
suited for color recognition, since we are usually only interested in the hue of
a color and can therefore better generalize the detection space. Another option
is posed by the \emph{CIE XYZ} color space which is applicable to situations
where adherence to how the human vision works is beneficial. For broadcasting
applications, color is often encoded using \emph{YCrCb}, where \emph{Y}
represents lightness and \emph{Cr} and \emph{Cb} represent $Y-R$ and $Y-B$
respectively. To find a dominant color within an image, we can choose to only
look at certain sections of the frame, e.g. the center or the largest continuous
region of color.  Another approach is to use a color histogram to count the
number of different hues within the frame.

Recognizing objects by their texture can be divided into three different
methods. One approach is to look at the direction pixels are oriented towards to
get a measure of \emph{directionality}. Secondly, \emph{rhythm} allows us to
detect if a patch of information (micro block) is repeated in its neighborhood
through \emph{autocorrelation}. Autocorrelation takes one neighborhood and
compares it—usually using a generalized distance measure—to all other
neighborhoods. If the similarity exceeds a certain threshold, there is a high
probability that a rhythm exists. Third, coarseness can be detected by applying
a similar process, but by looking at different window sizes to determine if
there is any loss of information. If there is no loss of information in the
compressed (smaller) window, the image information is coarse.

Shape detection can be realized using \emph{kernels} of different sizes and with
different values. An edge detection algorithm might use a sobel matrix to
compare neighborhoods of an image. If the similarity is high, there is a high
probability of there being an edge in that neighborhood.

Foreground and background detection relies on the assumption that the coarseness
is on average higher for the background than for the foreground. This only makes
sense if videos have been properly recorded using depth of field so that the
background is much more blurred out than the foreground.

For audio feature extraction, three properties are of relevance: loudness,
fundamental frequency and rhythm. Specific audio sources have a distinct
loudness to them where for example classical music has a higher standard
deviation of loudness than metal. The fundamental frequency can be particularly
helpful in distinguishing speech from music by analyzing the \emph{zero
crossings rate} (ZCR). Speech has a lower ZCR than music, because there is a
limit on how fast humans can speak. Audio signals can often times be made up of
distinct patterns which are described by the attack, sustain, decay and release
model. This model is effective in rhythm detection.

Motion in videos is easily detected using crosscorrelation between previous or
subsequent frames. Similarly to crosscorrelation in other domains, a similarity
measure is calculated from two frames and if the result exceeds a threshold,
there is movement. The similarity measurements can be aggregated to provide a
robust detection of camera movement.

\section{Classification 500 words}

\section{Evaluation 200 words}

\section{Perception and Psychophysics 600 words}

\section{Spectral Features 600 words}

\section{Semantic Modeling 200 words}

\section{Learning over Time 600 words}

\section*{References}

Please number citations consecutively within brackets \cite{b1}. The 
sentence punctuation follows the bracket \cite{b2}. Refer simply to the reference 
number, as in \cite{b3}---do not use ``Ref. \cite{b3}'' or ``reference \cite{b3}'' except at 
the beginning of a sentence: ``Reference \cite{b3} was the first $\ldots$''

Number footnotes separately in superscripts. Place the actual footnote at 
the bottom of the column in which it was cited. Do not put footnotes in the 
abstract or reference list. Use letters for table footnotes.

Unless there are six authors or more give all authors' names; do not use 
``et al.''. Papers that have not been published, even if they have been 
submitted for publication, should be cited as ``unpublished'' \cite{b4}. Papers 
that have been accepted for publication should be cited as ``in press'' \cite{b5}. 
Capitalize only the first word in a paper title, except for proper nouns and 
element symbols.

For papers published in translation journals, please give the English 
citation first, followed by the original foreign-language citation \cite{b6}.

\begin{thebibliography}{00}
\bibitem{b1} G. Eason, B. Noble, and I. N. Sneddon, ``On certain integrals of Lipschitz-Hankel type involving products of Bessel functions,'' Phil. Trans. Roy. Soc. London, vol. A247, pp. 529--551, April 1955.
\bibitem{b2} J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68--73.
\bibitem{b3} I. S. Jacobs and C. P. Bean, ``Fine particles, thin films and exchange anisotropy,'' in Magnetism, vol. III, G. T. Rado and H. Suhl, Eds. New York: Academic, 1963, pp. 271--350.
\bibitem{b4} K. Elissa, ``Title of paper if known,'' unpublished.
\bibitem{b5} R. Nicole, ``Title of paper with only first word capitalized,'' J. Name Stand. Abbrev., in press.
\bibitem{b6} Y. Yorozu, M. Hirano, K. Oka, and Y. Tagawa, ``Electron spectroscopy studies on magneto-optical media and plastic substrate interface,'' IEEE Transl. J. Magn. Japan, vol. 2, pp. 740--741, August 1987 [Digests 9th Annual Conf. Magnetics Japan, p. 301, 1982].
\bibitem{b7} M. Young, The Technical Writer's Handbook. Mill Valley, CA: University Science, 1989.
\end{thebibliography}

\end{document}